Nonnegative and compartmental dynamical systems

NNBook

September 1, 2009

Nonnegative and Compartmental Dynamical Systems

NNBook

September 1, 2009

NNBook

September 1, 2009

Nonnegative and Compartmental Dynamical Systems

Wassim M. Haddad VijaySekhar Chellaboina Qing Hui

PRINCETON UNIVERSITY PRESS PRINCETON AND OXFORD

NNBook

September 1, 2009

c 2010 by Princeton University Press Copyright Published by Princeton University Press 41 William Street, Princeton, New Jersey 08540 In the United Kingdom: Princeton University Press 6 Oxford St, Woodstock, Oxfordshire OX20 1TW All Rights Reserved Library of Congress Cataloging-in-Publication Data Haddad, Wassim M., 1961– Nonnegative and compartmental dynamical systems / Wassim M. Haddad, VijaySekhar Chellaboina, Qing Hui. p. cm. Includes bibliographical references and index. ISBN 978-0-691-14411-5 (hardcover : alk. paper) 1. Differential dynamical systems. I. Chellaboina, VijaySekhar, 1970- II. Hui, Qing, 1976- III. Title. QA614.8.H33 2010 515′ .39—dc22 2009022506

British Library Cataloging-in-Publication Data is available This book has been composed in LATEX The publisher would like to acknowledge the authors of this volume for providing the camera-ready copy from which this book was printed. Printed on acid-free paper. ∞ press.princeton.edu Printed in the United States of America 10

9

8

7

6

5

4

3

2

1

NNBook

September 1, 2009

To my father Mikhael Haddad, father-in-law Paul Katinas, and mentor and friend Dennis Bernstein, with gratitude and appreciation for instilling in me the fervor to always strive to reach ‚ret  W. M. H. To my father Nageswara Rao and the memory of my mother Andhra Jayashree for inspiring my passion for science and mathematics V. C. To my mother Guihua Li and my father Shifu Hui, with everlasting love Q. H.

NNBook

September 1, 2009

NNBook

September 1, 2009

FÔsi krÔptesjai file.

Nature loves to hide. Purì te ‚ntamoib˜ t€ pˆnta kaÈ pÜr pˆntwn.

All matter is exchanged for energy, and energy for all matter. EÚnai g€r ãn tä sofìn, âprtasjai gn¸mhn, åtèh âkubèrnhse pˆnta di€ pˆntwn.

Ultimate wisdom is reached when one achieves a fundamental understanding of the universal laws that govern all things and all forces in the universe. —Herakleitos of Ephesus, Ionia, Greece FÔsi oudenì estn eìntwn allˆ mìnon mxi te, diˆllax te migèntwn est, fÔsi d ep toi onomˆzetai anjr¸poisin.

There is no genesis with regards to any of the things in nature but rather a blending and alteration of the mixed elements; man, however, uses the word ‘nature’ to name these events. —Empedocles of Acragas, Sicily, Greater Greece H enìthta enai o nìmo tou Jèou, h exèlixh enai o nìmo th zw  , o arijmì enai o nìmo tou kìsmou.

Unity is the law of God, evolution is the law of life, and mathematics is the law of the universe. — Pythagoras of Samos, Samos, Greece

NNBook

September 1, 2009

NNBook

September 1, 2009

Contents

Preface

xv

Chapter 1. Introduction

1

Chapter 2. Stability Theory for Nonnegative Dynamical Systems 2.1 Introduction 2.2 Lyapunov Stability Theory for Nonnegative Dynamical Systems 2.3 Invariant Set Stability Theorems 2.4 Semistability of Nonnegative Dynamical Systems 2.5 Stability Theory for Linear Nonnegative Dynamical Systems 2.6 Nonlinear Compartmental Dynamical Systems 2.7 Compartmental Systems in Biology, Ecology, Epidemiology, and Pharmacology 2.8 Discrete-Time Lyapunov Stability Theory for Nonnegative Dynamical Systems 2.9 Discrete-Time Invariant Set Theorems and Semistability Theorems 2.10 Stability Theory for Discrete-Time Linear Nonnegative Dynamical Systems 2.11 Discrete-Time Nonlinear Compartmental Dynamical Systems

7

Chapter 3. Stability Theory for Nonnegative and Compartmental Dynamical Systems with Time Delay 3.1 Introduction 3.2 Lyapunov Stability Theory for Time-Delay Nonnegative Dynamical Systems 3.3 Invariant Set Stability Theorems 3.4 Stability Theory for Continuous-Time Nonnegative Dynamical Systems with Time Delay 3.5 Discrete-Time Lyapunov Stability Theory for Time-Delay Nonnegative Dynamical Systems

7 7 16 21 30 43 49 61 64 69 83

89 89 90 93 97 103

NNBook

x

September 1, 2009

CONTENTS

3.6

Stability Theory for Discrete-Time Nonnegative Dynamical Systems with Time Delay

Chapter 4. Nonoscillation and Monotonicity of Solutions of Nonnegative Dynamical Systems 4.1 Introduction 4.2 Nonoscillation and Monotonicity of Linear Nonnegative Dynamical Systems 4.3 Mammillary Systems 4.4 Monotonicity of Nonlinear Nonnegative Dynamical Systems 4.5 Monotonicity of Discrete-Time Linear Nonnegative Dynamical Systems 4.6 Monotonicity of Discrete-Time Nonlinear Nonnegative Dynamical Systems 4.7 Monotonicity of Nonnegative Dynamical Systems with Time Delay Chapter 5. Dissipativity Theory for Nonnegative Dynamical Systems 5.1 Introduction 5.2 Dissipativity Theory for Nonnegative Dynamical Systems 5.3 Feedback Interconnections of Nonnegative Dynamical Systems 5.4 Dissipativity Theory for Nonlinear Nonnegative Dynamical Systems 5.5 Feedback Interconnections of Nonnegative Nonlinear Dynamical Systems 5.6 Dissipativity Theory for Discrete-Time Nonnegative Dynamical Systems 5.7 Specialization to Discrete-Time Linear Nonnegative Dynamical Systems 5.8 Feedback Interconnections of Discrete-Time Nonnegative Dynamical Systems 5.9 Dissipativity Theory for Nonnegative Dynamical Systems with Time Delay 5.10 Feedback Interconnections of Nonnegative Dynamical Systems with Time Delay 5.11 Dissipativity Theory for Discrete-Time Nonnegative Dynamical Systems with Time Delay 5.12 Feedback Interconnections of Discrete-Time Nonnegative Dynamical Systems with Time Delay Chapter 6. Hybrid Nonnegative and Compartmental Dynamical Systems

106

111 111 112 119 123 127 132 135

143 143 145 153 158 164 166 173 177 183 188 191 194

197

NNBook

September 1, 2009

xi

CONTENTS

6.1 6.2 6.3 6.4 6.5 6.6

Introduction Stability Theory for Nonlinear Hybrid Nonnegative Dynamical Systems Hybrid Compartmental Dynamical Systems Dissipativity Theory for Hybrid Nonnegative Dynamical Systems Specialization to Linear Impulsive Dynamical Systems Feedback Interconnections of Nonlinear Hybrid Nonnegative Dynamical Systems

Chapter 7. System Thermodynamics, Irreversibility, and Time Asymmetry 7.1 Introduction 7.2 Dynamical System Model 7.3 Reversibility, Irreversibility, Recoverability, and Irrecoverability 7.4 Reversible Dynamical Systems, Volume-Preserving Flows, and Poincar´e Recurrence 7.5 System Thermodynamics 7.6 Entropy and Irreversibility 7.7 Semistability and the Entropic Arrow of Time 7.8 Monotonicity of System Energies in Thermodynamic Processes

197 199 203 207 215 217

223 223 226 228 233 243 247 254 259

Chapter 8. Finite-Time Thermodynamics 8.1 Introduction 8.2 Finite-Time Semistability of Nonlinear Nonnegative Dynamical Systems 8.3 Homogeneity and Finite-Time Semistability 8.4 Finite-Time Energy Equipartition in Thermodynamic Systems

263

Chapter 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8

281

9. Modeling and Analysis of Mass-Action Kinetics Introduction Reaction Networks The Law of Mass Action and the Kinetic Equations Nonnegativity of Solutions Realization of Mass-Action Kinetics Reducibility of the Kinetic Equations Stability Analysis of Linear and Nonlinear Kinetics The Zero-Deficiency Theorem

Chapter 10. Semistability and State Equipartition of Nonnegative Dynamical Systems 10.1 Introduction 10.2 Semistability and State Equipartitioning

263 264 268 275

281 282 284 288 290 293 297 301

315 315 315

NNBook

September 1, 2009

xii

CONTENTS

10.3 10.4 Chapter 11.1 11.2 11.3 11.4

Semistability and Equipartition of Linear Compartmental Systems with Time Delay Semistability and Equipartition of Nonlinear Compartmental Systems with Time Delay 11. Robustness of Nonnegative Dynamical Systems Introduction Nominal System Model Semistability and Homogeneous Dynamical Systems Uncertain System Model

Chapter 12. Modeling and Control for Clinical Pharmacology 12.1 Introduction 12.2 Pharmacokinetic Models 12.3 State Space Models 12.4 Drug Action, Effect, and Interaction 12.5 Pharmacokinetic Parameter Estimation 12.6 Pharmacodynamic Models 12.7 Open-Loop Drug Dosing 12.8 Closed-Loop Drug Dosing 12.9 Closed-Loop Control of Cardiovascular Function 12.10 Closed-Loop Control of Anesthesia 12.11 Electroencephalograph-Based Control 12.12 Bispectral Index-Based Control 12.13 Pharmacokinetic and Pharmacodynamic Models for Drug Distribution 12.14 Challenges and Opportunities in Pharmacological Control Chapter 13. Optimal Fixed-Structure Control for Nonnegative Systems 13.1 Introduction 13.2 Optimal Zero Set-Point Regulation for Nonnegative Dynamical Systems 13.3 Optimal Nonzero Set-Point Regulation for Nonnegative Dynamical Systems 13.4 Suboptimal Control for Nonnegative Dynamical Systems 13.5 Optimal Fixed-Structure Control for Nonnegative Dynamical Systems 13.6 Nonnegative Control for Nonnegative Dynamical Systems 13.7 Optimal Fixed-Structure Control for General Anesthesia Chapter 14. H2 Suboptimal Control for Nonnegative Dynamical Systems Using Linear Matrix Inequalities 14.1 Introduction

324 328 343 343 343 347 348 359 359 360 361 362 363 365 366 368 369 371 372 373 374 377

379 379 379 383 390 392 394 396

405 405

NNBook

September 1, 2009

xiii

CONTENTS

14.2 14.3 14.4

H2 Suboptimal Control for Nonnegative Dynamical Systems Suboptimal Estimation for Nonnegative Dynamical Systems H2 Suboptimal Dynamic Controller Design for Nonnegative Dynamical Systems

Chapter 15. Adaptive Control for Nonnegative Systems 15.1 Introduction 15.2 Adaptive Control for Linear Nonnegative Uncertain Dynamical Systems 15.3 Adaptive Control for Linear Nonnegative Dynamical Systems with Nonnegative Control 15.4 Adaptive Control for General Anesthesia: Linear Model 15.5 Adaptive Control for Nonlinear Nonnegative Uncertain Dynamical Systems 15.6 Adaptive Control for General Anesthesia: Nonlinear Model 15.7 Adaptive Control for Nonlinear Nonnegative Uncertain Dynamical Systems 15.8 Adaptive Control for Linear Nonnegative Uncertain Dynamical Systems with Time Delay 15.9 Adaptive Control for Linear Nonnegative Dynamical Systems with Nonnegative Control and Time Delay 15.10 Adaptive Control for Nonnegative Systems with Time Delay and Actuator Amplitude Constraints 15.11 Adaptive Control for General Anesthesia: Linear Model with Time Delay and Actuator Constraints Chapter 16. Adaptive Disturbance Rejection Control for Compartmental Systems 16.1 Introduction 16.2 Compartmental Systems with Exogenous Disturbances 16.3 Adaptive Control for Linear Compartmental Uncertain Systems with Exogenous Disturbances 16.4 Adaptive Control for Linear Compartmental Dynamical Systems with L2 Disturbances 16.5 Adaptive Control for Automated Anesthesia with Hemorrhage and Hemodilution Effects Chapter 17. Limit Cycle Stability Analysis and Control for Respiratory Compartmental Models 17.1 Introduction 17.2 Ultrametric Matrices, Periodic Orbits, and Poincar´e Maps 17.3 Compartmental Modeling of Lung Dynamics: Dichotomy Architecture 17.4 State Space Multicompartment Lung Model

405 413 418 425 425 427 435 438 444 451 454 463 476 480 484

491 491 492 493 502 512

523 523 524 528 531

NNBook

xiv

September 1, 2009

CONTENTS

17.5 17.6 17.7 17.8 17.9

Limit Cycle Analysis of the Multicompartment Lung Model A Regular Dichotomy Model A General Tree Structure Model Direct Adaptive Control for Switched Linear Time-Varying Systems Adaptive Control for a Multicompartment Lung Model

Chapter 18. Identification of Stable Nonnegative and Compartmental Systems 18.1 Introduction 18.2 State Reconstruction 18.3 Constrained Optimization for Subspace Identification of Stable Nonnegative Systems 18.4 Constrained Optimization for Subspace Identification of Compartmental Systems 18.5 Illustrative Numerical Examples

534 538 541 545 548

553 553 554 557 566 567

Chapter 19. Conclusion

571

Bibliography

573

Index

599

NNBook

September 1, 2009

Preface

Nonnegative and compartmental dynamical system models are derived from mass and energy balance considerations that involve dynamic states whose values are nonnegative. These models are widespread in biology, chemistry, ecology, economics, genetics, medicine, sociology, and engineering, and play a key role in the understanding of these disciplines. Specifically, since such disciplines give rise to systems that have numerous input, state, and output properties related to conservation, dissipation, and transport of mass and energy, nonnegative and compartmental models are conceptually simple yet remarkably effective in describing the essential phenomenological features of these dynamical systems. Furthermore, since such systems are governed by conservation laws (e.g., mass, energy, fluid, etc.) and are comprised of homogeneous compartments which exchange variable nonnegative quantities of material via intercompartmental flow laws, these systems are completely analogous to network thermodynamic (advection-diffusion) systems with compartmental masses or energies playing the role of heat and temperatures. Compartmental models have been widely used in biology, pharmacology, and physiology to describe the distribution of a substance (e.g., biomass, drug, radioactive tracer, etc.) among different tissues of an organism. In this case, a compartment represents the amount of the substance inside a particular tissue and the intercompartmental flows are due to diffusion processes. In engineering and the physical sciences, compartments typically represent the energy, mass, or information content of the different parts of the system, and different compartments interact by exchanging heat, work energy, and matter. In ecology and economics, compartments can represent soil and debris, or finished goods and raw materials in different regions, and the flows are due to energy and nutrient exchange (e.g., nitrates, phosphates, carbon, etc.), or money and securities. Compartmental systems can also be used to model chemical reaction systems. In this case, the compartments would represent quantities of different chemical substances contained within the compartment, and the compartmental flows would characterize transformation rates of reactants to products. In this monograph, we develop a unified stability and dissipativity

NNBook

xvi

September 1, 2009

PREFACE

(i.e., conservation, dissipation, and transport) analysis and control design framework for nonnegative and compartmental dynamical systems in order to foster the understanding of these systems as well as advancing the state of the art in active control of nonnegative and compartmental systems. This general framework is then applied to the fields of thermal sciences, biology, chemistry, and medicine to provide a dynamical systems perspective of these diverse disciplines. The monograph is written from a system-theoretic point of view and can be viewed as a contribution to dynamical system and control system theory. After a brief introduction to nonnegative and compartmental dynamical systems in Chapter 1, fundamental stability theory for linear and nonlinear nonnegative and compartmental dynamical systems is developed in Chapter 2. In Chapter 3, we extend the results of Chapter 2 to address nonnegative and compartmental systems with time delay. Chapter 4 provides necessary and sufficient conditions for identifying nonnegative and compartmental systems that admit nonoscillatory and monotonic solutions. A detailed treatment of dissipativity theory and stability of feedback interconnections of nonnegative dynamical systems is given in Chapter 5, whereas extensions of these results to impulsive nonnegative systems are given in Chapter 6. In Chapters 7 and 8 we use compartmental dynamical system theory to provide a system-theoretic foundation for thermodynamics. A detailed treatment of mass-action kinetics is given in Chapter 9, while Chapters 10 and 11 provide extensions to general compartmental models with directed and undirected intercompartmental flows, time delays, and model uncertainty. Next, in Chapters 12–16 we develop a control design framework for nonnegative and compartmental dynamical systems with application to drug dosing control for clinical pharmacology. In Chapter 17, we use compartmental dynamical system theory and Poincar´e maps to model, analyze, and control the dynamics of a pressure-limited respirator and lung mechanics system. Chapter 18 develops a constrained optimization framework for nonnegative and compartmental system identification. Finally, in Chapter 19 we present conclusions. The first author would like to thank James M. Bailey for his valuable discussions on pharmacokinetic and pharmacodynamic modeling in clinical pharmacology over the recent years. In addition, the authors thank Paul Katinas for several insightful and enlightening discussions on the statements quoted in ancient Greek on page vii. In some parts of the monograph we have relied on work we have done jointly with Elias August, James M. Bailey, Dennis S. Bernstein, Sanjay P. Bhat, Behnood Gholami, Tomohisa Hayakawa, Hancao Li, Sergey G. Nersesov, Tanmay Rajpurohit, Jayanthy Ramakrishnan, and Kostyantyn Y. Volyanskyy; it is a pleasure to acknowledge their contributions. The aphorisms by Herakleitos, Empedocles, and Pythagoras quoted in

NNBook

PREFACE

September 1, 2009

xvii

the epigraph of the book give the earliest perception of the complexity of nature and universe. For example, biology has shown that many species of animals such as insect swarms, ungulate flocks, fish schools, ant colonies, and bacterial colonies self-organize in nature. These biological aggregations give rise to remarkably complex global behaviors from simple local interactions between a large number of relatively unintelligent agents without the need for a centralized architecture. The spontaneous development (i.e., selforganization) of these autonomous biological systems and their spatiotemporal evolution to more complex states often appears without any external system interaction. In other words, structural morphing into coherent groups is internal to the system and results from local interactions among subsystem components that are independent of the physical nature of the individual components. These local interactions often comprise a simple set of rules that lead to remarkably complex and robust behaviors. Complexity here refers to the quality of a system wherein interacting subsystems self-organize to form hierarchical evolving structures exhibiting emergent system properties, whereas robustness refers to insensitivity of individual subsystem failures and unplanned behavior at the individual subsystem level. The connection between the local subsystem interactions and the globally complex system behavior is often elusive. This is true for nature in general and was most eloquently stated first by the ancient Greek philosopher Herakleitos in his 123rd fragment—Nature loves to hide (FÔsi krÔptesjai file). Herakleitos’ profound second statement—All matter is exchanged for energy, and energy for all matter (Purì te ‚ntamoib˜ t€ pˆnta kaÈ pÜr pˆntwn)—is a statement of the law of conservation of mass-energy and is a precursor to the principle of relativity. In describing the nature of the universe Herakleitos postulates that nothing can be created out of nothing, and nothing that disappears ceases to exist. This totality of forms, or massenergy equivalence, is eternal and unchangeable in a constantly changing universe (ta pˆuta re). Herakleitos’ last statement defines ultimate wisdom as knowledge and understanding of the intelligence which steers all things through all things. In the language of modern science, this statement defines ultimate wisdom as a fundamental understanding of the universal laws that govern all things and all forces in the universe. Like Herakleitos’ second statement, Empedocles’ statement is one of totality of forms in nature. He postulates that there is no genesis with regard to any of the things in nature but rather a blending and alteration of elements (stoiqea) through attractive and repulsive forces. He further postulates that the organic universe originated from spontaneous aggregations involving pattern interactions by which life emerged through autopoiesis (self-creation). Pythagoras’ statement attempting to explain our incomprehensible universe is as trenchant today as it was two and a half

NNBook

September 1, 2009

xviii

PREFACE

millennia ago, with parts of his statement resonating with creationists, evolution theorists, and intelligent designers. However, Pythagoras spoke of one god, and of God in many forms, and he did so without contradiction. And with God elicited as the universal forces (strong nuclear, electromagnetic, weak nuclear, and gravitational), his statement belongs to the scientist. The results reported in this monograph were obtained at the School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, and the Department of Mechanical, Aerospace, and Biomedical Engineering of the University of Tennessee, Knoxville, between June 2000 and May 2008. The research support provided by the Air Force Office of Scientific Research and the National Science Foundation over the years has been instrumental in allowing us to explore basic research topics that have led to some of the material in this monograph. We are indebted to them for their support.

Atlanta, Georgia, USA, July 2009, Wassim M. Haddad

NNBook

September 1, 2009

Chapter One

Introduction

With the ever-increasing influence of mathematical modeling and engineering on biological, social, and medical sciences, it is not surprising that dynamical system theory has played a central role in the understanding of many biological, ecological, and physiological processes [155, 171, 172, 235]. With this confluence it has rapidly become apparent that mathematical modeling and dynamical system theory are the key threads that tie together these diverse disciplines. The dynamical models of many biological, pharmacological, and physiological processes such as pharmacokinetics [19, 287], metabolic systems [50], epidemic dynamics [155, 157], biochemical reactions [57, 171], endocrine systems [50], and lipoprotein kinetics [171] are derived from mass and energy balance considerations that involve dynamic states whose values are nonnegative. Hence, it follows from physical considerations that the state trajectory of such systems remains in the nonnegative orthant of the state space for nonnegative initial conditions. Such systems are commonly referred to as nonnegative dynamical systems 1 in the literature [79, 164, 166, 233]. A subclass of nonnegative dynamical systems are compartmental systems [4, 5, 29, 43, 88, 100, 134, 152, 155–158, 162, 165, 188, 198, 208, 209, 211, 219, 220, 232, 252, 258, 259, 300]. Compartmental systems involve dynamical models that are characterized by conservation laws (e.g., mass, energy, fluid, etc.) capturing the exchange of material between coupled macroscopic subsystems known as compartments. Each compartment is assumed to be kinetically homogeneous, that is, any material entering the compartment is instantaneously mixed with the material of the compartment. The range of applications of nonnegative systems and compartmental systems is not limited to biological, social, and medical systems. Their usage includes chemical reaction systems [25, 60, 82, 187, 298], queuing systems [301], large-scale systems [274,275], stochastic systems (whose state variables 1 Some authors erroneously refer to nonnegative dynamical systems as positive systems. However, since the state of a nonnegative system can evolve in the nonnegative (closed) orthant of the state space, which is a proper cone (i.e., a closed, convex, solid, and pointed cone), and is not necessarily constrained to the positive (open) orthant of the state space, nonnegative dynamical systems is the appropriate expression for the description of such systems.

NNBook

2

September 1, 2009

CHAPTER 1

represent probabilities) [301], ecological systems [38, 141, 181, 211, 231], economic systems [21], demographic systems [155], telecommunications systems [90], transportation systems, power systems, heat transfer systems, thermodynamic systems [116], and structural vibration systems [175–177], to cite but a few examples. In economic systems the interaction of raw materials, finished goods, and financial resources can be modeled by compartments representing various interacting sectors in a dynamic economy. Similarly, network systems, computer networks, and telecommunications systems are all amenable to compartmental modeling with intercompartmental flow laws governed by nodal dynamics and rerouting strategies that can be controlled to minimize waiting times and optimize system throughput. Compartmental models can also be used to model the interconnecting components of power grid systems with energy flow between regional distribution points subject to control and possible failure. Road, rail, air, and space transport systems also give rise to compartmental systems with interconnections subject to failure and real-time modification. Since the aforementioned dynamical systems have numerous input, state, and output properties related to conservation, dissipation, and transport of mass, energy, or information, nonnegative and compartmental models are conceptually simple yet remarkably effective in describing the essential phenomenological features of these dynamical systems. Furthermore, since such systems are governed by conservation laws and are comprised of homogeneous compartments which exchange variable nonnegative quantities of material via intercompartmental flow laws, these systems are completely analogous to network thermodynamic (advection-diffusion) systems with compartmental masses, energies, or information playing the role of heat and temperatures. The goal of the present monograph is directed toward developing a general stability2 analysis and control design framework for nonlinear nonnegative and compartmental dynamical systems. However, as in general nonlinear systems, nonlinear nonnegative dynamical systems can exhibit a very rich dynamical behavior, such as multiple equilibria, limit cycles, bifurcations, jump resonance phenomena, and chaos, which can make general nonlinear nonnegative system analysis and control notoriously difficult. In addition, since nonnegative and compartmental dynamical systems have specialized structures, nonlinear nonnegative system stabilization has received very little attention in the literature and remains 2 Unlike standard stability theory, stability notions for nonnegative dynamical systems need to be defined with respect to relatively open subsets of the nonnegative orthant of the state space containing the system equilibrium point. See Definition 2.3.

NNBook

INTRODUCTION

September 1, 2009

3

relatively undeveloped. For example, biological and physiological systems typically possess a multiechelon hierarchical hybrid structure characterized by continuous-time dynamics at the lower levels of the hierarchy and discrete-time dynamics (logical decision-making units) at the higher levels of the hierarchy. This is evident in all living systems wherein control structures and hierarchies are present at the intracellular level, the intercellular level, the organs, and the organ system and organism level. Furthermore, biological and physiological systems are self-regulating systems, and hence, they additionally involve feedback (nested or interconnected) subsystems within their hierarchical structures. Finally, the complexity of biological and physiological system modeling and control is further exacerbated when addressing system modeling uncertainty inherent to system biology and physiology. Another complicating factor in the stability analysis of many nonnegative and compartmental dynamical systems is that these systems possess a continuum of equilibria. Since every neighborhood of a nonisolated equilibrium contains another equilibrium, a nonisolated equilibrium cannot be asymptotically stable. Hence, asymptotic stability is not the appropriate notion of stability for systems having a continuum of equilibria. Two notions that are of particular relevance to such systems are convergence and semistability. Convergence is the property whereby every system solution converges to a limit point that may depend on the system initial condition. Semistability is the additional requirement that all solutions converge to limit points that are Lyapunov stable. Semistability for an equilibrium thus implies Lyapunov stability, and is implied by asymptotic stability.3 The dependence of the limiting state on the initial state is seen in numerous stable nonnegative systems and compartmental systems. For these systems, every trajectory that starts in a neighborhood of a Lyapunov stable equilibrium converges to a (possibly different) Lyapunov stable equilibrium, and hence, these systems are semistable. The main objective of this monograph is to develop a general analysis and control design framework for nonnegative and compartmental dynamical systems. The main contents of the monograph are as follows. In Chapter 2, we establish notation and definitions, and develop stability theory for nonnegative and compartmental dynamical systems. Specifically, Lyapunov stability theorems as well as invariant set stability theorems are developed for linear and nonlinear, continuous-time and discrete-time nonnegative and compartmental dynamical systems. Chapter 3 provides an extension of the results of Chapter 2 to nonnegative and compartmental dynamical systems 3 It is important to note that semistability is not merely equivalent to asymptotic stability of the set of equilibria. Indeed, it is possible for a trajectory to converge to the set of equilibria without converging to any one equilibrium point as examples in [34] show.

NNBook

4

September 1, 2009

CHAPTER 1

with time delay. Specifically, stability theorems for linear and nonlinear nonnegative and compartmental dynamical systems with time delay are established using Lyapunov-Krasovskii functionals. Since nonlinear nonnegative and compartmental dynamical systems can exhibit a full range of nonlinear behavior, including bifurcations, limit cycles, and even chaos, in Chapter 4 we present necessary and sufficient conditions for identifying nonnegative and compartmental systems that admit only nonoscillatory and monotonic solutions. As a result, we provide sufficient conditions for the absence of limit cycles in nonlinear compartmental systems. In Chapter 5, using generalized notions of system mass and energy storage, and external flux and energy supply, we present a systematic treatment of dissipativity theory for nonnegative and compartmental dynamical systems. Specifically, using linear and nonlinear storage functions with linear supply rates, we develop new notions of dissipativity theory for nonnegative dynamical systems. In addition, we develop new Kalman-Yakubovich-Popov equations for nonnegative systems for characterizing dissipativeness with linear and nonlinear storage functions and linear supply rates. Finally, these results are used to develop general stability criteria for feedback interconnections of nonnegative dynamical systems. In Chapter 6, we extend the results of Chapters 2 and 5 to develop stability and dissipativity results for impulsive nonnegative and compartmental dynamical systems. Using the concepts developed in Chapters 2, 4, and 5, in Chapter 7 we use compartmental dynamical system theory to provide a systemtheoretic foundation for thermodynamics. Specifically, using a state space formulation, we develop a nonlinear compartmental dynamical system model characterized by energy conservation laws that are consistent with basic thermodynamic principles. In addition, we establish the existence of a unique, continuously differentiable global entropy function for our compartmental thermodynamic model, and using Lyapunov stability theory we show that the proposed thermodynamic model has convergent trajectories to Lyapunov stable equilibria with a uniform energy distribution determined by the system initial energies. Finally, using the system entropy, we establish the absence of Poincar´e recurrence for our thermodynamic model and develop a clear connection between irreversibility, the second law of thermodynamics, and the entropic arrow of time. In Chapter 8, we merge the theories of semistability and finitetime stability [32, 35] to develop a rigorous framework for finite-time thermodynamics. Specifically, using a geometric description of homogeneity theory, we develop intercompartmental flow laws that guarantee finite-time

NNBook

INTRODUCTION

September 1, 2009

5

semistability and energy equipartition for the thermodynamically consistent model developed in Chapter 7. Next, in Chapter 9, we address the problem of nonnegativity, realizability, reducibility, and semistability of chemical reaction networks. Specifically, we show that mass-action kinetics have nonnegative solutions for initially nonnegative concentrations, we provide a general procedure for reducing the dimensionality of the kinetic equations, and we present stability results based upon Lyapunov methods. In Chapter 10, we generalize the results of Chapter 7 to general compartmental systems that account for directional material flow between compartments as well as material in transit between compartments. Specifically, we develop compartmental models that guarantee semistability and state equipartitioning with directed and undirected thermal flow as well as flow delays between compartments. In Chapter 11, we consider robustness extensions of nonnegative dynamical systems; that is, sensitivity of system stability and state equipartitioning in the face of model uncertainty. In Chapters 12–16, we develop a general control design framework for nonnegative and compartmental dynamical systems with application to clinical pharmacology. Specifically, pharmacokinetic and pharmacodynamic models for drug distribution are formulated, and suboptimal, optimal, and adaptive control strategies are developed to address the challenging problem of active control for intraoperative anesthesia. In particular, using a constrained fixed-structure control framework we develop optimal output feedback control laws for nonnegative and compartmental dynamical systems that guarantee that the trajectories of the closed-loop system remain in the nonnegative orthant of the state space for nonnegative initial conditions. Output feedback controllers for compartmental systems with nonnegative inputs are also given. In addition, we develop H2 (sub)optimal controllers for nonnegative dynamical systems using linear matrix inequalities. Finally, a Lyapunov-based direct adaptive control framework is developed for nonnegative systems that guarantees partial asymptotic stability of the closed-loop system, that is, asymptotic stability with respect to part of the closed-loop system states associated with the physiological state variables. The adaptive controllers are constructed without requiring knowledge of the system dynamics or the system disturbances while providing a nonnegative control (source) input for system stabilization. In Chapter 17, we use compartmental dynamical system theory and Poincar´e maps to model, analyze, and control the dynamics of a pressurelimited respirator and lung mechanics system. Chapter 18 develops a constrained optimization framework for nonnegative and compartmental system identification that guarantees asymptotic stability of the plant system dynamics as well as the nonnegativity of the system matrices.

NNBook

6

September 1, 2009

CHAPTER 1

The approach is based on a subspace identification method wherein the resulting constrained optimization problem is cast as a convex linear programming problem with mixed equality, inequality, quadratic, nonnegative, and nonnegative-definite constraints. Finally, we draw conclusions in Chapter 19.

NNBook

September 1, 2009

Chapter Two

Stability Theory for Nonnegative Dynamical Systems

2.1 Introduction Even though numerous results focusing on compartmental systems have been developed in the literature (see [4, 29, 88, 100, 155, 158, 209, 211, 220, 259] and the numerous references therein), the development of nonnegative dynamical systems theory has received far less attention. In this chapter, we develop several basic mathematical results on stability of linear and nonlinear nonnegative dynamical systems. In addition, using linear Lyapunov functions, we develop necessary and sufficient conditions for Lyapunov stability, semistability, that is, system trajectory convergence to Lyapunov stable equilibrium points [31, 47], and asymptotic stability for linear nonnegative dynamical systems. The consideration of a linear Lyapunov function leads to a new Lyapunov-like equation for examining the stability of linear nonnegative systems. This Lyapunov-like equation is analyzed using nonnegative matrix theory [21,145]. The motivation for using a linear Lyapunov function follows from the fact that the state of a nonnegative dynamical system is nonnegative, and hence, a linear Lyapunov function is a valid Lyapunov function candidate. This considerably simplifies the stability analysis of nonnegative dynamical systems. Linear Lyapunov functions were first considered in [158] for compartmental systems and further explored in [25] to study the stability of mass action kinetics which exhibit nonnegative dynamics. For compartmental systems, a linear Lyapunov function corresponds to the total mass of the system.

2.2 Lyapunov Stability Theory for Nonnegative Dynamical Systems In this chapter, we introduce notation, several definitions, and some key results on stability of linear and nonlinear nonnegative dynamical systems needed for developing the main results of this monograph. In a definition or

NNBook

September 1, 2009

8

CHAPTER 2

when a word is defined in the text, the concept defined is italicized. Italics in the running text are also used for emphasis. The definition of a word, phrase, or symbol is to be understood as an “if and only if” statement. Lowercase letters such as x denote vectors, upper-case letters such as A denote matrices, upper-case script letters such as S denote sets, and lower-case Greek letters such as α denote scalars; however, there are a few exceptions to this convention. The notation S1 ⊂ S2 means that S1 is a proper subset of S2 , whereas S1 ⊆ S2 means that either S1 is a proper subset of S2 or S1 is equal to S2 . Throughout the monograph we use two basic types of mathematical statements, namely, existential and universal statements. An existential statement has the form: there exists x ∈ X such that a certain condition C is satisfied; whereas a universal statement has the form: condition C holds for all x ∈ X . For universal statements we often omit the words “for all” and write: condition C holds, x ∈ X . The notation used in this monograph is fairly standard. Specifically, R (respectively, C) denotes the set of real (respectively, complex) numbers, Z+ denotes the set of nonnegative integers, Z+ denotes the set of positive integers, Rn (respectively, Cn ) denotes the set of n × 1 real (respectively, complex) column vectors, Rn×m (respectively, Cn×m ) denotes the set of real (respectively, complex) n × m matrices, (·)T denotes transpose, (·)+ denotes the Moore-Penrose generalized inverse, (·)# denotes the group generalized inverse, (·)D denotes the Drazin inverse, ⊗ denotes Kronecker product, ⊕ denotes Kronecker sum, In or I denotes the n × n identity matrix, and e denotes the ones vector of order n, that is, e = [1, . . . , 1]T . For x ∈ Rq we write x ≥≥ 0 (respectively, x >> 0) to indicate that every component of x is nonnegative (respectively, positive). In this case we say that x is nonnegative or positive, respectively. Likewise, A ∈ Rp×q is nonnegative or positive if every entry of A is nonnegative or positive, respectively, which q is written as A ≥≥ 0 or A >> 0, respectively. Let R+ and Rq+ denote the q nonnegative and positive orthants of Rq , that is, if x ∈ Rq , then x ∈ R+ and x ∈ Rq+ are equivalent, respectively, to x ≥≥ 0 and x >> 0. Furthermore, L2 denotes the space of square-integrable Lebesgue measurable functions on [0, ∞) and L∞ denotes the space of bounded Lebesgue measurable functions on [0, ∞). In addition, we denote the boundary, the interior, and the closure ◦

of the set S by ∂S, S , and S, respectively.

We write k · k for the Euclidean vector norm, k · kF for the Frobenius matrix norm, ||| · ||| for the operator norm of an element in a Banach space B, R(A) and N (A) for the range space and the null space of a matrix A, respectively, spec(A) for the spectrum of the square matrix A including multiplicity, α(A) for the spectral abscissa of A (that is, α(A) = max{Re λ : λ ∈ spec(A)}), and ρ(A) for the spectral radius of A (that is, ρ(A) = max{|λ| : λ ∈ spec(A)}). For vectors x, y ∈ Rp and matrices A, B ∈ Rp×q

NNBook

September 1, 2009

9

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

we use x◦y and A◦B to denote component-by-component and entry-by-entry multiplication, respectively. For a matrix A ∈ Rp×q , rowi (A) and colj (A) denote the ith row and jth column of A, respectively, and rank A denotes the rank of A. Furthermore, we write V ′ (x) for the Fr´echet derivative of V at x, Bε (x), x ∈ Rn , ε > 0, for the open ball centered at x with radius ε, M ≥ 0 (respectively, M > 0) to denote the fact that the Hermitian matrix M is nonnegative (respectively, positive) definite, inf to denote infimum (that is, the greatest lower bound), sup to denote supremum (that is, the least upper bound), and x(t) → M as t → ∞ to denote that x(t) approaches the set M (that is, for each ε > 0 there exists T > 0 such that dist(x(t), M) < ε for all t > T , where dist(p, M) , inf x∈M kp − xk). Finally, the notions of openness, convergence, continuity, and compactness that we use throughout q the monograph refer to the topology generated on R+ (respectively, B) by the norm k · k (respectively, ||| · |||). In this section, we develop the fundamental results of Lyapunov stability theory for nonnegative dynamical systems. We begin by considering the general nonlinear autonomous dynamical system x(t) ˙ = f (x(t)),

x(0) = x0 ,

t ∈ I x0 ,

(2.1)

where x(t) ∈ D ⊆ Rn , t ∈ Ix0 , is the system state vector, D is a relatively open set (see Definition 2.2), f : D → Rn is continuous on D, and Ix0 = [0, τx0 ), 0 ≤ τx0 ≤ ∞, is the maximal interval of existence for the solution x(·) of (2.1). A continuously differentiable function x : Ix0 → D is said to be a solution to (2.1) on the interval Ix0 ⊆ R with initial condition x(0) = x0 if and only if x(t) satisfies (2.1) for all t ∈ Ix0 . We assume that for every initial condition x(0) ∈ D and every τx0 > 0, the dynamical system (2.1) possesses a unique solution x : [0, τx0 ) → D on the interval [0, τx0 ). We denote the solution to (2.1) with initial condition x(0) = x0 by s(·, x0 ), so that the flow of the dynamical system (2.1) given by the map s : [0, τx0 ) × D → D is continuous in x and continuously differentiable in t and satisfies the consistency property s(0, x0 ) = x0 and the semigroup property s(τ, s(t, x0 )) = s(t + τ, x0 ), for all x0 ∈ D and t, τ ∈ [0, τx0 ) such that t + τ ∈ [0, τx0 ). Unless otherwise stated, we assume f (·) is Lipschitz continuous on D. Furthermore, xe ∈ D is an equilibrium point of (2.1) if and only if f (xe ) = 0. In addition, a subset Dc ⊆ D is an invariant set relative to (2.1) if Dc contains the orbits of all its points. Finally, recall that if all solutions to (2.1) are bounded, then it follows from the Peano-Cauchy theorem [112, p. 76] that Ix0 = R. The following definition introduces the notion of essentially nonnegative functions and vector fields. n

Definition 2.1. Let f = [f1 , . . . , fn ]T : D ⊆ R+ → Rn . Then f is

NNBook

September 1, 2009

10

CHAPTER 2 n

essentially nonnegative if fi (x) ≥ 0, for all i = 1, . . . , n, and x ∈ R+ such that xi = 0, where xi denotes the ith component of x. The following definition and technical lemma are needed. n

n

Definition 2.2. A set Q ⊆ R+ is open relative to R+ if there exists an n open set R ⊆ Rn such that Q = R ∩ Rn+ . A set Q ⊆ R+ is closed relative n to R+ if there exists an closed set R ⊆ Rn such that Q = R ∩ Rn+ . A set n n Q ⊆ R+ is compact relative to R+ if there exists an compact set R ⊆ Rn such that Q = R ∩ Rn+ . Lemma 2.1. Consider the nonlinear dynamical system (2.1), and let Dc ⊂ D be closed relative to D. Then the following statements are equivalent: i) For all x ∈ Dc , limh→0+ inf y∈Dc kx + hf (x) − yk/h = 0. ii) Dc is an invariant set with respect to (2.1). Proof. Assume that i) holds. To show ii), let x0 ∈ Dc . Since f (·) is Lipschitz continuous it follows that there exist ε > 0 and L > 0 such that kf (x) − f (y)k ≤ Lkx − yk,

x, y ∈ B2ε (x0 ).

(2.2)

Let T ∈ [0, τx0 ) be such that s(t, x) ∈ B2ε (x0 ) and s(t, y) ∈ B2ε (x0 ) for all t ∈ [0, T ) and x, y ∈ Bε (x0 ). Now, it follows from Gronwall’s lemma [112, p. 81] that, ks(t, x) − s(t, y)k ≤ eLt kx − yk,

x, y ∈ Bε (x0 ),

(2.3)

for all t ∈ [0, T ). Next, let t1 ∈ [0, T ) be such that ks(t, x0 )−x0 k < ε/3 for all t ∈ (0, t1 ), and define ϕ(t) , dist(s(t, x0 ), Dc ) = inf y∈Dc ks(t, x0 ) − yk. Note that since x0 ∈ Dc , it follows that ϕ(0) = 0 and ϕ(t) ≤ ks(t, x0 ) − x0 k < ε/3 for all t ∈ (0, t1 ). Now, let t ∈ (0, t1 ) and let yt ∈ Dc be such that ks(t, x0 ) − yt k − ϕ(t) ≤ ε/3. Hence, kyt − x0 k = ≤ ≤
0 such that t + h ≤ t1 , since ks(t, yt ) − x0 k < ε/3 < ε and ks(h, yt ) − x0 k < ε, it follows from (2.3) that ϕ(t + h) = inf ks(t + h, x0 ), zk z∈Dc

NNBook

September 1, 2009

11

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

≤ inf

n

ks(t + h, x0 ) − s(h, yt )k + ks(h, yt ) − yt − hf (yt )k o +kyt + hf (yt ) − zk z∈Dc

= ks(t + h, x0 ) − s(h, yt )k + ks(h, yt ) − yt − hf (yt )k +dist(yt + hf (yt ), Dc ) ≤ eLh kyt − s(t, x0 )k + ks(h, yt ) − yt − hf (yt )k +dist(yt + hf (yt ), Dc ),

(2.4)

which implies that ϕ(t + h) − ϕ(t) ≤ h



s(h, yt ) − yt eLh − 1 − f (yt ) ϕ(t) +

h h dist(yt + hf (yt ), Dc ) + . h



Now, letting h → 0+ yields lim sup h→0+

ϕ(t + h) − ϕ(t) ≤ Lϕ(t). h

(2.5)

Next, by Gronwall’s lemma [112, p. 81], it follows from (2.5) that 0 ≤ ϕ(t) ≤ eLt ϕ(0), t ∈ (0, t1 ), and hence, since ϕ(0) = 0, it follows that ϕ(t) = 0 for all t ∈ (0, t1 ). Now, since x0 ∈ Dc is arbitrary, it follows that, for every τ1 > 0 such that ϕ(τ1 ) = 0, there exists h > 0 such that ϕ(t) = 0, t ∈ [τ1 , τ1 + h). Next, let τ = inf{t > 0 : ϕ(t) > 0} and suppose, ad absurdum, that τ < τx0 . Since ϕ(t) = 0 for all t ∈ [0, t1 ), it follows that τ ≥ t1 > 0 and, by the definition of τ , ϕ(t) = 0 for all t ∈ [0, τ ) or, equivalently, s(t, x0 ) ∈ Dc for all t ∈ [0, τ ). Hence, since s(τ, x0 ) = limt→τ − s(t, x) and Dc is relatively closed with respect to D, it follows that s(τ, x0 ) ∈ Dc . Therefore, ϕ(τ ) = 0, which implies that there exists h > 0 such that ϕ(t) = 0 for all t ∈ [τ, τ + h), contradicting the definition of τ . Thus, ϕ(t) = 0, t ∈ [0, τx0 ), establishing the result. Conversely, assume Dc is an invariant set with respect to (2.1) so that, for all x0 ∈ Dc and h 6= 0, dist(x0 + hf (x0 ), Dc ) ≤ ks(h, x0 ) − x0 − hf (x0 )k



s(h, x0 ) − x0

− f (x0 ) = |h|

. h

Now, the result follows by letting h → 0+ .

The flow-invariant set result given by Lemma 2.1 was first proved by Brezis [40] and uses the fact that the vector field f in (2.1) is Lipschitz continuous on D. This result was generalized independently by Crandall [61]

NNBook

September 1, 2009

12

CHAPTER 2

and Hartman [131] to the case where f is continuous on D and (2.1) has a unique right maximally defined solution. The case where f = f (t, x) is a time-varying vector field is subsumed under the case f = f (x) by representing the time-varying dynamical system as an autonomous system, where an additional state is appended to represent time. Flow-invariant sets and differential inequalities in normed spaces with Lebesgue measurable and locally essentially bounded vector fields, that is, vector fields that are bounded on a bounded neighborhood of every point excluding sets of measure zero, are discussed in [250]. Specifically, extensions to Lemma 2.1 are addressed for unique absolutely continuous solutions to (2.1) with an essentially one-sided Lipschitz condition on f of the Osgood type [249]. n

Our next result shows that the nonnegative orthant R+ is an invariant set with respect to (2.1) if and only if f is essentially nonnegative. In light of the above discussion, this result holds for the case where f is continuous on D and, more generally, f is measurable and locally essentially bounded on D. n

n

Proposition 2.1. Suppose R+ ⊂ D. Then R+ is an invariant set with respect to (2.1) if and only if f : D → Rn is essentially nonnegative. n

Proof. Define dist(x, R+ ) = inf y∈Rn kx−yk, x ∈ Rn . Now, suppose f : △

+

n

D → Rn is essentially nonnegative and let x ∈ R+ . For every i ∈ {1, . . . , q}, if xi = 0, then xi + hfi (x) = hfi (x) ≥ 0 for all h ≥ 0, whereas, if xi > 0, n then xi + hfi (x) > 0 for all |h| sufficiently small. Thus, x + hf (x) ∈ R+ for n all sufficiently small h > 0, and hence, limh→0+ dist(x + hf (x), R+ )/h = 0. n It now follows from Lemma 2.1, with x(0) = x0 , that x(t) ∈ R+ for all t ∈ [0, τx0 ). n

Conversely, suppose that R+ is invariant with respect to (2.1), let n x(0) ∈ R+ , and suppose, ad absurdum, x is such that there exists i ∈ {1, . . . , q} such that xi (0) = 0 and fi (x(0)) < 0. Then, since f is continuous, there exists sufficiently small h > 0 such that fi (x(t)) < 0 for all t ∈ [0, h), where x(t) is the solution to (2.1). Hence, xi (t) is strictly decreasing on [0, h), n and thus, x(t) 6∈ R+ for all t ∈ (0, h), which leads to a contradiction. It follows from Proposition 2.1 that if x0 ≥≥ 0, then x(t) ≥≥ 0, t ≥ 0, if and only if f is essentially nonnegative. In this case, we say that (2.1) is a nonnegative dynamical system. Henceforth, in this monograph, we assume that f is essentially nonnegative so that the nonlinear dynamical system (2.1) is a nonnegative dynamical system. As we see in Chapters 13, 15, and 16, in order to guarantee the nonnegativity of a controlled closed-loop system state associated with the

NNBook

September 1, 2009

13

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

physical system dynamics it may be necessary to use discontinuous feedback. In this case, the closed-loop system dynamics give rise to a discontinuous vector field. To address the nonnegativity of such systems we present a partial extension to Proposition 2.1 involving nonlinear time-varying dynamical systems of the form x(t) ˙ = f (t, x(t)),

x(t0 ) = x0 ,

t ≥ t0 ,

(2.6)

where x ∈ Rn and f : R × Rn → Rn is such that for every t ∈ R, f (t, ·) is essentially nonnegative. Proposition 2.2. Consider the nonlinear time-varying dynamical system (2.6) where f : R × Rn → Rn is such that for every t ∈ R, f (t, ·) is essentially nonnegative and for every (t0 , x0 ) ∈ R × Rn , (2.6) has a unique solution forward in time. Furthermore, for every (t0 , x0 ) ∈ R × Rn , assume that there exist δ > 0 and an increasing sequence {ti }∞ i=0 such that ti+1 − ti > δ for all i = 0, 1, . . ., and f (·, ·) is continuous at (t, x(t)) for all n t ∈ [ti , ti+1 ), i = 0, 1, . . .. Then R+ is an invariant set with respect to (2.6). Proof. The result is a direct consequence of Proposition 2.1 by equivalently representing the time-varying system (2.6) as an autonomous system by appending another state to represent time. Specifically, let n n x0 ∈ R+ and note that it follows from Proposition 2.1 that x(t) ∈ R+ , n t ∈ [t0 , t1 ). Hence, since x(t) is continuous for all t ≥ t0 , x(t1 ) ∈ R+ . n Now, applying Proposition 2.1 on the interval [t1 , t2 ) we obtain x(t) ∈ R+ , t ∈ [t1 , t2 ). The result now follows by repeating this procedure on every interval [ti , ti+1 ), i = 2, 3, . . .. Note that, if f (t, x) is piecewise continuous in t and essentially nonnegative in x, and (2.6) has a unique solution forward in time, then all the conditions of Proposition 2.2 are trivially satisfied. The following definition introduces several types of stability for the n equilibrium solution x(t) ≡ xe ∈ R+ of the nonlinear nonnegative dynamical system (2.1) for Ix0 = [0, ∞). n

Definition 2.3. i) The equilibrium solution x(t) ≡ xe ∈ R+ to (2.1) is n Lyapunov stable with respect to R+ if, for all ε > 0, there exists δ = δ(ε) > 0 n n such that if x0 ∈ Bδ (xe ) ∩ R+ , then x(t) ∈ Bε (xe ) ∩ R+ , t ≥ 0. n

ii) The equilibrium solution x(t) ≡ xe ∈ R+ to (2.1) is (locally) n asymptotically stable with respect to R+ if it is Lyapunov stable with respect n n to R+ and there exists δ > 0 such that if x0 ∈ Bδ (xe ) ∩ R+ , then limt→∞ x(t) = xe .

NNBook

September 1, 2009

14

CHAPTER 2 n

iii) The equilibrium solution x(t) ≡ xe ∈ R+ to (2.1) is globally n asymptotically stable with respect to R+ if it is Lyapunov stable with respect n n to R+ and, for all x0 ∈ R+ , limt→∞ x(t) = xe . The following result, known as Lyapunov’s direct method, gives sufficient conditions for Lyapunov and asymptotic stability of a nonlinear nonnegative dynamical system. For this result, let V : D → R be a continuously differentiable function with derivative along the trajectories of △ (2.1) given by V˙ (x) = V ′ (x)f (x). Note that V˙ (x) is dependent on the system d V (s(t, x)) t=0 = dynamics (2.1). Since, using the chain rule, V˙ (x) = dt V ′ (x)f (x) it follows that if V˙ (x) is negative, then V (x) decreases along the solution s(t, x0 ) of (2.1) through x0 ∈ D at t = 0. Theorem 2.1 (Lyapunov’s Theorem). Let D be an open subset n relative to R+ that contains xe . Consider the nonlinear dynamical system (2.1) where f is essentially nonnegative and f (xe ) = 0, and assume that there exists a continuously differentiable function V : D → R such that V (xe ) = 0, V (x) > 0, ′ V (x)f (x) ≤ 0,

x ∈ D, x ∈ D.

x 6= xe ,

(2.7) (2.8) (2.9)

Then the equilibrium solution x(t) ≡ xe to (2.1) is Lyapunov stable with n respect to R+ . If, in addition, V ′ (x)f (x) < 0,

x ∈ D,

x 6= xe ,

(2.10)

then the equilibrium solution x(t) ≡ xe to (2.1) is asymptotically stable with n respect to R+ . Finally, if V (·) is such that V (x) → ∞ as kxk → ∞,

(2.11)

then (2.10) implies that the equilibrium solution x(t) ≡ xe to (2.1) is globally n asymptotically stable with respect to R+ . n

n

Proof. Let ε > 0 be such that Bε (xe ) ∩ R+ ⊆ D. Since R+ ∩ ∂Bε (xe ) is compact and V (x), x ∈ D, is continuous, it follows that n △ α = minx∈Rn ∩∂Bε (xe ) V (x) exists. Note α > 0 since xe 6∈ R+ ∩ ∂Bε (xe ) + and V (x) > 0, x ∈ D, x 6= xe . Next, let β ∈ (0, α) and define Dβ to be the arcwise connected component of {x ∈ D : V (x) ≤ β} containing xe ; that is, Dβ is the set of all x ∈ D such that there exists a continuous function ψ : [0, 1] → D such that ψ(0) = x, ψ(1) = xe , and V (ψ(µ)) ≤ β for all n µ ∈ [0, 1].1 Note that Dβ ⊂ R+ ∩ Bε (xe ). To see this, suppose, ad absurdum, 1 Unless otherwise stated, in the remainder of the monograph we assume that sets of the form Dβ = {x ∈ D : V (x) ≤ β} correspond to the arcwise connected component of {x ∈ D : V (x) ≤ β} containing xe . This minor abuse of notation considerably simplifies the presentation.

NNBook

September 1, 2009

15

STABILITY THEORY FOR NONNEGATIVE SYSTEMS n

that Dβ 6⊂ R+ ∩ Bε (xe ). In this case, there exists a point p ∈ Dβ such n that p ∈ R+ ∩ ∂Bε (xe ), and hence, V (p) ≥ α > β, which is a contradiction. △ Now, since V˙ (x) = V ′ (x)f (x) ≤ 0, x ∈ Dβ , it follows that V (x(t)) is a nonincreasing function of time, and hence, V (x(t)) ≤ V (x(0)) ≤ β, t ≥ 0. Hence, Dβ is a positively invariant set (see Definition 2.6) with respect to (2.1). Next, since V (·) is continuous and V (xe ) = 0, there exists δ = δ(ε) ∈ n (0, ε) such that V (x) < β, x ∈ R+ ∩ Bδ (xe ). Now, let x(t), t ≥ 0, satisfy n n n (2.1) with x(0) ∈ R+ ∩ Bδ (xe ). Since, R+ ∩ Bδ (xe ) ⊂ Dβ ⊂ R+ ∩ Bε (xe ) ⊆ D and V ′ (x)f (x) ≤ 0, x ∈ D, it follows that Z t V (x(t)) − V (x(0)) = V ′ (x(s))f (x(s))ds ≤ 0, t ≥ 0, (2.12) and hence, for all x(0) ∈

0 n R+ ∩

Bδ (xe ),

V (x(t)) ≤ V (x(0)) < β, n

t ≥ 0.

Now, since V (x) ≥ α, x ∈ R+ ∩ ∂Bε (xe ), and β ∈ (0, α), it follows that n x(t) 6∈ R+ ∩ ∂Bε (xe ), t ≥ 0. Hence, since f is essentially nonnegative, for n all ε > 0 there exists δ = δ(ε) > 0 such that if x(0) ∈ Bδ (xe ) ∩ R+ , then n x(t) ∈ Bε (xe ) ∩ R+ , t ≥ 0, which proves Lyapunov stability with respect to n R+ of the equilibrium solution x(t) ≡ xe to (2.1). n

To prove asymptotic stability with respect to R+ of the equilibrium solution x(t) ≡ xe to (2.1), suppose that V ′ (x)f (x) < 0, x ∈ D, x 6= xe , n n and x(0) ∈ R+ ∩ Bδ (xe ). Then it follows that x(t) ∈ R+ ∩ Bε (xe ), t ≥ 0. However, V (x(t)), t ≥ 0, is decreasing and bounded from below by zero. Now, ad absurdum, suppose x(t), t ≥ 0, does not converge to xe . This implies that V (x(t)), t ≥ 0, is lower bounded, that is, there exists L > 0 such that V (x(t)) ≥ L > 0, t ≥ 0. Hence, by continuity of V (·) there exists n δ′ > 0 such that V (x) < L for x ∈ R+ ∩ Bδ′ (xe ), which further implies that n △ x(t) 6∈ R+ ∩ Bδ′ (xe ) for all t ≥ 0. Next, define L1 = min{−V ′ (x)f (x) : n δ′ ≤ kx − xe k ≤ ε, x ∈ R+ }. Now, (2.10) implies −V ′ (x)f (x) ≥ L1 , δ′ ≤ n kx − xe k ≤ ε, x ∈ R+ , or, equivalently, Z t V (x(t)) − V (x(0)) = V ′ (x(s))f (x(s))ds ≤ −L1 t, 0

and hence, for all x(0) ∈

n R+

∩ Bδ (xe ),

V (x(t)) ≤ V (x(0)) − L1 t. , it follows that V (x(t)) < L, which is a contradiction. Letting t > V (x(0))−L L1 Hence, x(t) → xe as t → ∞, establishing asymptotic stability with respect

NNBook

September 1, 2009

16

CHAPTER 2 n

to R+ . n

Finally, to prove global asymptotic stability with respect to R+ , let n △ x(0) ∈ R+ , and let β = V (x0 ). Now, the radial unboundedness condition n (2.11) implies that there exists ε > 0 such that V (x) > β for all x ∈ R+ such that kx − xe k ≥ ε. Hence, it follows from (2.12) that V (x(t)) ≤ V (x0 ) = β, n t ≥ 0, which implies that x(t) ∈ R+ ∩ Bε (xe ), t ≥ 0. Now, the proof follows as in the proof of the local asymptotic stability result. A continuously differentiable function V (·) satisfying (2.7) and (2.8) is called a Lyapunov function candidate for the nonnegative dynamical system (2.1). If, additionally, V (·) satisfies (2.9), V (·) is called a Lyapunov function for the nonnegative dynamical system (2.1). Finally, we note that as in standard nonlinear dynamical system theory [112], converse Lyapunov theorems for asymptotically stable nonlinear, nonnegative dynamical systems can also be established. Since the statement and proofs of these results are virtually identical to the standard converse Lyapunov stability proofs for nonlinear dynamical systems, they are not presented here.

2.3 Invariant Set Stability Theorems In this section, we introduce the Krasovskii-LaSalle invariance principle to relax one of the conditions on the Lyapunov function V (·) in the theorems given in Section 2.2. In particular, the strict negative-definiteness condition on the Lyapunov derivative can be relaxed while ensuring system asymptotic stability. Specifically, if a continuously differentiable function defined on a compact invariant set with respect to the nonlinear dynamical system (2.1) can be constructed whose derivative along the system’s trajectories is negative semidefinite and no system trajectories can stay indefinitely at points where the function’s derivative vanishes, then the system’s equilibrium point is asymptotically stable. To state and prove the main results of this section, several definitions and a key theorem are needed. First, we introduce the notion of invariance with respect to the flow of the nonlinear dynamical system (2.1). Given t ∈ R we denote the map s(t, ·) : D → D by st (x0 ). Hence, for a fixed t ∈ R the set of mappings defined by st (x0 ) = s(t, x0 ) for every x0 ∈ D gives the flow of (2.1). In particular, if D0 is a collection of initial conditions such that D0 ⊂ D, then the flow st : D → D is the motion of all points x0 ∈ D0 or, equivalently, the image of D0 ⊂ D under the flow st , that is, st (D0 ) ⊂ D, where st (D0 ) , {y : y = st (x0 ) for some x0 ∈ D0 }. Alternatively, if the initial condition x0 ∈ D is fixed and we let [t0 , t1 ] ⊂ R, then the mapping s(·, x0 ) : [t0 , t1 ] → D defines

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

17

the solution curve or trajectory of the dynamical system (2.1). Hence, the mapping s(·, x0 ) generates a graph in [t0 , t1 ] × D identifying the trajectory corresponding to the motion along a curve through the point x0 in a subset D of the state space. Given x ∈ D, we denote the map s(·, x) : R → D by sx (t). Identifying s(·, x) with its graph, the trajectory or orbit of a point x0 ∈ D is defined as the motion along the curve △

Ox0 = {x ∈ D : x = s(t, x0 ), t ∈ R}.

(2.13)

For t ≥ 0, we define the positive orbit through the point x0 ∈ D as the motion along the curve Ox+0 = {x ∈ D : x = s(t, x0 ), t ≥ 0}. △

(2.14)

Definition 2.4. The trajectory x(t), t ≥ 0, of (2.1) is bounded if there exists γ > 0 such that kx(t)k < γ, t ≥ 0. Definition 2.5. A point p ∈ D is a positive limit point of the trajectory s(·, x) of (2.1) if there exists a monotonic sequence {tn }∞ n=0 of positive numbers, with tn → ∞ as n → ∞, such that s(tn , x) → p as n → ∞. The set of all positive limit points of s(t, x), t ≥ 0, is the positive limit set ω(x) of s(·, x) of (2.1). Definition 2.6. A set M ⊂ D ⊆ Rn is a positively invariant set with respect to the nonlinear dynamical system (2.1) if st (M) ⊆ M for all t ≥ 0, △ where st (M) = {st (x) : x ∈ M}. A set M ⊆ D is an invariant set with respect to the dynamical system (2.1) if st (M) = M for all t ∈ R. Next, we state and prove a key theorem involving positive limit sets. For this result, we use the notation x(t) → M ⊆ D as t → ∞ to denote that x(t) approaches M, that is, for each ε > 0 there exists T > 0 such that △ dist(x(t), M) < ε for all t > T , where dist(p, M) = inf x∈M kp − xk. Theorem 2.2. Consider the nonlinear dynamical system (2.1) where f is essentially nonnegative. Suppose the solution x(t) to (2.1) corresponding n to an initial condition x(0) = x0 ∈ R+ is bounded for all t ≥ 0. Then the positive limit set ω(x0 ) of x(t), t ≥ 0, is a nonempty, compact, invariant, n and connected subset of R+ . Furthermore, x(t) → ω(x0 ) as t → ∞. Proof. Let x(t), t ≥ 0, or, equivalently, s(t, x0 ), t ≥ 0, denote the solution to (2.1) corresponding to the initial condition x(0) = x0 . Next, since x(t) is bounded for all t ≥ 0, it follows from the Bolzano-Weierstrass △ theorem [112, p. 27] that every sequence in the positive orbit Ox+0 = {s(t, x) : t ∈ [0, ∞)} has at least one accumulation point p ∈ D as t → ∞, and hence, ω(x0 ) is nonempty. Next, let p ∈ ω(x0 ) so that there exists an increasing unbounded sequence {tn }∞ n=0 , with t0 = 0, such that limn→∞ x(tn ) = p.

NNBook

18

September 1, 2009

CHAPTER 2

Now, since x(tn ) is uniformly bounded in n it follows that the limit point p is bounded, which implies that ω(x0 ) is bounded. To show that ω(x0 ) is closed let {pi }∞ i=0 be a sequence contained in ω(x0 ) such that limi→∞ pi = p. Now, since pi → p as i → ∞ for every ε > 0, there exists an i such that kp − pi k < ε/2. Next, since pi ∈ ω(x0 ), there exists t ≥ T , where T is arbitrary and finite, such that kpi − x(t)k < ε/2. Now, since kp − pi k < ε/2 and kpi − x(t)k < ε/2 it follows that kp − x(t)k ≤ kpi − x(t)k + kp − pi k < ε, and hence, p ∈ ω(x0 ). Thus, every accumulation point of ω(x0 ) is an element of ω(x0 ) so that ω(x0 ) is closed. Hence, since ω(x0 ) is closed and bounded, ω(x0 ) is compact. To show positive invariance of ω(x0 ) let p ∈ ω(x0 ) so that there exists an increasing unbounded sequence {tn }∞ n=0 such that x(tn ) → p as n → ∞. Now, let s(tn , x0 ) denote the solution x(tn ) of (2.1) with initial condition x(0) = x0 and note that, since f : D → Rn in (2.1) is Lipschitz continuous on D, x(t), t ≥ 0, is the unique solution to (2.1) so that by the semigroup property s(t + tn , x0 ) = s(t, s(tn , x0 )) = s(t, x(tn )). Now, since x(t), t ≥ 0, is continuous it follows that, for t + tn ≥ 0, limn→∞ s(t + tn , x0 ) = limn→∞ s(t, x(tn )) = s(t, p), and hence, s(t, p) ∈ ω(x0 ). Hence, st (ω(x0 )) ⊆ ω(x0 ), t ≥ 0, establishing positive invariance of ω(x0 ). To show invariance of ω(x0 ) let y ∈ ω(x0 ) so that there exists an increasing unbounded sequence {tn }∞ n=0 such that s(tn , x0 ) → y as n → ∞. Next, let t ∈ [0, ∞) and note that there exists N such that tn > t, n ≥ N . Hence, it follows from the semigroup property that s(t, s(tn − t, x0 )) = s(tn , x0 ) → y as n → ∞. Now, it follows from the Bolzano-Lebesgue theorem [112, p. 28] that there exists a subsequence {znk }∞ k=1 of the sequence zn = s(tn − t, x0 ), n = N, N + 1, . . ., such that znk → z ∈ D as k → ∞ and, by definition, z ∈ ω(x0 ). Next, it follows from the continuous dependence property that limk→∞ s(t, znk ) = s(t, limk→∞ znk ), and hence, y = s(t, z), which implies that ω(x0 ) ⊆ st (ω(x0 )), t ∈ [0, ∞). Now, using positive invariance of ω(x0 ) it follows that st (ω(x0 )) = ω(x0 ), t ≥ 0, establishing invariance of the positive limit set ω(x0 ). To show connectedness of ω(x0 ), suppose, ad absurdum, that ω(x0 ) is not connected. In this case, there exist two nonempty closed sets P1+ and P2+ such that P1+ ∩ P2+ = Ø and ω(x0 ) = P1+ ∪ P2+ . Since P1+ and P2+ are closed and disjoint there exist two open sets S1 and S2 such that S1 ∩S2 = Ø, P1+ ⊂ S1 , and P2+ ⊂ S2 . Next, since f : D → R is Lipschitz continuous on D it follows that the solution x(t), t ≥ 0, to (2.1) is a continuous function of ∞ t. Hence, there exist sequences {tn }∞ n=0 and {τn }n=0 such that x(tn ) ∈ S1 , x(τn ) ∈ S2 , and tn < τn < tn+1 , which implies that there exists a sequence {τn }∞ n=0 , with tn < τn < τn+1 , such that x(τn ) 6∈ S1 ∪ S2 . Next, since x(t) is

NNBook

September 1, 2009

19

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

bounded for all t ≥ 0, it follows that x(τn ) → pˆ 6∈ ω(x0 ) as n → ∞, leading to a contradiction. Hence, ω(x0 ) is connected. Finally, to show x(t) → ω(x0 ) as t → ∞, suppose, ad absurdum, x(t) 6→ ω(x0 ) as t → ∞. In this case, there exists a sequence {tn }∞ n=0 , with tn → ∞ as n → ∞, such that inf

p∈ω(x0 )

kx(tn ) − pk > ε,

n ∈ Z+ .

(2.15)

However, since x(t), t ≥ 0, is bounded, the bounded sequence {x(tn )}∞ n=0 ∗ ∗ contains a convergent subsequence {x(t∗n )}∞ n=0 such that x(tn ) → p ∈ ω(x0 ) as n → ∞, which contradicts (2.15). Hence, x(t) → ω(x0 ) as t → ∞ and, n since f is essentially nonnegative, ω(x0 ) ⊆ R+ . Note that Theorem 2.2 implies that if p ∈ D is an ω-limit point of a trajectory s(·, x) of (2.1), then all other points of the trajectory s(·, p) of (2.1) through the point p are also ω-limit points of s(·, x), that is, if p ∈ ω(x) then Op+ ⊂ ω(x). Furthermore, since every equilibrium point xe ∈ D of (2.1) satisfies s(t, xe ) = xe for all t ∈ R, all equilibrium points xe ∈ D of (2.1) are their own ω-limit sets. If a trajectory of (2.1) possesses a unique ω-limit point xe , then it follows from Theorem 2.2 that since ω(xe ) is invariant with respect to the flow st of (2.1), xe is an equilibrium point of (2.1). Next, we present the Krasovskii-LaSalle invariance principle for nonnegative dynamical systems. Theorem 2.3 (Krasovskii-LaSalle Theorem). Consider the nonlinear dynamical system (2.1) where f is essentially nonnegative, assume that Dc ⊂ n D ⊆ R+ is a compact positively invariant set with respect to (2.1), and assume there exists a continuously differentiable function V : Dc → R such △ that V ′ (x)f (x) ≤ 0, x ∈ Dc . Let R = {x ∈ Dc : V ′ (x)f (x) = 0} and let M be the largest invariant set contained in R. If x(0) ∈ Dc , then x(t) → M as t → ∞. Proof. Let x(t), t ≥ 0, be a solution to (2.1) with x(0) ∈ Dc . Since V ′ (x)f (x) ≤ 0, x ∈ Dc , it follows that Z t V (x(t)) − V (x(τ )) = V ′ (x(s))f (x(s))ds ≤ 0, t ≥ τ, τ

and hence V (x(t)) ≤ V (x(τ )), t ≥ τ , which implies that V (x(t)) is a nonincreasing function of t. Next, since V (·) is continuous on the compact set Dc , there exists β ∈ R such that V (x) ≥ β, x ∈ Dc . Hence, △ γx0 = limt→∞ V (x(t)) exists. Now, for all p ∈ ω(x0 ) there exists an increasing unbounded sequence {tn }∞ n=0 , with t0 = 0, such that x(tn ) → p as n → ∞. Since V (x), x ∈ Dc , is continuous, V (p) = V (limn→∞ x(tn )) =

NNBook

September 1, 2009

20

CHAPTER 2

limn→∞ V (x(tn )) = γx0 , and hence, V (x) = γx0 on ω(x0 ). Now, since Dc is compact and positively invariant it follows that x(t), t ≥ 0, is bounded, and hence, it follows from Theorem 2.2 that ω(x0 ) is a nonempty, compact invariant set. Hence, it follows that V ′ (x)f (x) = 0 on ω(x0 ) and thus ω(x0 ) ⊂ M ⊂ R ⊂ Dc . Finally, since x(t) → ω(x0 ) as t → ∞, it follows that x(t) → M as t → ∞. Next, using Theorem 2.3 we provide a generalization of Theorem 2.1 for local asymptotic stability of a nonlinear dynamical system. Corollary 2.1. Consider the nonlinear dynamical system (2.1) where f n is essentially nonnegative, assume that Dc ⊂ D ⊆ R+ is a compact positively invariant set with respect to (2.1) such that xe ∈ Dc , and assume that there exists a continuously differentiable function V : Dc → R such that V (xe ) = 0, V (x) > 0, x 6= xe , and V ′ (x)f (x) ≤ 0, x ∈ Dc . Furthermore, assume that the △ set R = {x ∈ Dc : V ′ (x)f (x) = 0} contains no invariant set other than the set {xe }. Then the equilibrium solution x(t) ≡ xe to (2.1) is asymptotically n stable with respect to R+ . n

Proof. Lyapunov stability with respect to R+ of the equilibrium solution x(t) ≡ xe to (2.1) follows from Theorem 2.1 since V ′ (x)f (x) ≤ 0, x ∈ Dc . Now, it follows from Theorem 2.3 that if x0 ∈ Dc , then ω(x0 ) ⊆ M, where M denotes the largest invariant set contained in R, which implies that M = {xe }. Hence, x(t) → M = {xe } as t → ∞, establishing asymptotic stability of the equilibrium solution x(t) ≡ xe to (2.1) with n respect to R+ . In Theorem 2.3 and Corollary 2.1, we explicitly assumed that there n exists a compact invariant set Dc ⊂ D ⊆ R+ of (2.1). Next, we provide a result that does not require the explicit assumption of the existence of a compact invariant Dc . Theorem 2.4. Consider the nonlinear dynamical system (2.1) where f is essentially nonnegative and assume that there exists a continuously n differentiable function V : R+ → R such that V (xe ) = 0, V (x) > 0, ′

n

V (x)f (x) ≤ 0,

x∈

x∈

n R+ , n R+ .

x 6= xe ,

(2.16) (2.17) (2.18)

Let R = {x ∈ R+ : V ′ (x)f (x) = 0} and let M be the largest invariant set contained in R. Then all solutions x(t), t ≥ 0, of (2.1) that are bounded approach M as t → ∞. △

n

Proof. Let x ∈ R+ be such that trajectory s(t, x), t ≥ 0, of (2.1)

NNBook

September 1, 2009

21

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

n

is bounded. Since f is essentially nonnegative it follows that s(t, x) ∈ R+ , t ≥ 0. Now, with Dc = Ox+ , it follows from Theorem 2.3 that s(t, x) → M as t → ∞. Next, we present the global invariant set theorem for guaranteeing global asymptotic stability of a nonlinear dynamical system. Theorem 2.5. Consider the nonlinear dynamical system (2.1) where f is essentially nonnegative and assume there exists a continuously differenn tiable function V : R+ → R such that V (xe ) = 0, V (x) > 0, ′

x∈

n R+ , n R+ ,

x 6= xe ,

V (x)f (x) ≤ 0, x ∈ V (x) → ∞ as kxk → ∞.

(2.19) (2.20) (2.21) (2.22)

n

Furthermore, assume that the set R = {x ∈ R+ : V ′ (x)f (x) = 0} contains no invariant set other than the set {xe }. Then the equilibrium solution n x(t) ≡ xe to (2.1) is globally asymptotically stable with respect to R+ . △

Proof. Since (2.19)–(2.21) hold, it follows from Theorem 2.1 that the equilibrium solution x(t) ≡ xe to (2.1) is Lyapunov stable with respect n to R+ , while the radial unboundedness condition (2.22) implies that all solutions to (2.1) are bounded. Now, Theorem 2.4 implies that x(t) → M as t → ∞. However, since R contains no invariant set other than the set {xe }, the set M is {xe }, and hence, global asymptotic stability with respect n to R+ is immediate.

2.4 Semistability of Nonnegative Dynamical Systems As discussed in Chapter 1, numerous nonnegative and compartmental systems give rise to systems that posses a continuum of equilibria. In this section, we develop a stability analysis framework for systems having a continuum of equilibria. Since every neighborhood of a nonisolated equilibrium contains another equilibrium, a nonisolated equilibrium cannot be asymptotically stable. Hence, asymptotic stability is not the appropriate notion of stability for systems having a continuum of equilibria. Two notions that are of particular relevance to such systems are convergence and semistability. Convergence is the property whereby every system solution converges to a limit point that may depend on the system initial condition. Semistability is the additional requirement that all solutions converge to limit points that are Lyapunov stable. Semistability for an equilibrium thus implies Lyapunov stability, and is implied by asymptotic stability.

NNBook

22

September 1, 2009

CHAPTER 2

It is important to note that semistability is not merely equivalent to asymptotic stability of the set of equilibria. Indeed, it is possible for a trajectory to converge to the set of equilibria without converging to any one equilibrium point [34]. Conversely, semistability does not imply that the equilibrium set is asymptotically stable in any accepted sense. This is because stability of sets (see [112]) is defined in terms of distance (especially in case of noncompact sets), and it is possible to construct examples in which the dynamical system is semistable, but the domain of semistability (see Definition 2.8) contains no ε-neighborhood (defined in terms of the distance) of the (noncompact) equilibrium set, thus ruling out asymptotic stability of the equilibrium set. Hence, semistability and set stability of the equilibrium set are independent notions. The dependence of the limiting state on the initial state is seen in numerous dynamical systems including compartmental systems [158] which arise in chemical kinetics [25], biomedical [155], environmental [231], economic [21], power [274], and thermodynamic systems [116]. For these systems, every trajectory that starts in a neighborhood of a Lyapunov stable equilibrium converges to a (possibly different) Lyapunov stable equilibrium, and hence these systems are semistable. Semistability is especially pertinent to networks of dynamic agents which exhibit convergence to a state of consensus in which the agents agree on certain quantities of interest [150]. Semistability was first introduced in [47] for linear systems, and applied to matrix second-order systems in [24]. References [34] and [33] consider semistability of nonlinear systems, and give several stability results for systems having a continuum of equilibria based on nontangency and arc length of trajectories, respectively. In this section, we develop necessary and sufficient conditions for semistability. We say that the dynamical system (2.1) is convergent with n n respect to R+ if limt→∞ s(t, x) exists for every x ∈ R+ . The following proposition gives a sufficient condition for a trajectory of (2.1) to converge to a limit. Proposition 2.3. Consider the nonlinear dynamical system (2.1) where n f is essentially nonnegative and let x ∈ R+ . If the positive limit set ω(x) of n (2.1) contains a Lyapunov stable (with respect to R+ ) equilibrium point y, then y = limt→∞ s(t, x), that is, ω(x) = {y}. n

Proof. Suppose y ∈ ω(x) is Lyapunov stable with respect to R+ and let n Nε ⊆ R+ be a relatively open neighborhood of y. Since y is Lyapunov stable n n with respect to R+ , there exists a relatively open neighborhood Nδ ⊂ R+ of y such that st (Nδ ) ⊆ Nε for every t ≥ 0. Now, since y ∈ ω(x), it follows that there exists τ ≥ 0 such that s(τ, x) ∈ Nδ . Hence, s(t + τ, x) = st (s(τ, x)) ∈

NNBook

September 1, 2009

23

STABILITY THEORY FOR NONNEGATIVE SYSTEMS n

st (Nδ ) ⊆ Nε for every t > 0. Since Nε ⊆ R+ is arbitrary, it follows that y = limt→∞ s(t, x). Thus, limn→∞ s(tn , x) = y for every sequence {tn }∞ n=1 , and hence, ω(x) = {y}. The following definitions and key proposition are necessary for the main results of this section. n

Definition 2.7. An equilibrium solution x(t) ≡ xe ∈ R+ to (2.1) is n n semistable with respect to R+ if it is Lyapunov stable with respect to R+ + and there exists δ > 0 such that if x0 ∈ Bδ (xe ) ∩ Rn , then limt→∞ x(t) exists n and corresponds to a Lyapunov stable equilibrium point with respect to R+ . n An equilibrium point xe ∈ R is a globally semistable equilibrium with respect n n + to R+ if it is Lyapunov stable with respect to R+ and, for every x0 ∈ Rn , limt→∞ x(t) exists and corresponds to a Lyapunov stable equilibrium point n with respect to R+ . The system (2.1) is said to be Lyapunov stable with n respect to R+ if every equilibrium point of (2.1) is Lyapunov stable with n n respect to R+ . The system (2.1) is said to be semistable with respect to R+ n if every equilibrium point of (2.1) is semistable with respect to R+ . Finally, n (2.1) is said to be globally semistable with respect to R+ if every equilibrium n point of (2.1) is globally semistable with respect to R+ . n

Definition 2.8. The domain of semistability with respect to R+ is the n set of points x0 ∈ R+ such that if x(t) is a solution to (2.1) with x(0) = n x0 , t ≥ 0, then x(t) converges to a Lyapunov stable (with respect to R+ ) n equilibrium point in R+ . Next, we present alternative equivalent characterizations of semistability of (2.1). For this result recall the definitions of class K and class L functions [112, p. 162]. Proposition 2.4. Consider the nonlinear dynamical system G given by (2.1) where f is essentially nonnegative. Then the following statements are equivalent: n

i) G is semistable with respect to R+ .

ii) For each xe ∈ f −1 (0), there exist class K and L functions α(·) and β(·), respectively, and δ = δ(xe ) > 0, such that if kx0 − xe k < δ, then kx(t) − xe k ≤ α(kx0 − xe k), t ≥ 0, and dist(x(t), f −1 (0)) ≤ β(t), t ≥ 0.

iii) For each xe ∈ f −1 (0), there exist class K functions α1 (·) and α2 (·), a class L function β(·), and δ = δ(xe ) > 0, such that if kx0 − xe k < δ, then dist(x(t), f −1 (0)) ≤ α1 (kx(t) − xe k)β(t) ≤ α2 (kx0 − xe k)β(t), t ≥ 0.

NNBook

24

September 1, 2009

CHAPTER 2

Proof. To show that i) implies ii), suppose (2.1) is semistable and let xe ∈ f −1 (0). It follows from Lemma 4.5 of [174] that there exist δ = δ(xe ) > 0 and a class K function α(·) such that if kx0 − xe k ≤ δ, then kx(t)−xe k ≤ α(kx0 −xe k), t ≥ 0. Without loss of generality, we may assume that δ is such that Bδ (xe ) is contained in the domain of semistability of (2.1). Hence, for every x0 ∈ Bδ (xe ), limt→∞ x(t) = x∗ ∈ f −1 (0) and, consequently, limt→∞ dist(x(t), f −1 (0)) = 0. For each ε > 0 and x0 ∈ Bδ (xe ), define Tx0 (ε) to be the infimum of T with the property that dist(x(t), f −1 (0)) < ε for all t ≥ T , that is, Tx0 (ε) , inf{T : dist(x(t), f −1 (0)) < ε, t ≥ T }. For each x0 ∈ Bδ (xe ), the function Tx0 (ε) is nonnegative and nonincreasing in ε, and Tx0 (ε) = 0 for sufficiently large ε. Next, let T (ε) , sup{Tx0 (ε) : x0 ∈ Bδ (xe )}. We claim that T is well defined. To show this, consider ε > 0 and x0 ∈ Bδ (xe ). Since dist(s(t, x0 ), f −1 (0)) < ε for every t > Tx0 (ε), it follows from the continuity of s that, for every η > 0, there exists an open neighborhood U of x0 such that dist(s(t, z), f −1 (0)) < ε for every z ∈ U . Hence, lim supz→x0 Tz (ε) ≤ Tx0 (ε), implying that the function x0 7→ Tx0 (ε) is upper semicontinuous at the arbitrarily chosen point x0 , and hence on Bδ (xe ). Since an upper semicontinuous function defined on a compact set achieves its supremum, it follows that T (ε) is well defined. The function T (·) is the pointwise supremum of a collection of nonnegative and nonincreasing functions, and is hence nonnegative and nonincreasing. Moreover, T (ε) = 0 for every ε > max{α(kx0 − xe k) : x0 ∈ Bδ (xe )}. Rε Let ψ(ε) , 2ε ε/2 T (σ)dσ + 1ε ≥ T (ε) + 1ε . The function ψ(ε) is positive, continuous, strictly decreasing, and ψ(ε) → 0 as ε → ∞. Choose β(·) = ψ −1 (·). Then β(·) is positive, continuous, strictly decreasing, and β(σ) → 0 as σ → ∞. Furthermore, T (β(σ)) < ψ(β(σ)) = σ. Hence, dist(x(t), f −1 (0)) ≤ β(t), t ≥ 0. Next, to show that ii) implies iii), suppose ii) holds and let xe ∈ f −1 (0). Then it follows from Lemma 4.5 of [174] that xe is Lyapunov stable. Choosing x0 sufficiently close to xe , it follows from the inequality kx(t) − xe k ≤ α(kx0 − xe k), t ≥ 0, that trajectories of (2.1) starting sufficiently close to xe are bounded, and hence, the positive limit set of (2.1) is nonempty. Since limt→∞ dist(x(t), f −1 (0)) = 0, it follows that the positive limit set is contained in f −1 (0). Now, since every point in f −1 (0) is Lyapunov stable, it follows from Proposition 2.3 that limt→∞ x(t) = x∗ , where x∗ ∈ f −1 (0) is Lyapunov stable. If x∗ = xe , then it follows using similar arguments as ˆ such that dist(x(t), f −1 (0)) ≤ above that there exists a class L function β(·)

NNBook

September 1, 2009

25

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

ˆ for every x0 satisfying kx0 − xe k < δ and t ≥ 0. Hence, kx(t) − xe k ≤ β(t) q p ˆ t ≥ 0. Next, consider the case dist(x(t), f −1 (0)) ≤ kx(t) − xe k β(t), ∗ where x 6= xe and let α1 (·) be a class K function. In this case, note that limt→∞ dist(x(t), f −1 (0))/α1 (kx(t) − xe k) = 0, and hence, it follows using similar arguments as above that there exists a class L function β(·) such that dist(x(t), f −1 (0)) ≤ α1 (kx(t) − xe k)β(t), t ≥ 0. Finally, note that α1 ◦ α is of class K (by Lemma 4.5 of [174]), and hence, iii) follows immediately. Finally, to show that iii) implies i), suppose iii) holds and let xe ∈ f −1 (0). Then it follows that α1 (kx(t) − xe k) ≤ α2 (kx(0) − xe k), t ≥ 0, that is, kx(t) − xe k ≤ α(kx(0) − xe k), where t ≥ 0 and α = α−1 1 ◦ α2 is of class K (by Lemma 4.2 of [174]). It now follows from Lemma 4.5 of [174] that xe is Lyapunov stable. Since xe was chosen arbitrarily, it follows that every equilibrium point is Lyapunov stable. Furthermore, limt→∞ dist(x(t), f −1 (0)) = 0. Choosing x0 sufficiently close to xe , it follows from the inequality kx(t) − xe k ≤ α(kx0 − xe k), t ≥ 0, that trajectories of (2.1) starting sufficiently close to xe are bounded, and hence, the positive limit set of (2.1) is nonempty. Since every point in f −1 (0) is Lyapunov stable, it follows from Proposition 2.3 that limt→∞ x(t) = x∗ , where x∗ ∈ f −1 (0) is Lyapunov stable. Hence, by definition, (2.1) is semistable. Next, we present a sufficient condition for semistability with respect n to R+ . Theorem 2.6. Consider the nonlinear dynamical system (2.1) where n f is essentially nonnegative. Let Q ⊆ R+ be a relatively open neighborhood n of f −1 (0) ∩ R+ and assume that there exists a continuously differentiable function V : Q → R such that V ′ (x)f (x) < 0,

x ∈ Q\f −1 (0).

(2.23)

n

If (2.1) is Lyapunov stable with respect to R+ , then (2.1) is semistable with n respect to R+ . Proof. Since (2.1) is Lyapunov stable by assumption, for every z ∈ f −1 (0), there exists a relatively open neighborhood Vz S of z such that s([0, ∞) × Vz ) is bounded and contained in Q. The set V , z∈f −1 (0) Vz is a relatively open neighborhood of f −1 (0) contained in Q. Consider x ∈ V so that there exists z ∈ f −1 (0) such that x ∈ Vz and s(t, x) ∈ Vz , t ≥ 0. Since Vz is bounded it follows that the positive limit set of x is nonempty and invariant. Furthermore, it follows from (2.23) that V˙ (s(t, x)) ≤ 0, t ≥ 0, and hence, it follows from Theorem 2.3 that s(t, x) → M as t → ∞, where M is the largest invariant set contained in the set R = {y ∈ Vz : V ′ (y)f (y) = 0}. Note that R = f −1 (0) is invariant, and hence, M = R, which implies

NNBook

September 1, 2009

26

CHAPTER 2

that limt→∞ dist(s(t, x), f −1 (0)) = 0. Finally, since every point in f −1 (0) is Lyapunov stable, it follows from Proposition 2.3 that limt→∞ s(t, x) = x∗ , where x∗ ∈ f −1 (0) is Lyapunov stable. Hence, by definition, (2.1) is semistable. Next, we present a slightly more general theorem for semistability wherein we do not assume that all points in the zero-level set of the derivative of V , that is, V˙ −1 (0), are Lyapunov stable but rather we assume that all points in the largest invariant subset of V˙ −1 (0) are Lyapunov stable. Theorem 2.7. Consider the nonlinear dynamical system (2.1) where f n is essentially nonnegative, and let Q ⊆ R+ be a relatively open neighborhood n of f −1 (0) ∩ R+ . Suppose the orbit Ox of (2.1) is bounded for all x ∈ Q and assume that there exists a continuously differentiable function V : Q → R such that V ′ (x)f (x) ≤ 0,

x ∈ Q.

(2.24)

If every point in the largest invariant subset M of {x ∈ Q : V ′ (x)f (x) = 0} n is Lyapunov stable with respect to R+ , then (2.1) is semistable with respect n to R+ . Proof. Since every solution of (2.1) is bounded, it follows from the hypotheses on V (·) that, for every x ∈ Q, the positive limit set ω(x) of (2.1) is nonempty and contained in the largest invariant subset M of {x ∈ Q : V ′ (x)f (x) = 0}. Since every point in M is a Lyapunov stable equilibrium, it follows from Proposition 2.3 that ω(x) contains a single point for every x ∈ Q and limt→∞ s(t, x) exists for every x ∈ Q. Now, since limt→∞ s(t, x) ∈ M is Lyapunov stable for every x ∈ Q, semistability is immediate. Example 2.1. Consider the nonlinear dynamical system given by x˙ 1 (t) = σ12 (x2 (t)) − σ21 (x1 (t)), x˙ 2 (t) = σ21 (x1 (t)) − σ12 (x2 (t)),

x1 (0) = x10 , x2 (0) = x20 ,

t ≥ 0,

(2.25) (2.26)

where x1 , x2 ∈ R+ , σij (·), i, j = 1, 2, i 6= j, are Lipschitz continuous, σ12 (x2 ) − σ21 (x1 ) = 0 if and only if x1 = x2 , and (x1 − x2 )(σ12 (x2 ) − 2 σ21 (x1 )) ≤ 0, x1 , x2 ∈ R+ . Note that f −1 (0) = {(x1 , x2 ) ∈ R+ : x1 = x2 = α, α ∈ R+ }. To show that (2.25) and (2.26) is a semistable system, consider the Lyapunov function candidate V (x1 , x2 ) = 12 (x1 − α)2 + 12 (x2 − α)2 , where α ∈ R+ . Now, it follows that V˙ (x1 , x2 ) = (x1 − α)[σ12 (x2 ) − σ21 (x1 )] + (x2 − α)[σ21 (x1 ) − σ12 (x2 )] = x1 [σ12 (x2 ) − σ21 (x1 )] + x2 [σ21 (x1 ) − σ12 (x2 )]

NNBook

September 1, 2009

27

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

= (x1 − x2 )[σ12 (x2 ) − σ21 (x1 )] ≤ 0, (x1 , x2 ) ∈ R+ × R+ ,

(2.27)

which implies that x1 = x2 = α is Lyapunov stable. 2

2

Finally, let R , {(x1 , x2 ) ∈ R+ : V˙ (x1 , x2 ) = 0} = {(x1 , x2 ) ∈ R+ : x1 = x2 = α, α ∈ R+ }. Since R consists of equilibrium points, it follows that M = R. Hence, for every x1 (0), x2 (0) ∈ R+ , (x1 (t), x2 (t)) → M as t → ∞. Hence, it follows from Theorem 2.7 that x1 = x2 = α is semistable △ for all α ∈ R+ . Next, we provide a converse Lyapunov theorem for semistability, which holds with a smooth (i.e., infinitely differentiable) Lyapunov function. Theorem 2.8. Consider the system (2.1). Suppose (2.1) is semistable with the domain of semistability D0 . Then there exist a smooth nonnegative function V : D0 → R+ and a class K∞ function α(·) such that i) V (x) = 0, x ∈ f −1 (0), ii) V (x) ≥ α(dist(x, f −1 (0))), x ∈ D0 , and iii) V ′ (x)f (x) < 0, x ∈ D0 \f −1 (0). Proof. For R t any given solution x(t) of (2.1), the change of time variable from t to τ = 0 (1 + kf (x(s))k)ds results in the dynamical system f (¯ x(τ )) d¯ x = , dτ 1 + kf (¯ x(τ ))k

x ¯(0) = x0 ,

τ ≥ 0,

(2.28)

where x ¯(τ ) = x(t). With a slight abuse of notation, let s¯(t, x), t ≥ 0, denote the solution of (2.28) starting from x ∈ D0 . Note that (2.28) implies that k¯ s(t, x) − s¯(τ, x)k ≤ |t − τ |, x ∈ D0 , t, τ ≥ 0. Next, define the function U : D0 → R+ by   1 + 2t −1 dist(¯ U (x) , sup s(t, x), f (0)) , 1+t t≥0

x ∈ D0 .

(2.29)

Note that U (·) is well defined since (2.28) is semistable. Clearly, i) holds with V (·) replaced by U (·). Furthermore, since U (x) ≥ dist(x, f −1 (0)), x ∈ D0 , it follows that ii) holds with V (·) replaced by U (·). To show that U (·) is continuous on D0 \f −1 (0), define T : D0 \f −1 (0) → [0, ∞) by T (z) , inf{h : dist(¯ s(t, z), f −1 (0)) < dist(z, f −1 (0))/2 for all t ≥ h > 0}, and denote Wε , {x ∈ D0 : dist(x, f −1 (0)) < ε}. Note that Wε ⊃ f −1 (0) is open. Consider z ∈ D0 \f −1 (0) and define λ , dist(z, f −1 (0)) > 0 and let xe , limt→∞ s¯(t, z). Since xe is Lyapunov stable, it follows that there exists a relatively open neighborhood V of xe such that all solutions of (2.28) in V remain in Wλ/2 . Since xe is semistable, it follows that there

NNBook

September 1, 2009

28

CHAPTER 2

exists h > 0 such that s¯(h, z) ∈ V. Consequently, s¯(h + t, z) ∈ Wλ/2 for all t ≥ 0, and hence, it follows that T (z) is well defined. Next, by continuity of solutions of (2.28) on compact time intervals, it follows that there exists a neighborhood U of z such that U ∩ f −1 (0) = Ø and s¯(T (z), y) ∈ V for all y ∈ U . Now, it follows from the choice of V that s¯(T (z) + t, y) ∈ Wλ/2 for all t ≥ 0 and y ∈ U . Then, for every t > T (z) and y ∈ U , [(1 + 2t)/(1 + t)]dist(¯ s(t, y), f −1 (0)) ≤ 2dist(¯ s(t, y), f −1 (0)) ≤ λ. Therefore, for each y ∈ U ,   1 + 2t −1 dist(¯ U (z) − U (y) = sup s(t, z), f (0)) 1+t t≥0   1 + 2t −1 dist(¯ − sup s(t, y), f (0)) 1+t t≥0   1 + 2t −1 dist(¯ = sup s(t, z), f (0)) 1+t 0≤t≤T (z)   1 + 2t −1 dist(¯ − sup s(t, y), f (0)) . (2.30) 1+t 0≤t≤T (z) Hence, 1 + 2t dist(¯ |U (z) − U (y)| ≤ sup s(t, z), f −1 (0)) 0≤t≤T (z) 1 + t
s(t, y), f −1 (0)) −dist(¯ s(t, z), f −1 (0)) ≤ 2 sup dist(¯ 0≤t≤T (z)

−dist(¯ s(t, y), f −1 (0)) s(t, z), s¯(t, y)), ≤ 2 sup dist(¯ 0≤t≤T (z)

z ∈ D0 \f −1 (0),

y ∈ U.

(2.31)

Now, it follows from the continuous dependence of solutions s¯(·, ·) on system initial conditions (Theorem 3.4 of Chapter I of [127]) and (2.31) that U (·) is continuous at z. Furthermore, it follows from (2.31) that, for every sufficiently small h > 0, |U (¯ s(h, z)) − U (z)| ≤ 2

sup 0≤t≤T (z)

= 2

sup 0≤t≤T (z)

≤ 2h,

k¯ s(t, s¯(h, z)) − s¯(t, z)k k¯ s(t + h, z) − s¯(t, z)k

which implies that |U˙ (z)| ≤ 2. Since z ∈ D0 \f −1 (0) was chosen arbitrarily, it follows that U (·) is continuous, |U˙ (·)| ≤ 2, and T (·) is well defined on D0 \f −1 (0).

NNBook

September 1, 2009

29

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

To show that U (·) is continuous on f −1 (0), consider xe ∈ f −1 (0). −1 (0) that converges to x . Since x Let {xn }∞ e e n=1 be a sequence in D0 \f is Lyapunov stable, it follows from Lemma 4.5 of [174] that x(t) ≡ xe is the unique solution to (2.28) with x0 = xe . By the continuous dependence of solutions s¯(·, ·) on system initial conditions (Theorem 3.4 of Chapter I of [127]), s¯(t, xn ) → s¯(t, xe ) = xe as n → ∞, t ≥ 0. Let ε > 0 and note that it follows from ii) of Proposition 3.1 that there exists δ = δ(xe ) > 0 such that, for every solution of (2.28) in Bδ (xe ), there exists Tˆ = Tˆ(xe , ε) > 0 such that s¯t (Bδ (xe )) ⊂ Wε for all t ≥ Tˆ. Next, note that there exists a positive integer N1 such that xn ∈ Bδ (xe ) for all n ≥ N1 . Now, it follows from (2.29) that for n ≥ N1 , U (xn ) ≤ 2 sup dist(¯ s(t, xn ), f −1 (0)) + 2ε.

(2.32)

0≤t≤Tˆ

Next, it follows from Lemma 3.1 of Chapter I of [127] that s¯(·, xn ) converges to s¯(·, xe ) uniformly on [0, Tˆ]. Hence, s(t, xn ), f −1 (0)) = sup dist( lim s¯(t, xn ), f −1 (0)) lim sup dist(¯

n→∞

0≤t≤Tˆ

0≤t≤Tˆ

n→∞

= sup dist(xe , f −1 (0)) 0≤t≤Tˆ

= 0, which implies that there exists a positive integer N2 = N2 (xe , ε) ≥ N1 such that sup0≤t≤Tˆ dist(¯ s(t, xn ), f −1 (0)) < ε for all n ≥ N2 . Combining (2.32) with the above result yields U (xn ) < 4ε for all n ≥ N2 , which implies that limn→∞ U (xn ) = 0 = U (xe ). Next, we show that U (¯ x(τ )) is strictly decreasing along the solution of (2.28) on D\f −1 (0). Note that for every x ∈ D0 \f −1 (0) and 0 < h ≤ 1/2 such that s¯(h, x) ∈ D0 \f −1 (0), it follows from the arguments preceding (2.30) that, for sufficiently small h, the supremum in the definition of U (¯ s(h, x)) is reached at some time tˆ such that 0 ≤ tˆ ≤ T (x). Hence, 1 + 2tˆ U (¯ s(h, x)) = dist(¯ s(tˆ + h, x), f −1 (0)) 1 + tˆ 1 + 2tˆ + 2h = dist(¯ s(tˆ + h, x), f −1 (0)) 1 + tˆ + h  h · 1− (1 + 2tˆ + 2h)(1 + tˆ)   h ≤ U (x) 1 − , 2(1 + T (x))2

(2.33)

which implies that U˙ (x) ≤ − 12 U (x)(1 + T (x))−2 < 0, x ∈ D0 \f −1 (0), and

NNBook

30

September 1, 2009

CHAPTER 2

hence, iii) holds with V (·) replaced by U (·). The function U (·) now satisfies all of the conditions of the theorem except for smoothness. To obtain smoothness, note that since |U˙ (x)| ≤ 2 for every x ∈ D0 , it follows that U˙ (x) satisfies a boundedness condition in the sense of Wilson [310]. By Theorem 2.5 of [310], there exists a smooth function W : D0 \f −1 (0) → R satisfying |W (x)−U (x)| < 14 U (x)(1+T (x))−2 < 12 U (x) ˙ (x) ≤ − 1 U (x)(1 + T (x))−2 < 0 for x ∈ D0 \f −1 (0). Next, we extend and W 4 W (·) to all of D0 by taking W (z) = 0 for z ∈ f −1 (0). Now, W (·) is a continuous Lyapunov function which is smooth on D0 \f −1 (0). Taking −2 V (x) = W (x)e−(W (x)) , and noting that W (x) > 12 U (x) > 12 dist(x, f −1 (0)), 2 x ∈ D0 \f −1 (0), so that V (·) satisfies ii) with α(r) , (r/2)e−4/r , we obtain the desired smooth Lyapunov function.

2.5 Stability Theory for Linear Nonnegative Dynamical Systems Linear nonnegative dynamical systems are of major importance in biological and physiological systems. For example, almost the entire field of distribution of tracer-labeled materials in steady-state systems can be captured by linear nonnegative dynamical systems. In this section, we develop necessary and sufficient conditions for stability of linear nonnegative dynamical systems. First, however, we introduce several definitions and some key results concerning nonnegative matrices [21, 22, 145, 218] that are necessary for developing the main results of this section. Definition 2.9. Let T > 0. A real function u : [0, T ] → Rm is a nonnegative (respectively, positive) function if u(t) ≥≥ 0 (respectively, u(t) >> 0) on the interval [0, T ]. Definition 2.10. Let A ∈ Rn×n . A is a Z-matrix if A(i,j) ≤ 0, i, j = 1, . . . , n, i 6= j. A is an M-matrix (respectively, a nonsingular M-matrix) if A is a Z-matrix and Re λ ≥ 0 (respectively, Re λ > 0) for all λ ∈ spec(A). A is essentially nonnegative if −A is a Z-matrix, that is, A(i,j) ≥ 0, i, j = 1, . . . , n, i 6= j. A is compartmental if A is essentially nonnegative and P n T i=1 A(i,j) ≤ 0, j = 1, . . . , n, or, equivalently, A e ≤≤ 0. Finally, A is 2 nonnegative (respectively, positive) if A(i,j) ≥ 0 (respectively, A(i,j) > 0), i, j = 1, 2, . . . , n. The following results are needed for developing several stability results for linear nonnegative dynamical systems. Theorem 2.9. Let A ∈ Rn×n be such that A ≥≥ 0. Then ρ(A) ∈ 2 In this monograph it is important to distinguish between a nonnegative (respectively, positive) matrix and a nonnegative-definite (respectively, positive-definite) matrix.

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

31

spec(A) and there exists x ≥≥ 0, x 6= 0, such that Ax = ρ(A)x. Proof. See [144, p. 503]. Lemma 2.2. Assume A ∈ Rn×n is a Z-matrix. Then the following statements are equivalent: i ) A is an M-matrix. ii ) Re λ ≥ 0, λ ∈ spec(A). iii ) There exist a scalar α > 0 and an n × n nonnegative matrix B ≥≥ 0 such that α ≥ ρ(B) and A = αI − B. iv ) If λ ∈ spec(A), then either λ = 0 or Re λ > 0. Furthermore, the following statements are equivalent: v ) A is a nonsingular M-matrix. vi ) det A 6= 0 and A−1 ≥≥ 0.

vii ) For every y ∈ Rn , y ≥≥ 0, there exists a unique x ∈ Rn , x ≥≥ 0, such that Ax = y. viii ) There exists x ∈ Rn , x ≥≥ 0, such that Ax >> 0. ix ) There exists x ∈ Rn , x >> 0, such that Ax >> 0. Proof. The equivalence of i) and ii) is by definition. To show that ii) implies iii), note that since A is a Z-matrix there exist α > 0 and B ≥≥ 0 such that A = αI − B. Next, it follows from Theorem 2.9 that ρ(B) ∈ spec(B) so that α − ρ(B) ∈ spec(A). Now, it follows from ii) that α ≥ ρ(B). Next, to show that iii) implies ii), let λ ∈ spec(A) and since A = αI − B there exists β ∈ spec(B) such that λ = α − β. Now, since α ≥ ρ(B) it follows that Re λ = α − Re β ≥ α − ρ(B) ≥ 0. To show that ii) implies iv), note that if λ ∈ spec(A), then Re λ ≥ 0. Suppose, there exists λ ∈ spec(A) such that Re λ = 0, that is, λ = ω for some ω ∈ R. Since ii) implies iii) it follows that there exist a scalar α > 0 and an n × n nonnegative matrix B ≥≥ 0 such that α ≥ ρ(B) and A = αI − B. Hence, there exists β ∈ spec(B) such that λ = α − β, which implies that 0 = Re λ = α − Re β ≥ α − ρ(B) ≥ 0, which further implies that α = β or, equivalently, λ = 0. Hence, either Re λ > 0 or λ = 0. Statement ii) follows trivially from iv), establishing the equivalence of ii) and iv).

NNBook

September 1, 2009

32

CHAPTER 2

To show that v) implies vi), note that since A is an M-matrix it follows from iii) that there exist α > 0 and B ≥≥ 0 such that A = αI − B and α ≥ ρ(B). Next, since A is nonsingular and α − ρ(B) ∈ spec(A) it follows that α > ρ(B) or, equivalently, ρ(B/α) < 1. Hence, I − α1 B is invertible P 1 k and (I − α1 B)−1 = ∞ k=0 ( α B) ≥≥ 0. The result now follows by noting that 1 1 −1 −1 A = α (I − α B) . To show that vi) implies vii), let y ≥≥ 0 and note that since A is nonsingular there exists a unique x such that Ax = y. Now, since A−1 ≥≥ 0 it follows that x = A−1 y ≥≥ 0. To show that vii) implies viii), let y = e and note that it follows from vii) that there exists x ≥≥ 0 such that Ax = y = e >> 0. To show that viii) implies ix), let α > 0 and B ≥≥ 0 be such that A = αI − B. Now, letting x ≥≥ 0 be such that 0 > Bx ≥≥ 0, establishing the result. To show that ix) implies v), let x >> 0 be such that Ax >> 0 and consider the linear dynamical system y(t) ˙ = −AT y(t),

y(0) = y0 ≥≥ 0,

t ≥ 0.

(2.34)

Note that since A is a Z-matrix it follows that f (y) = −AT y is essentially nonnegative. Hence, with V (y) = xT y it follows from Theorem 2.1 that T y(t) = e−A t y0 → 0 as t → ∞ for every y0 ≥≥ 0, which implies that −AT is Hurwitz. Now, the result is immediate by noting that −AT is Hurwitz if and only if Re λ > 0 for all λ ∈ spec(A) (see Definition 2.11). Note that if f (x) = Ax, where A ∈ Rn×n , then f is essentially nonnegative if and only if A is essentially nonnegative. To address linear nonnegative dynamical systems consider (2.1) with f (x) = Ax so that x(t) ˙ = Ax(t),

x(0) = x0 ,

t ≥ 0,

(2.35)

where x(t) ∈ Rn , t ≥ 0, and A ∈ Rn×n . The solution to (2.35) is standard and is given by x(t) = eAt x(0), t ≥ 0. The following proposition shows that A is essentially nonnegative if and only if the state transition matrix eAt is nonnegative on [0, ∞). Proposition 2.5. Let A ∈ Rn×n . Then A is essentially nonnegative if and only if eAt is nonnegative for all t ≥ 0. Furthermore, if A is essentially nonnegative and x0 ≥≥ 0, then x(t) ≥≥ 0, t ≥ 0, where x(t), t ≥ 0, denotes the solution to (2.35). Proof. The proof is a direct consequence of Proposition 2.1 with

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

33

f (x) = Ax. The proof can also be shown using matrix mathematics [29]. To prove necessity, note that, since A is essentially nonnegative it follows that △ Aα = A + αI is nonnegative, where α , − min{A(1,1) , . . . , A(n,n) }. Hence, eAα t = e(A+αI)t ≥≥ 0, t ≥ 0, and thus eAt = e−αt eAα t ≥≥ 0, t ≥ 0. absurdum, there Conversely, suppose eAt ≥≥ 0, t ≥ 0, and assume, adP −1 k k exist i, j such that i 6= j and A(i,j) < 0. Now, since eAt = ∞ i=1 (k!) A t , it follows that [eAt ](i,j) = I(i,j) + tA(i,j) + O(t2 ), where O(t)/t → 0 as t → 0. Thus, as t → 0 and i 6= j, it follows that [eAt ](i,j) < 0 for some t sufficiently small, which leads to a contradiction. Hence, A is essentially nonnegative. Finally, if A is essentially nonnegative and x0 ≥≥ 0, then x(t) = eAt x0 ≥≥ 0, t ≥ 0, is immediate. Since it follows from Proposition 2.5 that the linear dynamical system given by (2.35) is nonnegative if and only if A is essentially nonnegative, henceforth we assume that A is essentially nonnegative. Definition 2.11. Let A ∈ Rn×n . Then: i ) A is Lyapunov stable if spec(A) ⊂ {s ∈ C : Re s ≤ 0} and, if λ ∈ spec(A) and Re λ = 0, then λ is semisimple. ii ) A is semistable if spec(A) ⊂ {s ∈ C : Re s < 0} ∪ {0} and, if 0 ∈ spec(A), then 0 is semisimple. iii ) A is asymptotically stable or Hurwitz if spec(A) ⊂ {s ∈ C : Re s < 0}. The following proposition concerning Lyapunov stability, semistability, and asymptotic stability of (2.35) is immediate. This result holds whether or not A is an essentially nonnegative matrix. Proposition 2.6 ([23]). Let A ∈ Rn×n and consider the linear dynamical system (2.35). Then the following statements are equivalent: i ) xe = 0 is a Lyapunov-stable equilibrium of (2.35). ii ) At least one equilibrium of (2.35) is Lyapunov stable. iii ) Every equilibrium of (2.35) is Lyapunov stable. iv ) A is Lyapunov stable. v ) For every initial condition x(0) ∈ Rn , x(t) is bounded for all t ≥ 0.

vi ) keAt k is bounded for all t ≥ 0, where k · k is a matrix norm on Rn×n .

NNBook

September 1, 2009

34

CHAPTER 2

vii ) For every initial condition x(0) ∈ Rn , eAt x(0) is bounded for all t ≥ 0. The following statements are equivalent: viii ) A is semistable. ix ) limt→∞ eAt exists. In fact, limt→∞ eAt = I − AA# . x ) For every initial condition x(0) ∈ Rn , limt→∞ x(t) exists. The following statements are equivalent: xi ) xe = 0 is an asymptotically stable equilibrium of (2.35). xii) A is asymptotically stable. xiii ) α(A) < 0. xiv ) For every initial condition x(0) ∈ Rn , limt→∞ x(t) = 0. xv ) For every initial condition x(0) ∈ Rn , limt→∞ eAt x(0) = 0. xvi ) limt→∞ eAt = 0. The following theorem gives several properties of a nonnegative dynamical system when a Lyapunov-like equation is satisfied for (2.35). Note that it follows from Proposition 2.6 that if A is asymptotically stable, then N (A) = {0}. Theorem 2.10. Let A ∈ Rn×n be essentially nonnegative. If there exist vectors p, r ∈ Rn such that p >> 0 and r ≥≥ 0 satisfy 0 = AT p + r,

then the following statements hold: i ) −A is an M-matrix. ii ) If λ ∈ spec(A), then either Re λ < 0 or λ = 0. iii ) A is semistable and limt→∞ eAt = I − AA# ≥≥ 0. iv ) R(A) = N (I − AA# ) and N (A) = R(I − AA# ). Rt v ) 0 eAσ ds = A# (eAt − I) + (I − AA# )t, t ≥ 0.

vi ) A is nonsingular if and only if −A is a nonsingular M-matrix.

(2.36)

NNBook

September 1, 2009

35

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

vii ) If A is nonsingular, then A is asymptotically stable and A−1 ≤≤ 0. Proof. i) Consider the linear dynamical system x(t) ˙ = Ax(t),

x(0) = x0 ≥≥ 0,

t ≥ 0.

(2.37)

Note that since A is an essentially nonnegative matrix it follows that f (x) = Ax is essentially nonnegative. Hence, with V (x) = pT x it follows from Theorem 2.1 that the zero solution to (2.37) is Lyapunov stable, and hence, Re λ ≤ 0 for all λ ∈ spec(A). Now, i) follows by noting A is essentillay nonnegative if and only if −A is a Z-matrix. ii) The proof is a consequence of i)–iv) of Lemma 2.2. iii) It follows from i) and ii) that A is Lyapunov stable and Re λ < 0 or λ = 0 for all λ ∈ spec(A). Hence, the eigenvalue λ = 0 (if it exists) is semisimple and it follows from the Jordan decomposition that there exist invertable matrices J ∈ Rr×r , where r = rank A, and S ∈ Rn×n such that   J 0 A=S S −1 , 0 0 and J is Hurwitz. Hence, it follows that  Jt  e 0 At lim e = lim S S −1 0 In−r t→∞ t→∞   0 0 = S S −1 0 In−r    −1  J 0 J 0 −1 = In − S S S S −1 0 0 0 0 = In − AA# .

Next, since A is essentially nonnegative, it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0, which implies that I − AA# ≥≥ 0. iv) Let x ∈ R(A), that is, there exists y ∈ Rn such that x = Ay. Now, (I − AA# )x = x − AA# Ay = x − Ay = 0, which implies that R(A) ⊆ N (I − AA# ). Conversely, let x ∈ N (I − AA# ). Hence, (I − AA# )x = 0, or, equivalently, x = AA# x, which implies that x ∈ R(A), and hence, proves R(A) = N (I − AA# ). The equality N (A) = R(I − AA# ) can be proved in an analogous manner.   J 0 v) Note that A = S S −1 , and hence, 0 0  Z t Z t  Jσ e 0 Aσ e dσ = S S −1 dσ 0 In−r 0 0

NNBook

September 1, 2009

36

CHAPTER 2

= S

 Rt 0

eJσ dσ 0 Rt 0 I 0 n−r dσ



S −1

 J −1 (eJt − I) 0 = S S −1 0 In−r t   −1   Jt J 0 (e − Ir ) 0 −1 S −1 = S S S 0 0 0 0   0 0 +S S −1 0 In−r t 

= A# (eAt − In ) + (In − AA# )t,

t ≥ 0.

vi) The result follows from i). vii) Asymptotic stability of A is a direct consequence of ii). A−1 ≤≤ 0 follows from vi) of Lemma 2.2. It follows from Proposition 2.5 and iii) of Theorem 2.10 that lim x(t) = (I − AA# )x0 ≥≥ 0.

t→∞

Hence, the set of all equilibria of a semistable linear nonnegative dynamical system lies in N (A) = R(I − AA# ). Next, we show that linear compartmental dynamical systems [4,29,88, 100, 155, 158, 209, 211, 220, 259] are a special case of nonnegative dynamical systems. To see this, let xi (t), i = 1, . . . , n, denote the mass (and hence a nonnegative quantity) of the ith subsystem of the compartmental system shown in Figure 2.1, let aii ≥ 0 denote the loss coefficient of the ith subsystem, let wi (t) ≥ 0, i = 1, . . . , n, denote the flux (mass inflow) supplied to the ith subsystem, and let φij (t), i 6= j, i, j = 1, . . . , n, denote the net mass flow (or flux) from the jth subsystem to the ith subsystem given by φij (t) = aij xj (t) − aji xi (t), t ≥ 0, where the transfer coefficient aij ≥ 0, i 6= j, i, j = 1, . . . , n. A mass balance for the whole compartmental system yields x˙ i (t) = −aii xi (t) +

n X

φij (t) + wi (t),

j=1,i6=j

t ≥ 0,

i = 1, . . . , n,

(2.38)

or, equivalently, x(t) ˙ = Ax(t) + w(t),

x(0) = x0 ,

t ≥ 0,

(2.39)

where x(t) = [x1 (t), . . . , xn (t)]T , w(t) = [w1 (t), . . . , wn (t)]T , and for i, j =

NNBook

September 1, 2009

37

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

wi (t)

wj (t)

' ?

ith Subsystem xi (t) &

$ ) %

aij xj (t)

aji xi (t)

'

$

?

jth Subsystem xj (t) 1&

aii xi (t) ?

%

ajj xj (t) ?

Figure 2.1 Linear compartmental interconnected subsystem model.

1, . . . , n, A(i,j) =



−

Pn

k=1 aki , aij ,

i = j, i 6= j.

(2.40)

Thus, a compartmental system (with w(t) ≡ 0) satisfies x˙ i (t) ≤ 0 for all t ≥ 0 whenever xj (t) = 0 for all j 6= i and t ≥ 0, while a nonnegative system (with w(t) ≡ 0) satisfies x˙ i (t) ≥ 0 for all t ≥ 0 whenever xi (t) = 0 and xj (t) ≥ 0 for all j 6= i and t ≥ 0. Note that A is an essentially nonnegative matrix and hence the compartmental model given by (2.38) is a nonnegative dynamical system. Furthermore, note that AT e = [−a11 , −a22 , . . . , −ann ]T , and hence, with p = e and r = −AT e ≥≥ 0, it follows that (2.36) is satisfied, which implies that the compartmental model given by (2.38) (with w(t) ≡ 0) is semistable if A is singular and P asymptotically stable if A is nonsingular. In both cases, V (x) = eT x = ni=1 xi denoting the total mass of the system serves as a Lyapunov function for the undisturbed (w(t) ≡ 0) system (2.38) P n with V˙ (x) = eT Ax = − ni=1 aii xi ≤ 0, x ∈ R+ .

The compartmental system (2.38) with no inflows, that is, wi (t) ≡ 0, i = 1, . . . , n, is said to be inflow-closed [155]. Alternatively, if (2.38) possesses no losses (outflows) it is said to be outflow-closed [155]. A compartmental system is said to be closed if it is inflow-closed and outflown closed. Note that for a closed system V˙ (x) = 0, x ∈ R+ , which shows that the total mass inside a closed system is conserved. Alternatively, in the case where aii 6= 0 and wi (t) 6= 0, i = 1, . . . , n, it follows that (2.38) can be equivalently written as   ∂V T + w(t), x(0) = x0 , x(t) ˙ = [Jn (x) − D(x)] t ≥ 0, (2.41) ∂x where Jn (x) is a skew-symmetric matrix function with Jn(i,i) (x) = 0 and Jn(i,j) (x) = aij xj − aji xi , i 6= j, and D(x) = diag[a11 x1 , a22 x2 , . . . , ann xn ] n ≥≥ 0, x ∈ R+ . Hence, a linear compartmental system is a port-controlled

NNBook

September 1, 2009

38

CHAPTER 2 n

Hamiltonian system [203] with a Hamiltonian H(x) = V (x) = eT x, x ∈ R+ , n representing the total mass in the system, Jn (x), x ∈ R+ , representing the n internal system interconnection structure, D(x), x ∈ R+ , representing the outflow dissipation, and w(t) representing the supplied flux to the system. This observation shows that closed compartmental systems are conservative systems. This will be further elaborated in Chapter 5. Finally, we note that semistability of an inflow-closed compartmental system is equivalent to the existence of a trap in the system. A trap [158] is a compartment or a set of compartments which is outflow-closed and from which there are no transfers to any compartments outside the trap. A simple trap is a trap that has no traps inside it. Reference [83] shows that the compartmental matrix given by (2.40) has a zero eigenvalue if and only if the compartmental system has a trap. Furthermore, [84] shows that the algebraic multiplicity of λ = 0, where λ ∈ spec(A), corresponds to the number of simple traps in the system. Now, using the construction developed in the proof of iii) of Theorem 2.10, it can be shown that the algebraic and geometric multiplicity of λ = 0 ∈ spec(A) are equal. Hence, all solutions of inflow-closed linear compartmental systems are bounded and convergent. Next, motivated by the fact that for a compartmental system the total mass in the system can serve as a Lyapunov function, we give necessary and sufficient conditions for Lyapunov stability, semistability, and asymptotic stability for linear nonnegative dynamical systems using linear Lyapunov functions. Theorem 2.11. Consider the linear dynamical system given by (2.35) where A ∈ Rn×n is essentially nonnegative. Then the following statements hold: i ) A is Lyapunov stable if and only if A is semistable. ii ) If there exist vectors p, r ∈ Rn such that p >> 0 and r ≥≥ 0 satisfy 0 = AT p + r,

(2.42)

then A is semistable (and hence Lyapunov stable). iii ) If A is semistable, then there exist vectors p, r ∈ Rn such that p ≥≥ 0 and r ≥≥ 0 satisfy (2.42). iv ) If there exist vectors p, r ∈ Rn such that p ≥≥ 0 and r ≥≥ 0 satisfy (2.42) and (A, r T ) is observable, then p >> 0 and (2.35) is asymptotically stable.

Furthermore, the following statements are equivalent:

NNBook

September 1, 2009

39

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

v ) A is asymptotically stable. vi ) There exist vectors p, r ∈ Rn such that p >> 0 and r >> 0 satisfy (2.42). vii ) There exist vectors p, r ∈ Rn such that p ≥≥ 0 and r >> 0 satisfy (2.42). viii ) For every r ∈ Rn such that r >> 0, there exists p ∈ Rn such that p >> 0 satisfies (2.42). Proof. i) If A is semistable, then A is Lyapunov stable by definition. Conversely, suppose A is Lyapunov stable and essentially nonnegative. Then, it follows from ii) of Lemma 2.2 that −A is an M-matrix. Now, it follows from iv) of Lemma 2.2 that the real part of each nonzero λ ∈ spec(A) is negative, and hence, Re λ < 0 or λ = 0. This proves the equivalence between Lyapunov stability and semistability. ii) The proof is a direct consequence of iv) of Theorem 2.10. Alternatively, consider the linear Lyapunov function candidate V (x) = pT x. n Note that V (0) = 0 and V (x) > 0, x ∈ R+ , x 6= 0. Now, computing the Lyapunov derivative yields △ V˙ (x) = V ′ (x)Ax = pT Ax = −r T x ≤ 0,

n

x ∈ R+ ,

establishing Lyapunov stability. Semistability now follows from i). iii) If A is semistable, it follows as in the proof of i) that −AT is an M-matrix. Hence, it follows from ii) of Lemma 2.2 that there exist a scalar α > 0 and a nonnegative matrix B ≥≥ 0 such that α ≥ ρ(B) and AT = B − αI. Now, since B ≥≥ 0, it follows from Theorem 2.9 that ρ(B) ∈ spec(B), and hence, there exists p ≥≥ 0 such that Bp = ρ(B)p. Thus, AT p = Bp − αp = (ρ(B) − α)p ≤≤ 0, which proves that there exist p ≥≥ 0 and r ≥≥ 0 such that (2.42) holds. iv) Assume there exist p ≥≥ 0 and r ≥≥ 0 such that (2.42) holds and suppose (A, r T ) is observable. Now, consider the function V (x) = pT x, x ∈ n n R+ , and note that, since V (x) ≥ 0, x ∈ R+ , and V˙ (x) = pT Ax = −r T x ≤ 0, n △ it follows that if x(0) ∈ P = {x ∈ R+ : pT x = 0}, then V (x(t)) = 0, t ≥ 0, = 0. Specifically, which implies that dV (x(t)) dt dV (x(t)) = pT Ax(0) = 0. dt t=0 Hence, if x ˆ ∈ P then V˙ (ˆ x) = pT Aˆ x = −r T x ˆ = 0. Thus, if x ˆ ∈ P then n △ T x ∈ P it follows Aˆ x ∈ P and x ˆ ∈ Q = {x ∈ R+ : r x = 0}. Now, since Aˆ

NNBook

40

September 1, 2009

CHAPTER 2

that A2 x ˆ ∈ P and Aˆ x ∈ Q. Repeating these arguments yields Ak x ˆ ∈ Q, T k k = 0, 1, . . . , n, or, equivalently, r A x ˆ = 0, k = 1, 2, . . . , n. Now, since (A, r T ) is observable it follows that x ˆ = 0 and P = {0}, which implies that p >> 0. Asymptotic stability of (2.35) now follows as a direct consequence of the Krasovskii-LaSalle invariant set theorem with V (x) = pT x and using the fact that (A, r T ) is observable. To show the equivalence among v)–viii) first suppose there exist p ≥≥ 0 and r >> 0 such that (2.42) holds. Now, there exists sufficiently small ε > 0 such that AT (p + εe) > 0, which proves that vii) implies vi). Since vi) implies vii) it trivially follows that vi) and vii) are equivalent. Now, suppose vi) holds, that is, there exist p >> 0 and r >> 0 such that (2.42) holds, and consider the Lyapunov function candidate n V (x) = pT x, x ∈ R+ . Computing the Lyapunov derivative yields V˙ (x) = pT Ax = −r T x < 0, x 6= 0, and hence, it follows that (2.35) is asymptotically stable. Thus, vi) implies v). Next, suppose (2.35) is asymptotically stable. △ Hence, −A−T ≥≥ 0 and thus for every r ∈ Rn+ , p = −A−T r ≥≥ 0 satisfies (2.42), which proves that v) implies vii). Finally, suppose (2.35) is asymptotically stable. Now, as in the proof n given above, for every r ∈ Rn+ , there exists p ∈ R+ such that (2.42) holds. n Next, suppose, ad absurdum, there exists x ∈ R+ , x 6= 0, such that xT p = 0, that is, there exists at least one i ∈ {1, 2, . . . , n} such that pi = 0. Hence, −xT A−T r = 0. However, since −AT ≥≥ 0 it follows that −A−1 x ≥≥ 0 and, since r >> 0, it follows that −A−1 x = 0, which implies that x = 0 yielding a contradiction. Hence, for every r ∈ Rn+ , there exists p ∈ Rn+ such that (2.42) holds, which proves that v) implies viii). Since viii) implies vi) trivially, the equivalence of v)–viii) is established. Next, using Theorem 2.11, we show that every asymptotically stable linear nonnegative system is equivalent, modulo a similarity transformation, to a compartmental system. Proposition 2.7. Let A ∈ Rn×n be essentially nonnegative and asymptotically stable. Then there exist an invertible matrix S ∈ Rn×n and a matrix Aˆ ∈ Rn×n such that Aˆ = SAS −1 , Aˆ(i,j) ≥ 0, i 6= j, and Pn ˆ k=1 A(k,j) ≤ 0, i, j = 1, . . . , n.

Proof. It follows from v) and vi) of Theorem 2.11 that there exists △ p ∈ Rn+ such that AT p 0 and diagonal, implies that AˆT e > 0 satisfies AT p > 0, it follows that vec R >> 0, and hence, viii) of Theorem 2.11 implies that vec P >> 0, which establishes that P >> 0.

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

43

2.6 Nonlinear Compartmental Dynamical Systems Many applications in life sciences give rise to nonlinear compartmental dynamical systems. These include metobolic pathways, membrane transports, pharmacodynamics, epidemiology, and ecology. In this section, we develop linearization results for nonlinear nonnegative dynamical systems as well as several stability results for nonlinear compartmental dynamical systems. First, we show that if a nonlinear system is nonnegative, then its linearization is also nonnegative. Proposition 2.10. Consider the nonlinear dynamical system (2.1) where f (0) = 0 and f : D → Rn is essentially nonnegative and continuously n △ ∂f is essentially nonnegative. differentiable in R+ . Then, A = ∂x (x) x=0

Proof. Since f : D → Rn is essentially nonnegative it follows that n fi (x)|xi =0 ≥ 0, x ∈ R+ . Now, note that for all i 6= j, ∂fi (x) A(i,j) = ∂xj x=0 fi (0, . . . , h, . . . , 0) − fi (0) = lim+ h h→0 fi (0, . . . , h, . . . , 0) , = lim+ h h→0 where h in fi (0, . . . , h, . . . , 0) is in the jth location, which implies that fi (0, . . . , h, . . . , 0) ≥ 0. Hence, A(i,j) ≥ 0, i 6= j, which proves essential nonnegativity of A. Next, we present a key result on stability of a linearized nonlinear nonnegative dynamical system. First, however, note that the definition of a domain of attraction can be extended to nonlinear nonnegative dynamical n systems by restricting the domain to the nonnegative orthant R+ . Theorem 2.14. Let x(t) ≡ xe be an equilibrium point for the nonlinear dynamical system (2.1). Furthermore, let f : D → Rn be essentially nonnegative and let ∂f (x) A= . ∂x x=xe

Then the following statements hold:

i ) If Re λ < 0, where λ ∈ spec(A), then the equilibrium solution x(t) ≡ xe of the nonlinear dynamical system (2.1) is asymptotically stable. ii ) If there exists λ ∈ spec(A) such that Re λ > 0, then the equilibrium

NNBook

September 1, 2009

44

CHAPTER 2

wi (t) ' ?

ith Subsystem xi (t) &

wj (t) $ ) %

a ˆij (x(t))

a ˆji (x(t))

a ˆii (x(t)) ?

'

?

jth Subsystem xj (t) 1&

$ %

a ˆjj (x(t)) ?

Figure 2.2 Nonlinear compartmental interconnected subsystem model.

solution x(t) ≡ xe of the nonlinear dynamical system (2.1) is unstable. iii) Let xe = 0, let Re λ < 0, where λ ∈ spec(A), let p >> 0 be such that n △ △ AT p P n △ 0 : pT f (x) < 0, x ∈ R+ , kxk < ε} and kxk = ni=1 pi |xi |. Then DA is a subset of the domain of attraction for (2.1). Proof. Statements i) and ii) are restatements of Lyapunov’s indirect method [112] as applied to nonlinear nonnegative systems. To prove iii), note that it follows from Lemma 2.10 that if f : D → Rn is essentially nonnegative, then A is essentially nonnegative. Hence, since Re λ < 0, where λ ∈ spec(A), it follows from vi) of Theorem 2.11 that there exists p >> 0 such that AT p 0, n x ∈ R+ , x 6= 0, and by (2.47), the zero solution x(t) ≡ 0 to (2.45) (with w(t) ≡ 0) is asymptotically stable. As in the linear case, nonlinear compartmental systems are portcontrolled Hamiltonian systems. This follows from the fact that (2.44) can be equivalently written as x(t) ˙ = [Jn (x) − D(x)]



∂V ∂x

T

+ w(t),

x(0) = x0 ,

t ≥ 0,

(2.48)

where Jn (x) is a skew-symmetric matrix function with Jn(i,i) (x) = 0 and Jn(i,j) (x) = a ˆij (x) − a ˆji (x), i 6= j, and D(x) = diag[ˆ a11 (x), a ˆ22 (x), . . ., n a ˆnn (x)] ≥≥ 0, x ∈ R+ . n

Finally, if f : R+ → Rn in (2.45) is r times continuously differentiable and xe = 0 so that f (0) = 0, then representing f (x) =

Z

1 0

∂f (σx)xdσ ∂x

and using the fact that f is essentially nonnegative it follows that (see [158] for details) a ˆij (x) = aij (x)xj , where the state-dependent transfer coefficients n aij (x) ≥ 0, x ∈ R+ , i, j = 1, . . . , n, and aij (·) is at least r − 1 times

NNBook

September 1, 2009

47

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

continuously differentiable. In this case, (2.44) becomes   n X x˙ i (t) = − aii (x(t)) + +aji (x(t)) xi (t) j=1,i6=j

+

n X

aij (x(t))xj (t) + wi (t),

j=1,i6=j

t ≥ 0,

i = 1, . . . , n,

(2.49)

or, equivalently, x(t) ˙ = A(x(t))x(t) + w(t),

t ≥ 0,

x(0) = x0 ,

(2.50)

where, for i, j = 1, . . . , n, A(i,j) (x) =



−

Pn

k=1 aki (x), i = j, aij (x), i= 6 j.

(2.51)

n

Note that A(x) is essentially nonnegative for all x ∈ R+ . Hence, once again using the total mass V (x) = eT x as a Lyapunov function for (2.50) (with w(t) ≡ 0) it follows that V˙ (x) = eT x˙ = eT A(x)x = −

n X i=1

aii (x)xi ≤ 0,

n

x ∈ R+ ,

(2.52)

which shows that the zero solution x(t) ≡ 0 of the inflow-closed (w(t) ≡ 0) system given by (2.50) is Lyapunov stable. As in the case of (2.44), (2.50) can have a full range of nonlinear behavior including bifurcations, limit cycles, and even chaos. However, it is of interest to determine conditions under which masses/concentrations for a nonlinear compartmental system converge. In light of the above and (2.48), we have the following result on stability, convergence, and global existence and uniqueness of solutions for nonlinear inflow-closed compartmental systems. Theorem 2.15. Consider the inflow-closed nonlinear compartmental system given by (2.48) where V (x) = eT x. Then the following statements hold: n n i ) If D(x) > 0, x ∈ R+ \{0}, and D(0) = 0, then V˙ (x) ≤ 0 for all x ∈ R+ and V˙ (x) = 0 if and only if x = 0. n

ii ) If Jn (0) = 0, D(x) ≥≥ 0, x ∈ R+ , and D(0) = 0, then the zero solution x(t) ≡ 0 to (2.48) is Lyapunov stable. If, in addition, D(x) > 0, n x ∈ R+ \{0}, then the zero solution x(t) ≡ 0 to (2.48) is asymptotically stable.

NNBook

September 1, 2009

48

CHAPTER 2 △

n

iii ) Let Dc = {x ∈ R+ : V (x) ≤ β} be compact for every β ∈ R+ and let x0 ∈ Dc . Then there exists a unique solution to (2.48) (with w(t) ≡ 0) that is defined for all t ≥ 0. In addition, every solution x(t) → M as △ t → ∞, where M is the largest invariant set contained in R = {x ∈ n n R+ : D(x) = 0}. Finally, if D(i,i) (x) 6= 0, x ∈ R+ , i = 1, . . . , n, except for a finite number of equilibrium points {p1 , p2 , . . . , pr }, then for every x0 ∈ Dc , limt→∞ x(t) = pi , where i ∈ {1, . . . , r}. Proof. i) Note that ∂V x˙ V˙ (x) = ∂x   ∂V ∂V T = [Jn (x) − D(x)] ∂x ∂x T  ∂V ∂V = − D(x) ∂x ∂x ≤ 0,

n

x ∈ R+ ,

which, using D(0) = 0, shows that V˙ (x) = 0 if and only if x = 0. ii) Lyapunov stability follows by noting that V (x) = eT x is a ∂V T Lyapunov function candidate for (2.48) and V˙ (x) = − ∂V ∂x D(x)( ∂x ) ≤ 0, n n x ∈ R+ . To show asymptotic stability note that if D(x) > 0, x ∈ R+ \{0}, n then V˙ (x) = −eT D(x)e < 0, x ∈ R+ \{0}. iii) Suppose x0 ∈ Dc with β ≥ V (x0 ). Now, since V˙ (x) ≤ 0 for all x ∈ Dc , it follows that x(t) ∈ Dc for all t ≥ 0. Hence, since Dc is compact, it follows from Corollary 2.5 of [112] that there exists a unique solution to (2.48) that is defined for all t ≥ 0. The fact that x(t) → M as t → ∞ is a direct consequence of the Krasovskii-LaSalle invariant set theorem. n

Finally, if D(i,i) (x) 6= 0, x ∈ R+ , i = 1, . . . , n, except for a finite n number of equilibrium points {p1 , p2 , . . . , pr }, then R = {x ∈ R : V˙ (x) = n 0} = {x ∈ R+ : D(i,i) (x) = 0, i = 1, . . . , n} = {p1 , p2 , . . . , pr }. Now, it follows that the largest invariant set M contained in R is given by M = {p1 , p2 , . . . , pr } and by the Krasovskii-LaSalle invariant set theorem, x(t) → M = {p1 , p2 , . . . , pr } as t → ∞. Next, since there exists a unique solution to n (2.48) it follows that for every initial condition x0 ∈ R+ , the positive limit set ω(x0 ) of (2.48) is a connected set. Hence, since M consists of a set of finite isolated points it follows that x(t), t ≥ 0, approaches one of the points in M, which shows that limt→∞ x(t) = pi , where i ∈ {1, . . . , r}. n

Note that if for every β ∈ R+ , the set {x ∈ R+ : D(x) = 0, eT x =

NNBook

September 1, 2009

49

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

β} does not contain any closed orbits, then x(t), t ≥ 0, approaches an equilibrium point of (2.48). For two-dimensional (n = 2) nonlinear 2 compartmental systems, the set {x ∈ R+ : D(x) = 0, eT x = β} is an empty set, a finite set of points, or a line, for every β ∈ R+ , and hence, it 2 2 follows that (2.48) is semistable for every D(x), x ∈ R+ , and x0 ∈ R+ .

2.7 Compartmental Systems in Biology, Ecology, Epidemiology, and Pharmacology As mentioned in Chapter 1, nonnegative dynamical systems arise in a number of areas such as biomedicine, physiology, genetics, ecology, chemical kinetics, epidemiology, economics, and thermodynamics. Moreover, many traditional models, such as the logistic equation for population dynamics, Lotka-Volterra equations for predator-prey or host-parasite dynamics, Markov chains, and Leslie models used to model the age classes of a biological population, fall under the class of nonnegative dynamical systems. In this section, we provide several examples that draw from the aforementioned areas to demonstrate the utility of the basic mathematical results developed in this chapter. Example 2.2. This example considers a lipoprotein metabolism and potassium ion transfer model. Specifically, consider the general form of a two-compartment model with arbitrary inputs (inflows) and outflows (excretions) shown in Figure 2.3, where for i, j = 1, 2, xi is the size of compartment i in mass units, aij is the instantaneous transfer coefficient of material flow from j to i in units of (time)−1 , aii is the flow loss coefficient from compartment i out of the system in units of (time)−1 , and ui is the instantaneous rate of flow of material from outside the system (environment) into the compartment i in units of mass/time. A mass balance for the two-compartment system yields x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0 ,

where x = [x1 x2 ]T , u = [u1 u2 ]T , and   −(a11 + a21 ) a12 A= , a21 −(a22 + a12 )

t ≥ 0,

B=



(2.53)

1 0 0 1



.

This model is frequently used to analyze the distribution or flow of a drug or other material (the tracer) through the human body after injection into the bloodstream [155]. The flow of the material to be studied is assumed to be near steady state, that is, the material is mixed in the blood plasma so that the amount in the plasma is at a uniform concentration. Furthermore, the tracer is introduced in a comparatively small quantity so that even if

NNBook

September 1, 2009

50

CHAPTER 2

the original flows are nonlinear, the tracer flows can be modeled by linear equations. A simple example of such a model would be of a lipoprotein metabolism. In this case, the first compartment corresponds to the blood plasma and the second compartment corresponds to the extravascular space. Here, we assume that the tracer is introduced intravenously as an impulsive injection, that is, a bolus injection, into the bloodstream and hence u2 = 0. Furthermore, we assume that the tracer does not enter cells so that a22 = 0. Finally, we assume that the compound is removed from the plasma by excretion into the kidneys and the amount excreted is some fraction of the amount filtered in the renal glomeruli so that a11 > 0. u1 (t) ' ?

Compartment I x1 (t) &

u2 (t) $ ) %

a12 x2 (t)

a21 x1 (t)

'

?

$

Compartment II x2 (t) 1& %

a11 x1 (t) ?

a22 x2 (t) ?

Figure 2.3 General linear two-compartment model.

To analyze this system, note that A is nonsingular and essentially nonnegative. Furthermore, since the input material is a bolus injection, we can always reproduce the impulsive response with the free response by setting x(0) = Bv, where v ∈ R2 denotes the impulse strength. Hence, the above model can be analyzed as an input-closed compartmental model. Now, it follows from Proposition 2.5 that eAt ≥≥ 0 for all t ≥ 0 and consequently if x(0) is nonnegative, then the solution x(t) = eAt x(0) is nonnegative for all t ≥ 0. Furthermore, since A is a nonsingular matrix it follows that the set 2 of equilibria of (2.53) (with u(t) ≡ 0) is given by E = {(x1 , x2 ) ∈ R+ : Ax = 0} = N (A) = {(0, 0)}. Next, taking p = e >> 0 and r = [−a11 , 0]T ≥≥ 0, it follows that (2.42) holds, and hence, since (A, r T ) is observable, the system is asymptotically stable by iv) of Theorem 2.11. Finally, we use the two-compartment model shown in Figure 2.3 to study the behavior of a closed compartmental system, that is, u1 = u2 = 0 and a11 = a22 = 0. These systems can arise when studying potassium ions continually moving from the plasma into the red blood cells and vice versa in the human bloodstream. Since potassium is concentrated in red blood cells by a nonlinear active transport mechanism, the resulting compartmental

NNBook

September 1, 2009

51

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

model is inherently nonlinear. However, experimental data reported in [272] show that potassium levels in both the plasma and red cells stay relatively constant over time, and hence, at steady state the tracer distribution is linear. Hence, we use the model in Figure 2.3 with u1 = u2 = 0 and a11 = a22 = 0 to analyze this system with the total red blood cells containing an amount x1 of potassium (the tracee) designated as the first compartment, and blood plasma, containing a quantity x2 of potassium, as the second compartment. In this case, A is singular and essentially nonnegative. Hence, " ! # −(a21 +a12 )t 1 − e A x0 ≥≥ 0, t ≥ 0. x(t) = eAt x(0) = I2 + a21 + a12 2

Furthermore, the set of equilibria is given by E = {(x1 , x2 ) ∈ R+ : Ax = 2 0} = N (A) = {(x1 , x2 ) ∈ R+ : x2 = aa21 x1 }. Finally, taking p = e >> 0 and 12 T r = [0, 0] , it follows that (2.42) holds, and hence, the system is semistable by ii) of Theorem 2.11. Alternatively, semistability   also follows by noting 1 At △ that limt→∞ e exists and is equal to I2 + a21 +a12 A.

Example 2.3. This example models the flow of lead in the human body. Lead enters the human body via food, liquid, and air inhalation. A simple yet accurate model for the lead kinetics in the human body is captured by the three-compartment model shown in Figure 2.4 [301]. The first compartment corresponds to the red blood cells and to a lesser extent blood plasma. From the blood (Compartment I), lead is rapidly distributed to the tissues (e.g., liver and kidneys) denoted by Compartment II. Finally, at a slower rate, lead is eventually transferred to the bones (Compartment III). For low concentrations of lead, the intercompartmental flow diffusion process is linear. Thus, with xi , i = 1, 2, 3, denoting the amount of lead in micrograms of the ith compartment and u1 denoting the input rate in Compartment I, a mass balance of the three-state compartmental model shown in Figure 2.4 yields (2.53) with x = [x1 , x2 , x3 ]T , u = u1 , and     −a11 − a21 − a31 a12 a13 1    a −a − a 0 A= , B = 0 . 21 22 12 a31 0 −a13 0 To analyze this system, let u1 (t) ≡ 0 and note that A is nonsingular and essentially nonnegative. Now, it follows from Proposition 2.5 that eAt ≥≥ 0 for all t ≥ 0 and consequently if x(0) is nonnegative, then the solution x(t) = eAt x(0) is nonnegative for all t ≥ 0. Furthermore, since A is a nonsingular matrix it follows that the set of equilibria of this system is 3 given by E = {(x1 , x2 , x3 ) ∈ R+ : Ax = 0} = N (A) = {(0, 0, 0)}. Next, taking p = e >> 0 and r = [a11 a22 0]T ≥≥ 0, it follows that (2.42) holds

NNBook

September 1, 2009

52

CHAPTER 2

and hence, since (A, r T ) is observable, the system is asymptotically stable by iv) of Theorem 2.11. △ u1 (air, water, food) a31 x1 (t) a12 x2 (t) ' $ '? $ $ ' ) ) Compartment I Compartment III Compartment II blood (red cells, tissue skeleton plasma) & % 1& % 1& % a13 x3 (t) a21 x1 (t) ? ? a11 a22 (urine) (hair, nails, sweat)

Figure 2.4 Compartmental model of lead flow in the human body.

Example 2.4. Forest ecosystem dynamics are highly nonlinear and notoriously complex. However, a simple yet accurate linear model is given by the four-compartment model shown in Figure 2.5. The four compartments correspond to leaves, debris, soil, and wood. The intercompartmental flow dynamics are such that leaves fall and contribute to forest debris, which in turn decomposes into soil. The cycle is completed as the nutrients from the soil are converted into wood and leaves. Input and output flows of the system can enter and exit primarily to and from the soil compartment. Thus, with xi , i = 1, 2, 3, 4, denoting potassium concentrations in the ith compartment and assuming that the inputs u are balanced by the system outflows, that is, u = a33 x3 , a mass balance of the four-state compartment model yields (2.35) with x = [x1 , x2 , x3 , x4 ]T and   −a21 0 0 a14  a21 −a32 0 0  . A=  0 a32 −a43 0  0 0 a43 −a14 Note that A is essentially nonnegative and singular.

Now, it follows from Proposition 2.5 that eAt ≥≥ 0 for all t ≥ 0, and hence, for every nonnegative initial value x(0) the solution x(t) = eAt x(0) is nonnegative for all t ≥ 0. Furthermore, since A is a singular matrix it follows that the set of equilibria for this system is given by E = N (A) = 4 {(x1 , x2 , x3 , x4 ) ∈ R+ : x1 = aa14 x4 , x2 = aa14 x4 , x3 = aa14 x4 }. Finally, taking 21 32 43 T p = e >> 0 and r = [0, 0, 0, 0] it follows by ii) of Theorem 2.11 that the forest ecosystem is semistable. Semistability in a living forest reflects the fact that potassium concentrations converge to Lyapunov stable (nonzero) equilibrium points determined by the initial potassium concentrations in the ecosystem. △ Example 2.5. In this and the next three examples we consider various nonlinear compartmental models for epidemiology, that is, the spread

NNBook

September 1, 2009

53

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

'

$ )

'

&

%

&

$

'

a14 x4 (t)

Compartment I leaves

'

a21 x1 (t) ?

Compartment II debris

&

%

a32 x2 (t)

Compartment IV wood 6

a43 x3 (t)

$ % $

Compartment III  soil u(t) 1& % ?a33 x3 (t)

Figure 2.5 Compartmental model of potassium flow in a forest.

of epidemics. There are several such models in the literature and in this and the next three examples we consider the Susceptible-Infected (SI) model [157], the Susceptible-Infected-Removed (SIR) model [155], the Susceptible-Infected-Susceptible (SIS) model [157], and the SusceptibleInfected-Recovered-Susceptible (SIRS) model [301]. The Susceptible-Infected (SI) model is shown in Figure 2.6 and is given by cβ x2 (t)x1 (t) − µx1 (t) + u(t), x1 (0) = x01 , N cβ x2 (t)x1 (t) − (k + µ)x2 (t), x2 (0) = x02 , x˙ 2 (t) = N x˙ 1 (t) = −

t ≥ 0, (2.54) (2.55)

where x1 denotes the number of susceptibles, x2 denotes the number of infectives, µ > 0 is the background death rate coefficient, k is the additional death rate due to the disease, N is a constant denoting the total size of the population (that is, x1 (t) + x2 (t) = N ), u is the rate of recruitment of new members into the susceptible pool and, since x1 (t) + x2 (t) = N , is given by u(t) = µx1 (t) + (k + µ)x2 (t), c > 0 is the contact rate per individual, and β > 0 is the probability of transmission in a contact. Note that the nonlinear terms in (2.54) and (2.55) arise due to the fact that the rate of susceptibles becoming infected is given by the nonlinear transfer λ(x1 , x2 )x1 = cβ N x1 x2 ; that is, the product of the rate of infecteds that make contact and transmit the disease (βx2 ) and the fraction of susceptibles in the population (cx1 /N ). To analyze this model, first note that the vector field of (2.54) and (2.55) is essentially nonnegative. Furthermore, since x1 (t) + x2 (t) = N , (2.54) is superfluous and one need only consider (2.55). In addition, (2.55)

NNBook

54 -

u(t)

September 1, 2009

'

$

&

%

Compartment I x1 (t) µx1 (t) ?

λ(x1 , x2 )x1 (t)

-

CHAPTER 2

'

$

&

%

Compartment II x2 (t) µx2 (t) ?

-

kx2 (t)

Figure 2.6 SI compartmental model.

can be equivalently written as   x2 (t) ) − 1 x2 (t), x˙ 2 (t) = (k + µ) α(1 − N

x2 (0) = x02 ,

t ≥ 0,

(2.56)

cβ represents the basic reproduction number. Now, the set of where α , k+µ equilibria for (2.56) is E = {x2 ∈ R+ : x˙ 2 (t) = 0} = {0} if α ≤ 1 and E = {0, N (1 − α1 )} if α > 1.

In the case where α > 1, linearizing (2.56) about the two equilibria, it follows from Theorem 2.14 that the origin of the SI model is unstable and {N (1 − α1 )} is locally asymptotically stable. Hence, in this case, there exists a stable epidemic level. Finally, we consider the case where α ≤ 1. Consider the Lyapunov function candidate V (x2 ) = x2 > 0, x2 6= 0, and note that V˙ (x2 ) = (k + µ)[α(1 − xN2 ) − 1]x2 < 0, x2 ∈ R+ , x2 6= 0. Hence, in the case where α ≤ 1, a disease introduced into the population dies out, that is, it cannot give rise to an epidemic. △ Example 2.6. The Susceptible-Infected-Removed (SIR) model is shown in Figure 2.7 and is given by x˙ 1 (t) = − Nλ x1 (t)x2 (t) − µx1 (t) + u(t), λ N x1 (t)x2 (t)

x1 (0) = x01 ,

− (γ + µ)x2 (t), x2 (0) = x02 , x˙ 2 (t) = x˙ 3 (t) = γx2 (t) − µx3 (t), x3 (0) = x03 ,

t ≥ 0, (2.57)

(2.58) (2.59)

where x1 denotes the number of susceptibles, x2 denotes the number of infectives, x3 denotes the number of immunes, µ > 0 is the death rate coefficient, N is a constant denoting the total size of the population (that is, x1 (t) + x2 (t) + x3 (t) = N ), u is the rate of recruitment of new members into the susceptible pool and is assumed to be a constant rate that just makes up for the deaths (that is, u(t) = u = µN ), γ > 0 is a rate constant for recovery, and λ > 0 is the mean contact rate per person for contacts that transmit the disease. Note that the nonlinear terms in (2.57) and (2.58) arise due to the fact that the rate of susceptibles that become infected is λx1 x2 /N ; that is, the product of the rate of infecteds that make contact and transmit the disease (λx2 ) and the fraction of susceptibles in the population (x1 /N ).

NNBook

September 1, 2009

55

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

u(t) ' ?

$

'

&

%

&

Compartment I x1 (t) µx1 (t) ?

λ x (t)x2 (t) N 1 -

Compartment II x2 (t) µx2 (t) ?

$'

γx2 (t)

-

Compartment III x3 (t)

%&

µx3 (t) ?

$ %

Figure 2.7 SIR compartmental model.

To analyze this system, first note that the vector field of (2.57)–(2.59) is essentially nonnegative. Furthermore, since x1 (t) + x2 (t) + x3 (t) = N , (2.59) is superfluous and we need only consider (2.57) and (2.58). In addition note that (2.58) can be equivalently written as i hα x1 (t) − 1 x2 (t), x2 (0) = x02 , x˙ 2 (t) = (γ + µ) (2.60) N △

where α = λ/(γ+µ) represents the number of infections that are transmitted by an infective, over the lifetime of the infection, if all contacts are with susceptibles. Now, the set of equilibria for (2.57) and (2.58) is 2 E = {(x1 , x2 ) ∈ R+ : f (x) = 0} = {(N, 0)} if α ≤ 1 and E = µN (α−1) )} if α > 1, where f denotes the vector field of (2.57) {(N, 0), ( N α, λ and (2.58).

In the case where α > 1, linearizing (2.57) and (2.58) about the two equilibria and computing the eigenvalues of the resulting Jacobian matrix, it follows from Theorem 2.14 that the equilibrium point (N, 0) of the SIR model µN (α−1) ) is locally asymptotically is unstable and the equilibrium point ( N α, λ stable. Hence, in this case, there exists a stable epidemic level. Now, we consider the case where α ≤ 1. Consider the Lyapunov function candidate V (x) = 12 (x1 − N )2 + N x2 and note that since α ≤ 1 it follows that V˙ (x) = −µ(x1 − N )2 − Nλ x2 (x1 − N )2 + N (γ + µ)(α − 1)x2 ≤ 0, 2

x ∈ R+ , which shows that the equilibrium point (N, 0) is Lyapunov stable. Now, consider the function E(x)  = x2 and note that since x1 ≤ N it follows α ˙ x1 − 1 x2 ≤ 0. Next, using the Krasovskii-LaSalle that E(x) = (γ + µ) N theorem it can be shown that the largest invariant set M contained in the 2 △ ˙ set R = {x ∈ R+ : E(x) = 0} is given by M = {(N, 0)}, which shows that (N, 0) is a globally asymptotically stable equilibrium. Hence, in the case where α ≤ 1, a disease introduced into the population dies out, that is, it cannot propagate and give rise to an epidemic.

Next, we consider the case corresponding to zero death rate, that is, u = µ = 0. In this case, the system equilibria are given by E =

NNBook

September 1, 2009

56

CHAPTER 2 2

{(x1 , x2 ) ∈ R+ : x2 = 0}. First, using the Lyapunov function candidate V (x) = 12 (x1 − x1e )2 + γN λ x2 , it can be shown that every point in the 2

set Es = {(x1e , x2e ) ∈ R+ : x1e ≤ γN λ , x2e = 0} is Lyapunov stable. Alternatively, using Theorem 2.14 it can be shown that every point in 2 the set {(x1 , x2 ) ∈ R+ : x1 > γN λ , x2 = 0} is unstable. To further analyze these equilibria, consider the function E(x) = x1 + x2 and note that 2 ˙ E(x) = −γx2 ≤ 0, x ∈ R+ . Now, applying the Krasovskii-LaSalle theorem it △ can be shown that the largest invariant set M contained in the set R = {x ∈ 2 ˙ R+ : E(x) = 0} is given by M = E. Furthermore, since x1 (t) is bounded and monotonic it follows from the monotone convergence theorem [112, p. 37] 2 that limt→∞ x1 (t) exists. Hence, for every initial condition x(0) ∈ R+ , limt→∞ x(t) exists, limt→∞ x2 (t) = 0, and limt→∞ x1 (t) + x3 (t) = N . In addition, it can be shown that limt→∞ x1 (t) ≤ γN λ , which shows that every equilibrium point in Es is semistable. △ △

Example 2.7. In this example, we consider the Susceptible-InfectedSusceptible (SIS) model. In this model, a susceptible who becomes infected and recovers is then susceptible again. An example of an SIS disease is gonorrhea. The SIS model is shown in Figure 2.8 and is given by x˙ 1 (t) = − Nλ x1 (t)x2 (t) − µx1 (t) + δx2 (t) + u(t), x˙ 2 (t) =

λ N x1 (t)x2 (t)

− (δ + µ)x2 (t),

x1 (0) = x01 ,

x2 (0) = x02 ,

t ≥ 0, (2.61) (2.62)

where x1 denotes the number of susceptibles, x2 denotes the number of infectives, µ > 0 is the death rate coefficient, N is a constant denoting the total size of the population (that is, x1 (t) + x2 (t) = N ), u is the rate of recruitment of new members into the susceptible pool and is assumed to be a constant rate that just makes up for the deaths (that is, u(t) = u = µN ), λ > 0 is the mean contact rate per person for contacts that transmit the disease, and δ > 0 is a rate constant for recovery. Hence, δx2 gets fed back into the susceptible pool (Compartment I).

u(t)

'

Compartment I x1 (t) &

$ ) %

δx2 (t)

λ N x1 (t)x2 (t)

'

Compartment II x2 (t) 1&

µx1 (t) ?

$ %

µx2 (t) ?

Figure 2.8 SIS compartmental model.

To analyze this system, first note that the vector field of (2.61) and

NNBook

September 1, 2009

57

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

(2.62) is essentially nonnegative. Furthermore, since x1 (t) + x2 (t) = N , (2.61) is superfluous and we need only consider (2.62). In addition, note that (2.62) can be equivalently written as i h α x˙ 2 (t) = (δ + µ) − x2 (t) − 1 + α x2 (t), x2 (0) = x02 , (2.63) N △

where α = λ/(δ + µ). Now, the set of equilibria for (2.63) is E = {x2 ∈ R+ : f (x) = 0} = {0} if α ≤ 1 and E = {0, N (α−1) } if α > 1, where f denotes α the vector field of (2.63).

In the case where α > 1, linearizing (2.63) about the two equilibria, it follows from Theorem 2.14 that the origin of the SIS model is unstable and is locally asymptotically stable. Hence, in this the equilibrium point N (α−1) α case, there exists a stable epidemic level. Finally, we consider the case where α ≤ 1. Consider the Lyapunov function candidate V (x2 ) = x2 > 0, x2 6= 0, and note that since α ≤ 1 α x2 − 1 + α x2 < 0, x2 ∈ R+ , x2 6= 0, it follows that V˙ (x2 ) = (δ + µ) − N which shows that the origin is globally asymptotically stable. Hence, in the case where α ≤ 1, a disease introduced into the population dies out, that is, it cannot propagate and give rise to an epidemic. △ Example 2.8. In this example we consider the Susceptible-InfectedRecovered-Susceptible (SIRS) model. In this model, the recovered population can lose its immunity and again become susceptible. The SIRS model is shown in Figure 2.9 and is given by (with no death rate and no input) x˙ 1 (t) = − Nλ x1 (t)x2 (t) + δx3 (t),

x1 (0) = x01 ,

λ N x1 (t)x2 (t)

− γx2 (t), x2 (0) = x02 , x˙ 2 (t) = x˙ 3 (t) = γx2 (t) − δx3 (t), x3 (0) = x03 ,

t ≥ 0,

(2.64) (2.65) (2.66)

where x1 denotes the number of susceptibles, x2 denotes the number of infectives, x3 denotes the number of immunes, γ > 0 is a rate constant for recovery, λ > 0 is the mean contact rate per person for contacts that transmit the disease, and δ > 0 is the fraction of recovered members of the population which are not immune to the disease. '

Compartment I x1 (t)

&

6

$

'

$'

$

%

&

%&

%

λ x (t)x2 (t) N 1 -

Compartment II x2 (t) δx3 (t)

γx2 (t)

-

Compartment III x3 (t)

Figure 2.9 SIRS compartmental model.

To analyze this system, first note that the vector field of (2.64)–(2.66)

NNBook

September 1, 2009

58

CHAPTER 2

is essentially nonnegative. Furthermore, since x1 (t) + x2 (t) + x3 (t) = N , (2.66) is superfluous and we need only consider (2.64) and (2.65). In addition, note that (2.64) and (2.65) can be equivalently written as x˙ 1 (t) = − Nλ x1 (t)x2 (t) + δ(N − x1 (t) − x2 (t)), x˙ 2 (t) = γ △

hα

N

i x1 (t) − 1 x2 (t),

x1 (0) = x01 ,

x2 (0) = x02 ,

t ≥ 0, (2.67) (2.68)

where α = λ/γ. Now, the set of equilibria for (2.67) and (2.68) is 2 E = {(x1 , x2 ) ∈ R+ : f (x) = 0} = {(N, 0)} if α ≤ 1 and E = δN (α−1) {(N, 0), ( N α , α(γ+δ) )} if α > 1, where f denotes the vector field of (2.67) and (2.68). In the case where α > 1, linearizing (2.67) and (2.68) about the two equilibria and computing the eigenvalues of the resulting Jacobian matrix, it follows from Theorem 2.14 that the equilibrium point (N, 0) of δN (α−1) the SIRS model is unstable and the equilibrium point ( N α , α(γ+δ) ) is locally asymptotically stable. Hence, in this case, there exists a stable epidemic level. Finally, we consider the case where α ≤ 1. Consider the Lyapunov function candidate V (x) = 12 (x1 + x2 − N )2 + N x2 and note that since α ≤ 1 it follows that V˙ (x) = −δ(x1 + x2 − N )2 + γx2 (αx1 − x1 − x2 ) < 0, 2 x ∈ R+ , x 6= (N, 0), which shows that the equilibrium point (N, 0) is a globally asymptotically stable equilibrium. Hence, in the case where α ≤ 1, a disease introduced into the population dies out, that is, it cannot propagate and give rise to an epidemic. △ Example 2.9. One of the most classical areas in which nonnegative dynamical systems arise is population growth and predator-prey models. A classical model for two populations that interact in a predator-prey relationship without age structure in the populations is given by the LotkaVolterra equations [155] x˙ 1 (t) = αx1 (t) − βx1 (t)x2 (t), x1 (0) = x01 , t ≥ 0, x˙ 2 (t) = −γx2 (t) + δx1 (t)x2 (t), x2 (0) = x02 ,

(2.69) (2.70)

where α, β, γ, δ > 0, x1 denotes the number of prey, and x2 denotes the number of predators. Note that in the absence of predators (x2 = 0), (2.69) results in exponential growth of prey while in the absence of prey (x1 = 0), (2.70) results in exponential decay of predators. To analyze this system, first note that the vector field of (2.69) and (2.70) is essentially nonnegative. Furthermore, the set of all equilibria

NNBook

September 1, 2009

59

STABILITY THEORY FOR NONNEGATIVE SYSTEMS 2

of (2.69) and (2.70) is given by E = {(x1 , x2 ) ∈ R+ : f (x) = 0} = {(0, 0), ( γδ , αβ )}, where f denotes the vector field of (2.69) and (2.70). Now, linearizing (2.69) and (2.70) about the origin, it follows from ii) of Theorem 2.14 that the origin is unstable. To analyze the stability of the equilibrium xe = ( γδ , αβ ) define E(x) , 2 ˙ = 0 for all x ∈ R+ . Now, δx1 + βx2 − γln x1 − αln x2 and note that E(x) since xe is the only singular point in R2+ , E ′ (xe ) = 0, and E ′′ (xe ) > 0, it 2 follows that ( γδ , αβ ) is a local minimizer of E(x). Since E ′′ (x) > 0, x ∈ R+ , it follows that E(x) is convex, and hence, xe is a global minimizer. Next, using the Lyapunov function candidate V (x) = E(x)−E(xe ) with V (xe ) = 0 2 and V (x) > 0, x ∈ R+ \{xe }, it follows that V˙ (x) = 0, x ∈ R2+ , which shows that the equilibrium point ( γδ , αβ ) is Lyapunov stable. However, this point is neither asymptotically stable nor semistable. To see this, since 2 △ E(·) is locally convex at xe , it follows that the set S = {(x1 , x2 ) ∈ R+ : E(x1 , x2 ) = E(x01 , x02 )} is a closed curve for all (x01 , x02 ) sufficiently close to the equilibrium xe . Hence, since xe is the only equilibrium point in R2+ , the trajectory starting from all (x01 , x02 ) in the neighborhood of xe remains △ in S, which shows that S is a periodic orbit. Example 2.10. In this example, we consider a pharmacokinetic and pharmacodynamic model for clinical anesthesia. Almost all anesthetics are myocardial depressants which lower cardiac output (i.e., the amount of blood pumped by the heart per unit time). As a consequence, decreased cardiac output slows down redistribution kinetics, that is, the transfer of blood from the central compartments (heart, brain, kidney, and liver) to peripheral compartments (muscle and fat). In addition, decreased cardiac output could increase drug concentrations in the central compartments, causing even more myocardial depression and further decrease in cardiac output. To study the effects of pharmacological agents and anesthetics we propose the two-compartment model shown in Figure 2.10, where x1 denotes the drug concentration in the central compartment, which is the site for drug administration and is generally thought to be comprised of the intravascular blood volume as well as highly perfused organs such as the heart, brain, kidney, and liver. These organs receive a large fraction of the cardiac output. The state x2 is the concentration of drug in the peripheral compartment, comprised of muscle and fat, which receive a smaller proportion of the cardiac output.

NNBook

September 1, 2009

60

CHAPTER 2

'

6

u- Compartment I x1 (t) &

$

Ke (x1 (t))x1 (t)

6

%

k1 (x1 (t))x1 (t)

k2 (x1 (t))x2 (t)

-

'

$

&

%

Compartment II x2 (t)

Figure 2.10 Pharmacokinetic model for drug distribution during anesthesia.

A concentration balance of the two-state compartment model yields x˙ 1 (t) = −k1 (x1 (t))x1 (t) − Ke (x1 (t))x1 (t) + k2 (x1 (t))x2 (t), x1 (0) = x01 , t ≥ 0, x˙ 2 (t) = k1 (x1 (t))x1 (t) − k2 (x1 (t))x2 (t), x2 (0) = x02 ,

(2.71) (2.72)

where k1 (x1 ) is the rate of transfer of drug from Compartment I to Compartment II, k2 (x1 ) is the rate of transfer of drug from Compartment II to Compartment I, and Ke (x1 ) is the rate of drug metabolism and elimination (metabolism typically occurs in the liver). In this model we assume that each of these rate constants is proportional to the cardiac output. This reflects the fact that the drug transfer from the central compartment to the peripheral compartment (or the reverse) requires physical transport via the bloodstream from the heart, brain, etc., to muscle and fat (or the reverse). It is generally assumed that this transport in the vascular tree will be proportional to the cardiac output Q(x1 ). Furthermore, for many drugs the rate of metabolism (i.e., Ke (x1 )) will be proportional to the rate of transport of drug to the liver, and hence, we assume that Ke (x1 ) is also proportional to the cardiac output. Thus, we assume k1 (x1 ) = A1 Q(x1 ), k2 (x1 ) = A2 Q(x1 ), and Ke (x1 ) = Ae Q(x1 ), where A1 , A2 , and Ae are positive constants. Finally, since many anesthetics depress the heart, and thus decrease the cardiac output, the pharmacodynamic effects are almost universally described by a sigmoid relationship between drug concentration and effect, so that α Q0 C50 (2.73) Q(x1 ) = α + xα ) , (C50 1 where the effect is related to x1 (since that is the presumed concentration in the highly perfused myocardium), Q0 > 0 is a constant, C50 > 0 is the drug concentration associated with a 50% decrease in the cardiac output, and α > 1 determines the steepness of this curve (that is, how rapidly the cardiac output decreases with increasing drug concentration x1 ). Furthermore, this model assumes instantaneous mixing, and as x1 increases the rate constants decrease through their dependence on the cardiac output.

NNBook

September 1, 2009

61

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

To analyze this system, first note that the vector field f (x) of (2.71) and (2.71) is essentially nonnegative. Furthermore, the set of equilibria of 2 (2.71) and (2.71) is given by E = {(x1 , x2 ) ∈ R+ : f (x) = 0} = {(0, 0)}. To study the stability of the pharmacokinetic and pharmacodynamic system (2.71) and (2.71), consider the Lyapunov function candidate V (x) = x1 + x2 α 2 Ae Q0 C50 and note that V˙ (x) = x˙ 1 + x˙ 2 = (C α α x1 ≤ 0, (x1 , x2 ) ∈ R+ , which 50 +x1 ) implies that the origin is Lyapunov stable. Next, since the pharmacokinetic and pharmacodynamic system (2.71) and (2.71) is Lipschitz continuous, it △ follows that the largest invariant set M contained in the set R = {(x1 , x2 ) ∈ R+ × R+ : V˙ (x1 , x2 ) = 0} = {(x1 , x2 ) ∈ R+ × R+ : x1 = 0} is given by M = {0, 0}. Hence, it follows from the Krasovskii-LaSalle theorem that (x1 (t), x2 (t)) → M = {0, 0} as t → ∞, establishing global asymptotic stability of (2.71) and (2.71). △

2.8 Discrete-Time Lyapunov Stability Theory for Nonnegative Dynamical Systems Even though numerous results on continuous-time compartmental systems and, to a lesser extent, nonnegative systems have been developed in the literature (see [4, 20, 29, 88, 100, 111, 113, 155, 158, 209, 211, 220, 230, 233, 259], and the references therein), the development of discrete-time nonnegative and compartmental systems theory has received far less attention. In the remainder of this chapter, we develop several basic mathematical results on the stability of discrete-time linear and nonlinear nonnegative dynamical systems. In addition, using linear Lyapunov functions, we develop necessary and sufficient conditions for Lyapunov stability and asymptotic stability for linear nonnegative dynamical systems. The consideration of a linear Lyapunov function leads to a new Lyapunov-like equation for examining the stability of linear nonnegative systems. This Lyapunov-like equation is analyzed using nonnegative matrix theory [21, 145]. As in the continuoustime case, the motivation for using a linear Lyapunov function follows from the fact that the state of a nonnegative dynamical system is nonnegative, and hence, a linear Lyapunov function is a valid Lyapunov function candidate. This considerably simplifies the stability analysis of nonnegative dynamical systems. We begin by considering the general discrete-time nonlinear dynamical system x(k + 1) = f (x(k)),

x(0) = x0 ,

k ∈ Z+ ,

(2.74)

where x(k) ∈ D ⊆ Rn , k ∈ Z+ , is the system state vector, D is a relatively open set, and f : D → Rn . Furthermore, we denote the solution to (2.74) with initial condition x(0) = x0 by s(·, x0 ), so that the map of

NNBook

September 1, 2009

62

CHAPTER 2

the dynamical system given by s : Z+ × D → D is continuous on D and satisfies the consistency property s(0, x0 ) = x0 and the semigroup property s(κ, s(k, x0 )) = s(k + κ, x0 ), for all x0 ∈ D and k, κ ∈ Z+ . For discretetime systems we use the notation s(k, x0 ), k ∈ Z+ , and x(k), k ∈ Z+ , interchangeably as the solution of the nonlinear discrete-time system (2.74) with initial condition x(0) = x0 . Unless otherwise stated, we assume f (·) is continuous on D. Furthermore, xe ∈ D is an equilibrium point of (2.74) if and only if f (xe ) = xe . The following definition introduces the notion of nonnegative vector fields. n

Definition 2.13. Let f = [f1 , . . . , fn ]T : D ⊆ R+ → Rn . Then f is n nonnegative if f (x) ≥≥ 0 for all x ∈ R+ . n

The following proposition shows that R+ is an invariant set of (2.74) if and only if f is nonnegative. n

n

Proposition 2.11. Suppose R+ ⊂ D. Then R+ is an invariant set with respect to (2.74) if and only if f : D → Rn is nonnegative. n

Proof. Suppose f : D → Rn is nonnegative and let x(0) ∈ R+ . Then, for every i ∈ {1, . . . , n} it follows that xi (k+1) = fi (x(k)) ≥ 0. Thus, x(k) ∈ n n n R+ , k ∈ Z+ . Conversely, suppose x(k) ∈ R+ , k ∈ Z+ , for all x(0) ∈ R+ n and assume, ad absurdum, that there exists i ∈ {1, . . . , n} and x0 ∈ R+ such that fi (x0 ) < 0. In this case, with x(0) = x0 , xi (1) = fi (x(0)) = fi (x0 ) < 0, which is a contradiction. It follows from Proposition 2.11 that if x0 ≥≥ 0, then x(k) ≥≥ 0, k ∈ Z+ , if and only if f is nonnegative. In this case, we say that (2.74) is a discrete-time nonnegative dynamical system. Henceforth, in this monograph, we assume that f is nonnegative so that the discrete-time nonlinear dynamical system (2.74) is a nonnegative dynamical system. The following definition introduces several types of stability corresponding to the equilibrium solution x(k) ≡ xe of the discrete-time system (2.74). Definition 2.14. i) The equilibrium solution x(k) ≡ xe to (2.74) is n Lyapunov stable with respect to R+ if, for all ε > 0, there exists δ = δ(ε) > 0 n n such that if x0 ∈ Bδ (xe ) ∩ R+ , then x(k) ∈ Bε (xe ) ∩ R+ , k ∈ Z+ . ii) The equilibrium solution x(k) ≡ xe to (2.74) is (locally) asymptotn n ically stable with respect to R+ if it is Lyapunov stable with respect to R+ n and there exists δ > 0 such that if x0 ∈ Bδ (xe )∩R+ , then limk→∞ x(k) = xe .

NNBook

September 1, 2009

63

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

iii) The equilibrium solution x(k) ≡ xe to (2.74) is globally asymptotn n ically stable with respect to R+ if it is Lyapunov stable with respect to R+ n and, for all x(0) ∈ R+ , limk→∞ x(k) = xe . The following result gives sufficient conditions for Lyapunov and asymptotic stability of a discrete-time nonlinear dynamical system. n

Theorem 2.16. Let D be a relatively open subset of R+ that contains xe . Consider the discrete-time nonlinear dynamical system (2.74) where f is nonnegative and f (xe ) = xe , and assume that there exists a continuous function V : D → R such that V (xe ) = 0, V (x) > 0, V (f (x)) − V (x) ≤ 0,

x ∈ D, x ∈ D.

(2.75) (2.76) (2.77)

x 6= xe ,

Then the equilibrium solution x(k) ≡ xe to (2.74) is Lyapunov stable with n respect to R+ . If, in addition, V (f (x)) − V (x) < 0,

x ∈ D,

x 6= xe ,

(2.78)

then the equilibrium solution x(k) ≡ xe to (2.74) is asymptotically stable n with respect to R+ . Finally, if V (·) is such that V (x) → ∞ as kxk → ∞,

(2.79)

then (2.78) implies that the equilibrium solution x(k) ≡ xe to (2.74) is n globally asymptotically stable with respect to R+ . n

n

Proof. Let ε > 0 be such that Bε (xe ) ∩ R+ ⊆ D. Since R+ ∩ B ε (xe ) is compact and f (x), x ∈ D, is continuous, it follows that ( ) △

η = max ε,

max n

x∈R+ ∩Bε (xe )

△

kf (x) − xe k

(2.80)

exists. Next, let α = minx∈D: ε≤kxk≤η V (x). Note α > 0 since xe 6∈ ∂Bε (xe ) △ and V (x) > 0, x ∈ D, x 6= xe . Next, let β ∈ (0, α) and define Dβ = {x ∈ n R+ ∩ Bε (xe ) : V (x) ≤ β}. Now, for every x ∈ Dβ , it follows from (2.77) that V (f (x)) ≤ V (x) ≤ β, and hence, it follows from (2.80) that kf (x) − xek ≤ η, x ∈ Dβ . Next, suppose, ad absurdum, that there exists x ∈ Dβ such that kf (x) − xe k ≥ ε. This implies V (x) ≥ α, which is a contradiction. Hence, n since f is nonnegative, for every x ∈ Dβ it follows that f (x) ∈ R+ ∩Bε (xe ) ⊂ Dβ , which implies that Dβ is a positively invariant set (see Definition 2.16) with respect to (2.74). Next, since V (·) is continuous and V (xe ) = 0, there n exists δ = δ(ε) ∈ (0, ε) such that V (x) < β, x ∈ R+ ∩ Bδ (xe ). Now, let n n x(k), k ∈ Z+ , satisfy (2.74). Since R+ ∩ Bδ (xe ) ⊂ Dβ ⊂ R+ ∩ Bε (xe ) ⊆ D and Dβ is positively invariant with respect to (2.74) it follows that for all

NNBook

September 1, 2009

64

CHAPTER 2 n

n

x(0) ∈ Bδ (xe ) ∩ R+ , x(k) ∈ Bε (xe ) ∩ R+ , k ∈ Z+ , which proves Lyapunov n stability of xe with respect to R+ . n

To prove asymptotic stability with respect to R+ , suppose that n V (f (x)) < V (x), x ∈ D, x 6= xe , and x(0) ∈ Bδ (xe ) ∩ R+ . Then it n follows that x(k) ∈ Bε (xe ) ∩ R+ , k ∈ Z+ . However, V (x(k)), k ∈ Z+ , is decreasing and bounded from below by zero. Now, suppose, ad absurdum, that x(k), k ∈ Z+ , does not converge to xe . This implies that V (x(k)), k ∈ Z+ , is lower bounded by a positive number, that is, there exists L > 0 such that V (x(k)) ≥ L > 0, k ∈ Z+ . Hence, by continuity of V (x), n x ∈ D, there exists δ′ > 0 such that V (x) < L for x ∈ Bδ′ (xe ) ∩ R+ , n which further implies that x(k) 6∈ Bδ′ (xe ) ∩ R+ , k ∈ Z+ . Next, let n △ L1 = min{V (x) − V (f (x)) : δ′ ≤ kx − xe k ≤ ε, x ∈ R+ }. Now, (2.78) n implies V (x) − V (f (x)) ≥ L1 , δ′ ≤ kx − xe k ≤ ε, , x ∈ R+ , or, equivalently, V (x(k)) − V (x(0)) = n

k−1 X i=0

[V (f (x(i))) − V (x(i))] ≤ −L1 k,

and hence, for all x(0) ∈ R+ ∩ Bδ (xe ), V (x(k)) ≤ V (x(0)) − L1 k.

, it follows that V (x(k)) < L, which is a contradiction. Letting k > V (x(0))−L L1 Hence, x(k) → xe as k → ∞, establishing asymptotic stability with respect n to R+ . Finally, to prove global asymptotic stability with respect to R+ , let n △ x0 ∈ R+ , and let β = V (x0 ). Now, the radial unboundedness condition (2.79) implies that there exists ε > 0 such that V (x) ≥ β for kx − xe k ≥ ε, x ∈ Rn . Hence, since f is nonnegative it follows from (2.78) that V (x(k)) ≤ n V (x(0)) = β, k ∈ Z+ , which implies that x(k) ∈ R+ ∩ Bε(xe ), k ∈ Z+ . Now, the proof follows as in the proof of the local result. A continuous function V (·) satisfying (2.75) and (2.76) is called a Lyapunov function candidate for the discrete-time nonlinear dynamical system (2.74). If, additionally, V (·) satisfies (2.77), V (·) is called a Lyapunov function for the discrete-time nonlinear dynamical system (2.74).

2.9 Discrete-Time Invariant Set Theorems and Semistability Theorems In this section, we use the discrete-time Krasovskii-LaSalle invariance principle to relax one of the conditions on the Lyapunov function V (·) in the theorems given in Section 2.8. In particular, the strict negative-

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

65

definiteness condition on the Lyapunov difference can be relaxed while ensuring system asymptotic stability. To state the main results of this section several definitions and a key lemma analogous to the ones given in Section 2.3 are needed. Definition 2.15. The trajectory x(k), k ∈ Z+ , of (2.74) is bounded if there exists γ > 0 such that kx(k)k < γ, k ∈ Z+ . Definition 2.16. A set M ⊂ D ⊆ Rn is a positively invariant set for the nonlinear dynamical system (2.74) if sk (M) ⊆ M, for all k ∈ Z+ , where △ sk (M) = {sk (x) : x ∈ M}. A set M ⊆ D ⊆ Rn is an invariant set for the dynamical system (2.74) if sk (M) = M for all k ∈ Z+ . Definition 2.17. A point p ∈ D is a positive limit point of the trajectory x(k), k ∈ Z+ , of (2.74) if there exists a monotonic sequence {kn }∞ n=0 of nonnegative numbers, with kn → ∞ as n → ∞, such that x(kn ) → p as n → ∞. The set of all positive limit points of x(k), k ∈ Z+ , is the positive limit set ω(x0 ) of x(k), k ∈ Z+ . Note that if p ∈ D is a positive limit point of the trajectory x(·), then, for all ε > 0 and finite K ∈ Z+ , there exists k > K such that kx(k) − pk < ε. This follows from the fact that kx(k) − pk < ε for all ε > 0 and some k > K > 0 is equivalent to the existence of a sequence of integers {kn }∞ n=0 , with kn → ∞ as n → ∞, such that x(kn ) → p as n → ∞. Next, we state and prove a key lemma involving positive limit sets for discrete-time systems. Lemma 2.5. Consider the nonlinear dynamical system (2.74) where f is nonnegative. Suppose the solution x(k) to (2.74) corresponding to an initial condition x(0) = x0 is bounded for all k ∈ Z+ . Then the positive limit set ω(x0 ) of x(k), k ∈ Z+ , is a nonempty, compact, invariant subset of n R+ . Furthermore, x(k) → ω(x0 ) as k → ∞. Proof. Let x(k), k ∈ Z+ , denote the solution to (2.74) corresponding to the initial condition x(0) = x0 . Next, since x(k) is bounded for all k ∈ Z+ , it follows from the Bolzano-Weierstrass theorem [112, p. 27] that △ every sequence in the positive orbit Ox+0 = {s(k, x0 ) : k ∈ Z+ } has at least one accumulation point p ∈ D as k → ∞, and hence, ω(x0 ) is nonempty. Next, let p ∈ ω(x0 ) so that there exists an increasing unbounded sequence {kn }∞ n=0 , with k0 = 0, such that limn→∞ x(kn ) = p. Now, since x(kn ) is uniformly bounded in n it follows that the limit point p is bounded, which implies that ω(x0 ) is bounded. To show that ω(x0 ) is closed let {pi }∞ i=0 be a sequence contained in ω(x0 ) such that limi→∞ pi = p. Now, since pi → p as

NNBook

September 1, 2009

66

CHAPTER 2

i → ∞ for every ε > 0, there exists i such that kp − pi k < ε/2. Next, since pi ∈ ω(x0 ), there exists k ≥ K, where K ∈ Z+ is arbitrary and finite, such that kpi − x(k)k < ε/2. Now, since kp − pi k < ε/2 and kpi − x(k)k < ε/2, k ≥ K, it follows that kp − x(k)k ≤ kpi − x(k)k + kp − pi k < ε. Thus, p ∈ ω(x0 ). Hence, every accumulation point of ω(x0 ) is an element of ω(x0 ) so that ω(x0 ) is closed. Thus, since ω(x0 ) is closed and bounded, ω(x0 ) is compact. To show positive invariance of ω(x0 ) let p ∈ ω(x0 ) so that there exists an increasing sequence {kn }∞ n=0 such that x(kn ) → p as n → ∞. Now, let s(kn , x0 ) denote the solution x(kn ) of (2.74) with initial condition x(0) = x0 and note that, since f : D → D in (2.74) is continuous, x(k), k ∈ Z+ , is the unique solution to (2.74) so that s(k +kn , x0 ) = s(k, s(kn , x0 )) = s(k, x(kn )). Now, since s(k, x0 ), k ∈ Z+ , is continuous with respect to x0 , it follows that, for k + kn ≥ 0, limn→∞ s(k + kn , x0 ) = limn→∞ s(k, x(kn )) = s(k, p), and hence, s(k, p) ∈ ω(x0 ). Hence, sk (ω(x0 )) ⊆ ω(x0 ), k ∈ Z+ , establishing positive invariance of ω(x0 ). To show invariance of ω(x0 ) let y ∈ ω(x0 ) so that there exists an increasing unbounded sequence {kn }∞ n=0 such that s(kn , x0 ) → y as n → ∞. Next, let k ∈ Z+ and note that there exists N such that kn > k, n ≥ N . Hence, it follows from the semigroup property that s(k, s(kn − k, x0 )) = s(kn , x0 ) → y as n → ∞. Now, it follows from the Bolzano-Lebesgue theorem [112, p. 28] that there exists a subsequence zni of the sequence zn = s(kn −k, x0 ), n = N, N +1, . . ., such that zni → z ∈ D and, by definition, z ∈ ω(x0 ). Next, it follows from the continuous dependence property that limi→∞ s(k, zni ) = s(k, limi→∞ zni ), and hence, y = s(t, z), which implies that ω(x0 ) ⊆ sk (ω(x0 )), k ∈ Z+ . Now, using positive invariance of ω(x0 ) it follows that sk (ω(x0 )) = ω(x0 ), k ∈ Z+ , establishing invariance of the positive limit set ω(x0 ). Finally, to show x(k) → ω(x0 ) as k → ∞, suppose, ad absurdum, that x(k) 6→ ω(x0 ) as k → ∞. In this case, there exists a sequence {kn }∞ n=0 , with kn → ∞ as n → ∞, such that inf

p∈ω(x0 )

kx(kn ) − pk > ε,

n ∈ Z+ .

(2.81)

However, since x(k), k ∈ Z+ , is bounded, the bounded sequence {x(kn )}∞ n=0 ∗ ∗ contains a convergent subsequence {x(kn∗ )}∞ n=0 such that x(kn ) → p ∈ ω(x0 ) as n → ∞, which contradicts (2.81). Hence, x(k) → ω(x0 ) as k → ∞ and, n since f is nonnegative, ω(x0 ) ⊆ R+ . Next, we present a discrete-time version of the Krasovskii-LaSalle invariance principle.

NNBook

September 1, 2009

67

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

Theorem 2.17. Consider the discrete-time nonlinear dynamical sysn tem (2.74) where f is nonnegative, assume Dc ⊂ D ⊆ R+ is a compact invariant set with respect to (2.74), and assume that there exists a continuous function V : Dc → R such that V (f (x)) − V (x) ≤ 0, x ∈ Dc . Let △ R = {x ∈ Dc : V (f (x)) = V (x)} and let M denote the largest invariant set contained in R. If x(0) ∈ Dc , then x(k) → M as k → ∞. Proof. Let x(k), k ∈ Z+ , be a solution to (2.74) with x(0) ∈ Dc . Since V (f (x)) ≤ V (x), x ∈ Dc , it follows that V (x(k)) − V (x(κ)) =

k−1 X i=κ

[V (f (x(i))) − V (x(i))] ≤ 0,

k − 1 ≥ κ,

and hence, V (x(k)) ≤ V (x(κ)), k − 1 ≥ κ, which implies that V (x(k)) is a nonincreasing function of k. Next, since V (x) is continuous on the compact set Dc , there exists L ≥ 0 such that V (x) ≥ L, x ∈ Dc . Hence, △ γx0 = limk→∞ V (x(k)) exists. Now, for all p ∈ ω(x0 ) there exists an increasing unbounded sequence {kn }∞ n=0 , with k0 = 0, such that x(kn ) → p as n → ∞. Since V (x), x ∈ D, is continuous, V (p) = V (limn→∞ x(kn )) = limn→∞ V (x(kn )) = γx0 , and hence, V (x) = α on ω(x0 ). Now, since Dc is compact and invariant it follows that x(k), k ∈ Z+ , is bounded, and hence, it follows from Lemma 2.5 that ω(x0 ) is a nonempty, compact invariant set. Hence, it follows that V (f (x)) = V (x) on ω(x0 ), and hence, ω(x0 ) ⊂ M ⊂ R ⊂ Dc . Finally, since x(k) → ω(x0 ) as k → ∞ it follows that x(k) → M as k → ∞. Now, using Theorem 2.17 we provide a generalization of Theorem 2.16 for local asymptotic stability of a nonlinear dynamical system. Corollary 2.2. Consider the nonlinear dynamical system (2.74) where n f is nonnegative, assume Dc ⊂ D ⊆ R+ is a compact invariant set with respect to (2.74) such that xe ∈ Dc , and assume that there exists a continuous function V : Dc → R such that V (xe ) = 0, V (x) > 0, x 6= xe , and V (f (x)) − V (x) ≤ 0, x ∈ Dc . Furthermore, assume that the set △ R = {x ∈ Dc : V (f (x)) = V (x)} contains no invariant set other than the set {xe }. Then the equilibrium solution x(k) ≡ xe to (2.74) is asymptotically n stable with respect to R+ . Proof. The result is a direct consequence of Theorem 2.17. In Theorem 2.17 and Corollary 2.2 we explicitly assumed that there n exists a compact invariant set Dc ⊂ D ⊆ R+ of (2.74). Next, we provide a result that does not require the existence of a compact invariant set Dc .

NNBook

September 1, 2009

68

CHAPTER 2

Theorem 2.18. Consider the nonlinear dynamical system (2.74) where f is nonnegative and assume that there exists a continuous function n V : R+ → R such that V (xe ) = 0, V (x) > 0, V (f (x)) − V (x) ≤ 0,

x∈

x∈

n R+ , n R+ .

(2.82) (2.83)

x 6= xe ,

(2.84)

Let R = {x ∈ Rn : V (f (x)) = V (x)} and let M be the largest invariant set contained in R. Then all solutions x(k), k ∈ Z+ , to (2.74) that are bounded approach M as k → ∞. △

n

Proof. Let x ∈ R+ be such that trajectory s(k, x), k ∈ Z+ , of (2.74) n is bounded. Since f is nonnegative it follows that s(k, x) ∈ R+ , k ∈ Z+ . Now, with Dc = Ox+ , it follows from Theorem 2.17 that s(k, x) → M as k → ∞. Next, we present the global invariant set theorem for guaranteeing global asymptotic stability of a discrete-time nonlinear dynamical system. Theorem 2.19. Consider the nonlinear dynamical system (2.74) where n f is nonnegative and assume that there exists a continuous function V : R+ → R such that V (xe ) = 0, V (x) > 0,

x∈

n R+ , n R+ ,

(2.85) (2.86)

x 6= xe ,

V (f (x)) − V (x) ≤ 0, x ∈ V (x) → ∞ as kxk → ∞.

(2.87) (2.88)

△

Furthermore, assume that the set R = {x ∈ D: V (f (x)) = V (x)} contains no invariant set other than the set {xe }. Then the equilibrium solution n x(k) ≡ xe to (2.74) is globally asymptotically stable with respect to R+ . Proof. Since (2.85)–(2.87) hold, it follows from Theorem 2.16 that the equilibrium solution x(k) ≡ xe to (2.74) is Lyapunov stable, while the radial unboundedness condition (2.88) implies that all solutions to (2.74) are bounded. Now, Theorem 2.18 implies that x(k) → M as k → ∞. However, since R contains no invariant set other than the set {xe }, the set M is {xe }, and hence, global asymptotic stability is immediate. Finally, we introduce the notion of semistability for discrete-time nonnegative dynamical systems. n

Definition 2.18. An equilibrium solution x(k) ≡ xe ∈ R+ to (2.74) is

NNBook

September 1, 2009

69

STABILITY THEORY FOR NONNEGATIVE SYSTEMS n

n

semistable with respect to R+ if it is Lyapunov stable with respect to R+ and n there exists δ > 0 such that if x0 ∈ Bδ (xe )∩R+ , then limk→∞ x(k) exists and n corresponds to a Lyapunov stable equilibrium point with respect to R+ . An n equilibrium point xe ∈ R is a globally semistable equilibrium with respect n n n to R+ if it is Lyapunov stable with respect to R+ and, for every x0 ∈ R+ , limk→∞ x(k) exists and corresponds to a Lyapunov stable equilibrium point n with respect to R+ . The system (2.74) is said to be Lyapunov stable with n respect to R+ if every equilibrium point of (2.74) is Lyapunov stable with n respect to R+ . The system (2.74) is said to be semistable with respect to n n R+ if every equilibrium point of (2.74) is semistable with respect to R+ . n Finally, (2.74) is said to be globally semistable with respect to R+ if every n equilibrium point of (2.74) is globally semistable with respect to R+ . n

Theorem 2.20. Let Dc ⊂ R+ be a compact invariant set with respect to (2.74). Suppose there exists a continuous function V : Dc → R such that V (f (x)) − V (x) ≤ 0, x ∈ Dc . Let R , {x ∈ Dc : V (f (x)) = V (x)} and let M denote the largest invariant set contained in R. If every element in M is a Lyapunov stable equilibrium point with respect to Dc , then (2.74) is semistable with respect to Dc . Proof. Since every solution of (2.74) is bounded, it follows from the hypotheses on V (·) that, for every x ∈ Dc , the positive limit set ω(x) of (2.74) is nonempty and contained in the largest invariant subset M of R. Since every point in M is a Lyapunov stable equilibrium point, it follows that every point in ω(x) is a Lyapunov stable equilibrium point. Next, let z ∈ ω(x) and let Uε be a relatively open neighborhood of z. By Lyapunov stability of z, it follows that there exists a relatively open n n subset Uδ containing z such that sk (Uδ ∩ R+ ) ⊆ Uε ∩ R+ for every k ≥ k0 . n Since z ∈ ω(x), it follows that there exists h ≥ 0 such that s(h, x) ∈ Uδ ∩ R+ . n n Thus, s(k + h, x) = sk (s(h, x)) ∈ sk (Uδ ∩ R+ ) ⊆ Uε ∩ R+ for every k > k0 . Hence, since Uε was chosen arbitrarily, it follows that z = limk→∞ s(k, x). Now, it follows that limi→∞ s(ki , x) → z for every divergent sequence {ki }, and hence, ω(x) = {z}. Finally, since limk→∞ s(k, x) ∈ M is Lyapunov stable for every x ∈ Dc , it follows from the definition of semistability that every equilibrium point in M is semistable.

2.10 Stability Theory for Discrete-Time Linear Nonnegative Dynamical Systems In this section, we provide necessary and sufficient conditions for stability of linear discrete-time nonnegative dynamical systems. First, however, we

NNBook

September 1, 2009

70

CHAPTER 2

introduce several definitions and some key results concerning nonnegative matrices [21, 22, 145, 218] that are necessary for developing the main results of this section. n×n . Then A is (discrete-time) compartDefinition 2.19. Let A ∈ PR n mental if A is nonnegative and i=1 A(i,j) ≤ 1, j = 1, . . . , n, or, equivalently, eT (A − In ) ≤≤ 0.

Definition 2.20. A real function u : Z+ → Rm is a nonnegative (resp., positive) function if u(k) ≥≥ 0 (resp., u(k) >> 0), k ∈ Z+ . The following lemma is needed for developing several stability results of this monograph. Lemma 2.6. Let A ∈ Rn×n be nonnegative. statements are equivalent:

Then the following

i ) I − A is an M-matrix. ii ) ρ(A) ≤ 1. Furthermore, the following statements are equivalent: iii ) I − A is a nonsingular M-matrix.

iv ) det(I − A) 6= 0 and (I − A)−1 ≥≥ 0.

v ) For each y ∈ Rn , y ≥≥ 0, there exists a unique x ∈ Rn , x ≥≥ 0, such that (I − A)x = y.

vi ) There exists x ∈ Rn , x ≥≥ 0, such that x >> Ax.

vii ) There exists x ∈ Rn , x >> 0, such that x >> Ax. Proof. Since A ≥≥ 0 it follows from Theorem 2.9 that ρ(A) ∈ spec(A), and hence, ρ(A) = max{Re λ : λ ∈ spec(A)}. The equivalence of i) and ii) follows by noting that I − A is an M-matrix if and only if Re λ ≤ 1 for all λ ∈ spec(A) or, equivalently, max{Re λ : λ ∈ spec(A)} ≤ 1. The equivalence of statements iii)–vii) follows from v)–ix) of Lemma 2.2 with A replaced by I − A. Note that if f (x) = Ax, where A ∈ Rn×n , then f is nonnegative if and only if A is nonnegative. To address discrete-time linear nonnegative dynamical systems, consider (2.74) with f (x) = Ax so that x(k + 1) = Ax(k),

x(0) = x0 ,

k ∈ Z+ ,

(2.89)

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

71

where x(k) ∈ Rn , k ∈ Z+ , and A ∈ Rn×n . Since the solution to (2.35) is given by x(k) = Ak x0 it follows that x(k) ≥≥ 0, k ∈ Z+ , if and only if A is nonnegative. Henceforth, we assume that A is nonnegative. Definition 2.21. Let A ∈ Rn×n . Then: i ) A is (discrete-time) Lyapunov stable if spec(A) ⊂ {z ∈ C : |z| ≤ 1} and, if λ ∈ spec(A) and |λ| = 1, then λ is semisimple. ii ) A is (discrete-time) semistable if spec(A) ⊂ {z ∈ C : |z| < 1} ∪ {1} and, if 1 ∈ spec(A), then 1 is semisimple. iii ) A is (discrete-time) asymptotically stable or Schur if spec(A) ⊂ {z ∈ C : |z| < 1}. The following proposition concerning Lyapunov stability, semistability, and asymptotic stability of (2.89) is immediate. This result holds whether or not A is a nonnegative matrix. Proposition 2.12 ([23]). Let A ∈ Rn×n and consider the linear discrete-time dynamical system (2.89). Then the following statements are equivalent: i ) A is Lyapunov stable. ii ) For every initial condition x(0) ∈ Rn , the sequence {kx(k)k}∞ k=1 is bounded, where k · k is a vector norm on Rn . iii) For every initial condition x(0) ∈ Rn , the sequence {kAk x(0)k}∞ k=1 is bounded, where k · k is a vector norm on Rn . iv ) The sequence {kAk k}∞ k=1 is bounded, where k · k is a matrix norm on Rn×n . The following statements are equivalent: v ) A is semistable. vi ) limk→∞ Ak exists. In fact, limk→∞ Ak = I − (I − A)# (I − A). vii ) For every initial condition x(0) ∈ Rn , limk→∞ x(k) exists. The following statements are equivalent:

NNBook

September 1, 2009

72

CHAPTER 2

viii ) A is asymptotically stable. ix ) ρ(A) < 1. x ) For every initial condition x(0) ∈ Rn , limk→∞ x(k) = 0. xi ) For every initial condition x(0) ∈ Rn , limk→∞ Ak x(0) = 0. xii ) limk→∞ Ak = 0. The following theorem gives several properties of a discrete-time nonnegative dynamical system when a Lyapunov-like equation is satisfied for (2.89). Note that it follows from Proposition 2.12 that if A is asymptotically stable, then N (A − I) = {0}. Theorem 2.21. Let A ∈ Rn×n be nonnegative. If there exist p, r ∈ Rn such that p >> 0 and r ≥≥ 0 satisfy p = AT p + r,

(2.90)

then the following statements hold: i ) I − A is an M -matrix. Pk i # k+1 − I) + [I − (A − I)(A − I)# ](k + 1), k ∈ Z . ii ) + i=0 A = (A − I) (A

iii) A − I is nonsingular if and only if I − A is a nonsingular M-matrix.

Proof. i) The proof is a direct consequence of i) of Theorem 2.10 with A replaced by A − I. ii) Since I − A is an M-matrix is follows that Re λ < 1 or λ = 1 for all λ ∈ spec(A). Hence, the eigenvalue λ = 1 (if it exists) is semisimple and it follows from the Jordan decomposition that there exist invertible matrices J ∈ Rr×r and S ∈ Rn×n , where r = rank(I − A), such that   Ir − J 0 A=S S −1 . 0 In−r Hence, it follows that   k k X X (Ir − J)i 0 Ai = S S −1 0 In−r i=0 i=0  Pk  (Ir − J)i 0 i=0 = S S −1 0 In−r (k + 1)   −J −1 ([Ir − J]k+1 − Ir ) 0 = S S −1 0 In−r (k + 1)

NNBook

September 1, 2009

73

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

wi (k) ' ?

ith Subsystem xi (k) &

wj (k) $ ) %

aij xj (k)

aji xi (k)

'

?

jth Subsystem xj (k) 1&

aii xi (k) ?

$ %

ajj xj (k) ?

Figure 2.11 Discrete-time linear compartmental interconnected subsystem model.

  −J −1 0 ([Ir − J]k+1 − Ir ) −1 = S S S 0 0 0   0 0 +S S −1 0 In−r (k + 1)   J − Ir # k+1 − I) + In − S = (A − I) (A 0    (J − Ir )−1 0 ·S S −1 (k + 1) 0 0 

0 0

0 0





S −1

S −1

= (A − I)# (Ak+1 − I) + (I − (A − I)(A − I)# )(k + 1),

k ∈ Z+ .

iii) The result follows from i). Next, we show that discrete-time linear compartmental dynamical systems [4,29,88,100,155,158,209,211,220,259] are a special case of discretetime nonnegative dynamical systems. To see this, let xi (k), i = 1, . . . , n, denote the mass (and hence a nonnegative quantity) of the ith subsystem of the compartmental system shown in Figure 2.11, let aii ≥ 0 denote the loss coefficient of the ith subsystem averaged over the discretization interval h, let wi (k) ≥ 0, i = 1, . . . , n, denote the mass inflow supplied to the ith subsystem, and let φij (k), i 6= j, i, j = 1, . . . , n, denote the net mass flow from the jth subsystem to the ith subsystem given by φij (k) = aij xj (k) − aji xi (k), k ∈ Z+ , where the transfer coefficient aij ≥ 0, i 6= j, i, j = 1, . . . , n, is averaged over the discretization interval h. A mass balance for the whole compartmental system with time step h = 1 yields ∆xi (k) , xi (k + 1) − xi (k)

NNBook

September 1, 2009

74

CHAPTER 2

= −aii xi (k) +

n X

j=1,i6=j

φij (k) + wi (k), k ∈ Z+ , i = 1, . . . , n, (2.91)

or, equivalently, x(k + 1) = Ax(k) + w(k),

x(0) = x0 ,

k ∈ Z+ ,

(2.92)

where x(k) = [x1 (k), . . . , xn (k)]T , w(k) = [w1 (k), . . . , wn (k)]T , and for i, j = 1, . . . , n,  P 1 − nl=1 ali , i = j, A(i,j) = (2.93) aij , i 6= j.

Note that since at any given instant of time mass can only be transported, stored, or discharged but not created and the maximum amount of mass that can be transported and/or P discharged cannot exceed the mass in a compartment, it follows that 1 ≥ nl=1 ali . Thus A is a nonnegative matrix, and hence, the compartmental model given by (2.91) is a nonnegative dynamical system. Furthermore, note that AT e = [−a11 , −a22 , . . . , −ann ]T + e,

and hence, with p = e and r = (I − A)T e ≥≥ 0, it follows that (2.90) is satisfied, which implies that the compartmental model given byP(2.91) n (w(k) ≡ 0) is Lyapunov stable. In this case, V (x) = eT x = i=1 xi denoting the total mass of the system serves as a Lyapunov function for the undisturbed (w(k) ≡ 0) system (2.91) with ∆V (x) = V (Ax) − V (x) = eT (A − I)x n n X X xi = (1 − aii )xi − i=1

i=1

= − ≤ 0,

n X

aii xi

i=1

n

x ∈ R+ .

As in the continuous-time case, the compartmental system (2.91) with no inflows, that is, wi (k) ≡ 0, i = 1, . . . , n, is said to be inflow-closed [155]. Alternatively, if (2.91) possesses no losses (outflows) it is said to be outflowclosed [155]. A compartmental system is said to be closed if it is inflow-closed n and outflow-closed. Note that for a closed system ∆V (x) = 0, x ∈ R+ , which shows that the total mass inside a closed system is conserved. Alternatively, it follows that (2.91) can be equivalently written as  T ∂V (x(k)) ∆x(k) = [Jn (x(k)) − D(x(k))] + w(k), x(0) = x0 , k ∈ Z+ , ∂x

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

75

where Jn (x) is a skew-symmetric matrix function with Jn(i,i) (x) = 0 and Jn(i,j) (x) = aij xj − aji xi , i 6= j, and D(x) = diag[a11 x1 , a22 x2 , . . . , ann xn ] n ≥≥ 0, x ∈ R+ . Hence, a discrete-time linear compartmental system is a discrete-time port-controlled Hamiltonian system with a Hamiltonian n H(x) = V (x) = eT x, x ∈ R+ , representing the total mass in the system, n Jn (x), x ∈ R+ , representing the internal system interconnection structure, n D(x), x ∈ R+ , representing the outflow dissipation, and w(k) representing the supplied mass to the system. This observation shows that discrete-time closed compartmental systems are conservative systems. This will be further elaborated in Chapter 5. Next, motivated by the fact that for a discrete-time compartmental system the total mass in the system can serve as a Lyapunov function, we give necessary and sufficient conditions for Lyapunov stability and asymptotic stability for discrete-time linear nonnegative dynamical systems using linear Lyapunov functions. Theorem 2.22. Consider the discrete-time linear dynamical system given by (2.89) where A ∈ Rn×n is nonnegative. Then the following statements hold: i ) If there exist vectors p, r ∈ Rn such that p >> 0 and r ≥≥ 0 satisfy p = AT p + r,

(2.94)

then A is Lyapunov stable. ii ) If A is Lyapunov stable, then there exist vectors p, r ∈ Rn such that p ≥≥ 0, p 6= 0, and r ≥≥ 0 satisfy (2.94).

iii ) If there exist vectors p, r ∈ Rn such that p ≥≥ 0 and r ≥≥ 0 satisfy (2.94) and (A, r T ) is observable, then p >> 0 and (2.89) is asymptotically stable. Furthermore, the following statements are equivalent: iv ) A is asymptotically stable. v ) There exist vectors p, r ∈ Rn such that p >> 0 and r >> 0 satisfy (2.94). vi ) There exist vectors p, r ∈ Rn such that p ≥≥ 0 and r >> 0 satisfy (2.94). vii ) For every r ∈ Rn such that r >> 0, there exists p ∈ Rn such that p >> 0 satisfies (2.94).

NNBook

September 1, 2009

76

CHAPTER 2

Proof. i) Consider the linear Lyapunov function candidate V (x) = n Note that V (0) = 0 and V (x) > 0, x ∈ R+ , x 6= 0. Now, computing the Lyapunov difference yields

pT x.

∆V (x) , V (Ax) − V (x) = pT Ax − pT x = −r T x ≤ 0,

n

x ∈ R+ ,

establishing Lyapunov stability. ii) If A is Lyapunov stable it follows from i) of Lemma 2.6 that (I −A)T is an M-matrix. Since AT ≥≥ 0, it follows from Theorem 2.9 that ρ(A) ∈ spec(A), and hence, there exists p ≥≥ 0, p 6= 0, such that AT p = ρ(A)p. Thus, AT p − p = (ρ(A) − 1)p ≤≤ 0, which proves that there exist p ≥≥ 0, p 6= 0, and r ≥≥ 0 such that (2.94) holds. iii) Assume there exist p ≥≥ 0 and r ≥≥ 0 such that (2.94) holds and suppose (A, r T ) is observable. Now, consider the function V (x) = pT x, n n x ∈ R+ , and note that, since V (x) ≥ 0, x ∈ R+ , and ∆V (x) = pT Ax−pT x = n △ −r T x ≤ 0, it follows that if x(0) ∈ P = {x ∈ R+ : pT x = 0}, then V (x(k)) = 0, k ∈ Z+ , which implies that 0 ≥ ∆V (x(k)) = pT Ax(k) − pT x(k) = pT Ax(k) ≥ 0, k ∈ Z+ . Specifically, ∆V (x(0)) = pT Ax(0) = 0. Hence, if x ˆ ∈ P then ∆V (ˆ x) = pT Aˆ x = −r T x ˆ = 0. Thus, if x ˆ ∈ P n △ T then Aˆ x ∈ P and x ˆ ∈ Q = {x ∈ R+ : r x = 0}. Now, since Aˆ x ∈ P it follows that A2 x ˆ ∈ P and Aˆ x ∈ Q. Repeating these arguments yields Ak x ˆ ∈ Q, k = 0, 1, . . . , n−1, or, equivalently, r T Ak x ˆ = 0, k = 0, 1, . . . , n−1. Now, since (A, r T ) is observable it follows that x ˆ = 0 and P = {0}, which implies that p >> 0. Asymptotic stability of (2.89) now follows as a direct consequence of the Krasovskii-LaSalle theorem with V (x) = pT x and using the fact that (A, r T ) is observable. To show the equivalence among iv)–vii) first suppose there exist p ≥≥ 0 and r >> 0 such that (2.94) holds. Now, there exists sufficiently small ε > 0 such that AT (p + εe) > 0, which proves that vi) implies v). Since v) implies vi) it trivially follows that v) and vi) are equivalent. Now, suppose v) holds, that is, there exist p >> 0 and r >> 0 such that (2.94) holds, and consider the Lyapunov function n candidate V (x) = pT x, x ∈ R+ . Computing the Lyapunov difference yields n ∆V (x) = pT Ax − pT x = −r T x < 0, x ∈ R+ , x 6= 0, and hence, it follows that (2.89) is asymptotically stable. Thus, v) implies iv). Next, suppose (2.89) is asymptotically stable. Hence, (I − A)−T ≥≥ 0 and thus for every △ r ∈ Rn+ , p = (I − A)−T r ≥≥ 0 satisfies (2.94), which proves that iv) implies vi). Finally, suppose (2.89) is asymptotically stable. Now, as in the proof n given above, for every r ∈ Rn+ , there exists p ∈ R+ such that (2.94) holds. n Next, suppose, ad absurdum, that there exists x ∈ R+ , x 6= 0, such that

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

77

xT p = 0, that is, there exists at least one i ∈ {1, 2, . . . , n} such that pi = 0. Hence, xT (I − A)−T r = 0. However, since (I − A)−T ≥≥ 0 it follows that (I − A)−1 x ≥≥ 0 and, since r >> 0, it follows that (I − A)−1 x = 0, which implies that x = 0 yielding a contradiction. Hence, for every r ∈ Rn+ , there exists p ∈ Rn+ such that (2.94) holds, which proves iv) implies vii). Since vii) implies v) trivially, the equivalence of iv)–vii) is established. Next, we give necessary and sufficient conditions for semistability of a nonnegative matrix. Theorem 2.23. Let A ∈ Rn×n be nonnegative. A is semistable if and only if |λ| < 1 or λ = 1 and λ = 1 is semisimple, where λ ∈ spec(A). Furthermore, if A is semistable then limk→∞ Ak = I −(A−I)(A−I)# ≥≥ 0. Proof. The result is a direct consequence of Proposition 2.12. For completeness of exposition, we provide a proof here. If |λ| < 1 or λ = 1 and λ = 1 is semisimple, then A is Lyapunov stable. Now, it follows from the Jordan decomposition that there exists an invertible matrix S ∈ Cn×n such that   J 0 A=S S −1 , 0 In−r where J ∈ Cr×r , r = rank A, and ρ(J) < 1. Hence, it follows that  k  J 0 S −1 lim Ak = lim S 0 In−r k→∞ k→∞   0 0 = S S −1 0 In−r     J − Ir 0 (J − Ir )−1 0 −1 = In − S S S S −1 0 0 0 0 = In − (A − In )(A − In )# ,

which implies that limk→∞ Ak exists. Hence, A is semistable. Furthermore, since Ak ≥≥ 0, k ∈ Z+ , it follows that I − (A − I)(A − I)# ≥≥ 0. Conversely, suppose A is semistable, that is, limk→∞ Ak exists. Since A is semistable it follows that A is Lyapunov stable, that is, if λ ∈ spec(A), then either |λ| < 1, or |λ| = 1 and λ is semisimple. Now, it follows from the Jordan decomposition that there exists an invertible matrix S ∈ Cn×n such that   J1 0 A=S S −1 , 0 J2 where J1 ∈ Cr×r such that ρ(J1 ) < 1 and J2 ∈ C(n−r)×(n−r) is diagonal such

NNBook

September 1, 2009

78

CHAPTER 2

that |λ| = 1, λ ∈ spec(J2 ). Hence,  k  J1 0 k A =S S −1 . 0 J2k Now, since for λ ∈ C, |λ| = 1, limk→∞ λk exists if and only if λ = 1, it follows that limk→∞ Ak exists if and only if J2 = In−r , which proves the result. It is important to note that unlike the case of continuous-time linear nonnegative systems, Lyapunov stability of discrete-time linear nonnegative systems does not necessarily imply semistability of a discrete-time linear nonnegative system. To see this, let n = 2 and let   0 1 A= 1 0

so that A is nonnegative and discrete-time Lyapunov stable. In this case, limk→∞ Ak does not exist, which shows that A is not semistable. However, if the discrete-time linear nonnegative system is derived from the discretization of a continuous-time linear nonnegative semistable system, then it can be shown that the discrete-time system is semistable. To see this, consider the continuous-time linear nonnegative system x(t) ˙ = Ac x(t),

t ≥ 0,

x(0) = x0 ,

(2.95)

where x(t) ∈ Rn , t ≥ 0, Ac ∈ Rn×n is essentially nonnegative and semistable, so that the discretization of (2.95) (with sampling rate h = 1) is given by x(k + 1) = Ad x(k),

x(0) = x0 ,

eAc .

k ∈ Z+ ,

(2.96)

where Ad = In this case, since Ac is (continuous-time) semistable, it follows from the real Jordan decomposition that there exists an invertible matrix S ∈ Rn×n such that   Jc 0 Ac = S S −1 , 0 0 where Jc is Hurwitz, so that Ac

Ad = e

=S



eJc 0 0 I



S −1 .

Since ρ(eJc ) < 1, it follows that if λ ∈ spec(Ad ), then either |λ| < 1, or λ = 1 and λ is semisimple. Now, it follows from Theorem 2.23 that Ad is (discrete-time) semistable. Next, we show that every asymptotically stable discrete-time linear nonnegative system is equivalent, modulo a similarity transformation, to a compartmental system.

NNBook

September 1, 2009

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

79

Proposition 2.13. Let A ∈ Rn×n be nonnegative and asymptotically n×n stable. Then there Pn exists a diagonal invertible matrix S ∈ R −1 such that ˆ ˆ ˆ A(i,j) ≥ 0 and k=1 A(k,j) ≤ 1, i, j = 1, . . . , n, where A = SAS .

Proof. It follows from iv) and v) of Theorem 2.22 that there exists △ p ∈ Rn+ such that AT p − p 0 and diagonal, implies that AˆT e > 0 satisfies AT p 0 such that (2.97) holds, then asymptotic stability is immediate with the quadratic Lyapunov function V (x) = xT P x. To show necessity, assume that A is nonnegative and Schur. Then it follows from Lemma 2.7 that there exist vectors l, m ∈ Rn , with l >> 0 and m >> 0 such that AT l 0, and hence, vii) of Theorem 2.22 implies that vec P >> 0, which establishes that P >> 0. Example 2.11. This example models the flow of thyroxine when injected into the blood stream and then carried into the liver where it is converted into iodine which in turn is absorbed into the bile. A simple yet accurate model for the flow of thyroxine into the blood stream is captured

NNBook

September 1, 2009

81

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

by the three-compartment model shown in Figure 2.12. u1 ' ?

'

$

%

1 &

%

a12

Compartment I blood

&

)$

a21

Compartment II liver

a32

'

$

&

%

- Compartment III bile

Figure 2.12 Three-compartment thyroxine model.

The first compartment corresponds to the blood plasma, the second compartment to the liver, and the third compartment to the bile. Note that the feedback path between Compartments I and II reflects the fact that neither conversion nor absorption occurs instantaneously; that is, a fraction of the thyroxine that enters the liver is fed back to the blood before conversion to iodine. To analyze this compartmental system let, for i, j = 1, 2, 3, xi denote the concentration of thyroxine in compartment i, aij denote the instantaneous transfer coefficient of thyroxine flow rates from compartment j to compartment i in units of (time)−1 , and u1 denote a bolus (impulse) injection into the blood stream. Since at t = 0 no absorption or transfer of thyroxine can occur it follows that x2 (0) = x3 (0) = 0. Furthermore, since the input material is a bolus injection we can always reproduce the impulsive response with the free response by setting x3 (0) = bv, where v ∈ R denotes the impulse strength. Hence, a mass balance of the three-state compartment model shown in Figure 2.12 yields (2.95) with   −a21 a12 0 Ac =  a21 −(a12 + a32 ) 0  . (2.98) 0 a32 0

Now, it follows that the discretization of (2.95) is given by (2.96) where Ad = eAc h and h is the sampling rate. Note that since Ac is essentially nonnegative it follows from Proposition 2.5 that Ad ≥≥ 0. Hence, Akd ≥≥ 0, k ∈ Z+ , and consequently if x(0) is nonnegative, then the solution x(k) = Akd x(0) is nonnegative for all k ∈ Z+ . we assume that the sample rate h is small so that Ad = eAc h = P∞ Next, −1 i i=0 (i!) (Ac h) ≈ I + hAc . In this case,   1 − ha21 ha12 0 1 − h(a12 + a32 ) 0  . Ad =  ha21 (2.99) 0 ha32 1 Furthermore, since Ad − I is singular it follows that the set of equilibria

NNBook

September 1, 2009

82

CHAPTER 2 3

E = {(x1 , x2 , x3 ) ∈ R+ : Ad x = x} = N (Ad − I) = {(x1 , x2 , x3 ) ∈ 3 R+ : (0, 0, x3 )}. Now, taking p = e >> 0 and r = [0, 0, 0]T , it follows that (2.94) holds, and hence, the discretized system is Lyapunov stable by i) of Theorem 2.22. In addition, since spec(Ad ) = {1 − p h h (a12 + a21 + a32 )2 − 4a21 a32 , 1 − h2 (a12 + a21 + a32 ) − 12 + a21 + a32 ) − 2 2 (a p h (a12 + a21 + a32 )2 − 4a21 a32 , 1}, it follows from Theorem 2.23 that the 2 discretized system is semistable. △ Example 2.12. One of the key areas in which discrete-time nonnegative dynamical systems arise is age-structured population models. Typically in such systems the state variables can present biomass, population density, or the number of species in a population. One of the most classical models for developing demographic projections in age-structured population models is the Leslie model [51, 225]. In this model the discrete-time k denotes the year or reproduction season and the state variables xi (k), k ∈ Z+ , typically denote the number of females of age i at the beginning of year k. Alternatively, xi (k), k ∈ Z+ , can denote the number of males, individuals, or couples of each age class [51]. Assuming a balanced sex ratio with equal probability of survival between males and females, the aging process is characterized by xi+1 (k + 1) = σi xi (k),

i = 1, . . . , n − 1,

(2.100)

where σi is the probability of survival from age i to age i + 1. Note that δi = 1 − σi gives the probability of dying during the same age. The first state equation captures the reproduction process and is given by x1 (k + 1) = σ0 (φ1 x1 (k) + φ2 x2 (k) + · · · + φn xn (k)),

(2.101)

where σ0 is the probability of survival in the first year of life and φi ≥ 0, i = 1, . . . , n, is the fertility rate of females at age i. Hence, the Leslie model is given by (2.89) where   σ0 φ1 σ0 φ2 . . . σ0 φn−1 σ0 φn  σ1 0 ... 0 0     σ2 . . . 0 0  A= 0 (2.102) .  .. .. .. .. ..   . . . . .  0 0 . . . σn−1 0 Note that A ≥≥ 0, and hence, Ak ≥≥ 0, k ∈ Z+ . Hence, if x(0) is nonnegative, then demographic projections obtained from the Leslie model given by x(k) = Ak x(0) are nonnegative for all k ∈ Z+ . To study the stability of this model form χA (λ) = det(λIn − A + In )

NNBook

September 1, 2009

83

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

= (λ + 1)n − σ0 φ1 (λ + 1)n−1 − σ0 σ1 φ2 (λ + 1)n−2 −σ0 σ1 σ2 φ3 (λ + 1)n−3 − · · · − σ0 σ1 · · · σn−1 φn ,

(2.103)

or, equivalently using the binomial theorem,   n−2 χA (λ) = λn + (n − σ0 φ1 )λn−1 + n(n−1) − σ0 φ1 (n − 1) − σ0 σ1 φ2 λ 2  n(n − 1)(n − 2) (n − 1)(n − 2) − σ0 φ1 + 6 2  −σ0 σ1 φ2 (n − 2) − σ0 σ1 σ2 φ3 λn−3

+ · · · + (1 − σ0 φ1 − σ0 σ1 φ2 − σ0 σ1 σ2 φ3 − · · · − σ0 σ1 · · · σn−1 φn ). (2.104)

Now, it follows from Proposition 2.15 that the Leslie system is asymptotically stable if and only if the coefficients of χA (λ) are positive. Note that σ0 σ1 · · · σi−1 denotes the probability that a newly born female will survive to the age of i. Furthermore, the coefficient of (2.104) corresponding to the ith age is given by αi =

( 

n i



−

i X l=0

" 

n−l i−l



φl

l−1 Y

σj

j=0

# )

,

(2.105)

   n n n(n−1)···(n−i+1) and , 1. Hence, if αi > 0, i = where , i! 0 i 1, . . . , n, then the population will become extinct; otherwise if αi < 0, i = 1, . . . , n, the population will experience a demographic explosion. Clearly what governs the sign of αi are the fertility rates of the females up to age i and the probability that a newly born female will survive to the age of i. △ 

2.11 Discrete-Time Nonlinear Compartmental Dynamical Systems In this section, we develop linearization results for discrete-time nonlinear nonnegative dynamical systems as well as several stability results for nonlinear compartmental systems. First, we show that if a nonlinear system is nonnegative, then its linearization is also nonnegative. Lemma 2.8. Consider the nonlinear dynamical system (2.74) where f (0) = 0 and f : D → Rn is nonnegative and continuously differentiable in n △ R+ . Then, A = ∂f is nonnegative. ∂x (x) x=0

NNBook

September 1, 2009

84

CHAPTER 2

Proof. Since f : D → Rn is nonnegative it follows that fi (x) ≥ 0, n x ∈ R+ . Now, note that for all i, j ∈ {1, . . . , n}, ∂fi (x) A(i,j) = ∂xj x=0 fi (0, . . . , h, . . . , 0) − fi (0) = lim+ h h→0 fi (0, . . . , h, . . . , 0) , = lim+ h→0 h where h in fi (0, · · · , h, · · · , 0) is in the jth location. Hence, since fi (·) is nonnegative, A(i,j) ≥ 0, which proves nonnegativity of A. Next, we present a key result on stability of a linearized discrete-time nonlinear nonnegative dynamical system. Theorem 2.26. Let x(k) ≡ xe be an equilibrium point for the nonlinear dynamical system (2.74). Furthermore, let f : D → Rn be nonnegative and let ∂f (x) A= . ∂x x=xe Then the following statements hold:

i ) If |λ| < 1, where λ ∈ spec(A), then the equilibrium solution x(k) ≡ xe of the discrete-time nonlinear dynamical system (2.74) is asymptotically stable. ii ) If there exists λ ∈ spec(A) such that |λ| > 1, then the equilibrium solution x(k) ≡ xe of the discrete-time nonlinear dynamical system (2.74) is unstable. iii ) Let xe = 0, let |λ| < 1, where λ ∈ spec(A), let p >> 0 be such that n △ △ AT p P n △ 0 : pT f (x) < pT x, x ∈ R+ , kxk < ε} and kxk = ni=1 pi |xi |. Then DA is a subset of the domain of attraction for (2.74). Proof. Statements i) and ii) are restatements of Lyapunov’s indirect method [112] as applied to discrete-time nonlinear nonnegative dynamical systems. To prove iii) note that it follows from Lemma 2.8 that, if f : D → Rn is nonnegative, then A is nonnegative. Hence, since |λ| < 1, where λ ∈ spec(A), it follows from vi) of Theorem 2.22 that there exists p >> 0 such that AT p 0, x ∈ R+ , x 6= 0, and by (2.108), the zero solution x(k) ≡ 0 to (2.107) (with w(k) ≡ 0) is asymptotically stable. As in the linear case, nonlinear discrete-time compartmental systems are port-controlled Hamiltonian systems. This follows from the fact that (2.106) can be equivalently written as ∆x(k) = [Jn (x(k))−D(x(k))]



T ∂V (x(k)) +w(k), ∂x

x(0) = x0 ,

k ∈ Z+ ,

(2.109) where Jn (x) is a skew-symmetric matrix function with Jn(i,i) (x) = 0 and Jn(i,j) (x) = a ˆij (x) − a ˆji (x), i 6= j, and D(x) = diag[ˆ a11 (x), a ˆ22 (x), . . ., n a ˆnn (x)] ≥≥ 0, x ∈ R+ . n

Finally, if f : R+ → Rn is continuously differentiable and xe = 0 so that f (0) = 0, then it follows that (see [158] for details) a ˆij (x) = aij (x)xj , n where the state-dependent transfer coefficients aij (x) ≥ 0, x ∈ R+ , i, j = 1, . . . , n, and aij (·) is continuous. In this case, (2.106) becomes   n X aji (x(k)) xi (k) xi (k + 1) = xi (k) − aii (x(k)) + j=1,i6=j

NNBook

September 1, 2009

87

STABILITY THEORY FOR NONNEGATIVE SYSTEMS

+

n X

aij (x(k))xj (k) + wi (k),

j=1,i6=j

k ∈ Z+ ,

i = 1, . . . , n,

(2.110)

k ∈ Z+ ,

(2.111)

or, equivalently, x(k + 1) = A(x(k))x(k) + w(k),

x(0) = x0 ,

where, for i, j = 1, . . . , n, A(i,j) (x) =



1−

Pn

l=1 ali (x), i = j, aij (x), i= 6 j.

(2.112)

P Note that using identical arguments as in the linear case, 1 ≥ nl=1 ali (x), n n x ∈ R+ , and hence A(x) is nonnegative for all x ∈ R+ . Hence, once again using the total mass V (x) = eT x as a Lyapunov function for (2.111) (with w(k) ≡ 0) it follows that T

T

∆V (x) = e ∆x = e [A(x) − I]x = −

n X i=1

aii (x)xi ≤ 0,

n

x ∈ R+ , (2.113)

which shows that the zero solution x(k) ≡ 0 of the inflow-closed (w(k) ≡ 0) discrete-time system given by (2.111) is Lyapunov stable. In light of the above and (2.109) we have the following result on stability of solutions for discrete-time nonlinear inflow-closed compartmental systems. Theorem 2.27. Consider the inflow-closed nonlinear compartmental system given by (2.109) where V (x) = eT x. If Jn (0) = 0, D(x) ≥≥ 0, n x ∈ R+ , and D(0) = 0, then the zero solution x(k) ≡ 0 to (2.109) (with n w(k) ≡ 0) is Lyapunov stable. If, in addition, D(x) > 0, x ∈ R+ \{0}, then the zero solution x(k) ≡ 0 to (2.109) is asymptotically stable. Proof. Lyapunov stability follows by noting that V (x) = eT x is a Lyapunov function candidate for (2.109) and ∆V (x) = = = ≤

eT ∆xe eT [Jn (x) − D(x)] e −eT D(x)e n 0, x ∈ R+ .

n

To show asymptotic stability note that if D(x) > 0, x ∈ R+ \{0}, then n ∆V (x) = −eT D(x)e < 0, x ∈ R+ \{0}.

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Three

Stability Theory for Nonnegative and Compartmental Dynamical Systems with Time Delay

3.1 Introduction As discussed in Chapter 1, nonnegative and compartmental models play a key role in understanding many processes in biological and medical sciences [4,100,113,155,158,259]. Such models are composed of homogeneous interconnected subsystems (or compartments) which exchange variable nonnegative quantities of material with conservation laws describing transfer, accumulation, and outflows between compartments and the environment. A key physical limitation of such systems is that transfer between compartments is not instantaneous, and realistic models for capturing the dynamics of such systems should account for material, energy, or information in transit between compartments [106, 155, 210]. Hence, to accurately describe the evolution of compartmental systems, it is necessary to include in any mathematical model of the system dynamics some information about the past system states. This of course leads to (infinite-dimensional) delay dynamical systems [128, 229]. In this chapter, we develop stability theorems for time-delay nonnegative and compartmental dynamical systems. In addition, using linear Lyapunov-Krasovskii functionals, we develop necessary and sufficient conditions for asymptotic stability of linear nonnegative dynamical systems with time delay. The consideration of a linear Lyapunov-Krasovskii functional leads to a new Lyapunov-like equation for examining the stability of time-delay nonnegative dynamical systems. The motivation for using a linear Lyapunov-Krasovskii functional follows from the fact that the (infinite-dimensional) state of a retarded nonnegative dynamical system is nonnegative, and hence, a linear Lyapunov-Krasovskii functional is a valid candidate Lyapunov-Krasovskii functional. For a time-delay compartmental system, a linear Lyapunov-Krasovskii functional is shown to correspond to

NNBook

September 1, 2009

90

CHAPTER 3

the total mass of the system at a given time plus the integral of the mass flow in transit between compartments over the time intervals it takes for the mass to flow through the intercompartmental connections.

3.2 Lyapunov Stability Theory for Time-Delay Nonnegative Dynamical Systems In this section, we develop the fundamental results of Lyapunov stability theory for time-delay nonnegative dynamical systems. We begin by considering the general nonlinear autonomous time-delay dynamical system x(t) ˙ = f (x(t)) + fd (x(t − τd )),

x(θ) = φ(θ),

−τd ≤ θ ≤ 0, (3.1)

where x ∈ Rn , f : Rn → Rn , fd : Rn → Rn , f (0) = fd (0) = 0, and φ ∈ C = C([−τd , 0], Rn ), where C is a Banach space of continuous functions mapping the interval [−τd , 0] into Rn with topology of uniform convergence and designated norm given by |||φ||| = sup−τd ≤θ≤0 kφ(θ)k. Here, φ : [−τd , 0] → Rn is a continuous vector-valued function specifying the initial state of the system. Furthermore, let xt ∈ C defined by xt (θ) = x(t + θ), θ ∈ [−τd , 0], denote the (infinite-dimensional) state of (3.1) at time t corresponding to the piece of trajectories x between t − τd and t or, equivalently, the element xt in the space of continuous functions defined on the interval [−τd , 0] and taking values in Rn . Unless otherwise stated, we assume f (·) and fd (·) are locally Lipschitz continuous on Rn . Furthermore, xe ∈ Rn is an equilibrium point of (3.1) if and only if f (xe ) = 0 and fd (xe ) = 0. n

Proposition 3.1. R+ is an invariant set with respect to (3.1) if and only if f : Rn → Rn is essentially nonnegative and fd : Rn → Rn is nonnegative. n

n

Proof. Assume R+ is invariant with respect to (3.1), φ(0) = x ∈ R+ , φ(θ) = 0 for all θ ∈ [−τd , 0), and suppose, ad absurdum, that x is such that there exists i ∈ {1, . . . , q} such that xi = 0 and fi (x) < 0. Then, since f and fd are continuous and fd (0) = 0, there exists sufficiently small h > 0 such that x˙ i (t) < 0 for all t ∈ [0, h). Hence, xi (t) is strictly decreasing on [0, h), n and thus x(t) 6∈ R+ for all t ∈ (0, h), which leads to a contradiction. Next, n assume that φ(−τd ) = x ∈ R+ , φ(θ) = 0 for all θ ∈ (−τd , 0], and suppose, ad absurdum, x is such that fdi (x) < 0. Hence, there exists sufficiently small h > 0 such that x˙ i (t) < 0 for all t ∈ [0, h) and xi (t) is strictly decreasing on n [0, h), and thus, x(t) 6∈ R+ for all t ∈ (0, h), which leads to a contradiction. Conversely, assume that f is essentially nonnegative and fd is nonnegative. Now, consider the nonlinear dynamical system y(t) ˙ = f (y(t)) + w(t),

y(0) = φ(0),

t ≥ 0,

(3.2)

NNBook

September 1, 2009

91

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

where w(t) =



fd (φ(t − τd )), 0 ≤ t ≤ τd , 0, t > τd .

Note that the solution x(t), t ≥ 0, to (3.1) satisfies (3.2) for all t ∈ [0, τd ]. Furthermore, since f (·) is Lipschitz continuous it follows that y(t) = x(t), t ∈ [0, τd ]. Next, since f (·) is essentially nonnegative and w(t) ≥≥ 0 for all t ≥ 0, it follows from Proposition 4.3 that y(t) ≥≥ 0, t ≥ 0. Hence, x(t) ≥≥ 0, t ∈ [0, τd ]. Now, repeating the above procedure by replacing t with t − τd in (3.1), we can show that x(t) ≥≥ 0, t ∈ [τd , 2τd ]. Repeating this procedure iteratively, it follows that x(t) ≥≥ 0, t ≥ 0. It follows from Proposition 3.1 that if φ(θ) ≥≥ 0, θ ∈ [−τd , 0], then x(t) ≥≥ 0, t ≥ 0, if and only if f is essentially nonnegative and fd is nonnegative. In this case, we say that (3.1) is a nonnegative time-delay dynamical system. Henceforth, we assume that f is essentially nonnegative and fd is nonnegative so that the nonlinear time-delay dynamical system (3.1) is a nonnegative time-delay dynamical system. The following definition introduces several types of stability for the n equilibrium solution x(t) ≡ xe ∈ R+ of the nonlinear nonnegative dynamical n △ system (3.1). Define C+ = {ψ : [−τd , 0] → R+ : ψ ∈ C} and, for xe ∈ Rn , △ define Bˆδ (xe ) = {ψ ∈ C : kψ(θ) − xe k ≤ δ, −τd ≤ θ ≤ 0}. n

Definition 3.1. i) The equilibrium solution x(t) ≡ xe ∈ R+ to (3.1) is n Lyapunov stable with respect to R+ if, for all ε > 0, there exists δ = δ(ε) > 0 n such that if φ ∈ Bˆδ (xe ) ∩ C+ , then x(t) ∈ Bε (xe ) ∩ R+ , t ≥ 0. n

ii) The equilibrium solution x(t) ≡ xe ∈ R+ to (3.1) is (locally) n asymptotically stable with respect to R+ if it is Lyapunov stable with n respect to R+ and there exists δ > 0 such that if φ ∈ Bˆδ (xe ) ∩ C+ , then limt→∞ x(t) = xe . n

iii) The equilibrium solution x(t) ≡ xe ∈ R+ to (3.1) is globally n asymptotically stable with respect to R+ if it is Lyapunov stable with respect n to R+ and, for all φ ∈ C+ , limt→∞ x(t) = xe . The following result gives sufficient conditions for Lyapunov and asymptotic stability of a nonlinear nonnegative dynamical system with time delay. For this result, let V : C → R be a continuously differentiable functional with derivative along the trajectories of (3.1) given by 1 △ V˙ (ψ) = lim [V (xh (ψ)) − V (ψ)], h→0 h

NNBook

September 1, 2009

92

CHAPTER 3

where xh (ψ) denotes the state of (3.1) at t = h with initial condition ψ. Note that V˙ (ψ) is dependent on the system dynamics (3.1). First, the following definition is needed. Definition 3.2. A set Q ⊆ C+ is open relative to C+ if there exists an open set R ⊆ C such that Q = R ∩ C+ . A set Q ⊆ C+ is closed relative to C+ if there exists a closed set R ⊆ C such that Q = R ∩ C+ . A set Q ⊆ C+ is compact relative to C+ if there exists an compact set R ⊆ C such that Q = R ∩ C+ . Theorem 3.1. Let D be an open subset relative to C+ that contains xe . Consider the nonlinear dynamical system (3.1) where f is essentially nonnegative, fd is nonnegative, f (xe ) = 0, and fd (xe ) = 0, and assume that there exist a continuously differentiable functional V : D → R and a class K function α(·) such that V (xe ) = 0, α(kψ(0) − xe k) ≤ V (ψ), ψ ∈ D, V˙ (ψ) ≤ 0, ψ ∈ D.

(3.3) (3.4) (3.5)

Then the equilibrium solution x(t) ≡ xe to (3.1) is Lyapunov stable with n respect to R+ . If, in addition, there exists a class K function γ(·) such that V˙ (ψ) ≤ −γ(kψ(0) − xe k),

ψ ∈ D,

(3.6)

then the equilibrium solution x(t) ≡ xe to (3.1) is asymptotically stable n with respect to R+ . Finally, if, in addition, D = C+ and α(·) is a class K∞ function, then the equilibrium solution x(t) ≡ xe to (3.1) is globally n asymptotically stable with respect to R+ . △ Proof. Let ε > 0 be such that Bˆε (xe ) ∩ C+ ⊂ D, define η = α(ε), and △ define Dη = {ψ ∈ Bˆε (xe ) ∩ C+ : V (ψ) < η}. Since V (·) is continuous and V (xe ) = 0 it follows that Dη is nonempty and there exists δ = δ(ε) > 0 such that V (ψ) < η, ψ ∈ Bˆδ (xe ) ∩ C+ . Hence, Bˆδ (xe ) ∩ C+ ⊆ Dη . Next, since V˙ (ψ) ≤ 0 it follows that V (xt ) is a nonincreasing function of time, and hence, for every φ ∈ Bˆδ (xe ) ∩ C+ ⊆ Dη it follows that

α(kx(t) − xe k) ≤ V (xt ) ≤ V (φ) < η = α(ε). Thus, since f is essentially nonnegative and fd is nonnegative it follows from Proposition 3.1 that C+ is invariant with respect to (3.1). Hence, for every φ ∈ Bˆδ (xe ) ∩ C+ , x(t) ∈ Bε (xe ), t ≥ 0, establishing Lyapunov stability with n respect to R+ . Next, assume that (3.6) holds and note that Lyapunov stability follows from the first assertion. Now, to prove asymptotic stability with respect

NNBook

September 1, 2009

93

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

n to R+ , let ε > 0 and δ = δ(ε) be such that for every φ ∈ Bˆδ (xe ) ∩ C+ , x(t) ∈ Bˆε (xe ) ∩ C+ , t ≥ 0 (the existence of such a (δ, ε) pair follows from n Lyapunov stability with respect to R+ ). Note that V (xt ) is a nonincreasing function of time and, since V (·) is bounded from below, it follows from the monotone convergence theorem [112, p. 37] that there exists L ≥ 0 such that limt→∞ V (xt ) = L. Now, suppose that for some ψ ∈ Bˆδ (xe ) ∩ C+ , ad △ absurdum, L > 0 so that DL = {ψ ∈ Bˆε (xe ) ∩ C+ : V (ψ) ≤ L} is nonempty and x(t) 6∈ DL , t ≥ 0. Thus, as in the proof of Lyapunov stability, there exists δˆ > 0 such that Bˆδˆ(xe ) ∩ C+ ⊂ DL . Hence, it follows from (3.6) that, for a given φ ∈ Bˆδ (xe ) ∩ C+ \DL and t ≥ 0, Z t V (xt ) = V (φ) + V˙ (xs )ds 0 Z t ≤ V (φ) − γ(kx(s) − xe k)ds 0

Letting t >

V (φ)−L ˆ , γ(δ)

ˆ ≤ V (φ) − γ(δ)t.

it follows that V (xt ) < L, which is a contradiction. Hence, L = 0, and, since φ ∈ Bˆδ (xe ) ∩ C+ was chosen arbitrarily, it follows that V (xt ) → 0 as t → ∞ for all φ ∈ Bˆδ (xe ) ∩ C+ . Now, since V (xt ) ≥ α(kx(t) − xe k) ≥ 0, it follows that α(kx(t) − xe k) → 0 or, equivalently, n x(t) → xe as t → ∞, establishing asymptotic stability with respect to R+ . n

Finally, to prove global asymptotic stability with respect to R+ , let δ > 0 be such that φ ∈ Bˆδ (xe ) ∩ C+ and assume α(·) is a class K∞ function. Since α(·) is a class K∞ function it follows that there exists ε > 0 such that V (φ) < α(ε). Now, (3.6) implies that V (xt ) is a nonincreasing function of time, and hence, it follows from (3.4) that α(kx(t) − xe k) ≤ V (xt ) ≤ V (φ) < α(ε),

t ≥ 0.

Hence, x(t) ∈ Bε (xe ) ∩ C+ , t ≥ 0. Now, the proof follows as in the proof of the local asymptotic stability result.

3.3 Invariant Set Stability Theorems In this section, we introduce the Krasovskii-LaSalle invariance principle for time-delay dynamical systems. To state the main results of this section several definitions and a key theorem are needed. First, we introduce the notion of invariance with respect to the flow st (ψ) of a nonlinear dynamical system with time delay. Consider the nonlinear dynamical system (3.1) and for ψ ∈ D, let the map s(·, ψ) : R → D denote the solution curve or trajectory of (3.1) through the point ψ in D.

NNBook

September 1, 2009

94

CHAPTER 3

Identifying s(·, ψ) with its graph, the trajectory or orbit of a point φ ∈ D is defined as the motion along the curve △

Oφ = {ψ ∈ D : ψ = s(t, φ), t ∈ R}.

(3.7)

For t ≥ 0, we define the positive orbit through the point φ ∈ D as the motion along the curve Oφ+ = {ψ ∈ D : ψ = s(t, φ), t ≥ 0}. △

(3.8)

Definition 3.3. A point p ∈ D is a positive limit point of the trajectory s(·, ψ) of (3.1) if there exists a monotonic sequence {tn }∞ n=0 of positive numbers, with tn → ∞ as n → ∞, such that s(tn , ψ) → p as n → ∞. The set of all positive limit points of s(t, ψ), t ≥ 0, is the positive limit set ω(ψ) of s(·, ψ) of (3.1). Definition 3.4. A set M ⊂ D ⊆ C is a positively invariant set with respect to the nonlinear dynamical system (3.1) if st (M) ⊆ M for all t ≥ 0, △ where st (M) = {st (ψ) : ψ ∈ M}. A set M ⊆ D is an invariant set with respect to the dynamical system (3.1) if st (M) = M for all t ∈ R. Next, we state a key theorem involving positive limit sets. Furthermore, we use the notation xt → M ⊆ D as t → ∞ to denote that xt approaches M, that is, for each ε > 0 there exists T > 0 such that △ dist(xt , M) < ε for all t > T , where dist(p, M) = inf ψ∈M |||p − ψ|||. Theorem 3.2. Consider the nonlinear dynamical system (3.1) where f is essentially nonnegative and fd is nonnegative. Suppose the solution x(t) to (3.1) corresponding to an initial condition φ ∈ C+ is bounded for all t ≥ 0. Then the positive limit set ω(φ) of xt , t ≥ 0, is a nonempty, compact, and invariant subset of C+ . Furthermore, xt → ω(φ) as t → ∞. Proof. It follows from Lemma 1.4 of [128, p. 103] that for every φ ∈ C+ , + Oφ is compact. Furthermore, note that it is easy to show that ω(φ) =

\

+

{O ψ : ψ = s(t, φ), t ≥ 0}.

(3.9)

Hence, it follows from (3.9) that ω(φ) is nonempty and compact. To show positive invariance of ω(φ), let p ∈ ω(φ) so that there exists an increasing unbounded sequence {tn }∞ n=0 such that s(tn , φ) → p as n → ∞. Now, it follows from the semigroup property that s(t + tn , φ) = s(t, s(tn , φ)) for all t ≥ 0. Next, since s(·, φ) is continuous, it follows that, for t + tn ≥ 0, limn→∞ s(t + tn , φ) = limn→∞ s(t, s(tn , φ)) = s(t, p), and hence, s(t, p) ∈ ω(φ), establishing positive invariance of ω(φ).

NNBook

September 1, 2009

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

95

To show invariance of ω(φ) let y ∈ ω(φ) so that there exists an increasing unbounded sequence {tn }∞ n=0 such that s(tn , φ) → y as n → ∞. Next, let t ∈ [0, ∞) and note that there exists N such that tn > t, n ≥ N . Hence, it follows from the semigroup property that s(t, s(tn − t, φ)) = + s(tn , φ) → y as n → ∞. Now, since Oφ is compact there exists a subsequence {znk }∞ k=1 of the sequence zn = s(tn −t, φ), n = N, N +1, . . ., such that znk → z ∈ D as k → ∞ and, by definition, z ∈ ω(φ). Next, it follows from the continuous dependence property that limk→∞ s(t, znk ) = s(t, limk→∞ znk ), and hence, y = s(t, z), which implies that ω(φ) ⊆ st (ω(φ)), t ∈ [0, ∞). Now, using positive invariance of ω(φ) it follows that st (ω(φ)) = ω(φ), t ≥ 0, establishing invariance of the positive limit set ω(φ). Finally, to show s(t, φ) → ω(φ) as t → ∞, suppose, ad absurdum, s(t, φ) 6→ ω(φ) as t → ∞. In this case, there exists a sequence {tn }∞ n=0 , with tn → ∞ as n → ∞, and ε > 0 such that dist(x(tn ), ω(φ)) > ε, +

n ∈ Z+ .

(3.10)

However, since Oφ is compact there exists a convergent subsequence ∗ ∗ {s(t∗n , φ)}∞ n=0 such that s(tn , φ) → p ∈ ω(φ) as n → ∞, which contradicts (3.10). Hence, s(t, φ) → ω(φ) as t → ∞. Next, we present the Krasovskii-LaSalle theorem for nonnegative dynamical systems with time delay. Theorem 3.3. Consider the nonlinear dynamical system (3.1) where f is essentially nonnegative and fd is nonnegative, assume that Dc ⊂ D ⊆ C+ is a closed positively invariant set with respect to (3.1), and assume there exists a continuously differentiable functional V : Dc → R such that V (·) is △ bounded from below and V˙ (ψ) ≤ 0, ψ ∈ Dc . Let R = {ψ ∈ Dc : V˙ (ψ) = 0} and let M be the largest invariant set contained in R. If φ ∈ Dc and x(t), t ≥ 0, is bounded, then xt → M as t → ∞. Proof. Let x(t), t ≥ 0, be a solution to (3.1) with φ ∈ Dc . Since V˙ (ψ) ≤ 0, ψ ∈ Dc , it follows that Z t V (xt ) − V (xτ ) = V˙ (xs )ds ≤ 0, t ≥ τ, τ

and hence, V (xt ) ≤ V (xτ ), t ≥ τ , which implies that V (xt ) is a nonincreasing function of t. Next, since V (·) is continuous on the closed set Dc and bounded from below, there exists β ∈ R such that V (ψ) ≥ β, △ ψ ∈ Dc . Hence, γφ = limt→∞ V (xt ) exists. Now, for all p ∈ ω(φ), there exists an increasing unbounded sequence {tn }∞ n=0 , with t0 = 0, such that s(tn , φ) → p as n → ∞. Since V (ψ), ψ ∈ Dc , is continuous, V (p) = V (limn→∞ s(tn , φ)) = limn→∞ V (s(tn , φ)) = γφ , and hence, V (ψ) = γφ on

NNBook

September 1, 2009

96

CHAPTER 3

ω(φ). Now, since x(t), t ≥ 0, is bounded it follows from Theorem 3.2 that ω(φ) is a nonempty invariant set. Hence, it follows that V˙ (ψ) = 0 on ω(φ) and thus ω(φ) ⊂ M ⊂ R ⊂ Dc . Finally, since xt → ω(φ) as t → ∞, it follows that xt → M as t → ∞. Next, using Theorem 3.3 we provide a generalization of Theorem 3.1 for local asymptotic stability of a nonlinear dynamical system with time delay. Corollary 3.1. Consider the nonlinear dynamical system (3.1) where f is essentially nonnegative and fd is nonnegative, assume that Dc ⊂ D ⊆ C+ is a positively invariant set with respect to (3.1) such that xe ∈ Dc , and assume that there exist a continuously differentiable functional V : Dc → R and a class K function α(·) such that V (xe ) = 0, V (ψ) ≥ α(kψ(0)−xe k), and △ V˙ (ψ) ≤ 0, ψ ∈ Dc . Furthermore, assume that the set R = {ψ ∈ Dc : V˙ (ψ) = 0} contains no invariant set other than the set {xe }. Then the equilibrium n solution x(t) ≡ xe to (3.1) is asymptotically stable with respect to R+ . Proof. Since V˙ (ψ) ≤ 0, ψ ∈ Dc , it follows from Theorem 3.3 that if φ ∈ Dc , then ω(φ) ⊆ M, where M denotes the largest invariant set contained in R, which implies that M = {xe }. Hence, xt → M = {xe } as t → ∞, establishing asymptotic stability of the equilibrium solution x(t) ≡ n xe to (3.1) with respect to R+ . Finally, we present a global invariant set theorem for guaranteeing global asymptotic stability of a nonlinear nonnegative dynamical system with time delay. Theorem 3.4. Consider the nonlinear dynamical system (3.1) where f is essentially nonnegative and fd is nonnegative, and assume there exist a continuously differentiable functional V : C+ → R and class K∞ functions α(·) and γ(·) such that V (xe ) = 0, α(kψ(0) − xe k) ≤ V (ψ), ψ ∈ C+ , V˙ (ψ) ≤ −γ(kψ(0) − xe k),

ψ ∈ C+ .

(3.11) (3.12) (3.13)

△ Furthermore, assume that the set R = {ψ ∈ C+ : V˙ (ψ) = 0} contains no invariant set other than the set {xe }. Then the equilibrium solution n x(t) ≡ xe to (3.1) is globally asymptotically stable with respect to R+ .

Proof. Since (3.11)–(3.13) hold and α(·) is a class K∞ function, it follows from Theorem 3.1 that the equilibrium solution x(t) ≡ xe to (3.1) n is Lyapunov stable with respect to R+ and bounded. Now, Theorem 3.3 implies that xt → M as t → ∞. However, since R contains no invariant set

NNBook

September 1, 2009

97

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

other than the set {xe }, the set M is {xe }, and hence, global asymptotic n stability with respect to R+ is immediate.

3.4 Stability Theory for Continuous-Time Nonnegative Dynamical Systems with Time Delay In this section, we consider a linear time-delay dynamical system G of the form x(t) ˙ = Ax(t) + Ad x(t − τ ),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

t ≥ 0,

(3.14)

where x(t) ∈ Rn , t ≥ 0, A ∈ Rn×n , Ad ∈ Rn×n , τ ≥ 0, and φ(·) ∈ C = C([−τ, 0], Rn ) is a continuous vector-valued function specifying the initial state of the system. Note that since φ(·) is continuous it follows from Theorem 2.1 of [128, p. 14] that there exists a unique solution x(φ) defined on [−τ, ∞) that coincides with φ on [−τ, 0] and satisfies (3.14) for t ≥ 0. The following definition is needed for the main results of this section. Definition 3.5. The linear time-delay dynamical system G given by △ (3.14) is nonnegative if for every φ(·) ∈ C+ , where C+ = {ψ(·) ∈ C : ψ(θ) ≥≥ 0, θ ∈ [−τ, 0]}, the solution x(t), t ≥ 0, to (3.14) is nonnegative. Proposition 3.2. The linear time-delay dynamical system G given by (3.14) is nonnegative if and only if A ∈ Rn×n is essentially nonnegative and Ad ∈ Rn×n is nonnegative. Proof. The proof is a direct consequence of Proposition 3.1. The proof can also be shown using matrix mathematics. Specifically, the solution to (3.14) is given by Z t At x(t) = e x(0) + eA(t−θ) Ad x(θ − τ )dθ 0 Z t−τ = eAt φ(0) + eA(t−τ −θ) Ad x(θ)dθ. (3.15) −τ

Now, if A is essentially nonnegative, then it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0; and if φ(·) ∈ C+ and Ad is nonnegative, then it follows that Z t−τ x(t) = eAt φ(0) + eA(t−τ −θ) Ad φ(θ)dθ ≥≥ 0, t ∈ [0, τ ). −τ

Alternatively, for all τ < t, Aτ

x(t) = e

x(t − τ ) +

Z

τ 0

eA(τ −θ) Ad x(t + θ − 2τ )dθ,

NNBook

September 1, 2009

98

CHAPTER 3

and hence, since x(t) ≥≥ 0, t ∈ [−τ, τ ), it follows that x(t) ≥≥ 0, τ ≤ t < 2τ . Repeating this procedure iteratively, it follows that x(t) ≥≥ 0, t ≥ 0. Conversely, assume G is nonnegative and suppose, ad absurdum, that A is not essentially nonnegative. That is, suppose there exist I, J ∈ {1, 2, . . . , n}, I 6= J, such that A(I,J) < 0. Now, let φ(·) ∈ C+ be such that φ(θ) = 0, −τ ≤ θ ≤ τ − η, and φ(0) = eJ , where τ > η > 0 and eJ ∈ Rn is a vector of zeros with one in the Jth component. Next, it follows from (3.15) that x(t) = eAt eJ , 0 ≤ t < η. Hence, for sufficiently small T > 0, M(I,J) < 0, where M = eAT , which implies that xI (T ) < 0, leading to a contradiction. Now, suppose, ad absurdum, Ad is not nonnegative, that is, there exist I, J ∈ {1, 2, . . . , n} such that Ad(I,J) < 0. Next, let {vn }∞ n=1 ⊂ C+ denote a sequence of functions such that limn→∞ vn (θ) = eJ δ(θ + η − τ ), where 0 < η < τ and δ(·) denotes the Dirac delta function. In this case, it follows from (3.15) that Z η xn (η) = eAη vn (0) + eA(η−θ) Ad x(θ − τ )dθ, △

0

which implies that x(η) = limn→∞ xn (η) = eAη Ad eJ . Now, by choosing η sufficiently small it follows that xI (η) < 0, which is a contradiction. Note that it follows from Proposition 3.2 that, since Ad in (3.14) is required to be nonnegative, the linear time-delay dynamical system given by x(t) ˙ = Ax(t − τ ), x(θ) = φ(θ), −τ ≤ θ ≤ 0, t ≥ 0, (3.16)

where x(t) ∈ Rn , t ≥ 0, A ∈ Rn×n is essentially nonnegative, τ ≥ 0, and φ(·) ∈ C+ , fails to yield a nonnegative trajectory under arbitrarily small delays τ > 0. As an example, consider the scalar (n = 1) linear time-delay dynamical system G given by x(t) ˙ = −x(t − τ ),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

t ≥ 0.

Now, note that G is nonnegative if τ = 0, whereas G is not nonnegative if τ > 0 with φ(θ) = −θ. To see this, it need only be noted that Z 0 x(τ ) = x(0) + −x(θ)dθ = −τ 2 /2. −τ

For the remainder of this section, we assume that A is essentially nonnegative and Ad is nonnegative so that for every φ(·) ∈ C+ , the linear time-delay dynamical system G given by (3.14) is nonnegative. Next, we present necessary and sufficient conditions for asymptotic stability for the linear time-delay nonnegative dynamical system (3.14).

NNBook

September 1, 2009

99

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

Theorem 3.5. Consider the linear nonnegative time-delay dynamical system G given by (3.14) where A ∈ Rn×n is essentially nonnegative and Ad ∈ Rn×n is nonnegative, and let τ¯ > 0. Then G is asymptotically stable for all τ ∈ [0, τ¯] if and only if there exist p, r ∈ Rn such that p >> 0 and r >> 0 satisfy 0 = (A + Ad )T p + r. (3.17) Proof. To prove necessity, assume that the linear time-delay dynamical system G given by (3.14) is asymptotically stable for all τ ∈ [0, τ¯]. In this case, it follows that the linear nonnegative dynamical system n

x(0) = x0 ∈ R+ ,

x(t) ˙ = (A + Ad )x(t),

t ≥ 0,

(3.18)

or, equivalently, (3.14), with τ = 0, is asymptotically stable. Now, it follows from Theorem 2.11 that there exist p >> 0 and r >> 0 such that (3.17) is satisfied. Conversely, to prove sufficiency, assume that (3.17) holds and consider the candidate Lyapunov-Krasovskii functional V : C+ → R given by Z 0 V (ψ) = pT ψ(0) + pT Ad ψ(θ)dθ, ψ(·) ∈ C+ . −τ

Now, note that V (ψ) ≥ pT ψ(0) ≥ αkψ(0)k, where α = mini∈{1,2,...,n} pi > 0. Next, using (3.17), it follows that the Lyapunov-Krasovskii directional derivative along the trajectories of (3.14) is given by △

V˙ (xt ) = = = ≤

pT x(t) ˙ + pT Ad [x(t) − x(t − τ )] pT (A + Ad )x(t) −r T x(t) −βkx(t)k,

△

where β = mini∈{1,2,...,n} ri > 0 and xt (θ) = x(t + θ), θ ∈ [−τ, 0], denotes the (infinite-dimensional) state of the time-delay dynamical system G. Now, it follows from Theorem 3.1 that the linear nonnegative time-delay dynamical system G is asymptotically stable for all τ ∈ [0, τ¯]. The results presented in Proposition 3.2 and Theorem 3.5 can be easily extended to systems with multiple delays of the form x(t) ˙ = Ax(t) +

nd X i=1

Adi x(t − τi ),

x(θ) = φ(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

(3.19) where x(t) ∈ Rn , t ≥ 0, A ∈ Rn×n is essentially nonnegative, Adi ∈ Rn×n , i = 1, . . . , nd , is nonnegative, τ¯ = maxi∈{1,...,nd } τi , and φ(·) ∈ {ψ(·) ∈

NNBook

September 1, 2009

100

CHAPTER 3

'

$ )

&

%

ith Subsystem xi (t)

aij xj (t), τ

'

$

1&

%

jth Subsystem xj (t)

aji xi (t), τ

aii xi (t)

ajj xj (t)

?

?

Figure 3.1 Linear compartmental interconnected subsystem model with time delay.

τ , 0]}. In this case, (3.17) becomes C([−¯ τ , 0], Rn ) : ψ(θ) ≥≥ 0, θ ∈ [−¯ 0=

A+

nd X

Adi

i=1

!T

p + r,

(3.20)

which is associated with the Lyapunov-Krasovskii functional T

V (ψ) = p ψ(0) +

nd Z X i=1

0 −τi

pT Adi ψ(θ)dθ.

(3.21)

Next, we show that inflow-closed, linear compartmental dynamical systems with time delays [155] are a special case of the linear nonnegative time-delay systems (3.14). To see this, for i = 1, . . . , n, let xi (t), t ≥ 0, denote the mass (and hence a nonnegative quantity) of the ith subsystem of the compartmental system shown in Figure 3.1, let aii ≥ 0 denote the loss coefficient of the ith subsystem, and let φij (t − τ ), i 6= j, denote the net mass flow (or flux) from the jth subsystem to the ith subsystem given by φij (t−τ ) = aij xj (t−τ )−ajixi (t), where the transfer coefficient aij ≥ 0, i 6= j, and τ is the fixed time it takes for the mass to flow from the jth subsystem to the ith subsystem. For simplicity of exposition we have assumed that all transfer times between compartments are given by τ . The more general multiple-delay case can be addressed as shown above. Now, a mass balance for the whole compartmental system yields   n n X X x˙ i (t) = − aii + aji  xj (t)+ aij xj (t−τ ), t ≥ 0, i = 1, . . . , n, j=1,i6=j

j=1,i6=j

(3.22)

or, equivalently, x(t) ˙ = Ax(t) + Ad x(t − τ ),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

t ≥ 0,

(3.23)

NNBook

September 1, 2009

101

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

where x(t) = [x1 (t), . . . , xn (t)]T , φ(·) ∈ C+ , and, for i, j = 1, . . . , n,  Pn  − k=1 aki , i = j, 0, i = j, A(i,j) = Ad(i,j) = 0, i 6= j, aij , i 6= j.

(3.24)

Note that A is essentially nonnegative and Ad is nonnegative. Furthermore, A + Ad is a compartmental matrix, and hence, it follows from Theorem 2.10 that Re λ < 0 or λ = 0, where λ is an eigenvalue of A + Ad . Now, it follows from Theorems 2.11 and 3.5 that the zero solution x(t) ≡ 0 to (3.23) is asymptotically stable for all τ ∈ [0, τ¯] if and only if A + Ad is Hurwitz. Alternatively, asymptotic stability of (3.23) for all τ ∈ [0, τ¯] can be deduced using the Lyapunov-Krasovskii functional Z 0 T eT Ad ψ(θ)dθ, V (ψ) = e ψ(0) + ψ(·) ∈ C+ , (3.25) −τ

which captures the total mass of the system at t = 0 plus the integral of the mass flow in transit between compartments over the time intervals it takes for the mass to flow through the intercompartmental connections. In △ this case, it follows that V˙ (xt ) ≤ −βkx(t)k, where β = mini∈{1,...,n} aii and xt (θ) = x(t+θ), θ ∈ [−τ, 0]. This result is not surprising, since for an inflowclosed compartmental system the law of conservation of mass eliminates the possibility of unbounded solutions. Next, we present a nonlinear extension of Proposition 3.2 and Theorem 3.5. Specifically, we consider nonlinear time delay dynamical systems G of the form x(t) ˙ = Ax(t) + fd (x(t − τ )),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

t ≥ 0, (3.26)

where x(t) ∈ Rn , t ≥ 0, A ∈ Rn×n , fd : Rn → Rn is locally Lipschitz and fd (0) = 0, τ ≥ 0, and φ(·) ∈ C. Once again, since φ(·) is continuous, the existence and uniqueness of solutions to (3.26) follow from Theorem 2.3 of [128, p. 44]. Nonlinear time-delay systems of the form given by (3.26) arise in the study of physiological and biomedical systems [45], ecological systems [103], and population dynamics [184], as well as neural Hopfield networks [213]. For the nonlinear time-delay dynamical system (3.26), the definition of nonnegativity holds with (3.14) replaced by (3.26). Proposition 3.3. Consider the nonlinear time-delay dynamical system G given by (3.26). If A ∈ Rn×n is essentially nonnegative and fd : Rn → Rn is nonnegative, then G is nonnegative. Proof. The solution to (3.26) is given by Z t At x(t) = e x(0) + eA(t−θ) fd (x(θ − τ ))dθ 0

NNBook

September 1, 2009

102

CHAPTER 3 At

= e φ(0) +

Z

t−τ

−τ

eA(t−τ −θ) fd (x(θ))dθ.

(3.27)

Now, if A is essentially nonnegative, then it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0; and if φ(·) ∈ C+ and fd is nonnegative, then it follows that Z t−τ At x(t) = e φ(0) + eA(t−τ −θ) fd (φ(θ))dθ ≥≥ 0, t ∈ [0, τ ). −τ

Alternatively, for all τ < t, At

x(t) = e x(t − τ ) +

Z

τ 0

eA(τ −θ) fd (x(t + θ − 2τ ))dθ,

and hence, since x(t) ≥≥ 0, t ∈ [−τ, τ ), it follows that x(t) ≥≥ 0, τ ≤ t < 2τ . Repeating this procedure iteratively, it follows that x(t) ≥≥ 0, t ≥ 0. Next, we present sufficient conditions for asymptotic stability for nonlinear nonnegative dynamical systems given by (3.26). Theorem 3.6. Consider the nonlinear nonnegative time-delay dynamical system G given by (3.26) where A ∈ Rn×n is essentially nonnegative, n fd : Rn → Rn is nonnegative, and fd (x) ≤≤ γx, x ∈ R+ , where γ > 0, and let τ¯ > 0. If there exist p, r ∈ Rn such that p >> 0 and r >> 0 satisfy 0 = (A + γIn )T p + r,

(3.28)

then G is asymptotically stable for all τ ∈ [0, τ¯]. Proof. Consider the candidate Lyapunov-Krasovskii functional V : C+ → R given by Z 0 T pT fd (ψ(θ))dθ, ψ(·) ∈ C+ . V (ψ) = p ψ(0) + −τ

Now, note that V (ψ) ≥ pT ψ(0) ≥ αkψ(0)k, where α = mini∈{1,2,...,n} pi > 0. Next, using (3.28), it follows that the Lyapunov-Krasovskii directional derivative along the trajectories of (3.26) is given by △

V˙ (xt ) = = ≤ = ≤ △

pT x(t) ˙ + pT [fd (x(t)) − fd (x(t − τ ))] pT (Ax(t) + fd (x(t))) pT Ax(t) + γpT x(t) −r T x(t) −βkx(t)k,

where β = mini∈{1,2,...,n} ri > 0. Now, it follows from Theorem 3.1 that the nonlinear nonnegative time-delay dynamical system G is asymptotically stable for all τ ∈ [0, τ¯].

NNBook

September 1, 2009

103

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS n

The structural constraint fd (x) ≤≤ γx, x ∈ R+ , where γ > 0, in the statement of Theorem 3.6 is naturally satisfied for many compartmental dynamical systems. For example, in nonlinear pharmacokinetic models [12, 114] the transport across biological membranes may be facilitated by carrier molecules with the flux described by a saturable form fdi (xi , xj ) = φmax [(xαi /(xαi + β) − (xαj /(xαj + β)], where xi , xj are the concentrations of the ith and jth compartments, respectively, and φmax , α, and β are model parameters. This nonlinear intercompartmental flow model satisfies the structural constraint of Theorem 3.6.

3.5 Discrete-Time Lyapunov Stability Theory for Time-Delay Nonnegative Dynamical Systems In this section, we develop the fundamental results of Lyapunov stability theory for discrete-time time-delay nonnegative dynamical systems. We begin by considering the general discrete-time nonlinear autonomous timedelay dynamical system x(k + 1) = f (x(k)) + fd (x(k − κd )), x(κ) = φ(κ), −κd ≤ κ ≤ 0, (3.29) where x ∈ Rn , f : Rn → Rn , fd : Rn → Rn , f (0) = fd (0) = 0, and φ ∈ C = C([{−κ, . . . , 0}, Rn ]), where C is a Banach space of functions mapping the set {−κd , . . . , 0} into Rn with topology of uniform convergence and designated norm given by |||φ||| = sup−κd ≤κ≤0 kφ(κ)k. Here, φ : {−κd , . . . , 0} → Rn is a vector sequence specifying the initial state of the system. Furthermore, let xk ∈ C defined by xk (κ) = x(k + κ), κ ∈ {−κd , . . . , 0}, denote the state of (3.29) at time k corresponding to the piece of trajectories x between k − κd and k or, equivalently, the element xk in the space of functions defined on the interval {−κd , . . . , 0} and taking values in Rn . Furthermore, xe ∈ Rn is an equilibrium point of (3.29) if and only if f (xe ) = xe and fd (xe ) = 0. n

Proposition 3.4. R+ is an invariant set with respect to (3.29) if and only if f : Rn → Rn and fd : Rn → Rn are nonnegative. Proof. Note that (3.29) can be equivalently written as x(k + 1) = f (x(k)) + fd (yκd (k)), x(0) = φ(0), y1 (k + 1) = x(k), y1 (0) = φ(−1), y2 (k + 1) = y1 (k), y2 (0) = φ(−2), .. . yκd (k + 1) = yκd −1 (k), yκd (0) = φ(−κd ),

k ≥ 0,

(3.30) (3.31) (3.32) (3.33)

where yi (k) = x(k − i) for all i = 1, . . . , κd and k ≥ 0. Now, the result is a direct consequence of Proposition 2.11.

NNBook

September 1, 2009

104

CHAPTER 3

The following definition introduces several types of stability for the n equilibrium solution x(k) ≡ xe ∈ R+ of the nonlinear nonnegative dynamical n △ system (3.29). Define C+ = {ψ : {−κd , . . . , 0} → R+ : ψ ∈ C} and, for △ xe ∈ Rn , define Bˆδ (xe ) = {ψ ∈ C : kψ(κ) − xe k ≤ δ, −κd ≤ κ ≤ 0}. n

Definition 3.6. i) The equilibrium solution x(k) ≡ xe ∈ R+ to (3.29) is n Lyapunov stable with respect to R+ if, for all ε > 0, there exists δ = δ(ε) > 0 n such that if φ ∈ Bˆδ (xe ) ∩ C+ , then x(k) ∈ Bε (xe ) ∩ R+ , k ≥ 0. n

ii) The equilibrium solution x(k) ≡ xe ∈ R+ to (3.29) is (locally) n asymptotically stable with respect to R+ if it is Lyapunov stable with n respect to R+ and there exists δ > 0 such that if φ ∈ Bˆδ (xe ) ∩ C+ , then limk→∞ x(k) = xe . n

iii) The equilibrium solution x(k) ≡ xe ∈ R+ to (3.29) is globally n asymptotically stable with respect to R+ if it is Lyapunov stable with respect n to R+ and, for all φ ∈ C+ , limk→∞ x(k) = xe . The following result gives sufficient conditions for Lyapunov and asymptotic stability of a nonlinear nonnegative dynamical system with time delay. For this result, let V : C → R be a continuous function with difference △ along the trajectories of (3.29) given by ∆V (ψ) = [V (x1 (ψ)) − V (ψ)], where x1 (ψ) denotes the state of (3.29) at k = 1 with initial condition ψ. Note that ∆V (ψ) is dependent on the system dynamics (3.29). Theorem 3.7. Let D be an open subset relative to C+ that contains xe . Consider the nonlinear dynamical system (3.29) where f and fd are nonnegative, f (xe ) = xe , and fd (xe ) = 0, and assume that there exist a continuous functional V : D → R and a class K function α(·) such that V (xe ) = 0, α(kψ(0) − xe k) ≤ V (ψ), ψ ∈ D, ∆V (ψ) ≤ 0, ψ ∈ D.

(3.34) (3.35) (3.36)

Then the equilibrium solution x(k) ≡ xe to (3.29) is Lyapunov stable with n respect to R+ . If, in addition, there exists a class K function γ(·) such that ∆V (ψ) ≤ −γ(kψ(0) − xe k),

ψ ∈ D,

(3.37)

then the equilibrium solution x(k) ≡ xe to (3.29) is asymptotically stable n with respect to R+ . Finally, if, in addition, D = C+ and α(·) is a class K∞ function, then the equilibrium solution x(k) ≡ xe to (3.29) is globally n asymptotically stable with respect to R+ . Proof. The proof is similar to the proof of Theorem 3.1 and, hence,

NNBook

September 1, 2009

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

105

is omitted. Next, we introduce the Krasovskii-LaSalle invariance principle for discrete-time, time-delay nonnegative dynamical systems. To state this result, we first introduce the notion of invariance with respect to the map sk (ψ) of a discrete-time nonlinear dynamical system with time delay. Consider the nonlinear dynamical system (3.29) and for ψ ∈ D, let the map s(·, ψ) : R → D denote the solution curve or trajectory of (3.29) through the point ψ in D. Identifying s(·, ψ) with its graph, the trajectory or orbit of a point φ ∈ D is defined as the motion along the curve △

Oφ = {ψ ∈ D : ψ = s(k, φ), k ∈ Z}.

(3.38)

For k ∈ Z+ , we define the positive orbit through the point φ ∈ D as the motion along the curve Oφ+ = {ψ ∈ D : ψ = s(k, x0 ), k ∈ Z+ }. △

(3.39)

Definition 3.7. A point p ∈ D is a positive limit point of the trajectory s(·, ψ) of (3.29) if there exists a monotonic sequence {kn }∞ n=0 of positive numbers, with kn → ∞ as n → ∞, such that s(kn , ψ) → p as n → ∞. The set of all positive limit points of s(k, ψ), k ∈ Z+ , is the positive limit set ω(ψ) of s(·, ψ) of (3.29). Definition 3.8. A set M ⊂ D ⊆ C is a positively invariant set with respect to the nonlinear dynamical system (3.29) if sk (M) ⊆ M for all △ k ∈ Z+ , where sk (M) = {sk (ψ) : ψ ∈ M}. A set M ⊆ D is an invariant set with respect to the dynamical system (3.29) if sk (M) = M for all k ∈ Z. Theorem 3.8. Consider the nonlinear dynamical system (3.29) where f and fd are nonnegative, assume that Dc ⊂ D ⊆ C+ is a closed positively invariant set with respect to (3.29), and assume there exists a continuous functional V : Dc → R such that V (·) is bounded from below and ∆V (ψ) ≤ △ 0, ψ ∈ Dc . Let R = {ψ ∈ Dc : ∆V (ψ) = 0} and let M be the largest invariant set contained in R. If φ ∈ Dc and x(k), k ≥ 0, is bounded, then xk → M as t → ∞. Next, using Theorem 3.8 we provide a generalization of Theorem 3.7 for local asymptotic stability of a nonlinear dynamical system. Corollary 3.2. Consider the nonlinear dynamical system (3.29) where f and fd are nonnegative, assume that Dc ⊂ D ⊆ C+ is a closed positively invariant set with respect to (3.29) such that xe ∈ Dc , and assume that there exist a continuous functional V : Dc → R and a class K function α(·) such that V (xe ) = 0, V (ψ) ≥ α(kψ(0) − xe k), and ∆V (ψ) ≤ 0, ψ ∈ Dc . △ Furthermore, assume that the set R = {ψ ∈ Dc : ∆V (ψ) = 0} contains

NNBook

September 1, 2009

106

CHAPTER 3

no invariant set other than the set {xe }. Then the equilibrium solution n x(k) ≡ xe to (3.29) is asymptotically stable with respect to R+ . Finally, we present a global invariant set theorem for guaranteeing global asymptotic stability for a nonlinear discrete-time nonnegative system with time delay. Theorem 3.9. Consider the nonlinear dynamical system (3.29) where f and fd are nonnegative, and assume there exist a continuous functional V : C+ → R and class K∞ functions α(·) and γ(·) such that V (xe ) = 0, α(kψ(0) − xe k) ≤ V (ψ), ψ ∈ C+ , ∆V (ψ) ≤ −γ(kψ(0) − xe k),

ψ ∈ C+ .

(3.40) (3.41) (3.42)

△

Furthermore, assume that the set R = {ψ ∈ C+ : ∆V (ψ) = 0} contains no invariant set other than the set {xe }. Then the equilibrium solution n x(k) ≡ xe to (3.29) is globally asymptotically stable with respect to R+ .

3.6 Stability Theory for Discrete-Time Nonnegative Dynamical Systems with Time Delay In this section, we present a discrete-time analog to the results developed in Section 3.4. Specifically, we consider discrete-time dynamical systems G of the form k ∈ Z+ , (3.43) where x(k) ∈ Rn , k ∈ Z+ , A ∈ Rn×n , Ad ∈ Rn×n , κ ∈ Z+ , φ(·) ∈ C = C({−κ, . . . , 0}, Rn ) is a vector sequence specifying the initial state of the system, and C denotes the space of all sequences mapping {−κ, . . . , 0} into Rn with norm |||φ||| = maxk∈{−κ,...,0} kφ(k)k. The following definition is needed for the main results of this section. x(k + 1) = Ax(k) + Ad x(k − κ),

x(θ) = φ(θ),

−κ ≤ θ ≤ 0,

Definition 3.9. The discrete-time, linear time-delay dynamical system △ G given by (3.43) is nonnegative if for every φ(·) ∈ C+ , where C+ = {ψ(·) ∈ C : ψ(θ) ≥≥ 0, θ ∈ {−κ, . . . , 0}}, the solution x(k), k ∈ Z+ , to (3.43) is nonnegative. Proposition 3.5. The discrete-time, linear time-delay dynamical system G given by (3.43) is nonnegative if and only if A ∈ Rn×n and Ad ∈ Rn×n are nonnegative. Proof. The proof is a direct consequence of Proposition 3.4. The proof can also be shown using matrix mathematics. Specifically, the solution to

NNBook

September 1, 2009

107

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

(3.43) is given by k−1 X

x(k) = Ak x(0) + = Ak φ(0) +

Ak−θ−1 Ad x(θ − κ)

θ=0 k−κ−1 X

Ak−κ−θ−1 Ad x(θ).

(3.44)

θ=−κ

Now, if A is nonnegative, then it follows that Ak ≥≥ 0, k ∈ Z+ ; and if φ(·) ∈ C+ and Ad is nonnegative, then it follows that x(k) = Ak φ(0) +

k−κ−1 X θ=−κ

Ak−κ−θ−1 Ad φ(θ) ≥≥ 0,

k ∈ {0, . . . , κ}.

Alternatively, for all κ < k, κ

x(k) = A x(k − κ) +

κ−1 X θ=0

Aκ−θ−1 Ad x(k + θ − 2τ ),

and hence, since x(k) ≥≥ 0, k ∈ {−κ, . . . , κ}, it follows that x(k) ≥≥ 0, κ ≤ k < 2κ. Repeating this procedure iteratively, it follows that x(k) ≥≥ 0, k ∈ Z+ . Conversely, assume G is nonnegative and suppose, ad absurdum, that A is not nonnegative. That is, suppose there exist I, J ∈ {1, 2, . . . , n} such that A(I,J) < 0. Now, let φ(·) ∈ C+ be such that φ(−κ) = 0 and φ(0) = eJ . Next, it follows from (3.44) that x(1) = AeJ , which implies that xI (1) = A(I,J) < 0, leading to a contradiction. Now, suppose, ad absurdum, that Ad is not nonnegative, that is, there exist I, J ∈ {1, 2, . . . , n} such that Ad(I,J) < 0. Next, let φ ∈ C+ be such that φ(−κ) = eJ and φ(0) = 0. In this case, it follows from (3.44) that x(1) = Ad x(−κ), which implies that x(1) = Ad eJ and xJ (1) < 0, which is a contradiction. For the remainder of this section, we assume that A and Ad are nonnegative so that the discrete-time, linear time-delay dynamical system G given by (3.43) is nonnegative. Next, we present necessary and sufficient conditions for asymptotic stability of the discrete-time linear time-delay nonnegative dynamical system (3.43). Theorem 3.10. Consider the discrete-time, linear nonnegative timedelay dynamical system G given by (3.43) where A ∈ Rn×n and Ad ∈ Rn×n are nonnegative, and let κ ¯ > 0. Then G is asymptotically stable for all κ ∈ {0, . . . , κ ¯ } if and only if there exist p, r ∈ Rn such that p >> 0 and

NNBook

September 1, 2009

108

CHAPTER 3

r >> 0 satisfy

p = (A + Ad )T p + r.

(3.45)

Proof. To prove necessity, assume that the discrete-time, linear timedelay dynamical system G given by (3.43) is asymptotically stable for all κ ∈ {0, . . . , κ ¯ }. In this case, it follows that the discrete-time linear nonnegative dynamical system n

x(0) = x0 ∈ R+ ,

x(k + 1) = (A + Ad )x(k),

k ∈ Z+ ,

(3.46)

or, equivalently, (3.43), with κ = 0, is asymptotically stable. Now, it follows from Theorem 2.22 that there exist p >> 0 and r >> 0 such that (3.45) is satisfied. Conversely, to prove sufficiency, assume that (3.45) holds and consider the candidate Lyapunov-Krasovskii functional V : C+ → R given by V (ψ) = pT ψ(0) +

−1 X

pT Ad ψ(θ),

θ=−κ

ψ(·) ∈ C+ .

Now, note that V (ψ) ≥ pT ψ(0) ≥ αkψ(0)k, △

where α = mini∈{1,2,...,n} pi > 0. Next, using (3.45), it follows that the Lyapunov-Krasovskii difference along the trajectories of (3.43) is given by ∆V (xk ) = = = ≤

pT [x(k + 1) − x(k)] + pT Ad [x(k) − x(k − κ)] pT (A + Ad − I)x(k) −r T x(k) −βkx(k)k,

△

where β = mini∈{1,2,...,n} ri > 0 and xk (θ) = x(k + θ), θ ∈ {−κ, . . . , 0}, denotes the state of the time-delay dynamical system G. Now, it follows from Theorem 3.7 that the discrete-time, linear nonnegative time-delay dynamical system G is asymptotically stable for all κ ∈ {0, . . . , κ ¯ }. Next, we present a nonlinear extension of Proposition 3.5 and Theorem 3.10. Specifically, we consider nonlinear time-delay dynamical systems G of the form x(k + 1) = Ax(k) + fd (x(k − κ)),

k ∈ Z+ , (3.47) n×n n n n , fd : R → R is continuous and where x(k) ∈ R , k ∈ Z+ , A ∈ R fd (0) = 0, κ ≥ 0, and φ(·) ∈ C. Note that Definition 3.9 also holds for the nonlinear time-delay dynamical system G given by (3.47) with appropriate modifications. x(θ) = φ(θ),

−κ ≤ θ ≤ 0,

NNBook

September 1, 2009

109

STABILITY THEORY FOR TIME-DELAY NONNEGATIVE SYSTEMS

Proposition 3.6. Consider the discrete-time, nonlinear time-delay dynamical system G given by (3.47). If A ∈ Rn×n is nonnegative and fd : Rn → Rn is nonnegative, then G is nonnegative. Proof. The solution to (3.47) is given by k−1 X

k

x(k) = A x(0) + = Ak φ(0) +

Ak−θ−1 fd (x(θ − κ))

θ=0 k−κ−1 X

Ak−κ−θ−1 fd (x(θ)).

(3.48)

θ=−κ

Now, if A is nonnegative, then it follows that Ak ≥≥ 0, k ∈ Z+ ; and if φ(·) ∈ C+ and fd is nonnegative, then it follows that x(k) = Ak φ(0) +

k−κ−1 X θ=−κ

Ak−κ−θ−1 fd (φ(θ)) ≥≥ 0,

k ∈ {0, · · · , κ}.

Alternatively, for all κ < k, x(k) = Aκ x(k − κ) +

κ−1 X θ=0

Aκ−θ−1 fd (x(k + θ − 2κ)),

and hence, since x(k) ≥≥ 0, k ∈ {−κ, . . . , κ}, it follows that x(k) ≥≥ 0, κ ≤ k < 2κ. Repeating this procedure iteratively, it follows that x(k) ≥≥ 0, k ∈ Z+ . Finally, we present sufficient conditions for asymptotic stability of discrete-time, nonlinear nonnegative dynamical systems given by (3.47). Theorem 3.11. Consider the discrete-time, nonlinear nonnegative time-delay dynamical system G given (3.47) where A ∈ Rn×n is nonnegative, n fd : Rn → Rn is nonnegative, and fd (x) ≤≤ γx, x ∈ R+ , where γ > 0, and let κ ¯ > 0. If there exist p, r ∈ Rn such that p >> 0 and r >> 0 satisfy p = (A + γIn )T p + r,

(3.49)

then G is asymptotically stable for all κ ∈ {0, . . . , κ ¯ }. Proof. Consider the candidate Lyapunov-Krasovskii functional V : C+ → R given by T

V (ψ) = p ψ(0) +

−1 X

θ=−κ

pT fd (ψ(θ)),

ψ(·) ∈ C+ .

NNBook

September 1, 2009

110

CHAPTER 3

Now, note that V (ψ) ≥ pT ψ(0) ≥ αkψ(0)k, △

where α = mini∈{1,2,...,n} pi > 0. Next, using (3.49), it follows that the Lyapunov-Krasovskii difference along the trajectories of (3.47) is given by ∆V (xk ) = = ≤ = ≤ △

pT [x(k + 1) − x(k)] + pT [fd (x(k)) − fd (x(k − κ))] pT (Ax(k) − x(k) + fd (x(k))) pT Ax(k) − pT x(k) + γpT x(k) −r T x(k) −βkx(k)k,

where β = mini∈{1,2,...,n} ri > 0. Now, it follows from Theorem 3.7 that the discrete-time, nonlinear nonnegative time-delay dynamical system G is asymptotically stable for all κ ∈ {0, . . . , κ ¯ }.

NNBook

September 1, 2009

Chapter Four

Nonoscillation and Monotonicity of Solutions of Nonnegative Dynamical Systems

4.1 Introduction While compartmental systems have wide applicability in biology and medicine, their use in the specific field of pharmacokinetics [95, 297] is particularly noteworthy. The goal of pharmacokinetic analysis often is to characterize the kinetics of drug disposition in terms of the parameters of a compartmental model. This is accomplished by postulating a model, collecting experimental data (typically drug concentrations in blood as a function of time), and then using statistical analysis to estimate parameter values which best describe the data. Differences between the experimental data and those predicted by the model are attributed to measurement noise. Because the ultimate disposition of exogenous drugs is metabolism and elimination from the body, it is frequently assumed that drug concentrations will monotonically decline after discontinuation of drug administration. However, compartmental systems may admit nonmonotonic solutions (e.g., underdamped oscillations), that is, they can predict drug concentrations that do not decay monotonically with time after discontinuation of drug administration. Hence, it would be useful to identify compartmental systems which guarantee monotonicity of solutions in order to avoid attributing error (differences between model predictions and experimental data) to random noise, when the problem is in fact model misspecification. Similar considerations are also relevant to the other applications of nonnegative and compartmental dynamical systems. In this chapter, we present necessary and sufficient conditions for identifying nonnegative and compartmental dynamical systems that admit only nonoscillatory and monotonic solutions. Furthermore, we provide sufficient conditions that guarantee the absence of limit cycles in nonlinear compartmental systems.

NNBook

September 1, 2009

112

CHAPTER 4

4.2 Nonoscillation and Monotonicity of Linear Nonnegative Dynamical Systems In this section, we present sufficient conditions that guarantee nonoscillatory behavior of nonnegative systems as well as necessary and sufficient conditions for monotonicity of solutions. Consider the linear nonnegative dynamical system given by x(t) ˙ = Ax(t),

x(0) = x0 ,

t ≥ 0,

(4.1)

where x(t) ∈ Rn , t ≥ 0, and A ∈ Rn×n is essentially nonnegative. The following definition is needed. Definition 4.1. The linear nonnegative dynamical system (4.1) is nonoscillatory if there exist a finite time T > 0 and a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, 2, . . . , n, and, for every n x0 ∈ R+ , Qx(t2 ) ≤≤ Qx(t1 ), T ≤ t1 ≤ t2 , where x(t), t ≥ 0, denotes the solution to (4.1). Next, we present a result that shows that every two-dimensional compartmental dynamical system is nonoscillatory. First, however, the following lemma is needed. Lemma 4.1. Consider the linear nonnegative dynamical system (4.1) where n = 2 and A ∈ R2×2 is a compartmental matrix. Then the following statements hold: 2

˙ = Ax(t) ≤≤ 0, t ≥ 0. i ) For every x0 ∈ R+ such that Ax0 ≤≤ 0, x(t) 2

˙ ii ) For every x0 ∈ R+ , eT x(t) ≤ 0, t ≥ 0. iii ) For i = 1, 2, if x˙ i (0) < 0, then x˙ i (t) ≤ 0, t ≥ 0. Proof. i) Since A is essentially nonnegative it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0. The result now follows immediately by noting that x(t) ˙ = AeAt x0 = eAt Ax0 . ii) Since A is essentially nonnegative it follows from Proposition 2.5 that x(t) ≥≥ 0, t ≥ 0. Now, since A is additionally a compartmental matrix, it follows that " 2 # 2 X X T T e x(t) ˙ = e Ax(t) = A(i,j) xj (t) ≤ 0, t ≥ 0. j=1

i=1

iii) Assume x˙ 1 (0) < 0. If x˙ 2 (0) ≤ 0, then it follows from i) that

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

113

x(t) ˙ ≤≤ 0, t ≥ 0. Alternatively, let x˙ 2 (0) > 0 and suppose, ad absurdum, t ≥ 0, there exists a finite time T > 0 such that x˙ 1 (T ) > 0. Now, since x(t), ˙ is continuous it follows that there exists τ > 0 such that x˙ 1 (τ ) = 0. In this case, it follows from ii) that 0 ≥ eT x(τ ˙ ) = x˙ 1 (τ ) + x˙ 2 (τ ) = x˙ 2 (τ ). Hence, it follows from i) that x(t) ˙ ≤≤ 0, t ≥ τ , which is a contradiction. The proof of the case in which x˙ 2 (0) < 0 is completely analogous and, hence, is omitted. Theorem 4.1. Consider the linear nonnegative dynamical system (4.1) where n = 2 and A ∈ R2×2 is a compartmental matrix. Then the linear compartmental dynamical system (4.1) is nonoscillatory. 2

Proof. It follows from ii) of Lemma 4.1 that for every x0 ∈ R+ , T e x(0) ˙ ≤≤ 0, which implies that either (a) x˙ 1 (0) ≤ 0, x˙ 2 (0) ≤ 0, (b) x˙ 1 (0) > 0, x˙ 2 (0) < 0, or (c) x˙ 1 (0) < 0, x˙ 2 (0) > 0 holds. If (a) holds, then it follows from i) of Lemma 4.1 that x(t) ˙ ≤≤ 0, t ≥ 0. Alternatively, if (b) holds, then it follows from iii) of Lemma 4.1 that x˙ 2 (t) ≤ 0, t ≥ 0. In this case, if x˙ 1 (t) ≥ 0, t ≥ 0, then x˙ i (t), t ≥ 0, i = 1, 2, is monotonic. However, if there exists a finite time T > 0 such that x˙ 1 (T ) ≤ 0, then it follows from ˙ ≤≤ 0, t ≥ T . Similarly it can be shown that if (c) i) of Lemma 4.1 that x(t) holds, then there exists T > 0 such that x˙ i (t), t ≥ T , i = 1, 2, is monotonic. 2 Hence, for every x0 ∈ R+ , there exists T > 0 such that x˙ i (t), t ≥ T , i = 1, 2, is monotonic or, equivalently, there exists Q = diag[q1 , q2 ], qi = ±1, i = 1, 2, such that Qx(t) ˙ ≤≤ 0, t ≥ T . Now, it follows from (4.1) that for every T ≤ t1 ≤ t2 , Z t2 Qx(t)dt ˙ ≤≤ Qx(t1 ), Qx(t2 ) = Qx(t1 ) + t1

which proves that (4.1) is nonoscillatory. Theorem 4.1 shows that two-dimensional compartmental systems have monotonic solutions after a finite time T . However, this result is not true in the case of higher dimensions. Next, we give necessary and sufficient conditions for monotonicity of solutions over all time of linear n-dimensional nonnegative dynamical systems of the form x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0 ,

t ≥ 0,

(4.2)

where x ∈ Rn , u ∈ Rm , A ∈ Rn×n , and B ∈ Rn×m . First, however, the following two definitions are needed. Definition 4.2. The linear dynamical system given by (4.2) is nonnegn ative if for every x(0) ∈ R+ and u(t) ≥≥ 0, t ≥ 0, the solution x(t), t ≥ 0, to (4.2) is nonnegative.

NNBook

September 1, 2009

114

CHAPTER 4

Definition 4.3. Consider the linear nonnegative dynamical system n (4.2) where x0 ∈ X0 ⊆ R+ , A is essentially nonnegative, B is nonnegative, u(t), t ≥ 0, is nonnegative, and X0 denotes a set of feasible initial conditions n △ contained in R+ . Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . The linear nonnegative dynamical system (4.2) is partially monotonic with respect to x ˆ if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and, for every x0 ∈ X0 , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 , where x(t), t ≥ 0, denotes the solution to (4.2). The linear nonnegative dynamical system (4.2) is monotonic if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and, for every x0 ∈ X0 , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 . The notion of partial monotonicity with respect to part of the system states is very relevant in pharmacokinetic modeling since it allows for some compartmental masses/concentrations to exhibit oscillations while guaranteeing nonoscillation in the other compartments, which can include the central compartment. The following proposition is required for the main theorems of this section. Proposition 4.1. The linear dynamical system given by (4.2) is nonnegative if and only if A ∈ Rn×n is essentially nonnegative and B ∈ Rn×m is nonnegative. Proof. First, note that the solution x(t), t ≥ 0, to (4.2) is given by At

x(t) = e x0 +

Z

t 0

eA(t−s) Bu(s)ds,

t ≥ 0.

Now, if A is essentially nonnegative, then it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0, and if B ≥≥ 0, then it follows that x(t) ≥≥ 0 for all n t ≥ 0 and x(0) ∈ R+ , which implies that G is nonnegative. Conversely, suppose G is nonnegative. Now, let x0 = 0 and let u(t) = δ(t − tˆ)ˆ u, t ≥ 0, where u ˆ ≥≥ 0. In this case, since x(tˆ) = B u ˆ ≥≥ 0 for all m u ˆ ∈ R+ it follows that B ≥≥ 0. Finally, with u(t) = 0, t ≥ 0, x(t) = eAt x0 , and hence, it follows from Proposition 2.5 that if x(t) ≥≥ 0, t ≥ 0, for all n x0 ∈ R+ , then A is essentially nonnegative, which proves the result. Next, we present a sufficient condition for monotonicity of a linear n-dimensional nonnegative dynamical system. Theorem 4.2. Consider the linear nonnegative dynamical system

NNBook

September 1, 2009

115

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS n

given by (4.2) where x0 ∈ R+ , A ∈ Rn×n is essentially nonnegative, ˆ ≤ n, B ∈ Rn×m is nonnegative, and u(t), t ≥ 0, is nonnegative. Let n △ {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and QA ≤≤ 0 and QB ≤≤ 0. Then the linear nonnegative dynamical system (4.2) is partially monotonic with respect to x ˆ. Proof. It follows from (4.2) that Qx(t) ˙ = QAx(t) + QBu(t),

x(0) = x0 ,

t ≥ 0,

which further implies that Qx(t2 ) = Qx(t1 ) +

Z

t2

[QAx(t) + QBu(t)]dt. t1

Next, since A is essentially nonnegative, B is nonnegative, and u(t), t ≥ 0, is nonnegative, it follows from Proposition 4.1 that x(t) ≥≥ 0, t ≥ 0. Hence, since −QA and −QB are nonnegative it follows that QAx(t) ≤≤ 0 and n QBu(t) ≤≤ 0, t ≥ 0, which implies that, for every x0 ∈ R+ , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 . Corollary 4.1. Consider the linear nonnegative dynamical system n given by (4.2) where x0 ∈ R+ , A ∈ Rn×n is essentially nonnegative, B ∈ Rn×m is nonnegative, and u(t), t ≥ 0, is nonnegative. Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and −QA and −QB are nonnegative. Then the linear nonnegative dynamical system given by (4.2) is monotonic. Proof. The proof is a direct consequence of Theorem 4.2 with n ˆ=n and {k1 , . . . , knˆ } = {1, . . . , n}. Next, we present necessary and sufficient conditions for partial monotonicity and monotonicity of (4.2) in the case where u(t) ≡ 0. Theorem 4.3. Consider the linear nonnegative dynamical system n given by (4.2) where x0 ∈ R+ and A ∈ Rn×n is essentially nonnegative. △ ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x The linear nonnegative dynamical system (4.2) is partially monotonic with respect to x ˆ if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and QA ≤≤ 0. Proof. Sufficiency follows from Theorem 4.2 with u(t) ≡ 0. To show necessity, assume that the linear dynamical system given by (4.2) is partially

NNBook

September 1, 2009

116

CHAPTER 4

monotonic with respect to x ˆ. In this case, it follows from (4.2) that Qx(t) ˙ = QAx(t),

x(0) = x0 ,

t ≥ 0,

which further implies that Qx(t2 ) = Qx(t1 ) +

Z

t2

QAeAt x0 dt.

t1

Now, suppose, ad absurdum, there exist I, J ∈ {1, 2, . . . , n} such that n △ M(I,J) > 0, where M = QA. Next, let x0 ∈ R+ be such that x0J > 0 △ and x0i = 0, i 6= J, and define v(t) = eAt x0 so that v(0) = x0 and vJ (0) > 0. Now, it follows from continuity that there exists τ > 0 such that vJ (t) > 0, t ∈ [0, τ ). Thus, it follows that Z τ [Qx(τ )]J = [Qx(0)]J + [M v(t)]J dt Z0 τ = [Qx(0)]J + M(I,J) vJ (t)dt 0

> [Qx(0)]J ,

which is a contradiction. Hence, QA ≤≤ 0. Corollary 4.2. Consider the linear nonnegative dynamical system n given by (4.2) where x0 ∈ R+ and A ∈ Rn×n is essentially nonnegative. The linear nonnegative dynamical system (4.2) is monotonic if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, 2, . . . , n, and QA ≤≤ 0. Proof. The proof is a direct consequence of Theorem 4.3 with n ˆ=n and {k1 , . . . , knˆ } = {1, . . . , n}. Finally, we present a sufficient condition for weighted monotonicity of a linear nonnegative dynamical system. Proposition 4.2. Consider the linear dynamical system given by (4.2) △ where A is essentially nonnegative and x0 ∈ X0 = {SAx0 ≤≤ 0 : x0 ∈ Rn }, n×n −1 where S ∈ R is an invertible matrix. If SAS is essentially nonnegative, then for every x0 ∈ X0 , S x(t) ˙ = SAx(t) ≤≤ 0, t ≥ 0, and Sx(t2 ) ≤≤ Sx(t1 ), 0 ≤ t1 ≤ t2 . n

△

Proof. Let y(t) = −SAx(t) and note that y(0) = −SAx0 ∈ R+ . It follows from (4.2) that y(t) ˙ = −SAx(t) ˙ = −SA2 x(t) = SAS −1 y(t),

t ≥ 0.

Next, since SAS −1 is essentially nonnegative, it follows from Proposition n 2.5 that y(t) ∈ R+ , t ≥ 0. Now, the result follows immediately by noting

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

117

that y(t) = −SAx(t), ˙ t ≥ 0. Example 4.1. Consider the linear dynamical system (4.2) with   −(a + b) c A= , b −(c + d) where a, b, c, d > 0 so that A is a compartmental matrix. In this case, with S = I2 , Proposition 4.2 yields i) of Lemma 4.1. Alternatively, with   1 0 , S= b a+b ac+ad+bd

ac+ad+bd

it follows from Proposition 4.2 that if x˙ 1 (0) ≤ 0, then x˙ 1 (t) ≤ 0, t ≥ 0. A similar construction of S can be used to show that if x˙ 2 (0) ≤ 0, then △ x˙ 2 (t) ≤ 0, t ≥ 0, thus providing an alternative proof of Lemma 4.1. Example 4.2. To demonstrate the utility of Proposition 4.2 we consider a three-compartment mammillary system [155] in which the first compartment acts as the central compartment and the remaining two compartments exchange with the central compartment, which in turn outflows to the environment. Compartmental mammillary systems provide simple yet effective models of a distribution of a material which is injected into the plasma (the central compartment) and then distributes into the interstitial spaces of the organs of the body. An inflow-closed, three-compartment mammillary system is given by (4.2) with A given by   −(k11 + k21 + k31 ) k12 k13 k21 −k12 0 , A= k31 0 −k13

where the transfer coefficients k11 , k12 , k21 , k13 , k31 ≥ 0. Here we assume that x0 = [1, 0, 0]T (or a positive scalar multiple of [1, 0, 0]T ). Letting   1 0 0 k21 21  S =  k12k21k11 kk1112+k k12 k11 k11 k31 k13 k11

△

k31 k13 k11

k31 +k11 k13 k11

n

and noting that x0 ∈ X0 = {x ∈ R+ : SAx ≤≤ 0}, it can be shown that if k12 (k11 + k31 ) − k13 k31 ≤ 0, k13 (k11 + k21 ) − k12 k21 ≤ 0,

(4.3) (4.4)

then SAS −1 is essentially nonnegative. Hence, it follows from Proposition 4.2 that if (4.3) and (4.4) hold, then Sx(t2 ) ≤≤ Sx(t1 ), 0 ≤ t1 ≤ t2 , which implies that x1 (t2 ) ≤ x1 (t1 ), 0 ≤ t1 ≤ t2 , establishing that the concentrations/masses of the central compartment of the mammillary

NNBook

September 1, 2009

118

CHAPTER 4

model are monotonic. Note that (4.3) and (4.4) are only sufficient and not necessary conditions for monotonicity of concentrations/masses of the central compartment. △ Example 4.3. In this example, we use the results of this section to provide a taxonomy of linear three-dimensional, inflow-closed compartmental dynamical systems that exhibit monotonic solutions. A similar classification can be obtained for nonlinear and higher-order compartmental systems but we do not do so here for simplicity of exposition. To characterize the class of all three-dimensional monotonic compartmental systems, let △ Q = {Q ∈ R3×3 : Q = diag[q1 , q2 , q3 ], qi = ±1, i = 1, 2, 3}. Furthermore, let A ∈ R3×3 be a compartmental matrix and let x1 (t), x2 (t), and x3 (t), t ≥ 0, denote compartmental masses for compartments 1, 2, and 3, respectively. Note that there are exactly eight matrices in the set Q. Now, it follows from Corollary 4.2 that if QA ≤≤ 0, Q ∈ Q, then the corresponding compartmental dynamical system is monotonic. Hence, for every Q ∈ Q we seek all compartmental matrices A ∈ R3×3 such that qi A(i,j) ≤ 0, i, j = 1, 2, 3. First, we consider the case where Q = diag[1, 1, 1]. In this case, qi A(i,j) ≤ 0, i, j = 1, 2, 3, if and only if A(1,2) = A(1,3) = A(2,1) = A(3,1) = A(3,2) = A(2,3) = 0. This corresponds to a trivial (decoupled) case since there are no intercompartmental flows between the three compartments (see Figure 4.1 (a)). Next, let Q = diag[1, −1, −1] and note that qi A(i,j) ≤ 0, i, j = 1, 2, 3, if and only if A(1,2) = A(1,3) = A(2,2) = A(3,3) = 0. In addition, since 3 X i=1

A(i,j ) ≤ 0,

j = 1, 2, 3,

it follows that A(2,3) = A(3,2) = 0. Figure 4.1 (b) shows the compartmental structure for this case. Finally, let Q = diag[−1, 1, 1]. In this case, qi A(i,j) ≤ 0, i, j = 1, 2, 3, if and only if A(2,1) = A(3,1) = A(3,2) = A(2,3) = A(1,1) = 0. Figure 4.1 (c) shows the corresponding compartmental structure. It is important to note that in the case where Q = diag[−1, −1, −1], there does not exist a compartmental matrix satisfying QA ≤≤ 0 except for the zero matrix. This case would correspond to a compartmental dynamical system where all three states are monotonically increasing. Hence, the compartmental system would be unstable, contradicting the fact that all compartmental systems are Lyapunov stable. Finally, the remaining four cases corresponding to Q = diag[−1, 1, −1], Q = diag[−1, −1, 1], Q = diag[1, −1, 1], and Q = diag[1, 1, −1] are dual to the cases where Q = diag[1, −1, −1] and Q = diag[−1, 1, 1] and, hence, are not presented here. △

NNBook

September 1, 2009

119

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

a11 x1 (t) Compartment 1 x1 (t)

a11 x1 (t) Compartment 1 x1 (t) a21 x2 (t)

Compartment 2 x2 (t)

a31 x3 (t)

Compartment 3 x3 (t)

a22 x2 (t)

Compartment 2 x2 (t)

a33 x3 (t)

(a)

Compartment 3 x3 (t)

(b) Compartment 1 x1 (t) a12 x2 (t)

a13 x3 (t)

Compartment 2 x2 (t)

Compartment 3 x3 (t)

a22 x2 (t)

a33 x3 (t)

(c) Figure 4.1 Three-dimensional monotonic compartmental systems.

4.3 Mammillary Systems The most common pharmacokinetic models are linear and mammillary, that is, they are comprised of a central compartment from which there is outflow from the system and which exchanges material reversibly with one or more peripheral compartments. An inflow-closed mammillary system is given by (4.1) with A given by  Pn  − j=1 kj1 k12 . . . k1n  k21 −k12 . . . 0    A= (4.5) .. .. ..  , ..  . . . .  kn1

0

. . . −k1n

where the transfer coefficients kij , i 6= j, i, j = 1, . . . , n, and the loss coefficient k11 are positive. It follows from Theorem 4.2 and Corollary 4.1 that mammillary models are neither monotonic nor partially monotonic with respect to any compartment.

In numerous pharmacological applications, the initial state (at t = 0) is

NNBook

September 1, 2009

120

CHAPTER 4

characterized by a nonzero concentration in the central (first) compartment with all other compartments having zero initial concentrations. This, for example, can happen when a drug that is not synthesized by the body is administered as an impulsive intravenous injection. It is well documented via simulations that in this case the central compartment concentration monotonically decays. In the next theorem we state and prove this result. For the following theorem xj (t), j = 1, . . . , n, t ≥ 0, denotes the concentration of the jth compartment. Theorem 4.4. Consider the mammillary system given by (4.1) where △ A is given by (4.5) and x0 = e1 , where e1 = [1, 0, . . . , 0]T . Then the following statements hold: −1 is symmetric, where D = diag[1, d , . . . , d ] and d = i ) DAD 2 n j p k1j /kj1 , j = 2, . . . , n.

ii ) Let λ ∈ spec(A). Then λ ∈ R and λ ≤ 0.

iii ) x˙ 1 (t) ≤ 0, t ≥ 0, or, equivalently, the mammillary system is partially monotonic with respect to x1 . Proof. i) The proof follows by direct substitution. ii) Let λ ∈ spec(A). It follows from i) that A is similar to a symmetric matrix and, hence, λ ∈ R. Furthermore, since A is compartmental, and hence, Lyapunov stable, it follows that λ ≤ 0. iii) Since DAD −1 is symmetric it follows from the Schur decomposition that there exists a matrix S ∈ Rn×n such that SS T = In and DAD −1 = S T ΛS, where Λ = diag[λ1 , . . . , λn ], λi ∈ spec(DAD −1 ) = spec(A), i = △ 1, . . . , n. Now, let y(t) = Dx(t), t ≥ 0, and note that y(t) ˙ = DAD −1 y(t) = S T ΛSy(t),

t ≥ 0,

which further implies that y(t) = S T eΛt Sy(0), t ≥ 0. Next, note that x1 (t) = T T Λt y1 (t) = eT 1 y(t) = e1 S e Sy(0), t ≥ 0. Furthermore, y(0) = Dx0 = e1 , and hence, T Λt T Λt x1 (t) = eT 1 S e Se1 = z e z =

n X

eλj t zj2 ,

j=1

△

where z = Se1 . Hence, x˙ 1 (t) =

n X j=1

λj eλj t zj2 ≤ 0,

t ≥ 0,

t ≥ 0,

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

121

which proves that the mammillary system is partially monotonic with respect to x1 . While identification of monotonicity of the central compartment when it is the only compartment to have a nonzero initial condition is interesting, it should be noted that the results in this chapter provide necessary and sufficient conditions for monotonicity and partial monotonicity for all initial conditions. Thus, the main application of the results in this chapter is for models in which the initial conditions are less constrained, with nonzero initial states in multiple compartments. Noteworthy among potential applications is the exogenous administration of a pharmacological agent that is also endogenously synthesized. For example, consider the endogenous modulator of coagulation, antithrombin III (AT-3). The action of the exogenous anticoagulant heparin is dependent on activation of AT-3. AT-3 deficiency (congenital and acquired) is not uncommon and sometimes necessitates exogenous administration of AT-3. Hence, there has been interest in the pharmacokinetics of AT-3. In a study of transgenic AT-3, the authors in [205] assumed partial monotonicity of the central compartment with a two-compartment mammillary model. Since there is endogenous synthesis of AT-3, it is not immediately clear that the assumption of central compartment monotonicity after a single-bolus dose was valid. However, Lemma 4.1 indicates that this assumption was, indeed, valid since x˙ 1 (0) < 0. Note, however, that this lemma is applicable only to two-compartment models. There are numerous examples of other endogenously synthesized pharmacological agents, such as hormones, whose kinetics are described by more complex models after exogenous supplementation. Theorem 4.2 and Proposition 4.2 (weighted monotonicity) should aid in identifying monotonic behavior for these more complex models. Another area of potential applicability of our results is the kinetics of plasma volume expanders, especially hypertonic volume expanders. The kinetics of intravenous fluids, such as hypertonic saline, used for plasma volume expansion can be described by compartment models, and, obviously, the initial conditions will be comprised of material (in this case, fluid volume) in all compartments [69]. The dilution of various compartments could be complex depending on the exact initial conditions. An interesting, although more speculative, application is to the effect of cardiopulmonary bypass (CPB) on pharmacokinetics. The institution of CPB can be expected to result in a nearly stepwise decrease in the central compartment concentration of any drugs previously administered, due to the mixing of the patient’s circulating blood volume with the priming volume of

NNBook

September 1, 2009

122

CHAPTER 4

Figure 4.2 Nonmonotonicity of a mammillary model.

the extracorporeal circulation circuit. In contrast to bolus administration into the central compartment, in which the initial state is nonzero for this compartment only, the initial state at the institution of CPB may be characterized by a central compartment concentration that is less than the peripheral compartments due to hemodilution of the central compartment. As an example, consider a three-compartment mammillary CPB model. Application of Theorem 4.2 rules out partial monotonicity with respect to any compartment. Figure 4.2 shows a simulation of a threecompartment mammillary model with model parameters k11 = 0.001, k21 = 0.2, k12 = 0.2, k31 = 0.01, and k13 = 0.02, and initial conditions x1 (0) = 0.5, x2 (0) = 1, and x3 (0) = 1. While these parameters and initial conditions are arbitrary, they serve to illustrate the implications of Theorem 4.2. There is a monotonic decrease in the concentration of Compartment 3. However, the concentration of Compartment 1 increases, then decreases before settling into a monotonic decay. Alternatively, the concentration of Compartment 2 decreases, then increases before settling into a monotonic decay. If Compartment 2 is also identified as the site of the biological effect, for example a peripheral compartment comprised primarily of muscle, and hence, the site of action of muscle relaxants, then institution of CPB could be expected to result in a sudden decrease in the concentration of the muscle relaxant at its site of action, which, if the patient is treated with additional muscle relaxant, could result in a subsequent relative overdose.

NNBook

September 1, 2009

123

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

4.4 Monotonicity of Nonlinear Nonnegative Dynamical Systems In this section, we consider monotonicity of solutions over all time of nonlinear nonnegative dynamical systems. Specifically, we consider nonlinear dynamical systems G of the form x(t) ˙ = f (x(t)) + G(x(t))u(t),

x(0) = x0 ,

t ≥ 0,

(4.6)

where x ∈ Rn , u ∈ Rm , f : Rn → Rn , and G : Rn → Rn×m . We assume that f (·) and G(·) are continuously differentiable mappings and f (xe ) = 0, xe ∈ Rn . For the nonlinear dynamical system G given by (4.6) the definitions of monotonicity and partial monotonicity hold with (4.2) replaced by (4.6). The following definition generalizes Definition 4.2 to nonlinear systems. Definition 4.4. The nonlinear dynamical system given by (4.6) is n nonnegative if for every x(0) ∈ R+ and u(t) ≥≥ 0, t ≥ 0, the solution x(t), t ≥ 0, to (4.6) is nonnegative. The following proposition is required for the main theorem of this section. Proposition 4.3. Consider the nonlinear dynamical system G given by n (4.6). If f : Rn → Rn is essentially nonnegative and G(x) ≥≥ 0, x ∈ R+ , then G is nonnegative. △

△

Proof. Define x ˆ1 = x and x ˆ2 = t, so that (4.6) can be written as x1 (τ )) + G(ˆ x1 (τ ))u(ˆ x2 (τ )), x ˆ˙ 1 (τ ) = f (ˆ ˙x ˆ2 (τ ) = 1, x ˆ2 (0) = 0, or, equivalently,

x ˜˙ (τ ) = f˜(˜ x(t)),

x ˆ1 (0) = x0 ,

x ˜(0) = x ˜0 ,

τ ≥ 0,

(4.7) (4.8) (4.9)

where the differentiation in (4.7) and (4.8) is with respect to τ , x ˜ = [ˆ x1 , x ˆ2 ]T , T T x ˜0 = [x0 , 0] , and f˜(˜ x) = [f (ˆ x1 ) + G(ˆ x1 )u(ˆ x2 ), 1] . Next, define n+1

d(˜ x, R+ ) ,

infn+1 k˜ y−x ˜k,

y˜∈R+

x ˜ ∈ Rn+1 .

Now, since f (x) is essentially nonnegative and G(x) is nonnegative, it follows that for every i ∈ {1, . . . , n} such that xi = 0, xi + hx˙ i = hx˙ i ≥ 0 for h ≥ 0, whereas for every i ∈ {1, . . . , n} such that xi > 0, xi +hx˙ i > 0 for sufficiently n+1 small |h|. Thus, x ˜ + hf˜(˜ x) ∈ R+ for all sufficiently small h > 0, and hence, n+1 limh→0+ d(˜ x + hf˜(˜ x), R+ )/h = 0. Now, it follows from Lemma 2.1, with n+1 n x ˜(0) = x ˜, that x ˜(τ ) ∈ R+ for all τ ≥ 0. Hence, x(t) ∈ R+ for all t ≥ 0. Definition 4.5. Consider the nonlinear nonnegative dynamical system

NNBook

September 1, 2009

124

CHAPTER 4 n

(4.6) where x0 ∈ X0 ⊆ R+ , f : Rn → Rn is essentially nonnegative, G(x) ≥≥ n 0, x ∈ R+ , u(t), t ≥ 0, is nonnegative, and X0 denotes a set of initial n conditions contained in R+ . Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and △ T x ˆ(t) = [xk1 (t), . . . , xknˆ (t)] . The nonlinear nonnegative dynamical system (4.6) is partially monotonic with respect to x ˆ if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and, for every x0 ∈ X0 , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 , where x(t), t ≥ 0, denotes the solution to (4.6). The nonlinear nonnegative dynamical system (4.6) is monotonic if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and, for every x0 ∈ X0 , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 . Next, we present necessary and sufficient conditions for monotonicity of a nonlinear nonnegative dynamical system. Theorem 4.5. Consider the nonlinear nonnegative dynamical system n G given by (4.6) where x0 ∈ R+ , f : Rn → Rn is essentially nonnegative, n G(x) ≥≥ 0, x ∈ R+ , and u(t), t ≥ 0, is nonnegative. Let n ˆ ≤ n, △ T ˆ(t) = [xk1 (t), . . . , xknˆ (t)] . Then the {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x following statements hold: i ) If there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, Qf (x) ≤≤ 0, n n x ∈ R+ , and QG(x) ≤≤ 0, x ∈ R+ , then G is partially monotonic with respect to x ˆ. ii ) If u(t) ≡ 0, then G is partially monotonic with respect to x ˆ if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and Qf (x) ≤≤ 0, n x ∈ R+ . Proof. i) It follows from (4.6) that Qx(t) ˙ = Qf (x(t)) + QG(x(t))u(t),

x(0) = x0 ,

t ≥ 0,

which further implies that Qx(t2 ) = Qx(t1 ) +

Z

t2

[Qf (x(t)) + QG(x(t))u(t)]dt.

t1 n

Next, since f : Rn → Rn is essentially nonnegative, G(x) ≥≥ 0, x ∈ R+ , and u(t), t ≥ 0, is nonnegative it follows from Proposition 4.3 that x(t) ≥≥ 0, n t ≥ 0. Hence, since −Qf (x) and −QG(x) are nonnegative for all x ∈ R+ it follows that Qf (x(t)) ≤≤ 0 and QG(x(t))u(t) ≤≤ 0, t ≥ 0, which implies n that, for every x0 ∈ R+ , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 .

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

125

ii) Sufficiency follows from i) with u(t) ≡ 0. To show necessity assume that G with u(t) ≡ 0 is partially monotonic with respect to x ˆ. In this case, it follows from (4.6) that Qx(t) ˙ = Qf (x(t)),

x(0) = x0 ,

t ≥ 0,

which further implies that, for every τ > 0, Z τ Qx(τ ) = Qx0 + Qf (x(t))dt. 0

n

Now, suppose, ad absurdum, that there exist J ∈ {1, 2, . . . , n} and x0 ∈ R+ such that [Qf (x0 )]J > 0. Since the mapping Qf (·) and the solution x(t), t ≥ 0, to (4.6) are continuous it follows that there exists τ > 0 such that [Qf (x(t))]J > 0, 0 ≤ t ≤ τ , which implies that [Qx(τ )]J > [Qx0 ]J , which is n a contradiction. Hence, Qf (x) ≤≤ 0, x ∈ R+ . Corollary 4.3. Consider the nonlinear nonnegative dynamical system n G given by (4.6) where x0 ∈ R+ , f : Rn → Rn is essentially nonnegative, n G(x) ≥≥ 0, x ∈ R+ , and u(t), t ≥ 0, is nonnegative. Then the following statements hold: i ) If there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], n qi = ±1, i = 1, . . . , n, Qf (x) ≤≤ 0, x ∈ R+ , and QG(x) ≤≤ 0, n x ∈ R+ , then G is monotonic. ii ) If u(t) ≡ 0, then G is monotonic if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and n Qf (x) ≤≤ 0, x ∈ R+ . Proof. The proof is a direct consequence of Theorem 4.5 with n ˆ=n and {k1 , . . . , knˆ } = {1, . . . , n}. Corollary 4.3 provides some interesting ramifications with regards to the absence of limit cycles of inflow-closed nonlinear compartmental systems. To see this, consider the inflow-closed (u(t) ≡ 0) compartmental system (4.6) where f (x) = [f1 (x), . . . , fn (x)]T is such that fi (x) = −aii (x) +

n X

j=1,i6=j

[aij (x) − aji (x)],

i = 1, . . . , n,

(4.10)

and where the instantaneous rates of compartmental material losses aii (·), i = 1, . . . , n, and intercompartmental material flows aij (·), i 6= j, i, j = n 1, . . . , n, are such that aii (x) ≥ 0 and aij (x) ≥ 0, x ∈ R+ , i, j = 1, . . . , n. Since all mass flows as well as compartment sizes are nonnegative, it follows

NNBook

September 1, 2009

126

CHAPTER 4 n

that, for all i = 1, . . . , n, fi (x) ≥ 0 for all x ∈ R+ whenever xi = 0 and whatever the values of xj , j = 6 i. Hence, f is essentially nonnegative. As shown in Chapter 2, inflow-closed nonlinear compartmental systems are Lyapunov stable since the total mass in the system given by the sum of all components of the state x(t), t ≥ 0, is nonincreasing along the forward trajectories of (4.6). Recall that with V (x) = eT x and assuming aij (0) = 0, i, j = 1, . . . , n, it follows that V˙ (x) = eT f (x) n X = x˙ i i=1

= − = − ≤ 0,

n X

i=1 n X

aii (x) +

n n X X

i=1 j=1,i6=j

[aij (x) − aji (x)]

aii (x)

i=1

n

x ∈ R+ ,

(4.11)

which shows that the zero solution x(t) ≡ 0 of the inflow-closed nonlinear compartmental system (4.6) is Lyapunov stable, and hence, for every x0 ∈ n R+ , the solution to (4.6) is bounded. However, unlike the case of linear compartmental systems, the zero solution x(t) ≡ 0 to (4.6) with u(t) ≡ 0 is not necessarily convergent. In fact, (4.6) with f (x) given by (4.10) can exhibit limit cycles, bifurcations, and even chaos [158]. In light of the above, it is of interest to determine sufficient conditions under which masses/concentrations for nonlinear compartmental systems are Lyapunov stable and convergent, guaranteeing the absence of limit cycling behavior. The following result is a direct consequence of Corollary 4.3 and provides sufficient conditions for the absence of limit cycles in nonlinear compartmental systems. Theorem 4.6. Consider the nonlinear nonnegative dynamical system G given by (4.6) with u(t) ≡ 0 and f (x) = [f1 (x), . . . , fn (x)]T such that (4.10) holds. If there exists a matrix Q ∈ Rn×n such that Q = n diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and Qf (x) ≤≤ 0, x ∈ R+ , then, n for every x0 ∈ R+ , limt→∞ x(t) exists. n

Proof. Let V (x) = eT x, x ∈ R+ . Now, it follows from (4.11) that V˙ (x(t)) ≤ 0, t ≥ 0, where x(t), t ≥ 0, denotes the solution of G, which n implies that V (x(t)) ≤ V (x0 ) = eT x0 , t ≥ 0, and hence, for every x0 ∈ R+ , the solution x(t), t ≥ 0, of G is bounded. Hence, for every i ∈ {1, . . . , n},

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

127

xi (t), t ≥ 0, is bounded. Furthermore, it follows from Corollary 4.3 that xi (t), t ≥ 0, is monotonic. Now, since xi (·), i ∈ {1, . . . , n}, is continuous and every bounded nonincreasing or nondecreasing scalar sequence converges to a finite real number, it follows from the monotone convergence theorem [112, p. 37] that limt→∞ xi (t), i = 1, . . . , n, exists. Hence, limt→∞ x(t) exists.

4.5 Monotonicity of Discrete-Time Linear Nonnegative Dynamical Systems In this section, we present analogous results to Section 4.2 for discretetime, linear nonnegative dynamical systems. Specifically, we consider monotonicity of solutions of dynamical systems of the form given by x(k + 1) = Ax(k) + Bu(k),

x(0) = x0 ,

k ∈ Z+ ,

(4.12)

where x ∈ Rn , u ∈ Rm , A ∈ Rn×n , and B ∈ Rn×m . The following definition and proposition are needed for the main results of this section. Definition 4.6. The linear dynamical system given by (4.12) is n nonnegative if for every x(0) ∈ R+ and u(k) ≥≥ 0, k ∈ Z+ , the solution x(k), k ∈ Z+ , to (4.12) is nonnegative. Proposition 4.4. The linear dynamical system given by (4.12) is nonnegative if and only if A ∈ Rn×n is nonnegative and B ∈ Rn×m is nonnegative. by

Proof. First, note that the solution x(k), k ∈ Z+ , to (4.12) is given k

x(k) = A x0 +

k−1 X i=0

A(k−1−i) Bu(i),

k ∈ Z+ .

Now, if A is nonnegative, then it follows that Ak ≥≥ 0, k ∈ Z+ , and if n B ≥≥ 0, then it follows that x(k) ≥≥ 0 for all k ∈ Z+ and x(0) ∈ R+ , which implies that G is nonnegative. Conversely, suppose G is nonnegative. Now, with u(k) = 0, k ∈ Z+ , n x(k) = Ak x0 it follows that if x(1) ≥≥ 0 for all x0 ∈ R+ , then A is nonnegative. Finally, let x0 = 0 and let u(0) > 0, u(k) = 0, k ∈ Z+ , m k > 0. In this case, since x(1) = Bu(0) ≥≥ 0 for all u(0) ∈ R+ , it follows that B ≥≥ 0, which proves the result. Next, we give a discrete-time analog to Definition 4.3. Definition 4.7. Consider the discrete-time, linear nonnegative dynamn ical system (4.12) where x0 ∈ X0 ⊆ R+ , A is nonnegative, B is nonnegative,

NNBook

September 1, 2009

128

CHAPTER 4

u(k), k ∈ Z+ , is nonnegative, and X0 denotes a set of feasible initial n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, conditions contained in R+ . Let n △ T and x ˆ(k) = [xk1 (k), . . . , xknˆ (k)] . The discrete-time, linear nonnegative dynamical system (4.12) is partially monotonic with respect to x ˆ if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and, for every x0 ∈ X0 , Qx(k2 ) ≤≤ Qx(k1 ), 0 ≤ k1 ≤ k2 , where x(k), k ∈ Z+ , denotes the solution to (4.12). The discrete-time, linear nonnegative dynamical system (4.12) is monotonic if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and, for every x0 ∈ X0 , Qx(k2 ) ≤≤ Qx(k1 ), 0 ≤ k1 ≤ k2 . Next, we present a sufficient condition for monotonicity of a discretetime, linear nonnegative dynamical system. Theorem 4.7. Consider the discrete-time, linear nonnegative dynamn ical system given by (4.12) where x0 ∈ R+ , A ∈ Rn×n is nonnegative, B ∈ Rn×m is nonnegative, and u(k), k ∈ Z+ , is nonnegative. Let △ n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x ˆ(k) = [xk1 (k), . . . , xknˆ (k)]T . Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and QA ≤≤ Q and QB ≤≤ 0. Then the discrete-time, linear nonnegative dynamical system (4.12) is partially monotonic with respect to x ˆ. Proof. It follows from (4.12) that Qx(k + 1) = QAx(k) + QBu(k),

x(0) = x0 ,

k ∈ Z+ ,

which implies that Qx(k2 ) = Qx(k1 ) +

kX 2 −1

k=k1

[Q(A − I)x(k) + QBu(k)].

Next, since A and B are nonnegative and u(k), k ∈ Z+ , is nonnegative it follows from Proposition 4.4 that x(k) ≥≥ 0, k ∈ Z+ . Hence, since −Q(A − I) and −QB are nonnegative it follows that Q(A − I)x(k) ≤≤ n 0 and QBu(k) ≤≤ 0, k ∈ Z+ , which implies that, for every x0 ∈ R+ , Qx(k2 ) ≤≤ Qx(k1 ), 0 ≤ k1 ≤ k2 . Corollary 4.4. Consider the discrete-time, linear nonnegative dynamn ical system given by (4.12) where x0 ∈ R+ , A ∈ Rn×n is nonnegative, B ∈ Rn×m is nonnegative, and u(k), k ∈ Z+ , is nonnegative. Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and QA ≤≤ Q and QB ≤≤ 0 are nonnegative. Then the discrete-time, linear nonnegative dynamical system given by (4.12) is

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

129

monotonic. Proof. The proof is a direct consequence of Theorem 4.7 with n ˆ=n and {k1 , . . . , knˆ } = {1, . . . , n}. Next, we present partial converses to Theorem 4.7 and Corollary 4.4 in the case where u(k) ≡ 0. Theorem 4.8. Consider the discrete-time, linear nonnegative dynamn ical system given by (4.12) where x0 ∈ R+ , A ∈ Rn×n is nonnegative, △ ˆ(k) = and u(k) ≡ 0. Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x [xk1 (k), . . . , xknˆ (k)]T . The discrete-time, linear nonnegative dynamical system (4.12) is partially monotonic with respect to x ˆ if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and QA ≤≤ Q. Proof. Sufficiency follows from Theorem 4.7 with u(k) ≡ 0. To show necessity assume the discrete-time, linear dynamical system given by (4.12) with u(k) ≡ 0 is partially monotonic with respect to x ˆ. In this case, it follows from (4.12) that Qx(k + 1) = QAx(k),

x(0) = x0 ,

k ∈ Z+ ,

which further implies that Qx(k2 ) = Qx(k1 ) +

kX 2 −1

k=k1

[Q(A − I)Ak x0 ].

Now, suppose, ad absurdum, that there exist I, J ∈ {1, 2, . . . , n} such that n △ M(I,J) > 0, where M = QA − Q. Next, let x0 ∈ R+ be such that x0J > 0 △ and x0i = 0, i 6= J, and define v(k) = Ak x0 so that v(0) = x0 , v(k) ≥≥ 0, k ∈ Z+ , and vJ (0) > 0. Thus, it follows that [Qx(1)]J = [Qx0 ]J + [M v(0)]J = [Qx0 ]J + M(I,J) vJ (0) > [Qx0 ]J , which is a contradiction. Hence, QA ≤≤ Q. Corollary 4.5. Consider the discrete-time, linear nonnegative dynamn ical system given by (4.12) where x0 ∈ R+ , A ∈ Rn×n is nonnegative, and u(k) ≡ 0. The linear nonnegative dynamical system (4.12) is monotonic if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, 2, . . . , n, and QA ≤≤ Q. Proof. The proof is a direct consequence of Theorem 4.8 with n ˆ=n

NNBook

September 1, 2009

130

CHAPTER 4

and {k1 , . . . , knˆ } = {1, . . . , n}. Finally, we present a sufficient condition for weighted monotonicity of a discrete-time, linear nonnegative dynamical system. Proposition 4.5. Consider the discrete-time, linear dynamical system △ given by (4.12) where A is nonnegative, u(k) ≡ 0, x0 ∈ X0 = {x0 ∈ Rn : S(A − I)x0 ≤≤ 0}, where S ∈ Rn×n is an invertible matrix. If SAS −1 is nonnegative, then, for every x0 ∈ X0 , Sx(k2 ) ≤≤ Sx(k1 ), 0 ≤ k1 ≤ k2 . △

n R+ .

Proof. Let y(k) = −S(A−I)x(k) and note that y(0) = −S(A−I)x0 ∈ Hence, it follows from (4.12) that y(k + 1) = = = =

−S(A − I)x(k + 1) −S(A − I)Ax(k) −SAS −1 S(A − I)x(k) SAS −1 y(k). n

Next, since SAS −1 is nonnegative, it follows that y(k) ∈ R+ , k ∈ Z+ . Now, the result follows immediately by noting that y(k) = −S(A − I)x(k) >> 0,

k ∈ Z+ ,

and hence, S(A−I)x(k) ≤≤ 0, k ∈ Z+ , or, equivalently, Sx(k+1) ≤≤ Sx(k), k ∈ Z+ , which implies that Sx(k2 ) ≤≤ Sx(k1 ), 0 ≤ k1 ≤ k2 . Example 4.4. In this example, we use the results of this section to provide a taxonomy of linear three-dimensional, discrete-time inflowclosed compartmental dynamical systems that exhibit monotonic solutions. A similar classification can be obtained for nonlinear and higher-order compartmental systems but we do not do so here for simplicity of exposition. To characterize the class of all three-dimensional monotonic compartmental △ systems, let Q = {Q ∈ R3×3 : Q = diag[q1 , q2 , q3 ], qi = ±1, i = 1, 2, 3}. Furthermore, let A ∈ R3×3 be a compartmental matrix and let x1 (k), x2 (k), x3 (k), k ∈ Z+ , denote compartmental masses for compartments 1, 2, and 3, respectively. Note that there are exactly eight matrices in the set Q. Now, it follows from Corollary 4.5 that if QA ≤≤ Q, Q ∈ Q, then the corresponding compartmental dynamical system is monotonic. Hence, for every Q ∈ Q we seek all compartmental matrices A ∈ R3×3 such that qi A(i,i) ≤ qi , i = 1, 2, 3 and qi A(i,j) ≤ 0, i 6= j, i, j = 1, 2, 3. First, we consider the case where Q = diag[1, 1, 1]. In this case, qi A(i,i) ≤ qi , i = 1, 2, 3 and qi A(i,j) ≤ 0, i 6= j, i, j = 1, 2, 3, if and only if A(1,2) = A(1,3) = A(2,1) = A(3,1) = A(3,2) = A(2,3) = 0. This corresponds to a trivial (decoupled) case since there are no intercompartmental flows between the three compartments (see Figure 4.3 (a)). Next, let Q = diag[1, −1, −1]

NNBook

September 1, 2009

131

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

a11 x1 (k )

a11 x1 (k)

Compartment 1

Compartment 1

x1 (k )

x1 (k)

a21 x1 (k)

Compartment 2

Compartment 3

x2 (k )

Compartment 2

x3 (k )

a22 x2 (k )

a31 x1 (k)

Compartment 3

x2 (k)

x3 (k)

x2 (k)

a33 x3 (k )

x3 (k)

(a)

(b) x1 (k) Compartment 1

x1 (k)

a12 x2 (k)

a13 x3 (k)

Compartment 2

Compartment 3

x2 (k)

x3 (k)

a22 x2 (k)

a33 x3 (k)

(c) Figure 4.3 Three-dimensional monotonic compartmental systems.

and note that qi A(i,i) ≤ qi , i = 1, 2, 3 and qi A(i,j) ≤ 0, i 6= j, i, j = 1, 2, 3, if and only if A(2,2) = A(3,3) = 1 and A(1,2) = A(1,3) = A(2,3) = A(3,2) = 0. Figure 4.3 (b) shows the compartmental structure for this case. Finally, let Q = diag[−1, 1, 1]. In this case, qi A(i,i) ≤ qi , i = 1, 2, 3 and qi A(i,j) ≤ 0, i 6= j, i, j = 1, 2, 3, if and only if A(1,1) = 1 and A(2,1) = A(3,1) = A(3,2) = A(2,3) = 0. Figure 4.3 (c) shows the corresponding compartmental structure. It is important to note that in the case where Q = diag[−1, −1, −1], there does not exist a compartmental matrix satisfying QA ≤≤ Q except for the identity matrix. This case would correspond to a compartmental dynamical system where all three states are monotonically increasing. Hence, the compartmental system would be unstable, contradicting the fact that all compartmental systems are Lyapunov stable. Finally, the remaining four cases corresponding to Q = diag[−1, 1, −1], Q = diag[−1, −1, 1], Q = diag[1, −1, 1], and Q = diag[1, 1, −1] are dual to the cases where Q = diag[1, −1, −1] and Q = diag[−1, 1, 1] and, hence, are not presented here. △

NNBook

September 1, 2009

132

CHAPTER 4

4.6 Monotonicity of Discrete-Time Nonlinear Nonnegative Dynamical Systems In this section, we extend the results of Section 4.5 to nonlinear nonnegative dynamical systems. Specifically, we consider discrete-time, nonlinear dynamical systems G of the form x(k + 1) = f (x(k)) + G(x(k))u(k),

x(0) = x0 , n

k ∈ Z+ , (4.13)

where x(k) ∈ D, D is a relatively open subset of R+ with 0 ∈ D, u(k) ∈ Rm , f : D → Rn , and G : D → Rn×m . We assume that f (·) and G(·) are continuous in D and f (xe ) = xe , xe ∈ D. For the nonlinear dynamical system G given by (4.13) the definitions of monotonicity and partial monotonicity hold with (4.12) replaced by (4.13). The following definition generalizes Definition 4.6 to nonlinear systems. Definition 4.8. The nonlinear dynamical system (4.13) is nonnegative n if for every x(0) ∈ R+ and u(k) ≥≥ 0, k ∈ Z+ , the solution x(k), k ∈ Z+ , to (4.13) is nonnegative. Proposition 4.6. Consider the discrete-time, nonlinear dynamical system G given by (4.13). If f : D → Rn is nonnegative and G(x) ≥≥ 0, n x ∈ R+ , then G is nonnegative. n

Proof. Let x(0) ∈ R+ , u(k) ≡ 0, and suppose f : D → Rn is n nonnegative and G(x) ≥≥ 0, x ∈ R+ . For every i ∈ {1, . . . , n} it follows that xi (k + 1) = fi (x(k)) ≥≥ 0. Hence, x(k + 1) = f (x(k)) ≥≥ 0, k ∈ Z+ . Furthermore, for u(k) ≥≥ 0, k ∈ Z+ , it follows that x(k + 1) = n f (x(k)) + G(x(k))u(k) ≥≥ 0, k ∈ Z+ . Thus, x(k) ∈ R+ , k ∈ Z+ . Hence, G is nonnegative. Next, we present a sufficient condition for monotonicity of a nonlinear nonnegative dynamical system. Theorem 4.9. Consider the discrete-time, nonlinear nonnegative dyn namical system G given by (4.13) where x0 ∈ R+ , f : D → Rn is nonnegative, n G(x) ≥≥ 0, x ∈ R+ , and u(k), k ∈ Z+ , is nonnegative. Let n ˆ ≤ n, △ {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x ˆ(k) = [xk1 (k), . . . , xknˆ (k)]T . Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, n i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, Qf (x) ≤≤ Qx, x ∈ R+ , n and QG(x) ≤≤ 0, x ∈ R+ . Then the discrete-time, nonlinear nonnegative dynamical system G is partially monotonic with respect to x ˆ. Proof. The proof is similar to the proof of Theorem 4.7 with Proposition 4.6 invoked in place of Proposition 4.4.

NNBook

September 1, 2009

133

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

Corollary 4.6. Consider the discrete-time, nonlinear nonnegative dyn namical system G given by (4.13) where x0 ∈ R+ , f : D → Rn is nonnegative, n G(x) ≥≥ 0, x ∈ R+ , and u(k), k ∈ Z+ , is nonnegative. Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, n n Qf (x) ≤≤ Qx, x ∈ R+ , and QG(x) ≤≤ 0, x ∈ R+ . Then the discrete-time, nonlinear nonnegative dynamical system G is monotonic. Proof. The proof is a direct consequence of Theorem 4.9 with n ˆ=n and {k1 , . . . , knˆ } = {1, . . . , n}. Next, we present necessary and sufficient conditions for partial monotonicity and monotonicity of (4.13) in the case where u(k) ≡ 0. Theorem 4.10. Consider the discrete-time, nonlinear nonnegative n dynamical system G given by (4.13) where x0 ∈ R+ , f : D → Rn is nonnegative, and u(k) ≡ 0. Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, △ and x ˆ(k) = [xk1 (k), . . . , xknˆ (k)]T . The discrete-time, nonlinear nonnegative dynamical system G is partially monotonic with respect to x ˆ if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, n i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and Qf (x) ≤≤ Qx, x ∈ R+ . Proof. Sufficiency follows from Theorem 4.9 with u(k) ≡ 0. To show necessity, assume that the nonlinear dynamical system given by (4.13) with u(k) ≡ 0 is partially monotonic with respect to x ˆ. In this case, it follows from (4.13) that Qx(k + 1) = Qf (x(k)),

x(0) = x0 ,

k ∈ Z+ ,

which implies that, for every k ∈ Z+ , Qx(k2 ) = Qx(k1 ) +

kX 2 −1

k=k1

[Qf (x(k)) − Qx(k)]. n

Now, suppose, ad absurdum, there exist J ∈ {1, 2, . . . , n} and x0 ∈ R+ such that [Qf (x0 ))]J > [Qx0 ]J . Hence, [Qx(1)]J = [Qx0 ]J + [Qf (x0 ) − Qx0 ]J > [Qx0 ]J , n

which is a contradiction. Hence, Qf (x) ≤≤ Qx, x ∈ R+ . Corollary 4.7. Consider the discrete-time, nonlinear nonnegative dyn namical system G given by (4.13) where x0 ∈ R+ , f : D → Rn is nonnegative, and u(k) ≡ 0. The discrete-time, nonlinear nonnegative dynamical system G is monotonic if and only if there exists a matrix Q ∈ Rn×n such that n Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and Qf (x) ≤≤ Qx, x ∈ R+ .

NNBook

September 1, 2009

134

CHAPTER 4

Proof. The proof is a direct consequence of Theorem 4.10 with n ˆ=n and {k1 , . . . , knˆ } = {1, . . . , n}. Corollary 4.7 provides some interesting ramifications with regard to the absence of limit cycles of inflow-closed, discrete-time nonlinear compartmental systems. To see this, consider the inflow-closed (u(k) ≡ 0) compartmental system (4.13) where f (x) = [f1 (x), . . . , fn (x)] is such that fi (x) = xi − aii (x) +

n X

j=1,i6=j

[aij (x) − aji (x)],

i = 1, . . . , n,

(4.14)

and where the instantaneous rates of compartmental material losses aii (·), i = 1, . . . , n, and intercompartmental material flows aij (·), i 6= j, i, j = n 1, . . . , n, are such that aij (x) ≥ 0, x ∈ R+ , i, j = 1, . . . , n. Since all mass flows as well as compartment sizes are nonnegative, it follows that for all n i = 1, . . . , n, fi (x) ≥ 0 for all x ∈ R+ . Hence, f is nonnegative. As shown in Chapter 2, inflow-closed nonlinear compartmental systems are Lyapunov stable since the total mass in the system given by the sum of all components of the state x(k), k ∈ Z+ , is nonincreasing along the forward trajectories of (4.13). Recall that with V (x) = eT x and assuming aij (0) = 0, i, j = 1, . . . , n, it follows that ∆V (x) = eT f (x) − eT x n X = ∆xi i=1

= − = − ≤ 0,

n X

aii (x) +

i=1 j=1,i6=j

i=1

n X

n n X X

[aij (x) − aji (x)]

aii (x)

i=1

n

x ∈ R+ ,

(4.15)

which shows that the zero solution x(k) ≡ 0 of the inflow-closed nonlinear compartmental system (4.13) is Lyapunov stable, and hence, for every n x0 ∈ R+ , the solution to (4.13) is bounded. The following result is a direct consequence of Corollary 4.7 and provides sufficient conditions for the absence of limit cycles in discrete-time nonlinear compartmental systems. Theorem 4.11. Consider the nonlinear nonnegative dynamical system G given by (4.13) with u(k) ≡ 0 and f (x) = [f1 (x), . . . , fn (x)] such that (4.14) holds. If there exists a matrix Q ∈ Rn×n such that Q = n diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and Qf (x) ≤≤ Qx, x ∈ R+ , then, for n every x0 ∈ R+ , limk→∞ x(k) exists.

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

135

n

Proof. Let V (x) = eT x, x ∈ R+ . Now, it follows from (4.15) that ∆V (x(k)) ≤ 0, k ∈ Z+ , where x(k), k ∈ Z+ , denotes the solution of G, which implies that V (x(k)) ≤ V (x0 ) = eT x0 , k ∈ Z+ , and hence, for n every x0 ∈ R+ , the solution x(k), k ∈ Z+ , of G is bounded. Hence, for every i ∈ {1, . . . , n}, xi (k), k ∈ Z+ , is bounded. Furthermore, it follows from Corollary 4.7 that xi (k), k ∈ Z+ , is monotonic. Now, since xi (·), i ∈ {1, . . . , n}, is bounded and monotonic, it follows that limk→∞ xi (k), i = 1, . . . , n, exists. Hence, limk→∞ x(k) exists.

4.7 Monotonicity of Nonnegative Dynamical Systems with Time Delay Pharmacokinetic models with time delay can introduce oscillations in system solutions that may otherwise be nonoscillatory in the absence of time delay [155, 207]. Since drug concentrations should monotonically decline after discontinuation of drug administration, it is of interest to determine necessary and sufficient conditions under which these systems possess monotonic solutions. In this section, necessary and sufficient conditions are developed for identifying nonnegative and compartmental dynamical systems that admit only nonoscillatory and monotonic solutions in the presence of time lags. Consider the linear time-delay dynamical system G given by t ≥ 0, (4.16) where x(t) ∈ Rn , u(t) ∈ Rm , t ≥ 0, A ∈ Rn×n , Ad ∈ Rn×n , B ∈ Rl×m , τ ≥ 0, φ(·) ∈ C = C([−τ, 0], Rn ) is a continuous vector-valued function specifying the initial state of the system, and C([−τ, 0], Rn ) denotes a Banach space of continuous functions mapping the interval [−τ, 0] into Rn with the topology of uniform convergence. x(t) ˙ = Ax(t) + Ad x(t − τ ) + Bu(t),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

The following definition is needed for the main results in this section. Definition 4.9. The linear time-delay dynamical system G given by △ (4.16) is nonnegative if for every φ(·) ∈ C+ , where C+ = {ψ(·) ∈ C : ψ(θ) ≥≥ 0, θ ∈ [−τ, 0]}, and u(t) ≥≥ 0, t ≥ 0, the solution x(t), t ≥ 0, to (4.16) is nonnegative. Proposition 4.7. The linear time-delay dynamical system G given by (4.16) is nonnegative if and only if A ∈ Rn×n is essentially nonnegative, Ad ∈ Rn×n is nonnegative, and B ≥≥ 0.

NNBook

September 1, 2009

136

CHAPTER 4

Proof. First, note that the solution to (4.16) is given by Z t x(t) = eAt x(0) + eA(t−θ) [Ad x(θ − τ ) + Bu(θ)]dθ 0 Z t Z t−τ At A(t−τ −θ) eA(t−θ) Bu(θ)dθ. (4.17) = e φ(0) + e Ad x(θ)dθ + −τ

0

Now, if A is essentially nonnegative, then it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0; and if φ(·) ∈ C+ , Ad ≥≥ 0, and B ≥≥ 0, then it follows that Z t−τ Z t At A(t−τ −θ) x(t) = e φ(0) + e Ad x(θ)dθ + eA(t−θ) Bu(θ)dθ ≥≥ 0, −τ

0

t ∈ [0, τ ),

(4.18)

and y(t) ≥≥ 0 for all t ∈ [0, τ ). Alternatively, for all t > τ , Z τ Z t x(t) = eAτ x(t − τ ) + eA(τ −θ) Ad x(t + θ − 2τ )dθ + eA(t−θ) Bu(θ)dθ, 0

0

(4.19) and hence, since x(t) ≥≥ 0, t ∈ [−τ, τ ), it follows that x(t) ≥≥ 0, t ∈ [τ, 2τ ). Repeating this procedure iteratively, it follows that x(t) ≥≥ 0, t ≥ 0, which implies that G is nonnegative. Conversely, suppose G is nonnegative. Now, let φ(θ) = 0, −τ ≤ θ ≤ 0, and let u(t) = δ(t − tˆ)ˆ u, t, tˆ ∈ [0, τ ), where δ(·) denotes the Dirac delta m function and u ˆ ≥≥ 0. In this case, since x(tˆ) = B u ˆ ≥≥ 0 for all u ˆ ∈ R+ it follows that B ≥≥ 0. Furthermore, with u(t) = 0, φ(θ) = 0, −τ ≤ θ ≤ 0, x(t) = eAt φ(0), t ∈ [0, τ ), and hence, it follows from Proposition 2.5 that n if x(t) ≥≥ 0, t ≥ 0, for all φ(0) ∈ R+ , then A is essentially nonnegative. Finally, suppose, ad absurdum, that Ad is not nonnegative, that is, there exist I, J ∈ {1, 2, ...., n} such that Ad(I,J) < 0. Let u(t) = 0, t ≥ 0, and let {vn }∞ n=1 ⊂ C+ denote a sequence of functions such that limn→∞ vn (θ) = eJ δ(θ + η − τ ), where 0 < η < τ and δ(·) denotes the Dirac delta function. In this case, it follows from (4.17) that Z η Aη xn (t) = e vn (0) + eA(η−θ) Ad x(θ − τ )dθ, (4.20) 0

which implies that x(η) = limn→∞ xn (η) = eAη Ad eJ . Now, by choosing η sufficiently small it follows that xI (η) < 0, which is a contradiction. Next, we present a definition for the monotonicity of solutions over all time for a time-delay nonnegative dynamical system of the form given by (4.16). Definition 4.10. Consider the linear nonnegative time-delay dynam-

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

137

ical system (4.16) where φ(·) ∈ C+ , A is essentially nonnegative, Ad is nonnegative, B is nonnegative, and u(t), t ≥ 0, is nonnegative. Let n ˆ ≤ n, △ ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . The linear {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x nonnegative time delay dynamical system (4.16) is partially monotonic with respect to x ˆ if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and, for every φ(·) ∈ C+ , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 , where x(t), t ≥ 0, denotes the solution to (4.16). The linear nonnegative time delay dynamical system (4.16) is monotonic if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, . . . , n, and, for every φ(·) ∈ C+ , Qx(t2 ) ≤≤ Qx(t1 ), 0 ≤ t1 ≤ t2 . Next, we present necessary and sufficient conditions that guarantee partial monotonicity and monotonicity for nonnegative dynamical systems with time delay. Theorem 4.12. Consider the linear time-delay nonnegative dynamical system given by (4.16) where φ(·) ∈ C+ , A ∈ Rn×n is essentially nonnegative, Ad ∈ Rn×n is nonnegative, B ∈ Rl×m is nonnegative, and u(t), t ≥ 0, △ is nonnegative. Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, and QA ≤≤ 0, QAd ≤≤ 0, and QB ≤≤ 0. Then the linear nonnegative dynamical system (4.16) is partially monotonic with respect to x ˆ. Proof. The proof is a direct consequence of Theorem 4.2 with B replaced by [Ad B] and u(t), t ≥ 0, replaced by [xT (t − τ ) uT (t)]T , t ≥ 0. The following result gives necessary and sufficient conditions for monotonicity in the case where u(t) ≡ 0. Theorem 4.13. Consider the linear time-delay nonnegative dynamical system given by (4.16) where φ(·) ∈ C+ , A ∈ Rn×n is essentially nonnegative, Ad ∈ Rn×n is nonnegative, and u(t) ≡ 0. Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ △ {1, 2, . . . , n}, and x ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . Then the linear time delay dynamical system is partially monotonic with respect to x ˆ if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, QA ≤≤ 0, and QAd ≤≤ 0. Proof. Sufficiency follows from Theorem 4.12 with u(t) ≡ 0. To show necessity, assume the linear time-delay dynamical system given by (4.16) is partially monotonic with respect to x ˆ. In this case, it follows from (4.16) that Qx(t) ˙ = QAx(t) + QAd x(t − τ ),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

t ≥ 0,

NNBook

September 1, 2009

138

CHAPTER 4

which implies that, for some t1 ≥ 0 and t2 > t1 , Z t2 Qx(t2 ) = Qx(t1 ) + [QAx(t) + QAd x(t − τ )]dt. t1

Now, suppose, ad absurdum, that there exist I, J ∈ {1, 2, . . . , n} such that M(I,J) > 0, where M , QA. Next, let φ(·) ∈ C+ be such that φ(t) = 0, −τ < t < η − τ and let φ(0) = eJ , where τ > η > 0 and eJ ∈ Rn is a vector of zeros with one in the Jth component. Next, it follows from continuity of solutions that there exists 0 < ε ≤ η such that xJ (t) > 0, 0 ≤ t ≤ ε. Thus, for every t ∈ [0, η], it follows that Z t [Qx(t)]I = [Qx(0)]I + [QAx(s) + QAd x(s − τ )]I ds 0 Z t−τ = [Qx(0)]I + [QAx(s + τ ) + QAd x(s)]I ds −τ t−τ

= [Qx(0)]I + > [Qx(0)]I ,

Z

−τ

[QAx(s + τ )]I ds (4.21)

which is a contradiction. Hence, QA ≤≤ 0. Now, suppose, ad absurdum, that there exist I, J ∈ {1, 2, . . . , n} such that M(I,J) > 0, where M , QAd . Next, let {vn }∞ n=1 ⊂ C+ denote a sequence of functions such that limn→∞ vn (t) = eJ δ(t − η + τ ), where 0 < η < τ and δ(·) denotes the Dirac delta function. Furthermore, let φ(·) ∈ C+ be such that φ(t) = limn→∞ vn (t), −τ < t < 0. In this case, it follows from (4.21) that Z t Qxn (t) = Qvn (0) + [QAxn (s) + QAd xn (s − τ )]ds 0 Z t−τ = [QAvn (s + τ ) + QAd vn (s)]ds, −τ

where xn (·) denotes the solution to (4.16) with φ(·) = vn (t). Now, note that x ˆ(t) , limn→∞ [Qxn (t)]I = QAd eJ . Hence, x ˆI (t) = [Qx(t)]I > [Qx(0)]I = x ˆI (0) = 0. Thus, there exists n > 0 such that φ(t) = vn (t) implies [Qx(t)]I > [Qx(0)] and, hence, QAd ≤≤ 0. Corollary 4.8. Consider the linear time-delay dynamical system given by (4.16) where φ(·) ∈ C+ , A ∈ Rn×n is essentially nonnegative, Ad ∈ Rn×n is nonnegative, and u(t) ≡ 0. The linear time-delay dynamical system is monotonic if and only if there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = ±1, i = 1, 2, . . . , n, QA ≤≤ 0, and QAd ≤≤ 0. Proof. The proof is a direct consequence of Theorem 4.13 with n ˆ=n

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

139

and {k1 , . . . , knˆ } = {1, . . . , n}. Example 4.5. In this example, we use the results of this section to provide a taxonomy of linear three-dimensional, inflow-closed (i.e., u(t) ≡ 0) compartmental dynamical systems with time delay that exhibit monotonic solutions. To characterize the class of all three-dimensional monotonic compartmental systems with time delay, let Q , {Q ∈ R3×3 : Q = diag[q1 , q2 , q3 ], qi = ±1, i = 1, 2, 3}. It was shown in Chapter 3 that a compartmental dynamical system with time delay satisfies the property that A ∈ Rn×n is essentially nonnegative, Ad ∈ Rn×n is nonnegative, and A + Ad is a compartmental matrix. Let x1 (t), x2 (t), and x3 (t), t ≥ 0, denote compartmental masses for Compartments 1, 2, and 3, respectively. Note that there are exactly eight matrices in the set Q. Now, it follows from Corollary 4.8 that if QA ≤≤ 0 and QAd ≤≤ 0, Q ∈ Q, then the corresponding compartmental dynamical system is monotonic. Hence, for every Q ∈ Q we seek all matrices A ∈ R3×3 and Ad ∈ R3×3 such that qi A(i,j) ≤ 0 and qi Ad(i,j) ≤ 0, i, j = 1, 2, 3. First, we consider the case where Q = diag[1, 1, 1]. In this case, qi A(i,j) ≤ 0 and qi Ad(i,j) ≤ 0, i, j = 1, 2, 3, if and only if A(1,2) = A(1,3) = A(2,1) = A(3,1) = A(3,2) = A(2,3) = 0. Similarly, qi Ad(i,j) ≤ 0 if and only if Ad = 0. This corresponds to the trivial (decoupled) case since there are no intercompartmental flows between the three compartments (see Figure 4.1 (a)). Next, let Q = diag[1, −1, 1] and note that qi Ad(i,j) ≤ 0, i, j = 1, 2, 3, if and only if A(1,2) = A(1,3) = 0. In addition, since A + Ad is a compartmental matrix, it follows that A(2,2) = A(3,2) = A(2,3) = A(3,3) = 0. Figure 4.1 (b) shows the compartmental structure for this case. Finally, let Q = diag[−1, 1, 1]. In this case qi Ad(i,j) ≤ 0, i, j = 1, 2, 3, if and only if A(2,1) = A(2,3) = A(3,1) = A(3,2) = 0. Since, A + Ad is compartmental, A(2,2) = 0. Figure 4.1 (c) shows the corresponding compartmental structure. It is important to note that in the case where Q = diag[−1, −1, −1], there does not exist a compartmental matrix A + Ad satisfying QA ≤≤ 0 and QAd ≤≤ 0 except for the zero matrix. This case would correspond to a compartmental dynamical system where all three states are monotonically increasing. Hence, the compartmental system would be unstable contradicting the fact that all compartmental systems are Lyapunov stable [110, 113]. Finally, the remaining four cases corresponding to Q = diag[−1, 1, −1], Q = diag[−1, −1, 1], Q = diag[1, −1, 1], and Q = diag[1, 1, −1] are dual to the cases where Q = diag[1, −1, −1] and Q = diag[−1, 1, 1] and, hence, are not presented here. △ Example 4.6. Consider a three-compartment model with time delay between compartments. Here, for simplicity of exposition, we assume that

NNBook

September 1, 2009

140

CHAPTER 4

$ )

'

ith Compartment xi (t) & %

aji xi (t), τ

$

'

aij xj (t), τ

jth Compartment xj (t) 1& %

aii xi (t) ?

ajj xj (t) ?

Figure 4.4 Linear compartmental interconnected subsystem model with time delay.

all transfer times between compartments are equal and given by τ . Now, a mass balance for the compartmental system, with state variables and transfer coefficients aij ≥ 0, i = 1, 2, 3, defined as in Figure 4.4, yields (4.16) with u(t) ≡ 0, and A and Ad given by  P3  0, i = j, − k=1 aki , i = j, A(i,j) = Ad(i,j) = (4.22) aij , i 6= j, i 6= j, 0,

where i, j = 1, 2, 3. Note that A is essentially nonnegative, Ad is nonnegative, and A + Ad is a compartmental matrix. Here, we are interested in the monotonicity of the first compartment and hence we choose Q = diag[1, 0, 0]. Now, it follows from Theorem 4.13 that if QA ≤≤ 0 and QAd ≤≤ 0, then the first compartment is monotonic. Note that QA ≤≤ 0 and QAd ≤≤ 0 if and only if A(1,2) = A(1,3) = Ad(1,1) = Ad(1,2) = Ad(1,3) = 0. For our simulation we let   −5 0 0 A =  0 −5 0  , 0 0 −5

(4.23)



 0 0 0 Ad =  1 0 3  . 2 2 0

With τ = 2 and the initial condition φ(t) = [1, 0, 0]T , −τ ≤ t ≤ 0, Figure 4.5 shows that the mass of the first compartment is monotonic. Next, let 

 −5 0 0 A =  0 −5 0  , 0 0 −5



 0 2 2 Ad =  1 0 3  . 2 2 0

Figure 4.6 shows the mass of the first compartment with respect to time for τ = 2 and the initial condition φ(t) = [1, 0, 0]T , −τ ≤ t ≤ 0. However, since (4.23) is a necessary and sufficient condition for monotonicity of the first compartment, the mass in the first compartment does not exhibit a monotonic behavior. △

NNBook

September 1, 2009

NONOSCILLATION AND MONOTONICITY OF NONNEGATIVE SYSTEMS

141

Figure 4.5 Monotonicity of the first compartment.

Figure 4.6 Nonmonotonicity of the first compartment.

Next, we present a sufficient condition that guarantees partial monotonicity and monotonicity for nonlinear nonnegative dynamical systems with time delay. In particular, we consider nonlinear time-delay dynamical systems G of the form x(t) ˙ = Ax(t) + fd (x(t − τ )) + G(x(t))u(t), x(θ) = φ(θ), −τ ≤ θ ≤ 0, t ≥ 0,

(4.24)

where x(t) ∈ Rn , u(t) ∈ Rm , t ≥ 0, A ∈ Rn×n , fd : Rn → Rn is locally

NNBook

142

September 1, 2009

CHAPTER 4

Lipschitz and fd (0) = 0, τ ≥ 0, G : Rn → Rn×m, and φ(·) ∈ C. For the nonlinear dynamical system G given by (4.24) the definitions of monotonicity and partial monotonicity hold with (4.16) replaced by (4.24) Theorem 4.14. Consider the nonlinear time-delay nonnegative dynamical system given by (4.24) where φ(·) ∈ C+ , A ∈ Rn×n is essentially nonnegative, fd ∈ Rn×n is nonnegative, G(x) ≥≥ 0, and u(t), t ≥ 0, △ is nonnegative. Let n ˆ ≤ n, {k1 , k2 , . . . , knˆ } ⊆ {1, 2, . . . , n}, and x ˆ(t) = [xk1 (t), . . . , xknˆ (t)]T . Assume there exists a matrix Q ∈ Rn×n such that Q = diag[q1 , . . . , qn ], qi = 0, i 6∈ {k1 , . . . , knˆ }, qi = ±1, i ∈ {k1 , . . . , knˆ }, QA ≤≤ 0, Qfd (x) ≤≤ 0, and QG(x) ≤≤ 0, x ∈ Rn . Then the nonlinear nonnegative dynamical system (4.24) is partially monotonic with respect to x ˆ. Proof. The proof is similar to the proof of Theorem 4.5 and, hence, is omitted. Finally, it is important to note that discrete-time extensions for linear and nonlinear nonnegative dynamical systems with time delay possessing monotonic solutions can be derived in a similar fashion.

NNBook

September 1, 2009

Chapter Five

Dissipativity Theory for Nonnegative Dynamical Systems

5.1 Introduction In control engineering, dissipativity theory provides a fundamental framework for the analysis and control design of dynamical systems using an input, state, and output system description based on system-energy-related considerations [112]. The notion of energy here refers to abstract energy notions for which a physical system energy interpretation is not necessary. The dissipation hypothesis on dynamical systems results in a fundamental constraint on their dynamic behavior, wherein a dissipative dynamical system can deliver only a fraction of its energy to its surroundings and can store only a fraction of the work done to it. Many of the great landmarks of feedback control theory are associated with dissipativity theory. In particular, dissipativity theory provides the foundation for absolute stability theory which in turn forms the basis of the Lur´e problem, as well as the circle and Popov criteria, which are extensively developed in the classical monographs of Aizerman and Gantmacher [2], Lefschetz [194], and Popov [244]. Since absolute stability theory concerns the stability of a dynamical system for classes of feedback nonlinearities which, as noted in [107, 108], can readily be interpreted as an uncertainty model, it is not surprising that absolute stability theory (and, hence, dissipativity theory) also forms the basis of modern-day robust stability analysis and synthesis [107, 109, 123]. The key foundation in developing dissipativity theory for general nonlinear dynamical systems was presented by J. C. Willems [308, 309] in his seminal two-part paper on dissipative dynamical systems. In particular, Willems [308] introduced the definition of dissipativity for general dynamical systems in terms of a dissipation inequality involving a generalized system power input, or supply rate, and a generalized energy function, or storage function. The dissipation inequality implies that the increase in generalized system energy over a given time interval cannot exceed the generalized energy supply delivered to the system during this time interval. The

NNBook

144

September 1, 2009

CHAPTER 5

set of all possible system storage functions is convex and every system storage function is bounded from below by the available system storage and bounded from above by the required supply. The available storage is the amount of internal generalized stored energy which can be extracted from the dynamical system and the required supply is the amount of generalized energy which can be delivered to the dynamical system to transfer it from a state of minimum potential to a given state. Hence, as noted above, a dissipative dynamical system can deliver only a fraction of its stored generalized energy to its surroundings and can store only a fraction of generalized work done to it. Dissipativity theory is a system-theoretic concept that provides a powerful framework for the analysis and control design of dynamical systems based on generalized energy considerations. In particular, dissipativity theory exploits the notion that numerous physical dynamical systems have certain input, state, and output system properties related to conservation, dissipation, and transport of mass and energy. Such conservation laws are prevalent in dynamical systems such as mechanical systems, fluid systems, electromechanical systems, electrical systems, combustion systems, structural systems, biological systems, physiological systems, biomedical systems, ecological systems, economic systems, as well as feedback control systems. On the level of analysis, dissipativity can involve conditions on system parameters that render an input, state, output system dissipative. Or, alternatively, analyzing system stability robustness by viewing a dynamical system as an interconnection of dissipative dynamical subsystems. On the synthesis level, dissipativity can be used to design feedback controllers that add dissipation and guarantee stability robustness, allowing stabilization to be understood in physical terms [112]. In this chapter, we use linear and nonlinear storage functions and linear supply rates to develop new notions of classical dissipativity theory [308, 309] and exponential dissipativity theory [55, 112] for linear and nonlinear nonnegative dynamical systems. The overall approach provides a new interpretation of a mass balance for nonnegative systems with linear supply rates and linear and nonlinear storage functions. Specifically, we show that dissipativity of a nonnegative dynamical system involving a linear storage function and a linear supply rate implies that the system mass transport is equal to the supplied system flux minus the expelled system flux. In the special case where the linear supply rate is taken to be the excess input mass flux over the output mass flux the system dissipativity notion collapses to a nonaccumulativity system constraint, wherein the system mass transport is always less than or equal to the difference between the system flux input and system flux output. Furthermore, we show that all compartmental systems with measured outputs corresponding to material outflows are

NNBook

September 1, 2009

145

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

nonaccumulative. In addition, we develop new Kalman-Yakubovich-Popov equations for nonnegative systems to characterize dissipativeness with linear and nonlinear storage functions and linear supply rates. Next, using the concepts of dissipativity and exponential dissipativity with linear and nonlinear storage functions and linear supply rates we develop feedback interconnection stability results for nonnegative dynamical systems. General stability criteria are given for Lyapunov, semi-, and asymptotic stability of feedback linear and nonlinear nonnegative systems. These results can be viewed as a generalization of the positivity and the small gain theorems [112, 140] to nonnegative systems with linear and nonlinear storage functions and linear supply rates. A key observation of these results is that unlike the classical results on positivity and the small gain theorems requiring negative feedback interconnections, positive feedback interconnections are required in order to ensure that the resulting feedback system is a nonnegative dynamical system.

5.2 Dissipativity Theory for Nonnegative Dynamical Systems Dissipativity theory has been extensively developed for the analysis and design of control systems for engineering systems using input, state, and output system descriptions based on energy-related considerations without the consideration of nonnegative and compartmental models [112, 203, 308, 309]. Since biological and physiological systems have numerous input, state, and output1 properties related to conservation, dissipation, and transport of mass and energy, it seems natural to extend dissipativity theory to nonnegative and compartmental models which themselves behave in accordance to conservation laws. In this section, we extend the notion of dissipativity to nonnegative dynamical systems. Specifically, we consider dynamical systems G of the form2 x(t) ˙ = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t),

x(0) = x0 ,

t ≥ 0,

(5.1) (5.2)

where x ∈ Rn , u ∈ Rm , y ∈ Rl , A ∈ Rn×n , B ∈ Rn×m , C ∈ Rl×n , and D ∈ Rl×m . First, however, we provide definitions and several results concerning dynamical systems of the form (5.1) and (5.2) with nonnegative inputs and nonnegative outputs. 1 The outputs here refer to measured outputs or observations and may have nothing to do with material outflows of the nonnegative compartmental system. 2 In the life sciences such as biology, biochemistry, and physiology, input, state, and output system descriptions can be very useful in determining system structure and estimating system parameters.

NNBook

September 1, 2009

146

CHAPTER 5

Definition 5.1. The linear dynamical system G given by (5.1) and (5.2) with x(0) = 0 is input-output nonnegative if y(t), t ≥ 0, is nonnegative for every nonnegative input u(t), t ≥ 0. The following result shows that the dynamical system G is inputoutput nonnegative if and only if the impulse response function is nonnegative and D ≥≥ 0. Theorem 5.1. The linear dynamical system G given by (5.1) and (5.2) is input-output nonnegative if and only if D ≥≥ 0 and the impulse response △ matrix function H(t) = CeAt B, t ≥ 0, of G is nonnegative. Proof. Consider the dynamical system G given by (5.1) and (5.2) with x(0) = 0. Then, it follows that Z t y(t) = H(t − s)u(s)ds + Du(t), t ≥ 0. 0

Now, suppose the impulse response function H(t) ≥≥ 0, t ≥ 0, and D ≥≥ 0. Then, for every nonnegative input u(t) ≥≥ 0, t ≥ 0, the output is also nonnegative, that is, y(t) ≥≥ 0, t ≥ 0. Conversely, suppose G is input-output nonnegative and assume, ad absurdum, there exist i, j, with i 6= j, such that H(i,j)(τ ) < 0 at some τ > 0. By continuity, there exists 0 < t1 < τ such that H(i,j)(t) < 0, t ∈ [t1 , τ ]. Now, for every u(t), t ≥ 0, such that u(t) = 0, t 6∈ [0, τ − t1 ], and u(t) = u ˆ ej , t ∈ [0, τ − t1 ], where u ˆ > 0, it follows that Z τ −t1 yi (τ ) = H(i,j) (τ − s)uj (s)ds < 0, (5.3) 0

which is a contradiction. Hence, H(t) ≥≥ 0, t ≥ 0. Finally, for every u(t) ≥≥ 0, t ≥ 0, y(0) = Du(0) ≥≥ 0, which implies that D ≥≥ 0. Definition 5.2. The linear dynamical system G given by (5.1) and (5.2) n is nonnegative if for every x(0) ∈ R+ and u(t) ≥≥ 0, t ≥ 0, the solution x(t), t ≥ 0, to (5.1) and the output y(t), t ≥ 0, are nonnegative. n

Definition 5.2 states that the nonnegative orthant R+ of the state space is an invariant set with respect to (5.1) for all nonnegative inputs u(t), t ≥ 0. Furthermore, note that nonnegativity of G implies input-output nonnegativity; however, the converse is not generally true. Theorem 5.2. The linear dynamical system G given by (5.1) and (5.2) is nonnegative if and only if A is essentially nonnegative, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0.

NNBook

September 1, 2009

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

147

Proof. First, note that the solution x(t), t ≥ 0, to (5.1) is given by Z t At x(t) = e x0 + eA(t−s) Bu(s)ds, t ≥ 0. 0

Now, if A is essentially nonnegative, then it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0, and if B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0, then it follows n that x(t) ≥≥ 0 and y(t) ≥≥ 0 for all t ≥ 0 and x(0) ∈ R+ , which implies that G is nonnegative. Conversely, suppose G is nonnegative. Now, note that, with u(0) = 0, n y(0) = Cx0 and, since y(0) ≥≥ 0 for all x0 ∈ R+ , it follows that C ≥≥ 0. m Next, with x0 = 0, y(0) = Du(0) and, since y(0) ≥≥ 0 for all u(0) ∈ R+ , it follows that D ≥≥ 0. Now, let x0 = 0 and let u(t) = δ(t − tˆ)ˆ u, t ≥ 0, where m u ˆ ≥≥ 0. In this case, since x(tˆ) = B u ˆ ≥≥ 0 for all u ˆ ∈ R+ , it follows that B ≥≥ 0. Finally, with u(t) = 0, t ≥ 0, x(t) = eAt x0 , and hence, it follows n from Proposition 2.5 that if x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ , then A is essentially nonnegative, which proves the result. For the dynamical system G given by (5.1) and (5.2) defined on the state space Rn , let U and Y define input and output spaces, respectively, consisting of continuous bounded U -valued and Y -valued functions on the semi-infinite interval [0, ∞). The set U ⊆ Rm contains the set of input values, that is, for every u(·) ∈ U and t ∈ [0, ∞), u(t) ∈ U . The set Y ⊆ Rl contains the set of output values, that is, for every y(·) ∈ Y and t ∈ [0, ∞), y(t) ∈ Y . The spaces U and Y are assumed to be closed under the shift operator, that is, if u(·) ∈ U (respectively, y(·) ∈ Y), then the function △ △ defined by uT = u(t + T ) (respectively, yT = y(t + T )) is contained in U (respectively, Y) for all T ≥ 0. Similar definitions hold for the dynamical n system G given by (5.1) and (5.2) defined on the nonnegative orthant R+ with U and Y replaced by U+ and Y+ , where U+ and Y+ denote subsets of U and Y consisting of all nonnegative functions. A similar notation also holds for discrete-time as well as hybrid systems discussed later in this and the next chapter. For the dynamical system G given by (5.1) and (5.2), a function s : Rm × Rl → R such that s(0, 0) = 0 is called a supply rate [308] if it is locally integrable for all input-output pairs satisfying (5.1) and (5.2), that is, for all input-output pairs u(·) ∈ U and y(·) ∈ Y satisfying (5.1) and (5.2), s(·, ·) Rt satisfies t12 |s(u(σ), y(σ))|dσ < ∞, t1 , t2 ≥ 0. For the remainder of the results of this chapter we assume that G is a nonnegative dynamical system, and hence, by Theorem 5.2, A is essentially nonnegative, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. The following definition introduces the notion of dissipativity and

NNBook

September 1, 2009

148

CHAPTER 5

exponential dissipativity for a nonnegative dynamical system. Definition 5.3. The nonnegative dynamical system G given by (5.1) and (5.2) is exponentially dissipative (respectively, dissipative) with respect m l to supply rate s : R+ × R+ → R if there exist a continuous, nonnegativen definite function Vs : R+ → R+ , called a storage function, and a scalar ε > 0 (respectively, ε = 0) such that Vs (0) = 0 and the dissipation inequality Z t2 εt2 εt1 e Vs (x(t2 )) ≤ e Vs (x(t1 )) + eεt s(u(t), y(t))dt, t2 ≥ t1 , (5.4) t1

is satisfied for all t1 , t2 ≥ 0, where x(t), t ≥ t1 , is the solution of (5.1) with u(·) ∈ U+ . The nonnegative dynamical system G given by (5.1) and (5.2) is m l lossless with respect to the supply rate s : R+ × R+ → R if the dissipation inequality (5.4) is satisfied as an equality with ε = 0 for all t2 ≥ t1 and all u(·) ∈ U+ . If Vs (·) is continuously differentiable, then an equivalent statement for exponential dissipativity of a nonnegative dynamical system G is V˙ s (x(t)) + εVs (x(t)) ≤ s(u(t), y(t)),

t ≥ 0,

m

u ∈ R+ ,

l

y ∈ R+ ,

(5.5)

where V˙ s (x(t)) denotes the total derivative of Vs (x) along the state trajectories x(t), t ≥ 0, of (5.1). Since linear nonnegative dynamical systems are a subset of linear dynamical systems, standard dissipativity theory [112, 309] with quadratic storage functions and quadratic supply rates involving Kalman-YakubovichPopov conditions also hold for linear nonnegative dynamical systems. Hence, dissipativity with respect to a quadratic supply rate can be established by formulating a linear matrix inequality feasibility problem [39]. In this chapter, however, motivated by conservation of mass laws, we develop dissipativity notions for nonnegative dynamical systems with respect to linear and nonlinear storage functions and linear supply rates. For a linear dynamical system to be dissipative with respect to a linear supply rate it is necessary that the storage function is also linear. However, since all storage functions are nonnegative by definition, it follows that a storage function is nonnegative if and only if there exists a linear transformation such that the linear dynamical system is nonnegative in a transformed basis. Hence, dissipativity theory of linear dynamical systems with respect to linear supply rates is complete if we restrict our consideration to the class of linear nonnegative dynamical systems. The following result presents Kalman-Yakubovich-Popov conditions for linear nonnegative dynamical systems with linear supply rates of the

NNBook

September 1, 2009

149

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

form s(u, y) = q T y + r T u, where q ∈ Rl , q 6= 0, and r ∈ Rm , r 6= 0. For this result, we assume that there exists a continuously differentiable storage function Vs (x), x ∈ Rn , for the nonnegative dynamical system G. Theorem 5.3. Let q ∈ Rl and r ∈ Rm . Consider the nonnegative dynamical system G given by (5.1) and (5.2) where A is essentially nonnegative, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. Then G is exponentially dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y+r T u n n m if and only if there exist p ∈ R+ , l ∈ R+ , and w ∈ R+ , and a scalar ε > 0 (respectively, ε = 0) such that 0 = AT p + εp − C T q + l, 0 = B T p − D T q − r + w. n

(5.6) (5.7) n

m

Proof. Assume that there exist p ∈ R+ , l ∈ R+ , and w ∈ R+ , and a scalar ε > 0 such that (5.6) and (5.7) hold. Then, with Vs (x) = pT x it n m follows that for all x ∈ R+ and u ∈ R+ , V˙ s (x) + εVs (x) = pT (Ax + Bu) + εpT x = q T Cx − lT x + q T Du + r T u − wT u ≤ q T y + r T u,

which implies that G is exponentially dissipative with respect to the supply rate s(u, y) = q T y + r T u. Conversely, assume that G is exponentially dissipative with respect to the supply rate s(u, y) = q T y + r T u. Now, it follows that there exists a continuously differentiable function Vs (·) such that V˙ s (x) + εVs (x) = Vs′ (x)(Ax + Bu) + εVs (x) ≤ q T y + r T u. n

m

Next, let d : R+ × R+ → R+ be such that d(x, u) = −V˙ s (x) − εVs (x) + s(u, y) = −Vs′ (x)(Ax + Bu) − εVs (x) + q T (Cx + Du) + r T u n m ≥ 0, x ∈ R+ , u ∈ R+ , △

n

m

and note that d(x, u) is linear in u. Hence, since x ∈ R+ and u ∈ R+ are n n arbitrary, there exist continuous functions dx : R+ → R+ and du : R+ → 1×m R+ such that du (0) ≥≥ 0 and d(x, u) = dx (x) + du (x)u, which implies n m that, for all x ∈ R+ and u ∈ R+ , 0 = Vs′ (x)Ax + εVs (x) − q T Cx + dx (x) + [Vs′ (x)B − q T D − r T + du (x)]u.

Now, setting u = 0 yields 0 = Vs′ (x)Ax + εVs (x) − q T Cx + dx (x),

n

x ∈ R+ ,

(5.8)

NNBook

September 1, 2009

150

CHAPTER 5

which further implies that 0 = Vs′ (x)B − q T D − r T + du (x),

n

x ∈ R+ .

(5.9)

Next, expand Vs (·), dx (·), and du (·) as Vs (x) = pT x + Vsr (x), dx (x) = n n m + dxr (x), and du (x) = wT + dur (x), where p ∈ R+ , l ∈ R+ , w ∈ R+ , Vsr (x)/kxk → 0, and dxr (x)/kxk → 0 as kxk → 0. Now, substituting the above expressions into (5.8) and equating coefficients of equal powers yields (5.6). Furthermore, setting x = 0 in (5.9) yields (5.7) with w = dT u (0).

lT x

Finally, the proof of the equivalence between dissipativity with respect to the supply rate s(u, y) = q T y+r T u and (5.6) (with ε = 0) and (5.7) follows analogously with ε = 0. Next, we provide necessary and sufficient conditions for the case where G given by (5.1) and (5.2) is lossless with respect to the linear supply rate s(u, y) = q T y + r T u. Theorem 5.4. Let q ∈ Rl and r ∈ Rm . Consider the nonnegative dynamical system G given by (5.1) and (5.2) where A is essentially nonnegative, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. Then G is lossless with respect to the supply rate s(u, y) = q T y + r T u if and only if there exists n p ∈ R+ such that 0 = AT p − C T q, 0 = B T p − D T q − r.

(5.10) (5.11)

Proof. The proof is analogous to the proof of Theorem 5.3. Recall that in standard dissipativity theory if G is zero-state observable (see Definition 5.5) and there exists a function κ : Rl → Rm such that s(κ(y), y) < 0, y 6= 0, then the storage function Vs (·) satisfies Vs (x) > 0, x ∈ Rn , x 6= 0, [112, 139]. Similarly, for the nonnegative dynamical system G, it can be shown that if G is zero-state observable and there exists a l m l function κ : R+ → R+ such that s(κ(y), y) < 0, y ∈ R+ , y 6= 0, then n Vs (x) > 0, x ∈ R+ , x 6= 0. In the case of a linear supply rate, there exists a matrix K ∈ Rm×l such that q + Kr 0, x ∈ R+ , x 6= 0. For a given l ∈ Rn and w ∈ Rm , note that there exists p ∈ Rn such that (5.6) and (5.7) are satisfied if and only if rank[M, y] = rank M , where     (A + εIn )T C Tq − l △ △ M= , y= . BT DT q + r − w

NNBook

September 1, 2009

151

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

Now, there exist p ≥≥ 0, l ≥≥ 0, and w ≥≥ 0 such that (5.6) and (5.7) are satisfied if and only if the inequalities p ≥≥ 0, z − M p ≥≥ 0

(5.12) (5.13)

are satisfied, where △

z=



C Tq DT q + r



.

Equations (5.12) and (5.13) comprise a set of 2n linear inequalities with pi , i = 1, . . . , n, variables, and hence, the feasibility of p ≥≥ 0 such that (5.12) and (5.13) hold can be checked by standard linear matrix inequality (LMI) techniques [39]. As in the standard dissipativity theory with quadratic supply rates [112, 139], the concepts of linear supply rates and linear storage functions provide a generalized mass and energy interpretation. Specifically, using (5.6) and (5.7) it follows that Z t Z t T T T T [q y(σ) + r u(σ)]dσ = p x(t) − p x(t0 ) + [lT x(σ) + wT u(σ)]dσ, t0

t0

(5.14)

which can be interpreted as a generalized mass balance equation where pT x(t)−pT x(t0 ) is the stored mass of the nonnegative system and the second path-dependent term on the right corresponds to the expelled mass of the nonnegative system. Rewriting (3.22) as V˙ s (x) = pT x˙ = q T y + r T u − [lT x + wT u]

(5.15)

yields a mass flux conservation equation which shows that the system mass transport is equal to the supplied system flux minus the expelled system flux. Note that if a linear nonnegative dynamical system G is dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and if q ≤≤ 0 and u ≡ 0, then it follows that V˙ s (x(t)) ≤ q T y(t) ≤ 0, t ≥ 0. Hence, the undisturbed (u(t) ≡ 0) system G is Lyapunov stable, and hence, by Theorem 2.11, semistable. Furthermore, if a nonnegative dynamical system G is exponentially dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and if q ≤≤ 0 and u ≡ 0, then it follows that V˙ s (x(t)) ≤ −εVs (x(t)) + q T y(t) < 0, x(t) 6= 0, t ≥ 0, where ε > 0. Hence, the undisturbed (u(t) ≡ 0) system G is asymptotically stable. Next, we provide a key definition for nonnegative dynamical systems which are dissipative with respect to a very special supply rate.

NNBook

September 1, 2009

152

CHAPTER 5

Definition 5.4. A nonnegative dynamical system G of the form (5.1) and (5.2) is nonaccumulative (respectively, exponentially nonaccumulative) if G is dissipative (respectively, exponentially dissipative) with respect to the supply rate s(u, y) = eT u − eT y. If G is nonaccumulative, then it follows from (5.15) that V˙ s (x(t)) ≤ eT u(t) − eT y(t),

t ≥ 0,

(5.16)

where u(·) ∈ U+ and y(·) ∈ Y+ . If the components ui (·), i = 1, . . . , m, of u(·) denote flux inputs to the system G and the components yi (·), i = 1, . . . , l, of y(·) denote the flux outputs of the system G, then dissipativity with respect to the linear supply rate s(u, y) = eT u − eT y implies that the system mass transport is always less than or equal to the difference between the system flux input and system flux output. Finally, we show that all compartmental systems with measured outputs corresponding to material outflows are nonaccumulative. Specifially, consider (2.41) with w(t) = u(t) or, equivalently,   ∂Vs T + u(t), x(0) = x0 , t ≥ 0, (5.17) x(t) ˙ = [Jn (x) − D(x)] ∂x and with storage function Vs (x) = eT x and output   ∂Vs T = [a11 x1 , a22 x2 , . . . , ann xn ]T . y = D(x) ∂x Now, it follows that V˙ s (x) = e

T

"

[Jn (x) − D(x)]



∂Vs ∂x

T

+u

#

= eT u − eT y + eT Jn (x)e n = eT u − eT y, x ∈ R+ , which shows that all compartmental systems with outputs y = D(x) are lossless with respect to the supply rate s(u, y) = eT u − eT y.

(5.18)
∂Vs T ∂x

If, alternatively, the outputs y correspond to a partial observation of the material outflows, then it follows that the compartmental system is dissipative with respect to the supply rate s(u, y) = eT u − eT y. To see this, assume without loss of generality that the first l outflows are observed, that is, y = [a11 x1 , a22 x2 , . . . , all xl ]. Now, note that in this case     ∂Vs T ∂Vs T − Dr (x) , y = D(x) ∂x ∂x

NNBook

September 1, 2009

153

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS n

where Dr (x) = diag[0, . . . , 0, al+1l+1 xl+1 , . . . , ann xn ] ≥≥ 0, x ∈ R+ . Hence, # " T  ∂V s +u V˙ s (x) = eT [Jn (x) − D(x)] ∂x     ∂Vs ∂Vs T T T = e u−e y− Dr (x) ∂x ∂x ≤ eT u − eT y,

n

x ∈ R+ .

(5.19)

n Note that in the case where the system is closed, V˙ s (x) = 0, x ∈ R+ , indicating conservation of mass in the system.

Example 5.1. In this example, we show that the lipoprotein metabolism model considered in Example 2.2 is nonaccumulative. Since the material outflow of the first compartment corresponding to blood plasma cannot be measured directly, we measure the concentration of the material, that is, y = a11 x1 /v1 , where v1 is the volume of distribution of the blood plasma in the first compartment. Now, using the storage function Vs (x1 , x2 ) = x1 + x2 it follows that the lipoprotein metabolism model is lossless with respect to the supply rate s(u, y) = u1 − v1 y. △ 6 0, t ≥ 0, and Example 5.2. In this example, we show that for u(t) = output y(t) = a11 x1 (t) the lead kinetic model studied in Example 2.3 is nonaccumulative. This is immediate with storage function Vs (x1 , x2 , x3 ) = x1 + x2 + x3 . △

5.3 Feedback Interconnections of Nonnegative Dynamical Systems Feedback systems are pervasive in nature and can be found almost everywhere in living systems. In particular, control at the intercellular level, DNA replication and cell division, control of gene expression, control of enzyme activity, control at the organ system and organism level, humoral control, neural control, and regulation in biological systems all involve feedback systems [155]. To analyze these complex nonnegative systems, the notion of dissipativity, with appropriate storage functions and supply rates, can be used to construct Lyapunov functions by appropriately combining storage functions for each subsystem and examining their respective structure. In this section, we consider feedback interconnections of nonnegative dynamical systems. We begin by considering the nonnegative dynamical system G given by (5.1) and (5.2) with the nonnegative dynamical feedback system Gc given by x˙ c (t) = Ac xc (t) + Bc uc (t),

xc (0) = xc0 ,

t ≥ 0,

(5.20)

NNBook

September 1, 2009

154

CHAPTER 5

yc (t) = Cc xc (t),

(5.21)

where Ac ∈ Rnc ×nc , Bc ∈ Rnc ×mc , Cc ∈ Rlc ×nc , Ac is essentially nonnegative, Bc ≥≥ 0, and Cc ≥≥ 0. Theorem 5.5. Let q ∈ Rl , r ∈ Rm , qc ∈ Rlc , and rc ∈ Rmc . Consider the linear nonnegative dynamical systems G and Gc given by (5.1) and (5.2), and (5.20) and (5.21), respectively, with uc = y and yc = u. Assume that G is dissipative with respect to the linear supply rate s(u, y) = q T y+r T u and with a positive-definite, linear storage function Vs (x) = pT x, and assume that Gc is dissipative with respect to the linear supply rate s(uc , yc ) = qcT yc + rcT uc and with a positive-definite, linear storage function Vsc (xc ) = pT c xc . Then the following statements hold: i ) If there exists a scalar σ > 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is semistable. ii ) If (A, C) and (Ac , Cc ) are observable and there exists a scalar σ > 0 such that q + σrc 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is asymptotically stable. Proof. Note that the positive feedback interconnection of G and Gc is given by u = yc and uc = y so that the closed-loop dynamics of G and Gc is given by      x(t) ˙ A BCc x(t) = , x˙ c (t) Bc C Ac + Bc DCc xc (t) which implies that the closed-loop dynamics matrix   A BCc △ A˜ = Bc C Ac + Bc DCc is essentially nonnegative. Hence, the closed-loop system is also nonnegative, and thus, x(t) ≥≥ 0, xc (t) ≥≥ 0, u(t) ≥≥ 0, and y(t) ≥≥ 0, t ≥ 0.

NNBook

September 1, 2009

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

155

i) Consider the Lyapunov function candidate V (x, xc ) = pT x + σpT c xc and note that the Lyapunov derivative satisfies V˙ (x, xc ) ≤ q T y + r T u + σ(qcT yc + rcT uc ) = (q + σrc )T y + (r + σqc )T u ≤ 0, for all y ≥≥ 0 and u ≥≥ 0, establishing Lyapunov stability. semistability is an immediate consequence of Theorem 2.11.

(5.22) Now,

ii) With V (x, xc ) = pT x + σpT c xc Lyapunov stability follows as in i). Furthermore, if q + σrc 0 and u >> 0 such that either y 6= 0 or u 6= 0. The result now follows by applying the Krasovskii-LaSalle theorem and using the observability assumptions. iii) With V (x, xc ) = pT x + σpT c xc Lyapunov stability follows as in i). Next, if Gc is exponentially dissipative it follows that V˙ (x, xc ) ≤ −εVsc (xc ), nc n nc △ xc ∈ R+ . Now, define R = {(x, xc ) ∈ R+ × R+ : V˙ (x, xc ) = 0}. Since V˙ (x(t), xc (t)) ≤ −εVsc (xc (t)), it follows that xc (t) ≡ 0 and u(t) = yc (t) = Cc xc (t) ≡ 0 and, since rank Bc = mc , it follows that uc (t) = y(t) = 0 and Cx(t) = 0. Now, since (A, C) is observable it follows that x(t) ≡ 0. Hence, the largest invariant subset contained in R is given by M = {(0, 0)}, and hence, it follows from the Krasovskii-LaSalle theorem that (x(t), xc (t)) → (0, 0) as t → ∞. iv) With V (x, xc ) = pT x + σpT c xc Lyapunov stability follows as in i). Next, if G and Gc are exponentially dissipative it follows that n nc V˙ (x, xc ) ≤ −εVs (x) − εc Vsc (xc ) ≤ − min{ε, εc }V (x, xc ), (x, xc ) ∈ R+ × R+ , which establishes asymptotic stability. It is important to note that Theorem 5.5 also holds for the more general architecture of the feedback system Gc wherein yc (t) = Cc xc (t) + Dc uc (t), where Dc ∈ Rlc ×mc and Dc ≥≥ 0. In this case, however, we assume that the positive feedback interconnection of G and Gc is well posed, that is, det[Im − Dc D] 6= 0. Similar remarks hold for the rest of the results in this section as well as the results of Section 5.7. The following corollary to Theorem 5.5 addresses linear supply rates of the form s(u, y) = eT u − eT y. Corollary 5.1. Consider the linear nonnegative dynamical systems G and Gc given by (5.1) and (5.2), and (5.20) and (5.21), respectively, with uc = y and yc = u. Assume that G is nonaccumulative with a positive-definite, linear storage function Vs (x) = pT x, and assume that Gc is exponentially

NNBook

September 1, 2009

156

CHAPTER 5

nonaccumulative with a positive-definite, linear storage function Vsc (xc ) = pT c xc . Then the following statements hold: i ) If (A, C) is observable and rank Bc = mc , then the positive feedback interconnection of G and Gc is asymptotically stable. ii ) If G is exponentially nonaccumulative, then the positive feedback interconnection of G and Gc is asymptotically stable. Proof. The proof is a direct consequence of iii) and iv) of Theorem 5.5 with σ = 1, q = −rc = −e, and r = −qc = e. Next, we develop absolute stability criteria for nonnegative feedback systems with nonnegative time-varying memoryless input nonlinearities. Since absolute stability theory concerns the stability for classes of feedback nonlinearities which, as noted in [107], can readily be interpreted as an uncertainty model, the proposed framework can be used to analyze robustness of biological and physiological systems developed from data models. Specifically, given the linear nonnegative system G characterized by (5.1) and (5.2) we derive sufficient conditions that guarantee asymptotic stability of the feedback interconnection involving the linear nonnegative system G and the feedback nonnegative time-varying input nonlinearity σ(·, ·) ∈ Φ, where △

l

m

l

Φ = {σ : R+ × R+ → R+ : σ(·, 0) = 0, 0 ≤≤ σ(t, y) ≤≤ M y, y ∈ R+ , l

a.a. t ≥ 0, and σ(·, y) is Lebesgue measurable for all y ∈ R+ }, (5.23) M >> 0, and M ∈ Rm×l . Theorem 5.6. Consider the nonnegative dynamical system G given by (5.1) and (5.2), and assume that (A, C) is observable and G is exponentially dissipative with respect to the supply rate s(u, y) = eT u − eT M y, where M >> 0. Then, the positive feedback interconnection of G and σ(·, ·) is globally (uniformly)3 asymptotically stable for all σ(·, ·) ∈ Φ. l

Proof. Since σ(t, y) ≥≥ 0 for all t ≥ 0, y ∈ R+ , and (5.1) and (5.2) is a nonnegative dynamical system, it follows that the positive feedback interconnection of G and σ(·, ·) given by x(t) ˙ = Ax(t) + Bσ(t, y(t)),

x(0) = x0 ,

t ≥ 0,

3 Since the positive feedback interconnection of G and σ(·, ·) results in a time-varying system, it is important to distinguish between asymptotic stability and uniform asymptotic stability. As for the case of time-invariant systems, stability notions for time-varying nonnegative systems need to n be defined with respect to relatively open subsets of R+ .

NNBook

September 1, 2009

157

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

is a nonnegative dynamical system for all σ(·, ·) ∈ Φ. Next, since (A, C) is observable and G is exponentially dissipative with respect to the supply rate s(u, y) = eT u−eT M y, it follows from Theorem 5.3 of [112] and Theorem 5.3, n m with r = e and q = −M T e, that there exist p ∈ Rn+ , l ∈ R+ , and w ∈ R+ , and a scalar ε > 0 such that 0 = AT p + εp + C T M T e + l, 0 = B T p + D T M T e − e + w.

(5.24) (5.25)

Next, consider the Lyapunov function candidate Vs (x) = pT x and note that the Lyapunov derivative satisfies V˙ s (x) = pT (Ax + Bσ) = −εpT x + eT [σ − M y] − lT x − wT σ ≤ −εVs (x) + eT [σ − M y]. n

Now, since Vs (x) > 0, x ∈ R+ , x 6= 0, and σ ≤≤ M y for all σ(·, ·) ∈ Φ, n it follows that V˙ s (x) < 0, x ∈ R+ , x 6= 0. Hence, the positive feedback interconnection of G and σ(·, ·) is globally (uniformly) asymptotically stable for all σ(·, ·) ∈ Φ. To consider nonlinearities with upper and lower bounds of the form M1 y ≤≤ σ(t, y) ≤≤ M2 y, where σ(·, ·) ∈ Φ, we can use the standard loop shifting techniques discussed in [112, 174]. In this case, Theorem 5.6 holds with σ(t, y), A, B, C, D, and M replaced by σ(t, y) − M1 y, A + B(I − M1 D)−1 M1 C, B(I −M1 D)−1 , (I −DM1 )−1 C, (I −DM1 )−1 D, and M2 −M1 , respectively. To develop robust stability results for nonnegative dynamical systems, consider the set F defined by m×l

△

F = {F : R+ → R+

: F (·) is Lebesgue measurable and 0 ≤≤ F (t) ≤≤ M, a.a. t ≥ 0}.

(5.26)

That is, F includes σ in Φ of the form σ(t, y) = F (t)y. Furthermore, consider the uncertain system x(t) ˙ = (A + ∆A(t))x(t),

x(0) = x0 ,

t ≥ 0,

(5.27)

where A is a nominal essentially nonnegative, asymptotically stable matrix and ∆A(·) ∈ U , where U is the uncertainty set △

U = {∆A(·) : ∆A(t) = BF (t)C, F (·) ∈ F}.

(5.28)

In this case, it follows from Theorem 5.6 with D = 0 that the zero solution x(t) ≡ 0 to the linear nonnegative uncertain system (5.27) is globally (uniformly) asymptotically stable for all F (·) ∈ F.

NNBook

September 1, 2009

158

CHAPTER 5

The uncertain dynamical system model given by (5.27) considers an additive (absolute) uncertainty structure. Next, we show that nonnegative dynamical systems are also robust with respect to a multiplicative (relative) uncertainty characterization. Specifically, consider the uncertain system x(t) ˙ = F AGx(t),

x(0) = x0 ,

t ≥ 0,

(5.29)

where A ∈ Rn×n is essentially nonnegative and F ∈ Rn×n and G ∈ Rn×n are positive and diagonal but otherwise unknown, and (5.29) is asymptotically stable with F = G = In , that is, the nominal system is asymptotically stable. Since the nominal system is asymptotically stable, it follows from Theorem 2.12 that there exist a positive diagonal matrix P ∈ Rn×n and a positive-definite matrix R ∈ Rn×n such that 0 = AT P + P A + R.

(5.30)

Now, choosing the Lyapunov function candidate V (x) = xT GP F −1 x for the n uncertain system (5.29), it follows that V˙ (x) = −xT GRGx < 0, x ∈ R+ , x 6= 0, which proves asymptotic stability for the perturbed system (5.29). Finally, we present a straightforward but key property of cascade interconnections of nonnegative dynamical systems. Proposition 5.1. Consider the nonnegative dynamical systems G1 and G2 with input-output pairs (u1 , y1 ) and (u2 , y2 ), respectively. Assume G1 and G2 are nonaccumulative. Then the cascade interconnection G = G2 G1 with input-output pair (u, y) = (u1 , y2 ) is nonaccumulative. Proof. The result is a direct consequence of Definition 5.3 by noting that the interconnection constraint for a cascade interconnection is given by u = u1 and y = y2 . Proposition 5.1 is not surprising in light of the fact that nonaccumulativity of a nonnegative dynamical system implies that the system mass transport is always less than or equal to the difference between the system flux input and system flux output. In fact, Proposition 5.1 holds for n cascade-interconnected systems. However, it is important to note that a similar result to Proposition 5.1 does not hold for parallel and feedback interconnections.

5.4 Dissipativity Theory for Nonlinear Nonnegative Dynamical Systems In this section, we extend the notion of dissipativity to nonlinear nonnegative dynamical systems. Specifically, we consider nonlinear dynamical systems

NNBook

September 1, 2009

159

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

G of the form x(t) ˙ = f (x(t)) + G(x(t))u(t), y(t) = h(x(t)) + J(x(t))u(t),

x(0) = x0 ,

t ≥ 0,

(5.31) (5.32)

where x ∈ Rn , u ∈ Rm , y ∈ Rl , f : Rn → Rn , G : Rn → Rn×m , h : Rn → Rl , and J : Rn → Rl×m . We assume that f (·), G(·), h(·), and J(·) are continuously differentiable mappings and f (xe ) = 0 and h(xe ) = 0. For simplicity of exposition here we assume xe = 0.4 For the nonlinear dynamical system G given by (5.31) and (5.32) the definitions of inputoutput nonnegativity, nonnegativity, dissipativity, exponential dissipativity, and losslessness hold with (5.1) and (5.2) replaced by (5.31) and (5.32). The following result is now immediate. Proposition 5.2. Consider the nonlinear dynamical system G given by (5.31) and (5.32). If f : D → Rn is essentially nonnegative, h(x) ≥≥ 0, n G(x) ≥≥ 0, and J(x) ≥≥ 0, x ∈ R+ , then G is nonnegative. Proof. The nonnegativity of x(t), t ≥ 0, is a direct consequence of n Proposition 4.3. Now, since h(x) and J(x) are nonnegative for all x ∈ R+ , y(t) ≥≥ 0, t ≥ 0, is immediate. Hence, G is nonnegative. To state the main result of this section, the following definition is required. Definition 5.5. A nonnegative dynamical system G is zero-state observable if u(t) ≡ 0 and y(t) ≡ 0 implies x(t) ≡ 0. G is completely n reachable if for all x0 ∈ D ⊆ R+ there exist a finite time ti < 0 and a square integrable nonnegative input u(t) defined on [ti , 0] such that the state x(t), t ≥ ti , can be driven from x(ti ) = 0 to x(0) = x0 . G is completely null n controllable if for all x0 ∈ D ⊆ R+ there exist a finite time tf > 0 and a square integrable nonnegative input u(t) defined on [0, tf ] such that x(t), t ≥ 0, can be driven from x(0) = x0 to x(tf ) = 0. The following result presents a nonlinear extension to Theorem 5.3. As in the linear case, if G is zero-state observable and there exists a function l m n κ : R+ → R+ such that s(κ(y), y) < 0, y 6= 0, then Vs (x) > 0, x ∈ R+ , x 6= 0. Hence, for linear supply rates s(u, y) = q T y + r T u, where q ∈ Rl , q 6= 0, and r ∈ Rm , r 6= 0, it follows that there exists K ∈ Rm×l such that q + Kr 0, x ∈ R+ , x 6= 0. For this result we assume there exists a continuously differentiable 4 In the case where G is nonnegative, this assumption is not without loss of generality since shifting the equilibrium can destroy the essential nonnegativity of the vector field f and the nonnegativity of h. However, using minor modifications in the proofs of the theorems in this section, the results of this section can also be extended to the case where xe 6= 0.

NNBook

September 1, 2009

160

CHAPTER 5 n

storage function Vs (x), x ∈ R+ , for the nonnegative dynamical system G. Theorem 5.7. Let q ∈ Rl and r ∈ Rm . Consider the nonlinear nonnegative dynamical system G given by (5.31) and (5.32) where f : D → Rn is essentially nonnegative, G(x) ≥≥ 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, n x ∈ R+ . Then G is exponentially dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + r T u if and only if there exist n n n m functions Vs : R+ → R+ , ℓ : R+ → R+ , and W : R+ → R+ , and a scalar ε > 0 (respectively, ε = 0) such that Vs (·) is continuously differentiable, n Vs (0) = 0, and, for all x ∈ R+ , 0 = Vs′ (x)f (x) + εVs (x) − q T h(x) + ℓ(x), 0 = Vs′ (x)G(x) − q T J(x) − r T + W T (x). n

(5.33) (5.34) n

Proof. Assume that there exist Vs : R+ → R+ , ℓ : R+ → R+ , n m and W : R+ → R+ , and a scalar ε > 0 such that Vs (·) is continuously differentiable, Vs (0) = 0, and (5.33) and (5.34) hold. Then it follows that n for all x ∈ R+ and every admissible input u(·) ∈ U+ , V˙ s (x) + εVs (x) = Vs′ (x)(f (x) + G(x)u) + εVs (x) = q T h(x) − ℓ(x) + q T J(x)u + r T u − W T (x)u ≤ q T y + r T u,

which implies that G is exponentially dissipative with respect to the supply rate s(u, y) = q T y + r T u. Conversely, assume that G is exponentially dissipative with respect to the supply rate s(u, y) = q T y + r T u. Now, it follows that there exists a continuously differentiable function Vs (·) such that V˙ s (x) + εVs (x) = Vs′ (x)(f (x) + G(x)u) + εVs (x) ≤ q T y + r T u. n

m

Next, let d : R+ × R+ → R+ be such that d(x, u) = −V˙ s (x) − εVs (x) + s(u, y) = −Vs′ (x)[f (x) + G(x)u] − εVs (x) + q T [h(x) + J(x)u] + r T u n m ≥ 0, x ∈ R+ , u ∈ R+ , △

n

m

and note that d(x, u) is linear in u. Hence, since x ∈ R+ and u ∈ R+ are n n arbitrary, there exist continuous functions dx : R+ → R+ and du : R+ → 1×m n R+ such that du (x) ≥≥ 0, x ∈ R+ , and d(x, u) = dx (x) + du (x)u, which n m implies that, for all x ∈ R+ and u ∈ R+ , 0 = Vs′ (x)f (x)+εVs (x)−q T h(x)+dx (x)+[Vs′ (x)G(x)−q T J(x)−r T +du (x)]u.

Now, setting u = 0 yields (5.33) with ℓ(x) = dx (x), which further implies

NNBook

September 1, 2009

161

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

(5.34) with W(x) = dT u (x). Finally, the proof of the equivalence between dissipativity with respect to the supply rate s(u, y) = q T y + r T u and (5.33) (with ε = 0) and (5.34) follows analogously with ε = 0. As in the linear case, Theorem 5.7 provides a generalized mass and energy balance interpretation. In particular, using (5.33) and (5.34) it follows that Z t [q T y(σ) + r T u(σ)]dσ = Vs (x(t)) − Vs (x(t0 )) t0

+

Z

t

t0

[ℓT (x(σ))x(σ) + W T (x(σ))u(σ)]dσ, (5.35)

which can be interpreted as a generalized mass balance equation where Vs (x(t)) − Vs (x(t0 )) is the stored mass of the nonnegative system and the second path-dependent term on the right corresponds to the expelled mass of the nonnegative system. Rewriting (5.14) as V˙ s (x) = Vs′ (x)x˙ = q T y + r T u − [ℓT (x)x + W T (x)u]

(5.36)

yields a mass flux conservation equation which shows that the system mass transport is equal to the supplied system flux minus the expelled system flux. The next result is a nonlinear extension to Theorem 5.4. Theorem 5.8. Let q ∈ Rl and r ∈ Rm . Consider the nonlinear nonnegative dynamical system G given by (5.31) and (5.32) where f : D → Rn is essentially nonnegative, G(x) ≥≥ 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, n x ∈ R+ . Then G is lossless with respect to the supply rate s(u, y) = q T y+r T u n if and only if there exists a function Vs : R+ → R+ such that Vs (·) is n continuously differentiable, Vs (0) = 0, and, for all x ∈ R+ , 0 = Vs′ (x)f (x) − q T h(x), 0 = Vs′ (x)G(x) − q T J(x) − r T .

(5.37) (5.38)

Proof. The proof is analogous to the proof of Theorem 5.7. As in the linear case, it is important to note that all nonlinear compartmental systems with measured outputs corresponding to material outflows are lossless with respect to the supply rate s(u, y) = eT u − eT y.

NNBook

September 1, 2009

162

CHAPTER 5

This can be easily seen from (2.48) with w(t) = u(t) or, equivalently,   ∂Vs T + u(t), x(0) = x0 , t ≥ 0, (5.39) x(t) ˙ = [Jn (x) − D(x)] ∂x and with storage function Vs (x) = eT x and output   ∂Vs T = h(x) = [ˆ a11 (x), a ˆ22 (x), . . . , a ˆnn (x)]T . y = D(x) ∂x Specifically, "

V˙ s (x) = eT [Jn (x) − D(x)]



∂Vs ∂x

= eT u − eT y + eT Jn (x)e n = eT u − eT y, x ∈ R+ ,

T

+u

# (5.40)
∂Vs T

which shows that all compartmental systems with outputs y = D(x) ∂x are lossless with respect to the supply rate s(u, y) = eT u − eT y. If, alternatively, the outputs y correspond to a partial observation of the material outflows, then it follows that the compartmental system is dissipative with respect to the supply rate s(u, y) = eT u − eT y. To see this assume without loss of generality that the first l outflows are observed, that is, y = [ˆ a11 (x)x1 , a ˆ22 (x)x2 , . . . , a ˆll (x)xl ]. Now, note that     ∂Vs T ∂Vs T − Dr (x) , y = D(x) ∂x ∂x n

where Dr (x) = diag[0, . . . , 0, aˆl+1l+1 (x)xl+1 , . . . , a ˆnn (x)xn ] ≥≥ 0, x ∈ R+ . Hence, # " T  ∂V s T +u V˙ s (x) = e [Jn (x) − D(x)] ∂x     ∂Vs T ∂Vs T T T = e u−e y− Dr (x) ∂x ∂x ≤ eT u − eT y,

n

x ∈ R+ .

(5.41)

Example 5.3. In this example we show that for u(t) ∈ R+ , t ≥ 0, and measured output y(t) = µx2 (t) the SIR epidemic model considered in Example 2.6 is nonaccumulative. Specifically, with storage function Vs (x1 , x2 , x3 ) = x1 +x2 +x3 , it follows that V˙ s (x1 , x2 , x3 ) = u−y −µx1 −µx3 , which implies that V˙ s (x1 , x2 , x3 ) ≤ u − y. △ Finally, we present a key result on linearization of nonnegative dissipative dynamical systems. For this result we assume that the storage

NNBook

September 1, 2009

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

163

function Vs (·) is three times continuously differentiable. Theorem 5.9. Let q ∈ Rl and r ∈ Rm , and consider the nonlinear nonnegative dynamical system G given by (5.31) and (5.32) where f : D → Rn is essentially nonnegative, G(x) ≥≥ 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, n x ∈ R+ . Suppose G is exponentially dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + r T u. Then, there exist n n m p ∈ R+ , l ∈ R+ , and w ∈ R+ , and a scalar ε > 0 (respectively, ε = 0) such that (5.6) (respectively, (5.6) with ε = 0) and (5.7) are satisfied where ∂h ∂f , B = G(0), C = , D = J(0). A= (5.42) ∂x x=0 ∂x x=0 If, in addition, (A, C) is observable, then p >> 0.

Proof. First note that since G is exponentially dissipative (respectively, dissipative) with respect to the linear supply rate s(u, y) = q T y + r T u n it follows from Theorem 5.7 that there exist functions Vs : R+ → R+ , n n n m ℓ : R+ → R+ , W : R+ → R+ , and a scalar ε > 0 (respectively, ε = 0) such that (5.33) (respectively, (5.33) with ε = 0) and (5.34) are satisfied. Now, expanding Vs (·) via a Taylor series expansion about x = 0 and using the fact that Vs (·) is nonnegative and Vs (0) = 0, it follows that there exists n n a nonnegative p ∈ R+ such that Vs (x) = pT x + Vsr (x), where Vsr : R+ → R contains the higher-order terms of Vs (x). Next, let f (x) = Ax+fr (x), ℓ(x) = lT x+ℓr (x), and h(x) = Cx+hr (x), where fr (x), ℓr (x), and hr (x), contain the nonlinear terms of f (x), ℓ(x), and h(x), respectively, and let G(x) = B + Gr (x), J(x) = D + Jr (x), and △ W(x) = w + Wr (x), where w = W(0) and Gr (x), Jr (x), and Wr (x) contain the nonconstant terms of G(x), J(x), and W(x), respectively. Using the above expressions, (5.33) can be written as 0 = xT [AT p + εp − C T q + l] + γ(x),

(5.43)

where γ(x) = Vsr′ (x)f (x) + pT fr (x) + εVsr (x) − q T hr (x) + ℓr (x). △

Now, viewing (5.43) as the Taylor’s series expansion of (5.33) about x = 0 and noting that |γ(x)| = 0, kxk→0 kxk lim

it follows that p satisfies (5.6). Now, (5.7) follows from (5.34) by setting x = 0. Finally, suppose (A, C) is observable and note that it follows from

NNBook

September 1, 2009

164

CHAPTER 5

Theorem 5.3 that the linear system x(t) ˙ = Ax(t) + Bu(t), y(t) = Cx(t) + Du(t),

x(0) = x0 ,

t ≥ 0,

with storage function Vs (x) = pT x is exponentially dissipative (respectively, dissipative) with respect to supply rate s(u, y). Now, it follows from Theorem 5.3 of [112] that p >> 0. Linearization results for nonaccumulativity follow directly from Theorem 5.9.

5.5 Feedback Interconnections of Nonnegative Nonlinear Dynamical Systems In this section, we consider feedback interconnections of nonlinear nonnegative dynamical systems. The results of this section very closely parallel the results of Section 5.3 and, hence, detailed proofs are omitted. We begin by considering the nonlinear nonnegative dynamical system G given by (5.31) and (5.32) with the nonlinear nonnegative dynamical feedback system Gc given by x˙ c (t) = fc (xc (t)) + Gc (xc (t))uc (t), yc (t) = hc (xc (t)),

xc (0) = xc0 ,

t ≥ 0,

(5.44) (5.45)

where fc : Rnc → Rnc , Gc : Rnc → Rnc ×mc , hc : Rnc → Rlc , fc is essentially nc nonnegative, Gc (xc ) ≥≥ 0, and hc (xc ) ≥≥ 0, xc ∈ R+ . Theorem 5.10. Let q ∈ Rl , r ∈ Rm , qc ∈ Rlc , and rc ∈ Rmc . Consider the nonlinear nonnegative dynamical systems G and Gc given by (5.31) and (5.32), and (5.44) and (5.45), respectively, with uc = y and yc = u. Assume that G is dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and with a continuously differentiable, positive-definite storage function Vs (·), and assume that Gc is dissipative with respect to the linear supply rate s(uc , yc ) = qcT yc + rcT uc and with a continuously differentiable, positivedefinite storage function Vsc (·). Then the following statements hold: i ) If there exists a scalar σ > 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is Lyapunov stable. ii ) If G and Gc are zero-state observable and there exists a scalar σ > 0 such that q + σrc 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is asymptotically stable. Proof. Note that the positive feedback interconnection of G and Gc is given by u = yc and uc = y so that the closed-loop dynamics of G and Gc is given by     x(t) ˙ f (x(t)) + G(x(t))hc (xc (t)) = , x˙ c (t) fc (xc (t)) + Gc (xc (t))h(x(t)) + Gc (xc (t))J(x(t))hc (xc (t)) which implies that

△ f˜(x, xc ) =



f (x) + G(x)hc (xc ) fc (xc ) + Gc (xc )h(x) + Gc (xc )J(x)hc (xc )



is essentially nonnegative. Hence, the closed-loop system is also nonnegative and thus x(t) ≥≥ 0, xc (t) ≥≥ 0, u(t) ≥≥ 0, and y(t) ≥≥ 0, t ≥ 0. Now, with the Lyapunov function candidate V (x, xc ) = Vs (x)+σVsc (xc ), the proof is identical to that given in Theorem 5.5 and, hence, is omitted. The following corollary to Theorem 5.10 addresses linear supply rates of the form s(u, y) = eT u − eT y. Corollary 5.2. Consider the nonlinear nonnegative dynamical systems G and Gc given by (5.31), (5.32) and (5.44), (5.45), respectively. Assume that G is nonaccumulative with a continuously differentiable, positive-definite storage function Vs (·), and assume that Gc is exponentially nonaccumulative with a continuously differentiable, positive-definite storage function Vsc (·). Then the following statements hold: nc

i ) If G is zero-state observable and rank Gc (xc ) = mc , xc ∈ R+ , then the positive feedback interconnection of G and Gc is asymptotically stable. ii ) If G is exponentially nonaccumulative, then the positive feedback interconnection of G and Gc is asymptotically stable. Proof. The proof is a direct consequence of iii) and iv) of Theorem 5.10 with σ = 1, q = −rc = −e, and r = −qc = e.

NNBook

September 1, 2009

166

CHAPTER 5

Finally, we develop absolute stability criteria for nonlinear nonnegative feedback systems with nonnegative time-varying memoryless input nonlinearities. Specifically, given the nonlinear nonnegative system G characterized by (5.31) and (5.32) we derive sufficient conditions that guarantee asymptotic stability of the feedback interconnection involving the nonlinear nonnegative system G and the feedback nonnegative time-varying nonlinearity σ(·, ·) ∈ Φ, where Φ is given by (5.23). Theorem 5.11. Consider the nonlinear nonnegative dynamical system G given by (5.31) and (5.32), and assume that G is zero-state observable and exponentially dissipative with respect to the supply rate s(u, y) = eT u − eT M y, where M >> 0, and with a positive-definite, radially unbounded storage function Vs (·). Then, the positive feedback interconnection of G and σ(·, ·) is globally (uniformly) asymptotically stable for all σ(·, ·) ∈ Φ. Proof. The proof is similar to that of Theorem 5.6 and, hence, is omitted.

5.6 Dissipativity Theory for Discrete-Time Nonnegative Dynamical Systems In this section, we develop the notion of dissipativity for discrete-time nonlinear nonnegative dynamical systems. Specifically, we consider discretetime dynamical systems G of the form x(k + 1) = f (x(k)) + G(x(k))u(k), y(k) = h(x(k)) + J(x(k))u(k),

x(0) = x0 ,

k ∈ Z+ ,

(5.46) (5.47)

where x ∈ Rn , u ∈ Rm , y ∈ Rl , f : Rn → Rn , G : Rn → Rn×m , h : Rn → Rl , and J : Rn → Rl×m . We assume that f (·), G(·), h(·), and J(·) are continuous mappings and f (xe ) = xe and h(xe ) = xe . For simplicity of exposition here we assume xe = 0.5 First, we provide definitions and several results concerning dynamical systems of the form (5.46) and (5.47) with nonnegative inputs and nonnegative outputs. Definition 5.6. The nonlinear dynamical system G given by (5.46) and (5.47) with x(0) = 0 is input-output6 nonnegative if y(k), k ∈ Z+ , is nonnegative for every nonnegative input u(k), k ∈ Z+ . Definition 5.7. The nonlinear dynamical system G given by (5.46) 5 As in the continuous-time case, this assumption is not without loss of generality since shifting the equilibrium can destroy the nonnegativity of the vector field f and the nonnegativity of h. However, using minor modifications in the proofs of the theorems in this section, the results of this section also extend to the case where xe 6= 0. 6 As in the continuous-time case, the outputs here refer to measured outputs or observations and may have nothing to do with material outflows of the nonnegative compartmental system.

NNBook

September 1, 2009

167

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS n

and (5.47) is nonnegative if for every x(0) ∈ R+ and u(k) ≥≥ 0, k ∈ Z+ , the solution x(k), k ∈ Z+ , to (5.46) and the output y(k), k ∈ Z+ , are nonnegative. Proposition 5.3. Consider the nonlinear dynamical system G given by (5.46) and (5.47). If f : D → Rn is nonnegative, G(x) ≥≥ 0, h(x) ≥≥ 0, n and J(x) ≥≥ 0, x ∈ R+ , then G is nonnegative. Proof. The nonnegativity of x(k), k ∈ Z+ , is a direct consequence of n Proposition 4.6. Now, since h(x) ≥≥ 0 and J(x) ≥≥ 0, x ∈ R+ , it follows m that y(k) ∈ R+ , k ∈ Z+ . Hence, G is nonnegative. For the dynamical system G given by (5.46) and (5.47) all input-output pairs u(·) ∈ U+ , y(·) ∈ Y+ , are defined on Z+ . Hence, the supply rate s : Rm × Rl → R satisfying s(0, 0) = 0 is locally summable for all inputP 2 output pairs satisfying (5.46) and (5.47), that is, ki=k |s(u(i), y(i))| < ∞, 1 k1 , k2 ∈ Z+ . For the remainder of the results of this section we assume that n f (x) ≥≥ 0, G(x) ≥≥ 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, x ∈ R+ . The following definition introduces the notion of dissipativity and geometric dissipativity for a discrete-time nonnegative dynamical system. Definition 5.8. The nonnegative dynamical system G given by (5.46) and (5.47) is geometrically dissipative (respectively, dissipative) with respect m l to the supply rate s : R+ × R+ → R if there exist a continuous, nonnegativen definite function Vs : R+ → R+ , called a storage function, and a scalar ρ > 1 (respectively, ρ = 1) such that Vs (0) = 0 and the dissipation inequality ρk Vs (x(k)) ≤ ρk0 Vs (x(k0 )) +

k−1 X

ρi s(u(i), y(i)),

i=k0

k ≥ k0 ,

(5.48)

is satisfied for all k0 , k ∈ Z+ , where x(k), k ≥ k0 , is the solution of (5.46) with u(·) ∈ U+ . The nonnegative dynamical system G given by (5.46) and m l (5.47) is lossless with respect to the supply rate s : R+ × R+ → R if the dissipation inequality (5.48) is satisfied as an equality with ρ = 1 for all k ≥ k0 and all u(·) ∈ U+ . An equivalent statement for geometric dissipativity of a discrete-time nonnegative dynamical system G is ρVs (x(k + 1)) − Vs (x(k)) ≤ s(u(k), y(k)),

k ∈ Z+ ,

m

u ∈ R+ ,

l

y ∈ R+ . (5.49)

Since discrete-time nonnegative dynamical systems are a subset of discrete-time dynamical systems, standard discrete-time dissipativity theory

NNBook

September 1, 2009

168

CHAPTER 5

[54, 112] with quadratic storage functions and quadratic supply rates involving Kalman-Yakubovich-Popov conditions also holds for discrete-time nonnegative dynamical systems. In this section, however, motivated by conservation of mass laws, we develop dissipativity notions for discrete-time nonnegative dynamical systems with respect to linear supply rates. The following result presents Kalman-Yakubovich-Popov conditions for discrete-time nonnegative dynamical systems with linear supply rates of the form s(u, y) = q T y + r T u, where q ∈ Rl , q 6= 0, and r ∈ Rm , r 6= 0. First, however, the following definition is required. Definition 5.9. A discrete-time nonnegative dynamical system G is zero-state observable if u(k) ≡ 0 and y(k) ≡ 0 implies x(k) ≡ 0. G is n completely reachable if for all x0 ∈ R+ , there exist a ki ≤ 0 and square summable nonnegative input u(k) defined on [ki , 0] such that the state x(k), k ≥ ki , can be driven from x(ki ) = 0 to x(0) = x0 . G is completely null n reachable if for all x0 ∈ R+ , there exist a kf ≥ 0 and square summable nonnegative input u(k) defined on [0, kf ] such that the state x(k), k ≥ 0, can be driven from x(0) = x0 to x(kf ) = 0. Theorem 5.12. Let q ∈ Rl and r ∈ Rm . Consider the nonlinear nonnegative dynamical system G given by (5.46) and (5.47) where f : D → n Rn is nonnegative, G(x) ≥≥ 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, x ∈ R+ . n n n m If there exist functions Vs : R+ → R+ , ℓ : R+ → R+ , W : R+ → R+ , P1u : Rn → R1×m , and a scalar ρ > 1 (respectively, ρ = 1) such that Vs (·) is continuous and nonnegative definite, Vs (0) = 0, Vs (f (x) + G(x)u) = Vs (f (x)) + P1u (x)u, n

n

m

x ∈ R+ ,

u ∈ R+ ,

(5.50)

and, for all x ∈ R+ , 0 = Vs (f (x)) − ρ1 Vs (x) − q T h(x) + ℓ(x),

0 = P1u (x) − q T J(x) − r T + W T (x),

(5.51) (5.52)

then G is geometrically dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + r T u. n

n

Proof. Suppose that there exist functions Vs : R+ → R+ , ℓ : R+ → n m R+ , W : R+ → R+ , P1u : Rn → R1×m , and a scalar ρ > 1 (respectively, ρ = 1), such that Vs (·) is continuous, Vs (0) = 0, and (5.50)–(5.52) are n satisfied. Then, with Vˆs (x) , 1ρ Vs (x), it follows that, for all x ∈ R+ and m u ∈ R+ , ρVˆs (f (x) + G(x)u) − Vˆs (x) = Vs (f (x) + G(x)u) − ρ1 Vs (x)

= Vs (f (x)) + P1u (x)u − ρ1 Vs (x)

NNBook

September 1, 2009

169

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

= q T h(x) − ℓ(x) + q T J(x)u + r T u − W T (x)u ≤ q T y + r T u, which implies that G is geometrically dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + r T u. As in standard dissipativity theory with quadratic supply rates [112, 139], the concepts of linear supply rates and linear storage functions provide a generalized mass and energy balance interpretation. Specifically, using (5.50)–(5.52) with ρ = 1, it follows that k−1 X

κ=k0

[q T y(κ) + r T u(κ)] = Vs (x(k)) − Vs (x(k0 )) +

k−1 X

κ=k0

[ℓT (x(κ))x(κ) + W T (x(κ))u(κ)], (5.53)

which can be interpreted as a generalized mass balance equation where Vs (x(k)) − Vs (x(k0 )) is the stored mass of the discrete-time nonnegative system and the sum on the right corresponds to the expelled mass of the nonnegative system. Rewriting (5.53) as ∆Vs (x) = Vs (f (x)) − Vs (x) = q T y + r T u − [ℓT (x)x + W T (x)u]

(5.54)

yields a mass conservation equation which shows that the system mass transport is equal to the supplied system mass minus the expelled system mass. As noted in Section 5.5, if G is zero-state observable and there exists a function κ : Rl → Rm such that s(κ(y), y) < 0, y 6= 0, then the storage function Vs (·) satisfies Vs (x) > 0, x ∈ Rn , x 6= 0 [112]. Similarly, for the discrete-time nonnegative dynamical system G, it can be shown that if G is l m zero-state observable and there exists a function κ : R+ → R+ such that l

n

s(κ(y), y) < 0, y ∈ R+ , y 6= 0, then Vs (x) > 0, x ∈ R+ , x 6= 0. See Theorem 13.18 of [112] for details. Hence, in the case of a linear supply rate, there always exists a matrix K ∈ Rm×l such that q + K T r 0, x ∈ R+ , x 6= 0.

If a discrete-time nonnegative dynamical system G is zero-state observable and dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and if q ≤≤ 0 and u ≡ 0, then it follows that ∆Vs (x(k)) ≤ q T y(k) ≤ 0, k ∈ Z+ . Hence, the undisturbed (u(k) ≡ 0) system G is Lyapunov stable. Furthermore, if a discrete-time nonnegative dynamical system G is zero-state observable and geometrically dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and if q ≤≤ 0 and u ≡ 0, then

NNBook

September 1, 2009

170

CHAPTER 5

1 T it follows that ∆Vs (x(k)) ≤ − ρ−1 ρ Vs (x(k)) + ρ q y(k), k ∈ Z+ and ρ > 1. Hence, the undisturbed (u(k) ≡ 0) system G is asymptotically stable.

Next, we provide necessary and sufficient conditions for the case where G given by (5.46) and (5.47) is lossless with respect to the linear supply rate s(u, y) = q T y + r T u. Theorem 5.13. Let q ∈ Rl and r ∈ Rm . Consider the nonlinear nonnegative dynamical system G given by (5.46) and (5.47) where f : D → n Rn is nonnegative, G(x) ≥≥ 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, x ∈ R+ . Then m G is lossless with respect to the supply rate s(u, y) = q T y + r T u , u ∈ R+ , if n and only if there exist functions Vs : R+ → R+ and P1u : Rn → R1×m such n that Vs (·) is continuous, Vs (0) = 0, and, for all x ∈ R+ , (5.50) holds and 0 = Vs (f (x)) − Vs (x) − q T h(x), 0 = P1u (x) − q T J(x) − r T .

(5.55) (5.56)

If, in addition, Vs (·) is continuously differentiable, then P1u (x) = Vs′ (f (x))G(x).

(5.57)

Proof. Sufficiency follows as in the proof of Theorem 5.12. To show necessity, suppose that G is lossless with respect to the linear supply rate s(u, y) = q T y + r T u. Then, it follows that there exists a continuous function n Vs : R+ → R+ such that Vs (f (x) + G(x)u) = Vs (x) + s(u, y) = Vs (x) + q T y + r T u n m = Vs (x) + q T h(x) + (q T J(x) + r T )u, x ∈ R+ , u ∈ R+ . (5.58) Since the right-hand side of (5.58) is linear in u it follows that Vs (f (x) + G(x)u) is linear in u, and hence, there exists P1u : Rn → R1×m such that Vs (f (x) + G(x)u) = Vs (f (x)) + P1u (x)u,

n

x ∈ R+ ,

m

u ∈ R+ . (5.59)

Now, using (5.59) and equating coefficients of equal powers in (5.58) yields (5.55) and (5.56). Finally, if Vs (·) is continuously differentiable, application of a Taylor series expansion on (5.59) about u = 0 yields ∂Vs (f (x) + G(x)u) = Vs′ (f (x))G(x). P1u (x) = (5.60) ∂u u=0 This completes the proof.

NNBook

September 1, 2009

171

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

Next, we introduce the notion of nonaccumulativity for discrete-time nonnegative dynamical systems. Definition 5.10. A nonnegative dynamical system G of the form (5.46) and (5.47) is nonaccumulative (respectively, geometrically nonaccumulative) if G is dissipative (respectively, geometrically dissipative) with respect to the supply rate s(u, y) = eT u − eT y. If G is nonaccumulative, then it follows from (5.49) that ∆Vs (x(k)) ≤ eT u(k) − eT y(k),

k ∈ Z+ ,

(5.61)

where u(·) ∈ U+ and y ∈ Y+ . If the components ui (·), i = 1, . . . , m, of u(·) denote mass inputs to the system G and the components yi (·), i = 1, . . . , l, of y(·) denote the mass outputs of the system G, then dissipativity with respect to the linear supply rate s(u, y) = eT u− eT y implies that the change in system mass is always less than or equal to the difference between the system mass input and system mass output. Next, we show that all discrete-time compartmental systems with measured outputs corresponding to material outflows are nonaccumulative. Specifically, consider (2.109) with w(k) = u(k) or, equivalently,  T ∂Vs (x(k)) + u(k), x(0) = x0 , k ∈ Z+ , ∆x(k) = [Jn (x(k))− D(x(k))] ∂x (5.62) and with storage function Vs (x) = eT x and output   ∂Vs T = [a11 (x)x1 , a22 (x)x2 , . . . , ann (x)xn ]T . y = D(x) ∂x Now, it follows that "

∆Vs (x) = eT [Jn (x) − D(x)]



∂Vs ∂x

= eT u − eT y + eT Jn (x)e n = eT u − eT y, x ∈ R+ ,

T

+u

# (5.63)

which shows that all discrete-time compartmental systems with outputs y =
T s are lossless with respect to the supply rate s(u, y) = eT u − eT y. D(x) ∂V ∂x

If, alternatively, the outputs y correspond to a partial observation of the material outflows, then it follows that the discrete-time compartmental system is dissipative with respect to the supply rate s(u, y) = eT u − eT y. To see this, assume without loss of generality that the first l outflows are

NNBook

September 1, 2009

172

CHAPTER 5

observed, that is, y = [a11 (x)x1 , a22 (x)x2 , . . . , all (x)xl ]. Now, note that     ∂Vs T ∂Vs T − Dr (x) , y = D(x) ∂x ∂x n

where Dr (x) = diag[0, . . . , 0, al+1l+1 (x)xl+1 , . . . , ann (x)xn ] ≥≥ 0, x ∈ R+ . Hence, # "   ∂Vs T T +u ∆Vs (x) = e [Jn (x) − D(x)] ∂x     ∂Vs ∂Vs T T T = e u−e y− Dr (x) ∂x ∂x ≤ eT u − eT y,

n

x ∈ R+ .

(5.64) n

Note that, in the case where the system is closed, ∆Vs (x) = 0, x ∈ R+ , which corresponds to conservation of mass in the system. Finally, we present a key result on linearization of discrete-time nonnegative dissipative dynamical systems. For this result we assume that the storage function Vs (·) is two times continuously differentiable. Theorem 5.14. Let q ∈ Rl and r ∈ Rm , and assume G given by (5.46) and (5.47) is such that f : D → Rn is nonnegative, G(x) ≥≥ 0, n h(x) ≥≥ 0, and J(x) ≥≥ 0, x ∈ R+ . Suppose G is geometrically dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + n n m r T u. Then, there exist p ∈ R+ , l ∈ R+ , and w ∈ R+ and a scalar ρ > 1 (respectively, ρ = 1) such that 0 = AT p − 1ρ p − C T q + l,

(5.65)

0 = B p − D q − r + w,

(5.66)

T

T

where ∂f , A= ∂x x=0

B = G(0),

∂h , C= ∂x x=0

D = J(0).

(5.67)

If, in addition, (A, C) is observable, then p >> 0.

Proof. Assume that G is geometrically dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + r T u. Then, it follows that there exist a continuous function Vs : Rn+ → R+ and a scalar ρ > 1 (respectively, ρ = 1) such that ρVs (f (x) + G(x)u) − Vs (x) ≤ q T y + r T u,

n

x ∈ R+ ,

m

u ∈ R+ . (5.68) n

m

Now, it follows from (5.68) that there exists a function d : R+ × R+ → R+

NNBook

September 1, 2009

173

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

such that d(x, u) ≥ 0, d(0, 0) = 0, and

0 = ρVs (f (x) + G(x)u) − Vs (x) − q T y − r T u + d(x, u), n m x ∈ R+ , u ∈ R+ .

(5.69)

Next, expanding Vs (·) and d(·, ·) via a Taylor series expansion about x = 0, u = 0, and using the fact that Vs (·) and d(·, ·) are nonnegative definite n n and Vs (0) = 0, d(0, 0) = 0, it follows that there exist p ∈ R+ , l ∈ R+ , and m w ∈ R+ such that Vs (x) =

1 T ρp x T

+ Vsr (x),

(5.70)

T

d(x, u) = l x + w u + dsr (x, u), n R+

→ R+ and dsr : where Vsr : of Vs (·) and d(·, ·), respectively.

n R+

×

m R+

(5.71)

→ R+ contain higher-order terms

Next, let f (x) = Ax + fr (x) and h(x) = Cx + hr (x), where fr (x) and hr (x) contain the nonlinear terms of f (x) and h(x), respectively, and let G(x) = B + Gr (x) and J(x) = D + Jr (x), where Gr (x) and Jr (x) contain the nonconstant terms of G(x) and J(x), respectively. Using the above expressions, (5.69) can be written as 0 = pT Ax + pT Bu − ρ1 pT x − q T (Cx + Du) − r T u + lT x + wT u + δ(x, u),

(5.72)

where δ(x, u) is such that δ(x, u)/(kxk + kuk) → 0 as kxk + kuk → 0. Now, setting u = 0 in (5.72) and equating coefficients of equal powers yields (5.65). Alternatively, setting x = 0 in (5.72) and equating coefficients of equal powers yields (5.66). Finally, to show that p >> 0 in the case where (A, C) is observable, note that it follows from Theorem 5.12 that the linearized system G with storage function Vs (x) = pT x is geometrically dissipative (respectively, dissipative) with respect to the linear supply rate s(u, y) = q T y + r T u. Now, it follows from Theorem 13.18 of [112] that p >> 0.

5.7 Specialization to Discrete-Time Linear Nonnegative Dynamical Systems In this section, we specialize the results of Section 5.6 to linear discrete-time nonnegative dynamical systems. Specifically, setting f (x) = Ax, G(x) = B, h(x) = Cx, and J(x) = D, the nonlinear nonnegative dynamical system given by (5.46) and (5.47) specializes to x(k + 1) = Ax(k) + Bu(k),

x(0) = x0 ,

k ∈ Z+ ,

(5.73)

NNBook

September 1, 2009

174

CHAPTER 5

y(k) = Cx(k) + Du(k),

(5.74)

where x ∈ Rn , u ∈ Rm , y ∈ Rl , A ∈ Rn×n , B ∈ Rn×m , C ∈ Rl×n , and D ∈ Rl×m . Before providing linear dissipativity specializations, we present several results on linear nonnegative systems in the case where u(k) ≥≥ 0 and y(k) ≥≥ 0, k ∈ Z+ . Theorem 5.15. The linear dynamical system G given by (5.73) and (5.74) is input-output nonnegative if and only if D ≥≥ 0 and the impulse response matrix function H(k) = CAk B, k ∈ Z+ , of G is nonnegative. Proof. Consider the dynamical system G given by (5.73) and (5.74) with x(0) = 0. Then, it follows that y(k) =

k−1 X i=0

H(k − 1 − i)u(i) + Du(k),

k ∈ Z+ .

Now, suppose the impulse response function H(k) ≥≥ 0, k ∈ Z+ , and D ≥≥ 0. Then, for every input u(k) ≥≥ 0, k ∈ Z+ , the output is also nonnegative. Conversely, suppose G is input-output nonnegative and assume, ad absurdum, there exist i, j, with i 6= j, such that H(i,j) (κ) < 0 for some κ ∈ Z+ . Now, for every u(k), k ∈ Z+ , such that u(k) = 0, k 6= κ − 1, and u(κ − 1) = u ˆej , where u ˆ > 0, it follows that y(2κ) =

2κ−1 X i=0

H(i,j)(2κ − 1 − i)u(i) 0, u(k) = 0, k ∈ Z+ , k > 0. In this case, since m x(1) = Bu(0) ≥≥ 0 for all u(0) ∈ R+ , it follows that B ≥≥ 0, which proves the result. The following result presents necessary and sufficient Kalman-Yakubovich-Popov conditions for linear discrete-time nonnegative dynamical systems with linear supply rates of the form s(u, y) = q T y + r T u, where q ∈ Rl , q 6= 0, and r ∈ Rm , r 6= 0. Note that, as in the continuoustime case, for a discrete-time linear dynamical system to be dissipative with respect to a linear supply rate it is necessary that the storage function is also linear. However, since all storage functions are nonnegative by definition, it follows that a storage function is nonnegative if and only if there exists a linear transformation such that the discrete-time linear dynamical system is nonnegative in a transformed basis. Hence, dissipativity theory of discretetime linear dynamical systems with respect to linear supply rates is complete if we restrict our consideration to the class of discrete-time nonnegative dynamical systems. Theorem 5.17. Let q ∈ Rl and r ∈ Rm . Consider the nonnegative dynamical system G given by (5.73) and (5.74) where A ≥≥ 0, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. Then G is geometrically dissipative (respectively, dissipative) with respect to the supply rate s(u, y) = q T y + r T u if and only n n m if there exist p ∈ R+ , l ∈ R+ , and w ∈ R+ , and a scalar ρ > 1 (respectively, ρ = 1) such that 0 = AT p − ρ1 p − C T q + l,

0 = B T p − D T q − r + w.

(5.75) (5.76)

Proof. Sufficiency follows from Theorem 5.12 with f (x) = Ax, G(x) = B, h(x) = Cx, J(x) = D, and Vs (x) = pT x. To show necessity, note that, if the linear nonnegative dynamical system (5.73) and (5.74) is dissipative with respect to the linear supply rate s(u, y) = q T y + r T u, then it follows from Theorem 5.14, with f (x) = Ax, G(x) = B, h(x) = Cx, and n n m J(x) = D, that there exist p ∈ R+ , l ∈ R+ , and w ∈ R+ such that (5.75) and (5.76) are satisfied. For a given l ∈ Rn and w ∈ Rm , note that there exists p ∈ Rn such that (5.75) and (5.76) are satisfied if and only if rank[M, y] = rank M ,

NNBook

September 1, 2009

176

CHAPTER 5

where △

M=



(A − ρ1 In )T BT



,

△

y=



C Tq − l T D q+r−w



.

Now, there exist p ≥≥ 0, l ≥≥ 0, and w ≥≥ 0 such that (5.75) and (5.76) are satisfied if and only if the inequalities p ≥≥ 0, z − M p ≥≥ 0

(5.77) (5.78)

are satisfied, where △

z=



C Tq T D q+r



.

Equations (5.77) and (5.78) comprise a set of 2n + m linear inequalities with pi , i = 1, . . . , n, variables, and hence, the feasibility of p ≥≥ 0 such that (5.77) and (5.78) hold can be checked by standard linear matrix inequality (LMI) techniques [39]. Finally, we provide necessary and sufficient conditions for the case where G given by (5.73) and (5.74) is lossless with respect to the linear supply rate s(u, y) = q T y + r T u. Theorem 5.18. Let q ∈ Rl and r ∈ Rm . Consider the nonnegative dynamical system G given by (5.73) and (5.74) where A ≥≥ 0, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. Then G is lossless with respect to the supply rate n s(u, y) = q T y + r T u if and only if there exists p ∈ R+ such that 0 = AT p − p − C T q, 0 = B T p − D T q − r.

(5.79) (5.80)

Proof. The proof follows from Theorems 5.13 and 5.14 with f (x) = Ax, G(x) = B, h(x) = Cx, J(x) = D, and Vs (x) = pT x. As in the case of nonlinear compartmental systems, it is important to note that all linear compartmental systems with measured outputs corresponding to material outflows are lossless with respect to the supply rate s(u, y) = eT u − eT y. If, alternatively, the outputs y correspond to a partial observation of the material outflows, then the linear compartmental system is nonaccumulative. Example 5.4. In this example we consider the thyroxine model studied in Example 2.11. Specifically, we show that the discretized system is nonaccumulative. Here, we assume u1 (k) is an arbitrary discrete input and the bile is discharged into the duodenum so that y(k) = Cx(k), where C = [0, 0, a33 ] and a33 > 0. In this case, using the storage function

NNBook

September 1, 2009

177

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

Vs (x1 , x2 , x3 ) = x1 + x2 + x3 , it follows that ∆Vs (x1 , x2 , x3 ) ≤ u1 − y and, hence, the discretized system is nonaccumulative. △ Example 5.5. In this example, we explore dissipativity notions for the Leslie model considered in Example 2.12. To do this, we specify appropriate inputs and outputs for this model so that the system is given by (5.73) and n (5.74). Here, we assume u ∈ R+ so that the ith component of the vector Bu(k) in (5.73) represents an exogenous input or immigration to the ith age class. Furthermore, we set D = 0 and choose C to capture the number of retired and/or deceased individuals in the population. n

n

Now, it follows from Theorem 5.17 that if there exist p ∈ R+ , l ∈ R+ , n and w ∈ R+ such that 0 = AT p − p + C T e + l, 0 = B T p − e + w,

(5.81) (5.82)

then the Leslie model, with the given inputs and outputs and appropriate choice of immigration input matrix B, is nonaccumulative. That is, the change of population is always less than or equal to the difference between the immigration population and the retired and/or deceased population. Thus, with knowledge of the parameters in the system matrix A for a given population, immigration can be regulated to enforce nonaccumulativity or accumulativity, that is, ∆Vs (x) ≥ eT u − eT y, by regulating the immigration input vector Bu(k). △

5.8 Feedback Interconnections of Discrete-Time Nonnegative Dynamical Systems In this section, we consider feedback interconnections of nonnegative dynamical systems. We begin by considering the nonlinear nonnegative dynamical system G given by (5.46) and (5.47) with the nonlinear nonnegative dynamical feedback system Gc given by xc (k + 1) = fc (xc (k)) + Gc (xc (k))uc (k), yc (k) = hc (xc (k)),

xc (0) = xc0 ,

k ∈ Z+ , (5.83) (5.84)

where fc : Rnc → Rnc , Gc : Rnc → Rnc ×mc , hc : Rnc → Rlc , fc (xc ) ≥≥ 0, nc Gc (xc ) ≥≥ 0, and hc (xc ) ≥≥ 0, xc ∈ R+ . Theorem 5.19. Let q ∈ Rl , r ∈ Rm , qc ∈ Rlc , and rc ∈ Rmc . Consider the nonlinear nonnegative dynamical systems G and Gc given by (5.46) and (5.47), and (5.83) and (5.84), respectively, with uc = y and yc = u. Assume that G is dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and with a positive-definite storage function Vs (·), and assume that

NNBook

September 1, 2009

178

CHAPTER 5

Gc is dissipative with respect to the linear supply rate sc (uc , yc ) = qcT yc + rcT uc and with a positive-definite storage function Vsc (·). Then the following statements hold: i ) If there exists a scalar σ > 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is Lyapunov stable. ii ) If G and Gc are zero-state observable and there exists a scalar σ > 0 such that q + σrc 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is asymptotically stable. Proof. Note that the positive feedback interconnection of G and Gc is given by u = yc and uc = y so that the closed-loop dynamics of G and Gc is given by   x(k + 1) xc (k + 1)   f (x(k)) + G(x(k))hc (xc (k)) , = fc (xc (k)) + Gc (xc (k))h(x(k)) + Gc (xc (k))J(x(k))hc (xc (k)) which implies that f˜(˜ x) ,



f (x) + G(x)hc (xc ) fc (xc ) + Gc (xc )h(x) + Gc (xc )J(x)hc (xc )



,

T where x ˜ = [xT , xT c ] , is nonnegative. Hence, the closed-loop system is also nonnegative and thus x(k) ≥≥ 0, xc (k) ≥≥ 0, u(k) ≥≥ 0, and y(k) ≥≥ 0, k ∈ Z+ .

i) Consider the Lyapunov function candidate V (x, xc ) = Vs (x) + σVsc (xc ) and note that ∆V (x, xc ) ≤ q T y + r T u + σ(qcT yc + rcT uc ) = (q + σrc )T y + (r + σqc )T u

NNBook

September 1, 2009

179

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

≤ 0

(5.85)

for all y ≥≥ 0 and u ≥≥ 0, establishing Lyapunov stability. ii) With V (x, xc ) = Vs (x) + σVsc (xc ), Lyapunov stability follows as in i). Furthermore, if q + σrc 0 and u >> 0 such that either y 6= 0 or u 6= 0. The result now follows by applying the Krasovskii-LaSalle theorem and using the observability assumptions. iii) With V (x, xc ) = Vs (x) + σρc Vsc (xc ), Lyapunov stability follows as in i). Next, if Gc is geometrically dissipative it follows that ∆V (x, xc ) ≤ nc n nc △ −σ(ρc − 1)Vsc (xc ), xc ∈ R+ . Now, define R = {(x, xc ) ∈ R+ × R+ : ∆V (x, xc ) = 0}. Since ∆V (x(k), xc (k)) ≤ −σ(ρc − 1)Vsc (xc ), it follows that xc (k) ≡ 0 and u(k) = yc (k) = 0 and, since rank Gc (xc ) = mc , it follows that uc (k) = y(k) = 0. Now, since G is zero-state observable, it follows that x(k) ≡ 0. Hence, the largest invariant set contained in R is given by M = {(0, 0)}, and hence, it follows from the Krasovskii-LaSalle theorem that (x(k), xc (k)) → (0, 0) as k → ∞. iv) With V (x, xc ) = ρVs (x) + σρc Vsc (xc ), Lyapunov stability follows as in i). Next, if G and Gc are geometrically dissipative, it follows that ∆V (x, xc ) ≤ −(ρ − 1)Vs (x) − σ(ρc − 1)Vsc (xc ) ≤ − min{(ρ − 1), σ(ρc − 1)}V (x, xc ),

n

nc

(x, xc ) ∈ R+ × R+ ,

which establishes asymptotic stability. As in the continuous-time case, it is important to note that Theorem 5.19 also holds for the more general architecture of the feedback system Gc wherein yc (k) = hc (xc (k)) + Jc (xc (k))uc (k), where Jc : Rnc → Rlc ×mc nc and Jc (xc ) ≥≥ 0, xc ∈ R+ . In this case, however, we assume that the positive feedback interconnection of G and Gc is well posed, that is, det[Im − nc n Jc (xc )J(x)] 6= 0 for all xc ∈ R+ , x ∈ R+ . Similar remarks hold for the rest of the results in this section. The following corollary to Theorem 5.19 addresses linear supply rates of the form s(u, y) = eT u − eT y. Corollary 5.3. Consider the nonlinear nonnegative dynamical systems G and Gc given by (5.46) and (5.47), and (5.83) and (5.84), respectively. Assume that G is nonaccumulative with a positive-definite storage function Vs (·), and assume that Gc is geometrically nonaccumulative with a positivedefinite storage function Vsc (·). Then the following statements hold:

NNBook

September 1, 2009

180

CHAPTER 5

i ) If G is zero-state observable and rank Gc (0) = mc , then the positive feedback interconnection of G and Gc is asymptotically stable. ii ) If G is geometrically nonaccumulative, then the positive feedback interconnection of G and Gc is asymptotically stable. Proof. The result is a direct consequence of iii) and iv) of Theorem 5.19 with σ = 1, q = −rc = −e, and r = −qc = e. Next, we develop absolute stability criteria for discrete-time linear nonnegative feedback systems with nonnegative memoryless input nonlinearities. Specifically, given the discrete-time nonnegative system G characterized by (5.73) and (5.74) we derive sufficient conditions that guarantee asymptotic stability of the feedback interconnection involving the discrete-time linear nonnegative system G and the feedback nonnegative input nonlinearity σ(·, ·) ∈ Φ, where △

l

m

Φ = {σ : Z+ × R+ → R+ : σ(·, 0) = 0, 0 ≤≤ σ(k, y) ≤≤ M y, l

y ∈ R+ , k ∈ Z+ },

(5.86)

M >> 0, and M ∈ Rm×l . Theorem 5.20. Consider the nonnegative dynamical system G given by (5.73) and (5.74), and assume that (A, C) is observable and G is geometrically dissipative with respect to the supply rate s(u, y) = eT u − eT M y, where M >> 0. Then, the positive feedback interconnection of G and σ(·, ·) is globally (uniformly) asymptotically stable for all σ(·, ·) ∈ Φ. l

Proof. Since σ(k, y) ≥≥ 0 for all k ∈ Z+ , y ∈ R+ , and (5.73) and (5.74) is a discrete-time nonnegative dynamical system, it follows that the positive feedback interconnection of G and σ(·, ·) given by x(k + 1) = Ax(k) + Bσ(k, y(k)),

x(0) = x0 ,

k ∈ Z+ ,

is a nonnegative dynamical system for all σ(·, ·) ∈ Φ. Next, since (A, C) is observable and G is geometrically dissipative with respect to the supply rate s(u, y) = eT u − eT M y, it follows from Theorem 13.18 of [112] and Theorem n 5.17, with r = e and q = −M T e, that there exist p ∈ Rn+ , l ∈ R+ , and m w ∈ R+ , and a scalar ρ > 1 such that 0 = AT p − ρ1 p + C T M T e + l, 0 = B T p + D T M T e − e + w.

(5.87) (5.88)

Next, consider the Lyapunov function candidate Vs (x) = pT x and note

NNBook

September 1, 2009

181

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS n

that for x ∈ R+ , x 6= 0, ∆Vs (x) = pT (Ax + Bσ) − pT x + 1ρ pT x − 1ρ pT x ρ−1 T ρ p x Bσ) − 1ρ pT x T T

= pT (Ax + Bσ) − < pT (Ax +

− 1ρ pT x

= eT [σ − M y] − l x − w σ ≤ eT [σ − M y].

n

Now, since σ ≤≤ M y for all σ(·, ·) ∈ Φ, it follows that ∆Vs (x) < 0, x ∈ R+ , x 6= 0. Hence, the positive feedback interconnection of G and σ(·, ·) is globally (uniformly) asymptotically stable for all σ(·, ·) ∈ Φ. To develop robust stability results for discrete-time linear nonnegative dynamical systems consider the set F defined by m×l

△

F = {F : Z+ → R+

: 0 ≤≤ F (k) ≤≤ M,

k ∈ Z+ }.

(5.89)

That is, F includes σ in Φ of the form σ(k, y) = F (k)y. Furthermore, consider the uncertain system x(k + 1) = (A + ∆A(k))x(k),

x(0) = x0 ,

k ∈ Z+ ,

(5.90)

where A is a nominal nonnegative, asymptotically stable matrix and ∆A(·) ∈ U , where U is the uncertainty set △

U = {∆A(·) : ∆A(k) = BF (k)C, F (·) ∈ F}.

(5.91)

In this case, it follows from Theorem 5.20 with D = 0 that the zero solution x(k) ≡ 0 to the linear nonnegative uncertain system (5.90) is globally (uniformly) asymptotically stable for all F (·) ∈ F. The uncertain dynamical system model given by (5.90) considers an additive (absolute) uncertainty structure. Next, we show that discrete-time linear nonnegative dynamical systems are also robust with respect to a multiplicative (relative) uncertainty characterization. Specifically, consider the incremental uncertain system ∆x(k) = F (A − I)Gx(k),

x(0) = x0 ,

k ∈ Z+ ,

(5.92)

or, equivalently, x(k + 1) = [I − F G + F AG]x(k),

x(0) = x0 ,

k ∈ Z+ ,

(5.93)

where A ∈ Rn×n is nonnegative and F ∈ Rn×n , G ∈ Rn×n are positive and diagonal such that (5.93) remains nonnegative, that is, I − F G + F AG ≥ 0, and (5.92) is asymptotically stable with F = G = In , that is, the nominal system is asymptotically stable.

NNBook

September 1, 2009

182

CHAPTER 5

Since the nominal system is asymptotically stable, it follows from Theorem 2.23 that there exist a positive diagonal matrix P ∈ Rn×n and a positive-definite matrix R ∈ Rn×n such that P = AT P A + R.

(5.94)

Now, choosing the Lyapunov function candidate V (x) = xT GP F −1 x for the uncertain system (5.92), it follows after some algebraic manipulations that n ∆V (x) < 0, x ∈ R+ , x 6= 0, which proves asymptotic stability. Alternatively, it follows from Theorem 2.22 that there exist p ∈ Rn+ and r ∈ Rn+ such that p = AT p + r.

(5.95)

Now, with the Lyapunov function candidate V (x) = xT F −1 p for the n uncertain system (5.92), it follows that ∆V (x) = −xT Gr < 0, x ∈ R+ , x 6= 0, which proves asymptotic stability for the perturbed system (5.92). Theorem 5.21. Consider the nonlinear nonnegative dynamical system G given by (5.46) and (5.47), where f : D → Rn is nonnegative, G(x) ≥≥ n 0, h(x) ≥≥ 0, and J(x) ≥≥ 0, x ∈ R+ . Suppose there exist functions n n 1×m Vs : R+ → R+ and P1u : R+ → R+ , such that Vs (·) is positive definite, Vs (0) = 0, Vs (x) → ∞ as kxk → ∞, and 0 = Vs (f (x)) − 1ρ Vs (x) − eT M h(x) + ℓ(x), 0 = P1u (x) − eT M J(x) − eT + W T (x).

(5.96) (5.97)

Then, the positive feedback interconnection of G and σ(·, ·) is globally (uniformly) asymptotically stable for all σ(·, ·) ∈ Φ. Proof. The proof is similar to that of Theorem 5.20 and, hence, is omitted. Finally, we present a straightforward but key property of cascade interconnections of nonnegative dynamical systems. Proposition 5.4. Consider the discrete-time nonnegative dynamical systems G1 and G2 with input-output pairs (u1 , y1 ) and (u2 , y2 ), respectively. Assume G1 and G2 are nonaccumulative. Then the cascade interconnection G = G2 G1 with input-output pair (u, y) = (u1 , y2 ) is nonaccumulative. Proof. The result is a direct consequence of Definition 5.10 by noting the interconnection constraint for a cascade interconnection is given by u = u1 and y = y2 . As in the continuous-time case, it is important to note that a similar result to Proposition 5.4 does not hold for parallel and feedback interconnections.

NNBook

September 1, 2009

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

183

5.9 Dissipativity Theory for Nonnegative Dynamical Systems with Time Delay Exploiting the input, state, and output properties related to conservation, dissipation, and transport of mass and energy in nonnegative and compartmental dynamical systems, in the remainder of this chapter we develop dissipativity theory for nonnegative dynamical systems with time delay. Specifically, using linear storage functionals with linear supply rates, we develop sufficient conditions for dissipativity of nonnegative dynamical systems with time delay. The motivation for using linear storage functionals and linear supply rates follows from the fact that the (infinite-dimensional) state as well as the inputs and outputs of retarded nonnegative dynamical systems are nonnegative. The consideration of linear storage functionals and linear supply rates leads to new Kalman-Yakubovich-Popov equations for characterizing dissipativity of nonnegative systems with time delay. For a time-delay compartmental system, a linear storage functional is shown to correspond to the total mass of the system at a given time plus the integral of the mass flow in transit between compartments over the time intervals it takes for the mass to flow through the intercompartmental connections. In this case, dissipativity implies that the total system mass transport is equal to the supplied system flux minus the expelled system flux. Finally, using the concepts of dissipativity for retarded nonnegative dynamical systems, we develop feedback interconnection stability results for nonnegative systems with time delay. In particular, general stability criteria are given for Lyapunov and asymptotic stability of feedback nonnegative dynamical systems with time delays. In this section, we consider linear time-delay dynamical systems G of the form x(t) ˙ = Ax(t) + Ad x(t − τ ) + Bu(t), x(θ) = φ(θ), −τ ≤ θ ≤ 0, t ≥ 0, (5.98) y(t) = Cx(t) + Du(t), (5.99) where x(t) ∈ Rn , u(t) ∈ Rm , y(t) ∈ Rl , t ≥ 0, A ∈ Rn×n , Ad ∈ Rn×n , B ∈ Rl×n , C ∈ Rl×n , D ∈ Rl×m , τ > 0, and φ(·) ∈ C = C([−τ, 0], Rn ) is a continuous vector-valued function specifying the initial state of the system. The following definition is needed for the main results of this section. Definition 5.11. The linear time-delay dynamical system G given by △ (5.98) is nonnegative if for every φ(·) ∈ C+ , where C+ = {ψ(·) ∈ C : ψ(θ) ≥≥ 0, θ ∈ [−τ, 0]}, and u(t) ≥≥ 0, t ≥ 0, the solution x(t), t ≥ 0, to (5.98) and the output y(t), t ≥ 0, are nonnegative.

NNBook

September 1, 2009

184

CHAPTER 5

Proposition 5.5. The linear time-delay dynamical system G given by (5.98) is nonnegative if and only if A ∈ Rn×n is essentially nonnegative and Ad ≥≥ 0, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. Proof. First, note that the solution to (5.98) is given by Z t At x(t) = e x(0) + eA(t−θ) [Ad x(θ − τ ) + Bu(θ)]dθ 0 Z t−τ Z t At A(t−τ −θ) = e φ(0) + e Ad x(θ)dθ + eA(t−θ) Bu(θ)dθ. (5.100) −τ

0

Now, if A is essentially nonnegative, then it follows from Proposition 2.5 that eAt ≥≥ 0, t ≥ 0; and if φ(·) ∈ C+ , Ad ≥≥ 0, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0, then it follows that Z t−τ Z t At A(t−τ −θ) x(t) = e φ(0) + e Ad x(θ)dθ + eA(t−θ) Bu(θ)dθ ≥≥ 0, −τ

0

t ∈ [0, τ ),

(5.101)

and y(t) ≥≥ 0 for all t ∈ [0, τ ). Alternatively, for all t > τ , Z t Z τ eA(t−θ) Bu(θ)dθ, x(t) = eAτ x(t − τ ) + eA(τ −θ) Ad x(t + θ − 2τ )dθ + 0

0

(5.102) and hence, since x(t) ≥≥ 0, t ∈ [−τ, τ ), it follows that x(t) ≥≥ 0, t ∈ [τ, 2τ ). Repeating this procedure iteratively, it follows that x(t) ≥≥ 0, t ≥ 0, and hence, y(t) ≥≥ 0, t ≥ 0, which implies that G is nonnegative.

Conversely, suppose G is nonnegative. Now, note that with u(0) = 0, n y(0) = Cx(0) and, since y(0) ≥≥ 0 for all x(0) ∈ R+ , it follows that C ≥≥ 0. Next, with x(0) = 0, y(0) = Du(0) and, since y(0) ≥≥ 0 for l all u(0) ∈ R+ , it follows that D ≥≥ 0. Now, let φ(θ) = 0, −τ ≤ θ ≤ 0, and let u(t) = δ(t − tˆ)ˆ u, t, tˆ ∈ [0, τ ), where δ(·) denotes the Dirac delta function m and u ˆ ≥≥ 0. In this case, since x(tˆ) = B u ˆ ≥≥ 0 for all u ˆ ∈ R+ it follows that B ≥≥ 0. Furthermore, with u(t) = 0, φ(θ) = 0, −τ ≤ θ ≤ 0, x(t) = eAt φ(0), t ∈ [0, τ ), and hence, it follows from Proposition 2.5 that if x(t) ≥≥ 0, t ≥ 0, n for all φ(0) ∈ R+ , then A is essentially nonnegative. Finally, suppose, ad absurdum, that Ad is not nonnegative, that is, there exist I, J ∈ {1, 2, . . . ., n} such that Ad(I,J) < 0. Let u(t) = 0, t ≥ 0, and let {vn }∞ n=1 ⊂ C+ denote a sequence of functions such that limn→∞ vn (θ) = eJ δ(θ + η − τ ), where 0 < η < τ and δ(·) denotes the Dirac delta function. In this case, it follows from (5.100) that Z η Aη xn (t) = e vn (0) + eA(η−θ) Ad x(θ − τ )dθ, (5.103) 0

which implies that x(η) = limn→∞ xn (η) = eAη Ad eJ . Now, by choosing η

NNBook

September 1, 2009

185

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

sufficiently small it follows that xI (η) < 0, which is a contradiction. For the remainder of this section, we assume that A is essentially nonnegative and Ad , B, C, and D, are nonnegative, so that the linear timedelay dynamical system G given by (5.98) and (5.99) is nonnegative. Next, we present sufficient conditions for dissipativity of the linear time-delay nonnegative dynamical system (5.98) and (5.99). The following definition is needed for the next result. Definition 5.12. The nonnegative dynamical system (5.98) and (5.99) m l is dissipative with respect to the supply rate s : R+ × R+ → R if there exists a continuous, nonnegative-definite functional Vs : C+ → R, called a storage functional, such that the dissipation inequality Z t Vs (xt ) ≤ Vs (xt1 ) + s(u(σ), y(σ))dσ, (5.104) t1

is satisfied for all t1 , t ≥ 0, where x(t), t ≥ 0, is the solution to (5.98) with φ(·) ∈ C+ and u(·) ∈ U+ . The nonnegative dynamical system (5.98) and m l (5.99) is strictly dissipative with respect to the supply rate s : R+ × R+ → R if the dissipation inequality (5.104) is strictly satisfied. Theorem 5.22. Let q ∈ Rl and r ∈ Rm . Consider the nonnegative dynamical system G given by (5.98) and (5.99) where A ∈ Rn×n is essentially nonnegative, Ad ≥≥ 0, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0. If there exist n n m p ∈ R+ , l ∈ R+ (respectively, l ∈ Rn+ ), and w ∈ R+ such that 0 = (A + Ad )T p − C T q + l, 0 = B T p − D T q − r + w,

(5.105) (5.106)

then G is dissipative (respectively, strictly dissipative) with respect to the supply rate s(u, y) = q T y + r T u. n

n

m

Proof. Suppose that there exist p ∈ R+ , l ∈ R+ , and w ∈ R+ such that (5.105) and (5.106) hold. Then, with storage functional Z 0 T Vs (ψ) = p ψ(0) + pT Ad ψ(θ)dθ, ψ(·) ∈ C+ , (5.107) −τ

m

it follows that, for all xt ∈ C+ and u ∈ R+ , V˙ s (xt ) = = = = ≤

pT x(t) ˙ + pT Ad [x(t) − x(t − τ )] pT [Ax(t) + Ad x(t − τ ) + Bu(t)] + pT Ad x(t) − pT Ad x(t − τ ) pT (A + Ad )x(t) + pT Bu(t) q T Cx(t) − lT x(t) + q T Du(t) + r T u(t) − wT u(t) (5.108) q T y(t) + r T u(t),

NNBook

September 1, 2009

186

CHAPTER 5

which implies that G is dissipative with respect to the supply rate s(u, y) = q T y + r T u. Finally, in the case where l ∈ Rn+ , the inequality in (5.108) is strict, and hence, G is strictly dissipative with respect to the supply rate s(u, y) = q T y + r T u. The result presented in Theorem 5.22 can be easily extended to systems with multiple and distributed state delays. Specifically, consider the multiple pointwise delay system G given by x(t) ˙ = Ax(t) +

nd X i=1

Adi x(t − τi ) + Bu(t),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

y(t) = Cx(t) + Du(t),

t ≥ 0,

(5.109) (5.110)

where x(t) ∈ Rn , t ≥ 0, A ∈ Rn×n is essentially nonnegative, Adi ∈ Rn×n , △ i = 1, . . . , nd , is nonnegative, τ = maxi∈{1,...,nd } τi , and φ(·) ∈ C + = {ψ(·) ∈ C([−τ , 0], Rn ) : ψ(θ) ≥≥ 0, θ ∈ [−τ , 0]}. In this case, with (5.105) replaced by !T nd X 0= A+ Adi p − C Tq + l (5.111) i=1

and storage functional T

Vs (ψ) = p ψ(0) +

nd Z X i=1

0 −τi

pT Adi ψ(θ)dθ,

ψ(·) ∈ C + ,

(5.112)

it can be shown using a similar construction as in the proof of Theorem 5.22 that G given by (5.109) and (5.110) is dissipative with respect to the supply rate s(u, y) = q T y + r T u. Alternatively, for pure distributed delay systems of the form Z 0 x(t) ˙ = Ax(t) + Ad x(t + σ)dσ + Bu(t), x(θ) = φ(θ), −τ

y(t) = Cx(t) + Du(t),

−τ ≤ θ ≤ 0,

t ≥ 0,

(5.113) (5.114)

it can also be shown, with (5.105) replaced by 0 = (A + τ Ad )T p − C T q + l

(5.115)

and storage functional T

Vs (ψ) = p ψ(0) +

Z

0 −τ

Z

σ

0

pT Ad ψ(θ)dθdσ,

ψ(·) ∈ C+ ,

(5.116)

that the system (5.113) and (5.114) is dissipative with respect to the supply rate s(u, y) = q T y + r T u.

NNBook

September 1, 2009

187

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

ui (t)

uj (t)

' ?

$ )

&

%

ith Subsystem xi (t)

'

aij xj (t), τ

$

?

jth Subsystem xj (t)

aji xi (t), τ

1&

aii xi (t)

%

ajj xj (t)

?

?

Figure 5.1 Linear compartmental interconnected subsystem model with time delay.

Next, we show that linear compartmental dynamical systems with time delays are a special case of the linear nonnegative time-delay systems (5.98) and (5.99). To see this, for i = 1, . . . , n, let xi (t), t ≥ 0, denote the mass (and hence a nonnegative quantity) of the ith subsystem of the compartmental system shown in Figure 5.1, let aii ≥ 0 denote the loss coefficient of the ith subsystem, let φij (t−τ ), i 6= j, denote the net mass flow (or flux) from the jth subsystem to the ith subsystem given by φij (t − τ ) = aij xj (t − τ ) − aji xi (t), where the transfer coefficient aij ≥ 0, i 6= j, and τ is the fixed time it takes for the mass to flow from the jth subsystem to the ith subsystem, and let ui (t) ≥ 0, t ≥ 0, denote the input mass flux to ith compartment. For simplicity of exposition we have assumed that all transfer times between compartments are given by τ . The more general multipledelay case can be addressed as shown for the system G given by (5.109) and (5.110). Now, a mass balance for the whole compartmental system yields   n n X X aji  xi (t) + aij xj (t − τ ) + ui (t), x˙ i (t) = − aii + j=1,i6=j

j=1,i6=j

t ≥ 0,

i = 1, . . . , n,

(5.117)

or, equivalently, x(t) ˙ = Ax(t) + Ad x(t − τ ) + u(t),

x(θ) = φ(θ),

−τ ≤ θ ≤ 0,

t ≥ 0, (5.118)

where x(t) = [x1 (t), . . . , xn (t)]T , u(t) = [u1 (t), . . . , un (t)]T , φ(·) ∈ C+ , and for i, j = 1, . . . , n,  Pn  − k=1 aki , i = j, 0, i = j, A(i,j) = Ad(i,j) = (5.119) 0, i 6= j, aij , i 6= j.

Note that A is essentially nonnegative and Ad is nonnegative. Furthermore,

NNBook

September 1, 2009

188

CHAPTER 5

note that A + Ad is a compartmental matrix and (A + Ad )x = (Jn (x) − D(x))e,

(5.120)

where Jn (x) is a skew-symmetric matrix function with Jn(i,i) (x) = 0 and Jn(i,j) (x) = aij xj −aji xi , i 6= j, and D(x) = diag[a11 x1 , a22 x2 , . . . , ann xn ] ≥≥ n 0, x ∈ R+ . To show that all compartmental systems of the form (5.118) with measured outputs corresponding to material outflows y = D(x)e = [a11 x1 , a22 x2 , . . . , ann xn ]T are nonaccumulative, consider the storage functional Z 0 Vs (ψ) = eT ψ(0) + eT Ad ψ(θ)dθ.

(5.121)

−τ

Note that the storage functional Vs (ψ) captures the total mass of the system at t = 0 plus the integral of the mass flow in transit between the compartments over the time intervals it takes for the mass to flow through the intercompartmental connections. Now, it follows that V˙s (xt ) = eT x(t) ˙ + eT Ad [x(t) − x(t − τ )] + eT u(t) = eT [(A + Ad )x(t) + u(t)] = eT u(t) − eT y(t) + eT Jn (x(t))e = eT u(t) − eT y(t), xt ∈ C+ ,

(5.122)

which shows that all compartmental systems with outputs y = D(x)e are nonaccumulative, that is, dissipative with respect to the supply rate s(u, y) = eT u − eT y. Note that in the case where the system is closed, that is, u(t) ≡ 0 and y(t) ≡ 0, V˙s (xt ) = 0, xt ∈ C+ , which corresponds to conservation of mass in the system.

5.10 Feedback Interconnections of Nonnegative Dynamical Systems with Time Delay In this section, we consider feedback interconnections of dynamical systems with time delay. We begin by considering the positive feedback interconnection of the nonnegative dynamical system G given by (5.98) and (5.99) with the nonnegative dynamical feedback system Gc given by x˙ c (t) = Ac xc (t) + Adc xc (t − τ ) + Bc uc (t), xc (θ) = φc (θ), −τ ≤ θ ≤ 0, t ≥ 0, (5.123) yc (t) = Cc xc (t), (5.124) where Ac ∈ Rnc ×nc , Adc ∈ Rnc ×nc , Bc ∈ Rnc ×mc , Cc ∈ Rlc ×nc , Ac is △ essentially nonnegative, Adc ≥≥ 0, Bc ≥≥ 0, Cc ≥≥ 0, and φc (·) ∈ Cc+ =

NNBook

September 1, 2009

DISSIPATIVITY THEORY FOR NONNEGATIVE SYSTEMS

189

{ψc (·) ∈ C([−τ, 0], Rnc ) : ψc (θ) ≥≥ 0, θ ∈ [−τ, 0]}. Note that the positive feedback interconnection of G and Gc is given by u = yc and uc = y. The delay amount in the feedback system Gc need not be the same as the delay amount in the dynamical system G. The assumption of equal delay amounts in G and Gc is made for convenience only. If this is not the case, the closedloop system has the form given by (5.109) with u(t) ≡ 0 and thus can be easily addressed. For the following result we assume that rowi (C) 6= 0, i = 1, . . . , n, and rowi (Cc ) 6= 0, i = 1, . . . , nc , where rowi (·) denotes the ith row operator. Theorem 5.23. Let q ∈ Rl , r ∈ Rm , qc ∈ Rlc , and rc ∈ Rmc . Consider the linear nonnegative dynamical systems G and Gc given by (5.98) and (5.99), and (5.123) and (5.124), respectively, with u = yc and uc = y. Assume that G is dissipative with respect to the linear supply rate s(u, y) = q T y + r T u and with a linear storage functional Z 0 Vs (ψ) = pT ψ(0) + pT Ad ψ(θ)dθ, −τ

where p >> 0, and assume that Gc is dissipative with respect to the linear supply rate sc (uc , yc ) = qcT yc + rcT uc and with a linear storage functional Z 0 T Vsc (ψc ) = pc ψc (0) + pT c Adc ψc (θ)dθ, −τ

where pc >> 0. Then the following statements hold: i ) If there exists a scalar σ > 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is Lyapunov stable. ii ) If there exists a scalar σ > 0 such that q + σrc 0, and assume that Gc is dissipative with respect to the linear supply rate s(uc , yc ) = qcT yc + rcT uc and with a linear storage functional Vsc (ψc ) = pT c ψc (0) +

−1 X

pT c Adc ψc (θ),

θ=−κ

where pc >> 0. Then the following statements hold: i ) If there exists a scalar σ > 0 such that q + σrc ≤≤ 0 and r + σqc ≤≤ 0, then the positive feedback interconnection of G and Gc is Lyapunov stable. ii ) If there exists a scalar σ > 0 such that q + σrc 0. For further insights on Assumptions A1 and A2 the interested reader is referred to [117]. n

Next, we present a result which shows that R+ is an invariant set for (6.1) and (6.2) if fc : D → Rn is essentially nonnegative and fd : D → Rn is n such that x + fd (x) is nonnegative for all x ∈ R+ . n

Proposition 6.1. Suppose R+ ⊂ D. If fc : D → Rn is essentially nonnegative and fd : Zx → Rn is such that x + fd (x) is nonnegative, then n R+ is an invariant set with respect to (6.1) and (6.2). Proof. Consider the continuous-time dynamical system given by x˙ c (t) = fc (xc (t)),

xc (0) = xc0 ,

t ≥ 0.

(6.5)

Now, it follows from Proposition 2.1 that since fc : D → Rn is essentially n n nonnegative, R+ is an invariant set with respect to (6.5), that is, if xc0 ∈ R+ n then xc (t) ∈ R+ , t ≥ 0. Now, since, with xc0 = x0 , x(t) = xc (t), 0 ≤ t ≤ n τ1 (x0 ), it follows that x(t) ∈ R+ , 0 ≤ t ≤ τ1 (x0 ). Next, since fd : Zx → Rn is such that x + fd (x) is nonnegative it n follows that x+ 1 = x(τ1 (x0 )) + fd (x(τ1 (x0 ))) ∈ R+ . Now, since s(t, x0 ) = + s(t − τ1 (x0 ), x+ 1 ), τ1 (x0 ) < t ≤ τ2 (x0 ), with xc0 = x1 , it follows that x(t) = n xc (t − τ1 (x0 )) ∈ R+ , τ1 (x0 ) < t ≤ τ2 (x0 ), and hence, x+ 2 = x(τ2 (x0 )) + n fd (x(τ2 (x0 ))) ∈ R+ . Repeating this procedure for τi (x0 ), i = 3, 4, . . ., it n follows that R+ is an invariant set with respect to (6.1) and (6.2). It is important to note that unlike continuous-time nonnegative systems and discrete-time nonnegative systems, Proposition 6.1 provides n only sufficient conditions ensuring that R+ is an invariant set with respect n to (6.1) and (6.2). To see this, let Zx = ∂R+ and assume x + fd (x), x ∈ Zx ,

NNBook

September 1, 2009

201

HYBRID NONNEGATIVE SYSTEMS n

is nonnegative. Then, R+ remains invariant with respect to (6.1) and (6.2) irrespective of whether fc (·) is essentially nonnegative or not. Next, we specialize Proposition 6.1 to linear1 state-dependent impulsive dynamical systems of the form x(t) 6∈ Zx , x(t) ˙ = Ac x(t), x(0) = x0 , ∆x(t) = (Ad − In )x(t), x(t) ∈ Zx ,

(6.6) (6.7)

n

where t ≥ 0, x(t) ∈ R+ , Ac ∈ Rn×n is essentially nonnegative, Ad ∈ Rn×n is n nonnegative, and Zx ⊂ R+ . Note that in this case A2 implies that if x ∈ Zx , then Ad x 6∈ Zx . Proposition 6.2. Let Ac ∈ Rn×n and Ad ∈ Rn×n . If Ac is essentially n nonnegative and Ad is nonnegative, then R+ is an invariant set with respect to (6.6) and (6.7). Proof. The proof is a direct consequence of Proposition 6.1 with fc (x) = Ac x and fd (x) = (Ad − In )x. Next, we present several key results on stability of nonlinear hybrid nonnegative dynamical systems. We note that for addressing the stability of the equilibrium solution of a nonnegative impulsive dynamical system the stability definitions introduced in Chapter 2 are valid. In addition, the standard Lyapunov stability theorems [117] and invariant set theorems [117] for nonlinear hybrid dynamical systems can be used directly with the n required sufficient conditions verified on D ⊂ R+ , where D is an open subset n relative to R+ that contains xe . n

Theorem 6.1. Let D be an open subset relative to R+ that contains xe . n Suppose there exists a continuously differentiable function V : R+ → [0, ∞) satisfying V (xe ) = 0, V (x) > 0, x 6= xe , and V ′ (x)fc (x) ≤ 0, x 6∈ Zx , V (x + fd (x)) ≤ V (x), x ∈ Zx .

(6.8) (6.9)

Then the equilibrium solution x(t) ≡ xe of the hybrid nonnegative dynamical system (6.1) and (6.2) is Lyapunov stable. Furthermore, if the inequality (6.8) is strict for all x 6= xe , then the equilibrium solution x(t) ≡ xe of the hybrid nonnegative dynamical system (6.1) and (6.2) is asymptotically stable. Finally, if V (x) → ∞, as kxk → ∞, then asymptotic stability is global. 1 Impulsive dynamical systems with f (x) = Ax and f (x) = (A −I)x are not linear. However, c d d this minor abuse in terminology provides a natural way of differentiating between impulsive dynamical systems with nonlinear vector fields versus impulsive dynamical systems with linear vector fields, and considerably simplifies the presentation.

NNBook

September 1, 2009

202

CHAPTER 6

Proof. The proof is identical to the proof of Theorem 2.1 of [117] n n with D ⊆ R+ and Zx ⊂ R+ . Next, we present a generalized Krasovskii-LaSalle invariant set stability theorem for nonlinear hybrid dynamical systems. The following key assumption is needed for the statement of this result. Assumption 6.1 ([117]). Consider the impulsive nonnegative dynamical system G given by (6.1) and (6.2), and let s(t, x0 ), t ≥ 0, denote the solution to (6.1) and (6.2) with initial condition x0 . Then for every x0 ∈ D, there exists a dense subset Tx0 ⊆ [0, ∞) such that [0, ∞) \ Tx0 is (finitely or infinitely) countable and for every ε > 0 and t ∈ Tx0 , there exists δ(ε, x0 , t) > 0 such that if kx0 − yk < δ(ε, x0 , t), y ∈ D, then ks(t, x0 ) − s(t, y)k < ε. Assumption 6.1 is a generalization of the standard continuous dependence property for dynamical systems with continuous flows to dynamical systems with left-continuous flows and ensures continuous dependence over a dense subset of [0, ∞). Henceforth, we assume that fc (·), fd (·), and Zx are such that Assumption 6.1 holds. Necessary and sufficient conditions that guarantee that the nonlinear impulsive dynamical system G given by (6.1) and (6.2) satisfies Assumption 6.1 are given in [117]. For further discussion on Assumption 6.1, see [117]. Theorem 6.2. Consider the hybrid nonnegative dynamical system G n given by (6.1) and (6.2), assume Dc ⊂ D ⊆ R+ is a compact positively invariant set with respect to (6.1) and (6.2), and assume that there exists a continuously differentiable function V : Dc → R such that V ′ (x)fc (x) ≤ 0, x ∈ Dc , x 6∈ Zx , V (x + fd (x)) ≤ V (x), x ∈ Dc , x ∈ Zx .

(6.10) (6.11)

Let R , {x ∈ Dc : x 6∈ Zx , V ′ (x)fc (x) = 0} ∪ {x ∈ Dc : x ∈ Zx , V (x + fd (x)) = V (x)} and let M denote the largest invariant set contained in R. If x0 ∈ Dc , then x(t) → M as t → ∞. Proof. The proof is a direct consequence of Theorem 2.3 of [117] with n Dc ⊂ D ⊆ R+ . Finally, we give sufficient conditions for Lyapunov stability and asymptotic stability of linear hybrid nonnegative dynamical systems using linear Lyapunov functions. Theorem 6.3. Consider the linear hybrid dynamical system given by (6.6) and (6.7) where Ac ∈ Rn×n is essentially nonnegative and Ad ∈ Rn×n

NNBook

September 1, 2009

203

HYBRID NONNEGATIVE SYSTEMS

is nonnegative. Then the following statements hold: i ) If there exist vectors p, rc , rd ∈ Rn such that p >> 0, rc ≥≥ 0, and rd ≥≥ 0, satisfy 0 = xT (AT x 6∈ Zx , c p + rc ), T T 0 = x (Ad p − p + rd ), x ∈ Zx ,

(6.12) (6.13)

then the zero solution x(t) ≡ 0 to (6.6) and (6.7) is Lyapunov stable. ii ) If there exist vectors p, rc , rd ∈ Rn such that p >> 0, rc >> 0, and rd ≥≥ 0 satisfy (6.12) and (6.13), then the zero solution x(t) ≡ 0 to (6.6) and (6.7) is asymptotically stable. Proof. The result is a direct consequence of Theorem 6.1 with V (x) = pT x, fc (x) = Ac x, and fd (x) = (Ad − In )x. Specifically, in this case, V ′ (x)fc (x) = pT Ac x = −rcT x ≤ 0, x 6∈ Zx , and V (x + fd (x)) − V (x) = pT Ad x − pT x = −rdT x ≤ 0, x ∈ Zx , so that all the conditions of Theorem 6.1 are satisfied, which proves Lyapunov stability. In the case where rc >> 0 it follows that V ′ (x)fc (x) = pT Ac x = −rcT x < 0, x 6∈ Zx , which proves asymptotic stability. For asymptotic stability, conditions (6.12) and (6.13) are implied by T p >> 0, AT c p > 0, rc >> 0, and rd ≥≥ 0 such that (6.12) and (6.13) are satisfied if and only if p >> 0 and −Ap ≥≥ 0.

6.3 Hybrid Compartmental Dynamical Systems In this section, we specialize the results of Section 6.2 to hybrid compartmental dynamical systems. Specifically, we show that nonlinear hybrid compartmental dynamical systems are a special case of hybrid nonnegative dynamical systems. To see this, let xi (t), i = 1, . . . , n, denote the mass (and hence a nonnegative quantity) of the ith subsystem of the hybrid compartmental system shown in Figure 6.1, let acii (x) ≥ 0, x 6∈ Zx , denote

NNBook

September 1, 2009

204

CHAPTER 6

wci (t)

wdi (tk )

acij (x(t))

'? ?

)$ ) adij (x(tk )) ith Subsystem xi (t) adji (x(tk )) & % acii (x(t))

adii (x(tk )) ? ?

acji (x(t))

wcj (t)

wdj (tk )

'? ?

$

jth Subsystem xj (t) 3 & 3 acjj (x(t))

%

adjj (x(tk )) ? ?

Figure 6.1 Nonlinear hybrid compartmental interconnected subsystem model.

the rate of flow of material loss of the ith continuous-time subsystem, let wci (t) ≥ 0, t ≥ 0, i = 1, . . . , n, denote the rate of mass inflow supplied to the ith continuous-time subsystem, and let φcij (x(t)), t ≥ 0, i 6= j, i, j = 1, . . . , n, denote the net mass flow (or flux) from the jth continuous-time subsystem to the ith continuous-time subsystem given by φcij (x(t)) = acij (x(t)) − acji (x(t)), where the rates of material flows are such that acij (x) ≥ 0, x 6∈ Zx , i 6= j, i, j = 1, . . . , n. Similarly, for the resetting dynamics, let adii (x) ≥ 0, x ∈ Zx , denote the material loss of the ith discrete-time subsystem, let wdi (tk ) ≥ 0, i = 1, . . . , n, denote the mass inflow supplied to the ith discrete-time subsystem, and let φdij (tk ), i 6= j, i, j = 1, . . . , n, denote the net mass exchange from the jth discrete-time subsystem to the ith discrete-time subsystem given by φdij (x(tk )) = adij (x(tk )) − adji (x(tk )), where tk = τk (x0 ) and the material flows are such that adij (x) ≥ 0, x ∈ Zx , i 6= j, i, j = 1, . . . , n. A mass balance for the whole hybrid compartmental system yields x˙ i (t) = −acii (x(t)) +

∆xi (t) = −adii (x(t)) +

n X

φcij (x(t)) + wci (t),

n X

φdij (x(t)) + wdi (t),

x(t) 6∈ Zx ,

j=1,i6=j

i = 1, . . . , n,

j=1,i6=j

(6.17)

x(t) ∈ Zx ,

i = 1, . . . , n,

(6.18)

x(t) 6∈ Zx ,

(6.19) (6.20)

or, equivalently, x(t) ˙ = fc (x(t)) + wc (t), ∆x(t) = fd (x(t)) + wd (t),

x(0) = x0 , x(t) ∈ Zx ,

where x(t) , [x1 (t), . . . , xn (t)]T , wc (t) , [wc1 (t), . . . , wcn (t)]T , wd (t) ,

NNBook

September 1, 2009

205

HYBRID NONNEGATIVE SYSTEMS

[wd1 (t), . . . , wdn (t)]T , and, for i, j = 1, . . . , n, fci (x) = −acii (x) + fdi (x) = −adii (x) +

n X

j=1,i6=j n X

[acij (x) − acji (x)],

(6.21)

[adij (x) − adji (x)].

(6.22)

j=1,i6=j

Since all mass flows as well as compartment sizes are nonnegative, it follows that for all i = 1, . . . , n, fci (x) ≥ 0 for all x 6∈ Zx whenever xi = 0 and whatever the values of xj , j 6= i, and xi + fdi (x) ≥ 0 for all x ∈ Zx . The above physical constraints are implied by acij (x) ≥ 0, acii (x) ≥ 0, x 6∈ Zx , adij (x) ≥ 0, adii (x) ≥ 0, x ∈ Zx , wci ≥ 0, wdi ≥ 0, for all i, j = 1, . . . , n, and if xi = 0, then acii (x) = 0 and acji (x) = 0 for all i, j = 1, . . . , n, so that x˙ i ≥ 0. In this case, fc (x), x 6∈ Zx , is essentially nonnegative and x + fd (x) ≥≥ 0, x ∈ Zx , and hence, the hybrid compartmental model given by (6.17) and (6.18) is a hybrid nonnegative dynamical system. T Pn Taking the total mass of the compartmental system V (x) = e x = i=1 xi as a Lyapunov function for the undisturbed (i.e., wc (t) ≡ 0 and wd (tk ) ≡ 0) system (6.17) and (6.18), and assuming aij (0) = 0, i, j = 1, . . . , n, it follows that

V˙ (x) =

n X

x˙ i

i=1

= − = − ≤ 0,

n X

i=1 n X

acii (x) +

n n X X

i=1 j=1,i6=j

[acij (x) − acji (x)]

acii (x)

i=1

x 6∈ Zx ,

and ∆V (x) =

n X

∆xi

i=1

= − = − ≤ 0,

n X

adii (x) +

i=1

n X

n n X X

i=1 j=1,i6=j

adii (x)

i=1

x ∈ Zx ,

[adij (x) − adji (x)]

NNBook

September 1, 2009

206

CHAPTER 6

which, by Theorem 6.1, shows that the zero solution x(t) ≡ 0 of the nonlinear hybrid compartmental system given by (6.17) and (6.18) is Lyapunov stable. If (6.17) and (6.18) with wc (t) ≡ 0 and wd (tk ) ≡ 0 has losses (outflows) from all compartments over the continuous-time dynamics, then acii (x) > 0, x 6∈ Zx , x 6= 0, and hence, by Theorem 6.1, the zero solution x(t) ≡ 0 to (6.17) and (6.18) is asymptotically stable. It is interesting to note that in the linear case acii (x) = acii xi , φcij (x) = acij xj − acji xi , adii (x) = adii xi , and φdij (x) = adij xj − adji xi , where acij ≥ 0 and adij ≥ 0, i, j = 1, . . . , n, (6.19) and (6.20) become x(t) ˙ = Ac x(t) + wc (t), x(0) = x0 , x(t) 6∈ Zx , ∆x(t) = (Ad − In )x(t) + wd (t), x(t) ∈ Zx ,

(6.23) (6.24)

where, for i, j = 1, . . . , n, −

Pn

l=1 acli , i = j, i 6= j, acij ,  Pn 1 − l=1 adli , i = j, = i 6= j. adij ,

Ac(i,j) = Ad(i,j)



(6.25) (6.26)

Note that, since at any given instant of time compartmental mass can only be transported, stored, or discharged but not created and the maximum amount of mass that can be transported and/or P discharged cannot exceed the mass in a compartment, it follows that 1 ≥ nl=1 adli . Thus Ac is an essentially nonnegative matrix and Ad is a nonnegative matrix, and hence, the hybrid compartmental model given by (6.23) and (6.24) is a hybrid nonnegative dynamical system. The hybrid compartmental system (6.17) and (6.18) with no inflows, that is, wci (t) ≡ 0 and wdi (tk ) ≡ 0, i = 1, . . . , n, is said to be inflow-closed. Alternatively, if (6.17) and (6.18) possesses no losses (outflows) it is said to be outflow-closed. A hybrid compartmental system is said to be closed if it is inflow-closed and outflow-closed. Note that for a closed system V˙ (x) = 0, x 6∈ Zx , and ∆V (x) = 0, x ∈ Zx , which shows that the total mass inside a closed system is conserved. In the case where acii (x) 6= 0, x 6∈ Zx , adii (x) 6= 0, x ∈ Zx , wci (t) 6= 0, and wdi (tk ) 6= 0, i = 1, . . . , n, it follows that (6.17) and (6.18) can be equivalently written as  T ∂V (x(t)) x(t) ˙ = [Jcn (x(t)) − Dc (x(t))] + wc (t), x(t) 6∈ Zx , ∂x  T ∂V (x(t)) + wd (t), x(t) ∈ Zx , ∆x(t) = [Jdn (x(t)) − Dd (x(t))] ∂x

NNBook

September 1, 2009

207

HYBRID NONNEGATIVE SYSTEMS

where Jcn (x) and Jdn (x) are skew-symmetric matrix functions with Jcn(i,i) (x) = 0, Jdn(i,i) (x) = 0, Jcn(i,j) (x) = acij (x) − acji (x), and Jdn(i,j) (x) = adij (x) − adji (x), i 6= j, Dc (x) = diag[ac11 (x), ac22 (x), . . . , acnn (x)] ≥≥ n 0, and Dd (x) = diag[ad11 (x), ad22 (x), . . . , adnn (x)] ≥≥ 0, x ∈ R+ . Hence, a linear hybrid compartmental system is a hybrid port-controlled Hamiltonian system with a Hamiltonian H(x) = V (x) = eT x representing n the total mass in the system, Jcn (x), x ∈ R+ , representing the internal n system interconnection over the continuous-time dynamics, Jdn (x), x ∈ R+ , representing the internal system interconnection at the resetting instants, n Dc (x), x ∈ R+ , representing the outflow dissipation over the continuousn time dynamics, Dd (x), x ∈ R+ , representing the outflow dissipation at the resetting instants, wc (t) representing the supplied flux to the system over the continuous-time dynamics, and wd (tk ) representing the supplied mass to the system at the resetting instants. This observation shows that hybrid compartmental systems are conservative systems. This will be further elaborated in the following sections.

6.4 Dissipativity Theory for Hybrid Nonnegative Dynamical Systems In this section, we extend the notion of dissipativity to nonlinear impulsive nonnegative dynamical systems. Specifically, we consider nonlinear impulsive dynamical systems G of the form (x(t), uc (t)) 6∈ Z, (6.27) ∆x(t) = fd (x(t)) + Gd (x(t))ud (t), (x(t), uc (t)) ∈ Z, (6.28) yc (t) = hc (x(t)) + Jc (x(t))uc (t), (x(t), uc (t)) 6∈ Z, (6.29) yd (t) = hd (x(t)) + Jd (x(t))ud (t), (x(t), uc (t)) ∈ Z, (6.30) x(t) ˙ = fc (x(t)) + Gc (x(t))uc (t),

x(0) = x0 ,

n

where t ≥ 0, x(t) ∈ D ⊆ Rn , D is a relatively open set with respect to R+ and △ with 0 ∈ D, ∆x(t) = x(t+ ) − x(t), uc (t) ∈ Uc ⊆ Rmc , ud (tk ) ∈ Ud ⊆ Rmd , tk denotes the kth instant of time at which (x(t), uc (t)) intersects Z ⊂ D × Uc for a particular trajectory x(t) and input uc (t), yc (t) ∈ Yc ⊆ Rlc , yd (tk ) ∈ Yd ⊆ Rld , fc : D → Rn is Lipschitz continuous and satisfies fc (0) = 0, Gc : D → Rn×mc , fd : D → Rn is continuous, Gd : D → Rn×md , hc : D → Rlc and satisfies hc (0) = 0, Jc : D → Rlc ×mc , hd : D → Rld , Jd : D → Rld ×md , and Z ⊂ D × Uc . Here, we assume that uc (·) and ud (·) are restricted to the class of admissible inputs consisting of measurable functions such that (uc (t), ud (tk )) ∈ △ Uc × Ud for all t ≥ 0 and k ∈ Z[0,t) = {k : 0 ≤ tk < t}, where the constraint set Uc × Ud is given with (0, 0) ∈ Uc × Ud . Furthermore, we assume that

NNBook

208

September 1, 2009

CHAPTER 6 △

the set Z = {(x, uc ) : X (x, uc ) = 0}, where X : D × Uc → R. In addition, we assume that the system functions fc (·), fd (·), Gc (·), Gd (·), hc (·), hd (·), Jc (·), and Jd (·) are continuously differentiable mappings. Finally, for the nonlinear dynamical system (6.27) we assume that the required properties for the existence and uniqueness of solutions are satisfied such that (6.27) has a unique solution for all t ∈ R [16, 189]. Next, we provide definitions and several results concerning dynamical systems of the form (6.27)–(6.30) with nonnegative inputs and nonnegative outputs. Definition 6.1. The nonlinear dynamical system G given by (6.27)– (6.30) with x(0) = 0 is input-output nonnegative if the hybrid output (yc (t), yd (tk )), t ≥ 0, k ∈ Z+ , is nonnegative for every nonnegative hybrid input (uc (t), ud (tk )), t ≥ 0, k ∈ Z+ . Definition 6.2. The nonlinear dynamical system G given by (6.27)– n (6.30) is nonnegative if for every x(0) ∈ R+ and nonnegative hybrid input (uc (t), ud (tk )), t ≥ 0, k ∈ Z+ , the solution x(t), t ≥ 0, to (6.27) and (6.27) and the hybrid output (yc (t), yd (tk )), t ≥ 0, k ∈ Z+ , are nonnegative. Proposition 6.3. Consider the nonlinear dynamical system G given by (6.27)–(6.30). If fc : D → Rn is essentially nonnegative, fd : D → Rn is n such that x + fd (x) is nonnegative for all x ∈ R+ , Gc (x) ≥≥ 0, Gd (x) ≥≥ 0, n hc (x) ≥≥ 0, hd (x) ≥≥ 0, Jc (x) ≥≥ 0, and Jd (x) ≥≥ 0, x ∈ R+ , then G is nonnegative. Proof. The proof is similar to the proof of Proposition 6.1 and follows from Propositions 5.2 and 5.3. For the impulsive dynamical system G given by (6.27)–(6.30) a function (sc (uc , yc ), sd (ud , yd )), where sc : Uc × Yc → R and sd : Ud × Yd → R are such that sc (0, 0) = 0 and sd (0, 0) = 0, is called a hybrid supply rate if sc (uc , yc ) is locally integrable for all input-output pairs uc (t) ∈ Uc and yc (t) ∈ Yc satisfying (6.27)–(6.30), that is, for all input-output pairs uc (t) ∈ Uc , yc (t) ∈ Yc satisfying (6.27)–(6.30), sc (·, ·) satisfies R tˆ ˆ t |sc (uc (s), yc (s))| ds < ∞, t, t ≥ 0. Note that since all input-output pairs ud (tk ) ∈ Ud , yd (tk ) ∈ Yd , are defined for discrete instants, sd (·, ·) satisfies P △ ˆ k∈Z[t,tˆ) |sd (ud (tk ), yd (tk ))| < ∞, where k ∈ Z[t,tˆ) = {k : t ≤ tk < t}. The following definition introduces the notion of dissipativity and exponential dissipativity for a nonlinear hybrid nonnegative dynamical system. Definition 6.3. The impulsive dynamical system G given by (6.27)– (6.30) is exponentially dissipative (respectively, dissipative) with respect to

NNBook

September 1, 2009

209

HYBRID NONNEGATIVE SYSTEMS

the hybrid supply rate (sc , sd ) if there exist a continuous, nonnegativen definite function Vs : R+ → R+ , called a storage function, and a scalar ε > 0 (respectively, ε = 0) such that Vs (0) = 0 and the dissipation inequality εT

Z

T

Vs (x(t0 )) + eεt sc (uc (t), yc (t))dt t0 X + eεtk sd (ud (tk ), yd (tk )), T ≥ t0 , εt0

e Vs (x(T )) ≤ e

(6.31)

k∈Z[t0 ,T )

is satisfied for all T ≥ t0 , where x(t), t ≥ t0 , is the solution of (6.27)–(6.30) with (uc (·), ud (·)) ∈ Uc+ × Ud+ . The impulsive dynamical system given by (6.27)–(6.30) is lossless with respect to the hybrid supply rate (sc , sd ) if the dissipation inequality (6.31) is satisfied as an equality with ε = 0 for all T ≥ t0 and (uc (·), ud (·)) ∈ Uc+ × Ud+ . The following result gives necessary and sufficient conditions for dissipativity over an interval t ∈ (tk , tk+1 ] involving the consecutive resetting times tk and tk+1 . First, however, the following definition is required. Definition 6.4. An impulsive dynamical system G given by (6.27)– (6.30) is zero-state observable if (uc (t), ud (tk )) ≡ (0, 0) and (yc (t), yd (tk )) ≡ (0, 0), k ∈ Z+ , implies x(t) ≡ 0. An impulsive dynamical system G given by (6.27)–(6.30) is strongly zero-state observable if uc (t) ≡ 0 and yc (t) ≡ 0 implies x(t) ≡ 0. An impulsive dynamical system G is completely reachable if for all (t0 , xi ) ∈ R × D, there exist a finite time ti ≤ t0 , square integrable nonnegative inputs uc (t) defined on [ti , t0 ], and square summable nonnegative inputs ud (tk ) defined on k ∈ Z[ti ,t0 ) , such that the state x(t), t ≥ ti , can be driven from x(ti ) = 0 to x(t0 ) = xi . Finally, an impulsive dynamical system G is minimal if it is zero-state observable and completely reachable. Theorem 6.4. Assume G is completely reachable. Then G is dissipative with respect to the hybrid supply rate (sc , sd ) if and only if there exists n a continuous, nonnegative-definite function Vs : R+ → R+ such that, for all k ∈ Z+ , Vs (x(tˆ)) − Vs (x(t)) ≤

Z

t

tˆ

sc (uc (s), yc (s))ds,

tk < t ≤ tˆ ≤ tk+1 ,

(6.32)

Vs (x(tk ) + fd (x(tk )) + Gd (x(tk ))ud (tk )) − Vs (x(tk )) ≤ sd (ud (tk ), yd (tk )). (6.33) Furthermore, G is exponentially dissipative with respect to the hybrid supply rate (sc , sd ) if and only if there exists a continuous, nonnegative-definite

NNBook

September 1, 2009

210

CHAPTER 6 n

function Vs : R+ → R+ such that Z tˆ εtˆ εt eεs sc (uc (s), yc (s))ds, e Vs (x(tˆ)) − e Vs (x(t)) ≤ t

tk < t ≤ tˆ ≤ tk+1 ,

(6.34) Vs (x(tk ) + fd (x(tk )) + Gd (x(tk ))ud (tk )) − Vs (x(tk )) ≤ sd (ud (tk ), yd (tk )). (6.35)

Finally, G is lossless with respect to the hybrid supply rate (sc , sd ) if and n only if there exists a continuous, nonnegative-definite function Vs : R+ → R+ such that (6.32) and (6.33) are satisfied as equalities. Proof. The proof is identical to the proof of Theorem 3.2 of [117]. If Vs (·) is continuously differentiable, then an equivalent statement for dissipativeness of the impulsive dynamical system G with respect to the hybrid supply rate (sc , sd ) is V˙s (x(t)) ≤ sc (uc (t), yc (t)), tk < t ≤ tk+1 , ∆Vs (x(tk )) ≤ sd (ud (tk ), yd (tk )), k ∈ Z+ ,

(6.36) (6.37)

where V˙ s (·) denotes the total derivative of Vs (x(t)) along the state trajectories x(t), t ∈ (tk , tk+1 ], of the impulsive dynamical system (6.27)– △ (6.30) and ∆Vs (x(tk )) = Vs (x(t+ k )) − Vs (x(tk )) = Vs (x(tk ) + fd (x(tk )) + Gd (x(tk ))ud (tk )) − Vs (x(tk )), k ∈ Z+ , denotes the difference of the storage function Vs (x) at the resetting times tk , k ∈ Z+ , of the impulsive dynamical system (6.27)–(6.30). Furthermore, an equivalent statement for exponential dissipativeness of the impulsive dynamical system G with respect to the supply rate (sc , sd ) is given by V˙s (x(t)) + εVs (x(t)) ≤ sc (uc (t), yc (t)),

tk < t ≤ tk+1 ,

(6.38)

and (6.37). The following result presents Kalman-Yakubovich-Popov conditions for hybrid nonnegative dynamical systems with linear hybrid supply rates of the form (sc (uc , yc ), sd (ud , yd )) = (qcT yc +rcT uc , qdT yd +rdTud ), where qc ∈ Rlc , qc 6= 0, rc ∈ Rmc , rc 6= 0, qd ∈ Rld , qd 6= 0, rd ∈ Rmd , and rd 6= 0. For the mc md remainder of the section we assume that Uc = R+ , Ud = R+ , and Z = mc Zx ×R so that resetting occurs only when x(t) intersects Zx . Furthermore, we assume that there exists a continuously differentiable storage function Vs (x), x ∈ Rn , for the impulsive nonnegative dynamical system G. Theorem 6.5. Let qc ∈ Rlc , rc ∈ Rmc , qd ∈ Rld , and rd ∈ Rmd . Consider the nonlinear hybrid dynamical system G given by (6.27)–(6.30) where fc : D → Rn is essentially nonnegative, fd : Zx → Rn is such that

NNBook

September 1, 2009

HYBRID NONNEGATIVE SYSTEMS

211

x + fd (x) nonnegative, Gc (x) ≥≥ 0, Gd (x) ≥≥ 0, hc (x) ≥≥ 0, hd (x) ≥≥ 0, n n Jc (x) ≥≥ 0, and Jd (x) ≥≥ 0, x ∈ R+ . If there exist functions Vs : R+ → R+ , n n n mc n md ℓc : R+ → R+ , ℓd : R+ → R+ , Wc : R+ → R+ , Wd : R+ → R+ , and a scalar ε > 0 (respectively, ε = 0) such that Vs (·) is continuously differentiable, nonnegative definite, Vs (0) = 0, Vs (x + fd (x) + Gd (x)ud ) = Vs (x + fd (x)) + Vs′ (x + fd (x))Gd (x)ud , md (6.39) x ∈ Zx , ud ∈ R+ , and 0 0 0 0

= = = =

Vs′ (x)fc (x) + εVs (x) − qcT hc (x) + ℓc (x), x 6∈ Zx , Vs′ (x)Gc (x) − qcT Jc (x) − rcT + WcT (x), x 6∈ Zx , Vs (x + fd (x)) − Vs (x) − qdT hd (x) + ℓd (x), x ∈ Zx , Vs′ (x + fd (x))Gd (x) − qdT Jd (x) − rdT + WdT (x), x ∈ Zx ,

(6.40) (6.41) (6.42) (6.43)

then the nonlinear impulsive system G given by (6.27)–(6.30) is exponentially dissipative (respectively, dissipative) with respect to the linear hybrid supply rate (sc (uc , yc ), sd (ud , yd )) = (qcT yc + rcT uc , qdT yd + rdT ud ). Proof. For every admissible input uc (·) ∈ Uc+ , t, tˆ ∈ R, tk < t ≤ tˆ ≤ tk+1 , and k ∈ Z+ , it follows from (6.40) and (6.41) that, for all x 6∈ Zx and mc uc ∈ R+ , V˙ s (x) + εVs (x) = = = ≤ =

Vs′ (x)(fc (x) + Gc (x)uc ) + εVs (x) qcT hc (x) − ℓc (x) + qcT Jc (x)uc + rcT uc − WcT (x)uc qcT yc + rcT uc − ℓc (x) − WcT (x)uc qcT yc + rcT uc sc (uc , yc ). (6.44)

Next, it follows from (6.42), (6.43), and the structural storage function md constraint (6.39) that, for all x ∈ Zx and ud ∈ R+ , ∆Vs (x) = = = = ≤

Vs (x + fd (x) + Gd (x)ud ) − Vs (x) Vs (x + fd (x)) − Vs (x) + Vs′ (x + fd (x))Gd (x)ud qdT hd (x) − ℓd (x) + qdT Jd (x)ud + rdT ud − WdT (x)ud sd (ud , yd ) − ℓd (x) − WdT (x)ud sd (ud , yd ). (6.45)

Now, using (6.44) and (6.45), the result follows from Theorem 6.4. The structural constraint (6.39) on the system storage function is similar to the structural constraint invoked in standard nonlinear discrete-time dissipativity theory [112] and hybrid dissipativity theory [117]. However, n since Vs : R+ → R+ , we can take a first-order Taylor expansion in (6.39) as

NNBook

September 1, 2009

212

CHAPTER 6

opposed to the second-order Taylor expansion given in [44, 54, 115]. As in standard dissipativity theory with quadratic supply rates [112], the concepts of linear supply rates and linear storage functions provide a generalized mass balance interpretation. Specifically, using (6.40)–(6.43), it follows that, for tˆ ≥ t ≥ 0 and k ∈ Z[t,tˆ) , Z

tˆ t

[qcT yc (s) + rcT uc (s)]ds +

X

[qdT yd (tk ) + rdT ud (tk )]

k∈Z[t,tˆ)

Z tˆ T [ℓT = Vs (x(tˆ)) − Vs (x(t)) + c (x(s))x(s) + Wc (x(s))uc (s)]ds t X T + [ℓd (x(tk ))x(tk ) + WdT (x(tk ))ud (tk )], (6.46) k∈Z[t,tˆ)

which can be interpreted as a generalized mass balance equation where Vs (x(tˆ)) − Vs (x(t)) is the stored mass of the nonlinear hybrid dynamical system, the second path-dependent term on the right corresponds to the expelled mass of the nonnegative system over the continuous-time dynamics, and the third discrete term on the right corresponds to the expelled mass at the resetting instants. Equivalently, it follows from Theorem 6.4 that (6.46) can be rewritten as T V˙ s (x(t)) = qcT yc (t) + rcT uc (t) − [ℓT c (x(s))x(s) + Wc (x(s))uc (s)], tk < t ≤ tk+1 , (6.47) T T T T ∆Vs (x(tk )) = qd yd (tk ) + rd ud (tk ) − [ℓd (x(tk ))x(tk ) + Wd (x(tk ))ud (tk )], (6.48) k ∈ Z+ ,

which yields a set of generalized mass flux (respectively, mass) conservation equations. Specifically, (6.47) and (6.48) show that the system mass transport (respectively, change in system mass) over the interval t ∈ (tk , tk+1 ] (respectively, the resetting instants) is equal to the supplied system flux (respectively, mass) minus the expelled system flux (respectively, mass). If a nonnegative dynamical system G is dissipative with respect to the linear hybrid supply rate (qcT yc + rcT uc , qdT yd + rdT ud ) with a continuously differentiable, positive-definite storage function, and qc ≤≤ 0, qd ≤≤ 0, and (uc (t), ud (tk )) ≡ (0, 0), then it follows that V˙ s (x(t)) ≤ qcT yc (t) ≤ 0, t ≥ 0, and ∆Vs (x(tk )) ≤ qdT yd (tk ) ≤ 0, k ∈ Z+ . Hence, the undisturbed ((uc (t), ud (tk )) ≡ (0, 0)) system G is Lyapunov stable. Furthermore, if a nonnegative dynamical system G is exponentially dissipative with respect to the hybrid linear supply rate (qcT yc + rcT uc ; qdT yd + rdT ud ) with a continuously differentiable, positive-definite storage function, and qc ≤≤ 0, qd ≤≤ 0, and

NNBook

September 1, 2009

213

HYBRID NONNEGATIVE SYSTEMS

(uc (t), ud (tk )) ≡ (0, 0), then it follows that V˙ s (x(t)) ≤ −εVs (x(t))+qcT yc (t) < 0, x(t) 6= 0, t ≥ 0, where ε > 0, and ∆Vs (x(tk )) ≤ qdT yd (tk ) ≤ 0, k ∈ Z+ . Hence, the undisturbed ((uc (t), ud (tk )) ≡ (0, 0)) system G is asymptotically stable. Next, we provide necessary and sufficient conditions for the case where G given by (6.27)–(6.30) is lossless with respect to the linear hybrid supply rate of the form (sc (uc , yc ), sd (ud , yd )) = (qcT yc + rcT uc , qdT yd + rdT ud ). Theorem 6.6. Let qc ∈ Rlc , rc ∈ Rmc , qd ∈ Rld , and rd ∈ Rmd . Consider the nonlinear hybrid dynamical system G given by (6.27)–(6.30) where fc : D → Rn is essentially nonnegative, fd : Zx → Rn is such that x+fd (x) is nonnegative, Gc (x) ≥≥ 0, Gd (x) ≥≥ 0, hc (x) ≥≥ 0, hd (x) ≥≥ 0, n Jc (x) ≥≥ 0, and Jd (x) ≥≥ 0, x ∈ R+ . Then G is lossless with respect to the supply rate (sc (uc , yc ), sd (ud , yd )) = (qcT yc + rcT uc , qdT yd + rdT ud ) if and n only if there exists a function Vs : R+ → R+ such that Vs (·) is continuously md differentiable, nonnegative definite, Vs (0) = 0, and, for all x ∈ Zx , ud ∈ R+ , (6.39) holds, and 0 0 0 0

= = = =

Vs′ (x)fc (x) − qcT hc (x), x 6∈ Zx , Vs′ (x)Gc (x) − qcT Jc (x) − rcT , x 6∈ Zx , Vs (x + fd (x)) − Vs (x) − qdT hd (x), x ∈ Zx , Vs′ (x + fd (x))Gd (x) − qdT Jd (x) − rdT , x ∈ Zx .

(6.49) (6.50) (6.51) (6.52)

Proof. Sufficiency follows as in the proof of Theorem 6.5. To show necessity, suppose that the nonlinear impulsive dynamical system G is lossless with respect to the linear supply rate (sc , sd ). Then, it follows that for all k ∈ Z+ , Z tˆ ˆ Vs (x(t)) − Vs (x(t)) = sc (uc (s), yc (s))ds, tk < t ≤ tˆ ≤ tk+1 , (6.53) t

and

Vs (x(tk ) + fd (x(tk )) + Gd (x(tk ))ud (tk )) = Vs (x(tk )) + sd (ud (tk ), yd (tk )). (6.54) Now, dividing (6.53) by tˆ − t+ and letting tˆ → t+ , (6.53) is equivalent to V˙s (x(t)) = Vs′ (x(t))[fc (x(t)) + Gc (x(t))uc (t)] = sc (uc (t), yc (t)), tk < t ≤ tk+1 .

(6.55)

Next, with t = 0, it follows from (6.55) that mc

Vs′ (x0 )[fc (x0 ) + Gc (x0 )uc (0)] = sc (uc (0), yc (0)), x0 6∈ Zx , uc (0) ∈ R+ . (6.56)

NNBook

September 1, 2009

214

CHAPTER 6

Since x0 6∈ Zx is arbitrary, it follows that

Vs′ (x)[fc (x) + Gc (x)uc ] = qcT yc + rcT uc mc = qcT hc (x) + (rcT + qcT Jc (x))uc , x 6∈ Zx , uc ∈ R+ .

Now, setting uc = 0 yields (6.49), which further yields (6.50). Next, it follows from (6.54) with k = 1 that Vs (x(t1 ) + fd (x(t1 )) + Gd (x(t1 ))ud (t1 )) = Vs (x(t1 )) + sd (ud (t1 ), yd (t1 )). (6.57) Now, since the continuous-time dynamics (6.27) are Lipschitz, it follows that for arbitrary x ∈ Zx there exists x0 6∈ Zx such that x(t1 ) = x. Hence, it follows from (6.57) that Vs (x + fd (x) + Gd (x)ud ) = Vs (x) + qdT yd + rdT ud = Vs (x) + qdT hd (x) + (rdT + qdT Jd (x))ud , md (6.58) x ∈ Z x , ud ∈ R + . Since the right-hand side of (6.58) is linear in ud it follows that Vs (x + n fd (x) + Gd (x)ud ) is linear in ud , and hence, there exists P1ud : R+ → R1×md such that Vs (x + fd (x) + Gd (x)ud ) = Vs (x + fd (x)) + P1ud (x)ud .

(6.59)

Since Vs (·) is continuously differentiable, application of a Taylor series expansion on (6.59) about ud = 0 yields ∂Vs (x + fd (x) + Gd (x))ud = Vs′ (x + fd (x))Gd (x). (6.60) P1ud (x) = ∂ud ud =0

Now, using (6.59) and equating coefficients of equal powers in (6.58) yields (6.51) and (6.52). Next, we present the notion of nonaccumulativity for hybrid nonnegative dynamical systems. Definition 6.5. A hybrid nonnegative dynamical system G of the form (6.27)–(6.30) is nonaccumulative (respectively, exponentially nonaccumulative) if G is dissipative (respectively, exponentially dissipative) with respect to the supply rate (sc (uc , yc ), sd (ud , yd )) = (eT uc − eT yc , eT ud − eT yd ). If G is nonaccumulative, then it follows that

V˙ s (x(t)) ≤ eT uc (t) − eT yc (t), tk < t < tk+1 , ∆Vs (x(tk )) ≤ eT ud (tk ) − eT yd (tk ), k ∈ Z+ .

(6.61) (6.62)

If the components uci (·), i = 1, . . . , mc , and udi (·), i = 1, . . . , md , of uc (·)

NNBook

September 1, 2009

215

HYBRID NONNEGATIVE SYSTEMS

and ud (·), respectively, denote flux and mass inputs of the hybrid system G and the components yci (·), i = 1, . . . , lc , and ydi (·), i = 1, . . . , ld , of yc (·) and yd (·), respectively, denote flux and mass outputs of the hybrid system G, then nonaccumulativity implies that the system mass transport (respectively, change in system mass) is always less than or equal to the difference between the system flux (respectively, mass) input and system flux (respectively, mass) output. Next, we show that all hybrid compartmental systems with measured outputs corresponding to material outflows are nonaccumulative. Specifically, consider (6.27) and (6.27) with wc (t) = uc (t) and wd (tk ) = ud (tk ), storage function Vs (x) = eT x, and hybrid outputs   ∂Vs T = [ac11 (x), ac22 (x), . . . , acnn (x)]T yc = Dc (x) ∂x and yd = Dd (x)



∂Vs ∂x

T

= [ad11 (x), ad22 (x), . . . , adnn (x)]T .

Now, it follows that V˙ s (x) = e

T

"

[Jcn (x) − Dc (x)]



∂Vs ∂x

T

+ uc

#

= eT uc − eT yc + eT Jcn (x)e = eT uc − eT yc , x 6∈ Zx ,

(6.63)

and "

∆Vs (x) = eT [Jdn (x) − Dd (x)]



∂Vs ∂x

T

+ ud

#

= eT ud − eT yd + eT Jdn (x)e = e T ud − e T y d , x ∈ Z x ,

(6.64)

which shows that all hybrid compartmental systems are lossless with respect to the linear supply rate (sc , sd ) = (eT wc −eT yc , eT wd −eT yd ). Alternatively, if the hybrid outputs yc and yd correspond to a partial observation of the material outflows, then it can easily be shown that the nonlinear hybrid compartmental system is dissipative with respect to the supply rate (sc , sd ) = (eT uc − eT yc , eT ud − eT yd ).

6.5 Specialization to Linear Impulsive Dynamical Systems In this section, we specialize the results of Section 6.4 to the case of linear impulsive dynamical systems. Specifically, setting fc (x) = Ac x, Gc (x) = Bc ,

NNBook

September 1, 2009

216

CHAPTER 6

hc (x) = Cc x, Jc (x) = Dc , fd (x) = (Ad − In )x, Gd (x) = Bd , hd (x) = Cd x, and Jd (x) = Dd , the nonnegative state-dependent impulsive dynamical system given by (6.27)–(6.30) specializes to x(t) ˙ ∆x(t) yc (t) yd (t)

= = = =

Ac x(t) + Bc uc (t), x 6∈ Zx , (Ad − In )x(t) + Bd ud (t), x ∈ Zx , Cc x(t) + Dc uc (t), x 6∈ Zx , Cd x(t) + Dd ud (t), x ∈ Zx ,

(6.65) (6.66) (6.67) (6.68)

where Ac ∈ Rn×n is essentially nonnegative, Bc ∈ Rn×mc , Cc ∈ Rlc ×n , Dc ∈ Rlc ×mc , Ad ∈ Rn×n is nonnegative, Bd ∈ Rn×md , Cd ∈ Rld ×n , and Dd ∈ Rld ×md . Theorem 6.7. Let qc ∈ Rlc , rc ∈ Rmc , qd ∈ Rld , and rd ∈ Rmd . Consider the linear impulsive dynamical system G given by (6.65)–(6.68) and assume that Ac is essentially nonnegative, Ad is nonnegative, Bc ≥≥ 0, Bd ≥≥ 0, Cc ≥≥ 0, Cd ≥≥ 0, Dc ≥≥ 0, and Dd ≥≥ 0. If there exist n n n mc md vectors p ∈ R+ , lc ∈ R+ , ld ∈ R+ , wc ∈ R+ , wd ∈ R+ and a scalar ε > 0 (respectively, ε = 0) such that 0 0 0 0

= = = =

T xT (AT x 6∈ Zx , c p + εp − Cc qc + lc ), T T Bc p − Dc qc − rc + wc , T xT (AT d p − p − Cd qd + ld ), x ∈ Zx , BdT p − DdT qd − rd + wd ,

(6.69) (6.70) (6.71) (6.72)

then the linear impulsive dynamical system G given by (6.65)–(6.68) is exponentially dissipative (respectively, dissipative) with respect to the linear supply rate (sc (uc , yc ), sd (ud , yd )) = (qcT yc + rcT uc , qdT yd + rdT ud ). Proof. The proof follows from Theorem 6.5 with fc (x) = Ac x, Gc (x) = Bc , hc (x) = Cc x, Jc (x) = Dc , fd (x) = (Ad − In )x, Gd (x) = Bd , hd (x) = Cd x, Jd (x) = Dd , Vs (x) = pT x, ℓc (x) = lcT x, ℓd (x) = ldT x, Wc (x) = wc , and Wd (x) = wd . For a given lc ∈ Rn , wc ∈ Rmc , ld ∈ Rn , and wd ∈ Rmd , note that if rank [M, y] = rank M , where  T    CcT qc − lc Ac + ǫI    DcT qc + rc − wc  Bc ,  , M , y , (6.73) T  A −I    CdT qd − ld d BdT DdT qd + rd − wd

then there exists p ∈ Rn such that (6.69)–(6.72) are satisfied. Now, if there exists p ∈ Rn such that inequalities p ≥≥ 0,

(6.74)

NNBook

September 1, 2009

217

HYBRID NONNEGATIVE SYSTEMS

z − M p ≥≥ 0, where

 CcT qc  DcT qc + rc  , z,   CdT qd T Dd qd + rd 

(6.75)

(6.76)

are satisfied, then there exist lc ≥≥ 0, wc ≥≥ 0, ld ≥≥ 0, and wd ≥≥ 0 such that (6.69)–(6.72) hold. Equations (6.74) and (6.75) comprise a set of 3n + mc + md linear inequalities with pi , i = 1, . . . , n, variables, and hence, the feasibility of p ≥≥ 0 such that (6.74) and (6.75) hold can be checked by standard linear matrix inequality (LMI) techniques [39]. Next, we provide sufficient conditions for the case where G given by (6.65)–(6.68) is lossless with respect to the linear supply rate (sc (uc , yc ), sd (ud , yd )) = (qcT yc + rcT uc , qdT yd + rdT ud ). Theorem 6.8. Let qc ∈ Rlc , rc ∈ Rmc , qd ∈ Rld , and rd ∈ Rmd . Consider the linear impulsive dynamical system G given by (6.65)–(6.68) and assume that Ac is essentially nonnegative, Ad is nonnegative, Bc ≥≥ 0, n Bd ≥≥ 0, Cc ≥≥ 0, Cd ≥≥ 0, Dc ≥≥ 0, and Dd ≥≥ 0. If there exist p ∈ R+ such that 0 0 0 0

= = = =

T xT (AT x 6∈ Zx , c p − Cc qc ), T T Bc p − Dc qc − rc , T xT (AT d p − p − Cd qd ), x ∈ Zx , BdT p − DdT qd − rd ,

(6.77) (6.78) (6.79) (6.80)

then the linear impulsive dynamical system G given by (6.65)–(6.68) is lossless with respect to the linear supply rate (sc (uc , yc ), sd (ud , yd )) = (qcT yc + rcT uc , qdT yd + rdT ud ). Proof. The proof follows from Theorem 6.6 with fc (x) = Ac x, Gc (x) = Bc , hc (x) = Cc x, Jc (x) = Dc , fd (x) = (Ad − In )x, Gd (x) = Bd , hd (x) = Cd x, Jd (x) = Dd , and Vs (x) = pT x.

6.6 Feedback Interconnections of Nonlinear Hybrid Nonnegative Dynamical Systems In this section, we consider Lyapunov stability and asymptotic stability of feedback interconnections of hybrid nonnegative dynamical systems. We begin by considering the nonlinear impulsive hybrid dynamical system G given by (6.27)–(6.30) with the nonlinear impulsive nonnegative feedback

NNBook

September 1, 2009

218

CHAPTER 6

system Gc given by (xc (t), ucc (t)) 6∈ Zc , (6.81) ∆xc (t) = fdc (xc (t)) + Gdc (xc (t))udc (t), (xc (t), ucc (t)) ∈ Zc , (6.82) (6.83) ycc (t) = hcc (xc (t)), (xc (t), ucc (t)) 6∈ Zc , ydc (t) = hdc (xc (t)), (xc (t), ucc (t)) ∈ Zc , (6.84) x˙ c (t) = fcc (xc (t)) + Gcc (xc (t))ucc (t),

nc

xc (0) = xc0 ,

mcc

where t ≥ 0, xc (t) ∈ R+ , ∆xc (t) = xc (t+ ) − xc (t), ucc (t) ∈ Ucc ⊆ R+ , mdc udc (tk ) ∈ Udc ⊆ R+ , tk denotes the kth instant of time at which nc (xc (t), ucc (t)) intersects Zc ⊂ R+ × Ucc for a particular trajectory xc (t) △

lcc

ldc

and input ucc (t), ycc (t) ∈ Ycc ⊆ R+ , ydc (tk ) ∈ Ydc ⊆ R+ , fcc : Rnc → Rnc is Lipschitz continuous and is essentially nonnegative, Gcc : Rnc → Rnc ×mcc nc and satisfies Gcc (xc ) ≥≥ 0, xc ∈ R+ , fdc : Rnc → Rnc is continuous nc and is such that xc + fdc (xc ) is nonnegative for all xc ∈ R+ , Gdc : nc Rnc → Rnc ×mdc and satisfies Gdc (xc ) ≥≥ 0, xc ∈ R+ , hcc : Rnc → Rlcc nc and satisfies hcc (xc ) ≥≥ 0, xc ∈ R+ , hdc : Rnc → Rldc and satisfies nc hdc (xc ) ≥≥ 0, xc ∈ R+ , mcc = lc , mdc = ld , lcc = mc , ldc = md , and nc Zc , Zcxc × Zcucc ⊂ R+ × Ucc .

Here, we assume that ucc (·) ∈ Uc+ and udc (·) ∈ Ud+ are restricted to the class of admissible inputs consisting of measurable functions such △ that (ucc (t), udc (tk )) ∈ Ucc × Udc for all t ≥ 0 and k ∈ Z[0,t) = {k : 0 ≤ tk < t}, where the constraint set Ucc × Udc is given with (0, 0) ∈ Ucc × Udc . Furthermore, we assume that the set Zc = {(xc , ucc ) : Xc (xc , ucc ) = 0}, nc where Xc : R+ × Ucc → R. In addition, we assume that the system functions fcc (·), fdc (·), Gcc (·), Gdc (·), hcc (·), and hdc (·) are continuously differentiable mappings. Finally, for the nonlinear dynamical system (6.81) we assume that the required properties for the existence and uniqueness of solutions are satisfied such that (6.81) has a unique solution for all t ∈ R [16, 189]. Note that with the positive feedback interconnection given by Figure 6.2, (ucc , udc ) = (yc , yd ) and (ycc , ydc ) = (uc , ud ). Furthermore, even though the input-output pairs of the feedback interconnection shown on Figure 6.2 consist of two vector inputs/two vector outputs, at any given instant of time a single vector input/single vector output is active. Next, we define the closed-loop resetting set Z˜x˜ , Zx × Zcxc ∪ {(x, xc ) : (hc (x) + Jc (x)hcc (xc ), hcc (xc )) ∈ Zcucc × Zuc }. (6.85) Note that since the positive feedback interconnection of G and Gc is well posed, it follows that Z˜x˜ is well defined and depends on the closed-loop T states x ˜ , [xT , xT c ] . As in Section 6.2, here we assume that the solution s(t, x ˜0 ) to the dynamical system resulting from the feedback interconnection

NNBook

September 1, 2009

219

HYBRID NONNEGATIVE SYSTEMS

+

-

G

Gc



+

Figure 6.2 Feedback interconnection of G and Gc .

of G and Gc is such that Assumption 6.1 is satisfied. The following theorem gives sufficient conditions for Lyapunov and asymptotic stability of the positive feedback interconnection given by Figure 6.2. For the statement of this result let Txc0 ,uc denote the set of resetting times of G, let Tx0 ,uc denote the complement of Txc0 ,uc , that is, Tx0 ,uc = [0, ∞) \ Txc0 ,uc , let Txcc0 ,ucc denote the set of resetting times of Gc , and let Txc0 ,ucc denote the complement of Txcc0 ,ucc , that is, Txc0 ,ucc = [0, ∞) \ Txcc0 ,ucc . Theorem 6.9. Let qc ∈ Rlc , rc ∈ Rmc , qd ∈ Rld , rd ∈ Rmd , qcc ∈ rcc ∈ Rmcc , qdc ∈ Rldc , rdc ∈ Rmdc . Consider the nonlinear impulsive nonnegative dynamical systems G and Gc given by (6.27)–(6.30) and (6.81)– (6.84), respectively. Assume G and Gc are dissipative with respect to Ty + the linear hybrid supply rates (qcT yc + rcT uc , qdT yd + rdT ud ) and (qcc cc T T T rcc ucc , qdc ydc +rdc udc ) and with continuously differentiable, positive-definite storage functions Vs (·) and Vsc (·), respectively, such that Vs (0) = 0 and Vsc (0) = 0. Furthermore, assume there exists a scalar σ > 0 such that qc + σrcc ≤≤ 0, rc + σqcc ≤≤ 0, qd + σrdc ≤≤ 0, and rd + σqdc ≤≤ 0. Then the following statements hold:

Rlcc ,

i ) The positive feedback interconnection of G and Gc is Lyapunov stable. ii ) If G and Gc are strongly zero-state observable and qc + σrcc V (sx (t0 , u(t0 ))) + V (sx (t1 , u(t1 ))),

which is a contradiction. Hence, sx (t, u(t)) ∈ M, t ∈ [t0 , t1 ]. It is important to note that since V : D → R in Theorem 7.1 is not sign definite, Theorem 7.1 also holds for the case where the inequality in (7.4) is reversed. The following corollary to Theorem 7.1 is immediate. Corollary 7.1. Consider the dynamical system G defined on D. Let R : D → D be an involutive operator, let M ⊂ D, and let sx (·, u(·)) ∈ W[t0 ,t1 ] ,

NNBook

September 1, 2009

231

SYSTEM THERMODYNAMICS

where u(·) ∈ U . Assume there exists a continuous function V : D → R such that V (x) = V (Rx), x ∈ D, and for sx (t, u(t)) 6∈ M, t ∈ [t1 , t2 ], V (s(t, x0 , u(·))) is a strictly increasing (respectively, decreasing) function of time. If sx (·, u(·)) is an R-reversible trajectory of G, then sx (t, u(t)) ∈ M, t ∈ [t0 , t1 ]. Proof. The proof is a direct consequence of Theorem 7.1 with r(u, y) ≡ 0 and the fact that Theorem 7.1 also holds for the case when the inequality in (7.4) is reversed. It follows from Corollary 7.1 that if, for a given dynamical system G, there exists an R-reversible trajectory of G, then there does not exist a function of the state of the system that strictly decreases or strictly increases in time on any trajectory of G lying in M. In this case, the existence of a completely ordered time set having a topological structure involving a closed set homeomorphic to the real line cannot be established. Such systems, which include lossless Newtonian and Hamiltonian systems, are time-reversal symmetric and hence lack an inherent time direction. However, that is not the case with thermodynamic systems. Next, we present a notion of state recoverability of a dynamical system G. Definition 7.4. Consider the dynamical system G defined on D. Let r : U × Y → R, and let sx (·, u(·)) ∈ W[t0 ,t1 ] , where u(·) ∈ U . sx (·, u(·)) is a recoverable trajectory of G if there exist u− (·) ∈ U and t2 > t1 such that u− : [t1 , t2 ] → U , s(t2 , sx (t1 , u(t1 )), u− (t2 )) = sx (t0 , u(t0 )),

(7.8)

and Z

t1

r(u(t), y(t))dt + t0

Z

t2

r(u− (t), y − (t))dt = 0,

(7.9)

t1

where y − (·) denotes the read-out map for the trajectory s(·, sx (t1 , u(t1 )), u− (·)). Furthermore, G is a state recoverable dynamical system if for every x ∈ D, sx (·, u(·)) is a recoverable trajectory of G. It follows from the definition of state recoverability that the way in which the initial dynamical system state is restored may be chosen freely so long as (7.9) is satisfied. Hence, unlike R-state reversibility, it is not necessary for the dynamical system to recover the initial state of the system through an involutive transformation of the system trajectory. Furthermore, unlike state reversibility, it is not necessary for the dynamical system to retrace every stage of the original trajectory in the reverse order.

NNBook

September 1, 2009

232

CHAPTER 7

However, condition (7.9) ensures that the recoverable process completely restores the original dynamic state and at the same time restores the energy supplied by the environment back to its original condition. This notion of recoverability is closely related to Planck’s notion of complete reversibility, wherein the initial system state is restored in the totality of nature (“die gesamte Natur”). The following result provides a sufficient condition for the existence of a recoverable trajectory of a nonlinear dynamical system G, and hence, establishes sufficient conditions for state recoverability of G. Theorem 7.2. Consider the dynamical system G defined on D. Let ∈ W[t0 ,t1 ] , where u(·) ∈ U . Assume there exist a continuous function V : D → R and a function r : U × Y → R such that for every x ∈ D and all tˆ0 , tˆ1 , t0 ≤ tˆ0 < tˆ1 ≤ t1 , Z tˆ1 r(u(t), y(t))dt. (7.10) V (sx (tˆ1 , u(tˆ1 ))) ≥ V (sx (tˆ0 , u(tˆ0 ))) +

sx (·, u(·))

tˆ0

Furthermore, assume there exists M ⊂ D such that for all tˆ0 , tˆ1 , t0 ≤ tˆ0 < tˆ1 ≤ t1 , and sx (t, u(t)) 6∈ M, t ∈ [tˆ0 , tˆ1 ], (7.10) holds as a strict inequality. If sx (·, u(·)) is a recoverable trajectory of G, then sx (t, u(t)) ∈ M, t ∈ [t0 , t1 ]. Proof. Let sx (·, u(·)) ∈ W[t0 ,t1 ] , where u(·) ∈ U , be a recoverable trajectory of G so that there exist u− (·) ∈ U and t2 > t1 such that s(t2 , sx (t1 , u(t1 )), u− (t2 )) = sx (t0 , u(t0 )). Suppose, ad absurdum, there exists t ∈ [t0 , t1 ] such that sx (t, u(t)) 6∈ M. Now, it follows that there exists an interval [tˆ0 , tˆ1 ] ⊂ [t0 , t1 ] such that for t0 ≤ tˆ0 < tˆ1 ≤ t1 , Z tˆ1 x ˆ x ˆ ˆ ˆ r(u(t), y(t))dt, (7.11) V (s (t1 , u(t1 ))) > V (s (t0 , u(t0 ))) + tˆ0

which further implies that V (sx (t1 , u(t1 ))) > V (sx (t0 , u(t0 ))) +

Z

t1

r(u(t), y(t))dt.

(7.12)

t0

Next, it follows from (7.10) with t2 > t1 that V (s(t2 , sx (t1 , u(t1 )), u− (t2 ))) ≥ V (s(t1 , sx (t1 , u(t1 )), u− (t1 ))) Z t2 (7.13) + r(u− (t), y − (t))dt. t1

Now, adding (7.12) and (7.13), using the definition of s(t2 , sx (t1 , u(t1 ), u− (t2 ))), and using (7.9) yields V (sx (t0 , u(t0 ))) + V (sx (t1 , u(t1 ))) > V (sx (t0 , u(t0 ))) + V (sx (t1 , u(t1 ))), which is a contradiction. Hence, sx (t, u(t)) ∈ M, t ∈ [t0 , t1 ].

NNBook

September 1, 2009

233

SYSTEM THERMODYNAMICS

The following corollary to Theorem 7.2 is immediate. Corollary 7.2. Consider the dynamical system G defined on D. Let M ⊂ D, and let sx (·, u(·)) ∈ W[t0 ,t1 ] , where u(·) ∈ U . Assume there exists a continuous function V : D → R such that for sx (t, u(t)) 6∈ M, t ∈ [t0 , t1 ], V (s(t, x0 , u(·)) is a strictly increasing (respectively, decreasing) function of time. If sx (·, u(·)) is a recoverable trajectory of G, then sx (t, u(t)) ∈ M, t ∈ [t0 , t1 ]. Proof. The proof is a direct consequence of Theorem 7.2 with r(u, y) ≡ 0 and the fact that Theorem 7.2 also holds for the case when the inequality in (7.10) is reversed. As in the case of R-state reversibility and state reversibility, state recoverability can be used to establish a connection between a dynamical system evolving on a manifold M ⊂ D and the arrow of time. However, in the case of state recoverability, the recoverable dynamical system trajectory need not involve an involutive transformation of the system trajectory, nor is it required to retrace the original system trajectory in recovering the original dynamic state. It should be noted here that state recoverability is not implied by the concepts of reachability and controllability, which play a central role in control theory [116]. For example, one might envision, albeit with a considerable stretch of the imagination, perfectly controlled inputs that could reassemble a broken egg or even fuse water into solid cubes of ice. However, in all such cases, an external source of energy from the environment would be required to operate such an immaculate state recoverable mechanism and would violate condition (7.9). Clearly, state recoverability is a weaker notion than that of state reversibility since state reversibility implies state recoverability; the converse, however, is not generally true. Conversely, state irrecoverability is a logically stronger notion than state irreversibility since state irrecoverability implies state irreversibility. However, as we see in Section 7.6, these notions are equivalent for thermodynamic systems.

7.4 Reversible Dynamical Systems, Volume-Preserving Flows, e Recurrence and Poincar´ The notion of R-state reversibility introduced in Section 7.3 is one of the fundamental symmetries that arises in natural science. This notion can also be characterized by the flow of a dynamical system. In particular, consider the dynamical system given by x(t) ˙ = f (x(t)),

x(t0 ) = x0 ,

t ∈ I x0 ,

(7.14)

NNBook

September 1, 2009

234

CHAPTER 7

where x(t) ∈ D ⊆ Rq , t ∈ Ix0 , is the system state vector, D is an open subset of Rq , f : D → Rq is locally Lipschitz continuous on D, and Ix0 = [t0 , τx0 ), t0 < τx0 ≤ ∞, is the maximal interval of existence for the solution x(·) of (7.14). Note that since f (·) is locally Lipschitz continuous on D, it follows from Theorem 3.1 of [127, p. 18] that the solution to (7.14) is unique for every initial condition in D and jointly continuous in t and x0 . In this case, the semigroup property s(t + τ, x0 ) = s(t, s(τ, x0 )), t, τ ∈ Ix0 , and the continuity of s(t, ·) on D, t ∈ Ix0 , hold. Given t ∈ R, we denote the flow s(t, ·) : D → D of (7.14) by st (x0 ) for x0 ∈ D, and given x ∈ D, we denote the trajectory s(·, x) : R → D of (7.14) by sx (t). Now, in terms of the flow st : D → D of (7.14), the consistency and semigroup properties of (7.14) can be equivalently written as s0 (x0 ) = x0 and (sτ ◦ st )(x0 ) = sτ (st (x0 )) = st+τ (x0 ), where “◦” denotes the composition operator. Next, it follows from continuity of solutions and the semigroup property that the map st : D → D is a continuous function with a continuous inverse s−t . Thus, st , t ∈ Ix0 , generates a one-parameter family of homeomorphisms on D forming a commutative group under composition. To show that R-state reversibility can be characterized by the flow of (7.14), let R : D → D be a continuous map of (7.14) such that ˙ R(x(t)) = −f (R(x(t))), R(x(t0 )) = R(x0 ), t ∈ IR(x0 ) . (7.15) Now, it follows from (7.15) that R ◦ st = s−t ◦ R,

t ∈ I x0 .

(7.16)

Condition (7.16), with R(·) satisfying (7.15), defines an R-reversed trajectory of (7.14) in the sense of Definition 7.2 with τ (t) = t. In the context of classical mechanics involving the configuration manifold (space of generalized positions) Q = Rn , with governing equations given by   ∂H(q(t), p(t)) T , q(t0 ) = q0 , t ≥ t0 , q(t) ˙ = (7.17) ∂p(t)   ∂H(q(t), p(t)) T , p(t0 ) = p0 , p(t) ˙ = − (7.18) ∂q(t)

where q ∈ Rn denotes generalized system positions, p ∈ Rn denotes generalized system momenta, H : Rn × Rn → R is the system Hamiltonian given by H(q, p) , q˙T p − L(q, q), ˙ L(q, q) ˙ is the system Lagrangian,5 and T  q) ˙ , the reversing symmetry R : Rn × Rn → Rn × Rn p(q, q) ˙ , ∂L(q, ∂ q˙ is such that R(q, p) = (q, −p) and satisfies (7.15). In this case, R is an 5 Here we assume that the system Lagrangian is hyperregular [199] so that the map from the generalized velocities q˙ to the generalized momenta p is bijective (i.e., one-to-one and onto).

NNBook

September 1, 2009

235

SYSTEM THERMODYNAMICS

involution. This implies that if (q(t), p(t)), t ≥ t0 , is a solution to (7.17) and (7.18), then (q(−t), −p(−t)), t ≥ t0 , is also a solution to (7.17) and (7.18) with initial condition (q0 , −p0 ). In the configuration space this clearly shows the time-reversal nature of lossless mechanical systems. Reversible dynamical systems tend to exhibit a phenomenon known as Poincar´e recurrence [8]. Poincar´e recurrence states that if a dynamical system has a fixed total energy that restricts its dynamics to bounded subsets of its state space, then the dynamical system will eventually return arbitrarily close to its initial system state infinitely often. More precisely, Poincar´e [243] established the fact that if the flow of a dynamical system preserves volume and has only bounded orbits, then for each open set there exist orbits that intersect the set infinitely often. In order to state the Poincar´e recurrence theorem, the following definitions are needed. Definition 7.5. Let V ⊂ Rq be a bounded set. The volume Vvol of V is defined as Z Vvol , dV, (7.19) V

where the integration in (7.19) is the Lebesgue integral over V. Definition 7.6. Let V ⊂ Rq be a bounded set. A map g : V → Q, where Q ⊂ Rq , is volume preserving if for every V0 ⊂ V, the volume of g(V0 ) is equal to the volume of V0 . The following theorem, known as Liouville’s theorem [8], establishes sufficient conditions for volume-preserving flows. For the statement of this theorem, consider the nonlinear dynamical system (7.14) and define the divergence of f = [f1 , . . . , fq ]T : D → Rq by ∇ · f (x) ,

q X ∂fi (x) i=1

∂xi

,

(7.20)

where ∇ denotes the nabla operator, “ · ” denotes the dot product in Rq , and xi denotes the ith component of x. Theorem 7.3. Consider the nonlinear dynamical system (7.14). If ∇ · f (x) ≡ 0, then the flow st : D → D of (7.14) is volume preserving. Proof. Let V ⊂ Rq be a compact set such that its image at time t under the mapping st (·) is given by st (V). In addition, let dSV denote an infinitesimal surface element of the boundary of the set V and let n ˆ (z), z ∈ ∂V, denote an outward normal vector to the boundary of V. Then the

NNBook

September 1, 2009

236

CHAPTER 7

change in volume of st (V) at t = t0 is given by Z dst (V)vol = (f (x) · n ˆ (x))dtdSV ,

(7.21)

∂V

which, using the divergence theorem, implies that Z Z dst (V)vol = (f (x) · n ˆ (x))dSV = ∇ · f (x)dV. dt ∂V V t=t0

(7.22)

Hence, if ∇ · f (x) ≡ 0, then st (·) is a volume-preserving map.

Volume preservation is the key conservation law underlying statistical mechanics. The flows of volume-preserving dynamical systems belong to one of the Lie pseudogroups6 of diffeomorphisms. These systems arise in incompressible fluid dynamics, classical mechanics, and acoustics. Next, we state the well-known Poincar´e recurrence theorem. For this result, let g(n) (x), n ∈ Z+ , denote the n-time composition operator of g(x) with itself and define g(0) (x) , x. Theorem 7.4. Let D ⊂ Rq be an open bounded set, and let g : D → D be a continuous, volume-preserving bijective (one-to-one and onto) map. Then for every open set N ⊂ D, there exists n ∈ Z+ such that g(n) (N )∩N 6= Ø. Furthermore, there exists a point x ∈ N which returns to N , that is, g(n) (x) ∈ N for some n ∈ Z+ . Proof. The proof of this result is standard; see for example [8, p. 72]. For completeness of exposition, however, we provide a proof here. First, note that the images g(p) (N ), p ∈ Z+ , under the mapping g(·) of the neighborhood N ⊂ D have the same volume and are all contained in D. Next, define the union of all the images of N by V ,

∞ [

p=0

g(p) (N ) ⊂ D.

(7.23)

Since the volume of a union of disjoint sets is the sum of the individual set volumes, it follows that if g(p) (N ), p ∈ Z+ , are disjoint, then Vvol = ∞. However, V ⊂ D and D is a bounded set by assumption. Hence, there exist k, l ∈ Z+ , with k > l, such that g(k) (N ) ∩ g(l) (N ) 6= Ø. Now, applying the inverse g(−1) to this relation l times and using the fact that g(·) is a bijective map, it follows that g(k−l) (N ) ∩ N 6= Ø. Thus, g(n) (N ) ∩ N 6= Ø, where n = k − l. Hence, there exists a point x ∈ N such that g(n) (x) ∈ g(n) (N ) ∩ N ⊆ N . 6 A Lie group is a topological group that can be given an analytic structure such that the group operation and inversion are analytic. A Lie pseudogroup is an infinite-dimensional counterpart of a Lie group.

NNBook

SYSTEM THERMODYNAMICS

September 1, 2009

237

The next result establishes the existence of a point x in D ⊂ Rq such that limi→∞ g(ni ) (x) = x for some sequence {ni }∞ i=1 , with ni → ∞ as i → ∞, under a continuous, volume-preserving bijective mapping g(·) which maps a bounded region D of a Euclidean space onto itself. Hence, x returns infinitely often to any open neighborhood of itself under the mapping g(·). Theorem 7.5. Let D ⊂ Rq be an open bounded set, and let g : D → D be a continuous, volume-preserving bijective map. Then for every open neighborhood N ⊂ D, there exists a point x ∈ N such that limi→∞ g(ni ) (x) = x for some sequence {ni }∞ i=1 , with ni → ∞ as i → ∞. Hence, x ∈ N returns to N infinitely often, that is, there exists a sequence (ni ) (x) ∈ N for all i ∈ Z . {ni }∞ + i=1 , with ni → ∞ as i → ∞, such that g Proof. Let N ⊂ D be an open set, and let N1 , Bδ1 (x1 ) be such that N 1 ⊂ N for some δ1 > 0 and x1 ∈ N . Applying Theorem 7.4, with g(·) replaced by g(−1) (·), it follows that there exists n1 ∈ Z+ such that g(−n1 ) (N1 ) ∩ N1 6= Ø, which implies that g(−n1 ) (N 1 ) ∩ N 1 6= Ø. Now, let N2 = Bδ2 (x2 ) be such that N 2 ⊂ g(−n1 ) (N1 ) ∩ N1 for some δ2 > 0 and x2 ∈ g(−n1 ) (N1 ) ∩ N1 . Repeating the above arguments it follows that there exists n2 ∈ Z+ , n2 > n1 , such that g(−n2 ) (N2 ) ∩ N2 6= Ø and g(−n2 ) (N 2 ) ∩ N 2 6= Ø. Repeating this process recursively, it follows that there exist sequences ∞ {ni }∞ i=1 and {δi }i=1 , with ni → ∞ as i → ∞, δi → 0 as i → ∞, and δi > δi+1 , i = 1, 2, . . ., such that Ni ⊃ Ni+1 , i = 1, 2, . . ., and g(−ni ) (Ni ) ∩ Ni 6= Ø, where Ni = Bδi (xi ) for some xi ∈ g(−ni−1 ) (Ni−1 ) ∩ Ni−1 and where n0 , 0 and N0 , N . Now, since Ni 6= Ø, i ∈ T Z+ , it follows from the Cantor intersection theorem [7, p.56] that Z , ∞ i=1 N i 6= Ø. Furthermore, since δi → 0 as i → ∞, it follows that Z is a singleton. Next, let x ∈ Z = {x}, and since for every i ∈ Z+ , N i+1 ⊂ Ni , it follows that x ∈ Ni , i ∈ Z+ . Now, note that x ∈ Ni+1 ⊂ g(−ni ) (Ni ) ∩ Ni for all i ∈ Z+ , which implies that g(ni ) (z) ∈ Ni , i ∈ Z+ . Hence, since δi → 0 as i → ∞, it follows that limi→∞ g(ni ) (x) = x. The next theorem strengthens Poincar´e’s theorem by showing that for every open neighborhood N of D ⊂ Rq , there exists a subset of N that is dense7 in N so that almost every moving point in N returns repeatedly to the vicinity of its initial position under a continuous, volume-preserving bijective mapping which maps the bounded region D onto itself. Theorem 7.6. Let D ⊂ Rq be an open bounded set, and let g : D → D be a continuous, volume-preserving bijective map. Then for every open neighborhood N ⊂ D, there exists a dense subset V ⊂ N such that for every point z ∈ V, limi→∞ g(ni ) (x) = x for some sequence {ni }∞ i=1 , with 7 We say that V is dense in N if and only if N is contained in the closure of V; that is, V ⊆ N is dense in N if and only if N ⊆ V.

NNBook

September 1, 2009

238

CHAPTER 7

ni → ∞ as i → ∞. Proof. Let N ⊂ D be an open neighborhood and define V ⊂ N by V , {x ∈ N : there exists a sequence {ni }∞ i=1 , with ni → ∞ as i → ∞, such that limi→∞ g(ni ) (x) = x}.

(7.24)

and let {δi }∞ i=1 be a strictly decreasing → ∞ and Bδ1 (x) ⊂ N . It follows from

Now, let x ∈ N positive sequence with δi → 0 as i Theorem 7.5 that for every i ∈ Z+ , there exists xi ∈ Bδi (x) such that limk→∞ g(nk ) (xi ) = xi for some sequence {nk }∞ k=1 , with nk → ∞ as k → ∞, which implies that xi ∈ V, i ∈ Z+ . Next, since limi→∞ xi = x, it follows that x ∈ V, which implies that V ⊆ N ⊂ V, and hence, V is a dense subset of N . It follows from Theorem 7.6 that almost every point in D ⊂ Rq will return infinitely many times to any open neighborhood of itself under a continuous, volume-preserving bijective mapping which maps a bounded region D of a Euclidean space onto itself. The following theorem provides several equivalent statements for establishing Poincar´e recurrence. Theorem 7.7. Let D ⊂ Rq be an open bounded set, and let g : D → D be a continuous, bijective map. Then the following statements are equivalent: i ) For every open set N ⊂ D, there exists a dense subset V ⊂ N such that, for every point z ∈ V, limi→∞ g(ni ) (x) = x for some sequence {ni }∞ i=1 , with ni → ∞ as i → ∞. ii ) For every open set N ⊂ D, there exists a point x ∈ N such that limi→∞ g(ni ) (x) = x for some sequence {ni }∞ i=1 , with ni → ∞ as i → ∞. iii ) For every open set N ⊂ D, there exists a point x ∈ N which returns to N infinitely often, that is, g(ni ) (x) ∈ N , i ∈ Z+ , for some sequence {ni }∞ i=1 , with ni → ∞ as i → ∞. iv ) For every open set N ⊂ D, there exists a point x ∈ N which returns to N , that is, g(n) (x) ∈ N for some n ∈ Z+ . v ) For every open set N ⊂ D, there exists n ∈ Z+ such that g(n) (N )∩N 6= Ø. Proof. The implication i) implies ii) follows trivially and the proof of ii) implies i) is identical to that of Theorem 7.6. The implications ii) implies iii), iii) implies iv), and iv) implies v) follow trivially. The proof of v) implies ii) is identical to that of Theorem 7.5.

NNBook

September 1, 2009

239

SYSTEM THERMODYNAMICS

Note that it follows from Theorems 7.4, 7.5, and 7.6 that a continuous, bijective map g : D → D exhibits Poincar´e recurrence (that is, one of the statements in Theorem 7.7 holds) if g(·) is volume preserving. For the remainder of this section we consider the nonlinear dynamical system (7.14) and assume that the solutions to (7.14) are defined for all t ∈ R. Recall that if all solutions to (7.14) are bounded, then it follows from the Peano-Cauchy theorem [127, pp. 16, 17] that Ix0 = R. The following theorem shows that if a dynamical system preserves volume, then almost all trajectories return arbitrarily close to their initial position infinitely often. Theorem 7.8. Consider the nonlinear dynamical system (7.14). Assume that the flow st : D → D of (7.14) is volume preserving and maps an open bounded set Dc ⊂ Rq onto itself, that is, Dc is an invariant set with respect to (7.14). Then the nonlinear dynamical system (7.14) exhibits Poincar´e recurrence, that is, almost every point x ∈ Dc returns to every open neighborhood N ⊂ Dc of x infinitely many times. Proof. Since f : D → Rq is locally Lipschitz continuous on D and st (·) maps an open bounded set Dc ⊂ Rn onto itself, it follows that the solutions to (7.14) are bounded and unique for all t ∈ R and x0 ∈ Dc . Thus, the mapping st (·) is bijective. Furthermore, since the solutions of (7.14) are continuously dependent on the system’s initial conditions, it follows that st (·) is continuous. Now, the result follows as a direct consequence of Theorem 7.6 with g(·) = st (·) for every t ≥ t0 . It follows from Theorem 7.8 that a nonlinear dynamical system exhibits Poincar´e recurrence if one of the statements in Theorem 7.7 holds with g(·) = st (·) for every t ≥ t0 . Note that in this case it follows from ii) of Theorem 7.7 that Poincar´e recurrence is equivalent to the existence of a point x ∈ N ⊂ Dc such that x belongs to its positive limit set ω(x), that is, x ∈ ω(x). All Hamiltonian dynamical systems of the form (7.17) and (7.18) exhibit Poincar´e recurrence since they possess volume-preserving flows and are conservative in the sense that the Hamiltonian function H(q, p) remains constant along system trajectories. To see this, note that with x , [q T , pT ]T , (7.17) and (7.18) can be rewritten as x(t) ˙ =J



T ∂H (x(t)) , ∂x

x(t0 ) = x0 ,

t ≥ t0 ,

(7.25)

T 2n and where x0 , [q0T , pT 0] ∈R

J ,



0n In −In 0n



.

(7.26)

NNBook

September 1, 2009

240

CHAPTER 7

Now, since ˙ H(x) =



  T ∂H ∂H (x) J (x) = 0, ∂x ∂x

x ∈ R2n ,

(7.27)

the Hamiltonian function H(·) is conserved along the flow of (7.25). If H(·) is bounded from below and is radially unbounded, then every trajectory of the Hamiltonian system (7.25) is bounded. Hence, by choosing the bounded region D , {x ∈ R2n : H(x) ≤ η}, where η ∈ R and η > 0, it follows that the flow st (·) of (7.25) maps the bounded region D onto itself. Since η > 0 is arbitrary, the region D can be chosen arbitrarily large. Furthermore, since (7.25) possesses unique solutions over R, it follows that the mapping st (·) is one-to-one and onto. Moreover,  T X n n ∂H ∂ 2 H(q, p) X ∂ 2 H(q, p) (x) ∇·J = − = 0, x ∈ R2n , (7.28) ∂x ∂qi ∂pi ∂pi ∂qi i=1

i=1

which, by Theorem 7.3, shows that the flow st (·) of (7.25) is volume preserving. Finally, since the flow st (·) of (7.25) is volume preserving, continuous, and bijective, and st (·) maps a bounded region of a Euclidean space onto itself, it follows from Theorem 7.8 that the Hamiltonian dynamical system (7.25) exhibits Poincar´e recurrence. That is, in every open neighborhood N of every point x0 ∈ R2n there exists a point y ∈ N such that the trajectory s(t, y), t ≥ t0 , of (7.25) will return to N infinitely many times. Poincar´e recurrence has been the main source for the long and fierce debate between the microscopic and macroscopic points of view of thermodynamics [116]. In thermodynamic models predicated on statistical mechanics, an isolated dynamical system will return arbitrarily close to its initial state of molecular positions and velocities infinitely often. If the system entropy is determined by the state variables, then it must also return arbitrarily close to its original value, and hence, undergo cyclical changes. This apparent contradiction between the behavior of a mechanical system of particles and the second law of thermodynamics remains one of the hardest and most controversial problems in statistical physics. The resolution of this paradox lies in the controversial statement that as system dimensionality increases, the recurrence time increases at an extremely fast rate. Nevertheless, the shortcoming of the mechanistic world view of thermodynamics is the absence of the emergence of damping in lossless mechanical systems. The emergence of damping is, however, ubiquitous in isolated8 thermodynamic systems. Hence, the development of a viable 8A

key distinction between thermodynamics and mechanics is that thermodynamics is a theory of open systems, whereas mechanics is a theory of closed systems. The notions, however, of open and closed systems are different in thermodynamics and dynamical system theory. In particular, thermodynamic systems exchange matter and energy with the environment, and hence, interact

NNBook

September 1, 2009

241

SYSTEM THERMODYNAMICS

dynamical system model for thermodynamics must guarantee the absence of Poincar´e recurrence. The next set of results presents sufficient conditions for the absence of Poincar´e recurrence for the nonlinear dynamical system (7.14). For these results define the set of equilibria for the nonlinear dynamical system (7.14) in D by Me , {x ∈ D : f (x) = 0}. Theorem 7.9. Consider the nonlinear dynamical system (7.14) and assume that D \ Me 6= Ø. Assume that there exists a continuous function V : D → R such that for every x0 ∈ D \ Me , V (s(t, x0 )), t ≥ t0 , is a strictly increasing (respectively, decreasing) function of time. Then the nonlinear dynamical system (7.14) does not exhibit Poincar´e recurrence on D \ Me . That is, for some x ∈ D \ Me , there exists a neighborhood N ⊂ D \ Me such that for every y ∈ N , y 6∈ ω(y). Proof. Suppose, ad absurdum, there exists z ∈ D \ Me such that for every open neighborhood N containing x, there exists a point y ∈ N such that y ∈ ω(y). Now, let {ti }∞ i=1 be such that ti → ∞ as i → ∞ and s(ti , y) → y as i → ∞. Since V (·) is continuous, it follows that limi→∞ V (s(ti , y)) = V (y). However, since V (s(·, y)) is strictly increasing, it follows that V (s(ti , y)) > V (y), i ∈ Z+ , which is a contradiction. The proof for the case where V (s(t, x0 )), t ≥ t0 , is strictly decreasing is identical. For the remainder of this section let Dc ⊆ D be a closed invariant set with respect to the nonlinear dynamical system (7.14). The following definition for convergence is needed. Definition 7.7. The nonlinear dynamical system (7.14) is convergent with respect to Dc if limt→∞ s(t, x) exists for every x ∈ Dc . If the system (7.14) is convergent with respect to Dc , then the ω-limit set ω(x) of (7.14) for the trajectory sx (t) starting at x ∈ Dc is a singleton. Furthermore, it follows from continuity of solutions that for every h ≥ 0, dsh (ω(x)) = 0 and hence sh (ω(x)) , limt→∞ s(t + h, x) = ω(x). Thus, dh h=0

ω(x) is an equilibrium point of (7.14) for all x ∈ Dc . The next result relates the continuity of the function ω(·) at a point x to the stability of the equilibrium point ω(x). with the environment. Such systems are called open systems in the thermodynamic literature. Systems that exchange heat (energy) but not matter with the environment are called closed, whereas systems that do not exchange energy and matter with the environment are called isolated. Alternatively, in mechanics it is always possible to include interactions with the environment (via feedback interconnecting components) within the system description to obtain an augmented closed system in the sense of dynamical system theory. That is, the system can be described by an evolution law with, possibly, an output equation wherein past trajectories define the future trajectory uniquely and the system output depends on the instantaneous (present) value of the system state.

NNBook

September 1, 2009

242

CHAPTER 7

Proposition 7.1. Suppose the nonlinear dynamical system (7.14) is convergent with respect to Dc . If ω(x) is a Lyapunov stable equilibrium point for some x ∈ Dc , then ω : Dc → Dc is continuous at x. Proof. A proof of this result appears in [31]. For completeness of exposition, we provide a proof here. Suppose ω(x) is Lyapunov stable for some x ∈ Dc , and let Nε be an open neighborhood of ω(x). Moreover, choose open neighborhoods N and Nδ of ω(x) such that N ⊂ Nε and st (Nδ ) ⊆ N for all t ≥ t0 , and let {xi }∞ n=1 be a sequence in Dc converging to x. The existence of such neighborhoods follows from the Lyapunov stability of ω(x). Next, there exists h > 0 such that s(h, x) ∈ Nδ and, since the solutions to (7.14) are continuously dependent on the system initial conditions, it follows that there exists an open neighborhood Nδˆ , Bδˆ(x), δˆ > 0, of x such that s(h, y) ∈ Nδ for all y ∈ Nδˆ. Furthermore, it follows from the Lyapunov stability of ω(x) that s(t + h, y) ∈ N , y ∈ Nδˆ, t ≥ 0, and hence, ω(y) ∈ N ⊂ Nε , y ∈ Nδˆ, which proves that ω : Dc → Dc is continuous at x. The next result gives an alternative sufficient condition for the absence of Poincar´e recurrence in a dynamical system. Theorem 7.10. Consider the nonlinear dynamical system (7.14). Assume that Dc \Me 6= Ø and assume (7.14) is convergent and semistable in Dc . Then the nonlinear dynamical system (7.14) does not exhibit Poincar´e recurrence in Dc \ Me . That is, for some x ∈ Dc \ Me , there exists an open neighborhood N ⊂ Dc \ Me such that for every y ∈ N the trajectory s(t, y), t ≥ t0 , does not return to N infinitely many times. Proof. Let x ∈ Dc \ Me and let ω(x) ∈ Me be a limiting point for the trajectory s(t, x), t ≥ t0 , so that limt→∞ s(t, x) = ω(x). Since (7.14) is convergent and semistable, it follows from Proposition 7.1 that ω(x), x ∈ Dc \ Me , is continuous. Hence, for every ε > 0 there exists δ = δ(ε) > 0 such that ω(y) ∈ Bε (ω(x)) for all y ∈ Bδ (x). Choose ε > 0 and δ > 0 such that Bδ (x) ∩ Bε (ω(x)) = Ø. Furthermore, choose εˆ > 0 to be sufficiently small such that [

y∈Bδ (x)

Bεˆ(ω(y)) ∩ B δ (x) = Ø.

(7.29)

Since the dynamical system (7.14) is convergent in Dc , it follows that for all y ∈ Bδ (x) and εˆ > 0, there exists T (ˆ ε, y) > t0 such that s(t, y) ∈ Bεˆ(ω(y)) for all t > T (ˆ ε, y). Moreover, it follows from (7.29) that, for all y ∈ Bδ (x), s(t, y), t ≥ t0 , does not return to Bδ (x) infinitely many times, which proves the result with N = Bδ (x).

NNBook

SYSTEM THERMODYNAMICS

September 1, 2009

243

7.5 System Thermodynamics The fundamental and unifying concept in the analysis of thermodynamic systems is the concept of energy. The energy of a state of a dynamical system is the measure of its ability to produce changes (motion) in its own system state as well as changes in the system states of its surroundings. These changes occur as a direct consequence of the energy flow between different subsystems within the dynamical system. Heat (energy) is a fundamental concept of thermodynamics involving the capacity of hot bodies (more energetic subsystems) to produce work. As in thermodynamic systems, dynamical systems can exhibit energy (due to friction) that becomes unavailable to do useful work. This in turn contributes to an increase in system entropy, a measure of the tendency of a system to lose the ability to do useful work. In this section, we use the state space formalism to construct a mathematical model of a thermodynamic system that is consistent with basic thermodynamic principles. Specifically, we consider a large-scale system model with a combination of subsystems (compartments or parts) that is perceived as a single entity. For each subsystem (compartment) making up the system, we postulate the existence of an energy state variable such that the knowledge of these subsystem state variables at any given time t = t0 , together with the knowledge of any inputs (heat fluxes) to each of the subsystems for time t ≥ t0 , completely determines the behavior of the system for any given time t ≥ t0 . Hence, the (energy) state of our dynamical system at time t is uniquely determined by the state at time t0 and any external inputs for time t ≥ t0 and is independent of the state and inputs before time t0 . More precisely, we consider a large-scale dynamical system composed of a large number of units with aggregated (or lumped) energy variables representing homogenous groups of these units. If all the units comprising the system are identical (that is, the system is perfectly homogeneous), then the behavior of the dynamical system can be captured by that of a single plenipotentiary unit. Alternatively, if every interacting system unit is distinct, then the resulting model constitutes a microscopic system. To develop a middle-ground thermodynamic model placed between complete aggregation (classical thermodynamics) and complete disaggregation (statistical thermodynamics), we subdivide the large-scale dynamical system into a finite number of compartments, each formed by a large number of homogeneous units. Each compartment represents the energy content of the different parts of the dynamical system, and different compartments interact by exchanging heat. Thus, our compartmental thermodynamic model utilizes subsystems or compartments to describe the energy distribution among distinct regions in space with intercompartmental flows representing the heat

NNBook

September 1, 2009

244

CHAPTER 7

u1

σ11 (x)

G1

ui

σii (x)

Gi σji (x)

σij (x)

uj

σjj (x)

Gj

uq

σqq (x)

Gq

Figure 7.1 Large-scale dynamical system G.

transfer between these regions. Decreasing the number of compartments results in a more aggregated or homogeneous model, whereas increasing the number of compartments leads to a higher degree of disaggregation resulting in a heterogeneous model. To formulate our state space thermodynamic model, consider the large-scale dynamical system G shown in Figure 7.1 involving energy exchange between q interconnected subsystems. Let xi : [0, ∞) → R+ denote the energy (and hence a nonnegative quantity) of the ith subsystem, let ui : [0, ∞) → R denote the external power (heat flux) supplied to (or q extracted from) the ith subsystem, let σij : R+ → R+ , i 6= j, i, j = 1, . . . , q, denote the instantaneous rate of energy (heat) flow from the jth subsystem q to the ith subsystem, and let σii : R+ → R+ , i = 1, . . . , q, denote the instantaneous rate of energy (heat) dissipation from the ith subsystem to q the environment. Here, we assume that σij : R+ → R+ , i, j = 1, . . . , q, are q locally Lipschitz continuous on R+ and ui : [0, ∞) → R, i = 1, . . . , q, are bounded piecewise continuous functions of time. An energy balance for the ith subsystem yields xi (T ) = xi (t0 ) +

Z q X

T

j=1, j6=i t0

−

Z

T

t0

σii (x(t))dt +

[σij (x(t)) − σji (x(t))]dt Z

T

t0

ui (t)dt,

T ≥ t0 ,

(7.30)

NNBook

September 1, 2009

245

SYSTEM THERMODYNAMICS

or, equivalently, in vector form, Z T Z x(T ) = x(t0 ) + f (x(t))dt − t0

T

d(x(t))dt +

t0

Z

T t0

u(t)dt, T ≥ t0 , (7.31)

where x(t) , [x1 (t), . . . , xq (t)]T , d(x(t)) , [σ11 (x(t)), . . . , σqq (x(t))]T , u(t) , q [u1 (t), . . . , uq (t)]T , t ≥ t0 , and f = [f1 , . . . , fq ]T : R+ → Rq is such that fi (x) =

q X

[σij (x) − σji (x)],

j=1, j6=i

q

x ∈ R+ .

(7.32)

It is important to note that the exchange of energy between subsystems in (7.30) is assumed to be a nonlinear function of all the subsystems, that q is, σij = σij (x), x ∈ R+ , i 6= j, i, j = 1, . . . , q. This assumption is made for generality and would depend on the complexity of the diffusion process. For example, thermal processes may include evaporative and radiative heat transfer as well as thermal conduction giving rise to complex heat transport mechanisms. However, for simple diffusion processes it suffices to assume that σij (x) = σij (xj ), wherein the energy flow from the jth subsystem to the ith subsystem is only dependent (possibly nonlinearly) on the energy in the jth subsystem, resulting in a donor-controlled compartmental model. Similar comments apply to system dissipation. Note that (7.30) yields a conservation of energy equation and implies that the energy stored in the ith subsystem is equal to the external energy supplied to (or extracted from) the ith subsystem plus the energy gained by the ith subsystem from all other subsystems due to subsystem coupling minus the energy dissipated from the ith subsystem to the environment. Equivalently, (7.30) can be rewritten as x˙ i (t) =

q X

[σij (x(t)) − σji (x(t))] − σii (x(t)) + ui (t),

j=1, j6=i

xi (t0 ) = xi0 ,

t ≥ t0 ,

(7.33)

or, in vector form, x(t) ˙ = f (x(t)) − d(x(t)) + u(t),

x(t0 ) = x0 ,

t ≥ t0 ,

(7.34)

where x0 , [x10 , . . . , xq0 ]T , yielding a power balance equation that characterizes energy flow between subsystems of the large-scale dynamical system G. Equation (7.33) shows that the rate of change of energy, or power, in the ith subsystem is equal to the power input (heat flux) to the ith subsystem plus the energy (heat) flow to the ith subsystem from all other subsystems minus the power dissipated from the ith subsystem to the environment. q Furthermore, since f (·) − d(·) is locally Lipschitz continuous on R+ and u(·)

NNBook

September 1, 2009

246

CHAPTER 7

is a bounded piecewise continuous function of time, it follows that (7.34) has a unique solution over the finite time interval [t0 , τx0 ). If, in addition, the power balance equation (7.34) is input-to-state stable [112], then τx0 = ∞. Equation (7.31) or, equivalently, (7.34) is a statement of the first law of thermodynamics as applied to isochoric transformations (i.e., constant subsystem volume transformations) for each of the subsystems Gi , i = 1, . . . , q, with xi (·), ui (·), σij (·), i 6= j, and σii (·), i, j = 1, . . . , q, playing the role of the ith subsystem internal energy, rate of heat supplied to (or extracted from) the ith subsystem, heat flow between subsystems due to coupling, and the rate of energy (heat) dissipated to the environment, respectively. To further elucidate that (7.31) is essentially the statement of the principle of the conservation of energy, let the total energy in the largeq scale dynamical system G be given by U , eT x, x ∈ R+ , and let the net energy received by the large-scale dynamical system G over the time interval [t1 , t2 ] be given by Z t2 Q, eT [u(t) − d(x(t))]dt, (7.35) t1

where x(t), t ≥ t0 , is the solution to (7.34). Then, premultiplying (7.31) by eT and using the fact that eT f (x) ≡ 0, it follows that ∆U = Q,

(7.36)

where ∆U , U (t2 ) − U (t1 ) denotes the variation in the total energy of the large-scale dynamical system G over the time interval [t1 , t2 ]. This is a statement of the first law of thermodynamics for isochoric transformations of the large-scale dynamical system G and gives a precise formulation of the equivalence between the variation in system internal energy and heat. It is important to note that the large-scale dynamical system model (7.34) does not consider work done by the system on the environment nor work done by the environment on the system. Hence, Q can be physically interpreted as the net amount of energy that is received by the system in forms other than work. The extension of addressing work performed by and on the system can be easily addressed by including an additional state equation, coupled to the power balance equation (7.34), involving volume (deformation) states for each subsystem. Since this extension does not alter any of the conceptual results of this chapter, it is not considered here for simplicity of exposition. Work performed by the system on the environment and work done by the environment on the system is addressed in [116]. For our large-scale dynamical system model G, we assume that q σij (x) = 0, x ∈ R+ , whenever xj = 0, i, j = 1, . . . , q. In this case, q f (x)−d(x), x ∈ R+ , is essentially nonnegative. The above constraint implies

NNBook

September 1, 2009

SYSTEM THERMODYNAMICS

247

that if the energy of the jth subsystem of G is zero, then this subsystem cannot supply any energy to its surroundings nor dissipate energy to the environment. Moreover, we assume that ui (t) ≥ 0 whenever xi (t) = 0, t ≥ t0 , i = 1, . . . , q, which implies that when the energy of the ith subsystem is zero, then no energy can be extracted from this subsystem. Under these assumptions, it can be shown (see [116] for details) that the solution x(t), q t ≥ t0 , to (7.34) is nonnegative for all nonnegative initial conditions x0 ∈ R+ .

7.6 Entropy and Irreversibility The nonlinear power balance equation (7.34) can exhibit a full range of nonlinear behavior, including bifurcations, limit cycles, and even chaos. However, a thermodynamically consistent energy flow model should ensure that the evolution of the system energy is diffusive (parabolic) in character with convergent subsystem energies. As established in Section 7.4, such a system model would guarantee the absence of Poincar´e recurrence. Otherwise, the thermodynamic model would violate the second law of thermodynamics, since subsystem energies (temperatures) would be allowed to return to their starting state and thereby subverting the diffusive character of the dynamical system. Hence, to ensure a thermodynamically consistent energy flow model, we require the following axioms. For the statement of these axioms,9 we first recall the following graph-theoretic notions. Definition 7.8 ([21]). A directed graph G(C) associated with the connectivity matrix C ∈ Rq×q has vertices {1, 2, . . . , q} and an arc from vertex i to vertex j, i 6= j, if and only if C(j,i) 6= 0. A graph G(C) associated with the connectivity matrix C ∈ Rq×q is a directed graph for which the arc set is symmetric, that is, C = C T . We say that G(C) is strongly connected if for any ordered pair of vertices (i, j), i 6= j, there exists a path (i.e., a sequence of arcs) leading from i to j. Recall that the connectivity matrix C ∈ Rq×q is irreducible, that is, there does not exist a permutation matrix such that C is cogredient to a lower-block triangular matrix, if and only if G(C) is strongly connected (see q Theorem 2.7 of [21]). Let φij (x) , σij (x) − σji (x), x ∈ R+ , denote the net energy flow from the jth subsystem Gj to the ith subsystem Gi of the large-scale dynamical system G. 9 It can be argued here that a more appropriate terminology is assumptions rather than axioms since, as will be seen, these are statements taken to be true and used as premises in order to infer certain results, but may not otherwise be accepted. However, as we will see, these statements are equivalent (within our formulation) to the stipulated postulates of the zeroth and second laws of thermodynamics involving transitivity of a thermal equilibrium and heat flowing from hotter to colder bodies, and as such we refer to them as axioms.

NNBook

September 1, 2009

248

CHAPTER 7

Axiom i ) The connectivity matrix C ∈ Rq×q associated with the large-scale dynamical system G is defined by  0, if φij (x) ≡ 0, C(i,j) , i 6= j, i, j = 1, . . . , q, (7.37) 1, otherwise, and

C(i,i) , −

q X

k=1, k6=i

C(k,i) ,

i = 1, . . . , q,

(7.38)

and satisfies rank C = q − 1. Moreover, for every i 6= j such that C(i,j) = 1, φij (x) = 0 if and only if xi = xj . q

Axiom ii ) For i, j = 1, . . . , q, (xi − xj )φij (x) ≤ 0, x ∈ R+ . The fact that φij (x) = 0 if and only if xi = xj , i 6= j, implies that subsystems Gi and Gj of G are connected; alternatively, φij (E) ≡ 0 implies that Gi and Gj are disconnected. Axiom i) implies that if the energies in the connected subsystems Gi and Gj are equal, then energy exchange between these subsystems is not possible. This statement is consistent with the zeroth law of thermodynamics, which postulates that temperature equality is a necessary and sufficient condition for thermal equilibrium. Furthermore, it follows from the fact that C = C T and rank C = q −1 that the connectivity matrix C is irreducible, which implies that for every pair of subsystems Gi and Gj , i 6= j, of G there exists a sequence of connectors (arcs) of G that connect Gi and Gj . Axiom ii) implies that energy flows from more energetic subsystems to less energetic subsystems and is consistent with the second law of thermodynamics, which states that heat (energy) must flow in the direction of lower temperatures.10 Furthermore, note that φij (x) = −φji (x), q x ∈ R+ , i 6= j, i, j = 1, . . . , q, which implies conservation of energy between lossless subsystems. With u(t) ≡ 0, Axioms i) and ii) along with the fact q that φij (x) = −φji (x), x ∈ R+ , i 6= j, i, j = 1, . . . , q, imply that at a given instant of time, energy can only be transported, stored, or dissipated but not created, and the maximum amount of energy that can be transported and/or dissipated from a subsystem cannot exceed the energy in the subsystem. Next, we show that the classical Clausius equality and inequality for reversible and irreversible thermodynamics over cyclic motions are satisfied H for our thermodynamically consistent energy flow model. For this result denotes a cyclic integral evaluated along an arbitrary closed path of (7.34) 10 It is important to note that our formulation of the second law of thermodynamics as given by Axiom ii) does not require the mentioning of temperature nor the more primitive subjective notions of hotness or coldness. As we will see later, temperature is defined in terms of the system entropy after we establish the existence of a unique, continuously differentiable entropy function for G.

NNBook

September 1, 2009

249

SYSTEM THERMODYNAMICS q

in R+ ; that is, q x0 ∈ R + .

H

,

R tf t0

with tf ≥ t0 and u(·) ∈ U such that x(tf ) = x(t0 ) =

Proposition 7.2. Consider the large-scale dynamical system G with power balance equation (7.34), and assume that Axioms i) and ii) hold. q Then, for all x0 ∈ R+ , tf ≥ t0 , and u(·) ∈ U such that x(tf ) = x(t0 ) = x0 , I X Z tf X q q dQi (t) ui (t) − σii (x(t)) dt = ≤ 0, (7.39) c + xi (t) c + xi (t) t0 i=1

i=1

where c > 0, dQi (t) , [ui (t) − σii (x(t))]dt, i = 1, . . . , q, is the amount of net energy (heat) received or dissipated by the ith subsystem over the infinitesimal time interval dt, and x(t), t ≥ t0 , is the solution to (7.34) with initial condition x(t0 ) = x0 . Furthermore, I X q dQi (t) =0 (7.40) c + xi (t) i=1

if and only if there exists a continuous function α : [t0 , tf ] → R+ such that x(t) = α(t)e, t ∈ [t0 , tf ]. q

Proof. Since x(t) ≥≥ 0, t ≥ t0 , and φij (x) = −φji (x), x ∈ R+ , i 6= j, i, j = 1, . . . , q, it follows from (7.34) and Axiom ii) that P Z tf X I X q q x˙ i (t) − qj=1, j6=i φij (x(t)) dQi (t) = dt c + xi (t) c + xi (t) t0 i=1 i=1  Z tf X  q q q X X φij (x(t)) c + xi (tf ) − dt = loge c + xi (t0 ) c + xi (t) t0 i=1 j=1, j6=i

i=1

= −

Z

= −

Z

≤ 0,

tf

t0

  q q−1 X X φij (x(t)) φij (x(t)) − dt c + xi (t) c + xj (t) i=1 j=i+1

q−1 tf X

t0

q X φij (x(t))(xj (t) − xi (t)) dt (c + xi (t))(c + xj (t))

i=1 j=i+1

(7.41)

which proves (7.39). To show (7.40), note that it follows from (7.41), Axiom i), and Axiom ii) that (7.40) holds if and only if xi (t) = xj (t), t ∈ [t0 , tf ], i 6= j, i, j = 1, . . . , q, or, equivalently, there exists a continuous function α : [t0 , tf ] → R+ such that x(t) = α(t)e, t ∈ [t0 , tf ]. Inequality (7.39) is a generalization of Clausius’ inequality for re-

NNBook

September 1, 2009

250

CHAPTER 7

versible and irreversible thermodynamics as applied to large-scale dynamical systems and restricts the manner in which the system dissipates (scaled) heat over cyclic motions. It follows from Axiom i) and (7.34) that for the adiabatically isolated large-scale dynamical system G (that is, u(t) ≡ 0 and d(x(t)) ≡ 0), the energy states given by xe = αe, α ≥ 0, correspond to the equilibrium energy states of G. Thus, as in classical thermodynamics, we can define an equilibrium process as a process in which the trajectory of the large-scale dynamical system G moves along the equilibrium manifold q Me , {x ∈ R+ : E = αe, α ≥ 0} corresponding to the set of equilibria of the isolated11 system G. The power input that can generate such a trajectory can be given by u(t) = d(x(t)) + u ˆ(t), t ≥ t0 , where u ˆ(·) ∈ U is such that u ˆi (t) ≡ u ˆj (t), i 6= j, i, j = 1, . . . , q. Our definition of an equilibrium transformation involves a continuous succession of intermediate states that differ by infinitesimals from equilibrium system states and thus can only connect initial and final states, which are states of equilibrium. This process need not be slowly varying, and hence, equilibrium and quasistatic processes are not synonymous in this chapter. Alternatively, a nonequilibrium process is a process that does not lie on the equilibrium manifold Me . Hence, it follows from Axiom i) that for an equilibrium process φij (x(t)) = 0, t ≥ t0 , i 6= j, i, j = 1, . . . , q, and thus, by Proposition 7.2, inequality (7.39) is satisfied as an equality. Alternatively, for a nonequilibrium process it follows from Axioms i) and ii) that (7.39) is satisfied as a strict inequality. Next, we give a deterministic definition of entropy for the large-scale dynamical system G that is consistent with the classical thermodynamic definition of entropy. Definition 7.9. For the large-scale dynamical system G with power q balance equation (7.34), a function S : R+ → R satisfying Z t2 X q ui (t) − σii (x(t)) S(x(t2 )) ≥ S(x(t1 )) + dt (7.42) c + xi (t) t1 i=1

for every t2 ≥ t1 ≥ t0 and u(·) ∈ U is called the entropy function of G.

Next, we establish the existence of a unique, continuously differentiable entropy function for G for equilibrium and nonequilibrium processes. This result answers the long-standing question of how the entropy of a nonequilibrium state of a dynamical process should be defined [191, 217], and establishes its global existence and uniqueness. Theorem 7.11. Consider the large-scale dynamical system G with power balance equation (7.34), and assume that Axioms i) and ii) hold. 11 Since in our formulation we are not considering work performed by and on the system, the notions of an isolated system and an adiabatically isolated system are equivalent.

NNBook

September 1, 2009

251

SYSTEM THERMODYNAMICS q

q

Then the function S : R+ → R+ given by S(x) = eT loge (ce + x) − q loge c,

q

x ∈ R+ ,

(7.43)

where loge (ce + x) , [loge (c + x1 ), . . . , loge (c + xq )]T and c > 0, is a unique (modulo a constant of integration), continuously differentiable entropy function of G. Furthermore, for x(t) 6∈ Me , t ≥ t0 , where x(t), q t ≥ t0 , denotes the solution to (7.34) and Me = {x ∈ R+ : x = αe, α ≥ 0}, (7.43) satisfies S(x(t2 )) > S(x(t1 )) +

Z

t2

t1

q X ui (t) − σii (x(t))

c + xi (t)

i=1

dt

(7.44)

for every t2 ≥ t1 ≥ t0 and u(·) ∈ U . q

Proof. Since x(t) ≥≥ 0, t ≥ t0 , and φij (x) = −φji (x), x ∈ R+ , i 6= j, i, j = 1, . . . , q, it follows that q X

x˙ i (t) c + xi (t) i=1  q q X X ui (t) − σii (x(t))  + = c + xi (t)

˙ S(x(t)) =

i=1

=

q X i=1

=

≥

q X

i=1 q X i=1

j=1, j6=i

ui (t) − σii (x(t)) + c + xi (t)

ui (t) − σii (x(t)) + c + xi (t) ui (t) − σii (x(t)) , c + xi (t)

q−1 X



φij (x(t))  c + xi (t)

  q X φij (x(t)) φij (x(t)) − c + xi (t) c + xj (t)

i=1 j=i+1

q−1 X

q X φij (x(t))(xj (t) − xi (t)) (c + xi (t))(c + xj (t))

i=1 j=i+1

t ≥ t0 .

(7.45)

Now, integrating (7.45) over [t1 , t2 ] yields (7.42). Furthermore, in the case where x(t) 6∈ Me , t ≥ t0 , it follows from Axiom i), Axiom ii), and (7.45) that (7.44) holds. To show that (7.43) is a unique, continuously differentiable entropy function of G, let S(x) be a continuously differentiable entropy function of G so that S(x) satisfies (7.42) or, equivalently, ˙ S(x(t)) ≥ µT (x(t))[u(t) − d(x(t))], q

t ≥ t0 ,

(7.46)

1 1 , . . . , c+x ], x ∈ R+ , x(t), t ≥ t0 , denotes the solution to where µT (x) = [ c+x 1 q ˙ the power balance equation (7.34), and S(x(t)) denotes the time derivative

NNBook

September 1, 2009

252

CHAPTER 7

of S(x) along the solution x(t), t ≥ t0 . Hence, it follows from (7.46) that q

S ′ (x)[f (x) − d(x) + u] ≥ µT (x)[u − d(x)],

u ∈ Rq ,

x ∈ R+ ,

(7.47)

q

which implies that there exist continuous functions ℓ : R+ → Rp and W : q R+ → Rp×q such that 0 = S ′ (x)[f (x) − d(x) + u] − µT (x)[u − d(x)] q −[ℓ(x) + W(x)u]T [ℓ(x) + W(x)u], x ∈ R+ ,

u ∈ Rq . (7.48)

Now, equating coefficients of equal powers (of u), it follows that W(x) ≡ 0, q S ′ (x) = µT (x), x ∈ R+ , and 0 = S ′ (x)f (x) − ℓT (x)ℓ(x),

q

x ∈ R+ .

(7.49)

q

Hence, S(x) = eT loge (ce + x) − q loge c, x ∈ R+ , and 0 = µT (x)f (x) − ℓT (x)ℓ(x),

q

x ∈ R+ .

(7.50)

Thus, (7.43) is a unique, continuously differentiable entropy function for G. Note that it follows from Axiom i), Axiom ii), and the last equality in (7.45) that the entropy function given by (7.43) satisfies (7.42) as an equality for an equilibrium process and as a strict inequality for a nonequilibrium process. Hence, it follows from Theorem 7.9 that the isolated (i.e., u(t) ≡ 0 and d(x) ≡ 0) large-scale dynamical system G does not exhibit Poincar´e q recurrence in R+ \ Me . Furthermore, for any entropy function of G, it follows from Proposition 7.2 that if (7.42) holds as an equality for some transformation starting and ending at equilibrium points of the isolated system G, then this transformation must lie on the equilibrium manifold Me . However, (7.42) may hold as an equality for nonequilibrium processes starting and ending at nonequilibrium states. The entropy expression given by (7.43) is identical in form to the Boltzmann entropy for statistical thermodynamics. Due to the fact that the entropy given by (7.43) is indeterminate to the extent of an additive constant, we can set the constant of integration q loge c to zero by taking c = 1. Since S(x) given by (7.43) achieves a maximum when all the subsystem energies xi , i = 1, . . . , q, are equal [116], the entropy of G can be thought of as a measure of the tendency of a system to lose the ability to do useful work, lose order, and settle to a more homogenous state. Recalling that dQi (t) = [ui (t) − σii (x(t))]dt, i = 1, . . . , q, is the infinitesimal amount of the net heat received or dissipated by the ith subsystem of G over the infinitesimal time interval dt, it follows from (7.42)

NNBook

September 1, 2009

253

SYSTEM THERMODYNAMICS

that q X dQi (t) , dS(x(t)) ≥ c + xi (t) i=1

t ≥ t0 .

(7.51)

Inequality (7.51) is analogous to the classical thermodynamic inequality for the variation of entropy during an infinitesimal irreversible transformation with the shifted subsystem energies c + xi playing the role of the ith subsystem thermodynamic (absolute) temperatures. Specifically, note 1 i that since dS dxi = c+xi , where Si = loge (c + xi ) − loge c denotes the unique continuously differentiable ith subsystem entropy, it follows that dSi dxi , i = 1, . . . , q, defines the reciprocal of the subsystem thermodynamic temperatures. That is, dSi 1 , Ti dxi

(7.52)

and Ti > 0, i = 1, . . . , q. Hence, in our formulation, temperature is a function derived from entropy and does not involve the primitive subjective notions of hotness and coldness. Finally, using the system entropy function given by (7.43) we show that our large-scale dynamical system G with power balance equation (7.34) is state irreversible for every nontrivial (nonequilibrium) trajectory of G. For this result, let W[t0 ,t1 ] denote the set of all possible energy trajectories of G over the time interval [t0 , t1 ] given by q

W[t0 ,t1 ] , {sx : [t0 , t1 ] × U → R+ : sx (·, u(·)) satisfies (7.34)},

(7.53)

q

and let Me ⊂ R+ denote the set of equilibria of the isolated system G given q by Me = {x ∈ R+ : αe, α ≥ 0}. Theorem 7.12. Consider the large-scale dynamical system G with power balance equation (7.34), and assume Axioms i) and ii) hold. Furthermore, let sx (·, u(·)) ∈ W[t0 ,t1 ] , where u(·) ∈ U . Then sx (·, u(·)) is an Iq -reversible trajectory of G if and only if sx (t, u(t)) ∈ Me , t ∈ [t0 , t1 ]. Proof. First, note that it follows from Theorem 7.11 that if x(t) q 6 Me , t ≥ t0 , then there exists an entropy function S(x), x ∈ R+ , for G ∈ such that (7.44) holds. Now, sufficiency follows as a direct consequence of Theorem 7.1 with R = Iq , V (x) = S(x), and r(u, y) = r(u, d(x)) = Pq ui −σii (x) x i=1 c+xi . To show necessity, assume that s (t, u(t)) ∈ Me , t ∈ [t0 , t1 ]. In this case, it can be shown that u(t) = d(x(t)) + u ˆ(t), t ≥ t0 , where u ˆ(·) ∈ U is such that u ˆi (t) ≡ u ˆj (t), i 6= j, i, j = 1, . . . , q. Now, with u− (t) = d(x(t)) + u ˆ− (t), t ≥ t0 , where u ˆ− (t) = −ˆ u(t1 + t0 − t), t ∈ [t0 , t1 ], it follows x that s (t, u(t)) is an Iq -reversible trajectory of G.

NNBook

September 1, 2009

254

CHAPTER 7

Theorem 7.12 establishes an equivalence between (non)equilibrium and state (ir)reversible thermodynamic systems. Furthermore, Theorem 7.12 shows that for every x0 6∈ Me , the large-scale dynamical system G is state irreversible. In addition, since state irrecoverability implies state irreversibility and, by Theorem 7.12, state irreversibility is equivalent to x(t) 6∈ Me , t ≥ t0 , it follows from Theorem 7.2 that state (ir)reversibility and state (ir)recoverability are equivalent for our thermodynamically consistent large-scale dynamical system G. Hence, in the remainder of the chapter we use the notions of (non)equilibrium, state (ir)reversible, and state (ir)recoverable dynamical processes interchangeably.

7.7 Semistability and the Entropic Arrow of Time For the isolated large-scale dynamical system G, (7.45) yields the fundamental inequality S(x(t2 )) ≥ S(x(t1 )),

t2 ≥ t1 .

(7.54)

Inequality (7.54) implies that, for any dynamical change in an isolated largescale dynamical system G, the entropy of the final state can never be less than the entropy of the initial state. Inequality (7.54) is often identified with the second law of thermodynamics as a statement about entropy increase. Furthermore, it follows from (7.44) that for an isolated large-scale dynamical system G the entropy function (7.43) is a strictly increasing function of time q along the trajectories of (7.34) with initial conditions in R+ \ Me . Hence, it follows from Theorem 7.9 that the isolated large-scale dynamical system q G does not exhibit Poincar´e recurrence in R+ \ Me . This result can also be arrived at using the fact that our thermodynamically consistent large-scale dynamical system G is semistable. Since our thermodynamic compartmental model involves intercompartmental flows representing energy transfer between compartments, we can use graph-theoretic notions with undirected graph topologies (i.e., bidirectional energy flows) to capture the compartmental system interconnections. Graph theory [67, 101] can be useful in the analysis of the connectivity properties of compartmental systems. In particular, a directed graph can be constructed to capture a compartmental model in which the compartments are represented by nodes and the flows are represented by edges or arcs. In this case, the environment must also be considered as an additional node. Specifically, let G = (V, E, A) be a directed graph (or digraph) denoting the compartmental network with the set of nodes (or compartments) V = {1, . . . , q} involving a finite nonempty set denoting the compartments, the set of edges E ⊆ V × V involving a set of ordered pairs denoting the direction of energy flow, and an adjacency matrix A ∈ Rq×q such that A(i,j) = 1, i, j = 1, . . . , q, if (j, i) ∈ E, while A(i,j) = 0 if

NNBook

September 1, 2009

255

SYSTEM THERMODYNAMICS

(j, i) 6∈ E. The edge (j, i) ∈ E denotes that compartment j can obtain energy from compartment i, but not necessarily vice versa. Moreover, we assume A(i,i) = 0 for all i ∈ V. A graph or undirected graph G associated with the adjacency matrix A ∈ Rq×q is a directed graph for which the arc set is symmetric, that is, A = AT . Weighted graphs can also be considered here; however, since this extension does not alter any of the conceptual results in this chapter we do not consider this extension for simplicity of exposition. Finally, we denote the energy of the compartment i ∈ {1, . . . , q} at time t by xi (t) ∈ R+ . Proposition 7.3. Consider the large-scale dynamical system G with power balance equation (7.34) with d(x) ≡ 0 and u(t) ≡ 0, and assume Axioms i) and ii) hold. Then fi (x) = 0 for all i = 1, . . . , q if and only if x1 = · · · = xq . Furthermore, αe, α ≥ 0, is an equilibrium state of (7.34). Proof. If xi = xj for all (i, j) ∈ E, then fi (x) = 0 for all i = 1, . . . , q is immediate from Axiom i). Next, we show that fi (x) = 0 for all i = 1, . . . , q implies that x1 = · · · = xq . If fi (x) = 0 for all i = 1, . . . , q, then it follows from Axiom ii) that 0 = =

q X

xi fi (x) i=1 q X q X

xi φij (x)

i=1 j=1

=

q−1 X q X

(xi − xj )φij (x)

i=1 j=i+1

≤ 0,

where we have used the fact that φij (x) = −φji (x) for all i, j = 1, . . . , q. Hence, (xi − xj )φij (x) = 0 for all i, j = 1, . . . , q. Now, the result follows from Axiom i). Alternatively, the proof can also be shown using graph-theoretic concepts. Specifically, if xi = xj for all (i, j) ∈ E, then fi (x) = 0 for all i = 1, . . . , q is immediate from Axiom i). Next, we show that fi (x) = 0 for all i = 1, . . . , q implies that x1 = · · · = xq . If the values of all nodes are equal, then the result is immediate. Hence, assume there exists a node i∗ such that xi∗ ≥ xj for all j 6= i∗ , j ∈ {1, . . . , q}. If (i, j) ∈ E, then we define a neighbor of node i to be node j, and vice versa. Define the initial node set J (0) , {i∗ } and denote the indices of all the first neighbors of node i∗ by J (1) = Ni∗ . Then, fi∗ (x) = 0 implies that P j∈Ni∗ φi∗ j (xi∗ , xj ) = 0. Since xj ≤ xi∗ for all j ∈ Ni∗ and, by Axiom

NNBook

September 1, 2009

256

CHAPTER 7

ii), φij (zi , zj ) ≤ 0 for all zi ≥ zj , it follows that xi∗ = xj for all the first neighbors j ∈ J (1) . Next, we define the kth neighbor of node i∗ and show that the value of node i∗ is equal to the values of all kth neighbors of node i∗ for k = 1, . . . , q − 1. The set of kth neighbors of node i∗ is defined by J (k) , J (k−1) ∪ NJ (k−1) ,

k ≥ 1,

J (0) = {i∗ },

(7.55)

where NJ denotes the set of neighbors of the node set J ⊆ V. By definition, {i∗ } ⊂ J (k) ⊆ V for all k ≥ 1 and J (k) is a monotonically increasing sequence of node sets in the sense of set inclusions. Next, we show that J (q−1) = V. Suppose, ad absurdum, V\J (q−1) 6= Ø. Then, by definition, there exists one node m ∈ {1, . . . , q}, disconnected from all the other nodes. Hence, C(m,i) = C(i,m) = 0, i = 1, . . . , q, which implies that the connectivity matrix C has a row and a column of zeros. Without loss of generality, assume that C has the form   Cs 0(q−1)×1 , C= 01×(q−1) 0 where Cs ∈ R(q−1)×(q−1) denotes the connectivity matrix for the new undirected graph G which excludes node m from the undirected graph G. In this case, since rank Cs ≤ q − 2, it follows that rank C < q − 1, which contradicts Axiom i). Using mathematical induction, we show that the values of all the nodes in J (k) are equal for k ≥ 1. This statement holds for k = 1. Assuming that the values of all the nodes in J (k) are equal to the value of node i∗ , we show that the values of all the nodes in J (k+1) are equal to the value of node i∗ as well. Note that since G is strongly connected, Ni 6= Ø for all i ∈ V. If Ni ∩ (J (k+1) \J (k) ) = Ø for all i, then it follows that J (k+1) = J (k) , and hence, the statement holds. Thus, it suffices to show (k) with N ∩ (J (k+1) \J (k) ) 6= that xi = xi∗ for an arbitrary i P node i ∈ J Ø. For node i, note that j∈Ni φij (xi , xj ) = 0. Furthermore, note that Ni = (Ni ∩ J (k) ) ∪ (Ni ∩ (V\J (k) )), V\J (k) = V\J (k+1) ∪ (J (k+1) \J (k) ), J (k) ⊆ V for all k, and J (k+1) contains the set of first neighbors of node i, or Ni ⊆ J (k+1) . Then it follows that Ni ∩ (V\J (k) ) = Ni ∩ (J (k+1) \J (k) ) and X X φij (xi , xj ) + φij (xi , xj ) = 0. (7.56) j∈Ni ∩J (k)

j∈Ni ∩(J (k+1) \J (k) )

Since xj = xi for all nodes j ∈ NiP ∩ J (k) ⊆ J (k) , it follows that P j∈Ni ∩J (k) φij (xi , xj ) = 0, and hence, j∈Ni ∩(J (k+1) \J (k) ) φij (xi , xj ) = 0. However, since xi∗ = xi ≥ xj for all i ∈ J (k) and j ∈ V\J (k) , it follows that the values of all nodes in Ni ∩ (J (k+1) \J (k) ) are equal to xi∗ . Hence,

NNBook

September 1, 2009

257

SYSTEM THERMODYNAMICS

S the values of all nodes i in the node set i∈J (k) Ni ∩ (J (k+1) \J (k) ) = J (k+1) ∩ (J (k+1) \J (k) ) = J (k+1) \J (k) are equal to xi∗ , that is, the values of all the nodes in J (k+1) are equal. Combining this result with the fact that J (q−1) = V, it follows that the values of all the nodes in V are equal. The second assertion is a direct consequence of the first assertion. Theorem 7.13. Consider the large-scale dynamical system G with power balance equation (7.34) with u(t) ≡ 0 and d(x) ≡ 0, and assume that Axioms i) and ii) hold. Then for every α ≥ 0, αe is a semistable equilibrium state of (7.34). Furthermore, x(t) → 1q eeT x(t0 ) as t → ∞ and 1 T q ee x(t0 ) is a semistable equilibrium state. q

Proof. It follows from Proposition 7.3 that αe ∈ R+ , α ≥ 0, is an equilibrium state of (7.34). To show Lyapunov stability of the equilibrium state αe, consider the function V (x) = 12 (x − αe)T (x − αe) as a Lyapunov q function candidate. Now, since φij (x) = −φji (x), x ∈ R+ , i 6= j, i, j = q 1, . . . , q, and eT f (x) = 0, x ∈ R+ , it follows from Axiom ii) that V˙ (x) = (x − αe)T x˙ = (x − αe)T f (x) = xT f (x)   q q X X φij (x) = xi  i=1

=

=

q−1 X

j=1, j6=i

q X

(xi − xj )φij (x)

i=1 j=i+1 q X X i=1 j∈Ki

≤ 0,

(xi − xj )φij (x) q

x ∈ R+ ,

(7.57)

i−1 where Ki , Ni \ ∪l=1 {l} and Ni , {j ∈ {1, . . . , q} : φij (x) = 0 if and only if xi = xj }, i = 1, . . . , q, which establishes Lyapunov stability of the equilibrium state αe. q To show that αe is semistable, let R , {x ∈ R+ : V˙ (x) = 0} = {x ∈ q R+ : (xi − xj )φij (x) = 0, i = 1, . . . , q, j ∈ Ki }. Now, by Axiom i) the directed graph associated with the connectivity matrix C for the large-scale q dynamical system G is strongly connected, which implies that R = {x ∈ R+ : x1 = ··· = xq }. Since the set R consists of the equilibrium states of (7.34), it follows that the largest invariant set M contained in R is given by M = R.

NNBook

258

September 1, 2009

CHAPTER 7

Hence, it follows from the Krasovskii-LaSalle theorem that for every initial q condition x(t0 ) ∈ R+ , x(t) → M as t → ∞, and hence, αe is a semistable equilibrium state of (7.34). Next, note that since eT x(t) = eT x(t0 ) and x(t) → M as t → ∞, it follows that x(t) → 1q eeT x(t0 ) as t → ∞. Hence, with α = 1q eT x(t0 ), αe = 1q eeT x(t0 ) is a semistable equilibrium state of (7.34). Theorem 7.13 shows that the isolated (i.e., u(t) ≡ 0 and d(x) ≡ 0) large-scale dynamical system G is semistable. Hence, it follows from Theorem 7.10 that the isolated large-scale dynamical system G does not q exhibit Poincar´e recurrence in R+ \ Me . Next, using the system entropy function given by (7.43), we show that our large-scale isolated dynamical system G with power balance equation (7.34) is state irreversible for all nonequilibrium trajectories of G establishing a clear connection between our thermodynamic model and the arrow of time. Theorem 7.14. Consider the large-scale dynamical system G with power balance equation (7.34) with u(t) ≡ 0 and d(x) ≡ 0, and assume Axioms i) and ii) hold. Furthermore, let sx (·, 0) ∈ W[t0 ,t1 ] . Then for every q x0 6∈ Me , there exists a continuously differentiable function S : R+ → R such x that S(s (t, 0)) is a strictly increasing function of time. Furthermore, sx (·, 0) is an Iq -reversible trajectory of G if and only if sx (t, 0) ∈ Me , t ∈ [t0 , t1 ]. Proof. The existence of a continuously differentiable function S : q R+ → R, which strictly increases on all nonequilibrium trajectories of G, is a restatement of Theorem 7.11 with u(t) ≡ 0 and d(x) ≡ 0. Now, necessity is immediate, while sufficiency is a direct consequence of Corollary 7.1 with R = Iq and V (x) = S(x). Theorem 7.14 shows that for every x0 6∈ Me , the isolated dynamical system G is state irreversible. This gives a clear connection between our thermodynamic model and the arrow of time. In particular, it follows from Corollary 7.1 and Theorem 7.14 that there exists a function of the system state that strictly increases in time on every nonequilibrium trajectory of G if and only if there does not exist a nonequilibrium reversible trajectory of G. Thus, the existence of the continuously differentiable entropy function given by (7.43) for G establishes the existence of a completely ordered time set having a topological structure involving a closed set homeomorphic to the real line. This fact follows from the inverse function theorem of mathematical analysis and the fact that a continuous strictly monotonic function is a topological mapping (i.e., a homeomorphism), and conversely every topological mapping of a strictly monotonic function’s domain onto its codomain must be strictly monotonic. This topological property gives a clear time-reversal asymmetry characterization of our thermodynamic model

NNBook

September 1, 2009

SYSTEM THERMODYNAMICS

259

establishing an emergence of the direction of time flow.

7.8 Monotonicity of System Energies in Thermodynamic Processes Even though Theorem 7.13 gives sufficient conditions under which the subsystem energies in the large-scale dynamical system G converge, these subsystem energies may exhibit an oscillatory (hyperbolic) or nonmonotonic behavior prior to convergence. For certain thermodynamical processes, it is desirable to identify system models that guarantee monotonicity of the system energy flows. It is important to note that monotonicity of solutions does not necessarily imply Axiom ii), nor does Axiom ii) imply monotonicity of solutions. These are two disjoint notions. In this section, using the ideas developed in Chapter 4, we give necessary and sufficient conditions under which the solutions to (7.34) are monotonic. To develop necessary and sufficient conditions for monotonicity of solutions, note that the power balance equation (7.34) for the large-scale dynamical system G can be written as T  ∂H (x(t)) + Gu(t), x(t0 ) = x0 , x(t) ˙ = [J (x(t)) − D(x(t))] ∂x t ≥ t0 , (7.58) q

where x(t) ∈ R+ , H(x) = eT x, u(t) = [u1 (t), . . . , uq (t)]T , t ≥ t0 , J (x) is a skew-symmetric matrix function with J(i,i) (x) = 0 and J(i,j)(x) = σij (x) − σji (x), i 6= j, i, j = 1, . . . , q, D(x) = diag[σ11 (x), . . . , σqq (x)] ≥ 0, and G ∈ Rq×q is a diagonal input matrix that has been included for generality and contains zeros and ones as its entries. Hence, the power balance equation of the large-scale dynamical system G has a port-controlled Hamiltonian structure [203] with a Hamiltonian function H(x) = eT x = P q i=1 xi representing the sum of all subsystem energies, D(x) representing power dissipation in the subsystems, J (x) = −J T (x) representing energyconserving subsystem coupling, and u(t), t ≥ t0 , representing supplied system power. As noted in Section 7.6, the nonlinear power balance equation (7.58) can exhibit a full range of nonlinear behavior, including bifurcations, limit cycles, and even chaos. However, a thermodynamically consistent energy flow model ensures that the evolution of the system energy is diffusive in character with convergent subsystem energies. As shown in Section 7.6, Axioms i) and ii) guarantee a thermodynamically consistent energy flow model. In order to guarantee a thermodynamically consistent energy flow model, we assume Axiom ii) holds and seek solutions to (7.58) that exhibit

NNBook

September 1, 2009

260

CHAPTER 7

a monotonic behavior of the subsystem energies. This would physically imply that the energy of a subsystem whose initial energy is greater than the average system energy will decrease, while the energy of a subsystem whose initial energy is less than the average system energy will increase. This of course is consistent with the second law of thermodynamics with the additional constraint of monotonic heat flows. The following definition is needed. Definition 7.10. Consider the large-scale dynamical system G with power balance equation (7.58). The subsystem energies x(t), t ≥ t0 , of G q are monotonic for all x0 ∈ Dc ⊆ R+ , where Dc is a positively invariant set with respect to (7.58), if there exists a weighting matrix R ∈ Rq×q such that q R = diag[r1 , . . . , rq ], ri = ±1, i = 1, . . . , q, and, for every x0 ∈ Dc ⊆ R+ , Rx(t2 ) ≤≤ Rx(t1 ), t0 ≤ t1 ≤ t2 . The following result presents necessary and sufficient conditions that guarantee that the subsystem energies of the large-scale dynamical system G are monotonic. It is important to note that this result holds whether or not Axiom ii) holds. Theorem 7.15. Consider the large-scale dynamical system G with power balance equation (7.58). Then the following statements hold: i) If u(t) ≥≥ 0, t ≥ t0 , and there exists a matrix R ∈ Rq×q such that R = T diag[r1 , . . . , rq ], ri = ±1, i = 1, . . . , q, R[J (x) − D(x)]( ∂H ∂x (x)) ≤≤ 0, q x ∈ R+ , and RG ≤≤ 0, then the subsystem energies x(t), t ≥ t0 , of G q are monotonic for all x0 ∈ R+ . q

ii) Let u(t) ≡ 0 and let Dc ⊆ R+ be a positively invariant set with respect to (7.58). Then the subsystem energies x(t), t ≥ t0 , of G are q monotonic for all x0 ∈ Dc ⊆ R+ if and only if there exists a matrix R ∈ Rq×q such that R = diag[r1 , . . . , rq ], ri = ±1, i = 1, . . . , q, and q T R[J (x) − D(x)]( ∂H ∂x (x)) ≤≤ 0, x ∈ Dc ⊆ R+ . Proof. i) Let u(t) ≥≥ 0, t ≥ t0 , and assume there exists R = diag[r1 , . . . , rq ], ri = ±1, i = 1, . . . , q, such that R[J (x) − D(x)] q T ·( ∂H ∂x (x)) ≤≤ 0, x ∈ R+ . Now, it follows from (7.58) that Rx(t) ˙ = R[J (x(t)) − D(x(t))]



T ∂H (x(t)) + RGu(t), x(t0 ) = x0 , ∂x t ≥ t0 , (7.59)

NNBook

September 1, 2009

261

SYSTEM THERMODYNAMICS

which further implies that Rx(t2 ) = Rx(t1 ) + +

Z

t2

Z

t2 t1

R[J (x(t)) − D(x(t))]



T ∂H (x(t)) dt ∂x (7.60)

RGu(t)dt. t1

T Next, since [J (x) − D(x)]( ∂H ∂x (x)) is essentially nonnegative and u(t) ≥≥ 0, t ≥ t0 , it follows from Proposition 4.1 that x(t) ≥≥ 0, t ≥ t0 , for all x0 ∈ q q T R+ . Hence, since R[J (x) − D(x)]( ∂H ∂x (x)) ≤≤ 0, x ∈ R+ , and RG ≤≤ 0, it follows that  T ∂H (x(t)) + RGu(t) ≤≤ 0, t ≥ t0 , (7.61) R[J (x(t)) − D(x(t))] ∂x q

which implies that, for every x0 ∈ R+ , Rx(t2 ) ≤≤ Rx(t1 ), t0 ≤ t1 ≤ t2 . ii) To show sufficiency, note that since by assumption Dc is positively T invariant, then R[J (x(t)) − D(x(t))]( ∂H ∂x (x(t))) ≤≤ 0, t ≥ t0 , for all x0 ∈ q Dc ⊆ R+ . Now, the result follows by using identical arguments as in i) with q u(t) ≡ 0 and x0 ∈ Dc ⊆ R+ . To show necessity, assume that (7.58) with q u(t) ≡ 0 is monotonic for all x0 ∈ Dc ⊆ R+ . In this case, (7.59) implies that for every τ > t0 , T  Z τ ∂H (x(t)) dt. (7.62) Rx(τ ) = Rx0 + R[J (x(t)) − D(x(t))] ∂x t0 q

Now, suppose, ad absurdum, there exist J ∈ {1, . . . , q} and x0 ∈ Dc ⊆ R+ T such that [R[J (x0 ) − D(x0 )]( ∂H ∂x (x0 )) ]J > 0. Since the mapping R[J (·) − T and the solution x(t), t ≥ t , to (7.58) are continuous, it D(·)]( ∂H 0 ∂x (·)) follows that there exists τ > t0 such that h T i  ∂H (x(t)) R[J (x(t)) − D(x(t))] > 0, t0 ≤ t ≤ τ, (7.63) ∂x J which implies that [Rx(τ )]J > [Rx0 ]J , leading to a contradiction. Hence, q T R[J (x) − D(x)]( ∂H ∂x (x)) ≤≤ 0, x ∈ Dc ⊆ R+ .

It follows from i) of Theorem 7.15 that if G = Iq (that is, external power (heat flux) can be injected to all subsystems), then R = −Iq , and q T hence, [J (x) − D(x)]( ∂H ∂x (x)) ≥≥ 0, x ∈ R+ . This case would correspond to a power balance equation whose states are all increasing and can only q be achieved if D(x) = 0, x ∈ R+ . This, of course, implies that the dynamical system G cannot dissipate energy, and hence, the transfer of energy (heat) from a lower energy (temperature) level (source) to a higher energy (temperature) level (sink) requires the input of additional heat or

NNBook

September 1, 2009

262

CHAPTER 7

energy. This is consistent with Clausius’ statement of the second law of thermodynamics. The following result is a direct consequence of Theorem 7.15 and provides sufficient conditions for convergence of the subsystem energies of the isolated large-scale dynamical system G. Once again, this result holds whether or not Axiom ii) holds. Theorem 7.16. Consider the large-scale dynamical system G with q power balance equation (7.58) and u(t) ≡ 0. Let Dc ⊆ R+ be a positively invariant set. If there exists a matrix R ∈ Rq×q such that T R = diag[r1 , . . . , rq ], ri = ±1, i = 1, . . . , q, and R[J (x) − D(x)]( ∂H ∂x (x)) q q ≤≤ 0, x ∈ Dc ⊆ R+ , then, for every x0 ∈ Dc ⊆ R+ , limt→∞ x(t) exists. q

Proof. Since H(x) = eT x, x ∈ R+ , it follows that ∂H ˙ x˙ H(x) = ∂x   ∂H ∂H T = [J (x) − D(x)] ∂x ∂x T  ∂H ∂H = − D(x) ∂x ∂x ≤ 0,

q

x ∈ R+ ,

(7.64)

˙ and hence, H(x(t)) ≤ 0, t ≥ t0 , where x(t), t ≥ t0 , denotes the solution of (7.58). This implies that H(x(t)) ≤ H(x0 ) = eT x0 , t ≥ t0 , and hence, for q every x0 ∈ R+ , the solution x(t), t ≥ t0 , of (7.58) is bounded. Hence, for every i ∈ {1, . . . , q}, xi (t), t ≥ t0 , is bounded. Furthermore, it follows from q Theorem 7.15 that xi (t), t ≥ t0 , is monotonic for all x0 ∈ Dc ⊆ R+ . Now, since xi (·), i ∈ {1, . . . , q}, is continuous and every bounded nonincreasing or nondecreasing scalar sequence converges to a finite real number, it follows from the monotone convergence theorem [112, p. 37] that limt→∞ xi (t), i ∈ q {1, . . . , q}, exists. Hence, limt→∞ x(t) exists for all x0 ∈ Dc ⊆ R+ .

NNBook

September 1, 2009

Chapter Eight

Finite-Time Thermodynamics

8.1 Introduction In Chapter 7, we used a compartmental dynamical systems perspective to provide a system-theoretic approach to thermodynamics. Specifically, using a state space formulation, we developed a nonlinear compartmental dynamical system model characterized by energy conservation laws that is consistent with basic thermodynamic principles. In the case where the compartmental system is isolated we showed that the dynamical system asymptotically evolves toward a state of energy equipartition. However, in physical systems, energy and temperature equipartition is achieved in finite time rather than merely asymptotically. In this chapter, we merge the theories of semistability and finite-time stability developed in [32–34] to develop a rigorous framework for finite-time thermodynamics. First, we present the notions of finite-time convergence and finite-time semistability for nonlinear dynamical systems, and develop several sufficient Lyapunov stability theorems for finite-time semistability. Following [35], we exploit homogeneity as a means for verifying finite-time convergence. Our main result in this direction asserts that a homogeneous system is finite-time semistable if and only if it is semistable and has a negative degree of homogeneity. This main result depends on a converse Lyapunov result for homogeneous semistable systems, which we develop. While our converse result resembles a related result for asymptotically stable systems given in [35,255], the proof of our result is rendered more difficult by the fact that it does not hold under the notions of homogeneity considered in [35, 255]. More specifically, while previous treatments of homogeneity involved Euler vector fields representing asymptotically stable dynamics, our results involve homogeneity with respect to a semi-Euler vector field representing a semistable system having the same equilibria as the dynamics of interest. Consequently, our theory precludes the use of dilations commonly used in the literature on homogeneous systems (such as [255]), and requires us to

NNBook

September 1, 2009

264

CHAPTER 8

adopt a more geometric description of homogeneity (see [35] and references therein). Finally, using this framework we develop intercompartmental flow laws that guarantee finite-time semistability and energy equipartition for our thermodynamically consistent dynamical system model developed in Chapter 7.

8.2 Finite-Time Semistability of Nonlinear Nonnegative Dynamical Systems In this chapter, we consider nonlinear dynamical systems of the form x(t) ˙ = f (x(t)), n

x(0) = x0 ,

t ∈ I x0 ,

(8.1)

where x(t) ∈ D ⊆ R+ , t ∈ Ix0 , is the system state vector, D is a relatively n open set with respect to R+ , f : D → R is continuous and essentially nonnegative on D, f −1 (0) , {x ∈ D : f (x) = 0} is nonempty, and Ix0 = [0, τx0 ), 0 ≤ τx0 ≤ ∞, is the maximal interval of existence for the solution x(·) of (8.1). The continuity of f implies that, for every x0 ∈ D, there exist τ0 < 0 < τ1 and a solution x(·) of (8.1) defined on (τ0 , τ1 ) such that x(0) = x0 . A solution x is said to be right maximally defined if x cannot be extended on the right (either uniquely or nonuniquely) to a solution of (8.1). Here, we assume that for every initial condition x0 ∈ D, (8.1) has a unique right maximally defined solution, and this unique solution is defined on [0, ∞). Under these assumptions, the solutions of (8.1) define a continuous global semiflow on D, that is, s : [0, ∞) × D → D is a jointly continuous function satisfying the consistency property s(0, x) = x and the semigroup property s(t, s(τ, x)) = s(t + τ, x) for every x ∈ D and t, τ ∈ [0, ∞). Furthermore, we assume that for every initial condition x0 ∈ D\f −1 (0), (8.1) has a local unique solution for negative time. Given t ∈ [0, ∞), we denote the flow s(t, ·) : D → D of (8.1) by st (x0 ) or st . Likewise, given x ∈ D, we denote the solution curve or trajectory s(·, x) : [0, ∞) → D of (8.1) by sx (t) or sx . The image of U ⊂ D under the flow st is defined n as st (U ) , {y : y = st (x0 ) for all x0 ∈ U}. Finally, a set E ⊆ R+ is n connected if and only if every pair of open sets Ui ⊆ R+ , i = 1, 2, satisfying E ⊆ U1 ∪ U2 and Ui ∩ E 6= Ø, i = 1, 2, has a nonempty intersection. A n connected component of the set E ⊆ R+ is a connected subset of E that is not properly contained in any connected subset of E. Next, we establish the notion of finite-time semistability and develop sufficient Lyapunov stability theorems for finite-time semistability. Definition 8.1. An equilibrium point xe ∈ f −1 (0) of (8.1) is said to be

NNBook

September 1, 2009

265

FINITE-TIME THERMODYNAMICS

finite-time semistable if there exist a relatively open neighborhood Q ⊆ D of xe and a function T : Q\f −1 (0) → (0, ∞), called the settling-time function, such that the following statements hold: i) For every x ∈ Q\f −1 (0), s(t, x) ∈ Q\f −1 (0) for all t ∈ [0, T (x)), and limt→T (x) s(t, x) exists and is contained in Q ∩ f −1 (0). ii) xe is semistable. An equilibrium point xe ∈ f −1 (0) of (8.1) is said to be globally finiten time semistable if it is finite-time semistable with D = Q = R+ . The system (8.1) is said to be finite-time semistable if every equilibrium point in f −1 (0) is finite-time semistable. Finally, (8.1) is said to be globally finitetime semistable if every equilibrium point in f −1 (0) is globally finite-time semistable. It is easy to see from Definition 8.1 that, for all x ∈ Q, T (x) = inf{t ∈ R+ : f (s(t, x)) = 0},

(8.2)

where T (Q ∩ f −1 (0)) = {0}.

Lemma 8.1. Suppose (8.1) is finite-time semistable. Let xe ∈ f −1 (0) be an equilibrium point of (8.1) and let Q ⊆ D be as in Definition 8.1. Furthermore, let T : Q → R+ be the settling-time function. Then T is continuous on Q if and only if T is continuous at each ze ∈ Q ∩ f −1 (0). Proof. Necessity is immediate. To prove sufficiency, suppose that T is continuous at each ze ∈ Q ∩ f −1 (0). Let z ∈ Q\f −1 (0) and consider a − = lim inf sequence {zm }∞ m→∞ T (zm ) m=1 in Q that converges to z. Let τ + − + and τ = lim supm→∞ T (zm ). Note that both τ and τ are in R+ and τ − ≤ τ +.

(8.3)

+ ∞ + Next, let {zl+ }∞ l=1 be a subsequence of {zm }m=1 such that T (zl ) → τ + ∞ as l → ∞. The sequence {(T (z), zl )}l=1 converges in R+ × Q to (T (z), z). By continuity and

s(T (x) + t, x) = s(T (x), x)

(8.4)

for all x ∈ Q and t ∈ R+ , s(T (z), zl+ ) → s(T (z), z) = ze as l → ∞, where ze ∈ Q∩f −1 (0). Since T is assumed to be continuous at each ze ∈ Q∩f −1 (0), T (s(T (z), zl+ )) → T (ze ) = 0 as l → ∞. Note that T (s(t, x)) = max{T (x) − t, 0}

(8.5)

for all x ∈ Q and t ∈ R+ . Using (8.5) with t = T (z) and x = zl+ , we obtain max{T (zl+ ) − T (z), 0} → 0 as l → ∞. Hence, max{τ + − T (z), 0} = 0, that

NNBook

September 1, 2009

266

CHAPTER 8

is, τ + ≤ T (z).

(8.6)

− ∞ − Now, let {zl− }∞ l=1 be a subsequence of {zm }m=1 such that T (zl ) → τ − as l → ∞. It follows from (8.3) and (8.6) that τ ∈ R+ . Therefore, − the sequence {(T (zl− ), zl− )}∞ l=1 converges in R+ × Q to (τ , z). Since s is continuous, it follows that s(T (zl− ), zl− ) → s(τ − , z) as l → ∞. Equation (8.4) implies that s(T (zl− ), zl− ) ∈ Q∩f −1 (0) for each l. Hence, s(τ − , z) = ze , ze ∈ Q ∩ f −1 (0) and, by (8.2),

T (z) ≤ τ − .

(8.7)

It follows from (8.3), (8.6), and (8.7) that τ − = τ + = T (z), and hence, T (zm ) → T (z) as m → ∞. Next, we introduce a new definition which is weaker than finite-time semistability and is needed for the next result. Definition 8.2. The system (8.1) is said to be finite-time convergent to M ⊆ f −1 (0) for D0 ⊆ D if, for every x0 ∈ D0 , there exists a finite-time T = T (x0 ) > 0 such that x(t) ∈ M for all t ≥ T . The next result gives a sufficient condition for characterizing finitetime convergence. For the statement of this result, define 1 V˙ (x) , lim+ [V (s(h, x)) − V (x)] , h→0 h

x ∈ D,

(8.8)

for a given continuous function V : D → R and for every x ∈ D such that the limit in (8.8) exists. Proposition 8.1. Let D0 ⊆ D be positively invariant and M ⊆ f −1 (0). Assume that there exists a continuous function V : D0 → R such that V˙ (·) is defined everywhere on D0 , V (x) = 0 if and only if x ∈ M ⊂ D0 , and −c1 |V (x)|α ≤ V˙ (x) ≤ −c2 |V (x)|α ,

x ∈ D0 \M,

(8.9)

where c1 ≥ c2 > 0 and 0 < α < 1. Then (8.1) is finite-time convergent to M for {x ∈ D0 : V (x) ≥ 0}. Alternatively, if V is nonnegative and V˙ (x) ≤ −c3 (V (x))α ,

x ∈ D0 \M,

(8.10)

where c3 > 0, then (8.1) is finite-time convergent to M for D0 . Proof. Note that (8.9) is also true for x ∈ M. Application of the comparison lemma (Theorems 4.1 and 4.2 of [311]) to (8.9) yields µ(t, V (x), c1 ) ≤ V (s(t, x)) ≤ µ(t, V (x), c2 ), x ∈ {z ∈ D0 : V (z) ≥ 0},

NNBook

September 1, 2009

267

FINITE-TIME THERMODYNAMICS

where µ is given by ( 1 |z|1−α , α < 1, (|z|1−α − c(1 − α)t) 1−α , 0 ≤ t < c(1−α) µ(t, z, c) , (8.11) |z|1−α 0, t ≥ c(1−α) , α < 1. Hence, V (s(t, x)) = 0 for t ≥

|V (x)|1−α c2 (1−α) ,

|V (x)|1−α c2 (1−α) .

which implies that s(t, x) ∈ M for

t≥ The assertion follows. The second part of the assertion can be proved similarly. The next result establishes a relationship between finite-time convergence and finite-time semistability.

Theorem 8.1. Assume that there exists a continuous nonnegative function V : D → R+ such that V˙ (·) is defined everywhere on D, V −1 (0) = f −1 (0), and there exists a relatively open neighborhood Q ⊆ D such that Q ∩ f −1 (0) is nonempty and V˙ (x) ≤ w(V (x)),

x ∈ Q\f −1 (0),

(8.12)

where w : R+ → R is continuous, w(0) = 0, and z(t) ˙ = w(z(t)),

z(0) = z0 ∈ R+ ,

t ≥ 0,

(8.13)

has a unique solution in forward time. If (8.13) is finite-time convergent to the origin for R+ and every point in Q ∩ f −1 (0) is a Lyapunov stable equilibrium point of (8.1), then every point in Q ∩ f −1 (0) is finite-time semistable. Moreover, the settling-time function of (8.1) is continuous on a relatively open neighborhood of Q ∩ f −1 (0). Finally, if Q = D, then (8.1) is finite-time semistable. Proof. Consider xe ∈ Q ∩ f −1 (0). Since x(t) ≡ xe is Lyapunov stable, it follows that there exists a relatively open positively invariant set S ⊆ Q containing xe . Next, it follows from (8.12) that V˙ (s(t, x)) ≤ w(V (s(t, x))),

x ∈ S,

t ≥ 0.

(8.14)

Now, application of the comparison lemma (Theorem 4.1 of [311]) to the inequality (8.14) with the comparison system (8.13) yields V (s(t, x)) ≤ ψ(t, V (x)),

t ≥ 0,

x ∈ S,

(8.15)

where ψ : [0, ∞) × R → R is the global semiflow of (8.13). Since (8.13) is finite-time convergent to the origin for R+ , it follows from (8.15) and the nonnegativity of V (·) that V (s(t, x)) = 0,

t ≥ Tˆ(V (x)),

x ∈ S,

where Tˆ(·) denotes the settling-time function of (8.13).

(8.16)

NNBook

September 1, 2009

268

CHAPTER 8

Next, since s(0, x) = x, s(·, ·) is jointly continuous, and V (s(t, x)) = 0 is equivalent to f (s(t, x)) = 0 on S, it follows that inf{t ∈ R+ : f (s(t, x)) = 0} > 0 for x ∈ S\f −1 (0). Furthermore, it follows from (8.16) that inf{t ∈ R+ : f (s(t, x)) = 0} < ∞ for x ∈ S. Define T : S\f −1 (0) → R+ by T (x) = inf{t ∈ R+ : f (s(t, x)) = 0}. Then it follows that every point in S ∩ f −1 (0) is finite-time semistable and T is the settling-time function on S. Furthermore, it follows from (8.16) that T (x) ≤ Tˆ(V (x)), x ∈ S. Since the settling-time function of a one-dimensional finite-time stable system is continuous at the equilibrium, it follows that T is continuous at each point in S ∩ f −1 (0). Since xe ∈ Q ∩ f −1 (0) was chosen arbitrarily, it follows that every point in Q ∩ f −1 (0) is finite-time semistable, while Lemma 8.1 implies that T is continuous on a relatively open neighborhood of Q ∩ f −1 (0). The last statement follows by noting that, if Q = D, then Q is positively invariant by our assumptions on (8.1), and hence, the preceding arguments hold with S = Q. Theorem 8.2. Assume that there exists a continuous nonnegative function V : D → R+ such that V˙ (·) is defined everywhere on D, V −1 (0) = f −1 (0), and there exists a relatively open neighborhood Q ⊆ D such that Q∩f −1 (0) is nonempty and (8.10) holds for all x ∈ Q\f −1 (0). Furthermore, assume that there exists a continuous nonnegative function W : Q → R+ ˙ (·) is defined everywhere on Q, W −1 (0) = Q ∩ f −1 (0), and such that W ˙ (x), kf (x)k ≤ −c0 W

x ∈ Q\f −1 (0),

(8.17)

where c0 > 0. Then every point in Q ∩ f −1 (0) is finite-time semistable. Proof. For any xe ∈ Q ∩ f −1 (0), since W (x) ≥ 0 = W (xe ) for all x ∈ Q, it follows from i) of Theorem 5.2 of [33] that xe is a Lyapunov stable equilibrium and, hence, every point in Q ∩ f −1 (0) is Lyapunov stable. Now, it follows from the second assertion of Proposition 8.1 and Theorem 8.1, with w(x) = −c3 sgn(x)|x|α , that every point in Q ∩ f −1 (0) is finite-time semistable.

8.3 Homogeneity and Finite-Time Semistability In this section, we develop necessary and sufficient conditions for finite-time semistability of homogeneous dynamical systems. In the sequel, we will n need to consider a complete vector field ν on R+ such that the solutions of the differential equation y(t) ˙ = ν(y(t)) define a continuous global flow n n n ψ : R × R+ → R+ on R+ , where ν −1 (0) = f −1 (0). For each τ ∈ R, the map ψτ (·) = ψ(τ, ·) is a homeomorphism and ψτ−1 = ψ−τ . We define a function n V : R+ → R to be homogeneous of degree l ∈ R with respect to ν if and only n if (V ◦ ψτ )(x) = elτ V (x), τ ∈ R, x ∈ R+ . Our assumptions imply that every

NNBook

September 1, 2009

269

FINITE-TIME THERMODYNAMICS n

connected component of R+ \f −1 (0) is invariant under ν. n

The Lie derivative of a continuous function V : R+ → R with respect to ν is given by Lν V (x) , limt→0+ 1t [V (ψ(t, x)) − V (x)], whenever the limit on the right-hand side exists. If V is a continuous homogeneous function of degree l > 0, then Lν V is defined everywhere and satisfies Lν V = lV . We assume that the vector field ν is a semi-Euler vector field, that is, the dynamical system y(t) ˙ = −ν(y(t)),

y(0) = y0 , n

t ≥ 0,

(8.18) n

is globally semistable with respect to R+ . Thus, for each x ∈ R+ , limτ →∞ ψ(−τ, x) = x∗ ∈ ν −1 (0), and for each xe ∈ ν −1 (0), there exists n z ∈ R+ such that xe = limτ →∞ ψ(−τ, z). Finally, we say that the vector field f is homogeneous of degree k ∈ R with respect to ν if and only if ν −1 (0) = f −1 (0) and, for every t ∈ R+ and τ ∈ R, n R+

st ◦ ψτ = ψτ ◦ sekτ t .

(8.19)

→ R is a homogeneous function of degree l such that Note that if V : Lf V (x) is defined everywhere, then Lf V (x) is a homogeneous function of degree l + k.1 Finally, note that if ν and f are continuously differentiable in n a neighborhood of x ∈ R+ , then (8.19) holds at x for sufficiently small t and n τ if and only if [ν, f ](x) = kf (x) in a neighborhood of x ∈ R+ , where the ∂ν Lie bracket [ν, f ] of ν and f can be computed by using [ν, f ] = ∂f ∂x ν − ∂x f . The following lemmas are needed for the main results of this section. n

Lemma 8.2. Consider the dynamical system (8.18). Let Dc ⊂ R+ be a relatively compact set satisfying Dc ∩ ν −1 (0) = Ø. Then for every relatively open set Q satisfying ν −1 (0) ⊂ Q, there exist τ1 , τ2 > 0 such that ψ−t (Dc ) ⊂ Q for all t > τ1 and ψτ (Dc ) ∩ Q = Ø for all τ > τ2 . Proof. Let Q be a relatively open neighborhood of ν −1 (0) with n respect to R+ . Since every z ∈ ν −1 (0) is Lyapunov stable under ν, it follows that there exists a relatively open neighborhood VzScontaining z such that ψ−t (Vz ) ⊆ Q for all t ≥ 0. Hence, V , z∈ν −1 (0) Vz is relatively open and ψ−t (V) ⊆ Q for all t ≥ 0. Next, consider the collection of nested sets {Dt }t>0 , where Dt = {x ∈ Dc : ψh (x) 6∈ V, h ∈ n S [−t, 0]} = Dc ∩ (R+ \( h∈[−t,0] ψh−1 (V))), t > 0. For each t > 0, Dt is 1 In a geometric, coordinate-free setting, the only link between homogeneity of functions and vector fields is that the Lie derivative of a homogeneous function along a homogeneous vector field is also a homogeneous function. In the special case where the coordinate functions are homogeneous functions, the fact mentioned above can be used to relate the homogeneity of a vector field with that of the components (considered as functions) of its coordinate representation. Such a relation is very familiar in the case of conventional dilations seen in the homogeneity literature [255].

NNBook

September 1, 2009

270

CHAPTER 8

a relatively compact set. T Therefore, if Dt is nonempty for each t > 0, then there exists x ∈ t>0 Dt , that is, there exists x ∈ Dc such that ψ−t (x) 6∈ V for all t > 0, which contradicts the fact that the domain n of semistability of (8.18) is R+ . Hence, there exists τ > 0 such that S Dτ = Ø, that is, Dc ⊂ h∈[−τ,0] ψh−1 (V). Therefore, for every t > τ , S S ψ−t (Dc ) ⊂ h∈[−τ,0] ψ−t (ψh−1 (V)) = h∈[−τ,0] ψ−t−h (V) ⊆ Q. The second conclusion follows using similar arguments as above. n

Lemma 8.3. Suppose f : R+ → Rn is homogeneous of degree k ∈ R with respect to ν and (8.1) is (locally) semistable. Then the domain of n semistability of (8.1) is R+ . n

n

Proof. Let A ⊆ R+ be the domain of semistability and x ∈ R+ . Note that A is a relatively open neighborhood of ν −1 (0) with respect to n R+ . Since every point in ν −1 (0) is a globally semistable equilibrium under n −ν with respect to R+ , there exists τ > 0 such that z = ψ−τ (x) ∈ A. Then it follows from (8.19) that s(t, x) = s(t, ψτ (z)) = ψτ (s(ekτ t, z)). Since limt→∞ s(t, z) = x∗ ∈ f −1 (0), it follows that limt→∞ s(t, x) = limt→∞ ψτ (s(ekτ t, z)) = ψτ (limt→∞ s(ekτ t, z)) = ψτ (x∗ ) = x∗ , which implies n n that x ∈ A. Since x ∈ R+ is arbitrary, A = R+ . The following theorem presents a converse Lyapunov result for homogenous semistable systems. n

Theorem 8.3. Suppose f : R+ → Rn is homogeneous of degree k ∈ R with respect to ν and (8.1) is semistable. Then for every l > max{−k, 0}, n there exists a continuous nonnegative function V : R+ → R+ that is homogeneous of degree l with respect to ν, continuously differentiable on n n R+ \f −1 (0), and satisfies V −1 (0) = f −1 (0), V ′ (x)f (x) < 0, x ∈ R+ \f −1 (0), and for each xe ∈ f −1 (0) and each bounded, relatively open neighborhood n D0 containing xe with respect to R+ , there exist c1 = c1 (D0 ) ≥ c2 = c2 (D0 ) > 0 such that −c1 [V (x)]

l+k l

≤ V ′ (x)f (x) ≤ −c2 [V (x)]

l+k l

,

x ∈ D0 .

(8.20)

Proof. Choose l > max{−k, 0}. First, we prove that there exists a n continuous Lyapunov function V on R+ that is homogeneous of degree l with n respect to ν, continuously differentiable on R+ \f −1 (0), and V ′ (x)f (x) < 0 n for x ∈ R+ \f −1 (0). Choose any nondecreasing smooth function g : R+ → [0, 1] such that g(s) = 0 for s ≤ a, g(s) = 1 for s ≥ b, and g′ (s) > 0 on (a, b), where 0 < a < b are constants. It follows from Theorem 2.8 and Lemma 8.3 that there exists a continuously differentiable Lyapunov function U (·) on n R+ satisfying all of the properties in Theorem 2.8.

NNBook

September 1, 2009

271

FINITE-TIME THERMODYNAMICS

Next, define V (x) ,

Z

+∞

e−lτ g(U (ψ(τ, x)))dτ,

−∞

n

x ∈ R+ .

(8.21)

Let Q be a bounded, relatively open set satisfying Q ∩ f −1 (0) = Ø. Since every point in ν −1 (0) is a globally semistable equilibrium point under −ν n with respect to R+ , it follows that for each x ∈ Q, limτ →+∞ U (ψ(τ, x)) = +∞ and limτ →+∞ U (ψ(−τ, x)) = 0. Now, it follows from Lemma 8.2 that there exist time instants τ1 < τ2 such that for each x ∈ Q, U (ψ(τ, x)) ≤ a for all τ ≤ τ1 and U (ψ(τ, x)) ≥ b for all τ ≥ τ2 . Hence, Z τ2 e−lτ2 , x ∈ Q, (8.22) V (x) = e−lτ g(U (ψ(τ, x)))dτ + l τ1 which implies that V is well defined, positive, and continuously differentiable on Q. Next, since U (·) satisfies i) and ii) of Theorem 2.8 it follows from n (8.21) and (8.22) that V −1 (0) = f −1 (0). Since for any σ ∈ R and x ∈ R+ , Z +∞ V (ψ(σ, x)) = e−lτ g(U (ψ(τ + σ, x)))dτ = elσ V (x), (8.23) −∞

by definition, V is homogeneous of degree l. In addition, it follows from (8.19) and (8.22) that Z τ2 d V ′ (x)f (x) = e−lτ g′ (U (ψ(τ, x))) U (s(e−kτ t, ψ(τ, x))) dτ dt t=0 Zτ1τ2 = e−(l+k)τ g′ (U (ψ(τ, x)))U ′ (ψ(τ, x))f (ψ(τ, x))dτ τ1

< 0,

x ∈ Q,

(8.24)

which implies that V ′ f is negative and continuous on Q. Now, since Q is arbitrary, it follows that V is well defined and continuously differentiable, n and V ′ f is negative and continuous on R+ \f −1 (0). n

Next, to show continuity at points in f −1 (0), we define T : R+ \f −1 (0) → R by T (x) = sup{t ∈ R : U (ψ(τ, x)) ≤ a for all τ ≤ t}, and note that the n continuity of U implies that U (ψ(T (x), x)) = a for all x ∈ R+ \f −1 (0). Let n −1 (0) converging to xe ∈ f −1 (0), and consider a sequence {xk }∞ k=1 in R+ \f ∞ xe . We claim that the sequence {T (xk )}k=1 has no bounded subsequence so that limk→∞ T (xk ) = ∞. To prove our claim by contradiction, suppose, ad absurdum, that {T (xki )}∞ i=1 is a bounded subsequence. Without loss of generality, we may assume that the sequence {T (xki )}∞ i=1 converges to h ∈ R. Then, by joint continuity of ψ, limi→∞ ψ(T (xki ), xki ) = ψ(h, xe ) = xe , so that limi→∞ U (ψ(T (xki ), xki )) = U (xe ) = 0. However, this contradicts our

NNBook

September 1, 2009

272

CHAPTER 8 n

observation above that U (ψ(T (x), x)) = a for all x ∈ R+ \f −1 (0). The contradiction leads us to conclude that limk→∞ T (xk ) = ∞. Now, for each k = 1, 2, . . . , it follows that Z ∞ Z ∞ V (xk ) = e−lτ g(U (ψ(τ, xk )))dτ ≤ e−lτ dτ = l−1 e−lT (xk ) , T (xk )

T (xk )

so that limk→∞ V (xk ) = 0 = V (xe ). Since xe was chosen arbitrarily, it follows that V is continuous at every xe ∈ f −1 (0). To show that V possesses the last property, let xe ∈ f −1 (0), and choose n a bounded, relatively open neighborhood D0 of xe with respect to R+ . Let −1 W = ψ(R+ × D0 ). For every ε > 0, denote Wε = W ∩ V (ε). For every ε > n 0, define the continuous map τε : R+ \f −1 (0) → R by τε (x) , l−1 ln(ε/V (x)), n and note that, for every x ∈ R+ \f −1 (0), ψ(t, x) ∈ V −1 (ε) if and only n n if t = τε (x). Next, define βε : R+ \f −1 (0) → R+ by βε , ψ(τε (x), x). Note that, for every ε > 0, βε is continuous, and βε (x) ∈ V −1 (ε) for every n x ∈ R+ \f −1 (0). Consider ε > 0. Wε is the union of the images of connected components of D0 \f −1 (0) under the continuous map βε . Since every n connected component of R+ \f −1 (0) is invariant under ν, it follows that the n image of each connected component Q of R+ \f −1 (0) under βε is contained in Q itself. In particular, the images of connected components of D0 \f −1 (0) under βε are all disjoint. Thus, each connected component of Wε is the image of exactly one connected component of D0 \f −1 (0) under βε . Finally, if ε is small enough so that V −1 (ε)∩D0 is nonempty, then V −1 (ε)∩D0 ⊆ Wε , and hence, every connected component of Wε has a nonempty intersection with D0 \f −1 (0). We claim that Wε is bounded for every ε > 0. It is easy to verify that, for every ε1 , ε2 ∈ (0, ∞), Wε2 = ψh (Wε1 ) with h = l−1 ln(ε2 /ε1 ). Hence, it suffices to prove that there exists ε > 0 such that Wε is bounded. To arrive at a contradiction, suppose, ad absurdum, that Wε is unbounded for every ε > 0. Choose a bounded relatively open neighborhood V of D 0 and a sequence {εi }∞ i=1 in (0, ∞) converging to 0. By our assumption, for every i = 1, 2, . . ., n at least one connected component of Wεi must contain a point in R+ \V. On the other hand, for i sufficiently large, every connected component of Wεi has a nonempty intersection with D0 ⊂ V. It follows that Wεi has a nonempty intersection with the boundary of V for every i sufficiently ∞ large. Hence, there exists a sequence {xi }∞ i=1 in D0 , and a sequence {ti }i=1 in (0, ∞) such that yi , ψti (xi ) ∈ V −1 (εi ) ∩ ∂V for every i = 1, 2, . . .. Since V is bounded, we can assume that the sequence {yi }∞ i=1 converges to y ∈ ∂V. Continuity implies that V (y) = limi→∞ V (yi ) = limi→∞ εi = 0. Since V −1 (0) = f −1 (0) = ν −1 (0), it follows that y is Lyapunov stable under

NNBook

September 1, 2009

273

FINITE-TIME THERMODYNAMICS

−ν. Since y 6∈ D 0 , there exists a relatively open neighborhood Q of y such that Q ∩ D0 = Ø. The sequence {yi }∞ i=1 converges to y while ψ−ti (yi ) = n xi ∈ D0 ⊂ R+ \Q, which contradicts Lyapunov stability. This contradiction implies that there exists ε > 0 such that Wε is bounded. It now follows that Wε is bounded for every ε > 0. Finally, consider x ∈ D0 \f −1 (0). Choose ε > 0 and note that ψτε (x) (x) ∈ Wε . Furthermore, note that V ′ (x)f (x) < 0 for all x ∈ n n R+ \f −1 (0), V ′ (x)f (x) is continuous on R+ \f −1 (0), and W ε ∩ f −1 (0) = Ø. Then, by homogeneity, V (ψτε (x) (x)) = ε, and hence, min V ′ (z)f (z) ≤ V ′ (ψτε (x) (x))f (ψτε (x) (x)) ≤ max V ′ (z)f (z). (8.25)

z∈W ε

Since V that

z∈W ε

′ (ψ τε (x) (x))f (ψτε (x) (x))

is homogeneous of degree l + k, it follows

V ′ (ψτε (x) (x))f (ψτε (x) (x)) = e(l+k)τε (x) V ′ (x)f (x) = ε l+k

l+k l

V (x)−

l+k l

V ′ (x)f (x).

l+k

Let c1 , −ε− l minz∈W ε V ′ (z)f (z) and c2 , −ε− l maxz∈W ε V ′ (z)f (z). Note that c1 and c2 are positive and well defined since W ε is compact. Hence, the theorem is proved. The following result represents the main application of homogeneity [35] to finite-time semistability. Theorem 8.4. Suppose f is homogeneous of degree k ∈ R with respect to ν. Then (8.1) is finite-time semistable if and only if (8.1) is semistable and k < 0. In addition, if (8.1) is finite-time semistable, then the settlingtime function T (·) is homogeneous of degree −k with respect to ν and T (·) n is continuous on R+ . Proof. Since finite-time semistability implies semistability, it suffices to prove that if (8.1) is semistable, then (8.1) is finite-time semistable if and only if k < 0. Suppose (8.1) is finite-time semistable and let l > max{−k, 0}. Then for each xe ∈ f −1 (0), it follows from Theorem 8.3 that there exist a bounded, relatively open, and positively invariant set S containing xe , and a continuous nonnegative function V : S → R+ that is homogeneous of degree l + k and is such that V ′ (x)f (x) is continuous, negative on S\f −1 (0), homogeneous of degree l + k, and (8.20) holds. Now, ad absurdum, if k ≥ 0 and x ∈ S\f −1 (0), then application of the comparison lemma (Theorem 4.2 in [311]) to the first inequality in (8.20) yields V (s(t, x)) ≥ π(t, V (x)), where π is given by ( − 1  α−1 1 + c (α − 1)t , α > 1, sgn(x) 1 |x|α−1 π(t, x) = (8.26) e−c1 t x, α = 1,

NNBook

274

September 1, 2009

CHAPTER 8

and where sgn (x) , x/|x|, x 6= 0, and sgn (0) , 0, with α = 1 + k/l ≥ 1. Since, in this case, π(t, V (x)) > 0 for all t ≥ 0, we have s(t, x) 6∈ S ∩ f −1 (0) for every t ≥ 0; that is, xe is not a finite-time semistable equilibrium under f , which is a contradiction. Hence, k < 0. Conversely, if k < 0, choose xe ∈ f −1 (0) and choose a relatively open neighborhood D0 of xe such that (8.21) holds. Next, Sxe is chosen to be a bounded, positively invariant neighborhood of xe contained in D0 . Then it follows from Theorem 8.3 that there exists a continuous nonnegative function V (·) such that (8.20) holds on Sxe . Now, with c = c2 > 0, 0 < α = 1 + k/l < 1, D0 = Sxe , and w(x) = −csgn(x)|x|α , it follows from PropositionS8.1 and Theorem 8.1 that xe is finite-time semistable on Sxe . Define S , xe ∈f −1 (0) Sxe . Then S is a relatively open neighborhood of f −1 (0) such that every solution in S converges in finite time to a Lyapunov stable equilibrium. Hence, (8.1) is finite-time semistable. Lemma 8.3 then implies that (8.1) is globally finite-time semistable, and T (·) is defined on n R+ . By Proposition 8.1 with D0 = Sxe , and Theorem 8.1, it follows that T (·) is continuous on Sxe . Next, since xe ∈ f −1 (0) was chosen arbitrarily, it n follows from Lemma 8.1 that T (·) is continuous on R+ . n

Finally, let x ∈ R+ and note that, since every point in ν −1 (0) = f −1 (0) n is a globally semistable equilibrium under −ν with respect to R+ , there exists τ > 0 such that z , ψ−τ (x) ∈ S. Then it follows from (8.19) that s(t, x) = s(t, ψτ (z)) = ψτ (s(ekτ t, z)), and hence, f (s(t, x)) = 0 if and only if f (s(ekτ t, z)) = 0. Now, it follows that for x ∈ S, T (ψ−τ (x)) = T (z) = ekτ T (x). By definition, it follows that T (·) is homogeneous of degree −k with respect to ν. In order to use Theorem 8.4 to prove finite-time semistability of a homogeneous system, a priori information of semistability for the system is needed, which is not easy to obtain. To overcome this, we need to develop some sufficient conditions to establish finite-time semistability. Recall that a n function V : R+ → R is said to be weakly proper if and only if for every c ∈ R, n every connected component of the set {x ∈ R+ : V (x) ≤ c} = V −1 ((−∞, c]) is compact [34]. Proposition 8.2. Assume f is homogeneous of degree k < 0 with respect to ν. Furthermore, assume that there exists a weakly proper, n n continuous function V : R+ → R such that V˙ is defined on R+ and satisfies n V˙ (x) ≤ 0 for all x ∈ R+ . If every point in the largest invariant subset N of V˙ −1 (0) is a Lyapunov stable equilibrium point of (8.1), then (8.1) is finite-time semistable.

NNBook

September 1, 2009

275

FINITE-TIME THERMODYNAMICS

Proof. Since V (·) is weakly proper, it follows from Proposition 3.1 n n of [34] that the positive orbit sx ([0, ∞)) of x ∈ R+ is bounded in R+ . Since every solution is bounded, it follows from the hypotheses on V (·) that for n every x ∈ R+ , the omega limit set ω(x) is nonempty and contained in the largest invariant subset N of V˙ −1 (0). Since every point in N is a Lyapunov stable equilibrium point, it follows from Proposition 2.3 that the omega limit n set ω(x) contains a single point for every x ∈ R+ . And since limt→∞ s(t, x) ∈ n N is Lyapunov stable for every x ∈ R+ , by definition, the system (8.1) is semistable. Hence, it follows from Theorem 8.4 that (8.1) is finite-time semistable. Example 8.1. Consider the nonlinear compartmental dynamical system given by 1

1

1 3

1 3

1 3

1 3

x˙ 1 (t) = (x2 (t) − x1 (t)) 3 + (x3 (t) − x1 (t)) 3 ,

x˙ 2 (t) = (x1 (t) − x2 (t)) + (x3 (t) − x2 (t)) , x˙ 3 (t) = (x1 (t) − x3 (t)) + (x2 (t) − x3 (t)) ,

x1 (0) = x10 , x2 (0) = x20 , x3 (0) = x30 ,

t ≥ 0, (8.27) (8.28) (8.29)

where xi ∈ R+ , i = 1, 2, 3. For each a ∈ R+ , x1 = x2 = x3 = a is the equilibrium point of (8.29). We show that all the equilibrium points in (8.29) are finite-time semistable. Note that the vector field f of (8.29) is homogeneous of degree −2 with respect to the semi-Euler vector field,2 ν(x) = (2x1 − x2 − x3 )

∂ ∂ ∂ + (2x2 − x1 − x3 ) + (2x3 − x1 − x2 ) . ∂x1 ∂x2 ∂x3

Furthermore, f (·) is essentially nonnegative. Next, consider V (x) = 12 x21 + 4 1 2 x + 1 x2 . Then V˙ (x(t)) ≤ 0, t ≥ 0, and N = {x ∈ R : x1 = x2 = x3 = a}. 2 2

2 3

+

Now, it follows from the Lyapunov function candidate V (x − ae) = 12 (x1 − 4 4 a)2 + 12 (x2 − a)2 + 12 (x3 − a)2 that V˙ (x − ae) = −(x1 − x2 ) 3 − (x2 − x3 ) 3 − 4 (x3 − x1 ) 3 ≤ 0, which implies that every point in N is a Lyapunov stable equilibrium point of (8.29). Hence, it follows from Proposition 8.2 that the system (8.29) is finite-time semistable. In fact, x1 (t) = x2 (t) = x3 (t) = 1 3 (x10 + x20 + x30 ) for t ≥ T (x0 ). Figure 8.1 shows the state trajectories versus time. △

8.4 Finite-Time Energy Equipartition in Thermodynamic Systems In this section, we develop intercompartmental flow laws that guarantee finite-time semistability and energy equipartition for the thermodynamically consistent dynamical system model developed in Chapter 7. Specifically, 2 The differential operator notation in ν(x) is standard differential geometric notation used to write coordinate expressions for vector fields. This notation is based on the fact that there is a one-to-one correspondence between first-order linear differential operators on real-valued functions and vector fields.

NNBook

September 1, 2009

276

CHAPTER 8 10 x1 x

9

2

x3 8

States

7 6 5 4 3 2 1 0

1

2

3 Time

4

5

6

Figure 8.1 State trajectories versus time for Example 8.1.

consider the dynamical system G given by x˙ i (t) =

q X

φij (xi (t), xj (t)),

xi (t0 ) = xi0 ,

j=1, j6=i

t ≥ t0 ,

i = 1, . . . , q, (8.30)

q

where φij (x), x ∈ R+ , denotes the net energy flow from the jth compartment to the ith compartment defined in Chapter 7. In vector form, (8.30) becomes x(t) ˙ = f (x(t)),

x(t0 ) = x0 , q

t ≥ t0 ,

(8.31) q

where x(t) , [x1 (t), . . . ,P xq (t)]T ∈ R+ , t ≥ t0 , and f = [f1 , . . . , fq ]T : R+ → q R is such that fi (x) = qj=1, j6=i φij (xi , xj ). Theorem 8.5. Consider the dynamical system (8.31) and assume that Axioms i) and ii) of Chapter 7 hold. Furthermore, assume that φij (xi , xj ) = −φji (xj , xi ) for all i, j = 1, . . . , q, i 6= j. Then for every α ∈ R+ , αe is a semistable equilibrium state of (8.31). Furthermore, x(t) → 1q eeT x(t0 ) as t → ∞ and 1q eeT x(t0 ) is a semistable equilibrium state. Proof. The result is a direct consequence of Proposition 7.3 and Theorem 7.13. Theorem 8.5 implies that the steady-state values of the state in each compartment Gi of the dynamical system G are equal, that is, the steady-

NNBook

September 1, 2009

277

FINITE-TIME THERMODYNAMICS

state value of the dynamical system G given by # " q 1X 1 T xi (t0 ) e x∞ = ee x(t0 ) = q q i=1

is uniformly distributed over all compartments of G.

Next, we use the results of Section 8.3 to develop a compartmental model for finite-time thermodynamics. Specifically, consider the dynamical system given by x˙ i (t) =

q X

j=1,j6=i

φij (xi (t), xj (t)),

xi (0) = xi0 ,

t ≥ 0,

(8.32)

where for each i ∈ {1, . . . , q}, xi (t) ∈ R+ denotes an energy state for all t ≥ 0, φij (·, ·) satisfies Axioms i) and ii) of Chapter 7, and φij (xi , xj ) = −φji (xj , xi ) for all i, j = 1, . . . , q, i 6= j. Theorem 8.6. Consider the dynamical system G given by (8.32). Assume that Axioms i) and ii) of Chapter 7 hold, and φij (xi , xj ) = −φji (xj , xi ) for all i, j = 1, . . . , q, i 6= j. Furthermore, assume that the vector field f of the dynamical system (8.32) is homogeneous of degree i Pq h Pq ∂ k ∈ R with respect to ν(x) = − i=1 j=1,j6=i µij (xi , xj ) ∂xi , where x , q

[x1 , . . . , xq ]T ∈ R+ and µij (·, ·) satisfies Axiom ii), µij (xi , xj ) = −µji (xj , xi ), and µij (xi , xj ) = 0 if and only if xi = xj for all i, j = 1, . . . , q, i 6= j. Then, for every xe ∈ R+ , xe e is a finite-time semistable equilibrium state of G if and only if k < 0. Furthermore, if k < 0, then x(t) = 1q eeT x(0) for all t ≥ T (x(0)) and 1q eeT x(0) is a finite-time semistable equilibrium state, where T (x(0)) ≥ 0. q

Proof. Suppose k < 0. It follows from Theorem 8.5 that xe e ∈ R+ , xe ∈ R+ , is a semistable equilibrium state of the homogeneous system (8.32). Furthermore, x(t) → 1q eeT x(0) as t → ∞ and 1q eeT x(0) is a semistable equilibrium state. Next, it can be shown using similar arguments as in the proof of Theorem 8.5 that i (8.18) is globally semistable with ν(x) = Pq h Pq ∂ − i=1 j=1,j6=i µij (xi , xj ) ∂xi . Now, it follows from Theorem 8.4 that xe e isPa finite-time semistable equilibrium state by noting that the vector field qj=1,j6=i φij (xi , xj ) is homogeneous of degree k < 0 with respect to i P h Pq ∂ the semi-Euler vector field ν(x) = − qi=1 µ (x , x ) j=1,j6=i ij i j ∂xi . Hence, with xe = 1q eT x(0), xe e = 1q eeT x(0) is a finite-time semistable equilibrium state. The converse follows as a direct consequence of Theorem 8.4. The following corollary to Theorem 8.6 gives a concrete form for the

NNBook

September 1, 2009

278

CHAPTER 8

energy flow function φij (xi , xj ), i, j = 1, . . . , q, i 6= j. Corollary 8.1. Consider the dynamical system G given by (8.32) with energy flow function φij (xi , xj ) = C(i,j) sgn(xj − xi )|xj − xi |α ,

(8.33)

where α > 0 and C(i,j) is as in (7.37) with C = C T . Assume that Axioms i) and ii) of Chapter 7 hold. Then for every xe ∈ R+ , xe e is a finite-time semistable equilibrium state of G if and only if α < 1. Furthermore, if α < 1, then x(t) = 1q eeT x(0) for all t ≥ T (x(0)) and 1q eeT x(0) is a finitetime semistable equilibrium state, where T (x(0)) ≥ 0. Proof. First, note that the vector field f of G is essentially inonPq hPq ∂ negative. Next, the Lie bracket of ν(x) = − i=1 j=1,j6=i (xj − xi ) ∂xi and the vector field f of the dynamical system (8.32) with (8.33) is given iT hP Pq ∂fq ∂νq q ∂f1 ∂ν1 ν − f , ν − f . Since for each by [ν, f ] = . . . , i i i i i=1 ∂xi i=1 ∂xi ∂xi ∂xi i, j = 1, . . . , q, ∂νj ∂fj νi − fi ∂xi ∂xi hP i  q α−1  C α|x − x | (x − x )  i j s (j,i) s=1,s6=i i  P   + qk=1,k6=i C(i,k) sgn(xk − xi )|xk − xi |α , i 6= j, hP i hP i = q q α−1   s=1,s6=j (xs − xj ) k=1,k6=j C(j,k) α|xk − xj |   Pq  −(q − 1) k=1,k6=j C(j,k) sgn(xk − xj )|xk − xj |α , i = j,

and noting that C(i,j) = C(j,i) , i, j = 1, . . . , q, i 6= j, it follows that for each j = 1, . . . , q, q X ∂fj i=1

∂xi

νi −

q X ∂νj ∂fj ∂νj ∂νj ∂fj fi = νj − fj + νi − fi ∂xi ∂xj ∂xj ∂xi ∂xi i=1,i6=j    q q X X =  C(j,k) α|xk − xj |α−1   (xs − xj ) k=1,k6=j

−(q − 1) +

q X

i=1,i6=j

+

q X

s=1,s6=j

q X

k=1,k6=j

C(j,k)sgn(xk − xj )|xk − xj |α 

C(j,i) α|xi − xj |α−1  q X

i=1,i6=j k=1,k6=i

q X



(xi − xs )

s=1,s6=i

C(i,k) sgn(xk − xi )|xk − xi |α

NNBook

September 1, 2009

279

FINITE-TIME THERMODYNAMICS q X

= α

+

C(j,k) sgn(xk − xj )|xk − xj |α

k=1,k6=j q X

q X

k=1,k6=j s=1,s6=j,k q X

−(q − 1) +α

k=1,k6=j

q X

i=1,i6=j q X

+

C(j,k)sgn(xk − xj )|xk − xj |α

C(j,i) sgn(xi − xj )|xi − xj |α q X

i=1,i6=j s=1,s6=i,j

+

−

q q X X

i=1 k=1,k6=i q X

k=1,k6=j q X

= 2α

i=1,i6=j q X

C(j,i) α|xi − xj |α−1 (xi − xs )

C(i,k) sgn(xk − xi )|xk − xi |α

C(j,k)sgn(xk − xj )|xk − xj |α C(j,i) sgn(xi − xj )|xi − xj |α q X

+α

−q

C(j,k) α|xk − xj |α−1 (xs − xj )

i=1,i6=j s=1,s6=i,j q X

k=1,k6=j

= q(α − 1)

C(j,i) sgn(xi − xj )|xi − xj |α

C(j,k) sgn(xk − xj )|xk − xj |α

q X

i=1,i6=j

C(j,i) sgn(xi − xj )|xi − xj |α

= q(α − 1)fj ,

(8.34)

which implies that the vector field f is homogeneous of degree k = q(α − 1) with respect to the semi-Euler vector field   q q X X ∂  . ν(x) = − (xj − xi ) ∂xi i=1

j=1,j6=i

Now, the result is a direct consequence of Theorem 8.6.

Note that Example 8.1 serves as a special case of Corollary 8.1.

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Nine

Modeling and Analysis of Mass-Action Kinetics

9.1 Introduction Mass-action kinetics are used in chemistry and chemical engineering to describe the dynamics of systems of chemical reactions, that is, reaction networks [281]. These models are a special form of compartmental systems, which involve mass- and energy-balance relations [29, 158]. Aside from their role in chemical engineering applications, mass-action kinetics have numerous analytical properties that are of inherent interest from a dynamical systems perspective. For example, mass-action kinetics give rise to systems of differential equations having polynomial nonlinearities. Polynomial systems are notorious for their intricate analytical properties even in lowdimensional cases [159, 179, 180, 265]. Because of physical considerations, however, mass-action kinetics have special properties, such as nonnegative solutions, that are useful for analyzing their behavior [78, 91, 92, 280]. With this motivation in mind, this chapter has several objectives. First, we provide a general construction of the kinetic equations based on the reaction laws. We present this construction in a state space form that is accessible to the dynamical systems community. This presentation is based on the formulation given in [25, 78]. Next, we consider the nonnegativity of solutions to the kinetic equations. Since the kinetic equations govern the concentrations of the species in the reaction network, it is obvious from physical arguments that nonnegative initial conditions must give rise to trajectories that remain in the nonnegative orthant. To demonstrate this fact, we show that the kinetic equations are essentially nonnegative, and we prove that, for all nonnegative initial conditions, the resulting concentrations are nonegative. A related result is mentioned in [78]. In addition, we consider the realizability problem, which is concerned with the inverse problem of constructing a reaction network having specified essentially nonnegative dynamics. In particular, we provide an explicit construction of a reaction network for essentially nonnegative polynomial

NNBook

September 1, 2009

282

CHAPTER 9

dynamics involving a scalar state. Next, we consider the reducibility of the kinetic equations. In certain cases, such as in enzyme kinetics, the kinetic equations can be reduced in dimensionality by using constants involving initial concentrations. We provide a general statement of this procedure. We then consider the stability of the equilibria of the kinetic equations. To do this, we apply Lyapunov methods to the kinetic equations, and we obtain results that guarantee semistability, that is, convergence to a Lyapunov-stable equilibrium that depends on the initial concentrations. Semistability is the appropriate notion of stability for reaction networks, where the limiting concentrations may be nonzero and may depend on the initial concentrations. Finally, we revisit the zero deficiency result of [81,82], which provides rate-independent conditions that guarantee convergence of the species concentrations. In this regard we have two objectives. First, we present the zero-deficiency result for mass-action kinetics in standard matrix terminology, and, second, we prove semistability using the theory developed in Chapter 2.

9.2 Reaction Networks We begin by reviewing the general formulation of the kinetic equations that describe chemical reactions with mass-action kinetics. First, consider the familiar reaction k 2H2 + O2 → 2H2 O. (9.1) The quantities on the left-hand side of the reaction (9.1) are the reactants, the quantities on the right-hand side are the products, and k denotes the reaction rate. The reactants and products are collectively referred to as the species of the reaction. Equation (9.1) can be rewritten as 3 X j=1

k

Aj Xj →

3 X

Bj Xj ,

(9.2)

j=1

where X1 , X2 , and X3 denote the species H2 , O2 , and H2 O, respectively, A1 = 2, A2 = 1, A3 = 0, B1 = 0, B2 = 0, and B3 = 2 are the stoichiometric coefficients, and k denotes the reaction rate. Note that (9.2) can be written in a compact form using the matrix-vector notation k

AX → BX,

(9.3)

where X = [X1 , X2 , X3 ]T , A = [A1 , A2 , A3 ] = [2, 1, 0], and B = [B1 , B2 , B3 ] = [0, 0, 2]. Next, consider the reversible reaction k1

Na2 CO3 + CaCl2 ⇋ CaCO3 + 2NaCl, k2

(9.4)

NNBook

September 1, 2009

283

MASS-ACTION KINETICS

which is a concise notation for the forward and backward reactions k

Na2 CO3 + CaCl2 →1 CaCO3 + 2NaCl, k

CaCO3 + 2NaCl →2 Na2 CO3 + CaCl2 ,

(9.5) (9.6)

where k1 and k2 are the reaction rates for the forward and backward reactions, respectively. Now, let X1 , X2 , X3 , and X4 denote the species Na2 CO3 , CaCl2 , CaCO3 , and NaCl, respectively, so that (9.4) can be written as k

X1 + X2 →1 X3 + 2X4 ,

(9.7)

X3 + 2X4 →2 X1 + X2 ,

(9.8)

k

or, equivalently, as (9.3), where X = [X1 , X2 , X3 , X4 ]T , k = [k1 , k2 ], and     1 1 0 0 0 0 1 2 A= , B= . 0 0 1 2 1 1 0 0 Next, we formulate the kinetic equations for multiple chemical reactions such as (9.7) and (9.8). Specifically, consider s species X1 , . . . , Xs , where s ≥ 1, whose interactions are governed by r reactions, where r ≥ 1, comprising the reaction network s X j=1

k

Aij Xj →i

s X

Bij Xj ,

i = 1, . . . , r,

(9.9)

j=1

where, for i = 1, . . . , r, ki > 0 is the reaction rate ofPthe ith reaction, P s s j=1 Aij Xj is the reactant of the ith reaction, and j=1 Bij Xj is the product of the ith reaction. Note that each reaction in the reaction network (9.9) is represented as being irreversible. However, reversible reactions can be modeled by including the reverse reaction as a separate reaction, as in the case of the reaction (9.4). Each stoichiometric coefficient Aij and Bij is assumed to be a nonnegative integer. The reaction network (9.9) can be written compactly in matrix-vector form as k

AX → BX,

(9.10)

where X = [X1 , . . . , Xs ]T is a column vector of species, k = [k1 , . . . , kr ]T ∈ r R+ , and A and B denote the r × s nonnegative matrices A = [Aij ] and B = [Bij ]. To avoid vacuous cases, we assume that each species X1 , . . . , Xs appears in the reaction network (9.10) with at least one nonzero coefficient Aij or Bij . This assumption is equivalent to assuming that none of the

NNBook

September 1, 2009

284

CHAPTER 9

columns of 

A B



is zero. Furthermore, in special cases and only when specifically mentioned, we allow ki = 0, which effectively denotes the fact that the ith reaction is absent. Finally, we assume that, for each i = 1, . . . , r, rowi (A) 6= rowi (B) to k

k

avoid trivial reactions of the form X1 → X1 or X1 + X2 → X1 + X2 , whose kinetics equations are x˙ 1 (t) = 0 and x˙ 1 (t) = 0, x˙ 2 (t) = 0, respectively.

9.3 The Law of Mass Action and the Kinetic Equations To derive the dynamics of the reaction network, we invoke the law of mass action [281], which states that, for an elementary reaction, that is, a reaction in which all of the stoichiometric coefficients of the reactants are 1, the rate of reaction is proportional to the product of the concentrations of the reactants. In particular, consider the reaction k

X1 + X2 → bX3 ,

(9.11)

where X1 , X2 , X3 are the species and b is a positive integer. Then x˙ i (t) = −kx1 (t)x2 (t), xi (0) = xi0 , x˙ 3 (t) = bkx1 (t)x2 (t), x3 (0) = x30 ,

t ≥ 0,

i = 1, 2,

(9.12) (9.13)

where xi (t), i = 1, 2, 3, denotes the concentration of the species Xi . Now, writing (9.1) as the elementary reaction k

H2 + H2 + O2 → 2H2 O,

(9.14)

the law of mass action implies that x˙ 1 (t) = −2kx21 (t)x2 (t), x1 (0) = x10 , x˙ 2 (t) = −kx21 (t)x2 (t), x2 (0) = x20 , x˙ 3 (t) = 2kx21 (t)x2 (t), x3 (0) = x30 ,

t ≥ 0,

(9.15) (9.16) (9.17)

where x1 (t), x2 (t), and x3 (t) denote the concentrations of H2 , O2 , and H2 O, respectively, at time t. Similarly, let xi (t) denote the concentration of Xi , i = 1, . . . , 4, in (9.7) and (9.8) or, equivalently, the reversible reaction (9.4). In this case, it follows from the law of mass action that x˙ 1 (t) x˙ 2 (t) x˙ 3 (t) x˙ 4 (t)

= = = =

−k1 x1 (t)x2 (t) + k2 x3 (t)x24 (t), x1 (0) = x10 , t ≥ 0, −k1 x1 (t)x2 (t) + k2 x3 (t)x24 (t), x2 (0) = x20 , k1 x1 (t)x2 (t) − k2 x3 (t)x24 (t), x3 (0) = x30 , 2k1 x1 (t)x2 (t) − 2k2 x3 (t)x24 (t), x4 (0) = x40 .

(9.18) (9.19) (9.20) (9.21)

NNBook

September 1, 2009

285

MASS-ACTION KINETICS

To consider general reaction networks the following notation is needed. For x = [x1 , . . . , xq ]T ∈ Rq and nonnegative matrix A = [A(i,j) ] ∈ Rp×q , xA denotes the element of Rp whose ith component for i = 1, . . . , p is the Aiq i1 product xA 1 · · · xq . For example, if   1 2 , A= 3 4 then A

x =



x1 x22 x31 x42



.

We define 00 = 1. The matrix exponentiation operation has many convenient properties [23, 197]. For example, if A, B ∈ Rp×q , then x(A+B) = xA xB . If B ∈ Rn×p , then (xA )B = xBA . Furthermore, (x ◦ y)A = (xA ) ◦ (y A ) = xA y A . Note that xIp = x and x−A ◦ (xA ) = e. Alternatively, if A ∈ Rp×p , then x−Ip ◦ xA = xA−Ip . Furthermore, if det A 6= 0, x >> 0, and −1 y >> 0, then xA = y implies that y = xA . In addition, loge xA = Aloge x and eAloge x = xA , while xA = y implies Aloge x = loge y, where, for x = [x1 , . . . , xs ]T ∈ Rp+ , loge x denotes the vector in Rp whose ith component is loge xi . Finally, if f (x) = xA , then f ′ (x) = diag(xA )A[diag(x)]−1 , where   x1 . . . 0 △   diag[x1 , x2 , . . . , xn ] =  ... . . . ...  . 0 . . . xn △

For x = [x1 , . . . , xs ]T ∈ Rp , ex denotes the vector in Rp whose ith component is exi .

Now, more generally, consider the reaction (9.10) and, for j = 1, . . . , s, let xj (t) denote the concentration of the species Xj at time t. Then, by applying the law of mass action, the dynamics of the reaction network (9.10) are given by the kinetic equations x(t) ˙ = (B − A)T [k ◦ xA (t)],

x(0) = x0 ,

Defining K , diag[k1 , . . . , kr ], (9.22) can be written as x(t) ˙ = (B − A)T KxA (t),

x(0) = x0 ,

t ≥ 0. t ≥ 0.

(9.22)

(9.23)

P In mass-action kinetics the reaction order sj=1 Aij of the ith reaction is the sum of the stoichiometric coefficients of the species appearing in the reactant of the ith reaction. Equation (9.22), which is equivalent to (4.7) of [78], is a matrix-vector formulation of mass-action kinetics. It can be seen that the kinetic equations (9.22) are linear if and only if each row of A contains exactly one 1 with the remaining entries equal to zero, that is, if and only if each reaction is unimolecular. In this case, it can be seen that

NNBook

September 1, 2009

286

CHAPTER 9

xA = Ax, and thus (9.22) becomes x(t) ˙ = M x(t),

x(0) = x0 ,

where M ∈ Rs×s is defined by

t ≥ 0,

(9.24)

M , (B − A)T KA.

(9.25)

The reaction network (9.10) is not limited to closed systems for which conservation of mass holds. In fact, (9.10) can also be used to represent open systems in which mass removal and mass addition are allowed. For example, k either A = 0 or B = 0 (but not both) is allowed in the reaction AX1 →1 BX1 . k

k

The kinetic equations for the reactions X1 →1 0 and 0 →1 X1 , which represent the removal and addition of mass, are x˙ 1 (t) = −k1 x1 (t) and x˙ 1 (t) = k1 with solutions x1 (t) = x1 (0)e−k1 t and x1 (t) = k1 t + x1 (0), respectively. k

k

The reactions X1 →1 2X1 and 2X1 →1 3X1 , which also represent the addition of mass, have the kinetics x˙ 1 (t) = k1 x1 (t) and x˙ 1 (t) = k1 x21 (t) with solutions x1 (t) = x1 (0)ek1 t and x1 (t) = x1 (0)/(1 − k1 x1 (0)t), respectively. Note that the latter solution has finite escape time since it exists only on k k the interval [0, 1/(k1 x1 (0))). Finally, the reactions X → Y and 2X → 2Y , although stoichiometrically equivalent, have different kinetic equations, namely, x(t) ˙ = −kx(t), y(t) ˙ = kx(t) and x(t) ˙ = −kx2 (t), y(t) ˙ = kx2 (t), respectively. We adopt the convention that the law of mass action applies to the reaction involving the minimum number of molecules necessary for the reaction to occur. Example 9.1. Consider the reaction network k

X1 →1 X2 ,

(9.26)

X2 → X1 ,

(9.27)

k2

so that s = 2, r = 2, and A and B are given by     1 0 0 1 A= , B= . 0 1 1 0

(9.28)

The kinetic equations are thus given by x˙ 1 (t) = −k1 x1 (t) + k2 x2 (t), x1 (0) = x10 , x˙ 2 (t) = k1 x1 (t) − k2 x2 (t), x2 (0) = x20 , or in linear system form by (9.24), where   −k1 k2 M= . k1 −k2

t ≥ 0,

(9.29) (9.30)

(9.31) △

NNBook

September 1, 2009

287

MASS-ACTION KINETICS

Example 9.2. Consider the reaction network k

X1 + X2 →1 2X1 ,

(9.32)

2X1 →2 X1 + X2 ,

(9.33)

k

so that s = 2, r = 2, A=



1 1 2 0



,

B=



2 0 1 1



.

(9.34)

The kinetic equations are thus given by x˙ 1 (t) = k1 x1 (t)x2 (t) − k2 x21 (t), x1 (0) = x10 , t ≥ 0, x˙ 2 (t) = −k1 x1 (t)x2 (t) + k2 x21 (t), x2 (0) = x20 .

(9.35) (9.36) △

Example 9.3. The Lotka-Volterra reaction is given by X1 →1 2X1 ,

k

(9.37)

k2

X1 + X2 → 2X2 ,

(9.38)

k3

(9.39)

X2 → 0,

where x1 and x2 denote prey and predator species, respectively, so that s = 2 and r = 3. Furthermore, A and B are given by     1 0 2 0 (9.40) A =  1 1 , B =  0 2 . 0 1 0 0 Consequently, the kinetic equations have the form       k1 x1 (t) x10 1 −1 0  k2 x1 (t)x2 (t)  , x(0) = , x(t) ˙ = x20 0 1 −1 k3 x2 (t)

t ≥ 0, (9.41)

or, equivalently, x˙ 1 (t) = k1 x1 (t) − k2 x1 (t)x2 (t), x1 (0) = x10 , t ≥ 0, x˙ 2 (t) = −k3 x2 (t) + k2 x1 (t)x2 (t), x2 (0) = x20 .

(9.42) (9.43) △

Example 9.4. A widely studied reaction network [201] involves the interaction of a substrate S and an enzyme E to produce a product P by means of an intermediate species C. The reactions are given by S+E

k1

⇋ k2

k

C →3 P + E

(9.44)

NNBook

September 1, 2009

288

CHAPTER 9

so that s = 4 and r = 3. Letting X1 = S, X2 = C, X3 = E, and X4 = P, the corresponding reaction network can be written as k

X1 + X3 →1 X2 ,

(9.45)

X2 → X1 + X3 ,

(9.46)

X2 → X3 + X4 .

(9.47)

k2

k3

It thus follows that A and  1 0 A= 0 1 0 1

B are given by    1 0 0 1 0 0 0 0 , B =  1 0 1 0 . 0 0 0 0 1 1

Consequently, the kinetic equations have the form     −1 1 0  k x (t)x (t) 1 1 3   1 −1 −1    , x(0) =  k x (t) x(t) ˙ = 2 2   −1 1 1  k3 x2 (t) 0 0 1

 x10 x20  , x30  x40

(9.48)

t ≥ 0, (9.49)

or, equivalently, x˙ 1 (t) x˙ 2 (t) x˙ 3 (t) x˙ 4 (t)

= = = =

k2 x2 (t) − k1 x1 (t)x3 (t), x1 (0) = x10 , t ≥ 0, −(k2 + k3 )x2 (t) + k1 x1 (t)x3 (t), x2 (0) = x20 , (k2 + k3 )x2 (t) − k1 x1 (t)x3 (t), x3 (0) = x30 , k3 x2 (t), x4 (0) = x40 .

(9.50) (9.51) (9.52) (9.53) △

9.4 Nonnegativity of Solutions Since the states of the kinetic equations (9.22) represent concentrations, it is natural to expect that, for nonnegative initial concentrations, the concentrations remain nonnegative for as long as the solution exists. In this section, we show that the kinetic equations (9.22) are in fact essentially nonnegative, and hence, nonnegativity of solutions is a direct consequence of Proposition 2.1. For the following results we consider the system x(t) ˙ = f (x(t)),

x(0) = x0 ,

t ∈ [0, τx0 ),

(9.54)

where f : D → Rn is locally Lipschitz continuous, D is an open subset of Rn , x0 ∈ D, and [0, τx0 ), where 0 < τx0 ≤ ∞, is the maximal interval of existence for the solution x(·) of (9.54).

NNBook

September 1, 2009

289

MASS-ACTION KINETICS

Proposition 9.1. Define f : Rs → Rs by f (x) = (B − A)T (k ◦ xA ). Then f is locally Lipschitz continuous and essentially nonnegative. Proof. Since f is continuously differentiable it follows that f is locally n Lipschitz continuous. Next, let x ∈ R+ . For j ∈ {1, . . . , s} we have  k1 xrow1 (A)   .. fj (x) = [colj (B) − colj (A)]T   . row r (A) kr x r r X X Aij ki xrowi (A) . = Bij ki xrowi (A) − 

i=1

i=1

Note that the first summation is nonnegative since x is nonnegative. Next, A note that Aij ki xrowi (A) contains the factor Aij xj ij . Now, to verify essential A

nonnegativity, let xj = 0. If Aij > 0, then Aij xj ij = Aij (0Aij ) = 0, while, A

if Aij = 0, then Aij xj ij = limxj →0 0(x0j ) = limxj →0 0(1) = 0. Consequently, the second summation is zero for all nonnegative A1j , . . . , Arj whenever xj = 0. Thus, f is essentially nonnegative. n

Theorem 9.1. R+ is an invariant set with respect to (9.22). Proof. The result is an immediate consequence of Propositions 2.1 and 9.1. Corollary 9.1. Consider the linear kinetic reaction (9.24), where M = (B − A)T KA and A has exactly one nonzero entry in each row. Then s f (x) = M x is essentially nonnegative and R+ is an invariant set with respect to (9.24). Proof. Since A is nonnegative, K is nonnegative and diagonal, and A has exactly one nonzero entry in each row, it follows that AT KA is diagonal and nonnegative. Now, since B T KA is nonnegative it follows that M is essentially nonnegative, and hence, f (x) = M x is essentially nonnegative. s The invariance of R+ is a direct consequence of Theorem 9.1. In the linear case f (x) = M x, where M ∈ Rn×n is essentially nonnegative, Theorem 9.1 implies the following result. Proposition 9.2. Let M ∈ Rn×n . Then M is essentially nonnegative if and only if eM t ≥≥ 0 for all t ≥ 0. Proof. This is a restatement of Proposition 2.5.

NNBook

September 1, 2009

290

CHAPTER 9

Example 9.5. Consider Example 9.1. For the kinetic equations (9.24) with M given by (9.31) it can be seen that   −k1 k2 M= k1 −k2 is essentially nonnegative. The exponential of M is given by eM t = I2 + 1−e−(k1 +k2 )t M , which is nonnegative for all t ≥ 0. Consequently, if x(0) is k1 +k2 nonnegative, then the solution x(·) of (9.24) given by x(t) = eM t x(0) is nonnegative for all t ≥ 0. △ Example 9.6. Consider Examples 9.2–9.4. It can be seen that the function f for each of these examples is essentially nonnegative. △

9.5 Realization of Mass-Action Kinetics In this section, we consider the realization problem, which is concerned with the construction of a reaction network whose dynamics are given by specified kinetic equations. In this case, the reaction network is a realization of the kinetic equations. Note that the polynomial f (x) =

ν X

ai xi

(9.55)

i=0

in the real scalar x is essentially nonnegative if and only if a0 ≥ 0. Theorem 9.2. Consider the system (9.54), where n = 1 and f : R → R is an essentially nonnegative polynomial of degree ν of the form (9.55). Then there exists a reaction network of the form (9.10) with s = 1 and r ≤ ν + 1, and with stoichiometric coefficient matrices A and
B having nonnegative integer entries such that f (x) = (B − A)T k ◦ xA . ν+1

Proof. For i = 1, . . . , ν, define A, B, and k ∈ R+ △

Ai = i,

△

Bi = (i + sgn ai ),

△

ki = |ai |,

△

as

(9.56)

where sgn 0 = 0. Note that A ≥≥ 0 and, since a0 ≥ 0, it follows that B ≥≥ 0. Then the dynamics of the reaction network (9.10) are given by the kinetic equation   x(t) ˙ = (B − A)T k ◦ xA (t) ν X = (Bi − Ai )ki xAi (t) i=0

ν X = (sgn ai )|ai |xi (t) i=0

NNBook

September 1, 2009

291

MASS-ACTION KINETICS

=

ν X

ai xi (t)

i=0

= f (x(t)). Hence, (9.10) is a realization of (9.22), where f is given by (9.55). To demonstrate Theorem 9.2, let ν = 3. Then a realization of x˙ 1 (t) = a3 x31 (t) + a2 x21 (t) + a1 x1 (t) + a0 is given by the reaction network a

0 →0 X1 ,

(9.57)

|a1 |

X1 → (1 + sgn a1 )X1 ,

(9.58)

2X1 → (2 + sgn a2 )X1 ,

|a2 |

(9.59)

3X1 → (3 + sgn a3 )X1 ,

|a3 |

(9.60)

where we follow the convention that any reaction with rate constant 0 is 0 removed from the network to avoid trivial reactions of the form 0 → aX and 0 aX → aX. If n ≥ 2 and f is an essentially nonnegative multivariate polynomial in x1 , . . . , xn , then there does
not necessarily exist a reaction network such that f (x) = (B − A)T k ◦ xA . For example, consider the case n = 2 and the dynamic equations x˙ 1 (t) = x22 (t) − 2x32 (t) + x42 (t), x˙ 2 (t) = 0, x2 (0) = x20 . Then f (x1 , x2 ) =



x1 (0) = x10 ,

x22 − 2x32 + x42 0

t ≥ 0,

(9.61) (9.62)



is essentially nonnegative. However, (9.61) and (9.62) cannot be realized as a reaction network. To see this, suppose that (9.61) and (9.62) are the kinetic equations for a reaction network of r reactions involving the species X1 and X2 . Since f (·, ·) is independent of x1 , it follows that the reaction network must have the form k

ai X2 →i bi X1 + ci X2 ,

(9.63)

where ai , bi , and ci are nonnegative integers and ki ≥ 0 for all i = 1, . . . , r. Now, it follows from the law of mass action that the kinetic equations for (9.63) are given by x˙ 1 (t) =

r X i=1

bi ki xa2i (t),

x1 (0) = x10 ,

t ≥ 0,

(9.64)

NNBook

September 1, 2009

292

CHAPTER 9 r X x˙ 2 (t) = (ci − ai )ki xa2i (t),

x2 (0) = x20 .

(9.65)

i=1

Comparing (9.61) with (9.65), it follows that ai ∈ {2, 3, 4} for all i = 1, . . . , r. P △ Furthermore, i∈R bi ki = −2, where R = {i ∈ {1, . . . , r} : ai = 3}, which is a contradiction since bi ≥ 0 and ki ≥ 0 for all i = 1, . . . , r. Next, we present a necessary and sufficient condition that guarantees
a reaction network realization exists such that f (x) = (B − A)T k ◦ xA .

Theorem 9.3. Consider the system (9.54), where n > 1 and f : Rn → is a multivariate polynomial. Then there exists a reaction
network of the form (9.10) with s = n such that f (x) = (B − A)T k ◦ xA , where the stoichiometric coefficient matrices A and B have nonnegative integer entries, if and only if for each j ∈ {1, . . . , n}, fj (x1 , x2 , . . . , xj−1 , 0, xj+1 , . . . , xn ) is a multivariate polynomial with nonnegative integer coefficients.

Rn

Proof. To prove sufficiency, let j ∈ {1, . . . , n}. By assumption, fj (x) is a sum of terms either of the form p

aj xp11 xp22 · · · xj j · · · xpnn ,

(9.66)

where pi ≥ 0 for all i = 1, . . . , n and pj > 0, or of the form q

q

j−1 j+1 bj xq11 · · · xj−1 xj+1 · · · xqnn ,

(9.67)

with bj > 0. Next, note that the reaction n X i=1

|aj |

pi Xi →(pj + sgn aj )Xj +

n X

pi Xi

(9.68)

i=1,i6=j

contributes the term (9.66) to x˙ j and no terms to x˙ i for all i = 1, . . . , n such that i 6= j. Similarly, the reaction n X i=1

bj

qi Xi → Xj +

n X

qi Xi

(9.69)

i=1,i6=j

contributes the term (9.67) to the rate of x˙ j and zero terms to x˙ i for all i = 1, . . . , n, i 6= j. Hence, for all j = 1, . . . , n, each term of fj (x) can be realized as a valid reaction which establishes sufficiency. s

To prove necessity, let x ∈ R+ and let j ∈ {1, . . . , s}. Then fj (x) =

r X (Bij − Aij )ki xrowi (A) . i=1

Let xj = 0. If Aij > 0, then xrowi (A) and hence (Bij − Aij )ki xrowi (A) = 0,

NNBook

September 1, 2009

293

MASS-ACTION KINETICS

whereas, if Aij = 0, then A

A

i(j−1) i(j+1) i1 in (Bij − Aij )ki xrowi (A) = lim + Bij ki xA x0j xj+1 · · · xA n . 1 · · · xj−1

xj →0

Hence, fj (x) =

X

i∈Ij

A

A

i(j−1) i(j+1) i1 in xj+1 Bij ki xA · · · xA n , 1 · · · xj−1

△

where Ij = {i ∈ {1, . . . , r} : Aij = 0}, establishing the result.

9.6 Reducibility of the Kinetic Equations In this section, we provide a technique for reducing the number of kinetic equations needed to model the dynamics of the reaction network (9.10). The reduced-order kinetic equations model a subset of the species appearing in the original reaction network. This technique is based on the fact that, while x(t), t ≥ 0, is confined to the nonnegative orthant for nonnegative initial conditions, the structure of the kinetic equations (9.22) impose an additional constraint on the allowable trajectories. To state this result we define the stoichiometric subspace S by S , R((B − A)T ), which is a subspace of Rs . The dimension of this subspace is given by q , rank((B − A)T ) = rank(B − A), which is the rank of the reaction network. Note that q ≤ min{r, s}. The following result shows that the solution of the kinetic equations (9.22) is confined to an affine subspace that is parallel to the stoichiometric subspace. For convenience, we let P ∈ Rs×s denote the unique orthogonal projector whose range is S, and define P⊥ , Is −P . In terms of the generalized inverse (·)+ , P is given by P = (B − A)T [(B − A)T ]+ = (B − A)+ (B − A). Note that, if z ∈ Rs , then P z = z if and only if z ∈ S, and therefore P⊥ z = 0 if and only if z ∈ S. s

Proposition 9.3. Suppose x(0) ∈ R+ . Then, for all t ∈ [0, τx(0) ), the solution x(·) of (9.22) satisfies s

x(t) ∈ (x(0) + S) ∩ R+ .

(9.70)

Proof. It follows from Proposition 9.1 that, for all t ∈ [0, τx(0) ), x(t) is confined to the nonnegative orthant. To show that x(t) ∈ x(0) + S for all t ∈ [0, τx(0) ), note that x(t) ˙ ∈ S for all t ∈ [0, τx(0) ), which implies that d ˙ = 0 for all t ∈ [0, τx(0) ). Hence, P⊥ [x(t) − x(0)] dt P⊥ [x(t) − x(0)] = P⊥ x(t) is constant for all t ∈ [0, τx(0) ). Thus, for all t ∈ [0, τx(0) ), it follows that P⊥ [x(t) − x(0)] = P⊥ [x(0) − x(0)] = 0, and hence, x(t) − x(0) ∈ S, as required. s

s

Corollary 9.2. Suppose x(0) ∈ R+ . Then (x(0) + S) ∩ R+ is an invariant set with respect to (9.22).

NNBook

September 1, 2009

294

CHAPTER 9 s

Proof. Let x ˆ(0) ∈ (x(0) + S) ∩ R+ so that x ˆ(0) = x(0) + w, where w ∈ S, and let x ˆ(·) denote the corresponding solution to (9.22). Then, since s x ˆ(0) ∈ R+ , it follows from Proposition 9.3 that, for all t ∈ [0, τxˆ(0) ), s

s

s

x ˆ(t) ∈ (ˆ x(0) + S) ∩ R+ = (x(0) + w + S) ∩ R+ = (x(0) + S) ∩ R+ . (9.71) This completes the proof. Proposition 9.3 shows that the solution x(·) of the kinetic equations s (9.22) is confined to the stoichiometric compatibility class (x(0) + S) ∩ R+ , which is a q-dimensional manifold with boundary. (The set (x(0) + S) ∩ Rs+ is a positive stoichiometric compatibility class.) This fact suggests that the dynamics of the reaction network can be represented by a set of q species. In fact, the following result shows that, if q < s, then the number of species can be reduced from s to q. Since q ≤ min{r, s}, this reduction is always possible when r < s. For convenience, the following result assumes that the species x1 , . . . , xs are labeled such that the first q columns of B − A are linearly independent. Proposition 9.4. Assume that q < s. Furthermore, partition A = [A1 , A2 ] and B = [B1 , B2 ], where A1 , B1 ∈ Rr×q , and assume that rank(B1 − A1 ) = q. In addition, let F ∈ Rq×(s−q) satisfy A2 −B2 = (A1 −B1 )F . Finally, T partition x = [ˆ xT ˆT ˆ1 , [x1 , . . . , xq ]T and x ˆ2 , [xq+1 , . . . , xs ]T . 1 ,x 2 ] , where x Then x ˆ2 (t) = F T x ˆ1 (t) + γ, x ˆ2 (0) = x ˆ20 , t ≥ 0, (9.72) where γ , x ˆ2 (0) − F T x ˆ1 (0) ∈ Rs−q , and x ˆ1 (·) satisfies T 1 ˆ10 , t ≥ 0. (9.73) x ˆ˙ 1 (t) = (B1 −A1 )T [k◦ x ˆA ˆ1 (t)+γ)A2 ], x ˆ1 (0) = x 1 (t)◦(F x

Proof. Left multiplying (9.22) by [F T , −Is−q ] yields x ˆ˙ 2 (t) = F T x ˆ˙ 1 (t), which implies (9.72). Next, note that x ˆ˙ 1 (t) = (B1 − A1 )T [k ◦ xA (t)] = A1 A2 T (B1 − A1 ) [k ◦ x ˆ1 (t) ◦ x ˆ2 (t)], which, with (9.72), yields (9.73). Example 9.7. Consider Example 9.1. Note that s = 2, r = 2, and q = 1 < s, and thus Proposition 9.4 can be applied with F = −1. It thus △ follows that x2 (t) = −x1 (t) + γ for all t ≥ 0, where γ = x1 (0) + x2 (0). Application of Proposition 9.4 with x ˆ1 = x1 and x ˆ2 = x2 , (9.73) yields the scalar kinetic equation x˙ 1 (t) = −(k1 + k2 )x1 (t) + k2 γ,

x1 (0) = x10 ,

t ≥ 0,

(9.74)

which is essentially nonnegative. A reduced reaction network realization for these kinetic equations is given by 0 X1

k2 γ

→

X1 ,

(9.75)

→

0,

(9.76)

k1 +k2

NNBook

September 1, 2009

295

MASS-ACTION KINETICS

△

for which q = s = 1 and r = 2.

Example 9.8. Consider Example 9.2. Note that s = 2, r = 2, and q = 1 < s, and thus Proposition 9.4 can be applied with F = −1. It thus △ follows that x2 (t) = −x1 (t) + γ for all t ≥ 0, where γ = x1 (0) + x2 (0). Application of Proposition 9.4 with x ˆ1 = x1 and x ˆ2 = x2 , (9.73) yields the scalar kinetic equation x˙ 1 (t) = −(k1 + k2 )x21 (t) + k1 γx1 (t),

x1 (0) = x10 ,

t ≥ 0,

(9.77)

which is essentially nonnegative. A reaction network realization for this reduced-order kinetic equation is given by X1 2X1

k1 γ

→

2X1 ,

(9.78)

→

X1 ,

(9.79)

k1 +k2

△

for which q = s = 1 and r = 2.

Example 9.9. Consider Example 9.3. Note that s = 2, r = 3, and q = 2 = s, and thus reduction is not possible. △ Example 9.10. Consider Example 9.4. Note that s = 4, r = 3, and q = 2 < s, and thus Proposition 9.4 can be applied with   0 −1 F = . −1 −1 It thus follows that x3 (t) = −x2 (t) + γ1 and x4 (t) = −x1 (t) − x2 (t) + γ2 △ △ for all t ≥ 0, where γ1 = x2 (0) + x3 (0) and γ2 = x1 (0) + x2 (0) + x4 (0). Application of Proposition 9.4 with x ˆ1 = [x1 , x2 ]T and x ˆ2 = [x3 , x4 ]T , (9.73) yields the kinetic equations x˙ 1 (t) = −k1 γ1 x1 (t) + k2 x2 (t) + k1 x1 (t)x2 (t),

t ≥ 0, (9.80) x2 (0) = x20 , (9.81)

x1 (0) = x10 ,

x˙ 2 (t) = k1 γ1 x1 (t) − (k2 + k3 )x2 (t) − k1 x1 (t)x2 (t),

which is essentially nonnegative. The dynamics of the system (9.80) and (9.81) are discussed in [201] and the references given therein. A reaction network realization for these reduced-order kinetic equations is given by k1 γ1

X1 → X2 ,

(9.82)

X2 → X1 ,

(9.83)

X2 → 0,

(9.84)

k2 k3

k1

X1 + X2 → 2X1 , for which q = s = 2 and r = 4.

(9.85) △

NNBook

September 1, 2009

296

CHAPTER 9

Example 9.11. We now show that not every reduced-order kinetic equation can be realized as a reaction network. For convenience, we relabel the species of Example 9.4 as X1 = S, X2 = P, X3 = C, and X4 = E. The reaction network (9.45)–(9.47) can now be written as k

X1 + X4 →1 X3 ,

(9.86)

X3 →2 X1 + X4 ,

(9.87)

X3 →3 X4 + X2 ,

(9.88)

k

k

whose kinetic equations are x˙ 1 (t) x˙ 2 (t) x˙ 3 (t) x˙ 4 (t)

= = = =

−k1 x1 (t)x4 (t) + k2 x3 (t), x1 (0) = x10 , t ≥ 0, k3 x3 (t), x2 (0) = x20 , k1 x1 (t)x4 (t) − (k2 + k2 )x3 (t), x3 (0) = x30 , −k1 x1 x4 (t) + (k2 + k2 )x3 (t), x4 (0) = x40 .

(9.89) (9.90) (9.91) (9.92)

Since s = 4, r = 3, and q = 2 < s, Proposition 9.4 can be applied with x ˆ1 = [x1 , x2 ]T , x ˆ2 = [x3 , x4 ]T , and   −1 1 F = . −1 1 It thus follows that x3 (t) = −x1 (t) − x2 (t) + γ1 and x4 (t) = x1 (t) + x2 (t) + γ2 △ △ for all t ≥ 0, where γ1 = x1 (0)+x2 (0)+x3 (0) and γ2 = x4 (0)−x1 (0)−x2 (0). By applying Proposition 9.4, it follows from (9.73) that x˙ 1 (t) = −k1 x21 (t) − k1 x1 (t)x2 (t) − (k1 γ2 + k2 )x1 (t) − k2 x2 (t) + k2 γ1 , (9.93) x1 (0) = x10 , t ≥ 0, x˙ 2 (t) = −k3 x1 (t) − k3 x2 (t) + k3 γ1 , x2 (0) = x20 , (9.94) which have nonnegative solutions as long as the initial conditions coincide with the initial conditions of the original kinetic equations (9.89)–(9.92). However, due to the terms −k2 x2 and −k3 x1 , (9.93) and (9.94) are not essentially nonnegative, and hence, solutions may become nonnegative. Therefore, (9.93) and (9.94) are not realizable by a reaction network. △ The following proposition presents conditions that guarantee nonnegativity of the solutions to reduced-order kinetic equations. Proposition 9.5. Assume that q < s. Furthermore, partition A = [A1 , A2 ] and B = [B1 , B2 ], where A1 , B1 ∈ Rr×q , and assume that rank(B1 − A1 ) = q. In addition, let F ∈ Rq×(s−q) satisfy A2 −B2 = (A1 −B1 )F . Finally, T partition x = [ˆ xT ˆT ˆ1 , [x1 , . . . , xq ]T and x ˆ2 , [xq+1 , . . . , xs ]T . 1 ,x 2 ] , where x q s−q Then, for all x ˆ1 (0) ∈ R+ and γ ∈ Rs−q such that γ + F T x ˆ1 (0) ∈ R+ , the solution x ˆ1 (t) to (9.73) is nonnegative for all t ≥ 0.

NNBook

September 1, 2009

297

MASS-ACTION KINETICS

Proof. With x ˆ2 (0) = γ + F T x ˆ1 (0), it follows from Proposition 9.4 T that the solution to (9.22) is given by [ˆ xT ˆT 1 (t), x 2 (t)] for all t ≥ 0, where x ˆ2 (t) is given by (9.72). Hence, since x ˆ1 (0) ≥≥ 0 and x ˆ2 (0) ≥≥ 0, it follows that x ˆ1 (t) ≥≥ 0 for all t ≥ 0.

9.7 Stability Analysis of Linear and Nonlinear Kinetics We now consider the stability of equilibria of the kinetic equations (9.22). The following definition defines equilibria for the kinetic equations (9.22). s

Definition 9.1. A vector xe ∈ R+ satisfying (B − A)T (k ◦ xA e)=0

(9.95)

is an equilibrium of (9.22). If, in addition, xe ∈ Rs+ , then xe is a positive equilibrium of (9.22). Let E denote the set of equilibria of (9.22), and let E+ ⊆ E denote the set of positive equilibria of (9.22). The following result can be used to obtain additional equilibria from known equilibria. λz

Proposition 9.6. Let z ∈ N (A) and let λ ∈ (0, ∞). If xe ∈ E, then ◦ xe ∈ E. Furthermore, if xe ∈ E+ , then λz ◦ xe ∈ E+ . Proof. Note that (B − A)T K(λz ◦ xe )A = (B − A)T K((λz )A ◦ xA e) A T Az = (B − A) K(λ ◦ xe ) = (B − A)T KxA e.

The proof for the case xe ∈ E+ is identical. Note that if xe is an equilibrium but not a positive equilibrium, then at least one of the species has zero concentration for this solution. Furthermore, it can be seen that xe = 0 is an equilibrium of (9.22) if and only if (9.22) has k

no reaction of the form 0 → C, where C is a nonzero product and k > 0. Example 9.12. Consider Example 9.1. 2 {(x1 , x2 ) ∈ R : x2 = (k1 /k2 )x1 }.

For this example E = △

Example 9.13. Consider Example 9.2. For this example E = 2 {(x1 , x2 ) ∈ R : x1 = 0 or x2 = (k2 /k1 )x1 }. For the reduced system (9.77) E = {0, k1 γ/(k1 + k2 )}. △ Example 9.14. Consider Example 9.3.

For this example E =

NNBook

September 1, 2009

298

CHAPTER 9

{(0, 0), (k3 /k2 , k1 /k2 )}.

△

Example 9.15. Consider Example 9.4. For this example E = 4 {(x1 , x2 , x3 , x4 ) ∈ R : x2 = 0 and x1 x3 = 0}. For the reduced system (9.80) and (9.81), if γ1 = x2 (0) + x3 (0) > 0, then E = {(0, 0)}, whereas, if γ1 = x2 (0) + x3 (0) = 0, then E = {(x1 , 0) : x1 ≥ 0}. △ First we consider a stability analysis of the linear case, that is, the case in which (9.54) is of the form x(t) ˙ = M x(t),

x(0) = x0 ,

t ≥ 0,

(9.96)

where M ∈ Rn×n . In this case, the following results hold. An equilibrium xe of (9.96) is Lyapunov stable (respectively, semistable) if and only if every equilibrium xe of (9.96) is Lyapunov stable (respectively, semistable). Furthermore, if an equilibrium of (9.96) is asymptotically stable, then xe = 0. Thus, all three types of stability can be characterized independently of the equilibrium. Specifically, the equilibrium xe = 0 of (9.96) is asymptotically stable if and only if every eigenvalue of M has negative real part; an equilibrium xe of (9.96) is semistable if and only if every eigenvalue of M has negative real part or is zero and, if M is singular, the zero eigenvalue is semisimple; and an equilibrium xe of (9.96) is Lyapunov stable if and only if every eigenvalue of M has nonpositive real part and every eigenvalue with zero real part is semisimple. Now, we consider (9.24) with M given by (9.25). The following result follows from Theorem 2.10. For this proof we construct a linear Lyapunov function that can be interpreted as the mass of the system. To do this, let µi > 0, i = 1, . . . , s, denote the molecular mass of the ith species, and define µ , [µ1 , . . . , µs ]T . Then the function V (x) = µT x represents the total mass of the system. Note that arbitrary constants µi > 0 can be used, and thus “mass” need not be interpreted literally. Note that V is a positive-definite s function with respect to R+ . We note that the following result makes no use of the structure of M except that it is essentially nonnegative. For the proof of this result, recall that −M is an M-matrix if and only if −M is a Z-matrix and every eigenvalue of M has a nonnegative real part [21]. Proposition 9.7. Consider the following statements: i ) There exists µ >> 0 such that M T µ ≤≤ 0. ii ) M is Lyapunov stable. iii ) M is semistable. iv ) There exists µ ≥≥ 0, µ 6= 0, such that M T µ ≤≤ 0.

NNBook

September 1, 2009

MASS-ACTION KINETICS

299

Then i) implies ii), ii) is equivalent to iii), and iii) implies iv). Furthermore, the following statements are equivalent: v ) M is asymptotically stable. vi ) There exists µ >> 0 such that M T µ 0 and a nonnegative matrix Q ≥≥ 0 such that α ≥ ρ(Q) and M T = Q − αIs . Now, since Q ≥≥ 0, it follows from Theorem 2.9 that ρ(Q) ∈ spec(Q), and hence, there exists µ ≥≥ 0, µ 6= 0, such that Qµ = ρ(Q)µ. Thus, M T µ = Qµ − αµ = (ρ(Q) − α)µ ≤≤ 0, which proves that there exists µ ≥≥ 0, µ 6= 0, such that M T µ ≤≤ 0. To show the equivalence of v)–vii) first suppose there exists µ ≥≥ 0 such that M T µ 0 such that M T (µ + εe) > 0, which proves that vii) implies vi). Since vi) implies vii) it follows that vi) and vii) are equivalent. Now, suppose vi) holds, that is, there exists µ >> 0 such that M T µ 0 it follows that M µ = 0. Hence, M is semistable. △ The following result uses the Lyapunov function V (x) = µT x to analyze the stability of the zero solution of (9.22). Recall that xe = 0 is an equilibrium of (9.22) if and only if A has no zero rows, that is, if and only if 0 is not a reactant of the reaction network (9.10). Proposition 9.8. Assume that xe = 0 is an equilibrium of (9.22) and suppose there exists µ >> 0 such that Bµ ≤≤ Aµ. Then xe is Lyapunov

NNBook

September 1, 2009

300

CHAPTER 9 s

stable with respect to R+ . If, in addition, Bµ 0 for all s x ∈ R+ \{0}. Since (B − A)µ ≤≤ 0 it follows that V˙ (x) = µT (B − A)T (k ◦ xA ) ≤ 0,

s

x ∈ R+ .

(9.97)

Hence, Theorem 2.1 implies that xe = 0 is Lyapunov stable with respect to s s R+ . Now suppose that Bµ 0. By linearizing the system about this equilibrium, it can be seen that this equilibrium is not Lyapunov stable. Hence, it remains to determine the stability of an equilibrium of the form (δ, k2 δ/k1 ), where δ ≥ 0. To do 2 this, let Q be the closed set Q , {(x1 , x2 ) ∈ R+ : x2 − ax1 ≤ 0}, where d a > k2 /k1 . Note that Q is invariant since dt (x2 − ax1 ) is negative on the set {(x1 , x2 ) : x2 = ax1 , x2 ≥ 0}, while the point (0, 0) is an equilibrium. Note that all of the equilibria contained in Q are of the form (δ, k2 δ/k1 ). Next, define the Lyapunov function candidate V : Q → R by Vδ (x) = 12 (x1 − δ + x2 − k2 δ/k1 )2 + 12 (k1 x2 − k2 x1 )2 .

(9.98)

Then, for all δ ≥ 0, we have Vδ (δ, k2 δ/k1 ) = 0 and Vδ (x) > 0 for all x ∈ Q\{(δ, k2 δ/k1 )}. Since V˙ δ (x) = −(k1 + k2 )x1 (k1 x2 − k2 x1 )2 ≤ 0 for all x ∈ Q it follows that the equilibrium (δ, k2 δ/k1 ) is Lyapunov stable with respect to Q for all δ ≥ 0. Finally, to show semistability, define U (x) = x1 + x2 , which satisfies U (0) = 0, U (x) > 0, x ∈ Q\{0}, and U˙ (x) = 0, x ∈ Q. Hence, every trajectory in Q is bounded. Then V˙ δ−1 (0) = f −1 (0), which shows that V˙ δ−1 (0) is an invariant set. Thus, the largest invariant set M contained in V˙ δ−1 (0) ∩ Q is the set of equilibria {(δ, k2 δ/k1 ) : δ ≥ 0}, all of which are Lyapunov stable. Hence, by Theorem 2.7, the kinetic equations are semistable with respect to Q. △ Example 9.19. Consider Example 9.3. By linearizing the kinetic equations about the origin, it can be seen that the origin is not Lyapunov

NNBook

September 1, 2009

MASS-ACTION KINETICS

301

stable. To analyze the stability of the equilibrium xe = (k3 /k2 , k1 /k2 ), consider as in [80, p. 115] the function U : R2+ → R defined by U (x) = k2 (x1 + x2 ) − k3 ln x1 − k1 ln x2 , which satisfies U˙ (x) = 0 for all x ∈ R2+ . It can be seen from the form of the gradient and the Hessian of U that x = xe is an isolated local minimizer of U . Hence V (x) = U (x) − U (xe ) satisfies V (xe ) = 0 and V (x) > 0 for all x ∈ D\{xe }, where D is an open neighborhood of xe . Hence, the equilibrium xe = (k3 /k2 , k1 /k2 ) is Lyapunov stable with respect to R2+ . Since the solutions consist of closed orbits [80], this equilibrium is not semistable. △ Example 9.20. Consider Example 9.4. For this example let µ = [1 2 1 1]T >> 0 so that (A − B)µ = 0. It thus follows from Proposition 4 9.8 that xe = 0 is Lyapunov stable with respect to R+ . For the reduced kinetic equations (9.80) and (9.81), with x2 (0) + x3 (0) > 0, it follows that x1 = x2 = 0 is the only equilibrium. Now, consider the radially unbounded Lyapunov function V (x1 , x2 ) = 12 k3 x22 + 12 k1 γ1 (x1 +x2 )2 . Since V˙ (x1 , x2 ) ≤ 0 for all x1 , x2 ≥ 0, global asymptotic stability follows from the KrasovskiiLaSalle invariant set theorem. △

9.8 The Zero-Deficiency Theorem In this section, we analyze the stability of positive equilibria of the kinetic equations (9.22) using the zero-deficiency theorem [81, 82]. This result provides a sufficient condition for Lyapunov stability and semistability based on the structure of the reaction network and independent of the values of the rate constants. The following definitions are required. A complex is either a reactant or a product. For example, in Example 9.3, the complexes include the reactants X1 , X1 + X2 , and X2 as well as the products 2X1 , 2X2 , and 0. Let m ≥ 1 denote the number of distinct complexes of the reaction network (including the reactant or product 0 if present), and denote the complexes by their corresponding vectors c1 , . . . , cm of stoichiometric coefficients. Obviously, m ≤ 2r. We can identify each complex with ha row i A of A or B so that ci ∈ R1×s . Thus, m is the number of distinct rows of B . In Examples 9.1–9.4 the number of complexes is 2, 2, 6, and 3, respectively. In particular, Example 9.4 involves the three complexes c1 = [1, 0, 1, 0], c2 = [0, 1, 0, 0], and c3 = [0, 0, 1, 1] corresponding to S + E, C, and P + E, respectively. For the following definition, “ci → cj ” denotes the reaction k

ci X →l cj X, where we assume kl > 0. Recall that reactions of the form c → c are not allowed.

It is useful to represent the reaction network by a directed graph. Consider a directed graph G having m vertices and r edges such that the ith vertex represents the complex ci , and there exists a directed edge from

NNBook

302

September 1, 2009

CHAPTER 9

vertex i to vertex j if and only if the reaction network contains the reaction ci → cj . Each edge of cj is numbered according to the reaction that it represents. Definition 9.2. Let ci and cj be complexes of the reaction network (9.10). Then ci and cj are directly linked if either ci → cj or cj → ci . Furthermore, ci and cj are indirectly linked if there exist complexes ci1 , . . . , cip such that ci is directly linked to ci1 , ci1 is directly linked to ci2 , . . ., cip is directly linked to cj . Finally, ci and cj are linked if ci and cj are either directly or indirectly linked. The statement that complexes ci and cj are linked is an equivalence relation on the set of complexes. This relation induces a partitioning of the set of complexes into disjoint linkage classes. These linkage classes are the connected components of the directed graph G. Let ℓ denote the number of linkage classes of G, and denote these linkage classes by C1 , . . . , Cℓ . Since the reactant and product in each reaction belong to the same linkage class, it follows that ℓ ≤ r. Furthermore, since each linkage class contains at least two complexes it follows that ℓ ≤ m/2. As noted in Section 9.6, the rank q = rank(B − A) of the reaction network (9.22) satisfies q ≤ min{r, s}. The following result provides a bound for q that is sometimes better. Some additional notation is needed. For P i = 1, . . . , ℓ, let mi denote the number of complexes in Ci so that ℓi=1 mi = m. Furthermore, for convenience we order the complexes c1 , . . . , cm so that C1 = {c1 , . . . , cm1 }, C2 = {cm1 +1 , . . . , cm2 }, and so forth. Next, we reorder the reactions so that the first r1 rows of [A, B] include the complexes in C1 , rows r1 + 1, . . . , r1 + r2 of [A, B] include the complexes in C2 , and so forth. P Hence, ℓi=1 ri = r. For i = 1, . . . , ℓ, we define the rank qi of the linkage class Ci to be the number of linearly independent rows in the submatrix of B − A comprised of the rows of [A, B] corresponding to the complexes in Ci . P Note that q ≤ ℓi=1 qi . For i = 1, . . . , ℓ, it can be seen that mi ≤ ri + 1, and thus m ≤ r + ℓ. If qi = mi − 1, then the linkage class Ci has full rank. Lemma 9.1. Let i ∈ {1, . . . , ℓ}. Then qi ≤ mi − 1. Furthermore, qi = mi − 1 if and only if the complexes in Ci are the vertices of an (mi − 1)s dimensional simplex in R+ . Proof. For notational convenience, let i = 1 and order the first m1 − 1 reactions so that, for j = 1, . . . , m1 − 1, the jth reaction is either cj → cj+1 or cj+1 → cj . The span of the first m1 rows of B − A is thus equal to the span of {c2 − c1 , . . . , cm1 − cm1 −1 }. Furthermore, since C1 is a linkage class, it follows that rows m1 + 1, . . . , r1 of B − A are contained in the span of the first m1 rows of B − A. Thus, q1 ≤ m1 − 1.

NNBook

September 1, 2009

303

MASS-ACTION KINETICS

Next, note that the span of {c2 − c1 , . . . , cm1 − cm1 −1 } is equal to the span of {c2 −c1 , c3 −c1 , . . . , cm1 −c1 }, which has dimension m1 −1 if and only if the complexes in C1 are the vertices of an (m1 − 1)-dimensional simplex s in R+ [254, pp. 7,12]. In the terminology of [254], an affine subspace is the translate of a subspace. Furthermore, the affine hull of a set S is the smallest affine subspace that contains S. It can be seen that Ci has full rank if and only if the subspace parallel to the affine hull of Ci has dimension mi − 1. Proposition 9.9. q ≤ m − ℓ. P Proof. As noted above, q ≤ ℓi=1 qi , while Lemma 9.1 implies that Pℓ P qi ≤ mi − 1. Therefore, q ≤ i=1 qi ≤ ℓi=1 (mi − 1) = m − ℓ. Definition 9.3. The deficiency δ of the reaction network (9.10) is δ , m − ℓ − q.

(9.99)

It follows from Proposition 9.9 that the deficiency of a reaction network is a nonnegative integer. If the deficiency of a reaction network is zero, then the reaction network has zero deficiency. It can be seen that a reaction network has deficiency zero if and only if i) every linkage class has full rank, and ii) for every pair Ci , Cj of distinct linkage classes, the subspaces parallel to the affine hulls of the linkage classes Ci , Cj have trivial intersection. Example 9.21. Consider Example 9.1. For this reaction network, m = 2, ℓ = 1, q = 1, and thus δ = 0. △ Example 9.22. Consider Example 9.2. For this reaction network, m = 2, ℓ = 1, q = 1, and thus δ = 0. △ Example 9.23. Consider Example 9.3. For this reaction network, m = 6, ℓ = 3, q = 2, and thus δ = 1. △ Example 9.24. Consider Example 9.4. For this reaction network, m = 3, ℓ = 1, q = 2, and thus δ = 0. △ Now, define the matrix C ∈ Rm×s whose rows are c1 , . . . , cm . ˆ B ˆ ∈ Rr×m be the matrices whose rows are unit Furthermore, let A, coordinate vectors in Rm and that satisfy ˆ A = AC,

ˆ B = BC.

(9.100)

NNBook

September 1, 2009

304

CHAPTER 9

It follows that ˆ − A)C. ˆ B − A = (B

(9.101)

ˆ − A) ˆ T ) ⊆ N ((B − A)T ). Next, observe that Aˆij = 1 if Note that N ((B and only if the complex cj is the reactant of the ith reaction, that is, if and ˆij = 1 if and only if the ith edge of G originates from vertex i. Similarly, B only if the ith edge of G terminates at vertex j. Consequently, the matrix ˆ − A) ˆ T is the incidence matrix of the directed graph G (see [37, p. 24].) (B ˆ − Aˆ and shows that The following result gives some properties of B T T ˆ − A) ˆ ) ⊇ N ((B − A) ) holds if δ = 0. the reverse inclusion N ((B Proposition 9.10. The following statements hold: ˆ − A) ˆ = m − ℓ. i ) rank(B

ˆ − A) ˆ T ) ∩ N (C T )). ii ) δ = dim(R((B

ˆ − A)e ˆ Cµ . iii) If µ ∈ Rs , then eBµ − eAµ = (B

ˆ − A) ˆ T ). iv ) δ = 0 if and only if N ((B − A)T ) = N ((B

ˆ − Aˆ corresponding Proof. First, to prove i ) consider the rows of B to C1 . As in the proof of Lemma 9.1 we order the first m1 − 1 reactions so that, for j = 1, . . . , m1 − 1, the jth reaction is either cj → cj+1 or cj+1 → cj . ˆ − Aˆ is either ej − ej+1 or Therefore, for j = 1, . . . , m1 − 1, the jth row of B ej+1 −ej , where ej is the jth unit coordinate vector in Rm . Thus, the first r1 ˆ − Aˆ have rank m1 − 1. Using a similar argument for each linkage rows of B ˆ − Aˆ corresponding to different linkage classes class and noting that rows of B ˆ A) ˆ = Pℓ (mi −1) = m−ℓ. are linearly independent, it follows that rank(B− i=1 Next, to prove ii ) it follows from Sylvester’s theorem (see Fact 2.10.13 of [23]) that q = rank(B − A) ˆ − A) ˆ T) = rank(C T (B

ˆ − A) ˆ T ) − dim(R((B ˆ − A) ˆ T ) ∩ N (C T )) = rank((B ˆ − A) ˆ T ) ∩ N (C T )). = m − ℓ − dim(R((B

To prove iii), let j ∈ {1, . . . , r}. Now, since each row of B corresponds to a unique row of C it follows that Bj = rowkj (C) for some kj ∈ {1, . . . , m}. ˆj C, where B ˆjk = 1, k = kj , and B ˆjk = 0, k 6= kj . Hence, Hence, Bj = B ˆ ˆj eCµ . eBj µ = eBj Cµ = erowkj (C)µ = B

NNBook

September 1, 2009

MASS-ACTION KINETICS

305

ˆj − Aˆj )eCµ = eBj µ − Similarly, we can show that Aˆj eCµ = eAj µ . Hence, (B A jµ e . To prove iv), note that rank((B − A)T ) + dim(N ((B − A)T ) = r and ˆ − A) ˆ T ) + dim(N ((B ˆ − A) ˆ T ) = r. rank((B ˆ− Since δ = 0, it follows from i) that q = rank((B − A)T ) = m − ℓ = rank(B T T T ˆ ˆ ˆ ˆ ˆ A), and thus dim(N ((B − A) ) = dim(N ((B − A) ). Since N ((B − A) ) ⊆ ˆ − A) ˆ T ). The converse N ((B − A)T ) it follows that N ((B − A)T ) = N ((B follows by reversing the steps. Definition 9.4. Let ci and cj be complexes. Then there exists a direct path from ci to cj if ci → cj . Furthermore, there exists an indirect path from ci to cj if there exist complexes ci1 , . . . , cip such that ci → ci1 → ci2 → · · · → cip → cj . Finally, there exists a path from ci to cj if there exists either a direct path or an indirect path from ci to cj . Note that the existence of a path from ci to cj is stronger than the statement that ci and cj are linked since the former condition accounts for the directionality of the reactions. Definition 9.5. The reaction network (9.10) is weakly reversible if, for all pairs of complexes ci , cj , the existence of a path from ci to cj implies the existence of a path from cj to ci . Note that the existence of a path from ci to cj is equivalent to the existence of a directed path from vertex i to vertex j on the graph G. Consequently, weak reversibility is equivalent to the requirement that every vertex or, equivalently, every edge of G must be part of a directed cycle of G [37, p. 25]. In the terminology of [144, pp. 357-358], weak reversibility of (9.10) is equivalent to strong connectedness of each connected component of G. The following lemmas are needed. Furthermore, for l = 1, . . . , ℓ, let vl ∈ Rm (respectively, el ∈ Rr ) be such that the jth component of vl (respectively, el ) is 1 if the jth vertex (respectively, jth edge) of G belongs to the lth connected component of G and 0 otherwise. It is easy to see that ˆ l = Bv ˆ l = el for all l = 1, . . . , ℓ, which implies that vl ∈ N (B ˆ − A) ˆ for Av all l = 1, . . . , ℓ. Next, note that, since each vertex of G belongs to exactly one connected component of G, {v1 , . . . , vℓ } are linearly independent and ˆ − A) ˆ = m − ℓ, it follows that N (B ˆ − A) ˆ is the span of hence, since rank(B

NNBook

September 1, 2009

306

CHAPTER 9

{v1 , . . . , vℓ }. Finally, note that Pℓ

e where θ1 , . . . , θℓ ∈ R.

l=1

θl vl

=

ℓ X

eθl vl ,

(9.102)

l=1

ˆ − A) ˆ T (αeT ◦ A) ˆ ∈ Rm×m . Lemma 9.2. Let α ∈ Rr+ and define Γ = (B Then the following statements hold: △

i ) The reaction network (9.22) is weakly reversible if and only if there ˆ − A) ˆ T p = 0. exists p ∈ Rr+ such that (B ii ) Assume that the reaction network (9.22) is weakly reversible. Then rank Γ = m − ℓ and there exists p ∈ Rm + such that Γ(p ◦ vl ) = 0 for all l = 1, . . . , ℓ. iii ) If the reaction network (9.22) has zero deficiency, then rank[C v1 · · · vℓ ] = m. Proof. To prove i), note that it follows from Theorems 4.5 and 5.2 ˆ − A) ˆ T ) is the span of {η1 , . . . , ηnc }, where nc is the number of [37] that N ((B of directed cycles of the graph G and ηi is such that the jth component of ηi is 1 if the jth edge is part of ith directed cycle of G and 0 otherwise. Hence, if the reaction network is weakly reversible, then every edge of G is part of at least one directed cycle of G. Now, a positive linear combination ˆ − A) ˆ T p = 0. To prove of all the cycles of G yields p ∈ Rr+ such that (B the converse, assume that the reaction network is not weakly reversible or, equivalently, there exists an edge (say the Jth edge) that does not belong to any cycle of G. Hence, it follows that the Jth component of all vectors in ˆ − A) ˆ T ) is zero, which implies that there does not exist p ∈ Rr+ such N ((B ˆ − A) ˆ T p = 0. (B To prove ii), note that −ΓT is the Laplacian of the weighted directed graph [234] obtained by assigning the weight αi to the ith edge of G. △ ˆ = There exists a permutation matrix Π ∈ Rm×m such that Γ ΠT ΓΠ and m l ×ml ˆ ˆ ˆ ˆ Γ = block-diag(Γ1 , . . . , Γℓ ), where Γl ∈ R , l = 1, . . . , ℓ, are such that Pℓ T ˆ l=1 ml = m and −Γl is the Laplacian of Cl . Weak reversibility implies that each connected component of G is strongly connected. (Note that ˆ T is the Laplacian of G in the case where the vertices are reordered such −Γ that the lth connected component (linkage class) of G contains the vertices △ (complexes) numbered as ml−1 + 1, . . . , ml , l = 1, . . . , ℓ, where m0 = 0.) ˆ l = ml − 1 for all Hence, it follows from Theorem 1 of [234] that rank Γ ˆ = m − ℓ. l = 1, . . . , ℓ, which implies that rank Γ = rank Γ

NNBook

September 1, 2009

307

MASS-ACTION KINETICS

△

To prove the second assertion of ii), let l ∈ {1, . . . , ℓ}, let cl = △ − mini=1,...,ml γi , and let Xl = Γl + cl Iml , where γi denotes the (i, i)th entry of Γl . Now, note that Xl is a nonnegative matrix and, for i 6= j, the (i, j)th entry of Xl is positive if and only if there exists an edge from vertex j to vertex i of the linkage class l. Hence, since the reaction network is weakly reversible, it follows from Theorem 6.2.24 of [144] that Xl is an irreducible matrix [144, p. 361], which further implies that there exists pˆl ∈ Rm + such that Xl pˆl = ρ(Xl )ˆ pl (see Theorem 8.4.4 of [144]). Consequently, ˆ l pˆl = (Xl −cl Iml )ˆ ˆ l pˆl = (ρ(Xl )−cl )eT pˆl Γ pl = (ρ(Xl )−cl )ˆ pl and, since 0 = eT Γ T and e pˆl > 0, it follows that cl = ρ(Xl ). Thus, there exists a positive ˆ l pˆl = 0 for all l = 1, . . . , ℓ. Now, letting vector pˆl ∈ Rm satisfying Γ T T T T pˆ = [ˆ p1 , . . . , pˆℓ ] it can be shown that pˆ ◦ (ΠT vl ) = [0, . . . , pˆT l , . . . , 0] ˆ p ◦ (ΠT vl )) = 0. Finally, taking p = Πˆ so that Γ(ˆ p implies that Γ(p ◦ vl ) = T ˆ (p ◦ vl ) = ΠΓ(ˆ ˆ p ◦ (ΠT vl )) = 0, establishing the result. ΠΓΠ To prove iii), let x ∈ Rm be such that xT [C v1 · · · vℓ ] = 0 or, equivalently, x ∈ N (C T ) and xT vl = 0 for all l = 1, . . . , ℓ. Next, since ˆ − A) ˆ is the span of {v1 , . . . , vℓ }, it follows that x ∈ [N (B ˆ − A)] ˆ ⊥ = N (B ˆ − A) ˆ T ). Hence, x ∈ R((B ˆ − A) ˆ T ) ∩ N (C T ) and, since the reaction R((B network has zero deficiency, it follows from ii) of Proposition 9.10 that x = 0, which proves that rank[C v1 · · · vℓ ] = m. Lemma 9.3. Assume that the reaction network (9.22) has zero deficiency and assume that there exists α ∈ Rr+ such that (B − A)T α = 0. Then µ ∈ Rs satisfies (B − A)T (α ◦ eAµ ) = 0 if and only if µ ∈ S ⊥ . Proof. Since the reaction network (9.22) has zero deficiency, it follows ˆ − A) ˆ T ), and hence, from iv) of Proposition 9.10 that N ((B − A)T ) = N ((B T N ((B − A) ) is the span of {η1 , . . . , ηnc } defined in the proof ofP i) of Lemma c T 9.2. Furthermore, since α ∈ N ((B − A) ) it follows that α = ni=1 βi ηi for some βi ∈ R, i = 1, . . . , nc . Now, note that ηi ◦ el = ηi if the ith cycle of G belongs to the lth linkage class of G and zero otherwise. In both cases, ˆ − A) ˆ T (ηi ◦ el ) = 0 for all i = 1, . . . , nc and l = 1, . . . , ℓ. (B − A)T (ηi ◦ el ) = (B ˆ − A)Cµ ˆ To prove necessity, let µ ∈ N (B − A). Hence, (B = 0, which, P ˆ − A) ˆ is the span of {v1 , . . . , vℓ }, implies that Cµ = ℓ θl vl for since N (B l=1 some θ1 , . . . , θℓ ∈ R. Hence, it follows that

ˆ Cµ ) (B − A)T (α ◦ eAµ ) = (B − A)T (α ◦ Ae = (B − A) (α ◦ Aˆ T

ℓ X l=1

eθl vl )

NNBook

September 1, 2009

308

CHAPTER 9

=

=

ℓ X

(B − A)T (α ◦ eθl el )

l=1 ℓ X nc X l=1 i=1

(B − A)T (βi eθl (ηi ◦ el ))

= 0,

where iii) of Proposition 9.10 is used to obtain the first equality, (9.102) ˆ l = el for all is used to obtain the second equality, and the fact that Av l = 1, . . . , ℓ, is used to obtain the third equality. Conversely, assume that (B − A)T (α ◦ eAµ ) = 0, which implies that ˆ − A) ˆ T (α ◦ eAµ ) = 0. Hence, (B ˆ − A) ˆ T (αeT ◦ A)e ˆ Cµ = ΓeCµ , ˆ − A) ˆ T (α ◦ Ae ˆ Cµ ) = (B 0 = (B

(9.103)

where Γ is defined in Lemma 9.2. Next, note that, for all l = 1, . . . , ℓ, ˆ − A) ˆ T (αeT ◦ A)v ˆ l Γvl = (B ˆ − A) ˆ T (α ◦ Av ˆ l) = (B

ˆ − A) ˆ T (α ◦ el ) = (B nc X ˆ − A) ˆ T (ηi ◦ el ) = βi (B i=1

= 0.

Furthermore, it follows from ii) of Lemma 9.2 that rank Γ = m − ℓ, which implies that N (Γ) is the span of {v1 , . . . , vℓ }. Hence, it follows from (9.103) P that eCµ = ℓl=1 eθl vl for some θ1 , . . . , θℓ ∈ R, which implies that Cµ = Pℓ ˆ − A)Cµ ˆ θl vl . Now, the result follows by noting that (B − A)µ = (B = Pl=1 ℓ ˆ ˆ l=1 θl (B − A)vl = 0. The following result shows that weak reversibility is a necessary and sufficient condition for a reaction network with zero deficiency to have at least one positive equilibrium.

Proposition 9.11. Assume that the reaction network (9.22) has zero deficiency. Then the reaction network (9.22) is weakly reversible if and only if it has a positive equilibrium. Proof. To prove necessity, let xe be a positive equilibrium of (9.22). ˆ − A) ˆ T p = 0, where Hence, it follows from iv) of Proposition 9.10 that (B A r p = Kxe ∈ R+ . Now, it follows from i) of Lemma 9.2 that the reaction network is weakly reversible.

NNBook

September 1, 2009

309

MASS-ACTION KINETICS

To prove sufficiency, note that ˆ − A) ˆ T (keT ◦ A)x ˆ C = ΓxC , ˆ − A) ˆ T (k ◦ xA ) = (B ˆ − A) ˆ T (k ◦ Ax ˆ C ) = (B (B △ ˆ − A) ˆ T (keT ◦ A). ˆ where Γ = (B Now, it follows from ii) of Lemma 9.2 that there exists a positive vector p ∈ Rm such that Γ(p ◦ vl ) = 0 for all l = 1, . . . , ℓ. Next, we show that there exists a positiveP vector x ∈ Rs ℓ and scalars θl ∈ R, l = 1, . . . , ℓ, such that xC = p ◦ e l=1 θl vl . To see this, notePthat the existence of a positive vector x and scalars θl satisfying ℓ xC = p◦e l=1 θl vl isP equivalent to the existence of a solution x to the equation Cloge x = loge p + ℓl=1 θl vl or, equivalently,



  [C v1 · · · vℓ ]  

loge x −θ1 .. . −θℓ



   = loge p. 

(9.104)

Now, since the reaction network has zero deficiency, it follows from iii) of Lemma 9.2 that rank[C v1 · · · vℓ ] = m, and hence, (9.104) has a solution, which implies P ℓ that there exists a positive vector x and scalars θl such that xC = p ◦ e l=1 θl vl . Next, it follows from (9.102) that Pℓ

(B − A)T (k ◦ xA ) = C T ΓxC = C T Γ(p ◦ e

l=1

θl vl

)=

ℓ X l=1

eθl C T Γ(p ◦ vl ) = 0,

which implies that x is a positive equilibrium of the reaction network (9.22). Next, we show that every positive stoichiometric compatibility class contains exactly one equilibrium for a weakly reversibile reaction network with zero deficiency. The following lemma is needed for this result. Lemma 9.4. Let p, pˆ ∈ Rs+ , let X be a subspace of Rs , and define △ X ⊥ = {x ∈ Rs : xT y = 0 for all y ∈ X }. Then there exists a unique µ ∈ X ⊥ such that (p ◦ eµ − pˆ) ∈ X . Proof. Define ϕ : Rs → R by ϕ(x) = pT ex − pˆT x. It can be shown that limkxk→∞ ϕ(x) = ∞. Now, let r > 0 and, since limkxk→∞ ϕ(x) = △ ∞, it follows that Cr = {x ∈ Rs : ϕ(x) ≤ r} is a compact set. Hence, △ Cˆr = {x ∈ X ⊥ : ϕ(x) ≤ r} is also a compact set, which implies that there exists µ ∈ X ⊥ such that ϕ(µ) ≤ ϕ(x) for all x ∈ Cˆr . Now, since X ⊥ = Cˆr ∪ {x ∈ X ⊥ : ϕ(x) > r} it follows that ϕ(µ) ≤ ϕ(x) for all x ∈ X ⊥ . Specifically, ϕ(µ) ≤ ϕ(µ + θγ) for all θ ∈ R and γ ∈ X ⊥ . Thus, △

NNBook

September 1, 2009

310

CHAPTER 9 △

f (θ) = ϕ(µ + θγ) has a minimum at θ = 0, which implies that ∂ϕ df = γ. 0= dθ θ=0 ∂x x=µ ∂ϕ ⊥ = (p◦eµ −pˆ) ∈ X , which establishes Hence, since γ ∈ X is arbitrary, ∂x x=µ

existence.

To prove uniqueness, let µ ˆ ∈ X ⊥ be such that (p ◦ eµˆ − pˆ) ∈ X . Since ⊥ µ µ, µ ˆ ∈ X and (p ◦ e − pˆ), (p ◦ eµˆ − pˆ) ∈ X it follows that (µ − µ ˆ) ∈ X ⊥ and [p ◦ (eµ − eµˆ )] ∈ X , and hence, T

µ

µ ˆ

0 = (µ − µ ˆ) [p ◦ (e − e )] =

s X i=1

pi (µi − µ ˆi )(eµi − eµˆi ).

(9.105)

Next, since the exponential function is an increasing function it follows that (µi − µ ˆi )(eµi − eµˆi ) ≥ 0 for all i = 1, . . . , s, and, since p ∈ Rs+ , it follows from (9.105) that (µi − µ ˆi )(eµi − eµˆi ) = 0 for all i = 1, . . . , s, or, equivalently, µ=µ ˆ. The next result characterizes all positive equilibria of zero-deficiency, weakly reversible reaction networks. Proposition 9.12. Assume that the reaction network (9.22) has zero deficiency and is weakly reversible, and let xe be a positive equilibrium. Then E+ = {x ∈ Rs+ : loge x − loge xe ∈ S ⊥ }.

(9.106)

Furthermore, every positive stoichiometric compatibility class contains exactly one equilibrium. Proof. To prove that E+ has the form (9.106), let xe be a positive △ equilibrium, let x ∈ Rs+ , and define µ = loge x − loge xe . Then (k ◦ xA ) = = = =

(k ◦ xA ◦ x−A ◦ xA e e) Aloge x −Aloge xe (k ◦ e ◦e ◦ xA e) A Aµ (k ◦ e ◦ xe ) Aµ (k ◦ xA ). e ◦e

Now, assume that x is also a positive equilibrium so that (B − A)T (k ◦ Aµ ) = (B − A)T (k ◦ xA ) = 0. Since x is an equilibrium, we have xA e e ◦e A (B − A)T (k ◦ xA e ) = 0. It thus follows from Lemma 9.3, with α = k ◦ xe , that ⊥ ⊥ T A µ ∈ S . Conversely, assume that µ ∈ S . Since (B − A) (k ◦ xe ) = 0, it Aµ ) = (B − A)T (k ◦ xA ), follows from Lemma 9.3 that 0 = (B − A)T (k ◦ xA e ◦e which shows that x is an equilibrium.

NNBook

September 1, 2009

311

MASS-ACTION KINETICS △

To prove the second assertion, let Sp = {p + x : x ∈ S} denote a stoichiometric compatibility class, where p ∈ Rs+ . Now, with X = S, it follows from Lemma 9.4 that there exists a unique µ ∈ S ⊥ such that (xe ◦ eµ − p) ∈ S or, equivalently, (xe ◦ eµ ) ∈ Sp . Now, the result follows by noting that E+ = {xe ◦ eµ : µ ∈ S ⊥ } ⊂ Rs+ . We now have the main result of this section. Theorem 9.4. If the reaction network (9.22) has zero deficiency, then every positive equilibrium of (9.22) is semistable with respect to Rs+ . Proof. Let xe be a positive equilibrium of (9.22) and define the Lyapunov candidate V : Rs+ → R by △

V (x) =

s X [xi (loge xi − loge xei ) − (xi − xei )], i=1

where xi and xei are the ith components of x and xe , respectively. It follows from the inequality loge a ≤ a − 1 for all a > 0, with a = xei /xi , that V (x) ≥ 0 for all x ∈ Rs+ . Since loge a = a − 1 if and only if a = 1, it follows that V (x) = 0 if and only if x = xe . Next, for x ∈ Rs+ , define µ = loge x − loge xe , and note that it follows from loge a ≤ a − 1, a > 0, with a = erowi (Bµ) /erowi (Aµ) , that △

eAµ ◦ [(B − A)µ] ≤≤ eBµ − eAµ ,

(9.107)

with equality holding in (9.107) if and only if (B − A)µ = 0. Using (9.107), along with iii) and iv) of Proposition 9.10, yields V˙ (x) = = = = = ≤

µT (B − A)T KxA µT (B − A)T KeAloge x µT (B − A)T K(eAloge xe ◦ eAµ ) ([µT (B − A)T ] ◦ (eAµ )T )KxA e T Aµ (KxA ) (e ◦ [(B − A)µ]) e T Bµ (KxA − eAµ ) e ) (e T ˆ ˆ Cµ = (KxA e ) (B − A)e ˆ − A) ˆ T KxA ]T eCµ = [(B e

= 0,

(9.108)

which proves that every positive equilibrium of (9.22) is Lyapunov stable. Next, assume that the reaction network (9.22) has zero deficiency. If T Aµ ◦ x ∈ Rs+ satisfies V˙ (x) = 0, then it follows from (9.108) that (KxA e ) (e A T Bµ Aµ A [(B − A)µ]) = (Kxe ) (e − e ). Now, since Kxe >> 0, it follows

NNBook

September 1, 2009

312

CHAPTER 9

from (9.107) that that eAµ ◦ [(B − A)µ] = eBµ − eAµ , which implies that (B − A)µ = 0, and hence, (loge x − loge xe ) ∈ S ⊥ . It now follows from Proposition 9.12 that x is a positive equilibrium of (9.22) and, as shown above, x is Lyapunov stable. Thus, every element of the largest invariant set of {x ∈ Rs+ : V˙ (x) = 0} is a Lyapunov-stable equilibrium. Furthermore, for η > 0, let Dη denote the closure of the connected component of {x ∈ Rs+ : V (x) ≤ η} containing xe . Since V (·) is continuous in Rs+ and V (xe ) = 0, it follows that there exists β > 0 such that Dβ ⊂ Rs+ and is compact. Now, with Q = Dβ , Theorem 2.7 implies every solution to (9.22) with x(0) ∈ Dβ converges to an equilibrium that is semistable with respect to Dβ . Finally, the result follows from the definition of semistability with respect to Rs+ and the fact that Dβ has a nonempty interior. The following version of Theorem 9.4 is proved in [81, 82]. Theorem 9.5. Assume that the reaction network (9.22) has zero deficiency and is weakly reversible. Then every positive stoichiometric compatibility class contains exactly one equilibrium. This equilibrium is asymptotically stable with respect to the positive stoichiometric compatibility class that it is contained in, and there exist no nontrivial periodic orbits in Rs+ . Proof. The first assertion is a consequence of Propositions 9.11 and 9.12. The second assertation follows from Theorem 9.4 and using the facts that every positive stoichiometric compatibility class is invariant, contains exactly one positive equilibrium, and V (x(t)) is a (strictly) decreasing function on every nontrivial solution to (9.22) in Rs+ , where V (·) is the Lyapunov function defined in the proof of Theorem 9.4. The conclusions of Theorems 9.4 and 9.5 can be strengthened without any additional assumptions. Specifically, it can be shown that, for every initial condition in the nonnegative orthant, the positive limit set of the reaction network (9.22) is a subset of the set of nonnegative equilibria. Furthermore, if every positive stoichiometric compatibility class has no equilibria on its boundary, then every equilibrium is globally asymptotically stable relative to its positive stoichiometric compatibility class. Example 9.25. Consider Example 9.1. This reaction network has zero deficiency and is weakly reversible. Theorem 9.5 thus implies that every positive stoichiometric compatibility class contains exactly one equilibrium, s △ and this equilibrium is semistable with respect to R+ . Example 9.26. Consider Example 9.2. This reaction network has zero deficiency and is weakly reversible. Theorem 9.5 thus implies that every

NNBook

September 1, 2009

313

MASS-ACTION KINETICS

positive stoichiometric compatibility class contains exactly one equilibrium, s △ and this equilibrium is semistable with respect to R+ . Example 9.27. Consider Example 9.3. This reaction network has deficiency 1 and is not weakly reversible. Hence, Theorem 9.5 does not apply. △ Example 9.28. Consider Example 9.4. Although this reaction network has zero deficiency, it is not weakly reversible. Accordingly, Theorem 9.4 cannot be used to conclude semistability. However, Lyapunov methods, based on nontangency between the vector field and invariant subsets of the level sets of the Lyapunov function V (x) = αx1 +x2 , where α ∈ (1, 1+k3 /k2 ), 4 can be used to conclude semistability of every equilibrium in Eˆ = {x ∈ R+ : x1 = 0, x2 = 0, x3 > 0}. For details, see [34]. △ The following example is a modification of Example 9.4 to include weak reversibility. Example 9.29. Consider a modification of Example 9.4 in which all reactions are reversible, that is, S+E so that s = 4 and r = 4.  1  0 A=  0 0

k1

⇋ k2

C

k3

⇋ k4

P+E

It thus follows that A and   0 1 0 0 1  1 0 1 0 0  , B =   0 0 1 0 0  0 1 1 0 1

and the kinetic equations have the form

(9.109) B are given by  0 0 1 0  , (9.110) 1 1  0 0

x˙ 1 (t) = k2 x2 (t) − k1 x1 (t)x3 (t), x1 (0) = x10 , t ≥ 0, x˙ 2 (t) = −(k2 + k3 )x2 (t) + k1 x1 (t)x3 (t) + k4 x3 (t)x4 (t), x˙ 3 (t) = (k2 + k3 )x2 (t) − k1 x1 (t)x3 (t) − k4 x3 (t)x4 (t), x˙ 4 (t) = k3 x2 (t) − k4 x3 (t)x4 (t),

x4 (0) = x40 .

(9.111) x2 (0) = x20 , (9.112) x3 (0) = x30 , (9.113) (9.114)

Since this network has zero deficiency and is weakly reversible, Theorem 9.5 implies that every positive stoichiometric compatibility class contains exactly one equilibrium, and this equilibrium is semistable with respect to s R+ . △

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Ten

Semistability and State Equipartition of Nonnegative Dynamical Systems

10.1 Introduction In Chapter 7, we developed a compartmental dynamical systems approach to thermodynamics. Specifically, each compartment represented the energy content of the different parts of the dynamical system, and different compartments interacted by exchanging heat. A key assumption in Chapter 7 was that intercompartmental energy flows between connected compartments were bidirectional. However, in many applications of thermal and fluid sciences the assumption of bidirectional energy flow or fluid flow between compartments can be limiting. In addition, transfers between compartments are not instantaneous and realistic models for capturing the dynamics of fluid and thermal systems should account for material or energy in transit between compartments. In this chapter, we develop compartmental dynamical systems models that guarantee semistability and state equipartitioning with directed and undirected energy flow between compartments. Furthermore, intercompartmental flow delays are also considered.

10.2 Semistability and State Equipartitioning In this chapter, we use undirected and directed graphs to represent intercompartmental connections for nonlinear compartmental dynamical systems with directional and bidirectional flows. Specifically, let G = (V, E, A) be a weighted directed graph (or digraph) denoting the compartmental network with the set of nodes (or compartments) V = {1, . . . , q} involving a finite nonempty set denoting the compartments, the set of edges E ⊆ V × V involving a set of ordered pairs denoting the direction of energy flow, and a weighted adjacency matrix A ∈ Rq×q such that A(i,j) = αij > 0, i, j = 1, . . . , q, if (j, i) ∈ E, while αij = 0 if (j, i) 6∈ E. The edge (j, i) ∈ E denotes that compartment Gj can obtain energy from compartment Gi , but

NNBook

September 1, 2009

316

CHAPTER 10

not necessarily vice versa. Moreover, we assume that αii = 0 for all i ∈ V. Note that if the weights αij , i, j = 1, . . . , q, are not relevant, then αij is set to 1 for all (j, i) ∈ E. In this case, A is called a normalized adjacency matrix . A graph or undirected graph G associated with the adjacency matrix A ∈ Rq×q is a directed graph for which Pq the arc Pqset is symmetric, that is, T A = A . A graph G is balanced if j=1 αij = j=1 αji for all i = 1, . . . , q. Finally, we denote the energy or mass of the compartment i ∈ {1, . . . , q} at time t by xi (t) ∈ R+ . The state equipartitioning problem involves a dynamical model that guarantees energy or mass state equipartition, that is, limt→∞ xi (t) = α ∈ R+ for i = 1, . . . , q. The state equipartitioning problem can be characterized as a closed compartmental dynamical system G given by x˙ i (t) =

q X

φij (xi (t), xj (t)),

xi (t0 ) = xi0 ,

j=1, j6=i

t ≥ 0,

i = 1, . . . , q, (10.1)

where the flow functions φij (·, ·), i, j = 1, . . . , q, are locally Lipschitz continuous or, in vector form, x(t) ˙ = f (x(t)),

x(t0 ) = x0 ,

t ≥ 0,

(10.2)

n

where x(t) , [x1 (t), . . . , xq (t)]T ⊆ R+ , t ≥ 0, and f = [f1 , . . . , fq ]T : D → Rq P q is such that fi (x) = qj=1, j6=i φij (xi , xj ), where D ⊆ R+ . Here, we assume that f (·) is essentially nonnegative. Definition 10.1 ([21]). A directed graph G is strongly connected if for any ordered pair of vertices (i, j), i 6= j, there exists a path (i.e., sequence of arcs) leading from i to j. Recall that A ∈ Rq×q is irreducible, that is, there does not exist a permutation matrix such that A is cogredient to a lower-block triangular matrix, if and only if G is strongly connected (see Theorem 2.7 of [21]). Furthermore, note that for an undirected graph A = AT , and hence, every undirected graph is balanced. In this section, we develop a thermodynamically motivated state equipartitioning framework for nonlinear compartmental dynamical systems that achieve semistability and state equipartition. Specifically, consider the dynamical system given by (10.1) where for each i ∈ {1, . . . , q}, xi (t) ∈ R+ denotes the energy or mass state of the ith compartment and φij (·, ·), i, j = 1, . . . , q, are locally Lipschitz continuous and essentially nonnegative. The following assumptions are needed for the main results of the chapter.

NNBook

EQUIPARTITION AND SEMISTABILITY

September 1, 2009

317

Assumption 1. The connectivity matrix 1 C ∈ Rq×q associated with the compartmental dynamical system G is defined by  0, if φij (xi , xj ) ≡ 0, C(i,j) , i 6= j, i, j = 1, . . . , q, 1, otherwise, P and C(i,i) , − qk=1, k6=i C(i,k) , i = 1, . . . , q, and satisfies rank C = q − 1. Moreover, for every i 6= j such that C(i,j) = 1, φij (xi , xj ) = 0 if and only if xi = xj . R+ .

Assumption 2. For i, j = 1, . . . , q, (xi − xj )φij (xi , xj ) ≤ 0, xi , xj ∈

Note that (10.1) describes an interconnected compartmental system with a graph topology G. The fact that φij (xi , xj ) = 0 if and only if xi = xj , i 6= j, implies that the compartments Gi and Gj are connected; alternatively, φij (xi , xj ) ≡ 0 implies that the compartments Gi and Gj are disconnected. Assumption 1 implies that if the energies in the connected compartments Gi and Gj are equal, then energy exchange between these compartments is not possible. This statement is reminiscent of the zeroth law of thermodynamics, which postulates that temperature equality is a necessary and sufficient condition for thermal equilibrium. Furthermore, if C = C T and rank C = q−1, then it follows that the connectivity matrix C and the adjacency matrix A are irreducible, which implies that for any pair of the compartments Gi and Gj , i 6= j, of G there exists a sequence of energy connectors (arcs) of G that connect Gi and Gj . Assumption 2 implies that energy flows from more energetic compartments to less energetic compartments and is reminiscent of the second law of thermodynamics, which states that heat (energy) must flow in the direction of lower temperatures. Proposition 10.1. Consider the compartmental dynamical system (10.1) and assume that Assumptions 1 and 2 hold. Then fi (x) = 0 for all i = 1, . . . , q if and only if x1 = · · · = xq . Furthermore, αe, α ∈ R+ , is an equilibrium state of (10.1). Proof. The proof of the first assertion is identical to the proof of Proposition 7.3 with Cs in the proof of Proposition 7.3 denoting the connectivity matrix for the new directed graph G which excludes node m from the directed graph G. The second assertion is a direct consequence of the first assertion. The following results are needed for the main result of this section. For the statement of these results, (·)D denotes the Drazin generalized inverse. 1 The negative of the connectivity matrix, that is, −C, is known as the Laplacian of the directed graph G in the literature.

NNBook

September 1, 2009

318

CHAPTER 10

Recall that for a diagonal matrix A ∈ Rq×q the Drazin inverse AD ∈ Rq×q is D given by AD (i,i) = 0 if A(i,i) = 0 and A(i,i) = 1/A(i,i) if A(i,i) 6= 0, i = 1, . . . , q [23, p. 227]. Proposition 10.2. Let A ∈ Rq×q be an essentially nonnegative matrix such that A = AT . If there exists p ∈ Rq+ such that AT p ≤≤ 0, then A ≤ 0. Proof. The proof is a direct consequence of ii) of Theorem 2.10 by noting that if A is symmetric, then semistability implies that A ≤ 0. Lemma 10.1. Let X ∈ Rn×n and Z ∈ Rm×m be such that X = X T and Z = Z T , and let Y ∈ Rn×m be such that Y = Y Z D Z. Then   X Y ≤0 (10.3) M, YT Z if and only if Z ≤ 0 and X − Y Z D Y T ≤ 0. Proof. Define T ,



In −Y Z D 0 Im



and note that det T 6= 0. Now, noting that T M T T ≤ 0 if and only if M ≤ 0, and     X Y In 0 In −Y Z D T T MT = YT Z −Z D Y T Im 0 Im   X − Y Z DY T 0 = 0 Z ≤ 0, the result follows immediately.

or

Lemma 10.2. Let A ∈ Rq×q and Ad ∈ Rq×q be given by either  Pq − k=1,k6=i aik , i = j, A(i,j) = 0, i 6= j,  0, i = j, Ad(i,j) = i, j = 1, . . . , q, (10.4) aij , i 6= j, A(i,j) = Ad(i,j) =





−

Pq

k=1,k6=i aki ,

0,

0, i = j, aij , i = 6 j,

where aij ≥ 0, i, j = 1, . . . , q, i 6= j.

i = j, i 6= j,

i, j = 1, . . . , q, Assume that

(10.5) Pq

k=1,k6=i aik

=

NNBook

September 1, 2009

319

EQUIPARTITION AND SEMISTABILITY

Pq

a for each i = 1, . . . , q. Then for every Adi , i = 1, . . . , qd , such k=1,k6 P=qdi ki that i=1 Adi = Ad , there exist nonnegative definite matrices Qi ∈ Rq×q , i = 1, . . . , qd , such that 2A +

qd X D (Qi + AT di Qi Adi ) ≤ 0.

(10.6)

i=1

by

Proof. For each i ∈ {1, . . . , qd }, let Qi be the diagonal matrix defined Qi (l,l) ,

q X

Adi (l,m) ,

l = 1, . . . , q,

(10.7)

m=1,l6=m

Pq d

and note that A + i=1 Qi = 0, (Adi − Qi )e = 0, and Qi QD i Adi = Adi , i = 1, . . . , qd . Hence, M e = 0, where   Pd AT 2A + qi=1 Qi AT AT d1 d2 · · · dqd  Ad1 −Q1 0 ··· 0    M , (10.8) .. .. .. .. ..  .  . . . . .  Adqd 0 0 · · · −Qqd

Now, note that M = M T and M(i,j) ≥ 0, i, j = 1, . . . , q, i 6= j. Hence, by ii) of Theorem 2.10, M is semistable. Thus, by Proposition 10.2, M ≤ 0. Now, since Qi QD i Adi = Adi , i = 1, . . . , qd , it follows from Lemma 10.1 that M ≤ 0 if and only if (10.6) holds. Alternatively, if A ∈ Rq×q and Ad ∈ Rq×q are given by (10.5), then let Qi be the diagonal matrix defined by Qi (l,l) ,

q X

Adi (m,l) ,

l = 1, . . . , q.

(10.9)

m=1,l6=m

The result now follows using similar arguments as above. Next, we consider the case where (10.1) has a nonlinear structure of the form φij (xi , xj ) = aij (xj ) − aji (xi ),

(10.10)

where aij : R → R, i, j = 1, . . . , q, i 6= j, are such that aij (0) = 0 and aij (·), i, j = 1, . . . , q, i 6= j, is strictly increasing. For this result define P P fci (xi ) , − qj=1,j6=i aji (xi ), fdi (x) , ei qj=1 aij (xj ), i = 1, . . . , q, and fc (x) , [fc1 (x1 ), . . . , fcq (xq )]T , where ei ∈ Rq denotes the elementary vector of order q with 1 in the ith component and 0’s elsewhere. Theorem 10.1. Consider the nonlinear dynamical system given by

NNBook

September 1, 2009

320

CHAPTER 10

(10.1) or, equivalently, (10.2) where φij (xi , xj ), i, j = 1, . . . , q, i 6= j, is given by (10.10) and fci (·), i = 1, . . . , q, is strictly P Pdecreasing. Assume that q eT [fc (x)+ qi=1 fdi (x)] = 0, x ∈ R+ , and fc (x)+ qi=1 fdi (x) = 0 if and only if x = αe for some α ∈ R+ . Furthermore, assume thereP exist nonnegative diagonal matrices Pi ∈ Rq×q , i = 1, . . . , q, such that P , qi=1 Pi is positive definite, q

q X i=1

PiD Pi fdi (x) = fdi (x), x ∈ R+ , i = 1, . . . , q,

T fdi (x)Pi fdi (x) ≤ fcT (x)P fc (x),

q

x ∈ R+ .

(10.11) (10.12)

Then, for every α ∈ R+ , αe is a semistable equilibrium state of (10.2). Furthermore, x(t) → 1q eeT x(0) as t → ∞ and 1q eeT x(0) is a semistable equilibrium state. Proof. Consider the nonnegative function given by q Z xi X V (x) = −2 P(i,i) fci (θ)dθ. i=1

(10.13)

0

Since fci (·), P i = 1, . . . , q, is a strictly decreasing function it follows that V (x) ≥ 2 qi=1 P(i,i) [−fci (δi xi )]xi > 0 for all xi 6= 0, where 0 < δi < 1, and hence, there exists a class K function α(·) such that V (x) ≥ α(kxk). Now, note that the derivative of V (x) along the trajectories of (10.2) is given by V˙ (x) = −2fcT (x)P fc (x) − 2 ≤ −fcT (x)P fc (x) − 2 −

q X

q X

fcT (x)P fdi (x)

i=1

q X

fcT (x)P PiD Pi fdi (x)

i=1

fdi (x)Pi PiD Pi fdi (x)

i=1

q X = − [P fc (x) + Pi fdi (x)]T PiD [P fc (x) + Pi fdi (x)] i=1

≤ 0,

q

x ∈ R+ ,

(10.14)

where the first inequality in (10.14) follows from (10.11) and (10.12), and the last equality in (10.14) follows from the fact that fcT (x)P fc (x) = P q q T D i=1 fc (x)P Pi P fc (x), x ∈ R+ . q

Next, let R , {x ∈ R+ : P fc (x) + Pi fdi (x) = 0, i = 1, . . . , q}. Then it follows from the Krasovskii-LaSalle theorem that x(t) → M as t → ∞, where M denotes the largest invariant set contained in R. Now, since

NNBook

September 1, 2009

321

EQUIPARTITION AND SEMISTABILITY

eT [fc (x) +

Pq

q

= 0, x ∈ R+ , it follows that ( ) q X q ˆ , x ∈ R : fc (x) + R⊆R fdi (x) = 0 +

i=1 fdi (x)]

i=1

= {x ∈

q R+

: x = αe, α ∈ R+ },

(10.15)

ˆ as t → ∞. which implies that x(t) → R Finally, Lyapunov stability of αe, α ∈ R+ , follows by considering the Lyapunov function candidate q Z xi X V (x) = −2 P(i,i) (fci (θ) − fci (α))dθ (10.16) i=1

α

and noting that V (x) ≥ 2

q X i=1

P(i,i) [fci (α) − fci (α + δi (xi − α))](xi − α) > 0,

x 6= αe,

where 0 < δi < 1 and i = 1, . . . , q. In this case, it follows from Theorem 2.7 that, for every α ∈ R+ , αe is a semistable equilibrium state of (10.2). Furthermore, note that since eT x(t) = eT x(0), t ≥ 0, and x(t) → M as t → ∞, it follows that x(t) → 1q eeT x(0) as t → ∞. Hence, with α = 1q eT x(0), αe = 1q eeT x(0) is a semistable equilibrium state of (10.2). Theorem 10.2. Consider the nonlinear dynamical system (10.1) or, equivalently, (10.2), and assume that Assumptions 1 and 2 hold. i) Assume that φij (xi , xj ) = −φji (xj , xi ) for all i, j = 1, . . . , q, i 6= j. Then for every α ∈ R+ , αe is a semistable equilibrium state of (10.2). Furthermore, x(t) → 1q eeT x(0) as t → ∞ and 1q eeT x(0) is a semistable equilibrium state. ii) Let φij (xi , xj ) = C(i,j) [σ(xj ) − σ(xi )] for all i, j = 1, . . . , q, i 6= j, where σ(0) = 0 and σ(·) is strictly increasing. Assume that C T e = 0. Then, for every α ∈ R+ , αe is a semistable equilibrium state of (10.2). Furthermore, x(t) → 1q eeT x(0) as t → ∞ and 1q eeT x(0) is a semistable equilibrium state. Proof. i) This is a restatement of Theorem 8.5. ii) It follows from Lemma 10.2 that there exists Qi , i = 1, . . . , q, such that (10.6) holds with Qi given by (10.7), and A and Adi , i = 1, . . . , q, are given by (10.4) with aij replaced by C(i,j). Next, consider the nonnegative

NNBook

September 1, 2009

322

CHAPTER 10

function given by V (x) = 2

q Z X i=1

xi

σ(θ)dθ.

(10.17)

0

P Since σ(·) is a strictly increasing function it follows that V (x) ≥ 2 qi=1 σ(δi xi )xi > 0 for all x 6= 0, where 0 < δi < 1, and hence, there exists a class K function α(·) such that V (x) ≥ α(kxk). Now, the derivative of V (x) along the trajectories of (10.2) is given by V˙ (x) = 2ˆ σ T (x)Aˆ σ (x) + 2

q X

σ ˆ T (x)Adi σ ˆ (x)

i=1

≤ − = − ≤ 0, q

q X

D σ T (x)Qi σ ˆ (x) − 2ˆ σ T (x)Adi σ ˆ (x) + σ ˆ T (x)AT ˆ (x)] [ˆ di Qi Adi σ

i=1 q X

[−Qi σ ˆ (x) + Adi σ ˆ (x)]T QD ˆ (x) + Adi σ ˆ (x)] i [−Qi σ

i=1

q

x ∈ R+ ,

(10.18)

ˆ (x) , [σ(x1 ), . . . , σ(xq )]T . where σ ˆ : R+ → Rq is given by σ q

ˆ (x) + Adi σ Next, let R , {x ∈ R+ : −Qi σ ˆ (x) = 0, i = 1, . . . , q}. Then it follows from the Krasovskii-LaSalle theorem that x(t) → M as t → ∞, where P M denotes the largest invariant set contained in R. Now, since A + qi=1 Qi = 0, it follows that ( ) q X q ˆ , x ∈ R : Aˆ σ (x) + R⊆R Adi σ ˆ (x) = 0 . +

i=1

P Hence, since rank(A+ i=1 Adi ) = q − 1 and (A+ qi=1 Adi )e = 0, it follows ˆ contained in R ˆ is given by M ˆ = {x ∈ Rq+ : that the largest invariant set M ˆ ⊆ R ⊆ R, ˆ it follows that M = M. ˆ x = αe, α ∈ R}. Furthermore, since M Pq

Finally, Lyapunov stability of αe, α ∈ R+ , follows by considering the Lyapunov function candidate q Z xi X ˜ V (x) = 2 [σ(θ) − σ(α)]dθ (10.19) i=1

α

P and noting that V˜ (x) ≥ 2 qi=1 [σ(α + δi (xi − α)) − σ(α)](xi − α) > 0, for all xi 6= α, where 0 < δi < 1 and i = 1, . . . , q. In this case, it follows from Theorem 2.7 that, for every α ∈ R+ , αe is a semistable equilibrium state of (10.2). Furthermore, note that since eT x(t) = eT x(0), t ≥ 0, and x(t) → M as t → ∞, it follows that x(t) → 1q eeT x(0) as t → ∞. Hence,

NNBook

September 1, 2009

323

EQUIPARTITION AND SEMISTABILITY

with α = (10.2).

1 T q e x(0),

αe =

1 T q ee x(0)

is a semistable equilibrium state of

Note that the assumption φij (xi , xj ) = −φji (xj , xi ), i, j = 1, . . . , q, i 6= j, in i) of Theorem 10.2 implies that C = C T , and hence, the underlying graph for the compartmental dynamical system G given by (10.1) is undirected. Furthermore, note that φij (xi , xj ) is not restricted to a specified structure. Alternatively, in ii) of Theorem 10.2 the assumption C T e = 0 implies that the underlying directed graph of G is balanced. To see this, recall that for a directed graph G, Ae = AT e implies that G is balanced. Since C = hA − ∆, where A denotes i the normalized adjacency matrix and Pq Pq q×q , it follows that Ae = AT e if ∆ , diag j=1 α1j , . . . , j=1 αqj ∈ R and only if Ce = C T e. Hence, C T e = 0 implies that G is balanced.

Theorem 10.2 implies that the steady-state value of the state of each subsystem Gi of the compartmental dynamical system G is equal; that is, the steady state of the compartmental dynamical system G given by " q # 1 T 1X xi (0) e x∞ = ee x(0) = q q i=1

is uniformly distributed over all subsystems of G. Finally, we specialize Theorem 10.1 to the case where φij (xi , xj ) = aij σ(xj ) − aji σ(xi ),

(10.20)

where σ : R+ → R is such that σ(u) = 0 if and only if u = 0, aij ≥ 0, i, j = 1, . . . , q, i 6= j. In this case, (10.2) can be rewritten as x(t) ˙ = Aˆ σ (x(t)) +

q X i=1

q

Adi σ ˆ (x(t)),

x(0) = x0 ,

t ≥ 0,

(10.21)

where σ ˆ : R+ → Rq is given by σ ˆ (x) , [σ(x1 ), . . . , σ(xq )]T , and A and Adi , i = 1, . . . , q, are given by (10.5). Theorem 10.3. Consider the compartmental dynamical system given is such that σ(0) = 0Pand σ(·) is strictly by (10.21) where σ : R+ → R P increasing. Assume that (A + qi=1 Adi )T e = (A + qi=1 Adi )e = 0 and P rank(A + qi=1 Adi ) = q − 1. Then, for every α ∈ R+ , αe is a semistable equilibrium point of (10.2). Furthermore, x(t) → 1q eeT x(0) as t → ∞ and 1 T q ee x(0) is a semistable equilibrium state. Proof. It follows from Lemma 10.2 that there exists Qi , i P = 1, . . . , q, such that (10.6) holds with Qi given by (10.9). Now, since A = − qi=1 Qi =

NNBook

September 1, 2009

324

CHAPTER 10

P P − qi=1 PiD = −P −1 , where P = qi=1 Pi , it follows from (10.6) that, for all q x ∈ R+ , q X D 0 ≥ 2ˆ σ (x)Aˆ σ (x) σ (x) + σ ˆ (x) (Qi + AT di Qi Adi )ˆ T

T

i=1

= −fcT (x)P fc (x) +

q X

T fdi (x)Pi fdi (x),

i=1

q

where fc (x) = Aˆ σ (x) and fdi (x) = Adi σ ˆ (x), i = 1, . . . , q, x ∈ R+ . Furthermore, since PiD Pi Adi = Adi , i = 1, . . . , q, it follows that PiD Pi fdi (x) = q fdi (x), i = 1, . . . , q, x ∈ R+ . Now, the resultP is an immediate consequence T of Theorem 10.1 by noting that e [fc (x) + qi=1 fdi (x)] = 0 and fc (x) + Pq i=1 fdi (x) = 0 if and only if x = αe for some α ∈ R+ .

10.3 Semistability and Equipartition of Linear Compartmental Systems with Time Delay In this section, we extend the results of Section 10.2 to compartmental systems with time delay. We first consider linear, time-delay dynamical systems G of the form x(t) ˙ = Ax(t) +

qd X i=1

Adi x(t − τi ),

x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

(10.22) t ≥ 0, A ∈ Adi ∈ τi ∈ R, i = 1, . . . , qd , where x(t) ∈ τ¯ = maxi∈{1,...,qd } τi , η(·) ∈ C+ , {ψ(·) ∈ C([−¯ τ , 0], Rq ) : ψ(θ) ≥≥ 0, θ ∈ [−¯ τ , 0]} is a continuous vector-valued function specifying the initial state of the system, and C([−¯ τ , 0], Rq ) denotes a Banach space of continuous τ , 0] into Rq with the topology of uniform functions mapping the interval [−¯ convergence. In addition, note that since η(·) is continuous it follows from Theorem 2.1 of [128, p. 14] that there exists a unique solution x(η) defined on [−¯ τ , ∞) that coincides with η on [−¯ τ , 0] and satisfies (10.22) for all t ≥ 0. Finally, recall that if the positive orbit Oη+ of (10.22) is bounded, then Oη+ is precompact [126], that is, Oη+ can be enclosed in the union of a finite number of ε-balls around elements of Oη+ . q R+ ,

Rq×q ,

Rq×q ,

Definition 10.2. The linear time-delay dynamical system (10.22) is called a compartmental dynamical system if A ∈ Rq×q isPessentially d nonnegative, Adi ∈ Rq×q , i = 1, . . . , qd , is nonnegative, and A + qi=1 Adi is a compartmental matrix. Note that the linear time-delay dynamical system (10.22) is compart-

NNBook

September 1, 2009

325

EQUIPARTITION AND SEMISTABILITY

Pd mental if A and Ad , qi=1 Adi are given by   Pn 0, i = j, − k=1 aki , i = j, Ad(i,j) = A(i,j) = 0, i 6= j, aij , i = 6 j,

(10.23)

where aii ≥ 0, i ∈ {1, . . . , q}, denotes the loss coefficients of the ith compartment and aij ≥ 0, i 6= j, i, j ∈ {1, . . . , q}, denotes the transfer coefficients from the jth compartment to the ith compartment. Next, we present sufficient conditions for semistability and system state equipartition for linear compartmental dynamical systems with time delay. The following lemma is needed for the main theorem of this section. Pd Lemma 10.3.P Let A ∈ Rq×q and Ad = qi=1 Adi be given by (10.23). d Assume that (A + qi=1 Adi )e = 0. Then there exist nonnegative definite matrices Qi ∈ Rq×q , i = 1, . . . , qd , such that qd X D A+A + (Qi + AT di Qi Adi ) ≤ 0. T

(10.24)

i=1

by

Proof. For each i ∈ {1, . . . , qd }, let Qi be the diagonal matrix defined Qi (l,l) ,

qd X

Adi (l,m) ,

(10.25)

m=1,l6=m

and note that it follows from (10.25) and the definition of the Drazin inverse that (Adi − Qi )e = 0 and Qi QD di , i = 1, . . . , qd . Since A and i Adi = AP d Qi , iP= 1, . . . , qd , are diagonal and (A + qi=1 Adi )e = 0 it follows that qd A + i=1 Qi = 0. Hence, M e = 0, where   Pd AT A + AT + qi=1 Qi AT AT d1 d2 · · · dqd  Ad1 −Q1 0 ··· 0  △   M = (10.26) .. .. .. .. ..  .  . . . . .  Adqd

0

0

· · · −Qqd

Note that M = M T and M(i,j) ≥ 0, i, j = 1, . . . , qd , i 6= j. Hence, by ii) of Theorem 2.10, M is semistable. Now, it follows from Proposition 10.2 that M ≤ 0 and since Qi QD i Adi = Adi , i = 1, . . . , qd , it follows from Lemma 10.1 that M ≤ 0 if and only if (10.24) holds.

Theorem 10.4. Consider the linear time-delay dynamical system givenPby (10.22) where APand Ad are given by (10.23). Pqd Assume that d d (A + qi=1 Adi )T e = (A + qi=1 Adi )e = 0 and rank(A + i=1 Adi ) = q − 1. Then, for every α ≥ 0, αe is a semistable equilibrium point of (10.22).

NNBook

September 1, 2009

326

CHAPTER 10

Furthermore, x(t) → α∗ e as t → ∞, where P d R0 T eT η(0) + qi=1 e Adi η(θ)dθ ∗ Pqd −τi T . α = q + i=1 τi e Adi e

(10.27)

Proof. It follows from Lemma 10.3 that there exist nonnegative matrices Qi , i = 1, . . . , qd , such that (10.24) holds. Now, consider the Lyapunov-Krasovskii functional V : C+ → R given by qd Z 0 X T D ψ T (θ)AT V (ψ(·)) = ψ (0)ψ(0) + (10.28) di Qi Adi ψ(θ)dθ, i=1

−τi

and note that the directional derivative of V (xt ) along the trajectories of (10.22) is given by V˙ (xt ) = 2xT (t)x(t) ˙ +

qd X

D xT (t)AT di Qi Adi x(t)

i=1

−

qd X i=1

D xT (t − τi )AT di Qi Adi x(t − τi )

= 2xT (t)Ax(t) + 2xT (t)

qd X i=1

+

qd X

Adi x(t − τi )

D xT (t)AT di Qi Adi x(t)

i=1

− ≤ −

qd X i=1

qd X i=1 T

D xT (t − τi )AT di Qi Adi x(t − τi )

[xT (t)Qi x(t) − 2xT (t)Adi x(t − τi )

D +x (t − τi )AT di Qi Adi x(t − τi )] qd X = − [−Qi x(t) + Adi x(t − τi )]T QD i [−Qi x(t) + Adi x(t − τi )] i=1

≤ 0,

t ≥ 0.

(10.29)

Next, let R , {ψ(·) ∈ C+ : −Qi ψ(0) + Adi ψ(−τi ) = 0, i = 1, . . . , qd } and note that, since the positive orbit Oη+ of (10.22) is bounded, Oη+ belongs to a compact subset of C+ , and hence, it follows from Theorem 3.3 that xt → M, where set contained in R. Now, Pqd M denotes the largest invariant ˆ , {ψ(·) ∈ C+ : Aψ(0) + since A + Q = 0, it follows that R ⊂ R i i=1 Pq d Pq d i=1 Adi ) = q − 1 and (A + i=1 Adi ψ(−τi ) = 0}. Hence, since rank(A +

NNBook

September 1, 2009

327

EQUIPARTITION AND SEMISTABILITY

Pq d

ˆ ˆ contained in R = 0, it follows that the largest invariant set M ˆ = {ψ ∈ C+ : ψ(θ) = αe, θ ∈ [−¯ is given by M τ , 0], α ≥ 0}. Furthermore, ˆ ˆ ˆ since M ⊂ R ⊂ R, it follows that M = M. i=1 Adi )e

Next, define the functional E : C+ → R by qd Z 0 X E(ψ(·)) = eT ψ(0) + eT Adi ψ(θ)dθ, i=1

(10.30)

−τi

˙ t ) ≡ 0 along the trajectories of (10.22). Thus, for all t ≥ 0, and note that E(x qd Z 0 X T E(xt ) = E(η(·)) = e η(0) + eT Adi η(θ)dθ, (10.31) i=1

−τi

which implies that xt → M ∩ E, where E , {ψ(·) ∈ C+ : E(ψ(·)) = E(η(·))}. Hence, since M ∩ E = {α∗ e}, it follows that x(t) → α∗ e, where α∗ is given by (10.27). Finally, Lyapunov stability of αe, α ≥ 0, follows by considering the Lyapunov-Krasovskii functional V (ψ(·)) = (ψ(0) − αe)T (ψ(0) − αe) qd Z 0 X D + (ψ(θ) − αe)T AT di Qi Adi (ψ(θ) − αe)dθ i=1

−τi

and noting that V (ψ) ≥ kψ(0) − αek22 .

Note that if qd = q 2 − q, Ad = AT d , and (A + Ad )e = 0, then (10.22) can be rewritten as x˙ i (t) = −

q X

j=1,j6=i

aij [xi (t) − xj (t − τij )], x(θ) = η(θ), −¯ τ ≤ θ ≤ 0, t ≥ 0,

(10.32) where i = 1, . . . , q, and τij ∈ [0, τ¯], i 6= j, i, j = 1, . . . , q, which implies that the rate of material transfer from the ith compartment to the jth compartment is proportional to the difference xj (t − τij ) − xi (t). Hence, the rate of material transfer is positive (respectively, negative) if xj (t − τij ) > xi (t) (respectively, xj (t − τij ) < xi (t)). Equation (10.32) is an energy flow balance equation that governs the energy exchange among coupled subsystems and is completely analogous to the equations of thermal transfer with subsystem energy playing the role of temperatures. Furthermore, note that since aij ≥ 0, i 6= j, i, j = 1, . . . , q, energy flows from more energetic subsystems to less energetic subsystems, which is consistent with the second law of thermodynamics requiring that heat (energy) must flow in the direction of lower temperatures.

NNBook

September 1, 2009

328

CHAPTER 10

10.4 Semistability and Equipartition of Nonlinear Compartmental Systems with Time Delay In this section, we extend the results of Section 10.3 to nonlinear compartmental systems with time delay. Specifically, we consider nonlinear timedelay dynamical systems G of the form x(t) ˙ = f (x(t)) + fd (x(t − τ1 ), . . . , x(t − τqd )), x(θ) = η(θ), −¯ τ ≤ θ ≤ 0, t ≥ 0, (10.33) where x(t) ∈ Rq , t ≥ 0, f : Rq → Rq is locally Lipschitz continuous and f (0) = 0, fd : Rq × · · · × Rq → Rq is locally Lipschitz continuous and fd (0, . . . , 0) = 0, τ¯ = maxi∈{1,...,qd } τi , τi ≥ 0, i = 1, . . . , qd , and η(·) ∈ C = C([−¯ τ , 0], Rq ) is a continuous vector-valued function specifying the initial state of the system. Note that since η(·) is continuous it follows from Theorem 2.3 of [128, p. 44] that there exists a unique solution x(η) defined on [−¯ τ , ∞) that coincides with η on [−¯ τ , 0] and satisfies (10.33) for all t ≥ 0. In addition, recall that if the positive orbit Oη+ of (10.33) is bounded, then Oη+ is precompact [126]. Here, we assume that f (·) is essentially nonnegative and fd (·) is nonnegative so that for every η(·) ∈ C+ , the nonlinear time-delay dynamical system G given by (10.33) is nonnegative. Next, we consider a subclass of nonlinear nonnegative systems, namely, nonlinear compartmental systems. Definition 10.3. The nonlinear time-delay dynamical system (10.33) is called a compartmental dynamical system if F (·) is compartmental, where F (x) , f (x) + fd (x, x, . . . , x). Note that the nonlinear time-delay dynamical system is compartmental if f = [f1 , . . . , fq ]T and fd = [fd1 , . . . , fdq ]T are given by fi (x(t)) = − fdi (x(t − τ1 ), . . . , x(t − τqd )) =

q X

aji (x(t)),

j=1,j6=i q X

j=1,j6=i

aij (x(t − τij )),

where aii (x(·)) ≥ 0, x(·) ∈ C+ , aii (0) = 0, i ∈ {1, . . . , q}, denotes the instantaneous rate of flow of material loss of the ith compartment, aij (x(·)) ≥ 0, x(·) ∈ C+ , i 6= j, i, j ∈ {1, . . . , q}, denotes the instantaneous rate of material flow from the jth compartment to the ith compartment, τij , i 6= j, i, j ∈ {1, . . . , q}, denotes the transfer time of material flow from the jth compartment to the ith compartment, and aii (·) and aij (·) are such that q if xi = 0, then aii (x) = 0 and aji (x) = 0 for all i, j = 1, . . . , q, and x ∈ R+ .

NNBook

September 1, 2009

329

EQUIPARTITION AND SEMISTABILITY

Note that the above constraints imply that f (·) is essentially nonnegative and fd (·) is nonnegative. The next result generalizes Theorem 10.4 to nonlinear time-delay compartmental systems of the form x(t) ˙ = f (x(t)) +

qd X i=1

fd i (x(t − τi )),

x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

(10.34) q q where f : R+ → R+ is given by f (x) = [f1 (x1 ), . . . , fq (xq )]T , f (0) = 0, q q fd i : R+ → R+ , i = 1, . . . , qd , and fd (0) = 0. For this result, we assume that fi (·), i = 1, . . . , q, are strictly decreasing functions. Theorem 10.5. Consider the nonlinear time-delay dynamical system given by (10.34) where P fi (·), i = 1, . . . , q, is strictly decreasing fi (0) = 0. Pand q qd qd T Assume that e [f (x)+ i=1 fd i (x)] = 0, x ∈ R+ , and f (x)+ i=1 fd i (x) = 0 if and only if x = αe for some α ≥ 0. Furthermore, assume there exist q×q nonnegative diagonal matrices Pi ∈ R+ , i = 1, . . . , qd , such that P , Pq d i=1 Pi > 0, q

PiD Pi fd i (x) = fd i (x),

qd X i=1

x ∈ R+ ,

i = 1, . . . , qd , q

T fd T i (x)Pi fd i (x) ≤ f (x)P f (x),

x ∈ R+ .

(10.35)

(10.36)

Then, for every α ≥ 0, αe is a semistable equilibrium point of (10.34). Furthermore, x(t) → α∗ e as t → ∞, where α∗ satisfies ∗

qα +

qd X

∗

T

T

τi e fd i (α e) = e η(0) +

i=1

qd Z X i=1

0

−τ i

eT fd i (η(θ))dθ.

(10.37)

Proof. Consider the Lyapunov-Krasovskii functional V : C+ → R given by V (ψ(·)) = −2

q Z X i=1

ψi (0) 0

P(i,i) fi (ζ)dζ +

qd Z X i=1

0

−τi

fd T i (ψ(θ))Pi fd i (ψ(θ))dθ.

(10.38) Since, fi (·), i = 1, . . . , q, is a strictly decreasing function it follows that P V (ψ) ≥ 2 qi=1 P(i,i) [−fi (δi ψi (0))]ψi (0) > 0 for all ψ(0) 6= 0, where 0 < δi < 1, and hence, there exists a class K function α(·) such that V (ψ) ≥ α(kψ(0)k). Now, note that the directional derivative of V (xt ) along the trajectories of (10.34) is given by V˙ (xt ) = −2f T (x(t))P x(t) ˙ +

qd X i=1

fd T i (x(t))Pi fd i (x(t))

NNBook

September 1, 2009

330

CHAPTER 10

−

qd X i=1

fd T i (x(t − τi ))Pi fd i (x(t − τi ))

T

= −2f (x(t))P f (x(t)) − 2 +

qd X i=1

−

i=1

i=1

fd T i (x(t))Pi fd i (x(t)) −

≤ −f T (x(t))P f (x(t)) − 2 qd X

qd X

qd X i=1

f T (x(t))P fd i (x(t − τi )) qd X i=1

fd T i (x(t − τi ))Pi fd i (x(t − τi ))

f T (x(t))P PiD Pi fd i (x(t − τi ))

D fd T i (x(t − τi ))Pi Pi Pi fd i (x(t − τi ))

qd X = − [P f (x(t)) + Pi fd i (x(t − τi ))]T PiD i=1

·[P f (x(t)) + Pi fd i (x(t − τi ))] ≤ 0, t ≥ 0,

(10.39)

where the first inequality in (10.39) follows from (10.35) and (10.36), and the last equality in (10.39) follows from the fact that f T (x)P f (x) = P q qd T D i=1 f (x)P Pi P f (x), x ∈ R+ .

Next, let R , {ψ(·) ∈ C+ : P f (ψ(0)) + Pi fd i (ψ(−τi )) = 0, i = 1, . . . , qd } and note that, since the positive orbit Oη+ of (10.34) is bounded, Oη+ belongs to a compact subset of C+ , and hence, it follows from Theorem 3.3 that xt → M, where M denotes the largest invariant Pqd set (with T respect to (10.34)) contained in R. Now, since e (f (x) + i=1 fd i (x)) = q 0, x ∈ R+ , it follows that ˆ , {ψ(·) ∈ C+ : f (ψ(0)) + R⊂R

qd X

fd i (ψ(−τi )) = 0}

i=1

= {ψ(·) ∈ C+ : ψ(θ) = αe, θ ∈ [−¯ τ , 0], α ≥ 0}, ˆ as t → ∞. which implies that xt → R Next, define the functional E : C+ → R by T

E(ψ(·)) = e ψ(0) +

qd Z X i=1

0

−τ i

eT fd i (ψ(θ))dθ,

(10.40)

NNBook

September 1, 2009

331

EQUIPARTITION AND SEMISTABILITY

˙ t ) ≡ 0 along the trajectories of (10.34). Thus, for all t ≥ 0, and note that E(x qd Z 0 X T E(xt ) = E(η(·)) = e η(0) + eT fd i (η(θ))dθ, (10.41) i=1

−τ i

ˆ ∩ E, where E , {ψ(·) ∈ C+ : E(ψ(·)) = E(η(·))}. which implies that xt → R ˆ ∩ E = {α∗ e}, it follows that x(t) → α∗ e, where α∗ satisfies (10.37). Hence, R

Finally, Lyapunov stability of αe, α ≥ 0, follows by considering the Lyapunov-Krasovskii functional q Z ψi (0) X V (ψ(·)) = −2 P(i,i) (fi (ζ) − fi (α))dζ +

i=1 α q d Z 0 X i=1

−τi

[fd i (ψ(θ)) − fd i (αe)]T Pi [fd i (ψ(θ)) − fd i (αe)]dθ,

P and noting that V (ψ) ≥ 2 qi=1 P(i,i) [fi (α) − fi (α + δi (ψi (0) − α))](ψi (0) − α) > 0, for all ψi (0) 6= α, where 0 < δi < 1. Theorem 10.5 establishes semistability and state equipartition for the special case of nonlinear compartmental systems of the form (10.34) where f (·) and fdi (·), i = 1, . . . , q, satisfy (10.35) and (10.36). For general qdimensional nonlinear compartmental systems with time delay and vector fields given by (10.34) it is not possible to guarantee semistability and state equipartition. However, semistability without state equipartition may be shown. For example, consider the nonlinear time-delay compartmental dynamical system given by x˙ 1 (t) = −a21 (x1 (t)) + a12 (x2 (t − τ12 )), x1 (θ) = η1 (θ), −¯ τ ≤ θ ≤ 0, t ≥ 0, x˙ 2 (t) = −a12 (x2 (t)) + a21 (x1 (t − τ21 )), x2 (θ) = η2 (θ), −¯ τ ≤ θ ≤ 0, t ≥ 0,

(10.42) (10.43)

where x1 (t), x2 (t) ∈ R, t ≥ 0, a12 : R+ → R+ and a21 : R+ → R+ satisfy a12 (0) = a21 (0) = 0 and a12 (·) and a21 (·) are strictly increasing, τ12 , τ21 > 0, τ¯ = max{τ12 , τ21 }, and η1 (·), η2 (·) ∈ C+ = C([−¯ τ , 0], R+ ). Note that (10.42) and (10.43) can have multiple equilibria with all the equilibria lying on the curve a21 (u) = a12 (v), u, v ≥ 0. It follows from the conditions on a12 (·) and a21 (·) that all system equilibria lie on the curve x2 = a−1 12 (a21 (x1 )) in the −1 (x1 , x2 ) plane, where a12 (·) denotes the inverse function of a12 (·). Consider the functional E : C+ × C+ → R given by Z 0 Z 0 E(ψ1 , ψ2 ) = ψ1 (0) + ψ2 (0) + a12 (ψ2 (θ))dθ + a21 (ψ1 (θ))dθ. −τ12

−τ21

NNBook

September 1, 2009

332

CHAPTER 10

Now, it can be easily shown that the directional derivative of E(ψ1 , ψ2 ) along the trajectories of (10.42) and (10.43) is identically zero for all t ≥ 0, which implies that, for all t ≥ 0, E(x1t , x2t ) = E(η1 , η2 ) = η1 (0) + η2 (0) Z Z 0 + a12 (η2 (θ))dθ + −τ12

0 −τ21

a21 (η1 (θ))dθ.

Next, consider the functional V : C+ × C+ → R given by Z ψ1 (0) Z ψ2 (0) V (ψ1 , ψ2 ) = 2 a21 (θ)dθ + 2 a12 (θ)dθ 0 0 Z 0 Z 0 + a212 (ψ2 (θ))dθ + a221 (ψ1 (θ))dθ, −τ12

(10.44)

(10.45)

−τ21

and note that the directional derivative of V (ψ1 , ψ2 ) along the trajectories of (10.42) and (10.43) is given by V˙ (x1t , x2t ) = −[a21 (x1 (t)) − a12 (x2 (t − τ12 ))]2 −[a12 (x2 (t)) − a21 (x1 (t − τ21 ))]2 .

(10.46)

Now, using similar arguments as in the proof of Theorem 10.5 it follows ∗ ∗ that (x1 (t), x2 (t)) → (α∗ , a−1 12 (a21 (α ))) as t → ∞, where α is the solution to the equation ∗ ∗ α∗ + a−1 12 (a21 (α )) + (τ12 + τ21 )a21 (α ) Z 0 Z = η1 (0) + η2 (0) + a12 (η2 (θ))dθ + −τ12

0 −τ21

a21 (η1 (θ))dθ, (10.47)

∗ and (α∗ , a−1 12 (a21 (α ))) is a Lyapunov stable equilibrium state. The above analysis shows that all two-dimensional nonlinear compartmental dynamical systems of the form (10.42) and (10.43) are semistable with system states reaching equilibria lying on the curve x2 = a−1 12 (a21 (x1 )) in the (x1 , x2 ) plane.

To demonstrate the utility of Theorem 10.5 we consider a nonlinear two-compartment time-delay dynamical system given by x˙ 1 (t) = −

x˙ 2 (t) =

qd X i=1

qd X i=1

[ai (x1 (t)) + ai (x2 (t − τi ))],

x1 (θ) = η1 (θ),

−¯ τ ≤ θ ≤ 0,

[ai (x1 (t − τi )) − ai (x2 (t))],

t ≥ 0,

(10.48)

x2 (θ) = η2 (θ),

−¯ τ ≤ θ ≤ 0,

(10.49)

NNBook

September 1, 2009

333

EQUIPARTITION AND SEMISTABILITY

where ai : R+ → R+ , i = 1, . . . , qd , are such that, for every i = 1, . . . , qd , [ai (x1 ) − ai (x2 )](x1 − x2 ) > 0,

x1 6= x2 ,

(10.50)

and ai (0) = 0, i = 1, . . . , qd . If x1 and x2 represent system energies, then (10.48) and (10.49) capture energy flow balance between the two compartments, and (10.50) is consistent with the second law of thermodynamics, that is, energy flows from the more energetic compartment to the less energetic compartment. Furthermore, since ai (0) = 0, (10.50) implies that ai (·), i = 1, . . . , qd , is strictly increasing. Now, note that (10.48) and (10.49) can be written in the form of (10.34) with    Pq d  ai (x2 ) − Pi=1 ai (x1 ) , i = 1, . . . , qd , (10.51) f (x) = , fd i (x) = d ai (x1 ) − qi=1 ai (x2 ) which implies that fj (xj ), j = 1, 2, are strictly decreasing. Next, with Pi = In , i = 1, . . . , qd , (10.35) and (10.36) are trivially satisfied, and hence, it follows from Theorem 10.5 that x1 (t) − x2 (t) → 0 as t → ∞.

Next, we consider nonlinear compartmental time-delay dynamical systems of the form x˙ i (t) = −

q X

aji (xi (t)) +

j=1,j6=i

q X

j=1,j6=i

aij (xj (t − τi )),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

x(θ) = η(θ), (10.52)

where i = 1, . . . , q, aij : R+ → R+ , i 6= j, i, j ∈ {1, . . . , q}, are such that aij (0) = 0 and aij (·), i 6= j, i, j = 1, . . . , q, is strictly increasing. Note that since each transfer coefficient aij (·) is only a function of xj and not x, the nonlinear compartmental system (10.52) is a nonlinear donor-controlled compartmental system [158]. In this case, (10.52) can be written in the form given by (10.34) with qd = q, fi (xi ) = −

q X

aji (xi ),

fd i (x) = ei

q X

aij (xj ),

i = 1, . . . , q. (10.53)

j=1

j=1,j6=i

Next, with Pi = ei eT i , i = 1, . . . , q, so that P = Iq , it follows that (10.35) is trivially satisfied and (10.36) holds if and only if  2  2 q q q q X X X X q   (10.54) aij (xj ) ≤ aji (xi ) , x ∈ R+ . i=1

j=1,i6=j

i=1

j=1,i6=j

In the case where q = 2, (10.54) is trivially satisfied, and hence, it follows from Theorem 10.5 that x1 (t) − x2 (t) → 0 as t → ∞. In general, (10.54) does not hold for arbitrary strictly increasing functions aij (·). However, if

NNBook

September 1, 2009

334

CHAPTER 10

aij (·) = σ(·), i 6= j, i, j = 1, . . . , q, where σ : R+ → R+ is such that σ(0) = 0 and σ(·) is strictly increasing, (10.54) holds if and only if 2  2  q q q q X X X X   σ(xj ) ≤ σ(xi ) , x ∈ Rq+ . (10.55) i=1

i=1

j=1,i6=j

j=1,i6=j

In this case, since

0 ≥ (q − 1)

q X i=1

−(q − 1)2 = −(q − 2)

2

σ (xi ) + (q − 2)

q X

q q X X

σ(xi )σ(xj )

i=1 j=1,j6=i

σ 2 (xi )

i=1

q q X X

(σ(xi ) − σ(xj ))2 ,

i=1 j=1,j6=i

(10.55) holds, and hence, it follows from Theorem 10.5 that xi (t)−xj (t) → 0 as t → ∞, where i 6= j, i, j = 1, . . . , q. Next, we specialize Theorem 10.5 to nonlinear time-delay compartmental systems of the form x(t) ˙ = Aˆ σ (x(t)) +

qd X i=1

q R+

Adi σ ˆ (x(t − τi )),

x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0, t ≥ 0,

q R+

(10.56)

→ is given by σ ˆ (x) = [σ(x1 ), σ(x2 ), . . . , σ(xq )]T , where where σ ˆ : σ : RP + → R+ is such that σ(u) = 0 if and only if u = 0, and A and d Ad = qi=1 Adi are as given by (10.23).

Theorem 10.6. Consider the nonlinear time-delay system given by (10.56) where σ : R+ → R+ is that σ(0) = 0Pand σ(·) is strictly Psuch d d increasing.PAssume that (A + qi=1 Adi )T e = (A + qi=1 Adi )e = 0 and qd rank(A + i=1 Adi ) = q − 1. Then, for every α ≥ 0, αe is a semistable equilibrium point of (10.56). Furthermore, x(t) → α∗ e as t → ∞, where α∗ satisfies qd qd Z 0 X X ∗ ∗ T T eT Adi σ ˆ (η(θ))dθ. (10.57) nα + σ(α ) τi e Adi e = e η(0) + i=1

i=1

−τi

Proof. It follows from Lemma 10.3 that there exists Qi , i = 1, . . . , qP (10.24) holds with Qi given by Now, since d , such that P P (10.25). d d d Pi , it follows from A = − qi=1 Qi = − qi=1 PiD = −P −1 , where P = qi=1

NNBook

September 1, 2009

335

EQUIPARTITION AND SEMISTABILITY q

(10.24) that, for all x ∈ R+ , 0 ≥ 2ˆ σ T (x)Aˆ σ (x) + σ ˆ T (x) = −f T (x)P f (x) +

qd X

qd X D (Qi + Ad T σ (x) i Qi Adi )ˆ i=1

fd T i (x)Pi fd i (x),

i=1

q

where f (x) = Aˆ σ (x) and fd i (x) = Adi σ ˆ (x), i = 1, . . . , qd , x ∈ R+ . Furthermore, since PiD Pi Adi = Adi , i = 1, . . . , qd , it follows that PiD Pi fd i (x) = q fd i (x), i = 1, . . . , qd , x ∈ R+ . Now, the resultPis an immediate consequence d of Theorem 10.5 by noting that eT [f (x) + qi=1 fd i (x)] = 0 and f (x) + P qd f (x) = 0 if and only if x = αe for some α ≥ 0. i=1 d i Example 10.1. In this example, we apply Theorem 10.4 and 10.6 to a linear and nonlinear compartmental system with time delay. Specifically, consider x˙ i (t) = −

q X

aji xi (t) +

j=1,i6=j

q X

j=1,i6=j

aij xj (t − τij ),

xi (θ) = ηi (θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

(10.58)

x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

for all i = 1, . . . , q, or, equivalently, x(t) ˙ = Ax(t) +

qd X l=1

Adl x(t − τl ),

(10.59)

where qd , q 2 , A ∈ Rq×q , and Adl ∈ Rq×q , l = 1, . . . , qd , with   q−1 q X X A = diag − aj1 , . . . , − ajq  , j=2

(10.60)

j=1

Ad((i−1)q+j) = aij ei eT j , and τ((i−1)q+j) = τij , i, j = 1, . . . , q. Note that if (j, i) 6∈ E, then Ad((i−1)q+j) = 0. Furthermore, it can be easily shown that (A + Ad )T e = 0, where Pq d Ad , l=1 Adl , and rank(A + Ad ) = q − 1 if and only if for every pair of nodes (i, j) ∈ V there exists a path from compartment i to compartment j [105]. Here, we assume that the adjacency matrix A is chosen such that (A+Ad )e = 0 so that the linear time-delay compartmental dynamical system (10.59) satisfies all the conditions of Theorem 10.4. Hence, it follows from Theorem 10.4 that the compartmental system given by (10.59) achieves state equipartition, that is, limt→∞ xi (t) = limt→∞ xj (t) = α∗ , i, j = 1, . . . , q, i 6= j, where α∗ is given by (10.27).

NNBook

September 1, 2009

336

CHAPTER 10

- i - i i @ I@ I @ 6 @ @@ @ @ @@ @ @ a19@@ , a91 , a48 , a28@ ,@a82 , @ τ19 @τ91 τ28@@τ82 τ48 @ @ @ @ @ @ @ @ @ @ R @ Ri @  i i

1

a1,10 ,τ1,10

10

a21 ,τ21

a10,9 ,τ10,9

2

9

a32 ,τ32

a43 ,τ43

3

a98 ,τ98

- i 

a54 ,τ54

a57 , a75 , τ57 τ75

a84 , τ84

a87 ,τ87

8

4

 i

7

a76 ,τ76

- i 

5

a65 ,τ65 ? i

6

Figure 10.1 Compartmental model.

Alternatively, it follows from Theorem 10.6 that the nonlinear compartmental dynamical system given by x(t) ˙ = Aˆ σ (x(t)) +

qd X i=1

Adi σ ˆ (x(t − τi )),

x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

(10.61) also achieves state equipartition if σ(·) and σ ˆ (·) satisfy the conditions in Theorem 10.6. In this case, limt→∞ xi (t) = limt→∞ xj (t) = α∗ , i, j = 1, . . . , q, i 6= j, where α∗ is a solution to (10.57). Note that if σ(θ) = θ, (10.61) specializes to (10.59). To illustrate the two models given by (10.59) and (10.61) consider the compartmental system given by the graph shown in Figure 10.1 where aij and τij denote the weight and the time delay for each edge shown. Here, we choose ai,j = 1 if (i, j) ∈ E so that (A + Ad )e = 0. In addition, it can be easily shown that rank(A+Ad ) = q−1 = 9. With x0 = [1 2 3 4 5 6 7 8 9 10]T , Figures 10.2 and 10.3 demonstrate state equipartitioning of (10.58) and (10.61) with σ(θ) = tanh(θ) in (10.61). △ Next, we establish a sufficient condition for guaranteeing asymptotic stability of the zero solution xt ≡ 0 to (10.56). Theorem 10.7. Consider the nonlinear nonnegative time-delay dynamical system given by (10.56) where σ : R+ → R+ is such that σ(·) is positive, A ∈ Rq×q is essentially nonnegative, and Ad ∈ Rq×q is nonnegative. If there exist p, r ∈ Rq such that p >> 0 and r >> 0 satisfy 0=

A+

qd X i=1

Adi

!T

p + r,

(10.62)

then the zero solution xt ≡ 0 to (10.56) is asymptotically stable for all τ¯ ∈ [0, ∞).

NNBook

September 1, 2009

337

EQUIPARTITION AND SEMISTABILITY

10 9 8

Node Values

7 6 5 4 3 2 1 0

0

5

10

15

20 Time

25

30

35

40

Figure 10.2 Linear compartmental model.

Proof. Consider the Lyapunov-Krasovskii functional given by qd Z 0 X T V (ψ) = p ψ(0) + pT Adi σ ˆ (ψ(θ))dθ, (10.63) i=1

−τi

and note that V (ψ) ≥ mini∈{1,...,q} pi kψ(0)k. Next, using (10.62), it follows that the Lyapunov-Krasovskii directional derivative along the trajectories of (10.56) is given by V˙ (xt ) = pT x(t) ˙ + = p

T

A+

qd X

i=1 qd X

pT Adi [ˆ σ (x(t)) − σ ˆ (x(t − τi ))] Adi

i=1

= −r T σ ˆ (x(t)) ≤ −β(kx(t)k),

!

σ ˆ (x(t))

t ≥ 0,

(10.64)

where β : R+ → R+ is a class K function. Now, it follows from Theorem 3.1 that the zero solution xt ≡ 0 to (10.56) is asymptotically stable. Finally, we consider a class of nonlinear compartmental dynamical systems given by x˙ i (t) = −

q X j=1

aji (xi (t)) +

q X

j=1,j6=i

aij (xj (t − τij )),

x(θ) = η(θ),

NNBook

September 1, 2009

338

CHAPTER 10

10 9 8

Node Values

7 6 5 4 3 2 1 0

0

20

40

60

80

100

Time

Figure 10.3 Nonlinear compartmental model.

−¯ τ ≤ θ ≤ 0,

t ≥ 0,

(10.65)

where i = 1, . . . , q, and, for each i, j ∈ {1, . . . , q}, aij (·) satisfies aij (0) = 0 and aij (u) ≤ Lij , u > 0, (10.66) Mij ≤ u where Mij ≥ 0, i, j = 1, . . . , q. Next, using the inequality [130] ! r1 ! 1 m+1 m+1 Y p Pm+1 X − 1r Pm+1 akk ≤ pk ark , (10.67) k=1

k=1

P where ak ≥ 0, pk > 0, k = 1, . . . , m + 1, r > 0, and Pm+1 = m+1 k=1 pk , we establish sufficient conditions for Lyapunov and asymptotic stability of a nonlinear time-delay compartmental system given by (10.65).

Theorem 10.8. Consider the nonlinear time-delay dynamical system given by (10.65). Assume that (10.66) holds. If there exist constants rk > 0, k = 1, . . . , N , bi > 0, i = 1, . . . , q, pij ∈ R, qij ∈ R, i, j = 1, . . . , q, i 6= j, such that q N X X

j=1,j6=i k=1

ppij rk

rk Lij

q n X X bj pqji + L ≤p Mji , bi ji j=1,j6=i

i = 1, . . . , q, (10.68)

j=1

where pij and qij satisfy N pij + qij = 1, i, j = 1 . . . , q, i 6= j, and p = P 1+ N k=1 rk , then the zero solution xt ≡ 0 to (10.65) is Lyapunov stable

NNBook

September 1, 2009

339

EQUIPARTITION AND SEMISTABILITY

for all τij (10.68) is stable for constants

∈ [0, ∞), i, j = 1, . . . , q, i 6= j. If, in addition, the inequality in strict, then the zero solution xt ≡ 0 to (10.65) is asymptotically all τij ∈ [0, ∞), i, j = 1, . . . , q, i 6= j. Alternatively, if there exist bi > 0, i = 1, . . . , q, such that q n X X bj Lji ≤ Mji , bi

i = 1, . . . , q,

(10.69)

j=1

j=1,j6=i

then the zero solution xt ≡ 0 to (10.65) is Lyapunov stable for all τij ∈ [0, ∞), i, j = 1, . . . , q, i 6= j. Finally, if the inequality (10.69) is strict, then the zero solution xt ≡ 0 to (10.65) is asymptotically stable for all τij ∈ [0, ∞), i, j = 1, . . . , q, i 6= j. Proof. To show Lyapunov stability of the zero solution xt ≡ 0 to (10.65), consider the Lyapunov-Krasovskii functional given by V (ψ(·)) =

q X bi i=1

p

ψip (0) +

Z q q X X bi pqij 0 Lij ψjp (θ)dθ, p −τij

(10.70)

i=1 j=1,j6=i

Pq bi p p bi and note that V (ψ) ≥ i=1 p ψi (0) ≥ mini∈{1,...,q} p kψ(0)kp . Then the directional derivative of V (ψ) along the trajectories of (10.65) is given by q q X X bi pqij p L (xj (t) − xpj (t − τij )) + p ij i=1 i=1 j=1,j6=i   q q q X X X aij (xj (t − τij )) = bi xp−1 (t) − aji (xi (t)) + i

V˙ (xt ) =

q X

bi xp−1 (t)x˙ i (t) i

i=1

+

j=1

q X

q X

i=1 j=1,j6=i

≤

q X i=1

+



bi −

q X

q X

i=1 j=1,j6=i

=

q X i=1

+



bi −

q X

bi pqij p L (xj (t) − xpj (t − τij )) p ij Mji xpi (t) +

j=1

q X

q X j=1

q X

i=1 j=1,j6=i

j=1,j6=i

q X

j=1,j6=i



Lij xj (t − τij )xp−1 (t) i

bi pqij p L (xj (t) − xpj (t − τij )) p ij Mji xpi (t) +

q N  X Y

pij

Lijrk

j=1,j6=i k=1

bi pqij p L (xj (t) − xpj (t − τij )) p ij

  rk q xi (t) Lijij xj (t − τij )

NNBook

September 1, 2009

340

CHAPTER 10

≤ − +

q X i=1

q X



bi − q X

q X j=1

i=1 j=1,j6=i

= −

q X i=1

≤ −c

 q ppij N X X 1 r Mji xpi (t) + rk Lij k xpi (t) p j=1,j6=i k=1

bi pqij p L x (t) p ij j

  q q q N ppij X X X X b 1 1 j pqji  p r L bi  Mji − rk Lij k − xi (t) p p bi ji

q X

j=1

j=1,j6=i k=1

j=1,j6=i

xpi (t),

(10.71)

i=1

where c , min

1≤i≤q

q X i=1



bi 

q X j=1

 q q ppij N X X X 1 1 bj pqji  r L Mji − rk Lij k − ≥0 p p bi ji j=1,j6=i k=1

j=1,j6=i

(10.72)

and where inequality (10.67) was used in (10.71). This establishes Lyapunov stability of the zero solution xt ≡ 0 to (10.65). Next, if the inequality in (10.68) is strict, then it follows from (10.72) that c > 0. In this case, it follows from (10.71) that Z tX q c xpi (θ)dθ ≤ V (x0 ) − V (xt ), t ≥ 0, (10.73) 0 i=1

R ∞ Pq p which implies that 0 i=1 xi (θ)dθ < P∞, and hence, xi (t) and x˙ i (t) are p bounded on (0, ∞). Hence, xi (t) and qi=1 xpi (t) are uniformly continuous on (0, ∞). P Now, it follows from Barbalat’s Lemma [112, p. 221] that limt→∞ qi=1 xpi (t) = 0, and hence, the zero solution xt ≡ 0 to (10.65) is asymptotically stable. Finally, to show Lyapunov and asymptotic stability in the case where (10.69) holds, consider the Lyapunov-Krasovskii functional given by Z 0 q q q X X X V (ψ(·)) = bi ψi (0) + bi Lij ψj (θ)dθ (10.74) i=1

i=1 j=1,j6=i

−τij

and note that V (ψ) ≥ mini∈{1,...,q} bi kψ(0)k. Now, the result follows by using similar arguments as above. If we set rk = p − 1, N = 1, pij = p−1 p , and qij = then the following corollary is immediate.

1 p

in Theorem 10.8,

NNBook

September 1, 2009

341

EQUIPARTITION AND SEMISTABILITY

Corollary 10.1. Consider the nonlinear time-delay dynamical system given by (10.65). Assume that (10.66) holds. If there exist constants p ≥ 1 and bi > 0, i = 1, . . . , q, such that q X

q

X bj (p − 1)Lij + Lji ≤ p Mji , bi

j=1,j6=i

i = 1, . . . , q,

(10.75)

j=1

then the zero solution xt ≡ 0 to (10.65) is Lyapunov stable for all τij ∈ [0, ∞), i, j = 1, . . . , q, i 6= j. Alternatively, if the inequality (10.75) is strict, then the zero solution xt ≡ 0 to (10.65) is asymptotically stable for all τij ∈ [0, ∞), i, j = 1, . . . , q, i 6= j. It follows from Corollary 10.1 that the time-delay compartmental dynamical system (10.65) is asymptotically stable as long as Mii , i = 1, . . . , q, are sufficiently large.

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Eleven

Robustness of Nonnegative Dynamical Systems

11.1 Introduction Even though numerous analysis results for nonnegative and compartmental dynamical systems have been developed over the last several years in the literature, robustness properties of these systems have been largely ignored. Robustness here refers to sensitivity of the system stability in the face of model uncertainty. In this chapter, we build on the results of Chapters 8 and 10 to examine the robustness of several nonnegative and compartmental systems of a specified structure. In particular, we develop sufficient conditions for robust stability of compartmental systems involving higher-order perturbation terms that scale in a consistent fashion with respect to a scaling operation on an underlying space, with the additional property that the energy flow functions can be written as a sum of functions, each homogeneous with respect to a fixed scaling operation, that retain system semistability and equipartitioning. In addition, energy flow functions containing higher-order perturbation terms involving a thermodynamic structure are also explored.

11.2 Nominal System Model In this chapter, we consider nonlinear dynamical systems of the form x(t) ˙ = f (x(t)), n

x(0) = x0 ,

t ∈ I x0 ,

(11.1)

where x(t) ∈ D ⊆ R+ , t ∈ Ix0 , is the system state vector, D is a relatively n open set with respect to R+ , f : D → Rn is continuous on D, f −1 (0) , {x ∈ D : f (x) = 0} is nonempty, and Ix0 = [0, τx0 ), 0 ≤ τx0 ≤ ∞, is the maximal interval of existence for the solution x(·) of (11.1). The continuity of f implies that, for every x0 ∈ D, there exist τ0 < 0 < τ1 and a solution x(·) of (11.1) defined on (τ0 , τ1 ) such that x(0) = x0 . Here, we assume that for every initial condition x0 ∈ D, (11.1) has a unique right maximally defined

NNBook

September 1, 2009

344

CHAPTER 11

solution, and this unique solution is defined on [0, ∞). Furthermore, we assume that f (·) is locally Lipschitz continuous on D\f −1 (0). Note that the local Lipschitzness of f (·) on D\f −1 (0) implies local uniqueness in forward and backward time for nonequilibrium initial states. Under these assumptions on f , the solutions of (11.1) define a continuous global semiflow on D, that is, s : [0, ∞) × D → D is a jointly continuous function satisfying the consistency property s(0, x) = x and the semigroup property s(t, s(τ, x)) = s(t + τ, x) for every x ∈ D and t, τ ∈ [0, ∞). Given t ∈ [0, ∞) we denote the flow s(t, ·) : D → D of (11.1) by st (x0 ) or st . Definition 11.1. An equilibrium point x ∈ D of (11.1) is Lyapunov stable under f if for every relatively open subset Nε of D containing x, there exists a relatively open subset Nδ of D containing x such that st (Nδ ) ⊂ Nε for all t ≥ 0. An equilibrium point x ∈ D of (11.1) is semistable under f if it is Lyapunov stable under f and there exists a relatively open subset U of D containing x such that, for every initial condition z ∈ U , the trajectory of (11.1) converges to a Lyapunov stable equilibrium point, that is, limt→∞ s(t, z) = y, where y ∈ D is a Lyapunov stable equilibrium point n of (11.1). If, in addition, U = D = R+ , then an equilibrium point x ∈ D of (11.1) is a globally semistable equilibrium. The system (11.1) is said to be semistable if every equilibrium point of (11.1) is semistable under f . Finally, (11.1) is said to be globally semistable if every equilibrium point of (11.1) is globally semistable under f . Given a continuous function V : D → R, the upper right Dini derivative of V along the solution of (11.1) is defined by 1 V˙ (s(t, x)) , lim sup [V (s(t + h, x)) − V (s(t, x))]. h→0+ h

(11.2)

It is easy to see that V˙ (xe ) = 0 for every xe ∈ f −1 (0). In addition, note that V˙ (x) = V˙ (s(0, x)). Finally, if V (·) is continuously differentiable, then V˙ (x) = V ′ (x)f (x). n

In the sequel, we will need to consider a complete vector field ν on R+ , that is, a vector field ν such that the solutions of the differential equation n n n y(t) ˙ = ν(y(t)) define a continuous global flow ψ : R × R+ → R+ on R+ , where ν −1 (0) = f −1 (0). For each τ ∈ R, the map ψτ (·) = ψ(τ, ·) is a homeomorphism and ψτ−1 = ψ−τ . Our assumptions imply that every n connected component of R+ \f −1 (0) is invariant under ν. n

Recall that a function V : R+ → R is homogeneous of degree l ∈ R

NNBook

September 1, 2009

345

ROBUSTNESS OF NONNEGATIVE SYSTEMS

with respect to ν if and only if (V ◦ ψτ )(x) = elτ V (x),

τ ∈ R,

n

x ∈ R+ .

(11.3)

Note that if l 6= 0, then it follows from (11.3) that V (x) = 0 if x ∈ ν −1 (0). The following proposition provides a useful comparison between positive definite homogeneous functions with respect to an equilibrium set. Proposition 11.1. Assume V1 (·) and V2 (·) are continuous real-valued n functions on R+ , homogeneous with respect to ν of degrees l1 > 0 and l2 > 0, n respectively, and V1 (·) satisfies V1 (x) > 0 for x ∈ R+ \ν −1 (0). Then for each xe ∈ ν −1 (0) and each bounded relatively open neighborhood D0 containing xe , there exist c1 = c1 (D0 ) ∈ R and c2 = c2 (D0 ) ∈ R, where c2 ≥ c1 , such that l2

l2

c1 (V1 (x)) l1 ≤ V2 (x) ≤ c2 (V1 (x)) l1 , If, in addition, V2 (x) < 0 be chosen to additionally

n for x ∈ R+ \ν −1 (0), satisfy c1 ≤ c2 < 0.

x ∈ D0 .

(11.4)

then c1 and c2 in (11.4) may

Proof. Let xe ∈ ν −1 (0) and choose a bounded relatively open neighborhood D0 of xe . Let Q = ψ(R+ × D0 ). For every ε > 0, denote n Qε = Q ∩ V1−1 (ε), define the continuous map τε : R+ \ν −1 (0) → R n by τε (x) , l−1 ln(ε/V1 (x)), and note that, for every x ∈ R+ \ν −1 (0), n n ψ(t, x) ∈ V1−1 (ε) if and only if t = τε (x). Next, define βε : R+ \ν −1 (0) → R+ by βε , ψ(τε (x), x). Note that, for every ε > 0, βε is continuous, and n βε (x) ∈ V1−1 (ε) for every x ∈ R+ \ν −1 (0). Consider ε > 0. Qε is the union of the images of connected components of D0 \ν −1 (0) under the continuous map βε . Since every n connected component of R+ \ν −1 (0) is invariant under −ν, it follows that the n image of each connected component U of R+ \ν −1 (0) under βε is contained in U . In particular, the images of the connected components of D0 \ν −1 (0) under βε are all disjoint. Thus, each connected component of Qε is the image of exactly one connected component of D0 \ν −1 (0) under βε . Finally, if ε is small enough so that V1−1 (ε) ∩ D0 is nonempty, then V1−1 (ε) ∩ D0 ⊆ Qε , and hence, every connected component of Qε has a nonempty intersection with D0 \ν −1 (0). We claim that Qε is bounded for every ε > 0. It is easy to verify that, for every ε1 , ε2 ∈ (0, ∞), Qε2 = ψh (Qε1 ) with h = l−1 ln(ε2 /ε1 ). Hence, it suffices to prove that there exists ε > 0 such that Qε is bounded. To arrive at a contradiction, suppose, ad absurdum, that Qε is unbounded for every ε > 0. Choose a bounded open neighborhood V of D 0 and a sequence {εi }∞ i=1 in (0, ∞) converging to 0. By our assumption, for every i = 1, 2, . . .,

NNBook

September 1, 2009

346

CHAPTER 11 n

at least one connected component of Qεi must contain a point in R+ \V. On the other hand, for i sufficiently large, every connected component of Qεi has a nonempty intersection with D0 ⊂ V. It follows that Qεi has a nonempty intersection with the boundary of V for every i sufficiently large. ∞ Hence, there exist a sequence {xi }∞ i=1 in D0 and a sequence {ti }i=1 in (0, ∞) such that yi , ψti (xi ) ∈ V1−1 (εi ) ∩ ∂V for every i = 1, 2, . . .. Since V is bounded, we can assume that the sequence {yi }∞ i=1 converges to y ∈ ∂V. Continuity implies that V1 (y) = limi→∞ V1 (yi ) = limi→∞ εi = 0. Since V1−1 (0) = ν −1 (0), it follows that y is Lyapunov stable under −ν. Since y 6∈ D 0 , there exists an open neighborhood W of y such that W ∩ D0 = Ø. n The sequence {yi }∞ i=1 converges to y while ψ−ti (yi ) = xi ∈ D0 ⊂ R+ \W, which contradicts Lyapunov stability. This contradiction implies that there exists ε > 0 such that Qε is bounded. It now follows that Qε is bounded for every ε > 0. Finally, consider x ∈ D0 \ν −1 (0). Choose ε > 0 and note that ψτε (x) (x) ∈ Qε . Furthermore, note that V2 (x) is continuous on x ∈ n R+ \ν −1 (0) and Qε ∩ ν −1 (0) = Ø. Then, by homogeneity, V1 (ψτε (x) (x)) = ε, and hence, min V2 (z) ≤ V2 (ψτε (x) (x)) ≤ max V2 (z).

z∈Qε

(11.5)

z∈Qε

Since V2 (ψτε (x) (x)) is homogeneous of degree l2 , it follows that l

− l2

V2 (ψτε (x) (x)) = el2 τε (x) V2 (x) = ε

1

l

− l2

(V1 (x))

1

V2 (x).

l2

l2

Let c1 , ε l1 minz∈Qε V2 (z) and c2 , ε l1 maxz∈Qε V2 (z). Note that c1 and c2 are well defined, and hence, the first assertion is proved. Finally, if V2 (x) < 0 n for x ∈ R+ \ν −1 (0), then it follows from the definitions of c1 and c2 that c1 ≤ c2 < 0. n

Recall that the Lie derivative of a continuous function V : R+ → R with respect to ν is given by 1 Lν V (x) , lim+ [V (ψ(t, x)) − V (x)], (11.6) t→0 t whenever the limit on the right-hand side exists. If V is a continuous homogeneous function of degree l > 0, then Lν V is defined everywhere and satisfies Lν V = lV . We assume that the vector field ν is a semi-Euler vector field, that is, the dynamical system y(t) ˙ = −ν(y(t)),

y(0) = y0 , n

t ≥ 0,

(11.7)

is globally semistable. Thus, for each x ∈ R+ , limτ →∞ ψ(−τ, x) = x∗ ∈ n ν −1 (0), and for each xe ∈ ν −1 (0), there exists z ∈ R+ such that xe = limτ →∞ ψ(−τ, z). If ν −1 (0) = {0}, then the semi-Euler vector field becomes

NNBook

September 1, 2009

347

ROBUSTNESS OF NONNEGATIVE SYSTEMS

the Euler vector field given in [35]. Finally, as in Chapter 8, we say that the vector field f is homogeneous of degree k ∈ R with respect to ν if and only if ν −1 (0) = f −1 (0) and, for every t ∈ R+ and τ ∈ R, st ◦ ψτ = ψτ ◦ sekτ t .

n R+

(11.8)

→ R is a homogeneous function of degree l such that Note that if V : Lf V (x) is defined everywhere, then Lf V (x) is a homogeneous function of degree l + k. Finally, as noted in Chapter 8, if ν and f are continuously n differentiable in a neighborhood of x ∈ R+ , then (11.8) holds at x for sufficiently small t and τ if and only if [ν, f ](x) = kf (x) in a neighborhood n of x ∈ R+ , where the Lie bracket [ν, f ] of ν and f can be computed using ∂ν [ν, f ] = ∂f ∂x ν − ∂x f .

11.3 Semistability and Homogeneous Dynamical Systems As discussed in Chapter 8, homogeneity of dynamical systems is a property whereby system vector fields scale in relation to a scaling operation or dilation on the state space. In this section, we present a robustness result for a vector field that can be written as a sum of several vector fields, each of which is homogeneous with respect to a certain fixed dilation. Theorem 11.1. Let f = g1 + · · · + gp , where, for each i = 1, . . . , p, the vector field gi is continuous, homogeneous of degree mi with respect to ν, and m1 < m2 < · · · < mp . If every equilibrium point in g1−1 (0) is semistable under g1 and is Lyapunov stable under f , then every equilibrium point in g1−1 (0) is semistable under f . Proof. Let every point in g1−1 (0) be a semistable equilibrium under g1 . Choose l > max{−m1 , 0}. Then it follows from Theorem 8.3 that there n exists a continuous homogeneous function V : R+ → R of degree l such that n V (x) = 0 for x ∈ g1−1 (0), V (x) > 0 for x ∈ R+ \g1−1 (0), and Lg1 V satisfies n Lg1 V (x) = 0 for x ∈ g1−1 (0) and Lg1 V (x) < 0 for x ∈ R+ \g1−1 (0). For each i ∈ {1, . . . , p}, Lgi V is continuous and homogeneous of degree l + mi > 0 with respect to ν. Let xe ∈ g1−1 (0) and let Q be a bounded neighborhood of xe . Then it follows from Proposition 11.1 and Theorem 8.3 that there exist c1 > 0, c2 , . . . , cp ∈ R such that Lgi V (x) ≤ −ci (V (x))

l+mi l

,

x ∈ Q,

i = 1, . . . , p.

(11.9)

Hence, for every x ∈ Q, Lf V (x) ≤ − where U (x) , −

Pp

p X

ci (V (x))

l+mi l

= (V (x))

i=1

i=2 ci (V

(x))

mi −m1 l

.

l+m1 l

(−c1 + U (x)),

(11.10)

NNBook

September 1, 2009

348

CHAPTER 11

Since mi − m1 > 0 for every i ≥ 2, it follows that the function U (·), which takes the value 0 on the set g1−1 (0) ∩ Q, is continuous. Hence, there exists a relatively open neighborhood V ⊆ Q of xe such that U (x) < c1 /2 for all x ∈ V. Now, it follows from (11.10) that Lf V (x) ≤ −

l+m1 c1 (V (x)) l , 2

x ∈ V.

(11.11)

Since xe is Lyapunov stable, it follows that one can find a bounded neighborhood W of xe such that solutions in W remain in V. Take an initial condition in W. Since the solution is bounded (remains in Q), it follows from the Krasovskii-LaSalle invariance theorem that this solution converges to its compact positive limit set in f −1 (0). Since all points in f −1 (0) are Lyapunov stable, it follows from Proposition 2.3 that the positive limit set is a singleton involving a Lyapunov stable equilibrium in f −1 (0). Since xe was chosen arbitrarily, it follows that all equilibria in g1−1 (0) are semistable.

11.4 Uncertain System Model In this section, we apply the results of Chapter 10 and the results of Section 11.3 to develop sufficient conditions for robust stability of nonnegative and compartmental dynamical systems. In particular, using the thermodynamically motivated energy flow framework for nonlinear compartmental systems that achieve semistability and state equipartitioning developed in Chapter 10, we develop sufficient conditions for robust stability of nonlinear compartmental systems involving energy flow functions with higher-order perturbation terms. These higher-order terms involve energy flow functions that scale in a consistent fashion with respect to a scaling operation on an underlying space with the additional property that the energy flow functions can be written as a sum of homogeneous functions with respect to a fixed scaling operation. Consider the closed compartmental dynamical system G given by x˙ i (t) =

q X

φij (xi (t), xj (t)),

xi (0) = xi0 ,

j=1, j6=i

t ≥ 0,

i = 1, . . . , q, (11.12)

where the flow function φij (·, ·) satisfies the conditions in Theorem 10.2. In vector form (11.12) is given by x(t) ˙ = f (x(t)),

x(0) = x0 ,

t ≥ 0,

(11.13) q

where x(t) , [x1 (t), . . . , xq (t)]T , t ≥ 0, and f = [f1 , . . . , fq ]T : D ⊆ R+ → Rq

NNBook

September 1, 2009

349

ROBUSTNESS OF NONNEGATIVE SYSTEMS

is such that fi (x) =

q X

φij (xi , xj ).

(11.14)

j=1, j6=i

Furthermore, assume that φij (·, ·) satisfies Assumptions 1 and 2 of Chapter 10. Next, consider (11.12) or, equivalently, (11.13), and assume that the vector field f = [f1 , . . . , fq ] is homogeneous of degree k ∈ R with respect to ν. Finally, consider the perturbed nonnegative model z˙i (t) =

q X

φij (zi (t), zj (t)) + ∆i (z),

zi (0) = zi0 ,

i = 1, . . . , q,

j=1,j6=i

t ≥ 0, (11.15)

q

where ∆ = [∆1 , . . . , ∆q ]T : R+ → R is a continuous function such that ∆ is homogeneous of degree l ∈ R with respect to ν and (11.15) possesses unique q solutions in forward time for initial conditions in R+ \{αe : α ∈ R+ }. Theorem 11.2. Consider the nominal model (11.12) and the perturbed model (11.15). If {αe : α ∈ R+ } = ∆−1 (0), every equilibrium point in {αe : α ∈ R+ } is a Lyapunov stable equilibrium of (11.15), and k < l, then every equilibrium point in {αe : α ∈ R+ } is a semistable equilibrium of (11.12) and (11.15). Proof. It follows from Proposition 10.1 that for every α ∈ R+ , αe is an equilibrium point of (11.12). Next, it follows from Theorem 10.2 that αe is a semistable equilibrium state of (11.12). Now, the result is a direct consequence of Theorem 11.1. As a special case of Theorem 11.2, consider the nominal linear compartmental model given by x˙ i (t) =

q X

j=1,j6=i

C(i,j) [xj (t) − xi (t)],

xi (0) = xi0 ,

i = 1, . . . , q,

t ≥ 0, (11.16)

where, for each i ∈ {1, . . . , q}, xi ∈ R+ , C satisfies Assumption 1 of Chapter 10 and C T = C. Next, consider the perturbed nonnegative model given by z˙i (t) =

q X

j=1,j6=i

C(i,j) [zj (t) − zi (t)] + zi (0) = zi0 ,

q X

j=1,j6=i

δij (zj (t) − zi (t)),

i = 1, . . . , q,

t ≥ 0, (11.17)

NNBook

September 1, 2009

350

CHAPTER 11

and assume δij : R → R is continuously differentiable and satisfies δij ≡ 0 if C(i,j) = 0, δij (λz) = λ1+r δij (z) for all λ > 0 and for some r ≥ 0, and δij (z) = −δji (−z) forP z ∈ R and i, j = 1, . . . , q, i 6= j. Finally, let ∆ = [∆1 , . . . , ∆q ]T , where ∆i = qj=1,j6=i δij (zj − zi ), i = 1, . . . , q.

Proposition 11.2. For i, j = 1, . . . , q, i 6= j, let δij : R → R be continuously differentiable such that δij ≡ 0 if C(i,j) = 0 and δij (λz) = λ1+r δij (z) for all λ > 0 and some r ≥ 0, and δij (z) P = −δji (−z) for all z ∈ R. Furthermore, let ∆ = [∆1 , . . . , ∆q ]T , where ∆i = qj=1,j6=i δij (zj − zi ), i = 1, . . . , q. Then ∆ is homogeneous of degree qr with respect to the semi-Euler i Pq hPq ∂ vector field ν(x) = − i=1 j=1,j6=i (xj − xi ) ∂xi . Proof. First, note that the Lie bracket of   q q X X ∂  ν(x) = − (xj − xi ) ∂xi i=1

j=1,j6=i

and the vector field ∆ is given by [ν, ∆] =

"

q X ∂∆1 i=1

∂xi

νi −

q X ∂ν1 i=1

∂xi

∆i , . . . ,

q X ∂∆q i=1

∂xi

νi −

q X ∂νq i=1

∂xi

∆i

#T

.

Now, it follows from (11.8) and the assumptions on δij that ∆i , i = 1, . . . , q, is homogeneous of degree r with respect to the standard dilation of the form ∆λ (x1 , . . . , xq ) = (λx1 , . . . , λxq ) or, equivalently, the Euler vector field ν , ∆i ] = r∆i , i = 1, . . . , q, or, ν˜(x) = x1 ∂x∂ 1 + · · · + xq ∂x∂ q [35]. Hence, [˜ equivalently, q X ∂∆j

∂xi

i=1

xi = (r + 1)∆j ,

j = 1, . . . , q.

(11.18)

Next, note that νi = −

q X

(xj − xi ) = qxi −

j=1,j6=i

q X

xj ,

i = 1, . . . , q,

(11.19)

j=1

and q X ∂∆j i=1

∂xi

=

q q X X ∂δjs (xs − xj ) = 0, ∂xi i=1 s=1,s6=j

j = 1, . . . , q.

(11.20)

NNBook

September 1, 2009

351

ROBUSTNESS OF NONNEGATIVE SYSTEMS

Hence, it follows that q X ∂∆j i=1

∂xi

q X ∂∆j

νi =

i=1

∂xi



qxi −

q X ∂∆j

= q

i=1

∂xi

xi −

= q(r + 1)∆j ,

q X j=1



xj 

q X ∂∆j i=1

∂xi

 ! q X  xj  j=1

j = 1, . . . , q.

(11.21)

δij (xj − xi ) = 0,

(11.22)

Alternatively, note that q X

∆i =

i=1

q q X X

i=1 j=1,j6=i

and hence, q X ∂νj i=1

∂xi

∆i = (q − 1)∆j −

q X

i=1,i6=j

∆i = q∆j −

q X

∆i = q∆j ,

j = 1, . . . , q.

i=1

(11.23)

Thus, q X ∂∆j i=1

∂xi

νi −

q X ∂νj i=1

∂xi

∆i = qr∆j ,

j = 1, . . . , q,

(11.24)

or, equivalently, [ν, ∆] = qr∆, which implies that the vector field ∆ is homogeneoush of degree qr withi respect to the semi-Euler vector field P Pq ∂ ν(x) = − qi=1 j=1,j6=i(xj − xi ) ∂xi . Corollary 11.1. The vector field of (11.16) is homogeneous of degree k = 0 with respect to the semi-Euler vector field   q q X X ∂  . ν(x) = − (xj − xi ) ∂xi i=1

j=1,j6=i

Proof. The result is a direct consequence of Proposition 11.2 by setting r = 0. Corollary 11.2. Consider the linear nominal model (11.16) and the perturbed nonlinear model (11.17). Then every equilibrium point in {αe : α ∈ R+ } is a semistable equilibrium of (11.16) and (11.17). Furthermore, z(t) → 1q eeT z0 as t → ∞ and 1q eeT z0 is a semistable equilibrium state.

NNBook

September 1, 2009

352

CHAPTER 11

Proof. It follows from i) of Theorem 10.2 that αe, α ∈ R+ , is a semistable equilibrium of (11.16). Next, it follows from Corollary 11.1 that the right-hand side of (11.16) is homogeneous of degree k = 0 with respect to the semi-Euler vector field   q q X X ∂  . (xj − xi ) ν(x) = − ∂xi i=1

j=1,j6=i

To show that every point in {αe : α ∈ R+ } is a Lyapunov stable equilibrium of (11.17), consider the Lyapunov function candidate given by V (z − αe) = 1 2 2 kz − αek . Then it follows that V˙ (z − αe) = (z − αe)T z˙ q q X X = C(i,j) [zj − zi ] (zi − α) i=1

+

q X i=1

= − = −

q−1 X

(zi − α) q X

i=1 j=i+1

q−1 X

j=1,j6=i q X

q X

i=1 j=i+1

j=1,j6=i

δij (zj − zi )

C(i,j)[zi − zj ]2 + C(i,j)[zi − zj ]2 +

q q−1 X X

(zi − zj )δij (zj − zi )

i=1 j=i+1

q−1 X

q X

i=1 j=i+1

C(i,j) [zi − zj ]δij (zj − zi ), q

z ∈ R+ .

(11.25)

Next, since, by homogeneity of δij , δij (·) is such that limz→0 δij (z)/z = 0, it follows that for every γ > 0, there exists εij > 0 such that |δij (z)| ≤ γ|z| for all |z| < εij . Hence, q−1 X q X

i=1 j=i+1

C(i,j) [zi − zj ]δij (zj − zi ) ≤

q−1 X q X

i=1 j=i+1

γC(i,j) [zi − zj ]2 ,

|zi − zj | < εij .

(11.26)

Now, choosing γ ≤ 1, it follows from (11.25) and (11.26) that V˙ (z − αe) ≤ −

q−1 X q X

(1 − γ)C(i,j) [zi − zj ]2 ≤ 0,

i=1 j=i+1

|zi − zj | < εij ,

which establishes Lyapunov stability of the equilibrium state αe. Now, the result follows from Theorem 11.2. It is important to note that Corollary 11.2 still holds for the case where

NNBook

September 1, 2009

353

ROBUSTNESS OF NONNEGATIVE SYSTEMS

the perturbed nonlinear model has the nonlinear form z(t) ˙ = Cz(t) +

p X

gi (z(t)),

z(0) = z0 ,

i=1

t ≥ 0,

(11.27)

where, for each i ∈ P {1, . . .P , q}, gi (z) is homogeneous of degree li > 0 with q ∂ and l1 < · · · < lp . respect to ν(x) = − i=1 [ qj=1,j6=i (xj − xi )] ∂x i

As an application of Corollary 11.2, consider the nonlinear dynamical system given by x˙ 1 (t) = sinh(x2 (t) − x1 (t)), x˙ 2 (t) = sinh(x1 (t) − x2 (t)),

x1 (0) = x10 , x2 (0) = x20 .

t ≥ 0,

(11.28) (11.29)

Note that for sufficiently small x, sinh x can be approximated by x + x3 /3! + · · · + x2p−1 /(2p − 1)!, where p is a positive integer. The truncated system associated with (11.28) and (11.29) is given by 1 1 (x2 − x1 )3 + · · · + (x2 − x1 )2p−1 , 3! (2p − 1)! 1 1 x˙ 2 = x1 − x2 + (x1 − x2 )3 + · · · + (x1 − x2 )2p−1 , 3! (2p − 1)!

x˙ 1 = x2 − x1 +

or, equivalently, 

x˙ 1 x˙ 2



=



−1 1 1 −1



x1 x2



+

where gi (x1 , x2 ) ,

1 (2i + 1)!



(x2 − x1 )2i+1 (x1 − x2 )2i+1

p−1 X

(11.30) (11.31)

gi (x1 , x2 ),

(11.32)

i = 1, . . . , p − 1.

(11.33)

i=1



,

It can be easily shown that all the conditions of Corollary 11.2 hold for (11.32). Hence, it follows from Corollary 11.2 that every equilibrium point in {α[1, 1]T : α ∈ R+ } is a local semistable equilibrium of (11.30) and (11.31), which implies that the equilibrium set {α[1, 1]T : α ∈ R+ } of (11.30) and (11.31) has the same stability properties as the linear nominal system      x˙ 1 −1 1 x1 = . (11.34) x˙ 2 1 −1 x2 It should be noted that while our analysis above holds for every p, it does not imply that the exact model (11.28) and (11.29) is semistable. Note that Corollary 11.2 deals with nonlinear compartmental dynamical systems with intercompartmental connections generating an undirected graph G = (V, E, A), where A is a symmetric adjacency matrix. Next, we consider compartmental models where G is a directed graph with

NNBook

September 1, 2009

354

CHAPTER 11

nonhomogeneous higher-order perturbation terms. The following lemma is needed for the next result. Lemma 11.1. Suppose A ∈ Rq×q and Ad ∈ Rq×q satisfy   0, i = j, C(i,i) , i = j, Ad(i,j) = i, j = 1, . . . , q, A(i,j) = C(i,j) , i 6= j, 0, i 6= j, (11.35) T e = 0. Then for every A , i = 1, . . . , n , such that Assume that C d di Pn d q×q , i=1 Adi = Ad , there exist nonnegative definite matrices Qi ∈ R i = 1, . . . , nd , such that 2A +

nd X i=1

D (Qi + AT di Qi Adi ) ≤ 0.

(11.36)

Proof. The proof is identical to the proof of Lemma 10.2. Theorem 11.3. Consider the linear nominal model (11.16), where C satisfies Assumption 1 of Chapter 10 and C T e = 0, and the nonlinear perturbed model given by z˙i (t) =

q X

j=1,j6=i

C(i,j) [zj (t) − zi (t)] + zi (0) = zi0 ,

q X

j=1,j6=i

H(i,j) [σ(zj (t)) − σ(zi (t))],

i = 1, . . . , q,

t ≥ 0,

(11.37)

where σ(·) satisfies σ(0) = 0, σ : R+ → R+ is strictly increasing, and the matrix H = [H(i,j) ] satisfies H(i,j) ∈ {0, 1}, i, j = 1, . . . , q, i 6= j, HT e = 0, rank H = q − 1, H(i,j) = 0 whenever C(i,j) = 0, i, j = 1, . . . , q, i 6= j, P H(i,i) = − qk=1,k6=i H(i,k) , and H = C − L, where LT = L ∈ Rq×q . Then every equilibrium point in {αe : α ∈ R+ } is a semistable equilibrium of (11.16) and (11.37). Furthermore, z(t) → 1q eeT z0 as t → ∞ and 1q eeT z0 is a semistable equilibrium state. Proof. It follows from ii) of Theorem 10.2 that αe, α ∈ R+ , is a semistable equilibrium of (11.16). Next, note that (11.37) can be rewritten as q X z˙i (t) = H(i,j) [(zj (t) + σ(zj (t))) − (zi (t) + σ(zi (t)))] j=1,j6=i q X

+

j=1,j6=i

L(i,j) [zj (t) − zi (t)],

zi (0) = zi0 ,

i = 1, . . . , q,

t ≥ 0. (11.38)

q

q

ˆ : R+ → R+ by Note that L(i,j) ∈ {0, 1}, i, j = 1, . . . , q, i 6= j. Define σ

NNBook

September 1, 2009

355

ROBUSTNESS OF NONNEGATIVE SYSTEMS

σ ˆ (z) , [σ(z1 ), . . . , σ(zq )]T . Now, for C ∈ Rq×q and Cd ∈ Rq×q satisfying   H(i,i) , i = j, 0, i = j, i, j = 1, . . . , q, C(i,j) = Cd(i,j) = H(i,j), i 6= j, 0, i 6= j, (11.39) . . . , n , such that it follows from Lemma 11.1 that, for every C , i = 1, d di Pn d q×q , i = there exist nonnegative definite matrices Q ∈ R C = C , i d i=1 di 1, . . . , q, such that q X T D 2C + (Qi + Cdi Qi Cdi ) ≤ 0.

(11.40)

i=1

To show that every equilibrium point αe, α ∈ R+ , of (11.37) is Lyapunov stable, consider the Lyapunov function candidate given by 2

V (z − αe) = kz − αek + 2

q Z X i=1

zi α

[σ(θ) − σ(α)]dθ.

(11.41)

Now, the derivative of V (z − αe) along the trajectories of (11.37) is given by V˙ (z − αe) = 2[z − αe + σ ˆ (z) − σ ˆ (αe)]T C[z − αe + σ ˆ (z) − σ ˆ (αe)] q X ˆ (z) − σ ˆ (αe)] +2 [z − αe + σ ˆ (z) − σ ˆ (αe)]T Cdi [z − αe + σ i=1

q q X X +2 [zi − α + σ(zi ) − σ(α)] L(i,j)(zj − zi )

≤ − + −

i=1 q X i=1

[z − αe + σ ˆ (z) − σ ˆ (αe)]T Qi [z − αe + σ ˆ (z) − σ ˆ (αe)]

q X

i=1 q X i=1

j=1,j6=i

2[z − αe + σ ˆ (z) − σ ˆ (αe)]T Cdi [z − αe + σ ˆ (z) − σ ˆ (αe)]

T D [z − αe + σ ˆ (z) − σ ˆ (αe)]T Cdi Qi Cdi

·[z − αe + σ ˆ (z) − σ ˆ (αe)] −2 −2

q−1 X q X

i=1 j=i+1

q−1 X

q X

i=1 j=i+1

L(i,j) (zi − zj )[σ(zi ) − σ(zj )] L(i,j) (zi − zj )2

NNBook

September 1, 2009

356

CHAPTER 11

= −

q X i=1

(−Qi [z − αe + σ ˆ (z) − σ ˆ (αe)]

+Cdi [z − αe + σ ˆ (z) − σ ˆ (αe)])T QD i · (−Qi [z − αe + σ ˆ (z) − σ ˆ (αe)] + Cdi [z − αe + σ ˆ (z) − σ ˆ (αe)]) q X T D Qi Cdi − [z − αe + σ ˆ (z) − σ ˆ (αe)]T Cdi i=1

·[z − αe + σ ˆ (z) − σ ˆ (αe)] −2 −2 ≤ 0,

q−1 X q X

i=1 j=i+1

q−1 X q X

i=1 j=i+1 q z ∈ R+ ,

L(i,j) (zi − zj )[σ(zi ) − σ(zj )] L(i,j) (zi − zj )2 (11.42)

which establishes Lyapunov stability of αe. q

q

˜ , {x ∈ R+ : Finally, let R , {x ∈ R+ : V˙ (x) = 0} and R ˜ −Qi [x + σ ˆ (x)] + Cdi [x + σ ˆ (x)] = 0, i = 1, . . . , q}, and note that R ⊆ R. Then it follows from the Krasovskii-LaSalle invariant set theorem that x(t) → M as t → ∞, where M denotes the largest invariant set contained P ˆ , ˜ ⊆ R in C + qi=1 Qi = 0, it follows that R ⊆ R  R. q Now, since P Pq q ˆ (x) + i=1 Cdi σ x ∈ R+ : C σ ˆ (x) = 0 . Hence, since C + i=1 Cdi = H, rank H = q − 1, and He = 0, it follows that the largest invariant set ˆ contained in R ˆ is given by M ˆ = {x ∈ Rq : x = αe, α ∈ R+ }. M + ˆ ⊆ R ⊆ R, ˆ it follows that M = M. ˆ Hence, using Furthermore, since M similar arguments as in the proof of iii) ⇒ i) of Proposition 2.4, it follows that every equilibrium point in {αe : α ∈ R+ } is a semistable equilibrium of (11.16) and (11.37). Example 11.1. As an illustrative example for Theorem 11.3, consider the perturbed nonnegative system given by x˙ 1 (t) = x2 (t) − x1 (t) + x3 (t) − x1 (t) + aσ(x2 (t)) − aσ(x1 (t)), x1 (0) = x10 , t ≥ 0, x˙ 2 (t) = x3 (t) − x2 (t) + aσ(x3 (t)) − aσ(x2 (t)), x2 (0) = x20 , x˙ 3 (t) = x4 (t) − x3 (t) + x1 (t) − x3 (t) + aσ(x4 (t)) − aσ(x3 (t)), x3 (0) = x30 , x˙ 4 (t) = x5 (t) − x4 (t) + aσ(x5 (t)) − aσ(x4 (t)), x4 (0) = x40 , x˙ 5 (t) = x1 (t) − x5 (t) + aσ(x1 (t)) − aσ(x5 (t)), x5 (0) = x50 ,

(11.43) (11.44) (11.45) (11.46) (11.47)

NNBook

September 1, 2009

357

ROBUSTNESS OF NONNEGATIVE SYSTEMS

8 x1 x2

7

x

3

x

4

6

x

States

5

5

4

3

2 0

0.01

0.02

0.03

0.04

0.05

Time Figure 11.1 State trajectories versus time for (11.43)–(11.47).

where σ(x) = sgn(x)|x|α+1 , sgn(x) , x/|x| for x 6= 0, sgn(0) , 0, and α ≥ 0. Note that (11.43)–(11.47) can be rewritten in the form of (11.37) with     −2 1 1 0 0 −1 1 0 0 0  0 −1 1  0 −1 1 0 0  0 0         0 −2 1 0 , H =  0 0 −1 1 0  C= 1 ,  0  0 0 0 −1 1  0 0 −1 1  1 0 0 0 −1 1 0 0 0 −1   −1 0 1 0 0  0 0 0 0 0     L=C−H=  1 0 −1 0 0  .  0 0 0 0 0  0 0 0 0 0

Then it follows from Theorem 11.3 that every point in {(x1 , x2 , x3 , x4 , x5 ) ∈ R5 : x1 = x2 = x3 = x4 = x5 = c, c ∈ R} is a semistable equilibrium state of (11.43)–(11.47) with a > 0 and a = 0. Let [x10 , x20 , x30 , x40 , x50 ]T = [5, 3, 7, 8, 2]T , a = 6, and α = 2. Figure 11.1 shows the state trajectories versus time. △

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Twelve

Modeling and Control for Clinical Pharmacology

12.1 Introduction Control technology impacts modern medicine through robotic surgery, electrophysiological systems (pacemakers and automatic implantable defibrillators), life support (ventilators, artificial hearts), and image-guided therapy and surgery. An additional area of medicine suited for applications of control is clinical pharmacology, in which mathematical modeling plays a prominent role [155, 305]. Although numerous drugs are available for treating disease, proper dosing is often imprecise, resulting in increased costs, morbidity, and mortality. In this chapter, we discuss potential applications of control technology to clinical pharmacology, specifically the control of drug dosing [223]. We begin by considering how dosage guidelines are developed. Drug development begins with animal experimentation. Promising agents are moved to human trials, beginning with healthy volunteers and progressing to patients with the disease for which the drug is being developed. Early stages of these trials focus on safety, while the final trials usually involve administration of a placebo and different drug doses to evaluate efficacy. Efficacy is defined statistically, and aggregate therapeutic effects do not preclude the existence of individual patients for whom the drug either is not efficacious or causes side effects. If a therapeutic effect is observed, then the drug may be approved by the Food and Drug Administration. In general, the recommended dosage is the level found to be efficacious in the “average” patient. Herein lies the problem: The “average” patient does not exist. Substantial variability exists among patients in both the drug concentration at the locus of the effect (the effect-site concentration) resulting from a given dose, and in the therapeutic efficacy of a given effect-site concentration. Frequently, the appropriate dose for a specific patient is found by trial and error. For example, a physician treating a patient for

NNBook

September 1, 2009

360

CHAPTER 12

hypertension typically begins by prescribing the recommended dose, and later, by observing the effect of the drug on blood pressure, adjusts the dose empirically.

12.2 Pharmacokinetic Models Drug dosing can be made more precise by using pharmacokinetic and pharmacodynamic modeling [96]. Pharmacokinetics is the study of the concentration of drugs in tissue as a function of time and dose schedule, while pharmacodynamics is the study of the relationship between drug concentration and drug effect. By relating dose to resultant drug concentration (pharmacokinetics) and concentration to effect (pharmacodynamics), a model for drug dosing can be generated. The distribution of drugs in the body depends on transport and metabolic processes, many of which are poorly understood [96, 305]. However, compartmental models are widely used to model these processes [155]. Pharmacokinetic compartmental models typically assume that the body is comprised of more than one compartment. Within each compartment the drug concentration is assumed to be uniform due to perfect, instantaneous mixing. Transport to other compartments and elimination from the body occur by metabolic processes. For simplicity, the transport rate is often assumed to be proportional to drug concentration. Although the assumption of instantaneous mixing is an idealization, it has little effect on the accuracy of the model as long as we do not try to predict drug concentrations immediately after the initial drug dose. In a simple one-compartment model, the body is assumed to consist of a single compartment in which instantaneous mixing occurs, with subsequent elimination. It is usually assumed that elimination is linear, that is, the rate of elimination is directly proportional to the drug concentration in the compartment. This model is characterized by two parameters, namely, the compartmental volume Vd and the elimination rate constant ae . For this simple model the concentration C (in moles/volume or mass/volume) immediately after a dose of mass D is equal to D/Vd , and the drug is subsequently eliminated at the rate ae C with exponential decay. While the behavior of a few drugs can be described adequately by this model, the model is too simplistic for most drugs. Furthermore, for drugs taken orally, a model with two or more compartments is needed. One compartment represents the gastrointestinal tract, which receives the dose and transfers it to a second compartment. The second compartment represents intravascular blood (blood within arteries or veins) and other organ systems. A two-compartment mammillary model [155] can also be used for

NNBook

September 1, 2009

361

DRUG DOSING CONTROL

drugs administered intravenously. This model includes a central compartment, which receives the intravenous dose with instantaneous mixing and is typically identified with organs such as the heart, brain, liver, and kidney. These organ systems receive a large amount of blood flow per unit mass and hence are well mixed with intravascular blood. The drug is then transferred to a peripheral compartment comprised of muscle and fat or metabolized and eliminated from the body. For most drugs the enzyme systems responsible for drug metabolism are found in the liver or kidney so that metabolism in the peripheral compartment can be ignored. Drug in the peripheral compartment transfers back to the central compartment with linear kinetics.

12.3 State Space Models The two-compartment mammillary system is described by the state space model x(t) ˙ = Ax(t), where A=



x(0) = x0 ,

t ≥ 0,

−(a21 + a11 ) a12 a21 −a12



(12.1)

,

x = [x1 , x2 ]T is the state vector representing the masses in the two compartments, a12 and a21 are the nonnegative transfer coefficients from Compartment 2 to Compartment 1 and from Compartment 1 to Compartment 2, respectively, and the nonnegative coefficient a11 is the rate at which drug is eliminated from the system through the central Compartment 1. Entry (2,2) of A reflects no elimination from Compartment 2. An additional parameter is the volume V1 of the central compartment, for a total of four pharmacokinetic parameters. Note that with the assumption of instantaneous mixing, the concentration at t = 0+ after dose D is administered is D/V1 . The two-compartment mammillary model is useful for many drugs administered intravenously. The basic two-compartment model predicts that the drug concentration in the central compartment after an impulsive initial dose can be described by the sum of two terms decreasing exponentially with time. However, to fit the data, some drugs require three or more exponential terms, which motivates an extension of the two-compartment model [155]. Figure 12.1 shows an n-compartment mammillary model, which includes a central compartment that distributes drug into the interstitial spaces of several organs and tissues of the body. In most cases, the assumption of linear transfer is maintained so that the system is modeled by (12.1), where x ∈ Rn represents the system compartmental masses or system compartmental concentrations, and A ∈

NNBook

September 1, 2009

362

CHAPTER 12

3

n−1

2

n Central Compartment

Figure 12.1 n-compartment mammillary model.

Rn×n is a compartmental matrix when x represents compartmental masses, and an essentially nonnegative matrix when x represents compartmental concentrations. Hence, (12.1) describes a nonnegative or compartmental dynamical system [155]. Compartmental pharmacokinetic models, especially mammillary models, are coarse-grained oversimplifications. Consider the injection of a drug into a small peripheral vein in the hand. The drug is transported by the venous stream of flow to the right atrium and the right ventricle, binding to blood cells or proteins and mixing with venous streams as veins coalesce. Large-scale mixing occurs in the right atrium and ventricle, transporting the drug to the lung, where some of the drug can bind to tissue. From the lung, the drug returns to the left atrium and ventricle and is expelled into the aorta for transport to other inert tissues, where drug binding occurs, and to the liver and kidney, where the drug is metabolized. Modeling this process with a small number of compartments is thus an approximation.

12.4 Drug Action, Effect, and Interaction The clinical utility of pharmacokinetic models depends entirely on the time scale of the application. For example, these models work well for determining suitable dosing intervals for drugs administered orally as well as maintaining appropriate anesthetic concentrations during surgery. Using simplified mammillary models, it is possible to achieve median absolute performance errors (the normalized difference between target and measured anesthetic concentrations) of less than 20% when drug concentrations are sampled every 15 minutes. This level of performance is clinically acceptable since drug concentrations within this range of the target generally achieve the desired effect. We now consider the problem of predicting drug concentrations during

NNBook

September 1, 2009

363

DRUG DOSING CONTROL

the administration of anesthesia. Anesthesia is typically initiated by administering a bolus (impulsive) dose of a hypnotic drug intravenously. During the first few minutes after initiation of intravenous anesthesia by administration of a bolus dose, mammillary compartmental models fail to accurately predict drug concentrations because of the assumption of instantaneous mixing. More elaborate models postulate multiple compartments in series to approximate the transport of drug from the site of injection to the central circulation. Additional parallel compartments (similar to mammillary models) account for drug distribution to peripheral tissue (muscle and fat). These extensions help to describe drug concentration immediately after a bolus dose initiation of anesthesia [137]. While the most commonly used pharmacokinetic models are linear, the underlying processes that determine pharmacokinetic behavior are nonlinear. For example, the molecular processes of drug metabolism are described by Michaelis-Menten kinetics in which the rate of drug metabolism is given by Vm C/(Km +C), where Vm is the maximum rate of reaction, Km is the drug concentration that achieves 50% of the maximal effect, and C is the drug concentration. However, large-scale pharmacokinetic models assume linear drug metabolism or elimination. Similarly, most compartmental models assume that drug distribution between tissues is linear. However, delivery of drug to tissue per unit time is equal to the product of drug concentration in the blood and regional perfusion, that is, the blood flow to tissue per unit time. Regional perfusion may be nonlinearly related to the drug concentration since anesthetic drugs alter cardiovascular function.

12.5 Pharmacokinetic Parameter Estimation Data used for pharmacokinetic modeling are collected by administering a drug to patients, drawing blood samples at designated times after the initiation of dosing, and determining the concentration of the drug as a function of time, but not in real time. Consequently, most pharmacokinetic investigations focus on blood concentrations. One goal of this analysis is to derive an expression for the unit disposition function, the time-dependent blood concentration that results from a unit bolus dose. Assuming linear kinetics, if the unit disposition function fud is known then the resulting blood concentration is given by the convolution integral Z t C(t) = fud (τ )D(t − τ )dτ, (12.2) 0

where D(t) is the dose as a function of time [262]. It is not feasible to measure drug concentration in the tissue at the site of the therapeutic effect. Since drugs are distributed to the action site

NNBook

364

September 1, 2009

CHAPTER 12

by blood flow, the effect site rapidly equilibrates with blood, and it is often assumed that effect-site concentration and blood concentration are equal. If the equilibration time between the central intravascular blood volume and the effect site is clinically relevant, then the pharmacokinetic model must be revised to include an effect-site compartment distinct from the central compartment. Pharmacokinetic model parameters that comprise the system matrix A are estimated by fitting models to the data. There are numerous sources of noise in the data, from assay error to human recording error. Because of model approximation and noise, there is always an offset between the concentration predicted by the model and the observed data, namely, the prediction error. One method for estimating pharmacokinetic parameters is maximum likelihood [63]. This approach assumes a statistical distribution for the prediction error and then determines the parameter values that maximize the likelihood of the observed results. Suppose we conduct a study in a single patient from which we collect blood samples at ten different points in time after a single intravenous bolus. If we assume that the prediction error for an individual patient (intrapatient error model) has a normal or Gaussian distribution, then the likelihood L of the observed results is given by r Y 1 2 2 √ e−P Ei /2σ , (12.3) L= 2 2πσ i=1 where the prediction error P Ei of the ith observation is given by P Ei = Cp i − Cmi , where Cp i is the ith predicted drug concentration and Cmi is the ith measured drug concentration, σ 2 is the variance of the Gaussian distribution of prediction errors, and r is the number of observations (measured concentrations). The likelihood (12.3) of the observed results is a function of σ and the pharmacokinetic parameters. By maximizing (12.3) (or more commonly its logarithm) with respect to the pharmacokinetic parameters and σ, one can estimate the structural model parameters (the entries of the system matrix A) and the error model parameters (in this case, σ) that maximize the likelihood of the observed results. The above example reduces to least squares estimation when σ 2 is a constant. Use of a more sophisticated error model, in which σ 2 is proportional to a power of the predicted concentration, leads to weighted least squares estimation [63]. There are two distinct approaches to estimating mean pharmacokinetic parameters for a population of patients [269, 270]. In the first approach, models are fitted to data from individual patients, and the pharmacokinetic parameters are then averaged (two-stage analysis) to provide a measure of

NNBook

September 1, 2009

DRUG DOSING CONTROL

365

the pharmacokinetic parameters for the population. The second approach is to pool the data from individual patients, called mixed-effects modeling; in this situation the prediction error is determined by the stochastic noise of the experiment and by the fact that different patients have different pharmacokinetic parameters. The statistical model used to account for the discrepancy between observed and predicted concentrations must take into consideration not only variability between observed and predicted concentrations within the same patient (intrapatient variability), but also variability between patients (interpatient variability). Most commonly, it is assumed that the interpatient variability of pharmacokinetic parameters conforms to a log-normal distribution. This sophisticated method of analysis estimates the mean structural pharmacokinetic parameters as well as the statistical variability of these elements in the population. Analysis based on mixed-effects modeling is powerful for two reasons. First, this approach gives the clinician an estimate of both the pharmacokinetic parameters and their variance. These statistics are important for the clinician, since no matter how desirable the properties of a drug are, on average, if there is extreme variability in these parameters the drug may not be safe for clinical use. Second, mixed-effects modeling may allow a reduction in the amount of data needed from each patient. In a two-stage analysis, one must have enough data points from each patient to estimate the patient’s pharmacokinetic parameters. For example, a two-compartment mammillary model requires four pharmacokinetic parameters. With mixedeffects modeling, it is possible to estimate these parameters for any one patient with four or fewer data points.

12.6 Pharmacodynamic Models In contrast to pharmacokinetic modeling, pharmacodynamic modeling is less readily related to molecular processes. The molecular mechanism of action of many drugs is well understood in that most drugs act by binding to a receptor on or within target cells [96]. There is a well-developed theory of multiple equilibrium binding of ligands, such as drug molecules, to receptors on larger macromolecules, such as proteins. In theory, pharmacodynamics, which models the relationship between drug concentration and effect, should follow from models of molecular binding. However, the physiological effect is an interplay of numerous factors, and it is generally not possible to analytically relate the drug effect at the level of the intact organism to the number of receptors bound by the drug at the molecular level. Empirical models are thus needed. It might be assumed that drug effect is proportional to the drug concentration at the effect site, but this simple linear model is unrealistic since it admits the possibility of limitless drug effect as drug concentration increases, that is, saturation effects are ignored.

NNBook

September 1, 2009

366

CHAPTER 12

One empirical pharmacodynamic model is given by the Hill equation γ E = Emax C γ /(C γ + C50 ),

(12.4)

where E is the drug effect, Emax is the maximum drug effect, C is the drug concentration, C50 is the drug concentration associated with 50% of the maximum effect, and γ is a dimensionless parameter that determines the steepness of the concentration-effect relationship [138]. This model was developed in 1906 to describe a molecular interaction, namely, the binding of oxygen to hemoglobin. Since that time the Hill model has been applied to various phenomena that are far removed from explanations at the molecular level. A number of modifications of this model have been employed, including the case in which the drug effect is a binary, yes-orno, variable. An example of a binary variable is anesthesia, for which the patient is either responsive or not. In this case, Emax = 1, and the pharmacodynamic model becomes γ P = C γ /(C γ + C50 ),

(12.5)

where the effect is now the probability P that the patient does not respond to some noxious stimulus [204, 206]. In typical pharmacodynamic studies, drug is administered and the effect is measured at various points in time by taking a blood sample to determine the drug concentration at the time of observation of effect. The parameters of the pharmacodynamic model (Emax , C50 , γ) can then be estimated by the maximum likelihood or generalized least squares methods described above. Obviously, if drug concentrations in the blood and effect site have not equilibrated, this analysis does not apply. It should be noted that pharmacodynamic models are inherently nonlinear, in contrast to pharmacokinetic models, which are usually linear although the system may not be. However, the interplay with pharmacodynamics may also lead to nonlinear pharmacokinetics. For example, some intravenous anesthetics depress cardiac output, the volume of blood pumped by the heart per unit time. Since the basic transport processes that determine pharmacokinetic behavior depend on blood flow, administration of the drug alters its kinetics. Furthermore, since the pharmacodynamic relationship between drug concentration and depression of cardiac output is nonlinear, the pharmacokinetics of the drug are, in reality, also nonlinear.

12.7 Open-Loop Drug Dosing In addition to safety and efficacy, the Food and Drug Administration requires that drug manufacturers provide a pharmacokinetic evaluation before approval of any new drug to provide a basis for dosage guidelines.

NNBook

September 1, 2009

367

DRUG DOSING CONTROL

The concentration that results from drug administration is determined by the transport processes that distribute the drugs to various tissues as well as the metabolic processes that transform the drug. However, the vast majority of drugs are given for chronic conditions, and when the time scale of treatment greatly exceeds the time scale of the distributive processes, there will be equilibration of drug through the various tissues. The pharmacokinetics of most drugs given chronically are described by (12.1), where A is a scalar. Drug calculations are now greatly simplified. For example, consider an antihypertensive drug with a half-life, the time needed for the drug concentration to decrease by 50% after discontinuation of administration, of 12 hours. If a dose of 50 mg is efficacious in the average patient, then a suitable dosing schedule would be an initial dose of 50 mg with subsequent dosing of 25 mg every 12 hours. As another example, suppose that a blood concentration of an intravenous anesthetic of 100 µg/ml reliably produces unconsciousness and the clearance (the effective volume of blood cleared of drug per unit time) is 150 ml/min. An infusion rate of 100 µg/ml × 150 ml/min = 15,000 µg/min maintains the desired blood concentration, although this concentration is achieved only when distributive processes have equilibrated. Many of the dosing guidelines recommended by the manufacturers of drugs are based on simple calculations like these. There have been attempts to develop more precise open-loop control in acute care, especially in anesthetic pharmacology. Since the appearance in the 1980s of computers in the operating room, investigators have developed computer-controlled pump systems that continually adjust the drug infusion rate to achieve and maintain the drug concentration desired by the clinician [3, 14, 267, 268]. These systems use the pharmacokinetic model x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0 ,

t ≥ 0,

(12.6)

to calculate the dose u(t) needed to achieve and maintain the target drug concentration. To implement this approach it is necessary to know the pharmacokinetic parameters that define A and B. In open-loop control, specific pharmacokinetic parameters are not known for the individual patient. Instead, it is assumed that average parameter values, taken from the pharmacokinetic literature, are applicable to the individual patient. The dose calculated using these average parameter values is then delivered by a pump controlled by the computer. Despite the obvious fact that these systems ignore interpatient pharmacokinetic variability, studies have demonstrated that drug concentrations are better maintained in therapeutic ranges by open-loop control than with standard clinical practice.

NNBook

September 1, 2009

368

CHAPTER 12

12.8 Closed-Loop Drug Dosing While initial dosing guidelines are often based on the average patient, the significant interpatient pharmacokinetic and pharmacodynamic variability observed for most drugs suggests that precise drug dosing requires closedloop control. Most patients, especially those treated for chronic disease, are familiar with this closed-loop process. The physician prescribes an initial dose of drug, observes the response, and adjusts the dose. Although some physicians are adept at this process, it is usually time consuming, and the efficiency of the process depends on the experience of the clinician since there are not enough data available for many drugs to develop quantitative guidelines. The process of dose titration can be made more precise by using mixed-effects pharmacokinetic modeling and post hoc Bayesian estimation of individual patient pharmacokinetic parameters [63, 269, 270]. Recall that mixed-effects modeling provides not only estimates of pharmacokinetic parameters but also their variance within the population. Suppose one or more drug concentrations are measured in an individual patient. Using Bayesian probability principles, the likelihood of a given value of a pharmacokinetic parameter θ is proportional to P (C|θ)P (θ), where P (C|θ) is the probability of the observed concentration C as a function of θ. Furthermore, P (θ) is the a priori probability of a given value of θ, which is given by the assumed distribution of θ (as noted above, usually log-normal) with the variance of θ estimated from the mixed-effects analysis. By determining the mode of P (C|θ)P (θ) with respect to θ, that is, the value of θ at which P (C|θ)P (θ) is maximized, one can derive a maximum likelihood estimate of θ for the specific patient. The patient-specific parameter estimate can be used to calculate the dose needed to achieve a given drug concentration [212]. However, because of pharmacodynamic variability, more precise control of drug concentration does not necessarily lead to better control of drug effect. While the process of titrating drug dose to the desired effect may be acceptable for chronic outpatient therapy, in the acute care environment, such as the operating room or the intensive care unit, this process is often either dangerously slow or imprecise. Feedback control of drug effect, in contrast with drug concentration, has much to offer modern medicine. The remainder of this chapter focuses on drugs used in the acute care setting. To implement closed-loop control in an acute care environment, realtime measurement of drug effect is needed. Early attempts at closed-loop control focused on regulating variables that are conveniently measured.

NNBook

September 1, 2009

DRUG DOSING CONTROL

369

By their very nature, cardiovascular and central nervous system functions are critical in the acute care environment, and thus technologies have evolved for their measurement. The primary applications of closed-loop drug administration are hemodynamic management and control of consciousness. Next, we review closed-loop control of the cardiovascular system, which illustrates problems inherent in the application of control technology to physiological function.

12.9 Closed-Loop Control of Cardiovascular Function A major side effect of cardiac surgery is that patients can become hypertensive [196], requiring treatment to prevent cardiac dysfunction, pulmonary edema, myocardial ischemia, stroke, and bleeding from fragile sutures. Although drugs are available for treating postoperative hypertension, titration of these drugs to regulate blood pressure is often difficult. Underdosing leaves the patient hypertensive, whereas overdosing can reduce the blood pressure to levels associated with shock. There has been interest since the late 1970s in developing controllers for administering sodium nitroprusside (SNP), a commonly used and potent antihypertensive. The problems encountered in hemodynamic control are enlightening. Initial attempts used simple nonadaptive methods such as PD or PID controllers, which assume a linear relationship between infusion rate and effect [271, 277]. However, while the drug concentration is given by the convolution of the infusion rate and a transfer function as in (12.2), the relationship between effect and infusion rate is nonlinear as in (12.4). Also, a significant challenge to the design of a blood pressure controller is the time delay between administering the drug and the clinical effect, which can lead to system oscillations. Although early blood pressure controllers included time delays in the system model, the delays were assumed to be the same for each patient [271, 277]. While early controllers were successful in some patients, these techniques have not had wide clinical implementation due to nonlinear patient response and differences in drug sensitivity among patients. Interpatient variability, as well as a patient’s sensitivity to drugs, motivated the development of single-model and multiple-model adaptive controllers [9,135]. Single-model adaptive controllers are based on on-line estimation of system parameters using minimum variance or least squares methodologies. These controllers perform poorly due to large-amplitude transients [9]. Multiple-model adaptive controllers represent the system by means of a finite number of models. For each model there is a separate controller. The probability that the system is represented by each of the different models is

NNBook

370

September 1, 2009

CHAPTER 12

calculated from the relative offsets of the system response and the response predicted by each model. The output of the controller is the probabilityweighted sum of the outputs from each model [247, 248]. Multiple-model adaptive controllers have proven to be more satisfactory than single-model adaptive controllers and fixed-gain controllers [247]. Subsequent refinements to blood pressure control include single-model, reference adaptive control [240], which appears promising in simulations, and neural-network-based methods [56]. There is also interest in optimal control since SNP has toxic side effects when the dose is too high [18]. These investigations into controlling blood pressure reveal the challenges inherent to biological systems, specifically, nonlinearity, interpatient variability (system uncertainty), and time delay. Despite the refinements of closed-loop blood pressure controllers, such controllers are seldom used clinically. Although blood pressure control is important, cardiovascular function involves several other important variables, all of which are interrelated [196]. The intensive care unit clinician must ensure not only that blood pressure is within appropriate limits but also that cardiac output (the amount of blood pumped by the heart per minute) is acceptable and that the heart rate is within reasonable limits. Mean arterial blood pressure is proportional to cardiac output, with the proportionality constant denoting the systemic vascular resistance, in analogy with Ohm’s law. Cardiac output is equal to the product of heart rate and stroke volume, the volume of blood pumped with each beat of the heart. Stroke volume, in turn, is a function of contractility, the intrinsic strength of the cardiac contraction; preload, the volume of blood in the heart at the beginning of the contraction; and afterload, the impedance to ejection by the heart. The intensive care unit clinician must balance all of these variables. Inotropic agent drugs, that is, drugs that increase the strength of contraction of the heart, also have variable effects on heart rate and afterload. There are also vasopressor drugs, which increase afterload, and vasodilator drugs, which decrease afterload. Finally, stroke volume can be improved by giving the patient intravenous fluids and increasing preload. However, too much fluid can potentially be deleterious by impairing pulmonary function as fluid builds up in the lungs. The fact that closed-loop control of blood pressure has not been widely adopted by clinicians is not surprising when one considers the complex interrelationships among hemodynamic variables. Future applications of control theory in the form of adaptive and robust optimal controllers that control the administration of multiple drugs (inotropes, vasopressors, vasodilators) and fluids will be a major advance in critical care medicine. Preliminary investigation of the control of multiple hemodynamic drugs [136, 312] has already begun.

NNBook

September 1, 2009

DRUG DOSING CONTROL

371

12.10 Closed-Loop Control of Anesthesia Anesthesia involves several components, namely, analgesia, lack of reflex response, such as increased blood pressure or heart rate, to surgical stimulus, areflexia, lack of movement (which simplifies the task of the surgeon), and hypnosis, or lack of consciousness. Closed-loop control of anesthesia may be implemented by either controlling these components of anesthesia or, more simply, controlling the anesthetic concentration and assuming that the appropriate concentration will lead to the desired effect. Real-time spectroscopic methods for measuring the concentration of inhaled anesthetic agent in end-tidal (that is, exhaled) gases are now routinely available in most operating rooms. End-tidal anesthetic gas concentration is a reasonable surrogate for arterial blood anesthetic concentration [75]. End-tidal anesthetic agent concentrations can be measured in real time, which facilitates closed-loop control of end-tidal anesthetic concentration. However, anesthetic concentration cannot be equated with anesthetic effect. More recently, real-time processed electroencephalograph (EEG) measurement has offered the possibility of closed-loop control of anesthetic effect. It has been known for decades that the induction of anesthesia causes changes in the EEG [246]. In the last decade, there has been substantial progress in developing processed EEG monitors to measure the depth of anesthesia to provide performance variables for closed-loop controllers [246]. Inhaled anesthetic agents have been the mainstay of clinical practice since the first delivery of anesthesia. A fundamental characteristic of every inhaled anesthetic agent is its “MAC” value, for minimum (in suppressing the response to painful stimuli) alveolar (alveoli are the functional units of the lung) concentration, which is associated with a 50% probability of patient movement in response to surgical stimulus [312]. By maintaining end-tidal concentrations well above MAC, the practitioner is relatively assured of hypnosis. The ready availability of spectroscopic systems for measuring end-tidal anesthetic concentration in real time has led several investigators to develop closed-loop controllers. The earliest anesthesia controllers use fixed-gain PID controllers [253, 256], which assume that all patients are the same. In contrast, the adaptive model-based controllers in [160, 296] rely on least squares methods to estimate the patient-specific system parameters. In animal studies [248], the adaptive controllers have performed more effectively than the fixed-gain controllers. However, the adaptive controllers have not been widely adopted clinically since control of anesthetic concentration does not translate into control of anesthetic effect due to interpatient pharmacodynamic variability.

NNBook

September 1, 2009

372

CHAPTER 12

12.11 Electroencephalograph-Based Control Closed-loop control of anesthesia requires a monitor of anesthetic effect, specifically consciousness, which has been an elusive challenge for anesthesiologists. The EEG, which measures electrical activity in the brain, has been an obvious candidate. In particular, neurophysiologists have observed that the EEG of an anesthetized patient contains slower waves with higher amplitudes. However, the EEG is comprised of multiple time series and multiple spectra, and, while anesthesia induces characteristic changes in the EEG, it is not clear which, if any, characteristic of the EEG best reflects the anesthetic state. EEG-based closed-loop control of anesthesia was first proposed in [36]. Subsequently, a closed-loop model-based adaptive controller was developed and clinically tested in [263] for delivering intravenous anesthesia using the median frequency of the EEG power spectrum as the regulated variable. The model used in [263] assumes a two-compartment pharmacokinetic model for which the drug concentration C(t) after a single bolus dose is given by C(t) = Ae−αt + Be−βt ,

t ≥ 0,

(12.7)

where A, B, α, and β are patient-specific pharmacokinetic parameters. It is also assumed that the median EEG frequency E is related to the drug concentration by the modified Hill equation γ E = E0 − Emax [C γ /(C γ + C50 )],

(12.8)

where E0 is the baseline signal, Emax is the maximum decrease in signal with increasing drug concentration, C50 is the drug concentration associated with 50% of the maximum effect, and the parameter γ describes the steepness of the concentration-effect curve. From (12.8) it can be seen that the drug effect is a function of the pharmacokinetic parameters A, B, α, and β as well as the pharmacodynamic parameters E0 , Emax , C50 , and γ. If these parameters are known, it is straightforward to calculate the dose regimen needed to achieve the target EEG signal. However, these parameters are not known for individual patients, and interpatient variability may be significant. Estimates for the coefficients of variability for some parameters are as high as 100%. The algorithm in [263] assumes that the pharmacodynamic parameters E0 , Emax , C50 , and γ as well as the pharmacokinetic parameters α and β are equal to the mean values reported in prior studies. Using the mean values of the pharmacokinetic parameters A and B from prior studies as starting values, estimates of these parameters are refined by analyzing the difference ∆E between the target and observed EEG signal. Linearizing ∆E with

NNBook

September 1, 2009

DRUG DOSING CONTROL

373

respect to A and B yields ∆E = (∂E/∂A)δA + (∂E/∂B)δB,

(12.9)

where δA and δB represent the updates to the values of A and B in the adaptive control algorithm. In conjunction with minimizing (δA)2 + (δB)2 , (12.9) is used to estimate δA and δB. This algorithm is only partially adaptive in that A and B are the only parameters of the model that are updated. This algorithm was implemented for the intravenous anesthetic agents methohexital and propofol, but did not appear to offer great advantage over standard manual control [263, 264]. The observed performance might have been due to the approximations of the algorithm or the deficiencies of the median EEG frequency as a measure of the depth of anesthesia.

12.12 Bispectral Index-Based Control Since the work of [263], alternative EEG measures of depth of anesthesia have been developed. Possibly the most notable of these measures is the bispectral index or BIS [97,266]. The BIS is a single composite EEG measure, which appears to be closely related to the level of consciousness. The BIS signal is related to drug concentration by the empirical relationship   cγ (12.10) BIS(ceff ) = BIS0 1 − γ eff γ , ceff + EC50 where BIS0 denotes the baseline (awake state) value, which, by convention, is typically assigned a value of 100, ceff is the drug concentration in µg/ml in the effect-site compartment (brain), EC50 is the concentration at half maximal effect and represents the patient’s sensitivity to the drug, and γ determines the degree of nonlinearity in (12.10). Here, the effect-site compartment is introduced to account for finite equilibration time between the central compartment concentration and the central nervous system concentration [261]. In [283], closed-loop control is used to deliver the intravenous anesthetic propofol based on a model-based adaptive algorithm with the BIS as the regulated variable. The algorithm in [283] is based on a pharmacokinetic model that predicts the drug concentration as a function of infusion rate and time, and uses a pharmacodynamic model that relates the BIS signal to concentration. In contrast to [263], it is assumed in [283] that the pharmacokinetic parameters are always correct and that differences in individual patient response are due to pharmacodynamic variability. Moreover, the approach of [283] predicts the anesthetic concentration using the pharmacokinetic

NNBook

September 1, 2009

374

CHAPTER 12

model and then constructs a BIS-concentration curve using the observed BIS during induction and the predicted propofol concentration. During each time epoch, the difference between the target BIS signal and the observed BIS signal is used to update the pharmacodynamic parameters relating concentration and BIS signal for the individual patient. However, this algorithm does not update the pharmacokinetic parameters. The results in [283] demonstrate excellent performance as measured by the difference between the target and observed BIS signals. However, as pointed out in [99], the performance of the model-based adaptive controller may reflect the fact that the patient was not fully stressed. In [283] a high dose of the opioid remifentanil, a neurotransmitter inhibitor resulting in significant analgesic effect, was administered in conjunction with propofol. Consequently, central nervous system excitation due to surgical stimulus was blunted, and thus the need to adjust the propofol dose as surgical stimulus varied was diminished. It is unknown whether the control system would have been effective in the absence of deep narcotization. In contrast to the model-based adaptive controllers in [263,264,283], a PID controller using the BIS signal as the variable to control the infusion of propofol is considered in [1]. The median absolute performance error, that is, the median value of the absolute value of ∆E/Etarget , was good (8.0%), although in 3 of the 10 patients, oscillations of the BIS signal around the setpoint were observed, and anesthesia was deemed clinically inadequate in 1 of the 10 patients. The same system was used in [173] with an auditory evoked potential based on somatosensory information provided by auditory stimulation generating oscillations within the EEG signal as the regulated performance variable. Intravenous propofol anesthesia was delivered in [1] by means of a fuzzy logic closed-loop controller that uses both auditory (constant-frequency, constant-amplitude signal delivered by earphones) evoked responses and cardiovascular responses as the regulated variables. This system has had only minimal clinical testing [200]. More recently, [93] considers model-based controllers for inhalation anesthetic agents that attempt to control the BIS signal or mean arterial blood pressure, while keeping end-tidal anesthetic concentrations within prespecified limits.

12.13 Pharmacokinetic and Pharmacodynamic Models for Drug Distribution Almost all anesthetics are myocardial depressants, that is, they decrease the strength of the contraction of the heart and lower cardiac output. As a consequence, decreased cardiac output slows down the transfer of

NNBook

September 1, 2009

375

DRUG DOSING CONTROL

u ≡ Continuous infusion a12 (c)x2

a31 (c)x1 Intravascular Blood

Muscle a21 (c)x1

Fat a13 (c)x3

a11 (c)x1 ≡ Elimination (liver, kidney) Figure 12.2 Pharmacokinetic model for drug distribution during anesthesia.

blood from the central compartments comprising the heart, brain, kidney, and liver to the peripheral compartments of muscle and fat. In addition, decreased cardiac output can increase drug concentrations in the central compartments, compounding side effects. This instability can lead to overdosing which, at the very least, can delay recovery from anesthesia and, in the worst case, can result in respiratory and cardiovascular collapse. Alternatively, underdosing can cause psychological trauma from awareness and pain during surgery. Control of drug effect is clinically important since overdosing or underdosing incurs risk for the patient. To illustrate a pharmacokinetic model for drug distribution during anesthesia we consider a hypothetical model for the intravenous anesthetic propofol. The pharmacokinetics of propofol are described by the three-compartment model [121, 215] shown in Figure 12.2, where x1 denotes the mass of drug in the central compartment, which is the site for drug administration and includes the intravascular blood volume as well as highly perfused organs, that is, organs with high ratios of blood flow to weight such as the heart, brain, kidney, and liver, which receive a large fraction of the cardiac output. The remainder of the drug in the body is assumed to reside in two peripheral compartments, one identified with muscle and one with fat; the masses in these compartments are denoted by x2 and x3 , respectively. These compartments receive less than 20% of the cardiac output. A mass balance for the three-state compartmental model is of the form x˙ 1 (t) = −[a11 (c(t)) + a21 (c(t)) + a31 (c(t))]x1 (t) + a12 (c(t))x2 (t) +a13 (c(t))x3 (t) + u(t), x1 (0) = x10 , t ≥ 0, (12.11) x˙ 2 (t) = a21 (c(t))x1 (t) − a12 (c(t))x2 (t), x2 (0) = x20 , (12.12) x˙ 3 (t) = a31 (c(t))x1 (t) − a13 (c(t))x3 (t), x3 (0) = x30 , (12.13) where c(t) = x1 (t)/Vc , Vc is the volume of the central compartment (about 15 l for a 70 kg patient), aij (c) for i 6= j is the rate of transfer of drug

NNBook

September 1, 2009

376

CHAPTER 12

from the jth compartment to the ith compartment, a11 (c) is the rate of drug metabolism and elimination (metabolism typically occurs in the liver), and u(t) is the infusion rate of the anesthetic drug propofol into the central compartment. The transfer coefficients are assumed to be functions of the drug concentration c, since it is well known that the pharmacokinetics of propofol are influenced by cardiac output [289] and, in turn, cardiac output is influenced by propofol plasma concentrations, both due to venodilation (pooling of blood in dilated vains) [226] and myocardial depression [154]. Experimental data indicate that the transfer coefficients aij are nonincreasing functions of the propofol concentration [154, 226]. The most widely used empirical models for pharmacodynamic concentration-effect relationships are modifications of the Hill equation (12.4). Application of the Hill equation to the relationship between transfer coefficients and drug concentration implies that aij (c) = Aij Qij (c),

α

α

ij ij Qij (c) = Q0 C50,ij /(C50,ij + cαij ),

(12.14)

where, for distinct i, j ∈ {1, 2, 3}, C50,ij is the drug concentration associated with a 50% decrease in the transfer coefficient, αij is a parameter that determines the steepness of the concentration-effect relationship, and Aij are positive constants. Note that αij and Aij are functions of i and j, that is, there are distinct Hill equations for each transfer coefficient. Furthermore, since for many drugs the rate of metabolism a11 (c) is proportional to the rate of drug transport to the liver, we assume that a11 (c) is proportional to the cardiac output so that a11 (c) = A11 Q11 (c). Although propofol concentration in the blood is correlated with lack of responsiveness [168], the concentration cannot be measured in real time during surgery. Since we are more interested in drug effect (depth of hypnosis) than drug concentration, we consider a model involving pharmacokinetics and pharmacodynamics for controlling consciousness. We use an EEG signal, specifically the BIS signal, to access the effect of anesthetic compounds on the brain [97, 224, 273]. Furthermore, we utilize the modified Hill equation (12.10) to model the relationship between the BIS signal and the effect-site concentration. The effect-site compartment concentration is related to the concentration in the central compartment by the first-order model c˙eff (t) = aeff (c(t) − ceff (t)),

ceff (0) = c(0),

t ≥ 0,

where aeff in min−1 is a time constant. In reality, the effect-site compartment equilibrates with the central compartment in a few minutes. The parameters aeff , EC50 , and γ are determined by data fitting and vary from patient to patient. BIS index values of 0 and 100 correspond, respectively, to an isoelec-

NNBook

September 1, 2009

377

DRUG DOSING CONTROL

u ≡ Continuous infusion a12 (c)x2 Muscle a21 (c)x1

a31 (c)x1

Intravascular Blood

Fat a13 (c)x3

aeff c a11 (c)x1 ≡ Elimination (liver, kidney)

Effect Site Compartment ceff

BIS index

Pharmacodynamics

Figure 12.3 Combined pharmacokinetic and pharmacodynamic model with effect-site (brain) compartment.

tric EEG signal (no cerebral electrical activity) and an EEG signal of a fully conscious patient; the range 40 to 60 indicates a moderate hypnotic state [273]. Figure 12.3 shows the combined pharmacokinetic/pharmacodynamic model for the distribution of propofol.

12.14 Challenges and Opportunities in Pharmacological Control Closed-loop control for clinical pharmacology is in its infancy, with numerous challenges and opportunities ahead. An implicit assumption inherent in the control frameworks discussed in this chapter is that the control law is implemented without regard to actuator amplitude and rate constraints. In pharmacological applications, drug infusion rates vary from patient to patient, and, to avoid overdosing, it is vital that the infusion rate does not exceed the patient-specific threshold values. As a consequence, actuator constraints, that is, infusion pump rate constraints, need to be implemented in drug delivery systems [151]. Another important issue is measurement noise. In particular, EEG signals can have as much as 20% variation due to noise. For example, the BIS signal may be corrupted by electromyographic noise, that is, signals emanating from muscle rather than the central nervous system. Although electromyographic noise can be minimized by muscle paralysis, there are other sources of measurement noise, such as electrocautery, that need to be accounted for in the control design processes.

NNBook

378

September 1, 2009

CHAPTER 12

In pharmacokinetic and pharmacodynamic models the assumption of instantaneous mixing between compartments is not valid. For example, if a bolus of drug is injected, there is a time lag before the drug is detected in the extracellular and intercellular space of an organ [155]. Phase lag due to mixing times can be approximated by including additional compartments in series. To describe the distribution of pharmacological agents in the human body, information on the past system states can be modeled by delay dynamical systems [119]. This extension necessitates the development of adaptive control algorithms for compartmental systems with unknown time delays [53, 151]. Optimal and adaptive feedback controllers for nonnegative and compartmental dynamical systems with specific applications to drug dosing control for clinical pharmacology are discussed in the next several chapters.

NNBook

September 1, 2009

Chapter Thirteen

Optimal Fixed-Structure Control for Nonnegative Systems

13.1 Introduction In this chapter, we develop optimal output feedback controllers for set-point regulation of linear nonnegative and compartmental dynamical systems. In particular, we extend the optimal fixed-structure control framework [28, 30] to develop optimal output feedback controllers that guarantee that the trajectories of the closed-loop plant system states remain in the nonnegative orthant of the state space for nonnegative initial conditions. The proposed optimal fixed-structure control framework is a constrained optimal control methodology that does not seek to optimize a performance measure per se, but rather seeks to optimize performance within a class of fixedstructure controllers satisfying internal controller constraints that guarantee the nonnegativity of the closed-loop plant system states. Furthermore, since unconstrained optimal controllers are globally optimal but may not guarantee nonnegativity of the closed-loop plant system states, we additionally characterize domains of attraction contained in the nonnegative orthant for unconstrained optimal output feedback controllers [195] that guarantee nonnegativity of the closed-loop plant system trajectories. Specifically, domains of attraction contained in the nonnegative orthant for optimal output feedback controllers are computed using closed and open Lyapunov level surfaces [124]. It is also shown that the domains of attraction predicated on open Lyapunov level surfaces provide a considerably improved region of asymptotic stability in the nonnegative orthant as compared to regions of attraction given by closed Lyapunov level surfaces.

13.2 Optimal Zero Set-Point Regulation for Nonnegative Dynamical Systems In this section, we consider the problem of optimal output feedback zero setpoint stabilization of linear nonnegative dynamical systems. As discussed in

NNBook

September 1, 2009

380

CHAPTER 13

Chapter 1, it follows from physical considerations that the state trajectories of nonnegative dynamical systems remain in the nonnegative orthant of the state space for nonnegative initial conditions. Even though in certain applications of nonnegative systems, such as active control of drug delivery systems for clinical pharmacology, we additionally require control (source) inputs to be nonnegative, in many applications of nonnegative systems the positivity constraint on the control input is not natural. Hence, in the first part of this chapter we do not place any restriction on the sign of the control signal and design optimal output feedback controllers that guarantee that the system states remain in the nonnegative orthant and converge to a desired equilibrium state. In this chapter, we consider controlled dynamical systems of the form x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0 ,

t ≥ 0,

(13.1)

where x(t) ∈ Rn , t ≥ 0, u(t) ∈ Rm , t ≥ 0, A ∈ Rn×n , and B ∈ Rn×m . It follows from Proposition 4.1 that if A is essentially nonnegative, then the control input signal Bu(t), t ≥ 0, needs to be nonnegative to guarantee the nonnegativity of the state of (13.1). This is due to the fact that when the initial state of (13.1) belongs to the boundary of the nonnegative orthant, a negative input can destroy the nonnegativity of the state of (13.1). Alternatively, however, if the initial state is in the interior of the nonnegative orthant, then it follows from continuity of solutions with respect to system initial conditions that, over a sufficiently small interval of time, nonnegativity of the state of (13.1) is guaranteed irrespective of the sign of each component of the control input Bu(t) over this time interval. However, unlike open-loop control wherein lack of coordination between the input and the state necessitates nonnegativity of the control input, a feedback control signal predicated on the system state variables allows for the anticipation of loss of nonnegativity of the state. Hence, state feedback control signals can take negative values while ensuring nonnegativity of the system states. For further discussion of the above fact see [193]. Since stabilization of nonnegative systems naturally deals with equin librium points in the interior of the nonnegative orthant R+ , the following proposition provides necessary conditions for the existence of an interior equilibrium point xe ∈ Rn+ of (13.1) in terms of the stability properties of the system dynamics matrix A. Proposition 13.1. Consider the nonnegative dynamical system (13.1) m and assume there exist xe ∈ Rn+ and ve ∈ R+ such that 0 = Axe + Bve . Then, A is semistable.

(13.2)

NNBook

September 1, 2009

381

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

Proof. The proof is a direct consequence of Theorem 2.11 with A replaced by AT , p = xe , and r = Bve . It follows from Proposition 13.1 that the existence of an equilibrium point xe ∈ Rn+ for (13.1) implies that the system matrix A is semistable. m Hence, if (13.2) holds for xe ∈ Rn+ and ve ∈ R+ , A is asymptotically stable or 0 ∈ spec(A) is a simple eigenvalue of A and all other eigenvalues of A have negative real parts since −A is an M-matrix [21]. In light of the above constraints, it was shown in [193] using Brockett’s necessary condition for asymptotic stabilizability [42] that if 0 ∈ spec(A), then there does not exist a continuous stabilizing nonnegative feedback for set-point regulation in Rn+ for a nonnegative system. However, that is not to say that asymptotic feedback regulation using discontinuous feedback is not possible. Consider the controlled linear dynamical system given by x(t) ˙ = Ax(t) + Bu(t), y(t) = Cx(t),

x(0) = x0 ,

t ≥ 0,

(13.3) (13.4)

where x(t) ∈ Rn , t ≥ 0, is the state vector, u(t) ∈ Rm , t ≥ 0, is the control input, y(t) ∈ Rm , t ≥ 0, is the measurable output, A ∈ Rn×n is essentially nonnegative, B ∈ Rn×m and is assumed to be in the form   Bs B= , (13.5) 0(n−m)×m where Bs ∈ Rm×m is invertible, C = [Cs , 0m×(n−m) ], where Cs , diag[c1 , . . . , cm ], and the pair (A, B) is stabilizable. Alternatively, we can assume that Cs is invertible and not necessarily diagonal. The dynamical system structure (13.3)–(13.5) is natural for controlled compartmental dynamical systems wherein control inputs correspond to control inflows to subsystem compartments. For example, such system structures arise in pharmacokinetic and pharmacodynamic systems. For details, see Section 13.7. The control input u(·) in (13.3) is restricted to the class of admissible controls consisting of measurable functions such that u(t) ∈ Rm , t ≥ 0. Here, we assume that the control input is given by1 u(t) = Bs−1 Ky(t),

(13.6)

where K ∈ Rm×m and K , diag[k1 , . . . , km ]. Hence, the closed-loop system has the form ˜ x(t) ˙ = Ax(t), x(0) = x0 , t ≥ 0, (13.7) 1 In the case where C is invertible but not necessarily diagonal, the control input (13.6) is given s by u(t) = Bs−1 KCs−1 y(t).

NNBook

September 1, 2009

382

CHAPTER 13

where A˜ , A + BBs−1 KC. Clearly A˜ is essentially nonnegative, and hence, it follows from Proposition 2.5 that the trajectory x(t), t ≥ 0, of the closed-loop system (13.7) remains in the nonnegative orthant for all n initial conditions x0 ∈ R+ . Next, consider the performance functional Z ∞ J(x0 , u) = [x(t)T R1 x(t) + u(t)T R2 u(t)]dt,

(13.8)

0

n

where x0 ∈ R+ , and R1 ∈ Rn×n and R2 ∈ Rm×m are such that R1 ≥ 0 and R2 > 0. Here, we seek to determine feedback gain matrices K ∈ Rm×m such that the control law (13.6) stabilizes the closed-loop system (13.7), that is, the closed-loop system matrix A˜ is asymptotically stable, and minimizes (13.8). To eliminate the explicit dependence of the cost functional on the initial conditions x0 we average the performance obtained for a linearly independent set of initial conditions. In this case, the performance functional (13.8) can be rewritten as J(K) = tr P V,

(13.9)

where V , E [x0 xT 0 ] and E denotes expectation, and P ≥ 0 is the unique solution to the Lyapunov equation ˜ 0 = A˜T P + P A˜ + R,

(13.10)

where ˜ , R1 + C T K T B −T R2 B −1 KC. R s s As in the standard H2 -optimal control problem, the necessary conditions for optimality can now be derived by forming the Lagrangian ˜ L(P, Q, K) = tr{P V + Q[A˜T P + P A˜ + R]},

(13.11)

where Q ∈ Rn×n is a Lagrange multiplier. In particular, representing the gain matrix K ∈ Rm×m in the form K=

m X

ki E(i,i) ,

(13.12)

i=1

where E(i,i) ∈ Rm×m is an elementary matrix with unity in the (i, i) entry and zeros elsewhere, the gradient expressions of L(P, Q, K) with respect to the free parameters are given by ∂L ˜ + QA˜T + V, = AQ ∂P ∂L ˜ = A˜T P + P A˜ + R, ∂Q

(13.13) (13.14)

NNBook

September 1, 2009

383

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS m

X 1 ∂L T = tr CQP BBs−1E(i,i) + kj tr CQC T E(i,i) Bs−T R2 Bs−1 E(j,j) , 2 ∂ki j=1

i = 1, . . . , m.

(13.15)

The necessary conditions for optimality can now be obtained by equating the corresponding partial derivatives with zero, to yield 2n2 + m equations with 2n2 + m unknowns. This system of nonlinear algebraic equations can be solved numerically using a quasi-Newton gradient optimization method [59, 66]. The numerical procedure includes a constraintchecking subroutine which ensures that at each step of iteration the searched parameters yield a stable closed-loop system. Finally, the optimal gain matrix K ∈ Rm×m guarantees an asymptotically stable, essentially nonnegative closed-loop system where the performance functional (13.8) is minimized over the class of all fixed-structure controllers given by (13.6) with K = diag[k1 , . . . , km ]. Furthermore, it follows from Proposition 2.5 that the state trajectories of the closed-loop system (13.7) remain in the n nonnegative orthant of the state space for all x0 ∈ R+ . Note that if K ∈ Rm×m in (13.6) is constrained to be essentially nonnegative, then A˜ in (13.7) is essentially nonnegative, and hence, the trajectory x(t), t ≥ 0, of the closed-loop system (13.7) remains in the n nonnegative orthant for all initial conditions x0 ∈ R+ . In this case, m×m as K = {K}+ < K >, where representing Pm the gain matrix K ∈ R {K} = i=1 kii E(i,i) is the diagonal portion of K and < K > is the off2 leads to a fixed-structure diagonal portion of K, and enforcing (i,j)= kij optimal control problem that guarantees the essential nonnegativity of K ˜ and, hence, of A.

13.3 Optimal Nonzero Set-Point Regulation for Nonnegative Dynamical Systems Since stabilization of nonnegative systems typically involves stabilization of equilibrium points in the interior of the nonnegative orthant, in this section we consider the problem of optimal output feedback nonzero setpoint stabilization of an equilibrium point xe ∈ Rn+ . Specifically, we consider the controlled linear dynamical system x(t) ˙ = Ax(t) + Bu(t), y(t) = Cx(t),

x(0) = x0 ,

t ≥ 0,

(13.16) (13.17)

n

where x(t) ∈ Rn , t ≥ 0, x0 ∈ R+ , A ∈ Rn×n , B ∈ Rn×m , C ∈ Rl×n , and the pair (A, B) is stabilizable with the control input u(t) = K(y(t) − Cxe ) + ve ,

(13.18)

NNBook

September 1, 2009

384

CHAPTER 13

where K ∈ Rm×l and ve ∈ Rm is a constant vector. Here, it is important to note that we do not assume that A is essentially nonnegative nor do we assume any internal structure for B and C. Now, xe ∈ Rn+ is an equilibrium point for the closed-loop system (13.16), (13.17), and (13.18) if and only if there exists ve ∈ Rm such that 0 = Axe + Bve .

(13.19)

Hence, the closed-loop system is given by x(t) ˙ = (A + BKC)(x(t) − xe ),

x(0) = x0 ,

t ≥ 0.

(13.20)

Next, consider the performance functional Z ∞ J(x0 , u) = [(x(t) − xe )T R1 (x(t) − xe ) + (u(t) − ve )T R2 (u(t) − ve )]dt, 0

(13.21)

where R1 ∈ Rn×n , R2 ∈ Rm×m , R1 ≥ 0, R2 > 0, and the pair (A, R1 ) is observable. Here, we restrict K ∈ Rm×l to the class of admissible feedback gain matrices so that K ∈ S , {K ∈ Rm×l : A + BKC is asymptotically stable and CV C T > 0}, (13.22) where V , E [(x0 − xe )(x0 − xe )T ]. Now, it follows from standard linear optimal control theory that a static optimal output feedback controller minimizing (13.21) is given by [195] K = −R2−1 B T P QC T (CQC T )−1 ,

(13.23)

where the n × n positive-definite matrices P and Q satisfy

0 = (A − SP ν)Q + Q(A − SP ν)T + V, T 0 = AT P + P A + R1 − P SP + ν⊥ P SP ν⊥ ,

(13.24) (13.25)

where S , BR2−1 B T , ν , QC T (CQC T )−1 C, and ν⊥ , In − ν. In this case, the equilibrium point xe ∈ Rn+ for the closed-loop system (13.20) is globally asymptotically stable for all x0 ∈ Rn . Furthermore, V (x) = (x − xe )T P (x − xe ),

x ∈ Rn ,

(13.26)

is a Lyapunov function for the closed-loop system (13.20). Note that if C = In , then (13.24) and (13.25) specialize to 0 = AT P + P A + R1 − P SP

(13.27)

and the optimal feedback gain K is given by K = −R2−1 B T P.

(13.28)

NNBook

September 1, 2009

385

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

In this case, the existence of an n × n positive-definite P satisfying (13.27) such that A − SP is guaranteed to be asymptotically stable if and only if (A, B) is stabilizable and (A, R1 ) is observable. Note that the closed-loop system (13.20) is given by ˜ x(t) ˙ = A(x(t) − xe ),

x(0) = x0 ,

t ≥ 0,

(13.29)

where A˜ , A − SP ν. In general, the optimal output feedback controller (13.23) does not guarantee that the closed-loop state trajectories of (13.29) n will remain in the nonnegative orthant for all x0 ∈ R+ . However, since P > 0 and Q > 0 satisfying (13.24) and (13.25) are such that A˜ is asymptotically n stable, it follows that there exists a subset DA ⊆ R+ such that x(t) ∈ DA ⊆ n R+ , t ≥ 0, for all x0 ∈ DA . Furthermore, the control input (13.18), with K ∈ Rm×l given by (13.23), is an optimal stabilizing feedback control law for all initial conditions in DA . In the case where C = In , K ∈ Rm×n given by (13.28) is the only optimal stabilizing feedback control law for all n x0 ∈ DA . To characterize the subset DA ⊆ R+ , define the hyperplanes Γi , {x ∈ Rn : eT i x = 0}, where ei is a vector with unity in the ith component and zeros elsewhere, with associated minimum Lyapunov values VΓi , min V (x), x∈Γi

i ∈ {1, . . . , n},

(13.30)

where V (x) is given by (13.26). Proposition 13.2. If VΓi is given by (13.30), then VΓi =

2 (eT i xe ) . −1 e eT i i P

(13.31)

Proof. First, let β > 0 be sufficiently large such that the closed set Γβ , {x ∈ Γi : V (x) ≤ β} is not empty. Since V (x) = O(kx − xe k2 ), it follows that Γβ is bounded, and hence, compact. Hence, V (x) has a global minimum on Γβ , and hence, on Γi . Thus, suppose x∗ ∈ Γi solves minx∈Γi V (x). Next, to minimize V (x) subject to x ∈ Γi form the Lagrangian L(x, λ) , V (x) + λeT i x, where λ ∈ R is a Lagrange multiplier. Now, if x∗ solves (13.30), then ∂L = 2(x∗ − xe )T P + λeT 0= i , ∂x x=x∗

(13.32)

T and hence, V (x∗ ) = λe2i xe . Next, multiplying (13.32) by P −1 ei yields

−1 0 = 2(x∗ − xe )T ei + λeT ei , i P

(13.33)

NNBook

September 1, 2009

386

CHAPTER 13

x2

˜ eT 2 A(x − xe ) = 0

DA

xe Dcl ˜ eT 1 A(x − xe ) = 0 x1

Figure 13.1 Domains of attraction Dcl and DA based on closed and open Lyapunov surfaces.

which implies that λ = eT2ePi−1xeei . Thus, i T

VΓi = V (x∗ ) =

2 (eT i xe ) . T ei P −1 ei

(13.34)

This completes the proof. The closed set defined by Dcl , {x ∈ Rn : V (x) ≤ VΓ }, where VΓ , n mini=1,...,n VΓi , is such that Dcl ⊂ R+ . To see this, suppose, ad absurdum, ∗ T there exists x ∈ Dcl such that el x∗ < 0 for some l ∈ {1, . . . , n}. Then, V (x∗ ) > VΓl ≥ VΓ , which is a contradiction. Furthermore, note that Dcl is a subset of the domain of attraction for (13.29) since (A, R1 ) is observable, ˜ − xe ) < 0, x ∈ Rn , x 6= xe , where R ˜ , and hence, V˙ (x) = −(x − xe )T R(x n T R1 + ν P SP ν. The set Dcl ⊂ R+ provides a region of attraction contained in the nonnegative orthant for the optimal output feedback problem using closed Lyapunov level surfaces (see Figure 13.1). Next, using open Lyapunov level surfaces [124] we provide a considn erably improved region of attraction DA ⊆ R+ for optimal output feedback control of nonnegative set-point regulation. The following lemma is needed n for construction of DA ⊆ R+ predicated on open Lyapunov level surfaces (see Figure 13.1). Lemma 13.1. Let x∗ ∈ Γi be such that VΓi = V (x∗ ) for i ∈ {1, . . . , n}.

NNBook

September 1, 2009

387

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

Then ˜ ∗ eT i A(x − xe ) > 0.

(13.35)

Proof. Since, by assumption, x∗ minimizes V (x) on Γi it follows from (x∗ ) ∗ T ˙ ∗ ˜ ∗ (13.32) that λ = 2V eTi xe > 0. Next, since V (x ) = 2(x − xe ) P A(x − xe ) = ˜ ∗ − xe ) < 0, it follows that eT A(x ˜ ∗ − xe ) > 0. −λeT A(x i

i

Lemma 13.1 implies that at the point x∗ ∈ Γi such that VΓi = V (x∗ ), the trajectory of the closed-loop system (13.29) is directed away from the n hyperplane Γi and toward the nonnegative orthant R+ . Next, define the n ˜ intersection of Γi and the hyperplane eT i A(x − xe ) = 0, x ∈ R , for i ∈ {1, . . . , n} by ˜ − xe ) = 0}, Si , {x ∈ Γi : eT A(x (13.36) i

with associated minimum Lyapunov values  if Si = 6 Ø, minx∈Si V (x), VS i , ∞, if Si = Ø.

(13.37)

Proposition 13.3. Let VSi be given by (13.37) and suppose ˜ −1 A˜T ei ) − (eT P −1 A˜T ei )2 6= 0. (eT P −1 ei )(eT AP i

i

i

Then VS i =

2 T ˜ −1 A ˜T ei ) (eT i xe ) (ei AP . −1 e )(eT AP ˜ −1 A˜T ei ) − (eT P −1 A˜T ei )2 (eT i i P i i

(13.38)

Proof. To minimize V (x) subject to x ∈ Si form the Lagrangian ˜ − xe ), L(x, λ1 , λ2 ) = V (x) + λ1 eT x + λ2 eT A(x (13.39) i

i

where λ1 , λ2 ∈ R are the Lagrange multipliers. Now, if x∗ solves minx∈Si V (x), then ∂L T ˜ = 2(x∗ − xe )T P + λ1 eT 0= (13.40) i + λ2 ei A. ∂x x=x∗

Next, multiplying (13.40) by P −1 ei and (13.40) by P −1 A˜T ei yields, respectively, T −1 ˜ −1 ei = 0, −2eT e i + λ2 e T (13.41) i xe + λ1 e i P i AP T −1 ˜T T ˜ −1 ˜T λ1 ei P A ei + λ2 ei AP A ei = 0, (13.42) which further implies that λ1 =

T ˜ −1 A ˜T ei ) 2(eT i xe )(ei AP . −1 e )(eT AP ˜ −1 A˜T ei ) − (eT P −1 A˜T ei )2 (eT i i P i i

(13.43)

NNBook

September 1, 2009

388

CHAPTER 13

Hence, VSi = V (x∗ ) =

2 T ˜ −1 A ˜T ei ) (eT λ1 e T i xe ) (ei AP i xe = T . ˜ −1 A˜T ei ) − (eT P −1 A˜T ei )2 2 (ei P −1 ei )(eT i AP i (13.44)

This completes the proof. −1 e )(eT AP ˜ −1 A˜T ei ) − (eT P −1 A˜T ei )2 = 0, then the system of If (eT i i P i i equations (13.41) and (13.42) has no solution, which implies that Γi and the ˜ hyperplane eT i A(x − xe ) = 0 have no intersection and, hence, VSi = ∞.

Next, we present a result providing a guaranteed subset of the domain of attraction contained in the nonnegative orthant for (13.29). For the statement of this result, define n

DA , {x ∈ R+ : V (x) < VS },

(13.45)

where VS , mini=1,...,n {VSi }. Theorem 13.1. Let DA be given by (13.45). Then DA is a subset of the domain of attraction for (13.29). Proof. First, we show that DA is an invariant set with respect to (13.29). Suppose VS < ∞. In this case, in order to show that DA is an ˜ invariant set with respect to (13.29) it suffices to show that eT i A(x − xe ) > 0 for all x ∈ DA ∩ Γi , i = 1, . . . , n. Note that DA ∩ Γi = {x ∈ Rn : V (x) < VS , e T i x = 0} is a convex set, and hence, is connected. Now, suppose, ad ˜ x − xe ) ≤ 0. Then, since absurdum, there exists x ˆ ∈ DA ∩ Γi such that eT i A(ˆ ∗ ˜ ∗ by Lemma 13.1, there exists x ∈ DA ∩ Γi such that eT i A(x − xe ) > 0, n ˜ it follows from the continuity of eT i A(x − xe ), x ∈ R , that there exists T ˜ x ˜ ∈ DA ∩ Γi such that ei A(˜ x − xe ) = 0, and hence, V (˜ x) ≥ VS , which is a T ˜ contradiction. Hence, ei A(x − xe ) > 0 for all x ∈ DA ∩ Γi , i = 1, . . . , n. n

Next, suppose VS = ∞. In this case, DA = R+ , and hence, in order to show that DA is an invariant set with respect to (13.29) it suffices to n ˜ show that eT i A(x − xe ) > 0 for all x ∈ ∂R+ ∩ Γi , i = 1, . . . , n, which can be shown using identical arguments as above. Hence, DA is an invariant set with respect to (13.29). Finally, since V˙ (x) < 0 for all x ∈ Rn \ {xe }, it follows that x(t) → xe as t → ∞ for all initial conditions in DA , where x(t), t ≥ 0, is the solution to (13.29). The following theorem shows that the closure of DA is also a subset of the domain of attraction for (13.29). Theorem 13.2. Let DA be given by (13.45). Then D A is a subset of

NNBook

September 1, 2009

389

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

the domain of attraction for (13.29). ′ , Proof. Suppose VS < ∞. Note that in this case D A = DA ∪ DA n ′ where DA = {x ∈ R+ : V (x) = VS }. Furthermore, by Theorem 13.1, DA is a subset of the domain of attraction for (13.29). Hence, to show that D A is a subset of the domain of attraction for (13.29) it need only be ′ such that eT x∗ = 0 and shown that for every initial condition x∗ ∈ DA l ˜ ∗ eT l A(x − xe ) = 0 for some l ∈ {1, . . . , n}, the trajectory of (13.29) moves away from the hyperplane Γl and toward the interior of DA . ′ and let s(t, x), t ≥ 0, denote the solution To show this, let x∗ ∈ DA n to (13.29) with initial condition x ∈ R+ . Now, suppose, ad absurdum, there ∗ ∗ ′ exists εˆ > 0 such that for all ε ∈ (0, εˆ], eT l s(ε, x ) < 0. Since x ∈ DA there T T ˜ exists a sequence {x0k }∞ k=1 ∈ DA such that el x0k = 0, el A(x0k − xe ) > 0, ∗ T and limk→∞ x0k = x . Hence, el s(ε, x0k ) > 0 for all ε ∈ (0, εˆ] and k = 1, 2, . . . . Since the trajectory of (13.29) is continuous in time and initial conditions, it follows that T ∗ lim eT l s(ε, x0k ) = el s(ε, x ) ≥ 0,

k→∞

ε ∈ (0, εˆ],

(13.46)

∗ which is a contradiction. Moreover, eT ˆ]. To show this, l s(ε, x ) > 0, ε ∈ (0, ε ∗ ∗ ∗ suppose, ad absurdum, that there exists ε ∈ (0, εˆ] such that eT l s(ε , x ) = 0. ∗ ∗ ∗ ˜ Hence, eT l A(s(ε , x )−xe ) = 0 and, since x solves (13.37) for i = l, it follows that

V (s(ε∗ , x∗ )) ≥ V (x∗ ) = V (s(0, x∗ )),

(13.47)

which is a contradiction since V (·) is a strictly decreasing function for all ∗ initial conditions in Rn \ {xe }. Hence, eT ˆ], which l s(ε, x ) > 0, ε ∈ (0, ε ∗ ′ implies that at x ∈ DA , the trajectory of (13.29) moves away from the hyperplane Γl and towards the interior of DA . Hence, DA is an invariant set and since V˙ (x) < 0 for all x ∈ Rn \ {xe }, it follows that DA is a subset of the domain of attraction for (13.29). n

Finally, if VS = ∞, then DA = DA = R+ , and hence, the result is a direct consequence of Theorem 13.1. Since Si ⊂ Γi it follows that VSi ≥ VΓi , i = 1, . . . , n. Hence, the domain of attraction predicted by (13.45) will always be larger than or equal to the domain of attraction predicted by (13.45) with VS = mini=1,...,n {VΓi }. Moreover, the control law (13.18) with K ∈ Rm×l given by (13.23) is an optimal stabilizing control law for all initial conditions x0 ∈ D A . Finally, note that in the case when VS = ∞, the system (13.29) is n n asymptotically stable for all x0 ∈ R+ and x(t) ∈ R+ , t ≥ 0. Hence, in this

NNBook

September 1, 2009

390

CHAPTER 13

case the feedback control law (13.18) with K ∈ Rm×l given by (13.23) is an optimal stabilizing feedback control law for all initial conditions in the nonnegative orthant. Furthermore, it follows from Proposition 2.5 that A˜ is ˜ e is a positive vector. essentially nonnegative and −Ax

13.4 Suboptimal Control for Nonnegative Dynamical Systems In Section 13.3 we characterized optimal output feedback controllers for nonnegative dynamical systems that guarantee that the closed-loop state trajectory remains in the region of attraction D A contained in the nonnegative orthant of the state space for nonnegative initial conditions contained in D A . In this and the next section, we develop suboptimal and optimal fixed-structure controllers for nonnegative systems for the case where x0 ∈ n R+ \ DA . In particular, we develop time-optimal controllers that guarantee the closed-loop state trajectories x(t), t ≥ 0, enter D A in a finite time tˆ < tf n while ensuring that x(t) ∈ R+ , 0 ≤ t ≤ tf . Once x(t) ∈ ∂DA , t = tˆ, we switch the controller to the optimal static output feedback control law discussed in Section 13.3. Of course, such a switching controller scheme will work only if the state of the system can be accurately measured. To develop suboptimal controllers for nonnegative systems for the case n where x0 ∈ R+ \ D A , let xd (t), t ∈ [0, tf ], be a desired trajectory in the nonnegative orthant given by xd (t) = x0 + α(t)(xe − x0 ), t tf ,

n

x0 ∈ R+ \ D A ,

0 ≤ t ≤ tf ,

(13.48)

where α(t) = t ∈ [0, tf ], and tf > 0. Furthermore, for simplicity of exposition, assume that m = n and the matrix B ∈ Rn×n is invertible. The case where m 6= n is discussed below. Hence, ve = −B −1 Axe satisfies (13.19) and an open-loop control for (13.16) that generates the desired trajectory xd (t), t ∈ [0, tf ], is given by ud (t) = B −1 (x˙ d (t) − Axd (t)),

0 ≤ t ≤ tf .

(13.49)

To find the minimal terminal time tf we consider the performance functional Z tf J(x0 , tf ) = [(xd (t) − xe )T R1 (xd (t) − xe ) 0

+(ud (t) − ve )T R2 (ud (t) − ve )]dt.

Next, it can be easily shown that tf opt



∂J(x0 ,tf ) ∂tf tf =tf opt

(13.50)

= 0 yields

 12 3(xe − x0 )T B −T R2 B −1 (xe − x0 ) = (xe − x0 )T R1 (xe − x0 ) + (xe − x0 )T AT B −T R2 B −1 A(xe − x0 ) (13.51) 

NNBook

September 1, 2009

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

391

and ∂ 2 J(x0 , tf ) (xe − x0 )T B −T R2 B −1 (xe − x0 ) > 0. = ∂t2f t3f opt tf =tf opt

(13.52)

Hence, tf = tf opt minimizes the performance functional given by (13.50) with n xd (t), t ∈ [0, tf ], given by (13.48). Now, it follows that for all x0 ∈ R+ \ DA , the control law  i h t 0  − A(x + (x − x )) , 0 ≤ t < tˆ,  B −1 xtef−x 0 e 0 tf opt opt u(t) = (13.53)   KC(x(t) − xe ) + ve , tˆ ≤ t < ∞,

where x(t), t ≥ tˆ, is a solution to (13.16) with x(tˆ) = xd (tˆ), tˆ > 0, such that xd (tˆ) ∈ ∂D A and K ∈ Rn×l is given by (13.23), stabilizes the equilibrium point xe ∈ Rn+ of (13.16), minimizes the performance functional (13.21) with the lower limit in the integral replaced by tˆ, and ensures that the trajectory n x(t), t ≥ 0, of (13.16) remains in R+ . If B ∈ Rn×m and rank[B, Axe ] = rank B = m, then ve = −(B T B)−1 B T Axe

(13.54)

satisfies (13.19). In this case, if rank[B, A(t)] = rank B = m, 0 ≤ t ≤ tf opt , t 0 where A(t) , xtef−x − A(x0 + tf opt (xe − x0 )), 0 ≤ t ≤ tf opt , and tf opt is opt −T −1 given by (13.51) with B R2 B replaced by B(B T B)−T R2 (B T B)−1 B T , then the control law (13.53) is given by  0 ≤ t < tˆ,  (B T B)−1 B T A(t), u(t) = (13.55)  KC(x(t) − xe ) + ve , tˆ ≤ t < ∞, where ve is given by (13.54) and K ∈ Rm×l is given by (13.23). Alternatively, if rank[B, Axe ] = rank B < m, then ve = −B + Axe + (Im − B + B)z,

(13.56)

where B + ∈ Rm×n denotes the Moore-Penrose generalized inverse of B and z ∈ Rm , satisfies (13.19). Furthermore, if rank[B, A(t)] = rank B < m, 0 ≤ t ≤ tf opt , where tf opt is given by (13.51) with B −T R2 B −1 replaced by (B + )T R2 B + , then the control law (13.53) is given by  +  B A(t) + (Im − B + B)z, 0 ≤ t < tˆ, u(t) = (13.57)  ˆ KC(x(t) − xe ) + ve , t ≤ t < ∞, where ve is given by (13.56) and K ∈ Rm×l is given by (13.23).

NNBook

September 1, 2009

392

CHAPTER 13

Assuming that the desired trajectory (13.48) is given by the more general parameterization xd (t) = x0 + Aα (t)(xe − x0 ),

n

x0 ∈ R + \ D A ,

0 ≤ t ≤ tf ,

(13.58)

where Aα (t) , diag[α1 (t), . . . , αn (t)], t ∈ [0, tf ], the performance functional (13.50) can be rewritten as Z tf J(x0 , tf , α1 , . . . , αn ) = F (x0 , α1 (t), α˙ 1 (t), . . . , αn (t), α˙ n (t))dt, (13.59) 0

where F (·, . . . , ·) is the integrand of (13.50) with xd (·) given by (13.58). In this case, minimizing (13.59) with respect to tf , α1 (·), . . . , αn (·) subject to αi (t) ≥ −

eT i x0 , 0 ≤ t ≤ tf , T ei (xe − x0 )

if eT i (xe − x0 ) > 0,

(13.60)

αi (t) ≤ −

eT i x0 , 0 ≤ t ≤ tf , T ei (xe − x0 )

if eT i (xe − x0 ) < 0,

(13.61)

and

with αi (0) = 0 and αi (tf ) = 1, i = 1, . . . , n, results in a classical calculus of variations problem which can be solved using standard methods [304]. This minimization problem will in general give a better performance measure as compared to minimizing (13.50) with xd (·) given by (13.48). As in the minimization problem of (13.50) with xd (·) given by (13.48), solving for tf , α1 (·), . . . , and αn (·) yields an open-loop control law over [0, tˆ) that is dependent on the system initial conditions x0 . However, unlike the minimization problem of (13.50) with xd (·) given by (13.48), this may not be practical in practice since the minimization problem might not have a closed-form solution and will have to be solved numerically for each initial condition. If in (13.60) and (13.61) eT l (xe − x0 ) = 0 for some l ∈ {1, . . . , n}, T T then el xd (t) = el x0 , t ∈ [0, tf ], and hence, the function αl (·) does not appear in the performance integrand of (13.59). Hence, the problem of minimizing (13.59) with respect to tf and αi (·), i = 1, . . . , n, reduces to one of minimizing (13.59) with respect to tf and αi (·), i = 1, . . . , n, i 6= l.

13.5 Optimal Fixed-Structure Control for Nonnegative Dynamical Systems In this section, we consider an optimal fixed-structure output feedback nonzero set-point regulation problem wherein the set point can belong to the n boundary of the nonnegative orthant, that is, xe ∈ R+ , while guaranteeing n x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . For this problem we consider the linear

NNBook

September 1, 2009

393

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

dynamical system (13.3) and (13.4) with the performance functional (13.21), where A ∈ Rn×n is essentially nonnegative, B ∈ Rn×m is nonnegative and given by (13.5), C = [Cs , 0m×(n−m) ], where Cs , diag[c1 , . . . , cm ], ci ≥ 0, i = 1, . . . , m, and the pair (A, B) is stabilizable. Furthermore, we assume there m exists ve ∈ R+ such that (13.19) is satisfied. Here, we assume that the feedback control law is given by u(t) = Bs−1 K(y(t) − Cxe ) + ve ,

(13.62)

2 ]. Hence, the closed-loop where K ∈ Rm×m and K , diag[−k12 , . . . , −km system has the form

x(t) ˙ = (A + BBs−1 KC)x(t) + Bve − BBs−1 KCxe = A(x(t) − xe ), x(0) = x0 , t ≥ 0,

(13.63)

where A , A + BBs−1 KC. Clearly A is essentially nonnegative and Bve − n BBs−1 KCxe ∈ R+ . Hence, it follows from Proposition 4.1 that the trajectory n x(t), t ≥ 0, of (13.63) remains in the nonnegative orthant for all x0 ∈ R+ . As in Section 13.2, the performance functional (13.21) can be rewritten as J(K) = tr PV, where

(13.64)

0 = AT P + PA + R,

V , E [(x0 − xe )(x0 − xe )T ], and R , R1 + C T KT Bs−T R2 Bs−1 KC. Now, forming the Lagrangian L(P, Q, K) = tr{PV + Q[AT P + PA + R]},

(13.65)

where Q ∈ Rn×n is a Lagrange multiplier, it follows that the gradient expressions of L(P, Q, K) with respect to the free parameters are given by ∂L = AQ + QAT + V, (13.66) ∂P ∂L = AT P + PA + R, (13.67) ∂Q   m X 1 ∂L T = ki −tr CQPBBs−1 E(i,i) + Bs−T R2 Bs−1 E(j,j) , kj2 tr CQC T E(i,i) 4 ∂ki j=1

i = 1, . . . , m.

(13.68)

The optimal gains ki , i = 1, . . . , m, can be obtained numerically from the necessary conditions for optimization (13.66)–(13.68) using a quasiNewton optimization method. In this case, the control law (13.62) stabilizes

NNBook

394

September 1, 2009

CHAPTER 13

the closed-loop system (13.63), ensures that x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ n R+ , and minimizes (13.21) over the class of all fixed-structure controllers 2 ]. If x ∈ Rn , we can further given by (13.62) with K = diag[−k12 , . . . , −km e + n improve system performance by constructing the subset DA ⊆ R+ given by (13.45) of the domain of attraction for (13.63) with Bs−1 K ∈ Rm×m replaced by K ∈ Rm×m given by (13.23), where P > 0 and Q > 0 satisfy (13.24) and (13.25). Now, in the case where x0 ∈ D A we use the control law (13.18) and n in the case where x0 ∈ R+ \ DA we use the control law  −1  Bs KC(x(t) − xe ) + ve , 0 ≤ t < tˆ, u(t) = (13.69)  KC(x(t) − xe ) + ve , tˆ ≤ t < ∞, where x(t), t ≥ 0, is the solution to (13.3) such that x(tˆ) ∈ ∂D A and K ∈ Rm×m given by (13.62) satisfies (13.66)–(13.68). n

In general if xe ∈ Rn+ , then for all x0 ∈ D A (respectively, x0 ∈ R+ \D A ) the control law (13.18) (respectively, (13.69)) will yield better performance as compared to the fixed-structure control law (13.62).

13.6 Nonnegative Control for Nonnegative Dynamical Systems As discussed in Chapter 1, control (source) inputs for certain nonnegative dynamical systems are constrained to be nonnegative as are the system states. Hence, in this section we develop control laws for essentially nonnegative systems with nonnegative control inputs. However, since n m condition (13.2) is required to be satisfied for xe ∈ R+ and ve ∈ R+ , it follows from Brockett’s necessary condition for asymptotic stabilizability [193] that there does not exist a continuous stabilizing nonnegative feedback if 0 ∈ spec(A) and xe ∈ Rn+ . Hence, in this section we assume that A is asymptotically stable, and hence, without loss of generality, by Proposition 2.7 we further assume that A is an asymptotically stable compartmental matrix. Thus, we proceed with the aforementioned assumptions to design controllers for compartmental systems that guarantee that limt→∞ x(t) = n xe ≥≥ 0, where xe is a desired set point in R+ , while guaranteeing a nonnegative control input. Furthermore, we assume that control inputs are injected directly into m separate compartments so that Bs in (13.5) is given by Bs = diag[b1 , . . . , bm ], where bi ≥ 0, i = 1, . . . , m. For compartmental systems this assumption is not restrictive since control inputs correspond to control inflows to each individual compartment. Theorem 13.3. (13.17) where A ∈ matrix, B ∈ Rn×m diag[b1 , . . . , bm ], bi ≥

Consider the linear dynamical system (13.16) and Rn×n is an asymptotically stable compartmental is nonnegative and is given by (13.5) with Bs = 0, i = 1, . . . , m, and C = [Cs , 0m×(n−m) ], where

NNBook

September 1, 2009

395

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

Cs , diag[c1 , . . . , cm ], ci ≥ 0, i = 1, . . . , m. Assume that for a given desired n m equilibrium point xe ∈ R+ there exists ve ∈ R+ such that (13.19) is satisfied. Then the feedback control law ui (t) = max{0, uˆi (t)},

i = 1, . . . , m,

(13.70)

where u ˆi (t) = kˆi ci (xi (t) − xei ) + vei ,

kˆi ≤ 0,

i = 1, . . . , m,

(13.71)

guarantees that the equilibrium solution x(t) ≡ xe of the closed-loop system n (13.16) and (13.70) is asymptotically stable for all x0 ∈ R+ . Furthermore, n u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . Proof. Since A is essentially nonnegative and asymptotically stable it follows from Theorem 2.12 that there exists a positive diagonal matrix P , diag[p1 , . . . , pn ] and a positive-definite matrix R ∈ Rn×n such that 0 = AT P + P A + R.

(13.72)

To show asymptotic stability of the closed-loop system (13.16) and (13.70), consider the Lyapunov function candidate V (x) = (x − xe )T P (x − xe ). Note that V (xe ) = 0 and, since P is positive definite, V (x) > 0 for all x 6= xe . Furthermore, V (x) is radially unbounded. Now, letting x(t), t ≥ 0, denote the solution to (13.16) and using (13.19), it follows that the Lyapunov derivative along the closed-loop system trajectories is given by V˙ (x(t)) = 2(x(t) − xe )T P [A(x(t) − xe ) + B(u(t) − ve )] = −(x(t) − xe )T R(x(t) − xe ) + 2(x(t) − xe )T P B(u(t) − ve ) m X T = −(x(t) − xe ) R(x(t) − xe ) + 2 pi bi (xi (t) − xei )(ui (t) − vei ). i=1

(13.73)

Now, for the two cases u ˆi (t) < 0, t ∈ Ix0 , i = 1, . . . , m, where Ix0 , {t ∈ [0, ∞) : u ˆi (t) < 0}, and u ˆi (t) ≥ 0, t ∈ [0, ∞) \ Ix0 , i = 1, . . . , m, the last term on the right-hand side of (13.73) gives: i) If u ˆi (t) < 0, t ∈ Ix0 , i = 1, . . . , m, then it follows from (13.71) that xi (t) ≥ xei , t ∈ Ix0 , i = 1, . . . , m, and hence, pi bi (xi (t) − xei )(ui (t) − vei ) = −pi bi vei (xi (t) − xei ) ≤ 0,

t ∈ I x0 .

ii) Otherwise, u ˆi (t) ≥ 0, t ∈ [0, ∞) \ Ix0 , i = 1, . . . , m, and hence, pi bi (xi (t) − xei )(ui (t) − vei ) = pi bi kˆi ci (xi (t) − xei )2 ≤ 0, t ∈ [0, ∞) \ Ix0 .

NNBook

September 1, 2009

396

CHAPTER 13

Hence, it follows that in either case V˙ (x(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) < 0,

x(t) 6= xe ,

t ≥ 0, (13.74)

which proves that the equilibrium solution x(t) ≡ xe for the closed-loop n system (13.16) and (13.70) is asymptotically stable for all x0 ∈ R+ . Finally, u(t) ≥≥ 0, t ≥ 0, is a restatement of (13.70). Now, since Bu(t) ≥≥ 0, t ≥ 0, it follows from Proposition 4.1 that x(t) ≥≥ 0, t ≥ 0, n for all x0 ∈ R+ . m

It follows from Theorem 13.3 that if there exists ve ∈ R+ satisfying (13.19) and kˆi ≤ 0, i = 1, . . . , m, then the nonnegative control law (13.70) n stabilizes the equilibrium point xe ∈ R+ for the closed-loop system (13.16) and (13.70). Here, we can choose any values for kˆi so long as kˆi ≤ 0, i = 1, . . . , m. However, to improve system performance we can consider the performance functional (13.21) and solve for ki , i = 1, . . . , m, using the necessary conditions for optimality (13.13)–(13.15) with V = E [(x0 − xe )(x0 − xe )T ], Bs replaced by Im , and choose kˆi = min{0, ki }, i = 1, . . . , m.

13.7 Optimal Fixed-Structure Control for General Anesthesia To demonstrate the efficacy of the proposed constrained optimal control framework, in this section we develop optimal controllers for general anesthesia. As discussed in Chapter 12, propofol is an intravenous anesthetic that has been used for both induction and maintenance of general anesthesia [68]. A simple yet effective patient model for the disposition of propofol is based on the three-compartment mammillary model shown in Figure 13.2 with the first compartment acting as the central compartment and the remaining two compartments exchanging with the central compartment [200]. The three-compartment mammillary system provides a pharmacokinetic model for a patient describing the distribution of propofol into the central compartment (identified with the intravascular blood volume as well as highly perfused organs) and the other various tissue groups of the body. A mass balance for the whole compartmental system yields x˙ 1 (t) = −(a11 + a21 + a31 )x1 (t) + a12 x2 (t) + a13 x3 (t) + u(t), x1 (0) = x10 , t ≥ 0, (13.75) x˙ 2 (t) = a21 x1 (t) − a12 x2 (t), x2 (0) = x20 , (13.76) x˙ 3 (t) = a31 x1 (t) − a13 x3 (t), x3 (0) = x30 , (13.77) where x1 (t), x2 (t), x3 (t), t ≥ 0, are the masses in grams of propofol in the central compartment and Compartments 2 and 3, respectively, u(t), t ≥ 0, is

NNBook

September 1, 2009

397

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

u(t) ≡ Continuous infusion a12 x2 Compartment 2 a21 x1

Central Compartment

a31 x1 Compartment 3 a13 x3

aeff x1 Effect Site Compartment

a11 x1 ≡ Elimination

Figure 13.2 Compartmental mammillary model for disposition of propofol.

the infusion rate in grams/min of the anesthetic (propofol) into the central compartment, aij ≥ 0, i 6= j, i, j = 1, 2, 3, are the rate constants in min−1 for drug transfer between compartments, and a11 ≥ 0 in min−1 is the rate constant for elimination from the central compartment. It has been reported in [306] that a 2.5–6 µg/ml blood concentration level of propofol is required during the maintenance stage in general anesthesia depending on patient fitness and extent of surgical stimulation. Hence, continuous infusion control is required for maintaining this desired level of anesthesia. Our objective is to regulate the propofol concentration level of the central compartment to the desired level of 3.4 µg/ml while minimizing the propofol infusion rate into the central compartment and maximizing the convergence rate of the closed-loop system trajectory to the desired equilibrium. The propofol concentration in the central compartment is given by x1 /Vc , where Vc is the volume in liters of the central compartment. As noted in [200], Vc can be approximately calculated by Vc = (0.159 l/kg)(M kg), where M is the mass in kilograms of the patient. In our control design we assume M = 70 kg so that the desired level of propofol mass in the central compartment is given by xe1 = (4 µg/ml)(0.159 l/kg)(70 kg) = 44.52 mg. Note that in this case it follows from (13.19) that xe2 = aa21 xe1 , xe3 = aa31 xe1 , and ve = a11 xe1 . 12 13 Even though propofol concentration levels in the blood plasma are a good indication of the depth of anesthesia, they cannot be measured in real time during surgery. Furthermore, as discussed in Chapter 12, we are more interested in drug effect (depth of hypnosis) rather than drug concentration. Hence, we consider a more realistic model involving pharmacokinetics (drug concentration as a function of time) and pharmacodynamics (drug effect as a function of concentration) for control of anesthesia. Specifically, we use an electroencephalogram (EEG) signal as a measure of drug effect of anesthetic compounds on the brain [273]. Since electroencephalography provides real-

NNBook

September 1, 2009

398

CHAPTER 13

Table 13.1 Pharmacokinetic Parameters [98].

Set (min−1 ) A

a11 0.152

a21 0.207

a12 0.092

a31 0.040

a13 0.0048

time monitoring of the central nervous system activity, it can be used to quantify levels of consciousness, and hence, is amenable for feedback (closedloop) control in general anesthesia. As discussed in Chapter 12, the bispectral index (BIS), has been proposed as a measure of anesthetic effect [224]. This index quantifies the nonlinear relationships between the component frequencies in the electroencephalogram, as well as analyzing their phase and amplitude. The BIS signal is a nonlinear monotonically decreasing function of the depth of anesthesia and is given by   cγeff (t) , (13.78) BIS(ceff (t)) = BIS0 1 − γ ceff (t) + ECγ50 where BIS0 denotes the base line (awake state) value and, by convention, is typically assigned a value of 100, ceff is the propofol concentration in grams/liter in the effect-site compartment (brain), EC50 is the concentration at half maximal effect and represents the patient’s sensitivity to the drug, and γ determines the degree of nonlinearity in (13.78). Here, the effect-site compartment is introduced as a correlate between the central compartment concentration and the central nervous system concentration [261] (see Figure 13.2). Recall that the effect-site compartment concentration is related to the concentration in the central compartment by the first-order model c˙eff (t) = aeff (x1 (t)/Vc − ceff (t)),

ceff (0) = x1 (0)/Vc ,

t ≥ 0,

(13.79)

where aeff in min−1 is a positive time constant. Assuming x1 (0) = 0, it follows that Z t ceff (t) = e−aeff (t−s) aeff x1 (s)/Vc ds. (13.80) 0

In reality, the effect-site compartment equilibrates with the central compartment in a matter of a few minutes. The parameters aeff , EC50 , and γ are determined by data fitting and vary from patient to patient. BIS index values of 0 and 100 correspond, respectively, to an isoelectric EEG signal and an EEG signal of a fully conscious patient; while the range between 40 and 60 indicates a moderate hypnotic state [93]. For simplicity of exposition, in our first design we assume that the effect-site compartment equilibrates instantaneously with the central

NNBook

September 1, 2009

399

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS 100

90

80

BIS Index [score]

70

60

Target BIS 50

40

30

20

10

0

EC50 =3.4 [µg/ml] 0

1

2

3

4

5

6

Effect site concentration [ µg/ml]

7

8

9

10

Figure 13.3 BIS index versus effect-site concentration.

compartment, that is, we assume that aeff → ∞, so that (13.80) reduces to , t ≥ 0. Furthermore, we assume EC50 = 3.4 µg/ml, γ = 3, ceff (t) = x1V(t) c and BIS0 = 100 so that the BIS signal is shown in Figure 13.3. Finally, we use the average set of pharmacokinetic parameters [98] for 29 patients requiring general anesthesia for noncardiac surgery given in Table 13.1. In this case, (13.75)–(13.77) and (13.79) can be written in state space form (13.16) with x = [x1 , x2 , x3 ]T ,     −(a11 + a21 + a31 ) a12 a13 1 a21 −a12 0  , B =  0  , (13.81) A= a31 0 −a13 0 and C = [1/Vc , 0, 0].

For this design we use the framework developed in Section 13.6 to construct nonnegative (possibly optimal) controllers to minimize the performance functional (13.21) with R1 = I3 mg−2 and R2 = 4 min2 · mg−2 . Solving the necessary conditions for optimality (13.13)–(13.15) with V = E [(x0 − xe )(x0 − xe )T ] = I3 and Bs replaced by 1, we obtain an optimal gain l . k for the control law u ˆ(t) = Vkc (x1 (t) − xe1 ) + ve given by kopt = −17.99 min Since kopt < 0, it follows from Theorem 13.3 that the control input u(t) = max{0, uˆ(t)}, where k = kopt , stabilizes the equilibrium point xe

NNBook

September 1, 2009

400

CHAPTER 13 3

for the closed-loop system. Moreover, for all initial conditions x0 ∈ R+ such that u ˆ(t), t ≥ 0, is an optimal ˆ(t) ≥≥ 0, t ≥ 0, the control u(t) = u nonnegative stabilizing control. For the initial condition x0 = [0, 0, 0]T the propofol concentration in the central compartment and the optimal nonnegative control input are shown in Figure 13.4. Figure 13.5 shows the BIS index versus time. For our next design, we consider the case where aeff < ∞, which yields a four-state model given by (13.75)–(13.78) and (13.79) for the disposition of propofol. In this case, (13.75)–(13.77) and (13.79) can be written in state space form (13.16) with x = [ceff , x1 , x2 , x3 ]T ,   aeff 0 −aeff 0 Vc  0 −(a11 + a21 + a31 ) a12 a13  , A=  0 a21 −a12 0  0 a31 0 −a13    T 0 1  1   0     B=  0 , C =  0  . 0 0 Once again we set EC50 = 3.4 µg/ml, γ = 3, and BIS0 = 100, and assume aeff = 3.4657 min−1 . Furthermore, we use the pharmacokinetic parameters given in Table 13.1. The target (desired) BIS value, BIStarget , is set at 50, which implies that the desired value of ceff e is 3.4 µg/ml and xe1 , xe3 = aa31 xe1 , and xe1 = (3.4 µg/ml)(Vc l) = 37.842 mg, xe2 = aa21 12 13 ve = a11 xe1 . The control objective is to stabilize the desired BIS value via an optimal unconstrained static output feedback control law of the form u(t) = k(ceff (t) − ceff e ) + ve ,

(13.82)

while minimizing the performance functional (13.21) with R1 = diag[1 ml2 · µg−2 , I3 mg−2 ] and R2 = 0.01 min2 · mg−2 . It follows from (13.23)–(13.25) that the optimal value of the gain k in (13.82) and the solution P > 0 of (13.24) and (13.25) are given by l , kopt = −9.9595 min   5.6593 −1.8693 −3.2813 −11.3162  −1.8693 0.7887 1.1549 3.9424  . P =   −3.2813 1.1549 6.5896 3.8041  −11.3162 3.9424 3.8041 108.1090 The domain of attraction for the closed-loop system guaranteeing that the closed-loop system states remain in the nonnegative orthant is

NNBook

September 1, 2009

401

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

Concentration [µ g/ml]

4

3

2

1

0

0

10

20

30

40

50

60

70

80

90

100

60

70

80

90

100

Time [min]

Control signal [mg/min]

70 60 50 40 30 20 10 0

0

10

20

30

40

50

Time [min]

Figure 13.4 Concentration and control signal versus time. 100

90

80

BIS Index [score]

70

60

50

40

30

20

10

0

0

10

20

30

40

50

60

70

80

90

100

Time [min]

Figure 13.5 BIS index versus time.

4

given by DA = {x ∈ R+ : V (x) = (x − xe )T P (x − xe ) ≤ 473.2532}, where x = [ceff , x1 , x2 , x3 ]T . Since V (0) = 1.1079 × 107 , it follows that the initial condition x0 = [0, 0, 0, 0]T 6∈ DA . This, of course, indicates that the actual domain of attraction is larger than DA . However, for

NNBook

September 1, 2009

402

CHAPTER 13 4

all x0 ∈ R+ \ DA the optimal output feedback controller (13.82) can be used as long as x(t) ≥≥ 0, t ≥ 0, is achieved. For this example it can be shown that for ceff (t) ≤ 3, 9775µg/ml, t ≥ 0, the control input (13.82) with k = kopt is guaranteed to be nonnegative. Furthermore, for x0 = [0, 0, 0, 0]T , ceff (t) ≤ 3.4µg/ml, t ≥ 0. Hence, by Proposition 2.5, x(t) ≥≥ 0, t ≥ 0, which implies that in this case the control law (13.82) is an optimal output nonnegative feedback controller. Figure 13.6 shows propofol concentration in the central and site effect compartments and the rate of propofol infusion into the central compartment versus time. Note that the effect-site compartment equilibrates with the central compartment in a matter of minutes. Finally, Figure 13.7 shows the BIS index versus time.

Concentrations [ µg/ml]

3.5 3 2.5

ceff(t) x1(t)/ Vc

2 1.5 1 0.5 0

0

10

20

30

40

50

60

70

80

90

100

60

70

80

90

100

Time [min]

Control signal [mg/min]

40 35 30 25 20 15 10 5

0

10

20

30

40

50

Time [min]

Figure 13.6 Concentrations in the central and site effect compartments and control signal versus time.

NNBook

September 1, 2009

403

OPTIMAL CONTROL FOR NONNEGATIVE SYSTEMS

100

90

80

BIS Index [score]

70

60

50

40

30

20

10

0

0

10

20

30

40

50

60

70

Time [min]

Figure 13.7 BIS signal versus time.

80

90

100

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Fourteen

H2 Suboptimal Control for Nonnegative Dynamical Systems Using Linear Matrix Inequalities

14.1 Introduction Even though nonnegative systems are often encountered in numerous application areas, nonnegative orthant stabilizability and holdability has received little attention in the literature. Notable exceptions include [20, 193]. In addition, fixed-structure control for linear nonnegative dynamical systems, and adaptive and neuroadaptive control of nonnegative and compartmental systems have been recently developed in [120, 122, 132, 133, 228]. In this chapter, we use linear matrix inequalities (LMIs) to develop H2 (sub)optimal estimators and controllers for nonnegative dynamical systems. Linear matrix inequalities provide a powerful design framework for linear control problems [39,72,236,260]. Since LMIs lead to convex or quasiconvex optimization problems, they can be solved very efficiently using interiorpoint algorithms. An intersting feature of nonnegative orthant stabilizability is that it can be formulated as a solution to an LMI problem. However, H2 optimal nonnegative orthant stabilizability cannot, in general, be formulated as an LMI problem. In this chapter, we formulate a series of generalized eigenvalue problems subject to a set of LMI constraints for designing H2 suboptimal estimators, static controllers, and dynamic controllers for nonnegative dynamical systems.

14.2 H2 Suboptimal Control for Nonnegative Dynamical Systems In this section, we present a suboptimal control framework for minimizing the H2 cost of a linear nonnegative dynamical system while constraining the system states to the nonnegative orthant of the state space. First, however, we recall the following standard results. Theorem 14.1 ([70]). Let A ∈ Rn×n . A is Hurwitz if and only if there

NNBook

September 1, 2009

406

CHAPTER 14

exists an n × n matrix P > 0 such that

AT P + P A < 0.

(14.1)

Next, consider the linear dynamical system G given by x(t) ˙ = Ax(t) + Dw(t), z(t) = Ex(t),

x(0) = x0 ,

t ≥ 0,

(14.2) (14.3)

where A ∈ Rn×n , D ∈ Rn×d , E ∈ Rp×n , and w(t) ∈ Rd , t ≥ 0, is a standard white noise process. Theorem 14.2 ([70]). Consider the linear dynamical system G given by (14.2) and (14.3). Then A is Hurwitz and |||G|||22 < λ if and only if there exists Q > 0 such that tr EQE T < λ and 0 > AQ + QAT + DD T ,

(14.4)

where |||G|||2 denotes the H2 norm of the transfer function G(s) = E(sIn − A)−1 D. Recall that the H2 norm of G(s) is given by   Z t 1 T 2 z (s)z(s)ds . |||G|||2 = lim E t→∞ t 0

(14.5)

Next, we consider the static H2 optimal regulator problem and present the solution to this problem using LMIs. Problem 14.1 (Static H2 Optimal Regulator Problem). Given the linear controlled system x(t) ˙ = Ax(t) + Bu(t) + Dw(t),

x(0) = x0 ,

t ≥ 0,

(14.6)

where A ∈ Rn×n , B ∈ Rn×m , D ∈ Rn×d , x(t) ∈ Rn , u(t) ∈ Rm is the control input, w(t) ∈ Rd is a standard white noise process, and (A, B) is stabilizable, determine a static state feedback control law u(t) = Kx(t), where K ∈ Rm×n , that satisfies the following design criteria: i ) the closedloop system matrix given by A+BK is Hurwitz; and ii ) the H2 performance criterion   Z t 1 T z (s)z(s)ds (14.7) J(K) = lim E t→∞ t 0 is minimized, where the performance variable z(t) ∈ Rp is given by z(t) = E1 x(t) + E2 u(t), and where E1 ∈ Rp×n and E2 ∈ Rp×m are such that E1T E2 = 0.

(14.8)

NNBook

September 1, 2009

407

H2 THEORY FOR NONNEGATIVE SYSTEMS

Theorem 14.3 ([70]). Consider the Static H2 Optimal Regulator Problem given by Problem 14.1. Then the optimal control gain and optimal H2 cost are given by K ∗ = Z ∗ X ∗ −1 and J(K ∗ ) = tr W ∗ , respectively, where X ∗ = X ∗T ∈ Rn×n , W ∗ = W ∗T ∈ Rp×p , and Z ∗ ∈ Rm×n are the optimal solutions to the generalized eigenvalue problem (GEVP) inf

X∈Rn×n , Z∈Rm×n , W ∈Rp×p

tr W

(14.9)

subject to 0 > AX + XAT + BZ + Z T B T + DD T ,   X (E1 X + E2 Z)T 0 < . (E1 X + E2 Z) W

(14.10) (14.11)

It is important to note that the generalized eigenvalue problem formulated in Theorem 14.3 is a convex optimization problem, and hence, the infimum (14.9) is attained. Similar remarks hold for all the theorems in this chapter. Before stating the main theorem of this section we need the following definitions and proposition. Definition 14.1 ([20]). Consider the linear dynamical system given by x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0 ,

t ≥ 0.

(14.12)

Then the pair (A, B) is nonnegative orthant holdable if there exists a control n n input u : [ 0, ∞) → Rm such that, with initial condition x0 ∈ R+ , x(t) ∈ R+ for all t ≥ 0. Definition 14.2 ([20]). Consider the linear dynamical system given by (14.12). Then the pair (A, B) is nonnegative orthant feedback holdable if there exists a feedback control law of the form u(t) = Kx(t) such that, for n n every initial condition x0 ∈ R+ , x(t) ∈ R+ for all t ≥ 0. Recall that the linear dynamical system (14.12) is stabilizable if and only if there exists an n × n matrix Y > 0 such that 0 > (A + BK)T Y + Y (A + BK).

(14.13)

Note that (14.13) is not an LMI in K and Y . However, there exist an n × n matrix Y > 0 and a matrix K ∈ Rm×n such that (14.13) holds if and only if there exist an n × n matrix X > 0 and a matrix K ∈ Rm×n such that 0 > (A + BK)X + X(A + BK)T .

(14.14)

Now, defining Z , KX, it follows that there exist an n × n matrix X > 0 and a matrix Z ∈ Rm×n such that 0 > AX + XAT + BZ + Z T B T

(14.15)

NNBook

September 1, 2009

408

CHAPTER 14

if and only if there exist an n × n matrix X > 0 and a matrix K ∈ Rm×n such that (14.14) holds. Hence, the stabilizabiliy condition (14.13) holds if and only if the LMI condition (14.15) holds. Definition 14.3 ([20]). Consider the linear dynamical system given by (14.12). Then the pair (A, B) is stabilizable-nonnegative orthant feedback holdable if there exists a feedback control law of the form u(t) = Kx(t) such that the closed-loop system is asymptotically stable and, for every initial n n condition x0 ∈ R+ , x(t) ∈ R+ for all t ≥ 0. Proposition 14.1. Consider the linear dynamical system (14.12). Then the following statements are equivalent: i ) The pair (A, B) is stabilizable-nonnegative orthant feedback holdable. ii ) There exists K ∈ Rm×n such that −(A + BK) is a nonsingular Mmatrix. iii ) There exist a diagonal n × n matrix Y > 0 and a matrix K ∈ Rm×n such that A + BK is essentially nonnegative and (A + BK)T Y + Y (A + BK) < 0.

(14.16)

Proof. The equivalence of statements i ) and ii ) follows from Theorem 4.1 of [20, p. 133]. Next, note that statement i ) holds if and only if there exists K ∈ Rm×n such that A + BK is Hurwitz and essentially nonnegative or, equivalently, if and only if there exist a positive diagonal matrix Y ∈ Rn×n and a matrix K ∈ Rm×n such that A + BK is essentially nonnegative and (14.13) holds. This proves the equivalence of statements i ) and iii ), and hence, statements ii ) and iii ). Next, we present an optimal control framework for minimizing an H2 norm bound of a nonnegative dynamical system while constraining the system states to the nonnegative orthant of the state space. Problem 14.2 (Static Compensation for Nonnegative Systems). Consider the linear dynamical system (14.6) with performance variables (14.8) where (A, B) is stabilizable-nonnegative orthant feedback holdable, A is essentially nonnegative, and B and x0 are nonnegative. Determine K such that A + BK is essentially nonnegative and Hurwitz, and the feedback control law u(t) = Kx(t) minimizes the quadratic performance criterion (14.7). The H2 optimal regulator problem for nonnegative dynamical systems given in Problem 14.2 is computationally intractable as an LMI. Specifically,

NNBook

September 1, 2009

409

H2 THEORY FOR NONNEGATIVE SYSTEMS

if the solution to the GEVP (14.9)–(14.11) is such that A + BK (with K = ZX −1 ) is essentially nonnegative, then such a solution solves Problem 14.2. However, in general, A + BK is not guaranteed to be essentially nonnegative for Problem 14.1, and the optimization problem (14.9)–(14.11) does not remain an LMI if the essential nonnegativity constraint on A + BK is enforced. As noted above, the stabilizability condition (14.13) may be checked using the LMI condition (14.15). If we further require the essential nonnegativity of A + BK or, equivalently, (AX + BZ)X −1 , the stabilizability condition will no longer remain an LMI. However, if X is positive and diagonal, then AX + BZ is essentially nonnegative if and only if (AX + BZ)X −1 = A + BK is essentially nonnegative. It follows from Proposition 14.1 that there exist a positive diagonal matrix X ∈ Rn×n and a matrix Z ∈ Rm×n such that (14.15) holds and AX + BZ is essentially nonnegative, that is, (AX + BZ)(i,j) ≥ 0,

i 6= j,

i, j = 1, 2, . . . , n,

if and only if (A, B) is stabilizable-nonnegative orthant feedback holdable. Based on this observation we propose the following H2 suboptimal control design framework for Problem 14.2. Theorem 14.4. Consider the Static Compensation for Nonnegative Systems Problem given by Problem 14.2. Assume there exist matrices X ∈ Rn×n , Z ∈ Rm×n , W ∈ Rp×p, where X is diagonal and W = W T , such that (14.10) and (14.11) hold, and 0 ≤ (AX + BZ)(i,j) ,

i 6= j,

i, j = 1, 2, . . . , n.

(14.17)

Then K = ZX −1 is such that A + BK is Hurwitz and essentially nonnegative. Furthermore, J(Kopt ) ≤ J(K) < tr W,

(14.18)

where Kopt denotes the solution to Problem 14.2. Finally, the sharpest H2 bound satisfying (14.18) is given by J(Kopt ) ≤ J(K ∗ ) ≤ tr W ∗ ,

(14.19)

where K ∗ = Z ∗ X ∗−1 and X ∗ , Z ∗ , and W ∗ are the optimal solutions to the GEVP (14.9) subject to (14.10), (14.11), and (14.17). Proof. Note that (14.10) can be equivalently written as 0 > (A + BK)X + X(A + BK)T + DD T .

(14.20)

Hence, since X > 0 it follows that A + BK is Hurwitz. Furthermore, since X is diagonal it follows from (14.17) that A + BK = (AX + BZ)X −1 is essentially nonnegative. Next, using Schur complements, (14.11) holds if

NNBook

410

September 1, 2009

CHAPTER 14

and only if X > 0 and W > (E1 X + E2 Z)X −1 (E1 X + E2 Z)T = (E1 + E2 K)X(E1 + E2 K)T . (14.21) Now, with A replaced by A + BK and E replaced by E1 + E2 K it ˜ 2 < tr W , where G(s) ˜ = (E1 + follows from Theorem 14.2 that |||G||| 2 −1 E2 K) (sIn − (A + BK)) D is the closed-loop system transfer function. ˜ 2 < tr W . Hence, J(K) = |||G||| 2 Next, (14.18) follows trivially since K is a feasible controller gain satisfying all of the constraints of Problem 14.2. Finally, (14.19) follows by noting that the minimal cost of (14.9) subject to (14.10), (14.11), and (14.17) with diagonal X will be higher than the minimal cost of (14.9) subject to (14.10), (14.11), and A + BZX −1 essentially nonnegative without the restriction on X being diagonal. Problem 14.2 considered in Theorem 14.4 requires that the pair (A, B) is stabilizable-nonnegative orthant feedback holdable. Necessary and sufficient conditions for stabilizability and holdability in the nonnegative orthant via the feedback controller u = Kx are given in Chapters 7 and 8 of [20]. Although the stabilizable-nonnegative orthant feedback holdability condition is equivalent to an LMI condition with diagonal X, the computation of the H2 norm (and hence the optimal regulator control problem) of a nonnegative system cannot in general be formulated as an LMI. ¯ ≤ J(Kopt ) ≤ J(K ∗ ), where K ¯ and Kopt correspond to Note that J(K) the solutions to Problems 14.1 and 14.2, respectively, and K ∗ is the optimal solution given in Theorem 14.4. This follows directly from Theorem 14.3 by noting that the minimal cost of (14.9) subject to (14.10), (14.11), and (14.17) will be higher than the minimal cost of (14.9) subject to (14.10) and (14.11) due to the additional constraint (14.17) and restriction on X being diagonal. Hence, Theorem 14.4 gives a framework for minimizing an upper bound on the H2 norm of the closed-loop system while preserving closed-loop system nonnegativity. To develop static control laws for nonnegative systems with a nonnegative control input, it suffices to solve the optimization problem stated in Theorem 14.4 with the additional LMI constraint Z ≥≥ 0. To see this, note that u(t) ≥≥ 0, t ≥ 0, if and only if K ≥≥ 0. Since X > 0 is diagonal, K ≥≥ 0 if and only if Z = KX ≥≥ 0. The closed-loop dynamics for the system described in Problem 14.2

NNBook

September 1, 2009

411

H2 THEORY FOR NONNEGATIVE SYSTEMS 2 Nonnegative LQR LQR 1.5

x1

1

0.5

0

−0.5 0

2

4 time

6

8

Figure 14.1 Comparison of x1 (t) using the nonnegative LQR and LQR designs.

using the design method in Theorem 14.4 is nonnegative (whether or not A is essentially nonnegative and B is nonnegative) provided that x(0) = x0 ≥≥ 0. In this case, since A + BK is essentially nonnegative, it follows from Proposition 2.5 that x(t) ≥≥ 0, t ≥ 0, with w(t) ≡ 0. Example 14.1. Consider the nonnegative dynamical system (14.6) with 

 −10 1 5 −1 1  , A= 2 10 1 −1



 0 B =  0 , 1



 0 D =  0 . 1

(14.22)

Here, we design a suboptimal H2 controller for Problem 14.2. Furthermore, we formulate the H2 control problem in terms of the system free response with w(t) ≡ 0. To eliminate the explicit dependence of the H2 cost on the initial condition x0 we assume x0 x0 T has expected value DD T . For our numerical simulation we take x(0) = [2, 1, 1]T , E1 = [I3 03×1 ]T , and R E2 = [03×1 1]. The YALMIP [202] and SeDuMi [285] MATLAB toolboxes are used to solve the LMI optimization problem given in Theorem 14.4. The H2 suboptimal control gain is given by K = [−7.1440, −1, −8.4763]. Figures 14.1–14.4 show the controlled states and control input versus time for an LQR design and the controller design given by Theorem 14.4. Clearly, the controlled states x1 (t) and x3 (t) take on negative values in the LQR design whereas all states remain nonnegative for the controller design given by Theorem 14.4. The optimal H2 cost is J(Kopt ) = 6.9897, whereas the H2 cost for the controller given by Theorem 14.4 is J(K ∗ ) = 8.4763. △ It is important to note that if w(t) is a standard white noise

NNBook

September 1, 2009

412

CHAPTER 14

1.4 Nonnegative LQR LQR

1.2 1

x

2

0.8 0.6 0.4 0.2 0 0

2

4

time

6

8

10

Figure 14.2 Comparison of x2 (t) using the nonnegative LQR and LQR designs.

disturbance signal, then w(t) can take arbitrary large negative values, and hence, subvert the nonnegativity of the states of (14.6). To guarantee the nonnegativity of the state variables for the case where w(t) 6≡ 0, the disturbance model Dw(t) in (14.6) needs to be nonlinear with w(·) ∈ D ⊂ L2 . To further elucidate this, recall that an equivalent characterization of the H2 norm of G(s) is given by Z ∞ 1 ||G(ω)||2F dω, |||G|||22 = 2π −∞ where || · ||F denotes the Frobenius matrix norm. In addition the following theorem is needed. Theorem 14.5. Consider the linear dynamical system (14.2) and (14.3) where w(·) ∈ L2 . Then |||z|||∞ ≤ |||G|||2 |||w|||2 , where ||| · |||∞ and ||| · |||2 denote the L∞ and L2 norms, respectively. Proof. It follows from ii ) of Corollary 3.1 of [52] that |||z|||2∞ ≤ σmax (EQE T )|||w|||22 , where Q ∈ Rn×n is the solution to the Lyapunov equation 0 = AQ + QAT + DD T . Now, since σmax (X) ≤ tr(X), X ∈ Rp×p , it follows that

|||z|||2∞ ≤ σmax (EQE T )|||w|||22 ≤ tr(EQE T )|||w|||22 = |||G|||22 |||w|||22 .

This completes the proof. To present an optimal control framework for minimizing an H2 norm bound of a nonnegative dynamical system while constraining the system

NNBook

September 1, 2009

413

H2 THEORY FOR NONNEGATIVE SYSTEMS 1 Nonnegative LQR LQR

0.8 0.6

x3

0.4 0.2 0 −0.2 −0.4 0

1

2

3 time

4

5

6

Figure 14.3 Comparison of x3 (t) using the nonnegative LQR and LQR designs.

states to the nonnegative orthant of the state space in the presence of system disturbances, we consider controlled dynamical systems of the form x(t) ˙ = Ax(t) + Bu(t) + DG(x(t))w(t), ˆ

x(0) = x0 ,

t ≥ 0, (14.23)

where A ∈ Rn×n , B ∈ Rn×m , D ∈ Rn×d , G : Rn → Rd×q , x(t) ∈ Rn , u(t) ∈ Rm is the control input, and w(t) ˆ ∈ Rq is an L2 disturbance. Furthermore, we assume that G(x) is such that supx∈Rn σmax (G(x)) ≤ 1 and, for every i ∈ {1, 2, . . . , n}, if xi = 0, then rowi (DG(x)) = 0. Note that if u = Kx is such that A + BK is essentially nonnegative, then the solution to (14.23) n is nonnegative for all w(·) ˆ ∈ L2 and x0 ∈ R+ . In addition, note that if w(·) ˆ ∈ L2 , then w(t) , G(x)w(t) ˆ ∈ L2 for all x ∈ Rn . Hence, (14.23) can be rewritten as (14.6) with w(·) ∈ D ⊂ L2 , where D , {w(·) ∈ L2 : w(t) = G(x(t))w(t), ˆ w(·) ˆ ∈ L2 , n and x(t) is the solution to (14.23) for some x0 ∈ R+ }. ˜ 2 is an upper bound to the Now, it follows from Theorem 14.5 that |||G||| ˜ induced operator norm from D to L∞ , where G(s) is the closed-loop transfer function from L2 disturbances w(t) to L∞ performance variables z(t). In this case, Theorem 14.4 solves Problem 14.2 for the case where w(·) ∈ D ⊂ L2 in (14.6).

14.3 Suboptimal Estimation for Nonnegative Dynamical Systems In this section, we present a suboptimal estimation framework for minimizing the H2 norm of the error dynamics of a linear dynamical system while constraining the estimated states to the nonnegative orthant of the state

NNBook

September 1, 2009

414

CHAPTER 14

5 Nonnegative LQR LQR

control input u

0 −5 −10 −15 −20 −25 0

2

4 time

6

8

Figure 14.4 Comparison of u(t) using the nonnegative LQR and LQR designs.

space. Note that if the nonnegativity constraint on the estimated states is relaxed, the H2 optimal solution is given by the least squares Kalman filter problem. Before stating the main theorem of this section, we state the standard least squares estimation problem and present a solution to this problem using LMIs. Problem 14.3 (H2 Optimal Estimation Problem). Consider the linear dynamical system given by x(t) ˙ = Ax(t) + D1 w(t), y(t) = Cx(t) + D2 w(t),

x(0) = x0 ,

t ≥ 0,

(14.24) (14.25)

where A ∈ Rn×n , D1 ∈ Rn×d , D2 ∈ Rl×d, C ∈ Rl×n , x(t) ∈ Rn , y(t) ∈ Rl is the output measurement, and w(t) ∈ Rd is a standard white noise process. Furthermore, assume that (A, C) is detectable and D1 D2T = 0. Design an estimator of the form x˙ e (t) = Ae xe (t) + Be y(t), ye (t) = Ce xe (t),

xe (0) = xe0 ,

t ≥ 0,

(14.26) (14.27)

where xe (t) ∈ Rn , Ae ∈ Rn×n , Be ∈ Rn×l , and Ce ∈ Rn×n , that satisfies the following design criteria: i ) x(t) − xe (t) → 0 as t → ∞ when w(t) ≡ 0; and ii ) the H2 error performance criterion   Z t 1 T z (s)z(s)ds (14.28) J(Be ) = lim E t→∞ t 0 is minimized, where the weighted error z(t) is given by z(t) , E (x(t) − xe (t)). As in the standard Kalman filter design, we set Ae = A − Be C and

NNBook

September 1, 2009

415

H2 THEORY FOR NONNEGATIVE SYSTEMS

Ce = In so that the only free parameter in Problem 14.3 is the Kalman gain Be . Note that the constraint Ae = A − Be C automatically satisfies criterion i ) in Problem 14.3 by requiring that A − Be C be Hurwitz. In this case, the error state e(t) → 0 as t → ∞ for w(t) ≡ 0, where e(t) , x(t) − xe (t). Theorem 14.6. Consider the H2 Optimal Estimation Problem given by Problem 14.3. Then the optimal Kalman gain and the optimal H2 cost are given by Be∗ = −Y ∗ −1 V ∗ T and J(Be∗ ) = tr U ∗ , respectively, where Y ∗ = Y ∗ T ∈ Rn×n , U ∗ = U ∗ T ∈ Rd×d , and V ∗ ∈ Rl×n are the optimal solutions to the GEVP inf

Y ∈Rn×n , V ∈Rl×n , U ∈Rd×d

tr U

(14.29)

subject to 0 > AT Y + Y A + C T V + V T C + E T E,   Y (Y D1 + V T D2 ) 0 < . (Y D1 + V T D2 )T U

(14.30) (14.31)

Proof. The proof is dual to the proof of Theorem 14.3 and, hence, is omitted. Next, we present the optimal nonnegative estimator design problem for nonnegative systems. Problem 14.4 (Optimal Estimation for Nonnegative Systems). Consider the linear dynamical system given by (14.24) and (14.25), and assume that (AT , −C T ) is stabilizable-nonnegative orthant feedback holdable, A is essentially nonnegative, C and x0 are nonnegative, and D1 D2T = 0. Determine Be such that A − BeC is essentially nonnegative and Hurwitz, Be is nonnegative, and the dynamic estimator (14.26) and (14.27) minimizes the least square error criterion (14.28). Note that (14.24)–(14.26) can be rewritten as        x(t) ˙ A 0 x(t) D1 = + w(t), x˙ e (t) Be C Ae xe (t) Be D2     x(0) x0 = , t ≥ 0. xe (0) xe0

(14.32)

Now, to guarantee that (14.32) is a nonnegative dynamical system with w(t) ≡ 0, we require that Ae = A−Be C is essentially nonnegative, Be ≥≥ 0, n n x0 ∈ R+ , and xe0 ∈ R+ . Theorem 14.7. Consider the Optimal Estimation for Nonnegative Systems Problem given by Problem 14.4. Assume there exist matrices

NNBook

September 1, 2009

416

CHAPTER 14

Y ∈ Rn×n , V ∈ Rl×n , and U ∈ Rd×d , where Y is diagonal and U = U T , such that (14.30) and (14.31) hold, and
(14.33) Y A + V T C (i,j) ≥ 0, i 6= j, i, j = 1, 2, . . . , n, V ≤≤ 0.

(14.34)

Then Be = −Y −1 V T is such that A − Be C is Hurwitz and essentially nonnegative, and Be ≥≥ 0. Furthermore, J(Be,opt ) ≤ J(Be ) < tr U,

(14.35)

where Be,opt denotes the solution to Problem 14.4. Finally, the sharpest H2 bound satisfying (14.35) is given by J(Be,opt ) ≤ J(Be∗ ) ≤ tr U ∗ , where Be∗ = −Y ∗−1 V ∗T and Y ∗ , U , and V ∗ are the optimal solutions to the GEVP (14.29) subject to (14.30), (14.31), (14.33), and (14.34). Proof. Note that (14.30) can be equivalently written as 0 > (A − Be C)T Y + Y (A − Be C) + E T E.

(14.36)

Hence, since Y > 0 it follows that A − Be C is Hurwitz. Furthermore, since Y is diagonal it follows from (14.33) that A − Be C = Y −1 (Y A + V T C) is essentially nonnegative. Next, Be = −Y −1 V T is nonnegative if and only if (14.34) holds. The remainder of the proof now follows as in the proof of Theorem 14.4. Example 14.2. In this example, we consider an estimation problem for the four-compartment model for the disposition of the anesthetic drug propofol considered in Section 13.7. Recall that a mass balance for the compartmental system yields a linear system of the form (14.24) and (14.25) where    T −3.4657 0.3114 0 0 1    0  0 −0.3990 0.0920 0.0048  , C =   . (14.37) A=    0  0 0.2070 −0.0920 0 0 0.0400 0 −0.0048 0

Here, we assume that the system states as well as system measurements are driven by a standard white noise process so that D1 = [I4 04×1 ] and D2 = [01×4 1]. For our numerical simulation we take x(0) = [0.5, 0.5, 0.5, 0.5]T , xe (0) = [1, 1, 1, 1]T , and a weighting matrix E = I4 . The YALMIP [202] and SeDuMi [285] MATLAB toolboxes are used to solve the LMI optimization problem given by Theorem 14.7. The estimator parameters are given by     −3.6974 0.3114 0 0 0.2317    0 −0.3990 0.0920 0.0048  0  , Be =  . Ae =      0 0.2070 −0.0920 0 0 0 0.0400 0 −0.0048 0

NNBook

September 1, 2009

417

H2 THEORY FOR NONNEGATIVE SYSTEMS

1

x x

0.8

1 e1

x (Kalman) e1

0.6 0.4 0.2 0 0

2

4

time

6

8

10

Figure 14.5 Comparison of x1 (t) and xe1 (t) of the undisturbed system using standard and nonnegative Kalman filter design.

(14.38) Figures 14.5–14.8 show the actual states and estimated H2 optimal Kalman filter and H2 suboptimal nonnegative filter states of the system for the case where w(t) ≡ 0. Note that the estimated Kalman filter states take on negative values whereas all states remain nonnegative for the estimator design given by Theorem 14.7. The optimal H2 estimator error is J(Kopt ) = 137.01, whereas the H2 estimator error for the estimator given by Theorem △ 14.7 is J(K ∗ ) = 563.1. 1.2

x2

1

xe2 xe2(Kalman)

0.8 0.6 0.4 0.2 0 −0.2 0

20

40

time

60

80

100

Figure 14.6 Comparison of x2 (t) and xe2 (t) of the undisturbed system using standard and nonnegative Kalman filter design.

NNBook

September 1, 2009

418

CHAPTER 14

1.2

x

3

1

x

e3

x (Kalman)

0.8

e3

0.6 0.4 0.2 0 −0.2 0

20

40

time

60

80

100

Figure 14.7 Comparison of x3 (t) and xe3 (t) of the undisturbed system using standard and nonnegative Kalman filter design.

14.4 H2 Suboptimal Dynamic Controller Design for Nonnegative Dynamical Systems In this section, we present a suboptimal dynamic controller design framework for minimizing the H2 norm of a linear dynamical system while constraining the plant states to the nonnegative orthant of the state space. The standard H2 dynamic compensation problem is given by the following problem. Problem 14.5 (H2 Optimal Dynamic Compensation). Consider the linear dynamical system given by x(t) ˙ = Ax(t) + Bu(t) + D1 w(t), y(t) = Cx(t) + D2 w(t),

x(0) = x0 ,

t ≥ 0,

(14.39) (14.40)

with performance variables (14.8), where A ∈ Rn×n , B ∈ Rn×m , C ∈ Rl×n , D1 ∈ Rn×d , D2 ∈ Rl×d , E1 ∈ Rp×n , E2 ∈ Rp×m , x(t) ∈ Rn is the system state vector, u(t) ∈ Rm is the control input, y(t) ∈ Rl is the output measurement, and w(t) ∈ Rd is a standard white noise process. Furthermore, assume that (A, B) is stabilizable, (A, E1 ) is detectable, E1T E2 = 0, and D1 D2T = 0. Design a full-order dynamic compensator of the form x˙ c (t) = Ac xc (t) + Bc y(t), u(t) = Cc xc (t),

xc (0) = xc0 ,

t ≥ 0,

(14.41) (14.42)

where xc (t) ∈ Rn , Ac ∈ Rn×n , Bc ∈ Rn×m , and Cc ∈ Rl×n , that satisfies the following design criteria: i ) the undisturbed (i.e., w(t) ≡ 0) closed-loop system (14.39)–(14.42) is asymptotically stable; and ii ) the H2 performance

NNBook

September 1, 2009

419

H2 THEORY FOR NONNEGATIVE SYSTEMS

1.6

x x

1.4

4 e4

x (Kalman) e4

1.2 1 0.8 0.6 0.4 0

20

40

time

60

80

100

Figure 14.8 Comparison of x4 (t) and xe4 (t) of the undisturbed system using standard and nonnegative Kalman filter design.

criterion   Z t 1 T z (s)z(s)ds J(Ac , Bc , Cc ) = lim E t→∞ t 0

(14.43)

is minimized. Theorem 14.8. Consider the H2 Optimal Dynamic Compensation Problem given by Problem 14.5. Then the optimal compensator gains are given by A∗c = A + BCc∗ − Bc∗ C, Bc∗ = −Y ∗ −1 V ∗ T , and Cc∗ = Z ∗ X ∗ −1 , where X ∗ = X ∗ T ∈ Rn×n , W ∗ = W ∗ T ∈ Rp×p , Y ∗ = Y ∗ T ∈ Rn×n , U ∗ = U ∗T ∈ Rd×d , V ∗ ∈ Rl×n , and Z ∗ ∈ Rm×n are the optimal solutions to the GEVPs (14.9) and (14.29) subject to (14.10), (14.11), and (14.30), (14.31), respectively. Proof. The proof is a direct consequence of Theorems 14.3 and 14.6 by reducing the H2 Optimal Dynamic Compensation Problem to a combination of a full information problem and an output estimation problem. Next, we present the optimal dynamic compensation problem for nonnegative systems. Problem 14.6 (Dynamic Compensation with Sign-Indefinite Input). Consider the linear dynamical system given by (14.39) and (14.40) where (A, B) and (AT , −C T ) are stabilizable-nonnegative orthant feedback holdable, A is essentially nonnegative, B, C, and x0 are nonnegative, D1 D2T = 0, and E1T E2 = 0. Determine the controller gains (Ac , Bc , Cc ) of (14.41) and (14.42) such that the undisturbed (i.e., w(t) ≡ 0) closed-loop system

NNBook

September 1, 2009

420

CHAPTER 14 n

(14.39)–(14.42) is asymptotically stable, the plant state x(t) ∈ R+ , t ≥ 0, for w(t) ≡ 0, and the quadratic performance criterion (14.43) is minimized. Theorem 14.9. Consider the Dynamic Compensation with SignIndefinite Input Problem given by Problem 14.6. Let xc0 = 0 and assume there exist matrices X ∈ Rn×n , Z ∈ Rm×n , W ∈ Rp×p , Y ∈ Rn×n , V ∈ Rl×n , and U ∈ Rd×d , where X and Y are diagonal and W = W T and U = U T , such that (14.10), (14.11), (14.30), (14.31) hold, and (AX + BZ)(i,j) ≥ 0,

i 6= j,

i, j = 1, 2, . . . , n,

(14.44)

Y A + V TC

i 6= j,

i, j = 1, 2, . . . , n.

(14.45) (14.46)

BZ ≤≤ 0,
≥ 0, (i,j)

Then Ac = A + BCc − Bc C, Bc = −Y −1 V T , and Cc = ZX −1 are such that the undisturbed (i.e., w(t) ≡ 0) closed-loop system (14.39)–(14.42) is n n asymptotically stable and plant state x(t) ∈ R+ for all x0 ∈ R+ and t ≥ 0. Furthermore, J(Ac,opt , Bc,opt , Cc,opt ) ≤ J(Ac , Bc , Cc ),

(14.47)

where Ac,opt , Bc,opt , and Cc,opt denote the solution to Problem 14.6. Finally, the sharpest H2 bound satisfying (14.47) is given by J(Ac,opt , Bc,opt , Cc,opt ) ≤ J(A∗c , Bc∗ , Cc∗ ),

(14.48)

where A∗c = A + BCc∗ − Bc∗ C, Bc∗ = −Y ∗ −1 V ∗ T , and Cc∗ = Z ∗ X ∗ −1 , and X ∗ , Z ∗ , Y ∗ , V ∗ , W ∗ , and U ∗ are the optimal solutions to the GEVPs (14.9) and (14.29) subject to (14.10), (14.11), (14.44), and (14.45), and (14.30), (14.31), and (14.46), respectively. Proof. Note that the undisturbed (w(t) ≡ 0) closed-loop system (14.39)–(14.42) is given by ˜x(t), x ˜˙ (t) = A˜

t ≥ 0,

x ˜(0) = x ˜0 ,

(14.49)

where x ˜,



or, equivalently, 

x xc



x(t) ˙ e(t) ˙

,



x ˜0 ,

=





x0 xc0



,

A˜ ,



A BCc Bc C Ac



,

  A + BCc −BCc x(t) , 0 A − Bc C e(t)     x(0) x0 = , t ≥ 0, e(0) x0

(14.50)

where e(t) , x(t) − xc (t). Next, note that (14.10) and (14.30) can be

NNBook

September 1, 2009

H2 THEORY FOR NONNEGATIVE SYSTEMS

421

equivalently written as 0 > (A + BCc )X + X(A + BCc )T + DD T , 0 > (A − Bc C)T Y + Y (A − Bc C) + E T E. Since X > 0 and Y > 0 it follows that A+BCc and A−Bc C are Hurwitz, and hence, A˜ is Hurwitz. Furthermore, since X and Y are diagonal A + BCc = (AX+BZ)X −1 and A−Bc C = Y −1 (Y A+V T C) are essentially nonnegative. Moreover, BCc ≤≤ 0 holds if and only if (14.45) holds. Hence, it follows n n from (14.50) that x(t) ∈ R+ for all x0 ∈ R+ and t ≥ 0. The remainder of the proof follows using similar arguments as in the proof of Theorem 14.4. Note that in Problem 14.6, we only require that the plant states be nonnegative. Specifically, in the proof of Theorem 14.9 we show that the state x(t) and the error state e(t) = x(t) − xc (t) are nonnegative for w(t) ≡ 0, and hence, the compensator state xc (t) = x(t) − e(t) need not be nonnegative. However, if x(t) ≥≥ 0 and e(t) is sufficiently small, then xc (t) is nonnegative. In certain biological applications, control (source) inputs are usually constrained to be nonnegative as are the system states. Hence, next we develop dynamic control laws for nonnegative systems with nonnegative control inputs. Problem 14.7 (Dynamic Compensation with Nonnegative Input). Consider the linear dynamical system given by (14.39) and (14.40), where (A, B) and (AT , −C T ) are stabilizable-nonnegative orthant feedback holdable, A is essentially nonnegative, B, C, and x0 are nonnegative, D1 D2T = 0, and E1T E2 = 0. Determine the controller gains (Ac , Bc , Cc ) of (14.41) and (14.42) such that the undisturbed (i.e., w(t) ≡ 0) closed-loop system n m (14.39)–(14.42) is asymptotically stable, x(t) ∈ R+ and u(t) ∈ R+ for all t ≥ 0 and for w(t) ≡ 0, and the quadratic performance criterion (14.43) is minimized. To guarantee the nonnegativity of the control input u(t), t ≥ 0, it follows from (14.42) that Cc xc (t) should be nonnegative. Here, we consider two different cases, namely, i ) Cc and xc0 are nonnegative, and ii ) Cc and xc0 are non-positive. Theorem 14.10. Consider the Dynamic Compensation with Nonnegative Input Problem given by Problem 14.7. Let xc0 ≥≥ 0 and assume there exist matrices X ∈ Rn×n , Z ∈ Rm×n , W ∈ Rp×p , Y ∈ Rn×n , V ∈ Rl×n , U ∈ Rd×d , where X and Y are diagonal and W = W T and U = U T , such that (14.10), (14.11), (14.30), (14.31) hold, and BZ ≥≥ 0,

(14.51)

NNBook

September 1, 2009

422

CHAPTER 14

Y A + V TC




(i,j) T

≥ 0,

i 6= j,

i, j = 1, 2, . . . , n,

V C ≤≤ 0.

(14.52) (14.53)

Then Ac = A + BCc − Bc C, Bc = −Y −1 V T , and Cc = ZX −1 are such that the undisturbed (i.e., w(t) ≡ 0) closed-loop system (14.39)–(14.42) is n m n asymptotically stable, and x(t) ∈ R+ and u(t) ∈ R+ for all x0 ∈ R+ and t ≥ 0. Furthermore, J(Ac,opt , Bc,opt , Cc,opt ) ≤ J(Ac , Bc , Cc ),

(14.54)

where Ac,opt , Bc,opt , and Cc,opt denote the solution to Problem 14.7. Finally, the sharpest H2 bound satisfying (14.54) is given by J(Ac,opt , Bc,opt , Cc,opt ) ≤ J(A∗c , Bc∗ , Cc∗ ),

(14.55)

where A∗c = A + BCc∗ − Bc∗ C, Bc∗ = −Y ∗−1 V ∗T , and Cc∗ = Z ∗ X ∗−1 , and X ∗ , Z ∗ , Y ∗ , V ∗ , W ∗ , and U ∗ are the optimal solutions to the GEVPs (14.9) and (14.29) subject to (14.10), (14.11), and (14.51), and (14.30), (14.31), (14.52), (14.53), respectively. Proof. As in the proof of Theorem 14.9, it follows from (14.10) and (14.30) that A + BCc and A − Bc C are Hurwitz, and hence, A˜ is Hurwitz. Next, since Y > 0 is diagonal, (A − Bc C) = Y −1 (Y A + V T C) is essentially nonnegative, and BCc and Bc C are nonnegative if and only if (14.51) and 2n (14.53) hold. Hence, A˜ is essentially nonnegative. Thus, x ˜(t) ∈ R+ , t ≥ 0, and hence, x(t) and xc (t) are nonnegative for all t ≥ 0 and w(t) ≡ 0. m Furthermore, (14.51) implies that u(t) ∈ R+ for all t ≥ 0. The remainder of the proof now follows using similar arguments as in the proof of Theorem 14.4. Theorem 14.11. Consider the Optimal Dynamic Compensation with Nonnegative Input Problem given by Problem 14.7. Let xc0 ≤≤ 0 and assume there exist matrices X ∈ Rn×n , Z ∈ Rm×n , W ∈ Rp×p , Y ∈ Rn×n , V ∈ Rl×n , U ∈ Rd×d , where X and Y are diagonal and W = W T and U = U T , such that (14.10), (14.11), (14.30), (14.31) hold, and (AX + BZ)(i,j) ≥ 0,

BZ ≤≤ 0, V T C ≥≥ 0.

i 6= j,

i, j = 1, 2, . . . , n,

(14.56) (14.57) (14.58)

Then Ac = A + BCc − Bc C, Bc = −Y −1 V T , and Cc = ZX −1 are such that the undisturbed (i.e., w(t) ≡ 0) closed-loop system (14.39)–(14.42) is n m n asymptotically stable, and x(t) ∈ R+ and u(t) ∈ R+ for all x0 ∈ R+ and t ≥ 0. Furthermore, J(Ac,opt , Bc,opt , Cc,opt ) ≤ J(Ac , Bc , Cc ),

(14.59)

NNBook

September 1, 2009

H2 THEORY FOR NONNEGATIVE SYSTEMS

423

where Ac,opt , Bc,opt , Cc,opt denote the solution to Problem 14.7. Finally, the sharpest H2 bound satisfying (14.59) is given by J(Ac,opt , Bc,opt , Cc,opt ) ≤ J(A∗c , Bc∗ , Cc∗ ),

(14.60)

where A∗c = A + BCc∗ − Bc∗ C, Bc∗ = −Y ∗ −1 V ∗ T , and Cc∗ = Z ∗ X ∗ −1 , and X ∗ , Z ∗ , Y ∗ , V ∗ , W ∗ , and U ∗ are the optimal solutions to the GEVPs (14.9) and (14.29) subject to (14.10), (14.11), (14.56), and (14.57) and (14.30), (14.31), and (14.58). Proof. The proof is identical to the proof of Theorem 14.10 and, hence, is omitted. The suboptimal H2 estimation and static and dynamic compensation frameworks for nonnegative dynamical systems considered in Sections 14.2–14.4 can be extended to the mixed-norm H2 /H∞ case using similar techniques as presented above and the results given in [26, 27].

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Fifteen

Adaptive Control for Nonnegative Systems

15.1 Introduction One of the fundamental problems in feedback control design is the ability of the control system to guarantee robust stability and robust performance with respect to system uncertainties in the design model. To this end, adaptive control along with robust control theory have been developed to address the problem of system uncertainty in control-system design. The fundamental differences between adaptive control design and robust control design can be traced to the modeling and treatment of system uncertainties as well as the controller architecture structures. In particular, adaptive control [10,153,227] is based on constant linearly parameterized system uncertainty models of a known structure but unknown variation, whereas robust control [303, 313] is predicated on structured and/or unstructured linear or nonlinear (possibly time-varying) operator uncertainty models consisting of bounded variation. Hence, for systems with constant real parametric uncertainties with large unknown variations, adaptive control is clearly appropriate, whereas for systems with time-varying parametric uncertainties and nonparametric uncertainties with norm bounded variations, robust control may be more suitable. Furthermore, in contrast to fixed-gain robust controllers, which maintain specified constants within the feedback control law to sustain robust performance, adaptive controllers directly or indirectly adjust feedback gains to maintain closed-loop stability and improve performance in the face of system uncertainties. Specifically, indirect adaptive controllers utilize parameter update laws to identify unknown system parameters and adjust feedback gains to account for system variation while direct adaptive controllers directly adjust the controller gains in response to plant variation. In either case, the overall process of parameter identification and controller adjustment constitutes a nonlinear control law architecture. Even though advanced robust and adaptive control methodologies have been (and are being) extensively developed for highly complex engineering

NNBook

426

September 1, 2009

CHAPTER 15

systems, modern active control technology has received far less consideration in medical systems. The main reason for this state of affairs is the steep barriers to communication between mathematics/control engineering and medicine. However, this is slowly changing, and there is no doubt that control-system technology has a great deal to offer medicine. For example, critical care patients, whether undergoing surgery or recovering in intensive care units, require drug administration to regulate key physiological (state) variables (e.g., blood pressure, cardiac output, heart rate, glucose, etc.) within desired levels. The rate of infusion of each administered drug is critical, requiring constant monitoring and frequent adjustments. Openloop control (manual control) by clinical personnel can be very tedious, imprecise, time consuming, and sometimes of poor quality. Hence, the need for active control (closed-loop control) in medical systems is severe; with the potential for improving the quality of medical care as well as curtailing its increasing cost. The complex highly uncertain and hostile environment of surgery places stringent performance requirements for closed-loop set-point regulation of physiological variables. For example, during cardiac surgery, blood pressure control is vital and is subject to numerous highly uncertain exogenous disturbances. Vasoactive and cardioactive drugs are administered, resulting in large disturbance oscillations to the system (patient). The arterial line may be flushed and blood may be drawn, corrupting sensor blood pressure measurements. Low anesthetic levels may cause the patient to react to painful stimuli, thereby changing system response characteristics. The flow rate of vasodilator drug infusion may fluctuate, causing transient changes in the infusion delay time. Hemorrhage, patient position changes, cooling and warming of the patient, and changes in anesthesia levels will also affect system response characteristics. In light of the complex and highly uncertain nature of system response characteristics under surgery requiring controls, it is not surprising that reliable system models for many highperformance drug delivery systems are unavailable. In the face of such high levels of system uncertainty, robust controllers may unnecessarily sacrifice system performance, whereas adaptive controllers can tolerate far greater system uncertainty levels to improve system performance [10, 153, 183, 227]. In this chapter, we develop a direct adaptive control framework for adaptive set-point regulation of linear and nonlinear uncertain nonnegative and compartmental systems. Specifically, using nonnegative and compartmental model structures, a Lyapunov-based direct adaptive control framework is developed that guarantees partial asymptotic setpoint stability of the closed-loop system; that is, asymptotic set-point stability with respect to part of the closed-loop system states associated with the physiological state variables. In particular, adaptive controllers

NNBook

September 1, 2009

427

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

are constructed without requiring knowledge of the system dynamics while providing a nonnegative control (source) input for robust stabilization with respect to the nonnegative orthant. Furthermore, in certain applications of nonnegative and compartmental systems such as biological systems, population dynamics, and ecological systems involving positive and negative inflows, the nonnegativity constraint on the control input is not natural. In this case, we also develop adaptive controllers that do not place any restriction on the sign of the control signal while guaranteeing that the physical system states remain in the nonnegative orthant of the state space.

15.2 Adaptive Control for Linear Nonnegative Uncertain Dynamical Systems In this section, we consider the problem of characterizing adaptive feedback control laws for nonnegative and compartmental uncertain dynamical systems to achieve set-point regulation in the nonnegative orthant. Specifically, consider the controlled linear uncertain dynamical system G given by x(t) ˙ = Ax(t) + Bu(t),

x(0) = x0 ,

t ≥ 0,

(15.1)

where x(t) ∈ Rn , t ≥ 0, is the state vector, u(t) ∈ Rm , t ≥ 0, is the control input, A ∈ Rn×n is an unknown essentially nonnegative matrix, and B ∈ Rn×m is an unknown nonnegative input matrix. The control input u(·) in (15.1) is restricted to the class of admissible controls consisting of measurable functions such that u(t) ∈ Rm , t ≥ 0. As discussed in Chapter 1, it follows from physical considerations that the state trajectories of nonnegative and compartmental dynamical systems remain in the nonnegative orthant of the state space for nonnegative initial conditions. However, even though active control of drug delivery systems for physiological applications additionally require control (source) inputs to be nonnegative, in many applications of nonnegative systems such as biological systems, population dynamics, and ecological systems, the positivity constraint on the control input is not natural. Hence, in this section we do not place any restriction on the sign of the control signal and design an adaptive controller that guarantees that the system states remain in the nonnegative orthant and converge to a desired equilibrium state. n

Specifically, for a given desired set point xe ∈ R+ , our aim is to design a control input u(t), t ≥ 0, so that limt→∞ kx(t)−xe k = 0. However, since in many applications of nonnegative systems and in particular, compartmental systems, it is often necessary to regulate a subset of the nonnegative state variables which usually include a central compartment, here we require that limt→∞ xi (t) = xdi ≥ 0 for i = 1, . . . , m ≤ n, where xdi is a desired set point for the ith state xi (t). Furthermore, we assume that control inputs

NNBook

September 1, 2009

428

CHAPTER 15

are injected directly into m separate compartments so that the input matrix is given by   Bu B= , (15.2) 0(n−m)×m where Bu , diag[b1 , . . . , bm ] and bi ∈ R+ , i = 1, . . . , m. Here, we assume that for i ∈ {1, . . . , m}, bi is unknown. For the statement of T T T and our main result define xe , [xT d , xu ] , where xd , [xd1 , . . . , xdm ] T xu , [xu1 , . . . , xu(n−m) ] . First, however, the following propositions are needed. The following result shows that, for nonnegative initial conditions, the states of a time-varying linear dynamical system G of the form x(t) ˙ = A(t)x(t),

x(0) = x0 ,

t ≥ t0 ,

(15.3)

where t0 ∈ [0, ∞) and A : [0, ∞) → Rn×n is continuous and essentially nonnegative pointwise-in-time, remain nonnegative. Proposition 15.1. Consider the time-varying dynamical system (15.3) n where A : [0, ∞) → Rn×n is continuous. Then R+ is an invariant set with respect to (15.3) if and only if A : [0, ∞) → Rn×n is essentially nonnegative pointwise-in-time. Proof. The result is a direct consequence of Proposition 2.1 by equivalently representing the time-varying system (15.3) as an autonomous nonlinear system by appending an additional state to represent time. Specifically, defining y(t − t0 ) , x(t) and yn+1 (t − t0 ) , t, it follows that the solution x(t), t ≥ t0 , to (15.3) can be equivalently characterized by the solution y(τ ), τ ≥ 0, where τ , t − t0 , to the nonlinear autonomous system y(τ ˙ ) = A(yn+1 (τ ))y(τ ), y(0) = y0 , y˙ n+1 (τ ) = 1, yn+1 (0) = t0 ,

τ ≥ 0,

(15.4) (15.5)

where y(·) ˙ and y˙ n+1 (·) denote differentiation with respect to τ . Now, since y˙ i (τ ) ≥ 0, τ ≥ 0, for i = 1, . . . , n + 1, whenever yi (τ ) = 0, the result is a direct consequence of Proposition 2.1. Next, we present a time-varying extension to Proposition 4.1 needed for the main theorems of this chapter. Specifically, we consider the timevarying system x(t) ˙ = A(t)x(t) + Bu(t),

x(t0 ) = x0 ,

t ≥ t0 ,

(15.6)

where A : [0, ∞) → Rn×n is continuous. For the following result the definition of nonnegativity holds with (4.2) replaced by (15.6). Proposition 15.2. Consider the time-varying dynamical system (15.6)

NNBook

September 1, 2009

429

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

where A : [0, ∞) → Rn×n is continuous. If A : [0, ∞) → Rn×n is essentially nonnegative pointwise-in-time and B ∈ Rn×m is nonnegative, then the solution x(t), t ≥ t0 , to (15.6) is nonnegative. Proof. The proof is similar to the proof of Proposition 15.1 using Proposition 4.1 and, hence, is omitted. Theorem 15.1. Consider the linear uncertain dynamical system G given by (15.1) where A is essentially nonnegative and B is nonnegative and given by (15.2). For a given xd assume there exist nonnegative vectors n−m m and ue ∈ R+ such that xu ∈ R + 0 = Axe + Bue .

(15.7)

Furthermore, assume there exists a diagonal matrix Kg = diag[kg 1 , . . . , kg m ] ˜ g is asymptotically stable, where K ˜g , ∈ Rm×m such that As , A + B K [Kg , 0m×(n−m) ]. Finally, let qi and qˆi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law u(t) = K(t)(ˆ x(t) − xd ) + φ(t),

(15.8)

where K(t) = diag[k1 (t), . . . , km (t)], x ˆ(t) = [x1 (t), . . . , xm (t)]T , and φ(t) ∈ m R , t ≥ 0, or, equivalently, ui (t) = ki (t)(xi (t) − xdi ) + φi (t),

i = 1, . . . , m,

(15.9)

where ki (t) ∈ R, t ≥ 0, and φi (t) ∈ R, t ≥ 0, i = 1, . . . , m, with update laws k˙ i (t) = −qi (xi (t) − xdi )2 , ki (0) ≤ 0, t ≥ 0, i = 1, . . . , m, (15.10)  0, if φi (t) = 0 and xi (t) ≥ xdi , φ˙ i (t) = −ˆ qi (xi (t) − xdi ), otherwise, φi (0) ≥ 0, i = 1, . . . , m, (15.11)

guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ) of the closedloop system given by (15.1), (15.8), (15.10), and (15.11) is Lyapunov stable n and xi (t) → xdi , i = 1, . . . , m, as t → ∞ for all x0 ∈ R+ . Furthermore, n x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . Proof. Note that with u(t), t ≥ 0, given by (15.8) it follows from (15.1) that x(t) ˙ = Ax(t) + BK(t)(ˆ x(t) − xd ) + Bφ(t),

x(0) = x0 ,

˜ g, or, equivalently, using (15.7) and As = A + B K

t ≥ 0,

(15.12)

x(t) ˙ = As (x(t) − xe ) + B(K(t) − Kg )(ˆ x(t) − xd ) + B(φ(t) − ue ), x(0) = x0 , t ≥ 0. (15.13) Furthermore, since As is essentially nonnegative and asymptotically stable,

NNBook

September 1, 2009

430

CHAPTER 15

it follows from Theorem 2.12 that there exist a positive diagonal matrix P , diag[p1 , . . . , pn ] and a positive-definite matrix R ∈ Rn×n such that 0 = AT s P + P As + R.

(15.14)

To show Lyapunov stability of the closed-loop system (15.10), (15.11), and (15.13) consider the Lyapunov function candidate V (x, K, φ) = (x − xe )T P (x − xe ) + tr(K − Kg )T Q−1 (K − Kg ) ˆ −1 (φ − ue ), +(φ − ue )T Q (15.15) or, equivalently, V (x, K, φ) =

n X i=1

where

2

pi (xi − xei ) +

m X p i bi i=1

 q1 qm ,..., , Q = diag p 1 b1 p m bm 

qi

2

(ki − kg i ) +

ˆ = diag Q



m X p i bi i=1

qˆi

(φi − uei )2 ,

 qˆ1 qˆm ,..., . p 1 b1 p m bm

ˆ are positive definite, Note that V (xe , Kg , ue ) = 0 and, since P , Q, and Q V (x, K, φ) > 0 for all (x, K, φ) 6= (xe , Kg , ue ). Furthermore, V (x, K, φ) is radially unbounded. Now, letting x(t), t ≥ 0, denote the solution to (15.13) and using (15.10) and (15.11), it follows that the Lyapunov derivative along the closedloop system trajectories is given by V˙ (x(t), K(t), φ(t)) = 2(x(t) − xe )T P [As (x(t) − xe ) + B(K(t) − Kg )(ˆ x(t) − xd ) T −1 ˙ ˙ ˆ −1 φ(t) +B(φ(t) − ue )] + 2tr(K(t) − Kg ) Q K(t) + 2(φ(t) − ue )T Q = −(x(t) − xe )T R(x(t) − xe ) + 2 +2

m X i=1

+2

i=1

pi bi (ki (t) − kg i )(xi (t) − xdi )2

pi bi (xi (t) − xdi )(φi (t) − uei ) + 2

m X p i bi i=1

m X

qˆi

m X p i bi i=1

qi

(ki (t) − kg i )k˙ i (t)

(φi (t) − uei )φ˙ i (t)

= −(x(t) − xe )T R(x(t) − xe )   m X 1 ˙ +2 pi bi (φi (t) − uei ) (xi (t) − xdi ) + φi (t) . qˆi

(15.16)

i=1

Now, for each i ∈ {1, . . . , m} and for the two cases given in (15.11),

NNBook

September 1, 2009

431

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

the last term on the right-hand side of (15.16) gives: i ) If φi (t) = 0 and xi (t) ≥ xdi , t ≥ 0, then φ˙ i (t) = 0, and hence,   1 ˙ pi bi (φi (t) − uei ) (xi (t) − xdi ) + φi (t) = −pi bi uei (xi (t) − xdi ) ≤ 0. qˆi ii ) Otherwise, φ˙ i (t) = −ˆ qi (xi (t) − xdi ), and hence,   1 ˙ pi bi (φi (t) − uei ) (xi (t) − xdi ) + φi (t) = 0. qˆi Hence, it follows that in either case V˙ (x(t), K(t), φ(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) ≤ 0, t ≥ 0,

(15.17)

which proves that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ) to (15.10), (15.11), and (15.13) is Lyapunov stable. Furthermore, since R > 0 it follows n from Theorem 4.2 of [112] that x(t) → xe as t → ∞ for all x0 ∈ R+ . n

Finally, to show that x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ , note that the closed-loop system (15.1), (15.8), (15.10), and (15.11) is given by x(t) ˙ = Ax(t) + BK(t)(ˆ x(t) − xd ) + Bφ(t) = (A + B[K(t), 0m×(n−m) ])x(t) − BK(t)xd + Bφ(t) ˜ = A(t)x(t) + v(t) + w(t), x(0) = x0 , t ≥ 0, (15.18) where

˜ A(t) ,

 a + b k (t) 11 1 1    a21   .  .  .  am1   am+1 1    . .  . an1

a12 a22 + b2 k2 (t)

.. ... ... .. . ...

a1m . . .

a1 m+1 . . .

... .. .

amm + bm km (t) am+1 m . . . anm

am m+1 am+1 m+1 . . . an m+1

... ... .. . ...

.

a1n a2n . . . amn am+1 n . . . ann



       ,      

(15.19)

b1 k1 (t)xd1 ..   .      bm km (t)xdm  v(t) , −  , 0     ..   . 0 

...



b1 φ1 (t) ..   .      bm φm (t)  w(t) ,  . 0     ..   . 0 



(15.20)

NNBook

September 1, 2009

432

CHAPTER 15

Now, since, by (15.10) and (15.11), ki (t) ≤ 0, t ≥ 0, i = 1, . . . , m, and φi (t) ≥ 0, t ≥ 0, i = 1, . . . , m, it follows that v(t) ≥≥ 0, t ≥ 0, and ˜ w(t) ≥≥ 0, t ≥ 0. Hence, since A(t), t ≥ 0, is essentially nonnegative pointwise-in-time, it follows from Proposition 15.2 that x(t) ≥≥ 0, t ≥ 0, n for all x0 ∈ R+ . Note that the conditions in Theorem 15.1 imply that x(t) → xe as t → ∞, and hence, it follows from (15.10) and (15.11) that (x(t), K(t), φ(t)) → M , {(x, K, φ) ∈ Rn × Rm×m × Rn : x = xe , K˙ = 0, φ˙ = 0} as t → ∞. It is important to note that the adaptive control law (15.8), (15.10), and (15.11) does not require the explicit knowledge of the gain matrix Kg and the nonnegative constant vector ue ; even though Theorem 15.1 requires the existence of Kg and nonnegative vectors xu and ue such that As is essentially nonnegative and asymptotically stable and condition (15.7) holds. Furthermore, in the case where A is semistable and minimum phase with respect to the output y = x ˆ, or A is asymptotically stable, then there ˜ g ∈ Rm×n such that As is asymptotically always exists a diagonal matrix K stable. Necessary and sufficient conditions for set-point stabilization of the pair (A, B), where A is singular and compartmental are given in [193, 291]. Finally, note that for i = 1, . . . , m, the control input signal ui (t), t ≥ 0, can be negative depending on the values of xi (t), ki (t), and φi (t), t ≥ 0. However, as is required in nonnegative and compartmental dynamical systems the closed-loop plant states remain nonnegative. In the case where our objective is zero set-point regulation, that is, xe = 0, the adaptive controller given in Theorem 15.1 can be considerably simplified. Specifically, since in this case x(t) ≥≥ xe = 0, t ≥ 0, and condition (15.7) is trivially satisfied with ue = 0, we can set φ(t) ≡ 0 so that update law (15.11) is superfluous. Furthermore, since (15.7) is trivially satisfied, A can possess eigenvalues in the right half plane. Alternatively, exploiting a linear Lyapunov function construction for the plant dynamics, an even simpler adaptive controller can be derived. This result is given in the following theorem. Theorem 15.2. Consider the linear uncertain dynamical system G given by (15.1) where A is essentially nonnegative and B is nonnegative and given by (15.2). Assume there exists a diagonal matrix Kg = ˜ g is asymptotically stable, diag[kg 1 , . . . , kg m ] ∈ Rm×m such that As , A+B K ˜ g , [Kg , 0m×(n−m) ]. Furthermore, let qi , i = 1, . . . , m, be positive where K constants. Then the adaptive feedback control law u(t) = K(t)ˆ x(t),

(15.21)

where K(t) = diag[k1 (t), . . . , km (t)] and x ˆ(t) = [x1 (t), . . . , xm (t)]T , or,

NNBook

September 1, 2009

433

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

equivalently, ui (t) = ki (t)xi (t),

i = 1, . . . , m,

(15.22)

where ki (t) ∈ R, i = 1, . . . , m, with update law

˙ K(t) = − diag[q1 x1 (t), . . . , qm xm (t)],

K(0) ≤≤ 0,

(15.23)

guarantees that the solution (x(t), K(t)) ≡ (0, Kg ) of the closed-loop system given by (15.1), (15.21), and (15.23) is Lyapunov stable and x(t) → 0 as n n t → ∞ for all x0 ∈ R+ . Furthermore, x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . Proof. Note that with u(t), t ≥ 0, given by (15.21) it follows from (15.1) that x(t) ˙ = Ax(t) + BK(t)ˆ x(t) = (A + B[K(t), 0m×(n−m) ])x(t) ˜ = A(t)x(t), x(0) = x0 , t ≥ 0,

(15.24)

˜ ˜ where A(t), t ≥ 0, is given by (15.19). Now, since A(t), t ≥ 0, is essentially nonnegative pointwise-in-time, it follows from Proposition 15.1 that x(t) ≥≥ n ˜ g , note that (15.24) can 0, t ≥ 0, for all x0 ∈ R+ . Next, using As = A + B K be equivalently written as x(t) ˙ = As x(t) + B(K(t) − Kg )ˆ x(t),

x(0) = x0 ,

t ≥ 0.

(15.25)

Furthermore, since As is essentially nonnegative and asymptotically stable, it follows from Theorem 2.11 that there exist p, r ∈ Rn such that p >> 0 and r >> 0 satisfy 0 = AT (15.26) s p + r. To show the Lyapunov stability of the closed-loop system (15.23) and (15.25) consider the Lyapunov function candidate V (x, K) = pT x + 12 tr(K − Kg )T Q−1 (K − Kg ),

(15.27)

or, equivalently, m

V (x, K) = pT x +

1 X p i bi (ki − kg i )2 , 2 qi

(15.28)

i=1

i h where Q = diag pq11b1 , . . . , pmqmbm . Note that V (0, Kg ) = 0 and, since p >> 0 and Q > 0, V (x, K) > 0 for all (x, K) 6= (0, Kg ). Furthermore, V (x, K) is radially unbounded with respect to the nonnegative orthant. Now, letting x(t), t ≥ 0, denote the solution to (15.25) and using (15.23), it follows that the Lyapunov derivative along the closed-loop system trajectories is given by x(t)] V˙ (x(t), K(t)) = pT [As x(t) + B(K(t) − Kg )ˆ

NNBook

September 1, 2009

434

CHAPTER 15

˙ +tr(K(t) − Kg )T Q−1 K(t) m X = −r T x(t) + pi bi (ki (t) − kg i )xi (t) i=1

+

m X p i bi i=1 T

qi

(ki (t) − kg i )k˙ i (t)

= −r x(t) ≤ 0, t ≥ 0, which proves that the solution (x(t), K(t)) ≡ (0, Kg ) to (15.23) and (15.25) is Lyapunov stable. Furthermore, since r >> 0, it follows from Theorem 4.2 n of [112] that x(t) → 0 as t → ∞ for all x0 ∈ R+ . Finally, we generalize Theorem 15.1 to the case where the input matrix is not necessarily nonnegative. For the statement of the following result define sgn bi , bi /|bi |, bi 6= 0, and sgn 0 , 0. Theorem 15.3. Consider the linear uncertain dynamical system G given by (15.1) where A is essentially nonnegative and B is given by (15.2) where bi , i = 1, . . . , m, is an unknown constant, but sgn bi is known. For a n−m and a vector given xd assume there exist a nonnegative vector xu ∈ R+ ue ∈ Rm such that (15.7) holds with Axe ≤≤ 0. Furthermore, assume there exists a diagonal matrix Kg = diag[kg 1 , . . . , kg m ] ∈ Rm×m such that ˜ g is asymptotically stable, where K ˜ g , [Kg , 0m×(n−m) ]. As , A + B K Finally, let qi and qˆi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law (15.8) with update laws k˙ i (t) = −(sgn bi )qi (xi (t) − xdi )2 , i = 1, . . . , m, (15.29)  0, if φi (t) = 0 and xi (t) ≥ xd , φ˙ i (t) = qi (xi (t) − xdi ), otherwise, −(sgn bi )ˆ i = 1, . . . , m, (15.30) where ki (0) and φi (0) are such that (sgn bi )ki (0) ≤ 0 and (sgn bi )φi (0) ≥ 0, respectively, guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ) of the closed-loop system given by (15.1), (15.8), (15.29), and (15.30) is n Lyapunov stable and xi (t) → xdi , i = 1, . . . , m, as t → ∞ for all x0 ∈ R+ . n Furthermore, x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . ˆ Proof. The proof i of Theorem 15.1 i h is similar to that h with Q and Q ˆ = diag qˆ1 , . . . , qˆm , replaced by Q = diag p1q|b1 1 | , . . . , pmq|bmm | and Q p1 |b1 | pm |bm | respectively. Note that the adaptive controller given in Theorem 15.3 does not

NNBook

September 1, 2009

435

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

destroy nonnegativity with respect to the plant states. In particular, the closed-loop system dynamics are given by (15.18). Now, it can be seen that if bi is negative, then ki (t) ≥ 0, t ≥ 0, and φi (t) ≤ 0, t ≥ 0, and hence, v(t) ≥≥ 0, t ≥ 0, and w(t) ≥≥ 0, t ≥ 0. Hence, by Proposition 15.2, x(t) ≥≥ 0, t ≥ 0.

15.3 Adaptive Control for Linear Nonnegative Dynamical Systems with Nonnegative Control As discussed in Section 15.1, control (source) inputs of drug delivery systems for physiological processes are usually constrained to be nonnegative as are the system states. Hence, in this section we develop adaptive control laws for essentially nonnegative systems with nonnegative control inputs. However, as noted in Section 13.2, since condition (15.7) is required to be n m satisfied for xe ∈ R+ and ue ∈ R+ , it follows from Brockett’s necessary condition for asymptotic stabilizability [193] that there does not exist a continuous stabilizing nonnegative feedback if 0 ∈ spec(A) and xe ∈ Rn+ . Hence, in this section, we assume that A is asymptotically stable, and hence, without loss of generality, by Proposition 2.7 we further assume that A is an asymptotically stable compartmental matrix. Thus, we proceed with the aforementioned assumptions to design adaptive controllers for uncertain compartmental systems that guarantee that limt→∞ xi (t) = xdi ≥ 0 for i = 1, . . . , m ≤ n, where xdi is a desired set point for the ith compartmental state while guaranteeing a nonnegative control input. Theorem 15.4. Consider the linear uncertain dynamical system G given by (15.1), where A is an asymptotically stable compartmental matrix, and B is nonnegative and given by (15.2). For a given xd ∈ Rm + assume there m such that (15.7) holds. Furthermore, and u ∈ R exist vectors xu ∈ Rn−m e + + let qi and qˆi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law ui (t) = max{0, uˆi (t)}, i = 1, . . . , m, (15.31) where u ˆi (t) = ki (t)(xi (t) − xdi ) + φi (t),

i = 1, . . . , m,

(15.32)

ki (t) ∈ R, t ≥ 0, and φi (t) ∈ R, t ≥ 0, i = 1, . . . , m, with update laws  0, if u ˆi (t) < 0, k˙ i (t) = −qi (xi (t) − xdi )2 , otherwise, ki (0) ≤ 0, i = 1, . . . , m, (15.33)  0, if φi (t) = 0 and xi (t) ≥ xdi , or if u ˆi (t) ≤ 0, φ˙ i (t) = −ˆ qi (xi (t) − xdi ), otherwise, (15.34) φi (0) ≥ 0, i = 1, . . . , m,

NNBook

September 1, 2009

436

CHAPTER 15

guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ), where Kg = diag[kg 1 , . . . , kg m ] ≤ 0, of the closed-loop system given by (15.1), (15.31), (15.33), and (15.34) is Lyapunov stable and xi (t) → xdi , i = 1, . . . , m, as n t → ∞ for all x0 ∈ R+ . Furthermore, u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0, n t ≥ 0, for all x0 ∈ R+ . Proof. First, define Ku [φu 1 (t), . . . , φu m ]T , where  0, kui (t) = ki (t),  0, φu i (t) = φi (t),

, diag[ku1 (t), . . . , kum (t)] and φu (t) , if u ˆi (t) < 0, otherwise,

i = 1, . . . , m,

(15.35)

if u ˆi (t) < 0, otherwise,

i = 1, . . . , m.

(15.36)

Now, note that with u(t), t ≥ 0, given by (15.31) it follows from (15.1) that x(t) ˙ = Ax(t) + BKu (t)(ˆ x(t) − xd ) + Bφu (t),

x(0) = x0 ,

t ≥ 0, (15.37)

or, equivalently, using (15.7), x(t) ˙ = A(x(t) − xe ) + BKu (t)(ˆ x(t) − xd ) + B(φu (t) − ue ), x(0) = x0 , t ≥ 0. (15.38) Furthermore, note that since A is essentially nonnegative and asymptotically stable, it follows from Theorem 2.12 that there exist a positive diagonal matrix P , diag[p1 , . . . , pn ] and a positive-definite matrix R ∈ Rn×n such that 0 = AT P + P A + R. (15.39) To show Lyapunov stability of the closed-loop system (15.33), (15.34), and (15.38) consider the Lyapunov function candidate V (x, K, φ) = (x − xe )T P (x − xe ) + tr(K − Kg )T Q−1 (K − Kg ) ˆ −1 (φ − ue ), +(φ − ue )T Q (15.40) or, equivalently, V (x, K, φ) =

n X i=1

where

pi (xi − xei )2 +

m X p i bi i=1

 q1 qm ,..., , Q = diag p 1 b1 p m bm 

qi

(ki − kg i )2 +

ˆ = diag Q



m X p i bi i=1

qˆi

(φi − uei )2 ,

 qˆ1 qˆm ,..., . p 1 b1 p m bm

ˆ are positive definite, Note that V (xe , Kg , ue ) = 0 and, since P , Q, and Q V (x, K, φ) > 0 for all (x, K, φ) 6= (xe , Kg , ue ). Furthermore, V (x, K, φ) is

NNBook

September 1, 2009

437

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

radially unbounded. Now, letting x(t), t ≥ 0, denote the solution to (15.38) and using (15.33) and (15.34), it follows that the Lyapunov derivative along the closedloop system trajectories is given by V˙ (x(t), K(t), φ(t)) = 2(x(t) − xe )T P [A(x(t) − xe ) + BKu (t)(ˆ x(t) − xd ) + B(φu (t) − ue )] T −1 ˙ ˙ ˆ −1 φ(t) +2tr(K(t) − Kg ) Q K(t) + 2(φ(t) − ue )T Q = −(x(t) − xe )T R(x(t) − xe ) + 2 +2

m X i=1

+2

i=1

pi bi kui (t)(xi (t) − xdi )2

pi bi (xi (t) − xdi )(φu i (t) − uei ) + 2

m X p i bi i=1

m X

qˆi

m X p i bi i=1

qi

(ki (t) − kg i )k˙ i (t)

(φi (t) − uei )φ˙ i (t)

= −(x(t) − xe )T R(x(t) − xe )   m X 1 2 ˙ +2 pi bi kui (t)(xi (t) − xdi ) + (ki (t) − kg i )ki (t) qi i=1   m X 1 ˙ +2 pi bi (xi (t) − xdi )(φu i (t) − uei ) + (φi (t) − uei )φi (t) . qˆi i=1

(15.41)

Now, for each i ∈ {1, . . . , m} and for the two cases given in (15.33) and (15.34), the last two terms on the right-hand side of (15.41) give: i ) If u ˆi (t) < 0, t ≥ 0, then kui (t) = 0, φu i (t) = 0, k˙ i (t) = 0 and φ˙ i (t) = 0. Furthermore, since φi (t) ≥ 0 and ki (t) ≤ 0 for all t ≥ 0, it follows from (15.32) that u ˆi (t) < 0 only if xi (t) > xdi , t ≥ 0, and hence, 1 (ki (t) − kg i )k˙ i (t) = 0, qi 1 (xi (t) − xdi )(φu i (t) − uei ) + (φi (t) − uei )φ˙ i (t) = −(xi (t) − xdi )uei qi ≤ 0. kui (t)(xi (t) − xdi )2 +

ii ) Otherwise, kui (t) = ki (t) and φu i (t) = φi (t), and hence, kui (t)(xi (t) − xdi )2 +

1 (ki (t) − kg i )k˙ i (t) = kg i (xi (t) − xdi )2 ≤ 0, qi

NNBook

September 1, 2009

438

CHAPTER 15

1 (xi (t) − xdi )(φu i (t) − uei ) + (φi (t) − uei )φ˙ i (t) qˆi  −(xi (t) − xdi )uei ≤ 0, if φi (t) = 0 and xi (t) ≥ xdi , = 0, otherwise. Hence, it follows that in either case V˙ (x(t), K(t), φ(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) ≤ 0,

t ≥ 0, (15.42)

which proves that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ) to (15.33), (15.34), and (15.38) is Lyapunov stable. Furthermore, since R > 0 it follows n from Theorem 4.2 of [112] that x(t) → xe as t → ∞ for all x0 ∈ R+ . Finally, u(t) ≥≥ 0, t ≥ 0, is a restatement of (15.31). Now, since B ≥≥ 0 and u(t) ≥≥ 0, t ≥ 0, it follows from Propositions 2.2 and 4.1 that n x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . As in the case of Theorem 15.1, the conditions in Theorem 15.4 imply that x(t) → xe as t → ∞, and hence, it follows from (15.33) and (15.34) that (x(t), K(t), φ(t)) → M , {(x, K, φ) ∈ Rn × Rm×m × Rn : x = xe , K˙ = 0, φ˙ = 0} as t → ∞. It is important to note that the adaptive control law (15.31), (15.33), and (15.34) does not require the explicit knowledge of the constant vector ue ; even though Theorem 15.4 requires the existence of xu ∈ Rn−m and + ue ∈ R m such that condition (15.7) holds. Furthermore, the control input + ui (t), t ≥ 0, is always nonnegative regardless of the values of xi (t), ki (t), and φi (t), t ≥ 0, i = 1, . . . , m, which ensures that the closed-loop plant states remain nonnegative by Proposition 4.1. Finally, it should be noted that since A is asymptotically stable, the adaptive gains ki (t), t ≥ 0, i = 1, . . . , m, only change the performance of the closed-loop system and do not destroy stability even when we set k˙ i (t) = 0, t ≥ 0, with ki (0) ≤ 0, i = 1, . . . , m.

15.4 Adaptive Control for General Anesthesia: Linear Model In this section, we apply Theorem 15.2 to the anesthetic model considered in Section 13.7. Recall that the three-compartment mammillary system shown in Figure 15.1 provides a pharmacokinetic model for a patient describing the distribution of propofol into the central compartment (identified with the intravascular blood volume as well as highly perfused organs) and the other various tissue groups of the body. A mass balance for the whole compartmental system yields x˙ 1 (t) = −(a11 + a21 + a31 )x1 (t) + a12 x2 (t) + a13 x3 (t) + u(t), x1 (0) = x10 , t ≥ 0, (15.43)

NNBook

September 1, 2009

439

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

u(t) ≡ Continuous infusion a12 x2 Compartment 2 a21 x1

Central Compartment

a31 x1 Compartment 3 a13 x3

a11 x1

Figure 15.1 Three-compartment mammillary model for disposition of propofol.

x˙ 2 (t) = a21 x1 (t) − a12 x2 (t), x˙ 3 (t) = a31 x1 (t) − a13 x3 (t),

x2 (0) = x20 , x3 (0) = x30 ,

(15.44) (15.45)

where x1 (t), x2 (t), x3 (t), t ≥ 0, are the masses in grams of propofol in the central compartment and Compartments 2 and 3, respectively, u(t), t ≥ 0, is the infusion rate in grams/min of the anesthetic (propofol) into the central compartment, aij ≥ 0, i 6= j, i, j = 1, 2, 3, are the rate constants in min−1 for drug transfer between compartments, and a11 ≥ 0 in min−1 is the rate constant for elimination from the central compartment. Even though these transfer and loss coefficients are nonnegative, they can be uncertain due to patient gender, weight, preexisting disease, age, and concomitant medication. Hence, adaptive control for propofol regulation during surgery can significantly improve the outcome for drug administration over manual control. Here, we assume that the transfer and loss coefficients a11 , a12 , a21 , a13 , and a31 are unknown and our objective is to regulate the propofol concentration level of the central compartment to the desired level of 4 µg/ml in the face of system uncertainty. Furthermore, since propofol mass in the blood plasma cannot be measured directly, we measure the concentration of propofol in the central compartment, that is, x1 /Vc , where Vc is the volume in liters of the central compartment. As noted in Section 13.7, Vc can be approximately calculated by Vc = (0.159 l/kg)(M kg), where M is the mass in kilograms of the patient. In our control design we assume M = 70 kg so that the desired level of propofol mass in the central compartment is given by xd1 = (4 µg/ml)(0.159 l/kg)(70 kg) = 44.52 mg. Next, note that (15.43)–(15.45) can be written in state space form (15.1) with x = [x1 , x2 , x3 ]T ,     −(a11 + a21 + a31 ) a12 a13 1 a21 −a12 0  , B =  0  . (15.46) A= a31 0 −a13 0

NNBook

September 1, 2009

440

CHAPTER 15

Table 15.1 Pharmacokinetic Parameters [98].

Set (min−1 ) A B

a11 0.152 0.119

a21 0.207 0.114

a12 0.092 0.055

a31 0.040 0.041

a13 0.0048 0.0033

Now, it can be shown that for xd1 /Vc = 4 µg/ml, all the conditions of Theorem 15.4 are satisfied. Hence, it follows from Theorem 15.4 that the adaptive dynamic feedback controller (15.31) with update laws (15.33) and (15.34) guarantees that x1 (t) → xd1 as t → ∞ for any (uncertain) nonnegative values of the transfer and loss coefficients. To illustrate the properties of the proposed adaptive control law, we use the average set of pharmacokinetic parameters given in [98] for 29 patients requiring general anesthesia for noncardiac surgery. For our design we switch from Set A to Set B given in Table 15.1 at t = 25 min. With q1 = 1000 g−2 min−2 , qˆ1 = 0.5 min−2 , and initial conditions x(0) = [0, 0, 0]T g, k1 (0) = 0 min−1 , and φ1 (0) = 0.01 g/min−1 , Figure 15.2 shows the masses of propofol in all three compartments versus time. Figure 15.3 shows the propofol concentration in the central compartment and the control signal (propofol infusion rate) versus time. Finally, Figure 15.4 shows the adaptive gain history versus time. In the above simulations, the adaptive controller was designed using a pharmacokinetic model (a model describing drug concentrations as a function of time and dose) for the disposition of propofol. As noted in Chapter 12, even though propofol concentration levels in the blood plasma are a good indication of the depth of anesthesia, they cannot be measured in real time during surgery. Furthermore, we are more interested in drug effect (depth of hypnosis) rather than drug concentration. Hence, we consider a more realistic model involving pharmacokinetics (drug concentration as a function of time) and pharmacodynamics (drug effect as a function of concentration) for control of anesthesia. Specifically, we use an electroencephalogram (EEG) signal as a measure of drug effect of anesthetic compounds on the brain [273]. Specifically, we augment an effectsite compartment to our model, as in Figure 13.2, and use a linearized BIS measurement for feedback control. Recall that the BIS signal is a nonlinear monotonically decreasing function of the depth of anesthesia and is given by   cγeff (t) , (15.47) BIS(ceff (t)) = BIS0 1 − γ ceff (t) + ECγ50 where BIS0 denotes the baseline (awake state) value and, by convention, is typically assigned a value of 100, ceff is the propofol concentration in grams/liter in the effect-site compartment (brain), EC50 is the concentration

NNBook

September 1, 2009

441

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS 100

x1(t) x (t) 2 x3(t)

90

Compartmental masses [mg]

80

70

60

50

40

30

20

10

0

0

5

10

15

20

25

30

35

40

Time [min]

Figure 15.2 Compartmental masses versus time. Consentration [µg/ml]

5 4 3 2 1 0

0

5

10

15

20

25

30

35

40

25

30

35

40

Control signal [mg/min]

Time [min] 50 40 30 20 10 0

0

5

10

15

20

Time [min]

Figure 15.3 Drug concentration in the central compartment and control signal (infusion rate) versus time.

at half maximal effect and represents the patient’s sensitivity to the drug, and γ determines the degree of nonlinearity in (15.47). In the following numerical simulation we set EC50 = 3.4 µg/ml, γ = 3,

NNBook

September 1, 2009

442 Adaptive gain K1 [min−1]

CHAPTER 15 0

−0.2 −0.4 −0.6 −0.8 −1

−1.2 −1.4

0

5

10

15

20

25

30

35

40

25

30

35

40

30 25 20

1

Adaptive gain φ [mg/min]

Time [min]

15 10 5 0

0

5

10

15

20

Time [min]

Figure 15.4 Adaptive gain history versus time.

and BIS0 = 100, so that the BIS signal is shown in Figure 15.5. The target (desired) BIS value, BIStarget , is set at 50. In this case, the linearized BIS function about the target BIS value is given by γcγ−1 γ eff BIS(ceff (t)) ≃ BIS(EC50 ) − BIS0 · EC50 · γ (ceff + ECγ50 )2 ceff =EC50

·(ceff (t) − EC50 ) = 125 − 22.06ceff (t).

(15.48)

Furthermore, for simplicity of exposition, we assume that the effect-site compartment equilibrates instantaneously with the central compartment, that is, we assume that aeff → ∞, so that (12.15) reduces to ceff (t) = x1 (t)/Vc , t ≥ 0. Now, using the adaptive feedback controller u1 (t) = max{0, uˆ1 (t)},

(15.49)

u ˆ1 (t) = −k1 (t)(BIS(ceff (t)) − BIStarget ) + φ1 (t),

(15.50)

where k1 (t) ∈ R, t ≥ 0, and φ1 (t) ∈ R, t ≥ 0, with update laws  0, if u ˆ1 (t) < 0, ˙k1 (t) = −qBIS1 (BIS(ceff (t)) − BIStarget )2 , otherwise, k1 (0) ≤ 0,

(15.51)

NNBook

September 1, 2009

443

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS 100

90

80

BIS Index [score]

70

60

Target BIS

50

40

30

← Linearized range

20

10

EC50 = 3.4 [µg/ml] 0

0

1

2

3

4

5

6

7

8

9

10

Effect site concentration [µg/ml]

Figure 15.5 BIS index versus effect-site concentration.

   

if φ1 (t) = 0 and BIS(c eff (t)) > BIStarget , φ˙ 1 (t) = or if u ˆ1 (t) ≤ 0,    otherwise, qˆBIS1 (BIS(ceff (t)) − BIStarget ), φ1 (0) ≥ 0, (15.52) 0,

where qBIS1 and qˆBIS1 are arbitrary positive constants, it follows from Theorem 15.4 that BIS(ceff (t)) → BIStarget as t → ∞ for any (uncertain) nonnegative values of the transfer and loss coefficients in the range of ceff where the linearized BIS equation (15.48) is valid. It is important to note that during actual surgery the BIS signal is obtained directly from the EEG and not (15.47). Furthermore, since our adaptive controller only requires the error signal BIS(ceff (t)) − BIStarget over the linearized range of (15.47), we do not require knowledge of the slope of the linearized equation (15.48), nor do we require knowledge of the parameters γ and EC50 . Once again, for our design we assume M = 70 kg and we switch from Set A to Set B given in Table 15.1 at t = 25 min. Furthermore, we assume that at t = 25 min the pharmacodynamic parameters EC50 and γ are switched from 3.4 µg/ml and 3 to 4.0 µg/ml and 2, respectively. Here we consider noncardiac surgery since cardiac surgery often utilizes hypothermia, which itself changes the BIS signal. With qBIS1 = 1 × 10−6 g/min2 , qˆBIS1 = 1 × 10−3 g/min2 , and initial conditions x(0) = [0, 0, 0]T g, k1 (0) = 0 g/min, and φ1 (0) = 0.01 g/min,

NNBook

September 1, 2009

444

CHAPTER 15 90

x (t) 1 x (t) 2 x (t)

80

3

Compartmental masses [mg]

70

60

50

40

30

20

10

0

0

5

10

15

20

25

30

35

40

Time [min]

Figure 15.6 Compartmental masses versus time.

Figure 15.6 shows the masses of propofol in all three compartments versus time. Figure 15.7 shows the BIS Index versus time. Figure 15.8 shows the propofol concentration in the central compartment and the control signal (propofol infusion rate) versus time. Finally, Figure 15.9 shows the adaptive gain history versus time.

15.5 Adaptive Control for Nonlinear Nonnegative Uncertain Dynamical Systems In this section, we extend the results of Section 15.3 to consider the problem of characterizing adaptive feedback control laws for nonlinear nonnegative and compartmental uncertain dynamical systems to achieve set-point regulation in the nonnegative orthant. Specifically, we consider the controlled nonlinear uncertain system G given by x(t) ˙ = f (x(t)) + G(x(t))u(t),

x(0) = x0 ,

t ≥ 0,

(15.53)

where x(t) ∈ Rn , t ≥ 0, is the state vector, u(t) ∈ Rm , t ≥ 0, is the control input, f : Rn → Rn is an unknown essentially nonnegative function and satisfies f (0) = 0, and G : Rn → Rn×m is an unknown nonnegative input matrix function. The control input u(·) in (15.53) is restricted to the class of admissible controls consisting of measurable and locally bounded functions such that u(t) ∈ Rm , t ≥ 0. Furthermore, for the nonlinear system G, we assume that the required properties for the existence and uniqueness of

NNBook

September 1, 2009

445

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS 100

90

80

BIS Index [score]

70

60

50

40

30

20

10

0

0

5

10

15

20

25

30

35

40

Time [min]

Figure 15.7 BIS index versus time.

solutions are satisfied, that is, f (·), G(·), and u(·) satisfy sufficient regularity conditions such that (15.53) has a unique solution forward in time. As discussed in Section 15.1, control (source) inputs of drug delivery systems for physiological processes are usually constrained to be nonnegative as are the system states. Hence, in this section we develop adaptive control laws for nonnegative systems with nonnegative control inputs. Specifically, n for a given desired set point xe ∈ R+ , our aim is to design a control input u(t), t ≥ 0, such that limt→∞ kx(t) − xe k = 0. We assume that control inputs are injected directly into m separate compartments and the input matrix function is given by   Bu Gn (x) G(x) = , (15.54) 0(n−m)×m where Bu = diag[b1 , . . . , bm ] is an unknown nonnegative diagonal matrix n and Gn = diag[gn 1 (x), . . . , gn m (x)], where gn i : R+ → R+ , i = 1, . . . , m, is a known positive diagonal matrix function. For compartmental systems this assumption is not restrictive since control inputs correspond to control inflows to each individual compartment. For the statement of the next result we assume that for a given xe ∈ n m R+ , there exists a nonnegative vector ue ∈ R+ such that ˆ e, 0 = f (xe ) + Bu

(15.55)

NNBook

September 1, 2009

446

CHAPTER 15

Consentration [µg/ml]

5 4 3 2 1 0

0

5

10

15

20

25

30

35

40

25

30

35

40

Control signal [mg/min]

Time [min] 80

60

40

20

0

0

5

10

15

20

Time [min]

Figure 15.8 Drug concentration in the central compartment and control signal (infusion rate) versus time.

ˆ = [Bu , 0m×(n−m) ]T , and the equilibrium point xe of (15.53) is where B n globally asymptotically stable for all x0 ∈ R+ with Gn (x(t))u(t) ≡ ue . Theorem 15.5. Consider the nonlinear uncertain system G given by (15.53) where f : Rn → Rn is essentially nonnegative and G : Rn → Rn×m is nonnegative and is given by (15.54). Assume there exist continuously differentiable functions Vsi : R → R, i = 1, . . . , m, and Vˆs : Rn−m → R, Lipschitz continuous functions Fi : Rn → Rsi , i = 1, . . . , m, and a continuous function ℓ : Rn → Rp such that Vs (·) is positive definite, radially unbounded, Vs (0) = 0, ℓ(0) = 0, Fi (0) = 0, i = 1, . . . , m, and, for all e ∈ Rn , Vs ′i (ηi )Fi (η) ≥≥ 0, i = 1, . . . , m, 0 = Vs′ (η)fe (η) + ℓT (η)ℓ(η),

(15.56) (15.57)

where Vs (η) = Vs1 (η1 ) + . . . + Vsm (ηm ) + Vˆs (ηm+1 , . . . , ηn ) and fe (η) , f (η + xe ) − f (xe ). Furthermore, let qi and qˆi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law ui (t) = max{0, uˆi (t)},

i = 1, . . . , m,

(15.58)

where −1 T u ˆi (t) = gn −1 i (x(t))ki (t)Fi (x(t) − xe ) + gn i (x(t))φi (t),

i = 1, . . . , m, (15.59) ki (t) ∈ Rsi , t ≥ 0, i = 1, . . . , m, and φi (t) ∈ R, t ≥ 0, i = 1, . . . , m, with

NNBook

September 1, 2009

447

Adaptive gain K1 [mg/min2]

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS 0

−0.5

−1

−1.5

0

5

10

15

20

25

30

35

40

25

30

35

40

40

30

1

Adaptive gain φ [mg/min2]

Time [min]

20

10

0

0

5

10

15

20

Time [min]

Figure 15.9 Adaptive gain history versus time.

update laws k˙ iT (t) =



φ˙ i (t) =

   0,

0, if u ˆi (t) < 0, − q2i Vs ′i (xi (t) − xei )FiT (x(t) − xe ), otherwise, ki (0) ≤≤ 0, i = 1, . . . , m,

(15.60)

if φi (t) = 0 and Vs ′i (xi (t) − xei ) ≥ 0, ˆi (t) ≤ 0, or if u

  − qˆi V ′ (x (t) − x ), ei 2 si i

otherwise,

φi (0) = 0, i = 1, . . . , m,

(15.61)

guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ), where K(t) , T (t)] and K , block-diag[k T , . . . , k T ] ≤≤ 0, of block-diag[k1T (t), . . . , km g gm g1 the closed-loop system given by (15.53), (15.58), (15.60), and (15.61) is Lyapunov stable. If, in addition, ℓT (η)ℓ(η) > 0, e ∈ Rn , η 6= 0, then n x(t) → xe as t → ∞ for all x0 ∈ R+ . Furthermore, u(t) ≥≥ 0, t ≥ 0, and n x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . T (η)]T , Proof. First, define e(t) , x(t) − xe , F (η) , [F1T (η), . . . , Fm T T Ku (t) , block-diag [ku1 (t), . . . , kum (t)], and φu (t) , [φu 1 (t), . . . , φu m (t)]T , where  ˆi (t) ≤ 0, 0, if u i = 1, . . . , m, (15.62) kui (t) = ki (t), otherwise,

NNBook

September 1, 2009

448

CHAPTER 15



φu i (t) =

0, φi (t),

ˆi (t) ≤ 0, if u otherwise,

i = 1, . . . , m.

(15.63)

Now, note that with u(t), t ≥ 0, given by (15.58) it follows from (15.53) that ˆ u (t)F (x(t)−xe )+ Bφ ˆ u (t), x(t) ˙ = f (x(t))+ BK

x(0) = x0 ,

t ≥ 0, (15.64)

or, equivalently, using (15.55), ˆ u (t)F (e(t)) + B(φ ˆ u (t) − ue ), e(0) = x0 − xe , t ≥ 0. e(t) ˙ = fe (e(t)) + BK (15.65) To show Lyapunov stability of the closed-loop system (15.60), (15.61), and (15.65) consider the Lyapunov function candidate ˆ −1 (φ − ue ), V (e, K, φ) = Vs (e) + tr(K − Kg )T Q−1 (K − Kg ) + (φ − ue )T Q (15.66) or, equivalently, V (e, K, φ) = Vs (e) +

m X bi i=1

qi

(ki − kg i )T (ki − kg i ) +

m X bi i=1

qˆi

(φi − uei )2 , (15.67)

where  q1 qm ,..., , Q = diag b1 bm 

  qˆ1 qˆm ˆ Q = diag ,..., . b1 bm

ˆ are positive definite, Note that V (0, Kg , ue ) = 0 and, since Vs (·), Q, and Q V (e, K, φ) > 0 for all (e, K, φ) 6= (0, Kg , ue ). Furthermore, V (e, K, φ) is radially unbounded. Now, letting e(t), t ≥ 0, denote the solution to (15.65) and using (15.60) and (15.61), it follows that the Lyapunov derivative along the closedloop system trajectories is given by V˙ (e(t), K(t), φ(t)) h i ˆ u (t)F (e(t)) + B(φ ˆ u (t) − ue ) = Vs′ (e(t)) fe (e(t)) + BK ˙ ˙ ˆ −1 φ(t) +2tr(K(t) − Kg )T Q−1 K(t) + 2(φ(t) − ue )T Q m X T = −ℓT (e(t))ℓ(e(t)) + Vs ′i (ei (t))bi kui (t)Fi (e(t)) i=1

+

m X

bi Vs ′i (ei (t))(φu i (t)

i=1

+

m X 2bi i=1

qˆi

− uei ) +

(φi (t) − uei )φ˙ i (t)

m X 2bi i=1

qi

k˙ iT (t)(ki (t) − kg i )

NNBook

September 1, 2009

449

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

m i h X 2 T = −ℓ (e(t))ℓ(e(t)) + bi Vs ′i (ei (t))kui (t)Fi (e(t)) + k˙ iT (t)(ki (t) − kg i ) qi T

i=1

m i h X 2 + bi Vs ′i (ei (t))(φu i (t) − uei ) + (φi (t) − uei )φ˙ i (t) . qˆi

(15.68)

i=1

For each i ∈ {1, . . . , m} and for the two cases given in (15.60) and (15.61), the last two terms on the right-hand side of (15.68) give: i ) If u ˆi (t) ≤ 0, t ≥ 0, then kui (t) = 0, φu i (t) = 0, k˙ i (t) = 0 and φ˙ i (t) = 0. Furthermore, since φi (t) ≥ 0 and ki (t) ≤≤ 0 for all t ≥ 0, it follows from (15.59) that u ˆi (t) ≤ 0 only if Fi (x(t) − xe ) ≥≥ 0, t ≥ 0, which implies Vs ′i (ei (t)) ≥ 0, t ≥ 0, by (15.56), and hence, Vs ′ (ei (t))kT (t)Fi (e(t)) + 2 k˙ T (t)(ki (t) − kg ) = 0, i

Vs ′i (ei (t))(φu i (t) −

qi i

ui

uei ) +

2 qˆi (φi (t)

i

− uei )φ˙ i (t) = −Vs ′i (ei (t))uei ≤ 0.

ii ) Otherwise, kui (t) = ki (t) and φu i (t) = φi (t), and hence, 2 ˙T T k (t)(ki (t) − kg i ) = Vs ′i (ei (t))kgi (t)Fi (e(t)) qi i ≤ 0, 2 Vs ′i (ei (t))(φu i (t) − uei ) + (φi (t) − uei )φ˙ i (t) qˆi  if φi (t) = 0 and  −Vs ′i (ei (t))uei ≤ 0, Vs ′i (xi (t) − xei ) ≥ 0, =  0, otherwise.

T Vs ′i (ei (t))kui (t)Fi (e(t)) +

Hence, it follows that in either case

V˙ (e(t), K(t), φ(t)) ≤ −ℓT (e(t))ℓ(e(t)) ≤ 0,

t ≥ 0,

(15.69)

which proves that the solution (e(t), K(t), φ(t)) ≡ (0, Kg , ue ) to (15.60), (15.61), and (15.65) is Lyapunov stable. Thus, the solutions of the closed-loop system (15.60), (15.61), and (15.65) are bounded in Rn × Rm×s × Rm , and hence, since Vsi (·) is continuously differentiable and Fi (·) is Lipschitz continuous for i = 1, . . . , m, it follows from Corollary 2.5 of [112] that there exists a unique solution to (15.60), (15.61), and (15.65) that is defined for all t ≥ 0. Furthermore, it follows from Theorem 4.2 of [112] that ℓ(e(t)) → 0 as t → ∞. If, in addition, ℓT (e)ℓ(e) > 0, e ∈ Rn , e 6= 0, then x(t) → xe as t → ∞ for all n x0 ∈ R+ . Finally, u(t) ≥≥ 0, t ≥ 0, is a restatement of (15.58). Now, n since f : Rn → Rn is essentially nonnegative, G(x) ≥≥ 0, x ∈ R+ , and

NNBook

September 1, 2009

450

CHAPTER 15

u(t) ≥≥ 0, t ≥ 0, it follows from Proposition 4.3 that x(t) ≥≥ 0, t ≥ 0, for n all x0 ∈ R+ . In Theorem 15.5 the control input u(t), t ≥ 0, is always nonnegative regardless of the values of xi (t), ki (t), and φi (t), t ≥ 0, i = 1, . . . , m, which ensures that the closed-loop plant states remain nonnegative by Proposition 4.3 for nonnegative and compartmental dynamical systems. Furthermore, note that in Theorem 15.5 we assumed that the equilibrium point xe of (15.53) is globally asymptotically stable with Gn (x(t))u(t) ≡ ue . In general, however, unlike linear nonnegative systems with asymptotically stable plant n dynamics, a given set point xe ∈ R+ for the nonlinear nonnegative dynamical system (15.53) may not be asymptotically stabilizable with a constant m control Gn (x(t))u(t) ≡ ue ∈ R+ . However, if f (x) is homogeneous, (x) is essentially nonnegative for cooperative, that is, the Jacobian matrix ∂f∂x n n ∂f (x) all x ∈ R+ [278], the Jacobian matrix ∂x is irreducible for all x ∈ R+ [278], and the zero solution x(t) ≡ 0 of the undisturbed (u(t) ≡ 0) system (15.53) is globally asymptotically stable, then the set point xe ∈ Rn+ satisfying (15.55) is a unique equilibrium point with Gn (x(t))u(t) ≡ ue ∈ Rm + and is also n asymptotically stable for all x0 ∈ R+ [192]. This implies that the solution x(t) ≡ xe to (15.53) with Gn (x(t))u(t) ≡ ue is asymptotically stable. Finally, we note that if the equilibrium point xe of (15.53) is locally asymptotically n stable for all x0 ∈ D ⊂ R+ with Gn (x(t))u(t) ≡ ue , then Theorem 15.5 guarantees local asymptotic stability. It is important to note that the adaptive control law (15.58), (15.60), and (15.61) does not require the explicit knowledge of the nonnegative vector ue ; all that is required is the existence of the nonnegative constant vector ue and a partially component decoupled Lyapunov function Vs (e) such that (15.56) and (15.57) are satisfied and the equilibrium condition (15.55) holds. Furthermore, note that in the case where F (e) is only a function of eˆ , [e1 , . . . , em ]T , the adaptive feedback controller given in Theorem 15.5 can be viewed as an adaptive output feedback controller with outputs y = Cx, where C , [Im , 0m×(n−m) ]. In this case, it follows from (15.58) that the explicit knowledge of xu , [xm+1 , . . . , xn ]T and xeu = [xem+1 , . . . , xen ]T as well as ue ∈ Rm is not required. If f (·) in (15.53) is given by a linear function, that is, f (x) = Ax, where A ∈ Rn×n is essentially nonnegative and asymptotically stable, then fe (·) is given by fe (e) = Ae. In this case, it follows from Theorem 2.12 that there exist a positive diagonal matrix P ∈ Rn×n and a positive-definite matrix R ∈ Rn×n such that 0 = AT P + P A + R,

(15.70)

and hence, we can always construct a component decoupled function Vs (e) =

NNBook

September 1, 2009

451

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

eT P e which satisfies (15.57). Furthermore, in this case, we can always construct functions Fi (·), i = 1, . . . , m, such that (15.56) holds. Unlike linear asymptotically stable nonnegative systems, the existence of a component decoupled Lyapunov function is not necessarily guaranteed for nonlinear asymptotically stable nonnegative systems. Even though the existence of diagonal-type Lyapunov functions for asymptotically stable nonlinear nonnegative systems is not ensured, there do exist classes of nonnegative dynamical systems that do admit component decoupled Lyapunov functions. For details, see [166].

15.6 Adaptive Control for General Anesthesia: Nonlinear Model In this section, we apply Theorem 15.5 to the nonlinear anesthetic model considered in Section 12.13. Recall that a mass balance of the three-state compartmental model shown in Figure 15.10 yields x˙ 1 (t) = −[a11 (c(t)) + a21 (c(t)) + a31 (c(t))]x1 (t) + a12 (c(t))x2 (t) +a13 (c(t))x3 (t) + u(t), x1 (0) = x10 , t ≥ 0, (15.71) x˙ 2 (t) = a21 (c(t))x1 (t) − a12 (c(t))x2 (t), x2 (0) = x20 , (15.72) x˙ 3 (t) = a31 (c(t))x1 (t) − a13 (c(t))x3 (t), x3 (0) = x30 , (15.73) where c(t) = x1 (t)/Vc , Vc is the volume of the central compartment, aij (c), i 6= j, is the rate of transfer of drug from the jth compartment to the ith compartment, a11 (c) is the rate of drug metabolism and elimination (metabolism typically occurs in the liver), and u(t), t ≥ 0, is the infusion rate of the anesthetic drug propofol into the central compartment. The transfer coefficients are assumed to be functions of the drug concentration c since it is well known that the pharmacokinetics of propofol are influenced by cardiac output [289] and, in turn, cardiac output is influenced by propofol plasma concentrations, due to both venodilation (pooling of blood in dilated veins) and myocardial depression (decrease in cardiac output) [226]. As discussed in Section 12.13, experimental data indicates that the transfer coefficients should be nonincreasing functions of the propofol concentration [226]. Application of the Hill equation to the relationship between transfer coefficients and drug concentration gives aij (c) = Aij Qij (c),

α

α

ij ij Qij (c) = Q0 C50,ij /(C50,ij + cαij ),

(15.74)

where, for i, j ∈ {1, 2, 3}, i 6= j, C50,ij is the drug concentration associated with a 50% decrease in the transfer coefficient, αij is a parameter that determines the steepness of the concentration-effect relationship, and Aij are positive constants. Furthermore, since for many drugs the rate of metabolism a11 (c) is proportional to the rate of transport of drug to the

NNBook

September 1, 2009

452

CHAPTER 15

u ≡ Continuous infusion a12 (c)x2

a31 (c)x1 Central Compartment

Compartment II a21 (c)x1

Compartment III a13 (c)x3

a11 (c)x1 ≡ Elimination Figure 15.10 Pharmacokinetic model for drug distribution during anesthesia.

liver we assume that a11 (c) is also proportional to cardiac output so that a11 (c) = A11 Q11 (c). For simplicity of exposition and to provide a nonlinear model to illustrate implementation of our adaptive controller, we will assume that C50 and α are independent of i and j. Also, since decreases in cardiac output are observed at clinically utilized propofol concentrations, we will arbitrarily assign C50 a value of 4 µg/ml since this value is in the mid-range of clinically utilized values. We will also arbitrarily assign α a value of 3 [169]. This value is within the typical range of those observed for ligand-receptor binding (see the discussion in [73]). Note that these assumptions on C50 and α (both the independence from i and j and the assumed values) are done to provide a numerical framework for simulation. Even if these assumptions are incorrect, the basic Hill equations relating the transfer coefficients to propofol concentration are consistent with standard pharmacodynamic modeling. Even though the transfer and loss coefficients A12 , A21 , A13 , A31 , and A11 are positive, and α > 1, C50 > 0, and Q0 > 0, these parameters can be uncertain due to patient gender, weight, preexisting disease, age, and concomitant medication. Hence, adaptive control to regulate intravenous anesthetics during surgery is essential. For set-point regulation define e(t) , x(t) − xe , where xe ∈ R3 is the set point satisfying the equilibrium condition for (15.71)–(15.73) with x1 (t) ≡ xe1 , x2 (t) ≡ xe2 , x3 (t) ≡ xe3 , and u(t) ≡ ue , so that fe (e) = [fe 1 (e), fe 2 (e), fe 3 (e)]T is given by fe 1 (e) = −[ae (c) + a21 (c) + a31 (c)](e1 + xe1 ) + a12 (c)(e2 + xe2 ) +a13 (c)(e3 + xe3 ) − [ae (ce ) + a21 (ce ) + a31 (ce )]xe1 +a12 (ce )xe2 + a13 (ce )xe3 , (15.75) fe 2 (e) = a21 (c)(e1 + xe1 ) − a12 (c)(e2 + xe2 ) −[a21 (ce )xe1 − a12 (ce )xe2 ], (15.76)

NNBook

September 1, 2009

453

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

fe 3 (e) = a31 (c)(e1 + xe1 ) − a13 (c)(e3 + xe3 ) −[a31 (ce )xe1 − a13 (ce )xe3 ],

(15.77)

where ce , xe1 /Vc . The existence of this equilibrium point follows from the fact that the Jacobian matrix of (15.71)–(15.73) is essentially nonnegative and every solution of (15.71)–(15.73) is bounded. See Theorem 9 of [158] for details. Furthermore, let F (e) = e1 and Vs (e) = e21 + p2 e22 + p3 e23 , where p2 , p3 > 0, so that Vs ′1 (e)F (e) = 2e21 ≥ 0. Next, by linearizing fe (e) about 0 and computing the eigenvalues of the resulting Jacobian matrix, it can be shown that xe is asymptotically stable. Since we establish local asymptotic stability of xe , our results guarantee local asymptotic stabilizability. To consider drug effect rather than drug concentration, we use a linearized BIS measurement predicted on an effect-site compartment as in Section 15.4. In the following numerical simulation we set EC50 = 5.6 µg/ml, γ = 2.39, and BIS0 = 100, so that the BIS signal is shown in Figure 15.11. The target (desired) BIS value, BIStarget , is set at 50. In this case, the linearized BIS function about the target BIS value is given by γcγ−1 γ eff BIS(ceff ) ≃ BIS(EC50 ) − BIS0 · EC50 · γ (ceff + ECγ50 )2 ceff =EC50

·(ceff − EC50 ) = 109.75 − 10.67ceff .

(15.78)

Furthermore, for simplicity of exposition, we assume that the effect-site compartment equilibrates instantaneously with the central compartment; that is, we assume that ceff (t) = c(t) for all t ≥ 0. Now, using the adaptive feedback controller (15.58) with i = 1, F1 (x(t) − xe ) = BIS(t) − BIStarget , q1 = qBIS , and qˆ1 = qˆBIS , where qBIS1 and qˆBIS1 are positive constants, it follows from Theorem 15.5 that BIS(t) → BIStarget as t → ∞ for all (uncertain) nonnegative values of the pharmacokinetic transfer and loss coefficients (A12 , A21 , A13 , A31 , A11 ) as well as all (uncertain) nonnegative coefficients α, C50 , and Q0 in the range of ceff where the linearized BIS equation (15.78) is valid. Since our adaptive controller only requires the error signal BIS(t) − BIStarget over the linearized range of (15.47), we do not require knowledge of the slope of the linearized equation (15.78), nor do we require knowledge of the pharmacodynamic parameters γ and EC50 . For our simulation we assume Vc = (0.228 l/kg)(M kg), where M = 70 kg is the mass of the patient, A21 Q0 = 0.112 min−1 , A12 Q0 = 0.055 min−1 , A31 Q0 = 0.0419 min−1 , A13 Q0 = 0.0033 min−1 , A11 Q0 = 0.119 min−1 , α = 3, and C50 = 4 µg/ml [169, 215]. Note that the parameter values for α and C50 probably exaggerate the effect of propofol on cardiac output.

NNBook

September 1, 2009

454

CHAPTER 15 100

90

80

BIS index [score]

70

60

Target BIS

50

40

30

← Linearized range

20

10

EC50 = 5.6 [µg/ml] 0

0

2

4

6

8

10

12

14

16

18

20

Effect site concentration [µg/ml]

Figure 15.11 BIS index versus effect-site concentration.

They have been selected to accentuate nonlinearity but they are not biologically unrealistic. Furthermore, to illustrate the efficacy of the proposed adaptive controller we switch the pharmacodynamic parameters EC50 and γ, respectively, from 5.6 µg/ml and 2.39 to 7.2 µg/ml and 3.39 at t = 15 min and back to 5.6 µg/ml and 2.39 at t = 30 min. With qBIS1 = 2 × 10−6 g/min2 , qˆBIS1 = 3 × 10−4 g/min2 , and initial conditions x(0) = [0, 0, 0]T g, k1 (0) = 0 g/min, and φ1 (0) = 0.01 g/min, Figure 15.12 shows the masses of propofol in the three compartments versus time. Finally, Figure 15.13 shows the BIS index and the control signal (propofol infusion rate) versus time.

15.7 Adaptive Control for Nonlinear Nonnegative Uncertain Dynamical Systems In this section, we consider the problem of characterizing adaptive feedback control laws for nonlinear nonnegative and compartmental uncertain dynamical systems to achieve set-point regulation in the nonnegative orthant without a nonnegativity constraint on the control input. Specifically, consider the following controlled nonlinear uncertain system G given by x(t) ˙ = f (x(t)) + G(x(t))u(t),

x(0) = x0 ,

t ≥ 0,

(15.79)

where x(t) ∈ Rn , t ≥ 0, is the state vector, u(t) ∈ Rm , t ≥ 0, is the control input, f : Rn → Rn is an unknown essentially nonnegative function and

NNBook

September 1, 2009

455

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS 120

Compartmental masses [mg]

100

x1(t) x2(t) x (t) 3

80

60

40

20

0

0

5

10

15

20

25

30

35

40

Time [min]

Figure 15.12 Compartmental masses versus time.

BIS Index [score]

100 80 60 40 20 0

0

5

10

15

20

25

30

35

40

25

30

35

40

Time [min]

Control signal [mg/min]

150

100

50

0

0

5

10

15

20

Time [min]

Figure 15.13 BIS index versus time and control signal (infusion rate) versus time.

satisfies f (0) = 0, and G : Rn → Rn×m is an unknown input matrix function. The control input u(·) in (15.79) is restricted to the class of admissible controls consisting of measurable functions such that u(t) ∈ Rm , t ≥ 0.

NNBook

September 1, 2009

456

CHAPTER 15

Here, we design adaptive controllers that guarantee that the controlled system states remain in the nonnegative orthant and converge to a desired n equilibrium state. Specifically, for a given desired set point xe ∈ R+ , our aim is to design a control input u(t), t ≥ 0, such that limt→∞ kx(t) − xe k = 0. We assume that we have m control inputs and the input matrix function is given by   Bu Gn (x) G(x) = , (15.80) 0(n−m)×m

where Bu = diag[b1 , . . . , bm ] is an unknown diagonal matrix and Gn : Rn → Rm×m is a known nonnegative matrix function such that det Gn (x) 6= 0, x ∈ Rn . Furthermore, for the nonlinear system G we assume that the required properties for the existence and uniqueness of solutions are satisfied, that is, f (·), G(·), and u(·) satisfy sufficient regularity conditions such that (15.79) has a unique solution forward in time. For the main result of this section, the following proposition is needed. For this result, we consider the time-varying system x(t) ˙ = f (t, x(t)) + G(x(t))u(t),

x(t0 ) = x0 ,

t ≥ t0 ,

(15.81)

where x(t) ∈ Rn , t ≥ t0 , u(t) ∈ Rm , t ≥ t0 , f : [t0 , ∞) × Rn → Rn and satisfies f (t, 0) = 0, t ≥ t0 , and G : Rn → Rn×m . Proposition 15.3. Consider the time-varying dynamical system (15.81) where f (t, ·) : Rn → Rn is Lipschitz continuous on Rn for all t ∈ [t0 , ∞) and f (·, x) : [t0 , ∞) → Rn is piecewise continuous on [t0 , ∞) for all x ∈ Rn . If for every t ∈ [t0 , ∞), f (t, ·) : Rn → Rn is essentially nonnegative and G : Rn → Rn×m is nonnegative, then the solution x(t), t ≥ t0 , to (15.81) is n nonnegative for all x0 ∈ R+ . Proof. The proof is similar to the proofs of Propositions 2.2 and 4.3 by equivalently representing the time-varying system (15.81) as an autonomous nonlinear system by appending an additional state to represent time. Theorem 15.6. Consider the nonlinear uncertain system G given by (15.79) where f : Rn → Rn is essentially nonnegative and G : Rn → Rn×m is n given by (15.80). For a given xe ∈ R+ , assume there exists a vector ue ∈ Rm such that f (xe ) ≤≤ 0 and ˆ e, 0 = f (xe ) + Bu

(15.82)

ˆ , [Bu , 0m×(n−m) ]T . Furthermore, assume that bi is unknown but where B sgn bi is known for all i = 1, . . . , m, and assume there exist a rectangular T , . . . , k T ], where k ∈ Rsi is such block-diagonal matrix Kg , block-diag[kg1 gi gm that (sgn bi )kgi ≤≤ 0, i = 1, . . . , m, continuously differentiable functions Vsi : R → R, i = 1, . . . , m, and Vˆs : Rn−m → R, and continuous functions

NNBook

September 1, 2009

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

457

ℓ : Rn → Rp and Fi : Rn → Rsi , i = 1, . . . , m, with Fi (x − xe ) ≤≤ 0 whenever xi = 0 and Fi (0) = 0, i = 1, . . . , m, such that Vs (·) is positive definite, radially unbounded, Vs (0) = 0, ℓ(0) = 0, and, for all e ∈ Rn , Vs ′i (ηi )Fi (η) ≥≥ 0, i = 1, . . . , m, ˆ g F (η)] + ℓT (η)ℓ(η), 0 = V ′ (η)[fe (η) + BK s

(15.83) (15.84)

where Vs (η) = Vs1 (η1 ) + · · · + Vsm (ηm ) + Vˆs (ηm+1 , . . . , ηn ),

(15.85)

−1 u(t) = G−1 n (x(t))K(t)F (x(t) − xe ) + Gn (x(t))φ(t),

(15.86)

T (η)]T . Finally, let fe (η) , f (η + xe ) − f (xe ), and F (η) , [F1T (η), . . . , Fm qi and qˆi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law

T (t)], k (t) ∈ Rsi , i = 1, . . . , m, t ≥ 0, where K(t) , block-diag[k1T (t), . . . , km i m and φ(t) ∈ R , t ≥ 0, with update laws

k˙ iT (t) = −(sgn bi ) q2i Vs ′i (xi (t) − xei )FiT (x(t) − xe ), i = 1, . . . , m, (15.87)  if φi (t) = 0 and   0, Vs ′i (xi (t) − xei ) ≥ 0, φ˙ i (t) =   −(sgn bi ) qˆ2i Vs ′i (xi (t) − xei ), otherwise, i = 1, . . . , m, (15.88)

where ki (0) and φi (0) are such that (sgn bi )ki (0) ≤≤ 0 and (sgn bi )φi (0) ≥ 0, i = 1, . . . , m, guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , Kg , ue ) of the closed-loop system given by (15.79), (15.86)–(15.88) is Lyapunov stable. If, in addition, ℓT (η)ℓ(η) > 0, η ∈ Rn , η 6= 0, then x(t) → xe as t → ∞ for n n all x0 ∈ R+ . Furthermore, x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . Proof. Let e(t) , x(t) − xe and note that with u(t), t ≥ 0, given by (15.86) it follows from (15.79) that x(t) ˙ = f (x(t)) + G(x(t))G−1 n (x(t))K(t)F (x(t) − xe ) −1 +G(x(t))Gn (x(t))φ(t), x(0) = x0 , t ≥ 0,

(15.89)

or, equivalently, using (15.80) and (15.82), ˆ g F (e(t)) e(t) ˙ = fe (e(t)) + f (xe ) + BK ˆ ˆ +B(K(t) − Kg )F (x(t) − xe ) + Bφ(t)

ˆ ˆ = fs (e(t)) + B(K(t) − Kg )F (x(t) − xe ) + B(φ(t) − ue ), e(0) = x0 − xe , t ≥ 0, (15.90)

ˆ g F (η). where fs (η) , fe (η) + BK

NNBook

September 1, 2009

458

CHAPTER 15

To show Lyapunov stability of the closed-loop system (15.87), (15.88), and (15.90) consider the Lyapunov function candidate ˆ −1 (φ − ue ), V (e, K, φ) = Vs (e) + tr(K − Kg )T Q−1 (K − Kg ) + (φ − ue )T Q (15.91) or, equivalently, V (e, K, φ) = Vs (e) +

m X |bi | i=1

qi

T

(ki − kg i ) (ki − kg i ) +

m X |bi | i=1

qˆi

(φi − uei )2 ,

where hq qm i 1 ,..., , Q = diag |b1 | |bm |

  qˆ1 qˆm ˆ ,..., Q = diag . |b1 | |bm |

ˆ are positive definite, Note that V (0, Kg , ue ) = 0 and, since Vs (·), Q, and Q V (e, K, φ) > 0 for all (e, K, φ) 6= (0, Kg , ue ). Furthermore, V (e, K, φ) is radially unbounded. Now, letting e(t), t ≥ 0, denote the solution to (15.90) and using (15.87) and (15.88), it follows that the Lyapunov derivative along the closedloop system trajectories is given by V˙ (e(t), K(t), φ(t)) h i ˆ ˆ = Vs′ (e(t)) fs (e(t)) + B(K(t) − Kg )F (x(t) − xe ) + B(φ(t) − ue ) +

m X 2|bi | i=1

qi

k˙ iT (t)(ki (t) − kg i ) +

= −ℓT (e(t))ℓ(e(t)) + +

m X i=1

+

T

i=1

qˆi

i=1

qˆi

(φi (t) − uei )φ˙ i (t)

Vs ′i (ei (t))bi (ki (t) − kg i )T Fi (e(t))

bi Vs ′i (ei (t))(φi (t) −

m X 2|bi | i=1

m X

m X 2|bi |

uei ) +

m X 2|bi |

qi

i=1

k˙ iT (t)(ki (t) − kg i )

(φi (t) − uei )φ˙ i (t)

= −ℓ (e(t))ℓ(e(t)) +

m X i=1

(φi (t) − uei )



bi Vs ′i (ei (t))

 2|bi | ˙ φi (t) , + qˆi (15.92)

where in (15.92) we used |bi |(sgn bi ) = bi . Now, for each i ∈ {1, . . . , m} and for the two cases given in (15.88), the last term on the right-hand side of (15.92) gives:

NNBook

September 1, 2009

459

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

i ) If φi (t) = 0 and Vs ′i (xi (t) − xei ) ≥ 0, t ≥ 0, then φ˙ i (t) = 0, and hence, since, using (15.82), bi uei ≥ 0, it follows that   2|bi | ˙ ′ φi (t) = −bi uei Vs ′i (xi (t) − xei ) ≤ 0. (φi (t) − uei ) bi Vs i (ei (t)) + qˆi ii ) Otherwise, φ˙ i (t) = −(sgn bi ) qˆ2i Vs′ i (xi (t) − xei ), and hence,   2|bi | ˙ ′ φi (t) = 0. (φi (t) − uei ) bi Vs i (ei (t)) + qˆi Hence, it follows that in either case V˙ (e(t), K(t), φ(t)) ≤ −ℓT (e(t))ℓ(e(t)) ≤ 0,

t ≥ 0,

(15.93)

which proves that the solution (e(t), K(t), φ(t)) ≡ (0, Kg , ue ) to (15.87), (15.88), and (15.90) is Lyapunov stable. Furthermore, it follows from Theorem 4.2 of [112] that ℓ(e(t)) → 0 as t → ∞. If, in addition, n ℓT (η)ℓ(η) > 0, e ∈ Rn , η 6= 0, then x(t) → xe as t → ∞ for all x0 ∈ R+ . n

Finally, to show that x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ note that the closed-loop system (15.79), (15.86)–(15.88) is given by ˆ ˆ x(t) ˙ = f (x(t)) + BK(t)F (x(t) − xe ) + Bφ(t) ˆ ˆ = f˜(t, x(t)) + BK(t) F˜ (x(t) − xe ) + Bφ(t) = f˜(t, x(t)) + v(t) + w(t), x(0) = x0 , t ≥ 0,

(15.94)

T (x−x )]T , F ˜i (x−xe ) , Fi (x−xe )|xi =0 , where F˜ (x−xe ) , [F˜1T (x−xe ), . . . , F˜m e i = 1, . . . , m,

ˆ f˜(t, x) , f (x) + BK(t)[F (x − xe ) − F˜ (x − xe )],     b1 k1T (t)F˜1 (x(t) − xe ) b1 φ1 (t)   .. ..     . .        bm kT (t)F˜m (x(t) − xe )  b φ (t)   m m m  , w(t) ,  v(t) ,  .   0   0       . ..   ..   . 0 0

(15.95)

(15.96)

Now, since, by (15.83), (15.87), and (15.88), (sgn bi )kiT (t) ≤≤ 0, t ≥ 0, i = 1, . . . , m, and (sgn bi )φi (t) ≥ 0, t ≥ 0, i = 1, . . . , m, and since F˜i (x(t) − xe ) ≤≤ 0, t ≥ 0, i = 1, . . . , m, it follows that for every t ∈ [0, ∞), f˜(t, x(t)) is essentially nonnegative, v(t) ≥≥ 0, and w(t) ≥≥ 0. Hence, it follows from n Proposition 15.3 that x(t) ≥≥ 0, t ≥ 0, for all x0 ∈ R+ . In the case where F : Rm → Rs is only a function of eˆ , [e1 , . . . , em ]T ,

NNBook

September 1, 2009

460

CHAPTER 15

the adaptive feedback controller given in Theorem 15.6 can be viewed as an adaptive output feedback controller with outputs y = Cx, where ˆ 0m×(n−m) ] and Cˆ , diag[sgn b1 , . . . , sgn bm ]. In this case, it C , [C, follows from (15.86) that the explicit knowledge of xu , [xm+1 , . . . , xn ]T and xeu = [xem+1 , . . . , xen ]T as well as ue ∈ Rm is not required. In addition, if f (·) in (15.79) is such that fe (·) is continuously differentiable, fe (0) = 0, and fe (e) is given by   A11 + A11 (ˆ e) A12 fe (e) = e, (15.97) A21 A22 where A11 : Rm → Rm×m is a continuous and essentially nonnegative, A11 ∈ Rm×m is essentially nonnegative, A12 ∈ Rm×(n−m) is nonnegative, A21 ∈ R(n−m)×m is nonnegative, and A22 ∈ R(n−m)×(n−m) is essentially nonnegative, and if (15.79) is stabilizable and feedback linearizable, then there always exists a rectangular block-diagonal matrix Kg ∈ Rm×s such that (15.84) holds. Furthermore, in this case Vs (·) need not be known.

T To see this, let A11 (ˆ e)ˆ e be parameterized as A11 (ˆ e)ˆ e = [θ1T Fˆ1 (ˆ e), . . . , θm T (·)]T is a known function such that e)]T , where Fˆ (·) , [Fˆ1T (·), . . . , Fˆm Fˆm (ˆ Fˆi : Rm → Rsˆi satisfies Fˆi (ˆ x−x ˆe ) ≤≤ 0 whenever xi = 0 and Fˆi (0) = 0, i = 1, . . . , m, x ˆe , [xe1 , . . . , xem ]T , and θi ∈ Rsˆi , i = 1, . . . , m, are unknown constant parameters such that (sgn bi )θi ≥≥ 0, i = 1, . . . , m. Now, by viewing eˆ = Ce as an output, the zero dynamics of

e(t) ˙ = fe (e(t)) + G(e(t) + xe )u(t),

e(0) = e0 ,

t ≥ 0,

(15.98)

with fe (e) given by (15.97) are given by z(t) ˙ = A22 z(t),

z(0) = z0 ,

t ≥ 0,

(15.99)

ˆ = (C B) ˆ T > 0, it follows from where z , [em+1 , . . . , em ]T . Since C B Theorem 2 of [86] (see also [85, 87, 161]) that if A22 is asymptotically stable, then there exist matrices P ∈ Rn×n , L ∈ Rn×p , and Φ = αIm , where α < 0, and a positive constant ε such that P is positive definite and satisfies T ˆ ˆ 0 = (Aˆ + BΦC) P + P (Aˆ + BΦC) + εP + LT L, ˆ T P − C, 0 = B

where Aˆ is given by

Aˆ ,



A11 A12 A21 A22



,

(15.100) (15.101) (15.102)

ˆ and thus, Aˆ + BΦC is asymptotically stable. Note that it follows from (15.101) that P has the form P = block-diag[P1 , P2 ], where P1 , diag[p1 , . . . , pm ] ∈ Rm×m and P2 ∈ T = [−θ T /b , α] ≤≤ 0, i = 1, . . . , m, and R(n−m)×(n−m) . Now, with kgi i i

NNBook

September 1, 2009

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

461

e), ei ]T , it follows that Fi (e) = [FˆiT (ˆ ˆ g F (ˆ fs (e) = fe (e) + BK e)   A11 + A11 (ˆ e) A12 = e A21 A22   −θ1T Fˆ1 (ˆ e)/b1 + (sgn b1 )αe1   Bu   .. +  . 0(n−m)×m  TF ˆm (ˆ −θm e)/bm + (sgn bm )αem     A11 + A11 (ˆ e) A12 A11 (ˆ e) 0 = e− e A21 A22 0 0   (sgn b1 )αe1   Bu   .. +  . 0(n−m)×m  (sgn bm )αem ˆ (15.103) = (Aˆ + BΦC)e. In this case, with Vs (e) = eT P e, the adaptive feedback controller (15.86) with update laws (15.87) and (15.88), or, equivalently, k˙ iT (t) = −(sgn bi )qi (xi (t) − xei )FiT (ˆ x(t) − x ˆe ), i = 1, . . . , m, (15.104)  0, if φi (t) = 0 and xi (t) ≥ xei , φ˙ i (t) = qi (xi (t) − xei ), otherwise, −(sgn bi )ˆ (15.105) i = 1, . . . , m, where ki (0) and φi (0) are such that (sgn bi )ki (0) ≤≤ 0 and (sgn bi )φi (0) ≥ 0, i = 1, . . . , m, with qi and qˆi in (15.87) and (15.88) replaced by pqii and qˆi pi ,

respectively, guarantees global asymptotic stability of the nonlinear uncertain dynamical system (15.79) with f (x) = fe (e) + f (xe ), where fe (e) satisfies (15.97). It is important to note that the adaptive feedback controller (15.86) with update laws (15.104) and (15.105) does not require knowledge of the system dynamics (15.97). All that is required is that A22 in (15.97) be asymptotically stable. Finally, in the case where A11 (e) = 0 and Gn (x) = Im , we can simply take F (e) = eˆ. In this case, the adaptive feedback controller (15.86) with update laws (15.87) and (15.88) collapses to ui (t) = ki (t)(xi (t) − xei ) + φi (t), i = 1, . . . , m, (15.106) ˙ki (t) = −(sgn bi )qi (xi (t) − xei )2 , i = 1, . . . , m, (15.107)  0, if φi (t) = 0 and xi (t) ≥ xei , φ˙ i (t) = −(sgn bi )ˆ qi (xi (t) − xei ), otherwise, i = 1, . . . , m, (15.108)

NNBook

September 1, 2009

462

CHAPTER 15

where ki (0) and φi (0) are such that (sgn bi )ki (0) ≤ 0 and (sgn bi )φi (0) ≥ 0, i = 1, . . . , m. This is precisely the result given by Theorem 15.3. Example 15.1. In this example we demonstrate the utility of Theorem 15.6. Specifically, consider the controlled two-compartment nonnegative dynamical system given by x˙ 1 (t) = −a21 (x1 (t))x1 (t) + a12 (x1 (t))x2 (t) + bu(t), x˙ 2 (t) = a21 (x1 (t))x1 (t) − a12 (x1 (t))x2 (t),

x1 (0) = x10 ,

x2 (0) = x20 ,

t ≥ 0, (15.109) (15.110)

where a21 (x1 ) , c1 Q(x1 ), a12 (x1 ) , c2 + c3 Q(x1 ), Q(x1 ) , c4 x11+c5 , and c1 , . . . , c5 , and b are unknown positive constants. Note that with x = [x1 , x2 ]T , (15.109) and (15.110) can be written in the form of (15.79) with f (x) = [−a21 (x1 )x1 + a12 (x1 )x2 , a21 (x1 )x1 − a12 (x1 )x2 ]T and G(x) = ˆ = [b, 0]T . Here, our objective is to regulate x1 around the desired value B xe1 ≥ 0. Note that xe2 = c1 Q(xe1 )xe1 /(c2 + c3 Q(xe1 )) and ue = 0 satisfy the equilibrium condition (15.82) with xe = [xe1 , xe2 ]T . Next, define e(t) , x(t) − xe so that fe (e) is given by  −[a21 (e1 + xe1 ) + a12 (e1 + xe1 )(e2 + xe2 )  −[−(a21 (xe1 ) + a12 (xe1 )xe2 ]  fe (e) =   a21 (e1 + xe1 )(e1 + xe1 ) − a12 (e1 + xe1 )(e2 + xe2 ) −[a21 (xe1 )xe1 − a12 (xe1 )xe2 )]



  . 

(15.111)

Furthermore, let Kg = kg /b, F1 (e) = e1 , and Vs (e) = e21 + e22 so that Vs ′1 (e)F1 (e) = 2e21 ≥ 0. Next, note that ˆ g F1 (e)] Vs′ (e)[fe (e) + BK = e1 [fe 1 (e) + kg e1 ] + e2 fe 2 (e) = −[a21 (e1 + xe1 ) + ke (e1 + xe1 )]e21 + a12 (e1 + xe1 )e1 e2 −xe1 [a21 (e1 + xe1 ) − a21 (xe1 )]e1 − xe2 [a12 (e1 + xe1 ) − a12 (xe1 )]e1 +a21 (e1 + xe1 )e1 e2 − a12 (e1 + xe1 )e22 + xe1 [a21 (e1 + xe1 ) − a21 (xe1 )]e2 −xe2 [a12 (e1 + xe1 ) − a12 (xe1 )]e2 + kg e21 , (15.112) where fe i (·) denotes the ith component of fe (·), i = 1, 2, and −kg ∈ R+ . Now, since Q(·) is Lipschitz continuous there exist positive constants α and β such that |[Q(e1 + xe1 ) − Q(xe1 )]e1 | ≤ αe21 and |[Q(e1 + xe1 ) − Q(xe1 )]e2 | ≤ β|e1 ||e2 |, and hence, it follows that there exist γ1 , γ2 > 0 such that ˆ g F1 (e)] ≤ γ1 e2 + 2γ2 |e1 ||e2 | − c2 e2 + kg e2 Vs′ (e)[fe (e) + BK 1 2 1

NNBook

September 1, 2009

463

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS 12

x1(t) x2(t) 10

States

8

6

4

2

0

0

1

2

3

4

5

6

7

8

9

10

Time

Figure 15.14 State trajectories versus time.

= −c2

 2  γ2 |e1 | − |e2 | + γ1 − 22 + kg e21 . c2 c2

γ

Hence, there exists kg < 0 such that

2

ˆ g F1 (e)] < 0, Vs′ (e)[fe (e) + BK

e ∈ R2 ,

e 6= 0.

(15.113)

Now, it follows from Theorem 15.6 that x(t) → xe as t → ∞ for every positive constant c1 , . . . , c5 , and b. For xe1 = 2 and with c1 = 2, c2 = 0.1, c3 = 3, c4 = c5 = 1, b = 3, q1 = 0.01, qˆ1 = 0.1, and initial conditions x(0) = [5, 8]T , k1 (0) = 0, and φ1 (0) = 1, Figure 15.14 shows the state trajectories of the controlled system (15.109) and (15.110) versus time. Finally, Figure 15.15 shows the control signal and the adaptive gain history versus time. △

15.8 Adaptive Control for Linear Nonnegative Uncertain Dynamical Systems with Time Delay In the next several sections, we extend the results of Sections 15.2 and 15.3 to the case of nonnegative and compartmental dynamical systems with unknown system time delay. Specifically, we develop a LyapunovKrasovskii-based direct adaptive control framework for guaranteeing setpoint regulation for linear uncertain nonnegative and compartmental dynamical systems with unknown time delay. In particular, we develop direct

NNBook

September 1, 2009

464

CHAPTER 15 1

Input signal

0 −1 −2 −3 −4

0

1

2

3

4

5

6

7

8

9

10

Time

Adaptive gains

1

k1(t) φ1(t)

0.5

0

−0.5

−1

0

1

2

3

4

5

6

7

8

9

10

Time

Figure 15.15 Control signal and adaptive gain history versus time.

adaptive controllers with nonnegative control inputs as well as adaptive controllers with the absence of such a restriction. The following results are necessary for the development of the main results of this and the next sections. Lemma 15.1. Let A ∈ Rn×n be an essentially nonnegative matrix and ˆ let Adi ∈ Rn×n , i = 1, . . . , p, be Ppnonnegative matrices such that Ae ≤≤ 0 T ˆ ˆ and A e ≤≤ 0, where A , A+ i=1 Adi . Then there exist diagonal matrices Qi ≥ 0, i = 1, . . . , p, such that T

A+A +

p X i=1

by

# (Qi + AT di Qi Adi ) ≤ 0.

(15.114)

Proof. For each i ∈ {1, . . . , p}, let Qi be the diagonal matrix defined Qi (l,l) ,

n X

Adi(l,m) ,

m=1

and note that (Adi − Qi )e = 0, (A +

Pp

i=1 Qi )e

= (A +

Pp

i=1 Adi )e

≤≤ 0,

NNBook

September 1, 2009

465

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

and Qi Q# i Adi = Adi , i = 1, . . . , p. Hence, M e ≤≤ 0, where 

  M , 

P A + AT + pi=1 Qi AT AT AT d1 d2 . . . dp Ad1 −Q1 0 ... 0 .. .. .. .. .. . . . . . Adp 0 0 . . . −Qp



  . 

(15.115)

Now, it follows from Proposition 10.2 that M ≤ 0 and since Qi Q# i Adi = Adi , i = 1, . . . , p, it follows from Lemma 10.1 that M ≤ 0 if and only if (15.114) holds. Theorem 15.7. Let A ∈ Rn×n be an essentially nonnegative matrix n×n , i = 1, . . . , p, be nonnegative matrices such that A ˆ , A+ and Pp let Adi ∈ R n×n , i=1 Adi is Hurwitz. Then there exist diagonal matrices P and Qi ∈ R with P > 0 and Qi ≥ 0, i = 1, . . . , p, such that AT P + P A +

p X # (Qi + AT di P Qi P Adi ) < 0.

(15.116)

i=1

P Proof. Since Aˆ , A + pi=1 Adi is essentially nonnegative and Hurwitz, it follows from Theorem 2.11 that there exists l ∈ Rn+ , l = [l1 , . . . , ln ]T , such that AˆT l 0 and r >> 0 satisfying !T p X 0 = As + Adi p + r. (15.147) i=1

To show Lyapunov stability of the closed-loop system (15.144) and (15.146), consider the Lyapunov-Krasovskii functional candidate V : C+ × Rm×m → R given by p Z 0 X 1 T pT Adi ψ(θ)dθ + tr(K − Kg )T Q−1 (K − Kg ), V (ψ, K) = p ψ(0) + 2 i=1 −τi (15.148) where Q = diag[ pq11b1 , . . . , pmqmbm ]. Furthermore, note that V (ψe , Kg ) = 0, where ψe (θ) = 0, θ ∈ [−¯ τ , 0], and there exist class K functions α1 (·) and α2 (·) such that V (ψ, K) ≥ α1 (kψ(0)k) + α2 (kK − Kg kF ). Now, letting x(t), t ≥ 0, denote the solution to (15.146) and using (15.144), it follows that the Lyapunov-Krasovskii directional derivative of V (xt , K(t)) along the closed-loop system trajectories is given by V˙ (xt , K(t)) = pT As x(t) + pT

p X i=1

+p

T

p X i=1

Adi x(t − τi ) + pT B(K(t) − Kg )ˆ x(t)

Adi x(t) − p

T

p X i=1

Adi x(t − τi )

NNBook

September 1, 2009

476

CHAPTER 15

˙ +tr(K(t) − Kg )T Q−1 K(t)

= −r T x(t) ≤ 0, t ≥ 0,

which proves that the solution (x(t), K(t)) ≡ (0, Kg ) to (15.144) and + (15.146) is Lyapunov stable. Furthermore, since the positive orbit Oη,K 0 + belongs of the closed-loop system (15.144) and (15.146) is bounded, Oη,K 0 to a compact subset of C+ × Rm×m [126], and r >> 0, it follows from Theorem 3.4 that x(t) → 0 as t → ∞ for all η(·) ∈ C+ .

15.9 Adaptive Control for Linear Nonnegative Dynamical Systems with Nonnegative Control and Time Delay In this section, we develop adaptive control laws for nonnegative retarded systems with nonnegative control inputs. However, once again, since n m condition (15.122) is required to be satisfied for xe ∈ R+ and ue ∈ R+ , it follows from Brockett’s necessary condition for asymptotic stabilizability [193] that thereP does not exist a continuous stabilizing nonnegative feedback if 0 ∈ spec(A + pi=1 Adi ) and xe ∈ Rn+P(see [120] for further details). Hence, in this section we assume that A + pi=1 Adi is an asymptotically stable compartmental matrix. Thus, we proceed with the aforementioned assumptions to design adaptive controllers for uncertain time-delay compartmental systems that guarantee that limt→∞ xi (t) = xdi ≥ 0 for i = 1, . . . , m ≤ n, where xdi is a desired set point for the ith compartmental state while guaranteeing a nonnegative control input. Theorem 15.10. Consider the linear uncertain time-delay system G given by (15.124),P where A is essentially nonnegative, Adi , i = 1, . . . , p, is nonnegative, A + pi=1 Adi is asymptotically stable, and B is nonnegative and given by (15.125). For a given xd ∈ Rm , assume there exist vectors m xu ∈ Rn−m and ue ∈ R+ such that (15.126) holds. Furthermore, let qi and + qˆi , i = 1, . . . , m, be positive constants. Then, the adaptive feedback control law ui (t) = max{0, uˆi (t)}, i = 1, . . . , m, (15.149) where u ˆi (t) = ki (t)(xi (t) − xdi ) + φi (t),

i = 1, . . . , m,

(15.150)

ki (t) ∈ R, t ≥ 0, and φi (t) ∈ R, t ≥ 0, i = 1, . . . , m, with update laws  0, if u ˆi (t) < 0, ˙ki (t) = ki (0) ≤ 0, −qi (xi (t) − xdi )2 , otherwise, i = 1, . . . , m, (15.151)

NNBook

September 1, 2009

477

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

φ˙ i (t) =



if φi (t) = 0 and xi (t) > xdi , or if u ˆi (t) ≤ 0, otherwise, φi (0) ≥ 0, i = 1, . . . , m, (15.152)

0, −ˆ qi (xi (t) − xdi ),

guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , 0, ue ) of the closed-loop system given by (15.124), (15.149), (15.151), and (15.152) is Lyapunov stable and xi (t) → xdi , i = 1, . . . , m, as t → ∞ for all η(·) ∈ C+ . Furthermore, u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0, t ≥ 0, for all η(·) ∈ C+ . Proof. First, define Ku (t) , diag[ku1 (t), . . . , kum (t)] and φu (t) , [φu1 (t), . . . , φum (t)]T , where  ˆi (t) < 0, 0, if u kui (t) = i = 1, . . . , m, (15.153) ki (t), otherwise,

φui (t) =



ˆi (t) < 0, if u otherwise,

0, φi (t),

i = 1, . . . , m.

(15.154)

Now, note that with u(t), t ≥ 0, given by (15.149), it follows from (15.124) that p X x(t) ˙ = Ax(t) + Adi x(t − τi ) + BKu (t)(ˆ x(t) − xd ) + Bφu (t), i=1

x(θ) = η(θ), −¯ τ ≤ θ ≤ 0, t ≥ 0,

(15.155)

or, equivalently, using (15.126), x(t) ˙ = A(x(t) − xe ) +

p X i=1

x(t) − xd ) Adi (x(t − τi ) − xe ) + BKu (t)(ˆ

+B(φu (t) − ue ), x(θ) = η(θ), −¯ τ ≤ θ ≤ 0, t ≥ 0. (15.156) Pp Since A + i=1 Adi is essentially nonnegative and Hurwitz, it follows from ˜ i ≥ 0, i = Theorem 15.7 that there exist diagonal matrices P > 0 and Q n×n 1, . . . , p, and a positive-definite matrix R ∈ R such that T

0 = A P + PA +

p X

˜ i + AT P Q ˜ # P Adi ) + R. (Q di i

(15.157)

i=1

To show Lyapunov stability of the closed-loop system (15.151), (15.152), and (15.156), consider the Lyapunov-Krasovskii functional candidate V : C+ × Rm×m × Rm → R given by V (ψ, K, φ) = (ψ(0) − xe )T P (ψ(0) − xe ) p Z 0 X ˜# + (ψ(θ) − xe )T AT di P Qi P Adi (ψ(θ) − xe )dθ i=1

−τi

NNBook

September 1, 2009

478

CHAPTER 15

ˆ −1 (φ − ue ), +trK T Q−1 K + (φ − ue )T Q

(15.158)

or, equivalently, V (ψ, K, φ) =

n X i=1

+

pi (ψi (0) − xei )2

p Z X

0

˜# (ψ(θ) − xe )T AT di P Qi P Adi (ψ(θ) − xe )dθ

i=1 −τi m X p i bi 2 k + qi i i=1

+

m X p i bi i=1

qi

(φi − uei )2 ,

where  qm q1 ,..., , Q = diag p 1 b1 p m bm 

ˆ = diag Q



 qˆ1 qˆm ,..., . p 1 b1 p m bm

Note that V (ψe , 0, ue ) = 0, where ψe (θ) = xe , θ ∈ [−¯ τ , 0]. Furthermore, there exist class K functions α1 (·), α2 (·), and α3 (·) such that V (ψ, K, φ) ≥ α1 (kψ(0) − xe k) + α2 (kKkF ) + α3 (kφ − ue k). Next, letting x(t), t ≥ 0, denote the solution to (15.156) and using (15.151) and (15.152), it follows that the Lyapunov-Krasovskii directional derivative of V (xt , K(t), φ(t)) along the closed-loop system trajectories is given by V˙ (xt , K(t), φ(t)) = −(x(t) − xe )T R(x(t) − xe ) p X ˜ i (x(t) − xe ) − P Adi (x(t − τi ) − xe )]T Q ˜# − [Q i i=1

˜ i (x(t) − xe ) − P Adi (x(t − τi ) − xe )] ·[Q m X +2 pi bi kui (t)(xi (t) − xd )2 i=1

+2 +2 +2

m X

pi bi (xi (t) − xe )(φu i (t) − uei )

i=1 m X

p i bi (φi (t) − uei )φ˙ i (t) qˆi

i=1 m X

i=1

p i bi ki (t)k˙ i (t) qi

≤ −(x(t) − xe )T R(x(t) − xe )

NNBook

September 1, 2009

479

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

+2

m X i=1

+2

m X i=1

i h 1 pi bi kui (t)(xi (t) − xdi )2 + ki (t)k˙ i (t) qi h pi bi (xi (t) − xdi )(φui (t) − uei )

i 1 + (φi (t) − uei )φ˙ i (t) , qˆi

t ≥ 0.

(15.159)

For the two cases given in (15.151) and (15.152), the last two terms on the right-hand side of (15.159) give: i ) If u ˆi (t) < 0, t ≥ 0, then kui (t) = 0, φui (t) = 0, k˙ i (t) = 0, and φ˙ i (t) = 0, t ≥ 0. Furthermore, since φi (t) ≥ 0 and ki (t) ≤ 0 for all t ≥ 0 and i = 1, . . . , m, it follows from (15.150) that u ˆi (t) < 0 only if xi (t) > xdi , t ≥ 0, and hence, for all t ≥ 0, 1 kui (t)(xi (t) − xdi )2 + ki (t)k˙ i (t) = 0, qi 1 (xi (t) − xdi )(φui (t) − uei ) + (φi (t) − uei )φ˙ i (t) qi = −(xi (t) − xdi )uei ≤ 0. ii ) Otherwise, kui (t) = ki (t) and φui (t) = φi (t), and hence, for all t ≥ 0, kui (xi (t) − xdi )2 +

1 ki (t)k˙ i (t) = 0, qi

1 (xi (t) − xdi )(φui (t) − uei ) + (φi (t) − uei )φ˙ i (t) qˆi ( −(xi (t) − xdi )uei ≤ 0, if φi (t) = 0 and xi (t) ≥ xdi , = 0, otherwise. Hence, it follows that in either case V˙ (xt , K(t), φ(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) ≤ 0,

t ≥ 0,

(15.160)

which proves that the solution (x(t), K(t), φ(t)) ≡ (xe , 0, ue ) to (15.151), (15.152), and (15.156) is Lyapunov stable. Furthermore, since the positive + orbit Oη,K of the closed-loop system (15.151), (15.152), and (15.156) is 0 ,φ0 + bounded, Oη,K0 ,φ0 belongs to a compact subset of C+ × Rm×m × Rm [126], and R > 0, it follows from the Krasovskii-LaSalle invariant set theorem that x(t) → xe as t → ∞ for all η(·) ∈ C+ . Finally, u(t) ≥≥ 0, t ≥ 0, is a restatement of (15.149). Now, since B ≥≥ 0 and u(t) ≥≥ 0, t ≥ 0, it follows from Proposition 4.7 that x(t) ≥≥ 0,

NNBook

480

September 1, 2009

CHAPTER 15

t ≥ 0, for all η(·) ∈ C+ .

15.10 Adaptive Control for Nonnegative Systems with Time Delay and Actuator Amplitude Constraints An implicit assumption inherent in all the results in this chapter is that the control law is implemented without any regard to actuator amplitude and rate saturation constraints. Of course, any electromechanical control actuation device (e.g., a syringe pump) is subject to amplitude and/or rate constraints leading to saturation nonlinearities, enforcing limitations on control amplitude and control rates. More importantly, in pharmacological applications, drug infusion rates can vary from patient to patient, and, to avoid overdosing, it is vital that the infusion rate does not exceed the patientspecific threshold values. As a consequence, control constraints, that is, infusion pump rate constraints, need to be accounted for in drug delivery systems. In this section, we extend the results of Section 15.9 to the case of compartmental dynamical systems with unknown system time delays and control amplitude constraints. Specifically, we develop a LyapunovKrasovskii-based direct adaptive control framework for guaranteeing setpoint regulation for linear uncertain compartmental dynamical systems with unknown time delay and control amplitude constraints. In particular, we consider the controlled linear uncertain time-delay dynamical system G given by (15.124) wherein the control input u(·) in (15.124) is restricted to the class of admissible controls consisting of m measurable functions such that u(t) ∈ U ⊂ R+ , where U , {u(t) ∈ Rm : 0 ≤ ui (t) ≤ umax,i , i = 1, . . . , m, t ≥ 0}, and umax,i > 0, i = 1, . . . , m, are m given. The assumption that u(t) ∈ U ⊂ R+ is motivated by the fact that control (source) inputs for drug delivery systems of physiological processes are usually constrained to be nonnegative as are the system states. Since A is essentially nonnegative and Adi , i = 1, . . . , nd , and B are nonnegative, it follows from Proposition 4.7 that the state trajectories of (15.124) remain in the nonnegative orthant of the state space for nonnegative initial conditions and nonnegative inputs. Theorem 15.11. Consider the linear uncertain time-delay system G given by (15.124),P where A is essentially nonnegative, Adi , i = 1, . . . , p, is nonnegative, A + pi=1 Adi is asymptotically stable, and B is nonnegative and given by (15.125). For a given xd ∈ Rm + , assume there exist vectors xu ∈ Rn−m and u ∈ U such that (15.126) holds. Furthermore, let qi and e + qˆi , i = 1, . . . , m, be positive constants. Then, the adaptive feedback control

NNBook

September 1, 2009

481

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

law ui (t) = where

 1 u ˆi (t)+|ˆ ui (t)|+2umax,i − u ˆi (t)+|ˆ ui (t)|−2umax,i , 4 u ˆi (t) = ki (t)(xi (t) − xdi ) + φi (t),

i = 1, . . . , m,

i = 1, . . . , m, (15.161) (15.162)

ki (t) ∈ R, t ≥ 0, and φi (t) ∈ R, t ≥ 0, i = 1, . . . , m, with update laws  0, if u ˆi (t) < 0 or u ˆi (t) > umax,i , ˙ki (t) = 2 −qi (xi (t) − xdi ) , otherwise, ki (0) ≤ 0, i = 1, . . . , m, (15.163)  0, if φi (t) = 0 and xi (t) > xdi ,    0, if φi (t) = umax,i and xi (t) < xdi , φ˙ i (t) = 0, if u ˆi (t) < 0 or u ˆi (t) > umax,i ,    −ˆ qi (xi (t) − xdi ), otherwise, 0 ≤ φi (0) ≤ umax,i , i = 1, . . . , m, (15.164) guarantees that the solution (x(t), K(t), φ(t)) ≡ (xe , 0, ue ) of the closed-loop system given by (15.124), (15.161), (15.163), (15.164) is Lyapunov stable and xi (t) → xdi , i = 1, . . . , m, as t → ∞ for all η(·) ∈ C+ . Furthermore, m u(t) ∈ U ⊂ R+ , t ≥ 0, and x(t) ≥≥ 0, t ≥ 0, for all η(·) ∈ C+ . △

△

Proof. First, define Ku (t) = diag[ku1 (t), . . . , kum (t)] and φu (t) = [φu 1 (t), . . . , φu m (t)]T , where  ˆi (t) < 0 or u ˆi (t) > umax,i , 0, if u kui (t) = i = 1, . . . , m, ki (t), otherwise,  if u ˆi (t) < 0,  0, umax,i , if u ˆi (t) > umax,i , i = 1, . . . , m. φu i (t) =  φi (t), otherwise, Now, note that with u(t), t ≥ 0, given by (15.161) it follows from (15.124) that x(t) ˙ = Ax(t) +

p X i=1

Adi x(t − τi ) + BKu (t)(ˆ x(t) − xd ) + Bφu (t), x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0, (15.165)

or, equivalently, using (15.126), x(t) ˙ = A(x(t) − xe ) +

nd X i=1

+B(φu (t) − ue ),

Adi (x(t − τi ) − xe ) + BKu (t)(ˆ x(t) − xd ) x(θ) = η(θ),

−¯ τ ≤ θ ≤ 0,

t ≥ 0. (15.166)

NNBook

September 1, 2009

482

CHAPTER 15

P Since A + pi=1 Adi is essentially nonnegative and Hurwitz, it follows from Theorem 15.7 that there exist diagonal matrices P , diag[p1 , . . . , pn ] > 0 ˜ i ≥ 0, i = 1, . . . , p, and a positive-definite matrix R ∈ Rn×n such that and Q p X ˜ i + AT P Q ˜ # P Adi ) + R. 0 = A P + PA + (Q di i T

(15.167)

i=1

To show Lyapunov stability of the closed-loop system (15.163), (15.164), and (15.166), consider the Lyapunov-Krasovskii functional candidate V : C+ × Rm×m × Rm → R given by V (ψ, K, φ) = (ψ(0) − xe )T P (ψ(0) − xe ) p Z 0 X ˜ # (ψ(θ) − xe )dθ + (ψ(θ) − xe )T Q i i=1

−τi

ˆ −1 (φ − ue ), +tr K Q−1 K + (φ − ue )T Q T

(15.168)

or, equivalently, V (ψ, K, φ) =

n X

pi (ψi (0) − xei ) +

i=1 m X

+

i=1

where

2

p i bi 2 k + qi i

m X i=1

p Z X i=1

0 −τi

˜ # (ψ(θ) − xe )dθ (ψ(θ) − xe )T Q i

p i bi (φi − uei )2 , qi

  q1 qm ,..., , K = diag[k1 , . . . , km ], Q = diag p 1 b1 p m bm   qˆ1 qˆm ˆ Q = diag ,..., . p 1 b1 p m bm Note that V (ψe , 0, ue ) = 0, where ψe (θ) = xe , θ ∈ [−¯ τ , 0]. Furthermore, note that there exist class K functions α1 (·), α2 (·), and α3 (·) such that V (ψ, K, φ) ≥ α1 (kψ(0) − xe k) + α2 (kKkF ) + α3 (kφ − ue k). Now, letting x(t), t ≥ 0, denote the solution to (15.166) and using (15.163) and (15.164), it follows that the Lyapunov-Krasovskii directional derivative along the closed-loop system trajectories is given by V˙ (xt , K(t), φ(t)) = −(x(t) − xe )T R(x(t) − xe ) p X ˜ i (x(t) − xe ) − P Adi (x(t − τi ) − xe )]T − [Q i=1 ˜ # [Q ˜ i (x(t) ·Q i

− xe ) − P Adi (x(t − τi ) − xe )]

NNBook

September 1, 2009

483

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

+2

m X i=1

+2 +2

m X

i=1 m X i=1

+2

pi bi kui (t)(xi (t) − xd )2 pi bi (xi (t) − xe )(φu i (t) − uei ) p i bi ki (t)k˙ i (t) qi

m X p i bi i=1

qˆi

(φi (t) − uei )φ˙ i (t)

≤ −(x(t) − xe )T R(x(t) − xe ) m i h X 1 +2 pi bi kui (t)(xi (t) − xdi )2 + ki (t)k˙ i (t) qi +2

i=1 m X i=1

h pi bi (xi (t) − xdi )(φui (t) − uei )

i 1 + (φi (t) − uei )φ˙ i (t) . qˆi

(15.169)

Since the update laws given by (15.163) and (15.164) involve the cases u ˆi (t) < 0, u ˆi (t) > umax,i , or 0 ≤ u ˆi (t) ≤ umax,i , i = 1, . . . , m, it follows that, for each of these three cases, the last two terms on the right-hand side of (15.169) give: i ) If u ˆi (t) < 0, then kui (t) = 0, φu i (t) = 0, k˙ i (t) = 0, and φ˙ i (t) = 0. Furthermore, since φi (t) ≥ 0 and ki (t) ≤ 0 for all t ≥ 0 and i = 1, . . . , m, it follows from (15.162) that u ˆi (t) < 0 only if xi (t) > xdi , and hence, kui (t)(xi (t) − xdi )2 + (xi (t) − xdi )(φu i (t) − uei ) +

1 ki (t)k˙ i (t) = 0, qi

1 (φi (t) − uei )φ˙ i (t) = −(xi (t) − xdi )uei qi ≤ 0.

ii ) If u ˆi (t) > umax,i , then kui (t) = 0, φu i (t) = umax,i , k˙ i (t) = 0 and ˙ φi (t) = 0. Furthermore, since φi (t) ≤ umax,i and ki (t) ≤ 0 for all t ≥ 0, it follows from (15.162) that u ˆi (t) > umax,i only if xi (t) < xdi , and hence, kui (t)(xi (t) − xdi )2 +

1 ki (t)k˙ i (t) = 0, qi

NNBook

September 1, 2009

484

CHAPTER 15

1 (φi (t) − uei )φ˙ i (t) qi = (xi (t) − xdi )(umax,i − uei ) ≤ 0.

(xi (t) − xdi )(φu i (t) − uei ) +

iii) Otherwise, kui (t) = ki (t) and φu i (t) = φi (t), and hence, kui (t)(xi (t) − xdi )2 +

1 ki (t)k˙ i (t) = 0, qi

1 (xi (t) − xdi )(φu i (t) − uei ) + (φi (t) − uei )φ˙ i (t) qˆi  if φi (t) = 0 and xi (t) > xdi ,  −(xi (t) − xdi )uei ≤ 0, (xi (t) − xdi )(umax,i − uei ) ≤ 0, if φi (t) = umax,i and xi (t) < xdi , =  0, otherwise.

Hence, it follows that in any of the three cases V˙ (xt , K(t), φ(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) ≤ 0,

t ≥ 0,

(15.170)

which proves that the solution (xt , K(t), φ(t)) ≡ (xe , 0, ue ) to (15.163), (15.164), and (15.166) is Lyapunov stable. Furthermore, since the positive + orbit Oη,K of the closed-loop system (15.163), (15.164), and (15.166) is 0 ,φ0 + bounded, Oη,K0 ,φ0 belongs to a compact subset of C+ × Rm×m × Rm [126], and R > 0, it follows from the Krasovskii-LaSalle invariant set theorem that x(t) → xe as t → ∞ for all η(·) ∈ C+ . ˜i (t)} Finally, since max{0, uˆi (t)} = (ˆ ui (t) + |ˆ ui (t)|)/2 and min{umax,i , u = (˜ ui (t) + umax,i − |˜ ui (t) − umax,i |)/2, where u ˜i (t) = max{0, uˆi (t)}, (15.161) is equivalent to u ˜i (t) = max{0, uˆi (t)},

ui (t) = min{umax,i , u ˜i (t)},

t ≥ 0,

i = 1, . . . , m.

Hence, 0 ≤ ui (t) ≤ umax,i , i = 1, . . . , m, t ≥ 0. Now, since B ≥≥ 0 and u(t) ≥≥ 0, t ≥ 0, it follows from Proposition 4.7 that x(t) ≥≥ 0, t ≥ 0, for all η(·) ∈ C+ .

15.11 Adaptive Control for General Anesthesia: Linear Model with Time Delay and Actuator Constraints In this section, we apply results of Section 15.10 to the three-compartment drug delivery model considered in Section 15.4 with the additional constraints of time delay and actuator amplitude limitations. In particular, a mass balance for the three-state compartmental model shown in Figure 15.16 yields x˙ 1 (t) = −(a11 + a21 + a31 )x1 (t) + a12 x2 (t − τ1 ) + a13 x3 (t − τ2 ) + u(t),

NNBook

September 1, 2009

485

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

u ≡ Continuous infusion a12 x2 , τ1 Muscle a21 x1 , τ1

Intravascular Blood

a31 x1 , τ2 Fat a13 x3 , τ2

a11 x1 ≡ Elimination (liver, kidney) Figure 15.16 Three-compartment mammillary model for disposition of propofol.

x1 (θ) = η1 (θ), −¯ τ ≤ θ ≤ 0, t ≥ 0, (15.171) τ ≤ θ ≤ 0, (15.172) x˙ 2 (t) = −a12 x2 (t) + a21 x1 (t − τ1 ), x2 (θ) = η2 (θ), −¯ x˙ 3 (t) = −a13 x3 (t) + a31 x1 (t − τ2 ), x3 (θ) = η3 (θ), −¯ τ ≤ θ ≤ 0, (15.173) where x1 (t), x2 (t), x3 (t), t ≥ 0, are the masses in grams of propofol in the central compartment and Compartments 2 and 3, respectively, u(t), t ≥ 0, is the infusion rate in grams/min of the anesthetic drug propofol into the central compartment, τ1 > 0 is the transfer time between the central compartment and peripheral Compartment 2 (muscle), τ2 > 0 is the transfer time between the central compartment and peripheral Compartment 3 (fat), τ¯ = max{τ1 , τ2 }, aij > 0, i 6= j, i, j = 1, 2, 3, are the rate constants in min−1 for drug transfer between compartments, and a11 > 0 is the rate constant in min−1 of drug metabolism and elimination from the central compartment. Next, note that (15.171)–(15.173) can be written in the state space form (15.124) with state vector x = [x1 , x2 , x3 ]T ,     0 a12 0 −(a11 + a21 + a31 ) 0 0 0 −a12 0  , Ad1 =  a21 0 0  , A= 0 0 0 0 0 −a13     0 0 a13 1    0 0 0 Ad2 = , B = 0 . (15.174) a31 0 0 0

Now, it can be shown that for xd1 /Vc = 3.4 µg/ml, all the conditions of Theorem 15.11 are satisfied. Here, we assume that the transfer and loss coefficients a11 , a12 , a21 , a13 , and a31 are unknown and our objective is to regulate the propofol concentration level of the central compartment to the desired level of 3.4 µg/ml in the face of system uncertainty. To consider drug effect rather than drug concentration, we use a linearized BIS measurement predicted on an effect-site compartment as in Section 15.4. In the following numerical simulation we set EC50 = 3.4 µg/ml, γ = 3, and BIS0 = 100, so that the BIS signal is shown in Figure 15.17.

NNBook

September 1, 2009

486

CHAPTER 15 100

90

80

BIS Index [score]

70

60

Target BIS

50

40

30

← Linearized range

20

10

EC50 = 3.4 [µg/ml] 0

0

1

2

3

4

5

6

7

8

9

10

Effect site concentration [µg/ml]

Figure 15.17 BIS index versus effect-site concentration

The values for the pharmacodynamic parameters (EC50 , γ) are within the typical range of those observed for ligand-receptor binding [73, 169]. The target (desired) BIS value, BIStarget , is set at 50. In this case, the linearized BIS function about the target BIS value is given by γ−1 γc eff · ceff BIS(ceff ) ≃ BIS(EC50 ) − BIS0 · ECγ50 · γ (ceff + ECγ50 )2 ceff =EC50

= 125 − 22.06ceff .

(15.175)

Furthermore, for simplicity of exposition, we assume that the effect-site compartment equilibrates instantaneously with the central compartment, that is, we assume that ceff (t) = x1 (t)/Vc for all t ≥ 0.

Now, using the adaptive feedback controller  1 u ˆ1 (t) + |ˆ u1 (t)| + 2umax,1 − u u1 (t) = ˆ1 (t) + |ˆ u1 (t)| − 2umax,1 , (15.176) 4 where u ˆ1 (t) = −k1 (t)(BIS(t) − BIStarget ) + φ1 (t), (15.177) and k1 (t) and φ1 (t) are scalars for t ≥ 0, with update laws  0, if u ˆ1 (t) < 0 or u ˆ1 (t) > umax,1 , ˙k1 (t) = −qBIS1 (BIS(t) − BIStarget )2 , otherwise, k1 (0) ≤ 0, (15.178)

NNBook

September 1, 2009

487

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

Table 15.2 Pharmacokinetic Parameters [98]. Set A B

a11 (min−1 ) 0.152 0.119

a21 (min−1 ) 0.207 0.114

a12 (min−1 ) 0.092 0.055

 0,        0, ˙ φ1 (t) =     0,    −ˆ qBIS1 (BIS(t)) − BIStarget ),

a31 (min−1 ) 0.040 0.041

a13 (min−1 ) 0.0048 0.0033

if φ1 (t) = 0 and BIS(t) > BIStarget , if φ1 (t) = umax,1 and BIS(t) < BIStarget , if u ˆ1 (t) > umax,1 or u ˆ1 (t) < 0, otherwise, 0 ≤ φ1 (0) ≤ umax,1 ,

(15.179)

where qBIS1 and qˆBIS1 are positive constants, it follows from Theorem 15.11 that the control input (anesthetic infusion rate) satisfies 0 ≤ u1 (t) ≤ umax,1 for all t ≥ 0 and BIS(t) → BIStarget as t → ∞ for all nonnegative values of the transfer and loss coefficients in the range of ceff where the linearized BIS equation (15.175) is valid. It is important to note that during actual surgery or intensive care unit sedation the BIS signal is obtained directly from the EEG and not (15.47). Furthermore, since our adaptive controller only requires the error signal BIS(t) − BIStarget over the linearized range of (15.47), we do not require knowledge of the slope of the linearized equation (15.175), nor do we require knowledge of the parameters γ and EC50 . To numerically illustrate the efficacy of the proposed adaptive control law, we use the average set of pharmacokinetic parameters given in [98] for 29 patients requiring general anesthesia for noncardiac surgery. For our design we assume M = 70 kg, τ1 = 1 min, τ2 = 2 min, and use the data given in Table 15.2. Furthermore, to illustrate the adaptive controller we switch the pharmacodynamic parameters (EC50 , γ) and the pharmacokinetic parameters (the entries of the system matrices A, Ad1 , and Ad2 ) from 3.4 µg/ml, 3, and Set A to 4.0 µg/ml, 4, and Set B at t = 15 min, and back to 3.4 µg/ml, 3, and Set A at t = 30 min. Here we consider noncardiac surgery since cardiac surgery often utilizes hypothermia which itself changes the BIS signal. With qBIS1 = 1 × 10−4 g/min2 , qˆBIS1 = 1 × 10−3 g/min2 , umax,1 = 20 g/min, and initial conditions x(0) = [0, 0, 0]T g, k1 (0) = 0 min−1 , and φ1 (0) = 0.01 g/min−1 , Figure 15.18 shows the masses of propofol in all three compartments versus time. Figure 15.19 shows the BIS index versus time. Figure 15.20 shows the propofol concentration in the central compartment and the control signal (propofol infusion rate) versus time. Finally, Figure 15.21 shows the adaptive gain history versus time.

NNBook

September 1, 2009

488

CHAPTER 15

70

x (t) 1 x (t) 2 x (t)

60

Compartmental masses [mg]

3

50

40

30

20

10

0

0

5

10

15

20

25

30

35

40

35

40

Time [min]

Figure 15.18 Compartmental masses versus time.

100

90

80

BIS Index [score]

70

60

50

40

30

20

10

0

0

5

10

15

20

25

30

Time [min]

Figure 15.19 BIS index versus time.

NNBook

September 1, 2009

489

ADAPTIVE CONTROL FOR NONNEGATIVE SYSTEMS

Concentration [µg/ml]

5 4 3 2 1 0

0

5

10

15

20

25

30

35

40

35

40

Control signal [mg/min]

Time [min] 25 20

umax,1

15 10 5 0

0

5

10

15

20

25

30

Time [min]

2

Adaptive gain K1 [mg/min ]

Figure 15.20 Drug concentration in the central compartment and control signal (infusion rate) versus time.

0 −0.5 −1 −1.5 −2 −2.5 −3

0

5

10

15

20

25

30

35

40

25

30

35

40

Adaptive gain φ1 [mg/min2]

Time [min] 14 12 10 8 6 4 2 0

0

5

10

15

20

Time [min]

Figure 15.21 Adaptive gain history versus time.

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Sixteen

Adaptive Disturbance Rejection Control for Compartmental Systems

16.1 Introduction In Chapter 15 a direct adaptive control framework for linear and nonlinear nonnegative and compartmental systems was developed. The framework in Chapter 15 is Lyapunov based and guarantees partial asymptotic setpoint regulation, that is, asymptotic set-point stability with respect to the closed-loop system states associated with the plant. In addition, the adaptive controllers in Chapter 15 guarantee that the physical system states remain in the nonnegative orthant of the state space. In this chapter, we extend the results of Section 15.2 to develop a direct adaptive control framework for adaptive stabilization and disturbance rejection for compartmental dynamical systems with exogenous system disturbances. The main challenge here is to construct nonlinear adaptive disturbance rejection controllers without requiring knowledge of the system dynamics or the system disturbances while guaranteeing that the physical system states remain in the nonnegative orthant of the state space. While such an adaptive control framework can have wide applicability in areas such as economics, telecommunications, and power systems, its use in the specific field of anesthetic pharmacology is particularly noteworthy. Specifically, during stress (such as hemorrhage) in an acute care environment, such as the operating room, perfusion pressure falls and hypertonic saline solutions are typically intravenously administered to regulate hemodynamic effects and avoid hemorrhagic shock. This exogenous disturbance drives the system pharmacokinetics and pharmacodynamics and can be captured as a system disturbance. In addition, exogenous system disturbances can be used to capture unmodeled physiological and pharmacological system dynamics. Although the proposed framework develops adaptive controllers for general compartmental systems with exogenous disturbances, the specific focus of this chapter is on pharmacokinetic models with hemorrhage and hemodilution effects.

NNBook

September 1, 2009

492

CHAPTER 16

16.2 Compartmental Systems with Exogenous Disturbances In this chapter, we develop an adaptive disturbance rejection control framework for asymptotic set point regulation of a disturbed linear compartmental system. Specifically, we consider uncertain dynamical systems G of the form x(t) ˙ = Ax(t) + Bu(t) + d(x(t), t),

x(0) = x0 , n R+ ,

t ≥ 0,

(16.1)

u(t) ∈ Rm , t ≥ 0, is the where x(t) ∈ Rn , t ≥ 0, is the state vector, x0 ∈ n control input, d(x(t), t) ∈ R , t ≥ 0, is an unknown nonlinear disturbance signal, A ∈ Rn×n is an unknown compartmental matrix, and B ∈ Rn×m is an unknown matrix given by   Bu B= , Bu = diag[ b1 , . . . , bm ], bi > 0, i = 1, 2, . . . , m. 0(n−m)×m (16.2) Here we assume that for all i = 1, 2, . . . , m, bi is unknown. The structure of B implies that the control inputs are injected directly into m separate compartments. As discussed in Chapter 15, for compartmental systems this assumption is not restrictive since control inputs correspond to control inflows to each individual compartment. The control input u(·) is restricted to the class of admissible controls consisting of measurable functions on Rm , m or R+ if u(t), t ≥ 0, is constrained to be nonnegative. In this chapter, we consider two cases for the disturbance signal d : Rn × [0, ∞) → Rn . Namely, in the first case, the disturbance signal is given by d(x(t), t) = −BΨ∗ w(x(t), t), (16.3) where Ψ∗ is an unknown constant diagonal disturbance weighting matrix given by ∗ Ψ∗ = diag[ ψ1∗ , . . . , ψm ],

ψi∗ > 0,

i = 1, 2, . . . , m,

(16.4)

and w(x, t) = [ w1 (x, t), . . . , wm (x, t) ]T is a known disturbance signal satisfying sufficient regularity conditions so that (16.1) has a unique solution forward in time. Furthermore, we assume that for the uncontrolled (i.e., u(t) ≡ 0) system (16.1) the disturbance signal w(x, t) is such that for any n time t ∈ [0, ∞) such that xi (t) = 0, wi (x(t), t) = 0. In this case, x(t) ∈ R+ for all t ≥ 0 and u(t) ≡ 0. In the second case, the disturbance signal d : Rn × [0, ∞) → Rn is given by d(x(t), t) = J(x(t))w(t), (16.5) n

where J : R+ → Rn×d is an unknown bounded continuous function and w : [0, ∞) → Rd is an unknown continuous function such that w(·) ∈ L2 .

NNBook

September 1, 2009

493

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

Note that since J(·) is bounded, there exists α > 0 such that kJ(x)k ≤ α, n x ∈ R+ , where k · k is a matrix norm on Rn×d . Furthermore, since w(·) is continuous on [0, ∞) and w(·) ∈ L2 , there exists β > 0 such that kw(t)k ≤ β, t ≥ 0, where k · k is a vector norm on Rd . In addition, we assume that n n J(x)w(t) ≥≥ 0 for all x ∈ ∂R+ and t ≥ 0, where ∂R+ denotes the boundary of the nonnegative orthant. This assumption ensures that the uncontrolled n (i.e., u(t) ≡ 0) system (16.1) remains nonnegative for all x(0) ∈ R+ . Given a desired set point xe ∈ Rn+ \{x0 } our goal is to design a m measurable control law u : [0, ∞) → Rm (or u : [0, ∞) → R+ ) guaranteeing partial asymptotic set point stability of the closed-loop system; that is, asymptotic set point stability with respect to part of the closed-loop system state. Since in many applications of nonnegative systems and, in particular, compartmental systems, it is often necessary to regulate a subset of the nonnegative state variables, which usually include a central compartment, here we require that limt→∞ xi (t) = xdi ≥ 0 for i = 1, 2, . . . , m ≤ n, where xdi is the desired set point for the ith state xi (t). In addition, we require that the remainder of the state associated with the adaptive controller gains is n Lyapunov stable. Finally, we require that x(t) ∈ R+ for all t ≥ 0. In certain parts of the chapter we will use the following assumption regarding the existence of an equilibrium point of the undisturbed (i.e., d(x(t), t) ≡ 0) dynamical system (16.1). Assumption 16.1. For the undisturbed (i.e., d(x(t), t) ≡ 0) dynamical system (16.1) and a given desired set point xd ∈ Rm + , there exist nonnegative n n−m and ue ∈ R+ such that vectors xu ∈ R+ 0 = Axe + Bue ,

(16.6)

where xe , [ xd , xu ]T . It follows from Proposition 13.1 that Assumption 16.1 implies that A is semistable.

16.3 Adaptive Control for Linear Compartmental Uncertain Systems with Exogenous Disturbances In this section, we consider the problem of characterizing adaptive disturbance rejection feedback control laws for linear compartmental uncertain dynamical systems of the form given by (16.1) with the disturbance d(x(t), t), t ≥ 0, given by (16.3). Specifically, we consider the controlled uncertain dynamical system given by x(t) ˙ = Ax(t) + B[u(t) − Ψ∗ w(x(t), t)],

x(0) = x0 ,

t ≥ 0.

(16.7)

NNBook

September 1, 2009

494

CHAPTER 16

First, we consider the case where there is no restriction on the sign of the control input u(t), t ≥ 0. Theorem 16.1. Consider the linear uncertain dynamical system given by (16.7) where A is essentially nonnegative, B is nonnegative and given by (16.2), and Ψ∗ is given by (16.4). Suppose Assumption 16.1 holds and assume that there exists a diagonal matrix Kg = diag[ kg1 , . . . , kgm ] such ˜ g is Hurwitz, where K ˜ g , Kg , 0m×(n−m) . Furthermore, that As , A + B K let qi , qˆi , and γi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law u(t) = K(t)(ˆ x(t) − xd ) + φ(t) + Ψ(t)w(x(t), t),

t ≥ 0,

(16.8)

where K(t) , diag[ k1 (t), . . . , km (t)], x ˆ(t) , [ x1 (t), . . . , xm (t)], φ(t) ∈ Rm , and Ψ(t) , diag[ ψ1 (t), . . . , ψm (t)], t ≥ 0, or, equivalently, ui (t) = ki (t)(ˆ xi (t) − xdi ) + φi (t) + ψi (t)wi (x(t), t), t ≥ 0, i = 1, . . . , m, (16.9) with update laws k˙ i (t) = −qi (xi (t) − xei )2 , φ˙ i (t) =

t ≥ 0,

i = 1, . . . , m, (16.10)

  0, if φi (t) = 0 and xi (t) − xdi ≥ 0,

(16.11)

 ψ˙ i (t) =

ki (0) ≤ 0,

−ˆ qi (xi (t) − xdi ), otherwise, φi (0) ≥ 0,

t ≥ 0,

i = 1, . . . , m,

  0, if ψi (t) = 0 and (xi (t) − xdi )wi (x(t), t) ≥ 0, 

(16.12)

−γi (xi (t) − xdi )wi (x(t), t), otherwise, ψi (0) = 0, t ≥ 0, i = 1, . . . , m,

guarantees that the solution (x(t), K(t), φ(t), Ψ(t)) ≡ (xe , Kg , ue , Ψ∗ ) of the closed-loop system given by (16.7)–(16.12) is Lyapunov stable and xi (t) → n xdi , i = 1, . . . , m, as t → ∞ for all x0 ∈ R+ . Furthermore, x(t) ≥≥ 0 for n all x0 ∈ R+ and t ≥ 0. Proof. With u(t), t ≥ 0, given by (16.8), it follows from (16.6) that x(t) ˙ = As (x(t) − xe ) + B(K(t) − Kg )(x(t) − xe ) +B(φ(t) − ue ) + B(Ψ(t) − Ψ∗ )w(t), x(0) = x0 , t ≥ 0. (16.13) Furthermore, since As is essentially nonnegative and Hurwitz it follows from Theorem 2.12 that there exist a positive diagonal matrix P , diag[ p1 , . . . , pn ] and a positive-definite matrix R ∈ Rn×n such that 0 = AT s P + P As + R.

(16.14)

NNBook

September 1, 2009

495

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

To show Lyapunov stability of the closed-loop system (16.7)–(16.12), consider the Lyapunov function candidate V (x, K, φ, Ψ) = (x − xe )T P (x − xe ) + tr(K − Kg )T Q−1 (K − Kg ) ˆ −1 (φ − ue ) + tr(Ψ − Ψ∗ )T Γ−1 (Ψ − Ψ∗ ), +(φ − ue )T Q

(16.15)

where  qm q1 ,..., , Q , diag p 1 b1 p m bm   γ1 γm Γ , diag ,..., . p 1 b1 p m bm 

ˆ , diag Q



 qˆ1 qˆm ,..., , p 1 b1 p m bm (16.16)

ˆ and Γ are positive Note that V (xe , Kg , ue , Ψ∗ ) = 0 and, since P , Q, Q, ∗ definite, V (xe , Kg , ue , Ψ ) > 0 for all (x, K, φ, Ψ) 6= (xe , Ke , ue , Ψ∗ ). Furthermore, V (x, K, φ, Ψ) is radially unbounded. Now, letting x(t), t ≥ 0, denote the solution to (16.13) and using (16.10)–(16.12), it follows that the Lyapunov derivative along the trajectories of the closed-loop system (16.7)– (16.12) is given by V˙ (x(t), K(t), φ(t), Ψ(t)) = 2(x(t) − xe )T P [As (x(t) − xe ) +B(K(t) − Kg )(x(t) − xe ) +B(φ(t) − ue ) + B(Ψ(t) − Ψ∗ )w(t)] ˙ +2tr(K(t) − Kg )T Q−1 K(t) ˙ ˆ −1 φ(t) +2(φ(t) − ue )T Q ˙ +2tr(Ψ(t) − Ψ∗ )T Γ−1 Ψ(t)

= −(x(t) − xe )T R(x(t) − xe )   m X 1˙ 2 +2 pi bi (ki (t) − kgi ) (xi (t) − xdi ) + ki (t) qi i=1   m X 1 ˙ +2 pi bi (φi (t) − uei ) (xi (t) − xdi ) + φi (t) qˆi i=1

+2

m X i=1



pi bi (ψi (t) − ψi∗ )

 1 ˙ · (xi (t) − xdi )wi (x, t) + ψi (t) . γˆi Next, it follows from (16.10) that, for each i ∈ {1, . . . , m},   1˙ 2 pi bi (ki (t) − kgi ) (xi (t) − xei ) + ki (t) = 0, t ≥ 0. qi

NNBook

September 1, 2009

496

CHAPTER 16

Using (16.11) it follows that if φi (t) = 0 and xi (t) − xdi ≥ 0, t ≥ 0, then   1 ˙ pi bi (φi (t) − uei ) (xi (t) − xdi ) + φ(t) = −pi bi uei (xi (t) − xdi ) ≤ 0, qˆi t ≥ 0, and if φi (t) 6= 0 or xi (t) − xdi < 0, t ≥ 0, then   1 ˙ pi bi (φi (t) − uei ) (xi (t) − xdi ) + φ(t) = 0. qˆi From (16.12) it follows that if ψi (t) = 0 and (xi (t) − xdi )wi (x(t), t) ≥ 0, t ≥ 0, then   1 ˙ ∗ pi bi (ψi (t) − ψi ) (xi (t) − xdi )wi (x, t) + ψi (t) γˆi ∗ = −pi bi ψi (xi (t) − xdi )wi (x, t) ≤ 0, t ≥ 0, and if ψi (t) 6= 0, or (xi (t) − xdi )wi (x(t), t) < 0, t ≥ 0, then   1 ˙ ∗ pi bi (ψi (t) − ψi ) (xi (t) − xdi )wi (x, t) + ψi (t) = 0, γˆi

t ≥ 0.

Hence, V˙ (x(t), K(t), φ(t), Ψ(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) ≤ 0,

t ≥ 0,

which proves that the solution (x(t), K(t), φ(t), Ψ(t)) ≡ (xe , Kg , ue , Ψ∗ ) of the closed-loop system given by (16.7)–(16.12) is Lyapunov stable. Moreover, since R is positive definite, it follows from Theorem 4.2 of [112] that x(t) → xe as t → ∞. n

To show that x(t) ≥≥ 0 for all x0 ∈ R+ and t ≥ 0 note that the closed-loop system (16.7)–(16.12) is given by   x(t) ˙ = (A + B K(t), 0m×(n−m) )x(t) − BK(t)xd + Bφ(t) +BΨ(t)w(x(t), t) − BΨ∗ w(x(t), t) ˜ = A(t)x(t) + v(t) + h(t) + g(t) + d(x(t), t), x(0) = x0 , t ≥ 0, (16.17)   ˜ where A(t) , A + B K(t), 0m×(n−m) , v(t) , −BK(t)xd , h(t) , Bφ(t), and g(t) , BΨ(t)w(x(t), t). Now, since A is essentially nonnegative, B is nonnegative and diagonal, K(t) is diagonal, and, by (16.10), ki (t) ≤ 0, ˜ is essentially nonnegative pointwiset ≥ 0, i = 1, . . . , m, it follows that A(t) in-time and v(t) ≥≥ 0, t ≥ 0. Next, it follows from (16.11) and (16.12) that φi (t) ≥ 0, t ≥ 0, i = 1, . . . , m, and hence, h(t) ≥≥ 0, t ≥ 0. Now, if g(t) ≡ 0 and d(x(t), t) ≡ 0, then it follows from Proposition 15.2 that x(t) ≥≥ 0, m t ≥ 0, for all x0 ∈ R+ .

NNBook

September 1, 2009

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

497

Finally, we show that the signals g(·) and d(x(·) , ·) are such that (16.17) remains nonnegative. To see this, assume that for a given time tˆ ∈ [0, ∞), x(tˆ) ∈ Rn+ . In this case, it follows from continuity of solutions with respect to the system initial conditions that, over a sufficiently small interval of time, the nonnegativity of the state of (16.17) is guaranteed irrespective of the sign of the components of g(·) and d(x(·) , ·). Alternatively, suppose n that x(tˆ) ∈ ∂R+ . In this case, there exists i ∈ {1, . . . , m} such that xi (tˆ) = 0, and hence, by assumption (see the discussion in Section 16.2), wi (x(tˆ), tˆ) = 0. Hence, di (x(tˆ), tˆ) = 0 and gi (tˆ) = bi ψi (tˆ)wi (x(tˆ), tˆ) = 0. Thus, the signals g(·) and d(x(·) , ·) do not destroy the nonnegativity of (16.17). This completes the proof. In the case where (16.7) is such that w(x(t), t) ≡ 0, the controller (16.9) with update laws (16.10)–(16.12) collapses to ui (t) = ki (t)(xi (t) − xdi ) + φi (t),

t ≥ 0,

i = 1, . . . , m, (16.18)

with update laws (16.10) and (16.11). This is precisely the result given in Theorem 15.1, where an adaptive control framework for nonnegative dynamical systems is developed for the undisturbed case (i.e., w(x(t), t) ≡ 0). It is important to note that the adaptive control framework addressed in this section requires that the bounded disturbance w(x(t), t), t ≥ 0, can be accurately measured even though the disturbance signal d(x(t), t), t ≥ 0, is an unknown bounded disturbance since BΨ∗ is unknown. Such a disturbance model can, for example, address sinusoidal disturbances with unknown amplitude and phase. In the next section, we consider the more general problem of L2 disturbances. Since the dynamical system considered in this section is minimum phase, it is possible, in principle, to stabilize the system by simply employing the controller (16.9) with φi (t) ≡ 0 and ψi (t) ≡ 0, and with a sufficiently high gain ki (t), t ≥ 0, i = 1, . . . , m. However, this is a very unsafe strategy when the disturbance w(x(t), t), t ≥ 0, is not accurately known and unmodeled system dynamics are present. In this case, unsafe high gain levels can excite unmodeled dynamics and drive the system to instability. Next, we consider the case where the control input is constrained to be nonnegative. In this case, we assume that w(x(t), t) ≥≥ 0 for all x(t) ∈ Rn+ and t ≥ 0, and if xi (t) = 0 for some t ∈ [0, ∞), then wi (x(t), t) = 0. Theorem 16.2. Consider the linear uncertain dynamical system given by (16.7) where A is Hurwitz and compartmental, B is nonnegative and given by (16.2), and Ψ∗ is given by (16.4). Suppose Assumption 16.1 holds.

NNBook

September 1, 2009

498

CHAPTER 16

Furthermore, let qˆi and γi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law u(t) = φ(t) + Ψ(t)w(x(t), t),

t ≥ 0,

(16.19)

where φ(t) = [ φ1 (t), . . . , φm (t)] and Ψ(t) = diag[ ψ1 (t), . . . , ψm (t)], t ≥ 0, or, equivalently, ui (t) = φi (t) + ψi (t)wi (x(t), t),

t ≥ 0,

i = 1, . . . , m,

(16.20)

with update laws φi (t), t ≥ 0, and ψi (t) given by (16.11) and (16.12), respectively, guarantees that the solution (x(t), φ(t), Ψ(t)) ≡ (xe , ue , Ψ∗ ) of the closed-loop system given by (16.7), (16.19), and (16.20) is Lyapunov n stable and xi (t) → xdi , i = 1, . . . , m, as t → ∞ for all x0 ∈ R+ . n Furthermore, u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0 for all x0 ∈ R+ and t ≥ 0. Proof. The proof is analogous to the proof of Theorem 16.1 with K(t) ≡ 0, and, hence, is omitted. Finally, we consider the case where the control input is constrained to be nonnegative and the disturbance signal d(x(t), t) is sign indefinite over a finite time interval, and nonpositive otherwise. Theorem 16.3. Consider the linear uncertain dynamical system given by (16.7) where A is Hurwitz and compartmental, B is nonnegative and given by (16.2), and Ψ∗ is given by (16.4). Suppose Assumption 16.1 holds and there exists a finite time T > 0 such that w(x(t), t) ≥≥ 0 for all x(t) ∈ Rn+ and t ≥ T , and if xi (t) = 0 for some t ∈ [0, ∞), then wi (x(t), t) = 0. Furthermore, let qi , qˆi , and γi , i = 1, . . . , m, be positive constants. Then the adaptive feedback control law ui (t) = max{0, u ˆi (t)},

t ≥ 0,

i = 1, . . . , m,

(16.21)

where u ˆi (t) = ki (t)(xi (t) − xei ) + φi (t) + ψi (t)wi (x(t), t),

t ≥ 0,

with update laws ki (t), φi (t), and ψi (t) given by  ˆi < 0,  0, if u k˙ i (t) =  −qi (xi (t) − xdi )2 , otherwise, ki (0) ≤ 0, t ≥ 0, φ˙ i (t) =

i = 1, . . . , m, (16.22)

(16.23) i = 1, . . . , m,

 ˆi (t) < 0  0, if φi (t) = 0 and xi (t) − xdi ≥ 0, or if u 

−ˆ qi (xi (t) − xdi ), otherwise, φi (0) ≥ 0,

t ≥ 0,

(16.24)

i = 1, . . . , m,

NNBook

September 1, 2009

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

499

 0, if ψi (t) = 0 and (xi (t) − xdi )wi (x(t), t) ≥ 0,    or if u ˆi (t) < 0 (16.25) ψ˙ i (t) =    −γi (xi (t) − xdi )wi (x(t), t), otherwise, ψi (0) = 0, t ≥ 0, i = 1, . . . , m,

guarantees that the solution (x(t), K(t), φ(t), Ψ(t)) ≡ (xe , Kg , ue , Ψ∗ ), where Kg = diag[ kg1 , . . . , kgm ], kgi ≤ 0, i = 1, . . . , m, φ(t) , [ φ1 (t), . . . , φm (t)]T , and Ψ(t) , diag[ ψ1 (t), . . . , ψm (t)], of the closed-loop system given by (16.7), (16.21)–(16.25) is Lyapunov stable and xi (t) → xdi , i = 1, . . . , m, as t → ∞ n for all x0 ∈ R+ . Furthermore, u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0 for all n x0 ∈ R+ and t ≥ 0. Proof. First, note that the update laws (16.23)–(16.25) guarantee that for each i ∈ {1, . . . , m} and for all t ≥ 0 the adaptive gain ki (t) remains nonpositive, and adaptive gains φi (t) and ψi (t) remain nonnegative. Next, define Ku , diag[ ku1 , . . . , kum ], φu , [ φu1 , . . . , φum ]T , and Ψu , diag[ ψu1 , . . . , ψum ], where  ˆi (t) < 0  0, if u i = 1, . . . , m, kui (t) = (16.26)  ki (t), otherwise, φui (t) =

ψui (t) =

 ˆi (t) < 0  0, if u

i = 1, . . . , m,

(16.27)

 ˆi (t) < 0  0, if u

i = 1, . . . , m,

(16.28)



φi (t), otherwise,



ψi (t), otherwise.

Now, note that with u(t), t ≥ 0, given by (16.21), it follows from (16.7) that x(t) ˙ = A(x(t) − xe ) + BKu (t)(ˆ x(t) − xd ) + B(φu (t) − ue ) ∗ +B(Ψu (t) − Ψ )w(x(t), t), x(0) = x0 , t ≥ 0. (16.29) Furthermore, since A is essentially nonnegative and Hurwitz it follows from Theorem 2.12 that there exist a positive diagonal matrix P , diag[ p1 , . . . , pn ] and a positive-definite matrix R ∈ Rn×n such that 0 = AT P + P A + R.

(16.30)

To show Lyapunov stability of the closed-loop system (16.7), (16.21)– (16.25) consider the Lyapunov function candidate (16.15). Now, letting x(t),

NNBook

September 1, 2009

500

CHAPTER 16

t ≥ 0, denote the solution to (16.29) and using (16.23)–(16.25) it follows that the Lyapunov derivative along the trajectories of the closed-loop system (16.7), (16.21)–(16.25) is given by V˙ (x(t), K(t), φ(t), Ψ(t)) = 2(x(t) − xe )T P [A(x(t) − xe ) +B(Ku (t) − Kg )(x(t) − xe ) +B(φu (t) − ue ) + B(Ψu (t) − Ψ∗ )w(t)] ˙ +2tr(K(t) − Kg )T Q−1 K(t) ˙ ˆ −1 φ(t) +2(φ(t) − ue )T Q ˙ +2tr(Ψ(t) − Ψ∗ )T Γ−1 Ψ(t)

= −(x(t) − xe )T R(x(t) − xe ) m h X +2 pi bi kui (t)(xi (t) − xdi )2 i=1

i 1 + (ki (t) − kgi )k˙ i (t) qi m h X +2 pi bi (φui (t) − uei )(xi (t) − xdi ) i=1

i 1 + (φi (t) − uei )φ˙i (t) qˆi m h X +2 pi bi (ψui (t) − ψi∗ )(xi (t) − xdi )wi (x, t) i=1

i 1 ∗ ˙ + (ψi (t) − ψi )ψi (t) . γˆi

(16.31)

First, consider the case where w(x(t), t) >> 0, t ≥ 0. For each i ∈ {1, . . . , m} the last three terms in (16.31) give: i) If u ˆi (t) < 0, t ≥ 0, then kui (t) = 0, φui (t) = 0, and ψui (t) = 0. Furthermore, since ki (t) ≤ 0, φi (t) ≥ 0, and ψi (t) ≥ 0, for all t ≥ 0, it follows from (16.22) that if w(x(t), t) >> 0, t ≥ 0, then u ˆi (t) < 0, t ≥ 0, only if xi (t) > xdi . Hence, it follows from (16.23)–(16.28) that if w(x(t), t) >> 0, t ≥ 0, then   1 2 ˙ pi bi kui (t)(xi (t) − xdi ) + (ki (t) − kgi )ki (t) = 0, qi   1 ˙ pi bi (φui (t) − uei )(xi (t) − xdi ) + (φi (t) − uei )φi (t) qˆi = −pi bi uei (xi (t) − xdi ) ≤ 0, t ≥ 0, (16.32)

NNBook

September 1, 2009

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

501

i h 1 pi bi (ψui (t) − ψi∗ )(xi (t) − xdi )wi (x, t) + (ψi (t) − ψi∗ )ψ˙ i (t) γˆi ∗ = −pi bi ψi (xi (t) − xdi )wi (x, t) ≤ 0, t ≥ 0. (16.33) ii) Otherwise, if u ˆi (t) ≥ 0, t ≥ 0, then kui (t) = ki (t), φui (t) = φi (t), and ψui (t) = ψi (t), and hence,   1 2 ˙ pi bi kui (t)(xi (t) − xdi ) + (ki (t) − kgi )ki (t) qi = kgi (xi (t) − xdi )2 ≤ 0,

(16.34)

 1 ˙ pi bi (φui (t) − uei )(xi (t) − xdi ) + (φi (t) − uei )φi (t) qˆi   −pi bi uei (xi (t) − xdi ) ≤ 0, if φi (t) = 0 and xi (t) − xdi ≥ 0, = (16.35)  0, otherwise, 

i h 1 pi bi (ψui (t) − ψi∗ )(xi (t) − xdi )wi (x, t) + (ψi (t) − ψi∗ )ψ˙i (t) γˆi  ∗ −pi bi ψi (xi (t) − xdi )wi (x, t) ≤ 0, if ψi (t) = 0    and (xi (t) − xdi )w(x(t), t) ≥ 0, = (16.36)    0, otherwise.

Hence, for all t > T and for t ≤ T such that w(x(t), t) >> 0,

V˙ (x(t), K(t), φ(t), Ψ(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) ≤ 0.

(16.37)

Next, consider the case where for t ∈ [0, T ] and i ∈ {1, . . . , m}, wi (x(t), t) < 0. If u ˆi (t) ≥ 0, t ∈ [0, T ], then (16.34)–(16.36) hold, and hence, (16.37) holds. Alternatively, if u ˆi (t) < 0, t ∈ [0, T ], then, since wi (x(t), t) < 0, xi (t) − xd (t) ≥ 0 does not necessarily hold, and hence, (16.32) and (16.33) do not necessarily hold. Now, note that if u ˆi (t) < 0, t ∈ [0, T ], then ui (t) = 0, and hence, the disturbed system (16.29) is uncontrolled. Furthermore, if u ˆi (t) < 0, t ∈ [0, T ], then kui (t) = 0, φui (t) = 0, and ψui (t) = 0, and hence, the Lyapunov derivative (16.31) along the trajectories of the closed-loop system (16.7), (16.21)–(16.25) can be nonnegative over the finite-time interval [0, T ]. Since A is Hurwitz and the continuous bounded disturbance signal can take nonnegative values over the time interval [0, T ], the trajectory of the system (16.7) remains bounded on the time interval [0, T ]. Furthermore, since for all t ≥ T (16.37) holds, it follows that there exists an increasing unbounded sequence

NNBook

September 1, 2009

502

CHAPTER 16

ˆ ˆ {tn }∞ n=0 , with t0 = 0, such that 0 < tn+1 − tn ≤ T , T > 0, n = 0, 1, . . . , T and V (˜ x(tn+1 )) − V (˜ ˜ = [x , vecT (K), φT , vecT (Ψ)] x(tn )) ≤ 0, where x and vec(·) denotes the column stacking operator. In addition, for all t ≥ 0, V (˜ x(t)) satisfies α(k˜ x(t)k) ≤ V (˜ x(t)) ≤ β(k˜ x(t)k), where α(·) and β(·) are class K functions defined on [0, ε) for all ε > 0. Hence, by Theorem 4.28 of [112] (adapted to the nonnegative system case), the solution (x(t), K(t), φ(t), Ψ(t)) ≡ (xe , Kg , ue , Ψ∗ ) of the closed-loop system given by (16.7) and (16.21)–(16.25) is Lyapunov stable. Moreover, since R is positive definite, it follows from Theorem 4.28 of [112] using similar arguments as in Theorem 4.2 of [112] that x(t) → xe as t → ∞. Finally, u(t) ≥≥ 0, t ≥ 0, is a restatement of (16.21). The nonnegativity of x(t), t ≥ 0, trivially follows from the fact that A is essentially nonnegative, the control input is nonnegative, and the disturbance signal is such that nonnegativity is preserved. Example 16.1. As an illustrative numerical example for the proposed disturbance rejection adaptive controller given by Theorem 16.1, consider the uncertain compartmental dynamical system given by (16.7) with       −1 1 1 0 0.1 0 A= , B= , Ψ∗ = , 1 −1 0 0.4 0 0.2 and initial condition x0 = [0.5, 0.75]T . Here, the disturbance vector is given by w(x(t), t) = [sin(x1 (t)ω1 t), 1 − cos(x2 (t)ω2 t)]T , where ω1 = 1 rad/sec and ω2 = 5 rad/sec, and xe = [1, 1]T . For the given A, B, and xe , ue satisfying (16.6) is ue = [0, 0]T . Here, we consider the control law given by (16.8)–(16.12) with q1 = q2 = 3, k1 (0) = k2 (0) = 0, qˆ1 = qˆ2 = 1, φ1 (0) = φ2 (0) = 0.01, γ1 = γ2 = 7, and ψ1 (0) = ψ2 (0) = 0.01. Figure 16.1 shows the controlled system trajectories for the cases where Ψ(t), t ≥ 0, is given by (16.12) and Ψ(t) ≡ 0, t ≥ 0. Figure 16.2 shows the control input and disturbance signal time histories. △

16.4 Adaptive Control for Linear Compartmental Dynamical Systems with L2 Disturbances In this section, we consider the problem for characterizing disturbance rejection control laws for linear compartmental dynamical systems with L2 exogenous disturbances. Specifically, we consider the controlled system (16.1) with disturbance d(x(t), t) given by (16.5) so that x(t) ˙ = Ax(t) + Bu(t) + J(x(t))w(t),

x(0) = x0 , n

t ≥ 0.

(16.38)

Define the sets S(xe ) , {y ∈ Rn : y = x − xe , x ∈ R+ } and S0 (xe ) , {y ∈ S(xe ) : there exists i ∈ {1, . . . , n} : yi = 0}. (16.39)

NNBook

September 1, 2009

503

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

x1(t): no disturbance rejection x1(t): with disturbance rejection

x1(t)

1.2 1 0.8 0.6 0

5

10

15

25

30

35

40

x2(t): no disturbance rejection x2(t): with disturbance rejection

1.2 x2(t)

20 Time [sec]

1 0.8 0

5

10

15

20 Time [sec]

25

30

35

40

u1(t), d1(t)

Figure 16.1 System trajectories with and without (Ψ(t) ≡ 0) disturbance rejection.

1

u (t)

0.5

d1(t)

1

0 −0.5 0

5

10

15

20 Time [sec]

25

30

35

40

2

d (t) 2

2

u (t), d (t)

u (t) 2

2

1 0 0

5

10

15

20 Time [sec]

25

30

35

40

Figure 16.2 Control input and disturbance signal.

Next, partition S(xe ) into 2n+1 nonintersecting sets S0 (xe ), S1 (xe ), . . . , S2n (xe ), where Sq (xe ) ⊂ S(xe ), q = 1, . . . , 2n , are open orthants such that Sq (xe ) ∩ S0 (xe ) = ∅. Clearly, S(xe ) = S0 (xe ) ∪ S1 (xe ) ∪ · · · ∪ S2n (xe ) and

NNBook

September 1, 2009

504

CHAPTER 16

for all i, j = 0, . . . , 2n , i 6= j, Sj (xe ) ∩ Si (xe ) = ∅. Next, define the function Vs : S(xe ) → R+ by Vs (y) = kyk1 , where k·k1 denotes the absolute sum norm. Note that Vs (·) is continuous everywhere in S(xe ) and Vs (y) = 0 if and only if y = 0, and Vs (y) > 0 for all y 6= 0. Furthermore, note that for every y ∈ Sq (xe ), q = 1, . . . , 2n , Vs (·) is continuously differentiable, whereas for every y ∈ S0 (xe ), Vs (·) is continuous, but not continuously differentiable. Theorem 16.4. Consider the linear uncertain dynamical system given by (16.38) where A is compartmental and eT A 0 are as defined in Section n 16.2. Furthermore, u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0 for all x0 ∈ R+ and t ≥ 0. Proof. First, note that even though the update law given by (16.40) is not continuous, the adaptive control ui (t), t ≥ 0, i = 1, . . . , m, is continuous. Hence, for each i ∈ {1, . . . , m}, a function ui : Iui (0) → R is a solution to (16.40) on the interval Iui (0) ⊆ R with initial condition ui (0) if ui (·) is continuous and ui (t) satisfies (16.40) for all t ∈ Iui (0) .

NNBook

September 1, 2009

505

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

Next, note that the system dynamics (16.38) can be rewritten as x(t) ˙ = A(x(t) − xe ) + B(u(t) − ue ) + J(x(t))w(t),

x(0) = x0 ,

t ≥ 0. (16.43)

Consider the Lyapunov function candidate for the closed-loop system (16.43) and (16.40) given by (16.42) and note that V (x, u) ≥ 0 for all x and u, and V (x, u) = 0 if and only if x = xe and u = ue . Equation (16.42) can be written as V (x, u) = Vs (y) + (u − ue )T Γ−1 (u − ue ). (16.44) Now, suppose that at some time t ∈ [0, ∞), y(t) = x(t) − xe 6∈ S0 (xe ). In this case, there exists an index q ∈ {1, . . . , 2n } such that y(t) ∈ Sq (xe ), and hence, (16.44) is continuously differentiable for every y ∈ Sq (xe ). Next, for every set Sq (xe ) we associate a single vector pq consisting of the components ±1 defined as pq , sgn(y), y ∈ Sq (xe ), where the sgn(·) µ , µ 6= 0, and operator is taken componentwise and is defined as sgn(µ) , |µ| sgn(0) , 0.

T −1 Next, rewriting (16.44) as V (x, u) = pT q y + (u − ue ) Γ (u − ue ), the derivative of V (x, u) along the trajectories of closed-loop system (16.43) and (16.40) is given by T T V˙ (x(t), u(t)) = pT q Ay(t) + pq B(u(t) − ue ) + pq J(y(t) + xe )w(t)

+2(u − ue )T Γ−1 u(t) ˙ T T = pq Ay(t) + pq J(y(t) + xe )w(t) m X + bi (ui (t) − uei )(pqi + 2u˙ i (t)).

(16.45)

i=1

T T Note that pT q Ay can be written as pq Ay = γ1 y1 +· · ·+γn yn . Since e A 0 or, equivalently, pqi = 1, and ui (t) = 0, t ≥ 0, then u˙ i (t) = 0, t ≥ 0, and bi (ui (t) − uei )(pqi + 2u˙ i (t)) = −bi uei ≤ 0,

t ≥ 0.

ii) If yi > 0 and ui (t) 6= 0, t ≥ 0, then u˙ i (t) = − 12 , t ≥ 0, and bi (ui (t) − uei )(pqi + 2u˙ i (t)) = 0,

t ≥ 0.

iii) If yi < 0, or, equivalently, pqi = −1, then u˙ i (t) = 12 , t ≥ 0, and bi (ui (t) − uei )(pqi + 2u˙ i (t)) = 0, Hence,

m X i=1

bi (ui (t) − uei )(pqi + 2u˙ i (t)) ≤ 0,

Now, using (16.47) and (16.48), it follows that √ n X α n T ˙ w w, V (x, u) ≤ −|γj ||yj | + β j=1

t ≥ 0.

t ≥ 0.

t ≥ 0.

(16.49)

Since Sq (xe ) is open, it follows from continuity of the system trajectories that there exists a time interval [t, T ), T > t, such that y(σ) = x(σ) − xe ∈ Sq (xe ) for all σ ∈ [t, T ). Integrating (16.49) over σ ∈ [t, T ) yields √ Z α n T T w (σ)w(σ)dσ V (x(T ), u(T )) ≤ β t Z T n X − |γj | |yj (σ)|dσ + V (x(t), u(t)). j=1

t

Noting that V (x(T ), u(T )) ≥ 0 for all possible x(T ) and u(T ), it follows that √ Z Z T n X α n T T w (σ)w(σ)dσ + V (x(t), u(t)), (16.50) |γj | |yj (σ)|dσ ≤ β t t j=1

where w(·) ∈ L2 . Now, since |γi | > 0, the nonnegative expression on the lefthand side of the inequality (16.50) is zero only if yi = 0 for all i = 1, . . . , n. Next, suppose that on an arbitrary time interval [t, T ), T > 0, y(σ) ∈ S0 (xe ) for all σ ∈ [t, T ), y(σ) 6= 0, and all yi (σ) remain either strictly

NNBook

September 1, 2009

507

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

positive, strictly negative, or zero. In this case, there exists a set of indices I = {i1 , . . . , ik } such that for all j ∈ I, yj (σ) ≡ 0. Now, it follows from (16.40) that uj (t), j ∈ {1, . . . , m} ∩ I, remains bounded since it either remains constant or decreases and is always nonnegative. Since for all j ∈ I and σ ∈ [t, T ), xj (σ) ≡ xej , it follows that x˙ j (σ) ≡ 0. Now, to examine the stability of the closed-loop system (16.38) and (16.40), we can consider a reduced dynamical system obtained from (16.43) by deleting the equations corresponding to x˙ j , j ∈ I, since these states are constant, and hence, do not affect the nonexpansivity constraint (16.50). To see this, consider a Lyapunov function candidate having the same form as (16.42) with the states xj and control inputs uj , where j ∈ I, deleted. Specifically, consider the reduced vectors yr and ur with components yi and uj , respectively, where i ∈ {1, . . . , n}\I and j ∈ {1, . . . , m}\I. In this case, the Lyapunov function candidate is given by V (yr , ur ) = Vs (yr ) + (ur − uer )T Γ−1 r (ur − uer ),

(16.51)

= diag[ bj1 , . . . , bjk ] and j1 , j2 , . . . , jk ∈ {1, . . . , m}\I. Next, where Γ−1 r repeating the analysis above, it can be shown that √ Z Z T X X α n T |γj | |yj (σ)|dσ ≤ wj2 (σ)dσ β t t j∈{1, 2, ..., n}\I

j∈{1, 2, ..., n}\I

+V (yr (t), ur (t)).

P

RT

|yj (σ)|dσ = 0 to the left-hand side of the √ R P T 2 inequality (16.52), and the nonnegative term α β n t j∈I wj (σ)dσ to the right-hand side of the inequality (16.52), inequality (16.52) still holds and has the same form as inequality (16.50). Hence, inequality (16.50) holds on the time interval [0, t) for all t > 0, that is, √ Z Z t n X α n t T w (σ)w(σ)dσ + V (x0 , u(0)), (16.53) |γj | |yj (σ)|dσ ≤ β 0 0 Now, adding

j∈I

|γj |

(16.52)

t

j=1

where w(·) ∈ L2 . Now, it follows from (16.49) that the solution (x(t), u(t)) ≡ (xe , ue ) of the undisturbed (J(x(t))w(t) ≡ 0) closed-loop system (16.38) and (16.40) is Lyapunov stable. Furthermore, by Theorem 4.2 of [112], n y(t) = x(t) − xe → 0 as t → ∞ for all x0 ∈ R+ . n

Finally, note that, since, by assumption, J(x)w(t) ≥≥ 0, x ∈ ∂R+ and t ≥ 0, and the control inputs ui (t), t ≥ 0, defined by (16.40) are nonnegative, u(t) ≥≥ 0, t ≥ 0, and hence, the trajectory of the system (16.38) remains in the nonnegative orthant. This completes the proof. It is important to note that the adaptive feedback control law u(t), t ≥

NNBook

September 1, 2009

508

CHAPTER 16

0, characterized by (16.40) is continuous, but not continuously differentiable. n Namely, even though for each i ∈ {1, . . . , m}, u˙ i (·) is continuous on R+ \ n S(xe ) and discontinuous on S(xe ), ui (·) is continuous on R+ . Hence, the closed-loop system (16.43) generates a continuous closed-loop vector field. Next, we present a different control law for the disturbance rejection problem considered in this section. The following assumption is needed for the next result. Assumption 16.2. Consider the controlled system (16.38) and let k·k : n Rn×d → R. Assume that J(x), x ∈ R+ , in (16.38) is such that kJ(x)k ≤ α, n (R) and P > 0 and R > 0 satisfy (16.30). x ∈ R+ , where α < λmin kP k Theorem 16.5. Consider the linear uncertain dynamical system given by (16.38) where A is Hurwitz and compartmental, B is nonnegative and n n given by (16.2), J : R+ → Rn×d is continuous and bounded on R+ , and w(·) ∈ L2 . Suppose Assumption 16.2 holds. Then the control law u(t) = [u1 (t), . . . , um (t)]T , with ui (t), t ≥ 0, i = 1, . . . , m, satisfying   0, if ui (t) = 0 and yi (t) , xi (t) − xdi ≥ 0, u˙ i (t) = (16.54)  −γi yi (t), otherwise, ui (0) ≥ 0, γi > 0, t ≥ 0, i = 1, . . . , m, guarantees that the solution (x(t), u(t)) ≡ (xe , ue ) of the undisturbed (J(x(t))w(t) ≡ 0) closed-loop system (16.38) and (16.54) is Lyapunov stable n and xi (t) → xdi , i = 1, . . . , m, as t → ∞ for all x0 ∈ R+ . Moreover, the solution x(t), t ≥ 0, to the disturbed closed-loop system (16.38) and (16.54) satisfies the nonexpansivity constraint Z Z t 1 t T w (σ)w(σ)dσ + C, γ (x(σ) − xe )T (x(σ) − xe )dσ ≤ 2 0 0 γ > 0, C ≥ 0, t ≥ 0. (16.55) n

Furthermore, u(t) ≥≥ 0, t ≥ 0, and x(t) ≥≥ 0 for all x0 ∈ R+ and t ≥ 0. Proof. Since A is Hurwitz it follows from Theorem 2.12 that there exist a positive diagonal matrix P and a positive-definite matrix R ∈ Rn×n satisfying (16.30). Now, consider the Lyapunov function candidate V (x, u) = (x − xe )T P (x − xe ) + (u − ue )T Γ−1 (u − ue ), (16.56) i h . Note that V (x, u) is nonnegative for all x where Γ−1 = diag γb11 , . . . , γbm m and u, and V (x, u) = 0 if and only if x = xe and u = ue . The Lyapunov derivative along the trajectories of the closed-loop system (16.38) and (16.54)

NNBook

September 1, 2009

509

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

is given by V˙ (x(t), u(t)) = 2(x(t) − xe )T P A(x(t) − xe ) + 2(x(t) − xe )T P B(u(t) − ue ) +2(x(t) − xe )T P J(x(t))w(t)) + 2(u(t) − ue )T Γ−1 u(t) ˙ T T = −(x(t) − xe ) R(x(t) − xe ) + 2(x(t) − xe ) P J(x(t))w(t)   m X 1 +2 pi bi (ui (t) − ue ) yi (t) + u˙ i (t) , t ≥ 0. (16.57) γi j=1

Now, for each i ∈ {1, . . . , m} and for the two cases given in (16.54) the last term on the right-hand side of (16.57) gives: i) If ui (t) = 0 and yi (t) ≥ 0, t ≥ 0, then u˙ i (t) = 0, t ≥ 0, and hence,   1 pi bi (ui (t) − ue ) yi (t) + u˙ i (t) = −pi bi ue yi (t) ≤ 0, t ≥ 0. γi ii) Otherwise, u˙ i (t) = −γi yi (t), and hence,   1 pi bi (ui (t) − ue ) yi (t) + u˙ i (t) = 0, γi

t ≥ 0.

Thus, it follows that V˙ (x(t), u(t)) ≤ −(x(t) − xe )T R(x(t) − xe ) + 2(x(t) − xe )T P J(x(t))w(t) 1 ≤ −(x(t) − xe )T R(x(t) − xe ) + αkP kkx(t) − xe k2 + kw(t)k2 2 1 ≤ (−λmin (R) + αkP k)kx(t) − xe k2 + wT (t)w(t) 2 1 T 2 ≤ −γkx(t) − xe k + w (t)w(t), t ≥ 0, 2 where, by Assumption 16.2, γ , λmin (R) − αkP k > 0. Hence, V˙ (x(t), u(t)) ≤ −γkx(t) − xe k2 + wT (t)w(t),

t ≥ 0.

(16.58)

Integrating (16.58) over the time interval [0, t), t ≥ 0, yields Z Z t 1 t T w (σ)w(σ)dσ − γ (x(σ) − xe )T (x(σ) − xe )dσ V (x(t), u(t)) ≤ 2 0 0 +V (x0 , u(0)). Noting that V (x(t), u(t)) ≥ 0 for all possible x(t) and u(t) it follows that Z Z t 1 t T T w (σ)w(σ)dσ + V (x0 , u(0)), (x(σ) − xe ) (x(σ) − xe )dσ ≤ γ 2 0 0 γ > 0, (16.59)

NNBook

510

September 1, 2009

CHAPTER 16

where w(·) ∈ L2 . Finally, it follows from (16.58) that the solution (x(t), u(t)) ≡ (xe , ue ) of the undisturbed (J(x(t))w(t) ≡ 0) closed-loop system (16.38) and (16.54) is Lyapunov stable. Furthermore, by Theorem 4.2 of [112], x(t) − xe → 0 n as t → ∞ for all x0 ∈ R+ . Now, the nonnegativity of the dynamical system (16.38) follows trivially by noting that control input u(t) defined by (16.54) n is nonnegative and J(x)w(t) ≥≥ 0 for all x ∈ ∂R+ and t ≥ 0. Example 16.2. In this example, we consider the adaptive controller given by (16.40). Specifically, consider the dynamical system given by (16.38) with     −1.5 2.0 1.5 0.5 0 0 0.9  , B =  0 0.33 0 , A =  0.5 −3 0.75 0.5 −2.5 0 0 0.2   0.001 0.02 0.003  0.012 0.001 0.032  , J(x) = 0.022 0.003 0.007

and initial condition x0 = [0.5, 2, 3]T . Here, the disturbance vector is given by w(x(t), t) = [e−λ1 t sin(x1 (t)ω1 t), e−λ2 t (1+cos(ω2 t)]T ), e−λ3 t (1+ sin(ω3 t))]T , where ω1 = 1 rad/sec, ω2 = 2 rad/sec, ω3 = 3 rad/sec, λ1 = 0.01, λ2 = 0.02, and λ3 = 0.03. The desired set point is xe = [2.5, 1, 1]T . For the given A, B, and xe , ue satisfying (16.6) is ue = [0.5, 2.55, 0.625]T . Here, we consider the control law given by (16.40) with u1 (0) = u2 (0) = u3 (0) = 0. The controlled and uncontrolled system trajectories are shown in Figure 16.3. The control input and disturbance signal are shown in Figure 16.4. Note that the proposed controller achieves disturbance rejection and the trajectory of the system converges to the desired set-point. △ Example 16.3. In this example, we consider the adaptive controller given by (16.54). Specifically, consider the dynamical system (16.38) with the same parameters and disturbance signal as in Example 16.2, and A given by   −1.5 2.0 1.5 1 . A =  0.5 −3 0.75 0.5 −2.5

For the given A, B, and xe = [2.5, 1, 1]T , ue satisfying (16.6) is ue = [0.5, 2.25, 0.625]T . Here, we consider the controller given by (16.54) with γ1 = γ2 = γ3 = 2 and u1 (0) = u2 (0) = u3 (0) = 0.01. Figure 16.5 shows the controlled and uncontrolled system trajectories and Figure 16.6 shows the control input and disturbance versus time. Since the disturbance signal is an L2 signal and A is Hurwitz, the states of the uncontrolled system (u(t) ≡ 0)

NNBook

September 1, 2009

511

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

x 1 (t)

3 2

x1(t): no disturbance rejection x1(t): with disturbance rejection

1 0

0

5

10

15 Time [sec]

20

25

30

x 2 (t)

2 x2(t): no disturbance rejection x2(t): with disturbance rejection 1

0

0

5

10

15 Time [sec]

20

25

30

20

25

30

x 3 (t)

3 x3(t): no disturbance rejection x3(t): with disturbance rejection

2 1 0

0

5

10

15 Time [sec]

Figure 16.3 System trajectories with and without disturbance rejection.

Control

3 u (t)

2

1

u (t) 2

u (t) 3

1

0

0

5

10

15 Time [sec]

20

25

30

0.08 d (t) 1

Disturbance

0.06

d (t) 2

0.04

d (t) 3

0.02 0 −0.02

0

5

10

15 Time [sec]

20

25

Figure 16.4 Control input and disturbance signal.

30

NNBook

September 1, 2009

512

CHAPTER 16

converge to zero. Alternatively, the controlled system with the adaptive disturbance rejection controller given by (16.54) guarantees that the system trajectories converge to the desired set point. △

16.5 Adaptive Control for Automated Anesthesia with Hemorrhage and Hemodilution Effects To illustrate the adaptive disturbance rejection control framework developed in Section 16.3 for general anesthesia, we consider a hypothetical model for the intravenous anesthetic propofol. The pharmacokinetics of propofol are described by the three-compartment model developed in Section 13.7 and shown in Figure 16.7, where x1 denotes the mass of drug in the central compartment, which is the site of drug administration and is identified with tissues whose drug concentration equilibrates, within the assumptions of the model, instantaneously with the site of drug administration. This implies tissues with high ratios of blood flow to tissue mass, such as those found in the myocardium, brain, etc., although compartment models do not strictly equate the compartment with any specific organ. The remainder of the drug in the body is assumed to reside in two peripheral compartments, corresponding to tissues with progressively slower drug equilibration with the site of administration. The masses in these compartments are denoted by x2 and x3 , respectively. It should be noted that pharmacokinetic compartmental models may utilize any number of compartments and the decision about model complexity depends largely on the resolution of concentration measurements as a function of time. The three-compartment model shown in Figure 16.7 has been found to be effective for describing the disposition of propofol after intravenous injection [215]. A mass balance for the whole compartmental system yields x˙ 1 (t) = −(a11 + a21 + a31 )x1 (t) + a12 x2 (t) + a13 x3 (t) +u(t) + d(x(t), t), x1 (0) = x10 , t ≥ 0, x˙ 2 (t) = a21 x1 (t) − a12 x2 (t), x2 (0) = x20 , x˙ 3 (t) = a31 x1 (t) − a13 x3 (t), x3 (0) = x30 ,

(16.60) (16.61) (16.62)

where x1 (t), x2 (t), x3 (t), t ≥ 0, are the masses in grams of propofol in the central compartment and Compartments 2 and 3, respectively, u(t), t ≥ 0, is the infusion rate in grams/min of the anesthetic drug propofol into the central compartment, d(x(t), t) is an exogenuous disturbance signal in grams/min which has been included to model the effect of hemorrhage on the dynamics of the mass of propofol in the central compartment, aij > 0, i 6= j, i, j = 1, 2, 3, are the rate constants in min−1 for drug transfer between compartments, and a11 > 0 is the rate constant in min−1 of drug metabolism and elimination (metabolism typically occurs in the liver) from the central

NNBook

September 1, 2009

513

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

2

x1(t): no disturbance rejection

1

x1(t): with disturbance rejection

1

x (t)

3

0

0

10

20

30 Time [sec]

2

x (t)

2

40

50

60

x2(t): no disturbance rejection x2(t): with disturbance rejection

1 0

0

10

20

30 Time [sec]

40

50

60

x3(t): no disturbance rejection x3(t): with disturbance rejection

2

3

x (t)

3

1 0

0

10

20

30 Time [sec]

40

50

60

Figure 16.5 System trajectories with and without disturbance rejection. 2.5 2

u (t)

1.5

u (t)

1

u3 (t)

Control

1 2

0.5 0

0

10

20

30 Time [sec]

40

50

60

0.08

Disturbance

d (t) 1

0.06

d (t)

0.04

d (t)

2 3

0.02 0 −0.02

0

10

20

30 Time [sec]

40

50

Figure 16.6 Control input and disturbance signal.

60

NNBook

September 1, 2009

514

CHAPTER 16

u ≡ Continuous infusion d(x, t) ≡ Hemorrhage a12 x2

a31 x1 Compartment 1

Compartment 2

a21 x1

Compartment 3

a13 x3 a11 x1 ≡ Elimination (liver, kidney)

Figure 16.7 Three-compartment mammillary model for disposition of propofol.

compartment. Next, note that (16.60)–(16.62) can be written in state (16.1) with x = [x1 , x2 , x3 ]T ,    −(a11 + a21 + a31 ) a12 a13 a21 −a12 0 , B =  A =  a31 0 −a13   ∗ −ψ w(x, t) , 0 d(x, t) =  0

space form  1 0 , 0 (16.63)

where ψ ∗ is an unknown positive constant and the function w(x, t) represents blood loss due to hemorrhage. A model for the effect of hemorrhage on the dynamics of the mass of propofol in the central compartment is developed below. Since we are more interested in drug effect (depth of hypnosis) rather than drug concentration, we use an electroencephalogram (EEG) signal as a measure of the hypnotic effect of propofol on the brain [273]. In particular, we use the BIS signal to measure drug concentration given by the empirical relationship   cγeff , (16.64) BIS(ceff ) = BIS0 1 − γ ceff + ECγ50

where BIS0 denotes the baseline (awake state) value and, by convention, is typically assigned a value of 100, ceff is the propofol concentration in grams/liter in the effect-site compartment (brain), EC50 is the concentration at half maximal effect and represents the patient’s sensitivity to the drug, and γ determines the degree of nonlinearity in (16.64). As discussed in Section 12.13, the effect-site compartment concentration is related to the concentration in the central compartment by the first-order model [261] c˙eff (t) = aeff (x1 (t)/Vc − ceff (t)),

ceff (0) = x1 (0)/Vc ,

t ≥ 0,

(16.65)

where aeff in min−1 is an unknown positive time constant and Vc is the

NNBook

September 1, 2009

515

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS 100

90

80

BIS Index [score]

70

60

Target BIS

50

40

30

← Linearized range

20

10

EC

50

0

0

1

2

3

4

= 3.4 [µg/ml] 5

6

7

8

9

10

Effect site concentration [µg/ml]

Figure 16.8 BIS index versus effect-site concentration.

volume in liters of the central compartment. As noted in [200], Vc can be approximately calculated by Vc = (0.159 l/kg)(M kg), where M is the mass in kilograms of the patient, and aeff is obtained as aeff = 0.693/2.2 min = 0.3150 min−1 , where 2.2 min is the half-time ke0 value reported in [282]. In reality, the effect-site compartment equilibrates with the central compartment in a matter of a few minutes. However, in the case of significant blood loss, this equilibration can be slowed down. The parameters aeff , EC50 , and γ are determined by data fitting and vary from patient to patient. BIS index values of 0 and 100 correspond, respectively, to an isoelectric EEG signal (no cerebral electrical activity) and an EEG signal of a fully conscious patient; the range between 40 and 60 indicates a moderate hypnotic state [93]. In the following numerical simulation we set EC50 = 3.4 µg/ml, γ = 3, and BIS0 = 100, so that the BIS signal is shown in Figure 16.8. The values for the pharmacodynamic parameters (EC50 , γ) are within the typical range of those observed for ligand-receptor binding [73, 169]. The target (desired) BIS value, BIStarget , is set at 50. In this case, the linearized BIS function about the target BIS value is given by BIS(ceff ) ≃ BIS(EC50 ) −BIS0 ·

ECγ50

γcγ−1 eff · γ (ceff + ECγ50 )2

= bBIS + kBIS · ceff ,

where bBIS = 125 and kBIS = −22.06 ml/µg.

ceff =EC50

· (ceff − EC50 ) (16.66)

NNBook

September 1, 2009

516

CHAPTER 16

During surgery hemorrhage and hemodilution (i.e., increase in fluid content of blood resulting in reduced concentration of red blood cells in the blood) often take place, which can affect the concentration of a drug in the blood, and hence, the level of patient sedation [170]. Hence, it is of paramount importance that the adaptive controller is able to compensate for the effects of hemorrhage and hemodilution. In particular, during hemorrhage when perfusion pressure falls, perfusion of certain regions (e.g., brain and heart) takes precedence over perfusion of other regions, and blood flow to these other regions is significantly slowed down. Such an effect can be modeled by decreasing the transfer coefficients between compartments, as well as adding an exogenous disturbance to the baseline pharmacokinetic system to account for the effect of hemorrhage on the dynamics of the mass of propofol in the central compartment. The system equations (16.60)–(16.62) then take the form of (16.1). To develop a disturbance model for hemorrhage and hemodilution on the dynamics of the mass of propofol, we assume that the bleeding is arterial and the size of the holes in the bleeding vessels remain constant during the period of hemorrhaging. Assuming that blood loss occurs only through the central compartment, we model the disturbance signal (16.3) as d(x(t), t) = [βc(t)BL(x1 (t), t), 0, 0]T , where β is a dimensionless unknown positive constant coefficient, c(t) = x1 (t)/Vc (t) is the concentration of propofol in the central compartment in grams/liter, and BL(x1 (t), t) is the rate of blood loss in liters/min. Using [167], we model blood loss rate as BL(x1 (t), t) =

BL0 σ(t)M AP (x1 (t), t), M AP0

t ≥ 0,

(16.67)

where BL0 is the initial rate of blood loss, M AP0 is the initial mean arterial pressure, M AP (x1 (t), t) is the mean arterial pressure at time t ≥ 0, and σ : [0, ∞) → {0, 1} is a piecewise switching function describing a particular hemorrhage scenario, including hemorrhage start and stop times. Note that the blood pressure is a function of propofol mass in the central compartment. Using the linear approximation of the BIS index given by (16.66), the disturbance signal can be rewritten in the form 0 d(x(t), t) = [ψ ∗ w(x(t), t), 0, 0]T , where ψ ∗ = β BL kBIS is an unknown parameter

(x1 (t),t) (BIS(t)−bBIS ). In the numerical simulation, and w(x(t), t) = σ(t) M AP M AP0 the dimensionless parameter β is set to 8.25, BL0 is 0.216 l/min, and M AP0 = 80 mm Hg.

To proceed we need to develop a model for the relationships between blood pressure, blood volume, and propofol concentration. By definition (of

NNBook

September 1, 2009

517

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

vascular resistance), mean arterial blood pressure is given by [13] M AP (x1 (t), t) = CO(x1 (t), t) × SV R(x1 (t), t) + CV P (t),

t ≥ 0, (16.68)

where CO(x1 (t), t) is cardiac output (the volume of blood the heart pumps per minute), SV R(x1 (t), t) is systemic vascular resistance (an index of arteriolar compliance or constriction throughout the body), and CV P (t) is central venous pressure (the venous pressure of the right atrium of the heart). Since CV P (·) is usually an order of magnitude less than mean arterial pressure, (16.68) can be approximated as M AP (x1 (t), t) = CO(x1 (t), t) × SV R(x1 (t), t),

t ≥ 0.

(16.69)

Since cardiac output is equal to the product of heart rate HR and stroke volume SV (the volume of blood pumped per heart beat) it follows that M AP (x1 (t), t) = HR(x1 (t), t) × SV (x1 (t), t) × SV R(x1 (t), t),

t ≥ 0. (16.70) If the contractile strength of the heart remains constant during hemorrhage, to a first-order approximation, stroke volume can be modeled as SV (x1 (t), t) = (SV0 × BV (t))/BV0 ,

t ≥ 0,

(16.71)

where SV0 is the baseline stroke volume, BV (t) is the blood volume during hemorrhage, and BV0 is the baseline blood volume. From (16.70) and (16.71), it follows that mean arterial pressure is proportional to blood volume. However, there are physiological compensatory mechanisms that act to maintain blood pressure in the face of hemorrhage. The autonomic nervous system responds to blood loss with an increase in sympathetic nervous tone, leading to an increase in both heart rate and systematic vascular resistance, and also the contractile strength of the heart. In otherwise healthy conscious individuals, these mechanisms are so effective that blood pressure can be maintained even after significant blood loss. However, in the anesthetized individual, the situation is more complex, as anesthetic agents, including propofol, blunt these compensatory mechanisms. Thus, for our simulation we must consider the relationship between blood loss and blood pressure to be a spectrum with two extremes; namely, ranging from completely effective compensatory mechanisms with the blood pressure maintained at baseline levels despite blood loss, to completely blunted compensatory mechanisms in which blood pressure is proportional to the blood volume. To our knowledge, this relationship has never undergone mathematical modeling. Given its nearly ubiquitous value for modeling biological phenomena, we believe that using a modified Hill equation is a plausible approach for modeling the relationship between blood pressure, blood volume, and

NNBook

September 1, 2009

518

CHAPTER 16

propofol concentration. Specifically, for our simulations we assume that   M AP0 cα (t) BV (t) × α M AP (x1 (t), t) = M AP0 − M AP0 − α , BV0 c (t) + C50 t ≥ 0, (16.72) where c(t) is propofol concentration in the central compartment, C50 is an empirical constant which defines the midpoint of the relationship between propofol concentration and the blunting of compensatory mechanisms for the maintenance of blood pressure, and α is an empirical constant that describes the steepness of this relationship. Note that if C50 is zero, compensatory mechanisms are totally ineffective and mean arterial pressure is proportional to blood volume, while if C50 is large, blood pressure is largely maintained despite hemorrhage. We emphasize that this is a hypothetical relationship which we postulate in order to proceed with simulations. While the relationship is hypothetical, it is biologically plausible, and by appropriate choices of the empirical constants C50 and α, the spectrum of relationship between blood pressure, blood loss, and propofol concentration may be explored. In order to account for the two extremes between blood pressure being proportional to blood volume and blood pressure maintained at baseline levels despite blood loss, we have performed simulations using multiple values of C50 and α with C50 ranging from 0.5 to 10 in increments of 0.1 and α ranging from 2 to 8 in increments of 0.5. Our numerical study showed imperceptible differences indicating that the proposed disturbance rejection algorithm is very robust. In the simulation shown below we set C50 = 2 µg/ml and α = 3. Finally, we note that in actual surgery the mean arterial pressure is measured and does not need to be modeled. The dynamic behavior of the blood volume components involving the red blood cell volume y1 (t) and the plasma volume y2 (t) can be described by [167] y˙ 1 (t) = r(t) − y1 (t)BL(x1 (t), t)/BV (t), y1 (0) = y10 , t ≥ 0, (16.73) y˙ 2 (t) = CL(t) + CR(t) + T RAN S(t) − y2 (t)BL(x1 (t), t)/BV (t), y2 (0) = y20 , (16.74) where r(t) is the packed red blood cell infusion rate, CL(t) is the colloid infusion rate, CR(t) is the crystalloid infusion rate, and T RAN S(t) represents the effect of the Starling transcapillary refill [167], BL(x1 (t), t) is the rate of blood loss in liters/min, and BV (t) is the blood volume in the central compartment, which can be approximated by BV (t) = y1 (t) + y2 (t),

t ≥ 0.

(16.75)

NNBook

September 1, 2009

519

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

BL(t) r(t) CR(t)

0.2

0.15

0.1

0.05

0

0

20

40

60

80

100

120

Starling Transcapillary Refill Rate [liter/min]

Blood Loss and Infusion Rates [liter/min]

−3

0.25

x 10

1.5

1

0.5

0

0

20

40

Time [min]

60

80

100

120

Time [min]

6

85

5 80 4 3 2

0

20

40

60

80

100

120

Time [min] Hematocrit [%]

50 40

75 70 65 60 55

30 20

Blood Pressure [mmHg]

Blood Volume [liters]

Figure 16.9 Blood loss rate, infusion rates and transcapillary refill rate versus time.

0

20

40

60

Time [min]

80

100

120

50

0

20

40

60

80

100

120

Time [min]

Figure 16.10 Blood volume, hematocrit, and mean arterial pressure versus time.

For our simulation we assume that the initial blood volume BV0 is 5 liters. The initial red blood cell volume y10 is assumed to be 45% of BV0 and the initial plasma volume y20 is 55% of BV0 . The time histories of the blood loss rate, as well as the red blood cell and crystalloid infusion rates, and the Starling transcapillary refill rate are shown in Figure 16.9. In the simulation we assume that the colloid infusion rate is zero. For the chosen parameters, the dynamics of blood volume BV (t), as well as the hematocrit, that is, the ratio of red blood cell volume to the total blood volume, and mean arterial pressure M AP (x1 (t), t) are shown in Figure 16.10. During actual surgery neither the mass of propofol x1 (t) nor the concentration of propofol c(t) in the central compartment can be measured in real time. Moreover, due to hemorrhage and hemodilution, the blood volume and hence, the volume of the central compartment are not constant. As a result, the desired mass xd of propofol in the central compartment is not a fixed set point but rather a bounded unknown function of time.

NNBook

September 1, 2009

520

CHAPTER 16

This makes the automated anesthesia control problem with hemorrhage and hemodilution effects more challenging than the automated anesthesia problem without modeling these effects [120]. However, using the BIS signal it is possible to achieve the desired level of hypnotic state. In particular, using the linearized BIS given by (16.66), and assuming that the concentrations of propofol in the effect site and central compartment are equal, it follows that BIS(ceff ) − BIStarget ≃ kBIS (c(t) − ctarget ) =

kBIS (x1 (t) − xd1 (t)), (16.76) Vc (t)

where the volume of the central compartment Vc (t) and the desired level of mass of propofol in the central compartment xd1 (t) are bounded. Hence, the difference between the BIS index and its target value is approximately equal to the difference between the mass of propofol in the central compartment and its desired level multiplied by the bounded time-varying negative gain kBIS Vc (t) . In light of the above discussion we use the controller architecture of Theorem 16.3 with i = 1, x1 (t) − xd1 = BIStarget − BIS(ceff (t)), q1 = q, qˆ1 = qBIS1 , γ1 = qBIS2 , where q = 2.0 × 10−8 g/min2 , qˆBIS1 = 1.0 × 10−5 g/min2 , qˆBIS2 = 4.0 × 10−3 g/min2 , k(0) = 0, φ(0) = 0.01 g/min−1 , and ψ(0) = 0, for maintaining a desired constant level of depth of anesthesia while accounting for hemorrhage and hemodilution. It is important to note that during actual surgery the BIS signal is obtained directly from the EEG and not (16.64). Furthermore, since our adaptive controller only requires the error signal BIS(t) − BIStarget over the linearized range of (16.64), we do not require knowledge of the slope of the linearized equation (16.66), nor do we require knowledge of the parameters γ and EC50 . To numerically illustrate the efficacy of the proposed adaptive control law, we use the average set of pharmacokinetic parameters given in [98] for 29 patients. Specifically, we assume M = 70 kg, a11 = 0.152 min−1 , a21 = 0.207 min−1 , a12 = 0.092 min−1 , a31 = 0.040 min−1 , and a13 = 0.0048 min−1 [98]. Figures 16.11 and 16.12 show the central compartment and effect-site concentrations versus time, and the control, disturbance, and BIS signal versus time. Note that the effect-site compartment equilibrates with the central compartment in a matter of several minutes. In addition, note that when the adaptive controller does not account for hemorrhage and hemodilution the BIS index drops dangerously into the low 20s increasing the possibility of patient respiratory and cardiovascular collapse.

NNBook

September 1, 2009

521

DISTURBANCE REJECTION FOR COMPARTMENTAL SYSTEMS

Without Disturbance Rejection

With Disturbance Rejection 4.5

Propofol concentration [µg/ml]

Propofol concentration [µg/ml]

6 5 4 3 2 Central Compartment Effective Site

1 0

0

20

40

60

80

100

4 3.5 3 2.5 2 1.5 1

0

120

Central Compartment Effective Site

0.5 0

20

40

Time [min]

60

80

100

120

Time [min]

20

100 with disturbance rejection without disturbance rejection

15

5

70

0

20

40

60

80

100

120

Time [min] 8

BIS index [score]

80

0

60 50 40

6

30

4

20

2

10

0

0

20

40

60

Time [min]

80

100

with disturbance rejection without disturbance rejection

90

10

Disturbance [mg/min]

Propofol infusion rates [mg/min]

Figure 16.11 Concentration of propofol with and without disturbance rejection.

120

0

0

20

40

60

80

100

120

Time [min]

Figure 16.12 Control signal (infusion rate), system disturbance, and BIS index versus time.

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Seventeen

Limit Cycle Stability Analysis and Control for Respiratory Compartmental Models

17.1 Introduction Acute respiratory failure due to infection, trauma, and major surgery is one of the most common problems encountered in intensive care units, and mechanical ventilation is the mainstay of supportive therapy for such patients. Numerous mathematical models of respiratory function have been developed in the hope of better understanding pulmonary function and the process of mechanical ventilation [17, 46, 77, 214, 299]. However, the models that have been presented in the medical and scientific literature have typically assumed homogeneous lung function. For example, in analogy to a simple electrical circuit, the most common model has assumed that the lungs can be viewed as a single compartment characterized by its compliance (the ratio of compartment volume to pressure) and the resistance to air flow into the compartment [46, 214, 299]. While a few investigators have considered two-compartment models, reflecting the fact that there are two lungs (right and left), there has been little interest in more detailed models [62, 149, 276]. However, the lungs, especially diseased lungs, are heterogeneous, both functionally and anatomically, and are comprised of many subunits, or compartments, that differ in their capacities for gas exchange. Realistic models should take this heterogeneity into account. While more sophisticated models entail greater complexity, since the models are readily presented in the context of dynamical systems theory, sophisticated mathematical tools can be applied to their analysis. Compartmental lung models are described by a state vector, whose components are the volumes of the individual compartments. One interesting and important question is the stability, in the sense of dynamical systems theory, of the model. For a simple one-compartment model, it is easy to demonstrate that the model exhibits an asymptotically stable limit cycle behavior. And

NNBook

September 1, 2009

524

CHAPTER 17

indeed, in clinical practice it appears that the total lung volume converges to the steady-state end-inspiratory and end-expiratory values after the institution of mechanical ventilation. However, a more subtle question for a multicompartment lung model is whether the volumes in the individual compartments could be unstable, even when the total volume of the lung (the sum of all the compartment volumes) converges to a steady-state value. That is, is it possible that individual compartment volumes oscillate or even demonstrate chaotic behavior while the total lung volume is stable? This question has interesting clinical implications as there is also heterogeneity in the amount of blood flowing to individual subunits of the lung. If there is significant disparity in the ratio of ventilation (reflected in the compartment volume) to blood flow, gas exchange is impaired, resulting in decreases in the oxygen or increases in the carbon dioxide content of blood, which is a serious clinical problem. Instability of the compartment volumes could be reflected in unstable measures of basic pulmonary function, such as oxygen or carbon dioxide levels in the blood. In this chapter, we develop a generalized multicompartment lung model and subsequently analyze its stability properties. Specifically, we use compartmental dynamical system theory and Poincar´e maps to model and analyze the dynamics of a pressure-limited respirator and lung mechanics system, and show that the periodic orbit generated by this system is globally asymptotically stable. Furthermore, we show that the individual compartmental volumes, and hence the total lung volume, converge to steady-state end-inspiratory and end-expiratory values.

17.2 Ultrametric Matrices, Periodic Orbits, and Poincar´ e Maps In this section, we introduce several definitions and some key results that are necessary for developing the main results of this chapter. The following definition introduces the notion of strictly ultrametric matrices. Definition 17.1 ([216]). Let A ∈ Rn×n be nonnegative. A is strictly ultrametric if A is symmetric, A(i,i) > max{A(i,k) : k = 1, . . . , n, k 6= i}, i = 1, . . . , n, and A(i,j) ≥ min{A(i,k) , A(k,j) }, k = 1, . . . , n, i, j = 1, . . . , n, i 6= j. The following propositions and lemma are key in establishing the main results of the chapter. Proposition 17.1. The following statements hold: i ) Let λ1 , λ2 ≥ 0 be such that λ1 + λ2 > 0 and let A1 , A2 ∈ Rn×n be strictly ultrametric. Then λ1 A1 + λ2 A2 is strictly ultrametric.

NNBook

September 1, 2009

525

RESPIRATORY COMPARTMENTAL MODELS

ii ) Let x ∈ Rn be such that xi = 0 or 1, i = 1, . . . , n, and let P ∈ Rn×n be a positive diagonal matrix. Then P + xxT is a strictly ultrametric matrix. Proof. Statement i ) is a direct consequence of Definition 17.1. To show ii ) let A , P + xxT and note that A is symmetric and  P(i,i) + x2i , if i = j, A(i,j) = xi xj , if i 6= j. Hence, if xi = 0, then max{A(i,k) : k = 1, . . . , n, k 6= i} = 0, i = 1, . . . , n, which implies that A(i,i) = P(i,i) > max{A(i,k) : k = 1, . . . , n, k 6= i}, i = 1, . . . , n. Alternatively, if xi = 1, then A(i,i) = P(i,i) + 1 > max{xk : k = 1, . . . , n, k 6= i}, i = 1, . . . , n. Furthermore, for i 6= j, A(i,j) = xi xj and  0, if xi xj = 0, min{A(i,k) , A(k,j) } = xk , otherwise. In either case, A(i,j) ≥ min{A(i,k) , A(k,j) }, k = 1, . . . , n, i, j = 1, . . . , n, i 6= j, which implies that A is strictly ultrametric. Lemma 17.1 ([216]). Let A ∈ Rn×n be such that A ≥≥ 0. If A is strictly ultrametric, then −A−1 is essentially nonnegative and A−1 e ≥≥ 0. Proposition 17.2. Let A ∈ Rn×n and assume that there exists an n×n matrix P > 0 such that AT P + P A < 0. (17.1) T

Then eA P eA < P. Proof. Define R , −(AT P + P A) > 0 and note that (17.1) implies Z ∞ T P = eA t ReAt dt. (17.2) 0

T

Next, pre- and postmultiplying (17.2) by eA and eA , respectively, yields Z ∞ T T eA P eA = eA (t+1) ReA(t+1) dt Z0 ∞ T = eA t ReAt dt 1 Z ∞ Z 1 T T = eA t ReAt − eA t ReAt dt 0 0 Z 1 T = P− eA t ReAt dt 0

< P,

which proves the result.

NNBook

September 1, 2009

526

CHAPTER 17

It is well known that A is Hurwitz if and only if eA is Schur. Hence, it follows from Proposition 17.2 that the quadratic Lyapunov function V (x) = xT P x can be used to establish the stability of both A and eA . In this chapter, we analyze the stability of periodic orbits using Poincar´e maps [125, 307]. To state Poincar´e’s theorem, consider the nonlinear periodic dynamical system x(t) ˙ = f (t, x(t)),

x(0) = x0 ,

t ∈ I x0 ,

(17.3)

where x(t) ∈ D ⊆ Rn , t ∈ Ix0 , is the system state vector, D is an open set, f : [0, ∞) × D → Rn satisfies f (t, x) = f (t + T, x), x ∈ D, t ≥ 0, for some T > 0, and Ix0 = [0, τx0 ), 0 < τx0 ≤ ∞, is the maximal interval of existence for the solution x(·) of (17.3). It is assumed that f (·, ·) is such that the solution to (17.3) is unique for every initial condition in D and jointly continuous in t and x0 . A sufficient condition ensuring this is Lipschitz continuity of f (t, ·) : D → Rn for all t ∈ [0, t1 ] and continuity of f (·, x) : [0, t1 ] → Rn for all x ∈ D. Here, we assume that all solutions to (17.3) are bounded over Ix0 , and hence, by the Peano-Cauchy theorem can be extended to infinity. Next, we introduce the notions of periodic solutions and periodic orbits for (17.3). For the next definition, we denote the solution x(·) to (17.3) with initial conditon x0 ∈ D by s(t, x0 ).1 Definition 17.2. A solution s(t, x0 ) of (17.3) is periodic if there exists a finite time T > 0 such that s(t + T, x0 ) = s(t, x0 ) for all t ≥ 0. A set O ⊂ D is a periodic orbit of (17.3) if O = {x ∈ D : x = s(t, x0 ), 0 ≤ t ≤ T } for some periodic solution s(t, x0 ) of (17.3). Next, we introduce the notions of Lyapunov and asymptotic stability of a periodic orbit of the nonlinear dynamical system (17.3). For this definition, recall that dist(p, M) denotes the distance from a point p to any point in the set M, that is, dist(p, M) , inf x∈M kp − xk. Definition 17.3. A periodic orbit O of (17.3) is Lyapunov stable if, for all ε > 0, there exists δ = δ(ε) > 0 such that if dist(x0 , O) < δ, then dist(s(t, x0 ), O) < ε, t ≥ 0. A periodic orbit O is asymptotically stable if O is Lyapunov stable and there exists ε > 0 such that if dist(x0 , O) < ε, then dist(s(t, x0 ), O) → 0 as t → ∞. To proceed, we assume that for the point p ∈ D, the dynamical system (17.3) has a periodic solution s(t, p), t ≥ 0, with period T > 0 that generates 1 Note that since (17.3) is a time-varying dynamical system it is typical to denote its solution as sˆ(t, t0 , x0 ) to indicate the dependence on both the initial time t0 and the initial state x0 . In this chapter, we assume that t0 = 0 and define s(t, x0 ) , sˆ(t, 0, x0 ).

NNBook

September 1, 2009

527

RESPIRATORY COMPARTMENTAL MODELS

the periodic orbit O , {x ∈ D : x = s(t, p), 0 ≤ t ≤ T }. Next, let Q ⊂ D be a neighborhood of the point p and define the Poincar´e return map P : Q → D by P (x) , s(T, x), x ∈ Q. (17.4) Furthermore, define the discrete-time dynamical system given by z(k + 1) = P (z(k)), z(0) ∈ Q, k ∈ Z+ .

(17.5)

Clearly x = p is a fixed point of (17.5) since p = s(T, p) = P (p). Theorem 17.1. Consider the nonlinear periodic dynamical system (17.3) with the Poincar´e map defined by (17.4). Assume that the point p ∈ D generates the periodic orbit O , {x ∈ D : x = s(t, p), 0 ≤ t ≤ T }, where s(t, p), t ≥ 0, is the periodic solution with period T . Then the following statements hold: i ) p ∈ D is a Lyapunov stable fixed point of (17.5) if and only if the periodic orbit O generated by p is Lyapunov stable. ii ) p ∈ D is an asymptotically stable fixed point of (17.5) if and only if the periodic orbit O generated by p is asymptotically stable. △

△

Proof. Define x1 (t) = x(t) and x2 (t) = t, and note that the solution x(t), t ≥ 0, to the nonlinear periodic dynamical system (17.3) can be equivalently characterized by the solution x1 (t), t ≥ 0, to the nonlinear autonomous dynamical system x˙ 1 (t) = f (x2 (t), x1 (t)), x1 (0) = x0 , x2 (t) = t mod T, x2 (0) = 0.

t ≥ 0,

(17.6) (17.7)

Since p ∈ D generates a periodic solution to (17.3) it follows that the point [p, 0]T ∈ D × [0, T ] generates a periodic solution to (17.6) and (17.7). Next, it can be shown that the map P : Q → D given by (17.4) is a Poincar´e map for (17.6) and (17.7) (see [307, p. 127] for details). Now, the result is a direct consequence of the standard Poincar´e theorem [112, p. 293]. Finally, in this chapter, we develop a multicompartment lung model based on a directed tree architecture. The following definitions are necessary for the main results of this chapter. Definition 17.4 ([286]). A weighted directed graph G is a triple (V, E, W ), where V = {v1 , v2 , . . . , vN } is the set of vertices, E = {e1 , e2 , . . . , eM } ⊆ V × V is the set of edges, and W ∈ RN ×N is the weighted adjacency matrix. Every edge el ∈ E corresponds to an ordered pair of vertices (vi , vj ) ∈ V × V, where vi and vj are the initial and terminal vertices of

NNBook

528

September 1, 2009

' $

CHAPTER 17

c



R



papp Figure 17.1 Single-compartment lung model.

the edge el . In this case, el is incident into vj and incident out of vi . The adjacency matrix W is such that W(i,j) > 0, i, j = 1, . . . , N, if (vi , vj ) ∈ E, and W(i,j) = 0 otherwise. The in-degree di (vi ) of vi is the number of edges incident into vi and the out-degree do (vj ) of vj is the number of edges incident out of vj . A directed path from vi1 to vik is a set of distinct vertices {vi1 , vi2 , . . . , vik } such that (vij , vij+1 ) ∈ E, j = 1, . . . , k − 1. A vertex vi is a root of G if, for every vj 6= vi , there exist directed paths from vi to vj . G is connected if, for every pair of vi , vj ∈ V, there exists vk ∈ V such that there are directed paths from vk to vi and vk to vj . A vertex vi ∈ V is a leaf of G if do (vi ) = 0. Definition 17.5 ([286]). A weighted directed graph G is a weighted directed tree if G is connected and there exists a vertex vi ∈ V such that di (vi ) = 0 and di (vj ) = 1, vj ∈ V \ {vi }. Note that if G is a weighted directed tree, then there exists exactly one root vi ∈ V and exactly one directed path from vi to vj for all vj ∈ V \ {vi }. See [286] for further details.

17.3 Compartmental Modeling of Lung Dynamics: Dichotomy Architecture In this section, we develop a general mathematical model for the dynamic behavior of a multicompartment respiratory system in response to an arbitrary applied inspiratory pressure. Here, we assume that the bronchial tree has a dichotomy architecture [302], that is, in every generation each airway unit branches in two airway units of the subsequent generation. First, however, we start by considering a single-compartment lung model as shown in Figure 17.1. In this model, the lungs are represented as a single lung unit with compliance c connected to a pressure source by an airway unit with resistance (to air flow) of R. At time t = 0, an arbitrary pressure pin (t)

NNBook

September 1, 2009

529

RESPIRATORY COMPARTMENTAL MODELS

is applied to the opening of the parent airway, where pin (t) is determined by the mechanical ventilator. A typical choice for pin (t) is pin (t) = αt + β, where α and β are positive constants. This pressure is applied to the airway opening over the time interval 0 ≤ t ≤ Tin , which is the inspiratory part of the breathing cycle. At time t = Tin , the applied airway pressure is released and expiration takes place passively, that is, the external pressure is only the atmospheric pressure pex (t) during the time interval Tin ≤ t ≤ Tin + Tex , where Tex is the duration of expiration. The state equation for inspiration (inflation of lung) is given by 1 Rin x(t) ˙ + x(t) = pin (t), c

x(0) = xin 0 ,

0 ≤ t ≤ Tin ,

(17.8)

where x(t) ∈ R, t ≥ 0, is the lung volume, Rin ∈ R is the resistance to air flow during the inspiration period, xin 0 ∈ R is the lung volume at the start of the inspiration and serves as the system initial condition. We assume that expiration is passive (due to elastic stretch of the lung unit). During the expiration process, the state equation is given by 1 Rex x(t) ˙ + x(t) = pex (t), c

x(Tin ) = xex 0 ,

Tin ≤ t ≤ Tin + Tex , (17.9)

where x(t) ∈ R, t ≥ 0, is the lung volume, Rex ∈ R is the resistance to air flow during the expiration period, and xex 0 ∈ R is the lung volume at the start of expiration. Next, we develop the state equations for inspiration and expiration for a 2n -compartment model, where n ≥ 0. In this model, the lungs are represented as 2n lung units which are connected to the pressure source by n generations of airway units, where each airway is divided into two airways of the subsequent generation, leading to 2n compartments (see Figure 17.2 for a four-compartment model). Let ci , i = 1, 2, . . . , 2n , denote the compliance of each compartment in (respectively, Rex ), i = 1, 2, . . . , 2j , j = 0, . . . , n, denote the and let Rj,i j,i resistance (to air flow) of the ith airway in the jth generation during the in (respectively, Rex ) inspiration (respectively, expiration) period with R01 01 denoting the inspiration (respectively, expiration) of the parent (i.e., 0 generation) airway. As in the single-compartment model we assume that a pressure of pin (t) is applied during inspiration. Next, let xi , i = 1, 2, . . . , 2n , denote the lung volume in the ith compartment so that the state equations for inspiration are given by n−1

in x˙ i (t) Rn,i

X 1 in Rj,k + xi (t) + j ci j=0

kj 2n−j

X

l=(kj −1)2

n−j

x˙ l (t) = pin (t), +1

NNBook

September 1, 2009

530

CHAPTER 17 '

$

'

x2

x3

%

in R1,1

in R1,2

in R2,4

in R0,1

c4 x4

$

&

in R2,3

%

'

in R2,1

%

'

x1

& in R2,2

c3

% &

&

$

c2 c1

$

papp Figure 17.2 Four-compartment lung model.

xi (0) = xin i0 ,

0 ≤ t ≤ Tin ,

i = 1, 2, . . . , 2n ,

(17.10)

where kj+1 − 1 ⌋ + 1, j = 0, . . . , n − 1, kn = i, (17.11) 2 and ⌊q⌋ denotes the floor function, which gives the largest integer less than or equal to the positive number q. kj = ⌊

To further elucidate the inspiration state equation for a 2n -compartment model, consider the four-compartment model shown in Figure 17.2 corresponding to a two-generation lung model. Let xi , i = 1, 2, 3, 4, denote the compartmental volumes. Now, the pressure c1i xi (t) due to the compliance in the ith compartment will be equal to the difference between the external pressure applied and the resistance to air flow at every airway in the path leading from the pressure source to the ith compartment. In particular, for i = 3 (see Figure 17.2), 1 in x3 (t) = pin (t) − R0,1 [x˙ 1 (t) + x˙ 2 (t) + x˙ 3 (t) + x˙ 4 (t)] c3 in in −R1,2 [x˙ 3 (t) + x˙ 4 (t)] − R2,3 x˙ 3 (t), or, equivalently, in in R2,3 x˙ 3 (t) + R1,2 [x˙ 3 (t) + x˙ 4 (t)] in +R0,1 [x˙ 1 (t) + x˙ 2 (t) + x˙ 3 (t) + x˙ 4 (t)] +

1 x3 (t) = pin (t). c3

Next, we consider the state equation for the expiration process. As

NNBook

September 1, 2009

531

RESPIRATORY COMPARTMENTAL MODELS

in the single-compartment model we assume that the expiration process is passive and the external pressure applied is pex (t). Following an identical procedure as in the inspiration case, we obtain the state equation for expiration as ex x˙ i (t) Rn,i

xi (Tin ) =

+

n−1 X

kj 2n−j

X

ex Rj,k j

j=0 ex xi0 ,

x˙ l (t) +

l=(kj −1)2n−j +1

Tin ≤ t ≤ Tex + Tin ,

1 xi (t) = pex (t), ci

i = 1, 2, . . . , 2n ,

(17.12)

where kj satisfies (17.11).

17.4 State Space Multicompartment Lung Model In this section, we rewrite the state equations (17.10) and (17.12) for inspiration and expiration, respectively, as a switched dynamical system. To describe the dynamics of the multicompartment lung model in terms of a state space model, define the state vector x , [x1 , x2 , . . . , x2n ]T , where xi denotes the lung volume of the ith compartment. Now, the state equation (17.10) for inspiration can be rewritten as Rin x(t) ˙ + Cx(t) = pin (t)e,

x(0) = xin 0 ,

where C , diag[ c11 , . . . , c21n ] and

0 ≤ t ≤ Tin ,

(17.13)

j

Rin ,

n X 2 X

T in Zj,k Zj,k , Rj,k

(17.14)

j=0 k=1

n

where Zj,k ∈ R2 is such that the lth component of Zj,k is 1 for all l = (k − 1)2n−j + 1, (k − 1)2n−j + 2, . . . , k2n−j , k = 1, . . . , 2j , j = 0, 1, . . . , n, and zero elsewhere. Similarly, the state equation (17.12) for expiration can be rewritten as Rex x(t) ˙ + Cx(t) = pex (t)e, where

x(Tin ) = xex 0 ,

Tin ≤ t ≤ Tex + Tin , (17.15)

j

Rex ,

n X 2 X

ex T Rj,k Zj,k Zj,k .

(17.16)

j=0 k=1

Note that if Rin and Rex are invertible, then (17.13) and (17.15) can be equivalently written as x(t) ˙ = Ain x(t) + Bin pin (t), x(0) = xin 0 ≤ t ≤ Tin , (17.17) 0 , ex x(t) ˙ = Aex x(t) + Bex pex (t), x(Tin ) = x0 , Tin ≤ t ≤ Tex + Tin , (17.18)

NNBook

September 1, 2009

532

CHAPTER 17

−1 −1 −1 C, and B , R−1 e. e, Aex = −Rex where Ain = −Rin C, Bin = Rin ex ex △

△

△

The following proposition states and proves several important properties of Rin , Rex , Ain , and Aex that are essential for the main results of this chapter. Proposition 17.3. Consider the dynamical system (17.13) and (17.15) or, equivalently, (17.17) and (17.18). Then the following statements hold: i ) Rin > 0 and Rex > 0. ii ) AT in C + CAin < 0. iii) AT ex C + CAex < 0. iv ) Rin and Rex are strictly ultrametric. v ) Ain and Aex are compartmental and Hurwitz, and Bin ≥≥ 0 and Bex ≥≥ 0. Proof. Statement i ) follows from (17.14) by noting n

Rin ≥

2 X

in T in in Rn,k Zn,k Zn,k = diag[Rn,1 , . . . , Rn,2 n ] > 0,

k=1

since the lth component of Zn,k is 1 if l = k and zero otherwise. Similarly, it can be shown that Rex > 0. Statements ii ) and iii ) follow immediately by noting that −1 AT in C + CAin = −2CRin C < 0

and

−1 AT ex C + CAex = −2CRex C < 0.

To show iv ), define j

Rjin

△

=

εRnin

+

2 X k=1

in T Rj,k Zj,k Zj,k ,

j = 1, . . . , n − 1,

in , . . . , Rin ] and ε = 1 . Note that it follows from where Rnin = diag[Rn,1 n,2n n−1 Proposition 17.1 that for each j ∈ {1, . . . , n}, Rjin is strictly ultrametric, and P hence, Rin = nj=1 Rjin is strictly ultrametric. Similarly, it can be shown that Rex is strictly ultrametric. △

Finally, to show v ) note that since Rin and Rex are strictly ultrametric −1 −1 e ≥≥ 0, and it follows from Lemma 17.1 that Bin = Rin e ≥≥ 0, Bex = Rex

NNBook

September 1, 2009

533

RESPIRATORY COMPARTMENTAL MODELS

−1 −1 are essentially nonnegative. Hence, since C is a positive −Rin and −Rex diagonal matrix, Ain and Aex are essentially nonnegative. Finally, since −1 −1 e ≥≥ 0 it follows that AT e = −CR−1 e ≤≤ 0 and Rin e ≥≥ 0 and Rex in in T −1 Aex e = −CRex e ≤≤ 0, which implies that Ain and Aex are compartmental and, by ii) and iii), Ain and Aex are Hurwitz.

It follows from Proposition 17.3 that Rin and Rex are invertible. Hence, Ain and Aex are well defined, which implies that the state equations for inspiration and expiration given by (17.17) and (17.18), respectively, are well defined. In this chapter, we assume that the inspiration process starts from a given initial state xin 0 , followed by the expiration process whose initial state will be the final state of the inspiration. An inspiration followed by the expiration is called a single breathing cycle. We assume that each breathing cycle is followed by another breathing cycle, where the initial condition for the latter breathing cycle is the final state of the former breathing cycle. Furthermore, we assume that the duration of inspiration is Tin and that of expiration is Tex , so that the total duration of a breathing cycle is Tin + Tex . It is clear that this process generates a periodic dynamical system with a period T , Tin + Tex . Furthermore, the system dynamics switch from inspiration to expiration and back to inspiration. Hence, the dynamics for a breathing cycle can be characterized by the periodic switched dynamical system G given by x(t) ˙ = A(t)x(t) + B(t)u(t),

x(0) = xin 0 ,

t ≥ 0,

(17.19)

where A(t) = A(t + T ),  Ain , A(t) = Aex ,  Bin , B(t) = Bex ,  pin (t), u(t) = pex (t),

u(t) = u(t + T ), 0 ≤ t < Tin , Tin ≤ t < T, 0 ≤ t < Tin , Tin ≤ t < T,

0 ≤ t < Tin , Tin ≤ t < T.

t ≥ 0,

(17.20) (17.21) (17.22) (17.23)

The following result shows that the solution to the switched dynamical n system (17.19) is nonnegative, that is, for every xin 0 ∈ R+ , the solution x(t), t ≥ 0, to (17.19) satisfies x(t) ≥≥ 0, t ≥ 0. Theorem 17.2. Consider the switched dynamical system (17.19) where xin 0 ≥≥ 0. Then x(t) ≥≥ 0, t ≥ 0, where x(t) denotes the solution to (17.19).

NNBook

September 1, 2009

534

CHAPTER 17

Proof. Note that the solution to (17.19) over the time interval [0, T ] is given by ( R t A (t−τ ) in 0 ≤ t ≤ Tin , eAin t xin Bin pin (τ )dτ, 0 + 0 e R x(t) = t A ex (t−Tin ) ex e x0 + Tin eAex (t−τ ) Bex pex (τ )dτ, Tin ≤ t ≤ T, (17.24)

where xex 0 = x(Tin ). Now, since Ain and Aex are essentially nonnegative (by Proposition 17.3), it follows from Proposition 2.5 that eAin t ≥≥ 0 and eAex t ≥≥ 0 for all t ≥ 0. Hence, x(t) ≥≥ 0, 0 ≤ t ≤ T . Now, the nonnegativity of x(t) for all t ≥ 0 follows by mathematical induction.

17.5 Limit Cycle Analysis of the Multicompartment Lung Model In this section, we characterize and analyze the stability of periodic orbits of the switched dynamical system G given by (17.19). First, note that it follows from (17.24) that in xex 0 = x(Tin ) = Γin x0 + θ,

(17.25)

where Γin , eAin Tin , Z θ , eAin Tin

(17.26) Tin

e−Ain t Bin pin (t)dt.

(17.27)

0

Furthermore, note that x(T ) = Γex xex 0 + δ,

(17.28)

where Γex , eAex Tex , Z Aex T δ , e

(17.29) T

e−Aex t Bex pex (t)dt.

(17.30)

Tin

Next, let xin m denote the initial condition for the mth inspiration (and hence the mth breathing cycle) and let xex m denote the initial condition for the in mth expiration, that is, xm = x(mT ) and xex m = x(mT + Tin ), m = 0, 1, . . .. Hence, it follows from (17.25) and (17.28) that in xin 1 = Γei x0 + Γex θ + δ,

(17.31)

△

where Γei = Γex Γin . Similarly, it can be shown that ex xex 1 = Γie x0 + Γin δ + θ,

(17.32)

NNBook

September 1, 2009

535

RESPIRATORY COMPARTMENTAL MODELS △

where Γie = Γin Γex . More generally, in xin m+1 = Γei xm + Γex θ + δ, ex xex m+1 = Γie xm + Γin δ + θ,

m = 0, 1, . . . , m = 0, 1, . . . .

(17.33) (17.34)

The following proposition states and proves two key properties for Γei and Γie which are useful in characterizing a periodic orbit for the switched dynamical system G. Proposition 17.4. The following statements hold: T i ) ΓT ex CΓex < C and Γin CΓin < C. T ii ) ΓT ei CΓei < C and Γie CΓie < C.

Proof. It follows from Proposition 17.3 that Tin (AT in C + CAin ) < 0, Tex (AT ex C + CAex ) < 0. Hence, it follows from Proposition 17.2 that T

eAin Tin CeAin Tin < C, T

eAex Tex CeAex Tex < C, which proves i ). To prove ii), pre- and postmultiply the first inequality of i ) by ΓT in and Γin , respectively, to obtain T T ΓT in Γex CΓex Γin ≤ Γin CΓin < C,

where the last inequality follows from i ). This establishes the first inequality of ii ). The second inequality follows in an identical manner. For the next result, define x ˆin = (I − Γei )−1 (Γex θ + δ) and x ˆex = −1 (I − Γie) (Γin δ + θ). △

△

Proposition 17.5. Consider the switched dynamical system G given n by (17.19). Then, for every xin 0 ∈ R+ , the following statements hold: in = x i ) limm→∞ xm ˆin and limm→∞ xex ˆex . m =x

ii ) For every t ∈ [0, Tin ], Ain t

lim x(t + mT ) = e

m→∞

x ˆin +

Z

0

t

eAin (t−τ ) Bin pin (τ )dτ,

NNBook

September 1, 2009

536

CHAPTER 17

and, for every t ∈ [Tin , T ], lim x(t + mT + Tin ) = eAex t x ˆex +

m→∞

Z

t

eAex (t−τ ) Bex pex (τ + Tin )dτ.

0

Proof. It follows from ii) of Proposition 17.4 that Γei and Γie are m Schur, and hence, limm→∞ Γm ei = 0 and limm→∞ Γie = 0. Furthermore, (I − Γei )−1 and (I − Γie)−1 exist and are given by −1

(I − Γei )

=

∞ X

Γjei ,

−1

(I − Γie )

j=0

=

∞ X

Γjie .

j=0

Next, it follows from (17.33) and (17.34) that m in xin m = Γei x0 +

m−1 X

Γjei (Γex θ + δ),

j=0

m ex xex m = Γie x0 +

m−1 X

Γjie (Γin δ + θ),

j=0

which, by taking limits, yields i). Now, ii) follows from i) and (17.24). It follows from Proposition 17.5 that the individual compartmental volumes, and hence the total volume, converge to the steady-state endinspiratory and end-expiratory values of (I − Γei )−1 (Γex θ + δ) and (I − Γie )−1 (Γin δ + θ), respectively. ˆ = (I − Γei )−1 (Γex θ + δ) and define the orbit Next, let x △

△

n

ˆ), where s(t, x Oxˆ = {x ∈ R+ : x = s(t, x ˆ) is the solution to (17.19)}. (17.35) in = x With xin = x ˆ note that x ˆ , m = 1, 2, . . ., or, equivalently, x(mT )=x ˆ, m 0 m = 1, 2, . . ., which implies that Oxˆ is a periodic orbit of (17.19). The following theorem presents one of the main results of this chapter. Theorem 17.3. Consider the switched dynamical system G given by (17.19). Then the periodic orbit Oxˆ of G generated by x(0) = x ˆ = (I − −1 Γei ) (Γex θ + δ) is globally asymptotically stable. Proof. Note that for the periodic orbit Oxˆ generated by the point x ˆ = (I − Γei )−1 (Γex θ + δ), the Poincar´e map is given by z(k + 1) = s(T, z(k)) = Γei z(k) + Γex θ + δ, z(0) = xin 0 , k ∈ Z+ .

(17.36)

Since Γei is Schur (by Proposition 17.4) it follows that x ˆ is an asymptotically stable fixed point of (17.36). Hence, it follows from Theorem 17.1 that Oxˆ

NNBook

September 1, 2009

537

RESPIRATORY COMPARTMENTAL MODELS

is asymptotically stable. Next, let ε > 0 be such that dist(s(t, x0 ), Oxˆ ) → 0 for all x0 ∈ D and dist(x0 , Oxˆ ) < ε. (The existence of such an ε is guaranteed since Oxˆ is asymptotically stable.) Now, it follows from i) of Proposition 17.5 that in ˆk < ε. there exists m ∈ Z+ such that dist(s(mT, xin ˆ ) ≤ ks(mT, x0 ) − x 0 ), Ox Hence, lim dist(s(t, x0in ), Oxˆ ) = lim dist(s(t − mT, s(mT, xin ˆ ) = 0, 0 )), Ox

t→∞

t→∞

establishing global asymptotic stability of Oxˆ . Note that Theorem 17.3 is valid for arbitrary nonnegative functions (possibly discontinuous) pin (t) and pex (t) as long as Z Tin e−Ain t Bin pin (t)dt 0

and

Z

T

e−Aex t Bex pex (t)dt

Tin

are finite. In the case where pin (t) = αt + β and pex (t) = γ for some positive constants α, β, and γ, θ and δ are given by Ain Tin − I) − αAin Tin ]Bin , θ = A−2 in [(αI + βAin )(e Aex Tex δ = γA−1 − I)Bex . ex (e

The following result provides a generalization to Theorem 17.3. Theorem 17.4. Consider the switched dynamical system G given by (17.19). Let x(t) and y(t), t ≥ 0, denote the solutions to (17.19) with initial n conditions x(0) ∈ R+ and y(0) = x ˆ. Then, x(t) → y(t) as t → ∞. Proof. Let e(t) , x(t) − y(t) so that e(t) ˙ = A(t)e(t),

e(0) = x(0) − x ˆ,

t ≥ 0.

(17.37)

Now, consider the Lyapunov function candidate V : Rn → R given by V (e) = eT Ce so that the Lyapunov derivative of V (e) along the trajectories of (17.37) is given by V˙ (e(t)) = eT (t)[AT (t)C + CA(t)]e(t) −1 −1 ≤ max{−2eT (t)CRin Ce(t), −2eT (t)CRex Ce(t)}

≤ −2ηeT (t)e(t),

t ≥ 0,

−1 −1 C)}, which implies that e(t) → 0 where η , min{λmin (CRin C), λmin (CRex as t → ∞.

NNBook

September 1, 2009

538

CHAPTER 17

Note that Theorem 17.4 shows that the periodic solution given by Oxˆ is globally asymptotically stable (in the sense of stability of motion), and hence, Oxˆ is orbitally stable strengthening the conclusion of Theorem 17.3. Note that the error dynamics e(t), t ≥ 0, given by (17.37) is a switched dynamical system where each of the switched systems is a linear dynamical system, and V (e) = eT Ce is a common Lyapunov function for both linear systems.

17.6 A Regular Dichotomy Model In this section, we present results for a special class of models with a dichotomy architecture. Specifically, we assume that the bronchial tree has a regular dichotomy structure [302], that is, for a given branch generation all air flow resistances at the airway units are equal, and hence, for an nin = R ˆ in and Rex = R ˆ ex , generation model (2n -compartment model), Rj,k j j j,k ˆ in > 0 and R ˆ ex > 0, j = 0, . . . , n. k = 1, 2, . . . , 2j , j = 0, 1, . . . , n, where R j j Furthermore, we assume that ck = cˆ, k = 1, . . . , 2n , that is, the compliance of each compartment is equal. In this case, it can be shown that C = 1cˆ I2n , Rin =

n X


ˆ jin I2j ⊗ e2n−j eTn−j , R 2

(17.38)

n X


ˆ jex I2j ⊗ e2n−j eTn−j , R 2

(17.39)

j=0

and Rex =

j=0

−1 −1 −1 , and B −1 , Bin = Rin so that Ain = − 1cˆ Rin e, Aex = − 1cˆ Rex ex = Rex e. (Here, n en ∈ R denotes the ones vector of order n.) Furthermore, note that Rin e = ˆ in ˆ ex P P ˆ in e and Rex e = 2n R ˆ ex e, where R ˆ in , n Rjj and R ˆ ex , n Rjj , so 2n R

that Bin =

1 ˆ in e, Bex 2n R

1 e, 2n Rˆex

=

j=0 2

j=0 2

and

eAin (Tin −t) Bin =

(T ) 1 − in−t e cˆ2n Rˆ in e. ˆ in 2n R

(17.40)

Hence, T

θ=

− cˆ2nin ˆ R

e

ˆ 2n R

in

in

Z

Tin

t

e cˆ2n Rˆ in papp (t)dt e.

(17.41)

0

Now, using (17.41) it can be shown that x ˆin is of the form γe, where γ > 0, and hence, the limit cycle Oxˆ ⊂ {γe : γ ≥ 0}. Thus, it follows that the

NNBook

September 1, 2009

539

RESPIRATORY COMPARTMENTAL MODELS

limiting behavior of a regular dichotomy lung model exhibits equipartioning of the total volume, that is, xi (t) → xj (t) as t → ∞ for all i, j = 1, 2, . . . , 2n . Next, we provide a relation between m-generation and n-generation ˆ in and R ˆ ex denote the regular dichotomy models (m < n). Let R m,j m,j resistances to air flow at a jth generation airway unit, let cˆm denote the compliance of each compartment, and let xm i denote the ith compartmental volume in an m-generation model. Here, we assume that xm i =

L X

xn(i−1)L+j , i = 1, . . . , M,

(17.42)

j=1

where L , 2n−m and M , 2m ; that is, each compartment of the mgeneration model is equivalent to L compartments of the n-generation model, so that the total volumes in both models are equal. Note that (17.42) may be written as n xm = (IM ⊗ eT (17.43) L )x , m n n n n where xm = [xm 1 , . . . , xM ] and x = [x1 , . . . , xN ], and N , 2 .

Now, consider the n-generation state equation for inspiration given by n n Rin x˙ (t) +

1 n x (t) = pin (t)eN , cˆn

where n Rin

=

n X j=0

xn (0) = xnin,0 ,

0 ≤ t ≤ Tin ,

in ˆ n,j (I2j ⊗ e2n−j eT R 2n−j ).

(17.44)

(17.45)

In this case, it can be shown that (IM ⊗

eT L )(I2j

⊗ e2

n−j

eT 2n−j )

=



2L (I2j ⊗ e2m−j eT 2m−j ), j < m, (17.46) 2n−j (IM ⊗ eT ), j ≥ m. L

Now, premultiplying (17.45) by (IM ⊗ eT L ) and using (17.43) and (17.46) yields m−1 X j=0

in ˆ n,j 2L R (I2j ⊗ e2n−j eT ˙ n (t) 2n−j )x

+

n X

j=m

in m ˆ n,j 2n−j R x˙ (t) +

1 m x (t) = 2L pin (t)eM . cˆn

(17.47)

˙ m (t) so Next, note that (I2j ⊗ e2m−j eT ˙ n (t) = (I2j ⊗ e2m−j eT 2n−j )x 2m−j )x

NNBook

September 1, 2009

540

CHAPTER 17

that (17.47) can be written as m−1 X j=0

in ˆ n,j (I2j ⊗ e2m−j eT ˙ m (t) R 2m−j )x

+

n X

ˆ in x˙ m (t) + 2m−j R n,j

j=m

1 m x (t) = pin (t)eM . 2L cˆn

(17.48)

Comparison of (17.48) with the m-generation model given by m m Rin x˙ (t) +

1 m x (t) = pin (t)eM cˆm

(17.49)

yields cˆm = 2n−m cˆn and m Rin =

m−1 X j=0

ˆ in (I2j ⊗ e2m−j eTm−j ) + R n,j 2

n X

ˆ in IM , 2m−j R n,j

j=m

or, equivalently, ˆ in , ˆ in = R R n,j m,j in ˆ m,m R =

n X

j=m

j = 0, 1, . . . , m − 1,

ˆ in R n,j . 2j−m

(17.50) (17.51)

Similarly, it can be shown that ˆ ex = R ˆ ex , R m,j n,j ex ˆ m,m R =

n X

j=m

j = 0, 1, . . . , m − 1,

ˆ ex R n,j . j−m 2

(17.52) (17.53)

Example 17.1. In this example, we consider a four-compartment lung model and we numerically integrate (17.19) to illustrate convergence of the trajectories to a stable limit cycle. Here, we assume that the bronchial tree has a regular dichotomy. Anatomically the human lung has around 24 generations of airway units. A typical value for lung compliance is 0.1 l/cm H2 O, that is, cˆ0 = 0.1 l/cm H2 O. (Note that respiratory pressure is measured in terms of centimeters of water pressure.) The airway resistance varies with the branch generation and typical values can be found in [143]. Furthermore, the expiratory resistances will be higher than the inspiratory resistance by a factor of 2 to 3. Here, we assume that the factor is 2.5. Now, based on the values for the 24-generation model and using (17.50)–(17.53), we can obtain m-generation models for all m = 0, . . . , 23. Figures 17.3 and 17.4 provide the time responses of compartmental volumes of one-generation and two-generation lung models, respectively, where we assumed that the applied pressure pin (t) = 20t + 5 cm H2 O,

NNBook

September 1, 2009

541

RESPIRATORY COMPARTMENTAL MODELS

pex (t) = 0 cm H2 O, the inspiration time Tin = 1 sec, the expiration time Tex = 2 sec, and the initial total volume xtot (0) = 0.25 l. Figures 17.3 and 17.4 clearly show that the states of the one-generation and two-generation models converge to limit cycles. Furthermore, after an initial transient behavior, the steady-state volume in the lung is uniformly distributed over all the compartments, that is, the steady-state value of the volume in each compartment is equal (in both the one-generation and two-generation models). Finally, Figure 17.5 shows the phase portrait (x1 (t) versus x2 (t)) of the one-generation model, showing the asymptotic convergence of the state to a limit cycle. △ 0.5 0.45

Compartmental volume (liters)

0.4 0.35 0.3 0.25 0.2 x1

0.15

x2

0.1 0.05 0

0

2

4

6 Time(sec)

8

10

12

Figure 17.3 Compartmental volumes versus time: one-generation model.

17.7 A General Tree Structure Model In this section, we extend the model presented in Sections 17.3–17.5 to the case where the bronchial tree has a general tree architecture [146, 147, 178]. The general tree structure includes regular and irregular dichotomy [302]. Specifically, let the bronchial tree be represented by a weighted directed tree G = (V, E, R), where each vertex corresponds to a branching point of an airway unit or the terminal compartment (alveolus) of the lung. In this case, the trachea corresponds to the root v1 of the tree, and all the alveoli correspond to the leaves of the tree. Every edge (vl , vm ) ∈ E corresponds to an airway unit and R(l,m) , the weight of the edge, corresponds to the in and R ex resistance of the airway unit; we use R(l,m) = Rl,m (l,m) = Rl,m for resistance during inspiration and expiration, respectively.

NNBook

September 1, 2009

542

CHAPTER 17

0.25

Compartmental volume (liters)

0.2

0.15

x

0.1

1

x

2

x3 0.05

0

x4

0

2

4

6 Time(sec)

8

10

12

Figure 17.4 Compartmental volumes versus time: two-generation model. 0.5 0.45 0.4 0.35

x2 (liters)

0.3 0.25 0.2 0.15 0.1 0.05 0

0

0.05

0.1

0.15

0.2

0.25 0.3 x1 (liters)

0.35

0.4

0.45

Figure 17.5 x1 (t) versus x2 (t): one-generation model.

0.5

NNBook

September 1, 2009

543

RESPIRATORY COMPARTMENTAL MODELS

Let L , {vi ∈ V : vi is a leaf of G} and let the number of leaves of G (or, equivalently, compartments of the lung) be n so that L = {vi1 , vi2 , . . . , vin }, where ik ∈ {1, 2, . . . , N }, k = 1, 2, . . . , n, and N is the number of vertices of the graph. To develop the dynamical model for the inspiration process, let ck , k = 1, 2, . . . , n, denote the compliance of each compartment, and let xk , k = 1, 2, . . . , n, denote the lung volume in the kth compartment so that the state equations for inspiration are given by X X 1 in xk (t) + x˙ j (t) = pin (t), xi (0) = xin Rl,m k0 , 0 ≤ t ≤ Tin , ck vij ∈Ll,m

(vl ,vm )∈Pk

k = 1, 2, . . . , n,

(17.54)

where Pk , {(vl , vm ) ∈ E : (vl , vm ) belongs to the directed path from the root of G to vik }, (17.55) and, for each l, m ∈ {1, . . . , N } such that (vl , vm ) ∈ E,

Ll,m , {vik ∈ L : there exits a directed path from vm to vik , k = 1, . . . , n}. (17.56) Next, let x , [x1 , . . . , xn ]T so that (17.54) can be written as Rin x(t) ˙ + Cx(t) = pin (t)e, where C ,

diag[ c11 , . . . , c1n ]

x(0) = xin 0 ,

0 ≤ t ≤ Tin ,

and

Rin =

X

T in Zl,m Zl,m , Rl,m

(17.57)

(vl ,vm )∈E

Rn

where Zl,m ∈ is such that the kth component of Zl,m is 1 if vik ∈ Ll,m and 0 otherwise. An identical procedure yields the state equations for expiration given by Rex x(t) ˙ + Cx(t) = pex (t)e, where Rex =

X

x(Tin ) = xex 0 , Tin ≤ t ≤ T, T ex Rl,m Zl,m Zl,m .

(17.58) (17.59)

(vl ,vm )∈E

Note that it can be easily shown that Rin > 0 and Rex > 0, and it follows from (17.57), (17.59), and Proposition 17.1 that Rin and Rex are strictly ultrametric. Hence, for a general tree structure model all of the results of Sections 17.4 and 17.5 are valid with Rin and Rex given by (17.57) and (17.59), respectively. To illustrate the general tree structure lung model, consider the fivecompartment model shown in Figure 17.6. Here, the bronchial tree is

NNBook

September 1, 2009

544

CHAPTER 17

x3 , c3

x4 , c4

v7

v8

x2 , c2 v6 in R3,7

in R3,6

v3 in R3,5

in R4,8

in R2,3

in R2,4

v4 Rin 4,9

v9

v2

x5 , c5

v5 in R1,2

x1 , c1

v1 papp

Figure 17.6 Five-compartment tree structure model.

represented by a weighted directed tree G = (V, E, R) consisting of nine nodes V = {v1 , v2 , . . . , v9 } and eight edges E = {(v1 , v2 ), (v2 , v3 ), (v2 , v4 ), (v3 , v5 ), (v3 , v6 ), (v3 , v7 ), (v4 , v8 ), (v4 , v9 )}. In this case, the set of leaves L = {v5 , v6 , . . . , v9 } corresponds to the five compartments of the lung. Let vik = vk+4 , k = 1, . . . , 5. Now, the pressure 1 ck xk (t) due to the compliance in the kth compartment will be equal to the difference between the external pressure applied and the resistance to air flow at every airway in the path leading from the pressure source (the root v1 ) to the kth compartment. In particular, for k = 3 (see Figure 17.6), 1 in x3 (t) = pin (t) − R1,2 [x˙ 1 (t) + x˙ 2 (t) + x˙ 3 (t) + x˙ 4 (t) + x˙ 5 (t)] c3 in in −R2,3 [x˙ 1 (t) + x˙ 2 (t) + x˙ 3 (t)] − R3,7 x˙ 3 (t), or, equivalently, 1 x3 (t) + c3

X

(vl ,vm )∈P3

in Rl,m

X

x˙ j (t) = pin (t),

vij ∈Ll,m

where P3 L1,2 L2,3 L3,7

= = = =

{(v1 , v2 ), (v2 , v3 ), (v3 , v7 )}, {v5 , v6 , v7 , v8 , v9 }, {v5 , v6 , v7 }, {v7 }.

(17.60)

NNBook

September 1, 2009

545

RESPIRATORY COMPARTMENTAL MODELS

17.8 Direct Adaptive Control for Switched Linear Time-Varying Systems Mechanical ventilation of a patient with respiratory failure is one of the most common life-saving procedures performed in the intensive care unit. However, mechanical ventilation is physically uncomfortable due to the noxious interface between the ventilator and patient, and mechanical ventilation evokes substantial anxiety on the part of the patient. This will often be manifested by the patient “fighting the ventilator.” In this situation, there is dyssynchrony between the ventilatory effort of the patient and the ventilator. The patient will attempt to exhale at the time the ventilator is trying to expand the lungs or the patient will try to inhale when the ventilator is decreasing airway pressure to allow an exhalation. When patient-ventilator dyssynchrony occurs, at the very least there is excessive work of breathing with subsequent ventilatory muscle fatigue and in the worst case, elevated airway pressures that can actually rupture lung tissue. In this situation, it is a very common clinical practice to sedate patients to minimize fighting the ventilator. Sedative-hypnotic agents act on the central nervous system to ameliorate the anxiety and discomfort associated with mechanical ventilation and facilitate patient-ventilator synchrony. In this and the next section, we develop an adaptive feedback controller for addressing this dyssynchrony for intensive care unite sedation. In particular, we develop a model reference direct adaptive controller framework where the plant and reference model involve switching and timevarying dynamics. Then, we apply the proposed adaptive control framework to the multicompartmental model of a pressure-limited respirator and lung mechanics system developed in Section 17.4. Specifically, we develop an adaptive feedback controller that stabilizes a given limit cycle corresponding to a clinically plausible breathing pattern. We begin by considering the problem of adaptive tracking of uncertain linear time-varying switching systems. Specifically, consider the controlled uncertain switched linear time-varying system G given by x˙ p (t) = Ap (t)xp (t) + Bp (t)u(t),

xp (0) = xp0 ,

t ≥ 0,

(17.61)

where xp (t) ∈ Rn , t ≥ 0, is the state vector, u(t) ∈ Rp , t ≥ 0, is the control input, and Ap (t) ∈ Rn×n , t ≥ 0, and Bp (t) ∈ Rn×p , t ≥ 0, are unknown time-varying matrices. The control input u(·) in (17.61) is restricted to the class of admissible controls consisting of measurable functions such that u(t) ∈ Rp , t ≥ 0. Furthermore, for the uncertain linear time-varying system G, we assume that Ap (·) and Bp (·) are piecewise continuous functions and we assume that the required properties for the existence and uniqueness of solutions are satisfied; that is, Ap (·), Bp (·), and u(·) satisfy sufficient

NNBook

September 1, 2009

546

CHAPTER 17

regularity conditions such that (17.61) has a unique solution forward in time. Next, consider the reference model given by x˙ m (t) = Am (t)xm (t) + Bm (t)r(t),

xm (0) = xm0 ,

t ≥ 0,

(17.62)

where xm (t) ∈ Rn , t ≥ 0, is the state vector, r(t) ∈ Rp , t ≥ 0, is the reference input, and Am (t) ∈ Rn×n , t ≥ 0, and Bm (t) ∈ Rn×p , t ≥ 0, are known time-varying matrices. Moreover, let Am (t), t ≥ 0, satisfy AT m (t)Cm + Cm Am (t) ≤ −εm I,

t ≥ 0,

(17.63)

where εm > 0 and Cm ∈ Rn×n is positive definite. Furthermore, we assume that Am (·) and Bm (·) are piecewise continuous and are such that (17.63) has a unique solution for all t ≥ 0 and xm (t) is uniformly bounded for all xm0 ∈ Rn and t ≥ 0. For the next result, we assume that there exist a positive-definite matrix Q∗ ∈ Rp×p and a matrix Θ∗ ∈ Rp×n such that the compatibility conditions Bp (t)Q∗ = Bm (t), Ap (t) + Bp (t)Θ∗ = Am (t),

t ≥ 0, t ≥ 0,

(17.64) (17.65)

are satisfied. Theorem 17.5. Consider the uncertain linear time-varying system G given by (17.61) and the reference model given by (17.62), and assume the compatibility conditions (17.64) and (17.65) hold. Then the adaptive feedback control law u(t) = Θ(t)xp (t) + Q(t)r(t),

(17.66)

where Θ(t) ∈ Rp×n , t ≥ 0, and Q(t) ∈ Rp×p, t ≥ 0, with updated laws T ˙ Θ(t) = −Bm (t)Cm e(t)xT p (t)ΓΘ , Θ(0) = Θ0 , t ≥ 0, T ˙ Q(t) = −Bm (t)Cm e(t)r T (t)ΓQ , Q(0) = Q0 ,

(17.67) (17.68)

where ΓΘ ∈ Rn×n and ΓQ ∈ Rp×p are positive definite and e(t) , xp (t) − xm (t), guarantees that the solution (xp (t), Θ(t), Q(t)) of the closedloop system given by (17.61), (17.62), (17.66), (17.67), and (17.68) is uniformly bounded for all (xp0 , Θ0 , Q0 ) ∈ Rn × Rp×n × Rp×p and t ≥ 0, and xp (t) → xm (t) as t → ∞. Proof. Note that with u(t), t ≥ 0, given by (17.66) it follows from

NNBook

September 1, 2009

547

RESPIRATORY COMPARTMENTAL MODELS

(17.61) that x˙ p (t) = Ap (t)xp (t) + Bp (t)Θ(t)xp (t) + Bp (t)Q(t)r(t), xp (0) = xp0 , t ≥ 0, (17.69) or, equivalently, using (17.64) and (17.65), x˙ p (t) = Ap (t)xp (t) (17.70) ∗ ∗ ∗ ∗ +Bp (t)[Θ + Θ(t) − Θ ]xp (t) + Bp (t)[Q + Q(t) − Q ]r(t) = [Ap (t) + Bp (t)Θ∗ ]xp (t) +Bp (t)[Θ(t) − Θ∗ ]xp (t) + Bp (t)Q∗ r(t) + Bp (t)[Q(t) − Q∗ ]r(t) = Am (t)xp (t) + Bm (t)r(t) +Bp (t)[Θ(t) − Θ∗ ]xp (t) + Bp (t)[Q(t) − Q∗ ]r(t) = Am (t)xp (t) + Bm (t)r(t) + Bp (t)ΦT (t)xp (t) +Bp ΨT (t)(t)r(t), xp (0) = x0 ,

t ≥ 0,

(17.71)

where ΦT (t) , Θ(t) − Θ∗ and ΨT (t) , Q(t) − Q∗ . Now, it follows from (17.62) and (17.71) that e(t) ˙ = Am (t)e(t) + Bp (t)ΦT (t)xp (t) + Bp (t)ΨT (t)r(t), e(0) = xp0 − xm0 , t ≥ 0.

(17.72)

To show uniform boundedness of the closed-loop system (17.67), (17.68), and (17.72), consider the continuously differentiable function ∗−1 T ∗−1 T Φ , V (e, Φ, Ψ) = eT Cm e + tr Γ−1 Ψ + tr Γ−1 Θ ΦQ Q ΨQ

(17.73)

and note that V (0, 0, 0) = 0. Since Cm , ΓQ , ΓΘ , and Q∗ are positive definite, V (e, Ψ, Φ) > 0 for all (e, Φ, Ψ) 6= (0, 0, 0). In addition, V (e, Φ, Ψ) is radially unbounded. Now, using (17.67) and (17.68), it follows that the derivative of V (·, ·, ·) along the closed-loop system trajectories is given by V˙ (e(t), Φ(t), Ψ(t)) = eT (t)[AT m (t)Cm + Cm Am (t)]e(t) T +2e (t)Cm Bp (t)ΦT (t)xp (t)

+2eT (t)Cm Bp (t)ΨT (t)r(t) ∗ −1 ˙ T ∗ −1 ˙ T +2tr Γ−1 Φ (t) + 2tr Γ−1 Ψ (t) Θ Φ(t)Q Q Ψ(t)Q = eT (t)[AT m (t)Cm + Cm Am (t)]e(t) T +2e (t)Cm Bp (t)ΦT (t)xp (t) +2eT (t)Cm Bp (t)ΨT (t)r(t) T −2tr Ψ(t)Q∗ −1 Bm (t)Cm e(t)r T (t) −1 T −2tr Φ(t)Q∗ Bm (t)Cm e(t)xT p (t)

= eT (t)[AT m (t)Cm + Cm Am (t)]e(t) T ≤ −εm e (t)e(t), t ≥ 0.

(17.74)

NNBook

September 1, 2009

548

CHAPTER 17

Hence, it follows from Corollary 2.4 of [117, p. 68] that (e(t), Φ(t), Ψ(t)) is uniformly bounded for all t ≥ 0, and hence, (xp (t), Θ(t), Q(t)) is uniformly bounded for all (xp0 , Θ0 , Q0 ) ∈ Rn × Rp×n × Rp×p and t ≥ 0. Finally, with W1 (e, Φ, Ψ) = W2 (e, Φ, Ψ) = V (e, Φ, Ψ) and W (e, Φ, Ψ) = εm eT e, it follows from Theorem 2.5 of [117] that (e(t), Φ(t), Ψ(t)) → R as t → ∞, where R , {(e, Φ, Ψ) : W (e, Φ, Ψ) = 0} = {(e, Φ, Ψ) : e = 0}. In particular, note that ˙ (e(t), Φ(t), Ψ(t)) = 2εm eT (t)e(t) W ˙ T = 2εm e (t)[Am (t)e(t) + Bp (t)ΦT (t)xp (t) +Bp (t)ΨT (t)r(t)]

(17.75)

is bounded for all t ≥ 0, and hence, all the conditions of Theorem 2.5 of [117, p. 54] are satisfied proving that e(t) → 0 as t → ∞ or, equivalently, xp (t) → xm (t) as t → ∞. Although the form of the adaptive control law given in Theorem 17.5 is identical to that of the standard model reference adaptive controllers provided in the literature (see, for example, [227]), the dynamics of system considered in Theorem 17.5 are not Lipschitz continuous, and hence, standard arguments involving Barbalat’s lemma do not apply. Consequently, Theorem 17.5 requires the more general result given by Theorem 2.5 of [117]. Finally, it is important to note that the adaptive laws (17.67) and (17.68) do not require explicit knowledge of Q∗ or Θ∗ . In addition, no specific structure on the uncertain dynamics Ap (·) and Bp (·) is required as long as the compatibility conditions (17.64) and (17.65) are satisfied.

17.9 Adaptive Control for a Multicompartment Lung Model In this section, we demonstrate the utility of the proposed direct adaptive control framework for the multicompartmental lung model developed in Section 17.4. First, we choose the reference model (17.62) to correspond to a respiratory system producing a plausible breathing pattern. Specifically, −1 (t)C and B (t) = R−1 (t)e, where let Am (t) = −Rm m m m  0 ≤ t < Tin , Rin m , Rm (t) = (17.76) Rex m , Tin ≤ t < T, and where Rm (t) = Rm (t + T ), t > T . Here, Rin m , Rex m , Cm , and r(t) are chosen appropriately to obtain the desirable breathing pattern. It follows from Theorem 17.3 that xm (t), t ≥ 0, converges to a stable limit cycle, and hence, xm (t), t ≥ 0, is uniformly bounded. Next, we assume that the switched linear time-varying system (17.61)

NNBook

September 1, 2009

RESPIRATORY COMPARTMENTAL MODELS

is such that Ap (t) = −Rp−1 (t)Cp and Bp (t) = Rp−1 (t)e, where  0 ≤ t < Tin , Rin p , Rp (t) = Tin ≤ t < T, Rex p ,

549

(17.77)

and where Rp (t) = Rp (t + T ), t > T, so that (17.61) has the form of a lung mechanics model. Here, we assume that Rin p , Rex p , and Cp are unknown and we use Theorem 17.5 to design an adaptive controller u(t), t ≥ 0, such that xp (t) → xm (t) as t → ∞. In order to apply Theorem 17.5, we need to show that the compatibility conditions (17.64) and (17.65) hold. The following proposition provides sufficient conditions under which (17.64) and (17.65) hold for the compartmental lung model. Note that in this case p = 1. −1 Proposition 17.6. Let W = Rin p Rin m . Assume that the following conditions hold: △

−1 −1 i ) Rin p Rin m = Rex p Rex m .

ii ) There exists a positive scalar Q∗ such that W e = Q∗ e. iii) There exists Θ∗ ∈ R1×n such that Cp = W Cm + eΘ∗ . Then (17.64) and (17.65) hold. Proof. The proof follows by noting that i ) and ii ) imply (17.64) holds, whereas i ) and iii ) imply (17.65) holds. In the absence of switching, conditions ii ) and iii ) are standard for model reference adaptive control [227]. Condition i ) is an additional condition that ensures Theorem 17.5 holds for the switching periodic lung mechanics model. Example 17.2. In this example, we illustrative the adaptive controller framework on the four-compartment lung mechanics model of Example 17.1. The reference model is assumed to correspond to a bronchial tree which has a regular dichotomy architecture (see Section 17.6). Furthermore, we choose a reference model so that all the conditions of Proposition 17.6 hold, and hence, the compatibility conditions of Theorem 17.5 are satisfied. In addition, we let Θ0 = [75, 75, 75, 75] and Q0 = 5. Note that no explicit knowledge of the plant model is needed to generate the adaptive control input u(t), t ≥ 0, given by (17.66) and the update laws given by (17.67) and (17.68). Here, we assume that the applied pressure for the reference model is r(t) = sin(20t) + 5 cm H2 O and the inspiration time is Tin = 1 sec and

NNBook

September 1, 2009

550

CHAPTER 17

the expiration time is Tex = 2 sec. Figure 17.7 shows the error xp (t) − xm (t) versus time, verifying that xp (t) → xm (t) as t → ∞. Figures 17.8 and 17.9 show the controlled partial phase portrait. △ 15

e = xp−xm (liters)

10

5

0

−5

−10

0

5

10 Time (sec)

15

20

Figure 17.7 Error versus time

25

x2 (liters)

20

15

10

5

xm xp

0 −5

0

5

10 x (liters)

15

20

1

Figure 17.8 Controlled phase portrait: x1 versus x2

25

NNBook

September 1, 2009

551

RESPIRATORY COMPARTMENTAL MODELS

25

x3 (liters)

20

15

10

5

xm xp

0

0

5

10

15

20

x2 (liters)

Figure 17.9 Controlled phase portrait: x2 versus x3

25

NNBook

September 1, 2009

NNBook

September 1, 2009

Chapter Eighteen

Identification of Stable Nonnegative and Compartmental Systems

18.1 Introduction As discussed throughout this monograph, nonnegative and compartmental systems are essential in capturing the phenomenological behavior of biological and physiological systems, and their use in the specific field of pharmacokinetics is particularly noteworthy. The goal of pharmacokinetic analysis often is to characterize the kinetics of drug disposition in terms of the parameters of a compartmental model. This is accomplished by postulating a model, collecting experimental data (typically drug concentrations in blood as a function of time), and then using statistical analysis to estimate the parameter values that best describe the data. There are numerous sources of noise in the data, from assay error to human recording error. Because of model approximation and noise, there is always an offset between the concentration predicted by the model and the observed data, namely, the prediction error. One method for estimating pharmacokinetic parameters is maximum likelihood [63]. This approach assumes a statistical distribution for the prediction error and then determines the parameter values that maximize the likelihood of the observed results. As noted in Chapter 12, there are two distinct approaches to estimating mean pharmacokinetic parameters for a population of patients [269, 270]. In the first approach, models are fitted to data from individual patients, and the pharmacokinetic parameters are then averaged (two-stage analysis) to provide a measure of the pharmacokinetic parameters for the population. The second approach is to pool the data from individual patients, called mixed-effects modeling; in this situation the prediction error is determined by the stochastic noise of the experiment and by the fact that different patients have different pharmacokinetic parameters. The statistical model used to account for the discrepancy between observed and predicted concentrations must take into consideration not only variability between observed and predicted concentrations within the same patient

NNBook

September 1, 2009

554

CHAPTER 18

(intrapatient variability), but also variability between patients (interpatient variability). Most commonly, it is assumed that the interpatient variability of pharmacokinetic parameters conforms to a log-normal distribution. This method of analysis estimates the mean structural pharmacokinetic parameters as well as the statistical variability of these elements in the population. While system identifiability for nonnegative and compartmental systems has been widely explored in the literature [11, 58, 64, 65, 76, 89, 290], the system identification problem for these class of systems has received less attention. System identification refers to the overall problem of determining system structure as well as system parameter values from input-output data, whereas system identifiability refers to the narrower problem of existence and uniqueness of solutions. Namely, system identifiability concerns whether or not there is enough information in the observations to uniquely determine the system parameters [155]. In this chapter, we develop a system identification framework for stable nonnegative and compartmental dynamical systems within the context of subspace identification. Subspace identification methods [94, 186, 237–239, 292–295] differ from classical least squares identification methods [163, 241, 279] in that estimates of a state sequence are used to provide estimates of the system parameters. Our multivariable framework is based on a constrained weighted least squares optimization problem involving a stability constraint on the plant system matrix as well as a nonnegativity constraint on the system matrices. The resulting constrained optimization problem is cast as a convex linear programming problem over symmetric cones involving a weighted cost function with mixed equality, inequality, quadratic, and nonnegative-definite constraints. Our approach builds on the subspace identification technique presented in [186] guaranteeing system stability to address stable nonnegative and compartmental dynamical systems. Finally, to solve the resulting convex optimization problem, we apply the SeDuMi R MATLAB code [284] to the constrained least squares problem.

18.2 State Reconstruction In this section, we demonstrate how a sequence of state vectors may be constructed from input-output vectors. Consider a discrete-time linear multivariable state-space model given by x(k + 1) = Ax(k) + Bu(k), y(k) = Cx(k) + Du(k),

x(0) = x0 ,

k ∈ Z+ ,

(18.1) (18.2)

where u(k) ∈ Rm , y(k) ∈ Rp , and x(k) ∈ Rn denote the input, output, and state vector at time k, respectively, A ∈ Rn×n , B ∈ Rn×m , C ∈ Rp×n , and

NNBook

September 1, 2009

555

IDENTIFICATION OF NONNEGATIVE SYSTEMS

D ∈ Rp×m are unknown system matrices. Using (18.1) and (18.2), we can construct the block-matrix equation Yq (k) = Γq x(k : k + ℓ − 1) + Hq Uq (k),

(18.3)

where 

  Yq (k) ,  



y(k : k + ℓ − 1) y(k + 1 : k + ℓ) .. . y(k + q − 1 : k + ℓ + q − 2)

  Uq (k) ,   

  Γq ,  

u(k : k + ℓ − 1) u(k + 1 : k + ℓ) .. .



  ,  

  , 

u(k + q − 1 : k + ℓ + q − 2)   D 0 0 C  CB D 0  CA    CAB CB D  , Hq ,  ..   .. .. .. .  . . . q−1 CA CAq−2 B CAq−3 B CAq−4 B

... ... ... .. .

0 0 0 .. .

... D



   ,  

(18.4)

the notation u(k : k + ℓ − 1) denotes the matrix [u(k), . . . , u(k + ℓ − 1)], and likewise for y(k : k + ℓ − 1), x(k : k + ℓ − 1) and so on. Finally, we define the matrix Eq (k) ∈ Rq(p+m)×l as   Yq (k) Eq (k) , . (18.5) Uq (k) For V ∈ Rn×m let R(V ) denote the range (column space) of V . Then R(V is the row space of V . Let V L , (V T V )+ V T and V R , V T (V V T )+ . We also define the projection PV , V R V = V T (V V T )+ V and PV⊥ = I − PV . Note that V PV = V and V PV⊥ = 0. T)

Theorem 18.1 ([185, 221]). Assume that the following conditions are satisfied: i ) rank Γq = n. ii ) rank (x(k : k + ℓ − 1)PU⊥q (k) ) = rank (x(k : k + ℓ − 1)) for all k ∈ Z+ . iii) rank (x(k : k + ℓ − 1)) = n for all k ∈ Z+ . iv ) rank (Uq (k)) = mq for all k ∈ Z+ .

NNBook

September 1, 2009

556

CHAPTER 18

Then q ≥ n/p, ℓ ≥ mq + n, rank (Eq (k)) = mq + n, k ∈ Z+ ,

R(x(k + q : k + ℓ + q − 1)T ) = R(Eq (k)T ) ∩ R(Eq (k + q)T ),

(18.6) (18.7) k ∈ Z+ . (18.8)

In order to obtain the state vector sequence, some matrix computational algorithms are used such as the singular value decomposition (SVD) [185, 221] and the quotient singular value decomposition (QSVD) [222], combined with the cosine-sine decomposition (CSD) [102]. Here, we propose a generalized algorithm based on the URV decomposition [102]. Theorem 18.2. Assume i )–iv ) of Theorem 18.1 hold. Let M , L11 , L22 , −1 l×l , L11 − L12 L−1 22 L21 , and L22 − L21 L11 L12 be nonsingular, where M ∈ R q(p+m)×q(p+m) q(p+m)×q(p+m) q(p+m)×q(p+m) L11 ∈ R , L12 ∈ R , L21 ∈ R , and q(p+m)×q(p+m) L22 ∈ R . Consider the URV decomposition      Eq (k) U11 U12 R11 0 L M= V T, (18.9) Eq (k + q) U21 U22 0 0 where L=



L11 L12 L21 L22



∈ R2q(p+m)×2q(p+m) ,

R11 ∈ R(2mq+n)×(2mq+n) is nonsingular, U11 ∈ Rq(p+m)×(2mq+n) , U12 ∈ Rq(p+m)×(2pq−n) , U21 ∈ Rq(p+m)×(2mq+n) , U22 ∈ Rq(p+m)×(2pq−n) , and V ∈ Rl×l . Furthermore, let Ur ∈ R(2pq−n)×n , S11 ∈ Rn×n , and Vr ∈ R(2mq+n)×n be defined through the URV decomposition h −1 T T (U12 L11 + U22 L21 ) (L11 − L12 L−1 22 L21 ) U11 i −1 −1 −L−1 L (L − L L L ) U 22 21 11 12 21 R11 11 21   T    S11 0 Vr = Ur Us 0 0 VsT = Ur S11 VrT .

(18.10)

Then there exists a nonsingular T ∈ Rn×n such that

T T T x(k + q : k + q + ℓ − 1) = UrT (U12 L11 + U22 L21 )Eq (k).

(18.11)

Proof. The proof is similar to the proof of Proposition 2 of [185]. Note that if R11 and S11 are positive and diagonal, then Theorem 18.2 becomes Proposition 2 of [185]. Furthermore, if L = I and M = I, then

NNBook

September 1, 2009

557

IDENTIFICATION OF NONNEGATIVE SYSTEMS

Theorem 18.2 becomes Theorem 4 of [221]. In particular, we set T = I to obtain the estimate of the state matrix. Another interesting observation from Theorem 18.2 is that computing the complete form of R11 and S11 is not necessary. Since we do not need to know R11 and S11 , it is more desirable not to carry out these operations explicitly. Next, we present a Lanczos algorithm to compute U12 , U22 , and Ur for the case that the SVD is used in Theorem 18.2, without computing R11 and S11 .

18.3 Constrained Optimization for Subspace Identification of Stable Nonnegative Systems The identification problem for a discrete-time, linear nonnegative dynamical system involves estimating the entries of the system matrices of the dynamical system (18.1) and (18.2), where A ∈ Rn×n , B ∈ Rn×m , C ∈ Rp×n , and D ∈ Rp×m are nonnegative matrices, given the measurements of u(k : k + ℓ − 1), y(k : k + ℓ − 1), and an estimate of the state sequence x(k : k + ℓ) obtained from the subspace identification algorithms given in Section 18.2. Our aim is to obtain an estimate of the nonnegative system matrices A, B, C, and D while guaranteeing the asymptotic stability of A. The constrained matrix least squares problem involves the minimization of the cost function

   

Lx 0 x(k + 1 : k + ℓ) 2 J (A, B, C, D) ,

0 Ly y(k : k + ℓ − 1)    2

A B x(k : k + ℓ − 1) − W

C D u(k : k + ℓ − 1) F = J12 (A, B) + J22 (C, D),

(18.12)

where u(k : k +ℓ−1) ∈ Rm×ℓ , x(k : k +ℓ−1) ∈ Rn×ℓ , x(k +1 : k +ℓ) ∈ Rn×ℓ , y(k : k + ℓ − 1) ∈ Rp×ℓ , Lx ∈ Rs×n , Ly ∈ Rt×p , and W ∈ Rℓ×r are weighting matrices, and

  

  x(k : k + ℓ − 1)

J1 (A, B) , Lx x(k + 1 : k + ℓ) − A B W

, u(k : k + ℓ − 1) F (18.13)

  

  x(k : k + ℓ − 1) J2 (C, D) , W

Ly y(k : k + ℓ − 1) − C D

. u(k : k + ℓ − 1) F (18.14)

The following theorem gives necessary and sufficient conditions for the existence of a solution for the minimizers to (18.13) and (18.14).

NNBook

September 1, 2009

558

CHAPTER 18

Theorem 18.3. Consider the constrained optimization problem min kAXC − BkF , X∈C

(18.15)

where A ∈ Rm×n , X ∈ Rn×q , B ∈ Rm×p , C ∈ Rq×p , and C , {X ∈ Rn×q : X ≥≥ 0}. Then X is a solution to (18.15) if and only if X ≥≥ 0, A (AXC − B)C T ≥≥ 0, tr [AT (AXC − B)C T X T ] = 0. T

(18.16) (18.17) (18.18)

Proof. Note that f (X) , = = =

kAXC − Bk2F tr (C T X T AT AXC) − tr (B T AXC) − tr (C T X T AT B) + tr (B T B) tr (C T X T AT AXC) − 2tr (B T AXC) + tr (B T B) tr (C T X T AT AXC) − 2tr (CB T AX) + tr (B T B). (18.19)

Next, note that 1 ∂f (X) 1 ′ f (X) , = AT (AXC − B)C T . 2 2 ∂X

(18.20)

Now, it follows from Theorem 2.4 of [6] that X is a solution to (18.15) if and only if X ∈ C, f ′ (X) ∈ C ∗ , and tr (f ′ (X)X T ) = 0, where C ∗ , {Y ∈ Rn×q : tr (XY T ) ≥ 0, X ∈ C}.

(18.21)

Hence, to establish that X is a solution to (18.15) if and only if (18.16)– (18.18) hold, it need only be shown that C ∗ = C. C ∗ ⊆ C follows from [129, p. 353-354]. To show C ⊆ C ∗ , let X ∈ C. Since tr (XX T ) = tr (X T X) = kXk2F ≥ 0, X ∈ C ∗ , it follows that C ⊆ C ∗ . Hence, C ∗ = C. As noted in [6], in general explicit expressions do not exist for the solution to (18.15) in the case where A 6= In and C = Iq . However, for certain special cases of A and C, explicit closed-form expressions for the solution to (18.15) can be obtained. Next, we consider the minimization problem given by (18.15) for the case where A is a lower triangular matrix and C is a diagonal matrix. Theorem 18.4. Consider the constrained optimization problem min kAXC − BkF , X∈C

(18.22)

where A ∈ Rn×n is a nonsingular lower triangular matrix, C ∈ Rq×q is a nonsingular diagonal matrix, X ∈ Rn×q , B ∈ Rn×q , and C is as in

NNBook

September 1, 2009

559

IDENTIFICATION OF NONNEGATIVE SYSTEMS

Theorem 18.3. Then the solution to (18.22) is given by the recursive formula   −1 X(1,j) = max 0, A−1 B C (18.23) (1,1) (1,j) (j,j) , ! k−1 X −1 −1 −1 X(k,j) = max 0, A(k,k) B(k,j) C(j,j) − A(k,k) A(l,k) X(l,j) , l=1

j = 1, . . . , q,

k = 2, . . . , n.

(18.24)

Alternatively, if C is a singular diagonal matrix, then the solution to the modified constrained optimization problem min kXkF ,

(18.25)

X = {X ∈ C : min kAXC − BkF },

(18.26)

X∈X

where

is given by the recursive formula   + X(1,j) = max 0, A−1 B C (1,j) (j,j) , (1,1) X(k,j) = max

+ 0, A−1 (k,k) B(k,j) C(j,j)

j = 1, . . . , q,

− A−1 (k,k)

k−1 X

!

A(l,k) X(l,j) ,

l=1

k = 2, . . . , n,

(18.27)

+ + −1 where C(j,j) = C(j,j) if C(j,j) 6= 0, and C(j,j) = 0, j = 1, . . . , q, otherwise.

Proof. Let C = diag[C(1,1) , . . . , C(q,q) ],     A(1,1) X(1,1) . . . X(1,q)     .. .. .. .. .. A= , X =  , . . . . . A(n,1) . . . A(n,n) X(n,1) . . . X(n,q)   B(1,1) . . . B(1,q)   .. .. .. B= . . . . B(n,1) . . . B(n,q)

Then



A(1,1) X(1,1) C(1,1) − B(1,1)  ..  AXC − B =  . Pn k=1 A(k,n) X(k,1) C(1,1) − B(n,1)

... .. . ...

 A(1,1) X(1,q) C(q,q) − B(1,q)  .. .  . Pn A X C − B (k,n) (k,q) (q,q) (n,q) k=1

Since A(i,i) 6= 0 and C(j,j) 6= 0, i = 1, . . . , n, j = 1, . . . , q, it follows that " i #2 q n X X X 2 kAXC − BkF = A(k,i) X(k,j) C(j,j) − B(i,j) i=1 j=1

k=1

NNBook

September 1, 2009

560

CHAPTER 18

=

q X  j=1

+

A(1,1) X(1,j) C(j,j) − B(1,j)

q X  j=1

.. . +

A(1,2) X(1,j) C(j,j) + A(2,2) X(2,j) C(j,j) − B(2,j)

q h X j=1

=

j=1

+

q X

h 2 A2(2,2) C(j,j) A−1 (2,2) A(1,2) X(1,j) + X(2,j)

−1 −A−1 (2,2) B(2,j) C(j,j)

+

i2

i2 h −1 2 A2(1,1) C(j,j) X(1,j) − A−1 B C (1,j) (1,1) (j,j)

j=1

.. .

2

A(1,n) X(1,j) C(j,j) + · · ·

+A(n,n) X(n,j) C(j,j) − B(n,j)

q X

2

q X j=1

i2

h 2 A2(n,n) C(j,j) A−1 (n,n) A(1,n) X(1,j) + · · · + X(n,j)

i2 −1 B C −A−1 (n,j) (j,j) . (n,n)

(18.28)

Hence, the solution to (18.22) is given by   −1 X(1,j) = max 0, A−1 B C (1,1) (1,j) (j,j) ,   −1 −1 X(2,j) = max 0, A−1 B C − A A X (2,2) (1,2) (1,j) (2,2) (2,j) (j,j)  −1 = max 0, A−1 (2,2) B(2,j) C(j,j)   −1 −1 −A−1 A max 0, A B C , (1,2) (1,j) (2,2) (j,j) (1,1) .. .

X(n,j) = max

−1 0, A−1 (n,n) B(n,j) C(j,j)

−

A−1 (n,n)

n−1 X l=1

j = 1, . . . , q.

!

A(l,n) X(l,j) , (18.29)

Alternatively, if C is a singular diagonal matrix, then assume, without loss of generality, that C = diag[C(1,1) , . . . , C(r,r) , 0, . . . , 0], where C(j,j) 6= 0,

NNBook

September 1, 2009

561

IDENTIFICATION OF NONNEGATIVE SYSTEMS

j = 1, . . . , r. In this case, kAXC −

Bk2F

=

" i n X r X X i=1 j=1

=

n X r X i=1 j=1

+A−1 (i,i)

k=1

A(k,i) X(k,j)C(j,j) − B(i,j) h

i−1 X

A(l,i) X(l,j)

i2

+

q n X X

q n X X

2 B(i,j)

i=1 j=r+1

2 B(i,j) .

−1 0, A−1 (k,k) B(k,j) C(j,j)

i = 1, . . . , n,

(18.30)

i=1 j=r+1

Hence, the solution to (18.25) and (18.26) is given by   −1 X(1,j) = max 0, A−1 B C (1,j) (1,1) (j,j) ,

X(i,l) = 0,

+

−1 2 X(i,j) − A−1 A2(i,i) C(j,j) (i,i) B(i,j) C(j,j)

l=1

X(k,j) = max

#2

− A−1 (k,k)

k−1 X l=1

j = 1, . . . , r, l = r + 1, . . . , q,

!

A(l,k) X(l,j) , k = 2, . . . , n, (18.31)

which is equivalent to (18.27). If A ∈ Rn×n in Theorem 18.4 is diagonal, then Theorem 18.4 can be greatly simplified. Corollary 18.1. Consider the constrained optimization problem min kAXC − BkF , X∈C

(18.32)

where A ∈ Rn×n and C ∈ Rq×q are nonsingular diagonal matrices, X ∈ Rn×q , B ∈ Rn×q , and C is as in Theorem 18.3. Then the solution to (18.32) is given by X = max(0, A−1 BC −1 ),

(18.33)

where max(0, M ) denotes elementwise maximization. Alternatively, if A or C is a singular diagonal matrix, then the solution to the modified constrained optimization problem min kXkF ,

(18.34)

X = {X ∈ C : min kAXC − BkF },

(18.35)

X = max(0, A+ BC + ).

(18.36)

X∈X

where

is given by

NNBook

September 1, 2009

562

CHAPTER 18

Proof. It follows from Theorem 18.4 that the solution to (18.32) is given by −1 X(i,j) = max(0, A−1 (i,i) B(i,j) C(j,j) ),

i = 1, . . . , n,

j = 1, . . . , q, (18.37)

which is is equivalent to (18.33). Alternatively, if A or C is a singular diagonal matrix, assume, without loss of generality, that A = diag[A(1,1) , . . . , A(r,r) , 0, . . . , 0] and/or C = diag[C(1,1) , . . . , C(s,s) , 0, . . . , 0], where A(i,i) 6= 0 and C(j,j) 6= 0, i = 1, . . . , r, j = 1, . . . , s. In this case, kAXC −

Bk2F

=

r X s X i=1 j=1 n X

+

2 −1 2 A2(i,i) C(j,j) [X(i,j) − A−1 (i,i) B(i,j) C(j,j) ] q X

2 B(i,j) .

(18.38)

i=r+1 j=s+1

Hence, the solution to (18.34) and (18.35) is −1 X(i,j) = max(0, A−1 (i,i) B(i,j) C(j,j) ),

X(k,l) = 0,

k = r + 1, . . . , n,

i = 1, . . . , r,

j = 1, . . . , s, (18.39)

l = s + 1, . . . , q,

(18.40)

which is equivalent to (18.36). To simplify our minimization problem (18.12), we assume that Lx ∈ and Ly ∈ Rp×p in (18.12) are nonsingular diagonal matrices. Now, we focus on minimizing (18.12) subject to an asymptotic stability constraint on the nonnegative system matrix A. It follows from Theorem 2.24 that the asymptotic stability of A is equivalent to the linear matrix inequalities Rn×n

P − AT P A > 0,

P > 0,

P(i,j) = 0,

i, j = 1, . . . , n,

i 6= j. (18.41)

i, j = 1, . . . , n,

i 6= j, (18.42)

To enforce (18.41), we use the inequalities P − AT P A ≥ δI,

P ≥ δI,

P(i,j) = 0,

where δ > 0. Now, using Schur complements, the constraints in (18.42) are equivalent to   P − δI AT P ≥ 0, P(i,j) = 0, i, j = 1, . . . , n, i 6= j. (18.43) PA P The optimization problem can now be rewritten as min J(A, B, C, D) subject to (18.43) and A ≥≥ 0, B ≥≥ 0, C ≥≥ 0, and D ≥≥ 0.

(18.44)

NNBook

September 1, 2009

IDENTIFICATION OF NONNEGATIVE SYSTEMS

To solve this optimization problem, let  R   x(k : k + ℓ − 1) P W1 0 W = , u(k : k + ℓ − 1) 0 W2

563

(18.45)

where W1 ∈ Rn×n and W2 ∈ Rm×m are nonsingular diagonal matrices. The weighting matrix W allows us to convert our optimization problem to a convex optimization problem. Specifically, (18.13) and (18.14) become 2 2 J12 (A, B) = J11 (A) + J12 (B), 2 2 2 J2 (C, D) = J21 (C) + J22 (D),

(18.46) (18.47)

where 2 J11 (A) , kLx (X1 − A)P W1 k2F , 2 J12 (B) , kLx (X2 − B)W2 k2F , 2 J21 (C) , kLy (Y1 − C)P W1 k2F ,

2 J22 (D) , kLy (Y2 − D)W2 k2F , R  x(k : k + ℓ − 1) , [X1 , X2 ] , x(k + 1 : k + ℓ) u(k : k + ℓ − 1)  R x(k : k + ℓ − 1) [Y1 , Y2 ] , y(k : k + ℓ − 1) , u(k : k + ℓ − 1)

(18.48) (18.49) (18.50) (18.51) (18.52) (18.53)

X1 ∈ Rn×n , X2 ∈ Rn×m , Y1 ∈ Rp×n , and Y2 ∈ Rp×m. It follows from Corollary 18.1 that the solutions to (18.49) and (18.51) are ˆ = max(0, X2 ), B ˆ = max(0, Y2 ). D

(18.54) (18.55)

Now, redefining the cost function as 2 2 2 JAC (A, C) , J11 (A) + J21 (C) = kLx (X1 − A)P W1 k2F + kLy (Y1 − C)P W1 k2F



2     

Lx 0

X1 A

− P W1 =

, (18.56) Y1 C 0 Ly F

the optimization problem collapses to minimizing (18.56) subject to (18.43) and   A ≥≥ 0. (18.57) C

Let U1 = AP and U2 = CP . Note that P > 0 is diagonal, and U1 ≥≥ 0 and U2 ≥≥ 0 enforce A ≥≥ 0 and C ≥≥ 0. In this case, (18.56), (18.57),

NNBook

September 1, 2009

564

CHAPTER 18

and (18.43) become 2 JAC (A, C)



P − δI U1



2     

Lx 0

X1 U1

= P− W1

, (18.58) 0 Ly Y1 U2 F

U1 ≥≥ 0, U2 ≥≥ 0,  T U1 ≥ 0, P P(i,j) = 0,

(18.59) (18.60) (18.61)

i, j = 1, . . . , n, i 6= j.

(18.62)

This optimization problem is a quadratic programming problem with linear equality and inequality constraints, and a nonnegative definite constraint. This problem can be stated as the convex linear programming problem min cT xz subject to     0n Lx X1 0n Z4 0p×n Ly Y1 W1     In 0n In Z4 0n     0n 0n In Z4 − δIn In     0n×1 T 01×n ei Z4 ej Z2 Z1 Z4

(18.63)

= Z3 + = =





In In

= 0,



 Lx 0n×p Z1 W1 , 0p×n Ly  0n×p Z1 ,    In 0n Z4 , 0n

i, j = 1, . . . , n,

i 6= j,

≥ kZ3 kF , ≥≥ 0, ≥ 02n ,

(18.64) (18.65) (18.66) (18.67) (18.68) (18.69) (18.70)

where zi , vec Zi , vec (·) denotes the column stacking operator, cx , [01×(n2 +np) , 1, 01×(5n2 +np) ]T ∈ R6n z ,

[z1T , z2 , z3T , z4T ]T

∈R

6n2 +2np+1

,

2

+2np+1

,

(18.71) (18.72)

ei ∈ Rn is a vector whose ith component is 1 and the remaining components are zero, i = 1, . . . , n, Z1 = [U1T , U2T ]T ∈ R(n+p)×n , Z2 = z2 ∈ R represents the value of the cost function JAC (A, C), Z3 = block-diag[Lx , Ly ]([X1T , Y1T ]T P − [U1T , U2T ]T )W1 ∈ R(n+p)×n ,

and Z4 ∈ R2n×2n represents the matrix in (18.61).

To recast the minimization problem in a form suitable for use with a

NNBook

September 1, 2009

565

IDENTIFICATION OF NONNEGATIVE SYSTEMS

convex optimization code, we rearrange the equality constraints as functions of zi to obtain the optimization problem min cT xz

(18.73)

subject to Ax z z1 z2 Z4 7

2

1

= ≥≥ ≥ ≥

bx , 0, kz3 k2 , 02n ,

(18.74) (18.75) (18.76) (18.77)

2

where Ax ∈ R( 2 n +np− 2 n)×(6n +2np+1) is given by  W1 ⊗ E  0(n2 +np)×1  In ⊗ In 0n×p 0n2 ×1 Ax ,   0n2 ×(n2 +np) 0n2 ×1 0 1 n(n−1)×(n2 +np) 0 1 n(n−1)×1 2 2    In2 +np − 0n W1 ⊗ F  0n2 ×(n2 +np) K , (18.78)  0n2 ×(n2 +np) G 0 1 n(n−1)×(n2 +np) H  2 Lx 0n×p E , ∈ R(n+p)×(n+p) , (18.79) 0p×n Ly   0n Lx X1 F , ∈ R(n+p)×2n , (18.80) 0p×n Ly Y1         2 2 G , 0n In ⊗ 0n In − In 0n ⊗ In 0n ∈ Rn ×4n , (18.81) 1

2

T H , [H1T . . . Hn−1 ]T ∈ R 2 n(n−1)×4n , (18.82)          T 2 0n×1 0n×1 0n×1 0n×1 Hi , ⊗ ... ⊗ ∈ R(n−i)×4n , ei ei+1 ei en i = 1, . . . , n − 1, (18.83)     K , − In 0n ⊗ 0n In , (18.84) 7

and bx ∈ R 2 n

2

+np− 21 n

is given by 

 0(2n2 +np)×1 , vec In bx , δ  0 1 n(n−1)×1

(18.85)

2

where ⊗ denotes the Kronecker product and where we have used the identities vec(AXB) = (B T ⊗ A)vecX and (A⊗ B)T = AT ⊗ B T [23, pp. 248 and 249]. This optimization problem involves minimizing a linear function

NNBook

September 1, 2009

566

CHAPTER 18

over symmetric cones and can be solved by using the Self-Dual-Minimization R (SeDuMi) MATLAB package [284].

18.4 Constrained Optimization for Subspace Identification of Compartmental Systems In this section, we extend the constrained optimization method developed in Section 18.3 to linear compartmental dynamical systems. Specifically, consider the discrete-time, linear compartmental dynamical system (18.1) and (18.2), where A, B, C, D are nonnegative with A satisfying P n 1 − nl=1 ali , i = j, A(i,j) = (18.86) aij , i 6= j,

for i, j = 1, . . . , n. The additional constraint on A can be captured by eT (In −A) ≥≥ 0. Note that it follows from Theorem 2.21 that the constraint eT (In − A) ≥≥ 0 implies that A is Lyapunov stable. Letting Z1′ , eT (In − A)P , we obtain Z1′ = eT P − eT AP    T 0n In Z4 = e    = 01×n eT Z4    T = 01×n e Z4

Hence,

z1′ =



0n In

where z1′ = vec(Z1′ ).



⊗





0n In 0n In 0n In

01×n eT







− eT U1 − eT −







Z1

eT 01×p



In 0n×p

eT 01×p

z4 − In ⊗





Z1 .

(18.87)

z1 ,

(18.88)

Now, the optimization problem can be cast as min c¯T ¯ xz

(18.89)

subject to A¯x z¯ z¯1 z2 Z4

= ≥≥ ≥ ≥

¯bx , 0, kz3 k2 , 02n ,

(18.90) (18.91) (18.92) (18.93)

where c¯x , [01×(n2 +np+n) , 1, 01×(5n2 +np) ]T ∈ R6n z¯ ,

[¯ z1T , z2 , z3T , z4T ]T

∈R

6n2 +2np+n+1

,

2

+2np+n+1

,

NNBook

September 1, 2009

567

IDENTIFICATION OF NONNEGATIVE SYSTEMS

A¯x ∈ R(

7 2

2

z¯1 , [z1T , (z1′ )T ]T ∈ R(n 1 2

2

n +np+ n)×(6n +2np+n+1)

2

+np+n)×1

,

is given by

W1 ⊗ E  0(n2 +np)×n 0(n2 +np)×1  In ⊗ In 0n×p 0n2 ×n 0n2 ×1  T   ¯  I ⊗ e 0 I 0n×1 Ax ,  n 1×p n  2 0n2 ×(n2 +np) 0n ×n 0n2 ×1 0 1 n(n−1)×(n2 +np) 0 1 n(n−1)×n 0 1 n(n−1)×1 2 2  2  2 In +np − 0n W1 ⊗ F  0n2 ×(n2 +np) K  , 0n×(n2 +np) L   0n2 ×(n2 +np) G 0 1 n(n−1)×(n2 +np) H  2    L , − 0n In ⊗ 01×n eT , 

and ¯bx ∈ R

7 2

1 2

n2 +np+ n

is given by   0(2n2 +np+n)×1 ¯bx , δ  . vec In 1 0 n(n−1)×1

(18.94)

(18.95)

(18.96)

2

Once again, the optimization problem just presented can be solved by using R the SeDuMi MATLAB package.

18.5 Illustrative Numerical Examples In this section, we apply the constrained optimization algorithms of Sections 18.3 and 18.4 to two numerical examples. For each example, we add a zero-mean white noise w(k) to the output signal of the discrete-time system scaled such that the signal-to-noise ratio SNR , ky(1 : ℓ) − w(1 : ℓ)kF /kw(1 : ℓ)kF = 10. The input sequence is a realization of a zero-mean unit-variance noise sequence. For both examples, we let Lx = In , Ly = Ip , and W1 = In . The algorithm developed in Section 18.2 is used to estimate the state sequence. We ran the simulations for 100 realizations of the input and noise sequences and present the locations of the resulting eigenvalues, as well as the average values of the system matrices. For the first example consider the dynamical system (18.1) and (18.2) with     T 0 0.3 0.2 0 0.1 0 0.4  , B =  0.1  , C =  0  , D = 0, (18.97) A= 0 0.5 0 0.6 0 0.3 

so that m = 1, p = 1, and n = 3. Note that this system is a nonnegative

NNBook

September 1, 2009

568

CHAPTER 18 90 1 120

60 0.8 0.6

150

30 0.4 0.2

180

0

330

210

240

300 270

Figure 18.1 System poles: ◦ represent pole estimates from the constrained optimization algorithm.

dynamical system. Furthermore, the eigenvalues of A are 0.8136, −0.1068 + 0.2497j, and −0.1068 − 0.2497j, and (A, B) is controllable and (A, C) is observable. For our simulations we choose δ = 0.001 and ℓ = 256. The eigenvalues for 10 out of the 100 simulations are shown in Figure 18.1, while the average values of the system matrices are 

   0.0000 0.2959 0.1842 0.0000 A =  0.0000 0.0000 0.3929  , B =  0.1000  , 0.4829 0.0000 0.5782 0.0000  T 0.1334 C =  0.0295  , D = 0.0031. 0.3022

(18.98)

For the second example, we consider a third-order compartmental model given by (18.1) and (18.2) with 

   1 − a21 a12 0 1 , B =  0 , 1 − (a12 + a32 ) 0 A =  a21 0 a32 1 − a33 0

NNBook

September 1, 2009

569

IDENTIFICATION OF NONNEGATIVE SYSTEMS 90 1 120

60 0.8 0.6

150

30 0.4 0.2

180

0

330

210

240

300 270

Figure 18.2 System poles: ◦ represent pole estimates from the constrained optimization algorithm.

T 0 C =  0  , D = 0, a33 

(18.99)

so that m = 1, p = 1, and n = 3. As shown in Example 2.11, this system models the flow of thyroxine when injected into the blood stream and then carried into the liver where it is converted into iodine which in turn is absorbed into the bile. Note that n 1 1p (a12 + a21 + a32 )2 − 4a21 a32 , spec (A) = 1 − (a12 + a21 + a32 ) + 2 2 1 1p (a12 + a21 + a32 )2 − 4a21 a32 , 1 − (a12 + a21 + a32 ) − 2 o 2 1 − a33 .

(18.100)

The parameters a12 , a21 , a32 , and a33 are given in Table 18.1. For these parameters, the eigenvalues of A are 0.7, −0.2568, and 0.8568. Once again, we choose δ = 0.001 and ℓ = 256 for our simulations. The eigenvalues for 10 out of the 100 simulations are shown in Figure 18.2, while the average

NNBook

September 1, 2009

570

CHAPTER 18

Table 18.1 Three-compartment thyroxine model parameters.

a12 0.5

a21 0.6

a32 0.3

a33 0.3

values of the system matrices are     0.3998 0.4998 0.0000 1.0000 A =  0.5998 0.1999 0.0000  , B =  0.0000  , 0.0000 0.3000 0.7000 0.0000  T 0.0023 (18.101) C =  0.0048  , D = 0.0026. 0.3006

NNBook

September 1, 2009

Chapter Nineteen

Conclusion

In this monograph, we have developed a dynamical systems and control theory framework for nonnegative and compartmental systems. These systems are conceptually simple and yet remarkably effective in describing the essential features of a wide range of dynamical systems involving compartments containing variable nonnegative quantities of a particular substance coupled by connectors governed by intercompartmental flow laws. Such models include biomedical, demographic, epidemic, ecological, economic, pharmacological, telecommunications, transportation, power, heat transfer, fluid, structural vibration, network, and thermodynamic systems. In addition, suboptimal, optimal, and adaptive feedback control architectures for nonnegative and compartmental dynamical systems were developed. It was shown that these control architectures require specialized structures since control inputs are usually constrained to be nonnegative as are the system states of the compartments. The underlying intention of this monograph has been to present a general analysis and control design framework for nonnegative and compartmental dynamical systems in order to significantly increase our understanding of these systems as well as advance the state of the art in active control of these systems. It is hoped that this monograph will help stimulate increased interaction between physicists, economists, biologists, and physicians, and dynamical systems and control theorists. The potential and opportunities for applying and extending this work to engineering and life sciences are enormous. Extensions include infinite-dimensional flow compartmental systems involving advection-diffusion models, stochastic compartmental systems whose transfer rates are given in terms of probabilities of transfers, identifiability, identification, and model distinguishability of nonlinear nonnegative and compartmental systems, and parameter estimation and optimal filtering of nonnegative and compartmental systems. These extensions all play a critical role in the analysis and control design of the aforementioned applications of nonnegative and compartmental systems.

NNBook

September 1, 2009

NNBook

September 1, 2009

Bibliography

[1] A. R. Absalom, N. Sutcliffe, and G. N. Kenny, “Closed-loop control of anesthesia using bispectral index: Performance assessment in patients undergoing major orthopedic surgery under combined general and regional anesthesia,” Anesthesiology, vol. 96, pp. 67–73, 2002. [2] M. A. Aizerman and F. R. Gantmacher, Absolute Stability of Regulator Systems. San Francisco, CA: Holden-Day, 1964. [3] J. M. Alvis, J. G. Reves, A. V. Govier, P. G. Menkhaus, C. E. Henling, J. A. Spain, and E. Bradley, “Computer-assisted continuous infusions of fentanyl during cardiac anesthesia: Comparison with a manual method,” Anesthesiology, vol. 63, pp. 41–49, 1985. [4] D. H. Anderson, Compartmental Modeling and Tracer Kinetics. New York: Springer-Verlag, 1983. [5] D. H. Anderson, “Properties of the transfer functions of compartmental models,” Math. Biosci., vol. 71, pp. 105–119, 1984. [6] L. Andersson and T. Elfving, “A constrained procrustes problem,” SIAM J. Matrix Anal. Appl., vol. 18, pp. 124–139, 1997. [7] T. M. Apostol, Mathematical Analysis. Reading, MA: Addison-Wesley, 1974. [8] V. I. Arnold, Mathematical Models of Classical Mechanics. New York: Springer-Verlag, 1989. [9] J. M. Arnsparger, B. C. McInnis, J. R. Glover, and N. A. Normann, “Adaptive control of blood pressure,” IEEE Trans. Biomed. Eng., vol. 30, pp. 168–176, 1983. [10] K. J. ˚ Astr¨om and B. Wittenmark, Adaptive Control. Reading, MA: Addison-Wesley, 1989.

NNBook

September 1, 2009

574

BIBLIOGRAPHY

[11] S. Audoly and L. D’Angio, “On the identifiability of linear compartmental systems: A revisited transfer function approach based on topological properties,” Math. Biosci., vol. 66, pp. 201–228, 1983. [12] J. M. Bailey, personal communication, 2001. [13] J. M. Bailey, “Management of the hemodynamically unstable patient,” in Current Reviews in Clinical Anesthesia (F. Moya, ed.), vol. 20, lesson 12. Miami: Current Reviews, 2000. [14] J. M. Bailey and S. L. Shafer, “A simple analytical solution to the three compartment pharmacokinetic model suitable for computer controlled infusion pumps,” IEEE Trans. Biomed. Eng., vol. 38, pp. 522–525, 1991. [15] D. D. Bainov and P. S. Simeonov, Systems with Impulse Effect: Stability, Theory and Applications. Chichester, U.K.: Ellis-Horwood, 1989. [16] D. D. Bainov and P. S. Simeonov, Impulsive Differential Equations: Asymptotic Properties of the Solutions. Singapore: World Scientific, 1995. [17] P. Barbini, “Nonlinear models of the mechanics of breathing applied to the use and design of ventilators,” J. Biomed. Eng., vol. 4, pp. 294– 304, 1982. [18] K. Behbehani and R. R. Cross, “A controller for regulation of mean arterial blood pressure using optimum nitroprusside infusion rate,” IEEE Trans. Biomed. Eng., vol. 38, pp. 513–521, 1991. [19] R. Bellman, “Topics in pharmacokinetics. I. Concentration dependent rates,” Math. Biosci., vol. 6, pp. 13–17, 1970. [20] A. Berman, M. Neumann, and R. J. Stern, Nonnegative Matrices in Dynamic Systems. New York: Wiley, 1989. [21] A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences. New York: Academic, 1979. [22] A. Berman, R. S. Varga, and R. C. Ward, “ALPS: Matrices with nonpositive off-diagonal entries,” Linear Algebra Appl., vol. 21, pp. 233–244, 1978. [23] D. S. Bernstein, Matrix Mathematics. Princeton, NJ: Princeton Univ. Press, 2005.

NNBook

BIBLIOGRAPHY

September 1, 2009

575

[24] D. S. Bernstein and S. P. Bhat, “Lyapunov stability, semistability, and asymptotic stability of matrix second-order systems,” ASME Trans. J. Vibr. Acoustics, vol. 117, pp. 145–153, 1995. [25] D. S. Bernstein and S. P. Bhat, “Nonnegativity, reducibility, and semistability of mass action kinetics,” in Proc. IEEE Conf. Decision and Control, Phoenix, AZ, pp. 2206–2211, 1999. [26] D. S. Bernstein and W. M. Haddad, “LQG control with an H∞ performance bound: A Riccati equation approach,” IEEE Trans. Autom. Control, vol. 34, pp. 293–305, 1989. [27] D. S. Bernstein and W. M. Haddad, “Steady-state Kalman filtering with an H∞ error bound,” Syst. Control Lett., vol. 12, pp. 9–16, 1989. [28] D. S. Bernstein, W. M. Haddad, and C. N. Nett, “Minimal complexity control law synthesis. Part 2: Problem solution via H2 /H∞ optimal static output feedback,” in Proc. Am. Control Conf., Pittsburgh, PA, pp. 2506–2511, 1989. [29] D. S. Bernstein and D. C. Hyland, “Compartmental modeling and second-moment analysis of state space systems,” SIAM J. Matrix Anal. Appl., vol. 14, pp. 880–901, 1993. [30] D. S. Bernstein and D. C. Hyland, “Optimal projection approach to robust, fixed-structure control design,” in Mechanics and Control of Space Structures (J. Junkins, ed.). Washington, DC: AIAA, 1993. [31] S. P. Bhat and D. S. Bernstein, “Lyapunov analysis of semistability,” in Am. Control Conf., San Diego, CA, pp. 1608–1612, 1999. [32] S. P. Bhat and D. S. Bernstein, “Finite-time stability of continuous autonomous systems,” SIAM J. Control Optim., vol. 38, pp. 751–766, 2000. [33] S. P. Bhat and D. S. Bernstein, “Arc-length-based Lyapunov tests for convergence and stability in systems having a continuum of equilibria,” in Proc. Am. Control Conf., Denver, CO, pp. 2961–2966, 2003. [34] S. P. Bhat and D. S. Bernstein, “Nontangency-based Lyapunov tests for convergence and stability in systems having a continuum of equilibra,” SIAM J. Control Optim., vol. 42, pp. 1745–1775, 2003. [35] S. P. Bhat and D. S. Bernstein, “Geometric homogeneity with applications to finite-time stability,” Math. Control Signals Syst., vol. 17, pp. 101–127, 2005.

NNBook

September 1, 2009

576

BIBLIOGRAPHY

[36] R. G. Bickford, “Automatic electroencephalographic control of anesthesia (servo-anesthesia),” Electroenceph. Clin. Neurophysiol., vol. 3, pp. 83–86, 1951. [37] M. Biggs, Algebraic Graph Theory, 2nd ed. Cambridge Univ. Press, 1993.

Cambridge, U.K.:

[38] L. J. Bledsoe, “Linear and nonlinear approaches for ecosystem dynamic modeling,” in Systems Analysis and Simulation in Ecology (B. C. Patten, ed.), vol. 4, ch. 10, pp. 283–297. New York: Academic, 1976. [39] S. P. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory. Philadelphia, PA: SIAM, 1994. [40] H. Brezis, “On a characterization of flow-induced invariant sets,” Commun. Pure Appl. Math., vol. 23, pp. 261–263, 1970. [41] P. Bridgman, The Nature of Thermodynamics. Cambridge, MA: Harvard Univ. Press, 1941. Reprinted by Peter Smith: Gloucester, MA, 1969. [42] R. W. Brockett, “Asymptotic stability and feedback stabilization,” in Differential Geometric Control Theory (R. W. Brockett, R. S. Millman, and H. J. Sussmann, eds.). Boston, MA: Birkhauser, 1983. [43] R. F. Brown, “Compartmental system analysis: State of the art,” IEEE Trans. Biomed. Eng., vol. 27, pp. 1–11, 1980. [44] C. I. Byrnes and W. Lin, “Losslessness, feedback equivalence, and the global stabilization of discrete-time nonlinear systems,” IEEE Trans. Autom. Control, vol. 39, pp. 83–98, 1994. [45] S. A. Campbell and J. Belar, “Multiple-delayed differential equations as models for biological control systems,” in Proc. World Math. Conf., pp. 3110–3117, 1993. [46] D. Campbell and J. Brown, “The electrical analog of the lung,” Brit. J. Anaesth., vol. 35, pp. 684–693, 1963. [47] S. L. Campbell and N. J. Rose, “Singular perturbation of autonomous linear systems,” SIAM J. Math. Anal., vol. 10, pp. 542–551, 1979. [48] C. Carath´eodory, “Untersuchungen u ¨ber die Grundlagen der Thermodynamik,” Math. Annalen, vol. 67, pp. 355–386, 1909.

NNBook

BIBLIOGRAPHY

September 1, 2009

577

¨ [49] C. Carath´eodory, “Uber die Bestimmung der Energie und der absoluten Temperatur mit Hilfe von reversiblen Prozessen,” Sitzungsberichte der preuβischen Akademie der Wissenschaften, Math. Phys. Klasse, pp. 39–47, 1925. [50] E. R. Carson, C. Cobelli, and L. Finkelstein, The Mathematical Modeling of Metabolic and Endocrine Systems. New York: Wiley, 1983. [51] H. Caswell, Matrix Population Models: Construction, Analysis and Interpretation. Sunderland, MA: Sinauer Associates, 1989. [52] V. Chellaboina, W. M. Haddad, D. S. Bernstein, and D. A. Wilson, “Induced convolution operator norms of linear dynamical systems,” Math. Control Signals Syst., vol. 13, pp. 216–239, 2000. [53] V. Chellaboina, W. M. Haddad, J. Ramakrishnan, and T. Hayakawa, “Direct adaptive control of nonnegative and compartmental systems with time delay,” in Proc. Am. Control Conf., Boston, MA, pp. 1235– 1240, 2004. [54] V. Chellabonia and W. M. Haddad, “Stability margins of discrete-time nonlinear-nonquadratic optimal regulators,” Int. J. Syst. Sci., vol. 33, pp. 577–584, 2002. [55] V. Chellabonia and W. M. Haddad, “Exponentially dissipative nonlinear dynamical systems: A nonlinear extension of strict positive realness,” J. Math. Prob. Eng., vol. 2003, pp. 25–45, 2003. [56] C. T. Chen, W. L. Lin, and T. S. Kuo, “Adaptive control of arterial blood pressure with a learning controller based on multilayer neural networks,” IEEE Trans. Biomed. Eng., vol. 44, pp. 601–609, 1997. [57] Y. Cherruault, Mathematical Modelling in Biomedicine. Dordrecht, The Netherlands: Reidel, 1986. [58] C. Cobelli, A. Lepschy, and G. R. Jacur, “Identifiability of compartmental systems and related structural properties,” Math. Biosci., vol. 44, pp. 1–18, 1979. [59] E. G. Collins and D. Sadhukhan, “A comparison of descent and continuation algorithms for H2 optimal reduced-order control design,” Int. J. Control, vol. 69, pp. 647–662, 1998. [60] A. R. Cooper and G. V. Jeffreys, Chemical Kinetics and Reactor Design. Englewood Cliffs, NJ: Prentice-Hall, 1971.

NNBook

578

September 1, 2009

BIBLIOGRAPHY

[61] M. G. Crandall, “A Generalization of Peano’s Theorem and Flow Invariance,” MRC Technical Summary Report 1228, 1972. [62] P. S. Crooke, J. D. Head, and J. J. Marini, “A general twocompartment model for mechanical ventilation,” Mathematical and Computer Modeling, vol. 24, pp. 1–18, 1996. [63] M. Davidian and D. M. Giltinan, Nonlinear Models for Repeated Measurement Data. Boca Raton, FL: Chapman and Hall/CRC, 1995. [64] J. Delforge, “Necessary and sufficient structural condition for local identifiability of a system with linear compartments,” Math. Biosci., vol. 54, pp. 159–180, 1981. [65] J. Delforge, L. D’Angio, and S. Audoly, “On the use of the norm-coerciveness theorem in the identifiability problem of linear compartmental models,” IEEE Trans. Autom. Control, vol. 31, pp. 573–575, 1986. [66] J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Englewood Cliffs, NJ: Prentice-Hall, 1983. [67] R. Diestel, Graph Theory. New York: Springer-Verlag, 1997. [68] V. A. Doze, L. M. Westphal, and P. F. White, “Comparison of propofol with methohexital for outpatient anesthesia,” Aneth. Analg., vol. 65, pp. 1189–1195, 1986. [69] D. Drobin and R. G. Hahn, “Kinetics of isotonic and hypertonic plasma volume expanders,” Anethesiology, vol. 96, pp. 1371–1380, 2002. [70] G. E. Dullerud and F. Paganini, A Course in Robust Control Theory: A Convex Approach. New York: Springer-Verlag, 2000. [71] J. Earman, “Irreversibility and temporal asymmetry,” Journal of Philosophy, vol. 64, pp. 543–549, 1967. [72] Y. Ebihara and T. Hagiwara, “New dilated LMI characterizations for continuous-time multiobjective controller synthesis,” Automatica, vol. 40, pp. 2003–2009, 2004. [73] R. G. Eckenhoff and J. S. Johansson, “On the relevance of “clinically relevant concentrations” of inhaled anesthetics in in vitro experiments,” Anesthesiology, vol. 91, pp. 856–860, 1999.

NNBook

September 1, 2009

BIBLIOGRAPHY

579

[74] A. Eddington, The Nature of the Physical World. London, U.K.: Dent & Sons, 1935. [75] E. I. Eger, Anesthetic Uptake and Action. Baltimore, MD: Williams and Wilkins, 1997. [76] J. Eisenfeld, “On identifiability of impulse-response in compartmental systems,” Math. Biosci., vol. 47, pp. 15–33, 1979. [77] M. A. Epstein and R. A. Epstein, “Airway flow patterns during mechanical ventilation of infants: A mathematical model,” IEEE Trans. Biomed. Eng., vol. 26, pp. 299–306, 1979. [78] P. Erdi and J. Toth, Mathematical Models of Chemical Reactions: Theory and Applications of Deterministic and Stochastic Models. Princeton, NJ: Princeton Univ. Press, 1988. [79] L. Farina and S. Rinaldi, Positive Linear Systems: Theory and Applications. New York: Wiley, 2000. [80] M. Farkas, Periodic Motions. New York: Springer-Verlag, 1994. [81] M. Feinberg, “Chemical reaction network structure and the stability of complex isothermal reactors I: The deficiency zero and deficiency one theorems,” Chem. Eng. Sci., vol. 42, pp. 2229–2268, 1987. [82] M. Feinberg, “The existence and uniqueness of steady states for a class of chemical reaction networks,” Arch. Rational Mech. Anal., vol. 132, pp. 311–370, 1995. [83] D. Fife, “Which linear compartmental systems contain traps?,” Math. Biosci., vol. 14, pp. 311–315, 1972. [84] D. M. Foster and J. A. Jacquez, “Multiple zeros for eigenvalues and the multiplicity of traps of a linear compartmental system,” Math. Biosci., vol. 26, pp. 89–97, 1975. [85] A. L. Fradkov, “Quadratic Lyapunov functions in the adaptive stabilization problem of a linear dynamic plant,” Siberian Math. J., vol. 17, pp. 341–348, 1976. [86] A. L. Fradkov and D. J. Hill, “Exponential feedback passivity and stabilizability of nonlinear systems,” Automatica, vol. 34, pp. 697–703, 1998. [87] A. L. Fradkov, I. V. Miroshnik, and V. O. Nikiforov, Nonlinear and Adaptive Control of Complex Systems. Dordrecht, The Netherlands: Kluwer, 1999.

NNBook

580

September 1, 2009

BIBLIOGRAPHY

[88] R. E. Funderlic and J. B. Mankin, “Solution of homogeneous systems of linear equations arising from compartmental models,” SIAM J. Sci. Statist. Comput., vol. 2, pp. 375–383, 1981. [89] B. C. Gargash and D. P. Mital, “A necessary and sufficient condition of global structural identifiability of compartmental models,” Comput. Bio. Med., vol. 10, pp. 237–243, 1980. [90] M. R. Garzia and C. M. Lockhart, “Nonhierarchical communications networks: An application of compartmental modeling,” IEEE Trans. Communications, vol. 37, pp. 555–564, 1989. [91] K. Gatermann and B. Huber, “A family of sparse polynomial systems arising in chemical reaction systems,” J. Symbolic Computation, vol. 33, pp. 275–305, 2002. [92] K. Gatermann and M. Wolfrum, “Bernstein’s second theorem and Viro’s method for sparse polynomial systems in chemistry,” Adv. Appl. Math., vol. 34, pp. 252–294, 2005. [93] A. Gentilini, M. Rossoni-Gerosa, C. W. Frei, R. Wymann, M. Morari, A. M. Zbinden, and T. W. Schnider, “Modeling and closed-loop control of hypnosis by means of bispectral index (BIS) with isoflurane,” IEEE Trans. Biomed. Eng., vol. 48, pp. 874 –889, 2001. [94] T. Van Gestel, J. A. K. Suykens, P. Van Dooren, and B. De Moor, “Identification of stable models in subspace identification by using regularization,” IEEE Trans. Autom. Control, vol. 46, pp. 1416–1420, 2001. [95] M. Gibaldi and D. Perrier, Pharmacokinetics. New York: Marcel Dekker, 1975. [96] A. G. Gilman, J. G. Hardman, and L. E. Limbird, Goodman and Gilman’s The Pharmacological Basis of Therapeutics 10th ed. New York: McGraw-Hill, 1996. [97] P. S. Glass, M. Bloom, L. Kearse, C. Rosow, P. Sebel, and P. Manberg, “Bispectral analysis measures sedation and memory effects of propofol, midazolam, isoflurane, and alfentanil in normal volunteers,” Anesthesiology, vol. 86, pp. 836–847, 1997. [98] P. S. Glass, D. K. Goodman, B. Ginsberg, J. G. Reves, and J. R. Jacobs, “Accuracy of pharmacokinetic model-driven infusion of propofol,” Anesthesiology, vol. 71, p. A277, 1989.

NNBook

BIBLIOGRAPHY

September 1, 2009

581

[99] P. S. A. Glass and I. J. Rampil, “Automated anesthesia: Fact or fantasy?,” Anesthesiology, vol. 95, pp. 1–2, 2001. [100] K. Godfrey, Compartmental Models and their Applications. New York: Academic, 1983. [101] C. Godsil and G. Royle, Algebraic Graph Theory. New York: SpringerVerlag, 2001. [102] G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: The Johns Hopkins Univ. Press, 1996. [103] K. Gopalsamy, Stability and Oscillations in Delay Differential Equations of Population Dynamics. Dordrecht, The Netherlands: Kluwer, 1992. [104] A. Gr¨ unbaum, “The anisotropy of time,” in The Nature of Time (T. Gold, ed.). Ithaca, NY: Cornell Univ. Press, 1967. [105] J. L. Gross and J. Yellen, Handbook of Graph Theory. Boca Raton, FL: CRC, 2004. [106] I. Gyori, “Delay differential and integro-differential equations in biological compartment models,” Syst. Sci., vol. 8, no. 2–3, pp. 167– 187, 1982. [107] W. M. Haddad and D. S. Bernstein, “Explicit construction of quadratic Lyapunov functions for small gain, positivity, circle, and Popov theorems and their application to robust stability. Part I: Continuous-time theory,” Int. J. Robust. Nonlin. Control, vol. 3, pp. 313–339, 1993. [108] W. M. Haddad and D. S. Bernstein, “Explicit construction of quadratic Lyapunov functions for small gain, positivity, circle, and Popov theorems and their application to robust stability. Part II: Discrete-time theory,” Int. J. Robust Nonlinear Control, vol. 4, pp. 249–265, 1994. [109] W. M. Haddad and D. S. Bernstein, “Parameter-dependent Lyapunov functions and the Popov criterion in robust analysis and synthesis,” IEEE Trans. Autom. Control, vol. 40, pp. 536–543, 1995. [110] W. M. Haddad and V. Chellaboina, “Stability theory for nonnegative and compartmental dynamical systems with time delay,” in Proc. Am. Control Conf., Denver, CO, 2003. [111] W. M. Haddad and V. Chellaboina, “Stability and dissipativity theory for nonnegative dynamical systems: A unified analysis framework for

NNBook

582

September 1, 2009

BIBLIOGRAPHY

biological and physiological systems,” Nonlinear Analysis: Real World Applications, vol. 6, pp. 35–65, 2005. [112] W. M. Haddad and V. Chellaboina, Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach. Princeton, NJ: Princeton Univ. Press, 2008. [113] W. M. Haddad, V. Chellaboina, and E. August, “Stability and dissipativity theory for nonnegative dynamical systems: A thermodynamic framework for biological and physiological systems,” in Proc. IEEE Conf. Decision and Control, Orlando, FL, pp. 442–458, 2001. [114] W. M. Haddad, V. Chellaboina, E. August, and J. M. Bailey, “Nonnegative and compartmental dynamical systems in biology, medicine, and ecology,” in Proc. ASME Int. Mech. Eng. Cong., New Orleans, LA, November 2002. [115] W. M. Haddad, V. Chellaboina, and N. A. Kablar, “Nonlinear Impulsive Dynamical Systems. Part I: Stability and Dissipativity,” Int. J. Control, vol. 74, pp. 1631–1658, 2001. [116] W. M. Haddad, V. Chellaboina, and S. G. Nersesov, Thermodynamics: A Dynamical Systems Approach. Princeton, NJ: Princeton Univ. Press, 2005. [117] W. M. Haddad, V. Chellaboina, and S. G. Nersesov, Impulsive and Hybrid Dynamical Systems: Stability, Dissipativity, and Control. Princeton, NJ: Princeton Univ. Press, 2006. [118] W. M. Haddad, V. Chellaboina, and S. G. Nersesov, “Time-reversal symmetry, Poincar´e recurrence, irreversibility, and the entropic arrow of time: From mechanics to system thermodynamics,” Nonlinear Analysis: Real World Applications, vol. 9, pp. 250–271, 2008. [119] W. M. Haddad and V. Chellaboina, “Stability and dissipativity theory for nonnegative and compartmental dynamical systems with time delay,” in Advances in Time-Delay Systems (S.-I. Niculescu and K. Gu, eds.), New York: Springer, pp. 421-435, 2004. [120] W. M. Haddad, T. Hayakawa, and J. M. Bailey, “Adaptive control for nonnegative and compartmental dynamical systems with applications to general anesthesia,” Int. J. Adapt. Control Signal Process., vol. 17, pp. 209–235, 2003.

NNBook

September 1, 2009

BIBLIOGRAPHY

583

[121] W. M. Haddad, T. Hayakawa, and J. M. Bailey, “Nonlinear adaptive control for intensive care unit sedation and operating room hypnosis,” in Proc. Am. Control Conf., Denver, CO, pp. 1808–1813, 2003. [122] W. M. Haddad, T. Hayakawa, and J. M. Baily, “Adaptive control for nonlinear compartmental dynamical systems with applications to clinical pharmacology,” Syst. Control Lett., vol. 55, pp. 62–70, 2006. [123] W. M. Haddad, J. P. How, S. R. Hall, and D. S. Bernstein, “Extensions of mixed-µ bounds to monotonic and odd monotonic nonlinearities using absolute stability theory,” Int. J. Control, vol. 60, pp. 905–951, 1994. [124] W. M. Haddad, V. Kapila, and V. Chellaboina, “Guaranteed domains of attraction for multivariable Lur´e systems via open Lyapunov surfaces,” Int. J. Robust Nonlinear Control, vol. 7, pp. 935–949, 1997. [125] W. M. Haddad, S. G. Nersesov, and V. Chellaboina, “Lyapunov function proof of Poincar´es’ theorem,” Int. J. Syst. Sci., vol. 35, pp. 287–292, 2004. [126] J. K. Hale, “Dynamical systems and stability,” J. Math. Anal. Appl., vol. 26, pp. 39–59, 1969. [127] J. K. Hale, Ordinary Differential Equations, 2nd ed. New York: Wiley, 1980. Reprinted by Krieger, Malabar, 1991. [128] J. K. Hale and S. M. Verduyn Lunel, Introduction to Functional Differential Equations. New York: Springer-Verlag, 1993. [129] M. Hall, Combinatorial Theory. New York: Wiley, 1986. [130] G. H. Hardy, J. E. Littlewood, and G. Polya, Inequalities. Cambridge, U.K.: Cambridge Univ. Press, 1952. [131] P. Hartman, “On invariant sets and on a theorem of Wazewski,” Proc. Am. Math. Soc., vol. 32, pp. 511–520, 1972. [132] T. Hayakawa, W. M. Haddad, J. M. Baily, and N. Hovakimyan, “Passivity-based neural network adaptive output feedback control for nonlinear nonnegative dynamical systems,” IEEE Trans. Neural Networks, vol. 16, pp. 387–398, 2005. [133] T. Hayakawa, W. M. Haddad, N. Hovakimyan, and V. Chellaboina, “Neural network adaptive control for nonlinear nonnegative dynamical systems,” IEEE Trans. Neural Networks, vol. 16, pp. 399–413, 2005.

NNBook

584

September 1, 2009

BIBLIOGRAPHY

[134] J. Z. Hearon, “Monotonicity theorem for compartmental systems,” Math. Biosci., vol. 46, pp. 293–300, 1979. [135] W. G. He, H. Kaufman, and R. J. Roy, “Multiple model adaptive control procedure for blood pressure control,” IEEE Trans. Biomed. Eng., vol. 33, pp. 10–19, 1986. [136] C. M. Held and R. J. Roy, “Multiple drug hemodynamic control by means of a supervisory-fuzzy rule-based adaptive control system: Validation on a model,” IEEE Trans. Biomed. Eng., vol. 42, pp. 371– 385, 1995. [137] T. K. Henthorn, T. C. Krejcie, C. U. Niemann, C. Enders-Klein, C. A. Shanks, and M. J. Avram, “Ketamine distribution described by a recirculatory pharmacokinetic model is not stereo-selective,” Anesthesiology, vol. 91, pp. 1733–1743, 1999. [138] A. V. Hill, “The possible effects of the aggregation of the molecules of haemoglobin on its dissociation curves,” J. Physiol., vol. 40, pp. iv–vii, 1910. [139] D. J. Hill and P. J. Moylan, “The stability of nonlinear dissipative systems,” IEEE Trans. Autom. Control, vol. 21, pp. 708–711, 1976. [140] D. J. Hill and P. J. Moylan, “Stability results of nonlinear feedback systems,” Automatica, vol. 13, pp. 377–382, 1977. [141] H. Hirata, “Modeling and analysis of ecological systems: The largescale system viewpoint,” Int. J. Syst. Sci., vol. 18, pp. 1839–1855, 1987. [142] M. W. Hirsch, “Systems of differential equations which are competetive or cooperative. I: Limit sets,” SIAM J. Math. Anal., vol. 13, pp. 167–179, 1982. [143] W. F. Hofman and D. C. Meyer, “Essentials of Human Physiology,” in Respiratory Physiology, 2nd ed. (T. M. Nosek, ed.), Chicago, IL: Gold Standard Media, 1999. [144] R. A. Horn and R. C. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1985. [145] R. A. Horn and R. C. Johnson, Topics in Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1995. [146] K. Horsfield, “Diameters, generations, and orders of branches in the bronchial tree,” J. Appl. Physiol., vol. 68, pp. 457–461, 1990.

NNBook

September 1, 2009

BIBLIOGRAPHY

585

[147] K. Horsfield and G. Cumming, “Morphology of the bronchial tree in man,” J. Appl. Physiol., vol. 38, pp. 990–1975, 1975. [148] P. Horwich, Asymmetries in Time. Cambridge, MA: MIT Press, 1987. [149] J. R. Hotchkiss, P. S. Crooke, A. B. Adams, and J. J. Marini, “Implications of a biphasic two-compartment model of constant flow ventilation for the clinical setting,” Journal of Critical Care, vol. 9, pp. 114–123, 1994. [150] Q. Hui, W. M. Haddad, and S. P. Bhat, “Finite-time semistability and consensus for nonlinear dynamical networks,” IEEE Trans. Autom. Control, vol. 53, pp. 1887–1900, 2008. [151] Q. Hui, W. M. Haddad, V. Chellaboina, and T. Hayakawa, “Adaptive control of mammillary drug delivery systems with actuator amplitude constraints and system time delays,” Eur. J. Control, vol. 11, pp. 586– 600, 2005. [152] C. J. Hull, “How far can we go with compartmental models?,” Anesthesiology, vol. 72, pp. 399–402, 1990. [153] P. A. Ioannou and J. Sun, Robust Adaptive Control. Upper Saddle River, NJ: Prentice-Hall, 1996. [154] E. F. Ismail, S. J. Kim, and M. R. Salem, “Direct effects of propofol on myocardial contractility in situ canine hearts,” Anesthesiology, vol. 79, pp. 964–972, 1992. [155] J. A. Jacquez, Compartmental Analysis in Biology and Medicine, 2nd ed. Ann Arbor, MI: Univ. of Michigan Press, 1985. [156] J. Jacquez, “Compartmental modeling,” in IFAC Proc. Ser., vol. 1, pp. 31–37, 1989. [157] J. A. Jacquez, Modeling with Compartments. BioMedWare, 1999.

Ann Arbor, MI:

[158] J. A. Jacquez and C. P. Simon, “Qualitative theory of compartmental systems,” SIAM Rev., vol. 35, pp. 43–79, 1993. [159] A. S. Jarrah, R. Laubenbacher, B. Stigler, and M. Stillman, “Reverseengineering of polynomial dynamical systems,” Adv. Appl. Math., vol. 39, pp. 477–489, 2007. [160] G. I. Jee and R. J. Roy, “Adaptive control of multiplexed closed-circuit anesthesia,” IEEE Trans. Biomed. Eng., vol. 39, pp. 1071–1080, 1992.

NNBook

September 1, 2009

586

BIBLIOGRAPHY

[161] Z.-P. Jiang and D. J. Hill, “Passivity and disturbance attenuation via output feedback for uncertain nonlinear systems,” IEEE Trans. Autom. Control, vol. 43, pp. 992–997, 1998. [162] R. P. Jones and K. R. Godfrey, “Problems in the compartmental modelling of biomedical systems,” in IEE Conf. Publ., Int. Conf. on Control and Its Appl., vol. 194, pp. 43–47, 1981. [163] J.-N. Juang, Applied System Identification. Upper Saddle River, NJ: Prentice-Hall, 1993. [164] T. Kaczorek, Positive 1D and 2D Systems. London, U.K.: SpringerVerlag, 2002. [165] M. S. Kamel, M. A. Ismail, W. H. Enright, and T. Prasad, “Partitioning of compartmental models in biological systems,” in Proc. IFAC Symposium, Theory and Application of Digital Control, pp. 649– 652, 1982. [166] E. Kaszkurewicz and A. Bhaya, Matrix Diagonal Stability in Systems and Computation. Boston, MA: Birkhauser, 2000. [167] M. Kaya and J. Li, “Modeling aspect of hemorrhage and hemodilution,” in Proc. IEEE Bioeng. Conf., 28th Annual Northeast, pp. 177–178, 2002. [168] T. Kazama, K. Ikeda, and K. Morita, “The pharmacodynamic interaction between propofol and fentanyl with respect to the suppression of somatic or hemodynamic responses to skin incision, peritoneum incision, and abdominal wall retraction,” Anesthesiology, vol. 89, pp. 894–906, 1998. [169] T. Kazama, K. Ikeda, K. Morita, M. Kikura, M. Doi, T. Ikeda, and T. Kurita, “Comparison of the effect site KeOs of propofol for blood pressure and EEG bispectral index in elderly and young patients,” Anesthesiology, vol. 90, pp. 1517–1527, 1999. [170] T. Kazama, T. Kurita, K. Morita, J. Nakata, and S. Sato, “Influence of hemorrhage on propofol pseudo-steady state concentration,” Anesthesiology, vol. 97, pp. 1156–1161, 2002. [171] J. Keener and J. Sneyd, Mathematical Physiology. Springer-Verlag, 1998.

New York:

[172] J. Kemeny and J. Snell, Mathematical Models in the Social Sciences. Cambridge, MA: MIT Press, 1962.

NNBook

BIBLIOGRAPHY

September 1, 2009

587

[173] G. N. Kenny and H. Mantzardis, “Closed-loop control of propofol anaesthesia,” Brit. J. Anaesth., vol. 83, pp. 223–228, 1999. [174] H. K. Khalil, Nonlinear Systems, 3rd ed. Upper Saddle River, NJ: Prentice-Hall, 2002. [175] Y. Kishimoto and D. S. Bernstein, “Energy flow modeling of interconnected structures: A deterministic foundation for statistical energy analysis,” J. Sound Vibr., vol. 186, pp. 407–445, 1995. [176] Y. Kishimoto, D. S. Bernstein, and S. R. Hall, “Dissipative control of energy flow in interconnected systems, I Modal subsystems, II Structural subsystems,” in Proc. AIAA Guid. Nav. Contr. Conf., Fort Worth, TX, pp. 657–666, 1993. [177] Y. Kishimoto, D. S. Bernstein, and S. R. Hall, “Thermodynamic modeling of interconnected systems I: Conservative coupling, II: Dissipative coupling,” J. Sound Vibr., vol. 182, pp. 23–76, 1995. [178] H. Kitaoka and B. Suki, “Branching design of the bronchial tree based on a diameter-flow relationship,” J. Appl. Physiol., vol. 82, pp. 968– 976, 1997. [179] D. E. Koditschek and K. S. Narendra, “Limit cycles of planar quadratic differential equations,” J. Diff. Equat., vol. 54, pp. 181–195, 1984. [180] S. Kostios, “An algorithm for designing feedback stabilizers of nonlinear polynomial systems,” in Proc. Med. Conf. Contr. Autom., Athens, Greece, pp. T23–07, 2007. [181] J. N. Kremer and S. W. Nixon, A Coastal Marine Ecosystem. Berlin, Germany: Springer-Verlag, 1978. [182] P. Kroes, Time: Its Structure and Role in Physical Theories. Dordrecht, The Netherlands: Reidel, 1985. [183] M. Krsti´c, I. Kanellakopoulos, and P. V. Kokotovi´c, Nonlinear and Adaptive Control Design. New York: Wiley, 1995. [184] Y. Kuang, Delay Differential Equations with Applications in Population Dynamics. Boston, MA: Academic, 1993. [185] S. L. Lacy and D. S. Bernstein, “Subspace identification for nonlinear systems that are linear in unmeasured states,” in Proc. IEEE Conf. Decision and Control, Orlando, FL, pp. 3518–3523, 2001.

NNBook

September 1, 2009

588

BIBLIOGRAPHY

[186] S. L. Lacy and D. S. Bernstein, “Subspace identification with guaranteed stability using constrained optimization,” IEEE Trans. Autom. Control, vol. 48, pp. 1259–1263, 2003. [187] G. S. Ladde, “Cellular systems–I. Stability of chemical systems,” Math. Biosci., vol. 29, pp. 309–330, 1976. [188] G. S. Ladde, “Cellular systems–II. Stability of compartmental systems,” Math. Biosci., vol. 30, pp. 1–21, 1976. [189] V. Lakshmikantham, D. D. Bainov, and P. S. Simeonov, Theory of Impulsive Differential Equations. Singapore: World Scientific, 1989. [190] J. S. W. Lamb and J. A. G. Roberts, “Time reversal symmetry in dynamical systems: A survey,” Phys. D, vol. 112, pp. 1–39, 1998. [191] B. Lavenda, Thermodynamics of Irreversible Processes. London, U.K.: Macmillan, 1978. Reprinted by Dover: New York, 1993. [192] P. De Leenheer and D. Aeyels, “Stability properties of equilibria of classes of cooperative systems,” IEEE Trans. Autom. Control, vol. 46, pp. 1996–2001, 2001. [193] P. De Leenheer and D. Aeyels, “Stabilization of Positive Linear Systems,” Syst. Control Lett., vol. 44, pp. 259–271, 2001. [194] S. Lefschetz, Stability of Nonlinear Control Systems. Academic, 1965.

New York:

[195] W. S. Levine and M. Athans, “On the determination of the optimal output feedback gains for linear multivariable systems,” IEEE Trans. Autom. Control, vol. 15, pp. 44–48, 1970. [196] J. H. Levy, L. G. Michelsen, J. S. Shanewise, J. M. Bailey, and J. G. Ramsay, “Postoperative cardiovascular management,” in Cardiac Anesthesia, 4th ed. (J. Kaplan, ed.). Philadelphia, PA: WB Saunders, 1999. [197] D. C. Lewis, “A qualitative analysis of S-systems: Hopf bifurcations,” in Canonical Nonlinear Modeling: S-Systems Approach to Understanding Complexity (E. O. Voit, ed.). New York: Van Nostrand Reinhold, 1991. [198] R. M. Lewis and B. D. O. Anderson, “Insensitivity of a class of nonlinear compartmental systems to the introduction of arbitrary time delays,” IEEE Trans. Circuits Syst. I, vol. 27, pp. 604–612, 1980.

NNBook

BIBLIOGRAPHY

September 1, 2009

589

[199] P. Liberman and C. M. Marle, Symplectic Geometry and Analytical Mechanics. Dordrecht, The Netherlands: Reidel, 1987. [200] D. A. Linkens, M. F. Abbod, and J. E. Peacock, “Clinical implementation of advanced control in anaesthesia,” Trans. Inst. Meas. Control, vol. 22, pp. 303–330, 2000. [201] C. C. Lin and L. A. Segel, Mathematics Applied to Deterministic Problems in the Natural Sciences. New York: Academic, 1974. [202] J. L¨ofberg, “YALMIP : A toolbox for modeling and optimization in MATLAB,” in Proceedings of the CACSD Conference, Taipei, Taiwan, 2004. [203] R. Lozano, B. Brogliato, O. Egeland, and B. Maschke, Dissipative Systems Analysis and Control. London, U.K.: Springer-Verlag, 2000. [204] W. Lu and J. M. Bailey, “The reliability of pharmacodynamic analysis by logistic regression: A computer simulation study,” Anesthesiology, vol. 92, pp. 985–992, 2000. [205] W. Lu, T. G. K Mant, J. H. Levy, and J. M. Bailey, “The pharmacokinetics of recombinant transgenic antithrombin in volunteers,” Anesthesia and Analgesia, vol. 90, pp. 531–534, 2000. [206] W. Lu, J. G. Ramsay, and J. M. Bailey, “Reliability of pharmacodynamic analysis by logistic regression: Mixed-effects modeling,” Anesthesiology, vol. 99, pp. 1255–1262, 2003. [207] N. MacDonald, Time Lags in Biological Models. Berlin, Germany: Springer-Verlag, 1978. [208] H. Maeda and S. Kodama, “Qualitative analysis of a class of nonlinear compartmental systems: Nonoscillation and asymptotic stability,” Math. Biosci., vol. 38, pp. 35–44, 1978. [209] H. Maeda, S. Kodama, and F. Kajiya, “Compartmental system analysis: Realization of a class of linear systems with physical constraints,” IEEE Trans. Circuits Syst. I, vol. 24, pp. 8–14, 1977. [210] H. Maeda, S. Kodama, and T. Konishi, “Stability theory and existence of periodic solutions of time delayed compartmental systems,” Electron. Commun. Jpn., vol. 65, no. 1, pp. 1–8, 1982. [211] H. Maeda, S. Kodama, and Y. Ohta, “Asymptotic behavior of nonlinear compartmental systems: Nonoscillation and stability,” IEEE Trans. Circuits Syst. I, vol. 25, pp. 372–378, 1978.

NNBook

590

September 1, 2009

BIBLIOGRAPHY

[212] P. O. Maitre and D. R. Stanski, “Bayesian forecasting improves the prediction of intraoperative plasma concentrations of alfentanil,” Anesthesiology, vol. 69, pp. 652–659, 1988. [213] C. W. Marcus and R. M. Westervelt, “Stability of analog neural networks with delay,” Phys. Rev., vol. 34, pp. 347–359, 1989. [214] J. J Marini and P. S. Crooke, “A general mathematical model for respiratory dynamics relevant to the clinical setting,” Am. Rev. Respiratory Disease, vol. 147, pp. 14–24, 1993. [215] B. Marsh, M. White, N. Morton, and G. N. Kenny, “Pharmacokinetic model driven infusion of propofol in children,” Brit. J. Anaesth., vol. 67, pp. 41–48, 1991. [216] S. Martinez, G. Michon, and J. S. Martin, “Inverse of strictly ultrametric matrices are of Stieltjes type,” SIAM J. Matrix Anal. Appl., vol. 15, pp. 98–106, 1994. [217] J. Meixner, “On the foundation of thermodynamics of processes,” in A Critical Review of Thermodynamics (E. B. Stuart, B. Gal-Or, and A. J. Brainard, eds.), pp. 37-47. Baltimore, MD: Mono Book Corp., 1970. [218] C. D. Meyer and M. W. Stadelmaier, “Singular M-matrices and inverse positivity,” Linear Algebra Appl., vol. 22, pp. 139–156, 1978. [219] D. C. Mikulecky and S. R. Thomas, “Some network thermodynamic models of coupled dynamic physiological systems,” J. Franklin Inst., vol. 308, pp. 309–325, 1979. [220] R. R. Mohler, “Biological modeling with variable compartmental structure,” IEEE Trans. Autom. Control, vol. 1974, pp. 922–926, 1974. [221] M. Moonen, B. De Moor, L. Vandenberghe, and J. Vandewalle, “Onand off-line identification of linear state-space models,” Int. J. Control, vol. 49, pp. 219–232, 1989. [222] M. Moonen and J. Vandewalle, “QSVD approach to on- and off-line state-space identification,” Int. J. Control, vol. 51, pp. 1133–1146, 1990. [223] M. Morari and A. Gentilini, “Challenges and opportunities in process control: Biomedical processes,” AIChE Journal, vol. 47, pp. 2140– 2143, 2001.

NNBook

BIBLIOGRAPHY

September 1, 2009

591

[224] E. Mortier, M. Struys, T. De Smet, L. Versichelen, and G. Rolly, “Closed-loop controlled administration of propofol using bispectral analysis,” Anaesthesia, vol. 53, pp. 749–754, 1998. [225] S. Muratori and S. Rinaldi, “Equilibria, stability, and reachability of Leslie systems with nonnegative inputs,” IEEE Trans. Autom. Control, vol. 35, pp. 1065–1068, 1990. [226] M. Muzi, R. A. Berens, J. P. Kampine, and T. J. Ebert, “Venodilation contributes to propofol-mediated hypotension in humans,” Aneth. Analg., vol. 74, pp. 877–883, 1992. [227] K. S. Narendra and A. M. Annaswamy, Stable Adaptive Systems. Englewood Cliffs, NJ: Prentice-Hall, 1989. [228] S. G. Nersesov, W. M. Haddad, and V. Chellaboina, “Optimal fixedstructure control for linear nonnegative dynamical systems,” Int. J. Robust Nonlinear Control, vol. 14, pp. 487–511, 2004. [229] S. I. Niculescu, Delay Effects on Stability: A Robust Control Approach. New York: Springer-Verlag, 2001. [230] J. W. Nieuwenhuis, “About nonnegative realizations,” Syst. Control Lett., vol. 1, pp. 283–287, 1982. [231] E. P. Odum, Fundamentals of Ecology, 3rd ed. Philadelphia, PA: Saunders, 1971. [232] Y. Ohta, H. Maeda, and S. Kodama, “Asymptotic behavior of nonlinear compartmental systems,” Electron. Commun. Jpn., vol. 61, pp. 26–31, 1978. [233] Y. Ohta, H. Maeda, and S. Kodama, “Reachability, observability and realizability of continiuous-time postive systems,” SIAM J. Control Optim., vol. 22, pp. 171–180, 1984. [234] R. Olfati-Saber and R. M. Murray, “Consensus Problems in Networks of Agents with Switching Topology and Time-Delays,” IEEE Trans. Autom. Control, vol. 49, pp. 1520–1533, 2004. [235] M. Olinick, An Introduction to Mathematical Models in Social and Life Sciences. Reading, MA: Addison-Wesley, 1978. [236] M. C. De Oliveira, J. C. Geromel, and J. Bernussou, “Extended H2 and H∞ norm characterizations and controller parametrizations for discrete-time systems,” Int. J. Control, vol. 75, pp. 666–679, 2002.

NNBook

592

September 1, 2009

BIBLIOGRAPHY

[237] P. Van Overschee and B. De Moor, “N4SID: Subspace algorithms for the identification of combined deterministic-stochastic systems,” Automatica, vol. 30, pp. 75–93, 1994. [238] P. Van Overschee and B. De Moor, “A unifying theorem for three subspace system identification algorithms,” Automatica, vol. 31, pp. 1853–1864, 1995. [239] P. Van Overschee and B. De Moor, Subspace Identification for Linear Systems: Theory, Implementation, and Applications. Norwell, MA: Kluwer, 1996. [240] G. A. Pajunen, M. Steinmetz, and R. Shankar, “Model reference adaptive control with constraints for postoperative blood pressure management,” IEEE Trans. Biomed. Eng., vol. 37, pp. 679–687, 1990. [241] R. Pintelon and J. Schoukens, System Identification: A Frequency Domain Approach. Piscataway, NJ: IEEE Press, 2001. ¨ [242] M. Planck, “Uber die Begrundung des zweiten Hauptsatzes der Thermodynamik,” Sitzungsberichte der preuβischen Akademie der Wissenschaften, Math. Phys. Klasse, pp. 453–463, 1926. [243] H. Poincar´e, “Sur le probl´eme des trois corps et les ´equations de la dynamique,” Acta Math., vol. 13, pp. 1–270, 1890. [244] V. M. Popov, Hyperstability of Control Systems. New York: SpringerVerlag, 1973. [245] I. Prigogine, From Being to Becoming. San Francisco, CA: Freeman, 1980. [246] I. J. Rampil, “A primer for EEG signal processing in anesthesia,” Anesthesiology, vol. 89, pp. 980–1002, 1998. [247] R. R. Rao, B. Aufderheide, and B. W. Bequette, “Experimental studies on multiple-model predictive control for automated regulation of hemodynamic variables,” IEEE Trans. Biomed. Eng., vol. 50, pp. 277–288, 2003. [248] R. R. Rao, C. C. Palerm, B. Aufderheide, and B. W. Bequette, “Automated regulation of hemodynamic variables,” IEEE Eng. Medicine Biol. Magazine, vol. 20, pp. 24–38, 2001. [249] R. M. Redheffer, “The theorems of Bony and Brezis on flow-invariant sets,” Am. Math. Monthly, vol. 79, pp. 740–747, 1972.

NNBook

September 1, 2009

BIBLIOGRAPHY

593

[250] R. M. Redheffer and W. Walter, “Flow-invariant sets and differential inequalities in normed spaces,” Applicable Anal., vol. 5, pp. 149–161, 1975. [251] H. Reichenbach, The Direction of Time. California Press, 1956.

Berkeley, CA: Univ. of

[252] A. Rescigno and J. S. Beck, “Compartments,” in Foundation of Mathematical Biology (R. Rosen, ed.), pp. 255-322, New York: Academic, 1972. [253] R. G. Ritchie, E. A. Ernst, B. L. Pate, J. D. Pearson, and L. C. Shepherd, “Closed-loop control of an anesthesia delivery system: Development and animal testing,” IEEE Trans. Biomed. Eng., vol. 34, pp. 437–443, 1987. [254] R. T. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970. [255] L. Rosier, “Homogeneous Lyapunov function for homogeneous continuous vector field,” Syst. Control Lett., vol. 19, pp. 467–473, 1992. [256] J. A. Ross, R. T. Wloch, D. C. White, and D. W. Hawes, “Servocontrolled closed-circuit anaesthesia. A method for the automatic control anaesthesia produced by a volatile agent in oxygen,” Brit. J. Anaesth., vol. 55, pp. 229–230, 1982. [257] A. M. Samoilenko and N. A. Perestyuk, Impulsive Differential Equations. Singapore: World Scientific, 1995. [258] I. W. Sandberg, “A note on the properties of compartmental systems,” IEEE Trans. Circuits Syst. I, vol. 25, pp. 379–380, 1978. [259] I. W. Sandberg, “On the mathematical foundations of compartmental analysis in biology, medicine and ecology,” IEEE Trans. Circuits Syst. I, vol. 25, pp. 273–279, 1978. [260] T. Sato and K.-Z. Liu, “LMI solution to general H2 suboptimal control problems,” Syst. Control Lett., vol. 36, pp. 295–305, 1999. [261] T. W. Schnider, C. F. Minto, and D. R. Stanski, “The effect compartment concept in pharmocodynamic modelling,” Anaes. Pharmacol. Rev., vol. 2, pp. 204–219, 1994. [262] H. Schwilden, “A general method for calculating the dosage scheme in linear pharmacokinetics,” Eur. J. Clin. Pharmacol., vol. 20, pp. 379– 386, 1981.

NNBook

September 1, 2009

594

BIBLIOGRAPHY

[263] H. Schwilden, J. Schuttler, and H. Stoeckel, “Closed-loop feedback control of methohexital anesthesia by quantitative EEG analysis in humans,” Anesthesiology, vol. 67, pp. 341–347, 1987. [264] H. Schwilden, H. Stoeckel, and J. Schuttler, “Closed-loop feedback control of propofol anesthesia by quantitative EEG analysis in humans,” Brit. J. Anaesth., vol. 62, pp. 290–296, 1989. [265] S. K. Scott, Chemical Chaos. Oxford, U.K.: Oxford Univ. Press, 1991. [266] P. S. Sebel, E. Lang, I. J. Rampil, P.F. White, R. Cork, M. Jopling, N. T. Smith, P. S. Glass, and P. Manberg, “A multicenter study of bispectral electroencephalogram analysis for monitoring anesthetic effect,” Anesth. Analg., vol. 84, pp. 891–899, 1997. [267] S. L. Shafer and K. Gregg, “Algorithms to rapidly achieve and maintain stable drug concentrations at the site of drug effect with a computer-controlled infusion,” J. Pharmacokinetics Biopharm., vol. 20, pp. 147–169, 1992. [268] S. L. Shafer, J. R. Varvel, N. Aziz, and J. C. Scott, “Pharmacokinetics of fentanyl administered by computer-controlled infusion pump,” Anesthesiology, vol. 73, pp. 1092–1102, 1990. [269] L. B. Sheiner, “The population approach to pharmacokinetic data analysis: Rationale and standard data analysis methods,” Drug Metabolism Reviews, vol. 15, pp. 153–171, 1984. [270] L. B. Sheiner and S. L. Beal, “Evaluation of methods for estimating population pharmacokinetic parameters II. Biexponential model and experimental pharmacokinetic data,” J. Pharmacokinetics Biopharm., vol. 9, pp. 635–651, 1981. [271] L. C. Sheppard, “Computer control of the infusion of vasoactive drugs,” Ann. Biomed. Eng., vol. 8, pp. 431–444, 1980. [272] C. W. Sheppard and W. R. Martin, “Cation exchange between cells and plasma of mammalian blood,” J. Gen. Physiol., vol. 33, pp. 703– 722, 1950. [273] J. C. Sigl and N. G. Chamoun, “An introduction to bispectral analysis for the electroencephalogram,” J. Clin. Monit., vol. 10, pp. 392–404, 1994. [274] D. D. Siljak, Large-Scale Dynamic Systems. Holland, 1978.

New York: North-

NNBook

September 1, 2009

BIBLIOGRAPHY

595

[275] D. D. Siljak, “Complex dynamical systems: Dimensionality, structure and uncertainty,” Large Scale Syst., vol. 4, pp. 279–294, 1983. [276] T. Similowski and J. H. Bates, “Two-compartment modeling of respiratory system mechanics at low frequencies: Gas redistribution or tissue rheology,” Eur. Respiratory J., vol. 4, pp. 353–358, 1991. [277] J. B. Slate, “Model-based design of a controller for infusion of sodium nitroprusside during postsurgical hypertension.” Ph. D. thesis, Univ. of Wisconsin-Madison, 1980. [278] H. L. Smith, Monotone Dynamical Systems. Providence, RI: Am. Math. Soc., 1995. [279] T. Soderstrom and P. Stoica, System Identification. Upper Saddle River, NJ: Prentice-Hall, 1989. [280] E. D. Sontag, “Structure and stability of certain chemical networks and applications to the kinetic proofreading model of T-cell receptor signal transduction,” IEEE Trans. Autom. Control, pp. 1028–1047, 2001. [281] J. I. Steinfeld, J. S. Francisco, and W. L. Hase, Chemical Kinetics and Dynamics. Upper Saddle River, NJ: Prentice-Hall, 1989. [282] M. Struys, M. Coppens, N. De Neve, E. Mortier, A. Doufas, J. Van Bocxlaer, and S. Shafer, “Influence of administration rate on propofol plasma-effect site equilibration,” Anesthesiology, vol. 107, pp. 386–396, 2007. [283] M. Struys, T. De Smet, L. F. M. Versichelen, S. Van de Vilde, R. Van den Broecke, and E. P. Mortier, “Comparison of closed-loop controlled administration of propofol using BIS as the controlled variable versus ‘standard practice’ controlled administration,” Anesthesiology, vol. 95, pp. 6–17, 2001. [284] J. F. Sturm, Using SeDuMi 1.0x, a Matlab Toolbox for Optimization Over Symmetric Cones. Tilburg, The Netherlands: Department of Econometrics, Tilburg Univ., 2001. [285] SeDuMi 1.1R2, A Matlab Toolbox for Optimization Over Symmetric Cones. McMaster Univ., Canada: Advanced Optimization Lab, 2006. [286] K. Thulasiraman and M. N. S. Swamy, Graphs: Algorithms. New York: Wiley, 1992.

Theory and

NNBook

596

September 1, 2009

BIBLIOGRAPHY

[287] J. O. Tsokos and C. P. Tsokos, “Statistical modeling of pharmacokinetic systems,” ASME J. Dyn. Syst. Meas. Control, vol. 98, pp. 37–43, 1976. [288] J. Uffink, “Bluff your way in the second law of thermodynamics,” Stud. Hist. Phil. Mod. Phys., vol. 32, pp. 305–394, 2001. [289] R. N. Upton, G. I. Ludrook, C. Grant, and A. Martinez, “Cardiac output is a determinant of the initial concentration of propofol after short-term administration,” Aneth. Analg., vol. 89, pp. 545–552, 1999. [290] J. Van den Hof, “Structural identifiability of linear compartmental systems,” IEEE Trans. Autom. Control, vol. 43, pp. 800–818, 1998. [291] J. M. van den Hof, “Positive linear observers for linear compartmental systems,” SIAM J. Control Optim., vol. 36, pp. 590–608, 1998. [292] M. Verhaegen, “Subspace model identification part III: Analysis of the ordinary output-error state-space model identification algorithm,” Int. J. Control, vol. 58, pp. 555–586, 1993. [293] M. Verhaegen and P. Dewilde, “Subspace model identification part I: The output-error state-space model identification class of algorithms,” Int. J. Control, vol. 56, pp. 1187–1210, 1992. [294] M. Verhaegen and P. Dewilde, “Subspace model identification part II: Analysis of the elementary output-error state-space model identification algorithm,” Int. J. Control, vol. 56, pp. 1211–1241, 1992. [295] M. Viberg, “Subspace-based methods for the identification of linear time-invariant systems,” Automatica, vol. 31, pp. 1835–1851, 1995. [296] R. Vishnoi and R. J. Roy, “Adaptive control of closed-circuit anesthesia,” IEEE Trans. Biomed. Eng., vol. 38, pp. 39–47, 1991. [297] J. G. Wagner, Fundamentals of Clinical Pharmacokinetics. Hamilton, IL: Drug Intelligence Publications, 1975. [298] S. M. Walas, Reaction Kinetics for Chemical Engineers. New York: McGraw-Hill, 1959. [299] A. A. Wald, T. W. Murphy, and V. D. Mazzia, “A theoretical study of controlled ventilation,” IEEE Trans. Biomed. Eng., vol. 15, pp. 237– 248, 1968. [300] G. G. Walter, “Passage time, resilience, and structure of compartmental models,” Math. Biosci., vol. 63, pp. 199–213, 1983.

NNBook

September 1, 2009

BIBLIOGRAPHY

597

[301] G. G. Walter and M. Contreras, Compartmental Modeling with Networks. Boston, MA: Birkhauser, 1999. [302] E. R. Weibel, Morphometry of the Human Lung. New York: Academic, 1963. [303] A. Weinman, Uncertain Models and Robust Control. Springer-Verlag, 1991.

New York:

[304] R. Weinstock, Calculus of Variations with Applications to Physics and Engineering. New York: McGrow-Hill, 1952. [305] P. G. Welling, Pharmacokinetics: Processes, Mathematics, and Applications, 2nd ed. Washington DC: American Chemical Society, 1997. [306] M. White and G. N. C. Kenny, “Intravenous propofol anaesthesia using a computerised infusion system,” Anaesthesia, vol. 45, pp. 204–209, 1990. [307] S. Wiggins, Introduction to Applied Nonlinear Dynamical Systems and Chaos, 2nd ed. New York: Springer-Verlag, 2003. [308] J. C. Willems, “Dissipative dynamical systems part I: General theory,” Arch. Rational Mech. Anal., vol. 45, pp. 321–351, 1972. [309] J. C. Willems, “Dissipative dynamical systems part II: Quadratic supply rates,” Arch. Rational Mech. Anal., vol. 45, pp. 359–393, 1972. [310] F. W. Wilson, “Smoothing derivatives of functions and applications,” Trans. Am. Math. Soc., vol. 139, pp. 413–428, 1969. [311] T. Yoshizawa, Stability Theory by Liapunov’s Second Method. Tokyo, Japan: Math. Soc. Japan, 1966. [312] C. Yu, R. J. Roy, H. Kaufman, and B. W. Bequette, “Multiplemodel adaptive predictive control of mean arterial pressure and cardiac output,” IEEE Trans. Biomed. Eng., vol. 39, pp. 765–778, 1992. [313] K. Zhou, J. C. Doyle, and K. Glover, Robust and Optimal Control. Englewood Cliffs, NJ: Prentice-Hall, 1996.

NNBook

September 1, 2009

NNBook

September 1, 2009

Index A absence of limit cycles, 126 absence of Poincar´e recurrence, 241 absolute stability criteria, 156, 166, 180 absolute temperature, 253 absolute uncertainty structure, 158, 181 adaptive control, 425 adaptive control for systems with L2 disturbances, 502 adaptive control system with time delay, 463 adaptive control with actuator amplitude constraints, 480 adiabatically isolated system, 250 adjacency matrix, 254 affine hull, 303 affine subspace, 303 afterload, 370 alveoli, 371 analgesia, 371 antithrombin III, 121 arcwise connected component, 14 areflexia, 371 arrow of time, 224 asymptotically stable, 13 discrete-time system, 62

periodic orbit, 526 time-delay discrete system, 104 time-delay system, 91 asymptotically stable matrix, 33 B backward reaction, 283 balanced graph, 316 bispectral index, 373 bispectral-index-based control, 373 blood volume, 517 Boltzmann entropy, 252 bounded trajectory, 17 discrete-time system, 65 breathing cycle, 533 C cardiac output, 59, 366, 517 cardiopulmonary bypass, 121 cardiovascular function, 369 cascade interconnection, 158, 182 causality, 227 central compartment, 361 central venous pressure, 517 chemostat compartmental system, 45

classical thermodynamic inequality, 253 Clausius inequality, 248 clearance, 367 clinical anesthesia, 59 closed compartmental system, 37 discrete-time, 74 closed hybrid system, 206 n closed set relative to R+ , 10 closed system, 240 colloid infusion rate, 518 compact set relative to n R+ , 10 compartmental dynamical system, 468 compartmental function, 44 compartmental lung model, 528 compartmental matrix, 30 compartmental system, 1 complete reversibility, 224, 232 completely null controllable, 159 completely null reachable discrete-time system, 168 completely reachable, 159 discrete-time system, 168 hybrid system, 209 complex, 301

NNBook

September 1, 2009

600

INDEX

configuration manifold, 234 connected component, 264 connected set, 264 connectivity matrix, 247 conservation of energy, 246 consistency property, 9 constrained optimal control, 379 continuous nonnegative feedback, 381 continuously differentiable entropy function, 250 contractility, 370 convergence, 3 convergent system, 22, 241 converse Lyapunov theorem for semistability, 27 converse Lyapunov theorems, 16 crystalloid infusion rate, 518 D deficiency of a reaction network, 303 delay dynamical system, 89 dilation, 347 direct path, 305 directed graph, 254 directly linked complex, 302 discontinuous feedback, 381 discrete-time asymptotically stable matrix, 71 discrete-time compartmental matrix, 70

discrete-time Lyapunov stable matrix, 71 discrete-time semistable matrix, 71 dissipation hypothesis, 143 dissipation inequality, 143, 148 dissipative hybrid system, 208 dissipative system, 148 discrete-time, 167 dissipative time-delay system, 185 discrete-time, 192 dissipativity theory, 143 distributed delay system, 186 disturbance rejection control, 491 disturbance signal, 492 domain of semistability, 23 donor-controlled compartmental system, 45 drug concentration, 368 drug delivery systems, 468 drug effect, 368 dynamic compensation problem, 418 dynamical system, 227

elementary reaction, 284 emergence of time flow, 225 energy, 223 energy balance equation, 244 entropy, 250 equilibrium kinetic equation, 297 equilibrium point, 9 discrete-time system, 62 hybrid system, 199 time-delay discrete system, 103 time-delay system, 90 equilibrium process, 250 essentially nonnegative function, 10 essentially nonnegative matrix, 30 existential statement, 8 exponentially dissipative hybrid system, 208 exponentially dissipative system, 148 exponentially nonaccumulative hybrid system, 214 exponentially nonaccumulative system, 152 F

E edge, 254 effect-site compartment, 376 effect-site concentration, 359 electroencephalograph measurement, 371 electroencephalographbased control, 372 electromyographic noise, 377

feedback control, 380 feedback interconnections of discrete-time nonnegative system, 177 feedback interconnections of hybrid nonnegative system, 217 feedback interconnections of nonnegative system, 153, 164 feedback interconnections of nonnegative system

NNBook

September 1, 2009

601

INDEX

with time delay, 188 finite-time convergent, 266 finite-time energy equipartition, 275 finite-time semistable, 265 finite-time thermodynamics, 263 first law of thermodynamics, 246 flow, 9 forest ecosystem dynamics, 52 forward reaction, 283 Fr´echet derivative, 9 G generalized mass balance equation, 151, 161, 169, 212 generalized momenta, 234 generalized positions, 234 generalized power supply, 229 geometrically dissipative system discrete-time, 167 geometrically nonaccumulative, 171 global invariant set theorem, 21 discrete-time system, 68 time-delay discrete system, 106 time-delay system, 96 global semiflow, 264 globally asymptotically stable, 14 discrete-time system, 63 time-delay discrete system, 104 time-delay system, 91

globally finite-time semistable, 265 globally semistable equilibrium, 23, 69 globally semistable system, 23, 69 H H2 norm, 406 H2 optimal regulator problem, 406 half life, 367 Hamiltonian function, 259 Hamiltonian mechanics, 225 Hamiltonian system, 234 heart rate, 517 hematocrit, 519 hemodilution, 516 hemorrhage, 516 highly perfused organs, 375 Hill equation, 366 homogeneous function, 268 homogeneous vector field, 269 Hurwitz matrix, 33 hybrid compartmental system, 203 hybrid port-controlled Hamiltonian system, 207 hybrid supply rate, 208 hybrid system, 197 hypertonic volume expanders, 121 hypnosis, 371

indirectly linked complex, 302 infimum, 9 inflow-closed compartmental system, 37 discrete-time, 74 inflow-closed hybrid system, 206 initial mean arterial pressure, 516 input-output nonnegative hybrid system, 208 input-output nonnegative system, 146 discrete-time, 166 interpatient variability, 365 intrapatient error model, 364 intrapatient variability, 365 intravascular blood, 360 invariant set, 17 discrete-time system, 65 time-delay discrete system, 105 time-delay system, 94 invariant set stability theorems, 16 discrete-time system, 64 time-delay dynamical system, 93 irreducible matrix, 316 irreversibility, 224 isochoric transformation, 246 isoelectric EEG signal, 377

I K identifiability, 554 identification, 554 in-degree, 528 indirect path, 305

Kalman filter problem, 414 kinetic equation, 285

NNBook

September 1, 2009

602

INDEX

Krasovskii-LaSalle theorem, 19 discrete-time system, 66 time-delay discrete system, 105 time-delay system, 95 L L2 disturbance, 497 L2 , 8 L∞ , 8 Lagrangian, 234 Laplacian, 317 law of mass action, 284 lead kinetic model, 153 lead kinetics, 51 leaf, 528 Leslie model, 82, 177 Lie bracket, 269 Lie derivative, 269 Lie group, 236 Lie pseudogroup, 236 linear compartmental dynamical system discrete-time, 73 linear compartmental system, 36 linear Lyapunov function, 38 discrete-time system, 75 Linear matrix inequality, 405 linearization of discrete-time nonnegative system, 172 linearization of nonnegative dissipative system, 162 linearization of nonnegative system, 43 discrete-time, 83

linearization stability theorem, 43 discrete-time system, 84 linkage class, 302 linked complex, 302 Liouville’s theorem, 235 lipoprotein metabolism model, 49, 153 lossless hybrid system, 209 lossless system, 148 discrete-time, 167 Lotka-Volterra compartmental system, 45 Lotka-Volterra equations, 58 Lotka-Volterra reaction, 287 Lyapunov function, 16 discrete-time system, 64 Lyapunov function candidate, 16 discrete-time system, 64 Lyapunov stability theory, 7 Lyapunov stable, 13 discrete-time system, 62 periodic orbit, 526 time-delay discrete system, 104 time-delay system, 91 Lyapunov stable matrix, 33 Lyapunov stable under f , 344 Lyapunov theorem, 14 discrete-time system, 63 semistability, 25 time-delay discrete system, 104 time-delay system, 92

M M-matrix, 30 mammillary model, 360 mammillary system, 117, 119 mass conservation equation, 169 mass flux conservation equation, 151, 161 mass-action kinetics, 281 maximal interval of existence, 9 mechanical ventilation, 523, 545 Michaelis-Menten kinetics, 363 mixed-effects modeling, 365 monotonic system, 114, 124 discrete-time, 128 monotonic system energies, 260 monotonic time-delay system, 137 multiple pointwise delay system, 186 N Newtonian mechanics, 225 node, 254 nonaccumulative hybrid system, 214 nonaccumulative system, 152 nonaccumulativity, 144 nonequilibrium process, 250 nonlinear compartmental dynamical system, 43, 44 discrete-time, 85 nonlinear port-controlled Hamiltonian system, 46

NNBook

September 1, 2009

603

INDEX

nonnegative control, 476 nonnegative dynamical system, 1, 12 discrete-time, 62 nonnegative function, 30, 62 discrete-time, 70 nonnegative hybrid system, 208 nonnegative matrix, 8, 30 nonnegative orthant, 8 nonnegative orthant feedback holdable, 407 nonnegative orthant holdable, 407 nonnegative orthant stabilizability, 405 nonnegative system, 146 discrete-time, 167 nonnegative time-delay dynamical system, 91 nonnegative time-delay system, 183 discrete-time, 191 nonnegative vector, 8 nonoscillatory system, 112 nonsingular M-matrix, 30 normalized adjacency matrix, 316 null space, 8 O ω-limit set, 17 open ball, 9 open Lyapunov level surface, 386 n open set relative to R+ , 10 open system, 240 orbit, 17 time-delay discrete system, 105 time-delay system, 94 out-degree, 528

outflow-closed compartmental system, 37 discrete-time, 74 outflow-closed hybrid system, 206 P parent airway, 529 partially monotonic system, 114, 124 discrete-time, 128 partially monotonic time-delay system, 137 path, 305 periodic orbit, 526 periodic solution, 526 peripheral compartment, 361 pharmacodynamic model, 59, 365 pharmacokinetic model, 59, 360 Poincar´e map, 527 Poincar´e recurrence, 235, 240 Poincar´e recurrence theorem, 236 port-controlled Hamiltonian system, 38, 259 discrete-time, 75 positive equilibrium kinetic equation, 297 positive function, 30 discrete-time, 70 positive limit point, 17 discrete-time system, 65 time-delay discrete system, 105 time-delay system, 94 positive limit set, 17 discrete-time system, 65

time-delay discrete system, 105 time-delay system, 94 positive matrix, 8, 30 positive orbit, 17 time-delay discrete system, 105 time-delay system, 94 positive orthant, 8 positive stoichiometric compatibility class, 294 positive vector, 8 positively invariant time-delay discrete system, 105 positively invariant set, 17 discrete-time system, 65 time-delay system, 94 potassium ion model, 49 power balance equation, 245, 259 precompact set, 324 predator-prey model, 58 preload, 370 product, 282, 283 Q quadratic Lyapunov function, 41 discrete-time system, 79 quasistatic process, 250 quasistatic transformation, 224 R R-state reversibility, 228 R-state reversible dynamical system, 229 range space, 8 reactant, 282, 283

NNBook

September 1, 2009

604

INDEX

reaction network, 282, 283 reaction network rank, 293 reaction order, 285 reaction rate, 282, 283 realization of mass-action kinetics, 290 recipient-controlled compartmental system, 45 recoverable trajectory, 231 regional perfusion, 363 relative uncertainty structure, 158, 181 resetting law, 199 resetting times, 199 respiratory failure, 523 reversibility, 224 reversible, 282 reversible trajectory, 229 right maximally defined solution, 264 robust control, 425 robust stability, 157, 181 S Schur matrix, 71 second law of thermodynamics, 248 semi-Euler vector field, 269 semigroup property, 9 semistability, 3, 21 semistable, 23, 69, 226 semistable matrix, 33 semistable system, 23, 69 semistable under f , 344 set-point regulation, 427 settling-time function, 265 simple trap, 38 solution curve, 17 solution of a differential equation, 9 species, 282, 283

spectral abscissa, 8 spectral radius, 8 stabilizable, 407 stabilizable-nonnegative orthant feedback holdable, 408 Starling transcapillary refill, 518 state irrecoverability, 226, 254 state irreversibility, 226, 254 state recoverability, 228 state recoverable dynamical system, 231 state reversibility, 228 state reversible dynamical system, 229 statistical thermodynamics, 252 stoichiometric coefficient, 282, 283 stoichiometric compatibility class, 294 stoichiometric subspace, 293 storage function, 148 storage functional, 185, 193 strictly dissipative time-delay system, 185 discrete-time, 193 strictly ultrametric matrix, 524 stroke volume, 370, 517 strongly connected graph, 316 strongly zero-state observable hybrid system, 209 subspace identification, 554, 557 supply rate, 147 discrete-system, 167

supremum, 9 susceptible-infected model, 53 susceptible-infectedrecovered-susceptible model, 57 susceptible-infectedremoved model, 54 susceptible-infectedsusceptible model, 56 switched dynamical system, 533 systemic vascular resistance, 517 T temperature, 253 thermodynamics, 223 thyroxine model, 80, 176 time-reversal asymmetry, 226 time-reversal symmetry, 225 trajectory, 17 trap, 38 tree structure model, 541 U undirected graph, 255 unimolecular reaction, 285 unit disposition function, 363 universal statement, 8 upper right Dini derivative, 344 V vascular resistance, 517 vasodilator drug, 370 vasopressor drug, 370 venodilation, 376 volume-preserving map, 235

NNBook

September 1, 2009

605

INDEX

W weakly proper function, 274 weakly reversible reaction, 305 weighted adjacency matrix, 315 weighted directed tree,

528 weighted monotonicity, 116 discrete-time system, 130 Z Z-matrix, 30

zero deficiency, 303 zero-state observable, 159 discrete-time system, 168 zero-state observable hybrid system, 209 zeroth law of thermodynamics, 248