Quasilinearization and invariant imbedding: with applications to chemical engineering and adaptive control

QUASILINEARIZATION AND INVARIANT IMBEDDING With Applications to Chemical Engineering and Adaptive Control

E . Stanley Lee PHILLIPS PETROLEUM COMPANY BARTLESVILLE, OKLAHOMA KANSAS STATE UNIVERSITY MANHATTAN, KANSAS

1968

A C A D E M I C P R E S S New York and London

COPYRIGHT

8 1968, BY ACADEMIC PRESS INC.

ALL RIGHTS RESERVED. NO PART OF THIS BOOK MAY BE REPRODUCED IN ANY FORM, BY PHOTOSTAT, MICROFILM, OR ANY OTHER MEANS, WITHOUT WRITTEN PERMISSION FROM THE PUBLISHERS.

ACADEMIC PRESS INC. 11 1 Fifth Avenue, New York, New York 10003

United Kingdom Edition published by ACADEMIC PRESS INC. (LONDON) LTD. Berkeley Square House, London W.l

LIBRARY OF CONGRESS CATALOG CARDNUIVIBW: 68-18674

PRINTED IN THE UNITED STATES OF AMERICA

To Mayanne, Linda, and Margaret

PREFACE

When the complete initial conditions are given, modern digital computers are efficient tools for solving differential equations. But, unfortunately, many problems in engineering and physical sciences are two-point or multipoint nonlinear boundary-value problems, in which the conditions are not all given at one point. Problems of this type are most subtle and difficult and are not well suited for modern digital computers. This book presents a study on the use of two recently developed concepts for obtaining numerical solutions of boundary-value problems. Quasilinearization and invariant imbedding represent two completely different approaches to these problems. T h e invariant imbedding approach reformulates the original boundary-value problem into an initial value problem by introducing new variables or parameters ; while the quasilinearization technique represents an iterative approach combined with linear approximations. Certain problems can be treated more advantageously by quasilinearization, others by invariant imbedding. A combination of these two approaches also is used. Our aim is to produce various efficient algorithms which are suited for various types of boundary-value problems. This is a numerical study of boundary-value problems. Emphasis is placed upon computational instead of analytical aspects. Most of our discussions are concerned with the actual convergence rates and computational requirements. Various numerical experiments are performed and detailed computational procedures are given. No discussion will be given concerning the uniqueness and existence problems unless these topics are concerned directly with our results. T h e quasilinearization technique is introduced in Chapter 2. I n Chapter 3, this technique is applied to some boundary-value problems in ordinary differential equations. Since boundary-value problems are encountered in almost every branch of engineering and physical sciences, ix

X

PREFACE

these examples are necessarily restricted to the areas of interest to the author. Much interest has been shown in the literature on adaptive control. T h e problems in several areas in adaptive control can be treated as boundary-value problems. An important area in adaptive control is the identification or estimation problem. I n Chapter 4, this problem is treated as a two-point or a multipoint boundary-value problem by the quasilinearization technique. Another important area in adaptive control is optimization. T h e boundary-value difficulties severely limit the usefulness of the calculus of variations and the maximum principle in obtaining numerical solutions. I n Chapter 5 , the quasilinearization technique is shown to be a useful tool in overcoming these difficulties encountered in optimization. Another problem treated in Chapter 5 is the simultaneous optimization of parameters and control variables. I n Chapter 6, the invariant imbedding concept is introduced. I n Chapter 7, this concept is combined with quasilinearization to form some useful predictor-corrector formulas. Invariant imbedding also is used to avoid the numerical solution of linear algebraic equations in the quasilinearization procedure. Some interesting comparisons between the combined techniques and quasilinearization are obtained. I n Chapter 8, the estimation problem is treated by the invariant imbedding concept. This approach is compared with the quasilinearization approach treated in Chapter 4. A third area in adaptive control is the problem of stability. T h e dynamic equations used to study the stability of fixed bed chemical reactors are treated by quasilinearization in Chapter 9. Emphasis is placed upon the comparison between the present approach and those found in the literature. No detailed calculations are given for the study of the stability of fixed bed reactors. Except for the last few sections of Chapter 2, we have avoided all the theoretical aspects of the problems treated in this work. Those who wish to learn more about these theoretical aspects should consult the references listed at the end of the various chapters. For those who are interested mainly in obtaining numerical solutions, Sections 10 to 15 of Chapter 2 can be omitted during the first reading. This book is written in sufficient detail that it can be used as an introductory text on the subjects of quasilinearization and invariant imbedding, and every effort has been made to maintain an elementary level of mathematics throughout the book. T h e approach is formal and no attempt has been made to give a rigorous mathematical treatment. T h e numerical examples discussed in this book are of direct interest to

PREFACE

xi

chemical and control engineers. However, the basic principles illustrated by the various examples and the materials in Chapters 2 and 6 , where quasilinearization and invariant imbedding are introduced, should be useful to all scientists and engineers who are interested in obtaining numerical solutions of boundary-value problems in their particular fields. Except for the last chapter on parabolic partial differential equations, this work is primarily concerned with the numerical solution of boundary value problems in ordinary differential equations. Although the basic equations in invariant imbedding are partial differential equations, we have discussed only those problems whose invariant imbedding equations can be reduced to ordinary differential equations. T h e numerical aspects of partial differential equations are much more complex. We wish to treat these equations together with differential-difference and functional differential equations in another volume. Many important topics related to invariant imbedding, quasilinearization, and boundary value problems are not included. Some of these topics are the use of invariant imbedding in the analytical formulation of physical problems such as neutron transport and wave propagation, and analytical techniques for treating boundary value problems such as the method of Blasius. Furthermore, we have restricted our discussion to deterministic processes. Although the estimation problems are essentially stochastic in nature, we have avoided any discussion on the statistics of these problems. Most of the computational work has been done at Phillips Petroleum Company. I am grateful for its support of my research work in the fields of applied mathematics and optimization theory. I also wish to thank Dr. Richard Bellman of the University of Southern California and Dr. Robert Kalaba of the RAND Corporation for their encouragement. Their books and papers furnished a large fraction of the source material for this book. Finally I wish to express my gratitude to my wife who not only provides a constant source of inspiration and encouragement but also typed the complete manuscript-a difficult task considering the fact that she has never had any training in typing. E. S . LEE Manhattan, Kansas November, 1967

CONTENTS

PREFACE.

...............................

ix

Chapter 1 . Introductory Concepts 1 . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Quasilinearization . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Invariant Imbedding ........................ 4. Invariant Imbedding versus the Classical Approach . . . . . . . . . . . 5 . Numerical Solution of Ordinary Differential Equations . . . . . . . . . 6 Numerical Solution Terminologies . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

.

. . .

1 2 2 3 4 7 8

Chapter 2. Quasilinearization 1 . Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Nonlinear Boundary-Value Problems . . . . . . . . . . . . . . . . . 3 Linear Boundary-Value Problems . . . . . . . . . . . . . . . . . . 4 Finite-Difference Method for Linear Differential Equations . . . . . . . 5 . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. Newton-Raphson Method . . . . . . . . . . . . . . . . . . . . . . . 7. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8. Quasilinearization . . . . . . . . . . . . . . . . . . . . . . . . . . 9. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10. Existence and Convergence . . . . . . . . . . . . . . . . . . . . . . 11 . Existence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. Convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Maximum Operation and Differential Inequalities . . . . . . . . . . . 14. Construction of a Monotone Sequence . . . . . . . . . . . . . . . . 15 . Approximation inpolicy Space and DynamicProgramming 16. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17. Systems of Differential Equations . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. .

. . .

.

.

.

........

xiii

.

9 10 11 14 16 17 20 21 23 24 25 26 28 31 32 34 35 38

XiV

CONTENTS

Chapter 3. Ordinary Differential Equations 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. A Second-Order Nonlinear Differential Equation . . . . . . . . . . . . . 3. Recurrence Relation . . . . . . . . . . . . . . . . . . . . . . . . . 4. Computational Procedure . . . . . . . . . . . . . . . . . . . . . . . 5 . Numerical Results ......................... 6. Stability Problem in Numerical Solution-The Fixed Bed Reactor . . . . . 7. Finite-Difference Method . . . . . . . . . . . . . . . . . . . . . . . 8. Systems of Algebraic Equations Involving Tridiagonal Matrices . . . . . . 9. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 10. Stability Problem with High Peclet Number . . . . . . . . . . . . . . . 11. Adiabatic Tubular Reactor with Axial Mixing . . . . . . . . . . . . . . 12. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 13. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. Unstable Initial-Value Problems . . . . . . . . . . . . . . . . . . . . 15. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16. Systems of Differential Equations . . . . . . . . . . . . . . . . . . . 17. Computational Considerations . . . . . . . . . . . . . . . . . . . . . 18. Simultaneous Solution of Different Iterations . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

40 41 42 43 46 51 53 56 58 61 62 67 71 72 73 73 78 79 81

Chapter 4 . Parameter Estimation 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Parameter Estimation and the “Black Box” Problem . . . . . . . . . . . 3. Parameter Estimation and the Experimental Determination of Physical Constants . . . . . . . . . . . . . . . . . . . . . . . . . . 4. A Multipoint Boundary-Value Problem . . . . . . . . . . . . . . . . . 5. The Least Squares Approach . . . . . . . . . . . . . . . . . . . . . 6. Computational Procedure for a Simpler Problem . . . . . . . . . . . . . 7. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 8. Nonlinear Boundary Condition . . . . . . . . . . . . . . . . . . . . 9. Random Search Technique . . . . . . . . . . . . . . . . . . . . . . 10. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 11. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12. Parameter Up-dating . . . . . . . . . . . . . . . . . . . . . . . . . 13. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14. Estimation of Chemical Reaction Rate Constants . . . . . . . . . . . . . 15. Differential Equations with Variable Coefficients . . . . . . . . . . . . . 16. An Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17. Ill-Conditioned Systems . . . . . . . . . . . . . . . . . . . . . . . 18. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 19. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20. An Empirical Approximation . . . . . . . . . . . . . . . . . . . . . 21. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 22. A Second Approximation . . . . . . . . . . . . . . . . . . . . . . . 23. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . .

83 83 84 85 86 86 90 92 95 97 99 100 102 103 105 106 109 111 115 116 118 119 120

xv

CONTENTS

. .

24 Differential Approximation . . . . . . . . . . . . . . . . . . . . . . 25. A Second Formulation . . . . . . . . . . . . . . . . . . . . . . . . 26 Computational Aspects . . . . . . . . . . . . . . . . . . . . . . . . 27.Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

122 123 125 126 126

Chapter 5. Optimization 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Optimum Temperature Profiles in Tubular Reactors . . . . . . . . . . 3. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Back and Forth Integration . . . . . . . . . . . . . . . . . . . . . . . 6. Two Consecutive Gaseous Reactions . . . . . . . . . . . . . . . . . 7 Optimum Pressure Profile in Tubular Reactor . . . . . . . . . . . . . 8. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 9. Optimum Temperature Profile with Pressure as Parameter . . . . . . . . 10. Numerical Results and Procedures . . . . . . . . . . . . . . . . . . 11. Calculus of Variations with Control Variable Inequality Constraint . . . . 12. Calculus of Variations with Pressure Drop in the Reactor . . . . . . . . 13. Pontryagin’s Maximum Principle . . . . . . . . . . . . . . . . . . 14. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15. Optimum Feed Conditions . . . . . . . . . . . . . . . . . . . . . . 16. Partial Derivative Evaluation . . . . . . . . . . . . . . . . . . . . . 17.Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. .

. .

.

. . . .

.

129 130 135 144 146 147 149 151 153 160 170 171 174 175 175 176 176 177

Chapter 6. Invariant Imbedding 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. The Invariant Imbedding Approach . . . . . . . . . . . . . . . . . . 3. AnExample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. The Missing Final Condition . . . . . . . . . . . . . . . . . . . . . 5 . Determination of x and y in Terms of r and s . . . . . . . . . . . . . . 6. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .; . . . . . 7. Alternate Formulations-I 8. Linear and Nonlinear Systems . . . . . . . . . . . . . . . . . . . . . 9. The Riccati Equation . . . . . . . . . . . . . . . . . . . . . . . . 10. Alternate Formulations-I1 . . . . . . . . . . . . . . . . . . . . . . 11. The Reflection and Transmission Functions . . . . . . . . . . . . . . . 12. Systems of Differential Equations . . . . . . . . . . . . . . . . . . . 13. Large Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . 14. Computational Considerations . . . . . . . . . . . . . . . . . . . . . 15. Dynamic Programming . . . . . . . . . . . . . . . . . . . . . . . . 16 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

179 180 182 188 189 191 192 195 196 197 201 203 205 206 208 211 21 3

xvi

CONTENTS

.

Chapter 7

Quasilinearization and Invariant Imbedding

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. The Predictor-Corrector Formula . . . . . . . . . . . . . . . . . . 3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. Linear Boundary-Value Problems . . . . . . . . . . . . . . . . . . 5. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 6. Optimum Temperature Profiles in Tubular Reactors . . . . . . . . . . 7. Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . 8. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9. Dynamic Programming and Quasilinearization-I . . . . . . . . . . . 10. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11. Linear Differential Equations . . . . . . . . . . . . . . . . . . . . . 12. Dynamic Programming and Quasilinearization-I1 . . . . . . . . . . . 13. Further Reduction in Dimensionality . . . . . . . . . . . . . . . . . 14. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

. . . .

. .

217 218 223 224 226 229 232 235 236 238 238 239 243 244 244

Chapter 8 . Invariant Imbedding. Nonlinear Filtering. and the

Estimation of Variables and Parameters

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 2. An Estimation Problem . . . . . . . . . . . . . . . . . . . . . . 3. Sequential and Nonsequential Estimates . . . . . . . . . . . . . . 4. The Invariant Imbedding Approach . . . . . . . . . . . . . . . . 5. The Optimal Estimates . . . . . . . . . . . . . . . . . . . . . . 6. Equation for the Weighting Function . . . . . . . . . . . . . . . . 7. A Numerical Example . . . . . . . . . . . . . . . . . . . . . . 8. Systems of Differential Equations . . . . . . . . . . . . . . . . . 9. Estimation of State and Parameter-An Example . . . . . . . . . . . 10. A More General Criterion . . . . . . . . . . . . . . . . . . . . . 11. An Estimation Problem with Observational Noise and Disturbance Input 12. The Optimal Estimate-A Two-Point Boundary-Value Problem . . . . 13. Invariant Imbedding ...................... 14. A Numerical Example . . . . . . . . . . . . . . . . . . . . . . 15. Systems of Equations with Observational Noises and Disturbance Inputs 16. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . ..

.. . .

246 247 248 249 250 252 254 257 261 265 266 267 268 270 272 276 276

............ . . . . . . . . . . . . . . . . . . . . . . . . . . ............ . . . . . . . . . . . . .

278 279 280 283 283

.. .. .. ..

. . . .

. . . .

Chapter 9. Parabolic Partial Differential Equations-

Fixed Bed Reactors with Axial Mixing

1. 2. 3. 4 5.

.

Introduction . . . . . . . . . . . . . . . . Isothermal Reactor with Axial Mixing . . . . An Implicit Difference Approximation . . . . Computational Procedure . . . . . . . . . . . Numerical Results-Isothermal Reactor . . . .

xvii

CONTENTS 6. Adiabatic Reactor with Axial Mixing . . . . . . . . . . . . . . . . . . 7. Numerical Results-Adiabatic Reactor . . . . . . . . . . . . . . . . . 8. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Influence of the Packing Particles . . . . . . . . . . . . . . . . . . . 10. The Linearized Equations . . . . . . . . . . . . . . . . . . . . . . 11. The Difference Equations . . . . . . . . . . . . . . . . . . . . . . 12. Computational Procedure-Fixed Bed Reactor . . . . . . . . . . . . . . 13. Numerical Results-Fixed Bed Reactor . . . . . . . . . . . . . . . . . 14 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.

.

285 289 292 292 294 296 300 301 304 305

Appendix I . Variational Problems with Parameters 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Variational Equations with Parameters . . . . . . . . . . . . . . . . 3. Simpler End Conditions . . . . . . . . . . . . . . . . . . . . . . . 4. Calculus of Variations with Control Variable Inequality Constraint . . . . 5 . Pontryagin’s Maximum Principle . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . .

306 306 309 310 312 313

Appendix I1. The Functional Gradient Technique 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. The Recurrence Relations . . . . . . . . . . . . . . . . . . . . . . 3. Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . . 4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

315 315 320 321 322

.............................

323

AUTHOR INDEX

SUBJECT INDEX.

............................

326

Chapter

1

INTRODUCTORY CONCEPTS

1. Introduction

In engineering and physical sciences there occur many two-point or multipoint boundary-value problems. Since these problems usually are nonlinear, they are accompanied by various analytical and numerical difficulties. Analytically, there is no general proof for the existence and uniqueness of the solutions. Numerically, we possess no convenient technique for obtaining the numerical solutions on modern digital computers. These numerical difficulties are caused by the fact that not all the conditions are given at one point. T o obtain the missing condition, a trial-and-error procedure is generally used. Not only does this procedure have a relatively slow convergence rate; but also, owing to its trialand-error nature, it is not suited to modern digital computers. Furthermore, for a large number of problems, the starting or guessed missing condition must be very close to the correct and yet unknown condition before the procedure will converge. Quasilinearization and invariant imbedding are two useful techniques for obtaining numerical solutions for this type of problem. These two techniques present two systematic approaches to the boundary-value problems. It should be emphasized that quasilinearization and invariant imbedding are two completely different concepts. Quasilinearization is a numerical technique while invariant imbedding represents a completely different formulation of the original problem. The purpose of this chapter is twofold. First the basic concepts used throughout the book will be introduced. Some of these introductions are necessarily abstract. More detailed explanations and applications appear in later chapters. Second, some of the formulas for the numerical integration of ordinary differential equations of the initial value type will be reviewed briefly. Derivations of these formulas will not be given. 1

1.

2

INTRODUCTORY CONCEPTS

2. Quasilinearization

I n the quasilinearization technique, instead of being solved directly, the nonlinear differential equation is solved recursively by a series of linear differential equations. The main advantage of this technique is that if the procedure converges, it converges quadratically to the solution of the original equation. Quadratic convergence means that the error in the (n 1)st iteration tends to be proportional to the square of the error in the nth iteration. The advantage of quadratic convergence, of course, lies in the rapidity of convergence. The linear equation is obtained by using the first and second terms in the Taylor’s series expansion of the original nonlinear equation. This technique is a generalized Newton-Raphson formula for functional equations. Since linear differential equations of the boundary-value type with variable coefficients can be solved fairly routinely on modern computers by the superposition principle, an efficient recursive formula has been developed. However, this technique also has its difficulties. The main difficulty arises from the fact that in using the superposition principle, a set of algebraic equations must be solved. Thus, the ill-conditioning phenomenon in solving a set of linear algebraic equations can make the superposition principle useless.

+

3. Invariant Imbedding

T o illustrate the invariant imbedding approach, let us consider the simple second-order differential equation d2x _ -0 dta

with boundary conditions

< < .

(1)

x(0) = xo

with 0 t tf Equations (1) and (2) form a two-point boundary-value problem. In order to integrate Eq. (1) numerically, we must know the missing initial condition, which is the slope of x at t = 0. It is not easy to obtain this missing initial condition for most applicational problems. Invariant imbedding involves a completely different approach to formulating the problem. Instead of only considering a single problem

4.

INVARIANT IMBEDDING VERSUS THE CLASSICAL APPROACH

3

with duration tf , the invariant imbedding approach is to consider a family of problems, with duration of the process ranging from zero to the value of tf . Then, these problems are imbedded to obtain the particular original problem. Since if the process represented by Eq. (1) had a zero duration we would know the missing initial condition, the original two-point boundary-value problem becomes an initial-value problem in the invariant imbedding formulation. Since we are solving a family of problems instead of one original problem, more computation may be needed to obtain the solution. This is the price we have to pay for avoiding the two-point boundary-value difficulty. However, for some problems, the invariant imbedding approach has been found to be much superior to the usual approach. Furthermore, since generally we are not computing the solutions of the whole family of problems one by one, the computational requirements are not as formidable as they seem. Frequently we are interested in investigating the behavior of the solution of the neighboring processes of the original problem. Thus, the need of solving a family of problems in the invariant imbedding approach may constitute an advantage for certain applicational problems. Furthermore, this generality of solutions overcomes, at least partly, one serious criticism of numerical solution: namely, the lack of generality. Invariant imbedding is only a concept; it is not a technique or method. This concept can be applied to a variety of different physical problems. Because of its completely different approach, it frequently gives some different insights to the same physical problem which has been treated by the usual or classical method. 4. Invariant Imbedding versus the Classical Approach

For the ease of reference, the techniques and concepts usually used in treating a boundary-value problem will be referred to as the usual or classical approach. This is in contrast to the invariant imbedding approach, which requires a completely different concept. We shall use the term “classical approach’’ loosely throughout the book. I n general, it means that a problem is formulated and solved as a boundary-value problem, not in terms of the invariant imbedding concept. Thus, the quasilinearization technique can be considered as a classical approach. Note the difference between the computational philosophies of quasilinearization and invariant imbedding. Quasilinearization represents an iterative computational procedure while invariant imbedding solves the original problem by expanding it into a family of problems.

4

1.

INTRODUCTORY CONCEPTS

5. Numerical Solution of Ordinary Differential Equations Since most of this work will be based on numerical methods of obtaining solutions of ordinary differential equations of the initial-value type, it is helpful to review briefly the general methods. Consider the first-order ordinary differential equation dx _ - x'

dt

=f(x, t )

with the initial condition

In numerical approaches the value of the dependent variable x is calculated at discrete values of the independent variable t. I n other words, x is calculated at t, , t, ,..., with tk+, - t k = d t , where d t is called the integration step, interval, or grid spacing; and t , , t, ,... are called the grid points. Generally the size of d t is controlled by the accuracy desired in the numerical results. Computer limitations and the stability problem involved in solving a particular problem also control the value of A t . With the initial value xo at t = 0 known, Eq. (1) can be integrated and the values of x at t, , t, ,... can be obtained. There are various methods for obtaining these values of x;only some of those most frequently used will be mentioned [l-81. These can be separated into single-step and multiple-step methods.

A. SINGLE-STEP METHODS The formulas of this type of method can be represented by

with to = 0. This method starts with the known value of xo and uses A t andf(x(t,), to) to calculate x(tl). Once x(tl) is obtained, the process can be repeated by using d t andf(x(t,), t,) to calculate x(t,). This process is continued until t = tf , where tt is the final value of the independent variable t. Since the calculation is performed from one point to the next in a direct and orderly sequence, these methods are also known as marching techniques. Among the various single-step integration formulas the Runge-Kutta scheme is perhaps the best known and most frequently used. For Eq. (I),

5.

NUMERICAL SOLUTION OF ORDINARY DIFFERENTIAL EQUATIONS

5

the fourth-order Runge-Kutta formula, which is known simply as the Runge-Kutta formula, is m1 = M t k ) , t k )

At

+ +ml , t k + + A t ) A t + &n2 , t k + + A t ) A t + m3 , t k + A t ) A t + Q(m, + 2m2 + 2m, + m4) + O ( 4

m2 = f ( x ( t k ) m3 = f(x(tk)

(4)

m4 = f ( ~ ( t k ) ~ ( t k , , ) = x(tk)

with k = 0, 1, 2, ... . T h e last term in the last equation indicates that the local truncation error for this formula is in the order of O ( A 6 ) . This term is not used directly in the application of the formula, but only as an indicator of the accuracy of the results. Starting at x = x(to) = xo, t = t o , and the specified A t , the values of m, , m 2 , m 3 , and m4 are calculated successively. T h e value of x(tk+lkl) is then obtained from the last equation. Because of its low truncation error, the Runge-Kutta forrnula will be used in most of the calculations of this work. For some types of single-step formulas, the unknown x(tk+l) also can be present implicitly on the right-hand side of Eq. (3). A good example would be the modified Euler's method. Obviously, iteration must be used when x(t,+,) is present implicitly in the function h. One of the difficulties in using the single-step methods is that they are unstable under certain conditions. That is to say, if an error is committed in calculating x(tk+l), the error will propagate with increasing magnitude through the remainder of the calculation and thus render an unstable solution.

B. MULTIPLE-STEP METHODS T h e formulas of this type of method for Eq. (1) can be roughly represented by X(tk,l)

-4tk-T)

= h(A t , f ( X ( t k ) , tk),f(X(tk-l),

tk-l>,"',f(X(tk-IE>,

tk-n))

r , n = positive integers

(5) In order to evaluate ~ ( t ~ +the ~ )values , of x ( t k ) , x(tk-l),..., x ( t k J , and x(tkPr) must be known. Thus, it is not possible to calculate x(tk+l) directly from the initial value xo. T o start the calculation, the points x(tkVl),x ( tkP2),... must be obtained by another integration method. T o increase the accuracy, generally two integration formulas are used in the multiple-step methods. T h e first formula, which is known as the open-end integration formula and is in the same form as Eq. ( 5 ) , is used to predict the approximate value of x(tk+l). Then the second

1.

6

INTRODUCTORY CONCEPTS

formula, which is known as the closed-end formula, is used to generate a more accurate ~ ( t , + ~This ) . closed-end formula may be iterated to obtain as accurate an answer as desired. The closed-end formula is in ~ ) on the same form as Eq. ( 5 ) except that the unknown ~ ( t ~is+present the right-hand side also. These two formulas form a predictor-corrector scheme which is a powerful numerical tool. Other predictor-corrector schemes will be formulated by the use of quasilinearization and invariant imbedding in later chapters. Milne’s method is probably the best known multiple-step integration formula. The predictor for this method is X(tk,,)

= x(tt-3)

+ 4Wf(x(t,),

t k ) -f(X(tk-l),

tk-1)

+2f(X(tk--2),

t7C-211

(6)

and the corrector is X(h+,)

= X(tk-1)

+ S4f(x(tk+l),

+O ( 4

bfl)

+ 4f(X(?d,

tk)

-tf(4k-l),

tk-J

+ W5)

(7) T o begin the integration, the starting values at the three grid points t , , tk--l , and tk--2can be obtained by a single-step integration formula or by using Taylor’s series. Note that Eq. (7) is Simpson’s rule. In addition to the stability problem, which is common to both the single-step and multiple-step methods, the convergence rate of the iterating corrector equation must be considered. The stability and the convergence rate problems are discussed in various numerical analysis texts listed at the end of the chapter. The single-step methods have a number of advantages in terms of the use of digital computers. First, in using the multiple-step methods the starting values must be calculated by some other methods, but no such calculations or predictions of the starting values are necessary for the single-step methods. Second, during the integration process, several different values of the integration step d t may be necessary in solving the same equations. It is not easy to reduce the integration step A t for the multiple-step methods as the integration proceeds. Some kind of interpolation formula must be used to reduce this step size.

C. SYSTEMS OF EQUATIONS The above equations can be generalized to the treatment of a set of simultaneous equations. This generalization involves relatively little that is new. T o illustrate this, consider the two simultaneous differential equations

6.

NUMERICAL SOLUTION TERMINOLOGIES

7

with initial conditions x(0)

= xo

y ( 0 ) =yo

(9)

6. Numerical Solution Terminologies

The present work is primarily concerned with obtaining numerical solutions of differential equations by digital computer. Owing to the discrete nature of the computer, the solutions are always obtained at discrete points of the independent variable. These discrete points have been called the grid points in the previous section. Thus, by numerical solution we mean a numerical table in which the values of the dependent variable are tabulated at the grid points of the independent variable. For convenience, some of the terminologies frequently used in connection with obtaining numerical solutions will be summarized in the rest of this section.

A. ACCURACY OF

THE

SOLUTION

Various errors are involved in obtaining numerical solutions. For example, errors are introduced when we.change the original continuous problem into a discrete one. Errors also are introduced by the numerical

8

1.

INTRODUCTORY CONCEPTS

integration formulas discussed in the preceding section. I n the present work, no effort will be made to estimate these errors. However, in order to describe the numerical results, the word accuracy will be used frequently. T h e accuracy at nth iteration for x, , e x , will be defined as foIlows:

I Xn+l(t,)

- Xn(t7c)i


f’(uo>+

.*’

(2)

Equation (2) is obtained by the use of Taylor series with the second- and higher-order terms neglected. A second approximation to r can now be obtained by solving the linear equation for u :

f ( 4 + (U - U o ) f ’ ( U o ) = 0 (3) Call this approximation u1 . By the use of u l ,a third approximation u2 can be obtained by solving the following equation for u2 :

f ( 4 + (% - U d f ’ ( U 1 )

=

0

(4)

This process is continued and the general recurrence relation is

f(4+ (%+I

- %Jf’(Un)

=

0

(5)

2. QUASILINEARIZATION

18

where u, is always known and is obtained from the previous calculation and u , + ~is the unknown. It should be noted that Eq. ( 5 ) is a linear equation in the unknown Geometrically, Eq. ( 5 ) represents a tangent line to the functionf(u) at u, , and u , + ~represents the intersection of this tangent line with the u axis, This is shown in Fig. 2.1. If we solve for u % + ~Eq. , ( 5 ) becomes

We see that Eq. (6) is the familiar Newton-Raphson equation. Now we wish to examine some of the important properties of the Newton-Raphson method. First, observe that Eq. (6) does not hold if f'(un) = 0. Second, from Fig. 2.1 it is clear that

< u1 < ue * * * < r

(7) The property expressed by Eq. (7) is known as monotone convergence. The values of the sequence {urn}*increase monotonically to the root r . This is an important property computationally and is especially suited for modern computers, because it provides an upper or lower bound for the convergent interval and ensures automatic improvement of the initial approximation at each iteration. This monotone increasing property follows directly from the inequalities. 110

f(%) > 0

fY%) < 0

(8)

If the initial approximation uo is such that f(uo) < 0 as shown in Fig. 2.2, then the initial approximation uo no longer has the monotone

FIG.2.2.

Newton-Raphson method withf(u,)

< 0.

* The symbol {un} will be used throughout this book to denote a sequence of values ,

u1 up

,....

6.

NEWTON-RAPHSON METHOD

19

property. However, the sequence {un}, for n = 1, 2, ..., is still monotone convergent. By simple geometric constructions, it can be shown that this monotone convergence property also exists if the functions f’(u) > 0 and f(u) are concave functions. T h e other two cases-namely thatf(u) is convex and f’(u) > 0, and that f ( u ) is concave and f’(u) < 0-produce a sequence u1 , u2 ,..., which has a monotone decreasing property. If the functionf(u) is not a monotone increasing or monotone decreasing function, or if f ( u ) is not a strictly convex or concave function, this monotone convergence property may not exist. I n fact, the calculation may never converge to the desired root. Whether Eq. (6) converges or not depends on the initial estimate uo and the particular shape of f ( u ) . Fig. 2.3 illustrates one case in which the functionf(u) has turning points and inflections and in which Eq. ( 6 ) may not converge to the root r .

---FIG. 2.3. A function with points of inflection.

For computational purposes, the rate of convergence is another important property. T h e Newton-Raphson method is a second-order process. Most of the other methods generally used for solving algebraic equations numerically are first-order processes. This second-order process is also known as quadratic convergence. If the iteration converges, the error in (n 1)st iteration tends to be proportional to the square of the error in the nth iteration, or

+

u , , ~ - r a~ (u, - r)2

(9)

whereas for first-order iteration processes the two successive errors generally tend to be in a constant ratio. T o derive Eq. (9), rewrite Eq. ( 6 ) :

2. QUASILINEARIZATION

20 sincef(r)

=

0. By the use of the mean value theorem, we obtain

where v lies between r and u, . Substituting Eq. (1 I) into Eq. (lo), we obtain the following expression:

Iff"(.) andf'(u,) exist and iff'(u,) # 0, Eq. (12) reduces to Eq. (9). Computationally, this quadratic convergence means that after a large number of iterations, the number of correct digits for the root Y is approximately doubled for each iteration. Suppose u, has an accuracy of 0.1; then the quadratic convergence means that u,+~ has an accuracy of 0.01 for large n. Tt follows that as u, approaches 7 , there is an enormous acceleration in the convergence rate. 7. Discussion

From the computational standpoint, the Newton-Raphson technique has two important properties, namely monotone convergence and quadratic convergence. Whether Eq. (6.6) possesses the monotone convergence property depends on the property of the function f ( u ) . The monotone convergence property exists for the Newton-Raphson formula only if f ( u ) is a monotone decreasing or a monotone increasing function and, in addition, the function must be strictly convex or concave. However, in general, the Newton-Raphson formula always has the quadratic convergence property provided that it converges. This quadratic convergence is a consequence of using the first and second terms in the Taylor series expansion. It is important to note that the Newton-Raphson equation (6.6) is always linear even if the original function f ( u ) is nonlinear. We have pointed out before that linear boundary-value problems can be solved fairly routinely on modern digital computers. It is a natural question to ask whether we can generalize the Newton-Raphson method to differential equations. If so, does the resulting linear equation possess the important property, namely quadratic convergence ? As we shall see in the next few sections, the answers to all these questions are positive. I n fact the generalized Newton-Raphson technique for functional equations, which is known as the quasilinearization technique, is almost equivalent, at least abstractly, to the NewtonRaphson method for algebraic equations.

8. QUASILINEARIZATION

21

8. Quasilinearization

For concrete illustration, let us consider the nonlinear second-order differential equation:

-d2x _dt2

with the boundary conditions

The functionf now is a function of the function x(t). Choose a reasonable initial approximation of the function x(t); call it xo(t). Notice that we are now choosing a function, not just a single value as in Section 6. This approximate function can be obtained in a variety of ways. For an engineering problem, this approximation can be obtained from the physical situation and by exercising engineering judgment. For problems where convergence requires better initial approximations, this approximation can be obtained by some mathematical devices such as the invariant imbedding technique. However, for a number of problems, a very rough initial approximation is enough for the procedure to converge. I n this case, any intelligent guess can be used to obtain x,(t). T h e most obvious t tf . I n other words, we assume a one would be x,(t) = c l , for 0 constant function for xo(t). As we shall see later, the given boundary value will frequently be used as the initial approximation for the function. T h e function f can now be expanded around the function xo(t)by the use of the Taylor series

<

=

J(xn

(8)

2,)

+

2,) - J ( x n G)X, JAxn > ~ n ) ( ~ n + l2,) (9) Using Eqs. (8) and (9), the solution of (7) can be represented by Eq. (1 1.8). Introducing the Lagrange multiplier A, the problem becomes the maximization of

~ ( t= ) f (xn

J

7

= + ( ~ l ( t f )x*~ t f ) , * - *Xvm ( t r > ) -

J”

tf 0

f ( z )dt

(10)

over all z ( t ) ,satisfying Eqs. (2)-(4). Using Eq. (11.8), Eq. (10) becomes

where z,+,(t) are the unknown control variables after n iterations. If we consider X as a known parameter, the maximum value of Eq. (11) depends only on c1 , c2 ,..., c, , and tj . Furthermore, by examining Eqs. (11.5), (11.7), and (8), it can be seen that c is independent of the control variables zn+l(t). Thus, we wish to imbed the original problem with particular values of cl, c2 ,..., c, and duration tf within a family of processes in which cl, c2 ,..., c, and t j are parameters. Notice that if the explicit solution, Eq. (1 1.8), were not used, the maximum value of Eq. (10) would depend on c l , c2 ,..., cM and tf . We have reduced the number of variables from M to m. If m is equal to one or two, a feasible computational procedure has been obtained. Following the approach used in Chapter 6, we define g(c,

i

the maximum value of J where the process begins

> ~2 >.*.>

cm

2

a) = at t

=

u with starting state c1

,..., c,

.

1

Since the process is nonstationary, we have fixed the final time tf . A family of processes with different starting points a will be considered. T h e new maximization problem is

242

7.

QUASILINEARIZATION A N D INVARIANT IMBEDDING

T h e maximization is excuted by choosing the proper values of z over the interval [a, tf]. Applying the principle of optimality, we obtain the desired recurrence relation

T h e terms under the integral sign may be approximated by

M

cm

+ c hmj(a)Pj(a) j=l

a

+A)]

(16)

T o obtain the final condition for Eq. (16), observe that if the process had zero duration or a = tf , then the maximum value of Eq. (11) would be equal to zero. Thus g(c, > cz ~

. * cm * ~

7

tj) = 0

(17)

Notice that we have assumed that the duration of the process tf is divided into small intervals of A width. Let tf = AN, then a = 0, d, 24, ..., N d . Thus, Eq. (16) can be solved in a backward recursive fashion starting with the known final condition, Eq. (17), at a = tf . T h e computational procedure can now be summarized as follows: (a) Estimate a reasonable control policy ~ ~ = ~satisfying ( t ) , Eqs. (4) and ( 5 ) .

13.

FURTHER REDUCTION I N DIMENSIONALITY

243

(b) Calculate xn=,(t)from Eqs. (2) and (3), using the newly obtained values of z,=,(t). (c) Obtain z,,,(t) by maximizing Eq. (1l), using the newly obtained values of z,=,(t) and ~ , = ~ ( tT) h. e values of z,,,(t) must satisfy Eq. (4)(d) Return to (b) with n = 1. Equation (11) can be maximized by using the recurrence relation, Eq. (16), remembering that the solution of the linearized equation, Eq. (7), has been used in obtaining this recurrence relation.

13. Further Reduction in Dimensionality

Owing to the limited rapid-access memory of current computers, the above algorithms cannot be used if m is larger than three. However, for a large number of problems, a further reduction of the dimensionality can be obtained. Consider the problem of maximizing the function

Let us introduce a new state variable, xM+,(t), defined by

Differentiating Eq. (2) with respect to t , we have

T h e initial condition is X M + m = H(x(0))

(4)

If we consider Eqs. (12.2) and (3) as the system of differential equations, the objective function, Eq. (12. l), becomes a function of one variable

4 = XM+l(tf)

(5)

If the algorithms obtained in the previous section are used, a problem with a dimensionality of one is obtained. This is a significant reduction in terms of computational requirements.

244

7.

QUASILINEARIZATION AND INVARIANT IMBEDDING

14. Discussion

T h e above procedure can be generalized easily to problems with more general objective function. For example, the problem of maximizing the integral

J

=

/”f(x, 0 z) dt

(1)

can be treated by the above procedure if we introduce a new state variable, ~ ~ + ~defined ( t ) , by

with initial condition %4+1(0) = 0

(3)

T h e problem now becomes the maximization of ~ ~ + ~A( variety t ~ ) .of other forms of objective functions also can be treated by the algorithms obtained in the previous section. More discussion can be found in the references listed at the end of the chapter [ll-131. Let us consider briefly how Eq. (12.16) may be solved. I n order to obtain the values of c and K(a), the homogeneous differential equation, Eq. (1 1.5), must be solved first. Since Q ( t ) does not contain the unknowns x , + ~and z,+~ , Eq. (1 1.5) can be solved easily. Thus, in actual computations c and K(a) in Eq. (12.16) can be considered known. T h e functions f and p contain the unknown control variables Z ~ + ~ ( U T ) . h e problem is to find Z,+~(U) so that the expression inside the square bracket on the right-hand side of Eq. (12.16) is maximized. Since dynamic programming is especially suited for discrete processes, the present approach in the reduction of dimensionality appears to be a promising tool for solving stagewise processes. REFERENCES 1. Bellman, R., Kagiwada, H. H., and Kalaba, R., Numerical studies of a two-point nonlinear boundary-value problem using dynamic programming, invariant imbedding, and quasilinearization. RM-4069-PR. RAND Corp., Santa Monica, California, March, 1964. 2. Bellman, R., and Kalaba, R., Dynamic programming, invariant imbedding and quasilinearization: Comparisons and interconnections. RM-4038-PR. RAND Corp., Santa Monica, California, March, 1964; see also Bellman and Kalaba in “Computing Methods in Optimization Problems.” Balakrishnan, A. V., and Neustadt, L. W., eds., Academic Press, New York, 1964.

REFERENCES

245

3. Bellman, R., and Kalaba, R., “Quasilinearization and Nonlinear Boundary-Value Problems.” American Elsevier, New York, 1965. 4. Bellman, R., and Dreyfus, S., “Applied Dynamic Programming.” Princeton Univ. Press, Princeton, New Jersey, 1962. 5. Kalaba, R., On some communication network problems. “Combinatorial Analysis.” Am. Math. SOC.,Providence, Rhode Island, 1960. 6. Kalaba, R., Graph theory and automatic control, in “Applied Combinatorial Mathematics” (E. F. Beckenbach, ed.). Wiley, New York, 1964. 7. Bellman, R., “Introduction to Matrix Analysis.” McGraw-Hill, New York, 1960. 8. Bellman, R., “Stability Theory of Differential Equations.” McGraw-Hill, New York,

1953. 9. Bellman, R., Some new techniques in the dynamic programming solution of variational problems, Quart. Appl. Math. 16, 295 (1958). 10. Bellman, R., and Kalaba, R., Reduction of dimensionality, dynamic programming, and control processes, J . Basic Eng. 83, 82 (1961). 11. Rozonoer, L. I., The maximum principle of L. S. Pontryagin in optimal system theory, Automatika Telemekhhanika 20, 1320, 1441, 1561 (1959), [English transl. Automation Remote Control 20, 1288, 1405, 1517 (1960)l. 12. Katz, S., Best operating point for staged systems, Ind. Eng. Chem. Fundamentals 1, 226 (1962). 13. Fan, L. T., “The Continuous Maximum Principle.” Wiley, New York, 1966.

Chapter 8

INVARIANT IMBEDDING, NONLINEAR FILTERING, AND THE ESTIMATION OF VARIABLES AND PARAMETERS

1. Introduction

The invariant imbedding concept has been applied to various twopoint boundary-value problems in the two previous chapters. This concept will be used to derive some useful results in nonlinear filtering theory in this chapter. Since Wiener’s pioneering work [l] on the theory of optimal filtering and prediction, also known as the Wiener-Kolomogorov theory, many extensions and new developments hwe been made in this field. Among them, the works of Bode and Shannon [2], Pugachev [3], Kalman and Bucy [4, 51, Ho [6], Cox [7], and Bryson and Frazier [8] may be cited. For detailed treatment of Wiener’s theory refer to Levinson [9], Davenport and Root [lo], and Lee [ll]. The work of Kalman and Bucy is concerned with the estimation of state variables for linear systems. Later Cox treated the estimation problem in a formal fashion by dynamic programming. Bryson and Frazier treated a nonlinear version of this problem. Lee [12] and Deutsch [16] discussed this linear prediction problem in detail. The problem treated in this chapter is essentially an extension of this well-known linear problem. In the literature, most of the works have been limited to linear systems and have assumed that complete statistical knowledge concerning the system is available. Generally, white Gaussian noise has been assumed for the disturbances. Since the invariant imbedding approach is different from the usual classical approach, several advantages have been gained. First, the present approach is applicable to a wide variety of nonlinear problems. 246

2.

AN ESTIMATION PROBLEM

247

Second, a sequential estimation scheme is obtained. T h e usual classical approach results in nonsequential estimation schemes. T h e sequential estimation scheme has two advantages over the nonsequential one for dynamic systems. First, for nonsequential estimation schemes, each time additional observations or measurements are to be included, all the calculations must be repeated entirely. Second, because of these repeated calculations, nonsequential estimation schemes are much more difficult to implement in real time than sequential schemes. No statistical assumptions will be made concerning the noises or disturbances, because for most practical problems the determination of valid statistical data concerning these disturbances is a difficult problem in itself. T h e generally used least squares criterion will be employed to obtain the optimal estimates. If the statistics concerning the disturbances are known, criteria better than the least squares may be obtained. Essentially, two problems are treated in this chapter. They are the estimation of state variables and parameters with measurement errors only, and the same estimation problem with both measurement errors and unknown disturbance inputs. T h e estimator equations were originally obtained by Bellman and co-workers [13], and by Detchmendy and Sridhar [14]. Following the approaches used in the previous chapters, the derivations will be completely formal. T h e simpler scalar case is obtained first. Considerable details are given for these simpler cases. 2. An Estimation Problem

Consider a system whose dynamic behavior can be represented by the nonlinear differential equation

T h e state of the system, x, is being measured or observed starting at an initial time to = 0 and continuing to the present time tr . Owing to the presence of noises or measurement errors, the observed state, x, of the system does not represent the true state. Let

+ (measurement or observation errors) (2) On the basis of this observed signal z(t) in the interval 0 < t < tf , we z ( t )= x(t)

may seek an estimate of the present state x(t,), a past state x ( t l ) , or a

8.

248

NONLINEAR FILTERING

future state x(t2). Let us first consider the problem of estimating the present or current state x ( t t ) of the system. The other two problems can be solved in the same way. In fact, the entire trajectory or profile is estimated during the course of the estimation of x(tr). The problem is to estimate the current state of the system at tr , using the classical least squares criterion, so that the following integral is minimized:

J

=

f’(x(t)

-

~ ( t dt ) ) ~

0

(3)

where z(t) is the observed function. T h e function x ( t ) is determined on the interval 0 t tr by the differential equation (1). This problem can be stated differently as follows: On the basis of the observation z(t),0 t < tt , estimate the unknown condition

< <


+ 2[hc(c, a)lThc(c,a)

(29)

T h e elements of the matrix represented by the first term on the righthand side are scalar or inner products of the vectors hcic,and [z - h]. Thus

where

T h e term 5 represents terms that consist of terms of the form of rc or re,, . When c takes on its optimal estimate e, from Eq. (1l), rc = 0. The

9.

ESTIMATION OF STATE A N D PARAMETER-AN

EXAMPLE

261

terms involving rccc are negligible. Consequently, the term 5 drops out in the neighborhood of the optimal estimate. Combining Eqs. (24), (28), and (29), the desired differential equations for q are obtained:

_ dq - f4e, a)q(a) + q(4[ fc(e, 41= + q(a){hcc(e,a"(4 da -

x q(a) - q(a"c(e,

-

h(e, 41>

(31)

a>lThc(e, ah(4

The matrix {hcc[z- h]} is defined by Eq. (30). T o obtain a uniform notation, replacing subscript c by e, Eqs. (22) and (31) become

+ q(a)[he(e,a)l'[z(a) - Me, all dq - fe(e, a)q(a>+ q(a)[ fe(e, a)]' da de da

- = f(e, a)

(324

--

+ q(a){hee(e,a ) [ z ( a )-

a)lh(a)

(32b)

-q(a)[he(e, a)IThe(e,a)q(a)

T h e equations (32) are the desired estimator equations. Notice that Eq. (32a) represents M differential equations and (32b) represents M 2 differential equations. T h e matrices with subscript e are the same as the matrices with subscript c , as previously defined.

9. Estimation of State and Parameter-An

Example

The problem of simultaneous estimation of state and parameters of a system also can be solved by the present approach. T h e unknown parameters can be considered as part of the state of the system and, as has been discussed in Chapter 4, differential equations for these parameters can be established. To illustrate this approach, the problem solved in Section 7 will be considered. Both the state x and the parameter k are to be estimated from the noisy measurements on the concentration of component A. T h e noisy data for x(t), x ( t ) , are generated in the same way as before with Eqs. (7.2) and (7.3). Equation (7.4) is replaced by the equation z(t,)

=

.(tk)[l

+ O.lR(t,)]

k

= 0 , 1,2

,... N )

(1)

By using Eq. (I), the measurement error is approximately proportional to the true value of x.

262

8.

NONLINEAR FILTERING

The equations corresponding to Eq. (8.1) are dx _ dt

4 x 2

dk _ -0 dt

with M = 2 and m = 1. Since

h(x, t )

=x

(3)

Eq. (8.2) is identical with the scalar case for the present example. The estimator equations can be obtained from Eq. (8.32)

where 0 represents the null matrix. The functions e, and e2 represent the optimal estimates of x and K, respectively. Equations (4) and ( 5 ) represent six simultaneous estimator equations. With the following initial conditions for Eqs. (4) and ( 5 )

the results shown in Fig. 8.4 are obtained with the Runge-Kutta integration scheme and with A t = 0.1. Only the estimated values for the reaction rate constant are shown. Since the measurements at t = 0 are used as the initial conditions for the state variable, the true value of this variable is obtained very quickly. I n a way, the value of qza(0)

9.

ESTIMATION OF STATE AND PARAMETER-AN

263

EXAMPLE

represents the confidence one has in the initial value of k(0) and the observed signal x ( t ) . Note that the best filtering action is obtained with qS2(O) = 5. Too large a value for qZ2(O), such as q22(0)= 20, results in overestimation of the value of e 2 .

0.08 0.04 cu 0 L

-

0.0

0

- 0.04 -0.08 -0.12

-

e, 10) = z(01

qn lol=q1~o)=q21~o)= 1

-

'

1

0

I

10

I

I

20

I

t or a

1

I

30

I

40

I

50

FIG.8.4. Estimated parameter as a function of qZa(0).

Instead of using Eq. ( l ) , some experiments also have been performed with the noisy measurements generated by Eqs. (7.2), (7.3), and (7.4). The initial conditions used are:

The results for these experiments are shown in Figs. 8.5 and 8.6. In spite of the large oscillations of e2 at time near zero, the true value of the reaction rate.constant is obtained by time t = 30. T h e values of q(0) are very important in the calculations. T h e true values of x and k cannot be estimated by t = 50 if the following initial values are used:

A minus value of k is estimated for the present case.

8.

264

NONLINEAR FILTERING

2.21

I

t or

FIG. 8.5.

I

Estimated state as a function of el(0).

Only every tenth integration point is plotted in Figs. 8.4-8.6. Consequently, not all the oscillations can be shown in these figures.

0.28 F e210)=0.1

0.24

q,,(o)=q2210)= 5

q,210)=q21(o) =1

0.20 0.16 (v

Y

-

0 L -I

-

-. I

I

I

I

I

I

I

I

FIG.8.6. Estimated parameter as a function of el(0).

I

10.

A MORE GENERAL CRITERION

265

10. A More General Criterion

Instead of the criterion (8.4), the following weighted criterion can be used:

J

=

S"[z(t) - h(x, t)ITQ(t)[z(t)- h(x, t)l dt

where [z - hITQ ( t ) [ z - h] represents the quadratic form [I51 associated with the matrix Q ( t ) . T h e expression 11 z - h ;1 will be used to denote this quadratic form. T h e vector [z(t)- h(x, t)] represents the column vector

The matrix Q ( t ) is a symmetric m x m matrix.* I n addition, this matrix is positive semidefinite. T h e expansion of this quadratic form leads to a weighted sum of squares of the elements of z - h, with the weighting determined by the elements of Q ( t ) . If Q ( t ) is a unit matrix, Eq. (1) reduces to Eq. (8.4). This quadratic form has been used by Kalman [4, 121. Note the versatility of the criterion (1). By choosing the elements in Q suitably, the observations on any variable can be made more important than observations on any other variable. With the criterion (l), the estimator equations for the problem formulated in Section 8 can be obtained in the same way. With terms involving rcCcneglected, the estimator equations are

* As can be shown easily, it entails no loss of generality in the treatment of quadratic forms to assume that the matrix Q is symmetric.

8.

266

NONLINEAR FILTERING

Except for the replacement of the inner product of each element by - h] is similar to

h&, Q[z - h], the definition of the matrix he, Q[z that of matrix (8.30). Thus

The other matrices are defined in the same way as those defined in Section 8.

11.

An Estimation Problem with Observational Noise and Disturbance Input

The problem of estimating state variables and parameters in the presence of both observational errors and disturbances in the input also can be treated by invariant imbedding [14]. Consider the system represented by

The measurements or observations on the output are z ( t ) = h(x, t )

+ (measurement errors)

(2)

where u ( t ) is the unknown disturbance input. Since the function f depends on t explicitly, any known inputs such as control inputs or test signals can be included in this function. The above problem has been treated by Cox [7] by assuming that both the measurement errors and disturbance input are white Gaussian noise. No statistical assumption will be made concerning these disturbances in the present treatment. Thus, the disturbance input may be either random input with unknown statistics or a constant disturbance input. Following the treatments in earlier sections, this estimation problem can be stated as follows: On the basis of the measurement z(t), 0 t tf , estimate the unknown condition

<
These equations are integrated by the Runge-Kutta integration scheme with the following initial conditions:

T h e results are shown in Figs. 8.7 and 8.8. Constant values are assumed for the weighting factor w . T h e true values of x obtained by integrating

--

-TRUE VALUE, x ESTIMATED VALUE, e q(0)= 5 0

1.2

L

0 X

I

I

10

I

I

20

I

t or 3

1

30

I

I

I

40

FIG. 8.7. Estimation of current state with unknown input, w = 10.

1 50

8.

272

1.8

NONLINEAR FILTERING

--

-TRUE VMUE. x ESTIMATED VALUE, e e101=2 q@l= 5

I

1

ob

I

I

io

1

2o

I

t or a

I

I

30

1

I

FIG.8.8. Estimation of current state with unknown input, w

I 50

40

100.

=

Eq. (7.2) also are shown in the figures. Another experiment with q(0) = 1 and w = 10 is shown in Fig. 8.9. The disturbance input generated by Eq. (2) is fairly high compared with the term Ax2. This is especially true at t near t t , where the true

-TRUE VALUE, x "ESTIMATED VALUE, e eU=2

i.8

:

\

w(OI= 10

i2- \

\

I

Ob

I

io

I

I

'O

I

t or a

1

30

I

I

I

40

FIG.8.9. Estimation of current state with unknown input, q(0) = 1.

15. Systems of Equations with Observational Noises

and Disturbance Inputs Consider the system of nonlinear differential equations

15.

SYSTEMS OF EQUATIONS WITH OBSERVATIONAL NOISES

273

with the measurements on the output z ( t ) = h(x, t )

+ (measurement errors)

(2)

where x and f are M-dimensional vectors, z and h are m-dimensional vectors, and g is an M x k matrix. T h e k-dimensional vector u(t) represents the disturbance inputs. This problem can be formulated, again, in terms of optimization ) that the vector function problems. Find that vector function ~ ( t such x(t) given by Eq. (1) minimizes the integral (3)

where Q and W are symmetric m x m and M x M matrices, respectively. T h e first term under the integral sign represents the quadratic form associated with the matrix Q ( t ) and the second term represents the quadratic form associated with the matrix W(t). Both Q and W are positive semidefinite matrices. T h e elements of the matrices Q and W represent the weighting factors for the measurement errors and the disturbance inputs. Using Eq. (l), we can write Eq. (3) as (4)

T h e Euler-Lagrange equations for the above optimization problem can be obtained easily. From Eq. (2.17) of Appendix I, we obtain

T h e last term in the above equation is the inner product between the M-dimensional column vector h and the column vector within the square bracket. T h e Euler-Lagrange equations are

8.

274

NONLINEAR FILTERING

where

and

gxu =

An expression for u can be obtained from Eq. (7). Substituting this expression into Eqs. (1) and ( 6 ) , one obtains dh dt

- = -2[hx(x, t>lTQ(t)[z(t) - h ( x , t)] -

dx dt

- --

f ( x ,t )

+ 4g [ g T W g l - l g T A

T. fdx, t)lTX + 5

(lla) (1lb)

T h e term 5 represents terms involving A"-. We shall see later that terms involving powers of h higher than the first will be neglected. T h e boundary conditions for Eqs. (1 1) can be obtained from the free boundary condition h(0) = 0

(124

A(tf) = 0

(12b)

T h e missing final condition x(tr) for the boundary-value problem represented by Eqs. (11) and (12) can be obtained by the invariant

15.

SYSTEMS OF EQUATIONS W I T H OBSERVATIONAL NOISES

275

imbedding approach. Consider the problem with the more general boundary condition h(0) = 0 A(a) = c

with 0

< t < a. If we let X(U) =

r(c, a)

then the invariant imbedding equation is r,(c, a ) -

+ r,(c, a){--2[hr(r(c,

a), a)l'Q(a)[z(a) - h(r(c,a),

+

.>I

[ fr(r(c,a>,a>l*c S> =f

(r(c,a), a )

+ 4g[gWgl-lgTc

where ra is an M-dimensional vector and r, is the matrix

Using the same argument as that used for the scalar case in Section 13, the vector r can be approximated by r(c, a)

=

e(a)

+ p(a>c

(17)

where e is an M-dimensional vector and p ( u ) is an M x M matrix. Using the same analysis as that used in the scalar case, we obtain the following estimator equations:

276

8.

NONLINEAR FILTERING

16. Discussion

Invariant imbedding has been used in two different ways. I n Section 4, imbedding is used to obtain the missing final condition, which represents the nominal value of the integral criterion to be minimized. T h e optimal estimate is then obtained by differentiation. I n Sections 12 and 13, equations for the optimal estimate are obtained first by treating the problem as an optimization problem in the calculus of variations. Then the invariant imbedding equation for the missing final conditions, which represent the optimal estimates, is obtained. These different uses are due to the different natures of the two estimation problems. T h e problem discussed in Section 4 represents a minimization problem in which only the final value of the state variable is within our control. However, the problem discussed in Sections 12 and 13 represents a minimization problem in which the entire trajectory of the unknown disturbance input is assumed to be within our control. Note that Eq. (2.1) is completely determined once a boundary condition is given. However, Eq. (1 1.1) cannot be determined completely until the complete trajectory for u ( t ) is given. Although the invariant imbedding approach appears to be an effective tool to treat nonlinear estimation problems, much more research and computational experiments are needed. For example, it would be interesting to see whether an optimal weighting factor w ( t ) can be obtained for Eq. (13.11). T h e numerical experiments performed in Section 14 seem to indicate that a weighting factor w which varies with time t is preferable to a constant weighting factor. Although all the numerical experiments are performed on digital computers, analog computers also can be used if the estimator equations are not too complex. Economical advantages can be gained by using analog equipment for on-line estimation purposes. This is due to the fact that the estimator equations, which are ordinary differential equations of the initial-value type, are especially suited for real time analog computation.

REFERENCES

1. Wiener, N., “The Extrapolation, Interpolation, and Smoothing of Stationary Time Series.” Wiley, New York, 1949. 2. Bode, H. W., and Shannon, C. E., A simplified derivation of linear least-squares smoothing and prediction theory. Proc. I R E 38, 417 (1950). 3. Pugachev, V. S., “Theory of Random Functions and Its Application to Automatic Control Problems” (in Russian). Gostekhizdat, Moscow, 1960.

REFERENCES

277

4. Kalman, R. E., and Bucy, R. S., New results in linear filtering and prediction theory. J . Basic Eng. 83, 95 (1961). 5 . Kalman, R. E., A new approach to linear filtering and prediction problems. J. Basic Eng. 82, 35 (1960). 6 . Ho, Y., The method of least squares and optimal filtering theory. RM-3329-PR. RAND Corp., Santa Monica, California, October, 1962. 7. Cox, H., Estimation of state variables via dynamic programming. Presented at Joint Autom. Control Conf., Stanford, California, June 24-26, 1964. 8. Bryson, A. E., and Frazier, M., Smoothing for linear and nonlinear dynamic systems. Proc. Optimum System Syn. Conf., Wright-Patterson Air Force Base, Ohio, September, 1962, AST-TDR-63-119. 9. Levinson, N., A heuristic exposition of Wiener’s mathematical theory of prediction and filtering. J . Math. Phys. 26, 110 (1947). 10. Davenport, W. B., and Root, W. L., “An Introduction to the Theory of Random Signals and Noise.” McGraw-Hill, New York, 1958. 11. Lee, Y. W., “Statistical Theory of Communication.” Wiley, New York, 1960. 12. Lee, R. C. K., “Optimal Estimation, Identification, and Control,” The M. I. T. Press, Cambridge, Mass., 1964. 13. Bellman, R., Kagiwada, H. H., Kalaba, R., and Sridhar, R., Invariant imbedding and nonlinear filtering theory. RM-4374-PR. RAND Corp., Santa Monica, California, December, 1964. 14. Detchmendy, D. M., and Sridhar, R., Sequential estimation of states and parameters in noisy nonlinear dynamical systems. Presented at Joint Autom. Control Conf., Troy, New York, June 22-25, 1965. 15. Courant, R., and Hilbert, D., “Methods of Mathematical Physics,” Vol. I. Wiley (Interscience), New York, 1953. 16. Deutsch, R., “Estimation Theory,” Prentice-Hall, Inc., Englewood Cliffs, New Jersey, 1965.

Chapter 9

PARABOLIC PARTIAL DIFFERENTIAL EQUATIONS-FIXED BED REACTORS WITH AXIAL MIXING

1. Introduction

In this chapter, it will be shown that the quasilinearization technique is also a useful tool for solving parabolic partial differential equations encountered in the study of the dynamics of chemical reactors. Specifically, partial differential equations of fixed bed reactors with axial mixing will be treated under various simplifying assumptions. First, the reactor will be assumed to be an isothermal reactor. Then, the influence of temperature upon reaction rate equations will be considered. And, finally, the transient equations of the packing particle will be included. The last case has been solved by Liu and Amundson [l] by the commonly used first-order convergent method, and thus a comparison can be made between the present approach and the other method. Although we have encountered partial differential equations in connection with invariant imbedding, we have avoided numerical solutions for these equations. The computational solution of partial differential equations is much more complicated. For ordinary differential equations of the initial-value type, a large number of stable methods of numerical integration exists and the truncation errors of these numerical methods are of fairly high order in the integration step size. Thus, the advantage of a second-order convergent process over the first-order is obvious. However, for partial differential equations, especially of the elliptic and parabolic types, the commonly used stable numerical methods usually have higher truncation errors, which are very sensitive to the mesh or integration step sizes used. Because of these numerical inaccuracies, the advantage of the second-order convergent process such as the quasilinearization technique over the commonly used 278

2. ISOTHERMAL

REACTOR WITH AXIAL MIXING

279

first-order convergent process is not so obvious. Consequently, numerical experiments furnish the only source of comparison. We shall not discuss the problems of existence and convergence of the recurrence relations of partial differential equations resulting from the application of the generalized Newton-Raphson formula. This is due to the fact that any detailed discussion of these aspects would carry us too deeply into the theory of partial differential equations. Those interested can consult Bellman and Kalaba [2, 31. I n general, if a fairly accurate and stable numerical integration procedure is available, quadratic convergence should be expected. T h e equations treated in this chapter are extensions of the equations solved in Chapter 3. Consequently, the finite-difference method used there can be applied to these partial differential equations at each integration step in the time direction without modification. However, owing to stability problems, an implicit difference formula by Crank and Nicolson [4] is used to obtain the difference equations in the time direction. T h e numerical aspects of elliptic and hyperbolic partial differential equations will not be discussed in this volume [2, 3, 5-9, 11-13]. This chapter follows the treatment of Lee [lo]. 2. Isothermal Reactor with Axial Mixing

Instead of the steady state case, consider the transient equation of the fixed bed chemical reactor treated in Section 6 of Chapter 3. T h e dynamics of this isothermal reactor with axial mixing can be represented by the equation

where NPe= D,v/D is the Peclet group; z = x / D , is the dimensionless reactor length variable; t = Ov/D, is dimensionless time; and x , 0 are reactor length and time variables, respectively. T h e reaction rate group, r, and other symbols have the same meaning as those defined in Section 6 of Chapter 3. T h e variable p represents the partial pressure of the reactant A in the interstitial fluid. I t has been assumed that the packing has no influence on the reaction except its contribution to the axial mixing. T h e boundary conditions are

280

9.

PARABOLIC PARTIAL DIFFFRENTIAL EQUATIONS

where zfis the total dimensionless length x r / D , of the reactor and p , represents the concentration of A before it enters the reactor. T h e total reactor length is xf. T h e initial condition is p(z, 0 ) = p"2) t = 0 0 < 2 < Zf (2c) T h e partial pressure p is the dependent variable, and z and t are the two independent variables. Since only the term rp2 is nonlinear and all the differentials appear linearly, Eq. (1) is called a quasilinear partial differential equation. Applying the generalized Newton-Raphson formula to the nonlinear term in Eq. (l), we obtain the following linear equation:

where P , + ~ is the unknown variable, and p , is known and is obtained from the previous iteration. T h e boundary and initial conditions for (3) are

3. An Implicit Difference Approximation

At any fixed time t , Eq. (2.3) can be solved by the same finite-difference method discussed in Chapter 3. However, because of the stability problems, it is not at all straightforward to take the difference of (2.3) in the time direction. If explicit difference is used, in order to ensure numerical stability, the integration step size in the time direction must be very small. This problem has been discussed in detail by Forsythe and Wasow [5]. T o avoid these difficulties, the implicit formula of Crank and Nicolson will be used [4]. Suppose that now the bed length zf is divided into M equal increments of width dz, the partial derivatives in Eq. (2.3) can be replaced by the following Crank-Nicolson difference operator:

3.

28 1

A N IMPLICIT DIFFERENCE APPROXIMATION

T h e other unknown variables can be replaced by the simple difference expressions aPn+l -

at P,+l

1 At

+ I)

[pn+l(m,

=$[Pn+1h

k

- P(m9

')I

(3)

+ 1) + P ( m ')I

(4)

where p(m, k ) represents the value of p at axial position m dz of the reactor and at time k At. I n the above equations, the value of p at time step k is considered known; the current unknown time step is k 1. I n this unknown time step, n quasilinearization iterations have been calculated. 1)st time step and (n 1)st iteration Thus, only the values of p at ( k are the unknown values. T h e subscript which represents the number of iterations has no meaning for the values of the previous kth time step. T h e function p , in Eq. (2.3) can be replaced by either of the following two difference operators:

+

+

Pn =

P,

mdm

=P n b ,k

'+

+

1) + P ( m ,

1'1

+ 1)

(5)

(6)

In the present calculation, Eq. (5) is used. Note that in the implicit Crank-Nicolson method the average values of the difference operators at (k 1) d t and k d t are used for the partial derivatives. T h e stability and convergence requirements are much less severe for this method than for the explicit methods. T h e boundary conditions, Eqs. (2.4a) and (2.4b), now become

+

Pn+1W,

'+

1) = Pn+dM

-

1,

'+

1)

(8)

Substituting Eqs. (1)-(5) and Eqs. (7) and (8) into Eq. (2.3) for m = 1, 2, ..., ( M - l), we can obtain the following system of ( M - 1) simultaneous equations:

AP,+l(k + 1) = -(BPW where

A=

+ E)

9.

282

PARABOLIC! PARTIAL DIFFERENTIAL EQUATIONS

I

e

L o

f

E=

with

1 1 +--NPe49 242

1

(17)

At

1/(2Nped4

f

=

h

=f -

1/(2dz)

+

In the above equations the partial pressure, p,,, (k l), is the unknown variable. The value of p , (k 1) is known and is obtained from previous iterations. The partial pressure p ( m , k) is the preceding time step and is considered known. Since the matrix A is tridiagonal,

+

5.

NUMERICAL RESULTS-ISOTHERMAL

REACTOR

283

Eq. (9) can be solved by the straightforward Thomas method discussed in Section 8 of Chapter 3. 4, Computational Procedure

Assuming that all values at kth time step are known, we can calculate 1)st time step as follows: the (k

+

+

Assume the values for pnz0 (m,k I), m = 0, 1, 2,..., M . (2) Calculate P , , ~ (m,k l), m = 1, 2,..., M - 1, from Eq. (3.9). 1) and ( M ,k 1) from Eqs. (3.7) (3) Calculate p,=l (0, k and (3.8), respectively. (4) With the values for (m,k 1) known, obtain the values 1) from steps (2) and (3). Repeat this process until the for P , , ~(m,k required accuracy is obtained. (1)

+ +

+

+

+

Let E be the maximum error allowed; the required accuracy can be defined by the following equation:

1 pn+1(m,

+ 1)

+ 1) I