Theory of Difference Equations Numerical Methods and Applications
~
Lakshmikantham
Department of Mathematics Universi...
63 downloads
569 Views
8MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Theory of Difference Equations Numerical Methods and Applications
~
Lakshmikantham
Department of Mathematics University of Texas at Arlington Arlington, Texas
D. Trigiante Dipartimento di Matematica Universita di Bari Bari, Italy
ACADEMIC PRESS, INC. Harcourt Brace Jovanovich, Publishers Boston San Diego New York Berkeley London Sydney Tokyo Toronto
Copyright © 1988 by Academic Press, Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher.
ACADEMIC PRESS, INC. 1250 Sixth Avenue, San Diego, CA 92101
United Kingdom Edition published by ACADEMIC PRESS, INC. (LONDON) LTD. 24-28 Oval Road, London NWI 7DX
Library of Congress Cataloging-in-Publication Data Lakshmikantham, V. Theory of difference equations. (Mathematics in science and engineering) Includes index. Bibliography: p. 1. Difference equations. 2. Numerical analysis. I. Trigiante, D. II. Title. QA431.L28 1987 515'.625 87-16783 ISBN 0-12-434100-4
88 89 90 91 9 8 7 6 5 4 3 2 1 Printed in the United States of America
Preface
Difference equations appear as natural descriptions of observed evolution phenomena because most measurements of time evolving variables are discrete and as such these equations are in their own right important mathematical models. More importantly, difference equations also appear in the study of discretization methods for differential equations. Several results in the theory of difference equations have been obtained as more or less natural discrete analogues of corresponding results of differential equations. This is especially true in the case of Lyapunov theory of stability. Nonetheless, the theory of difference equations is a lot richer than the corresponding theory of differential equations. For example, a simple difference equation resulting from a first order differential equation may have a phenomena often called appearance of "ghost" solutions or existence ofchaotic orbits that can only happen for higher order differential equations. Consequently, the theory of difference equations is interesting in itself and it is easy to see that it will assume greater importance in the near future. Furthermore, the application of the theory of difference equations is rapidly increasing to various fields such as numerical analysis, control theory, finite mathematics and computer science. Thus, there is every reason for studying the theory of difference equations as a well deserved discipline. The present book offers a systematic treatment of the theory of difference equations and its applications to numerical analysis. It does not treat in detail the classical applications of difference calculus to numerical analysis such as interpolation theory, numerical quadrature and differentiation, which can be found in many classical books. Instead, we devote our attention to iterative processes and numerical methods for differential equations. The
ix
x
Preface
investigation of these subjects from the point of view of difference equations allows us to systematize and clarify the ideas involved and, as a result, pave the way for further developments of this fruitful union. The book is divided into seven chapters. The first chapter introduces difference calculus, deals with preliminary results on difference equations and develops the theory of difference inequalities. In the second chapter, we present the essential techniques employed in the treatment of linear difference equations with special reference to equations with constant coefficients. Chapter 3 deals with the basic theory of systems of linear difference equations. Chapter 4 is devoted to the theory of stability of Lyapunov including converse theorems and total and practical stability. Chapters 5 and 6 discuss the application of the theory of difference equations to numerical analysis. In Chapter 7, we present applications of difference equations to many fields such as economics, chemistry, population dynamics and queueing theory. The necessary linear algebra used in the book is given in Appendices. Finally, several carefully selected problems at the end of each chapter complement the material of the book. Some of the important features of the book include the following: (i)
(ii) (iii) (iv) (v)
development of the theory of difference inequalities and the various comparison results; a unified treatment of stability theory through Lyapunov functions and comparison method; stressing the important role of the theory of difference equations in numerical analysis; demonstrating the versatility of difference equations by various models of the real world; timely recognition of the importance of the theory of difference equations and of presenting a unified treatment.
The book can be used as a textbook at the graduate level and as a reference book. Readers interested in discrete dynamical systems will also find the material, which is not available in other books on difference equations, useful. We wish to express our immense thanks to Professors Ilio Galligani, Vincenzo Casulli, Giuseppe Piazza and S. Leela for their helpful comments and suggestions. We are happy to convey our thanks to Ms. Sandra Weber for her excellent typing of the manuscript at several stages. We express our appreciation and thanks to Mr. William Sribney, Editor, Academic Press, for his helpful cooperation.
Preliminaries
CHAPTER 1
1.0. Introduction This chapter is essentially introductory in nature. Its main aim is to introduce certain well-known basic concepts in difference calculus and to present some important results that are not as well-known. Sections 1.1 to 1.4 contain needed difference calculus and some notions related to it, most of which is found in standard books on difference equations. Section 1.5 deals with preliminary results on difference equations such as existence and uniqueness and also discusses solving some simple difference equations. In Section 1.6, we develop theory of difference inequalities and prove a variety of comparison theorems that play a prominent role in the development of the book. Several problems are given in Section 1.7 that, together with the material of Sections 1.1 to 1.4, cover the necessary theory of difference calculus.
1.1.
Operators A and E
We shall consider functions defined on a discrete set of points given by
r; or J;b,h
r; = {xo, Xo + 1, ... , Xo + k, ...}.
n.,
=
{xb, xb + h, ... , xb + kh, . ..},
1
2
I.
Preliminaries
where x~ = hxo E Rand Xo E R (sometimes Xo E C). Here Rand C denote as usual, the real and complex numbers respectively. The set N+ of natural numbers that is isomorphic to both n; J;;"h is also used as the definition set when it is desirable to exhibit the dependence of the function on n E N+. If y: N+ ~ C is the discrete function, y( n) is the sequence denoted by Yn' If the discrete function represents the values of a continuous function at discrete points, then the set J;" or J;;'.h is preferred since in this case, the dependence on Xo is shown explicitly. The advantage in using J;;"h as the definition set is that, for a discrete function which is the approximation of a function defined on R, the dependence of the approximation error on the step h is indicated. The correspondence between functions defined on N+ and is given by Yn = y(n) = y(xo + n), where y is the discrete function with values in R or C. In some cases, the following sets are also used:
r;
n.,
=
{xo, Xo ± h,
I;o = {xo, Xo ± 1, N
, Xo ± kh,
,Xo ± k,
},
},
= {a, ±1, ±2, ... , ±k, .. .},
N~o =
{no, no ± 1, ... , no ± k, ...},
with k E N+ and no E N. In this chapter, we shall often use J; as the definition set whenever we need the dependence on x E R (or x E C) in order to consider derivatives (or differences) with respect to x. There will be no difficulty in translating the results in terms of other notations. As a rule, we shall only use the sequence notation in the problems at the end of chapters. Definition 1.1.1.
defined by
Let y: J; ~ C. Then ll: C ~ C is the difference operator lly(x)
and E : C
~
= y(x + 1) -
y(x)
(1.1.1)
C is the shift operator defined by Ey(x)
=
y(x + 1).
(1.1.2)
It is easy to verify that the two operators II and E are linear and that they commute. That is, for any two functions y, z and J; and any two scalars a, {3, we have
+ {3z(x» E(ay(x) + {3z(x» ll(ay(x)
and llEy(x)
=
Elly(x).
= =
+ {311z(x), aEy(x) + {3Ez(x),
ally(x)
1.2.
3
Negative Powers of .1
The second difference on y(x) can be defined as a 2y(x)
= a(ay(x)) = y(x + 2) -
In general, for every k aky(x)
E
2y(x
+ 1) + y(x).
N+,
= a(ak-Iy(x) and
Eky(x)
= y(x + k),
= EOy(x) = Iy(x),
I being the identity operator such that Iy(x) = y(x). In the case when the definition set is N+, one has aYn = Yn+t - Yn and EYn = y(n + 1) for (1.1.1), (1.1.2) respectively. It is easy to
with aOy(x)
see that the formal relation between a and E is a = E - I and thus, powers of a can be expressed in terms of powers of E and vice versa. In fact, (1.1.3)
and (1.1.4)
where
(~)
are the binomial coefficients.
Definition 1.1.2. Let y: J~ ~ C. The function y is said to be periodic of period kif y(x + k) = y(x). The constant functions are particular periodic functions and y(x) = e i2 11"x is a periodic function of period 1. It is easy to see that ay(x) = 0 for any periodic function of period 1.
1.2.
Negative Powers of L1
Consider the equation ay(x) = g(x),
(1.2.1)
where y, g : J~ ~ C. The solution of this equation is a function y such that (1.2.1) is satisfied. We shall denote this solution by y(x) = a-Ig(x). It is not unique because y(x) can also be expressed as a -lg(X) + w(x), where w(x) is an arbitrary function of period 1. The operator a-I is called the antidifference and it is linear. Moreover, the operators a and a - I do not commute since
4
l.
Preliminaries
If f, g: J; ~ C are two functions such that t:.f(x) = t:.g(x), then it is clear that f(x) = g(x) + w(x). In particular, if f(x) and g(x) are polynomials, t:.f = t:.g implies f(x) = g(x) + c, where c is a constant. We shall now state a fundamental result that enables us to compute the finite sum 2:~~of(x + i) in terms of the antidifference t:.-If(x). Theorem 1.2.1.
f
Let F(x)
= A-If(x).
Then
+ i) = F(x + n + 1) -
f(x
F(x) ;;;: F(x
i~O
Proof.
+ i)I::~+I.
(1.2.2)
Since by hypothesis we have f(x) = t:.F(x), it is easy to see that n
L f(x
n
+ i) =
i~O
L t:.F(x
+ i)
i~O
n
= L [F(x + i + 1) - F(x + i)] i=O
= F(x + n + 1) -
F(x).
•
Note that (1.2.2) can also be written as
f f(x + i) = t:.-If(x + i)1::~+I.
(1.2.3)
j=O
If we leave the sum to remain indefinite, we can express the relation (1.2.3) in the form Lf(x)
= t:.-If(x) + w(x)
in analogy with the notation for indefinite integrals. In the case when the definition set is N+, the foregoing formulas reduce to n
" Y."= A-I y,"li=n+1 'i=O
;=0
and
"y," s: = t:.-Iy."
+w
respectively.
1.3. Factorial Powers and Discrete Taylor Formulas Analogous to the role of functions x" in differential and integral calculus, one has the factorial powers of x, denoted by x(n).
1.3. Factorial Powers and Discrete Taylor Formulas
Definition 1.3.1.
Let x
E
x(n)
5
R. The nth factorial power of x is defined by
= x(x -
1) ... (x - n
+ 1).
It is easy to verify from the above definition that 6x(n)
= nx(n-I)
and
6 -l x(n-l)
1 = -x(n) + w(x).
n
(1.3.1)
We also have x(m+n) = x(m)(x - m)(n) = x(n)(x - n)(m). For m = 0, this yields x(O+n) = x(o)x(n\ which shows that X(O) = 1. Moreover, for m = -n, we get 1 = x(O) = x(-n)(x + n )(n\ which allows one to define the negative factorial power x(-n) by 1
x(-n) = (x
+ 1)(x + 2) ... (x + n)'
The relations (1.3.1) suggest that it wilI be convenient to express other functions in terms of factorial powers whenever possible. For example, in the case of polynomials we have the following result. Theorem 1.3.1.
The powers x" and the factorial powers are related by
(1.3.2) where
S~
are the Sterling numbers (of second kind) that satisfy the relation S7+1 = S7-1 + iS7
with S: =
1.
S~ =
Proof. Clearly (1.3.2) holds for n = 1. Suppose it is true for some n, multiplying both sides of (1.3.2) by x, we get x n+ 1
=
n
I
1=1 n
=
S7x' xli)
I
iS7x(')
+
;=1
n
= "i... [iSnI 1=2
= n
I
n
I
1=1
S7x(x -1) ... (x - i
+ 1)(x - i + i)
S7x(i+I)
i=1
+ S~1-1 ]x(i) + Sn+1 x(I) + sn+lx(n+I) 1 n+l
n+l
= "J:. Sn+1 X(i), I i=l
showing that (1.3.2) holds for n + 1. Hence the proof is complete by induction. • Sterling numbers S7 for i, n = 1,2, ... ,6 are given in Table 1.
1.
6 Sterling numbers of Second Kind
1.
TABLE
Preliminaries
n;
2
1 4
1 3 7
5 6
15 31
2
3
5
3
4
1 6 25 90
10 65
6
1
1 15
Using the relation (1.3.2), it is immediate to derive the differences and the antidifferences of a polynomial. Theorem 1.3.2. The first difference ofa polynomial ofdegree k is a polynomial ofdegree k - 1 and in general, the s th difference is a polynomial ofdegree k - s.
Proof.
It is not restrictive to consider x k • From (1.3.2) we have
!:J.sx k =
k
"
~
Sk!:J.sX(i) J
k
= ~ i.J
i(s)Skx(i-s) I
,
i=1
i=l
which is of degree k - s. It is easy to check that for x
E
Nt, ~(;) = (;)
and that !:J.(;)
=
C~ J,
2:: 1. We shall now establish some discrete analogues of the Taylor formula. Using the formula (1.1.4), we obtain the first results. •
j
Theorem 1.3.3.
Let n
E
Nt u;
Theorem 1.3.4.
Let k; n
u.;
E
and
be defined on
~ = E n Uo = c:
;=0
Nt,
k-l(n); .!:J.
= ,~o L
Un
I
(n).!:J. uo.
k :s n and Uo
+
Nt.
Then
(1.3.3)
i
I
Un
be defined on
n-k(n-S-l) k L k_ !:J. s=o
1
US'
Nt.
Then
(1.3.4)
7
1.4. Bernoulli Numbers and Polynomials
By using the identity (see problem 1.11)
one obtains (1.3.4), which is called discrete Taylor formula. A generalization of Theorem 1.3.4 is as follows. Theorem 1.3.5. Then
Let j, k, n .
tJ/un
E N~,
j :s k - 1, k
n and Un be defined on N~.
:S
n).a'uo + n-k+j (n - s - 1) akus' I k'
= i=j I I. - J. k-l (
(U.5)
- J - 1
s=o
Proof. For j = 0, (1.3.5) reduces to (1.3.4). Suppose that (1.3.5) holds for some j: Then, one has
aj+l Un
=
~l
n )Ll
i
(
L.
i- j - 1
i=j+l
Uo
+ n-~+j L.
(n - s -
1) a
k
k - j - 2
s=o
Us
(kk -- J-l ~ - I)LlkUn+l_k+j kf (i - jn- 1)LliU + ny:'+j (nk -- js -- 2I)Ll kU
+
i=j+l
s=o
0
2)
k k- j _ j _ 2 Ll un+1-k+j
+(k
k-l (
I
i=j+1
which is (1.3.5) for j
n ).a'uo+ n-k+j+l I (n - s -
i - j - 1
+ 1.
1)
k - j - 2
s=o
s
k s-«;
•
1.4. Bernoulli Numbers and Polynomials From (1.3.2) one has that for every n a-1x n
n
=I
i=l
E
N+, x
E
R
x(i+l)
S~-.--+
1+1
Wn(X),
(1.4.1)
where wn(x) are periodic functions. Ifwe require a-Ix n to be a polynomial, . 1 then we must choose wn(x) as constants with respect to x. Let - - Cn + I n+l be such constants, and let us write Bn+l(x) = (n
+ 1)a- I x n
n
= (n
+ 1) I
i=l
x(i+l)
S~-.-+ Cn+1
1+1
(1.4.2)
8
1. Preliminaries
with Bo(x)
= 1. The
polynomials Bn(x) satisfy the relation tlBn(x)
= nx n- 1•
(1.4.3)
They are not uniquely defined because the constants en are arbitrary. Usually it is convenient to avoid the Sterling numbers in the determination of Bn(x). This can be accomplished as follows.
= 1 and Bn(x)
Theorem 1.4.1. Let n E N+, Bo(x) (1.4.3). Then the two functions Fn(x)
be polynomials satisfying
.
n-I
= j~O
(n)i Bj(x),
(1.4.4)
and (1.4.5)
differ by a constant. Proof.
From (1.4.3) one has tlFn(x) =
II (~)iXi-l I
i=O
and tlGn(x)
= n[(x + o n - l -
Hence, it follows that tlFn(x) nomials, we have
= tlGn(x)
Fn(x)
where d; are constants.
x
n- I
]
= n-I j~O
(n)i iX
i I - ,
and since Fn(x), Gn(x) are poly-
= Gn(x) + dn,
(1.4.6)
•
When the constants d; have been fixed, (1.4.4) and (1.4.6) allow us to construct the polynomials Bn(x). The constants d; are fixed by imposing one more condition to be satisfied by Bn(x). The most commonly used condition is (1.4.7) or for n One, in fact, has
t~e
following result.
= 1,2, ....
(1.4.8)
9
1.4. Bernoulli Numbers and Polynomials
Theorem 1.4.2. If for every n E N+, the polynomials Bn(x) satisfy (1.4.3) with Bo(x) = 1, and either (1.4.7) or (1.4.8) is satisfied, then d n = 0, and
I (n) . Bj(x) =
nx n- I •
n-I
I
j=O
(1.4.9)
Let us start with (1.4.7). Differentiating (1.4.6) and using (1.4.7), we have nFn_l(x) = F~(x) = G~(x) = nGn_l(x). This implies that 0 = n(Fn_l(x) - Gn_l(x» = ndn_1 from which it follows d n- I = O. Let us now suppose that (1.4.8) holds. From (1.4.4) we obtain g Fn(x) dx = J~ Bo(x) dx = 1 and J~ Gn(x) dx = 1. Because of (1.4.6) we now get 1 = 1 + d.; which implies d; = O. • Proof.
As we have already observed, (1.4.9) and Bo(x) = 1 define the polynomials Bn(x) uniquely, and these are called Bernoulli polynomials. The first five of these polynomials are as follows:
= 1, B1(x) = X B2 (x ) = x 2 - X +~, Bo(x)
-!,
B3 (x )
= x 3 - h 2 + 4x,
Bix)
= x4 -
and 2x
+ x 2 -~.
3
The values of Bn(O) are called Bernoulli numbers and are denoted by B n. As an easy consequence of (1.4.9), we see that the Bernoulli numbers satisfy the relation
I
n-I i=O
(n). s, =0, I
n
=
2,3, ... ,
which can be considered as the expansion of (l + B)" - B", where the powers B i are replaced by B i • This property is often used to define Bernoulli numbers. It can be shown that the Bernoulli numbers of odd index, except for B 1 , are zero. The values of the first ten numbers are B o = 1, B 1 =
-!, B 2 = t B 4 = -~, B 6 = -A, B g = -f, B IO = i6.
From (1.4.3), applying
;1-1
to both sides, we get
;1-1X n - 1
= Bn(x).
(1.4.10)
n
A simple application of (1.4.10) is the following. Suppose that x takes integer values. Then, from (1.2.3) and (1.4.9), we see that
f
x=O
x n- I
= Bn(x) • • = .!.[Bn(m + 1) n
x~O
n
B n],
10
1. Preliminaries TABLE
2. Differences and Antidifferences
f(x)
Af(x)
e
0
A-1f(x) ex eX
eX
(e - I)e x
- - e'i'l c- l '
xc"
(e - I)xe x + e x +1
(x + b)(n)
n(x + b)(n-l)
-c"- ( x - -e -) ,e'i' 1 e-1 e-1 (x + b)(n+ 1J , n 'i'-1
C)
C~J
C:J
cos(ax + b)
-2 sin
sin(ax + b)
2 sin
~sin( ax + b +~)
~ cos( ax + b +~)
xn
nf 1-0
(~)Xi I
IOg(l +_1_) x+e
log(x + e)
n+l
sin( ax + b
-~)
a 2sin2
-cos( ax + b
-~)
a 2sin2 Bn+,(x) - - - n'i'-l n+1 '
log I'(x + e)
from which we get the sum of the (n _1)th powers of integer numbers. When n = 3, for example, we have m
I
x 2 = 1(m
+ I? -
~(m
+ I? + !(m + 1) = ~m(m + 1)(2m + 1).
X~O
In Table 2 we list differences and antidifferences of the most common functions, omitting the periodic function w(x).
1.5.
Difference Equations
We have seen that the knowledge of Ll-1g(x) allows us to solve the equation (1.2.1), which is a very simple difference equation. In general, a difference equation of order k will be a functional relation of the form F(x, y(x), Lly(x), ... , Llky(X), g(x» = 0,
where y, g : i; ~ C.
1.5. Difference Equations
11
More often, instead of the operator il, one uses the operator E. The difference equation is then written in the form G(x, y(x), Ey(x), ... , Eky(x), g(x»
=::
O.
If the function F (or G) is linear with respect to y(x), ily(x), ... , ilky(x) (or respectively y(x), Ey(x), ... , Eky(x», then the difference equation is said to be linear. The theory of linear difference equations will be presented in Chapter 2. Except for some specific cases, we shall consider, in the following, difference equations that can be written in the normal form Eky(x)
=::
(x, y(x), Ey(x), ... , Ek-1y(X), g(x»,
(1.5.1)
where is a function uniquely defined in its arguments in some subset D c J~o x ex ... x C. With the equation (1.5.1), we associate k initial conditions y(x)
=::
Ct.
y(x
+ 1) =
C2,
... ,
y(x
+ k -1)
=::
Ck.
(1.5.2)
An existence and uniqueness result for the problem (1.5.1), (1.5.2) is the following.
Theorem 1.5.1. The difference equation (1.5.1) with the initial conditions (1.5.2) has a unique solution y(x + n), if the arguments of are in D. Proof. From (1.5.1) and (1.5.2), we get y(x + k). By changing x into x + 1 in the value y(x + k) and using the last (k - 1) values from (1.5.2), we obtain y(x + 1 + k) in view of (1.5.1). Repetition of this procedure yields • the unique solution y(x + n).
The possibility of obtaining the values of solutions of difference equations recursively is very important and does not have a counterpart in other kinds of equations. Having at our disposal machines that can do a large number of calculations in a second, we can get, in a short time, a great number of values of the solution of difference equations. For this reason, one reduces continuous problems to approximate discrete problems. This way of obtaining solutions, however, although very efficient for some purposes, is insufficient for others. For example, it does not give information on asymptotic behavior of solutions, unless one is willing to accept costs that are exceedingly high. Hence, it is of great importance to have solutions in a closed analytical form, or, at least, to deduce information on the qualitative behavior of the solutions in some other way.
12
1. Preliminaries
Next, we shall discuss solutions of some simple difference equations. We have already seen that the solution of ~y(x) = g(x) is ~ -1 g(X). It is sometimes possible to reduce certain other equations to this form. This is the case, for example, with the equation z(x
+ 1) -
= q(X),
p(X)z(x)
Z(Xo) = zoo
(1.4.3)
In fact, setting P(x) = rr::~op(t), P(xo) = 1 and dividing (1.5.3) by + 1), we have
P(x
z(x + 1) z(x) q(x) P(x + 1) - P(x) = P(x + 1)' .
z(x) q(x) g(x) = ( )' equation (1.5.3) now takes P(x) P x+l the form ~y(x) = g(x). The solution of (1.5.3) is then given by
If we wnte y(x)
= - - and
z(x)
= p(X)~-1 x-I
= P(x)
=
t
X
) ) P x+ 1
+ zoP(x)
q(s)
I ( ) + ZoP(x) s=Xo P s + 1 x-I
x~l
I q(s) I1 S=Xo
pet)
t=s+l
+ Zo
(1.5.4)
x-I
I1
t=xo
pet).
Let us now consider the following nonlinear difference equation y(x)
= x~y(x) + Uk+l and Ys :5 Us for s :5 k. But, k
Yk+l - Uk+!:5
I
s=o
which is a contradiction.
[g(k •
Yn+l
:5
Yno +
Let n
n
I [ksys + Ps]' s=no
Then, n-l
:5
Yno
I1
s=no
N;o such that
+ 1, s, Ys) - g(k + 1, s, us)] :5 0
Corollary 1.6.2. (Discrete Gronwall inequality).
Yn
E
(l
+ k.) +
n-l
I
s=no
n-l
p,
I1
7'"=s+1
(l
+ kr )
E
N;o' k; ;;::: 0 and
16
Proof.
1. Preliminaries
The comparison equation is n-I
Un
= U"o + l:
5=no
= Y"o·
[ksus + Ps], U"o
This is equivalent to aUn = knun + Pn, the solution of which is n-l
= U"o I1
Un
s=no
(l
+ ks) +
n-l
n-l
l: Ps JT=s+l I1 (l + k s=no
T ) .
The proof is complete by observing that 1 + k, :::; exp(k.). We can prove the following Corollary in a similar way. Corollary 1.6.3.
Let n
E
•
N~o' k; ~ 0 and
Yn+1 :::; Pn+1 +
n
l: ksY., Y"o:::; P"o' s=no
Then, n-l
Yn:::;
n-l
n-l
r; s=no 11 (l + ks ) + s=no l: qs II (1 + k r=s+l
T )
Another form of the foregoing result is as follows. Corollary 1.6.4.
Let n
E
N~o' k;
> 0 and n
Yn+1 s Pn+1
+ l: k»; Y"o:::; 5=no
PlIo'
Then, Yn :::; r;
Proof.
+
n-]
l:
s=no
n-l
Psks
l:
T=s+l
(1 + k
T )
By setting n-I
Vn
= l:
5=no
k.y.,
V"o =
O.
(1.6.7)
17
1.6. Comparison Results
we have (1.6.8)
a to both sides of (1.6.7), a vn = knYn ::s; knPn + k nVn
Applying the operator
we get
from which by Corollary 1.6.1, it follows that n-I
Vn::s;
L
s=no
n-I
k.p,
I1
T=s+l
(1 + k.).
In view of (1.6.8), we obtain the desired estimate.
•
Corollary 1.6.5. Let k(n, s, x): N~o x N~o ~ R+ ~ R+ be monotonic nondecreasing in x and g(n, u):N~o x R+ ~ R+ be monotonic nondecreasing in u. Suppose that (1.6.9) Then
where Un is the solution of Un
=
n-I
L
k(n, s, g(s, us)),
s=no
Proof. The relation (1.6.9) can be written as Yn ::s; g(n, rn), where rn = I::~o k(n, s, Ys)' But this yields rn::s; I::~o k(n, s, g(s, rs))' Now applying Theorem 1.6.2, we obtain r« ::s; Un, which in tum shows Yn ::s; g(n, rn) ::s; g(n, un)'
•
Theorem 1.6.3. Suppose that g 1(n, u) and gi n, u) are two functions defined on N~o x R+ and nondecreasing with respect to u. Let gin, un) es Un+1 ::s; gl(n, un)' Then,
where Pn and rn are the solutions of the difference equations:
18
1. Preliminaries
Proof. Applying estimate. •
Theorem
1.6.1
twice,
we
obtain
the
needed
Theorem 1.6.4. (Discrete Bihari inequality). Suppose that h; is a nonnegative function defined on N~o' M > 0 and W is a positive strictly increasing function defined on R+. If for n ~ no, Yn S Vn where Vn = Yo + M L;:~o h, W(Ys), then, for n E N 1 , Yn S G-
1
(
G(yo)
+M
~nt:o hs)'
where G is the solution of
AG(V)=~ n W(V ) n
and N 1
= {n E N;ol M L;:~o h, S
Proof.
We have A V(n) = Mh nW(Yn)
G( Vn + 1 )
G(oo) - G(xo)}.
S
u», W( Vn). It follows that
S
G( Vn )
+ Mh n ,
from which, in view of Theorem 1.6.1, we get G( V n )
S
G(yo)
+M
n-I
L
s=no
hs '
Theorem 1.6.5. Let an, b; be two nonnegative functions defined on N~o and P; be a positive non decreasing function defined on N no . Let Yn S
r; + snt:o «s, + snt:o asC~:o bkYk).
Then,
Proof.
Since P, is positive and nondecreasing, we obtain
nIl
nIl
nf
Yn S 1 + as Ys + as( b; Yk). P; s~no P, s=no k=no Pk
We let Un
= 1+
L
n-I
s=no
Ys n-I ( n-l Yk) as- + as bk, P, s=no k=no Pk
L
L
Uno = 1,
19
1.6. Comparison Results
so that we have
This in turn yields llun ::;; an[ Un
+ knt~o bkUk
J.
By setting Xn = Un + I::~o bkuk, we obtain llXn = llu n + b.u; ::;; a.x; + b.u; ::;; (an + bn)x n, from which one gets Xn+ 1 ::;; (1 + an + bn)xn, and therefore, n-I
x n::;;
TI
(l
k=no
+ ak + bk).
Consequently, we arrive at n-I
TI
llun ::;; an
(l
k=no
+ ak + bk),
which implies Un ::;; 1 +
n-]
I
s=no
This in turn yields Yn ::;; Pnun. Theorem 1.6.6.
n-1
as
I
k=no
(1 + ak + bk).
•
Let Yj(j = 0, 1, ...) be a positive sequence satisfying Yn+I ::;; g(Yn,
~~~ n.
~~: Yj),
where g(y, z, w) is a nondecreasing function with respect to its arguments. If Yo::;; Uo and
then, for all n z::
°
Proof. The proof is by induction. The claim is true for n is true for n = k. Then, we have YHI ::;;
g(yk,
If we take Uk
:~~ Yj,
= tH
I -
:~: Yj)
::;;
g(
Uk,
~~ Uj,
=
O. Suppose it
:~: Uj) = Uk+I'
•
tk = llt b to = 0, the comparison equation becomes lltk = g(Mk, tk, tk- I).
20
1. Preliminaries
Theorem 1.6.7. inequality
Let Yi (i
= 0,1, .. '.)
be a positive sequence satisfying the
Yn+l =s g(Yn, Yn-l' ..• ,Yn-d, where g is nondecreasing with respect to its arguments. Then,
where Un is the solution of j = 0, I, ... , k:
Proof. Suppose that the conclusion is not true. Then there exists an index m =:: k such that Ym+l > Um+l and Yj =s uj, j =s m. It follows that g(Ym, Ym-l' ... ,Ym-k) =:: Ym+l > Um+ 1
which is a contradiction.
= g( Um' Um- I, ... , Um-k),
•
The next theorem deals with the higher difference of the unknown function. Theorem 1.6.8. Let Yi, Pi, qi be nonnegative sequences defined on N; and hij be a nonnegative sequence defined on N+ x N+. Suppose that, for k =:: 0, Ji kUn =s Pn + qn
k
n-l
I L
hjsJijus'
(1.6.10)
j~Os=O
Then, Jikun=spn+qn where
n-l
n-l
s=O
7=s+1
L s
(l
+ o/T)'
and from (1.6.12), it follows that D..kU n ::; v:
+ q.;
n-l
n-l
5=0
l'"=s+1
L 4>s Il o + o/T)'
•
22
1. Preliminaries
1.7. Problems 1.1.
Let Yl = -2, Yz = -2, Y3 = 0, Y4 = 4. Compute Y5 supposing that = O.
~4YI
1.2.
Show that
~ -I
1.3.
ei(ax+b)
=
ei(ax+b) ia
e - 1
+ w(x).
Using the result of problem 1.2, show that
~ cos(ax + b) = -2 sin ~sin( ax + b +~), ~ sin(ax + b) = 2 sin ~cos( ax + b + ~).
tn2(n + If.
1.4.
Show that L.;~I/ =
1.5.
Prove that L~1 cos ql
1.6.
The factorial powers can be defined for values of x that are not integers. Letting m = 1 and n = x-I, in the relation x(m+m) = x(m)(x - m)(nl, we get x(x) = x(x -1)(X-l). Setting y(x) = X(X), the previous expression can be written as y(x) = xy(x -1). The solution of this equation is y(x) = I'(x + 1), where f(x) is the Euler function, which is defined for all real values of x except the negative integers. This allows us to define x(x) for x = O. In fact, we have 0(0) = I'(L) = 1.
n
From
x(-n)
=
•
=
sin(n
+ !)q -
2 . 1 sm"2q
sin !q
.
1 we also have (x+1)(x+2) ... (x+n)
o(-n)
=~. n!
f(x + 1) ( ) fx+l-n
1.7.
Usmg the previous result, show that x' ) =
1.8.
Show that ~ log I'(x) = log x and ~ -I log x = log I'(x) + w(x).
1.9.
Let ljJ(x)
• •
f'(x)
= f(x) ,
n
where f'(x)
1
. , . IS the derivative of I'(x), Show that
The function ljJ(x) has many interesting properties. x Among them we recall lim (ljJ(x) -log(x» = o. ~ljJ(x) = -.
.x-e cc
23
1.7. Problems
1.10. Show that, if p(x) is a degree k polynomial, it can be written in the (i)
form p(x)
=
L~=o ~llip(O), which is the Newton form. /.
1.11. Show that for
q, n
N~, L;~; (-l)j( n rr] q.)
E
the Sterling formula ( 1.12. Show that for q, I, n
=
(qn-1 -1). (Hint: Use
q.) = (qn -rr]~) + ( n+]-l q~ 1 ).
nvi
E N~,
one has
'-l( n+j q )(J)1 = (q n-1 - 1- 1) .
q-n
S(q,l,n)==Lj=1 (-I)'
Note that S(q, 0, n)
= (q -
1) as established in the previous exercise. n -1
1.13. Verify that, if {Yn} and {zn} are two sequences, one has (a) Il(Ynzn) = Yn+lllzn + znllYn
the last formula is called summation by parts. By setting Ilzn = b; we have, ( d ) "N-I L.n~O Yn bn formula.
=
YN "N L./=o bi
- "N-I("n L.n=o L.i~O
b)A h eAb i .:.J.Yn, known as t eI
1.14. Let f(x) E COO[a, b]. Putting EJ(x) = f(x + h), with X,X+hE [a, b], prove that E; = e''" = 1+ Il h , where Ilh!(x) = f(x
+ h)
df(x) - f(x) and Df(x) =~.
1.15. Expand in power series the function f(x) = f(x)
= L~~o B~xn, n.
+-
e - 1
and verify that
where B n are the Bernoulli numbers.
24
1.16.
1. Preliminaries
x X X coshxj2 Prove that - x - - + - = . h j and using the result of problem e -1 2 2 sm X 2 1.13, show that B 2 k + 1 = 0, k = 1,2, ....
B 1.17. Using the result of problem 1.12, show that d- I = L~~o ~Dn-I.
n.
1.18. (Euler-McLaurin formula).
From the previous results, show that
D-1f(x) = ff(X) dx = d -If(x) -
f
B n D n - 1f(x). n=1 n!
This formula can be used either to approximate an integral or to approximate a sum using an integral. 1.19.
Using the result of problem 1.14 prove that D- 1 = d - I + 1/- f2d + ~d2 - ... , from which one obtains the Newton integration formula f f(x) dx
= d -If(x) + 1f(x) -
f2df(x)
+ ~d2f(x) ....
1.20. Show that 1[ 1 2 1 3 ] 1 ~ idi D=- d--d +-d ... =- '- (-0--:-. h 2 3 h ;=1 I
This formula can be used to approximate numerically the derivative of a function. 1.21. (Horner rule).
Find the solution of the difference equation Yn
= ZYn-1 + bn, + Yn
Yo = boo
+ 1.
1.22.
Find the solution of Yn+1
1.23.
Solve the Clairaut equation Y n = 0 there are two solutions.
1.24.
Solve the following nonlinear equation
1.25.
In many applications (for example in computing the cost of Fast Fourier transform algorithm) one meets the following equation C(n) =
2C(~) + fen),
C(2)
= n =
ndy - ¢(dy) and verify that for
= 2.
Yn+l(l
+ Yn) =
Yn'
Transform this equation into a
linear one and find the solution. 1.26. The Newton method to compute the square root of a positive number
Areduces to the difference equation
Zn+1
= !(Zn +
A). Transform
Zn 2 this equation to a linear one and find the solution.
25
1.8. Notes
1.27. Solve the following nonlinear equations (A) -Yn+1
= Y~,
(B) Yn+1
= 2Yn(l
Zo -
- Yn),
(C) Yn+1 = 1
1.28. Find the first integral of Yn+2
= Yn
!y~
- Yn
(l + YnYn+1) 1+ 2 Yn+1
1.29. Suppose that c ~ 0; k; ~ 0, k; < e for n ~ no and Yn ~ c + I;=no ksYs' Show that Yn
~
(1
c n-I k, ) . --exp --Is=n 1-8 l-e 0
1.30. Let be h > 0, M > 0, 5> 0 and 0< Uo:S; 5. Find a bound for
Un
~ h
1/2
n-I M Ij=o (
1 .)1 Uj n - ] /2
+ 8,
Un'
1.31. Suppose that c ~ 0, k n ~ 0 for n ~ no and Yn that Yn ~ c exp(I:=n+1 k.). 1.32. Study the following inequality Y~+I and a < 1.
:s;
ay~
:s;
c + I:=n+1 ksYs' Show
+ bn(Yn + Yn+l) for
a
> 1
1.8. Notes Most of the contents of Sections 1.1 to 1.4 may be found in many classical books on difference equations, such as Jordan [83], Milne and Thomson [116], Fort [49], Brand [16] and Miller [114]. Propositions 1.3.4 and 1.3.5 have been adapted from Agarwal [5]. Condition 1.4.8 can be found in Schoenberg [153]. Theorem 1.6.1 and many other comparison results on difference inequalities are taken from Sugiyama [163]. Theorem 1.6.2 was established by Maslovskaya [13]. Theorems 1.6.3, 1.6.4 and 1.6.5 are due to Pachpatte [133, 134, 135]. Theorem 1.6.6 is adapted from a result of Rheinboldt [147]. Theorem 1.6.7 is essentially new. Theorem 1.6.8 is due to Agarwal and Thandapani [1]. A generalization of this theorem can be found in Agarwal and Thandapani [1,2]. For allied results on difference inequalities, see Popenda [142, 143], Mate' and Nevai [101] and Patula [139].
CHAPTER 2
2.0.
Linear Difference Equations
Introduction
This chapter investigates the essential techniques employed in the treatment of linear difference equations. We begin Section 2.1 with the fundamental theory of linear difference equations and develop the method of variation of constants in Section 2.2. We then specialize our discussions to linear difference equations with constant coefficients, since this is an important class in itself. We exhibit, in Sections 2.3 to 2.5, methods of obtaining solutions of such equations by the use of difference operators as well as generating functions that offer elegant methods of solving difference equations. In Section 2.6, we present theory of stability for difference equations with constant coefficients and discuss in Section 2.7 linear multistep methods as an application of the theory of absolute stability. Section 2.8 considers the scalar boundary value problem that is needed later on. Section 2.9 completes the treatment by filling in the gaps with interesting problems.
2.1.
Fundamental Theory
Since it is not essential to specify the properties of functions with respect to x E C, we shall use in this section N+ or as definition set. We shall denote a sequence by {Yn}, which is the set of all values of the function Y on N~o'
»;
27
28
Definition 2.1.1. Let po(n) = 1, Pl(n), ... ,Pk(n), gn be k defined on N;o' An equation of the form Yn+k
+ Pl(n)Yn+k-l + ... + Pk(n)Yn = gn
2.
Introduction
+2
functions (2.1.1)
is called a linear difference equation of order k, provided that Pk(n)
~
o.
With (2.1.1), the following k initial conditions
Yno+l
=
C2,"" Yno+k-l
=
(2.1.2)
Ck
are associated, where c, are real or complex constants. For convenience, we shall state the following theorem, which is a special case of Theorem 1.5.1. Theorem 2.1.1. The equation (2.1.1) with the initial conditions (2.1.2) has a unique solution. • We shall denote by y(n, no, c) the solution of (2.1.1) and (2.1.2), where k C = (c\, C2," ., Ck) E C . Thus, we have
= 0, 1, ... , k -
j
Definition 2.1.2. If for every n is said to be homogeneous.
E
N;o' g(n)
1.
= 0, then the equation (2.1.1)
Introducing the operator L defined by
LYn =
k
L Pi(n)Yn+k-i i=O
(2.1.3)
equation (2.1.1) assumes the form
LYn = g(n)
(2.1.4)
and the homogeneous one becomes (2.1.5)
LYn = O. It is easy to verify that L is linear, since
Li ay;
+ {3zn)
=
al.y;
+ f3Lzn ,
when a, f3 E C and {Yn}, {z.} are two sequences. Let S be the space of solutions of (2.1.5). Because of linearity of L, we have the following result. Lemma 2.1.1. Any linear combination of elements of S lies in S. Let y(n, no, E 1 ) , ••• ,y(n, no, E k) be solutions of (2.1.5), where E1
= (1,0, ... ,0),
E2
= (0, 1,0, ... ,0), ... , E k = (0,0, ... ,0,1),
and therefore, Ly(n, no, E i) = 0,
i
= 1,2, ... , k.
(2.1.6)
29
2.1. Fundamental Theory
Lemma 2.1.2. Any other solution y(n, no, c) of (2.1.5) can be expressed as a linear combination ofy(n, no, E;), i = 1,2, ... , k.
Proof. If y(n, no, c) is the solution with initial conditions then the sum
Cl> C2,""
Ck,
k
I
z; =
Cjy(n, no, E;}
(2.1.7)
i=l
is a solution of (2.1.5) by Lemma 2.1.1 and also satisfies the same initial conditions z"" = C\, z",,+\ = C2,' " , Z",,+k-\ = Ck' By Theorem 2.1.1 the two solutions must coincide. • Let us now take k functions};( n) defined on N~o and define the matrix f\(n) ft(n + 1)
f2(n)
fin + 1)
K(n) =
Definition 2.1.3. The function };(n), i dent if for all n, k
I
i=l
=
1,2, ... , k are linearly indepen-
O!j};(n) = 0
(2.1.8)
implies a, = 0, i = 1,2, ... , k. Theorem 2.1.2. A sufficient conditionfor the functions fiin), i = 1,2, ... , k, to be linearly independent is that there exists an n ~ no such that det K(n) ;1= O.
Proof.
If (2.1.8) holds, then k
I
O!j};(n) = 0,
j=\
k
I
i=1
k
I
i=l
O!d;(n + 1) = 0,
(2.1.9) O!j};(n + k - 1) = O.
30
2.
Introduction
This linear homogeneous system of k equations in k unknowns has the coefficient matrix K (n). Thus, iffor n = ii, det K (n) ;c 0, the unique solution of the system is a, = 0, i =; 1, .. , k, and the functions are linearly independent. • Theorem 2.1.3. If j;(n), i = 1,2, ... , k; are solutions of (2.1.5), then det K (n) ;c 0, for n 2:: no, provided that det K (no) ;c O. Proof.
From the definition of the matrix k(n), we get
1)
det Ktn., + 1) = det
(
1) ...
f l( no + fl(nO + 2)
f2(nO + f2(n O+ 2)
h(no + k)
f2(n O+ k)
...
fl(n O+
. ..
1))
fk(nO + A(no + 2)
...
A(no + k)
1)
, A(no + 1)
fl(n O+ 2)
A(n o + 1)
fk(n O+ k-1) fk(nO)
fl(n O+ k-1) fl(nO) = (-1)kpk(no) det K(no).
Since Pk(nO) ;c 0, it follows that det K(no + 1) ;c O. In general, for every n E N~o., we have det K(n)
= (-1)kpk(n) det
K(n -1),
and thus by induction it is now easy to see that det K(n) ;c O. Corollary 2.1.1. independent. Proof.
The solutions y(n, no, EJ, i
In this case, det K (no)
=
(2.1.10)
•
1,2, ... , k, are linearly
= 1, since
K(no) = ( ; :
o
~... ~)..
". ". 0 ... "·0".1
In view of Corollary 2.1.1, the base y(n, no, EJ, i = 1,2, ... , k, is said to be a canonical base. Using Lemma 2.1.2 and Corollary 2.1.1, we have the following result. Theorem 2.1.4. dimension k.
The space S of solutions of (2.1.5) is a vector space of
2.1.
31
Fundamental Theory
If a, E c', i = 1,2, ... , k, are linearly independent, the set of solutions y(n, no, aJ can also be used as a base of the space S. In fact, in this case K(n o) = (a], ... , ak) where a, is the ith column of the matrix K(no) and because they are linearly independent, it follows that det K (n) ,e. 0 for all n E N;o' The matrix K (n) is called the matrix of Casorati and it plays in the theory of difference equations the same role as the Wronskian matrix in the theory of linear differential equations. Definition 2.1.4. Given k linearly independent solutions of (2.1.5), any linear combination of them is said to be a general solution of (2.1.5). The term, general, means that such a solution can satisfy any set of initial conditions.
Lemma 2.1.3. The difference between two solutions Yn and Yn of (2.1.4) satisfies (2.1.5). Proof.
o.
•
One has LYn
= gn
and LYn
= gn,
which implies L(Yn - Yn) =
Theorem 2.1.5. Let y(n, no, aJ, i = 1,2, ... ,k, be k linearly independent solutions of (2.1.5) and Yn be a solution of (2.1.4). Then, any other solution of (2.1.4) can be written as Yn = Yn +
k
I
a.yt n, no, aJ.
(2.1.11)
i=1
Proof. From Lemma 2.1.3, one has that Yn - Yn E S and therefore, it can • be expressed as a linear combination of y(n, no, a i ) .
The foregoing theorem also means that the general solution of (2.1.4) is obtained by adding to the general solution of (2.1.5) any solution of (2.1.4). Associated with the operator L, one defines the adjoint operator L * by L*Yn = I~=oPj(n + i)Yn+i and the adjoint equations L*Yn = 0,
(2.1.12)
and (2.1.13) There are some interesting properties connecting the solutions of the equations and its adjoint. We shall consider, however, because of its use
32
2. Introduction
in Numerical Analysis, (see section 5.4) the transpose operator L T defined by
LTy" =
k
I Pi(n - k + i)y"+i i=O
(2.1.14)
and the transpose equation (2.1.15) Theorem 2.1.8. Let u; be the solution of the homogeneous equation (2.1.5) and y" the solution of (2.1.15) with YN+j = 0 for j = 1,2, ... , k and N> k: Then N
j
k-j
I u"g" = j=O I Yj i=O I pJj ,,=0 Proof.
We have N N N I u"g" = I u"L Ty" = I ,,=0
Setting j
k)Uj_i
k
I U"Pi(n - k + i)y,,+i' ,,=0 i=O
,,~o
= n + i and s = min(j, k), one has N
N+k
I u.g; = I n=O
j~O
s
Yj
I
Pi(j - k)Uj_io
Pi"'" 0
for
i
> k:
i~O
Since Yj = 0 for j > N and 'I;~OPi(j - k)uj_ i = 0 for s;::: k, the conclusion follows immediately. •
2.2.
The Method of Variation of Constants
It is possible to find a particular solution of (2.1.4) knowing the general solution of (2.1.5). This may be accomplished by the method of variation of constants. Let y(n, no, c) be a solution of (2.1.5) and y(n, no, E;), i = 1,2, ... , k, be the canonical base in the space S of solutions of (2.1.5). Then, k
y(n, no, c)
=
I
cjy(n, no, Ej).
(2.2.1)
j=l
We shall consider now the cj as functions of n with
cj(no) = cj
(2.2.2)
and require that the functions k
y(n, no, c(n»
=
I
j=l
cin)y(n, no, Ej)
(2.2.3)
33
2.2. The Method of Variation of Constants
satisfies the equation (2.1.4). From (2.2.3) it now follows that yen
+ 1, no, c(n + 1))
k
I
=
cj(n
j=!
+ 1)y(n + 1, no, Ej)
k
I
=
c/n)y(n
j=!
+ 1, no, Ej)
k
I
+
j=!
+ 1, no, Ej ).
Ilcj(n)y(n
By setting k
I
j=!
+ 1, no, Ej ) = 0,
Ilcj(n)y(n
(2.2.4)
we have
+ 1, no, c(n + 1))
y(n Similarly, for i
k
=
I
j=!
+ 1, no, Ej).
c/n)y(n
1,2, ... , k - 1, we can get
=
y(n
+ i, no, c(n + i))
k
I
=
cj(n)y(n
+ i, no, Ej),
(2.2.5)
j~!
if we set k
I
Ilcj(n)y(n
+ i, no, E)
i = 1,2, ... , k - 1.
= 0,
(2.2.6)
j~!
Therefore, in the end, we obtain y(n
+ k, no, c(n + k))
k
=
I
c/n)y(n
+ k, no, EJ
j~!
+
k
I
j=!
Ilcj(n)y(n
+ k, no, Ej).
By substituting in (2.14), there results k
I
pj(n)y(n
+k-
i, no, c(n
+k -
i))
j~O
k
=
I
j~O
k
Pi(n)
I cj(n)y(n + k j=!
i, no, Ej)
+
k
I
Ilc/n)y(n
+ k, no, Ej)
j~!
= gn·
Since y(n, no, Ej) are solutions of (2.1.5), one has k
I
j=!
Ilcj(n)y(n
+ k, no, Ej)
=
gn·
(2.2.7)
34
2. Introduction
The equations (2.2.6) and (2.2.7) form a linear system of k equations in k unknowns cj(n), whose coefficient matrix is the Casorati matrix K(n + 1). The solution is given by
IlCI(n))
Ilc~(n) = K-1(n + 1)
( Ilck(n) Denoting by Mik(n (2.2.8) becomes
(2.2.8)
gn
+ 1) the (i, k) element of the adjoint matrix of K (n + 1),
Ilc.(n) I
(0)f .
Mik(n + 1) = detK(n+1) g n,
i
= 1,2, ... , k,
(2.2.9)
from which it follows that cj(n) = 11
-I
Mik(n + 1) det K(n + 1) gn + Wj,
cj(nO)
= c;
By substituting the values of ci(n) in (2.2.3), we see that z(n, no, c) y(n, no, c(n)) satisfies (2.1.4).
2.3.
=
Linear Equations with Constant Coefficients
If in equation (2.1.1) the coefficients Pi(n) are constants with respect to n, we obtain an important class of difference equations k
I PiYn+k-i = j=O
s-
Po
=
1.
(2.3.1)
The corresponding homogeneous equation is k
I
i=O
Theorem 2.3.1.
PiYn+k-i
= O.
The equation (2.3.2) has the solution of the form Yn = z",
(2.3.2)
(2.3.3)
C, and satisfies (2.3.4).
where z
E
Proof.
Substituting (2.3.3) in (2.3.2), we have
n"L. Pi k
Z
Z k-j
i=O
= 0
from which it follows that k
I
i=O
PiZ k -
i
= O.
(2.3.4)
2.3.
35
Linear Equations with Constant Coefficients
Equation (2.3.4) is a polynomial and it has k solutions in the complex field. Furthermore, it is said to be the characteristic equation of (2.3.2), and the polynomial p(z) = I~~o PiZk-i is called the characteristic polynomial. • If the roots z}, Z2,"" Zk of p(z) are distinct, then z;, ... , z~ are linearly independent solutions of (2.3.2).
Theorem 2.3.2. z~,
Proof. It is easy to verify that in this case the Casorati determinant is proportional to the determinant of the matrix
1
1
ZI
Zk
Z2 2
zi
Z2
k-I ZI
k-I Z2
V(ZI> Z2,"" Zk) =
1
zi
(2.3.5)
k-I
Zk
which is known as Cauchy-Vandermonde matrix (or the Vandermonde matrix). Its determinant is given by det V(z}, Z2, ... , zd
=
n (z; - Zj),
(2.3.6)
i>j
which is different from zero if z, "r:. Zj for all i and j. From Theorem 2.1.4 it follows that if the roots of p(z) are distinct, any solution of (2.3.2) can be expressed in the form k
Yn =
I
;=1
(2.3.7)
ciz7·
When p(z) has multiple roots, the solutions z7 corresponding to distinct roots are linearly independent. But they are not enough to form a base in • S. It is possible however, to find other solutions and to form a base. Theorem 2.3.3.
Let m, be the multiplicity of the root z, of p(z). Then the
functions
(2.3.8)
ys(n) = u,(n)z:,
where Us (n) are generic polynomials in n whose degree does not exceed m s - 1, are solutions of (2.3.2) and they are linearly independent.
Proof.
If z, has multiplicity m s as a root of p(z), we have p(zs) = 0, p'(zs) = 0, ... , and p(m,-I)(zs)
= 0.
(2.3.9)
Let us look for a solution of (2.3.2) of the form Yn
= u,(n)z:.
(2.3.10)
36
2. Introduction
By substituting, one gets k
L PiUs(n + k -
i)z:-i
= O.
(2.3.11)
i~O
By the relation (1.1.4), we get
us(n
+
k- i) kf (k ~ i),~Jus(n) ]
j~O
and, from (2.3.11), we have
it
i
PiZ : -
%~
r:
(2.3.12)
=
i)tijus(n) =
jt
tijus(n)
~~ (k; i)Piz:-i (j)(
k
)
= L tijus(n)z1~.
(2.3.13)
]!
j=O
In view of (2.3.9), one has it that the terms of (2.3.13) corresponding to j = 0, 1, ... , m, -1 are zero for all functions u(n). To make the other k - m, + 1 terms equal to zero, it is necessary that tiju s(n) = 0 for m, :5, j :5, k. This can be accomplished by taking u.(n) as a polynomial of degree not greater than m s - 1. The proof that they are linearly independent is left as an exercise. •
The general solution of (2.3.2) is given by
Corollary 2.3.1. d
Yn where Aij
=L
i=1
= ajcj,
d
a juj(n)z7
=L
i=1
mj - l
aj
L
cjn jz7
j~O
d
mi-l
=L L
i=1 j=O
Aijn
iz7,
and d is the number of distinct roots.
The next theorem is useful in recognizing if a sequence Yn, n solution of a difference equation. Theorem 2.3.4. all n E N;o'
(2.3.14)
A sequence Yn, n
E
E
N;o is the
N;o satisfies the equation 2.3.2 iff, for
Yn D(Yn, Yn+1, ... , Yn+k) = det Yn+1 ( Yn+k
Yn+1 Yn+2 Yn+k+1
... ...
Yn+k) Yn+k+1 Yn+2k
=
0
and moreover, D(Yn, Yn+l, ... , Yn+k-I) '" O. Proof. Suppose that Yn satisfies 2.3.2. One has Yn+k+j = -I:;=1 PiYn+k+j-i' By substituting in the last row of D(Yn, Yn+1, ... , Yn+k), one easily obtains
2.3.
37
Linear Equations with Constant Coefficients
that D(Yn, ... , Yn+k) = O. Conversely, if this determinant is zero, one has, by developing with respect to the first row I~~o Yn+k-Ai(n) = 0, where Ai(n) are the cofactors of the k-i'" elements. If in D one substitutes to the first row, the second one, and then the other rows, one obtains determinants identically zero. By developing these determinants again with respect to the elements of the first row one obtains: k
I Yn+j+k-Ai(n) ;=0
0
=
j = 1,2, ... , k - 1.
The determinant Ak(n) is not zero by hypothesis. One has, setting _ Ai(n) Pi( n ) - Ak(n) k
Yn = -
I Pi(n)Yn+k-i, ;=0 k
Yn+1 = -
I Pi(n)Yn+J+k-b ;=0
Yn+k-I = -
I Pi(n)Yn+2k-I-i. ;=0
n
By setting n + 1 instead of n in the first relation and subtracting the second one, we have k
0= -
I ilpi(n)Yn+k+l-i ;=0
and proceeding similarly for the others, one arrives at an homogeneous system of equations k
I
i=1
ilpi(n)Yn+k+j-i = 0
j = 1,2, ... , k - 1,
whose determinant is not zero by hypothesis. It follows that the solution is ilpi(n) = 0, that is the Pi are constant with respect to n and then the conclusion follows. • Example 1.
Consider the equation Yn+1 - aYn = 0,
a E Co
(2.3.15)
The characteristic polynomial is p(z) = z - a and its unique root is z The general solution of (2.3.15) is then Yn = can.
=
a.
38
2. Introduction
Example 2.
Consider the equation
= O.
Yn+Z - Yn+1 - Yn
The characteristic polynomial is p(z) ZI -
I+VS --v-; Zz
=
=
(2.3.16)
ZZ -
1, which has roots
Z -
I-VS
--2-'
Therefore, the general solution of (2.3.16) is
_ (1 + vs)n + (1 -VS)n
Yn -
Cz
2
CI
2
'
and is known as the Fibonacci sequence. Example 3.
Consider the equation
From Example 1, it follows that the general solution of the homogeneous equation is ca ", Applying the method of variation of constants it follows that (c(n + 1) - c(n))a n+1 = gn and C(
n)
=
A-I
u.
g(n) = n~1 g(j) a
n+1
L.
j=O
a
j+\'
from which, we obtain Yn = Yoan +
n-I
I
g(j)a n- j - I.
j=O
Example 4. The following equation often arises in discretization of second order differential equations, namely,
Yn+Z - 2QYn+1
where q
E
+ Yn = In,
(2.3.17)
C. The homogeneous equation has the general solution Yn =
+ czz~, where
Z\ and Zz are the distinct roots of the second degree equation ZZ - 2qz + 1 = O. It is useful, in the applications, to write the general solution in two different forms. In the first form, the linearly independent solutions
CIZ~
( I)
Yn
n
=
n
ZZZI - ZIZZ ; Zz - Z\
(Z)
Yn
z~ - z~
= Zz -
ZI
are used, which give, as a general solution of the homogeneous equation Yn = CIy~1)
+ czY~Z).
(2.3.18)
2.4.
39
Use of Operators 4. and E
In the second case, one uses the Chebyshev polynomials (see Appendix C) Tn(q) and Un(q) as linearly independent solutions, obtaining (2.3.19) The advantage of using (2.3.18) is that the base y~l), Y~Z) is a canonical one, that is,
y&l) = 1,
Y&Z) = 0,
y~l) = 0,
Y~Z)
= 1,
from which it follows that for the initial value problem (2.3.18) we have
Yn = Yoy~l)
+ YIY~Z),
The advantage of the form (2.3.19) lies in the fact that the functions Tn(q) and U; (q) have many interesting properties that make their use especially helpful in Numerical Analysis and Approximation Theory. The solution of (2.3.17) can then be written in the following form, (I)
Yn = YOYn Example 5.
+ YIYn(Z) +
n-Z '\'
L.
j=O
I"
(Z)
Yn-j-I . J]>
Consider the equation
POYn+Z
+ PIYn+1 + PZYn
=
O.
(2.3.20)
The solution can be written in terms of the roots of the polynomial Pozz + PI Z + P: = 0 as usual. It is interesting, however, to give the solution in terms of Chebyshev and q = (PI) I/Z' Po 2 POP2 One easily verifies that pnTn(q) and pnUn(q) are solutions of (2.3.20). It follows then that Yn = clpnTn(q) + CZpnUn_l(q) is the general solution. polynomials. Suppose POP2 > 0 and let p
.2.4.
= (Pz) I/Z
Use of Operators L1 and E
The method of solving difference equations with constant coefficients becomes simple and elegant when we use the operators ~ and E. Using the operator E equations (2.3.1) and (2.3.2) can be rewritten in the form
p(E)Yn = gn, p(E)Yn
= 0,
(2.4.1) (2.4.2)
40
2. Introduction
where k
=I
p(E)
(2.4.3)
PiEk-i.
;=0
It is immediate to verify that p(E)zn
= znp(z)
and
k
p(E)
= I1
(2.4.4)
(E - z.I),
i=1
where z\> Z2,"" Zk are the zeros of p(z). If there are s distinct roots with multiplicity mj,j = 1,2, ... , s, then p(E) can be written as p(E) = rt., (E - zJ)m, and (2.4.2) becomes s
I1 (E -
zJ)m'Yn = 0,
(2.4.5)
i=1
from which it is seen that the homogeneous equation can be split into s difference equations of order mj • In fact, the commutability of the operators (E - zJ) implies the following result. Proposition 2.4.1.
The solution X n of the equation (E - z/)mjX n = 0
(2.4.6)
is a solution of (2.4.5).
• The problem simplifies further since it is possible to define the inverses of (E - zJ), i = 1, 2, ... , k.
Proposition 2.4.2. Let {Yn} be a sequence and let fez) polynomial of degree m. Then,
= L~~o a.z'
be a
(2.4.7) Proof.
By definition of fez), it follows that f(E)(zn yn)
m
=I
i=O
= z"
aiEi(zn yn) m
I
;=0
m
=I
ai(ziEi)Yn
aiZn+iEiYn
i=O
= znf(zE)Yn-
•
Definition 2.4.1. The inverse of the operator (E - zI) is the operator (E - zI)-1 such that (E - zI)(E - zI)-1 = 1.
2.4. Use of Operators
Theorem 2.4.1.
~
41
and E
Let z
E
Co Then, the inverse of E - zI is given by
(E -
av> = zn- 1a-
1z-
n.
(2.4.8)
Applying (E - zI) to both sides of (2.4.8) and using the result of Proposition 2.4.2, one gets
Proof.
(E - zI)(E - zI)-l
= (E - zI)zn-1a-1z- n = zn-1 z (E - I)a-1zn
Corollary 2.4.1.
For m
= 1,2, ... ,
one has
(2.4.9)
The equation (2.4.9) allows us to find very easily the solutions of (2.4.6) and then of (2.4.1). In fact, from (2.4.6) and (2.4.9) we have But, we know that a-mj. 0 = qj(n), where qj(n) is a polynomial of degree less than mj' Hence, the solution x, is given by x, = z'j-mjqj(n), and this can be repeated for j = 1,2, ... , s. Usually, because mj is independent of n, one prefers to consider the previous solution multiplied by z'j'j. The solutions corresponding to the multiple root Zj are then z'jqj(n), and hence, the general solution of (2.4.1) is Yn = "2.;=1 ajqj(n)z'j, which is equivalent to (2.3.14). In general, to get a solution of the nonhomogeneous equation (2.4.1), one can proceed as described in the previous section by applying the method of variation of constants. Usually this way of proceeding is too long and in some cases can be avoided using the definition of p-1(E). Proposition 2.4.3. f(z) ¥- O. Then
Let f( z) be a polynomial of degree k and z E C with
f Proof.
-1
(E)z
n
z"
= f(z)'
(2.4.10)
By applying f(E) to both sides of (2.4.10), one obtains f(E)f-1(E)zn = f(E) f~;) = zn.
•
42
2.
Introduction
Proposition 2.4.4. Let f(z) be a polynomial of degree k and ZI E C be a root of multiplicity m. Then, setting g(z) = (z - zl)-mf(z), one has n-m (m) f-I(E)z~ = Zl n (2.4.11) g(zl)m!
Proof.
By applying f(E) to both sides and using (2.4.7), one obtains
f( E )f ( E
-I) n
ZI =
z~-mf(ZIE)n(m)
g(z\)m! z~g(zIE)' 1
g(zl)
n"'g_..:...(z...:...IE----'--')(_E_-_I'--)m_n_(m_) g(z\)m!
= ZI-
n
= Zl'
•
Proposition 2.4.5. Let f( z) be a polynomial ofdegree k and Yn be a sequence. • Then for every n E N, we havef-I(E)znYn = znf-l(zE)Yn. These results can be used to obtain particular solutions of the equation p(E)Yn = g(n). Let us consider the most frequent cases: (a) g(n)
=
g constant. If p(l)
0 from (2.4.10) one obtains
;i:
y = p-I(E)g = gp-I(E) .1 n = --L = _g_ p(l)
n
~ L..
i=O
(b) g(n) = I:=I aiz7 with p(za _
Yn
=P
-1
(E)
;i:
O. From (2.4.10) one has
L a.z, = i~1 L a.p S
n
S
i=1
Pi
-1
n
(E)Zi
=
z7 L ap(za i-' n
i=\
(c) Same as in (b), but Zj is a root of p(z) of multiplicity m. From (2.4.10) and (2.4.11) we obtain a.z n zn-mn(m) Yn = L _(_I_I + aj J , jyt.J g(zj)m! f(z) where g(z) = ( )m
za
Z -
zJ
(d) g(n) = e in a • This case can be treated as in (b) or (c) by putting z = e ia • (e) g(n) = cos na, g(n) = sin na. We can proceed as in case (d) taking the real or imaginary part.
2.5.
Method of Generating Functions
The method of generating functions is another elegant method for solving linear difference equations with constant coefficients. Its importance is growing in discrete mathematics.
43
2.5. Method of Generating Functions
Definition 2.5.1. Given a sequence {Yn}, we shall call a formal series generated by it the expression 00
y=
L Yi X i,
(2.5.1)
i=O
where x is a symbol. Only in the case where x will be a complex value, the problem of convergence of (2.5.1) will arise. The formal series are often called generating functions of {Yn}' In the set of all formal series, we can define operations that make such a set algebraically similar to the set of rational numbers. Definition 2.5.2.
Given two formal series Y and Z, their sum is defined by Y
+Z =
00
L (Yn + zn)x n.
n~O
Definition 2.5.3.
The product of two formal series Y and Z is given by (2.5.2)
where en
=
n
n
L Yizn-i = L i=O
ZiYn-i'
(2.5.3)
i~O
We list some simple properties of formal series: (i) (ii) (iii) (iv) (v) (vi)
The product of two formal series is commutative; Given three formal series Y, Z, T we have (Y + Z) T = IT + ZT; The unit element with respect to the product is the formal series I = 1 + Ox + Ox2 + ... ; The zero element is the formal series 0 + Ox + Ox2 + ... ; The set of all the formal series is an integral domain; I Let Y = 0 YiXi. If Yo r6 0 then there exists the formal series yI such that y- Y = 1.
2:::
The polynomials are particular formal series with a finite number of terms. Consider now the linear difference equation k
L PiYn+k-i = 0, i=O
with Po = 1,
(2.5.4)
and we shall associate with it the two formal series
= Po + PIX + y = Yo + Ylx + P
+ p~\ and .
(2.5.5)
44
2. Introduction
P is different from the characteristic polynomial. In fact, one has P
= Xkp(~) == p(x),
(2.5.6)
where p(z) is the characteristic polynomial. The product Q of the two series is (2.5.7) where n
I PiYn-i' ;=0
qn =
(2.5.8)
In view of (2.5.4) it is easy to see that 0 = qk = qk+1 = ... , which means that Q is a formal series with a finite number of terms. Moreover, because P is invertible (Po = 1), one has
y
= P-1Q.
(2.5.9)
If we consider the symbol x as an element z of the complex plane, then (2.5.9) gives the values of Y as ratio of two polynomials Y
= q(z) = p(z)
where
q(z)
(2.5.10)
p (; )'
Zk
p(D is the characteristic polynomial and q(z) = I7:~ q.z'. The roots
of p(z) are Z;-I, where z, are the roots of p(z). It is known in the theory of complex variables that every expression like (2.5.10) is equal to the sum of the principal parts of its poles. The poles of (2.5.10) are the roots..!.. of the denominators, and therefore, z, s
00
'" YnZ n Y =_ L...
_
n=O
=
i=lj=1
mi
s
I I
m,
'\' ' " L... L...
-
aij(-1)
aij( z - z,-I)-j
(2.5.11)
jzl(l-
ZiZ)-j,
;=1 j=1
where s is the number of distinct roots of p(z) and mj their multiplicity. The coefficients aij are the coefficients in the Laurent series of (2.5.10). For Iz;zl < 1, i = 1,2, ... , k, (l - z;z)-j can be expressed as (1 - z.z)
-j
=
I
00
(
n=O
n
z; z
n
)j = I
00
n=O
(
n +.] n
1) z z , n j
n
(2.5.12)
45
2.5. Method of Generating Functions
and substituting in (2.5.10, we get
I
I
Ynz n =
n=O
t I aij(-O-i(n +
z"
i=1 i=1
n=O
If we write q;(n)
=
Im,
i=1
j -1)Z?+i. n
(n + j - 1) (-I)1z?, ..
au
n
we obtain Yn =
s
I
i=1
(2.5.13)
qi(n)z?,
which is equivalent to (2.3.14). Theorem 2.5.1. Suppose that the roots of the characteristic polynomial p (z) are inside the unit disk ofthe complex plane. Then the formal series Y converges inside the unit disk. Proof. The polynomial p(z) has zeros outside the unit disk and q(z)/p(z) has no poles in it. Y must then coincide with the Taylor series in the unit disk. •
Theorem 2.5.2. lfthe characteristic polynomial p(z) has no roots outside the unit disk and those on the unit circle are simple, then the coefficients Yn of Y are bounded. Proof. From (2.5.13), it follows that q;(n) corresponding to Iz;1 = 1 are constants with respect to n. The method of generating functions can also be used to obtain solutions of nonhomogeneous equation (2.1.4). One can proceed as before with the difference that qk, qk+h ... , are not zero. In fact from (2.5.8) and (2.1.4) one has qk = I~=OPiYk+i = go and, in general, for n = 1,2, ... k
qn+k
=I
;=0
PiYn+k-;
= gn,
(2.5.14)
since Pi = 0 for i> k. The series (2.5.7) can be written as k-I n co Q = I zn I PiYn-i + Zk I gnzn =
n=O
;=0
QI(Z)
+ ZkQ2(Z),
n=O
(2.5.15)
where as (2.5.9) becomes
I
;=0
YiZi = QI(Z) ~ ZkQ2(Z). p(z)
•
(2.5.16)
46
2.
Introduction
The polynomial QI (z ) depends only on the initial values Yo, YI, ... , Yk-I , while Q2(Z) is a formal series defined by the sequence {gn}. Proceeding as in (2.5.10), (2.5.11) and (2.5.12) one obtains the solution {Yn}. This procedure can be further simplified by considering that inside the region of convergence, it represents a functionf(z), which is said to be the transformed function of the sequence {Yn}. For example, in the unit disk, the function (1- Z)-I is the transformed function of the constant sequence {l}, since (1- Z)-I =
00 I z'.
i~O
On Table 1, transformed functions of some important sequences are given. Now suppose that Qiz) is the transformed function of {gn}. After doing the necessary algebraic operations, one obtains from (2.5.16), o y.z' = G(z), where G(z) is the function resulting in the right-hand side of (2.5.16). By expanding G(z) in Taylor series and equating the coefficients of the powers of the same orders on both sides, we arrive at the solution {Yn}.
I:
Example. Consider the QI(Z) = 1, Q2(Z) =
equation Yn+1 + Yn = -(n + 1), Yo = 1. Here 1
-I:=o (n + l Iz" = ( l-z )2 and
z
G(z)
1---~
=
(1- Z)2 1+ z
1 - 3z + Z2 (1 + z)(1 - Z)2
From Table 2.1, we find that _1_ l+z (1 -
z
,,00 z? =
"'-n=O
n
nz.
f
There ore,
5
1
1
1 1 z = 4 1 + z - 4 1 - z - 2 (1 - z?'
= I:=o (_1)n zn, _1_ = I:~o z", and l-z
,,00 ,,00 [5( ) 1 1 ] "'-n=oYn z = "'-n~O 4 -1 n - 4- 2n z, n
n
. h we obtain . Y = -5()n 1 -1 - -1 - -n f rom whic n 4 4 2' In some applications, especially in system theory, instead of generating functions defined in (2.5.1), generating functions called Z transform defined
by X(z) therefore,
=
I:=o Yn z- n
are used. It is evident that X(z)
=
yC)
and
2.6.
Stability of Solutions
47 TABLE
1.
Domain of Convergence
Yn
f(z)
n (n + m)(m) n(m) nm kn
(1 - Z)-I Z(1-Z)-2 m!(1 - z )-m-l m Izm(1 - z)-m-I zPm(z)(I- Z)-n-I(*) (l-kz)-1 m!(l- kz)-m-I (1 - eaz)-I
(n
+ m)(m)k n e an
Izi < 1
Izi < k-
Izi < e:" Izl < k-
l-kzcosa
k" cos an
1
1 - 2kz cos a + k 2z 2 kz sin a
k" sin an
1 - 2kz cos a
Bn
z
nl
e' - 1
(;)
+ k 2z2
Izi < 21r Izi < 1
zm(1 - z)-m-I
{I +
C) (-I)'C)
Izi < 1
z)k
Izi < 1 Ixl:5 1, Izi < 1
(1 - Z)k
(1 - 2xz + Z2)-1 (1 - xz)(1- 2xz + Z2)-1
Un(x) (**) Tn(x)
I
(*) Pm(Z) is a polynomial of degree m satisfying the recurrence relationpm+l(z) = (mz + l)Pm(z) + z(I - z)p;"(z), PI = J. (**) Tn(x) and Un(x) are the Chebyshev polynomials (see Appendix C).
where
Q2(;) is the Z transform of {gn} and
p(z) is the characteristic
polynomial. Using the table of Z transforms, everything goes similarly as before.
2.6.
Stability of Solutions
The stability problem will be studied in a more general setting in a later chapter. In this section we shall consider only the stability problem for linear difference equations, which is very important in applications.
Definition 2.6.1. The solution Yn of (2.1.4) is said to be stable if, for any other solution Yn of (2.1.4), the following difference is bounded:
In
= Yn
-
Yn,
n E N~o'
(2.6.1)
48
2. Introduction
Definition 2.6.2. The solution Yn of (2.1.4) is said to be asymptotically stable if for any other solution Yn of (2.1.4), one has limn~oo In = O. Definition 2.6.3. The solution Yn of (2.1.4) is said to be unstable if it is not stable. From Lemma 2.1.3 it follows that the difference In satisfies the homogeneous equations (2.1.5). In the case oflinear equations with constant coefficients we have the following results.
Therorem 2.6.1. The solution Yn of (2.3.1) is asymptotically stable if the roots of the characteristic polynomial are within the unit circle in the complex plane. Proof.
From (2.3.14) we have s
lim IYn - Ynl = lim
n ..... co
mi-l
L L
n_OO i=1 j=O
IAijlnilz?l·
If IZil < 1 one has limn~oo IYn - Ynl = 0 and vice versa.
•
Theorem 2.6.2. The solution Yn of (2.3.1) is stable if the module of the roots of the characteristic polynomial is less than or equal to 1 and that those with modules equal to 1 are simple roots. Proof. From (2.3.14), it is evident that the terms coming from roots with modules less than 1 gives a vanishing contribution for n ~ 00, while the terms coming from roots with unit modules, give a bounded contribution to In' since j = O. • It can happen that for some initial conditions, the solution remains bounded even in presence of multiple roots on the unit circle, as shown in the next example.
Example 1.
Consider the equation
Yn+2 - 2Yn+l + Yn
=
0,
Yo
= Yi = c.
This equation admits the solution Yn = c. Often in applications it becomes necessary to study the stability of a constant solution which exists, as we have seen, if gn is constant.
2.7.
49
Absolute Stability
Example 2.
Consider the equation
Yn+2 - Yn+l
+ ~Yn = 2.
We have p(z) = (z - !)2 and all solutions will be asymptotically stable. In particular the constant solution Y = 8 is asymptotically stable. In fact, the general solution is given by Yn = (c 1 + c2n)Tn + 8 and limn~oo (Yn - 8) = O. From definitions 2.6.1, 2.6.2 and 2.6.3, we see that the properties of stability and instability are usually referred with respect to a particular solution Yn' In the case where all solutions tend to a unique solution Yn as n ~ 00, it is often said (especially in numerical analysis) that the difference equation itself (or the numerical method represented by it) is asymptotically stable. Moreover, in some branches of applications, a special terminology is used that is becoming more and more popular and it is worthwhile to mention it. Definition 2.6.4. A polynomial with roots within the unit disk in the complex plane is called a Schur polynomial. Definition 2.6.5. A polynomial with roots in the unit disk in the complex plane with only simple roots on the boundary is called a Von Neumann polynomial.
Using this terminology, the Theorems 2.6.1 and 2.6.2 can be restated as follows. Theorem 2.6.1. The solution Yn is asymptotically stable, if the characteristic polynomial is a Schur polynomial. Theorem 2.6.2. The solution Yn is stable if the characteristic polynomial is a Von Neumann polynomial.
2.7.
Absolute Stability
One of the main application of linear difference equations is the study of discretization methods for differential equations. The difference equations, as we have seen, can be solved recursively. This is not possible for the differential equations and these are usually solved approximately using difference equations that satisfy some suitable conditions. Let us consider
Y'
= f(t, y),
y(to) = Yo,
(2.7.1)
2.
50
where t
Introduction
[to, T) and suppose that this continuous problem has a unique T solution yet). Let h > and t, to + ih with i 0, 1, ... , N E
°
=
=
=-,;.
Let the discrete problem approximating (2.7.1) be denoted by
r; (Yn , Yn+1,""
= 0, (2.7.2) 1, n + k ~ N. We sup-
Yn+k>fn,fn,··. ,fn+k)
where YI = y(tJ + O(h ) , q> 1 and i = 0, 1, ... , k pose that (2.7.2) has a unique solution Yn' As the discrete problem is represented by a difference equation of order k, it needs k initial conditions, only one of which is given from the continuous problem. The others are approximately found in some way. q
Definition 2.7.1. The problem (2.7.2) is said to be consistent with the problem (2.7.1) if
Fh(y(tn), y(tn+1), ... ,y(tn+d'/( tn, y(tn)), ... ,/(tn+k> y(tn+k))) "" Tn
with p
2::
= 0(h P+ 1 )
(2.7.3)
1.
The quantity Tn is called the truncation error. The equation (2.7.3) can be considered as a perturbation of (2.7.2). Definition 2.7.2. The discrete problem (2.7.2) is said to be convergent to the problem (2.7.1) if the solution Yn of (2.7.2) tends to the solution yet) of (2.7.1) for n ~ 00, and tn - to = nh ~ T.
Since the solution of the continuous problem satisfies (2.7.3), which is a perturbation of (2.7.2), the convergence will occur when (2.7.2) will be insensitive to such a perturbation, that is, when (2.7.2) is stable under perturbation. As a consequence, the consistency is not enough to guarantee the convergence. We shall study the problem in some detail for the main class of methods called linear multistep methods (LMF). These methods are obtained when F h is linear in its arguments, namely, k
I aiYn+i ;=0
I e.c.; = 0, ;=0 k
h
(2.7.4)
with (ik = 1 and coefficients ai, f3i are real numbers. Using the shift operator E and the two polynomials p and a given by k
p(z) =
u(z) =
I
j=O k
I
;=0
a.z',
(2.7.5)
f3i z i,
(2.7.6)
2.7.
SI
Absolute Stability
equation (2.7.4) can be written as p(E)Yn - ha(E)f" = O.
(2.7.7)
The two polynomials p(z) and a(z) characterize the method (2.7.2) uniquely and one often refers to them as (p, a) method. The relation (2.7.3) becomes (2.7.8) Theorem 2.7.1. Suppose that f is smooth enough. Then the quantity Tn is infinitesimal of order 2 with respect to h if the following two conditions are verified: p(q)
=
k
I
CI';
;=0
= 0,
(2.7.9)
and p'(1) - 0-(1)
=
k
k
I
iCl'; -
;~O
I
f3;
= o.
(2.7.10)
;~O
(For the proof of Theorem 2.7.1, see problem 2.18). The conditions (2.7.9) and (2.7.10) are said to be consistency conditions. Iff is nonlinear, then the study of stability of (2.7.7) is in general difficult. Usually, one studies the behavior of solutions of (2.7.7) for particular linear functions f, which are called test functions. The most used test functions are
f(y)
=
0,
(2.7.11)
Re A ::; O.
(2.7.12)
and
f(y)
= Ay,
The use of test function (2.7.11) is justified by considering that in (2.7.7) the values of f are multiplied by h and then, in the limit as h ~ 0, the contribution to solutions of the terms containing fn+i can be disregarded. Also, one sees that the methods give good results when applied to the simple equation y' = O. The use of test function (2.7.12) is justified by considering that in the neighborhood of an asymptotically stable solution of (2.7.7), the first order approximation theorem says that the behavior of any solution is established by the linear part that looks like (2.7.12). Let us first consider the test equation (2.7.11). Then (2.7.7) becomes p(E)Yn
= O.
(2.7.13)
52
2. Introduction
Definition 2.7.3. The method (p, u) is said to be O-stable if the solution Yn = 0, n E N+ of (2.7.13) is stable. As a simple consequence of Theorem 2.6.1, we have the following result.
Theorem 2.7.2. polynomial.
The method (p, u) is O-stable if p(z) is a Von-Neumann
Theorem 2.7.3.
The method (p, o ) is convergent in the finite interval (0, T) O-stable.
iff it is consistent and
Proof. Let us write f(ln, y(tn» - fn = Cnen, where en = y(tn) - Yn' Then, subtracting (2.7.4) from (2.7.8), one obtains the error equation
with q
~
1,
j
= 0, 1, ... , q -
1.
The necessity part of the proof will be left as an exercise (see problem 2.20 and 2.21). Suppose now that the method is O-stable and consistent, we shall prove the convergence. We will use the formal series method, from which we get
where QJ(z)
=
k-J
L
qP)Zi,
i~O
co
Qz(z)
=L
qj2)Zi,
i~O
co
Qiz) =
L
'Yn z n.
n~O
By Theorem 2.5.2 and by O-stability we see that 'Yn are bounded. By
2.7.
S3
Absolute Stability
multiplying the formal series we get 00
QI(Z)Q3(Z) =
I
n=O
8~l)zn,
=I
f q~I)'V/n-".
=
8(1) n
l.J
00
ZkQw(Z)Q3(Z)
n=O
8~2)zn,
8(2) n
By equating the coefficients, we have 18~llkl + 18~zlkl. But 18~llkl
k-I
s
J
i=O
k-I
= min(n, k -l),
O, n = 0, 1, ... , k - 1
=
nr-k:
(Z)
i~O qi
{
en+k =
'Yn-k-i>
n
===
k.
+ 8~zlk and len+kl:5
8~llk
i
k-I
j~O
i~O
i
:5 r I /qP)I:5 r i=LO II aA-jl :5 r H I Ij lajl = r HA i~O
where H
and
= 05;;sk-) max leil,
k-I
i
k-I
I I lajl = I
A =
i=Oj~O
it
(k -
i~O
it
18~Zlkl :5 r Iq\2)1 :5 r(to ITil + h IU(E)Cieil)
r( On + h :5 r( On + hL :5
with
itO
to Il3jIICi+jllei+jl)
itO
to Il3jllei+jl),
On = Ln~o hi and L = 0OO
case p
=
i t h e second case p 2,While 1 e In
2.12. Solve the difference equation 2.13.
Zn+!
+ z;
1+ v'5. = --
2
=
-(n
+ 1).
(Bernoulli Method). The solution of the linear difference equation has been obtained considering the polynomial p(z). It also happens that for finding the roots of a polynomial it is useful to consider a linear difference. equation. In fact, supposing that the roots ZI, Z2,"" Zk are all simple and that IZ11 > IZ21 > ... from (2.3.7) we have
from which it follows · Yn+1 Il m - - = n->OO
Yn
ZI'
Solving then the difference equation recursively, the ratio of two successive values of the solution will give an approximation of the first root. (a) How can we approximate the root of minimum modulus? (b) What happens if ZI = Z2? (c) How do we choose the initial conditions in order to avoid the effect of multiple zeros of the characteristic polynomial? 2.14 Suppose one has to perform the sum Sn = l:7~1 a, by using the following algorithm: So = 0, Si+1 = S, + ai+I' If one performs the sum not using the real numbers, but approximation of them (floating point numbers), that is, instead of the number a = m' 10 q , where 0.1 ~ m < 1, one uses the number ii = m' 10 q , where 0.1 ~ m < 1 but m has only t digits, this implies that la - iii < 1m - mj . 10q :S 10 q - ,. Study the behavior of the errors, considering that Sj = (Si-1 + ii;)(l + 10-').
60
2.15.
2.
If
An
-2
1
1
-2
o
O. 1.
O.
=
··0 ...1
o
··0
-2 nXn
·1
Show that D; = det An satisfies the equation D n+2 + 2Dn+1 and that D; = (_1)n(n + 1). 2.16.
Introduction
+ D;
=
0
If
2a
b
0
0
£:
An
=
O. 0 0
··0
·b ·e ··2a
nxn
and D; (A) = dett A, - AI), find the eigenvalues of An in the case a 2 ~ be. 2.17.
If {Yn} has generating functionj(z), for Izl < R, show that v.J:» + 1) has generating function II z J~j(t) dt in the same region. (Hint: integrate term by term.)
2.18.
Prove (2.7.9). (Hint: expand in Taylor series starting from y(tn ) and equate to zero the coefficients of h O and h.)
2.19.
3 Show that the solution of Yn+2 - - Yn+1 2 is unbounded.
1
1
+ - Yn = - - , Yo = 0, YI = 0, 2 n+1
2.20.
Prove that if the method (p, u) is convergent (for all j satisfying the hypothesis stated in the text), then it is O-stable. (Hint: take j = 0.)
2.21.
With the hypothesis of Problem 2.20, prove that the method is consistent.
2.22.
Suppose that the relation (2.7.8) is given by p(E)Yn - hu(E)jn = En' where En is a small bounded quantity but not infinitesimal with respect to h. (This happens in practice when we solve the difference equation on the computer.) Using a similar procedure used in the text prove that len+kl ::5 E 1 + E 2, where E 1 and E 2 have respectively a zero and a pole for h = O. Deduce for this that it is not convenient in practice to use h arbitrarily small.
2.10.
61
Notes
2.23. The linear multistep method Zn+1 - 2z n+1 + Zn = h[fn+1 - In] is consistent. It is not O-stable. Show, however, that for the some set of initial conditions the solutions of p(E)zn = 0 remain bounded. (Hint: try with constant initial conditions.) 2.24. Find the region of absolute stability for the following methods: (a) Yn+2 - Yn = 2hfn+l, (midpoint),
h
(b) Yn+2 - Yn ="3 (fn+2 + 41n+1
.
+ In), (Simpson rule).
2.25. Find the solution of the boundary value problem Yn+2 - 2ZYn+1 0, YI - ZYo = 0; YN-I - ZYN = O.
2.10.
+ Yn =
Notes
The material of Sections 2.1 to 2.4 is classical and can be found in 'many classical books. Theorem 2.1.8 is essentially a compact form of a result in Clenshaw [25] and Luke [99], see also Section 5.4. The content of Section 2.5 is also classical but the notation is adapted from Henrici's book [13]. Section 2.6 consists of a collection of results that are scattered in many publications, essentially dealing with Numerical Analysis. The material of Section 2.7 is based on Dahlquist's papers starting from the fundamental one [33] (see also references given in Chapter 6). The books of Lambert [92] and Gear [54] give more detailed arguments on the subject. Theorem 2.7.3 can be found in Henrici [75]. Section 2.8 deals with material that can be found either in some books on numerical analysis (see for example [150]) or in some books on difference equations, see Fort [49]. The classical reference is the book of Atkinson [9] where both the continuous and discrete cases are treated as well as many applications.
CHAPTER 3
Linear Systems of Difference Equations
3.0. Introduction In this chapter, we shall treat systems of linear difference equations. Some results discussed in Chapter 2 are presented here in an elegant form in terms of matrix theory. After investigating the basic theory, method of variation of constants and higher order systems in Sections 3.1 to 3.3, we shall consider the case of periodic solutions in Section 3.4. Boundary value problems are dealt with in Section 3.5, where the classical theory of Poincare is also included. The elements of matrix theory that are necessary for this chapter may be found in Appendix A. Some useful problems are given in Section 3.6.
3.1. Basic Theory Let A(n) be an s x s matrix whose elements aij(n) are real or complex functions defined on N;o and Yn E R S (or C S ) with components that are functions defined on the same set N;o' A linear equation Yn+l = A(n)Yn
+ bn,
(3.1.1)
where b; E R and Yno is a given vector is said to be a nonhomogeneous linear difference equation. The corresponding homogeneous linear difference equation is S
Yn+l = A(n)Yn'
(3.1.2)
63
64
3.
Linear Systems of Difference Equations
When an initial vector y"" is assigned, both (3.1.1) and (3.1.2) determine the solution uniquely on the set N~o as can be seen easily as induction. For example, it follows from (3.1.2) that the solution takes the form (3.1.3) from which follows the uniqueness of solution passing through y"", because n;:~o A(i) is uniquely defined for all n. Sometimes, in order to avoid confusion, we shall denote by y(n, no, y",,) the solution of (3.1.1) or (3.1.2) having y"" as initial vector. Let us now consider the space S of solutions of (3.1.2). It is a linear space since by taking any two solutions of (3.1.2), we can show that any linear combination of them is a solution of the same equation. Let E lo E 2, ... , E, be the unit vectors of R S- and y(n, no, E i), i = 1, 2, ... , s, the s solutions having E, as initial vectors. Lemma 3.1.1. Any element of S can be expressed as a linear combination of y(n, no, E i), i = 1,2, ... , s. Proof. Let y(n, no, c) be a solution of (3.1.2) with y"" = c E R S • From the linearity of S and from c = L:=I ciE;, it follows that the vector
z, =
s
I
i=1
ciy(n, no, E i)
satisfies (3.1.2) and has c as initial vector. Then, by uniqueness, • coincide with y(n, no, c).
Zn
must
Definition 3.1.1. Let j;(n), i = 1,2, ... , s, be vector valued functions defined on N~o' They are linearly dependent if there exists constants a., i = 1,2, ... , s not all zero such that L:=I a;j;(n) = 0, for all n 2:: no. Definition 3.1.2. The vectors};(n), i if they are not linearly dependent.
=
1,2, ... , s, are linearly independent
Let us define the matrix K(n) = (f1(n)'/2(n), ... ,fs(n» whose columns are the vectors };(n). Also let a be the vector (aI, a2,"" aJT.
Theorem 3.1.1. If there exists an ii E N~o such that det K (ii) "" 0 then the vectors };(n), i = 1,2, ... , s are linearly independent.
3.1.
Basic Theory
Proof.
65
Suppose that for n
2=
no s
K(n)a =
I
aJ;(n) = O.
i=l
Since det K(n) .,e 0, it follows that a linearly dependent. •
=
0 and the functions t(n) are not
Theorem 3.1.2. If t(n), i = 1,2, ... , s, are solutions of (3.1.2) with det A(n) .,e 0 for n E N;o' and if det K(no) .,e 0, then det K (n) .,e 0 for all n E N;o' Proof.
For n
2=
no,
det K(n
+ 1) =
det(f)(n
+ 1),fzCn + 1), ... ,!sen + 1))
= det A(n) det K(n),
(3.1.4)
from which it follows that detK(n)
= C~~ detA(i))
detK(no).
•
(3.1.5)
Corollary 3.1.1. The solutions yen, no, E;), i = 1,2, ... , s, of (3.1.2) with det A(n) .,e 0 for n 2= no, are linearly independent. Proof. In this case det K (no) 3.1.1, the result follows. •
= I,
the identity matrix and by Theorem
Corollary 3.1.2. If the columns of K(n) are linearly independent solutions of (3.1.2) with det A(n) .,e 0, then det K(n) ¥- 0 for all n 2= no. Proof. The proof follows from the fact that there exists an det K(n) .,e 0 and the relation (3.1.4). •
n at
which
The matrix K(n), when its columns are solution of (3.1.2) is called Casorati matrix or fundamental matrix. We shall reserve the name of fundamental matrix for a slightly different matrix, and call K (n) the Casorati matrix. Its determinant is called Casoratean and plays the same role as the Wronskian in the continuous case. The Casorati matrix satisfies the equation K(n
Theorem 3.1.3. dimension s.
+ 1) =
A(n)K(n).
(3.1.6)
The space S of all solutions of (3.1.2) is a linear space of
66
3.
Linear Systems of Difference Equations
The proof is an easy consequence of Lemma 3.1.1 and Corollary 3.1.1. Definition 3.1.3. Given s linearly independent solutions of (3.1.2), and a vector C E R of arbitrary components, the vector valued function Yn = K(n)c is said to be the general solution of (3.1.2). S
c
Fixing the initial condition Y"", it follows from definition 3.1.3 that and
= K-1(no)y""
y(n, no, y",,) and in general, for s
E
=
K(n)K-1(no)y""
(3.1.7)
N;o' Ys = c,
y(n, s, c)
= K(n)K-1(s)c.
(3.1.8)
= K(n)K-1(s)
(3.1.9)
The matrix
(n, s)
satisfies the same equation as K(n), i.e., (n + 1, s) = A(n)(n, s). Moreover, (n, n) = I for all n ;:::: no. We shall call the fundamental matrix. In terms of the fundamental matrix, (3.1.7) can be written as y(n, no, y",,) = (n, no)y"". Other properties of the matrix are (i) (n, s)(s, t) = (n, t) and (ii) if -I(n, s) exists then -I(n, s) = (s, n).
(3.1.10)
The relation (3.1.10) allows us to define (s, n), for s < n. Let us now consider the nonhomogeneous equation (3.1.1). Lemma 3.1.2. The difference between any two solutions Yn and )in of (3.1.1) is a solution of (3.1.2).
Proof.
From the fact that
Yn+i
=
A(n)Yn + bn,
)in+! = A( n) )in+! + bn,
one obtains
Yn+1 - )in+1 which proves the lemma.
=
A(n)(Yn - )in+1),
•
Theorem 3.1.4. Every solution of (3.1.1) can be written in the form Yn = )in + (n, no)c, where y; is a particular solution of (3.1.1) and (n, no) is the fundamental matrix of the homogeneous equation (3.1.2).
Proof. From Lemma 3.1.2, Yn - )in E S and an element in this space can be written in the form (n, no)c. If the matrix A is independent of n, the fundamental matrix simplifies because lI>(n, no) = (n - no, 0). •
3.2.
67
Method of Variation of Constants
3.2.
Method of Variation of Constants
From the general solution of (3.1.2), it is possible to obtain the general solution of (3.1.1). The general solution of (3.1.2) is given by y(n, no, c) = ll>(n, no)c. Let c be a function defined on N~o and let us impose the condition that y(n, no, cn) satisfy (3.1.1). We then have
y(n
+ 1, no, cn + l ) = ll>(n + 1, nO)cn+1 = A(n)ll>(n, no)cn+t = A(n)ll>(n, nO)cn + bn,
from which, supposing that det A(n) ."r:. 0 for all n ;;::: no, we get
c, + ll>(n o, n + 1)b n. The solution of the above equation is Cn + 1
=
n = Cno +
C
n-I
I
j=no
ll>(no,j + 1)bj.
The solution of (3.1.1) can now be written as
y(n, no, Cno ) = ll>(n, no)cno + =
ll>(n, no)cno +
n-I
I
ll>(n, no)ll>(no,j + 1)bj
j=no n-I
I
ll>(n,j + 1)bj,
j=no
from which, setting Cn = Yno' we have n-I
y(n, no, Yno)
= (n, no)yno + I
j=no
ll>(n,j + 1)bj.
By comparing (3.1.5) and (3.1.9), it follows that ll>(n, no) = n~~~~ A(i) = 1. We can rewrite (3.2.1) in the form
y(n, no, Yno)
=
(3.2.1)
rr.; A(i), where
CD: A(i) )Yno + jngoCD~I A(s) )bj.
(3.2.2)
In the case where A is a constant matrix ll>(n, no) = An-no and, of course, (n, no) = ll>(n - no, 0). The equation (3.2.2) reduces to n-I
y ( n, no, Yno )
'\' An-j-1b = A n- n°Yno + j=no ~
j'
(3.2.3)
Let us now consider the case where A(n) as well as b; are defined on N±. Theorem 3.2.1. Suppose that R+, j E N±. Then,
L;=-oo IIK- I (j + 1)11 < +00 and IlbJ
0, e > 0 and m 1 chosen I such that I;'::mt- r IIK- (j + 1)11 < e and Ily(n, ml -
T,
Ilj~~-r K(n)K-1(j + l)b ll
0) - y(n, ml> 0)11 =
j
~ IIK(n)llb~.
It follows that the sequence will converge as m given by
Yn
=
n-I
L
K(n)K-1(j
j=-co
~ -00.
Let Yn be the limit
+ 1)bj ,
which is again a solution of (3.1.1). By setting s = n - j - 1, we obtain 00
Yn
=L S~O
K(n)K-1(n - s)bn- s- I '
In the case of constant coefficients this solution takes the form co
Yn
=L
(3.2.5)
A n_S _ 1> Sb
S~O
which exists if the eigenvalues of A are inside the unit circle.
•
Let us close this section by giving the solution in a form that corresponds to the one given using the formal series in the scalar case. Let (3.2.6) By multiplying with z", with z E C, and summing formally from zero to infinity, one has
Letting co
Y(z)
=
L Yn z n,
n~O
B(z)
=
co
L b.s" n=O
69
3.3. Systems Representing High Order Equations
and substituting, one obtains Z-I(y(Z) - Yo) == AY(z)
+ B(z)
from which
+ zB(z)
(I - zA) Y(z) == Yo
and (3.2.7) When the formal series is convergent, the previous formula furnishes the solutions as the coefficient vectors of Y(z). The matrix R(z-1, A) == (Z-IJ - A)-I is called resolvent of A (see A.3). Its properties reflect the properties of the solution Yn'
3.3. Systems Representing High Order Equations Any k t h -order scalar linear difference equation Yn+k
+ PI(n)Yn+k-1 + ... + Pk(n)Yn
can be written as a first order system in R
Y n == (
Y~:I),
Yo == (
Yn;k-I
k
,
~:
(3.3.1)
== gn
by introducing the vectors
),
a, == (
~)
(3.3.2)
~n
Y:-I
and the matrix
o o
1
o
o 1
o o
o
A(n) ==
(3.3.3)
Using this notation equation, (3.3.1) becomes Yn + 1 == A(n) Yn
+ G;
(3.3.4)
where Yo is the initial condition. The matrix A(n) is called the companion (or Frobenius) matrix and some of its interesting properties that characterize the solution of (3.3.4) are given: (i)
The determinant of A(n)-H is the polynomial (_1)k (A k + PI(n)A k-I + ... + Pk(n». When A is independent of n this polynomial coincides with the characteristic polynomial;
3.
70
(ii) detA(n)
Linear Systems of Difference Equations
and is nonsingular if (3.3.1) is really a k th
= (-1)k pk ( n )
order equation; (iii) There are no semisimple eigenvalues of A(n) (see Appendix A). This implies that both the algebraic and geometric multiplicity of the eigenvalues of A coincide. This property is important in determining the qualitative behavior of the solutions; (iv) When A is independent of n and has simple eigenvalues ZI, Z2, ... , z., it can be diagonalized by the similarity transformation A = VDV-\ where V is the Vandermonde matrix V(Zh Z2,' .. , z.) and D = diagf z., Z2, ... , zs). The solution of(3.3.4) is deduced by (3.2.1), which in the present notation becomes n-I
Yn
= (n, no) Yo + I
j= r1o
(n,} + 1)Gj •
The fundamental matrix ( n, no) is given by (n, no) = K(n)K- 1(no),
where the Casorati matrix K (n) is given in terms of k independent solutions fl(n)J2(n), ... Jdn) of the homogeneous equation corresponding to (3.3.4), i.e., fk(n)
f2(n
+k
- 1)
)
f,(n+k-l) .
The solution Yn of (3.3.4) has redundant information concerning the solution of (3.3.1). It is enough to consider any component of Y n for n === no + k to get the solution of (3.3.1). For example, if we take the case Yo = 0, from (3.2.1) we have n-·1
Yn =
I
(n,} + I)Gj ,
(3.3.5)
j=no
where, by (3.1.9), (n,} + 1) = K(n)K- 1(j + 1). To obtain the solution y(n + k; no, 0) of (3.3.1), it is sufficient to consider the last component of the vector Yn +1 and we,get n
Yn+k
= I
E[(n
+ I,} + 1)Gj
E[K(n
+ 1)K- 1(j + 1)Ek gj ,
j=no
==
n
I
j=no
where E k = (0,0, ... , 0, 1) T.
(3.3.6)
3.3.
71
Systems Representing High Order Equations
Introducing the functions
H(n
+ le,j) = EI(n + I)K- I (j + 1)Ek ,
(3.3.7)
the solution (3.3.6) can be written as n
I
Yn+k =
r- no
H(n
+ k,j)gj'
(3.3.8)
The function H(n + le,j), which is called the one-sided Green's function has some interesting properties. For example, it follows easily from (3.3.7) that
H(n+k,n)=l.
(3.3.9)
In order to obtain additional properties, let us consider the identity k
I
=I
i=1
(3.3.10)
EjET,
where E, are the unit vectors in R k and I is the identity matrix. From (3.3.7), one has k
H(n
+ le,j) = I ErK(n + 1)EjETK- I (j + 1)EkJ
(3.3.11)
i=1
which represents the sum of the products of the elements of the last row of k(n + 1) and the elements of the last column of k-l(j + 1). By observing that the elements in the last column of the matrix K-l(j + 1) are the cofactors of the elements of the last row of the matrix K (j + 1), it follows that
H(n+kJ')= 1 , det K(j + 1) fl(j fl(j
+ I) + 2)
fk(j fk(j
+ 1) + 2)
x det
.
ft(j+k-1) fl(n + k)
fin
+ k)
(3.3.12)
fk(j+k-l) fk(n + k)
As a consequence, one finds
H(n H(n
+ k, n)
+k -
(3.3.13)
= 1,
i, n) = 0,
i = 1,2, ... , k - 1,
(3.3.14)
and
H(n+k,n+k)=(-I)
k-l
d
det K(n + k) 1 ( k )=- ( k)' et K n + + 1 P« n +
(3.3.15)
72
3.
Linear Systems of Difference Equations
Proposition 3.3.1. The function H(n,j),for fixed], satisfies the homogeneous equation associated with (3.3.1), that is, k
I
p;(n)H(n
+k
- j,j)
= o.
;~O
Proof. It follows easily from (3.3.12) and the properties of the determinant. •
The solution (3.3.8) can also be written as Yn
n-k =
I
i v n«
(3.3.16)
H(n,j)gj,
with the usual convention L~:no = O. For the case of arbitrary initial conditions together with equation (3.3.1), one can proceed in similar way. From the solution Yn + 1
n
= K(n + I)K- 1(no)Yo + I
K(n
+ 1)K- 1(j + 1)q,
j=no
by taking the k tb component we have Yn+k
= »: K(n + 1)K-
1(n
o)Yo +
n
I
j=no
H(n
+ k,j)gj'
In the case of constant coefficients, the expression for H(n,j) can be simplified. Suppose that the roots Zi of the characteristic polynomial are all distinct. We then have, from (3.3.11), 1 1 k H(n
+ k,j) =
TI z{'+1
i~1
k
TI z{+1 det K(O)
det
i=1
k-2 ZI (n-j)+k-I
ZI k "
z:
= i=1
n-j+k-l,,( Z; "i ZI,""
det K(O)
Zk
)
=
I
k
n-j+k-I
_Z::.. .i_ _ ;=1 p'(z;)-'
(3.3.17)
where V;(ZI,"" zd are the cofactors of the jtb elements of the last row and P'(Zi) is the derivative of the characteristic polynomial evaluated at z.. In this case, as can be expected, one has H(n +k,j) = H(n + k - j, 0). By denoting with H(n + k - j) the function H(n + k - j, 0), the solution
73
3.3. Systems Representing High Order Eqnations
of the equation k
I PJln+k-i = gn i=1 such that Yi
= 0, i = 0, 1, ... , k - 1, is given by Yn =
n-k
I
(3.3.18)
H(n - j)gj'
j~O
The properties (3.3.12), (3.3.13), (3.3.14) reduce to H(k) == 1, H(k - s) s
1
=
0,
.
= 1, ... , k -1, and H(O) = --, respectively. Pk
We shall now state two classical theorems on the growth of solutions of (3.3.1) with gn = O. If lim p.In) = Pi> i
Theorem 3.3.1. (Poincare).
n-+CO
= 1,2, ... , k,
and if the
roots of k
I
i-«
i PiAk- = 0,
(3.3.19)
Po = 1
have distinct moduli, then for every solution Yn,
. Yn+l , II m - = As> Yn
n-+OO
where As is a solution of (3.3.19). Proof. Let pi(n) = Pi + lli(n) where, by hypothesis, lli ~ 0 as n ~ matrix A(n) can be split as A(n) = A + E kll T(n), where
A=
o o
1
o
1
o
The
o
o
o -Pk
o
00.
-Pk-l
-PI
11 T(n) = (-llk(n), llk-l(n), .... , -lll(n)) and E[ == (0,0, ... ,1). Equation (3.3.4) then becomes Y n + 1 = AYn + E kll T (n) Yn' Now A = VA V-I, where V is the Vandermonde matrix made up of the eigenvalues. AI, A2 , ••• , Ak of A (which are the roots of (3.3.19)) and A = diag(A), A2 , ••• , Ak ) . We suppose that IA 11 < IA 2 1 < ... < IAkl. Changing variables u(n) = V-I Y n and letting f n = V- IEk ll T(n)V, we get u(n+1)=Au(n)+r nu(n). The elements of I'(n), being linear combinations of lli(n), tend to zero as n ~ co. This implies that for any matrix norm, we have IIrn I ~ O. Suppose now that
74
3.
max
l'$i:'Sk
IUj(n)1 = lus(n)l.
The index s will depend on n. We shall show that an
no can be chosen such that for n
In fact, we know that for i < j,
IAil1+ e
N suitably chosen, the function s(n) will then assume a fixed value less or equal to k. We shall show now that the ratios
IUj(n)1 jus(n)I'
(3.3.20)
j~s
tend to zero. In fact we know that for n > N,
lu(n)1
lu:(n)1 ~
. a < 1. This means
that a is an upper limit for (3.3.20). We extract for n 2': N a subsequence n1, n2, ... , for which (3.3.20) converges to a. Suppose first that j > s. Then luj(np + lus(np +
IAjlluj(np)l_ e ::lus(np)1 01- IAsl + e
01
:>
We take the limit of subsequence, obtaining a lower limit . luj(np + 01 hm p_oo Ius(np + 0 12':
This implies
IAjia - e IAs I+ e .
75
3.4. Periodic Solutions
for arbitrary small a
= O.
E.
Since
I~~II > 1, the
previous relation holds only for
In the case j < s, similar arguments, starting from IUj(np Ius(n p
+ 1)1 (N,j + 1)bj "r:. O. Consequently, for every solution Yn of (3.1.1), we have
VTYN = vTet>(N,O)yo+ v T
N-l
I
j=O
= vTyo+ v T
et>(N,j+ 1)bj
N-l
I
et>(N,j+ 1)bj •
j=O
Moreover, by considering the periodicity of b, and (3.4.1) we get
V
TY2N=V TYN+V T
N-l
I
j=O
et>(N,j+1)bj
N-l
I
= v TYo+2v T
j=O
et>(N,j+1)bj>
and in general, for k > 0, N-l
I
VTYkN=VTYO+kv T
j=O
showing that Yn cannot be bounded.
et>(N,j+1)bj
•
The matrix U"" et>(N, 0) has relevant importance in discussing the stability of periodic solutions. From
et>(n
+ N,O)
= et>(n
+ N, N)et>(N, 0)
and (3.4.1), it follows that:
et>(n
+ N,O)
=
et>(n, 0) U
(3.4.10)
and in general, for k > 0, et>~n
+ kN,O)
= et>(n, 0)
tr.
(3.4.11)
Suppose that p is an eigenvalue of U and v the corresponding eigenvector. Then, Letting et>(n, o)v
et>(n
+ N, O)v = et>(n, 0) Uv = pet>(n, O)v.
= o,
for n
;;=:
0, we get (3.4.12)
This means that the solution of the homogeneous equation having initial value vn , after one period, is multiplied by p. For this reason the eigenvalues of U are usually called muItiplicators. The converse is also true. If Yn is a solution such that Yn+N = PYn, for all n, then in particular is YN = PYo and that means UYo = PYo from which follows that Yo is an eigenvector of U.
79
3.5. Boundary Value Problems
3.5.
Boundary Value Problems
The discrete analog of the Sturm-Liouville problem is the following: (3.5.1) (3.5.2) where all the sequences are of real numbers,
rk
> 0,
ao ¢
0,
aM ¢
0 and
o:s; k :s; M. The problem can be treated by using arguments very similar to the continuous case. We shall transform the problem into a vector form, and reduce it to a problem of linear algebra. Note that the equation (3.5.1) can be rewritten as Let k = 2, .... , M -1,
Y
= (Yl , Y2, ... , YM ) T,
and
A=
al
-PI
0
-PI
a2
-P2
0
0
0
(3.5.3)
-PM-l
0
0
-:PM-l
aM
then the problem (3.5.1), (3.5.2) is equivalent to (3.5.4)
Ay = ARy.
This is a generalized eigenvalue problem for the matrix A. The condition for existence of solutions to this problem is
det(A - AR)
=
0,
(3.5.5)
which is a polynomial equation in A. Theorem 3.5.1.
Proof.
The generalized eigenvalues of (3.5.4) are real and distinct.
Let S = R- 1/2 . It then follows that the roots of (3.5.5) are roots of det(SAS - AI) = O.
(3.5.6)
Since the matrix SAS is symmetric, it will have real and distinct eigenvalues.
80
3.
Linear Systems of Difference Equations
For each eigenvalue Aj , there is an eigenvector v'. which is the solution of (3.5.4). By using standard arguments, it can be proved that if / and are two eigenvectors associated with two distinct eigenvalues, then
y
•
M
(
i j - 0. Y,i RY i) -= '"' L.. rsJ!.Ys
s=1
Definition 3.5.1. R-orthogonaI.
(3.5.7)
Two vectors u and v such that (u, Rv) = 0 are called
Since the Sturm-Liouville problem (3.5.1), (3.5.2) is equivalent to (3.5.4), we have the following result. Theorem 3.5.2.
Two solutions of the discrete Sturm-Liouville problem corresponding to two distinct eigenvalues are R-orthogonal. Consider now the more general problem
Yn+l
=
A(n)Yn
+ bn,
(3.5.8)
where Yn, b; E R S and A(n) is an s x s matrix. Assume the boundary conditions are given by N
L LJ'n, =
i=O
(3.5.9)
w,
where n, E Nt, n, < nj+1, no = 0, w is a given vector in R S and L j are given s x s matrices. Let (n,j) be the fundamental matrix for the homogeneous problem Yn+1
= A(n)Yn,
(3.5.10)
such that (0,0) = 1. The solutions of (3.5.8) are given n-l
Yn
= (n, O)yo + L
j=O
(n,j + l)bj,
(3.5.11)
where Yo is the unknown initial condition. The conditions (3.5.9) will be satisfied if N
L
i=O
LJ'n, =
N
L
i=O
Lj(nj, O)yo +
N
L
i=O
n,-1
i;
L
j=O
(nj,j + l)bj
= w,
3.5.
81
Boundary Value Problems
which can be written as N
"N- I
N
I Lj(nj,j + 1) T(j
+ 1, nJbj = w,
j~O
(3.5.12) where the step matrix T(j, n) is defined by
T(j, n)
By introducing the matrix Q = becomes
Qyo =
W -
I 0
= {
N
"N- I
;=0
j~O
I I
forj f ' or)
S
>
n, n.
I: Lj 0 there exists 5> 0, N(e) > 0 such that for Yo E B o'
1I(n, no)Yoll < 10 for n
~
no + N( e).
As before, it is easy to see that
1I(n, no)1I
(n, no)11 S a:1]n-n o for some suitable a: > 0 and 0 < 1] < 1. Then and this shows that the solution x = 0 is uniformly asymptotically stable • because Un is bounded. The proof of other cases is similar. We shall merely state another important variant of Theorem 4.5.1 * which is widely used in numerical analysis. Theorem 4.5.3.
Given the difference equation Yn+l
= Yn + hA(n)Yn + f(n, Yn),
(4.5.7)
where h is a positive constant, suppose that (1) f(n,O) = 0 for n (2) Ilf(n, y)1I S g(n,
2=
no, and with g(n, u) nondecreasing in u, g(n, 0)
Ilyll)
= O.
Then the stability properties of the null solution of (4.5.8)
imply the corresponding stability properties of the null solution of (4.5.7). In this form the foregoing theorem is used in the estimation of the growth of errors in numerical methods for differential equations. Instead of (4.5.8), one uses usually the comparison equation (4.5.9)
4.
98
Stability Theory
which is less useful because (1 + hiIAII) > 1. The form (4.5.8) is more interesting because when the eigenvalues of A have all negative real parts, I 1+ hA I can be less than 1. This will happen, for example, if the logarithmic norm: JL(A(n)) = lim
III + hA(n)ll- 1 h
h-O
is less than zero. From the definition it follows that
III + hAil = 1 + hJL(A(n)) + O(h 2A(n)). Letting the comparison equation becomes U n:r1 =
(1 + ,1)u n
+ g(n,
un).
(4.5.10)
The next theorem requires essentially a condition on the variation of 1, but it does not require the a-priori knowledge of the existence of the critical point. Let us consider (4.5.11)
Theorem 4.5.4. Suppose f: Dc R S ~ R S is continuous and g is a positive function nondecreasing with respect to its arguments, defined on J I x J2 X J3 where I, are subsets of R+ containing the origin. Further suppose that for X o E D, the sequence x, is contained in D, (4.5.12) and the comparison equation U n+1 =
g(
n
Un,
n-1)
j~O Uj, j~O u j
(4.5.13)
has an exponentially stable fixed point at the origin. Then (4.5.11) has a fixed point that is asymptotically stable.
Let us say
Proof.
j
j
Ilxn +! - x, II. Then we have Ilxn + 1 Since g is nondecreasing, it follows that
Yn =
I;=o Ilx + 1 - x I :s: I;=o Yj'
n
Yn+1 :S:
g(
By Theorem 1.6.6 we then obtain
Yn,
n-I)
j~O Yj, j~O Yj
•
xoll :s:
4.6.
99
Variation of Constants
where Un is the solution of (4.5.13), provided that Yo S U00 If the origin is exponentially stable for (4.5.13), it follows that for suitable Uo the sequence Un will tend to zero and the same will happen to Yn = [[x n+ 1 - x, II. Moreover for all p ;;:: 0 Ilxn+ p
-
x, II s
p
I
Yn+j'
j~1
Now by exponential stability of the origin of (4.5.13) and by Theorem 4.1.1 it follows that it is also II-stable. Then the series I Yj is convergent and then for suitable n, I%=I Yn+j can be taken arbitrarily small, showing that Xk is a Cauchy sequence. •
4.6.
Variation of Constants
Consider the equation Yn+1 = A(n)Yn
+ f(n,Yn),
Y""
where A( n) is an s x s nonsingular matrix and f: Theorem 4.6.1.
= Yo,
»; x R
(4.6.1) S
~ R
S
•
The solution y(n, no, Yo) of (4.6.1) satisfies the equation n-I
Yn
= (n, no)Y"" + I
j=no
(n,j + l)f(j, Yj),
(4.6.2)
where (n, no) is the fundamental matrix of the equation x n+ 1 = A(n)xno Proof.
(4.6.3)
Let y(n, no, Yo) = (n, no)xn,
x"" = Yo.
(4.6.4)
Then substituting in (4.6.1), we get (n
+ 1, no)x n+ 1 = A(n)(n, no)x n + f(n,Yn)
from which we see that
aXn = -l(n + 1, no)f(n, Yn), and n-I
x,
=
I
j=no
(no,j + 1)f(j, Yj) + x"".
•
From (4.6.4), it follows that I
y(n, no, Yo)
n-]
= (n, no)yo + I
j=no
(n,j + 1)f(j, Yj)'
(4.6.5)
100
4.
Stability Theory
Consider now the equation (4.6.6)
Lemma 4.6.2. Assume that f: N;o x R S ~ R S and f possesses partial derivatives on N;o x R S • Let the solution x(n) == x(n, no, xo) of (4.6.6) exist for n
~
no and let _ af(n, x(n, no, xo)) H ( n, no, X o) . ax
(4.6.7)
Then
(4.6.8) exists and is the solution of (n
+ 1, no, xo)
= H(n, no, xo)(n, no, x o),
(4.6.10)
(no, no, x o) = 1.
Proof.
(4.6.9)
By differentiating (4.6.6) with respect to X o we have aXn + 1 = afaxn axo aXn axo
Then (4.6.9) follows from the definition of . We are now able to generalize Theorem 4.6.1 to the equation Yn+l = f(n, Yn) + F(n,
yJ.
•
(4.6.11)
Theorem 4.6.2. Let f, F: N;o x R S ~ R S and let af/axexistand be continuous and invertible on N;o x R S • If x(n, no, x o) is the solution of X no
(4.6.12)
= Xo,
then any solution of (4.6.11) satisfies the equation y(n, no, x o) = where
x( n, no, Xo + j~~ l/J-l(j + 1, no, vj, vj+ )F (j, Yj)) 1
(4.6.13)
4.6.
Variation of Constants
101
and vj satisfies the implicit equation (4.6.14). Proof.
yen
Let us put yen, no, xo)
•
= x(n, no, vn) and
Vo
= Xo'
Then
+ 1, no, xo) = x(n + 1, no, Vn+l) - x(n + 1, no, vn) + x(n + 1, no, vn) = fen, x(n, no, vn + F(n, x(n, no, vn
»
»
from which we get
x(n
+ 1, no, Vn+l) - x(n + 1, no, vn) = F(n, Yn).
Applying the mean value theorem we have
f f
l aX(n + l , no, sVn+I + (1 - S)Vn) d ( -)- ( ) S Vn+1 Vn - F n, Yn o axo .
and hence by (4.6.8)
cI>(n
+ 1, no, SVn+1 + (1- s)vn) ds(v n+1 -
vn) = F(n, Yn),
which is equivalent to
l/J(n
+ 1, no, Vn, Vn+I)(Vn+1 - vn) = F(n, Yn).
(4.6.14)
It now follows that
Vn+1 - Vn = l/J-I(n + 1, no, Vn, vn+t)F(n, Yn) and n-I
e,
= Vo + L
l/J-I(j + 1, no, vh vj+I)F(j, Yj)
(4.6.15)
j=no
from which the conclusion results.
•
Corollary 4.6.1. Under the hypothesis of Theorem 4.6.2, the solution y( n, no, xo) can be written in the following form.
yen, no, xo) = x(n, no, x o) + l/J(n, no, Vn, xo) n-I
.L
l/J-l(j
+ 1, no, Vj,
vj+I)F{j,
v)
(4.6.16)
j=no
Proof.
Apply the mean value theorem once more to (4.6.13).
Corollary 4.6.2. Proof.
Iff(n,x)
In this case Xn =
= A(n)x,
cI>~n,
•
then (4.6.16) reduces to (4.6.2).
no)xo and
(n, no, xo) == l/J(n, no) == l/J(n, no, Vn, Vn+I),
102
4.
Stability Theory
and therefore, we have Vn+ 1 - Vn = l/J-l(n + 1, no, Vn, vn+l)F(n, Yn) and Vn = Xo + L:;':-~o l/J -l(j + i, no, Vj, vj+l)F(j, Yj) from which follows the claim. •
4.7.
Stability by First Approximation
Consider the equation Yn+l = A(n)Yn
+ f(n, Yn),
(4.7.1)
where y E R A(n) is an s x s matrix, f: N no x B a ~ B, and f(n, 0) = O. When f is small in the sense to be specified, one can consider (4.7.1) as a perturbation of the equation S
,
X n+ 1
(4.7.2)
A(n)xn
=
and the question arises whether the properties of stability of (4.7.2) are preserved for (4.7.1). The following theorems offer an answer to such a question. Theorem 4.7.1.
Assume that (4.7.3)
where gn are positive and L:~=no gn < 00. Then if the zero solution of (4.7.2) is uniformly stable (or uniformly asymptotically stable), then the zero solution of (4.7.1) is uniformly stable (or uniformly asymptotically stable). Proof.
By (4.6.2) we get Yn
=
(n, no)Yo +
n-l
I
j=O
(n,j + 1)f(j, Yj)'
Because of Theorem 4.2.1 we have, using (4.7.3), llYn II ~ MllYol1
+M
n-I
I
j=no
gJyJ.
Corollary 1.6.2 yields
from which follows the proof, provided that Xo is small enough such that MllYol1 exp(ML:;:nogJ < a.
4.7.
103
Stability by First Approximation
In the case of uniform asymptotic stability, it follows that for n > N,
1I(n, no)Yoll < e, for every e > 0, the previous inequality can be written
llYn II :5 e exp ( M ~ gj) from which we conclude that lim Yn
=
0.
•
Corollary 4.7.1. If the matrix A is constant such that the solutions of (4.7.2) are bounded, then the solutions of the equation
Yn+1
= (A + B(n»Yn
(4.7.4)
are bounded provided that co
I IIB(n)11 < 00. n=»o Theorem 4.7.2.
(4.7.5)
Assume that
°
° °
(4.7.6)
where L> sufficiently small, and the solution x, = of (4.7.2) is uniformly asymptotically stable. Then the solution Yn = of (4.7.1) is exponentially asymptotically stable.
Proof.
By using the result of Theorem 4.2.2, we have
I (n, no)11
O,
and because of (4.7.6), we get llYn I
:5
HT/n-noilYoll + LHTJ n- 1
By introducing the new variable P«
=
Pn:5 HTJ-noilYoil
n-I
I
TJ -jllYjll·
j=no
TJ -n llYn II, we see that n-I
+ LHTJ-I I
Pj'
j=no
Using Corollary 1.6.2 again, we arrive at Pn
:5
HTJ-noilYoil
n-I
I1
(I
+ LHT/-I)
j=no
which implies (4.7.8)
104
If T/
4. Stability Theory
+ LH
00
Assume that the zero solution ofxn + 1 = AX n is unstable and
Ilf(n, . . .IS unsta . ble fior (471) I Yn)11 I =.0 171en the ongtn .. . Yn
4.8. Lyapunov Functions The most powerful method for studying the stability properties of a critical point is the Lyapunov's second method. It consists in the use of an auxiliary function, which generalizes the role of the energy in mechanical systems. For differential systems the method has been used since 1892, while for difference equations its use is much more recent. In order to characterize such auxiliary functions, we need to introduce a special class of functions.
4.8. Lyapunov Functions
105
Definition 4.8.1. A function 4> is said to be of class K if it is continuous in [0, a), strictly increasing and 4>(0) = o. It is easy to check that the product of any two functions of class K is in the same class and the inverse of such a function is in the same class. Let V(n, x) be a function defined on /"" x Ba , which assumes values in R+.
Definition 4.8.2. The function V(n, x) is positive definite (or negative. definite) if there exists a function 4> E K such that
4>(llxll) ~ for all (n, x)
E N~o
V(n, x)
(or V(n, x)
2:
-4>(llxll»
x Ba •
Definition 4.8.3. A function V(n, x) exists 4> E K such that
2:
0 is said to be decrescent if there
V(n, x)
~
4>(x)
for all (n, x)
E N~o
x Ba .
Let us consider the equation (4.8.1)
Yn+l = f(n, Yn),
where f: N~o x B; ~ R f(n,O) = 0 and f(n, x) is continuous in x. Let y(n, no, Yo) be the solution of (4.8.1), having (no, Yo) as initial condition, which is defined for n E N;o' We shall now consider the variation of the function V along the solutions of (4.8.1) S
,
A V(n, Yn)
= V(n + 1, Yn+l) -
V(n, Yn)'
(4.8.2)
If there is a function w : N~o x R ~ R such that A V(n, Yn) ~ w(n, V(n, Yn),
then we shall consider the inequality V(n
+ 1, Yn+l) ~
V(n, Yn) + w(n, V(n, Yn)
== g(n, V(n, Yn»
(4.8.3)
to which we shall associate the comparison equation U n+1
= g(n, un) ==
Un
+ w(n, Un)'
(4.8.4)
The auxiliary functions V(n, x) are called Lyapunov functions. In the following, we shall always assume that such functions are continuous with respect to the second argument.
106
4.
Stability Theory
Theorem 4.8.1. Suppose there exists twofunction V(n, x) and g(n, u) satisfying the following conditions: (1) g: N;o x R+ ~ R, g(n, 0) = 0, g(n, u) is nondecreasing in u; (2) V:N;oxBa~R+, V(n, 0) =0, and V(n, x) is positive definite and continuous with respect to the second argument; (3) V satisfies (4.8.3).
Then (a) the stability of u = 0 for (4.8.4) implies the stability of Yn = 0; (b) the asymptotic stability of u = 0 implies the asymptotic stability of Yn = O. Proof.
By Theorem 1.6.1, we know that
provided that V(n o, Yo) we obtain for ¢ E K,
:5
Uo' From the hypothesis of positive definiteness
If the zero solution of the comparison equation is stable, we get that Un < ¢ (e) provided that Uo < T/ (s, no), which implies
from which we get
llYn II
(IIYnll):5 V(n,Yn):5 u; provided that V(no, Yn,,) :5 uo:5 7](e). If we take Jot (IIY""II) = I is Ily""II < 8(e) == Jot- ( 1]( e) ), then llYn I < e for all n , Jot, II E K and the function w == llJot - I is a Lipshitz function with constant less than one then Yn = 0 is uniformly asymptotically stable. Proof.
Clearly llYn I e > O. We then have
+ l,y(n + 1, no,Yo»:S V(n,Yn) - ,u(e), Jno or, since V is decreasing, V(n + 1, Yn+l) :s V(n, Yn). Summing we V(n
if n E have
V(n
+ 1, y(n + 1, no, Yo» :s
V(no, Yno) - ke,
where k is the number of elements in Jno . Taking the limit we get lim V(n, Yn) = -00, n->oo
which contradicts the hypothesis that V is positive definite. • The next theorem concerns the lp stability and is related to the previous one. Theorem 4.8.4.
Assume that there exists a function V such that
(1) V: N~o x B,
-7 R+, V(n, 0) = 0, positive definite and continuous with respect to the second argument; (2) .:lV(n, Yn) :S -ellYn liP,
where p, c are positive constant. Then Yn = 0 is lp stable.
Proof. By Theorem 4.8.1 we know that Yn = 0 is stable, that means that for n 2:: no, e 2:: 0, there exists 8( s, no) such that for lIyno ll < 8, llYn I < e. Let us define the function G(n) = V(n, Yn) + c L;:~o IlyJP. Then .:lG(n) =.:l V(n,Yn)
+ clIYnIIP:s O.
4.8.
109
Lyapunov Functions
Therefore G(n)
~
G(no) = V(no, Y no) for n ;;::: no and
o ~ G(n) =
yen, Yn)
+c
n-)
I
j=no
llYn liP ~ V(no, Yno)
from which it follows
and 00
I
j=no
llYn liP
0 for Yn ¥ O. Then for any Yo E R , either y( n, Yo) is unbounded 2 or it tends to zero for n -'; 00. Likewise if ~ V(Yn) < 0, Yn ¥ O. S
-';
Proof.
Put ~2V > 0 in the form V(Yn+2) - V(Yn+l) > V(Yn+l) - V(Yn).
If there exists a k E N;o such that V(Yk+l) - V(Yk) ~ 0, then V(Yn+l)V(Yn) > 0 for n > k, otherwise we get V(Yn+l) < V(Yn) for all n. In both cases V is a monotone function. Suppose that V is not increasing. Consider the positive limit set n(yo) of y(n, no, Yo). If n(yo) is empty then y(n, no, Yo) is unbounded and the theorem is proved. If n(yo) is not empty, V(Yn) must be constant on n(yo) because the limit of a monotone function is unique. But this is impossible because ~2 V> 0, unless n(yo) = {O}. The other cases are proved similarly. •
4.9.
Domain of Asymptotic Stability
All the previous results say that if the initial value is small enough, then the origin has some kind of stability. In applications one is more interested
112
4. Stability Theory
in the domain of asymptotic stability since one needs to know where to start with the iterations. In other words, one needs to know the domain in R S, containing the initial values, starting from which solutions will eventually tend to the fixed point. This problem is a difficult one and will also be discussed in the next chapter. We shall give some results in this direction in the case of autonomous difference equations Yn+l = f(Yn),
where f
E
C[Ba , R
S ] ,
and f(O) =
(4.9.1)
o.
Theorem 4.9.1. Suppose that there exists a continuous function V: Ba ~ R+, V(O) == 0 and il V(Yn) < O. Then the origin is asymptotically stable. Moreover if B a == R S and V(y) ~ 00 as y ~ 00, then the origin is globally asymptotically stable. Proof. The proof of stability follows from the same arguments used in Theorem and Corollary 4.8.1, except that continuity of V is used instead of positive definiteness of V. For proving asymptotic stability, we observe that for Yn E Bel where o< e < CI', V(Yn) is strictly decreasing and it must converge to zero. Again by continuity of V, the sequence Yn itself must converge to zero. Now suppose that the last hypothesis is true. Then it is clear that we are not restricted to take llYn II < CI' and hence the proof is complete. •
As an example, consider (4.9.1) when f is linear,
(4.9.2) and let us take
(4.9.3) where B is a symmetric positive definite matrix. The demand that il V(Yn) < 0 becomes
that is, we must have
(4.9.4) where C is any positive definite matrix. This leads to the following result. Corollary 4.9.1. If there is a symmetric positive definite matrix B such that .(4.9.4) is verified, then the origin is asymptotically stable.
4.9.
Domain of Asymptotic Stability
113
The converse of this corollary is also true. Theorem 4.9.2. Suppose (4.9.2) is asymptotically stable, then there exists two positive definite matrices E and C such that (4.9.4) is verified.
The statement analogous to (4.9.4) in the continuous case is STO
+ as =
-at.
(4.9.5)
where a and 0 1 are symmetric positive definite matrices, and S has the eigenvalues with negative real part. Equation (4.9.5) is called Lyapunov matrix equation. There is, of course, a correspondence between (4.9.4) and (4.9.5). In fact by putting S = (A + I)(A - I)-\ (4.9.5) is transformed into an equation of the form (4.9.4) and vice versa by putting A = (1 - S)-I(1 + S). To find the matrix E, one needs to solve the matrix equation (4.9.4) where A is given and C is chosen appropriately. The methods of solution of the equation (4.9.4) (or alternatively 4.9.5) have been studied extensively in the last ten years. The following theorem also gives the region of the asymptotic stability for the zero solution of (4.9.1) and it is the discrete version of the Zubov's theorem. Theorem 4.9.3. Assume that there exists two functions V and ¢> satisfying the following conditions:
(1) (2) (3) Then
Proof.
V: C[R R+], V(O) = 0, . V(x) > 0 for x s« 0, S ¢>: C[R , R+], ¢>(O) = 0, ¢>(x) > 0 for x,e. 0, V(Yn+l) = (l + ¢>(Yn)) V(Yn) - ¢>(Yn). D = {x I V (x) < I} is the domain of asymptotic stability. S
,
By condition(3), which can be written as 1 - V(Yn+l) = (l
+ ¢>(Yn))(l - V(Yn)),
one obtains 1 - V(Yn)
n-l
= I1
(l
+ ¢>(Yj))(1 - V(Yo))·
(4.9.6)
j~O
Suppose now that Yo lies in the region of asymptotic stability. Then, the left-hand side tends to one by hypothesis, the right-hand side will also be
= n (1 + ¢>(Yj) 00
convergent and will tend to C(1 - V(Yo)), where C
j~O
> 1.
= (C - 1)/ C < 1, which shows that Yo ED. Conversely if Yo ED, then again from (4.9.6) we see that V(Yn) will remain
It follows then that V(yo)
114
4.
Stability Theory
always outside D and V will never be zero. This implies that Yn will not tend to zero. • To use this theorem, one needs to know the solution Yj of (4.9.1), and then from (4.9.6) find the function V, which will define the set D. Unfortunately this can be done only for some cases. Theorems 4.8.5 and 4.8.7 and their corollaries also give the asymptotic stability domains in terms of D( 1]) for the set E. When the maximum invariant set contained in E, reduces to a point, Theorem 4.8.7 gives the asymptotic stability domain for a critical point. Theorem 4.8.7 can also be used to obtain the domain of asymptotic stability. Suppose that the condition d 2 V> 0 holds true only in an open region H containing the origin. Then put d max
= max{d V(x) [x E boundary H}
and Ej
for j
=
{x
E
Hid V(y(j, x)) > d max}
= 0, 1,2, ... , where y(j, x) == y(j, no, x).
Theorem 4.9.4. If the regions Ej are bounded and nonempty, then they are domains of asymptotic stability for (4.9.1). Proof. If E, is not empty and x E Ej , y(j + k, x) decreasing along any trajectory in Ej • Moreover d V(y(j
+ k + 1, x)) - d V(y(j + k; x))
= d
E
H, since d V is not
2V(y(j
+ k, x)) > 0
and d V(y(j + k; x)) > d V( y(j, x)) 2 -00364 } 17 17 . . I(y2 __4 - -
Converse Theorems
This section will be devoted to the construction of Lyapunov functions when certain stability properties hold. In this construction of Lyapunov functions, however, one uses the solutions of the problem and this implies that this construction is of little use in practice. The real importance of converse theorems lies therefore in the fact that by means of such theorems, it is possible to prove some results on total stability, which is a very important concept in applications. For this reason we shall present only those results that we shall use later. Suppose that the zero solution of (4.8.1) is uniformly stable, then there exists a function V, positive definite and decrescent, such that ~ V ~ 0 along the solutions.
Theorem 4.10.1.
Proof.
Consider the function yen, Yn) = sup k2:.n
Ily(k, n, Yn)ll·
(4.10.1)
As usual, Yn ;;;; yen, no, Yo). From (4.10.1) it is immediate that yen, Yn) ~
llYn"
showing that V is positive definite. From the definition of uniform stability we know thatlly(k,n,Yn)ll<e if llYn I 0, G"(r) > 0 and let a> 1. Since G(r)
G(;) = LIU du 1
fr
dco ao
f61
IU
G"(v) dv, we have,
=
1 fr dco ao
G"(v) dv < -
0
V(n,Yn) For k
Lr
Consider a function G(r) defined for r> 0, G(O) = 0, G'(O) = 0,
Proof.
=
du
G"(v) dv and
setting u = or]a,
f61 G"(v) dv = -1 G(r). a
0
G(;) =
Define
1 + ak = sup G(lly(n + k, n,Yn)II)--k. k2=O 1+
0, we have
The uniform stability ofthe origin implies that (see previous theorem) there exists a ¢J E k such that
Ily(n
+ k, n,Yn)11 < ¢J(IIYnll)
and therefore
G(lly(n
+ k, n, Yn)ll)
O(lIy(n + k«, n, y")ID we obtain V(n, y') - V(n, y") -s aAllY' y' and y" one gets similarly
y"II.
1 + ak, 1+
k' I
By interchanging the roles of
V(n, y") - V(n, y') ~ -aAllY' - y"ll,
4.11.
119
Total and Practical Stability
which shows that
IV(n, y') proving the theorem.
- V(n, Y")I ::; aAIIY' -
y"ll,
•
The following theorem is the converse in the case of lp-stability. Theorem 4.10.3. Suppose that the trivial solution of (4.10.1) is lp-stable and Ily(n, no, Yo) I ::; gn,B(IIYoll) with,B E K and L~=no s, = g. Then there exists a function V: N~o x B, ~ R+ such that it is positive definite, decrescent and ~ V(n, Yn) ::; -llYn liP.
Proof.
Let 00
V(n,Yn)
=
I
k=O
Ily(n
+ k, n,Yn)lIp •
It follows that
V(n, Yn) 2: llYn liP, which shows that V is positive ,BP(IIYnll) L~~no s; = g,BP(IIYnll) and 00
Ll V(n, Yn) =
I
k=O
Ily(n
+ 1 + k, n + 1, Yn+l)IIP
Ily(n
+ 1 + k, no, Yo)IIP
00
=I
k=O
=
Moreover, 00
I
-
k=O
00
-
I
k=O
Ily(n
lIy(n
V(n, Yn) ::;
+ k, n, Yn)IIP
+ k, no, Yo)[JP
-lly(n, no, Yo) liP,
which completes the proof.
4.11.
definite.
•
Total and Practical Stability
Let us consider the equations
Yn+l
=
f(n, Yn) + R(n, Yn)
(4.11.1)
Yn+l
=
f(n, Yn),
(4.11.2)
where R is a bounded, Lipschitz function in B a and R(n, 0) = o. We shall consider (4.11.1) as a perturbation of equation (4.11.2). Suppose that the zero solution of (4.11.2) has some kind of stability property, we want to know under what conditions on R the zero solution preserves some stability.
Definition 4.11.1. The solution Y = 0 of (4.11.2) is said to be totally stable (or stable with respect to permanent perturbations) ir'for every e > 0, there
120
4.
Stability Theory
exists two positive numbers 8 1 = 8 1 (e) and 8 2 = 8i e) such that every solution y( n, no, Yo) of (4.11.1) lies in Be for n 2= no, provided that
IIYol1
0,8(80 ) < a such that Yn E BIJ(lJ o), lim y, = 0 and moreover there exist the functions a, b, c E K and V such that for n E N~o' (a)
a(llfll) ~
V(n,
(b) 11 V(n, Yn) ~
y) ~ b(llfll),
-c(IIYn+111),
and (c) Iv(n, y') - V(n, y")1 ~ MIY' - y"l,
for y',
y"
E BIJ(lJ o)'
M>O'
Let 0 < e < 8(80 ) , Choose 8 1 > 0, 82 > 0 such that
b(28 1 ) < a(e),
82 ~ 8 1 ,
c(8 1 ) 82 < M'
(4.11.3)
Suppose that e is sufficiently small so that Lee + 8 2 < 8(80 ) , Let IIR(n, y)11 < 8 2 for y E Be. One then finds for Ilyll ~ e, Ily(n + 1, n, y)11 = If(n, y)1 ~ Lee, Ily(n + 1, n,y) - y(n + 1, n,y)11 =:' IIR(n,y)1I < 82 , and Ily(n + 1, n,y)11 ~ Lee + 8 2 < 8(80 ) , Thus, for Ilyll < e, we have V(n + l,y(n + 1, n,y)) - V(n,y) = V(n + l,y(n + 1, n,y)) - V(n,y)
+ V(n + l,y(n + 1, n,y)) - V(n + l,y(n + 1, n,y)) ~ -c(lly(n + 1, n, y))11 + Mlly(n + 1, n, y) - y(n + 1, n,y)1I ~ -c(lly(n + 1, n, y)11) + M8 2 < -c(lly(n + 1, n, y)ll) + c(8 1 ) . (4.11.4)
121
4.11. Total and Practical Stability
Now suppose that there is an index n\:?: no and a y; E Bat such that y(nj, no, y') e Be and y(n, no, y') E Be for n < n\. It then follows that V(nj, y(nl> no, y'»
:?:
a(e) > b(28\)
:?:
b(8\
+ 82 ) ,
and V(no, y') < b(8\). Then there exists an index n2 E [no, n\ - 1] such that V(n2,y(n2' no,y'»::s
s(s, + 8z) (4.11.5)
and V(n2 + 1, y(n2 Thus
+ 1, no, y'»
:?:
b(8\
+ 82 ) ,
Ily(n2 + 1, no, y')11 :?: 8\ + 8z from which we get 11Y'(n2 + 1, nz , y(n z , no, y')11 :?: IIy(nz + 1, no, y')11 -lly(n z + 1, »«, y(n2, no, y'» - Y(n 2 + 1, »«. y(n z , no, y'))II :?:
8\ + 8z - 82
= 8\.
Then from (4.11.5) and (4.11.4), it follows that
o~
+ 1, y(nz + 1, no, y'» - V(n z , y(n z, no, y'» < -c(IIY'(n2 + 1, n«, y(n2' no, y')II) + c(8\) ::s 0, V(nz
which is a contradiction.
•
Corollary 4.11.1. Suppose that the hypothesis of Theorem 4.11.1 is verified and moreooer.for y; E B; one has IIR(n, Yn)II < gnilYn II with gn ~ 0 monotonical/y. Then the solution of the perturbed equation is uniformly asymptotically stable. Proof.
From (4.11.4) one has
-c(IIY'(n + 1, n, Yn) II) + Mg nllYn II. Suppose 0 < r < 8(80), and r < IIY(n + 1, n, Yn)11 < 8( 80), by the hypothesis on gn, it can be chosen an n\ E N:o such that for n :?: n\ one has Mg; llYn II < ~ V(n, Yn) -s
2-\c(r) and then
Ll V(n, Yn)::S
Then apply Theorem 4.8.5.
-!c(IIY'(n + 1, n, Yn).
•
Connected to the problem of total stability is the problem of practical stability of (4.11.2) and (4.11.1). In this case we no longer require that R(n, 0) ¥- 0 so that (4.11.1) does not have the fixed point at the origin, but it is known that IIR(n, 0)11 is bounded for all n. This kind of stability is very important in numerical analysis, where certain kinds of errors cannot be made arbitrarily small.
122
4.
Stability Theory
Definition 4.11.2. The solution Y = 0 of (4.11.2) is said to be practically stable, if there exists a neighborhood A of the origin and fl ~ no such that for n ~ fl the solution y(n, no, Yo) of (4.11.2) remains in A.
Theorem 4.11.2. Consider the equation (4.11.1) and suppose that in a set D c R the following conditions are satisfied (1) IIf(n, y) - f(n, y')11 ~ Lily - y'll,
L< 1,
(2) IIR(n, y)11 < 8. Then the origin is practically stable for (4.11.2).
Proof. Let Yn and Yn be the solution of (4.11.1) and (4.11.2) respectively. Set m; = llYn - Yn II then by hypothesis mn+1 -s Lm; + 8 from which it follows n-I 8 j llYn - )in II ~ L n IIyo - Yoll + 8 j~O L = L n II Yo - Yoll + 1 _ L' If Yo = Yo, we see that the distance between the two solutions will never exceed _8_. Thus choosing n > fl suitably, both of the solutions will l-L remain in the ball
B(O, _8_) and the proof is complete. 1- L
•
The next theorem generalizes the previous result. Theorem 4.11.3. Consider the equation (4.11.1) and a set D c R S • Suppose there exists two continuous real valued functions defined on D such that, for all XED (1) V(x) ;:::0,
(2) .:l V = V(f(n, x)
+ R(n, x» -
V(x)
~
w(x)
~
a,
for some constant a;::: O. Let S = {x E 151w(x);::: O}, b = sup{V(x)lx E S} and A = {x E 15 IV(x) ~ b + a}. Then every solution, which remains D and enters A for n = ii; remains in A for n ;::: fl.
Proof. Suppose that Yn = y(fl, no, Yo) E A, then V(Yn) ~ b + a and V(Yn+I) -s V(Yn) + w(Yn). If w(Yn) is less than zero, Yn e S, then V(Yn+I) ~ b + a from which it follows that Yn+1 E A. If Yn E S, then, because V(Yn) ~ b, again it follows that V(Yn+I) ~ b + a. The proof is complete by induction. • Corollary 4.11.2. If 8 = sup{ w(x) IXED - A} < 0 then each solution y(n) of (4.11.1), which remains in D enters A in a finite number of steps.
4.12.
123
Problems
Proof. From 11 v s w(y) we get V(Yn):S; V(Yn) + 1:;=no w(Yj»:S; V(Y"o) + I)(n - no), from which it follows that V(y(n» ~ -<X) as n ~ 00, and this is a contradiction because V(Yn) 2': b + a for Yn E D - A. •
4.12.
Problems
4.1.
Show that if in the equation X n+1 = anxn, the product bounded, then the zero solution is uniformly stable.
4.2.
Consider the following equation, Yn+1 = n
In:no ail is
~ 1 y~. Compute, by using
a calculator, the two solutions starting from Yo = 10- 2 \ n = 0 and Yo = -1, no = 210 2 • Conclude that the zero solution is not uniformly stable. 4.3.
Show that for the system (4.1.8), the origin is the only fixed point that is attractive but not stable.
4.4.
Let Y = (x,
z) be such that Yn+1
=
(x~2x- z~).
Show that the set
nzn
D = {(x, z) Ix 2 + Z2 :s; 1} is the region of asymptotic stability of the origin. (Hint: Apply Theorem 4.9.3. Take (y) = x 2 + Z2 and show that if there is convergence, then which V(Yo) 4.5.
=
x~
n:}
1
(1 + (Yj»
=
1
1-
2
Xo -
2
Zo
from
+ z~).
. , . log(n + 2) . Consider the linear scalar equation Yn+1 = ( ) Yn' Find the log n + 3 solutions and show that the zero solution is asymptotically but not exponentially stable. (Hint: Put z; = Yn log(n + 2».
4.6.
Show that the zero solution of the previous problem is not Ip stable for any p 2': 1.
4.7.
Show that the function 8(e, no) in the definition of stability can be taken of class K. (Hint: Take 81(10 ) = sup 8(10, no). The function 1)1(e) is positive and strict increasing. Take l/J E K such that l/J( e) :s; I)I(e). This function can be used instead of I) in the definition.)
4.8.
Show that an alternative definition of stability is: The trivial solution is stable if there exists E K and nl E N;o such that for n 2': nl, Ily(n, no, Yo) I :s; (1IYoll, no). (Hint: Take as the inverse of rJ! defined in the previous problem.)
4.9.
Using Theorem 4.8.5 study the behavior of the solutions of the equation Yn+1 = y~2.
124
4. Stability Theory
4.10. Consider the equation Yn
o
+ b, where
AYn-l
=
o 0 0
A=
0 0 1
-2:
o
o
o
SXS
and study the stability of the critical point. The problem arises in economics treating oligopoly systems. The case s = 2 was treated by Cournot in finding asymptotic stability. 4.11. Show that the function g(n, u) = u; - w(un ) in Corollary 4.8.3 is not decreasing and prove that the origin is asymptotically stable for (4.8.6). 4.12. Show that O(Yo), the limit set of Yo E R 4.13.
.
•
=
Consider the equation Yn+1 of
1]
1] -
h~
1 - Yn
S ,
is invariant and closed.
.
, Yo = 1]. Determine the values
for which Yn converges and find the limit point.
4.14. Consider the system Xn + 1 =
Yn
Yn+1 = x,
+ f(x n),
»
and suppose that a(xJ(xn > 0 for all n. Show, by using Theorem 4.8.7 that the solutions are either unbounded or tend to zero. (Hint: Take Vn = xnYn.)
4.13.
Notes
The definitions in Section 4.1 are modified versions of those given for differential equations, for example, see Corduneanu [30] and Lakshmikantham and Leela [90]. Theorem 4.1.1 is adapted from Gordon [61]. The
4.13.
Notes
125
results of Section 4.2 are adapted from Corduneanu [29]. Most of the material of Section 4.4 has been taken from Halanay [68,69,70], see also [46]. The contents of Sections 4.5, 4.7, and 4.8 have been adapted from Halanay [68] and Lakshmikantham and Leela [90]. For Theorem 4.8.4, see Gordon [61]. Theorems 4.8.5 and 4.8.6 are due to Hurt [81] (see also LaSal1e [88]). Theorem 4.9.3 is the discrete version of Zubov's theorem and can be found in O'Shea [120] and Ortega [125]. Theorem 4.9.4 is due to Diamond [43,44]. The converse theorems are adapted from Lakshmikantham and Leela [90] and Halanay [68]. The total stability in the discrete case is treated by Halanay [68] and Ortega [125]. Practical stability is discussed in Hurt [81] and Ortega [125].
CHAPTER 5
5.0.
Applications to Numerical Analysis
Introduction
Despite the fact that the two theories have been developed almost independently, the connections between numerical analysis and the theory of difference equations are several. In this chapter, we shall explore some of these connections. In Sections 5.1 to 5.3, iterative methods for solving nonlinear equations are discussed and the importance of employing the theory of difference inequalities is stressed. Sections 5.4 and 5.5 deal with certain classical algorithms from the point of view of the theory of difference equations. Sections 5.6 and 5.7 are devoted to the study of monotone iterative techniques, which offer monotone sequences that converge to multiple solutions of nonlinear equations. This study also includes extension of monotone iterative methods to nonlinear equations with singular linear part as well as applications to numerical analysis. In Section 5.8 we provide problems of interest.
5.1.
Iterative Methods
By iterative methods one usually means methods by which one is able to reach a root of a linear or nonlinear equation. Let us consider the problem of solving the algebraic equation of the form F(x) = 0,
(5.1.1)
127
128
5.
where F: R
S
~
R
S ,
Applications to Numerical Analysis
which is usually transformed into the iterative form:
Xno = Xo. (5.1.2) The function f(x) is called the iterative function and is defined such that the fixed points of f are solutions of (5.1.1). There are many ways to define the iterative function f. The essential criterion is that the root x* of (5.1.1), which we are interested in, is asymptotically stable for (5.1.2), although it may not be sufficient to ensure rapidity of convergence. This, however, will imply that if Xo lies in the region of asymptotic stability of x*, the sequence will converge. If f is linear, we know that asymptotic stability can be recognized by looking at the eigenvalues of A (the iterative matrix) and indeed this is what is done in the study of linear iterative methods such as Jacobi, Gauss-Seidel, SOR, etc. Moreover, in the linear case, asymptotic stability implies global asymptotic stability. For nonlinear equations the situation becomes more difficult. In this case there are essentially three kinds of results that one can discuss: Xn+l
= f(x n ) ,
A. LOCAL RESULTS. These results ensure asymptotic stability of x*, but they have nothing to say on the region of asymptotic stability. Usually theorems in this category start with the unpleasant expression "If Xo is sufficiently near to x* ...." B. SEMI LOCAL RESULTS. The results in this category verify that an auxiliary positive function (usually the norm of the difference of two consecutive terms of the sequence) is decreasing along the sequence itself. The sequence is supposed to (or one proves that it) lie in a closed set D c R Then one can infer that the auxiliary function has a minimum in D and this minimum is located at x*. We shall see that this usually requires that x* is exponentially stable. Connected to these types of results are those requiring the stronger condition of contractivity off in a closed set D c R Contractivity implies exponential stability of the fixed point x*. C. GLOBAL RESULTS. These results say that the sequence x, given by (5.1.2) is almost (except, on a set of measure zero) globally convergent. S
•
S
•
Of course, results in the class (C) are very few in the nonlinear case. One such result is due to Barna and Smale, which says that for a polynomial with only real root, Newton's method is almost globally convergent, which we shall not present here. We shall, however, discuss in the next two sections some of the most important results in the classes (A) and (B).
5.2. Local Results
"
Let us begin with the following main result. Here f: R x* = f(x*).
S
~
R and x* satisfies S
5.2.
129
Local Results
Theorem 5.2.1. Suppose that f is differentiable with Lipshitz derivative, and the spectral radius off'(x*) is less than one. Then x* is asymptotically stable. Proof. By the transformation eTj difference equation en + 1
= f(x n) - f(x*) = f'(x*)en + =
where
f'(x*)en
=
xTj - x*, from (5.1.2) we obtain the
L
{f'(x*
+ s(xn -
x") - f'(x*)} ds en
(5.2.1)
+ g(en)
l~~ II~~:II)II = 0,
(see problem 5.2). The equation (5.2.1) is in the
form required by Corollary 4.7.2, which assures the exponential asymptotic stability of x*. Note that the exponential asymptotic stability does not imply in general that /lxl - x*11 < Ilxo - x"], unless (see section 4.2) aT] < 1, where the constant a depends on the norm used. The result established in Chapter 4 can be used to prove the next theorem on the convergence of nonstationary iterative methods defined by (5.2.2) where q : N;o x D c R ~ R , XO E D. It is obvious that nonstationary iterative methods are, in our terminology, nonautonomous difference equations. • S
Theorem 5.2.2.
S
Consider the difference equation
yno = X o ,
(5.2.3)
where f: D ~ R S , and f is locally Lipshitzian. Suppose that y* E D is an asymptotic stable solution for (5.2.3) and that the solutions y(n, no, x o) of (5.2.3) and x(n, no, xo) of (5.2.2) are in D for n E N;o' Assume further that
Ilq(n, x) - f(x)11 ~ Lnllx where L; ~ 0 for n ~ suitably chosen.
Proof.
y*ll,
(5.2.4)
Then the solution x(n, no, xo) ~ y*, when
00.
Xo
is
Rewrite (5.2.2) as Xn+I
= f(x n ) + R(n, x,)
(5.2.5)
and consider (5.2.5) as the perturbed equation 4felative to (5.2.3) with (5.2.6)
130
5.
Applications to Numerical Analysis
as the perturbation, which tends to zero as x, 4.11.1. •
~
y*. Then apply Corollary
Other results of local type can be obtained by using the comparison principle given in section 1.6. For example: Theorem 5.2.3.
Let f: D c R S ~ R S continuous, g: R+ ~ R+. Suppose that
(1) there exists x*
E
Int D such that for all x
E
B(x*, ll) < D, II
> 0,
(5.2.7) Ilf(x) - x*11 :5 [x - x*11 + g(llx - x*Ii); u + g(u) is nondecreasing with respect to u and g(O) = 0;
(2) G(u) = (3) the trivial solution of
Un + 1 =
G(u n ) ,
Uo = Ilxo - x*ll,
is asymptotically stable, with U o in the domain of asymptotic stability, and u, :5 Uo for all i. Then x* is an asymptotically stable fixed point for the equation
(5.2.8) Proof. Let Xo E B(x*, ll). Then by Theorem 1.6.1, we have [x, - x*11 :5 u; and, because Un is decreasing to zero, all x, E B(x*, 8) and the sequence will tend to zero. •
The next two theorems are applications of LaSalle's Theorem 4.8.6. Here we study the convergence of secant and Newton methods on the real line. These two methods are widely used in numerical analysis to find roots of equations and their generalizations are almost countless. On the real line, these two methods are defined by (5.2.9) and (5.2.10) respectively. Iff(zk) ¥- f(Zk+I), the difference equation (5.2.9) is well defined. Suppose that f is differentiable and that a is the simple root of f( z) = O. Using the transformation (5.2.11) and the mean value theorem f(a
+ e)
= f'(a)
+ g(a,
e)e 2 ,
(5.2.12)
S.2.
Local Results
131
we write the new difference equation corresponding to (5.2.9) having the fixed point at the origin in the form (5.2.13) where _ g(a, en+l)en+ l - g(a, en)e n M( a, en, en+ l ) ( . / a + en+!) - /(a + en)
(5.2.14)
If a is a simple root and g( a, e) is continuous and bounded, then Mt;«, ek, ek-l) is continuous and bounded if ek, ek-l are small enough. The second order difference equation (5.2.13) can be transformed into a first order system by introducing the variables x, = en and Yn = en+ l • Then the system is
= Yn, Yn+l = M(a, xn,Yn)xnYn' Lyapunov function V(x, y) = Ix[q + Iylq, X n+ l
By using the
q
2':
LlV(xn, Yn) = -(1 -IM(a, x n, Yn)Ynlq)lxnl
1, one obtains
q,
which is negative if W(x,y);;;: (l-IM(a,x,y)yj) > O. Let D(1) = {(x,y)I(!xl q + Iylq)l/q < 1)}. Since y = 0 implies W(x, y) = 1 > 0, there exists some 1) > 0 such that W(x, y) 2': 0 in D( 1). If the initial values Zo and ZI are such that (x o, Yo) and (Xl> Yl) E D( 1), then (x n , Yn) will remain in D (see Corollary 4.8.5) and will converge to the maximum invariant set contained in the set E = {(x, y) E 15(1) Ix = O}. The only invariant set with Xl = 0 is the origin, so we obtain (xn, Yn) ~ (0,0) for n ~ 00. Corollary 4.8.5 can be used to study the convergence of (5.2.10). Because there is no more difficulty involved we shall consider the case where (5.2.10) is defined in R S • The root a, which we are interested in, is simple. The change of variable similar to (5.2.11) allows us to write the method in the form where M l ( en) -_ [d/(a It can be
+ en)]-l and M ( en) -_ [d/(a + en)] . 2
ax shown that for I enII B( Xo, f31") c
D.
Then the sequence defined by (5.3.6) is well defined and converges to a root x* of F(x) = 0 contained in B(xo, I/f3'Y)· Proof.
For every
X E
Bt x«,
II {3'Y) we have
IIF'(x) - F'(xo)11 ~
'Yllx - xoll
< {3-I.
134
5.
Applications to Numerical Analysis
A bound of IIF'(x)-111 can then be obtained by using the Banach lemma (see Corollary A5.2), which yields Z*)
and (5) B( Xo, f31 c: Dare y satisfied. Then the sequence x, is well defined and converges quadratically to
x* contained in B( xo, (31y z*). Newton's method is invariant under affine transformation, that is, it is invariant under the transformation G = AF where A is an invertible matrix,
136
S. Applications to Numerical Analysis
whereas the hypotheses in the Newton-Kantarovich theorem are not. In fact, one has 0- 10 = F-1A-1AF = F- 1 F, while for example, the hypothesis (3) is not invariant under the same transformation, The following variant of Theorem 5.3.3 takes care of this situation. Theorem 5.3.3(b). and suppose that
Assume that the hypotheses (1) of Theorem 5.3.3 holds
II F'(XO)-I F(xo) II :s; a; IIF'(xo)-I(F'(y) - F'(x»11 :s; wllx - yll;
(2) (3) (4) (5)
aw:s;!; B(xo, w-1z*)
E
D, where z* = 1 - (1- 2wa)I/Z:s; 1.
Then the sequence x, remains in B(xo, w-1z*) and converges quadratically to a root x* of F(x).
Proof.
We have
Xn+Z - Xn+1 = -F'(Xn+I)-IF(xn+l)
= -F'(Xn+I)-I[F(xn) + F(xn+ 1 )
-
F(xn)]
-F'(Xn+I)-I{ F(x n) + F'(Xn)(Xn+1 - x n)
=
+ = -
f
[F'(x n + t(xn+1 - x n» - F'(x n)] dt(x n+1 - x n)}
F'(X n+1)-1 F'(xo){I F'(XO)-I[F'(xn + t(xn+1
-
»
xn
- F'(xn)] dt(x n+1 - Xn)}, from which it follows that Ilxn+z - xn+lll -s IIF'(xn+I)-1 F'(xo) II . ~ Ilxn+1 - x,
liZ,
Since F'(XO)-IF'(xn+ l ) == F'(XO)-I F'(Xn+l) - F'(xo)-I F'(xo) + F'(XO)-I F(xo), using hypothesis (3), we get 11F'(xO)-I(F'(xn+l) - F'(xo» II < wllxnn - xoll < z*:s; 1, if Xn+l E B(xo , w-1z*). Then, by the Banach lemma it follows that F'(x n+ l) is invertible and II F'(Xn+I)-1 F'(xo) II :s; 1-
II 1
W Xn + 1 -
II' Finally, we arrive at
Xo
IIXn+Z -
w Ilxn+1 - xnf xn+lll :s; -2 1 _ I _ II' w Xn+1 Xo
137
5.3. Semilocal Results
which is the same as before with f3y replaced by w. All the previous results then apply, and we are done. • The next result also shows the familiar quadratic decay of the errors in the Newton method, but it requires more assumptions on the inverse ofthe Jacobian.
Assume that F: Do c R S Moreover, suppose that
Theorem 5.3.4. (1) (2) (3) (4) (5)
~
R S and Do c R S is a convex set.
for x E Do, F is differentiable, for x E Do, F'(x) is invertible and IIF'(x)-111 for x, y E Do, 11F'(x) - F'(y)11 :5 yllx - yll, for Xo E Do, IIF'(xo)-IF(xo)II:5 TJ, and (\' = 4f3YTJ < 1.
:5
f3,
Then the Newton iterates remain in ei«; ro), where ro converge to a solution x* of F(x) = O. Proof.
= TJ 2:~~0 (\'ZH
and
From
IIx n+z -
xn+III =
IIF'(xn+ I)-IF(x n+I)11
:5
f3IIF(xn +1 ) II
and from the estimate of II F(xn + l ) I obtained in the previous theorem, we get (5.3.10) The comparison equation is therefore
f3y
Z
Un + 1 = TUn,
Uo
=
(5.3.11)
TJ,
whose solution is u; = TJ(\'zn_ l • Now applying comparison Theorem 1.6.1 we get (see problem 5.4)
[x, - xn+11 1:5
Un
= "1(\'zn_ l.
(5.3.12)
Since (\' < 1, the trivial solution 01 (5.3.11) is exponentially stable and hence it follows that x, is a Cauchy sequence. Furthermore, we also have [x, xoll :52:';:0 Uj = ro, which means that all the iterates remain in Bt x«, ro) and converge to x*. • Corollary 5.1.1.
The error [x" - Xn II satisfies the inequality
Ilx* - x, I :5 en IIxn -
Xn-I
liz,
for n
= 1,2, ....
(5.3.13)
As an application of Theorem 1.6.7, let us consider the two-step iterative method
138
5.
Applications to Numerical Analysis
Theorem 5.3.5. Suppose that G: D x D c R x R set Do c R the following inequality holds true S
S
~
R S and on some closed
S
,
IIG(x,y) - G(y,
z)ll:5 allx - yll + ~lly - zll, let there exist x o, XI E D such
where a + ~ < 1. Furthermore, that Xk E Do for k e: 1. Then the iteratives converge to the unique fixed point of G(x) = G(x, x). Proof. Setting Yk = Il xk + l - xkll, we obtain Yk+l:5 aYk + ~Yk-l' By Theorem 1.6.7, we then get Yk :5 Uk, where Uk is the solution of Uk+l = auc + ~Uk-I' which is Uk = coz~ + CIZ~, where ZI and Z2 are the roots of Z2 - az - ~ = O. Because of the assumption on a and ~, the two roots ZI and Z2 are less than 1 in modulus, which means that the zero solution of the comparison equation is exponentially stable. Using similar arguments as in the previous theorems, we can conclude that the sequence Xk is a Cauchy sequence. Since
IIG(x*) - Xk+111 :5IIG(x*, x") - G(x*, Xk-I)II + IIG(x*, Xk-I) - G(Xk' Xk-I)II :5 ~llx* - xk-t11 + allx* - xkll, the convergence of Xk shows that Xk+l ~ G(x*). The uniqueness of x* now • follows very easily and the proof is complete. The effect of perturbation on the iterative methods are of two different kinds. If the perturbations are small and tend to zero as n tends to 00, then the theorems on total stability will ensure that the fixed points continue to have asymptotic stability. In numerical analysis, however, perturbations may remain bounded for all n (roundoff errors, for example). In this case, a more convenient concept is the practical stability. If an iterative procedure is practically stable, the sequence of iterates will not tend to the solution x* but to a ball B(x*, B) surrounding x*. Inside B, the solution may oscillate but what is important is that it never leaves B. Of course it would be nice to have B as small as possible. Let us consider for example the iterative method
= Xo'
(5.3.14)
Ilf(x) - f(y) II :5 allx - yll
(5.3.15)
Xn+ 1 = f(x n ) ,
We assume that for x, y
E
Do c R
x na
S ,
with a < 1. Suppose that the errors in the computations perturb (5.3.14) by a bounded perturbation R n , that is
xn + 1 = f(x n ) + R n ,
xna = xo,
(5.3.16)
5.4.
139
Unstable Problems
with X n , xn E Do for all n. Then Theorem 4.11.2 can be applied. The conclusion is that the difference of the two solutions [x, - xn II will not exceed Rj(l- a), where R
B(Xn,~) I-a
=
sup Rn- The condition that all the balls
must be contained in Do may be very restrictive. In fact if a
is close to one, it follows that these balls are very large and the method is considered a bad one.
5.4.
Unstable Problems: Miller's, Olver's and Clenshaw's Algorithms
The situation that we shall analyze in this section and in the next is in a certain sense the opposite of that treated in the previous ones. In fact, there the limit point was important (and indeed in the theory of iterative processes the "solution" is the limit point), where the intermediate values z; are considered an unavoidable noise. Here the limit point will not be important (generally will not exist) and the important part ("the solution") will be a finite subset of the sequence itself. These problems are very typical in that part of numerical analysis which concerns the study of special functions, orthogonal polynomials, quadrature formulas, and numerical methods for ordinary differential equations. Consider the homogeneous scalar linear equation (see (2.1.5)) of order k. (5.4.1) It has k linearly independent solutions/l(n)'/in), ... '/k(n). Suppose that the solution Yn = 0 (n ~ 0) is unstable, and
. II(n) lim 1"( n ) = 0,
i = 2,3, ... , k.
(5.4.2)
n~oo.li
When this happens, the solution II(n) is said to be minimal. The problem is how to find II or a multiple of it. Even if we know the exact initial condition that generates the minimal solution, small errors in the calculations will, as usual, lead us to solve a perturbed equation LYn = en, whose solution (see (2.1.11)) will contain all the j;(n) and this will destroy the process (see for example [52], [175]), because the h(n) (i ~ 2) grow faster than II(n). The same problem appears, of course, in the nonhomogeneous case, where it is not excluded a priori that there exists a bounded solution even if all the hen) are unbounded. One is often interested in following this solution.
140
5.
Applications to Numerical Analysis
Consider, for example, the following first order equation Yn+l - (n
+ 1)Yn
= -1,
Yo = e.
(5.4.3)
The solution of the homogeneous part is Yn = en!, while the complete solution is Yn = n!( e -
2:.;=oh) ~ 2:.:=1 (n +l s
) (S) ,
which is bounded for
n ~ 00. If one tries to follow this solution, a small error in the initial data e (always existing because it has infinite digits), will be amplified by the factor n!. A simple idea is to find the way to look for the solution of the problem backwards (for n ~ -00), because in this case the origin becomes asymptotically stable for the homogeneous equation and the errors will be damped. This idea indeed works, as we will show in a simple case, which is the original proble~ to which it was applied (see [52] for references). Consider (5.4.6) where an, en ¥- 0 for all n, and the nonhomogeneous problem LYn = gn'
(5.4.7)
Suppose that two linearly independent solutions ¢n and l/Jn of the homogeneous equation are such that ¢n is minimal, that is
· ¢n 0. IIm-= "_00 t/Jn
(5.4.8)
Of course all solutions that are multiple of ¢n are minimal and vice versa. That means that only one appropriate initial condition is needed to determine the minimal solution we are interested in. The general solution of the nonhomogeneous problem is then (5.4.9) where
Yn
is a particular solution that will be supposed minimal too; that is,
· Yn 0 IIm-= I/Jn '
n_OO
(5.4.10)
and Yo = O. We are interested in the solution Yn
=
Yo,/, +_ ¢o't'n Yn'
A class of methods are based on the following result.
(5.4.11)
5.4.
141
Unstable Problems
Theorem 5.4.1.
Consider the boundary value problem Ly 0 and f( wo) :s; 0 it follows that Using the fact that B o is a subinverse of M (w o), we find for any [v, w],
Proof.
WI :s; W o'
u
E
u
+ Bof(u) =
WI -
:s; WI -
Hence, in particular, vo:S; Vo f( WI)
:s;
(w o - u) - Bo[f(w o) - f(u)J
[I - BoM(wo)](wo - u):s;
+ Bof( vo) :s;
f( wo) + M( woH Wo -
WI) =
WI'
WI'
Similarly, we obtain
[I - M( wo)BoJf(wo) s
o.
Proceeding similarly we see by induction that f(w n ) =:: 0,
n = 1,2, ....
(5.6.4)
Consequently, as a monotone nonincreasing sequence that is bounded below, {w n } has a limit r 2: Vo. If u is any solution of (5.6.1) such that u E [v, w J, then u = u + Bof( u) :s; WI' then by induction u :s; W n for all n. Hence u :s; r. Finally, the continuity of f and the fact lim inf B; =:: B, where B =:: 0 is nonsingular, (5.6.4) yields n
= -(lim inf B; )f( r) =:: -
Bf( r)
2:
0,
n~OO
which implies f(r) = 0 completing the proof.
•
By following the argument of Theorem 5.6.2, one can prove the next corollary.
5.7.
147
Monotone Iterative Methods
Corollary 5.6.1.
Let the assumptions of Theorem 5.6.2 hold. Suppose that M(y) is monotone non increasing in y. Then the sequence {v n } with Vo = v, given by V n+1
= Vn + Bnf(vn)
is well defined, monotone nondecreasing such that lim minimal solution of (5.6.1). n-+OO
Vn
= p, and is the
The case of most interest is when p = r, because then the sequences {v n }, {w n } constitute lower and upper bounds for the unique solution of (5.6.1). The following uniqueness result is of interest.
Theorem 5.6.3. Suppose that f(y) - f(x):5 N(x)(y - x), where N(x) is nonsingular matrix and N(X)-l ~ O. Then if (5.6.1) has either maximal or minimal solution in [v, w], then there are no other solutions in
[v, w]. Proof. Suppose r E [v, w] is the maximal solution of (5.6.1) and u is any other solution of (5.6.1). Then
0= f(r) - f(u)
:5
E [v, r]
N(u)(r - u).
Since N(U)-l ~ 0, it follows that r ~ u and hence r = u. A similar proof holds if the minimal solution exists. The proof is complete. •
5.7.
Monotone Iterative Methods (Continued)
Consider the problem of finding solutions of Ax = f(x),
(5.7.1)
where A is an n x n matrix and f E C[R n, R n ] , which arises as finite difference approximation to nonlinear differential equations. If A is nonsingular, writing F(x) = f(x) - Ax = 0, one can study existence of multiple solutions by employing the method of upper and lower solutions and the monotone iterative technique, described in Section 5.6. In this section we extend such existence results to equation (5.7.1) when A is singular. For convenience let us split the system (5.7.1) and write in the form (5.7.2)
148
5.
Applications to Numerical Analysis
where for each i, 1 ~ i ~ n, u = (Ui' [U]Pi' [u]q') with Pi + qj = n - 1 and (Au); represents the i l h component of the vector Au. As before, a function f E C[R", R"] is said to be mixed quasi-monotone if, for each i, /; is monotone non decreasing relative to [u ]p, components and monotone nonincreasing with respect to [u ]q, components. Let v, w E R" be such that v ~ w. Then v, ware said to be coupled quasi lower and upper solutions of (5.7.1) if
Coupled quasi-extremal solutions and solutions can be defined with equality holding. We are now in a position to prove the following result. Theorem 5.7.1. Assume that (i) f E C[R n , R n ] and f is mixed quasi-monotone; (ii) v, ware coupled quasi lower and upper solutions of (5.7.1); (iii) /;(u;, [u]p" [u]q') - /;(u;, [u]p" [u]q,) ~ -Mj(u j - uJ whenever v ~ u ~ u ~ wand M, > 0, for each i; (iv) A is an n x n singular matrix such that A + M = C is nonsingular where M is a diagonal matrix with M, > 0 and C- I ~ O. Then there exist monotone sequences {v n }, {w n } such that V n ~ p, W n ~ r as n ~ 00 and p, r are coupled quasi-extremal solutions of (1.1) such that if u is any solution of (1.1), then v :5 P ~ u ~ r ~ w.
Proof. For any YJ, I-t system
E
[v, w]
=
[x
E
Rn:v
~
x
~
w], consider the linear
(5.7.3)
Cu = F( YJ, I-t),
where C=A+M and for each i, Fj(YJ,I-t) =/;(YJk, [YJ]p,,[I-t]q,) + MjYJj. Clearly u can be uniquely defined given YJ, I-t E [v, w] since the matrix C is nonsingular. Furthermore, we have Av ~ F(v, w), Aw ~ F(w, v), and F is mixed monotone. Consequently we can define a mapping T such that T[ YJ, I-t] = u and show easily, using the fact that C- I ~ 0, that (a) v ~ T[v, w], w ~ T[w, v]; (b) T is mixed monotone on [v, w]. Then the sequences {v n } , {wn } with follows:
Vo
= v,
Wo
=w
can be defined as
S.7. Monotone Iterative Methods
149
Furthermore, it is evident from the properties (a) and (b) that {vn } , {w n } are monotone such that Vo ~ VI ~ ••• ~ vn ~ Wn ~ ••• ~ WI ~ Wo0 Consequently, lim e, = p, lim Wn = r exist and satisfy the relations n-JoOO
n-JoCO
By induction, it is also easy to prove that if (u l , U2) is a coupled quasisolution of (5.7.1) such that v ~ UI, U2 ~ w, then V n ~ ul , U2 ~ Wn for all nand hence (p, r) are coupled quasi-extremal solutions of (5.7.1). Since any solution u of (5.7.1) is a coupled quasi-solution, the conclusion of the theorem follows and the proof is complete. • The case of most interest is when p
=
r, because then the sequences {vn },
{w n } constitute lower and upper bounds for the unique solution of (5.7.1).
The following uniqueness result is thus of interest. Corollary 5.7.1. for each i, that
If, in addition to the hypotheses of Theorem 5.7.1, we assume (5.7.5)
where B is an n x n matrix such that (A - B) is nonsingularand (A - B)-I O. Then u = p = r is the unique solution of (5.7.1) such that v ~ u ~ w. Proof.
Since p
~
r, it is enough to prove that p
2::
2::
r. We have by (5.7.5)
[A(r - p)]j ~j;(ri' [r]pi' [p]q) - j;(Pj, [P]Pi' [r]q) ~ [B(r - p)]j.
Consequently, (A - B)(r - p)
~
0, which implies r ~ p.
•
Remark 5.7.1. If qi = 0 for each i, then Theorem 5.7.1 shows that r, pare maximal and minimal solutions of 5.7.1 and Corollary 5.7.1 gives the unique solution.
If f does not possess the mixed quasi monotone property, we need a different set of assumptions to generate monotone sequences that converge to a solution. This is the content of the next result. Theorem 5.7.2. Assume that (iv) of Theorem 5.7.1 holds. Further, suppose that V, W E R" such that v ~ W, Av
~f(v)
- B(w - v),
-B(x - y)
~f(x)
Aw 2::f(w) - f(y)
~
+ B(w -
B(x - y)
v),
(5.7.6) (5.7.7)
150
5.
Applications to Numerical Analysis
whenever v ~ y ~ x ~ w, B being an n x n matrix of nonnegative elements. Then there exist monotone sequences {v n }, {w n } that converge to a solution u of (5.7.1) such that
provided that (A - B) is a nonsingular matrix. Proof.
We define F(y, z)
= ![f(y) + fez) + B(y -
z)).
(5.7.8)
It is easy to see that F(y, z) is mixed monotone and
-B(z - z)
~
F(y, z) - F(y, z)
whenever z, z, y, Y E [v, w] and i
~
~
B(y - y)
(5.7.9)
z, Y ~ y. In particular, we have
F(y, z) - F(z, y)
= B(y -
z).
(5.7.10)
From (5.7.8) we obtain -B(w - v) ~ F(v, w) - f(v), which yields Av ~ f(v) - B(w - v) ~ F(v, w).Similarly Aw ~ F(w, v). Finally, it follows from (5.7.8) that F(x, x) = f(x). Consider now for any Tl, JL E [v, w], the linear system given by Cu = O( Tl, JL) where C = A + M, M being the diagonal matrix with M; > 0 and for each i, OJ( Tl, JL) = F; ( Tl, JL) + MjTli' Proceeding as in proof of Theorem 5.7.1, we arrive at Ap = F(p, r), Ar = F(r, p). Using (5.7.9), we see that A(r - p) = F(r, p) - F(p, r) = B(r - p) and this implies u = r = p is a solution of (5.7.1). The proof is complete. • Corollary 5.7.2. If in addition to the hypotheses of Theorem 5.7.2 we also have f(x) - fey) ~ C(y)(x - y) for x, y E [v, w], where C(y) is an n x n matrix such that [A - C(y)] is nonsingular and [A - C(y)r 1 ~ 0, then u is the unique solution of (5.7.1).
Proof.
If ii is another solution of 5.7.1, we get A(u - ii) = feu) - f(ii)
which yields u (5.7.1). •
~
ii. Similarly, ii
~
~
C(ii)(u - ii),
u and hence u is the unique solution of
Equations of the form (5.7.1) arise as finite difference approximations to nonlinear partial differential equations as well as problems at resonance where the matrix is usually singular, irreducible, M-matrix. For such matrices the fol1owing result is known.
5.8.
151
Problems
Theorem 5.7.3. Let A be an n x n singular, irreducible, M-matrix. Then (i) A has rank (n - 1); (ii) there exists a vector u > 0 such that Au = 0; (iii) Av ~ 0 implies Av = 0; and (iv) for any nonnegative diagonal matrix D, (A + D) is an M-matrix and (A + D)-t is a nonsingular M-matrix, if d jj > 0 for some i, 1 ~ i ~ n. As a consequence of Theorem 5.7.3, if we suppose that A in (5.7.1) is n x n singular, irreducible, M-matrix, then (A + M) is a nonsingular M-
matrix. Therefore assumption (iv) of Theorem 5.7.1 holds since nonsingular M-matrix has the property that its inverse exists and is greater than or equal to zero. Furthermore, in some applications to partial differential equations, the function f in (5.7.1) is of special type, namely feu) = (ft(Ut),/zCU2),'" '/n(u n». We can find in this special case lower and upper solutions such that assumption (ii) of Theorem 5.7.1 holds whenever we have lim sup};(uJ Sig(uj) < 0 IuJ... co
for each i.
In fact, by Theorem 5.7.3, there exists a ~ > 0 such that Ker A = and hence we can choose a A > 0 so large that f(A~) ~
0
Span(~)
and f( -A~) ~ O.
Letting v=-A~, w=Ag, we see that v~w, Av~f(v) and Aw~f(w). Assumption (iii) is simply a one-sided Lypschitz condition and in assumption (5.7.5), B is a diagonal matrix.
5.8. Problems 5.1.
Show that p( G'(x*» = 0 where G(x) is a simple root of F(x) = O.
5.2.
Show that if F: R S
=x -
(F'(X»-l F(x) and x*
R has a Lypshitz derivative then
~
IIF(x) - F(y) - F'(x)(y -
x)11 ~ hllx - y112.
5.3.
Show that if in the Newton-Kantarovich theorem one supposes IIF'(x)-tll bounded for all x E R S , then [x, - x*II ~ cllxn - 1 - x*II2•
5.4.
Show that in the hypothesis of Theorem 5.3.4, the error satisfies the inequalities (5.3.12) and (5.3.13).
5.5.
Suppose that
x, E R
S ,
n
~ 0 and Il.Ilxnll ~
n
.Ilt
.Il
n- t
lI.1lxn-t1I, where
a positive converging sequence with to = 0 and lim tn n~CO
that
Xn
converges to a limit x".
= r:
tn is
Show
152
5.
Applications to Numerical Analysis
IlllxnI s; ( Iltn)'Y (1IIlxn-lll)Y Ilt
5.6.
As in the previous exercise suppose that
5.7.
Show that the solution of
5.8.
Find the constant k in the previous problem and deduce that for Newton's method one has:
Zo -
n- I
~z~
IS given by Zn = 1 - Zn 1 - (1 - 2zo) 1/ 2 cotght cz"), with k appropriately chosen.
'I
X
*-
Xn
Zn+1
I s; -(1 2 f3y
=
2'
2zo)1/2 - -(J2 - " 1 - (J
where 1 - Zo - (1 - 2Z0) 1/ 2 (J = ---=--'-----=..:--:-= 1 - Zo (1- 2Z0 ) 1/ 2 '
+
5.9.
Obtain the result of Theorem 5.3.3 by considering the first integral with Zo = ~. (Hint: in this case the first order equation becomes Zn+1 = ~(1 + zJ.)
5.10. Consider the iterative method
= JL[AG(x n ) + (1- A)G(xk, Xk-I)] + (1- JL)Xk, A, JL E [0,1] and G, G are defined as in Theorem 5.3.5. Show Xk+1
where that JL and A can be chosen such that the convergence becomes faster. 5.11. Solve equation (5.5.1) directly and show the growth of the errors. 5.12. Obtain the relations (5.5.2), (5.5.3) and (5.5.4). 5.13. Show that the sweep method is equivalent to the Gaussian elimination of the problem Ay = g when the problem 5.5.1 is stated in vector form, see (5.4.16).
5.9. Notes The discussions of Sections 5.1 to 5.3 have been introduced only to show examples of application of difference equations to numerical analysis. More details can be found in the excellent books by Ortega and Rheinboldt [127] and Ostrowsky [132]. Theorem 5.3.3 is a simplified version of the original one, see [129], [127], while the estimates given before 5.3.4 (b) seem to be
5.9.
Notes
153
new. For other estimates of the error bounds for the Newton's method and Newton-like methods, see also [41], [109], [111], [179], [144]. For the effect of errors in the iterative methods, see Urabe [168]. A very large source of material on the problem discussed in Section 5.4 can be found in Gautschi's review paper [52]. Theorem 5.4.1 has been taken from Olver [123]. A detailed analysis, in a more general setting, of the Miller's algorithm is given in Zahar [180]. Applications to numerical methods for ODE's can be found in Cash's book [23], while a large exposition of applications as well as theoretical results are in Wimp's book [56], see also Mattheji [104]. The "sweep" method can be found in the Godunov and Rayabenki's book [59], where it is also presented for differential equations. An improvement of the Olver's algorithm can be found in [169]. An application of the sweep method to the problem of Section 5.4 can be found in Trigiante-Sivasundaram [167]. The contents of Section 5.6 are adapted from [89, 126, 127], while the material of Section 5.7 is taken from [91], see also [85, 87, 89].
CHAPTER 6
6.0.
Numerical Methods for Differential Equations
Introduction
Numerical methods for differential equations is one of the very rich fields of application of the theory of difference equations where the concepts of stability playa prominent role. Because of the successful use in recent years of computers to solve difficult problems arising in applications such as stiff equations, the connection between the two areas has become more important. Furthermore, in a fundamental work Dahlquist did emphasize the importance of stability theory in the study of numerical methods of differential equations. We shall consider in this chapter some of the most relevant applications. In Section 6.1 we discuss linear multistep methods again and show that the problem can be reduced to the study of total or practical stability when the roundoff errors are taken into account. In Section 6.2 we deal with the case of a finite interval where we shall find a different form of Theorem 2.7.3. Section 6.3 considers the situation when the interval is infinite restricting to the linear case. The nonlinear case when the nonlinearity is of monotone type is investigated in Section 6.4, while in Section 6.5, we show how one can utilize nonlinear variation of constants formula for evaluting global error. Sections 6.6 and 6.7 are devoted to the extension of previous results to partial differential equations via the method of lines, which leads to the consideration of the spectrum of a family of matrices in order to obtain the right stability conditions. The problems given in Section 6.8 complete the picture.
155
156
6.
Numerical Methods for Differential Equations
6.1. Linear Multistep Methods We have already seen that a linear multistep method (LM), which approximates the solution of the differential equation y'
= f(t, y),
(6.1.1)
is defined by
p(E)zn - hu(E)f(tn, zn) = 0,
(6.1.2)
where p(E) and u(E) are defined in section 2.7. For convenience we shall suppose that f is defined on IR X IR and it is continuously differentiable. The general case can be treated similarly but for some notational difficulties. The values of the solution y( t) on the knots tj = to + jh satisfy the difference equation
p(E)y(tn) - hu(E)f(tn, y(tn))
= Tn,
(6.1.3)
where Tn is the local truncation error. The global error
In
=
y(tn) - z;
(6.1.4)
satisfies then the difference equation
p(E)ln - hu(E)[f(tn, z; + In) - f(tn, Zn)]
= Tn·
(6.1.5)
In practice, equation (6.1.2) is solved on the computer and instead of (6.1.2) one really solves the perturbed equation (6.1.6) where en represent the roundoff errors. The equation for the global error then becomes (6.1.7) where (6.1.8) It is worth noting the different nature of the two kinds of errors. If the method is consistent Tn depend on some power of h, which is greater than one, while en is independent on such a parameter (it depends on the machine precision). For simplicity, we assume that en is bounded. (We note that en can grow as a polynomial in n, see problem 1.14). The equation (6.1.7) can be considered as the perturbation of the equation
(6.1.9) which has the trivial solution In = 0. The problem is then a problem of total stability or of practical stability if the roundoff errors Cn are taken in account. One needs to study equation
157
6.1. Linear Multistep Methods
(6.1.9) and verify whether the zero solution is uniformly asymptotically stable and then apply, for example, a theorem similar to 4.11.1 in order to get total stability. This has been done indirectly in the linear case. Recently some efforts have been devoted to certain nonlinear cases. The techniques used to solve the problem are different. One method is to use the Z-transform (with arguments similar to those used in the proof of Theorem 2.7.3 (See for example [117]) and the other is to transform the difference equation of order k to a first order system in R". We shall follow the latter approach. Let us put f(tn, Zn
+ In) -
(6.1.10)
f(tn, Zn) = cnln·
Equation (6.1.9) can be written (we take ak = 1) as In+k - h{3kcn+k1n+k
k-I
+I
j=O
(aj - h{3jCn+j)ln+j =
W n,
where b\n) = {3jCn+j - {3k a jcn+k J 1 - h{3k Cn+k .
By introducing the k-dimensional vectors En
= (1m, In+l, ... , In+k-I) T,
Wn = b(n)
=
(0, 0, ... ,1 -
(6.1.12)
hWn ) T (3kCn+k b(n) = (b~n), b~n), •.. , b~~I)T;
and the matrices
o o
1
o
o o
A=
o
o
1
(6.1.13)
IS8
6.
Numerical Methods for Differential Equations
° Bn =
(6.1.14)
equation (6.1.7) reduces to the form E n+ 1
=
(A + hBn)En + Wn.
(6.1.15)
The problem is now to study total stability for the zero solution of this equation, which depends crucially on the nonlinearity off contained in the matrix Bn • Historically this problem has been studied under some simplifying hypothesis that we shall summarize as follows: finite interval, infinite interval with f linear, infinite interval with f of monotone type.
6.2.
Finite Interval T
If the interval of integration (to, to + T) is bounded, one takes h = Nand
n = 0, 1, ... ,N This implies that h becomes smaller and smaller as N increases and the quantitiy hBnEn, like the term Wn, can be considered as small perturbation of the equation (6.2.1) The eigenvalues of the matrix A are roots of the characteristic polynomial p(z) and the requirement of the stability of the null solution of (6.2.1) is equivalent to O-stability. We cannot assume asymptotic stability of the zero solution and apply theorem 4.11.1 because one of the conditions of consistency impose p(l) = 0, which means that at least one of the eigenvalues of A is on the boundary of the unit disk in the complex plane. The analysis of the stability of the zero solution of (6.1.15) needs to be done directly. This is not difficult if we use (4.6.2). In fact n-l '\' An-j-l(W) + hE-E.) E n = AnE0 + L. (6.2.3) )). j~O
Taking the norms, we get
liEN I ~ IIA NII I Eoll + I IIA N N-l
- j -
j~O
111(II"'ili
+ hIIBjIIIIEJ).
(6.2.4)
Now suppose that the zero solution of (6.2.1) is stable (that is the same to demand that the method is O-stable). This means that the spectral radius
159
6.2. Finite Interval
of A is one and moreover the eigenvalues on the boundary of the unit circle are simple. It follows then (see Theorem 12.4) that the powers of A are bounded. Let us take for simplicity that IIAjl1 = 1,j = 1,2, ... , N, (see also Theorem A5.2). The inequality (6.2.3) becomes
N-l liEN I ~ IIEol1 + j=O L (II "'ill + hIIBjIIIIEJ).
By Corollary 1.6.2 we obtain liEN I
~ I Eoll ex p( h ;tolIIBjll) + JOl I "'ill ex p( h T:~:l IIBTII) ~
IIEol1 e
NLh
+
N-l
L I "'ill
(6.2.5)
ehL(N-j-l),
j~O
where L
=
max
}:$T::sN
IIBTII. We let max
O::SJsN
I "'ill = 8 + r(h),
(6.2.6)
Then (6.2.5) becomes e
LT
-1
liEN I ~ IIEol1 e L T + e hL -1
(8 + r(h)).
(6.27)
Usually the initial points are chosen such that lim
N->oo
Hence, taking the limit as h lim liEN I h->O
~
IIEol1 = o.
(6.2.8)
0, we have
~ (e L T
-1) lim h->O
e
e
~L r(h), - 1
from which it is clear that if 8 ,e 0 (the roundoff errors are considered), the right-hand side is unbounded. It is possible to derive for I EN I a lower bound with a similar behavior. This means that the zero solution is not practically stable. If the roundoff errors are not considered, and if the method is consistent (see definition 2.7.1) then 8 = 0 and r(h) = O(h P + l ) , P ;::: 1 and therefore lim h->O
IIEnl1 = 0
(6.2.9)
and this implies that the null solution is totally stable. It is possible to give a more refined analysis of (6.2.3) by considering in the limit only the components of the matrix A (see Appendix A), corresponding to the eigenvalues on the unit circle, that have nonzero limit for n ~ 00 (see problem 6.1).
160
6.
Numerical Methods for Differential Equations
This analysis allows us to weaken the conditions that we impose on the local error. If, as usually happens in the modern packages of programs designed to solve the problem, one allows the stepsize to vary at each step, the difference equation for the global error is no longer autonomous even for autonomous differential equations. Gear and Tu obtain the difference equation (6.2.10) where the matrix S(n) takes into account many other factors that define the method. It is shown that (6.2.11) Suppose that the two conditions hold: (1) (2)
1I 0 and TJ < 1 such that (6.3.8) It then follows that
liEn II -s a77"-"oIIEoll + a
n-I
I
77"-j- 1 11«jll,
j=no
from which one derives at once the stability of En = 0 for the perturbed equation. We have treated so far the case of a single equation. The case of autonomous systems y'
= Ay,
y(to) = Yo,
(6.3.9)
can be treated similarly. In fact, suppose that the matrix A can be reduced to the diagonal form by means of a similarity transformation A = T-1AT where A = diag(A 1 , A2 , ••• , A). Then the system (6.3.9) reduces to uncoupled scalar equations and the condition becomes that each hs, must be inside the absolute stability region.
163
6.4. Nonlinear Case
6.4.
Nonlinear Case
The study in the nonlinear case of equation (6.1.15) is difficult and has been made for only nonlinearities of a special kind, namely, monotone. The techniques used require essentially to bound norms of the matrix A + hBn. We shall be content to present only the case of the implicit Euler method and the one-leg methods, since they are good examples of the use of the equations employed here. Let f: R x R ~ R and suppose that for u, v E R, S
S
(6.4.1) with p., :::; O. The difference equation (6.1.15) becomes in this case In+1 - In - h[f(tn+l'
Zn+l
+ In+1) - f(t n, zn)] == W n·
(6.4.2)
By multiplying both sides by 1~+1 we have
II In +1 11 2 -s 1~+1 . In + hu. Il/n+t11 2 + 1~+1 W n (6.4.3) where the Cauchy-Schwarz inequality (6.4.4) has been used. From (6.4.3) one gets
Il/n+t11 :::; 1 _
1 hp., II In I
+1_
1 hp.,
II W n II·
(6.4.5)
By using Corollary 1.6.1, we then obtain (6.4.6) Since the quantity TJ ==
~h is less than one by hypothesis, the 1 - p.,
uniform
stability of the solution In == 0 follows. The previous arguments can be generalized to the class of methods defined by (6.4.7) This class of methods was proposed by Dahlquist who gave the name of one-leg methods. There is a connection between the solutions of the difference equation (6.1.2) and the solutions of (6.4.7) (See problems 6.6,
164
6.
Numerical Methods for Differential Equations
6.7). For the sake of simplicity, we shall suppose f: R x R ~ R. In this case the quantities in (5.4.1) are scalars. The error equation for (6.4.7) is p(E)ln - h[f«(J"(E) tn, (J"(E)zn
+ (J"(E)ln) - f«(J"(E)t n, (J"(E)zn)] = W n· (6.4.8)
Multiplying by (J"(E)ln one obtains (J"(E)ln' p(E)ln ~ hP,I(J"(E)lnj2 + W n ' (J"(E)ln'
Let us switch now to the vectorial notation (See 6.1.12) and suppose that there exists a symmetric positive definite matrix a such that is the Lyapunov function satisfies the inequality (6.4.9) It then follows that
Vn+ 1 - Y n ~ 2(J"(E)ln' p(E)ln 2
~ 2hp,1(J"(E)ln 1
+ 2Iwn(J"(E)lnl
2+ 2I(J"(E)lnllwnl.
~ 2hp,1(J"(E)ln 1
By considering that (here 1 0 and /3i > 0 for all indices. In this case it is easy to verify that AM> 0 (see sec. A.6). By Perron-Frobenius theorem (see Theorem A.6-2), it follows that A has a real simple eigenvalue AI associated with a positive eigenvector U I• The remaining eigenvalues are inside the circle BA ( in the complex plane. By using the expression (A2.1-6) for the powers of A and considering that in the present case ml = 1, one has A" = A~ [ ZII
+
s mk-1A"-i(n) ] Ik=2 Ii~O ~ . Zk; , AI I
from which it follows that the double sum in brackets tends to zero as n ~ 00. Considering that Zll is the projection to eigenspace corresponding to AI> one has A"yo ~ cA "UI> where c is a constant. The population tends to grow as A~ and is close to U I, that means that the distribution between the age groups tends to remain fixed. In fact, y;(n)
I Yi(n)
U;
(7.1.4)
~--
I
Uj'
;=1
The vector UI is called the stable age vector and rate. Of course the overall population will grow for guish, for AI < 1.
AI AI
the natural growth
> 1 and will extin-
178
7. Models of Real World Phenomena
Special cases of interest are those where some lli can be zero. To make the study simpler, it is useful to change variables. Letting II = 1, h = n~:J Pi (k = 2, ... , M), a, = li ll i (i = 1,2, ... , M), D = diag(lt. 12 " , . , 1M ) , x(n) = D-Iy(n) and B ='D-IAD, one verifies that
o
o
o
o
1
o
which is in the companion matrix (see section 3.3). The model is now x(n
+ 1) = Bx(n),
(7.1.5)
and the characteristic equation of B is then (see 3.3.3): (7.1.6) The following theorem, due to Cauchy, is very useful in this case. Theorem 7.1.1. If ai is nonnegative and the indices of the positive aj have the greatest common divisor 1, then there is a unique simple positive root Al of the greatest modulus. For a proof, see Ostrowski [131]. Since the eigenvector U I of B is = (A~-\A~, •.. , 1)T, it follows that U1 is a positive vector and all the previous results are still valid. When the hypothesis on the indices of Theorem 7.1.1 are not satisfied, then some eigenvalues may have the same modulus of Al giving rise to the interesting phenomenon of population waves. In this case, there are oscillating solutions to the model. To see this in a simpler way, let us consider the scalar equation for the population of the first group. This can be seen from (7.1.5). The first equation of the system gives U\
x\(n
+ 1) =
M
I
aixi(n)
(7.1.7)
i=1
and from the following equations, choosing appropriately the index n, we obtain xi(n) = xl(n - i + 1). By substitutions one has M
x\(n
+ 1) = I ajxl(n - i + 1), i=1
(7.1.8)
7.2. The Logistic Equation
179
which is a scalar difference equation of order M. The characteristic polynomial coincides with (7.1.6). The solution of (7.1.8) in terms of the roots of (7.1.6) is given by (2.3.7) with Ai instead of z.. If there are two distinct roots with the same modulus, say At = p e in 8 , A2 = p e- in8 , one has
+ c2p e- in 8 + other terms. It is possible to find two new constants a and r/J such that xt(n) = c.p" e
C2
= a e-io/J
in 8
and then
Ct =
a e'",
+ r/J) + other terms,
n
x\(n) = 2 pa cos(n6
from which follows that xt(n) is an oscillating function. In the general case more than one period can coexist. In the extreme case a, = 0, i = 1,2, ... , M - 1, aM ,e 0 a number proportional to M of periods can coexist and for large M a phenomena similar to chaos (see next section) may appear. The population waves have been observed in insect populations.
7.2.
The Logistic Equation
The discrete nonlinear model, which we are going to present for the dynamics of populations and which takes into account the limitation of resources, has been used successfully in many areas such as meteorology, fluid dynamics, biology and so on. Its interest consists in the fact that in spite of its simplicity, it presents a very rich behavior of the solutions that permits to explain and predict a lot of experimental phenomena. Let N(n) be the number of individuals at time n and suppose that N (n + 1) is a function of N(n), that is, N(n
+ 1) = F(N(n)).
(7.2.1)
This hypothesis is acceptable if two generations do not overlap. For small N, (7.2.1) must recover the Malthusian model, that is,
F(N(n))
=
aN(n)
+ nonlinear terms.
The nonlinear terms must take into account the fact that when the resources are bounded there must be a competition among the individuals, which is proportional to the number of encounters among them. The number of encounters is proportional to N 2 ( n). The model becomes N(n
+ 1) =
aN(n) - bN2(n),
(7.2.2)
with a and b positive. The parameter a represents the growth rate and b is a parameter depending on the environment (resources). The quantity a] b is said to be the carrying capacity of the environment. For a > 1, this
7.
180
equation has a critical point
N=
a
Models of Real World Phenomena
~ 1 which, as shown below, is globally
asymptotically stable for Yo E (0, 1) and a in a suitable interval. This means that lim N (n) = Nand N is a limit size of the population. N depends on n_oo
a and b, which can vary with the time: b can diminish because the population learns to better use the existing resources or to discover new ones; a can grow because the population learns how to live longer or how to accelerate the birth ratio. Anyway, taking into account the variation of N (which happens with a longer time scale), one sees the evolution of the population can be described as a sequence of equilibrium points with increasing values of N. In case of two similar species, it can be shown that the species with longer N will survive. This is the Darwinian law of the survival of the "fittest." Almost all the previous results are similar to those obtained in the continuous case. A further analysis of the logistic equation shows solutions that are unexpected if one thinks the discrete equation merely as an approximation of the continuous one. To see this, let us change the variable and
simplify the equation. Let Yn = ~ N(n). The model becomes (7.2.3 ) The new variable Yn represents the population size expressed in unit of carrying capacity a/ b. The equation (7.2.3) has two critical points (the term equilibrium points is used more in this context), the origin and ji = (a - 1)/ a, which has physical meaning only for a > 1. For a < 1, by using the theorems of stability by first approximation (see Section 4.7), one sees at once that the origin is asymptotically stable. It can be shown (see problem 7.3), that it is asymptotically stable for Ao in the interval (0,1). For a > 1, the second critical point becomes positive and the origin becomes unstable. The stability of ji can be studied locally by the linearization methods using again one of the theorems of Section 4.7. In fact Yn+l - ji = f'(ji)(Yn - ji)
+f'~) (Yn -
ji)2
= (2 -
a)(Yn - ji) - a(Yn _ ji)2.
If the coefficient of the linear term is less than one in absolute value then ji is asymptotically stable.
As a consequence, one has that for 1 < a < 3 the point ji is asymptotically stable. For a = 2, the solution can be found explicitly (see Problem 1.2.7). For a > 3, a new phenomenon appears. There exists a couple of points Xl and X 2 such that (7.2.4)
7.2. The Logistic Equation
The couple follows that
Xl
and
181 X2
is called a cycle of period two. From (7.2.4) it (7.2.5)
that is,
Xl
and X2 are fixed points of the difference equation (7.2.6)
One can determine the fixed points by using the Equation (7.2.5), which is a fourth degree equation (it contains as roots the origin and y, which are circles of any period). The following theorem permits us to simplify the problem. Let f[x, y] be the usual divided difference, that is, f[ X,Y ]
= f (x ) - f (Y ) . X-Y
Theorem 7.2.1. The solutions ofperiod two for the difference equation Yn+1 f(Yn) exist iff there exist values ofx E (0,1) such thatf[s,f(x)] =-1.
Proof.
=
By definition f[ x, f( X )]
= f ( x ) - f (2)(X) X _ f(x) .'
(7.2.7)
If f(2)(X) = X then (7.2.7) is equal to -1. On the other hand, if (7.2.7) is • equal to -1, then it follows at once that X = f(2)(X). Applying the previous results one obtains in the present casef[x,f(x)] == a(l - x - ax + ax 2) = -1. This equation has real roots for a 2: 3. It can be shown, for example, considering the linear part of f(2)(X), that the circle is asymptotically stable for values of a in a certain range, while y becomes unstable. One says in this case that the solution y bifurcates in the circle of order two. As a result, it follows that if y is greater than ~ (in units of carrying capacity) then the system tends to oscillate between two values. We cannot go into details on what happens for a > 3 (see [106] for details), but we shall qualitatively sketch the picture. For higher values of a, a cycle of order four appears and then a cycle of order eight and so on all the cycles of even period Z" will appear. All this happens as a goes from 3 to 3.57, where the last point is an accumulation point of cycles of Z" periods. What is the solution like to the left of this point?
182
7.
Models of Real World Phenomena
For a near 3.57 one can find a very large number of points that are parts of some solution of even period (which can be very large and hardly distinguishable from the periodic solutions). In this region even if two similar populations evolve starting from two very close initial points, their history will be completely different. After a long time every subinterval of (0,1) will contain a point of the trajectory and if one maps the density of the number of occurrences of Yn in subintervals of (0, 1) the pictures are very similar to sample functions of stochastic processes. This behavior has been called chaos. After the value 3.57 of a, the solution of period three appears. There is a result of Li and Yorke that states if there is a cycle of period three, then there are solutions of any integer periods and for the same reasons discussed above, the term chaos is appropriate in this case as well. Almost all of the previous results can be extended to more general functions f leading to the conclusion that the qualitative results are widely independent from the particular function chosen to describe the discrete model.
7.3.
Distillation of a Binary Liquid
The distillation of a binary ideal mixture of two liquids is often realized by a set of N-plates (column of plates) at the top of which there is a condenser and at the bottom there is a heater. At die base of the column there is a feeder of new liquid to still. A stream of vapor, whose composition becomes richer from the more volatile component, proceeds from one plate to the next one until it reaches the condenser from which part of the liquid is removed and part returns to the last plate. On each plate, which is at different temperature, the vapor phase will be in equilibrium with a liquid phase. Because of this, a liquid stream will proceed from the top to the bottom. We suppose that the liquids are ideal (that is the Raoult's law applies) as well as the vapors (that is the Dalton's law applies). Let Yi (i = 1,2) be the mole fraction of the i th component in the vapor phase and x, the mole fraction of the same component in the liquid phase. Of course Yl + Yz = 1 and Xl + Xz = 1 (the sum of the mole fraction in each phase is 1 by definition). In this hypothesis the quantity, called relative volatility, Yl Xz
a=--
Yz
Xl
·can be considered constant in a moderate range of the temperature (see [173]). If a > lone says that the first component is more volatile. For
183
7.3. Distillation of a Binary Liquid
simplicity, we shall consider as reference only the more volatile component. Setting Yt = Y and Xl = X, one has a=
y(l- x) x(l - y)'
(7.3.1)
which will be considered valid every time the two phases are in equilibrium. Let us see what happens on the nth plate. Here the two components are in equilibrium in the two phases. Let x, be the mole fraction of the more volatile component in the liquid phase, y~ the mole fraction in vapor phase of the same component, and Yn the mole fraction of the same component leaving the plate n. If we assume that the plate efficiency is 100 percent, then y~ = Yn' Moreover, part of the liquid will fall down with mole rate d and the vapor will go up with mole rate V. Let D be the mole rate of the product, which is withdrawn from the condenser. Consider now the system starting from the nth plate (above the point where new liquid enters into the apparatus) and the condenser. We can write the balance equation (7.3.2) where XD is the mole fraction of the liquid withdrawn from the condenser. To this equation one must add the definition of relative volatility that will hold for the equilibrium of the two phases at each plate a=
Yn(l- x n )
xn(l - Yn)
(7.3.3)
.
From the last relation we obtain
«x,
Y =-----:..:..-n l+(a-1)xn
and, after substitution in (7.3.2) we get Xn XnXn-1 + a - I
+
-1) - Va d(a -1) Xn+l
DXD(a
DXD
+ dt a -1) =
0,
(7.3.4)
from which, by letting 1
a=--' a-I'
b = DXD(a -1) - Va. dl a - 1) ,
c=
DXD
d(a -1)
one obtains (7.3.5) This is a difference equation of Riccati type (see 1.5.8). Let us consider the boundary conditions. The initial condition depends on how the apparatus is fed from the bottom and we will leave this
184
7.
Models of Real World Phenomena
undetermined. The other boundary condition will depend on the condition that we impose on the composition of the fluid on some plate. In the literature there are two different types of such conditions. The first one (see [19]) requires that there is a plate i for which X i- 1 = Xi' We shall show that this condition either leads to a constant solution or to no solutions (according to the initial conditions). The second boundary condition (which can be deduced from Fig. 16 of [20]) requires that YN = XD, XN+1 = XD' Let us start with the first condition, that is Xi = Xi-I for some i E [2, N]. Then Xi must satisfy the equation (obtained from (7.3.5» x;
+ (a + bvx, + c = O.
(7.3.6)
Now the Riccati equation can be transformed to a linear one by setting (7.3.7) to obtain Zn+1
+ (b + a)zn + (c - ab)zn_1 =
O.
(7.3.8)
If Al and A2 are two solutions of the polynomial associated with the previous equation one has (if Al ,p A2)
and therefore (7.3.9) Put n = i and n = i-I. The only way to obtain x, = Xj-l is taking either = 0 or C2 = 0, that is, x, = Al - a or x, = A2 - a. One verifies at once that these two solutions satisfy (7.3.6). If the initial condition does not match one of the two constant solutions, then there does not exist any solution. Concerning the second boundary conditions, which is consistent with the condition V = d + D, necessary for other physical considerations, one has CI
VYN
Imposing
YN = XD
=
LXN+I
+ DXD'
we get
(7.3.10)
185
7.4. Models from Economics
One obtains an equation whose unknowns are
Cl C2
(the Riccati equation is
offirst order and its general solution must contain only an arbitrary constant) and N (the number of plates needed to complete the process). C
Let us put, for simplicity, K = 2. One obtains, after some simple algebraic C2 manipulations,
XD) -XD
A2 - a Al - a
Al
(7.3.11)
log-
A2
One can verify that in order to obtain X D = 1 (at this value only the more volatile component is present in the distillate), one needs N = 00. The value of K is determined from the condition of the feed plate (which can be assumed as the zeroth plate).
7.4.
Models from Economics
A model called the cobweb model concerns the interaction of supply and demand for a single good. It assumes that the demand is a linear function of the price at the same time, while the supply depends linearly on the price at the previous period of time. The last assumption is based on the fact that the production is not instantaneous, but requires a fixed period of time. Let the above period of time be unitary. We shall then have
d; = -apn + do, Sn
= bpn-l + So,
where dn , s; are respectively the demand and supply functions and a, b, do, So are positive constants. This model is based on the following assumptions: (l) The producer hopes that the price will be the same in the next period and produces according to this. (2) The market determines at each time the price such that (7.4.2)
From (7.4.1) and (7.4.2) one obtains Pn
b)
= ( -;
do - So Pn-I +--a-'
186
7.
Models of Real World Phenomena
The equilibrium price Pe is obtained by setting P« = Pn-l = Pe, which gives
do Pe
= a
So
+b
and the solution of (7.4.3) is given by (7.4.4) which is deduced from (1.5.4). b If - < 1, it follows that the equilibrium price Pe is asymptotically stable, a
_!!:. is negative, we see that
that is lim P« = p., Since
b
n_OO
P« will oscillate
a
approaching Pe. For b > 1 the equilibrium price is unstable and the process will never reach this value (unless Po = Pe). As a more realistic model, take s; as a linear function of a predicted price at time n. This means that the producer tries to predict the new price using information on the past evolution of the price. For example, he can assume as the new price P« = Pn-l + P(Pn-l - Pn-2) obtaining
Sn
b(Pn-l
=
+ P(Pn-l - Pn-2)) + So,
while the first equation remains unchanged. By equating the demand and supply, one obtains
-apn
+ do =
+bpn-l
+ bpbn- 1 -
bPPn-2 + So
from which
Pn
b (l a
+-
+ P)Pn-l -
b So - do - PPn-2 + - - = a a
o.
(7.4.6)
. .. do - So 1 num pnce IS now Pe = - - . Th e equiilib a-b The homogeneous equation is
Pn
b
+-
a
+ P)Pn-l -
(l
b - PPn-2 a
=
0
(7.4.7)
and the characteristic polynomial reduces to
z
2
b
+-
a
(l
+ p)z -
bp - = o. a
(7.4.8)
7.4.
187
Models from Economics
Let Zj and Z2 be the roots of this equation. The general solution of the homogeneous equation is
while the general solution of (7.4.6) is (7.4.9) where C j and C2 are determined by using the initial conditions. From (7.4.9) it follows that if both IZjl and /z21 are less than one, one has lim P« = Pe. If n ....OO
at least one of the two roots has modulus greater than one, then lim P« = . n-HXJ
00
(for generic initial conditions). The problem is now reduced to derive the conditions on the coefficients of the equation in order to have IZjl and IZ2/ both less than one. A necessary condition is The other condition is (see Theorem B1)
b
bp
a
a
bap l < 1, SInce . -bp = -ZjZ2'
I
a
-+1 _ bp
< 1,
a
.
which is satisfied for
a
a
I; < 3 and -I;
CO
The probability of P; will describe the steady state of the problem. In order to get this information one observes that in the steady state the derivatives will be zero. It follows that the limiting probabilities P; will satisfy the difference equations (7.5.3) (7.5.4) To this one must add the condition L~=o P; = 1, which simply states that it is certain that in the system we must have either no items, or more items. Let p
=~. !J-
The solution of (7.5.3) is given in terms of the roots of
the polynomial equation Z2 -
which are
ZI
(I
= 1, Z2 = p. If p
Z2, ••• ,Zn are the eigenvalues of A, with multiplicity mi , letting
°
rr.,
peA) =
s
II (A -
z;I)m,
(Al.2)
i=1
one has peA) = 0. The polynomial p(z) is called the characteristic polynomial. Let ljJ(z) be the monic polynomial of minimal degree such that IjJ(A) = 0,
(AU)
and let n be its degree.
195
196
Appendix A
Theorem A.l.l. Every root of the minimal polynomial is also a root of the characteristic polynomialand vice versa. Proof. that
Dividing p(z) by l/J(z) we find two polynomials q(z) and r(z) such p(z) = q(z)l/J(z)
+ r(z)
(AlA)
with degree of r(z) less than n. For the corresponding matrix polynomials we have 0 = peA) = q(A)l/J(A) + rCA). Since l/J(A) = 0 and it is minimal, it follows that rCA) is identically zero. Thus (AlA) becomes p(z) = q(z)l/J(z) proving that the roots of l/J(z) are necessarily roots ofp(z). Suppose now that ,\ E C is a root of p(z) and therefore it is also an eigenvalue of A. For every j E N+, it follows that A'x = ,\jx, where x is the corresponding eigenvector. Then we have l/J(A)x = l/J(,\)x from which it follows l/J(,\) = 0, proving that ,\ is also a root of l/J(z). • Let h(z) be any other polynomial of degree greater than n. We can write h(z) = qi z)l/J(z) + r(z) where the degree of r(z) is less than n. Consequently h(A) = rCA).
(Al.5)
The last result shows that a polynomial of degree greater than n and a polynomial of degree less than n may represent the same matrix. Let h(z) and g(z) be two polynomials such that h(A) polynomial d(z)
= h(z) -
g(z)
= g(A). Then the (A.1.6)
annihilates A; that is, d(A) = O. It follows that it is possible to find q(z) such that d(z) = q(z)l/J(z). Let ml> m2,"" m, be the multiplicities of the roots of l/J(z). Then we have l/J(Zj) = l/J'(z;) = ... = l/J(m,-I)(Zj) = 0, i = 1,2, ... , s, from which d(zj) = d'(zj) = ... = d(m,-l)(z;) = 0, i = 1,2, ... , s. This result shows that h(z;)
= g(Zj),
h'(z;)
= g'(z;),
(A1.7)
h(m,-O(Zj) = g(m,-iJ(Zj),
for i = 1,2, ... , s. Two polynomials satisfying the above condition are said to assume the same values on the spectrum of A. Now suppose that the two polynomials assume the same values on the spectrum of A. It follows that d (z) has multiple roots z, of multiplicity m, and then it will be divided exactly by l/J(z); that is, d(z) = q(z)l/J(z). Then d(A) = g(A) - h(A) = 0 and the two polynomials represent the same matrix. The foregoing arguments prove the following result.
A.I.
Function of Matrices
197
Theorem A.1.2. Let g(z) and h(z) be two polynomials on iC and A E C. Then g(A) = h(A) iff they assume the same values on the spectrum of A. Definition A.I.I. Let f(z) be a complex valued function defined on the spectrum of A, and let g(z) be the polynomial assuming the same values on the spectrum of A. The matrix functionf(A) is defined by f(A) = g(A). The problem of definingf(A) is then solved once we find the polynomial g(z) such that g(i)(zd = fi)(Zk) for k = 1,2, ... , s, i = 0, 1, ... , mk-I' The last problem is solved by constructing the interpolating polynomial (Lagrange-Hermite polynomial) (A 1.8)
where cPki(Z) are polynomials of degree n - 1 such that ",('-1)( ) _ 'I' Zj -
"
ki
For example for m, = 1, i
{j, k = 1,2, ... , s,
"
UkjU'i,
i, r
= 1,2, ... , mk'
= 1,2, ... , n,
n (z - Zj)
cPkl(Z)
j>"k ="-----
n (z, -
j>"k
Zj)
It can be proved that the functions cPki(Z) are linearly independent. The matrix polynomial is then s
g(A)
=
mk
L: L: f(i-I)(Zk)cPki(A).
(A 1.9)
k=1 i=1
The matrices cPki(A) are said to be components of A. They are independent of the function f(z). As usual in this field we shall put (Al.10)
These matrices, being polynomials of A, commute with A. With this notation (Al.10) becomes f(A) == g(A)
s
mk
= L: L:
f(i-I)(Zk)Zki'
(Al.1t)
k=1 i=1
The set of matrices Zki are linearly independent. In fact if there exist constants Cki not all zero such that I~=I I~:I Ck;Zki = 0, the associated polynomial s(z) = I~=I I~:I CkicPki(Z) would annihilate A.
198
A.2.
Appendix A
Properties of Component Matrices and Sequences of Matrices
Let us list some properties of the component matrices Takingf(z) = 1, we get from (A.Lll)
Zki'
(A.2.l)
Also, taking f(z)
=
z, we see that s
A =
s
L ZkZkl + L Zk2' k=l
(A.2.2)
k~l
from which s
2
A =
s
L (ZkZkl + Z d j=l L (Zj2.il + 2.i2) k=l s
s
L L
=
(Zk ZjZkl2.il
k=lj~l
Starting directly from f( z) A
2
= Z2,
one has
s
= L
k~l
+ 2ZkZk l2.i2 + Zk22.i2)'
(z~Zk1
+ 2ZkZk 2 + 2Zk3)·
Comparing the two results, it follows that Zkl2.il = l)kjZkl, Zkl2.i2 = l)kj Zk2, Zk22.i2 = 2l)kjZk3'
Proceeding in a similar way, it can be proved in general that ZkpZ;r
=0
if k ;c i,
ZkpZkl
= Zkp
p;;::: 1,
Zkp Zk2
= pZk,p+l
p;;::: 2.
(A.2.3)
From the last relation it follows easily that Zkp
= ( 1 ) ZE 1 , P -1 !
P ;;::: 2.
It is worth noting that, from the second relation of (A.2.3), we get
(A.2.4)
199
A.2. Properties of Component Matrices
showing that the matrices Zkl are projections. Multiplying the expression A - z.I = L~=I (Zk - z;}ZkJ + L~=I Zk2' by Zit and considering (A.2.3), one gets, s
=I
(A - ZJ)Zil
k=1
(Zk - Z;)ZkIZil
+
s
I
k=1
Zk2Zil
= Zi2'
(A.2.6)
Because of (A.2.4) and (A.2.6), we obtain Z
1 Zp-I 1 (A I)P-IZ kp-(p-1)! k2 -(p-1)! -Zk kl'
(A.2.7)
From this result, it follows that (A. 1. 11) can be written as f(A) =
t I f(:-l)(z~)
k=1 i~1
(I
-1).
Let us consider now the functionf(z) f(A)
= (A -
(A - ZkI) i-I Zkl'
(A.2.8)
_1_, where A ."r:. Zk' The function
=
z-A
H)-I is expressed by s
mk
I I
(A - H)-I = -
k=1 i=1
(i -1)I(A - Zk)-iZki
(A.2.9)
from which, considering (A.2.4), we get (A - AI)-I
s
=- I
k=1
[(A -
zd- I Zkl + (A -
+ (A -
Zk)-3Z7.2 + ... + (A - Zk)-mkZ~~-I],
we then obtain because of (A - AI) = -
t
1=
k=1
[Zkl
+ (A -
+ (A -
Zk)-2Zk2
s
I
k=1
[(A - Zk)Zkl - Zd,
Zk)-IZkIZk2 + (A - Zk)-2ZkIZ7.2 + ...
ak)-mk+IZkIZ~~-1
- (A - Zk)-I ZklZk2 - (A - Zk)-2Z7.2 - (A - Zk)-3Zt2 - ...
- (A - Zd-mkZ~n s
I
[Zkl - (A - Zk)-mkZ~n
k~1
from which results
Z;; =
0,
showing that the matrices Zk2 are nilpotent.
(A.2.10)
200
Appendix A
This result allows us to extend the internal sum in formula (A.2.8) up to
mb the multiplicity of the characteristic polynomial. In fact, since m; ::'5 mk, Z~k+i = 0 for j = 0, 1, .. , , mk - mi, and hence (A.2.8) can be written as f(A) =
f
I
f:-I)(Zk) (A - ZkI)i-1 Zkl' k=li=1 (1-1)!
(A.2.1l)
Now consider a sequence of complex valued functions fl(z), f2(Z)"" defined on the spectrum of A. Definition A.2.1. The sequence fl J2, .. " converges on the spectrum of A if, for k = 1,2, ... ,s, limf(zk) = f(Zk), limf;(zd = f'(Zk), and so on i-+oo
until !imflmj-I)(zd
=
i-+oo
f(m j-I)(Zk)' The following theorem is almost evident
' .... 00
(see Lancaster), Theorem A.2.t. A sequence ofmatricesf(A) converges if the sequencef(z) converges on the spectrum of A. Corollary A.2.t. Let A be an s x s complex matrix having all the eigenvalues inside the complex unit disk. Then (I - A)-I = I:o Ai. Proof. Consider for i 2:: 0 the sequencef(z) = I;=o z'. This sequence (and the sequences obtained by differentiation) converges if Izl < 1 to (1- Z)-I. It follows that there exists the limit of f(A) and the limit is (I - A)-I, •
Corollary A.2.2.
Proof.
Let A be an s
x s complex matrix.
Then e A =
Consider for i > 0, the sequence f(z) = I;=o
converges for all z • to e/:
E
Definition A.2.2.
An eigenvalue Zk is said simple if mk
~.
J.
Ai
I';:o -:,. J.
This sequence
C. Likewise for fls)(z). It follows thatf(A) converges
= 1.
Definition A.2.3. An eigenvalue Zk is said semisimple if Zk2 = O. A semisimple eigenvalue is not simple (or degenerate) if mk = 1 and mk ¥:- 1.
In both cases the terms containing (A - ZkI)P-IZkl with p> 2 are not present in the expression of f(A).
201
A.2. Properties of Component Matrices
Example A.2.1. An important class of matrices are the so called companion matrices, defined by 0 0
1 0
0 1
0
A= 0 -an
0 1
0 -a l
For this class of matrices m, = mj (for an values of i), that is, the characteristic polynomial and the minimal polynomial coincide (see Lancaster) and this implies that there are no semisimple roots. Example A.2.2.
Let f( z) = e z t and A be an n x n matrix. Then s mk ti- I (A.2.l2) e At = " f.., " s: (. ) e zk teA - Zk I)i-1Zkl' k=1 i=1 I - 1 !
If the eigenvalues are all simple, the previous relation becomes if
e At - " f.., e k=1
If
Zj
vz.
(A.2.l3)
kl'
is a semisimple eigenvalue, (A.2.H) becomes (A.2.14)
Theorem A.2.2. Proof.
If Rez, < 0 for i
= 1,2, ... , ii,
From (A.2.12) it follows easily.
then lim IAt = O. (-+00
•
Theorem A.2.3. If for i = 1,2, ... , ii, Rez, s;; 0 and those for which Rez, = 0 are semisimple eigenvalues, then eAt tends to a bounded matrix as t ~ 00. Proof.
It follows easily from (A.2.l3).
Example A.2.3.
Letf(z)
= z"
and
•
202
Appendix A
If the eigenvalues
Zj
of A are all simple, then (A.2.I5) becomes ii
=I
An
If
Zj
(A.2.I6)
Z~Zk"
k~'
is semisimple, (A.2.I4) reduces to
Is mI k
An =
Theorem A.2.4. If
k=1 k¢j
~ z~-i(A - ZkI)iZkl + Z;~I' )
- , (
IZil < 1 for i = 1,2, ... , n,
then lim An n-+OO
The proof is easy to see from (A.2.I5).
Proof.
(A.2.I7)
I
i=O
= O.
•
Theorem A.2.S. Iffor i = 1,2, ... , ii, IZil ~ 1 and the eigenvalues for which IZil = 1 are semisimple, then An tends to a bounded matrix as n ~ 00. The proof is easy to see from (A.2.I7).
Proof.
•
It may happen, however, that for multiplicity of higher order and consequently higher dimension of the matrix, the terms in (A.2.I7) may become large. For example, let us take the matrix ,\
{3
0
A
0 {3
0 0
0
= AI + {3H
A=
0
0
0 A
(A.2.I8)
sXs
with 0 0
1 0
0 1
0 0 ... 0
H=
0 1 0
0
(A.2.I9)
sXs
and H = O. Then (A.2.I7) becomes S
An =
I
s-I
i=O
(n). I
A n-i{3iH i•
(A.2.20)
A.3. Integral Form of a Function of Matrix
203
Multiplying by E = (1,1, ... , l)T and taking n = s - 1, we have A"E
= itl
(;)A
"-if3
i
i
(A.2.2l)
H E.
The first component of this vector is (A"E)I
=
it(;)A
= (A + er,
"-if3i
(A.2.22)
from which it is seen that
IIA"EII2: IA + f31". (A.2.23) This implies that the component of A"E will grow even if IAI < 1, but IA + f31 > 1. Eventually they will tend to zero for n > s - 1, because from (A.2.20) the exponents of A become bigger and bigger while those of f3 remain bounded by s - 1. But in the cases (as in the matrix arising in the discretization of P.D.E.), where s grows itself, the previous example shows that the eigenvalues are not enough to describe the behavior of A". This will lead to the introduction of the concept of spectrum of a family of matrices. A.3.
Integral Form of a Function of Matrix
Let M(z) be an s x s matrix whose components mij(z) are functions of the variable z. In the following, we shall assume that the functions mij(z) are analytic functions in some specified domain of the complex plane. Definition A.3.1. The derivative of the matrix M(z) with respect to z, is the matrix M'(z) whose elements are mij(z). Definition A.3.2. The integral of the matrix M(z) is the matrix whose coefficients are the integrals of mij(z).
Let us consider now the matrix R(z, A) == (zI - A)-I, called
resolvent
matrix.
From
(A.3.1)
(A.2.9),
we
get
R(z, A) = (k = not containing other
I~=I I~~I (i -l)!(z - Zk)-iZki, which has singularities for
1,2, ... ,s). Suppose now that I' h is a circle around eigenvalues. Then (A.2.9) becomes R(z, A)
mh
=I
(j -l)!(z -
Zh)-jZhj
j=1
mh
=
I
j~l
+
S
mk
I I
k~lj=1
k,,"
(5)
-
2,
(5») we
., Xo , •.• , Xm,-l ,
0
ZI Zz
Since X is nonsingular, a matrix J, similar to A can be defined as J = X-lAX = diag(Jj, J2 ,
••• ,
Js ) .
The structure of B is a block diagonal, each block has the form
i: =
(~k. .:. : '.
o
0
0 ).
(A.4.9)
1 Ak
(4) At least an mj is different from 1 and ZJ2 ¢ 0 for 0 $ t < mj' Choosing xo(j) E M·J' we define the set of vectors x(j) = ZiJ 2XO (j) for i = 0, 1, ... , i -s t. These f vectors are linearly independent (see Lemma l
207
A.5. Norms of Matrices and Related Topics
1), but they are not enough to form a base of M]. Choose another vector xV) independent of the previous ones and define x;~\ = Z;2XY) and so on until a set of mj linearly independent vectors has been found. Then we proceed as in the previous case.
The matrix J is block diagonal as in the previous case, but the block corresponding to the subspaces M, can be decomposed in different subblocks with each one corresponding to one chain of vectors. Each subblock is of the type (A.4.9). The matrix J is said to be the Jordan canonical form of the matrix A. Each chain associated to M, contains an eigenvector associated to Zj (the last vector of the chain). The number of eigenvectors associated to Zj is the geometric multiplicity of the chain. The dimension of M, is the algebraic multiplicity of Zj.
A.5.
Norms of Matrices and Related Topics
The definitions of norms for vectors and matrices can be found on almost every book on matrix theory or numerical analysis. We recall that the most used norms in a finite dimensional space R S are: (I)
Ilvll
S
L Iv;l,
=
l
i~1
S
1/2
)
(2)
Ilvlb =
(3)
Ilvllco = max IVil.
(
;~I
v7
,
l~r$s
By means of each vector norm, one defines consistent matrix norm as follows
I Ax II
IIAII = ~~~N· The corresponding matrix norms are
f
la;jl,
(1')
IIAII
(2')
IIAlb = (p(A
(3')
IIAllco = max j=l L layl,
I
= max
i=1
ls}::=:;s
HA»I/2,
S
]SI$S
(A.5.I)
208
Appendix A
where A is any s x s complex matrix, A H is the transpose conjugate of A, and p(A) is the spectral radius of A. Given a nonsingular matrix T, one can define other norms starting from the previous ones, namely, IIvllT = I Tv I and the related consistent matrix norms IIAIIT = IIT-IATII. If T is a unitary matrix and the norm chosen is 11·lb, it follows that
IIAII = IIAIIT'
(A.5.2)
For all the previous defined norms the consistency relations
IIAxII :5 IIAllllxll,
(A.5.3)
IIABII :5I1AII IIBII, where A, B, x are arbitrary, hold.
(A.5.4)
and
Theorem A.5.l.
For every consistent matrix norm one has p(A)
:5
IIAII.
(A.5.5)
Proof. Let A be an eigenvalue of A and v the associated eigenvector. Then we have
IAlllvl1 :5IIAvll :5IIAllllvll, from which (A5.3) follows. Corollary A.5.l.
•
If IIAII < 1, then the series
(A.5.6)
converges to
(1 -
A)-I, and
11(1 - A)-III < 1/(1-IIAII).
Proof. From Theorem A.5.1 it follows that p(A) < 1 and the results follow from Theorem A1.6 (see also exercise ALl). • Another simple consequence is the following (known as Banach Lemma). Corollary A.5.2. Let A, B be s x s complex matrices such that IIA-I II :5 a, IIA - BII :5 13, with 0'13 < 1. Then B- 1 exists and liB-III < 0'/(1- 0'13).
Proof. Let be C = A-I(A - B). By hypothesis IICiI < 0'13 < 1. Then (1C)-I exists and 11(1 - C)-III :5 1/(1- 0'13). But B- 1 = (1 - c)-lA-I and • then liB-III :5 0'/(1 - a(3). Theorem A.5.2. Given a matrix A and e > 0, there exists a norm IIAII such that IIAIl :5 p(A) + e.
209
A.6. Nonnegative Matrices
Proof.
Consider the matrix A' =
.!.e A
and let X be the matrix that trans-
forms A' into the Jordan form (see section A.4). Then 1
-X e
-I
1 AX=Je==-D+H, e
where D is a diagonal matrix having on the diagonal the eigenvalues of A and H is defined by (A.2.19)'. For anyone of the norms (1'), (2'), (3'). IIHII = 1, and IIDII:s: p(A). It follows that IIX-IAXII == IIAllx :s: IIDII + ellHl1 :s: p(A) + e, which completes the proof. •
A.6.
Nonnegative Matrices
In many applications one has to consider special classes of matrices. We define some of the needed notions. Definition A.6.1.
An s x s matrix A is said to be
(1) positive (A> 0) if aij > 0 for all indices, (2) nonnegative (A ~ 0) if aij ~ 0 for all indices. A similar definition holds for vectors x
E
R S considered as matrices s x 1.
Definition A.6.2. An s x s nonnegative matrix A is said to be reducible if there exists a permutation matrix P such that PAp
T
= (~
~)
(A.6.l)
where B is an r x r matrix (r < s) and D an (s - r) x (s - r) matrix. Since P" = p- I , it follows that the eigenvalues of B form a subset of the eigenvalues of A. Definition A.6.3. A nonnegative matrix A, which is not reducible is said to be irreducible. Of course if A > 0 it is irreducible. Proposition A.6.1. Proof.
If A is reducible then all its powers are reducible.
It is enough to consider that
PA 2p T and proceed by induction.
=
PApTpAP •
=
B2 ( G
210
Appendix A
Definition A.6.4. An irreducible matrix is said to be primitive if there exists an m > 0 such that Am> o. Definition A.6.S. primitive.
An irreducible matrix A is said to be cyclic if it is not
It is possible to show that the cyclic matrices can be transformed, by means of a permutation matrix P, to the form
0
PAp T
=
Or
01
0
0
O2
0
...
0 (A.6.2) Or-I
0
In case when A is irreducible we have the following result, due to PerronFrobenius. Theorem A.6.2. If A > 0 (or A ~ 0 and primitive) then there exist Ao E R+ and xo ~ 0 such that
(a) Axo = Aoxo, (b) if A ~ Ao is an eigenvalue of A then IA 1 < Ao, (c) Ao is simple. If A ~ 0 but not primitive, then (a) is still valid but
IAI ~ Ao in (b).
Theorem A.6.3. Let A ~ 0 be irreducible and Ao its spectral radius. Then the matrix (Al- A)-I exists and is positive if IAI > Ao .
Proof. Suppose IAI > Ao. The matrix A* = A-IA has eigenvalues inside the unit circle. Hence it follows (see Corollary A2.l) that (1 - A*)-I is given by (1 - A*)-I =
L 00
A*i
i=O
from which it follows (Al- A)-I
= A-1(1 -
A*)-I
= A-I L 00
(A-IA)i.
i=O
The second part of the proof is left as an exercise.
•
(A.6.3)
Appendix B-The Schur Criterium
We have seen in Chapters 2 and 3 that the asymptotic stability problem for autonomous linear difference equations is reduced to the problem of establishing when a polynomial has all the roots inside the unit disk in the complex plane. This problem can be solved by using the Routh method (see for example [152] and [12]). In this Appendix we shall present the Schur criterium, the one most-used for such a problem. Let p(z) = L~=OPiZi be a polynomial of degree k. The coefficients Pi can be complex numbers. Let q(z) = L~=OPiZk-i, (Pi are the conjugates of p;), be the reciprocal complex polynomial: q(z) = Zkp(Z-I). Let S be the set of all the Schur polynomials (see Definition (2.6.4)). Consider the polynomial of degree k - 1: p(l)(z)
= fikp(z) -
Poq(z).
z
. easy t 0 see t h at p (1)() I t IS z Theorem B.t.
Proof.
p(z) E S
Suppose p(z)
E
k (- ) i-I = "':"'i=1 hPi - POPk-i Z •
iff (a)
IPol < IPkl and p(l)(z) E S.
S and let
ZI, Z2, •.• , Zk
IPol = IPk
III
I
z, < IPkl
and condition (a) is verified. On the unit circle
Iq(z)1 =
be its roots. Then
Izl = lone has
IitO iI = IitoPiZ- iI = litPizil = pZ
k-
Ip(z)l·
211
212
Appendix B-The Schur Criterium
Since condition (a) is verified, we get IPkP(z)l> IPop(z)1 = IPop(z)1
= IPoq(z)1 = I-Poq(z)l·
Applying Rouche's theorem (see [76]) it follows that the polynomial PkP(Z) and Pkq(Z) - Poq(z) = zp(l)(z) have the same number of roots in Izi < 1, that means that p(l)(z) E S. Suppose now that p(l)(z) E Sand IPol < Ipkl. It follows that on Izi = 1, IPkP(z)l> I-Poq(z)1 and again by Rouche's theorem the polynomial PkP(Z) has n roots inside the unit disk, that is p(z) E S. The previous theorem permits to define an algorithm to check recursively if a polynomial is a Schur polynomial. The algorithm is very easily implemented on a computer. Next theorem, which is similar to the previous one, gives the possibility of finding the number of roots inside the unit disk. Consider the polynomial of degree k - 1 Tp(z) == Pop(z) - Pkq(z) =
k-t
I
;=0
i
(POPi - Pkfik_Jz •
The polynomial Tp(z) is called the Schur transform of p(z). The transformation can be iterated by defining T'p = T(T'-tp), for s = 2, 3, ... , k: Let 'Ys = T'p(O), s = 1,2, ... , k: • Theorem B.2. Let 'Ys ;t' 0, s = 1,2, ... , k, and let SI, S2, ••. , s.; an increasing sequence of indices for which 'Ys < O. Then the number ofroots inside the unit disk is given by h(p) = Lj':t (~IY-1(k + 1 - Sj)' Proof.
See Henrici [76].
•
Analogous results can be given for Von-Neumann's polynomials. Let N be such a set. Theorem B.3.
A polynomial p(z) is a Von-Neumann polynomial
(1) IPol < IPkl or
and
(I') p(l)(z) == 0 and
Proof.
See [113].
•
(2) p(l)(z)
E
(2') p'(z)
S.
E
N
if either
Appendix CChebyshev Polynomials
C.l.
Definitions
The solutions of the second order linear difference equation where
Z E
Yn+Z - 2ZYn+l + Yn = 0, C, corresponding to the initial conditions Yo
= 1,
Yl
(Cil )
= z,
(C2)
and Y-l = 0, Yo = 1, (C.3) are polynomials as functions of Z and are called Chebyshev polynomials of first and second kind respectively. They are denoted by Tn(z) and Un(z). We list the first five of them below. To(z)
=1
U_ 1 = 0
T1(z) = Z Tz(z) T3(z)
= 2z z = 4z 3 -
T4(z) = 8z
4
-
Since the Casorati determinant
Uo(z) = 1 1
3z
8z z + 1
I
K(O) = To(z) T1(z)
U1(z) Uz(z) U3(z)
= 2z = 4z z -
= 8z
U_1(z) Uo(z)
3
-
1
4z.
I
is equal to 1, the general solution of (Cil ) can be written as Yn(z)
= Cj
Tn(z)
+ czUn-1(z).
(C.4)
213
214
Appendix C-Chebyshev Polynomials
Let WI and (C.1), that is,
W2
be the roots of the characteristic polynomial associate to 2 W -
2zw
+ 1 = O.
It is easy to express Tn(z) and Un(z) in terms of obtains, considering that W2 = w~\
Tn(z)
=
w n + w- n I 2 I = cosh n log(z
Un(z)
=
1 (Z2 -1)1/ 2
W~+I -
(C.S) w~
and
w~.
In fact one
+ (Z2 _1)1/2),
(C.6)
w~n-I
2 (C.7)
( 2 Z
For
Z E [
-
1
sinh(n + 1) log(z + (Z2 -
1)1/
e, it follows that WI = e" = cosh nlog e'" = cosh nU} = cos ne,
-1, 1], by setting Tn(z)
)1/2
1
2).
Z
= cos
_ sin(n + 1)e
Un ( Z ) -
•
sin
e
and
,
which are the classical Chebyshev polynomials.
C.2.
Properties of Tn(z) and Un(z)
One easily proves that the roots of Tn(z) and Un_l(z) are 2k + 1 1T n 2
k=O,I, ... ,n-l,
Zk=COS---,
(C.8)
and Zk =
k1T
cos-, n
k
= 1,2, ... , n -
1,
(C.9)
respectively. Other properties of T; (z) and U; (z) are the following, which can be verified easily. (1) (2) (3) (4) (S)
(6) (7) (8)
Tn(z) + (T~(z) - 1)1/2, Tn(z) = Ln(z), (symmetry) 1jn(z) = 1j(Tn(z)), (semigroup property) ~n-I(Z) = ~_I (Tn(z)), Uc; = -Un_l(z), Tn- 1(z) - zTn(z) = (1- Z2)Un_l(z), Un-I(z) - zUn(z) = -Tn+l(z), Un+iz) + Un_j(z) = 21j(z) Un(z).
w~ =
C.2.
Properties of T.(z) and U.(z)
215
From the last property one can derive many others. For example, (9) U.+j_1(z) + U._j_1(z) = 27j(z)U._1(z), (10) U.+j(z) + U._ j_ 2(z) = 27j+l(Z)Un_1(z), (11) 2T.(z) U.(z) = U2.(z) + 1, (12) 2T.+1(z)Un- t(z) = U2.(z) -1, (13) 2T.(z) = U.(z) - U.-iz).
Among the properties of Chebyshev polynomials the following has a fundamental importance in approximation theory. (14) Let P; be the set of all n degree polynomials having as leading coefficient 1. Then for any p(z) E P«, max
-l~zSl
12 1 - · Tn (z)l :::; max
-lsz~l
Ip(z)l.
The proof of this property is not difficult and can be found in several books on numerical analysis. (15) T.(z) and U.(z), as function of z, satisfy the differential equations T~(z)
= nUn-1(z);
= _(Z2 _l)-l[ZU.(Z) - (n + l)T.+1(z)], (1- z2)T~(z) - zT~(z) + n 2T.(z) = 0, Z2) U~(z) - 3zU~(z) + n(n + 2) U.(z) = O.
(C.l0)
U~(z)
(1 -
(C.lI) (CoI2)
The first two are consequences of the definitions (C.6) and (C.?) and the other can be derived from them. Finally T.(z) and U.(z) satisfy the orthogonal properties. for n = 0, for n > 0,
j7T . where Zj = cos - , ] = 1'0' 0' n - 1. n
Solutions to the Problems
Chapter 1 1.1.
From (1.2.12) Ys =
YI
Y2 Y3 Y4
we get Ys 1.3.
E = I;=o e)ll 4
jYI and from the scheme
YI
110
III
-2 -2
o
11 2
11 3
11 4
2 2
o
o
2 4
0 4
= 10.
The first result of problem 1.2 can also be written 11
ei(ax+bl
= 2 e i(ax+b+a/2) ( e ia/2 ~ e -ia/2) = 2i sin ~ e i(ax+b+a/2).
By taking the real and imaginary parts one gets the result. 1.4.
Using the Sterling transform one has / 11-1/
=
·(2)
.(4)
2
4
=/0 + 3/2 ) + /3)
and
L + f3) + L from which one obtains:
I/ = 11n
j=O
1 /
Ij=n+1 j=O
n 2(n + I? = -'---'--
4
217
218
1.5.
Solutions to the Problems
From the result of problem 1.3 we have
a-1cos(ax+b+a/2) Setting a
=
sin(ax + b) . / . 2sma 2
= q, b = -q/2 and x = n, we have: A-I
/.1
cos
qn
=
sin(qn-q/2) .
2 Sill q/2
and
f
L..-
;=1
.
cos ql =
A-I /.1
.1 i=n+1
cos ql
;=1
=
sin[(n +
1)q - q/2] - sin q/2 . . 2 Sill q/2
1.7.
and x(x) = f(x + 1), f(x + 1) n)(x-n) = f(x - n + 1) we have x(n) = ----'-----'-f(x + n + 1)
1.10.
Using the Sterling transformation, p(x) can be written in the form p(x) = I~=o ax'", Applying a,a2 , ••• , ak and putting x = 0 one aip(O) gets a, = -.-,-.
From
x(n+x-n) = x(n)(x - n)(x-n)
(x-
1.
1.11.
One has:
.)=Q-f (_IY( n +]q .)+(_1)q-n 1
Qf(-I Y( q j=O n +]
j=O
= q-n-I L (-1)1"[(q-l) + (q-l)] j=O n +j n +j - 1 + (_1)Q-n
=
q-f- (-ly(qn -+]~) + q1:1
j-O
1
(-ly(q -
1)
q- 1
j=O
x (_1)q-n q-n "( q-l) = q-n-I L (-1)1"(q-l) . + L (-1)1 . j-O n +] j=O n +] - 1 q-n-I
= j~O +
"(q - 1)
(-1)1 n + j
(qn-l -1).
+
q-n
j~1
"(
q- 1)
(-1)1 n + j - l
Chapter 1
219
By setting j - 1 = i in the second term, it is easily seen that the two sums cancel. 1.12.
One has:
= q-fl (-l)j-1(
S(q, I, n)
j~1
(j) + (_l)q-n-l(q -I n)
q .) n +] I
= q-fl (_l)j-l(q - ~)(j) n +]
j=/
+
)
I
q-fl (-I)j-1( n q~" (j) +] -1 I j=l
+
qf (_l)j-l(
j~l+l
= q~f-l (-l)j-t(q -~) J~/
n
(j)
q- 1 ) +( q- 1 ) n +j - 1 I n+I- 1
+]
[(j)I _(j +I 1)]
( q-l )= (q-l) n+l-l .
+ n+l-l
en - e2 •
1.23.
The solutions are Yn
1.24.
the equation becomes Zn+1 = z.; + 1. m, Letting n = 2 Ym = C(2 m) and gm-l = f(2 m), one has Ym m) 2Ym-1 + f(2 =::: 2Ym-1 + gm-h Yt =::: 2. The solution is Ym 2m - s - l 2 m - l Yl + ",m-l 4..s~1 gs .
1.25.
Letting Yn
=
= Z;;-l
1.26.
Letting Zn
1.27A.
From log Yn+1 2" Yn =::: Yo .
1.27B.
. b ecomes Zn+1 = Zn, 2 f rom 1 - z; t h e equation By su bstituti stituting Yn = -2-
=:::
A 1/2 cot Yn one has Yn+1
=::: =:::
= 2 log Yn
which one obtains Yn 1.27C.
Let z;
= 1-
1.28.
Yn+l(l
+ y~) + Yn
=:::
=:::
2Yn.
one obtains log Yn
= 2n log Yo and
![1 - (l - 2Yo)2"].
Yn and obtain the equation of problem 1.26. =:::
k.
then
220
Solutions to the Problems
1.30. Un
From the result of problem 1.29 one has
~ 13 eXP(H I/2M J=O nf ( _1')1/2) n ]
s=O 2 It is known that I~=I k- / < 2n l / 2 1
Un
+ 13 nf eXP(h l / 2M nf ( _1 )1/2)'
~ 13( (exp(h 1/2 M(2n 1/2 -
-
r=s+1 n 1. Then one has
r
1)) + ~~~ exp(h 1/2 M(2(n - s - 1)1/2 -1))
1.31.
Let Vn = C + I~=n+l k,Vs ' One has Yn ~ Vn and Vn = Vn+ 1 + kn+1 Vn+l' that is Vn = n:=n+l (l + k.) V, ~ ~ exp(I:=n+1 ks ) and for t ~ 00, Yn -s Voo exp(I~=n+l k s ) = C exp(I~=n+l k s ) '
1.32.
For a < 1 one obtains Y~+l < y~ + bn(Yn + Yn+l) and then Yn+1 < Yn + b.; from which Yn ~ Yo + I;~ b.. For a > 1 one has Y~+l aYn2 --< bn(Yn+ Yn+1 ) an d t h en Yn+1 - a 1/2Yn
bn 1/2 Yn + Yn+l < bn, a Yn + Yn+1 from which one obtains Yn < (a 1/2)nyo + I;:~ (a 1/2r-j - 1b.. ~
Chapter 2 2.8c. 2.l3c.
Write the equation (2.3.1) in the form LlYn+l = Yn and apply Ll- I to both sides. In (2.5.10) one imposes q(x) = p'(x) from which it follows YoPo = + POYI = 2p2, .... Let ei+1 = Si+l - 8i+1> r, = ai+1 - ai+1> l3i = 1O-'(5i + ai+I)' One has ei+l = e, + ri - l3 i, from which Ienl ~ k; + 10-' II~:~ l3i l. From this one deduces that one must keep II 13;1 as small as possible and this can be achieved summing first the smaller a, in absolute value. If lail < a, then Is;1 ~ ia. Finally one has leNI -s k N + lO-'a I~~I i ~ . that t h e error grows lik N 2. k + 10- ta N(N - 1) showing 1 e
PI> YOPI
2.14.
N
2.19
2
From (2.5.18) we have
221
Chapter 3
where q~2) = q\2) = 0 and qn = 1/ n. The roots of denominator are 1 and 2. In the unit circle it can be written:
1 ( -2 Z2
-
3 -2 Z
+2
)-1
with 'Yi bounded and lim r,~ 'Yi 1=0
00 I
'YiZ i
i=O
h were Max
lss~n
-
Ci -
00 I
00 Ii=O 'YiZi
=
= 00
q~2)zn+2 =
(why?). Then one has
00 I
CiZi+2,
Yn
= Cn-2 = 4..j=o j + 1 'Yn-j-2 $
i=O
n=O
"\'i (2) 4..n=2 qn 'Yi-n'
Then
1
,,\,n-2
r,;=1 Y): The last inequality follows from the result of problem
1.13a. 2.20.
Taking f = 0, the error equation becomes p(E)en = p(1) whose solution is the sum of the general solution of the homogeneous equation and a particular solution. The general solution is given by (2.3.14) from which it follows that p(z) must be a Von Neumann polynomial.
2.21.
Taking f = 0 as in the previous problem and eo = e1 = ... = ek-l = 0, we have (see 2.4.11) en = p(l)/ p(l) and this cannot be zero for n ~ co (nh $ T) unless p(1) = O. Similarly for f == 1: p(E)en = h(p'(1) - 0"(1».
2.24.
Use Theorems B.l and B.2 of Appendix B. For case a) one finds D = [-i, i].
Chapter 3 3.7.
Exchanging
with
care
sum in the expression one has gi r,~~opH(n + ki,j) + gn = g.; (Remember that some values of H are zero).
r,~=OPi(n) r,;:~o H(n + k -
3.11.
Izsl ~ 1, S =
3.12.
It depends on the roots of
H(n
the
ti:
t.D«;
1,2. Z2
+ PIZ + P2' It will be
+ 1) + PIH(n) + P2 H(n -1) = 0 H(l)
+ PIH (O) + P2 H( -1)
= 1.
for n
~
0
222
Solutions to the Problems
According to the values of the roots, several cases are possible. For example: (a) IZII < 1, IZ21 > 1
H(n)
I)-I
Z; {(ZI + PI + P2 Z(ZI + PI + P2Z21)-IZ~
=
(b) IZII < 1, IZ2/ < 1, H(n) H(n) = 0 for n ~ O.
= (ZI -
for n ~ 0 for n ~ 0
Z2)-I(Z~ -
zD for n ~ 0 and
(n;,j
;=0
N
- 2:
+ 1)T(j + 1, n;)
LjcI>(nj,O)Q-I
;=0 N
X N
=
2:
2: LscI>(n"j+1)T(j+l,ns) s=O
LscI>(n"j + 1)T(j + 1, ni)
i~O
N
- 2: 3.22.
From Yn+1
=
LscI>(n"j + 1)T(j + 1, n.)
=
O.
s~O
cI>(n
+ 1, O)Q-I W +
nN-l
2:
G(n
+ 1, svb,
5=0
= AcI>(n, O)Q-I W+
"N- 1
2:
s=O
AG(n, s)bs + b;
+ AG(n, n'ib;
s>'n = A[cI>(n, O)Q-I W
+
"N- 1
2:
s=O
G(n, s)bsl + b;
= AYn + b.:
224
Solutions to the Problems
For what concerns the boundary conditions one has: N
I
i=O
N
LJ'n,
=I
Lj(nj, O)Q-I W +
i~O
N
= w + i~O
n
-I
+
n
-I
-Jo
i=O s=O
L j (n j, s + I)T(s
%0
%0
nN-I
(
- (n;, O)Q-I = W
N
I I t.ot»; sib,
Jo
+ 1, nj)
Lj(nj, s + I) T(s
+ 1, nj)
)bs
(N
;~o Lj(n;, s + I) T(s + 1, n;)
Lj(nj, s + 1) T(s
+ 1, nj »)b, = w.
Chapter 4 4.3.
The system is symmetric with respect to the origin: gi( -x, -y) = -gi(X, y) for i = 1,2. This allows us to consider only the upper half plane. According to the signs of gl and g2, let us consider the following sets: A
= {(x, y) Iy 2: 2x}
B
= {(x,y)ly < 2x
c
= {( x, y) x (y - x)
I
2
where gl(X, y) 2: 0, g2(X, y) > 0 and x 2(y - x) + y5 2: O}
+ Y 5 < O}
where gl(X, y) 2: 0, gix,y) < 0 where gl(X, y) < 0, g2(X, y) < O.
It is clear that for (Xb Yk) E A, LiXk =:: 0, LiYk =:: 0, that is both the sequences Xb Yk are not decreasing. If they remain in A, they will · . gix,y) h b never cross t h e 1me y = 2x and the ratio ( ) as to e greater gl x,y ,
2
;( (y - ~X) 5> 2, which is impossxy-x+y ible because the y in the denominator has degree larger than the y in the numerator. The sequences must cross the line y = 2x and enter in B. In a similar way it can be shown that they cannot remain in than 2 for all k. This means that
225
Chapter 4
B. They will enter in C, where both ~Xk and ~Yk are negative and the sequences are decreasing. Now if Yk > 0 it follows that
Yk+1
= Yk +
=
4.5.
Y~(Yk - 2Xk) 2
'k
+
6
'k
=
Yk«Xk - Yk)2 + y~ +
,i+ r%
Yk('~ + '% + y~ - 2XkYk) 2
,n >0
'k
+
6
'k
and similarly for xi, This shows that the sequences must remain in C and must converge to a point where both ~Xk and ~Yk are zero, which is the origin. Starting from (-8,0) for small positive 8, it is easy to check that in the following iterations the points (Xk, Yk) have increasing distance from the origin until they reach the regions B or C, showing that the origin is unstable. The solution is log(no + 2) y(n, no, Yo) = log(n + 2) Yo· The series L:=no Iy(n, no, Yo)1 does not converge, showing that the origin is not II-stable.
4.9.
In this case D is the set of all positive numbers. If Yo E D, then Yn E D for all n. Consider V(y) = Y J(l + y 2). It is V(y) 2: 0 and
~V() Yn
=
Yn(1-Yn?(Yn-l) (l + y~)(l + y~)
=-
() W Yn < O.
The set E is {O, I} and W(x) ~ 0 for x ~ 00. Then, according to the theorem Yn is either unbounded or tends to E. In fact the solution is Yn = y~_2)n and it tends to zero if Yo < 1 and n even and it is unbounded for n odd. If Yo = 1 then Yn = 1 for all n. 4.10. The eigenvalues of A are Al =! with multiplicity s - 1 and A2 = (1 - s)J2. They can be obtained easily by considering that A is a circulant matrix and the eigenvalues are the values assumed by the polynomial p(z) = -!(z + Z2 + ... + ZS-I) on the Slh roots of the unity. There is global asymptotic stability for s = 2, only stability for s = 3 and instability for s > 3. 4.11. In order to have Vn+ 1 2: 0, it must be Vn - w( Vn) 2: O. If u and v are such that U 2: n, U - w(u) 2: 0, V - w(v) 2: 0, and U - v 2: w(u) - w(v) one has u - w(u) 2: V - w(v). The solution satisfies Un = Uo - L;:~ w(Uj)' Since w is increasing and Un must remain positive, it follows that Uj ~ O.
226
Solutions to the Problems
4.12. Let x E O(Yo), there exists a sequence n, ~ 00 such that y(n j , no, Yo) ~ x. But y(ni+l,no,Yo)=f(y(nj,no,Yo» and limy(nj+j,no,Yo)= "j""'OO
f(x), showing thatf(x) E O(Yo) and that O(Yo) is invariant. Now let Yk be a sequence in O(Yo) converging to y. We shall prove that y E O(Yo). For each index k, there is a sequence m~ ~ 00 such that y(m~, no, Yo) ~ Jk. Suppose for simplicity that dist(Yk, y(m~, no, Yo» < k- 1 and m~ ~ k for i ~ k: Consider the sequence mk = m~. Then dist(y, y(mb no, Yo» $ dist(y, yd + which implies dist(y, y(mb no, Yo» ~ 0 and then y E O(Yo).
u:',
Chapter 5 5.2.
From the mean value theorem one has
F(x) - F(y) - F'(x)(y - x) =
IIF(x) 5.4.
Let ao
f
[F'(x
F(y) - F'(x)(y -
= ~f3'Y.
One has
I/Xn+l -
x, II
$
x)11
+ s(y - x) $
'Y
fs
F'(x)] ds(y - x);
dslly -
x11
2
•
aollxn - xn_111 2
and I/X k+m -
xkll $ $
$
$
k+m-l
Ilxj+! - xJ $ I a~J-lllxk - xk_dl 21 j=k j=1 Ilxk - Xk_111 2 f: a~1-11Ixk - Xk_111 2L2 j=l m [x, - Xk_111 2I a~1-1ut~12 j=l Ilxk - Xk_111 2I a~1-11J21-2( a 2k- 1?1- 1 j=l
I
m
227
Chapter 5
5.5.
. atn u; Un-I C onsiid er t h e equation Un = --Un-I' One has A = - - , that is ~tn-I /.Jotn ~tn-I U
~ is constant and must maintain its initial value UI/ t., Then
txt;
Un = ~tn(UI/tl)' It follows then that ~Xn:5 Un' Moreover decreasing that is for m , ,
Ilxn+p -
x, I
= t +
n
- t tn + 1 - tn
I ~xm I :5 I ~xn I
'~tn
~tn
:5 I;':~ Ilxn+j+1 - xn+J :5 "p-I at .11~Xn+jll:5 ,,~-I ~t n+J
L.J~I
n p
n
:2:
~t. n+)
L.J=I
lI~xnll
--A-
tu;
.
1S
and
.11~xn I
n-r]
~t
n
II~xn I
from which it follows that
lim Ilxn+ p
r-«
Being II ~xn I
:5 Un
xnll:5
-
one has
II x *- X 5.6.
t* - tn Ilaxnll. tn + 1 - tn
n
I 11«t*-tn)u • tl
In this case the comparison equation is ~tn y _ u; - (~tn-If Un-I>
whose solution is Un
5.7.
Let Yn
=1 -
= ~tn (~:) yH.
As before one has
. Zn. The equation becomes Yn+1
1 - 2Zo) ="21( Yn + ~
and
apply the result of problem 1.26.
5.8.
From the given solution one has Zo = 1 - (l - 2Z0 ) 1/ 2 coth k from which k = log 0- 1/ 2 and then
z; = 1 -
1+ 0
2"
- - 2 " (l
1-
e
1/2
- 2zo)
228
Solutions to the Problems
from which [x" - x
1
n
II ~ -(z* P1
Z ) = n
2 -(1 - 2z
P1
0
(}2" )1/2 _ _ • 1 - (}2"
5.12. The equation (5.5.2) is the homogeneous equation related to (5.5.1) when one takes
x;
Yn-I Yn
=--.
Imposing that (5.5.4) satisfies the nonhomogeneous equation one gets (5.5.3). In fact one has Yn+1 - 20Yn + xnYn + Zn = gn from which Yn =
Yn+1 20 - X n
z; -
+ 20 -
gn Xn
== Xn+IYn+1 + Zn+I'
Chapter 6 6.1.
Consider the term I;:~ A n-j-I Wi and the decomposition (A2.2). It follows that A = ZlI + I~=2 (AkZ kl + Zk2) == 8 + 8], where d is the number of distinct eigenvalues of A. Using the properties of the component matrices Zkj it is seen that 8 j = 8 for all j and jim S{ = O. )->00
Moreover Aj = sj + 8{. The sum I;:~ An-j-I Wi becomes 8 I;:~ Wi + I;:~ 8~-j-1 Wi. The quantity 8Wi is called essential local error. If the errors are such that SWi = 0 then one can proceed as usual. 6.5.
Applying the theorem B3 one has p(l)(z) = -4 Req and p'(z) 2(z - q). It follows that p(z) E N iff Req = 0 and Iql < 1.
6.8.
Rewrite the equation (6.1.15) such that the linear autonomous matrix A is the companion matrix of O'(z), obtaining En + 1 = (A + Bn)En + W n, where B; = 0, Jour. Austral. Soc., 20B (1978), pp.280-284. . [45] Di Lena, G. and Trigiante D., On the stability and convergence of lines method, Rend. di Mat., 3 (1982), pp. 113-126.
References
233
[46] Driver, R. D., Note on a paper of Halanay on stability offinite difference equations, Arch. Rat. Mech., 18 (1965), pp. 241-243. [47] Fielder, M. and Ptak, V., On matrices with nonpositive off-diagonal elements and positive principal minors, Czechoslovakia Math. lour., 12 (1962), pp.382-400. [48] Fielder, M. and Ptak, V., Some generalizations of positive definiteness and monotonicity, Num. Math., 9 (1966), pp.163-172. [49] Fort, T., Finite Differences and Difference Equations in the Real Domain, Oxford, (1948). [50] Gandolfo, G., Mathematical Methods and Models in Economics Dynamics, NorthHolland, Amsterdam, (1971). [51] Gantmacher, F. R., The Theory of Matrices, Vol. 1-2, Chelsea, (1959). [52] Gautschi, W., Computational aspects of three terms recurrence relations, SIAM Rev., 9 (1967), pp.24-82. [53] Gautschi, W., Minimal solutions of three term reccurrence relations and orthogonal poly. nomials, Math. of Computation, 36 (1981), pp. 547-554. (54] Gear, G. W., Numerical Initial Value Problems in Ordinary Differential Equations, Prentice-Hail, (1971). [55] Gear, G. W. and Tu K. W., The effect of variable mesh size on the stability of multistep methods, SIAM lNA, 1 (1974), pp. 1025-1043. [56] Gekerel, E., Discretization methods for stable initial value problems, Lecture Notes in Math., Springer, (1984). [57] Gelfand, A. 0., Calcul des Difference Finies, Dunod, Paris, (1963). [58] Gelfand, A. P., Calculus ofFinite Differences, Hindusten Publishing Corp., Delhi, (1971). [59] Godunov, S. K. and Ryabenki, V. S., Theory of Difference Schemes, North-Holland, (1984). [60] Goldberg, S., Introduction to Difference Equations, 1. Wiley, New York, (1958). [61] Gordon, S. P., Stability and summability of solutions of difference equations, Math. Syst, Theory,S (1971), pp. 56-75. [62] Grobner, W., Die Lie-Reiben und Ihre Anwendungen, Berlin, DVW, (1960). [63] Grujic, L. 1. T. and Siljak, D. D., Exponential stability oj large scale discrete systems, 1. Control, 19 (1976), pp.481-491. [64] Guckenheimer, 1., Oster, G., and Ipaktchi, A., The dynamics of density dependent population models, 1. Math. Biology, 4 (1977), pp. 101-147. [65] Hahn, W., Stability of Motion, Springer, Berlin, (1967). [66] Haight, F. A., Mathematidal Theories oj Traffic Flow, Academic Press, (1963). [67] Hairer, E. and Wanner, G., Algebraically stable and implementable Runge-Kutta methods of higher order, SIAM lNA, 18 (1981), pp. 1098-1108. [68] Halanay, A. and Wexler, D., Teoria Calitative a Sistemlor cu Impulsuri, Bucharest, (1968). [69] Halanay, A., Solution periodiques et presque-periodiques des systems d'equationes aux difference finies, Arch. Rat. Mech., 12 (1963), pp. 134-149. [70] Halanay, A., Quelques questions de la theorie de la stabilite pour les systemes aux differences finies, Arch. Rat. Mech., 12 (1963), pp. 150-154. [71] Hang-Chin, Lau and Pou-Yah, Wu, Error bounds for Newton type process on Banach spaces, Num. Math., 39 (1982), pp, 175-193. [72] Hartman, P. and Wintner, A., On the spectre of Toeplitz matrices, Am. 1. of Math., 72 (1950), pp.359-366. [73] Hartman, P., Difference equations: disconjuancy, principal solutions, Green's Junctions, complete monotonicity, Trans. Am. Math. Soc., 246 (1978), pp. 1-30. [74] Hebrici, P., Discrete Variable Methods for Ordinary Differential Equations, 1. Wiley, (1962). [75] Henrici, P., Error Propagation for Difference Methods,l. Wiley, (1963).
234
References
[76] Henrici, P., Applied and Computational Complex Analysis, Vol. 1, J. Wiley, New York, (1974). [77] Hildebrand, F. 8., Finite Difference Equations and Simulations, Prentice-Hall, (1968). [78] Hildebrand, F. 8., Methods of Applied Mathematics, Prentice-Hall, (1952). [79] Hoppensteadt, F. C. and Hyman, J. M., Periodicsolutions ofa logistic difference equation, SIAM JAM 32 (1977), pp. 73-81. [80] Hoppensteadt, F. c., Mathematical Methods of Population Biology, Courant Inst. of Math. Science, (1976). [81] Hurt, J., Some stability theorems for ordinary difference equations, SIAM JNA, 4 (1967) pp. 582-596. [82] Jones, G. S., Fundamental Inequalities for discrete and discontinuous functional equations, J. Soc. Ind. Appl. Math., 12 (1964), pp.43-57. [83] Jordan, c., Calculus of Fnite Difference, Chelsea, New York, (1950). [84] Kalman, E. and Bertram, J. E., Control System analysis and design via the'second method' of Lyapunov, Part 11, Discrete time Syst., Trans. ASME, Ser. D.J. Basic Enr., 82 (1960), pp. 394-400. [85] Kannan, R. and Ray, M. B., Monotone iterative methods for nonlinear equations involving a non-invertible linear part, Num. Math., 45 (1984), pp. 219-225. [86] Kato, T., Perturbation Theory for Linear Operators, Springer, (1966). [87] Khavanin, M. and Lakshmikantham, V., The method of mixed monotony and first order differential systems, Nonlinear Analysis, 10 (1986), 873-877. [88] LaSalle, J. P., The stability ofdynamical systems, Regional Conference Series in Applied Mathematics, SIAM (1979). [89] Ladde, G. S., Lakshmikantham, V., and Vatsala, A. S., Monotone Iterative Techniques for Nonlinear Differential Equations, Pitman Publishers Co., (1985). [90] Lakshmikantham, V. and Leela, S., Differential and Integral Inequalities, Academic Press, New York, Vol. I & II, (1969). [91] Lakshmikantham, V.and Vatsala, A. S., Method ofmixed monotonyfor nonlinear equations with a singular linear part, Appl. Math. and Computations, 23 (1987),235-241. [92] Lambert, J. A., Computational Methods in Ordinary Differential Equations, J. Wiley, (1973). [93] Lancaster, P., Theory of Matrices, Academic Press, (1969). [94] Levy, H. and Lessmann, F., Finite Difference Equations, McMillan, New York, (1961). [95] Li, T. Y. and Yorke, J.A., Period three implies chaos, Am. Math. Monthly, 82 (1975), pp. 985-992. [96] Liniger, W., Numerical solution of ordinary and partial differential equations, Notes of a cource taught at the Suisse Federal Institute of Technology, Lausanne, (1971-1973). [97] Lorenz, E. N., The problem ofdeducing the climatefrom the governing equations, TELLUS, 16 (1964), pp. 1-11. [98] Luenberger, D. G., Introduction to Dinamic Systems, J. Wiley, (1979). [99] Luke, Y. L., The Special Functions and Their Approximations, Academic Press, Vol. I, (1969). [100] Maslovskaya, L. V., The stability of difference equations, Diff. Equations, 2 (1966), pp.608-611. [101] Mate, A. and Nevai, P., Sublinear perturbations of the differential equation y = 0 and the analogous difference equation, J. Diff. Eq., 53 (1984), pp. 234-257. [102] Mattheij, R. M. and Vandersluis, A., Error estimates for Miller's algorithm, Num. Math., 26 (1976), pp.61-78. [103] Mattheij, R. M., Characterizations of dominant and dominated solutions of linear recursions, Num. Math., 35 (1980), pp.421-442.
References
235
[104] Mattheij, R. M., Stability of block LU-decompositions of the matrices arising from BVP, SIAM J. Alg. Dis. Math., 5 (1984) pp. 314-331. [105] Mattheij, R. M., Accurate estimates for the fundamental solutions of discrete boundary value problems, J. Math. Anal. and Appl., 101 (1984), pp.444-464 [106] May, R. M., Simple mathematical models with very complicated dynamics, Nature, 261 (1976), pp.459-467. [107] May, R. M., Biological populations with nonoverlapping generations. Stable points, Stable Cucles and Chaos, Science, 186 (1974), pp. 546-647. [108] McKee, S., Gronwall Inequalities, Zamm., 62 (1982), pp.429-431. [109] Meil, G. J., Majorizing sequences and error bounds for iterative methods, Math. of Computations,34 (1960), pp. 185-202. [110] Mickley, H. S., Sherwood, T. K., and Reed, C. E., Applied Mathemaitcs in Chemical Engineering, McGraw-Hill, N. Y., (1967). [Ill] Miel, G., An updated version ofthe Kantarovich theoremfor Newton's method, Computing, 27 (1981), pp.237-244. [112] Miller, J.C. P., Bessel Functions, Part II, Math. Tables, Vol X, British Association for the Advacement of Sciences, Cambridge Univ. Press., (1952) [113] Miller, J. J. H., On the location of zeros ofcertain classes ofpolynomials with applications to numerical analysis, J. Inst. Math. Applic., 8 (1971), pp.397-406. [114] Miller, K. S., An Introduction to the Calculus of Finite Differences and Difference Equations, Hold and Company, New York, (1960). [115] Miller, K. S., Linear Difference Equations, Benjamin, New York, (1968). [116] Milne-Thomson, L. M., The Calculus of Finite Differences, McMillan & Co., London, (1933). [117] Nevanlinna, o. and Odeh, F., Multiplier techniques for linear multistep methods, Num. Funct. Anal. and Optimiz., 3 (4) (1981), pp. 377-423. [118] Nevanlinna, o. and Liniger, W., Contractive methods for stiff differential equations, BIT, 19 (1979), pp. 53-72. [119] Nevanlinna 0., On the behaviour of the lobal errors at infinity in the numerical interaction of stable IVp' Num. Math., 28 (1977), pp.445-454. [120] O'Shea, R, P., The extention of Zubov method to sampled data control systems described by difference equations, IEEE, Trans Auto. Conf., 9 (1964), pp. 62-69. [121] Odeh, F. and Liniger, W., Non Linear fixed h stability of linear multistep formulas, J. Math. Anal. Appl., 61 (1977), pp. 691-712. [122] Olver, F. W. and Sookne, D. J., Note on backward recurrence algorithms, Math. of Computation, 26 (1972), pp.941-947. [123] Olver, F. W., Numerical solutions of second order linear difference equations, Jour. of Research, NBS, 7t8 (1967), pp. 111-129. [124] Olver, F. W., Bounds for the solution of second-order linear difference equations, Journal of Research, NBS, 718 (1967), pp. 161-166. [125] Ortega, J. M., Stability ofdifference equations and convergence ofiterative processes, SIAM JNA, 10 (1973), pp.268-282 [126] Ortega, J.M. and Rheinboldt, W. C., Monotone iterations for nonlinear equations with application to Gauss-Seidel methods, SIAM JNA, 4 (1967), pp. 171-190. [127] Ortega, J. M. and Rheinboldt, W. C., Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, (1970). [128] Ortega, J. M. and Voigt, G., Solution ofpartial differential equations on vector and parallel computers, SIAM Rev., 27 (1985), pp. 149-240. [129] Ortega, J. M., The Newton-Kantarovich theorem, Am. Math. Monthly, 75 (1968), pp. 658660. [130] Ortega, J. M., Numerical Analysis, Academic Press, New York, (1972).
236
References
[131] Ostrowski, A., Les points d'otraction et de repulsion pour l'iteration dans /'esace a n dimension, C. R. Acad. Sciences, Paris, 244 (1957), pp. 288-289. [132] Ostrowski, A., Solution of Equations and Systems of Equations, Academic Press, New York, (1960). [133] Pachpatte, B. 8., Finite difference inequalities and an extension of Lyabunov method, Michigan Math. J., 18 (1971), pp. 385-391. [134] Pachpatte, B. G., On some fundamental inequalities and its applications in the theory of difference equations, Ganita, 27 (1976), pp. 1-11. [135] Pachpatte, 8. G., On some discrete inequalities of Bellman-Bihari type, Indian J. Pure and Applied Math., 6 (1975), pp.1479-1487. [136] Pachpatte, B. G., On the discrete generalization ofGronwall's inequality, J. Indian Math. Soc., 37 (1973), pp. 147-156. [137] Pachpatte, B. G., Finite difference inequalities and their applications, Proc, Nat. Acad. Sci., India, 43 (1973), pp.348-356. [138] Pasquini, L. and Trigiante, D., A globally convergent method for simultaneously finding polynomial roots, Math. of Computation, 44 (1985), 135-150. [139] Patula, W., Growth, oscillations and comparison theorems for second order difference equations, SIAM J. Math. An., 10 (1979) pp.1272-1279. [140] Piazza, G. and Trigiante, D., Propagazione degli errori nella integrazione numerica di equazioni differenziali ordinarie, Pubbl. lAC III, Roma 120 (1977). [141] Pollard, J. H., Mathematical Methods for the Growth of Human Populations, Cambridge Univ. Press, (1973) [142] Popenda, J, Finite difference inequalities, Fasciculi Math., 13 (1981), pp: 79-87. [143] Popenda, J., On the boundness of the solutions of difference equations, Fasciculi Math., 14 (1985), pp. 101-108. [144] Potra, F. A., Sharp error bounds for a class of Newton-like methods, Libertas Matematica, 5 (1985), pp. 71-84. [145] Potts, R. B., Nonlinear difference equations, Nonlinear Anal. TMA, 6 (1982), pp. 659-665. [146] Redheffer, R. and Walter, W., A comparison theorem for difference inequalities, Jour. of Dill. Eq., 44 (1982), pp. 111-117. [147] Rheinboldt, W. C., Unified convergence theory for a class of iterative processes, SIAM JNA,5 (1968), pp.42-63. [148] Saaty, T. L., Modern Nonlinear Equations, Dover, New York, (1981). [149] Saaty, T. L., Elements of Queuing Theory with Applications, Dover, New York, (1961). [150] Samarski, A. and Niloaiev, E., Methodes de Resolution des Equation des Mailles, MIR, Moscow, (1981). [151] Sand, J., On one leg and linear multistep formulas with variable stepsize, Report, TRITANA-8112, (1981). [152] Schelin, C. W., Counting zeros of real polynomials within the unit disk, SIAM JNA, 5 (1983), pp. 1023-1031. [153] Schoenberg, I. J., Monosplines and Quadrature Formulae, In T.N.E. Greville, Theory and Applications of Spines Functions, Academic Press, (1969). [154] Scraton, R. E., A modification of Miller's recurrencealgorithm, BIT, 12 (1972), pp. 242251. [155] Sharkovskii, A. N., Coexistence of cycles of continuous map of line into itself, Ukrainian Math. J., 16 (1964), pp. 61-71. [156] Shintani, H., Note on Miller's recurrence algorithm, Jour. of Science of the Hiroshima University, Ser. A. 1,29 (1965), pp. 121-133. [157] Skeel, R., Analysis offixed step-size methods, SIAM JNA, 13 (1976), pp. 664-685. [158] Smale, S., The fundamental theorem of algebra and complexity theory, Bull. Am. Math. Soc.,4 (1981), pp. 1-36.
References
237
[159] Smith, R. A., Sufficient conditions for stability of a class of difference equations. Duke Math. Jour., 33 (1966), pp.725-734. [160] Spiegl, M. R., Finite Difference and Difference Equations, Schaum Series, New York, (1971). [161] Stephan, P., A" theorem of Sharkovskii on the existence of periodic orbits of continuous endomorphisms of the real line, Comm. Math. Phys., 54 (1977), pp. 237-248. [162] Straffin, P. D., Periodicpoints ofcontinuous functions, Math. Mag., 51 (1978), pp. 99-105. [163] Sugiyama, S., Difference inequalities and their applications to stability problems, Lec-· tures Notes in Math., Springer, 243 (1971), pp. 1-15. [164] Svirezhev, Y. M. and Logofet, D.O., Stability of Biological Community, MIR, Moscow, (1983). [165] Toeplitz, 0., Zur Theorie der Quadratischen Fozunlu von unendlichrillen Yerenderlichen: Gitinger Nachtrichten (1910), pp. 351-376. [166] Traub, J. F., Iterative Methods for the Solution of Equations" Prentice-Hall, (1964). [167] Trigiante, D. and Sivasundaram, S., A new algorithm for unstable three term recurrence relations, Appl. Math. and Compo 22 (1987), pp. 277-289. [168] Urabe, M., Nonlinear Autonomous Oscillations, Academic Press, (1976). [169] Van der Cruyssen, P., A reformulation of Olver's algorithm for the numerical solution of second order difference equations, Num. Math., 32(1979), pp. 159-166. [170] Varga, R., Matrix Iterative Analysis, Prentice-Hall, (1962). [171] Wanner, G. and Reitberger, H., On the perturbation formulas of Grobner and Alekseev, Bul. Inst. Pol. Iasi, XIX (1973), pp. 15-25. [172] Weinitschke H., Uber eine kass von iterations oerfahren, Num. Math, 6 (1964), pp, 395404. [173] Weissberger, A. (Editor), Technique of Organic Chemistry, Vol IV. Distillation, Interscience, N.Y., (1951). [174] Willet, D. and Wong, J. S. W., On the discrete analogues of some generalizations of Gronwall's inequality, Monatsh. Math., 69 (1965), pp. 362-367. [175] Wimp, J., Computation with Recurrence Relations, Pitman, (1984). [176] Yamaguti, M. and Ushiki, S., Discretization and chaos, C.R. Acad. Sc. Paris, 290 (1980), pp.637-640. [177] Yamaguti, M. and Hushiki, S., Chaos in numerical analysis of ordinary differential equations, Phisica, 3D (1981), pp.618-626. [178] Yamaguti, M. and Matano, H., Euler's finite difference scheme and chaos, Proc. Japan Acad., 55A (1979), pp. 78-80. [179] Yamamoto, T., Error bounds for Newton's iterated, derivedfrom the Kantarovich theorem, Num. Math., 48 (1986), pp. 91-98. [180] Zahar, R. V. M., Mathematical analysis of Miller's algorithm, Num. Math., 27 (1977), pp.427-447.
Subject Index
A
C
A-stability, 55, 162 Abel formula, 23 Absolute stability, 49, 55, 161, 167 region, 61, 162 Actractivity, 88 Adjoint equation, 31, 77, 223 Affine transformation, 135 Age class, 176 vector, 177 Antidifference, 3, 4, 6 Asymptotic stability, 48, 88,92,93, 108, 123, 128 Autonomous difference equation, 110, 112 Autonomous systems, 92
Canonical base, 30 Capacity of a channel, 189 Carrying capacity, 179 Casorati matrix, 31, 65, 70 Cauchy sequence, 99, 133, 137 Cauchy-Ostrowsky theorem, 178 Central differences, 167 Chaos, 179, 182 Characteristic polynomial, 35, 69, 72,143, 158 Chebyshev polynomials, 39, 47, 57, 142, 173, 213-215 Circular matrix, 225 Clairaut equation, 12 Class K, 105 Clenshaw's algorithm, 139, 142 Cobweb, 185, 193 Companion matrix, 69, 92, 201, 222 equation, 41 Comparison principle, 14, 95, 130 Component matrix, 92, 197, 198 Consistency, 50, 60, 61 Contractivity, 132 Convergence, 50, 60 Converse theorems, 115-119 Courant condition, 168 Cournot, 124 Critical points, 88 Cyclic matrix, 210
B
Backward differences, 167 Balance equation, 183 Banach lemma, 134, 208 Barna, B., 128 Bernoulli method, 59 numbers, 7, 9, 23 polynomials, 7, 9 Biomathematics, 114, 175-182 Boundary value problems, 56, 79, 173 Bounded solutions, 77
239
240
Subject Index
D
Dahlquist, G., 155, 163 Dahlquist polynomial, 55 Dalton's law, 182 Darwinian law, 180 Decrescent functions, 105, 107 Definition set, 2 Demand, 185 Derivative of a matrix, 203 Difference equations definition, 10 normal form, 11 Difference operator, 2 negative powers, 3 Discrete Bihari inequality, 18 Discrete Gronwall inequality, 15 Distillation, 182 Domain of stability, 111-115 E
Economics, 185 Eigenvalues, 60, 94 generalized, 79 semisimple, 92, 94 simple, 92 Equilibrium price, 186 Erlang's loss distribution, 191 Error bounds, 135, 151, 152 Euler function, 22 Euler method, 172 explicit, 55, 167 implicit, 55, 163 Euler-Mel.aurin formula, 24 Expected number, 191 Expenditure, 187 Exponential stability, 88, 90, 103, 123 F
Factorial powers, 4, 5 negative, 5 Fast Fourier Transform, 24 Fibonacci sequence, 38, 58, 189 Finite interval, 161 Finite sum, 4 First approximation, 102 First integral, 14, 135 Fixed points, 88 Floating point, 59 Floquet solutions, 95
Formal series, 43, 68 Frobenius matrix, 69 Functions of matrices, 195, 203 integral form, 203 Fundamental matrix, 65, 66, 91, 96 G
Gautschi, W., 153 Gear, G. W., 160 General solution, 31, 36, 66, 67 Generalized inverse, 82 Generating functions, 42 Geometric distribution, 190 Global attractivity, 88 asymptotic stability, 88, 112, 128 convergence, 128 error, 156 results, 128 Golden section, 58 Green's function, 84 one sided, 71 Green's matrix, 82 H
High order equations, 69 Homogeneous equation, 63, 66, 77 Homer rule, 24 Hyperbolic equation, 173
Infinite interval, 161 Infinite matrix, 84, 222 Integral of a matrix, 203 Interest rate, 58 Invariant set, 111, 131 Investments, 187 Irreducible matrix, 150, 209 Iterative function, 128 matrix, 128 methods, 127-153 non stationary, 129 J
Jordan canonical form, 204 K
Kantorovich, 1. V., 133
241
Subject Index L
La Salle invariance principle, 108, 130 Laurent series, 44 Leslie model, 176 Li and Yorke, 182 Limit set, 110 Limiting probability, 190 Linear difference equation, 27-61, 63-85, 96, constant coefficients, 34 stability, 47, 48 space of solutions, 28, 64 Linear independence, 29,64 Linear multistep methods, 50, 156 Local error, 160 Local results, 128, 129 Logistic equation, 179, 180, 193 Lp stability, 88, 108 Lyapunov function, 87, 104-124, 164 Lyapunov matrix equation, 113 Lypshitz derivative, 151 M
M-matrix, 150 Malthus, T. R., 175, 176 Market, 185 Matrix norm, 91, 207 logarithmic, 98 Matrix polynomial, 195 Method of lines, 166-172 Midpoint method, 61 Miller's algorithm, 139, 141 Minimal polynomial, 195 Minimal solution, 139, 140 Mixed monotone, 145 Models, 175-193 Mole fraction, 182, 183 Mole rate, 183 Monotone iterative methods, 144-153 Monotone sequences, 145, 149 Morse alphabet, 188 Multiple root, 35 Multiplicators, 78 N
National income, 187, 193 Negative definite functions, 105, 107 Newton formula, 23 integration formula, 24 iterative function, 134 method, 24, 58, 128, 130, 135, 152
Newton-Kantarovich, 133, 136, 151 Nonhomogeneous equation, 63, 77 Nonlinear case, 163 Nonnegative matrix, 93, 209 vector, 93 Normal matrices, 169, 173 Null space, 82 Number of messages, 188 Numerical methods, 155-173
o Oligopoly, 124 Olver's algorithm, 139, 142 One-leg methods, 163, 172 Operators A and E, 1, 39 Order of convergence, 59, 135 p
Parking lot problem, 192 Period two cycle, 181 Periodic coefficients, 93 equations, 94 function, 3 solutions, 75, 77, 94 Perron, 0., 75, 104 Perron-Frobenius theorem, 93, 177, 193,210 Perturbation, 102, 119, 138, 161 Perturbed equation, 139 Poincare theorem, 73 Poisson distribution, 192 process, 189, 191 Polynomials p and (T, 50-53, 60, 156 Population dynamics, 175 Population waves, 178, 179 Positive definite functions, 105, 106 Positive definite matrix, 112 Positive matrix, 209 Positive solution, 93 Practical stability, 119, 122, 138, 144, 159 Principal parts, 44
Q Quasi extremal solutions, 145 Quasi lower solutions, 144 Quasi upper solutions, 144 Queuing theory, 188, 189 R
R-Orthogonality, 80 Range, 82
242 Raoult's law, 182 Reducible matrices, 209 Reflecting states, 191 Region of convergence, 132 Relative volatility, 182 Renewal theorem, 193 Resolvent, 69, 203 Riccati equation, 13, 183, 184 Roundoff errors, 155, 159 Runge- Kutta, 173
s Samuelson, 187 Schur criterium, 211 polynomial, 49, 170,212 transform, 212 Secant method, 130 Second differences, 111, 114 Semilocal results, 128, 132 Sequence of matrices, 200 Shift operator, 2 Similarity tranformation, 70 Simpson rule, 61 Smale, S., 128 Spectral radius, 93, 158 Spectrum of a family, 168, 171, 173,203 Spectrum of a matrix, 196, 200 Stability, 88, 106, 123 Step matrix, 81 Stepsize, 160, 162 Sterling numbers, 5 Stiff, 161 Sturm-Liouville problem, 79, 80
Subject Index
Supply, 85 Sweep method, 142, 144
T Taylor formula, 4, 6 Test functions, 51 Toeplitz matrices, 173 Total stability, 119, 159 Traffic in channels, 188 Transpose equation, 32, 142 Trapezoidal method, 55, 172 Truncation error, 50 U
Uniform lp stability, 88 asymptotic stability, 88,91, 107, 157 stability, 88, 91, 107 Unstable problems, 139-144
v Vandermonde matrix, 35, 70, 73 Vapor stream, 182 Variance, 191 Variation of constant method, 32, 38, 67, 99, 11 Volatile component, 182 Von Neumann polynomial, 49, 52, 212
z z- transform, 46 Zubov theorem, 113 O-stability, 52, 158