Theory of Difference Equations Numerical Methods and Applications

THEORY OF DIFFERENCE EQUATIONS NUMERICAL METHODS AND APPLICATIONS Second Edition V. Lakshmikantham Florida Institute of...

Author: V. Lakshmikantham | Donato Trigiante

146 downloads 952 Views 11MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

THEORY OF DIFFERENCE EQUATIONS NUMERICAL METHODS AND APPLICATIONS Second Edition

V. Lakshmikantham Florida Institute of Technology Melbourne, Florida

Donate Trigiante University of Florence Florence, Italy

n MARCEL

MARCEL DEKKER, INC.

D E K K E R

Copyright © 2002 Marcel Dekker, Inc.

NEW YORK • BASEL

The first edition was published by Academic Press as Theory of Difference Equations: Numerical Methods and Applications, 1988. ISBN: 0-8247-0803-2 This book is printed on acid-free paper. Headquarters Marcel Dekker, Inc. 270 Madison Avenue, New York, NY 10016 tel: 212-696-9000; fax: 212-685-4540 Eastern Hemisphere Distribution Marcel Dekker AG Hutgasse 4, Postfach 812, CH-4001 Basel, Switzerland tel: 41-61-261-8482; fax: 41-61-261-8896 World Wide Web http://www.dekker.com The publisher offers discounts on this book when ordered in bulk quantities. For more information, write to Special Sales/Professional Marketing at the headquarters address above. Copyright © 2002 by Marcel Dekker, Inc. All Rights Reserved. Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, microfilming, and recording, or by any information storage and retrieval system, without permission in writing from the publisher. Current printing (last digit): 10 9 8 7 6 5 4 3 2 1 PRINTED IN THE UNITED STATES OF AMERICA


Preface Difference equations appear as natural descriptions of observed evolution phenomena because most measurements of time evolving variables are discrete, and as such these equations are in their own right important mathematical models. More importantly, difference equations also appear in the study of discretization methods for differential equations. Several results in the theory of difference equations have been obtained as more or less natural discrete analogues of corresponding results of differential equations. This is especially true in the case of the Liapunov theory of stability. Nonetheless, the theory of difference equations is a lot richer than the corresponding theory of differential equations. For example, a simple difference equation resulting from a first-order differential equation may have a phenomena often called the appearance of ghost solutions or the existence of chaotic orbits that can only happen for higher order differential equations. Consequently, the theory of difference equations is interesting in itself, and it is easy to see that it will assume greater importance in the near future. Furthermore, the application of the theory of difference equations is rapidly increasing to various fields such as numerical analysis, control theory, finite mathematics, and computer science. Thus, there is every reason for studying the theory of difference equations as a well-deserved discipline. The present book offers a systematic treatment of the theory of difference equations and its applications with special emphasis on numerical analysis. For example, we devote special attention to iterative processes and numerical methods for differential equations. The investigation of these subjects from the point of view of difference equations allows us to systematize and clarify the ideas involved and, as a result, pave the way for further developments of this fruitful union. Moreover, the deep connections with the Pascal matrix, which is a basic notion in combinatorics, are presented in Chapter 1. With respect to the previous edition of the book ([105]), we have added two new chapters (5 and 9) and revised and added additional sections in the remaining material. The newly added Chapter 5 contains the relations among difference equations and linear algebra. Such relations are usually not included in the existing books, leading to the erroneous conclusions that the two fields have no intersections. Finally, in Chapter 9 we have added some classical difference equations of relevant historical interest, such as the in Copyright © 2002 Marcel Dekker, Inc.

iv

PREFACE

Gaussian arithmetic-geometric mean. The book is divided into nine chapters and four appendices. The first chapter introduces difference calculus, deals with preliminary results on difference equations, develops the theory of difference inequalities and introduce the Pascal matrix along with many of its countless properties chosen in order to give a large overview of its application to many problems. In the second chapter, we present the essential techniques employed in the treatment of linear difference equations with special reference to equations with constant coefficients. Chapter 3 deals with the basic theory of systems of linear difference equations. Chapter 4 is devoted to the Liapunov theory of stability including converse theorems and total and practical stability. In Chapters 5, the relations between difference equations and banded matrices are presented. This gives us the opportunity to present both the theory of linear difference equations from another point of view and to give an overview of classical problems such as orthogonal polynomials, the euclidean algorithm, roots of polynomials, and the problem of well-conditioning. Chapters 6 and 7 deal with some applications of the theory of difference equations relevant in numerical analysis. In Chapter 8, we present applications of difference equations to many fields such as economics, chemistry, population dynamics, and queueing theory. Finally, in Chapter 9 we present some historically important uses of difference equations, i.e. the arithmetic-geometric mean and its generalizations, the Weierstrass iteration, and some applications of difference equations in number theory. The necessary linear algebra used in the book, as well as the relevant notions concerning the Schur criteria and the Chebyshev polynomials, are given in Appendices. Finally, several carefully selected problems at the end of each chapter complement the material of the book. Some of the important features of the book include: (i) development of the theory of difference inequalities and the various comparison results; (ii) unified treatment of stability theory through Liapunov functions and the comparison method; (iii) emphasis on the important role of the theory of difference equations in numerical analysis and some basic notions of combinatorics (the Pascal matrix and its properties); (iv) demonstration of the versatility of difference equations by various models in the real world; (v) timely recognition of the importance of the theory of difference equations and of presenting a unified treatment. The book can be used as a textbook at the graduate level and as a reference book.


PREFACE

v

We wish to express our immense thanks to S. Leela, F. Mazzia, F. lavernaro, P. Amodio, and L. Aceto for their helpful comments and suggestions. There are many changes with respect to the first edition. All chapters have been revised. Moreover, Chapter 1 has been enlarged and partially rewritten. Chapter 3 has been enlarged, and Chapters 5 and 9 are new. Many new problems have been added to almost all the chapters. After the publication of the first edition of this book, a rapid increase in the activity on this subject has occurred. In fact, a new completely dedicated Journal (Journal of Difference Equation and Application, Gordon and Breach) has been created. Moreover, many new books, some of a more general type (Agarwal [8]), others exploiting particular aspects of the subject (e.g., Kocic and Ladas [102], Ahlbrandt and Peterson [12], Kelley and Peterson [99], Elaydi [62], Jagermann [96]), have been written. This shows the vitality of the subject and its increasing importance in modern applications.

V. Lakshmikantham and D. Trigiante


Contents Preface 1 Discrete Calculus 1.0 Introduction 1.1 Discrete Calculus 1.2 Summation and Negative Powers of A 1.2.1 Equations reducible to simple form 1.3 Factorial Powers and Stirling Numbers 1.4 Bernoulli Numbers and Polynomials 1.5 Matrix Form 1.5.1 Pascal matrix and combinatorics 1.5.2 Pascal matrix and Bernoulli polynomials 1.5.3 Pascal matrix and Bernstein polynomials 1.5.4 Pascal matrix and Stirling numbers 1.6 Comparison Principle 1.7 Problems and Remarks 1.8 Notes 2 Linear Difference Equations 2.0 Introduction 2.1 Preliminarie 2.2 Fundamental Theory 2.2.1 Adjoint and transposed equations 2.3 The Method of Variation of Constants 2.4 Linear Equations with Constant Coefficients 2.5 Use of Operators A and E 2.6 Method of Generating Functions 2.7 Stability of Solutions 2.8 Absolute Stability 2.9 Boundary Value Problems 2.10 Problems and Remarks 2.11 Notes


viii 3 Linear Systems of Difference Equations 3.0 Introduction 3.1 Basic Theory 3.2 Method of Variation of Constants 3.3 Autonomous Systems 3.4 Systems Representing High-Order Equations 3.4.1 One-sided Green's functions 3.5 Poincare Theorem 3.6 Periodic Solutions 3.7 Boundary Value Problems 3.8 Problems 3.9 Notes 4 Stability Theory 4.0 Introduction 4.1 Stability Notions 4.2 The Linear Case 4.3 Autonomous Linear Systems 4.4 Linear Equations with Periodic Coefficients 4.5 Use of the Comparison Principle 4.6 Variation of Constants 4.7 Stability by First Approximation 4.8 Liapunov Functions 4.9 Domain of Asymptotic Stability 4.10 Converse Theorems 4.11 Total and Practical Stability 4.12 Problems 4.13 Notes 5 Difference Equations as Banded Matrices 5.0 Introduction 5.1 Initial Value Problems 5.2 Boundary Values Problems 5.2.1 Invertibility of tridiagonal matrices 5.2.2 Sufficient conditions for well-conditioning 5.3 Cyclic Reduction 5.3.1 The case of Toeplitz tridiagonal matrices 5.4 Problems and Remarks 5.5 Notes 6 Applications to Numerical Analysis 6.0 Introduction 6.1 Iterative Methods 6.2 Local Results


CONTENTS 6.3

6.4 6.5 6.6 6.7 6.8 6.9

Semilocal Results 6.3.1 Newton-Kantorovich-like theorems 6.3.2 Effect of perturbations Miller's, Giver's, and Clenshaw's Algorithms Boundary Value Problems Monotone Iterative Methods Monotone Approximations Problems Notes

7 Numerical Methods for Differential Equations 7.0 Introduction 7.1 Linear Multistep Methods 7.2 Finite Interval 7.3 Infinite Interval 7.4 Nonlinear Case 7.5 Other Techniques 7.6 The Method of Lines 7.7 Spectrum of a Family of Matrices 7.8 Problems 7.9 Notes 8 Models of Real World Phenomena 8.0 Introduction 8.1 Linear Models for Population Dynamics 8.2 The Logistic Equation 8.3 Distillation of a Binary Liquid 8.4 Models from Economics 8.5 Models of Traffic in Channels 8.6 Problems 8.7 Notes 9 Historically Important Equations 9.0 Introduction 9.1 Combinations of Means 9.1.1 Arithmetic-harmonic mean 9.2 Arithmetic-Geometric (Borchard) 9.2.1 Arithmetic-geometric mean II 9.3 The Weierstrass Method 9.4 Difference Equations and Prime Numbers 9.5 Problems 9.6 Notes Appendices


x A Function of Matrices A.I Introduction A.2 Properties of Component Matrices A.3 Particular Matrices A.4 Sequence of Matrices A.5 Jordan Canonical Form A.6 Norms of Matrices and Related Topics A.7 Nonnegative Matrices B The Schur Criteria B.I The Schur'Criteria C The Chebyshev Polynomials C.I Definitions C.2 Properties of Tn(z) and Un(z) D Solutions to the Problems D.I Chapter 1 D.2 Chapter 2 D.3 Chapter 3 D.4 Chapter 4 D.5 Chapter 5 D.6 Chapter 6 D.7 Chapter 7 D.8 Chapter 8 D.9 Chapter 9 Bibliography


Chapter 1

Discrete Calculus 1.0

Introduction

This chapter is essentially introductory in nature. Its main aim is to introduce certain well-known basic concepts in difference calculus and to present some important results that are not as well-known. Sections 1.1 to 1.4 contain needed difference calculus and some notions related to it, most of which are found in standard books on difference equations. Section 1.5 deals with a more modern approach consisting of the systematic use of the vector and matrix notation. This not only permits us to rewrite many results obtained in the previous sections in a shorter and more elegant form, but also to get new surprising ones. The central role in such an approach is played by the Pascal matrix. Applications to Combinatorics and to Computer graphics (Bernstein polynomials) are also presented. In Section 1.6 we develop the theory of difference inequalities and prove a variety of comparison theorems that play a prominent role in the development of the book. Several problems are given in Section 1.7 which, together with the material of Sections 1.1 to 1.5, cover the necessary theory of difference calculus.

1.1

Discrete Calculus

Let

where no is an integer number. The generic element in such a set will be denoted by n. We shall consider functions defined on N^Q and assuming values on 1R (or, when explicitly mentioned, on C). They are also called sequences and denoted by f ( n ] or by fn (sequence notation). However, any discrete set of points on which a one-to-one correspondence with N+0 can be established, may be used as definition set. For example, particular


2

CHAPTER 1. DISCRETE CALCULUS

circumstances may require the use of the following discrete sets:

where XQ € IR. The generic element in the above sets will usually be denoted by x. They are used when it is desirable to exhibit the explicit dependence of the function on the initial point. The advantage in using J^ h as the definition set is that, for a discrete function which is the approximation of a function defined on El, the dependence of the approximation on the parameter h, usually called stepsize, is explicit. In this chapter, we shall often use J.+ as the definition set whenever we need the dependence on x 6 El (or x e C) in order to consider, for example, derivatives with respect to x. There will be no difficulty in translating the results in terms of other notations. As a rule, we shall only use the sequence notation in the problems at the end of chapters. Definition 1.1.1 Let y : J+0 ~^ IR- Then A is the difference defined by Ay(x) = y ( x + l ) - y ( x )

operator, (1.1)

and E is the shift operator, defined by Ey(x)=y(x+l).

(1.2)

It is easy to verify that the two operators A and E are linear, and that they commute. That is, for any two functions y ( x ) , z(x) and any two scalars a, (3, we have

&(ay(x) + j3z(x)) E(cty(x) + 0z(x}}

= aAy(x) + /3Az(x), = aEy(x) + (3Ez(x),

and &Ey(x] = EAy(x). The second difference on y(x] can be defined as

In general, for every k G A^ + , A fc ,y(x) = A(A f c ~ 1 y(a:))

and

Eky(x] = y(x + k ) ,

with A°y(x) = E°y(x) — I y ( x ) , I being the identity operator such that Iy(x] = y ( x } . In the case when the definition set is N£QI one has Ay n = yn+i — yn and Eyn = yn+\ • It is easy to see that the formal relation between A and E is A = E — I and thus, powers of A can be expressed in terms of powers of E and vice versa. In fact, k

Z-A ?=o


. I/

i fc

\\

\ i I V /

1.1. DISCRETE

CALCULUS

and A',

(1.4)

where (^) are the binomial coefficients. Additional properties of the operator A are reported in the Problem section. Concerning the binomial coefficients, we recall that (°) = 1 and (.) = 0 for j ± 0. The above relations (1.3) and (1.4) are usually used to express the value of the generic term of a discrete function by means of its the variations at the previous points. We report as examples a few such relations (many others are presented in the problem section). It is worth mentioning that these kinds of relations have been considered very important in the past, and many of them are associated with the names of famous mathematicians. The reason is that they permit us to simplify many hand computations and to save time when hand calculating. But hand calculations are over, fortunately. Theorem 1.1.1 Let un be defined on NQ . Then

(1.5) Elu0.

(1.6)

i=0

Proof.

Just apply (1.3) and (1.4) to UQ.

Theorem 1.1.2 (Discrete Taylor formula) Let k, n £ be defined on NQ~ . Then

n-k v—\ n — s — 1 E k-l 3=0

k-i

E

• n 1=0

Proof.

n

From (1.5) it follows that

fc-i

E

A

n—k

+ i=0


fr-l-1

A

-

n

k < n and un

(1.7)

4

CHAPTER 1. DISCRETE

CALCULUS

By using the identity (see Problem 1.14)

one obtains (1.6). D A generalization of Theorem 1.1.2 is as follows. Theorem 1.1.3 Let j, k, n e N^, j < k — 1, k < n and un be defined on JV0+. Then k-l /

A

- '«O + \

n~k+j

J "

1.2

/

~

8

..

Summation and Negative Powers of A

Definition 1.2.1 Let uj : J^Q —> IR. The function LJ(X] is said to be periodic of period fc if u(x + k) — LJ(X). For example, u(x) — e z2vrx is a periodic function of period 1. The constant functions are particular periodic functions. It is easy to see that ACJ(.X) = 0 for any periodic function of period 1. When the function u(x] needs to be a polynomial, it must be a constant since the only polynomial taking infinite times the same value is the one of degree zero. Consider the equation Ay(x)

= g(x),

(1.9)

where g : J^o —> IR is a known function. The function y ( x ) . denned on the same set of points, is unknown. We shall denote by y(x) = A~lg(x) a particular solution of the above equation. It is not unique because y(x] = y(x] +uj(x), where uj(x) is an arbitrary periodic function of period 1. is also a solution of (1.9). The operator A""1 is called the antidifference operator and it is linear. Moreover, the operators A and A"1 do not commute since, when writing the above considerations in operator form, we have, AA- 1 - / and A'1 A - / + u(x).

Although the last expression is not formally correct, since u(x) is not an operator, nevertheless it is useful because it expresses in a compact form the fact that the operation A~~ T is defined up to an arbitrary periodic one function. If /. g : J+Q —>• IR are two functions such that A/(x) = A#(:r), then it is clear that f ( x ) = CJ(X)+LJ(X). In particular, if f ( x ) and g(x) are polynomials, A/ = A# implies f ( x ) = g(x] + c, where c is a constant.


1.2. SUMMATION AND NEGATIVE POWERS OF A

5

We shall now state the relation between the finite sum £]i=o f ( x + i) and the antidifference A~lf(x). Theorem 1.2.1 Let AF(z) = f ( x ) . Then n

V^ f(x + i} — F(x + n + 1) - FCx1) = F(x + z)l i= ^ +1 7

J

\ JL>

n

t' /

-»•

V **^

T

' fr 1 ^ -*- /

V

/

~"~"

V

'^

/17

0

*

(1 10") \

/

i=0

Proof. that

Since by hypothesis we have f ( x ) — AF(x), it is easy to see

i=0

i—O

Note that (1.10) can also be written as

i=0

If we leave the sum to remain indefinite and consider x as the discrete variable, we can express the relation (1.10) in the form

in analogy with the notation for indefinite integrals. In the case when the definition set is N+ , the foregoing formulas reduce to v—^

>

—

yi = A

_i

yi

• i -i i=n+\

^-^

and > y{ = A ^—'

_,

j/ + u;,

respectively. If to the solutions of 1.9 is imposed to satisfy a condition, for example to assume an assigned value y$ at XQ initial condition, then u is equal to yo and the solution becomes x-l

y(x} = yo+ Y^9(s}. S=XQ


(1.12)


1.2.1

Equations reducible to simple form

It may happen that more difficult linear difference equations or even nonlinear ones may be reduced, by more or less judicious transformations, to the simple linear form considered above. Consider, for example, the equation z(x + 1) -p(x)z(x)

= q(x),

Z(XQ) = ZQ.

(1-13)

By setting P(x) = l f i ~ * Q p ( t ) , P ( x o ) = 1 and dividing (1.13) by P(x + l ) , we have z(x+l) z(x) q(x) If we write y(x] = pr\ and g(x] = p ( + i \ > equation (1.13) now takes the form Ay(x) = g ( x ) . The solution of (1.13) is then given by

__

=

I

s=x0 r^b "^ l)

+z

(1-14)

s ^(s) n ?(*) ° n ?(*)•

X—1

X— 1

.S = XQ

t= 5+ l

X—1

t — XQ

We present a few examples below taken from the applications. Example 1 The following equation is often encountered in the study of propagation of errors of iterative processes. The reduction to the easy linear form is almost trivial (see Problem 1.28): yn+i = Q>ynThe solution is 1

"

a yn — -( yo) 2 • a

Example 2 (1.15) This equation is solved by yn — T2«(z), where Tj(z) are the Chebyshev polynomials of the first kind described in Appendix C. and z is a complex value to be determined later. In order to check the assertion, we need the so-called semigroup property of Chebyshev polynomials, i.e. Tjm(z] — T3(Tm(z)}. In fact, considering that T^z] — 2z2 — 1, one has = r 2 « 2 (z) = T2(T2n(z}) = T2(yn) = 2y2n - 1.


1.2. SUMMATION AND NEGATIVE POWERS OF A

7

Furthermore, by considering that yo = T\(z], we have z — yo, and then yn = T2"(yo). It is well known that when \z\ < 1, |Tj(z)| < 1, for all j. The oscillatory nature of yn is then easily recognizable from the above expression. More than that, small variations of the initial condition yo may drastically change the solution (see Problem 1.5). We shall see in the following example that Equation (1.15) is related to the so-called chaotic behavior. Example 3 2/n+i = cyn(l - yn). This equation is the easiest equation whose solutions may have chaotic behavior (see Chapter 8). For a generic value of c, it is not possible to write the solution in closed form. It is however possible in the cases c = 2 and c = 4. In fact, the substitution yn = ^f^ transforms the above equation to zn+i = l + -(4- 1). The case c = 2 leads to Example 1. The case c = 4 has been considered in Example 2. It seems worthwhile to focus on the different behavior of the two solutions. In the case c = 2 the solution is yn — ^ (l — (1 — 2yo) 2 ")- It is then evident that for 0 < yo < 1/2, one has limy n = 1/2. In the second case the behavior is oscillating. As matter of fact, the value c = 2 is outside the chaos window, while the value c = 4 is just inside. The following equation arises often in the theory of algorithms, especially when dealing with the class of divide and conquer algorithms. Example 4 yn = ky^ + f ( r i ) . Depending on the particular applications, /(n) may assume different forms. Particular important cases are

2. k = 2, f ( n ) = n Iog2 n; 3. fc = 7, f ( n ) = n2.

In the first case yn represents the maximum possible cost of a binary search. The second case arises in many different applications, for example in the odd-even merge sort. The third case arises in evaluating the complexity of certain algorithm of matrix multiplications. In all applications, the initial


8


value y\ is known. The solution is obtained by setting n — 2 m , zm — y^™ and gm = f(1m}. The resulting equation is z

m ^ KZm — i + 9m,

whose solution is m —1

j=0

i.e. / (log2 yn = k og2 n \ y\ + y^

(

h

/

The above mentioned cases correspond respectively to the solutions: 1- yn = y\ +log 2 n; / 2.

llr, —

The following example is taken from Amer. Math. Monthly (1999) (Problem N. 10578)

Example 5 (n + l)(n - 2)y n+1 - n(n2 -n-

l}yn + (n - l)3yn-i = 0 ,

n> 2

with j/2 — Z/3 = 1- The change of variable xn — nyn transforms the equation to •En+l ~~ '-En / -\\'^-"n ~~ xn — 1 ,-, (n - 1 ~— = 0, n —1 n —2 which, in turn, by setting zn — X "^'_^ XT ' , gives zn- (n- l)2 n -i = 0, whose solution is zn = (n — 1)!. The solution of the original equation is then ( n - 1)1 + 1 Un -

n

•

It is interesting to note that a theorem of number theory (the Wilson theorem) may be used to state that yn is an integer if and only if n is prime. In Table 1.2 we list differences arid antidifferences of the most common functions, omitting the periodic function uj(x}.


1.3. FACTORIAL POWERS AND STIRLING NUMBERS

1.3

9

Factorial Powers and Stirling Numbers

The factorial powers defined below have, in the discrete calculus, the same role that the functions xn have in differential and integral calculus. Definition 1.3.1 Let x E IR. The n-th factorial power of x is defined by

It is easy to verify from the above definition that ^^(n) _

nx(n-\]

and

i

AA —1,^(71—1) *< > = - *_(7"i) ( > +i „ .

(1-16)

/ i 1 T\ (i. 17)

According to the observation made in the last section, the periodic function is actually a constant, since both sides of (1.16) are polynomials. We also have x(m+n"> = x^m\x - m)( n) . When m = 0, this yields x (0+n) = z (0 M n) , which shows that x^ — 1. Moreover, for m — —n, we get 1 = x^°^ — x^~ n ^(x + n}(n\ which allows us to define the negative factorial power x^~ n ^ by

from which we also derive that 0^~ n ^ = ^. Moreover, from the definition, we have (-x) (n) = (-l) n (z + n - l)^. (1.18) Relations (1.16) and (1.19) suggest that it will be convenient to express other functions in terms of factorial powers whenever possible. For example, in the case of polynomials we have the following result. Theorem 1.3.1 Let n G NQ~ . The powers xn and the factorial powers are related by n

xn = '£tS?x®

(1.19)

i=0

where 5" are the Stirling numbers (of the second kind) that satisfy the relation

with S% = S? = 1, 5£ = 0, for n ^ 0.



10

Table 1.1: Stirling Numbers of the Second Kind

n\i 1 2 3 4 5 6

1 2 1 1 1 1 3 1 7 1 15 1 31

3

4

5

6

1 6 25 90

1 10 65

1 15

1

Proof. Clearly (1.19) holds for n — I . Suppose it is true for some n, multiplying both sides of (1.19) by x, we get n

n+l r=l n \

N

i=2 n+1

i=l

showing that (1.19) holds for n + 1. induction. D

Hence the proof is completed by

Stirling numbers Sf for i.n = 1 , 2 , . . . ,6 are given in Table 1.1. Using the relation (1.19), one can immediately derive the differences and the antidifferences of a polynomial. Theorem 1.3.2 The first difference of a polynomial of degree k is a polynomial of degree k — I and in general the s-th difference is a polynomial of degree k — s. Proof. It is not restrictive to consider xk instead of a polynomial of degree k. From (1.19) we have


1.4. BERNOULLI NUMBERS AND POLYNOMIALS which is of degree k — s.

11

D

By using Equation (1.16), it is easy to check that for x e ij

j- v 1

j

:_ -«••

The latter result is often called Stirling identity, often used in constructing the binomial coefficients table (Pascal table), i.e.

x\ (x — 1\ (x - l\ ( ;)-(j-G-i)-

(L2o)

Example 6 From (1.11) and (1.20) one readily obtains

b

£ 1.4

Bernoulli Numbers and Polynomials

From (1.19) one has that for every n e NQ~, x £ IR

where o;n is constant with respect to x. Let ujn = ^rjCn+i and let us set JL ^(i+i) Bn+1(x) = (n + l)A~1xn = (n + 1) ]T 5J1— -r- + Cn+1 i=l

(1.22)

l+

with BQ(X} = 1. The polynomials Bn(x) satisfy the relation ABn(x) = nxn~l.

(1.23)

They arc not uniquely defined because the constants Cn are arbitrary. Usually it is convenient to avoid the Stirling numbers in the determination of Bn(x). This can be accomplished as follows. Theorem 1.4.1 Let n G -/V^ , BQ(X) = 1 and Bn(x) be polynomials satisfying (1.23). Then the two functions

i=0

and Gn(x) = nxn-1 differ by a constant.


(1.25)

12

CHAPTER 1. DISCRETE Proof.

CALCULUS

From (1.23) one has n-l /

n x

( ) ~~ 2^ {i i=Q \

and

AG n (:c) = n[(x + I)71"1 - xn~l Hence, it follows that AF n (x) — AG n (x) and since F n (x), Gn(x) are polynomials, we have F71 (x} (1.26) \ / — G 71(x} V / +' d 71 ? \ * / where dn are constants (with respect to x). D When the constants dn have been fixed, (1-24) allows us to construct the polynomials Bn(x). The constants dn are fixed by imposing one more condition to be satisfied by Bn(x). The most commonly used condition is dBn(x) ~rf7~~ n 5 n - l ( < r ) '

(

27)

or

I11 / B n (x)dx = 0, f o r n = l , 2 , . . . . Jo One, in fact, has the following result.

(1.28)

Theorem 1.4.2 If for every n e N+, the polynomials Bn(x] satisfy (1.23) with BQ(X) — 1 and either (1.27) or (1.28) is satisfied, then (1-29) Proof. Let us start with (1-27). Differentiating (1.26) and using (1.27), we have nFn-i(x} = F^(x) — G'n(x] = nGn-i(x}. This implies that 0 = n(Fn_i(x) — Gn-\(x)} = ndn^i from which it follows dn-\ = 0. Let us now suppose that (1.28) holds. From (1.24) we obtain J() Fn(x)dx = /Q1 B()(x}dx = 1 and f^ Gn(x)dx = 1. Because of (1.26), we now get 1 — 1 + d n , which implies dn = 0. n As we have already observed. (1.29) define the polynomials Bn(x) uniquely. They are called Bernoulli polynomials. The first five of such polynomials are as follows:


B0(x)

= 1,

Bi(x]

- z-i,

B2(x)

— x 2 — x H—1 , 6

1.5. MATRIX FORM

13

_ -

3

3 1 - -x2 + -x,

~4 rr

o 3+ , x2 -2ar

3Q .

The values of Bn(0) are called Bernoulli numbers and are denoted by Bn. As an easy consequence of (1.29), we see that the Bernoulli numbers satisfy the relation n-l / \

(1.30) which can be considered as the expansion of (1 -f B)n — Bn, where the powers Bl are replaced by Bi. This property is often used to define Bernoulli numbers. It can be shown that the Bernoulli numbers of odd index, except for J9i, are zero (see Problem 1.19). The values of the first ten numbers are 5

o = l,51 = --,52 = - 1 B 4 = -35,B6 = -,B8 = - - , B 1 o = g g .

From (1.23), applying A"1 to both sides, we get

n

A simple application of (1.31) is the following: suppose that x takes integer values. Then, from (1.11) and (1.29), we see that x

Bn(x]

= -[Bn(m

x=0n

from which we get the sum of the (n - l)-th powers of integer numbers. When n — 3, for example, we have m

E x=0 1.5

o

-•

J-

o

Matrix Form

Discrete calculus has many applications and constitutes the background of many old and new disciplines such as numerical analysis, combinatorics, umbral calculus, theory of algorithms, wavelets, etc. The use of vectors and matrices permits us not only to state results in a shorter and more compact form, but also to generalize such results. In this section we shall develop such an elegant approach.


14

CHAPTER 1. DISCRETE

CALCULUS

Table 1.2: Differences and Antidifferences /(x) c

A / ( x ) A ~ 0

a

:

)

ex

(r 1 \rx \C — 1 )C

C:r / i ~r^l • > £ / * •

xcxT

/ -1 \ T -4- 1 (c - l)x(r 4- cTx+i

C^' C \ ^rrj (I x - ^TI ) , c // -1I

/XN

/

Vri-l^

cos(ax + b]

—2 sin f sin (ax + bx + f )

/

/ (

rx C

ln^

•

1

, i \

sm(ax + o)

o - n

X

I

X

, i , o \

2sm ^ cos (ax + b + |)

cosfax+fe-^'

—

log(x + c)

1.5.1

Pascal matrix and combinatorics

In most applications the central role is played by the so-called Pascal matrix and its countless properties. Even if such a matrix is the oldest known matrix, the systematic study of its properties is very recent. Most results already obtained and used in the previous sections may be deduced in a more elegant and easy way by using the matrix notation and, of course, the Pascal matrix. We shall give here a short presentation of the main properties of such a matrix along with a few examples of its applications. Let n > 0. The entries of the Pascal matrix of dimension n are defined as follows:

(1.32)


15

1.5. MATRIX FORM

It is a lower triangular matrix whose entries on each row are the binomial coefficients. It is strictly related to the creation matrix H defined by

\

( 0 1 H =

(1.33)

n-l

0)

To see the relation between the two matrices, let us state a few properties of the matrix H. Let e^, i = 0, 1, . . . n — I be the unit vectors in IRn. Whenever the index of such vectors will result greater than n — 1, the corresponding vector will be assumed to be the null vector. From the definition of H one has Het = i + l e i i and Theorem 1.5.1 The Pascal matrix is given by P — eH . Proof.

Since Hn — 0 (i.e. it is equal to the zero matrix), we have n-l

i=0 Such a matrix has the same entries of P. In fact,

^•_^C7+^ r s=0

5=0

s=0

where Sij is the Kronecker symbol1 which is one if both indices are equal and zero otherwise. The above expression is then equal to zero if i < j. Otherwise one has

3 and this completes the proof.

D

T

The matrix PP is called the symmetric Pascal matrix. Its entries are

(PPT),, = 1

(1.34)

The Kronecker symbol is a sort of alien in this setting, where there exists the corresponding symbol T° ).


16

CHAPTER 1. DISCRETE

CALCULUS

This relation contains the famous Vandermonde convolution formula, i.e. (see Riordan [157] and Problem 1.32))

The power Px, for all x G IR, is easily defined as Px — exp(xH). With arguments similar to those used above, one easily checks that the nonzero entries of Px are (1-35) It follows that all the powers of P, included the negative ones, are easily provided. Combining such powers appropriately, many famous combinatorial identities can be established. Here we report some examples.

2z

where Q are the Catalan numbers. The first identity is obtained from P-P = P 2 , by using (1.35). The second is similarly obtained by multiplying P and its inverse obtained again by (1.35) by taking x = — 1. Other identities are not so easy, but their proof in matrix form is always easier, or at least more elegant. To give an example of more involved results, we give here a proof of the identity used in the previous section. To do this, we need to introduce the shifted Pascal matrices. Let s be an integer. The shifted Pascal matrix Ps is defined by

(1.36)

The entries of the inverses of such matrices have also a simple expression, i.e.

(see Problem 1.33).


1.5. MATRIX FORM

17

By considering the shift matrix / 0 1

K= \

, 1

(1.37)

0 // n x n

one may easily check the following result.

Theorem 1.5.2 For all values of s one has (i) Ps=e(H+sK); (n) Ps_! =(I-K)P3; (in) PS = PS^(I + K)Proof. The proof of the first relation is similar to the one provided in Theorem 1.5.1. The remaining two are easy consequences of the Stirling identity (1.20) on the combinatorial coefficients. We leave as an exercise the remainder of the proof. D From them, the following relations (valid for all integers s) easily follow:

P = (I-K)SPS, Ps = (I - K)P8(I + K), PSK-KPS KPS

= KPSK, = P^K,

Let e = (1,1,..., 1)T. From (ii) one has Ps-\P~le = GQ. Suppose now we want to prove the first identity used in Section 1.1, i.e. q—n

j=0

n-l

The left hand side can be written as

t

\ q —n + n

where m — q — n. In matrix form, it is equivalent to


m +n

18


Many other properties of P can be established by considering the following differential equation in IRn:

~ = Hy, ax

(1-38)

whose solution is y ( x ) = Pxy(0). By varying the initial condition, one obtains many known functions all satisfying the above equation (see [4]). For example, the vector £(x) = ( x ^ x 1 , . . . ,xn~l)T is one of such functions. It corresponds to the choice y(0) = CQ. One has P£(x) = exp((x -f l)H)eo — £(x + 1) and, in general,

This very simple relation may be used to establish many combinatorial identities. For example, by taking x — 1 and j — 1 we have Pe = £(2), which is a short form of writing fc */ ;IS* \\

k

\=2k

Analogously, from £(x — j ] — P

J

^(x) one obtains, for example, £(0) =

A less trivial example is the following (see Am. Math. Monthly (1997) Problem 10632). Example 7 Evaluate the expression

The expression can be rewritten as

I" C(x - l)xmdx

Jo


1.5. MATRIX FORM

19

It is an easy matter to verify that F ' ( y ) = 0, i.e. that F ( y ) is independent of y. The original expression is then equal to m'r?'

/"I F(0) - / xn(l - x)mdx =

(JTl ~~r 71 ~r 1J-

JO

If instead of the real numbers x in the definition of £, we use either the operator E or A, the relations (1.5) and (1.6) among the operators A and E are essentially obtained. In fact, one has £(E) = Pf (A).

(1.39)

When applied to a first term of a sequence U Q , W I , . . . , the s-th row of the above relation provides the values of u s _i in terms of the differences A ? WO, i — 0 , . . . , s — 1. It is worth noting that ax

px = HPX,

(1.40)

since Px is a fundamental matrix for (1.38).

1.5.2

Pascal matrix and Bernoulli polynomials

Let us consider now the Bernoulli polynomials Bi(x) and the vector

By (1.27), this vector also satisfies (1.38), and then b(x) = Pxb(Q). It follows that the values of the polynomials are easily obtained once the entries (Bernoulli numbers) of 6(0) are known. Moreover, for all integers j, we have b(x + j ] = Pjb(x). The property (1.28) is imposed by considering the matrix n

~l

rl Li —

/

±

(IX

—

/

f]i -

7TT .

&S (* + !)!

Jo

The mentioned property becomes now £6(0) - e 0 ,

and then 6(0) = L~I€Q, i.e. the Bernoulli numbers are the entries of the first column of the matrix L~1 . Moreover, since both P and L are polynomials of the same matrix H, they commute. This permits the assertion that 6(x) = F X 6(0) = PxL-leQ = L~lPxeQ =


20

CH AFTER 1. DISCRETE

CALCULUS

which shows that the matrix L is the transformation matrix between the Taylor expansion and the Bernoulli expansion. In other words, if a function f(x] has a Taylor expansion f ( x ) = fT£,(%) + high order terms, where /7 is the vector containing the coefficients of the expansion, the expansion in terms of Bernoulli polynomials will be: /(:/;) — f7 Lb(x) + high order terms. From the definition of L it easily follows that LH = HL = P-I.

(1.41)

This relation permits us to obtain many known relations. We give a few examples below. Example 8 By taking the difference between two successive vectors 6(.x), we have b(x + l)-b(x) = (P-I)b(x) = (P-I}PxL-le0 = (PThis is nothing but the property (1.23) in vector form.

Example 9 By multiplying on the right by b(x). we get (P — I}b(x] HLb(x) = H£(x) which is equivalent to (1.29).

Example 10 By multiplying on the right by 6(0). we get (P - /)6(0) — HLb(0) = H£(0) = He.Q which is equivalent to (1.30). 1.5.3

Pascal matrix and Bernstein polynomials

To quote one more application of the Pascal matrix, we discuss the Bernstein polynomials which are the foundations of modern computer graphics (see Farin [64]). Such polynomials B^(x) are defined as if

0

otherwise,

i,j — 0 , . . . . n — 1.

We define the Bernstein matrix Be(x] by setting (Be(x}}ij — B-j(x). By using the Pascal matrix, the matrix Be assumes the very simple form


1.5. MATRIX FORM

21

~l 2

n

(1.42)

1

where Dx = diag(l,x,x :... ,x ~ }. The easy check is left as exercise (see also [4]). Prom (1.42), it is very simple to derive the properties of the Bernstein polynomial. For example it can be seen as a similarity transformation to the diagonal form of Be. The eigenvalues of Be are then the diagonal entries of Dx. The first eigenvalue is 1, to which corresponds the eigenvector e — i.e the first column of P which is e = ( 1 , 1 , . . . , 1)T. It then follows that (1.43)

Bfe = e.

In vector form the above expression states a well known property, namely that the Bernstein polynomials form a partition of unity. Almost trivial is the proof of the so called subdivision property stating that Be(ct) = Be(c}Be(t). By the way, we note the interesting and useful relations among the newly defined matrix Dx and the matrices H and P: Px = DXPD-X,

_X = xH.

Moreover, since

/1 rl

1

P -i

>-i1 = P Be(x) dx = P / Dxdx P" o Jo by considering that

fl P / D-r-dx = / DxdxPi, Jo 10 where PI is the shifted Pascal matrix (see (1.36)), and by using a result in Theorem 1.5.2, we obtain

Be(x}dx

= '0

/1

\

/I

n /

, 1 \

1

0

... 0 \

2

i \

(

l

1

° 1

2

2

..

'•• 1


0 \

0

I

'•. o

1 i/

CHAPTER 1. DISCRETE

22

CALCULUS

This is a compact form of writing a very important property of Bernstein polynomials, i.e. (1.44) Finally, we note the lesser known result \fx £ IR\{0}

(1.45)

and, more in general terms, for all integers j (1.46)

1.5.4

Pascal matrix and Stirling numbers

The Stirling numbers may be also patterned in a matrix form, i.e.

/ s° 0

S=

0

0

0

\

1

S ^l

0

V o

-1 /

The nonvanishing entries of S are the Stirling numbers of the second kind defined in Section 1.3. Considering the above defined vector £(x) and the analogous vector, defined by means of factorial powers,

the Stirling transformation considered in Section 1.3 is written in matrix form as £(z) = Sri(x}. The reverse relation is, of course, defined by means of the inverse matrix S~l whose entries are called Stirling numbers of the first kind. By considering the Vanclermoride matrix, defined by its columns

and the analogous matrix Vx = (r/(x), 7](x + 1), . . . , r,(x + n ~ 1)) . the above expression leads to Wr = SVT.


1.5.

MATRIX FORM

23

The matrix Vb is upper triangular, as can be easily checked from the definition of the factorial powers. We have then found an LU type factorization of the Vandermonde matrix WQ. The result may by refined by introducing the diagonal factorial matrix Df = diag(l, 1!, 2!, . . . , n!). In fact one has Vb = DfPT. Moreover, since Wx satisfies the differential equation ^-Wx = HWX, ax

(1.47)

as can be immediately checked by considering that each column of Wx satisfies (1.38), one then has Wx = PXW0 = PxSDfPT, which is a factorization of type LDU. Such factorization is used in numerical analysis to solve efficiently the Vandermonde systems of large dimension (see Golub[78]). Moreover, we get also Vx = S-PXSV0,

(1.48)

which is obtained by considering that ax

X

ax

X

and that es~~lHSx = S~1PXS. A deeper relation between the matrices 5 and P can be stated as follows. The relation (1.16) in vector form becomes ATJ(X) = HTJ(X). Since the columns of Vx are just made by the successive vectors ^(x), we also have AFX - HVX,

from which we obtain V\ = VQ + HVo and then ViV^"1 = I + H. By considering (1.48) for x = 1, we obtain I +H

(1.49)

or P — I = SHS~l. Moreover, since DJ1HD/ — K, we also have (SDf)-1P(SDf)

= I + K.

(1.50)

The above discussion proves the following theorem. Theorem 1.5.3 The matrix SDf transforms the Pascal matrix P to the Jordan bidiagonal form.


24

1.6


Comparison Principle

One of the most efficient methods of obtaining information on the behavior of solutions of difference equations, even when they cannot be solved explicitly, is the comparison principle. In general, the comparison principle is concerned with estimating a function satisfying a difference inequality by the solution of the corresponding difference equation. In this section, we shall present various forms of this principle. Theorem 1.6.1 Let n e N+0,r > 0 andg(n,r) be a nondecreasing function with respect to r for any fixed n. Suppose that for n > HQ, the inequalities

yn+\

< g(n,yn),

(1.51)

un+i

> g(n,un)

(1.52)

hold. Then

yno < uno

(1.53)

implies that yn

< un,

n > n0.

(1-54)

Proof. Suppose that (1.54) is not true. Then, because of (1.53) there exists a k e N+0 such that yk+i > Wfc+i- It follows, using (1.51), (1.52), and the monotone character of g, that

g(k.Uk) < uk+i < yk+i < g ( k , y k ) < g(k,uk) which is a contradiction. Hence the proof. D Usually in applications, (1.52) is an equation and the corresponding result is called the comparison principle. Corollary 1.6.1 Let n G N£Q, kn > 0 and yn+\ < knyn + pn. Then, for n > HQ, we have n —l

yn < yno n

s~n0

n— l

n—l

ks +

S

Ps

s=no

II

kr

r=s+l

-

( li55 )

Proof. Because kn > 0 the hypotheses of Theorem 1.6.1 are verified. Hence yn < unj where un is the solution of the linear difference equation un+i = knun+pn,

uno = yno.

(1.56)

By (1.12), we see that the right-hand side of (1.55) is the solution of (1.56). D


1.6. COMPARISON PRINCIPLE

25

Theorem 1.6.2 Let g(n, s, y] be defined on N+0 x N£Q x IR and nondecreasing with respect to y. Suppose that for n G n-l

, yno < pno implies yn < un,n > no, where un is the solution of the equation n-l

s=no

Proof. If the claim is not true, then there exists a k e N+0 such that > Wfc+i and ys < us for s < k. But, + 1, s, y5) - g(k + 1, s, us)] < 0 s=no

which is a contradiction.

D

Corollary 1.6.2 (Discrete Gronwall inequality). Let n £ N+Q,kn > 0 and n

yn+i no, n— 1

yn

Theory of Difference Equations Numerical Methods and Applications

Theory of Difference Equations: Numerical Methods and Applications

Numerical Methods and Applications

Galois theory of difference equations

Galois Theory of Difference Equations