Markov Set-Chains (Lecture Notes in Mathematics)

Lecture Notes in Mathematics Editors: A. Dold, Heidelberg E Takens, Groningen B. Teissier, Paris

1695

Springer Berlin Heidelberg New York Barcelona Budapest Ho ng Ko ng London Milan Paris Singapore Tokyo

Darald J. Hartfiel

Markov Set-Chains

~ Springer

Author Darald J. Hartfiel Department of Mathematics Texas A&M University College Station, Texas 77843-3368, USA e-mail: hartfiel @math.tamu.edu

Cataloging-in-Publication Data applied for

Die Deutsche Bibliolhek - CIP-Einheitsaufnahme Hartfiei, Darald J.: M a r k o v set-chains / Darald J. Hartfiel, - Berlin ; Heidelberg ; N e w York ; Barcelona ; Budapest ; H o n g K o n g ; L o n d o n ; Milan ; Paris ; Santa Clara ; Singapore ; T o k y o : Springer, 1998 (Lecture notes in mathematics ; 1695) ISBN 3-540-64775-9

Mathematics Subject Classification (1991): 60J 10, 15A51, 52B55, 92H99, 68Q75

ISSN 0075- 8434 ISBN 3-540-64775-9 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. 9 Springer-Verlag Berlin Heidelberg 1998 Printed in Germany The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typesetting: Camera-ready TEX output by the author SPIN: 10650093 46/3143-543210 - Printed on acid-free paper

Preface These notes give an account of the research done in Markov set-chains. Research papers in this area appear in various journals and date back to 1981. This monograph pulls together this research and shows it as a cohesive whole. I would like to thank Professor Eugene Seneta for his help in writing Chapter 1. In addition, I would like to thank him for the co-authored research which is the basis for Chapter 4. I would also like to thank Ruvane Marvit for his help in locating references for some of the probability results in the Appendix. Finally, I need to thank Robin Campbell whose typing, formatting, etc. produced the camera ready manuscript for this monograph.

Contents

Introduction

3

1

Stochastic Matrices and Their Variants 1.1 A v e r a g i n g effect of stochastic matrices . . . . . . . . . . . . . . . 1.2 T h e coefficient of ergodicity . . . . . . . . . . . . . . . . . . . . . 1.3 N o n n e g a t i v e matrices . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Markov a n d n o n h o m o g e n o u s Markov chains . . . . . . . . . . . . 1.5 Powers of stochastic matrices . . . . . . . . . . . . . . . . . . . . 1.6 N o n h o m o g e n e o u s p r o d u c t s of matrices . . . . . . . . . . . . . . .

3 3 5 8 13 15 19

Introduction to Markov Set-Chains 2.1 Intervals of m a t r i c e s . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Markov set-chains . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Comi6uting steps in a Markov set-chain . . . . . . . . . . . . . . 2.4 Hi-Lo m e t h o d , for c o m p u t i n g b o u n d s at each step . . . . . . . .

27' 28 35 40 46

Convergence of Markov Set-Chains 3.1 U n i f o r m coefficient of ergodicity . . . . . . . . . . . . . . . . . . . 3.2 C o n v e r g e n c e of Markov set-chains . . . . . . . . . . . . . . . . . . 3.3 Eigenvector description of a limit set . . . . . . . . . . . . . . . . 3.4 P r o p e r t i e s of the limit set . . . . . . . . . . . . . . . . . . . . . . 3.5 Cyclic Markov set-chains . . . . . . . . . . . . . . . . . . . . . . . 3.6 Cesaro set-sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Cesaro s e t - s u m c o m p u t a t i o n s . . . . . . . . . . . . . . . . . . . . 3.8 L u m p i n g in Markov set-chains . . . . . . . . . . . . . . . . . . . .

59 59 62 67 69 72 75 82 84

Behavior in Markov Set-Chains 4.1 Classification a n d convergence . . . . . . . . . . . . . . . . . . . . 4.2 Weak law of large n u m b e r s . . . . . . . . . . . . . . . . . . . . . 4.3 Behavior in Markov set-chains . . . . . . . . . . . . . . . . . . . . I M o v e m e n t w i t h i n the t r a n s i e n t states . . . . . . . . . . . II M o v e m e n t from a t r a n s i e n t s t a t e to a n a b s o r b i n g s t a t e . . III M o v e m e n t w i t h i n a n ergodic class . . . . . . . . . . . . .

91 91 100 106 106 108 110

CONTENTS

viii

Appendix A.1 A.2 A.3 A.4 A.5

Norms . . . . . . . . . . . . . . . . . . . . . . . . . . O p e r a t i o n s on c o m p a c t sets . . . . . . . . . . . . . . C o n v e x sets, p o l y t o p e s , a n d vertices . . . . . . . . . . . . . . . . D i s t a n c e c n c o m p a c t sets . . . . . . . . . . . . . . . P r o b a b i l i t y results . . . . . . . . . . . . . . . . . . .

115 . . . . . . . . . . . . . . . . . . . . . . . . . . . .

115 115 116 118 121

Bibliography

123

Index

131

Chapter 0

Introduction One of the major problems in mathematics is to predict the behavior of a system. Traditionally, in the mathematical model of a system, the required data is assumed to be exact while in practice this data is estimated. Thus, the mathematical prediction of the system can only be viewed as an approximation. A more realistic way to describe a system is to use intervals which contain the required data. This approach is used in systems of linear equations

Ax = b by assuming that the required data satisfies

P~A O. Set ~i' = a -

G(ep

-

eq).

Then (i',~, a n ( 1 % - % ar(; sign coml)atible, 6' E C, and c(6') < c(~). By the induction hylmthesis

~' = Z

~

((~i - ej)

(z,j)Ez where JV'/j > 0 and all ei - ej sign (:ompatible to 5 ~ and thus sign compatible to (f. Thus,

,~ = ,~,,(e,, - ,;,,) + ~

@ - ( e , - ej)

(~,j)Ez t,tl(~ desired result.

[]

We extend the definition of T to any generalize(l stoehasti(: matrix A, with row sums a. Using that ak is the k-th row of A, T(A)

1 IIo, - ,,,~ II 2 i,j 1 = ~ nlax Z lac~ - o,j.~[ =

- max

i,j

= a -- ll.lill Z

nlin{ai.,, ajs }.

T h e th,,or,,nl giving (1.4), f(,r generalized stochastic matrices, follows. Theorem

1.2. l, br any genc.ralizcd stoeh, astic matri:r, A, T(A)

= .,a• ll,~ II

IlaAII. I

1.2. T H E C O E F F I C I E N T O F E R G O D I C I T Y

7

Proof. T h e technique we use is as in establishing formulas for n o r m s of matrices in general. Using L e m m a 1.1, for any ~ E C, II~AII =

Z

1

; .......

I..T.~.+;;~....~.~.+.~.,?..-.......T~..+.,=~.....~.~.+.~ ...... 9 ..... .-...~

L Th,1

Th,2

...

Th,g

Th,g+l

Thg+2

...

(1.9) ThJ

1.4. M A R K O V

AND NONHOMOGENOUS

MARKOV

CHAINS

13

where each Ti, i = 1 , . . . ,g, with g > 1, corresponds to an essential class, each Ti, k = g + 1 , . . . , h, corresponds to an inessential class and for each of these k's there is some j such that Tkj ~ 0 (if inessential classes are present). D e f i n i t i o n 1.5. A nonnegative matrix is irreducible if it consists of a single essential class. (Thus, 1 x 1 irreducible matrices contain a positive entry. Some authors allow the 1 x 1 zero matrix to be irreducible). Thus, in particular, all matrices T 1 , T 2 , . . . ,Tg in the canonical form (1.9) are irreducible as are any of the T g + l , . . . ,Th (if present) which are not 1 x 1 zero matrices. The following result, which relates to inessential vertices, if any, is important for stochastic matrices. L e m r n a 1.3. For a nonnegative matrix T with no zero rows and some inessential vertices, there is a path from any inessential vertex to an essential vertex. Proof. Similar to Lemma 1.2.

[]

We need one more definition before we focus on the special properties of powers of stochastic matrices. D e f i n i t i o n 1.6. A nonnegative matrix T is primitive if for some positive integer k, T k = ,(t! k.))_~ > 0 (that is, each tl k) > 0). Clearly, a primitive matrix is irreducible.

1.4

M a r k o v chains and n o n h o m o g e n o u s M a r k o v chains

Markov chains and nonhomogeneous Markov chains are covered in Isaacson and Madsen (1976), Iosifescu (1980), as well as in Seneta (1981). In this section we briefly describe these two chains. A Markov chain is usually described in terms of random variables. For this we consider a sequence of experiments, done in steps of time t l , t 2 , . . . . The outcome of any experiment is one of n states, say S l , . . . , sn. Without loss of generality we can assume that Sk = k for all k. Set S = { S l , . . . , sn} and let X k : S ~ S be a random variable defined for the k-th experiment. We assume P r { X k = sik I X k - 1 = sik_l,.-. ,Xo = Sio} = P r { X k = sik I X k - 1 = 8ik_l} for all k and all states s i ~ , . . . , sio. Simply put, the probability distribution of X k depends only on the state occupied at the end of the previous experiment. We also assume P r { X a = sik I X k - 1 = s ~ _ l } = P r { X a + t = si~ I X k + t - 1 ---- 8i~-1}

14

C H A P T E R 1. S T O C H A S T I C M A T R I C E S A N D T H E I R V A R I A N T S

for all positive integers t and thus P r { X k = sj I X k - 1 = s~} = aij

for all k. These probabilities determine a transition matrix A = [aij] which, using laws of probability, can be used to show P r { X k = sj IX0 = si} = a ~2 !k)

(1.10)

where A k = [a!~)l In addition, if what is known about the process initially is L t 3 J" that it is in si with probability Yi, that is, P r { X o = si} = Yi,

for all i, then, under these conditions, Pr{Xk = sj} = yAkej

where y = ( Y l , . . . , Yn). The sequence {Xk}k_>0 is called a Markov chain. Often, a Markov chain is simply depicted in a diagram form. 81 82

E x a m p l e 1.6. The Markov chain with transition matrix A = ~ [:62:4s] can be diagramed as shown in Fig. 1.4. .2

.6

Figure 1.4: Diagram of A. We will say that the Markov chain is in state sj at step k if Xk = sj. The behavior of a Markov chain concerns its movement among the states as the number of steps increase. From (1.10) it is clear that powers of stochastic matrices play a fundamental role in studying the behavior of a Markov chain. Nonhomogeneous Markov chains adjust the classical Markov chain description by allowing the probabilities for each Xk to change at different steps. Thus, there is a transition matrix at each step, say A1, A 2 , . . . respectively. Analyzing as with Markov chains, if P r { X o = si} = Yi

for all i, then Pr{X~. = sj} = yA1 ... A k e j

(1.11)

where y = ( y l , . . . , y,~). Thus from (1.11), for nonhomogeneous Markov chains, products of stochastic matrices play an important role.

1.5. P O W E R S

1.5

OF STOCHASTIC

Powers

MATRICES

of stochastic

15

matrices

Much of Markov chain theory concerns limits of powers of stochastic matrices. We describe a basic result. For it, we need the following theorem about inessential vertices. T h e o r e m 1.5. Let A be a stochastic matrix with some inessential vertices, and written in canonical f o r m (1.9). Let Q denote the bottom right submatrix of A associated with the inessential vertices. Then, as k --+ 0% Qk __~ O.

Proof. Let the totality of essential vertices be denoted by E and of inessential vertices by I. Then Qk = ,[ a i(k), j ), i , j E I, is the submatrix in the lower right corner of A k = Laij , (k) ), again a stochastic matrix. Taking i C I, by Lemma 1.3, there is a ko(i) and a j ( i ) E E such that a(kO(O) ij(O > O. Thus, E

(ko(i))

aij

> O.

jcE (k+l) (k) Now, since the row sums of Q are at most 1, ~ aij = ~ ~ ais asj _< jEI jGI scI _(k) 9 Thus ~ ~ij ^(k) is non-increasing with k. Now, for ~is

sEl

jEI

k _>ko(i) (ko(i))

E

alk) ko and i E I. So, for any integer m _> 1,

E jEI

_(mko+ko) = E E "is-(rnk~a(kO)sj ttiJ jEI sEl < 0E _

Amko) (Zis

sEI < 0 m+l -+ 0 as m --+ ec. Thus, for all i, the non-increasing nonnegative sequence ~ ctij _(k) -+ 0 jEI as k --+ cx~ since a subsequence tends to O. Hence, Qk __+ 0 as k --+ co. [] The type of matrix used in our limit result follows.

16

CHAPTER 1. STOCHASTIC MATRICES AND THEIR VARIANTS

D e f i n i t i o n 1.7. A stochastic matrix A is said to be regular if it corresponds to precisely one essential class of vertices, together with, possibly, some inessential vertices, and the stochastic submatrix A1 of A in the canonical form corresponding to the essential class is primitive. Thus, in canonical form (1.9), A1 appears as follows A21

A2 "

A description of the limit of powers of regular stochastic matrices depends on the following two lemmas. L e m m a 1.4. If a stochastic matrix A is regular then for all sufficiently large k, A k has a positive column. Proof. Considering A in canonical form, it is clear from the definition of primitive that for sufficiently large kl, A~ ~ > 0 and, by Theorem 1.5, that for each i E I there is a j(i) E E such that a ij(i) (s~) > 0. Then A s~+l = A A kl has all columns corresponding to essential vertices strictly positive, and similarly for A s, k > kl. [] The next lemma is a consequence of the averaging effect of a stochastic matrix on a column vector. L e m m a 1.5. Let w be an arbitrary vector and A a stochastic matrix. If z = Aw, then minwj < m i n z j , maxzj <maxwj. J J ~ Pro@ Since zi

=

~aijwj j

and ~ a i y = 1, m ! n w j < ~ a i j w j j 3 j

for all i.

= zi

~_ m a x w j J []

The following theorem is fundamental in the theory of Markov chains. T h e o r e m 1.6. (Ergodic Theorem) If the stochastic matrix A is regular, then A k ---+ ey where y is the stochastic eigenvector of A belonging to the eigenvalue 1. _(k+l)

Proof. Writing A k = (a~k)) we have that uij

(k)

= ~ aisasj , so for fixed j, from 8

Lemma 1.5

9

(k+l)

m/ina~ ) ~ mlnaij

(k+l)

,maxaij

_~ m/axal k)

so the min and max sequences of entries both approach finite limits. Using Lemma 1.4, let k* be an integer such that A s* has a positive column so T ( A k" ) < 1. Then by Theorem 1.1, since A (m+l)k* -~ AS*A ink*, maxal• k'+k*) - ,,~m _ : _ uij _(ink" +k')_< T(AS* )( maxaij(mS') 0, h > 1, define

Fp,h = (f~p,h)) --_ Ap+lAp+2... Ap+h

(Forward Product)

where A1, A 2 , . . . is a fixed sequence of stochastic matrices, and Ak = (aij (k)). We shall say that a forward product is weakly ergodic if and only if

f!p,h) ~,s _ f(p,h) J js --+ 0 as h -+ oc for each i,j, s,p. This is clearly equivalent (see (1.2)) to saying

T(Fp,h) -~ 0

for each

p _> 0

as h - ~ oc. To extend Theorem 1.6 to nonhomogeneous Markov chains requires two lemmas. 1.6. If P and Q are stochastic matrices, Q regular, and PQ or Q P have the same graph (positions of positive entries) as P, then P has a positive

Lemma

col~tmn. Proof. Since Q is regular, by L e m m a 1.4, Qk has a positive column for some k. If PQ ~ P (have the same graphs) then p Q k ~ p so P has at least those columns positive which are positive in Qk. The case Q P ~ P is done similarly. [] 1.7. If Fp,h is regular for each p > O, h > 1, then Fp,h has a positive column for h > t where t is the total number of distinct graphs (nonzero matrix patterns) corresponding to regular matrices. Lemma

C H A P T E R 1. S T O C H A S T I C M A T R I C E S A N D T H E I R V A R I A N T S

20

Proof. For fixed p there must be numbers a, b, 1 < a < b < t + 1 such that Ap+l Ap+2

. . 9A p + a + l

. . 9A p + b

"~ A p + l A p + 2

9 .. Ap+a

since the number of distinct graphs is t. By Lemma 1.6, Ap+l Ap+2 9.. Ap+a has a positive column. Thus, Fp,h has a positive column for h >_ t (this column need not be the same for all h). [] The following is a generalization of Theorem 1.6. T h e o r e m 1.9. If Fp,h is regular for each p >_ O, h > 1, and

min+aij(k) ~ '7 > 0

(1.12)

uniformly for all k >_ 1 (where rain + is the minimum over all positive entries), then Fp,h is weakly ergodie. Proof. Writing h = k t + r , 0 O, h > 1 and m i n + a o ( k ) _> 7 > 0

uniformly for all k > 1 (where rain + is the minimum over all positive entries), then lim Bp,h -~ Y , a rank one matrix that depends on p. Further, there are h-+oo constants K and 13, 0 < ~ < 1, such that IIBp,t,, - Yll < g~ h-

22

C H A P T E R 1. S T O C H A S T I C M A T R I C E S AND THEIR V A R I A N T S

Proof. Adjusting the proof of Theorem 1.6 for the sequence {Bp,h}h>_l shows that this sequence converges to a matrix, say Y. The proof of T h e o r e m 1.9 shows that Bp,r is weakly ergodic and thus Y is rank one. Since {Bp,h}h>_l converges to a rank one matrix for any p _> 0, for any fixed h >_ 1, {Bp+h,9}g>_lconverges to a rank one matrix, say X. Thus, XBp,h = Y.

(1.14)

If each row in Y is the vector y, (1.14) says that y is in the convex hull of the rows of Bp,h. Thus, for all k,

Nb(kp'h)

-

where b!p'h) is the s-th row of

Yll

-
1, set pi

. We consider . the . interval . [p, q].

=

1

1

n-

Note t h a t (-~,

1) E

[p, q] so [p, q] ~ O. To construct a vertex v, for i > 1 choose vi a s Pi or qi- Set vl = 1 - ~ v i . i:>l

T h e n vl >_ 1 1 -

(n - 1)[~

~q~

= 1-(n-

1)[~+

~ ]

= 0 and vl _< 1 -

EPi

i>1

i>1

~( 1_1)] =2K. Finally, by construction, it is clear t h a t

[p,q]

at least 2 n-1 vertices.

=

has

CHAPTER 2. INTRODUCTION TO M A R K O V SET-CHAINS

34

E x a m p l e 2.6. Let p = (.I, .2, .4) and q = (.3, .4, .6). We c o m p u t e vertices of the tight interval [p, q]. To find vertices, according to L e m m a 2.3, choose a position in the vector for the free variable. For all other positions, put in the lowest or highest possible values. This leads to four feasible vectors. On each vector, insert in the free variable position the value, if possible, t h a t determines a stochastic vector. Check to see if the resulting vector lies in the interval [p, q]. F o r / = 1, (a)

( Z l , .2, .4). Set zl = 1 - .6 = .4. Note .1 _< zl _< .3 is false so this is not a vertex.

(b) (zl, .2, .6). Set Z 1 = 1 - .8 = .2. Since .1 < zl < .3, (.2, .2, .6) is a vertex. (c) (Zl, .4, .4). Set zl = 1 - .8 = .2. Since .1 < zl < .3, (.2, .4, .4) is a vertex. (d) ( z l , . 4 , . 6 ) . vertex.

Set zl = 1 - 1 = 0. Since .1 < zl < .3 is false, this is not a

For i = 2 we have (.1, .3, .6), (.3, .3, .4). For i = 3 we have (.1, .4, .5), (.3, .2, .5). A g r a p h depicting these vertices is given in Fig. 2.5. x=.l

y=.2

~=.4

Figure 2.5: G r a p h of [(.1, .2, .4), (.3, .4, .6)]. In M a r k o v set-chains, m a t r i x intervals are also i m p o r t a n t . Definition

2.4. Define 12n = {A: A is an n x n stochastic matrix}.

Let P and Q be n x n nonnegative matrices with P < Q. Define the corresponding interval in f~n as

[P,Q] -- {A: A is an n x n stochastic m a t r i x with P < A < Q}. We will assume t h a t P and Q are such t h a t [P, Q] r 0.

2.2. M A R K O V SET-CHAINS

35

If P and Q satisfy pij--

min

ai~

AE[P,Q]

and

qij=

"

max a~j AE[P,Q]

for all i and j, then [P, Q] is called tight9 The interval [P, Q] can be constructed by rows9 L e m m a 2.4. Let [P, Q] be an interval9 Then [P, Q] = {A: Ai E [Pi, Q,] ]or all i where Ai

Pi, Qi are the i-th rows o] A, P, Q respectively}9 Thus, the results on vector intervals can be applied, by rows, to any interval [P, Q]. We can use vector intervals to determine if [P, Q] is tight, to show [P, Q] is a convex polytope, and determine its vertices. In addition we can note t h a t [P, Q] has at most (n2'~-1) n vertices and may have (2n-1) n vertices. It is important, in later work, to note that [P, Q] being tight implies simultaneous tightness of the entries in any column of P and of Q. C o r o l l a r y 2.3. Suppose the interval [P, Q] is tight9 Then ]or any integer i,

1 < i < n, there is a matrix A E [P, Q] with i-th column the i-th column o] P and a matrix B E [P, Q] with i-th column the i-th column o] Q.

2.2

Markov set-chains

In this section we define Markov set-chains and provide some basic results about them. D e f i n i t i o n 2.5. Let M be a compact set of n x n stochastic matrices 9 Let s l , . . . , sn be states and consider the set of all nonhomogeneous Markov chains, with these states, having all their transition matrices in M. Thus, we call M a transition set. Define

M 2=MM={A1A2: M k+l = M M k = { A I A 2 :

A1,A2EM} A I E M , A2 E M k}

= {A1...A~:

A1,...,AkEM}.

We call the sequence M, M 2, ... a Markov set-chain. Let So be a compact set of 1 • n stochastic vectors. This set will contain the set of all possible initial distribution vectors for our nonhomogeneous Markov chains9 Thus, we call So an initial distribution s e t .

36

CHAPTER

2.

INTRODUCTION

TO MARKOV

SET-CHAINS

Define

SI

= SoM

Sk+l = SkM

= {x:

x = y A where y C So and A E M}

= {x: x = y A w h e r e y E S k a n d A E M } ---- {X: x = y A 1 . . . A k

where y E So and A 1 , . . . ,Ak E M}.

We call S k the k-th distribution set and So, S o M , . . .

a Markov set-chain with initial distribution set So. An example is helpful. E x a m p l e 2.7. Suppose a plant provides a technical course on safety procedures for its employees. The probability of passing this course is approximated as .9, with some fluctuation, no more than .01, from offering to offering. To model this problem as a Markov set-chain we provide two states, g = graduated and c = taking the course. The transition set is

M=

{ [ c] A = g c

~ a

0 1-a

: .89_l converges to a rank one matrix (A regular matrix A would assure this), then t h a t rank one matrix has rows formed from the stochastic eigenvector for A. Thus, the set of stochastic eigenvectors belonging to the set of A's is important. The above papers describe, under various settings, this set. Wesselkamper (1982) showed how some of this work can be applied to program schemata.

Chapter 3

Convergence of Markov Set-Chains In this chapter we provide conditions which assure t h a t a Markov set-chain, or related such chain, converges. In addition, we give a description of the limit set.

3.1

U n i f o r m coefficient of ergodicity

To o b t a i n convergence criteria for Markov set-chains, we extend the notion of coefficient of ergodicity 7- to a transition set M . Of course, 7- can vary over M. A b o u n d on this variation, for intervals, follows. Theorem

3.1. L e t M be the interval [P, Q] and A , B E M .

Then

IT(A)- 7-(B)I < IIA- Bll _< IIQ- PII. Proof. For each i, define ~i = bi - ai where ai and bi are the i-th rows of A, B respectively. T h e n 1

T ( A ) = ~ m ~ IIa, - ~11 1

= - m a x It(hi - ei) - (bj - ej)ll 2 i5 1 1 < -~ m a x Nbi - bjll + - m a x llei -

i,j

2

i5

1

< T ( B ) + ~ max([le~ll + Ilejll) %3

1

< T ( B ) + ~(2)m/ax Ileill

< T ( B ) + lib - All.

ejll

60

CHAPTER

3.

CONVERGENCE

OF MARKOV

SET-CHAINS

Similarly, T(B) < T(A) +

lib

- All.

Thus, IT(A) - T(B)I

_< l i b - All.

Finally, using t h a t Pi and qi are the i-th rows of P and Q respectively, l i b - All = m a x Ilbi - ai[I _ miax Ilqi - pill = I1Q - PII.

T h e result follows.

[]

We extend 7- to M by using the largest value of 7- there. D e f i n i t i o n 3.1. For any M , define 7-(M) = m a x 7-(A).

(3.1)

AEM

We call T ( M ) the uniform coefficient of ergodicity for M . Note t h a t since 7- is continuous and M c o m p a c t , the m a x i m u m of 7-(A) over all A E M is achieved and thus equation (3.1) defines 7-(M). Also, note t h a t since 7-(A) _< 1 for all A E M it follows t h a t 7-(M) >_e for all

i.

Since Soo C A M k' for all i, there is a zi E A M k~ such that d({zi}, Soo) > e. Now, since A is compact and each zi E A it follows t h a t zl, z 2 , . . , has a limit point z C A. And, by the continuity of d, d({z}, Soo) _> e. But, since A M k+l C_ A M k for all k, for any k, Zk E A M k~ for all k > ki, and thus z E A M k'. H e n c e z E A M k for a l l k and so z E Soo. This yields a contradiction from which it follows t h a t AM, A M 2 , . . . converges to Soo. []

3.2. CONVERGENCE OF M A R K O V SET-CHAINS

63

General convergence criteria requires the following notion. D e f i n i t i o n 3.3. Suppose r is an integer such that T ( A 1 . . . A ~ ) < 1 for all A1,.. 9 , Ar E M. Then M is said to be product scrambling and r its scrambling integer. The importance of product scrambling follows. L e m m a 3.2. If M is product scrambling and r its scrambling integer then

T ( M ~) < 1. Further, if s is a positive integer, then T ( M ~+s) N1,

IIAk~Ak2... Akk --All < -~.

(3.4)

Since lim M k = M ~ there is a positive integer N2 such that if k > N2 then k--+oo

d(M k, Moo) < ~. Thus, for all k > N2, there is a Ak E M ~ such t h a t JJAklAk~ ...Akk---AkJl < 2" Let N = m a x { N 1 , N 2 ) and k > N. Then by (3.4) and (3.5), J I A - AkJl < e. Thus, A is a limit point of Moo and hence, A E M ~ . Hence, P C_ M ~176 Now let A E Moo. For each k there is a A k l A k 2 . . . Akk E M k such t h a t JlAklAk2... Akk -- All ~_ d(M k, Moo). Since lim (M k, M ~ ) = 0, A is the limit k-+oo

of the point sequence All,A21A22,... 9 Thus, A E P and hence M ~ C P. Putting together, P = Moo.

[]

The stochastic eigenvector description of Moo uses the following9 D e f i n i t i o n 3.7. Let M be product scrambling. For each A E M , let YA denote the stochastic eigenvector, belonging to the eigenvalue 1, of A. Let YA = eyA and define

EI = {YA: A E M} E 2 = {YA1A2: A1,A2 e M )

Ek = {YA1...A~ : A1,... ,Ak E M}

oo

Further define Coo =

[.J Ek. Thus, Coo is the set of rank one matrices YA k----1

where A E M k for some k. Finally, define E = E o o , the closure of E ~ . Theorem

3.7. Let M be product scrambling. Then E --- M ~.

3.4. P R O P E R T I E S OF T H E L I M I T S E T

69

Proof. T h r o u g h o u t the proof we let r denote the scrambling integer for M. We first show that E C M ~ . For this, let Y ~ E ~ . Then there is a positive integer m such that Y C Era. Write Y = ey. Then there is a matrix A1 ... Am E M m having y as an eigenvector. Thus, (A1 ... Am) r is scrambling and by Theorem 1.6 k__, . . l i m(A1 . Am) rk = Y. Let s be a positive integer less than rm. Choose any A 1 , . . . , Arm-1 E M . T h e n - A 1 . . . - A s ( A 1 . . . A m ) ~k E M krm+s. Since l i m [ - A 1 . . . A s ( A 1 . . . A m ) rk] = k-~c~

Y for all s < m r , we have a point sequence A 1 , A 1 A 2 , . . . , A 1 . . . A ~ m - 1 , (A1...Am)~,-AI(A1...Am)r,... ,A1...A~m-I(A1...Am) r, (A1...Am)2r,... , with Y as its limit point. Thus, by L e m m a 3.3, Y E M ~ and so E ~ c_ M ~ . Since M ~ is compact, E C M ~ as well. We now show t h a t M ~ C E. For this, let e be a positive constant. Let A E M ~ . Then since lim M k = M ~ there is a positive integer N1 such t h a t k--~oo

if k > N1, d ( M k, M ~ ) < e/2. Since T ( M r) < 1, there is a positive integer N2 such t h a t if k >_ N2, T ( M k) < e/4. Now let N = max{N1, N2, r}. Then there is a matrix A1 ... AN E M N such that IIA1... AN - All < e/2.

(3.6)

Since M is product scrambling, and N > r, A 1 . . . AN is scrambling and thus by T h e o r e m 1.6, lim (A1 . . . A N ) k = ey k--+cx~

where y is the stochastic eigenvector for A 1 . . . A N . rem 1.8,

Furthermore, by Theo-

IIA1... AN -- eyll < 2 T ( M N) < e/2.

(3.7)

Putting (3.6) and (3.7) together, I l e y - All < e. Thus, A is a limit point of E ~ and so A E E. From this it follows t h a t M ~ C_ E. Putting together, E = M ~ . []

3.4

P r o p e r t i e s of the limit set

In this section we give some qualitative information about limit sets. We use b o t h the diameter A of a set as well as the Hausdorff metric d. We first consider intervals. Theorem Then

3.8. Let M be the interval'[P,Q] and M ' be the interval [P',Q']. d(M, M ' ) _ 0 it follows t h a t 5 ( C ~ + , , C ~ ) 1r For this, let now show t h a t 5 ( C ~ , C~k+,) < - 2v 1_--:-

= 0. We

x ( I + . . . + A 1 . . . A ~ k _ l ) / w k C Cw~. Using any A ~ E M ~ , define A=I+--.+A1...A~k_I

and u s i n g w t copies of A,

B = (I + . . . + A1 . . . A ~ _ I )

+ AI...Aw~(I+""

+ (A1...Aw,)~'-I(I +-"

+ A1...Aw~_I) +""

+ A1...A~,-,).

Note t h a t x B / w k+t E C~k+~ for any x E A~. Now, define A = A + ... + A, the sum of w t copies of A. Then, applying R e m a r k 2,

I I x A / w k - :~:B/w T M II = IlxA/wk+~

- xB/wk+~ll

< 2wtT(A) --

< -

wk+t

2(1 + . . . + 7 ~ - 1 ) W k

-

1 < 2 -wkl_T" Thus,/i(C~,~ Cw~+,) < '

-- ~

2

1

1---:-~"

Finally, since lim C~,~+~ = Coo, it follows t h a t t-~oo

2

1

d(C~,, Coo) < wk 1 --7-" [] We extend the previous work to Cesaro set-sums in general. We use the notation that if m, w are positive integers then we can write

m = uo w~

+ Ul wl

~- 9 9 9 -~- U k w k

where 0 < ui < w for all i. L e m m a 3.6. If a Cesaro set-sum o/cycle length w is such that T ( M i ) < T < 1 for all i, then

d(u~176 m

m

< - 2 w ( km+ l )

1-T

where Cm has initial distribution set So and Cwo , . . 9 , Cwk have initial distribution sets An.

80

C H A P T E R 3.

CONVERGENCE

P r o @ We first show that Cm C ~ ~ 1 7 6 --

x ( I + A1 + ' "

m

+

t a w ~

" "

"

OF MARKOV

SET-CHAINS

/-* + UkWk m t~wk' For this, let

+ A1 . . . A m - 1 ) / m

9 Cm.

Now, group the terms so that there are uow ~ terms in the first group followed by UlW 1 terms in the second group, etc. Thus, we have x ( I + A1 + . . . + A 1 . . . Auo~,O_l)/m + ' "

+ x ( A 1 . . . A8 + . . . + A 1 . . . A m - 1 ) / r n

where uow ~ + 999 + U k _ l w k - 1 = 8. Now arrange the first group into Uo groups of w ~ terms, the second group into ul groups of w 1 terms, etc. Thus, we have (x + xA1 + ... + x A i . . . A u o w O _ l ) / m + ... + [ x ( A t . . . A , + . . . + At...A,+,,,~_t) + . . . + x ( A l . . . A t

+... + AI...A,,,_I)llm

where t = uo w~ + . . . + (uk -- 1)w k. Rewrite this as [(x)I + ( x A , ) I + . . . + ( x A 1 . . . A~,o,oO_l)I]/m + . . - + [ ( x A 1 . . . A , ) ( I + A~+, + . . " + A~+I . . . A ~ + ~ - I ) +... + (xA1...At)(I

+ At+l + " " + A t + l . . . A m - 1 ) ] / m .

Thus, since Cwo,... , C ~ are all convex, x( I + A1 + . . . +

U0 w0 _

A1 . . . A m - 1 ) / m 9 - - U w o m

~1 w l _.

+ --C~1 m

+ ...+

?.tk w k

m

C~.

Hence, we have that Cm c_ So, a ( C m , :Cwo + . . . + We now consider

noW ~

m

Uk W k

,(u:o

C~o + . . . +

U k wk

m

C~.

) =o

C~o + ' "

+

m

C~,Cm

)

9

For this let y o , . . . , Yk 9 So and U0 w0

y =

m

yoI +

UlW 1

Yl (I + ".. +

m

A~1) . . . . . A(1) w l _ 9 2~ I w

UkWkyk(I + ... + A~k) . . . A ~ 2 _ l ) l w k 9 m

UO wO

1 + " ' " "tUk w k

C~o + ... +

m

cw~.

m

Construct a sequence from M1, M 2 , . . . by repeating A~k),.

9

9

,

_(k) Uk 2tw~

times,

then repeating A~ k - l ) , . . . , A(wks times and finally A~~ , A(~ This gives a sequence of m - 1 matrices. Using this sequence, define for any z 9 So, x

=

z(I

zl(k) "*2 zl(k) + A~k! + "'1

+''"

zl(k)a(k) ~'2

+ "*1

A~o)

. . . . . .

A U(o)o - - 1~1 ] l rn

9 Crn.

81

3.6. C E S A R O S E T - S U M S

Then, arrange corresponding terms, by Remark 2, ltx -

yll < 2. u~176+... + 2 ukwk (1 +.-- + T ~ - 1 ) -

m

m

9

wk

1

2000 the bounds (approximate), independent of i, are

44
O, and

otherwise.

Define A = [x+A~jfj]. We now need to show t h a t A E M . To show t h a t P < A < Q, note t h a t

?tij =- x + A i j f j < x + Q i j f j -= X+(lijfi =- Klij(x+ fi) = (tij.

86

C H A P T E R 3. C O N V E R G E N C E OF M A R K O V S E T - C H A I N S

Similarly, Pij _< gij. To show that A is stochastic, observe that

k----1

k----1

k=l

Hence, A E M. Finally, 9 -A=

~2kakj =

Xkfk)(X Aks

=

xkfkX

k----1

-=

Hence, z M C_ ~ M . From this it follows that ~M = x M .

[]

A consequence of this theorem follows. C o r o l l a r y 3.8. For k = 0, 1,... , Sk+l = S k M . And from this corollary, we have the desired result. C o r o l l a r y 3.9. If lim Sk = Soo exists, then so does lim Sk. And lim Sk = k -~ ~

k --~ o o

k --* o o

-Soo. Proof. Note that S k F = -Sk by (3.8). Thus, lim Sk = lim S k F = SooF = k---~c~

k---~oo

500.

[]

An example demonstrating lumping follows. E x a m p l e 3.10. Let M be an interval where p =

.5 :4 .4

.2 .2 .3

.2 .2 .1

Q =

.6 .6 .6

.3 .3 .2

.2 ] .3 . .4

Then, using the partitioning,

Looking at M and computing bounds on lim Sk = S ~ yields that if (Yl, 92) E k---~oo

Soo then .444444444 _< Yl -< .6

and

.4 _< 92 _< .555555556.

Thus, for any (yl,Y2,Y3) E Soo, .444444444_ Yl _< .6

and

.4 < Y2 + Y3 _< .555555556.

3.8. LUMPING IN M A R K O V SET-CHAINS

87

Computing bounds on Soo yields that if (yl,y2,y3) E Soo, .444444444 < yl _ l , (2) Q~. is n.i • ni and irreducible for all k = 1 , . . . ,r, and (3) if t > r, Qt,k # 0 for some k = 1 , . . . t - 1. In tile remaining work we will assume t h a t P, Q and all matrices in [P, Q] have undergone the same simultaneous row and column p e r m u t a t i o n s and t h a t each of these matrices has been partitioned, with corresponding notation, as the form (4.1). In addition, if Q is a submatrix of Q we use the n o t a t i o n A to denote the corresponding submatrix in A, for any n x n matrix A. Using this notation, define

[p, Q] = {,4: A 9 [P, 0]}. It is easily seen t h a t [P, Q] is a c o m p a c t convex set. T h e classification of states follows. D e f i n i t i o n 4.1. Let M be an interval [P,Q] for a Markov set-chain. Let Ci be the set of states corresponding to Qi. We classify the states of the Markov set-chain as follows.

(i) If i < r then Ci is called a closed. If lim [Pi,Qi] k exists and consists k--+ o o

only of positive rank 1 matrices, then Ci, and each of its states, is called ergodic. If Ci is ergodic and a singleton, then Ci, and its state, are called absorbing. (ii) If i > r then Ci is called open. Further, if l i m [Pi, Qi]k = {0}, then C,, and each of its states, is called transient. There are some rather simple sufficient conditions which can be used to determine if a class is ergodic or transient. Theorem

4.1. Let M be an interval [P, Q] for a Markov set-chain.

(i) For i r, Ci is transient if lim Q/k = 0. k -4 oo

Proof. For the proof of (i), since Pi is primitive, there is a positive integer s such t h a t P [ has positive entries. Thus, P~ p

for all i,j and A 1 , . . . , A k + , 9 [Pi,Qi]. Thus, every matrix in [Pi,Qi] ~176 has all its entries at least, as large as p and thus, all matrices in [Pi, Qi] ~176 are positive. P r o p e r t y (ii) is obvious. []

4.1. CLASSIFICATION AND CONVERGENCE

93

The classification of states can now be used to describe the fundamental types of Markov set-chains. D e f i n i t i o n 4.2. Let M be an interval [P,Q] for a Markov set-chain. Markov set-chain is

The

(i) ergodic if it has only one class and that class is ergodic. (ii) regular if it has only one closed class, and that class is ergodic, while all open classes are transient. (iii) absorbing it each of its closed classes is a singleton and each of its open classes is transient. Of course, if the Markov set-chain is ergodic then lim M k = M ~ k---+oo

We now

show that convergence also holds for regular and absorbing Markov set-chains. The results require a preliminary theorem. T h e o r e m 4.2. Suppose a Markov set-chain, with ]lI an interval [P, Q], has all of its open classes transient. Let [P,Q] be the submatrix of [P,Q] that corresponds precisely to those transient classes. Then lira [P, ~]k = {0}. k ---+o o

Proof. The proof is by induction on the number of transient classes of the Markov set-chain. If there is only one transient class the result follows immediately from the definition of transient class. Now, suppose the theorem holds for m transient classes. If the Markov setchain has m + 1 transient classes, any matrix B E [P, Q] can be partitioned as is Q and then repartitioned Br+l Br+2,~+l

0 Br+2

0 0 =

.............................

Bs,r+l

[~1

Bs,r+2

0 ]

B21

B2

999 Bs

where s = r + m + 1 and B2 = Bs. Compatibly, repartition ~=

[

el P21

0 P2

]

and

0]

Q=

Q2 1

"

Then, since the class belonging to [P2, Q2] is a transient class, lim [P2,Q2] k = k---~oo

{0}. And, by the induction hypothesis, lira [P1,Q1] k = {0}. Thus, given any k--+oo

positive constant e, there is a positive integer N such that if k > N then

d({O},[P1,Q1] k) < e / 3

and

d({0},[P2,Q2] k) < e / 3 .

o ] and T2 = [D1 Now let T1 = [ ~U2~U2 ~ 2 ~ 2o ] be in [fi,~]k, partitioned as is B. Then IIClll

< e/3,

[[C2][ < e/3,

[]Dl[[ < e/3,

and

[]D2[] < e/3.

C H A P T E R 4. B E H A V I O R I N M A R K O V S E T - C H A I N S

94 Calculating,

T1T2=

IV1 0] [9 1 0] [ C1DI 0 ] C21 C2 921 D2 --C21DI+C2D21 62L)2 "

Since IIC2~II _l is Cauchy. Thus, lim M k exists. [] k--+(x~

C o r o l l a r y 4.3. Using the hypothesis of the theorem and that L =

lim M k, k--~ c ~

there are constants K and/3, 0 u, K

d(M ~, M ~) < K ~ 3 ~ _ ~ + K~3~ + g~3 ~ K2 _ 1_ ~u

_0 be a nonhomogeneous Markov chain with transition matrices in M. Let f be a real valued function defined on { s l , . . . , Sn}. Then, for any given constant ~,

Theorem

Pr

~

i=1 k

,L

>e

--+0 as k --~ c~.

Proof First observe t h a t if a, b E R then (f(a, L) = rain la - II _0

converges. k>0

We can also convert Poisson's WLLN into a set-theoretic format.

C H A P T E R 4. B E H A V I O R I N M A R K O V S E T - C H A I N S

104 Theorem

4.7. Let {Xi}i>o be a sequence o/independent random variables with k

P r { X i = 1} = Pi and Pr{X~ = 0} = qi where Pi + qi = 1. Let X k = E X i / k , i=1 k

Pk = ~ p i / k and suppose a < Pi O, i--=1

Pr{a-e

< Xk < b+c}-+ l

as k--~ oc. Proof. Applying Poisson's WLLN, given any ( > 0, P r { f k -- e 0 be a nonhomogeneous Markov chain with transition matrix An in M at each step k. We partition each An according to the ergodic and transient states, An =

0 ]"

Ark

Qn

Finally, we use that Fo,n = AI 9.. Ak for all positive integers k. The behavior in the Markov set-chain is the movement of the nonhomogeneous Markov chain among the states, as the steps increase. In this section, we describe three kinds of movement: movement within the transient states, movement from the transient states to the ergodic states, and movement within the ergodic states. I

Movement

within

the

transient

states

We assume the nonhomogeneous Markov chain is initially in a transient state si. Let sj be another transient state. We intend to find the expected number of visits the chain makes to sj as the number of steps increase. For this, let

u(n)

if the chain is in sj at step k,

~1

ij = [ 0

otherwise.

Further define t

(i)

_(t) 7~iJ

:

. (k) Z aiJ and k=O O0

( i i ) - ( ILij ~)

:

~ _ ( kl~ij) k=O

which represent the number of visits of the chain to sj in t steps and for all steps, respectively. We compute the expected values as follows. For (i), we have t

k=O t

k=0

n)

4.3. B E H A V I O R I N M A R K O V S E T - C H A I N S

107

where f}o,o) = {10 i f i = j otherwise. For (ii), by Fubini's Theorem (x)

n (~) x--" ,~ (4) l:.,nij = ~.~ E'nij k----O oo

V" :!0 ,4) " ~ J~3 k=0

In matrix

form we have

(or [Eni i ] = (I + QI + Q1Q2 + .. . )ij. Corollary 4.1 can be used to show this sum converges. To compute tight bounds on I + Q1 + Q~Q2 + "'" we compute bounds on the last n - r columns of I + A1 + A1A2 + . " and delete the first r components in these bounds. To compute bounds for column j, begin by choosing a positive integer s. Then the j - t h column of P and the j - t h column of Q bound that of As. We write 11 0. Then X+e={y:

]lx-yH <e

for some

x6X}.

A . 7 . Let X and Y be compact subsets of V. Then ~ ( X , Y ) _, converges to s. (Kemeny,

{Xk} is a sequence of nonnegative random variables, E

Xk

E EXk.

k=0

k:0

(Kemeny, Snell, and Knapp, p. 53 or Loeve, p. 125) C h e b y s h e v ' s I n e q u a l i t y . Let, X be a random variable having mean # and let t be a positive number. Then

Pr{IX

-

1 Var(X)2. #l >_t} ~_ -~

(Clark and Disney, p. 147) P o i s s o n W L L N . If in a sequence of independent trials, the probability of occurrence of an event A in the k-th trial is equal to Pk, then lim n--+ c~

Pr{l# ?2

Pl+'"+Pnt

<e} = 1

n

where, as usual, it denotes the number of occurrences of A in the first n trials. (Gnedenko, p. 201)

Bibliography Alefeld, G., Herzberger, J. (1983): Introduction to interval computations. Academic Press, New York Alefeld, G., Kreinovich, V., Mayer, G. (1997): On the shape of the symmetric, persymmetric, and skew-symmetric solution set. SIAM J. Matrix Anal. Appl. 18, 693-705 Altham, P.M.E. (1970): The measurement of association of rows and columns for the rxs contingency table. J. Roy. Statist. Soc. Set B 32, 63-73 Anthonisse, Jac.M., Tijms, H. (1977): Exponential convergence of products of stochastic matrices. J. Math. Anal. Appl. 59, 360-364 Artzrouni, M. (1983): A theorem on products of matrices. Linear Algebra Appl. 49, 153-159 Artzrouni, M. (1986): On the convergence of infinite products of matrices. Linear Algebra Appl. 74, 11-21 Artzrouni, M. (1991): On the growth of infinite products of slowly varying primitive matrices. Linear Algebra Appl. 145, 33-57 Artzrouni, M., Li, X. (1995): A note on the coefficient of ergodicity of a columnallowable nonnegative matrix. Linear Algebra Appl. 214, 93-101 Barmish, B.R., Hollot, C.V. (1984): Counter-examples to a recent result on the stability interval matrices by S. Bialas. Int. J. Control 39, 1103 1104 Barnsley, M. (1988): Fractals everywhere. Academic Press, New York Bauer, F.L., Deutsch, E., Stoer, J. (1969): Absch/izungen ffir die Eigenwerte positiver linearer operatoren. Linear Algebra Appl. 2, 275-301 Ben-Haim, Y., Elishakoff, I. (1990): Convex models of uncertainty in applied mathematics, Elsevier, New York Berge, C. (1963): Topological spaces. Oliver and Boyd, Edinburgh Berman, A., Plemmons, R. (1979): Nonnegative matrices in the mathematical sciences. Academic Press, New York Bialas, S. (1983): A necessary and sufficient condition for the stability of interval matrices. Int. J. Control 37, 717-722 Birkhoff, G. (1967): Lattice theory, 3rd edn. American Mathematical Society Colloquium Publications XXV, Providence, Rhode Island Bone, T., Jeffries, C, Klee, V. (1988): A qualitative analysis of x ~ = A x + b. Disc. Appl. Math. 20, 9-30

124

BIBLIOGRAPHY

Bowerman, B., David, H.T., Isaacson, D. (1977): The convergence of Cesaro averages for certain non-stationary Markov chains. Stoch. Proc. Appl. 5, 221-230 Brualdi, R.A., Shader, B.L. (1995): Matrices of sign-solvable linear systems. Cambridge University Press, Cambridge Bushell, P. (1973): Hilberts metric and positive contraction in Banach space. Arch. Rational Mech. Anal. 52, 330-338 Carmeli, M. (1983): Statistical theory and random matrices. M. Dekker, New York Chatterjee S., Seneta, E. (1977): Towards consensus: some convergence theorems on repeated averaging. J. Appl. Prob. 14, 89-97 Christian, Jr., F.L. (1979): A K-measure of irreducibility and doubly stochastic matrices. Linear Algebra Appl. 24, 225-237 Clarke, B.L. (1975): Theorems on chemical network stability. J. Chem. Phys. 62, 773-775 Clarke, B.L., Disney, R.L. (1970): Probability and random processes for engineers and scientists. John Wiley, New York Cohen, J.E. (1979): Contractive inhomogeneous products of non-negative matrices. Math. Proc. Camb. Phil. Soc. 86, 351-364 Cohen, J.E., Sellers, P.H. (1982): Sets of nonnegative matrices with positive inhomogeneous products. Linear Algebra Appl. 47, 185-192 Cohen, J.E., Kesten, H., Newman, C.M. (Editors) (1986): Random matrices and their applications. Contemporary Mathematics 50, Amer. Math. Soc., Providence Courtois, P.J., Semal, P. (1984): Error bounds for analysis by decomposition of nonnegative matrices. Mathematical Computer Performance and Reliability. Elsevier Science Publishers, North Holland, New York, 209-224 Crisanti, A., Paladin, G., Vulpiani, A. (1993): Products of random matrices in statistical physics. Springer-Verlag, Berlin-Heidelberg-New York Cull, P., Vogt, A. (1973): Mathematical analysis of the asymptotic behavior of the Leslie population matrix model. Bull. Math. Bio. 35, 645-661 Datta, B.N. (1995): Numerical linear algebra and applications. Brooks/Cole Publishing Company, Pacific Grove, California Daubechies, I., Langarias, J.C. (1992): Sets of matrices all infinite products of which converge. Linear Algebra Appl. 161,227-263 Deutsch, E., Zenger, C. (1971): Inclusion domains for the eigenvalues of stochastic matrices. Numer. Math. 18, 182-192 Diamond, P., Kloeden, P., Pokrouskii, A. (1994): Interval stochastic matrices and simulation of chaotic dynamics. Contemporary Mathematics 172, Amer. Math. Soc., Providence, 203-215 Diamond, P., Kloeden, P., Pokrouskii, A. (1995): Interval stochastic matrices, a combinatorial lemma and the computation of invarient measures of dynamicals systems. J. Dynam. Diff. Eq. 7, 341-364.

MARKOV-SET CHAINS

125

Dobrushin, R.L. (1956): Central limit theorem for non-stationary Markov chains, I, II. Theor. Prob. Appl. 1, 63-80, 329-383 Doeblin, W. (1938): Expos@ dela th~orie das chaines simples constantes de Markoff s un hombre fini d'etates. Reune Math@matiqde (Union Interbolk amique) 2, 77-105 Eggleston, H.G. (1969): Convexity. Cambridge University Press, Cambridge Elsner, L. Koltracht, I., Neuman, M. (1990): On the convergence of asynchromous paracontractions with applications to tomographic reconstructions from incomplete data. Linear Algebra Appl. 130, 65-82 Eschenbach, C.A. (1993): Sign patterns that require exactly one real eigenvalue and patterns that require n - 1 non real eigenvalues. Linear and Multilinear Algebra 35, 213-223 Eschenbach, C.A., Johnson, C.R. (1991): Sign patterns that require real, non real, and pure imaginary eigenvalues. Linear and Multilinear Algebra 29, 299-311 Eschenbach, C.A., Johnson C.R. (1993): Sign patterns that require repeated eigenvalues. Linear Algebra Applic. 190, 169-179 Faedo, S. (1953): Un nuovo problema di stabilit/~ per le equazioni algebriche a coefficienti reali. Ann. Scuola Norm. Super. Piza 3, 53-63 Fenner, T.I., Loizou, G. (1971): On fully indecomposable matrices, J. of Computer and System Sciences 5,607-622 Fiedler, M. (1972): Bounds for eigenvalues of doubly stochastic matrices. Linear Algebra Appl. 5, 299-310 Fiedler, M. (1995): An estimate for the nonstochastic eigenvalues of doubly stochastic matrices. Linear Algebra Appl. 214, 133-143 Franklin, J. (1968): Matrix theory. Prentice-Hall, Englewood Cliffs, New Jersey Freese, R., Johnson, C.R. (1974): Comments on the discrete matrix model of population dynamics. J. of Research of the National Bureau of Standards B. Mathematical Sciences 788, No. 2, 73-78 Friedland, S., Schneider, H. (1980): The growth of powers of a nonnegative matrix. SIAM J. Alg. Disc. Meth. 1, 185-200 Funderlic, R.E., Meyer, C.D. (1985): Sensitivity of the stationary distribution vector of an ergodic Markov chain. ORNL-6098, Oak Ridge National Laboratory, Oak Ridge, Tennessee Funderlic, R.E., Meyer, C.D. (1986): Sensitivity of the stationary distribution vector of an ergodic Markov chain. Linear Algebra Appl. 76, 1-17 Gantmacher, F.F. (1964): The theory of matrices. (vol. 1 and vol. 2). Chelsea Publishing Co., New York Garloff, J., Schwierz, K-P. (1980): A bibliography on interval mathematics II. J. Comput. Appl. Math. 6, 67-79 Geramita, J.M., Pullman, N.J. (1984): Classifying the asymptotic behavior of some linear models. Math. Bio. 69, 189-198. Glass, D.V., Hall, J.R. (1954): Social mobility in Great Britain: A study of intergeneration changes in status. Social Mobility in Great Britain. Routledge

126

BIBLIOGRAPHY

and Kegan Paul, London Gnedenko, B.V. (1982): The theory of probability. (translated by George Yankovsky), Mir Publishers, Moscow Golubitsky, M., Keeler, E.B., Rothschild, M. (1975): Convergence of age structure: applications of the projective metric. Theor. Pop. Bio. 7, 84-93 Gorman, W.M. (1964): More scope for qualitative economics. R. Econ. Studies 31, 65-68 Gregory, D.A., Kirktand, S.J., Pullman, N.J. (1992): Row stochastic matrices with common left fixed-vector. Linear Algebra Appl. 169, 131-149 Hajnal, J. (1976): On products of nonnegative matrices. Math. Proc. Cambridge Phil. Soc. 79, 521-530 Harary, F., Lipstein, B., Styan, G. (1979): A matrix approach to nonstationary chains. Oper. Res. 18, 1168-1181 Hartfiel, D.J. (1973): Bounds on eigenvalues and eigenvectors of a nonnegative matrix which involve a measure of irreducibility, SIAM J. Appl. Math. 24, 83-85 Hartfiel, D.J. (1974a): On infinite products of nonnegative matrices. SIAM J. Appl. Math. 26,297-301 Hartfiel, D.J. (1974b): A study of convex sets of stochastic matrices induced by probability vectors. Pacific J. of Math. 52,405-418 Hartfiel, D.J. (1975a): Results on measures of irreducibility and full indecomposibility. Trans. Amer. Math. Soc. 202, 357-368 Hartfiel, D.J. (1975b): Two theorems generalizing the mean transition probability results in the theory of Markov chains. Linear Algebra Appl. 11, 181-187 Hartfiel, D.J. (1980): Concerning the solution set to A x = b where P