This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
0.
Show that the random variable
Y given by
a a p(a) =
X
if l X I
0, Y > 0, 0) = 81 4rr1 Z
>
- + - {sin -
l
PI + sin - l P2 + sin - l P3 }·
Y, Z have the standard trivariate normal density of Exercise (8), with PI JE(Z I X, Y) var(Z I X, Y)
= { (P3 - PIP2 )X = {I - Pt - i p
(P2 - PIP3 )Y} /(1 - P t ), - pj + 2 PI P2 P3 }/(I - p h ·
35
+
=
p (X,
Y). Show
[4.10.1]-[4.11.7]
Continuous random variables
Exercises
4.10 Exercises. Distributions arising from the normal distribution 1. Let Xl and X2 be independent variables with the Show that X l + X2 has the X 2 + distribution.
(m n) t (r) s
x 2 (m) x 2 (n) and
distributions respectively.
2.
Show that the mean of the distribution is 0, and that the mean of the - 2) if > 2. What happens if ::::: 2?
3.
Show that the
s /(s
s
YF
t(1)
F (r, s)
distribution is
distribution and the Cauchy distribution are the same.
4. Let X and be independent variables having the exponential distribution with parameter that X/ Y has an distribution. Which?
1.
Show
5. Use the result of Exercise (4.5.7) to show the independence of the sample mean and sample variance of an independent sample from the (7 2 ) distribution.
N(/L , N(O, 1) Jr, B
1 r / B(�, � n - �),
E Jr]
6. Let {Xr : ::::: ::::: n } be independent variables. Let \11 [0, be the angle n between the vector (X l , X2 , . . . , Xn ) and some fixed vector in ]R . Show that \11 has density n ° ::::: 1/f < where is the beta function. f (1/f ) (sin 1/f) - 2
=
4.11 Exercises. Sampling from a distribution
U
1.
Uniform distribution. If I?
2.
Random permutation. Given the first
LnUJ +
is uniformly distributed on [0, 1 ] , what is the distribution of X
n
=
integers in any sequence So, proceed thus:
(a) pick any position Po from { I , 2, . . . , n } at random, and swap the integer in that place of So with the integer in the nth place of So, yielding S l .
,n
(b) pick any position PI from { I , 2, . . . - 1 } at random, and swap the integer in that place of S l with the integer in the - l ) th place of S 1 . yielding S2 ,
(n
(r
(c) at the - l)th stage the integer in position Pr - 1 , chosen randomly from { I , 2, . . . i s swapped with the integer at the + l)th place o f the sequence Sr - 1 . Show that Sn - 1 is equally likely to be any of the permutations of { I , 2, . . . , } .
(n - r
n!
n
,n-r+
ro
I},
, t)
Gamma distribution. Use the rejection method to sample from the gamma density where may not be assumed integral. [Hint: You might want to start with an exponential random variable with parameter
3.
..
t (:::: 1)
l/t.]
:::: 1. x = 6x(1 - x), x ::::: 1. probability vector n
4. Beta distribution. Show how to sample from the beta density f3 (a, (3) where a, f3 Use Exercise
(3).]
5.
Describe three distinct methods of sampling from the density f ( )
6. Aliasing method. A finite real vector is called a with sum Show that a probability vector p of length
1.
n
P = n -1 1 L --
r= l
[Hint:
0 :::::
if it has non-negative entries may be written in the form
Vr ,
where each Vr is a probability vector with at most two non-zero entries. Describe a method, based on this observation, for sampling from p viewed as a probability mass function. 7.
let
Box-Muller normals. Let
Tj
= Uj - 1. 2
UI U2 and
be independent and uniformly distributed on [0, 1 ] , and
Show that, conditional on the event that R X
= �V
-2 10g R 2 ,
36
= VTl + Ti 1,
Y= �V
:::::
-2 10g R 2 ,
Coupling and Poisson approximation are independent standard normal random variables. 8. Let U be uniform on distribution.
[0, 1] and 0
1
{ (X
r,c: exp -
u ", 2rr
- J.t) 2 2u 2
O.
}
'
-00
O.
Show that IP'(X >
39
x+
a/x I X >
x
> 0.
x) -+ e -a x -+ O. as
[4.14.2]-[4.14.11]
Let
2.
X
Exercises
be continuous with density function
(a) What are the possible values of a and f3 ?
Continuous random variables f(x) C(x - x 2), x C O. =
where a
(b) What i s C ? Let
3.
X
b e a random variable which takes non-negative values only. Show that 00
� )i - 1 ) IAj
::; X
i -I where Ai =
{i - 1 ::; X i}.
+ > = > for This is the 'lack of memory ' property again. Show that the exponential distribution is the only continuous distribution with this property. You may need to use the fact that the only non-negative monotonic solutions of the functional equation + = for ::: with = are of the form = Can you prove this? Let
5.
6.
f
g (s) elLs . X f(x, y) g(x)h(y) x y X f(x, y) 2e -x -y 0 x y
Show that and Y are independent continuous variables if and only if their joint density function = factorizes as the product of functions of the single variables and alone.
7. Let and Y have joint density function = , < independent? Find their marginal density functions and their covariance.
42
Problems
Exercises
[4.14.28]-(4.14.34]
to deduce the Cauchy-Schwarz inequality JE(Xy) 2 � JE(X2 )JE( y2 ). (b) Minkowski's inequality. Show that, if p � 1, then Set p
=
q
=
2
{JE(IX YIP) } I /p � {JEIXP I } I /p {JEl yP I } I /p .
+
+
Note that in both cases your proof need not depend on the continuity of X and Y; deduce that the same inequalities hold for discrete variables. 28. Let Z be a random variable. Choose X and Y appropriately in the Cauchy-Schwarz (or Holder) inequality to show that g (p) 10g JEIZP I is a convex function of p on the interval of values of p such that JEIZP I < 00 . Deduce Lyapunov's inequality: {JEIZr I} I / r � (JEIZs I} I /s whenever r � s 0. You have shown in particular that, if Z has finite rth moment, then Z has finite sth moment for all positive s � r. 29. Show that, using the obvious notation, JE{JE(X I Y, Z) I Y} JE(X I Y). 30. Motor cars of unit length park randomly in a street in such a way that the centre of each car, in turn , is positioned uniformly at random in the space available to it. Let m (x) be the expected number of cars which are able to park in a street of length x. Show that m(x + 1) x� Jor { m(y) +m(x - y) + l } dy . =
>
=
=
It is possible to deduce that m (x) is about as big as i x when x is large. 31. ButTon's needle revisited: ButTon's noodle.
(a) A plane is ruled by the lines y = nd (n 0, ± 1, . . . ) . A needle with length L « d) is cast randomly onto the plane. Show that the probability that the needle intersects a line is 2L/(rrd). (b) Now fix the needle and let C be a circle diameter d centred at the midpoint of the needle. Let A be a line whose direction and distance from the centre of C are independent and uniformly distributed on [0, 27r] and [0, �d] respectively. This is equivalent to 'casting the ruled plane at random'. Show that the probability of an intersection between the needle and A is 2L / (rr d). (c) Let S be a curve within C having finite length L(S). Use indicators to show that the expected number of intersections between S and A is 2L (S)/(rrd). This type of result is used in stereology, which seeks knowledge of the contents of a cell by studying its cross sections. 32. ButTon's needle ingested. In the excitement of calculating rr, Mr Buffon (no relation) inadver tently swallows the needle and is X-rayed. If the needle exhibits no preference for direction in the gut, what is the distribution of the length of its image on the X-ray plate? If he swallowed Buffon's cross (see Exercise (4.5 . 3)) also, what would be the joint distribution of the lengths of the images of the two arms of the cross? 33. Let X l , X2 , " " Xn be independent exponential variables with parameter A, and let X (1) � X (2) � . . . � X (n) be their order statistics. Show that =
YI
=
nX ( 1) , Yr
=
(
n + 1 - r)(X (r) - X (r - l ) ),
also independent and have the same joint distribution as the Xi . 34. Let X (1) , X (2) , . . . , X (n ) be the order statistics of a family of independent variables with common continuous distribution function F. Show that 1 � r < n, are
43
Continuous random variables are independent and uniformly distributed on [0, 1]. This is equivalent to Problem (4.14. 3 3). Why? [4.14.35]-[4.14.42]
Exercises
You are permitted to inspect the n prizes at a rete in a given order, at each stage either rejecting or accepting the prize under consideration. There is no recall, in the sense that no rejected prize may be accepted later. It may be assumed that, given complete information, the prizes may be ranked in a strict order of preference, and that the order of presentation is independent of this ranking. Find the strategy which maximizes the probability of accepting the best prize, and describe its behaviour when n is large. 36. Fisher's spherical distribution. Let R 2 = X 2 + y 2 + Z 2 where X, Y, Z are independent normal random variables with means A , /1- , v , and common variance 0' 2 , where (A , /1-, v) '" (0, 0,0). Show that the conditional density of the point (X, Y, Z) given R = r, when expressed in spherical polar coordinates relative to an axis in the direction e = (A , /1- , v) , is of the form 35. Secretary/marriage problem.
f(O, l/J) = 4rr s�mh a ea cos (} sinO,
0 ::::
° < rr,
0 :::: l/J
< 2rr
,
where a = r i e l . 37. Let l/J be the N(O, 1) density function, and define the functions Hn, n � 0, by HO = 1, and ( _ l)n Hnl/J = l/J (n) , the nth derivative of l/J. Show that: n (a) Hn (x) is a polynomial of degree n having leading term x , and O ifm ", n, H (x)Hn(x)l/J(x)dx = . m 00 n . tfm = n. 1-
{
00
,
Let X and Y have a standard bivariate normal distribution with zero means, unit variances, and correlation coefficient p, and suppose U = u (X) and V = v(Y) have finite variances. Show that I p (U, V ) I :::: Ipl. [Hint: Use Problem (4.14.37) to expand the functions u and v. You may assume that u and v lie in the linear span of the Hn.] 39. Let X ( 1 ) , X (2) , . . . , X (n ) be the order statistics of n independent random variables, uniform on [0, 1]. Show that: (a) lE(X (r» ) = n +r l ' (b) cov(X(r) , X(s» ) = (nr(n+ 1)-2s(n++1)2) for r :::: s. 40. (a) Let X, Y, Z be independent N(O, 1) variables, and set R = y'X 2 + y 2 + Z2 . Show that X2 / R2 has a beta distribution with parameters ! and 1, and is independent of R2 . (b) Let X, Y, Z be independent and2 uniform on [-1, 1] and set R = y'X2 + y2 + Z2 . Find the 2 2 density of X / R given that R =::: 1. 41. Let l/J and be the standard normal density and distribution functions. Show that: (a) (x ) = 1 - ( -x), (b) f(x) = 2l/J (X) (AX) , -00 < x < 00 , is the density function of some random variable (denoted by Y), and that IYI has density function 2ifJ. (c) Let X be a standard normal random variable independent of Y, and define Z = (X+A I Y I ) / y' 1 + A2 . Write down the joint density of Z and IYI , and deduce that Z has density function f. 42. The six coordinates (Xi , Yi), 1 =::: i =::: 3, of three points A, B, C in the plane are independent N(O, 1). Show that the the probability that C lies inside the circle with diameter AB is 1 . 38. Lancaster's theorem.
44
s
Problem
Exercises
[4.14.43]-[4.14.49]
N(O.
1 ) . Show 3. of three points A. B. C are independent 3 that the probability that C lies inside the sphere with diameter AB is ! . 3 44. Skewness. Let X have variance u 2 and write m k = lE(X k ) . Define the of X by skw(X) = lE[(X - m l ) 3 ]/u 3 . Show that: (a) skw(X) = (m 3 - 3m l m 2 + 2mV/u 3 . (b) skw(Sn ) = skw(X I ) / ...;n . where Sn = l:�=1 Xr is a sum of independent identically distributed 43. The coordinates (Xi . Yi . Zi ) . 1 ::: i :::
.../3 4 7r
random variables.
p .jnpq.
(c) skw(X) = (1 - 2 ) /
when X i s bin(n .
p)
where
(d) skw(X) = 1 /./I. when X i s Poisson with parameter (e) skw(X) = 2/.fi. when X is gamma 45. Kurtosis. Let X have variance lE[(X - m l )4]/u4. Show that:
(c) (d)
t). t
A.
= 1.
and is integral.
u 2 and lE(Xk )
=
mk .
Define the
N(/L .
3. when X is ( 2 ). kur(X) = 9. when X is exponential with parameter A. I kur(X) = 3 + A - . when X is Poisson with parameter A . kur(Sn ) = 3 + {kur(X I ) - 3}/n. where Sn = l:�=1 Xr
(a) kur(X) =
(b)
r(A .
p+q
distributed random variables.
kurtosis
of X by kur(X) =
is a sum of independent identically
46. Extreme value. Fisher-Gumbel-Tippett distribution. Let X r. 1 ::: exponentially distributed with parameter 1 . Show that X (n ) = max {Xr : 1
Hence show that
skewness
r::::::r n.
be independent and ::: n} satisfies
lim IP'(X(n} - log n ::: x) = exp(-e -X ) . n� oo
Iooo { 1 - exp( _e -X ) } dx = y where y i s Euler' s constant.
47. Squeezing. Let S and X have density functions satisfying b(x) ::: fs (x ) ::: a (x ) and fs (x ) ::: fx (x) . Let U be uniformly distributed on [0. 1 ] and independent of X. Given the value X. we implement the following algorithm:
if Ufx (X) > a (X).
reject X ;
otherwise: i f Ufx (X) < b (X) .
accept X ;
otherwise: i f Ufx (X) ::: fs (X) .
accept X ;
otherwise: reject X. Show that. conditional on ultimate acceptance. X is distributed as S . Explain when you might use this method of sampling. 48. Let X. Y. and {Ur
:
r
2: I } be independent random variables. where:
IP'(X = x ) = (e - l)e -x • IP'(Y = y ) = and the Ur are uniform on [0. exponentially distributed.
1].
Let
M
1 for x . y = I , 2 • . . . • (e - 1 ) y !
= max{U l . U2 • . . . • Uy } . and show that Z = X -
M is
I 49. Let U and V be independent and uniform on [0. 1 ] . Set X = -a - log U and Y = - log V where a >
O. O.
(a) Show that. conditional on the event Y 2: for x >
1
! (X _ a) 2 . X has density function f (x ) = ../2l1ie - Zx 45
2
[4.14.50]-[4.14.56]
Continuous random variables
Exercises
f, X
Y i(X
0,
(b) In sampling from the density function it is decided to use a rejection method: for given a > we sample and V repeatedly, and we accept the first time that :::: a) 2 . What is the optimal value of a ? 1 ) distribution. (c) Describe how to use these facts i n sampling from the
U
50. Let S be a semicircle of unit radius on a diameter
D. X
D.
N (O ,
-
(a) A point P is picked at random on If is the distance from P to S along the perpendicular to show = rr /4. (b) point Q is picked at random on S . If is the perpendicular distance from Q to show JE(Y) = 2/rr .
D,
JE(X)
A
Y
D,
51. (Set for the Fellowship examination of St John's College, Cambridge in 1 858.)
'A
large quantity of pebbles lies scattered uniformly over a circular field; compare the labour of collecting them one by one: (i) at the centre 0 of the field, (ii) at a point on the circumference.' To be precise, if Lo and LA are the respective labours per stone, show that JE(Lo) = �a and JE(L A ) = 32a / (9rr ) for some constant a. (iii) Suppose you take each pebble to the nearer of two points or at the ends of a diameter. Show in this case that the labour per stone satisfies
A
A B
JE(L AB )
=
4a { "3 16
3rr
17 1 - (; -12 + 2 10g(1 + -12)
} :::
1.13
x
2 3 a.
(iv) Finally suppose you take each pebble to the nearest vertex of an equilateral triangle ABC inscribed in the circle. Why is it obvious that the labour per stone now satisfies JE(L ABc ) < JE(Lo)? Enthusiasts are invited to calculate JE(L ABc ) . 52. The lines L , M, and are parallel, and P lies on L . line picked at random through P meets M at Q. A line picked at random through Q meets at R. What is the density function of the angle e that RP makes with L? [Hint: Recall Exercise (4.8.2) and Problem (4. 14.4).]
N
N
A
53. Let !:l. denote the event that you can form a triangle with
three given parts of a rod R. (a) R is broken at two points chosen independently and uniformly. Show that lP'(!:l.) = ! . (b) R is broken in two uniformly at random, the longer part is broken in two uniformly at random. Show that lP'(!:l.) = log(4/e) . (c) R is broken in two uniformly at random, a randomly chosen part is broken into two equal parts. Show that lP'( !:l.) = ! .
(d) In case (c) show that, given !:l., the triangle is obtuse with probability 3 - 2../2.
54. You break a rod at random into two pieces. Let R be the ratio of the lengths of the shorter to the
longer piece. Find the density function
fR ,
together with the mean and variance of R.
55. Let R be the distance between two points picked at random inside a square of side a. Show that
JE(R 2 )
=
l a2 , and that R 2 /a 2 has density function
f(r) { r =
- 4.vr + rr
4 .Jr=1 - 2 - + 2 sin - 1 � - 2 sin - 1
r
56. Show that a sheet of paper of area
such a way that at least
if
J1 - r 1
O
�
if 1 �
r r
� 1, � 2.
A cm2 can be placed on the square lattice with period 1 cm in
rA 1 points are covered.
46
Problems
Exercises
[4.14.57]-[4.14.63]
57. Show that it is possible to position a convex rock of surface area S in sunlight in such a way that its shadow has area at least ! S . 58. Dirichlet distribution. Let
{Xr
:
1 ::; r ::; k I} be independent r (>.. , f3r) random variables 2 ::; r ::; k 1, are independent random variables. 1 ::; r ::; k, have the joint Dirichlet density +
(respectively). (a) Show that Yr = Xr /(X 1 + . . . + Xr ), (b) Show that Zr = Xr /(X 1 + . . . + Xk+1 ),
+
Xr = (X 1 r , X2r , . . . , Xm r ), be independent multivariate normal random vectors having zero means and the same covariance matrix V = (Vij ). Show that the two random variables
1 ::; r ::; n,
59. HoteUing's theorem. Let
Tij
=
n- 1
L Xi r Xjr ,
r=l
are identically distributed. 60. Choose P, Q, and R independently at random in the square S (a) of side a. Show that JEIPQRI four points picked at random in a parallelogram form a convex quadrilateral lla2 /144. Deduce that with probability ( � ) 2 . Choose P, Q, and R uniformly at random within the convex region C illustrated beneath. By considering the event that four randomly chosen points form a triangle, or otherwise, show that the =
61.
mean area of the shaded region is three times the mean area of the triangle PQR.
n n matrix, and Cholesky decomposition of Let (X , X , . . . , X ) be a vector of independent random variables distributed as N (O, 1). Show that
62. Multivariate normal sampling. Let V be a positive-definite symmetric a lower-triangular matrix such that V = L'L; this is called the
X=
l 2
the vector Z matrix V.
n
= IL + XL
x
L
V.
has the multivariate normal distribution with mean vector IL and covariance
63. Verifying matrix multiplications. We need to decide whether or not AB = C where A, B, C are given x matrices, and we adopt the following random algorithm. Let x be a random -valued vector, each of the possibilities being equally likely. If (AB - C)X = 0, we decide that AB = C, and otherwise we decide that AB '# C. Show that
n n
to, l}n
2n
lP' (the decision is correct)
{ 11 =
� 2
if AB = c , if AB ,# C.
Describe a similar procedure which results in an error probability which may be made as small as desired.
47
5 Generating functions and their applications
5.1 Exercises.
Generating functions
1. Find the generating functions of the following mass functions, and state where they converge. Hence calculate their means and variances. (a) � o.
f(m ) = (n+;::- I )pn (1 _ p)m ,form -1 f(f(mm )) == p{m((1 -mp)p+ 1)}lm0l 1(1,p+ p),m1. � 1.m = ... , -1, 0, 1,
(b) (c) The constant
for
for
satisfies
for the O} is
'tail'
I
Show that
G x(s) =
a2 G x y (s, t) . as at ' I 8=t= 1
-
Find the joint generating functions of the following joint mass functions, and state for what values of the variables the series converge. (a) k) for � k � where < < < (b) k) for k � o.
4.
(c)
k-i - 1 , 0 j, 0 a 1, a p. f(j, = (1 -a)(-(p 2-a)a i p + k 1 f(j, = (e - l+)ek _ ) kki-Ij! , j, f(j, ) = (Y) pj (l p) i / [ p) )], j � 1, 0 p 1 . n p H T G H, T (X, y) = {px + (1 - p)y}n . (3.5. 1 ). X (0, 1). X 1,2, ... ,n}. U), U G(X, y,z, w) = � (xyzw +xy + yz + z w +ZX + yw +xz + 1)
k k log{ I / ( I for O � � k, k Deduce the marginal probability generating functions and the covariances.
where
= o.
=
O. Show that lE(Tb Tb < 00) = 8.
p
(= 1
I
- q).
bl i p ql ·
5.4 Exercises.
b
b
Branching processes
LetZn
Zo f,L, n -m lE(Z ) lE(ZnZ ) f,L n. m p(Zm , Zn) f,L. Consider a branching process with generation sizes Zn satisfying Zo 1 and IP'( Z I 0) O. Pick two individuals at random (with replacement) from the nth generation and let L be the index of the generation which contains their most recent common ancestor. Show that IP'(L r ) lE( Z I ) lE( Z;� I ) for 0 :::: r < n. What can be said if IP'( Z I 0) O? Consider a branching process whose family sizes have the geometric mass function I (k) qpk , k � 0, where p q 1 , and let Zn be the size of the nth generation. Let T min { n : Zn O} be the extinction time, and suppose that Zo 1 . Find IP'(T n). For what values of p is it the case that lE(T) oo? Let Zn be the size of the nth generation of a branching process, and assume Zo 1. Find an expression for the generating function Gn of Zn, in the cases when Z has generating function given 1. be the size of the nth generation in an ordinary branching process with = 1 , lE(Z I ) = and var(Z I ) > O. Show that = � for m :::: Hence find the correlation coefficient in terms of
=
2.
=
=
=
3.
+
=
=
=
=
=
=
=
4.
I
by: (a) (b)
=
=
G(s) 1 - a(1 - s)fJ, 0 1. G(s) 1 - { P ( f (s))}, where P is a probability generating function, and I is a suitable function satisfying 1 ( 1 ) 1 . (c) Suppose in the latter case that I (x ) x m and pes) s{y -(y -1)s} - 1 where y 1 . Calculate the answer explicitly. =
=
< a, f3
Branching with immigration. Each generation of a branching process (with a single progenitor) is augmented by a random number of immigrants who are indistinguishable from the other members of the population. Suppose that the numbers of immigrants in different generations are independent of each other and of the past history of the branching process, each such number having probability generating function Show that the probability generating function of the size of the nth generation satisfies where is the probability generating function of a l = + typical family of offspring.
5.
GH(s). n (s) Gn(G(s))H(s),
G
Zn
Gn
sZ
s-
6. Let be the size of the nth generation in a branching process with lE( I ) = (2 - ) 1 2 and = 1 . Let Vr be the total number of generations of size r. Show that lE( Vl ) = ! ;rr , and 2 4 lE(2V2 - V3) = ! ;rr - Jo ;rr .
Zo
51
[5.5.1]-[5.6.5]
Generating junctions and their applications
Exercises
5.5 Exercises.
Age-dependent branching processes
1. Let Zn be the size of the nth generation in an age-dependent branching process Z(t), the lifetime distribution of which is exponential with parameter A.. If Z = show that the probability generating function of Z (t) satisfies
(0) 1,
Gt (s)
a t (s) -G at
G(Gt (s» - Gt (s) } . Show in the case of 'exponential binary fission', when G(s) s 2 , that =
A. {
=
(t) at time t. Solve the differential equation of Exercise (1) when A. 1 and G(s) � (1 + s 2 ), to obtain Gt (s) 2s2 ++tt((11 -s)s) . Hence find IP' (Z(t ) :::: k), and deduce that IP' ( Z (t )/ t :::: x I Z (t ) > 0) -+ e - 2x as t -+ 00. and hence derive the probability mass function of the population size Z =
2.
=
=
-
5.6 Exercises. Expectation revisited
convex
a
1. Jensen's inequality. function u : IR -+ IR is called if for all real there exists A., depending on such that u (x ) :::: u ) + A. (x for all x. (Draw a diagram to illustrate this definition.) Show that, if u is convex and X is a random variable with finite mean, then lE(u (X» :::: u (lE(X» .
a,
2.
A
(a
-a)
Let X l , X2 , ' " be random variables satisfying lE ( L� 1 I Xi I )
< 00.
Show that
Let {Xn } be a sequence of random variables satisfying Xn .::: Y a.s. for some Y with lE l Y I Show that
3.
(
lE lim sup Xn n ---+ oo
)
< 00.
:::: lim sup lE(Xn ) . n ---+ oo
that x r lP'(I X I :::: x) -+ 0 as x -+ 00 . Conversely, 0 r-+> 00O. Deduce where r 0, and show that lE I X8 1 < 00 for 0 .::: s < r. Show that lE l X I < 00 i f and only i f th e following holds: for all E > 0, there exists 8 > 0, such that lE(I X I IA ) < E for all such that IP' < Suppose that lE l X r l < 00 where suppose that x r lP'(I X I :::: x) -+ as x
4.
5.
A
::::
(A)
8.
52
Characteristic functions
Exercises
[5.7.1]-[5.7.9]
5.7 Exercises. Characteristic functions
c/>x + y (t) c/>x (t)c/>y (t) - c/>(t)} - c/>(2t)},
t.
Find two dependent random variables X and Y such that = for all ::: !Re{1 and deduce that 2. If is a characteristic function, show that Re{ l 1 :::; 8{1 3. The cumulant generating function of the random variable X is defined by = the logarithm of the moment generating function of X. If the latter is finite in a neigh bourhood of the origin, then has a convergent Taylor expansion: 1.
c/> - I c/> (2t)I
- 1 c/> (t)l } .
10gE(eO x ), kn
Kx «(})
Kx
Kx «(})
the nth cumulant semi-invariant) kl k2 k3
and (X) is called (or of X. (a) Express (X), (X), and (X) in terms of the moments of X. (b) If X and Y independent random variables, show that (X + Y) = (X) + (Y). X = 1, km (X) = for =1= 4. Let X be N(O, 1), and show that the cumulants of X are 5. The random variable X is said to have a if there exist and such that X takes values in the set L(a, = { = ±1, . The of such a variable X is the maximal value of for which there exists such that X takes values in (a) Suppose that X has a lattice distribution with span Show that = 1 , and that < 1 for O < < (b) Suppose that = 1 for some =1= O. Show that X has a lattice distribution with span for some integer Show that -+ as -+ ±oo. 6. Let X be a random variable with density function 7. Let Xt . X2 , . . . , Xn be independent variables, Xi being N(J,Li , 1), and let Y = Xf + X� + . . +X� . Show that the characteristic function of Y is
b I c/>x (t) I 2nk/(}
are
b)
t I c/>x «(})I
kn kn kn k2 ( ) 0 m 2. a +bma : m 0,lattice distribution . . }. spanL(a, b). a b b. I c/>x (2n/b)I 2n/b. () k. f. l c/>x (t)1 0 t 1 c/>y (t) = (1 2it) n/2 exp ( 1 -it()2it ) _
() = J,Lf + J,L� + . . . + J,L� . The random variables Y is said to have the non-central chi-squared distribution with n degrees of freedom and non-centrality parameter (), written x 2 (n; ()). Let X be N(J,L, 1) and let Y be x 2 (n), and suppose that X and Y are independent. The random variable T = X/ '/Y/n is said to have the non-central t-distribution with n degrees of freedom and non-centrality parameter J,L. If and V are independent, being x 2 (m; (}) and V being x 2 (n), then F = / m) / ( V/ n) is said to have the non-central F -distribution with m and n degrees of freedom and non-centrality parameter (), written F(m, n; (}). (a) Show that T 2 is F ( 1 , n; J,L2 ) . (b) Show that + (}) E(F) = n(m m(n - 2) if n > 2. Let X be a random variable with density function f and characteristic function c/>. Show, subject to an appropriate condition on f, that where
8.
(U
U
U
9.
00
00
2 dx = 2n1 I c/> (t)1 2 dt. f(x) 1 1 - 00
- 00
53
[5.7.10]-[5.8.11] 10.
Generatingfunctions and their applications
Exercises
If X and Y are continuous random variables, show that
i: cfJx (y) fy (y) e -ity dy i: cfJy (x t) fx (x) dx . =
-
(a) Let X have distribution function F and let t' be such that M(t') e' x ) < 00 . Show that F,(x) M (t' ) - l J::'oo e'Y dF(y) is a distribution function, called a 'tilted distribution' of X, and find its moment generating function . (b) Suppose X and Y are independent and lEe e' X ), lEe e' y ) < 00. Find the moment generating function of the tilted distribution of X + Y in terms of those of X and Y. 11. Tilted distributions.
lE (
=
=
5 . 8 Exercises. Examples o f characteristic functions
is a characteristic function, show that �, cfJ2 , IcfJ1 2 , Re(cfJ) are characteristic functions . Show that IcfJl is not necessarily a characteristic function. 2. Show that lP'(X ::: ::: inf { }, 1.
If cfJ
Mx
x)
t�O
e -tx MX (t)
where is the moment generating function of X. Let X have the r o.. , distribution and let Y be independent of X with the beta distribution with parameters n and - where and are non-negative integers satisfying ::: Show that Z = X Y has the r 0.. , distribution. 2 2 4. Find the characteristic function of X when X has the N (/-L , (1 ) distribution. variables. Use characteristic functions to find the distri 5. Let Xl , X2 , . . . be independent N (O, bution of: (a) X f , (b) 2:1=1 Xl , (c) XI I X2 , (d) XI X2, (e) XI X2 + X3 X4. 6 . Let Xl , X2 , . . . , Xn be such that, for all aI , 2 , JR, the linear combination alXI + a2 X2 + . . . + anXn has a normal distribution. Show that the joint characteristic function of the Xm is exp( tll,' - � tVt') , for an appropriate vector /L and matrix V. Deduce that the vector (Xl , X2 , . . . , Xn) has a multivariate normal so long as V is invertible . 7. Let X and Y be independent N (O, variables, and let U and V be independent of X and Y. Show that Z = (UX + VY)/ "';U 2 + V 2 has the N (O, distribution. Formulate an extension of this result to cover the case when X and Y have a bivariate normal distribution with zero means, unit variances, and correlation p . 8 . Let X be exponentially distributed with parameter A. Show by elementary integration that lE(e i tX ) = A/(A 9 . Find the characteristic functions of the following density functions: (a) = � for JR, (b) = � for JR. 10. Is it possible for X, Y, and Z to have the same distribution and satisfy X = U(Y + Z), where U is uniform on and Y, Z are independent of U and of one another? (This question arises in modelling energy redistribution among physical particles . ) 11. Find the joint characteristic function of two random variables having a bivariate normal distribution with zero means. (No integration is needed.) 3.
mn) m)n,
m n
n
1)
a . . . , an E
i
density function 1)
1)
it). f(x) e - Ix l I l x E f(x) I x l e - x x E [0, 1],
54
m.
Inversion and continuity theorems
Exercises
[5.9.1]-[5.9.8]
5.9 Exercises. Inversion and continuity theorems
2, 0 ::: 1. lP'(n - 1 Xn ::: Let Xn have distribution function x) Fn (x) = X - sin(2mr 2nrr ' 0 ::: x ::: 1. (a) Show that Fn is indeed a distribution function, and that Xn has a density function. (b) Show that, as n -+ Fn converges to the uniform distribution function, but that the density function of Fn does not converge to the uniform density function.
1. Let Xn be a discrete random variable taking values in { I , . . . , n}, each possible value having l probability . Show that, as -+ 00, y) -+ y, for y :::
n-
n
2.
00,
A
coin is tossed repeatedly, with heads turning up with probability p on each toss. Let N be the 3. minimum number of tosses required to obtain k heads. Show that, as p -J, the distribution function of p converges to that of a gamma distribution.
0,
2N
4.
If X is an integer-valued random variable with characteristic function f/>, show that
lP'(X
rr: e - i t k f/> (t) dt. = k) = � 2rr J-rr:
What is the corresponding result for a random variable whose distribution is arithmetic with span A (that is, there is probability one that X is a multiple of A, and A is the largest positive number with this property)? 5.
Use the inversion theorem to show that
100 -00
sin(at ) sin(bt) dt t2
=
rr IDln. { a, b} .
n (x)n x x
6. Stirling's formula. Let f be a differentiable function on ffi. with a a global maximum at a > and such that fgc> exp{f ( ) ) d < 00. Laplace's method of steepest descent (related to Watson's lemma and saddlepoint methods) asserts under mild conditions that
0,
1000 n x ) x rv 1000 exp{f ( ) d
exp { Jn (a) +
! (x - a) 2 f�' (a) } dx
as
n -+ 00.
n x = n log x -x, prove Stirling's formula: n! rv n n e -n ,J2rrn. Let X = (Xl , X2 , . . . , Xn) have the multivariate normal distribution with zero means, and covariance matrix = (vij) satisfying IVI > 0 and Vij > 0 for all i, j. Show that 8Xi8xj if i =I j , 8f = -.!!.L 8 vij I 8 2 f If. . = . _2 8xi2 and deduce that lP'(maxk ::;: n Xk ::: u) ::: I1�= I lP'(Xk ::: u). 8.p. UseLettheX}inversion , X2 have a bivariate normal distribution with zero means, unit variances, and correlation theorem to show that 8 I -lP'(X 8p I > 0, X2 > 0) = � 1 - p2 Hence find lP'(XI > 0, X2 > 0). By setting f ( ) 7.
V
{
I
2rr y
55
j,
.
[S.10.1]-[S.10.9]
Generating functions and their applications
Exercises
5.10 Exercises. Two limit theorems 1.
n
Prove that, for x ::: 0, as --+ 00,
(a)
nk rv en jX -1 k! -x ..fiii e - z u du .
-
(b)
1
2
It is well known that infants born to mothers who smoke tend to be small and prone to a range of ailments. It is conjectured that also they look abnormal. Nurses were shown selections of photographs of babies, one half of whom had smokers as mothers; the nurses were asked to judge from a baby's appearance whether or not the mother smoked. In 1500 trials the correct answer was given 910 times. Is the conjecture plausible? If so, why? 3. Let X have the r (1, distribution; given that X = x, let Y have the Poisson distribution with parameter x. Find the characteristic function of Y, and show that Y - JE(Y) as --+ 00. � N(O, 1) Jvar(Y) Explain the connection with the central limit theorem. 4. Let Xl , X2 , . . . be independent random variables taking values in the positive integers, whose common distribution is non-arithmetic, in that gcd{n : lP'(XI = > O} = 1 . Prove that, for all integers x, there exist non-negative integers r = r(x), S = s (x), such that lP' (XI + . . . + Xr - Xr+I - . . . - Xr+s = x ) > O. S. Prove the local central limit theorem for sums of random variables taking integer values. You may assume for simplicity that the summands have span 1, in that gcd { Ix I : lP'(X = x) > O } = 1 . 6. Let Xl , X2, . . . be independent random variables having common density function f(x) = 1/{2Ix l (10g Ix 1) 2 } for Ix l < Show that the Xi have zero mean and finite variance, and that the density function fn of Xl + X2 + . . . + Xn satisfies fn (x) --+ 00 as x --+ O. Deduce that the Xi do not satisfy the local limit theorem. I 7. First-passage density. Let X have the density function f(x) = J2nx- 3 exp(-{2x} - ), x > O. Show that = JE( -s X ) = S > 0, and deduce that X has characteristic function if ::: 0, exp{ -(1 ¢ (t ) = exp{-(I i).JitT} if � O. + t [Hint: Use the result of Problem (5. 12. 1 8).] Let {Xr : r ::: I } be independent with the distribution of the preceding Exercise (7). Let Un = n - I E�= I Xr, and Tn = n-i Un. Show that: (a) lP'(Un < --+ 0 for any < 00, (b) Tn has the same distribution as Xl. 9 . A sequence of biased coins is flipped; the chance that the rth coin shows a head is where is a random variable taking values in (0, 1). Let Xn be the number of heads after flips. Does Xn obey the central limit theorem when: (a) the are independent and identically distributed? (b) = for all r, where is a random variable taking values in (0, I)? 2.
s)
s
n)
e- I .
¢(is)
e --I2i,
e
i) .Ji)
{
t
8.
c)
c
n E>r ,
E>r
E>r E>
E>
56
E>r
Problems
Exercises
[5.11.1]-[5.12.4]
5.11 Exercises. Large deviations 1. A fair coin is tossed n times, showing heads Show that
"
What happens if a Show that
2.
as n
Hn
times and tails
Tn
times. Let
lP(Sn > l /n --+ yI(l + l +a (l - -a 0 < � I? r,ln /n --+ -ryI(l +a)==;=;1 +;=4 = - =;=-a an )
==
--+ 00, where 0 < a < 1 and
Find the asymptotic behaviour of
if
a) l
a)
==
a(l
a) l
a
< 1.
Sn = Hn - Tn .
=
Tn = � (�). Ik- ! n l > ! an T� /n where
Tn =
where a
> o.
Show that the moment generating function of X is finite in a neighbourhood of the origin if and only if X has exponentially decaying tails, in the sense that there exist positive constants A and /1- such that lP(IXI � a ) ::: /1- e - A for a > O. [Seen in the light of this observation, the condition of the large deviation theorem is very natural] .
3.
a
(5.11.4)
4.
Xl
Let X , X2 , . . . be independent random variables having the Cauchy distribution, and let
+ X2 +l . . . + Xn . Find
lP(Sn >
an) .
Sn =
5.12 Problems 1.
A die is thrown ten times. What is the probability that the sum of the scores is
2.
A coin is tossed repeatedly, heads appearing with probability p on each toss.
27?
(a) Let X be the number of tosses until the first occasion by which three heads have appeared successively. Write down a difference equation for f(k) k) and solve it. Now write down an equation for E(X) using conditional expectation. (fry the same thing for the first occurrence of IITH). (b) Let N be the number of heads in n tosses of the coin. Write down G N (S). Hence find the probability that: (i) N is divisible by 2, (ii) N is divisible by
= lP(X = 3.
3.
A coin is tossed repeatedly, heads occurring on each toss with probability p . Find the probability
4.
Find the generating function of the negative binomial mass function
T f (k) = (k - 1 ) (l r-1
generating function of the number
of tosses before a run of n heads has appeared for the first time.
pr
_
p )k-r , 57
k
= r, r + 1, . . .
,
[5.12.5]-[5.12.13]
Generating functions and their applications
Exercises
0 < p < 1 and r is a positive integer. Deduce the mean and variance. For the simple random walk, show that the probability Po (2n ) that the particle returns to the origin at the (2n)th step satisfies po (2n ) rv (4 p ) n /.,;rrn, and use this to prove that the walk is persistent if n1 and only if p 1 . You will need Stirling's formula: n ! n + e n ./2if. 6. A symmetric random walk in two dimensions is defined to be a sequence of points {(Xn, Yn) : n O} which evolves in the following way: if (Xn, Yn) (x, y) then (Xn + l, Yn+ l) is one of the four points (x ± 1, y), (x, y ± 1), each being picked with equal probability ! . If (Xo, Yo) (0, 0): (a) show that E(X� + Y;) n, (b) find the probability Po (2n ) that the particle i s at the origin after the (2n)th step, and deduce that the probability of ever returning to the origin is 1. Consider the one-dimensional random walk {Sn} given by with probability p, Sn+ 1 { SnSn +- 21 with probability 1 - p, where 0 < p < 1. What is the probability of ever reaching the origin starting from So a where a > O? 8. Let X and Y be independent variables taking values in the positive integers such that where
5.
q
=
Z
rv
:::
-
=
=
=
7.
=
q =
=
9.
0
k
X and Y have Poisson distributions. In a branching process whose family sizes have mean and variance u 2 , find the variance of Zn, the size of the nth generation, given that Zo 1. A group { A I , A2 , . . . , Ar } o f r (> 2) people play the following game. wager on the toss of a fair coin. The loser puts £ 1 in the pool, the winner goes on to play AA I . andIn theA2 next wager, the loser puts £1 in the pool, the winner goes on to play A 4 , and so on. The 3 winner of the (r - l)th wager goes on to play A I , and the cycle recommences. The first person to beat all the others in sequence takes the pool.
for some p and all
.::s
.::s
n.
Show that
J1,
=
10. Waldegrave's problem.
(a) Find the probability generating function of the duration of the game. (b) Find an expression for the probability that wins. (c) Find an expression for the expected size of the pool at the end of the game, given that wins. (d) Find an expression for the probability that the pool is intact after the nth spin of the coin. This problem was discussed by Montmort, Bernoulli, de Moivre, Laplace, and others.
Ak
Ak
Hn Hn
11. Show that the generating function of the total number of individuals in the first n generations of a branching process satisfies (s) = sG(Hn _ 1 (s».
Show that the number Zn of individuals i n the nth generation of a branching process satisfies JP'(Zn > N I Zm O) .::s Gm (O) N for n < m. (a) A hen lays N eggs where N is Poisson with parameter A. The weight o f the nth egg is Wn , where WI , W2 , ' " are independent identically distributed variables with common probability generating function G(s). Show that the generating function Gw of the total weight W 2:� 1 Wj is given by Gw(s) exp{ -A + A G ( ) } . W is said to have a compound Poisson distribution. Show l further that, for any positive integral value of n, Gw (s ) / n is the probability generating function of 12.
=
13.
=
=
some random variable;
S
W (or its distribution) is said to be infinitely divisible in this regard. 58
Problems
Exercises
[5.12.14]-[5.12.19]
(b) Show that if R(s) is the probability generating function of some infinitely divisible distribution on the non-negative integers then R(s) = exp{-A + A G (S)} for some A (> and some probability generating function G (s ) .
0)
14. The distribution of a random variable X is called infinitely divisible if, for all positive integers n, n n n there exists a sequence YI( ) , yi ) , . . . , yJ ) of independent identically distributed random variables n n such that X and y? ) + yi ) + . . . + yJ ) have the same distribution. (a) Show that the normal, Poisson, and gamma distributions are infinitely divisible. (b) Show that the characteristic function ¢ of an infinitely divisible distribution has no real zeros, in that ¢ (t ) :f. for all real t.
0
0
X I , X2 , . . , be independent variables each taking the values or 1 with probabilities 1 - P and where < P < 1. Let N be a random variable taking values in the positive integers, independent of the Xj , and write = X l + X2 + . . . + XN . Write down the conditional generating function of N given that = N, in terms of the probability generating function G of N. Show that N has a Poisson distribution if and only if lE(x N )P = lE(x N I = N) for all P and x . 15. Let
p,
16. If
0 S
S
S
X and Y have joint probability generating function where
PI + P2 � 1 ,
find the marginal mass functions of X and Y, and the mass function of X + Y . Find also the conditional probability generating function G X I Y (s I y) = lE (s X I Y = y ) of X given that Y = y. The pair X, Y is said to have the bivariate negative binomial distribution. 17. If
X and Y have joint probability generating function G X ,Y (s, t) = exp { ex(s - 1) + {J (t - 1) + y (st
- I)}
find the marginal distributions of X , Y, and the distribution of X + Y , showing that X and Poisson distribution, but that X + Y does not unless y = O.
Y have the
18. Define
for a, b > O. Show that (a) l(a, b) = a - I I(1, ab), (b) 8 1 /8b = -2I ( l , ab), (c) /(a, b) = ,.fiTe - 2a b / (2a ) . (d) If X has density function (d / .jX) e -c/x-gx for x > then
lE (e - t X ) = d
V g rr+ t
0,
exp ( -2 y'c (g
+ t » ) , t > -g.
(e) If X has density function (2rr x3) - Z e -I/(2x) for x > then X has moment generating function given by lE (e - t X ) = exp { t ::: O. [Note that lE(Xn ) = 00 for n ::: 1 .]
0,
1
-51},
19. Let X, Y, Z be independent N(O, 1) variables. Use characteristic functions and moment gener ating functions (Laplace transforms) to find the distributions of (a) U = X/ Y, (b) V = X - 2 ,
(c)
W = XYZ/y'X2 y2 + Y2 Z2 + Z2 X2 .
59
Generatingfunctions and their applications Let X have density function f and characteristic function tP, and suppose that f�oo I tP (t) I dt < 00. Deduce that 1 1 e -lfx tP(t)dt. f (x) = 2rr [5.12.20]-[5.12.30]
Exercises
20.
00
.
- 00
21. Conditioned branching process. Consider a branching process whose family sizes have the geometric mass function f (k) = k ::: where = / > 1 . Let be the size of the nth generation, and assume o = 1. Show that the conditional distribution of / , given that > converges as -+ 00 to the exponential distribution with parameter 1 .
A
n
Z
qpk , 0,
/1- p q
symmetric
ZnZn1 /1-n /1--
Zn 0,
random variable X is called if X and - X are identically distributed. Show that X is symmetric if and only if the imaginary part of its characteristic function is identically zero.
22.
X and Y be independent identically distributed variables with means 0 and variances 1 . Let tP(t) Letbe their common characteristic function, and suppose that X + Y and X - Y are independent. Show that tP(2t) = tP(t) 3 tP( -t), and deduce that X and Y are N(O, 1 ) variables. More generally, suppose that X and Y are independent and identically distributed with means 0 and variances 1 , and furthermore that E(X - Y I X + Y) = 0 and var(X - Y I X + Y) = 2. Deduce " that tP(s) 2 = tP ' (s) 2 - tP(s)tP (s), and hence that X and Y are independent N(O, 1) variables. Show that the average Z = n- 1 L: i= 1 Xi of n independent Cauchy variables has the Cauchy distribution too. Why does this not violate the law of large numbers? 23.
24.
Let X and Y be independent random variables each having the Cauchy density function f(x) = {rr( 1 + x 2 ) } - I , and let = i (X + Y). (a) Show by using characteristic functions that has the Cauchy distribution also. (b) Show by the convolution formula that has the Cauchy density function. You may find it helpful to check first that
25.
Z
Z Z f(x)f(y - x) = f (x)rr (4+ +f(yy2-) x) + g(y) {xf(x) + (y - x)f(y - x) } where g(y) = 2/{rry(4 + y 2 )}. Let X I , X2 , . . . , Xn be independent variables with characteristic functions tPl, t/1z , . . . , tPn . De scribe random variables which have the following characteristic functions: t/1z (t) . . · tPn(t), (b) I tP l(t) 1 2 , ((c)a) tPl(t) L: PjtPj (t) where Pj 0 and L:'i Pj = 1 , (d) ( 2 - tPl (t)) - I , 'i oo (e) fo tPl (ut)e-U duo
26.
:::
Find the characteristic functions corresponding to the following density functions on (-00, 00 ) : (b) (1 - cOS X)/(7rX 2 ), (a) 1 / cosh(rrx), exp(-x (d) i (c) Show that the mean of the 'extreme-value distribution' in part (c) is Euler's constant y .
27.
- e-X ),
28.
e- ix i .
Which of the following are characteristic functions: (a) =1 if .::s 1 , = otherwise, (b) = (1 + (c) = exp( (e) = 2(1 - cos )/ 2 . (d) = cos t,
tP(t) - It I4 ItI I tP(t) 0 4 tP(t) t )- , tP(t) _t ), tP(t) t t tP(t) Show that the characteristic function tP of a random variable X satisfies 1 1 - tP(t) 1 E l t X I . Suppose X and Y have joint characteristic function tP(s, t). Show that, subject to the appropriate conditions of differentiability, 29.
30.
60
.::s
Problems
Exercises
for any positive integers m and n. 31.
If X has distribution function
[5.12.31]-[5.12.39]
F and characteristic function ifJ, show that for t 0 >
(a) (b)
[0,
32. Let Xl , X2 , . . . be independent variables which are uniformly distributed on 1]. Let Mn = max{X I , X2 , . . . , Xn } and show that n ( l - Mn ) .s X where X is exponentially distributed with parameter 1 . You need not use characteristic functions.
If X is either (a) Poisson with parameter A, or (b) r ( l , A), show that the distribution of Y... 1) distribution as A � 00. (X - EX)/"';var X approaches the (c) Show that n2 nn 1 e -n 1 + n + as n � 00. +...+� -
33.
(
N(O,
n!
2!
)
=
2
34. Coupon collecting. Recall that you regularly buy quantities of some ineffably dull commodity. To attract your attention, the manufacturers add to each packet a small object which is also dull, and in addition useless, but there are n different types. Assume that each packet is equally likely to contain any one of the different types, as usual. Let Tn be the number of packets bought before you acquire a complete set of n objects. Show that n - 1 (Tn - n log n) .s T, where T is a random variable with distribution function JP(T :s x) = exp( _ e -X ), -00 < x < 00.
Find a sequence (ifJn ) of characteristic functions with the property that the limit given by ifJ (t) limn ...... oo ifJn (t) exists for all t, but such that ifJ is not itself a characteristic function. 35.
=
Use generating functions to show that it is not possible to load two dice in such a way that the sum of the values which they show is equally likely to take any value between 2 and 12. Compare with your method for Problem (2.7.12). 36.
N
N
37. A biased coin is tossed times, where is a random variable which is Poisson distributed with parameter A. Prove that the total number of heads shown is independent of the total number of tails. Show conversely that if the numbers of heads and tails are independent, then has the Poisson distribution.
N
38. A binary tree is a tree (as in the section on branching processes) in which each node has exactly two descendants. Suppose that each node of the tree is coloured black with probability p, and white otherwise, independently of all other nodes. For any path T( containing n nodes beginning at the root of the tree, let B(T( ) be the number of black nodes in T(, and let Xn (k) be the number of such paths T( for which B (T( ) � k. Show that there exists f3c such that
E{Xn (f3n)} � and show how to determine the value f3c . Prove that JP ( Xn (f3n)
�
{0
if f3 > f3c , 00 if f3 < f3c ,
1) �
{ 01
if f3 > f3c , if f3 < f3c .
Use the continuity theorem (5.9.5) to show that, as n � 00, (a) if Xn is bin(n, A/n) then the distribution of Xn converges to a Poisson distribution,
39.
61
[5.12.40]-[5.12.46]
Generating junctions and their applications
Exercises
(b) if Yn is geometric with parameter p = nential distribution. 40.
A/n then the distribution of Yn/n converges to an expo
Let Xl , X2 , . . . be independent random variables with zero means and such that JE I X] I < 00 for
all j . Show that Sn = Xl
+ X2 + . . . + Xn satisfies Sn / Jvar(Sn) � N (O, I) as n -+ 00 if
The following steps may be useful. Let al = var(Xj ), a (n ) 2 = var(Sn ), Pj = JEI X] I , and ifJj and 1frn be the characteristic functions of Xj and Sn/a (n) respectively. (i) Use Taylor's theorem to show that lifJj (t) - 1 1 � 2t 2 al and lifJj (t) - 1 + � al t 2 1 � I t l 3 pj for j
:::
1.
(ii) Show that I log (1 + z) - z l � I d if Iz l � � , where the logarithm has its principal value. (iii) Show that a] � Pj , and deduce from the hypothesis that maxl �j :� n aj /a(n) -+ as n -+ 00, implying that maXl �j �n lifJj (t /a(n» - 1 1 -+ O. (iv) Deduce an upper bound for I log ifJj (t/a(n» - � t 2 al /a(n) 2 1 , and sum to obtain that log 1frn(t) -+ l t2 2 41. Let Xl , X2 , . . . be independent variables each taking values +1 or - 1 with probabilities � and i . Show that n "3 L kXk � N(O, 1) as n -+ 00 . n k= l
0
_
v;
Let Xl , X2 , . . . , Xn be independent N(/-L, ( 2 ) random variables. Define X = l n - �'i Xi and Zi = Xi - X. Find the joint characteristic function of X, Zl , Z2 , . . . , Zn , and hence prove that X and S 2 = (n 1 ) - 1 �'i (Xi - X) 2 are independent. 42. Normal sample.
_
(0,
Let X be N 1), and let Y = distribution. Show that the density function of Y is
43. Log-normal distribution.
f (x) =
1
xvJ;C" 2rr
exp { - i (log x) 2 } ,
e X ; Y is said to have the log-normal x > O.
For l a l � 1, define fa (x) = { I +a sin(2rr log x ) } f (x). Show that fa is a density function with finite moments of all (positive) orders, none of which depends on the value of a. The family {fa : la I � I} contains density functions which are not specified by their moments. Consider a random walk whose steps are independent and identically distributed integer-valued random variables with non-zero mean. Prove that the walk is transient. 44.
Let {Xr : r ::: I} be the integer-valued identically distributed intervals between the times of a recurrent event process. Let be the earliest time by which there has been an interval of length a containing no occurrence time. Show that, for integral a,
45. Recurrent events.
L
biased coin shows heads with probability p (= 1 - q). It is flipped repeatedly until the first time Wn by which it has shown n consecutive heads. Let JE(s Wn ) = G n (s ). Show that G n =
46.
A
62
Problems
Exercises
[5.12.47]-[5.12.52]
psG n - l /(l - qsG n - l ), and deduce that
47. In n flips of a biased coin which shows heads with probability of the longest run of heads. Show that, for r ::: 1,
p (= 1 - q), let Ln be the length
The random process {Xn : n ::: I} decays geometrically fast in that, in the absence of external input, Xn+l = i Xn. However, at any time n the process is also increased by Yn with probability i, where {Yn : n ::: I} is a sequence of independent exponential random variables with parameter A. Find the limiting distribution of Xn as n -+ 00. 48.
Let G(s) = E(s X ) where X ::: Show that E{(X + 1) - 1 } = Jd G(s) ds, and evaluate this when X is (a) Poisson with parameter A, (b) geometric with parameter p, (c) binomial bin(n , p), (d) logarithmic with parameter p (see Exercise (5.2.3)). Is there a non-trivial choice for the distribution of X such that E{(X + 1) - 1 } = {E(X + l)} - I ?
O.
49.
50. Find the density function of � �= 1 Xr , where {Xr : r ::: I } are independent and exponentially distributed with parameter A, and tv. is geometric with parameter p and independent of the Xr . 51.
Let X have finite non-zero variance and characteristic function cp(t). Show that
is a characteristic function, and find the corresponding distribution. 52.
Let X and Y have joint density function
Show that cpx (t)cpy (t)
Ix l
=
0 for all j . 4. Let X be a Markov chain with state space S = { I , 2 , 3 } and transition matrix
where 0
0, in agreement
and family-size generating function G. Find a way of describing the process
1
Let Zn be the size of the nth generation of a branching process with Zo = and lP'(Z 1 = k) = q k for k � 0, where P + q = and P > ! . Use your answer to Exercise (2) to show that, if we condition on the ultimate extinction of Z, then the process grows in the manner of a branching process with k generation sizes satisfying = 1 and lP'( I = k) = q for k � O. 3.
Zn
Zo
Z
p
70
p
Birth processes and the Poisson process
Exercises
[6.7.41-[6.8.71
(a) Show that lE(X I X > � lE(X 2 )/lE(X) for any random variable X taking non-negative values. (b) Let Zn be the size of the nth generation of a branching process with Zo = 1 and lP'(Z I = k) = qpk for k � where P > t . Use part (a) to show that lE(Zn/ J-tn I Zn > � 2p/(p - q), where J-t = p /q .
0)
4.
0,
0)
(c) Show that, in the notation of part (b), lE(Zn/ J-tn I
Zn
>
0)
--+
p/(p - q) as n --+ 00.
6.8 Exercises. Birth processes and the Poisson process 1. Superposition. Flies and wasps land on your dinner plate in the manner of independent Poisson processes with respective intensities A and J-t. Show that the arrivals of flying objects form a Poisson process with intensity A + J-t. 2. Thinning. Insects land in the soup in the manner of a Poisson process with intensity A, and each such insect is green with probability p, independently of the colours of all other insects. Show that the arrivals of green insects form a Poisson process with intensity Ap. 3. Let Tn be the time of the nth arrival in a Poisson process N with intensity A, and define the excess lifetime process E (t) = TN (t )+ 1 - t, being the time one must wait subsequent to t before the next arrival. Show by conditioning on Tl that
lP' ( E (t)
>
x)
=
e-J.. (t +x ) +
fot
lP' ( E (t
- u)
>
x)A
e - J.. duo
u
Solve this integral equation in order to find the distribution function of E (t). Explain your conclusion. Let B be a simple birth process (6.S . l I b) with B (O) = I; the birth rates are A n down the forward system of equations for the process and deduce that 4.
k� Show also that lE (B (t »
=
I eAt and var(B (t»
=
=
nA. Write
I.
I e 2A t ( 1 - e - A t ) .
5. Let B be a process of simple birth with immigration (6.S. l I c) with parameters A and v , and with B (O) = the birth rates are A n = nA + v . Write down the sequence of differential-
0,
(0)
Pi,i+ 1 (h ) = Pi,i - l (h ) = j; Ah + o(h ) . Show that the walk i s persistent. Let T be the time spent visiting m during an excursion from o . Find the distribution of T. 72
Birth-death processes and imbedding
Exercises
Let i be a transient state of a continuous-time Markov chain X with X (0) total time spent in state i has an exponential distribution. 9.
=
[6.9.9]-[6.11.4]
i.
Show that the
10. Let X be an asymmetric simple random walk in continuous time on the non-negative integers with retention at 0, so that
pjj (h) = Ah + o(h ) JLh + o(h )
{
if j if j
= =
1,
i + i � 0, i - I, i �
1.
Suppose that X (0) = 0 and A > JL. Show that the total time Vr spent in state r is exponentially distributed with parameter A - JL. Assume now that X (0) has some general distribution with probability generating function G . Find the expected amount of time spent at 0 in terms of G . = {X(t) : t � O} be a non-explosive irreducible Markov chain with generator G and unique stationary distribution 11: . The mean recurrence time JLk is defined as follows. Suppose X (0) = k, and let U = inf{s : Xes) :f. k}. Then JLk = E(inf{t > U : X(t) = k}). Let Z = {Zn : n � O} be the imbedded 'jump chain ' given by Zo = X (O) and Zn is the value of X just after its nth jump. (a) Show that Z has stationary distribution it satisfying
1 1 . Let X
where gj = -gjj , provided L: j 7rjgj < 00. When is it the case that it = 11: ? (b) Show that 7rj = I/(JLjgj) if JLj < 00, and that the mean recurrence time ilk of the state k in the jump chain Z satisfies ilk = JLk L: j 7rjgj if the last sum is finite. 12. Let Z be an irreducible discrete-time Markov chain on a countably infinite state space S, having transition matrix H = (h ij ) satisfying hjj = 0 for all states i , and with stationary distribution v . Construct a continuous-time process X on S for which Z i s the imbedded chain, such that X has no stationary distribution.
6. 1 1 Exercises. 1.
Birth-death processes and imbedding
Describe the jump chain for a birth-death process with rates An and JLn.
Consider an immigration-death process X, being a birth-death process with birth rates A n = A and death rates JLn = nJL. Find the transition matrix of the jump chain Z, and show that it has as stationary distribution
2.
7rn =
2(�!) (1 + �) pne-p
AI JL. Explain why this differs from the stationary distribution of X. 3 . Consider the birth-death process X with A n = nA and JLn = n JL for all n � O. Suppose X (0) = and let TI (t ) = lP' ( X (t ) = 0) . Show that TI satisfies the differential equation where p
=
Hence find TI (t ) , and calculate lP'(X(t )
< t < u. 4. For the birth-death process of the previous exercise with A < JL, show that the distribution of X(t), conditional on the event {X(t ) OJ, converges as t � 00 to a geometric distribution. =
0 I X (u )
1
=
>
73
0) for 0
[6.11.5]-[6.13.2]
Markov chains
Exercises
{
5. Let X be a birth-death process with An = nA and fJ.-n the time T at which X(t) first takes the value 0 satisfies
E(T I T < 00) = What happens when A
=
.!. Iog A
=
nfJ.-, and suppose X (0)
(�) ( )
fJ.- - A A .!.. Iog __ A - fJ.fJ.-
fJ.-?
if)..
< fJ.-,
if A
>
=
1 . Show that
fJ.-.
Let X be the birth-death process of Exercise (5) with A =1= fJ.-, and let Vr (t) be the total amount of time the process has spent in state r ::: 0, up to time t. Find the distribution of VI (00 ) and the generating function 'Er s r E(Vr (t)). Hence show in two ways that E(VI (00)) = [max{A , fJ.-} ] - I . Show further that E(Vr (oo)) = A r - 1 r - 1 [max {A , fJ.-lr r .
6.
7.
Repeat the calculations of Exercise (6) in the case A
=
fJ.-.
6.12 Exercises. Special processes 1. Customers entering a shop are served in the order of their arrival by the single server. They arrive in the manner of a Poisson process with intensity A, and their service times are independent exponentially distributed random variables with parameter fJ.-. By considering the jump chain, show that the expected duration of a busy period B of the server is (fJ.- - A) - 1 when A < fJ.-. (The busy period runs from the moment a customer arrives to find the server free until the earliest subsequent time when the server is again free.) Disasters. Immigrants arrive at the instants of a Poisson process of rate v , and each independently founds a simple birth process of rate A . At the instants of an independent Poisson process of rate 8, the population is annihilated. Find the probability generating function of the population X (t), given that X (0) = o. 2.
More disasters. In the framework of Exercise (2), suppose that each immigrant gives rise to a simple birth-death process of rates A and fJ.-. Show that the mean popUlation size stays bounded if and only if 8 > A - fJ.-.
3.
4. The queue MlG/oo. (See Section 1 1 . 1 .) An ftp server receives clients at the times of a Poisson process with parameter A, beginning at time O. The i th client remains connected for a length Sj of time, where the Sj are independent identically distributed random variables, independent of the process of arrivals. Assuming that the server has an infinite capacity, show that the number of clients being serviced at time t has the Poisson distribution with parameter A Ici [ I - G (x)] dx, where G is the common distribution function of the Sj .
6.13 Exercises.
Spatial Poisson processes
1. In a certain town at time t = 0 there are no bears. Brown bears and grizzly bears arrive as independent Poisson processes B and G with respective intensities f3 and (a) Show that the first bear i s brown with probability f3 / (f3 + (b) Find the probability that between two consecutive brown bears, there arrive exactly r grizzly bears. (c) Given that B (l) = I, find the expected value of the time at which the first bear arrived.
y).
2.
y.
Campbell-Hardy theorem. Let IT be the points of a non-homogeneous Poisson process on = g (x) where g is a smooth function which we assume for
with intensity function A. Let S
jRd
'Ex e n
74
Markov chain Monte CarZo
Exercises
convenience to be non-negative. Show that E(S) provided these integrals converge.
[6.13.3]-[6.14.4]
= JlR.d g(U)A(U) du and var(S) = JlR.d g(u) 2 A(U) du ,
Let n be a Poisson process with constant intensity A on the surface of the sphere of lR.3 with radius 1 . Let P be the process given by the (X, Y) coordinates of the points projected on a plane passing through the centre of the sphere. Show that P is a Poisson process, and find its intensity function. 3.
4.
Repeat Exercise (3), when
xi + xj � I } .
n
is a homogeneous Poisson process on the ball { (X l , x2, X3 )
: xt +
You stick pins in a Mercator projection of the Earth in the manner of a Poisson process with constant intensity A. What is the intensity function of the corresponding process on the globe? What would be the intensity function on the map if you formed a Poisson process of constant intensity A of meteorite strikes on the surface of the Earth?
5.
6. Shocks. The rth point Tr of a Poisson process N of constant intensity A on lR.+ gives rise to an effect Xr e -a(t - Tr ) at time t ::: Tr , where the Xr are independent and identically distributed with finite variance. Find the mean and variance of the total effect S(t) = 'E�? Xr e - a(t - Tr ) in terms of the first two moments of the Xr , and calculate cov(S(s ) , S(t». What is the behaviour of the correlation p(S(s), S(t» as s -+ 00 with t - s fixed? 7. Let N be a non-homogeneous Poisson process on lR.+ with intensity function A. Find the joint density of the first two inter-event times, and deduce that they are not in general independent. S. Competition lemma. Let {Nr (t) : r ::: l } be a collection ofindependent Poisson processes on lR.+ with respective constant intensities {A r : r ::: I }, such that 'Er A r = A < 00. Set N(t) = 'Er Nr (t), an d let I denote the index o f the process supplying the first point i n N, occurring at time T. Show that
' lP'(I = i, T ::: t) = lP'(I = i)lP'(T ::: t) = A� e -J...t ,
i :::
1.
6.14 Exercises. Markov chain Monte Carlo 1. Let P be a stochastic matrix on the finite set e with stationary distribution :n: . Define the inner product (x, y) = 'Ek e El XkYk :n:b and let Z 2 (:n:) = {x E lR. El : (x, x) < oo}. Show, in the obvious notation, that P is reversible with respect to :n: if and only if (x, Py) = (Px , y) for all x, y E Z 2 (:n:). 2.
Barker's algorithm. Show that a possible choice for the acceptance probabilities in Hastings's
general algorithm is
b IJ
. .
where G
-
_
= (gij ) is the proposal matrix.
:n:j gj i , :n:i gij + :n:j gj i
Let S be a countable set. For each j E S, the sets Aj k, k E S, form a partition of the interval [0, 1 ] . Let g : S x [0, 1 ] -+ S be given by g(j, u) = k if U E Aj k . The sequence {Xn : n ::: O} of random variables is generated recursively by Xn+ 1 = g( Xn , Un+ 1 ), n ::: 0, where { Un : n ::: I } are independent random variables with the uniform distribution on [0, 1 ] . Show that X is a Markov chain, and find its transition matrix. 3.
4.
Dobrushin's bound. Let
coefficient is defined to be
(a) Show that, if V is a finite
U = (u s t ) be a finite I S I I T I stochastic matrix. Dobrushin's ergodic x
d(U) = i I TI
x
sup l ) Ui t - ujt l . i,jeS te T l U I stochastic matrix, then d(UV) � d(U)d(V). 75
[6.15.1]-[6.15.7]
(b) Let
Markov chains
Exercises
X and Y be discrete-time Markov chains with the same transition matrix LIk lP'(Xn k) - lP'(Yn k)1 ::: d(p)n LIk lP'(Xo k) - lP'(Yo =
=
=
P, and show that =
k)l·
6.15 Problems 1. Classify the states of the discrete-time Markov chains with state space S transition matrices
( D 1 1 2: 1
"3"
(a)
4
0
2
"3"
1
2:
0 0
( ! i)1
2:
0 0 1
(b)
4
0
=
0 0 0
{I, 2, 3, 4} and
1
2:
0 0 1
In case (a), calculate h4 (n), and deduce that the probability of ultimate absorption in state 4, starting from 3, equals �. Find the mean recurrence times of the states in case (b). 2. A transition matrix is called doubly stochastic if all its column sums equal 1 , that is, if E i Pij = 1 for all j E S. (a) Show that if a finite chain has a doubly stochastic transition matrix, then all its states are non-null persistent, and that if it is, in addition, irreducible and aperiodic then Pij (n) -+ N - 1 as n -+ 00, where N is the number of states. (b) Show that, if an infinite irreducible chain has a doubly stochastic transition matrix, then its states are either all null persistent or all transient. 3.
Prove that intercommunicating states of a Markov chain have the same period.
(a) Show that for each pair i, j of states of an irreducible aperiodic chain, there exists N = N(i, j) such that Pij (n) > 0 for all n � N. (b) Let and be independent irreducible aperiodic chains with the same state space S and transition = matrix P. Show that the bivariate chain n � 0, is irreducible and aperiodic. (c) Show that the bivariate chain may be reducible if and are periodic.
4.
X Y
Z
{Xn
Zn (Xn, Yn), X Y
Xo
Suppose : n � O} is a discrete-time Markov chain with of visits made subsequently by the chain to the state j . Show that
5.
lP'
and deduce that (N
=
00)
=
1 if and only if lij
=
=
i . Let N be the total number
Ijj = 1 .
6. Let i and j be two states of a discrete-time Markov chain. Show that if i communicates with j , then there is positive probability of reaching j from i without revisiting i in the meantime. Deduce that, if the chain is irreducible and persistent, then the probability lij of ever reaching j from i equals 1 for all i and j .
{Xn
Let : n � O } b e a persistent irreducible discrete-time Markov chain on the state space S with transition matrix P, and let x be a positive solution of the equation x = xP . (a) Show that i, j E S, n � 1 ,
7.
76
Problems
Exercises
[6.15.8]-[6.15.13]
defines the n-step transition probabilities of a persistent irreducible Markov chain on S whose first-passage probabilities are given by i =1= j, n
� 1,
0 : Xm
where lji (n) = lP' (Xn = i , T > n I Xo = j ) and T = min {m > (b) Show that x i s unique up to a multiplicative constant. (c) Let 1j = min{n � 1 : Xn = j} and define h ij = lP'( 1j :::; Tj I Xo for all i, j E S . 8.
=
j}.
i ) . Show that Xi h ij
=
Xj hji
{u n : n � o} is called a 'renewal sequence' if n Un = Ji u n - i for n � 1 , i=l
Renewal sequences. The sequence U
Uo
=
=
=
L
1,
for some collection f = Un : n � I } of non-negative numbers summing to 1 . (a) Show that U is a renewal sequence if and only if there exists a Markov chain X on a countable state space S such that U n = lP' (Xn = s I Xo = s), for some persistent S E S and all n � 1 . (b) Show that if U and v are renewal sequences then so is {u n Vn n � o}. 9.
:
Consider the symmetric random walk in three dimensions on the set of points { (x
0, ± 1 , ±2, . . . } ; this process is a sequence {Xn : n � o} of points such that lP'(Xn+l for E (± 1 , 0, 0), (0, ± 1 , 0), (0, 0, ± 1 ) . Suppose that Xo (0, 0, 0). Show that =
lP'(X2n =
, y, z) : =
y, z = Xn + E) = i x,
( 1 ) 2n (2n ) L ( n n !· ) 2 ! (0, 0, 0)) ( 61 ) 2n i+j+k=n L C '(2n) · ' k ,. ) 2 2 n i+j+k=n 3 · , ' k '. =
=
I.J.
I.J.
=
and deduce by Stirling's formula that the origin is a transient state. 10. Consider the three-dimensional version of the cancer model (6. 1 2. 1 6). If K of Theorem (6. 1 2. 1 8) inevitable in this case? 11. Let X be a discrete-time Markov chain with state space S
(
=
=
1 , are the empires
,
{ 1 , 2} and transition matrix
p -- l -f3 a Classify the states of the chain. Suppose that af3 > ° and af3 =1= 1 . Find the n-step transition probabilities and show directly that they converge to the unique stationary distribution as n --+ 00. For what values of a and f3 is the chain reversible in equilibrium? 12. Another diffusion model. N black balls and N white balls are placed in two urns so that each contains N balls. After each unit of time one ball is selected at random from each urn, and the two balls thus selected are interchanged. Let the number of black balls in the first urn denote the state of the system. Write down the transition matrix of this Markov chain and find the unique stationary distribution. Is the chain reversible in equilibrium? 13. Consider a Markov chain on the set S = {O, 1 , 2, . . . } with transition probabilities Pi,i+ 1 = a i , Pi,O = 1 - ai , i � where (ai i � is a sequence of constants which satisfy ° < ai < 1 for all i . Let bo = 1 , bi = aO a l . . . ai - l for i � 1 . Show that the chain is (a) persistent if and only if bi --+ ° as i --+ 00, (b) non-null persistent if and only if L:: i bi < 00,
0,
:
0)
and write down the stationary distribution if the latter condition holds. 77
[6.15.14]-[6.15.19]
Markov chains
Exercises
Let A and f3 be positive constants and suppose that ai chain is (c) transient if f3 > 1 , (d) non-null persistent if f3 < 1 . Finally, if f3 = 1 show that the chain is (e) non-null persistent if A > 1 , (0 null persistent if A :::; 1 .
= 1 -
Ai - P for all large i . Show that the
X be a continuous-time Markov chain with countable state space S and standard semigroup Show that Pij (t) is a continuous function of t . Let g(t) = - log Pii (t); show that g is a continuous function, g(O) = 0, and g(s + t) :::; g(s) + g(t). We say that g is 'subadditive' , and a well known theorem gives the result that 14. Let
{PrJ.
lim
g(t)
t.J,O t
=
A
g(t) A = sup -t
exists and
t>O
:::; 00.
limt .J, O t - I { Pii (t)
- I } exists, but may be -00. 15. Let X be a continuous-time Markov chain with generator G = (g ij ) and suppose that the transition semigroup Pt satisfies Pt = exp(tG) . Show that X is irreducible if and only if for any pair i, j of states there exists a sequence k I , k2 , . . . , kn of states such that gi,k l gk l ,k2 . . . gkn ,j =1= O. 16. (a) Let X = {X ( t ) : -00 < t < oo} be a Markov chain with stationary distribution :n:, and suppose that X (0) has distribution :n: . We call X reversible if X and Y have the same joint distributions, where Y(t) = X(-t). (i) If X (t) has distribution :n: for all t, show that Y is a Markov chain with transition probabilities P�j (t) = (Jrj /Jri ) Pji (t), where the Pji (t) are the transition probabilities of the chain X. (ii) If the transition semigroup {PrJ of X is standard with generator G, show that Jrigij = Jrj gji (for all i and j) is a necessary condition for X to be reversible. (iii) If Pt = exp(tG) , show that X (t) has distribution :n: for all t and that the condition in (ii) is
Deduce that gii
=
sufficient for the chain to be reversible. (b) Show that every irreducible chain X with exactly two states is reversible in equilibrium. (c) Show that every birth-death process X having a stationary distribution is reversible in equilibrium.
17. Show that not every discrete-time Markov chain can be imbedded in a continuous-time chain. More precisely, let p _ -
(
01
1 - 01
1 (1 01
)
for some 0
< 01
1 .
78
Problems
Exercises
[6.15.20]-[6.15.27]
Xl , X2 , . . . , having density function f and distribution function F. Let Yl = X l , let Y2 be the first offer exceeding Yl , and generally let Yn+ 1 be the first offer exceeding Yn . Show that Yl , Y2 , . . . are the times of arrivals in a non-homogeneous Poisson process with intensity function A (t ) = f ( t ) / ( l - F(t)). The Yi are called 'record values' . Now let Z I be the first offer received which is the second largest to date, and let Z2 be the second such offer, and so on. Show that the Zi are the arrival times of a non-homogeneous Poisson process with intensity function A. 21. Let N be a Poisson process with constant intensity A, and let Yl , Y2 , . . . be independent random variables with common characteristic function c/J and density function f. The process N* (t ) = Yl + Y2 + . . . + YN (t) is called a compound Poisson process. Yn is the change in the value of N* at the nth arrival of the Poisson process N. Think of it like this. A 'random alarm clock' rings at the arrival times of a Poisson process. At the nth ring the process N* accumulates an extra quantity Yn . Write down a forward equation for N* and hence find the characteristic function of N* (t ) . Can you 20. Successive offers for my house are independent identically distributed random variables
see directly why it has the form which you have found?
A of a non-homogeneous Poisson process N is itself a random process, then N is called a doubly stochastic Poisson process (or Cox process). Consider the case when A (t ) = A for all t, and A is a random variable taking either of two values A l or A 2 , each being picked with equal probability i . Find the probability generating function of N (t), and deduce its mean and variance. 22. If the intensity function
X with parameter A is a doubly stochastic Poisson process with AX (t). 24. The Markov chain X = {X(t) : t ::: O} is a birth process whose intensities A k ( t ) depend also on the time t and are given by 1 + JLk lP' (X (t + h) = k + 1 1 X(t) = k) = 1 + JLt h + o(h ) as h .j, O. Show that the probability generating function G(s, t) = E (s X(t) ) satisfies aG s - 1 G + JLs a G = -a; ' 0 < s < 1 . at 1 + JLt Hence find the mean and variance of X (t) when X (0) = I. 25. (a) Let X be a birth-death process with strictly positive birth rates A O , A I , . . , and death rates JL l , JL2 , . . . . Let TJi be the probability that X (t) ever takes the value 0 starting from X (0) = i . Show that j ::: 1 , Aj TJj+l - (Aj + JLj )TJj + JLj TJj - l = 0, and deduce that TJi = 1 for aH i so long as Ef ej = 00 where ej = JL I JL 2 ' " JLj /(A I A 2 " · Aj ).
23. Show that a simple birth process
intensity function A (t )
=
}
{
(b) For the discrete-time chain on the non-negative integers with
find the probability that the chain ever visits 0, starting from 1 . 26. Find a good necessary condition and a good sufficient condition for the birth-death process X of Problem (6. 15 .25a) to be honest.
X be a simple symmetric birth-death process with An extinction. Show that
27. Let
lP'(T ::: x I X (0) = l) = 79
(
1
=
JLn = nA, and let T be the time until
AX + AX
)1 '
[6.15.28]-[6.15.33]
Markov chains
Exercises
and deduce that extinction is certain if 11"( X (0) < (0) = I . Show that lP'(AT/ I ::s x I X (O) = I) � e - 1 /x as I �
00 .
28. Immigration-death with disasters. Let X be an immigration-death-disaster process, that is, a
birth-death process with parameters A i = A, fJ.-i = i fJ.-, and with the additional possibility of 'disasters' which reduce the population to O. Disasters occur at the times of a Poisson process with intensity 8, independently of all previous births and deaths. (a) Show that X has a stationary distribution, and find an expression for the generating function of this distribution. (b) Show that, in equilibrium, the mean of X (t) is .1.. / (8 + fJ.-) . 29. With any sufficiently nice (Lebesgue measurable, say) subset B of the real line lR is associated a random variable X (B) such that (i) X (B) takes values in {O, 1 , 2, . . . }, (ii) if Bb B2, . . . , Bn are disjoint then X (B l ) , X ( B2 ) , . . . , X (Bn ) are independent, and furthermore X (BI U B2 ) = X ( B l ) + X (B2) , (iii) the distribution o f X (B) depends only o n B through its Lebesgue measure ( 'length') I B I , and
lP'(X (B) � I) lP'(X (B) = I)
� �
as I B I � o.
I
Show that X is a Poisson process. 30. Poisson forest. Let
N be a Poisson process in lR2 with constant intensity A, and let R( 1 ) < R(2)
O.
A. Show that the volume of the largest (n-dimensional) sphere centred at the origin which contains no point of X is exponentially distributed. Deduce the density function of the distance R from the origin to the nearest point of X. Show that E(R) = rO/n)/{n(Ac) l / n } where c is the volume of the unit ball of lRn and r is the gamma function. 31. Let X be a n-dimensional Poisson process with constant intensity
N + I people suffers an epidemic. Let X (t) be the number of ill people at time t, and suppose that X (O) = I and X is a birth process with rates A i = Ai (N + I - i). Let T be the length of time required until every member of the popUlation has succumbed to the illness. Show that
32. A village of
N
E(T) = iI L k(N +I I - k) k= l and deduce that where
y
y)
N+ O -2 E(T) = 2(log A(N + I ) + (N ) i s Euler's constant. It i s striking that E(T) decreases with N, for large N.
{
33. A particle has velocity V (t) at time t, where V (t) i s assumed to take values i n {n + Transitions during (t, t + h) are possible as follows:
lP' ( V (t + h) = w I V (t) = v ) =
�
(v + )h + o(h) 1 - 2vh + o(h)
if w = v + l , if w = v ,
(v - )h + o(h)
if w = v - I .
�
80
� : n � OJ.
Problems Initially
Exercises
V (0) = � . Let
[6.15.34]-[6.15.39]
00
G(s, t) = I > n lP'( V(t) = n + �) . n =O (a) Show that
aG at
=
(1
_
s) 2 aG as
_
(1
and deduce that G(s, t) = {1 + ( 1 - s ) t } - l . (b) Show that the expected length m n (T) of time for which V is given by m n (T)
=
loT lP'(V t
( )
=
- s)G =
n + � during the time interval [0, T]
n + � ) dt
and that, for fixed k, m k (T) - log T -+ 'E f= l i - I as T -+ (c) What is the expected velocity of the particle at time t? -
34. A random sequence of non-negative integers produced by Xn + Xn - l Xn+l =
{Xn
{
IXn - Xn - l l
:
n
00 .
� O} begins Xo
with probability with probability
=
0,
Xl
1 , and is
�,
I
2·
Show that Yn = (Xn - l , Xn ) is a transient Markov chain, and find the probability of ever reaching ( 1 , 1) from ( 1 , 2) . 35. Take a regular hexagon and join opposite corners by straight lines meeting at the point C. A particle performs a symmetric random walk on these 7 vertices, starting at A. Find: (a) the probability of return to A without hitting C, (b) the expected time to return to A, (c) the expected nmber of visits to C before returning to A, (d) the expected time to return to A, given that there is no prior visit to C. 36. Diffusion, osmosis. Markov chains are defined by the following procedures at any time
n:
(a) Bernoulli model. Two adjacent containers A and B each contain m particles; m are of type I and m are of type II. A particle is selected at random in each container. If they are of opposite types they are exchanged with probability a if the type I is in A, or with probability p if the type I is in B. Let Xn be the number of type I particles in A at time (b) Ehrenfest dog-flea model. Tw o adjacent containers contain m particles i n all. A particle is selected at random. If it is in A it is moved to B with probability a, if it is in B it is moved to A with probability p . Let Yn be the number of particles in A at time In each case find the transition matrix and stationary distribution o f the chain.
n.
n.
X be an irreducible continuous-time Markov chain on the state space S with transition prob abilities Pjk (t) and unique stationary distribution 11: , and write lP'(X (t) = j) = aj (t). If c(x) is a concave function, show that d(t) = 'Eje S 7rj c(aj (t)/7rj ) increases to c ( 1 ) as t -+ 00 . 37. Let
Uk (t) = lP' (X (t ) = k I X (O) the chain is reversible in equilibrium (see Problem (6. 1 5 . 1 6». Show that uo(2t) = and deduce that uo(t) decreases to 7ro as t -+ 00 . 38. With the notation of the preceding problem, let
=
0) , and suppose
'E/7rO /7rj )Uj (t) 2 ,
39. Perturbing a Poisson process. Let [J be the set of points in a Poisson process on ]Rd with constant intensity A. Each point is displaced, where the displacements are independent and identically distributed. Show that the resulting point process is a Poisson process with intensity A.
81
[6.15.40]-[6.15.46]
Markov chains
Exercises
40. Perturbations continued. Suppose for convenience in Problem (6. 1 5 .39) that the displacements have a continuous distribution function and finite mean, and that d = 1 . Suppose also that you are at the origin originally, and you move to a in the perturbed process. Let LR be the number of points formerly on your left that are now on your right, and RL the number of points formerly on your right that are now on your left. Show that lE(LR) = lE(RL) if and only if a = J1, where J1, is the mean displacement of a particle. Deduce that if cars enter the start of a long road at the instants of a Poisson process, having independent identically distributed velocities, then, if you travel at the average speed, in the long run the rate at which you are overtaken by other cars equals the rate at which you overtake other cars.
A.; they each visit the pantry and then the sink, and leave. The rth ant spends time Xr in the pantry and Yr in the sink (and Xr + Yr in the kitchen altogether), where the vectors Vr = (Xr , Yr ) and Vs are independent for r #- s. At time t = 0 the kitchen is free of ants. Find the joint distribution of the numbers A(t) of ants in the pantry and B(t) of ants in the sink at time t. Now suppose the ants arrive in pairs at the times of the Poisson process, but then separate to behave independently as above. Find the joint distribution of the numbers of ants in the two locations. 41. Ants enter a kitchen at the instants of a Poisson process N of rate
I}
{Xr r :::: be independent exponential random variables with parameter A., and set Sn = E�= l Xr . Show that: (a) Yk = Sk i Sn , 1 � k � n - 1 , have the same distribution as the order statistics of independent variables {Uk : 1 � k � n - which are uniformly distributed on (0, 1), (b) Zk = Xk l Sn , 1 � k � n, have the same joint distribution as the coordinates of a point ( UI , . . . , Un) chosen uniformly at random on the simplex E�=l U r = 1 , U r :::: 0 for all r. 43. Let X be a discrete-time Markov chain with a finite number of states and transition matrix P = ( Pij ) where Pij 0 for all i, j . Show that there exists A. E (0, 1) such that I Pij (n) - lZ"j I < A.n , 42. Let
:
I}
>
where 1C is the stationary distribution.
44. Under the conditions of Problem (6. 1 5 .43), let Vi ( n ) the chain to i before time n. Show that
(/ L-
=
E�,;;;J I( Xr =iJ be the number of visits of
/2)
Show further that, if f is any bounded function on the state space, then lE
1
;;
n l r=O
f(Xr ) - � f (i ) 7ri
-+ o.
IES
B
45. Conditional entropy. Let A and = (Bo, B I , . . . , Bn ) b e a discrete random variable and vector, respectively. The conditional entropy of A with respect to is defined as H(A I lE (lE{- log f(A I where f(a I b) = JP(A = a I = b) . Let X be an aperiodic Markov chain on a finite state space. Show that
B) I B})
and that
H(Xn+1 I Xn) -+
-L
B
7ri L Pij 10g Pij j
B
as n -+
B)
=
00 ,
if X is aperiodic with a unique stationary distribution 1C .
X and Y be independent persistent birth-death processes with the same parameters (and no explosions). It is not assumed that Xo = yo. Show that:
46. Coupling. Let
82
Problems
Exercises
[6.15.47)-[6.15.49J
(a) for any A � JR, IIP'(X t E A) - 1P'(Yt E A) I -+ 0 as t -+ 00 , (b) if lP'(Xo :::; yo) = 1, then lE[g(X t )] :::; lE[g(Yt )] for any increasing function g.
47. Resources. The number of birds in a wood at time t is a continuous-time Markov process X. Food resources impose the constraint 0 :::; X (t) :::; n. Competition entails that the transition probabilities obey Pk,k+ 1 (h ) = >"(n - k) h + o(h), Pk,k - I (h ) = /Lkh + o(h ) .
Find
lE(s X(t » ), together with the mean and variance o f X(t),
t -+ oo?
when
X(O)
= r.
What happens as
48. Parrando's paradox. A counter performs an irreducible random walk on the vertices 0, 1 , 2 of the triangle in the figure beneath, with transition matrix
P=
(�
Po qo I o PI P 2 q2 0
)
where Pi + qi = 1 for all i . Show that the stationary distribution 11: has
1 7ro = 3 q - qq2 PI q , - l PO - 2 PI - OP2 with corresponding formulae for 7r 1 , 7r2 .
2
o
Suppose that you gain one peseta for each clockwise step of the walk, and you lose one peseta for each anticlockwise step. Show that, in equilibrium, the mean yield per step is Y
=
E. (2Pi - 1)7ri = I
3(2pOPIP2 - POPI - PI P2 - P2 PO + Po + PI + P2 - 1) . 3 - ql Po - q2 PI - QOP2
-
Consider now three cases of this process:
i - a for each i , where a > O . Show that the mean yield per step satisfies YA < O. We have that Po = to a, PI = P2 = i - a, where a > O. Show that JIB < 0 for sufficiently small a.
A. We have Pi =
B.
-
C . A t each step the counter i s equally likely to move according to the transition probabilities of case A or case B, the choice being made independently at every step. Show that, in this case, Po = to a, PI = P2 = i - a . Show that YC > 0 for sufficiently small a. The fact that two systematically unfavourable games may be combined to make a favourable game is called Parrando's paradox. Such bets not available in casinos.
are
49. Cars arrive at the beginning of a long road in a Poisson stream of rate >.. from time t = 0 onwards. A car has a fixed velocity V > 0 which is a random variable. The velocities of cars are independent and identically distributed, and independent of the arrival process. Cars can overtake each other freely. Show that the number of cars on the first x miles of the road at time t has the Poisson distribution with parameter >"lE[V - 1 min{x , Vt}].
83
[6.15.50]-[6.15.51]
Marlwv chains
Exercises
50. Events occur at the times of a Poisson process with intensity A, and you are offered a bet based on the process. Let t > O. You are required to say the word 'now' immediately after the event which you think will be the last to occur prior to time t. You win if you succeed, otherwise you lose. If no events occur before t you lose. If you have not selected an event before time t you lose. Consider the strategy in which you choose the first event to occur after a specified time s, where 0 < s < t. (a) Calculate an expression for the probability that you win using this strategy. (b) Which value of s maximizes this probability? (c) If A t � 1 , show that the probability that you win using this value of s is e - 1 .
A
new Oxbridge professor wishes to buy a house, and can afford to spend up to one million pounds. Declining the services of conventional estate agents, she consults her favourite internet property page on which houses are announced at the times of a Poisson process with intensity A per day. House prices may be assumed to be independent random variables which are uniformly distributed over the interval (800,000, 2,000,000) . She decides to view every affordable property announced during the next 30 days. The time spent viewing any given property is uniformly distributed over the range ( 1 , 2) hours. What is the moment generating function of the total time spent viewing houses? 51.
84
7
Convergence of random variables
7.1 Exercises. Introduction
Let r ::: 1 , and define II X ll r = {El x r l } l / r . Show that: (a) Il e X llr = l e i · II X l i r for e E R, (b) II X + Y IIr � II X li r + II Y ll r , (c) II X li r = 0 if and only if JP>(X = 0) = 1 . This amounts to saying that I . I r is a norm on the set of equivalence classes of random variables on a given probability space with finite rth moment, the equivalence relation being given by X Y if and only if JP>(X = Y) = 1 . 1.
�
Define (X, Y ) = E(XY) for random variables X and Y having finite variance, and define II X II J(X, X) . Show that: (a) (aX + bY, Z ) = a (X, Z ) + b (Y, Z) , (b) II X + Y 11 2 + II X - Y 11 2 = 2( II X 1 I 2 + II Y II 2 ), the paralleiogram property, (c) if (Xi , Xj ) = 0 for all i =f:. j then
2.
Let E > O. Let g, h : [0, 1 ] --+ R, and define dE (g, h) = IE dx where Ig (u) - h (u) 1 > d. Show that dE does not satisfy the triangle inequality.
3.
E
=
=
{u E [0, 1 ] :
F and G, let d(F, G) = inf{ 8 0 : F(x - 8) - 8 � G(x) � F (x + 8) + Hor all x E R} . Show that d is a metric on the space of distribution functions.
4.
Levy metric.
For two distribution functions >
5. Find random variables X, Xl , X 2 , . . . such that E(I Xn - X 1 2 ) --+ 0 as for all
n.
7.2 Exercises. Modes of convergence 1.
(a) Suppose Xn � X where r ::: 1 . Show that E I X� I --+ E l x r l .
I
(b) Suppose Xn
-+
X . Show that E(Xn) --+ E(X) . Is the converse true?
(c) Suppose Xn
-+
X . Show that var(Xn ) --+ var(X) .
2
85
n --+
00,
but E I Xn l
= 00
[7.2.2]-[7.3.3] 2.
Convergence 01 random variables
Exercises
Dominated convergence.
then Xn
I
--+
Suppose l Xn l ::: Z for all n, where E(Z)
X.
< 00 . Prove that if Xn � X
Give a rigorous proof that E(XY) = E(X)E(Y) for any pair X, Y of independent non-negative random variables on (Q , J", lP) with finite means. [Hint: For k � 0, n � 1 , define Xn = kin if kin ::: X < (k + l)ln, and similarly for Yn . Show that Xn and Yn are independent, and Xn ::: X, and Yn ::: Y . Deduce that EXn -+ EX and EYn -+ EY, and also E(Xn Yn ) -+ E(XY).]
3.
Show that convergence in distribution is equivalent to convergence with respect to the Levy metric of Exercise (7. 1 .4).
4.
5.
(a) Suppose that Xn S X and Yn � Xn l Yn S X l c if c # O.
c, where c is a constant.
Show that Xn Yn S cX, and that
(b) Suppose that Xn S 0 and Yn � Y, and let g : JR2 -+ JR be such that g(x, y) is a continuous p function of y for all x, and g(x, y) is continuous at x = 0 for all y. Show that g (Xn , Yn) --+ g(O, Y). [These results are sometimes referred to as 'Slutsky's theorem(s)' .J Let Xl , X2 , . . . be random variables on the probability space (Q , :F, lP) . Show that the set A = {w E Q : the sequence Xn (w) converges} is an event (that is, lies in :F), and that there exists a random variable X (that is, an F-measurable function X : Q -+ JR) such that Xn (w) -+ X (w) for w E A. 6.
Let {Xn } be a sequence of random variables, and let { en } be a sequence of reals converging to the limit c. For convergence almost surely, in rth mean, in probability, and in distribution, show that the convergence of Xn to X entails the convergence of cn Xn to cX. 7.
8.limitLetX. {Xn } be a sequence of independent random variables which converges in probability to the Show that X is almost surely constant.
9. Convergence in total variation. The sequence of discrete random variables Xn , with mass functions In , is said to converge in total variation to X with mass function I if
L l in (x ) - l (x ) 1
-+
x
Suppose Xn
-+
X in total variation, and u :
0
as
n -+ 00 .
JR -+ JR is bounded.
Show that E(u(Xn))
-+
E(u (X)).
10. Let {X r : r � I} be independent Poisson variables with respective parameters {A r : r � I}. Show that L:� l Xr converges or diverges almost surely according as L:� l A r converges or diverges.
7.3 Exercises.
Some ancillary results
(a) Suppose that Xn � X. Show that {Xn } is Cauchy convergent in probability in that, for all 0, lP( I Xn - Xm I > E ) -+ 0 as n, m -+ 00. In what sense is the converse true? (b) Let {Xn } and { Yn } be sequences of random variables such that the pairs (Xj , Xj ) and (Yj , Yj )
1.
E >
have the same distributions for all i, j . If Xn � X, show that Yn converges in probability to some limit Y having the same distribution as X.
Show that the probability that infinitely many of the events {An : lP(An i.o.) � lim sUPn � oo lP(An ) .
2.
n �
I } occur satisfies
Let {Sn : n � O} b e a simple random walk which moves to the right with probability step, and suppose that So = O. Write Xn = Sn - Sn - l .
3.
86
p at each
Some ancillary results
Exercises
[7.3.4]-[7.3.10]
(a) Show that {Sn = ° i.o.} is not a tail event of the sequence {Xn } . (b) Show that JP>(Sn = ° i.o.) = ° if p i= �. (c) Let Tn = Sn / -/ii, and show that
{ lim inf Tn n �oo
.:s
-x
is a tail event of the sequence {Xn }, for all x
P -_ 21 '
} n { lim sup Tn ::: x } n �oo
>
0, and deduce directly that JP>(Sn
=
° i.o.)
=
I if
Hewitt-Savage zerQ-()ne law. Let X l , X2 , . . . be independent identically distributed random variables . The event A, defined in terms of the Xn , is called exchangeable if A is invariant un der finite permutations of the coordinates, which is to say that its indicator function fA satisfies fA (X 1 , X2 , . . . , Xn , . . . ) = fA (Xi 1 ' Xi2 ' . . . , Xin , Xn+ 1 , . . . ) for all n ::: 1 and all permutations (i I , i 2 , . . . , in ) of ( 1 , 2, . . , n). Show that all exchangeable events A are such that either JP>(A) = ° or JP>(A) = 1 . 4.
,
5. Returning to the simple random walk S of Exercise (3), show that {Sn = ° i.o.} i s an exchangeable event with respect to the steps of the walk, and deduce from the Hewitt-Savage zero--one law that it has probability either ° or 1 .
6. Weierstrass's approximation theorem. Let I : [0, 1 ] -+ lR be a continuous function, and let Sn be a random variable having the binomial distribution with parameters n and x. Using the formula lE(Z) = lE(ZfA ) + lE(ZfAc) with Z = I(x) - l (n - 1 Sn ) and A = { I n - 1 Sn - x l > 8}, show that sup nlim �oo O:'S :'S l
x
I / (X) k=Ot I(k/n) (:) -
n x k ( 1 - x) - k
I
=
0.
You have proved Weierstrass's approximation theorem, which states that every continuous function on [0, 1] may be approximated by a polynomial uniformly over the interval. 7.
Complete convergence.
convergent to X if
A sequence X l , X2 , . "
of random variables is said to be for all E
>
completely
0.
Show that, for sequences of independent variables, complete convergence is equivalent to a.s. conver gence. Find a sequence of (dependent) random variables which converges a.s. but not completely.
8.and finite Let X l , X2 , ' " be independent identically distributed random variables with common mean f..t variance. Show that as n
9.
Let {Xn
:
n :::
I } be independent and exponentially distributed with parameter 1 . Show that
(
Xn . JP> hm sup -n � oo log n
(.
10. Let {Xn
:
-+ 00 .
=
1
)
=
1.
n ::: I } be independent N(O, 1 ) random variables.
I Xnl r;;)
(a) JP> lim sup r.;;;:-:: n �oo ", log n
= '"
2
=
1, 87
Show that:
[7.3.11J-[7.5.1J
(b) JP(Xn
>
Convergence of random variables
Exercises
{
an i.o.) = °
if E n JP(XI if En JP(XI
1
an) < 00, an) = 00.
> >
11. Construct an example to show that the convergence in distribution of Xn to X does not imply the convergence of the unique medians of the sequence Xn .
....
12. (i) Let {Xr : r ::: I } be independent, non-negative and identically distributed with infinite mean. Show that lim supr oo Xr lr = 00 almost surely. (ii) Let {Xr } be a stationary Markov chain on the positive integers with transition probabilities
Pj k =
{
j - if k = J. + 1 , + j 2 j
....
!2
if k = 1 .
(a) Find the stationary distribution of the chain, and show that it has infinite mean. (b) Show that lim supr 00 Xr I r � 1 almost surely. 13. Let {X r : I
-
� r � n } be independent and identically distributed with mean J1, and finite variance
n 2 a . Let X = n - 1 E r=1 Xr . Show that
t
r =1
(Xr - J1,)
/ t
(Xr - X) 2
r =1
converges in distribution to the N(O, 1) distribution as n -+ 00.
7.4 Exercise. 1.
Laws of large numbers
Let X2 , X3 , . . . be independent random variables such that JP(Xn =
n) = JP(Xn = -n) =
-2n n
1 , log
JP(Xn = 0) = 1 -
-n n
1 . log
Show that this sequence obeys the weak law but not the strong law, in the sense that converges to ° in probability but not almost surely. 2.
Construct a sequence {Xr
:
r
:::
n
-
1 E1 X i
I } of independent random variables with zero mean such that
n - 1 E �=1 Xr -+ -00 almost surely, as n -+ 00.
Let N be a spatial Poisson process with constant intensity )., in ]Rd , where d ::: 2. Let S be the ball of radius r centred at zero. Show that N(S) / I S I -+ )., almost surely as r -+ 00, where l S I is the volume of the ball.
3.
7.5 Exercises. The strong law
1. Entropy. The interval [0, 1] is partitioned into n disjoint sub-intervals with lengths P I , P2 , . . . , Pn , and the entropy of this partition is defined to be n
h = - iL Pi log Pi . =1
88
Martingales
Exercises
[7.5.2]-[7.7.4]
Let Xl , X2 , . . . be independent random variables having the uniform distribution on [0, 1], and let Zm (i) be the number of the Xl , X2 , . . . , Xm which lie in the ith interval of the partition above. Show that n Rm - II Pizm (i ) i=l l satisfies m log Rm --+ -h almost surely as m --+ 00 .
Recurrent events. Catastrophes occur at the times Tl , T2 , . . . where Ti = Xl + X2 + . . . + Xi and the Xi are independent identically distributed positive random variables. Let N (t) = max. { n : Tn � t } be the number of catastrophes which have occurred by time t. Prove that if lE(Xl ) < 00 then N(t) --+ 00 and N(t)/t --+ l /lE(X as t --+ 00, almost surely.
2.
1)
Random walk. Let Xl , X2 , . . . be independent identically distributed random variables taking values in the integers Z and having a finite_mean. Show that the Markov chain S = {Sn } given by Sn = �1 Xi is transient if lE(X =1= O.
3.
1)
7.6 Exercise. Law of the iterated logarithm
1. A function cp (x ) is said to belong the 'upper class' if, in the notation of this section, IP'(Sn cpupper (n)..jn i.o.) O. A consequence of the law of the iterated logarithm is that .Ja log log x is in the for all a 2 . Use the first Borel-Cantelli lemma to prove the much weaker fact that .Ja log x is in the upper class for all a 2, in the special case when the Xi are independent cpN(O,(x) 1)classvariables. =
=
to
>
>
>
7.7 Exercises. Martingales
1. Let Xl , X2 , . . . be random variables such that the partial sums Sn determine a martingale. Show that lE(Xi Xj ) = 0 if i =1= j .
=
Xl + X2 + . . . + Xn
2. Let Zn b e the size of the nth generation of a branching process with immigration, i n which the family sizes have mean J1, (=1= 1) and the mean number of immigrants in each generation is m . Suppose that lE(Zo) < 00, and show that
is a martingale with respect to a suitable sequence of random variables. Let Xo , XI . X2 , . . . be a sequence of random variables with finite means and satisfying lE(Xn+ l I XO , XI . . . . , Xn) = aXn + bXn - l for n ::: 1 , where 0 < a, b < 1 and a + b = 1 . Find a value of a for which Sn = aXn + Xn - l , n ::: 1 , defines a martingale with respect to the sequence X. 3.
Let Xn be the net profit to the gambler of betting a unit stake on the nth play in a casino; the Xn may be dependent, but the game is fair in the sense that lE(Xn+l I Xl , X2 , . . . , Xn) = 0 for all n. The gambler stakes Y on the first play, and thereafter stakes fn (X X2 , . . . , Xn) on the (n + l)th play, where fl , h, . . . are given functions. Show that her profit after n plays is n Sn = "L: Xi fi - l (Xl , X2 , · · · , Xi - d , i=l
4.
1,
where fo = Y. Show further that the sequence S = {Sn } satisfies the martingale condition lE(Sn+ l Xl , X2 , . . . , Xn) = Sn, n ::: 1 , if Y is assumed to be known throughout. 89
I
[7.8.1]-[7.9.5]
Corwergence of random variables
Exercises
7.8 Exercises. Martingale convergence theorem 1.
Kolrnogorov's inequality.
and finite variances, and let Sn show that
IP
Let Xl , X2 , . . . be independent random variables with zero means Xl + X2 + . . . + Xn . Use the Doob-Kolmogorov inequality to
=
( Im� I Sj > E ) ::: ; t var(Xj ) � J �n l E J. = 1
for E >
O.
Let Xl , X 2 , . . . be independent random variables such that L:n n - 2 var(Xn ) mogorov's inequality to prove that 2.
t Xi - �(Xi ) � Y i= l
I
as n
-+ 00,
as n
-+ 00.
o. (d) Show that N is differentiable in probability but not in mean square.
h) - N (t ) } 2 ) � 0 as - N(t)1
>
E
)
� 0 as
6. Prove that n - 1 2:7=1 Xj � 0 whenever the X i are independent identically distributed variables with zero means and such that E(Xt) < 00 . 7. 8. 9.
Show that Xn �
X whenever 2:n E( I Xn - Xn < 00 for some r O. Show that if Xn S X then aXn + b S aX + b for any real a and b. If X has zero mean and variance 0" 2 , show that 0" 2 for t > O. lP' ( X t ) ::s: � 0" + t 2 >
;:::
10. Show that
Xn � 0 if and only if E
C ���n l ) �
0
as n
� 00 .
{Xn } is said to be mean-square Cauchy convergent if E{(Xn - X ) 2 } � 0 as Show that {Xn } converges in mean square to some limit X if and only if it is mean-square
m
11. The sequence
m, n �
00.
Cauchy convergent. Does the corresponding result hold for the other modes of convergence?
{Xn } is a sequence of uncorrelated variables with zero means and uniformly bounded variances. Show that n - 1 2:7=1 Xi � O. 13. Let X l , X2 , . . . be independent identically distributed random variables with the common dis tribution function F, and suppose that F(x) < 1 for all x. Let Mn = max {X l , X2 , . . . , Xn} and suppose that there exists a strictly increasing unbounded positive sequence a I , a2 , . . . such that lP' (Mn/an ::s: x) � H(x) for some distribution function H . Let us assume that H is continuous with 0 < H (1) < 1 ; substantially weaker conditions suffice but introduce extra difficulties. (a) Show that n[1 - F(an x)] � - log H(x) as n � 00 and deduce that 1 F(anx) � log H(x) if x O. 1 F(an) log H ( I ) 12. Suppose that
(b) Deduce that if x
>
0
----
1 - F(tx) � 1 - F (t )
>
log H(x) log H ( 1 )
--=--'--
as t
� 00 .
(c) Set x = X l x2 and make the substitution
g (x) = to
find that g (x
+
log H(eX ) log H(1)
y) = g(x)g(y), and deduce that H(x) = exp(- ax
{
o
92
fJ )
if x if x
;::: 0,
< 0,
Problems
Exercises
[7.11.14]-[7.11.19]
for some non-negative constants a and {J. You have shown that H is the distribution function of y - I , where Y has a Weibull distribution. 14. Let X I , X2 , . . . , Xn be independent and identically distributed random variables with the Cauchy distribution. Show that Mn = max{XI , X2 , . . . , Xn } is such that ]l' Mn /n converges in distribution, the limiting distribution function being given by H (x) = e- I /x if x � O.
Let XI , X2 , . . . be independent and identically distributed random variables whose common I characteristic function ,p satisfies ,p' (0) = ilL. Show that n = 1 Xj � IL . 15.
E'j
16. Total variation distance.
X and Y is defined by
The total variation distance dTV (X, Y) between two random variables
dTV(X, Y) =
sup
I E(u (X» - E(u(Y» I
u :ll u lloo = 1
where the supremum is over all (measurable) functions u : lR -+ lR such that l I u li oo = supx l u (x ) 1 satisfies l I u ll oo = (a) If X and Y are discrete with respective masses In and gn at the points Xn , show that
l.
dTV(X, Y) = L l in - gn l = 2 sup I IP'(X E A) - 1P'(Y E A) I · A�1R
n
(b) If X and Y are continuous with respective density functions I and g, show that
dTV(X, Y) =
-+
1
00
- 00
I I (x) - g(x) 1 dx
=
2 sup I IP'(X E A) - 1P'(Y E A) I . A�1R
-+
(c) Show that dTV(Xn , X) 0 implies that Xn X in distribution, but that the converse is false. (d) Maximal coupling. Show that IP'(X '" Y) � ! dTV(X, Y), and that there exists a pair X', yl having the same marginals for which equality holds. (e) If Xi , Yj are independent random variables, show that
17.
Let g : lR
-+ lR be bounded and continuous. Show that 00
k
(n)., ) - n J... g(k/n) L k .,-e -+ g().,) k=O
as n
-+
00 .
18. Let Xn and Ym be independent random variables having the Poisson distribution with parameters n and m, respectively. Show that
(Xn - n) - (Ym - m) ,JXn + Ym
D
...:.-'....-�;:::===;: := ;:=---'- -+
N (O, 1 )
as m , n
-+ 00 .
(a) Suppose that X I , X2 , . . . is a sequence of random variables, each having a normal distribution, and such that Xn � X. Show that X has a normal distribution, possibly degenerate. (b) For each n � 1 , let (Xn , Yn) be a pair of random variables having a bivariate normal distribution. Suppose that Xn � X and Yn � Y, and show that the pair (X, Y) has a bivariate normal distribution.
19.
93
[7.11.20]-[7.11.26]
Convergence of r
andom variables
Exercises
20. Let X I , X2 , . . . be random variables satisfying var(Xn) < c for all n and some constant c. Show that the sequence obeys the weak law, in the sense that n - l �1 (Xj - lEXj ) converges in probability to 0, if the correlation coefficients satisfy either of the following: (i) p (Xj , Xj ) ::0 0 for all i =1= j , (ii) p (Xj , Xj ) � 0 as Ii - j l � 00 . 2t. Let X I , X 2 , . . . be independent random variables with common density function
f (x)
{
=
0
c
x 2 10g Ix l
if Ix l ::0 2, if Ix l > 2,
where c is a constant. Show that the Xj have no mean, but n - l �?= l Xj � 0 as n � 00 . Show that convergence does not take place almost surely. 22. Let Xn be the Euclidean distance between two points chosen independently and unifonnly from the n-dimensional unit cube. Show that lE(Xn)/Jn � 1 /J6 as n � 00 . 23. Let Xl , X 2 , . . . be independent random variables having the unifonn distribution on [- 1, 1]. Show that
24.
Let X I , X2 , . . . be independent random variables, each Xk having mass function given by IP'(Xk
=
k)
=
IP'(Xk
=
1) = IP'(Xk = - 1 ) =
IP'(Xk
=
=
-k)
1
-2 ' 2k
� (1 - ; ) k
if k >
1.
Show that Un = �1 Xj satisfies Un/ In � N(O, 1) but var(Un / In) � 2 as n � 00 . 25. Let Xl , X 2 , . . . be random variables, and let NI , N2 , . . . be random variables taking values in the positive integers such that Nk � 00 as k � 00 . Show that: (i) if Xn � X and the Xn are independent of the Nb then XNk � X as k �
(ii) if Xn � X then XNk � X as k �
00,
00 .
26. Stirling's formula.
(a) Let a (k, n)
=
n k / (k - I ) ! for i ::0 k ::0 n + 1. Use the fact that I - x ::0 e -x if x � 0 to show that a(n - k, n) e -k2 j(2n ) if k � O. a(n + 1 , n)
Deduce Kolmogorov's inequality:
O.
c>
O.
30. Let Xl , X2 , . . . be independent random variables with zero means, and let Sn = Xl + X 2 +
. . . + Xn . Using Kolmogorov's inequality or the martingale convergence theorem, show that: (i) �� l Xi converges almost surely if �� l lF.(Xt ) < (ii) if there exists an increasing real sequence (bn ) such that bn and satisfying the inequality l t �� l lF.(X )lbt < then b;; �� l Xk � 0 as n
00,
� 00, � 00.
00,
31. Estimating the transition matrix. The Markov chain XO , Xl , . . . , Xn has initial distribution = lP' (Xo = and transition matrix P. The function )., (P) is defined as )., (P) =
i)
fi
log-likelihood
log ( fxo PXO , Xj PXj ,X2 ' " PXn _ j ,Xn )' Show that: (a) )., (P) = log fxo + � i , j Nij log Pij where Nij is the number of transitions from i to j ,
Pij ' )., (P) i s maximal when Pij = Pij where Pij = Nij I �k Nib (c) if X is irreducible and ergodic then Pij � Pij as n � 00.
(b ) viewed as a function of the
32. Ergodic theorem in discrete time. Let X be an irreducible discrete-time Markov chain, and let
be the mean recurrence time of state i . Let (n) = ��;;;J I{Xr=iJ be the number of visits to i up to n 1 , and let be any bounded function on S. Show that: l (a) n � as n (b) if < for all i, then
/Li
f
-
Vi(n) /Li 00
Vi
/Li l
� 00,
n l 1 n 0
- L f (Xr) � L f (i ) //Li i eS ,=
as n
� 00 .
33. Ergodic theorem in continuous time. Let X be an irreducible persistent continuous-time Markov
chain with generator G and finite mean recurrence times 95
/Lj .
[7.11.34]-[7.11.37]
(a) S how that
lot
1 t 0 I{ X(s ) =j } ds
-
Convergence of random variables
Exercises a.s.
1 /-Lj gj
-+ --
as t --+
00;
(b) deduce that the stationary distribution :n: satisfies 11) (c) show that, if f i s a bounded function on S,
=
1/(/-Ljgj);
�t r f(X(s)) ds � L 1fi /(i ) Jo i 34. Tail equivalence. Suppose that the sequences
00.
{Xn : n �
which is to say that E� I lP'(Xn =F Yn ) < Show that: (a) E� 1 Xn and E� 1 Yn converge or diverge together, (b) E� 1 (Xn - Yn ) converges almost surely,
as t --+
00 .
I } and {Yn :
n � I } are tail equivalent,
(c) if there exist a random variable X and a sequence an such that an t oo and a; l then
E�= 1 Xr � X,
1 n L Yr � X. an r= 1
35. Three series theorem. Let {Xn : n � I } be independent random variables. Show that E� 1 converges a.s. if, for some a > 0, the following three series all converge: (a) En lP'( I Xn l > a), (b) En var (Xn II l Xn l :::;a } ) (c) En E ( XnII l Xn l ::: a } ) · [The converse holds also, but is harder to prove.]
Xn
,
{X : n �
n We call Xk a record value for the sequence if Xk > Xr for 1 F.indicator function of the event that X is a record value.
36. Let
I} be independent random variables with continuous common distribution function :::: r < k, and we write h for the
k
(a) Show that the random variables h are independent. =
Ek= l lr satisfies Rm /log m � 1 as m --+ 00 . 37. Random harmonic series. Let {Xn : n � I } be a sequence of independent random variables with lP'(Xn = 1) = lP'(Xn = -1) = !. Does the series E�= 1 Xr /r converge a.s. as n --+ oo? (b) Show that Rm
96
8 Random processes
8.2 Exercises. 1.
Flip-flop. Let
Stationary processes
{Xn } be a Markov chain on the state space S = {O, I } with transition matrix p
=
( l -f3 a
)
a I - f3 '
where a + f3 > O. Find: (a) the correlation p(Xm ,
Xm+n ), and its limit as m --+ 00 with n remaining fixed, 1 limn-+oo n E�=l JP>(Xr = 1).
(b) Under what condition i s the process strongly stationary?
Random telegraph. Let (N(t) : t � O} be a Poisson process of intensity A, and let To be an independent random variable such that JP>(To = ±1) = 1 . Define T(t) = To (- I) N (t) . Show that (T(t) : t � O} is stationary and find: (a) p(T(s), T(s + t» , (b) the mean and variance of 2.
X(t) = Ici T(s) ds. 3. Korolyuk-Khinchin theorem. An integer-valued counting process (N(t) : t � O} with N(O) = o is called crudely stationary if Pk (S, t) = JP>(N(s + t) - N(s) = k) depends only on the length t and not on the location s. It is called simple if, almost surely, it has jump discontinuities of size 1 only. Show that, for a simple crudely stationary process N, limt ,l.O t - 1 JP>(N(t) 0) = lE(N(1». >
8.3 Exercises. Renewal processes
Let (fn : n � 1) be a probability distribution on the positive integers, and define a sequence (un : n � 0) by uo = 1 and Un = E�=l fr un -r , n � 1 . Explain why such a sequence is called a renewal sequence, and show that u is a renewal sequence if and only if there exists a Markov chain U and a state s such that Un = JP>(Un = s I Uo = s). 2. Let {Xj : i � I} be the inter-event times of a discrete renewal process on the integers. Show that the excess lifetime Bn constitutes a Markov chain. Write down the transition probabilities of the sequence {Bn} when reversed in eqUilibrium. Compare these with the transition probabilities of the chain U of your solution to Exercise (1). 3. Let (un : n � 1) satisfy Uo = 1 and Un = E�=l fr u n - r for n � 1, where ( fr : r � 1) is a non-negative sequence. Show that: n (a) Vn = p n un is a renewal sequence if p 0 and E� l p fn = 1, n (b) as n --+ 00, p u n converges to some constant c. 1.
>
97
[8.3.4]-[8.4.5]
Random processes
Exercises
Events occur at the times of a discrete-time renewal process N (see Example (5 .2. 15» . Let U n be the probability of an event at time n, with generating function U(s), and let F(s) be the probability generating function of a typical inter-event time. Show that, if I s I < 1 :
4.
5.
Prove Theorem (8.3.5): Poisson processes are the only renewal processes that are Markov chains.
8.4 Exercises. Queues 1. The two tellers in a bank each take an exponentially distributed time to deal with any customer; their parameters are A and /.t respectively. You arrive to find exactly two customers present, each occupying a teller. (a) You take a fancy to a randomly chosen teller, and queue for that teller to be free ; no later switching is permitted. Assuming any necessary independence, what is the probability p that you are the last of the three customers to leave the bank? (b) If you choose to be served by the quicker teller, find p. (c) Suppose you go to the teller who becomes free first. Find p.
Customers arrive at a desk according to a Poisson process of intensity A. There is one clerk, and the service times are independent and exponentially distributed with parameter /.t. At time 0 there is exactly one customer, currently in service. Show that the probability that the next customer arrives before time t and finds the clerk busy is
2.
3. Vehicles pass a crossing at the instants of a Poisson process of intensity A; you need a gap of length at least a in orde.r to cross. Let T be the first time at which you could succeed in crossing to the other side. Show that lE(T) = (e aA - 1) / A , and find lE(e9 T ). Suppose there are two lanes to cross, carrying independent Poissonian traffic with respective rates A and /.t. Find the expected time to cross in the two cases when: (a) there is an island or refuge between the two lanes, (b) you must cross both in one go. Which is the greater?
Customers arrive at the instants of a Poisson process of intensity A, and the single server has exponential service times with parameter /.t. An arriving customer who sees n customers present (including anyone in service) will join the queue with probability (n + 1 ) / (n + 2) , otherwise leaving for ever. Under what condition is there a stationary distribution? Find the mean of the time spent in the queue (not including service time) by a customer who joins it when the queue is in equilibrium. What is the probability that an arrival joins the queue when in equilibrium? 4.
5. Customers enter a shop at the instants of a Poisson process of rate 2. At the door, two represen tatives separately demonstrate a new corkscrew. This typically occupies the time of a customer and the representative for a period which is exponentially distributed with parameter 1 , independently of arrivals and other demonstrators. If both representatives are busy, customers pass directly into the shop. No customer passes a free representative without being stopped, and all customers leave by another door. If both representatives are free at time 0, show the probability that both are busy at time 2 15 - 5 t · t is �5 - �e 3 - t + �e
98
Problems
Exercises
[8.5.1]-[8.7.4]
8.S Exercises. The Wiener process
(0) 0, show that - l ([. lP'(W(s) > 0, Wet) > 0) !4 + 22n sin V i Calculate lP'(W(s) > 0, Wet) > 0, W(u) > 0) when s < t < u. 1.
For a Wiener process
W with W
=
=
for s
< t.
Let W be a Wiener process. Show that, for s < t < u, the conditional distribution of W(t) given W(s) and W(u) is normal - t) W(s) + (t - s)W(u) , (u - t)(t N u-s u-s Deduce that the conditional correlation between W(t) and W(u), given W(s) and W(v), where s < t < u < v, is (v - u)(t - s) (v - t)(u - s) 2.
- S)) .
((U
For what values of a and b is aWl + independent standard Wiener processes?
3.
Show that a Wiener process is to say that 4.
bW2 a standard Wiener process, where WI
W2 are
W with variance parameter a 2 has finite quadratic variation, which
n- l L { W ((j + l)t/n) - W(j t/n) } 2 � a 2 t j =O
5.
and
as n
--+
00.
Let W be a Wiener process. Which of the following define Wiener processes? (a) -W(t), (b) ,Jt W(l), (c) W(2t) - W(t). 8.7 Problems
1. Let {Zn } be a sequence of uncorrelated real-valued variables with zero means and unit variances, and define the 'moving average' r
Yn = L ai Zn - i , i =O for constants aO , a I , . . . , ar . Show that Y is stationary and find its autocovariance function. 2. Let {Zn } be a sequence of uncorrelated real-valued variables with zero means and unit variances. Suppose that {Yn } is an 'autoregressive' stationary sequence in that it satisfies Yn = aYn - 1 + Zn , -00 < n < 00, for some real a satisfying lal < 1 . Show that Y has autocovariance function c(m) = a 1m l /(l - a 2 ). 3 . Let {Xn } be a sequence of independent identically distributed Bernoulli variables, each taking values and 1 with probabilities 1 - p and p respectively. Find the mass function of the renewal process N(t) with interarrival times {Xn }.
0
Customers arrive in a shop in the manner of a Poisson process with parameter A . There are infinitely many servers, and each service time is exponentially distributed with parameter JL. Show that the number Q(t) of waiting customers at time t constitutes a birth-death process. Find its stationary distribution.
4.
99
[8.7.5]-[8.7.7]
Random processes
Exercises
5. Let X(t) = Y cos(Ot) + Z sin(Ot) where Y and Z are independent N (O, 1) random variables, and let X (t) = R cos(Ot + qt) where R and qt are independent. Find distributions for R and qt such that the processes X and X have the same fdds. 6. Bartlett's theorem. Customers arrive at the entrance to a queueing system at the instants of an inhomogeneous Poisson process with rate function ).. ( t). Their subsequent service histories are independent of each other, and a customer arriving at time s is in state A at time s + t with prob ability p(s, t). Show that the number of customers in state A at time t is Poisson with parameter
t oo ).. (u)p(u, t u) duo -
7. In a Prague teashop (U Mysaka), long since bankrupt, customers queue at the entrance for a blank bill. In the shop there are separate counters for coffee, sweetcakes, pretzels, milk, drinks, and ice cream, and queues form at each of these. At each service point the customers' bills are marked appropriately. There is a restricted number N of seats, and departing customers have to queue in order to pay their bills. If interarrival times and service times are exponentially distributed and the process is in equilibrium, find how much longer a greedy customer must wait if he insists on sitting down. Answers on a postcard to the authors, please.
1 00
9 Stationary processes
9.1 Exercises.
Introduction
Let . . . , Z - 1 ' Zo, Z 1 , Z2 , . . . be independent real random variables with means 0 and variances 1, and let a, f3 E lR. Show that there exists a (weakly) stationary sequence {Wn } satisfying Wn = aWn - 1 + f3Wn -2 + Zn , = . . . , - 1 , 0, 1 , . . . , if the (possibly complex) zeros of the quadratic equation z 2 - az - f3 = 0 are smaller than 1 in absolute value. 1.
n
Let U be uniformly distributed on [0, 1] with binary expansion U the sequence 2.
00
Vn = E Xi +n 2 - i , i=1
=
L: � 1 Xi r i .
Show that
n :::: 0,
is strongly stationary, and calculate its autocovariance function.
Let {Xn : = . . , - 1 , 0, 1 , . . . } be a stationary real sequence with mean 0 and autocovariance function c(m). (i) Show that the infinite series L:�o an Xn converges almost surely, and in mean square, whenever
n .
3.
L:�o lan l
< 00.
(ii) Let
00
Yn = E ak Xn -k , k=O where L:�o that
lak l
n = . . . , - 1 , 0, 1 , . . .
< 00. Find an expression for the autocovariance function c y of Y, and show 00
E Icy(m) 1 < 00. m= - oo Let X = {Xn : O} be a discrete-time Markov chain with countable state space S and stationary distribution 11: , and suppose that Xo has distribution 11: . Show that the sequence {f(Xn ) :::: O} is strongly stationary for any function f : S lR.
4.
n 2:
:n
--+
9.2 Exercises. 1.
Linear prediction
Let X be a (weakly) stationary sequence with zero mean and autocovariance function c(m).
(i) Find the best linear predictor Xn+1 o f Xn+1 given Xn.
(ii) Find the best linear predictor Xn+1 of Xn+1 given Xn and Xn - 1 . 101
[9.2.2]-[9.4.2]
Exercises
Stationary processes lE{(Xn+ l - Xn+ l )2 } - lE{(Xn+ l - Xn+ l )2 },
(iii) Find an expression for D = expression when: (a) = cos(n U) where U is uniform on (b) is an autoregressive scheme with c(k)
XXn
and evaluate this
[-n, n),k
1.
+ a2)X-n1 . Xn {XXnn-} l ,
Assuming that such a sequence exists, find the best linear predictor . . . , and show that the mean squared error of prediction is (1 (weakly) stationary.
Xn-2 ,
given of Verify that
is
9.3 Exercises. Autocovariances and spectra
xn
+
= A cos(n.i.. ) 1. Let B sin(n.i.. ) where A and B are uncorre1ated random variables with zero means and unit variances. Show that is stationary with a spectrum containing exactly one point.
X
(-n, n),
2. Let U be uniformly distributed on and let V be independent of U with distribution func = e i ( U - V n ) defines a stationary (complex) sequence with spectral distribution tion F. Show that function F.
Xn
Find the autocorrelation function of the stationary process density function is: (i) N(O, l), (ii) f(x) = i e l i x
3.
4.
Let
X l , X2 ,
c(m). Show that
- x , -00 < < 00. t Xj ) = C(O) j ( �n j=l
oo} whose spectral
. . . be a real-valued stationary sequence with zero means and autocovariance function . var
sm(.i..
� 0 if and only if
function. Deduce that
c(O){F(O) - F(O-)} =
9.4 Exercises.
( nSi�(n.i..j� ) 2 d F(.i..) n - 1 'L-'j= l Xj n'"- l c(j ). n-+oo n j =O
(-1r,1rj
F is the spectral distribution F(O) - F(O-) = 0, and show that
where
S
(X(t) : -00 < t
O. [Hint: Consider the renewal process with interarrival times Xi = EI{ Xe : € } for some suitable E.]
2. Let N be a renewal process and let W be the waiting time until the length of some interarrival time has exceeded s. That is, W = inf{t : C (t) > s}, where C (t) is the time which has elapsed (at time t) since the last arrival. Show that
Fw (x ) =
{
0 1
- F(s) + J� Fw (x - u) dF(u)
if x < if x 2:
s, s,
where F is the distribution function of an interarrival time. If N is a Poisson process with intensity A., show that A. - e JE(e9 W ) = A. ee (J.. -9 ) s for e < A. , and JE(W) = (eJ.. s - 1)/1.. . You may find it useful to rewrite the above integral equation in the form of a renewal-type equation. 3.
Find an expression for the mass function of N(t) in a renewal process whose interarrival times
are: (a) Poisson distributed with parameter A., (b) gamma distributed, r (A. , b).
Let the times between the events of a renewal process N b e uniformly distributed o n (0, 1 ) . Find the mean and variance of N (t) for 0 ::::: t ::::: 1 .
4.
10.2 Exercises. Limit theorems 1. Planes land at Heathrow airport at the times of a renewal process with interarrival time distribution function F. Each plane contains a random number of people with a given common distribution and finite mean. Assuming as much independence as usual, find an expression for the rate of arrival of passengers over a long time period.
Let Z l , Z2 , . . . be independent identically distributed random variables with mean 0 and finite variance (1 2 , and let Tn = L:i=l Zi . Let M be a finite stopping time with respect to the Zi such that JE(M) < 00. Show that var(TM) = JE(M)(1 2 .
2.
1 07
[10.2.3]-[10.4.1]
Show that lE(TN(t) k ) + ILm (t) .
3.
Renewals
Exercises =
lL (m (t) + k) for all k ::: l , but that it is not generally true that lE(TN(t» )
Show that, using the usual notation, the family {N(t)/t How might one make use of this observation?
4.
:
0
::;
=
t < oo} i s unifonnly integrable.
5. Consider a renewal process N having interarrival times with moment generating function M, and N( let T be a positive random variable which is independent of N. Find lE(s T » ) when: (a) T is exponentially distributed with parameter v, (b) N is a Poisson process with intensity J.. , in terms of the moment generating function of T . What is the distribution of N(T) in this case, if T has the gamma distribution r (v, b)?
10.3 Exercises. Excess life 1. Suppose that the distribution of the excess lifetime E (t) does not depend on t. Show that the renewal process is a Poisson process. 2.
Show that the current and excess lifetime processes, C (t) and E (t), are Markov processes.
Suppose that X I is non-arithmetic with finite mean JL. (a) Show that E (t) converges in distribution as t 00, the limit distribution function being
3.
�
H(x)
=
r .!. [ 1 - F (y)] dy .
Jo
IL
(b) Show that the rth moment of this limit distribution is given by I lE(X rI + ) r ' dH(x) = x IL (r + 1 ) o
10 00
assuming that this is finite. (c) Show that lE(E(t l)
=
lE ({ (X I - t)+ V ) +
lo
t
h (t - x) dm (x)
for some suitable function h to be found, and deduce by the key renewal theorem that lE(E(tt) I lE(X1 + )f{IL(r + I ) } as t 00, assuming this limit is finite.
�
�
4. Find an expression for the mean value of the excess lifetime E (t) conditional on the event that the current lifetime C (t) equals x . 5. Let M(t) = N(t) + 1 , and suppose that Xl has finite non-zero variance a 2 . (a) Show that var(TM(t ) - ILM(t)) = a 2 (m (t) + 1). (b) In the non-arithmetic case, show that var(M(t))/t a 2 / 1L 3 a s t 00 .
�
�
10.4 Exercise. Applications 1. Find the distribution of the excess lifetime for a renewal process each of whose interarrival times is the sum of two independent exponentially distributed random variables having respective parameters J.. and JL. Show that the excess lifetime has mean 1 J.. - (H /L) t + IL - + -:-c-:----:-� J.. (J.. + JL) IL
e
1 08
Problems
Exercises
[10.5.1]-[10.5.6]
10.5 Exercises. Renewal-reward processes 1. IT X(t) is an irreducible persistent non-null Markov chain, and u(·) is a bounded function on the integers, show that
�t r u(X(s)) ds � L Jl'iU(i), 10 i eS where is the stationary distribution of X(t). 2 . Let M(t) be an alternating renewal process, with interarrival pairs {Xr , Yr 1 r JEXl a.s . as t � 00 . t 10 II M(s ) is even } ds JEX 1 + JEYl 1C
:
r ::: I}.
Show that
---+
3.
X.
Let C(s) be the current lifetime (or age) o f a renewal process N(t) with a typical interarrival time Show that 1 t a.s. JE(X 2 ) - C (s ) ds ---+ --- as t 00 .
lo
t 0
2JE(X)
�
Find the corresponding limit for the excess lifetime. Let j and k be distinct states of an irreducible discrete-time Markov chain distribution 1C . Show that 4.
X with stationary
1 / Jl'k -I Xo = k) = ---JE(1j I Xo = k) + JE(Tk I Xo = j) where Ti = min {n ::: 1 X = i } is the first passage time to the state i . [Hint: Consider the times of return to j having made an intermediate visit to k.] IP'(T·J < Tk :
n
10.5 Problems
�
�
(a) Show that IP'( N (t) 00 as t (0) = 1 . (b) Show that m(t) < 00 if J1, =p O. (c) More generally show that, for all k > 0, JE(N(t) k ) < 00 if J1, =p O.
1.
2.
Let v(t)
=
IE(N(t) 2 ). Show that
v(t) = m(t) + 2
lot m(t - s) dm(s).
Find v(t) when N i s a Poisson process. 3.
Suppose that 0' 2
=
var(X l )
>
O. Show that the renewal process N satisfies
N(t) - (t/J1,) � N(O, 1 ) , Vt O' 2 /J1,3 4.
as t
� 00 .
Find the asymptotic distribution of the current life C (t) of N as t
� 00 when X l is not arithmetic.
5. Let N be a Poisson process with intensity A. Show that the total life D(t) at time t has distribution function lP'(D(t) :::: x) = 1 - ( 1 + A min{t, x})e -AX for x ::: O. Deduce that IE(D(t)) = (2 - e -At )/A. 6. A Type 1 counter records the arrivals of radioactive particles. Suppose that the arrival process is Poisson with intensity A, and that the counter is locked for a dead period of fixed length T after
1 09
[10.5.7]-[10.5.12]
Renewals
Exercises
each detected arrival. Show that the detection process N is a renewal process with interarrival time distribution F(x) 1 - - J.. ( - T ) if x :::: T . Find an expression for lP'(N(t) :::: k) .
=
x
e
Particles arrive at a Type 1 counter in the manner of a renewal process N; each detected arrival locks the counter for a dead period of random positive length. Show that 7.
= fox
lP'(X l :::: x)
- F(x - y)] h (y) dm(y)
FL is the distribution function of a typical dead period. (a) Show that m(t) 1 At - ! (1 - e -2J..t ) if the interarrival times have the gamma distribution
where S.
[1
=
r (J.. , 2) . (b) Radioactive particles arrive like a Poisson process, intensity J.. , at a counter. The counter fails to register the nth arrival whenever is odd but suffers no dead periods. Find the renewal function iii of the detection process N.
n
9. Show that Poisson processes are the only renewal processes with non-arithmetic interarrival times having the property that the excess lifetime E(t) and the current lifetime C(t) are independent for each choice of t. 10. Let Nl b e a Poisson process, and let N2 be a renewal process which i s independent of Nl with non-arithmetic interarrival times having finite mean. Show that Nl (t) + N2 (t) is a renewal process if and only if N2 is a Poisson process.
N(t) =
11. Let N be a renewal process, and suppose that F is non-arithmetic and that a 2 var(X l) < 00. Use the properties of the moment generating function F* ( - 0 ) of X 1 to deduce the formal expansion
=
m * (O)
1 J1,
-+
=o
a 2 - J1,2 2J1,2
+ 0( 1 )
as O
�
o.
Invert this Laplace-Stieltjes transform formally to obtain
m(t)
=t
-+
J1,
a 2 - J1,2 2J1, 2
+ 0( 1)
ast �
00 .
Prove this rigorously by showing that
m(t)
= t - FE (t) -
J1,
+
lot 0
[1 -
FE (t - x)] dm(x),
where FE is the asymptotic distribution function of the excess lifetime (see Exercise (10.3.3)), and applying the key renewal theorem. Compare the result with the renewal theorems. 12. Show that the renewal function
m d of a delayed renewal process satisfies
where m is the renewal function of the renewal process with interarrival times X2 , X3 , 1 10
....
Problems
Exercises
[10.5.13]-[10.5.19]
13. Let m (t) be the mean number of living individuals at time t in an age-dependent branching process with exponential lifetimes, parameter A, and mean family size v (> 1). Prove that m(t) = I e (v- l ) A t where I is the number of initial members. 14. Alternating renewal process. The interarrival times of this process are Zo , Yl , Zl , Y2 , . . . , where the Yj and Zj are independent with respective common moment generating functions and Let p (t ) be the probability that the epoch t of time lies in an interval of type Z. Show that the Laplace-Stieltjes transform p* of p satisfies
My
Mz.
p*
I_: --,M.-z)(-,:---::::,- Mz(-(j -_(j:.".)--:-:-) «(j) -:--1 ---:My(-(j =
15. Type 2 counters. Particles are detected by a Type 2 counter of the following sort. The incoming particles constitute a Poisson process with intensity A. The jth particle locks the counter for a length Yj of time, and annuls any after-effect of its predecessors. Suppose that Y1 , Y2 , . . . are independent of each other and of the Poisson process, each having distribution function G. The counter is unlocked at time o. Let L be the (maximal) length of the first interval of time during which the counter is locked. Show that H(t) = lP'(L > t) satisfies
H(t) = e -At [1 - G(t)] +
l H(t - x)[1 - G(x)]Ae -AX dx.
Solve for H in terms of G, and evaluate the ensuing expression in the case J1, > 0.
G(x) = 1 - e - f.LX where
N, M(t)
16. Thinning. Consider a renewal process and suppose that each arrival is 'overlooked' with probability q, independently of all other arrivals. Let be the number of arrivals which are detected up to time t / p where p = 1 q . (a) Show that i s a renewal process whose interarrival time distribution function Fp i s given by Fp (x) = E� l pq r - l Fr (x/p), where Fn is the distribution function of the time of the nth arrival in the original process (b) Find the characteristic function of Fp in terms of that of F, and use the continuity theorem to show that, as p + 0, Fp (s) ----+ l - e - s / f.L for S > 0, so long as the interarrival times in the original process have finite mean J1,. Interpret! (c) Suppose that p < 1, and and are processes with the same fdds. Show that is a Poisson process. -
M
N.
M N
N
17. (a) A PC keyboard has 100 different keys and a monkey is tapping them (uniformly) at random.
Assuming no power failure, use the elementary renewal theorem to find the expected number of keys tapped until the first appearance of the sequence of fourteen characters 'W. Shakespeare' . Answer the same question for the sequence 'omo ' . (b) A coin comes u p heads with probability p o n each toss. Find the mean number o f tosses until the first appearances of the sequences (i) HHH, and (ii) HTH.
18. Let
N
be a stationary renewal process. Let s be a fixed positive real number, and define X(t) Show that X is a strongly stationary process.
N(s + t) - N(t) for t ::: O.
=
19. Bears arrive in a village at the instants of a renewal process; they are captured and confined at a cost of $c per unit time per bear. When a given number B bears have been captured, an expedition (costing $d) is organized to remove and release them a long way away. What is the long-run average cost of this policy?
111
11 Queues
1 1.2 Exercises.
MIMIl
1. Consider a random walk on the non-negative integers with a reflecting barrier at 0, and which moves rightwards or leftwards with respective probabilities (1 + and 1 I (1 + when at 0, the particle moves to 1 at the next step. Show that the walk has a stationary distribution if and only if < 1 , and in this case the unique such distribution Jl is given by rro = ( 1 - ) , rr = for n 2: 1 .
pi
p)
p);
i
p
i ( 1 _ p 2 )pn - l
p n
2 . Suppose now that the random walker of Exercise ( 1 ) delays its steps in the following way. When at the point n , it waits a random length of time having the exponential distribution with parameter before moving to its next position; different 'holding times' independent of each other and of further information concerning the steps of the walk. Show that, subject to reasonable assumptions on the the ensuing continuous-time process settles into an equilibrium distribution v given by = for some appropriate constant
are
On
Vn
On , Crrn lOn
C.
By applying this result to the case when 00 = A, distribution of the M(A)/M(/1,)/1 queue is = (1
On = A + JL for n 2: 1 , deduce that the equilibrium p )p n , n 2: 0, where p = AI JL < 1 . 3 . Waiting time. Consider a M(A)/M(JL)/l queue with p = A IJL satisfying p < 1 , and suppose that the number of people in the queue at time 0 has the stationary distribution rrn = (1 p)pn , Vn
O. Q(O)
-
_
n 2: Let W be the time spent by a typical new arrival before he begins his service. Show that the distribution of W is given by ll:'(W x) = 1 for x 2: 0, and note that ll:'(W = 0) = 1 -
::::
i
4.
p.
pe-X(!-L-A) are
A box contains red balls and j lemon balls, and they drawn at random without replacement. Each time a red (respectively lemon) ball is drawn, a particle doing a walk on 1 , 2, . . . moves one step to the right (respectively left); the origin is a retaining barrier, so that leftwards steps from the origin suppressed. Let rr (n ; j) be the probability that the particle ends at position n, having started at the origin. Write down a set of difference equations for the rr (n ; j), and deduce that
are
i,
i, j) = A (n ; i, j) - A (n + 1 ; i, j) n where A (n ; i, j) = (�) / ( i � ) . Let
Q
where the rr (n ; 6. of
=
for
i :::: j + n =
I,
J)
Show that
.,
l .
. J ,.
i, j) are given in the previous exercise.
Let be the length of an M(A)/M(JL)/1 queue at time and let = Explain how the stationary distribution of may be derived from that of
Q . Q(t)
= n ) satisfies
Q(O) O. Pn (t) lI:'( Q(t) Pn(t) i,"j ?O . . ( At)i e-At) ( JLt)j e-!-Lt ) Q t, Z {ZnZ,}
be a M(A)/M(JL)/1 queue with
= L...J rr (n ;
}
i,
rr (n ;
5.
{O,
1 12
be the jump chain and vice versa.
GIGI1
Exercises
[11.2.7]-[11.5.2]
7. Tandem queues. 1\vo queues have one server each, and all service times are independent and exponentially distributed, with parameter JLi for queue Customers arrive at the first queue at the instants of a Poisson process of rate A « min {JL , JL 2 n, and on completing service immediately enter the second queue. The queues are in equilibrium. Show that:
i.
1
(a) the output of the first queue is a Poisson process with intensity A, and that the departures before time t are independent of the length of the queue at time t , (b) the waiting times o f a given customer i n the two queues are not independent.
1 1 .3 Exercises. MlG!l = Ad < 1 . Show that the mean queue length at moments of
p -p)p -p) . - s)
1. Consider M(A)ID(d)/I where departure in equilibrium is (2
�
I(1
2. Consider M(A)IM(JL)/1 , and show that the moment generating function of a typical busy period is given by (A + JL - (A + JL 4AJL MB (S ) = 2A
for all sufficiently small but positive values of
V
- s)2 -
s.
3. Show that, for a MlG/l queue, the sequence of times at which the server passes from being busy to being free constitutes a renewal process.
1 1.4 Exercises. GlMIl
E«JLX) j e - f.L X
1. Consider GIM(JL)Il , and let = Ij !) where is a typical interarrival time. Suppose the traffic intensity is less than 1 . Show that the equilibrium distribution 1C of the imbedded chain at moments of arrivals satisfies
p
(Xj
X
00
7rn = l::: (Xi 7rn+i - 1 for n � 1 . i=O Look for a solution of the form 7rn = o n for some 0 , and deduce that the unique stationary distribution is given by 7rj = (1 - TJ)TJ j for j � 0, where TJ is the smallest positive root of the equation S = MX (JL (s - 1)).
2. x
Consider a GIM(JL )/1 queue in eqUilibrium. Let TJ be the smallest positive root of the equation
= MX (JL (x - 1 )) where Mx is the moment generating function of an interarrival time. Show that
I the mean number of customers ahead of a new arrival is TJ ( l - TJ) - , and the mean waiting time is I TJ{JL(1 - TJ)} - .
3. Consider D(1 )IM(JL)1l where JL > 1 . Show that the continuous-time queue length Q (t) does not converge in distribution as t even though the imbedded chain at the times of arrivals is ergodic.
� 00,
1 1.5 Exercises. G/G/l 1. Show that, for a G/G/l queue, the starting times of the busy periods of the server constitute a renewal process. 2. Consider a GIM(JL)/1 queue in equilibrium, together with the dual (unstable) M(JL)/GIl queue. Show that the idle periods of the latter queue are exponentially distributed. Use the theory of duality
1 13
[11.5.3]-[11.7.5]
Queues
Exercises
of queues to deduce for the former queue that: (a) the waiting-time distribution is a mixture of an exponential distribution and an atom at zero, and (b) the equilibrium queue length is geometric.
G/M(JL)I1,
G
- X where S and X are typical Wiener-Hop! equation
3. Consider and let be the distribution function of S (independent) service and interarrival times. Show that the
F(x) =
lXoo F(x - y) dG(y), F x
1 1.6 Exercise.
where
::::
O. Here,
TJ
X is the moment generating
Heavy traffic
1. Consider the queue with = equilibrium queue distribution, and show that distribution being exponential with parameter
M(>")/M(JL)/1
- TJe -/L ( I - TJ)x , x
=
M (JL(x -F1)),(x) 1 M
for the limiting waiting-time distribution is satisfied by is the smallest positive root of the equation = X function of
X.
x :::: 0,
p(1 -p)>"/JL
1.
< Let Qp be a random variable with the Q p converges in distribution as t the limit
1.
p
1,
1 1.7 Exercises. Networks of queues 1. Consider an open migration process with c stations, in which individuals arrive at station j at rate individuals move from i to j at rate (nj), and individuals depart from i at rate (nj), where nj denotes the number of individuals currently at station i . Show when cPj n = nj for all i that the system behaves as though the customers move independently through the network. Identify the explicit form of the stationary distribution, subject to an assumption of irreducibility, and explain a connection with the Bartlett theorem of Problem (8.7.6).
Vj ,
>"jj c/Jj
Q
M(>")/M(JL)/s
( j)
Q
SJL,
JLjc/Jj
2. Let and assume is in equilibrium. Show that queue where >.. < be an the process of departures is a Poisson process with intensity >.. , and that departures up to time are independent of the value of
t
Q(t).
3. Customers arrive i n th e manner o f a Poisson process with intensity >.. i n a shop having two servers. The service times of these servers are independent and exponentially distributed with respective parameters JL I and Arriving customers form a single queue, and the person at the head of the queue moves to the first free server. When both servers are free the next arrival is allocated a server chosen according to one of the following rules:
JL2 .
,
(a) each server is equally likely to be chosen, (b) the server who has been free longer is chosen.
JL
JL2 ,
Assume that >.. < I + and the process is in equilibrium. Show in each case that the process of departures from the shop is a Poisson process, and that departures prior to time t are independent of the number of people in the shop at time t.
M(>")/M(JL)/1
4. Difficult customers. Consider an queue modified so that on completion of service the customer leaves with probability 8, or rejoins the queue with probability 8. Find the distribution of the total time a customer spends being served. Hence show that equilibrium is possible if >.. < 8 and find the stationary distribution. Show that, in eqUilibrium, the departure process is Poisson, but if the rejoining customer goes to the end of the queue, the composite arrival process is not Poisson.
1
-
JL,
5. Consider an open migration process in eqUilibrium. If there is no path by which an individual at station k can reach station j , show that the stream of individuals moving directly from station j to station k forms a Poisson process.
1 14
Problems
Exercises
[11.8.1]-[11.8.7]
1 1.8 Problems 1. Finite waiting room . Consider M(J... )/M(J.L)lk with the constraint that arriving customers who see customers in the line ahead of them leave and never return. Find the stationary distribution of queue length for the cases k 1 and k
N
=
= 2.
n p(n) p(n) = (n Find the stationary distribution 1C of queue length if p(n) = 2 - n , and show that the probability
2. Baulking. Consider M(J... )/M(J.L)/1 with the constraint that ifan arriving customer sees customers in the line ahead of him, he joins the queue with probability and otherwise leaves in disgust. (a) Find the stationary distribution of queue length if + 1)-1 .
(b)
that an arriving customer joins the queue (in equilibrium) is J.L ( l - Iro)/J... .
3. Series. In a Moscow supermarket customers queue at the cash desk to pay for the goods they want; then they proceed to a second line where they wait for the goods in question. If customers arrive in the shop like a Poisson process with parameter J... and all service times are independent and exponentially distributed, parameter J.Ll at the first desk and J.L2 at the second, find the stationary distributions of queue lengths, when they exist, and show that, at any given time, the two queue lengths are independent in equilibrium.
4. Batch (or bulk) service. Consider M/G/ 1 , with the modification that the server may serve up to m customers simultaneously. If the queue length is less than m at the beginning of a service period then she serves everybody waiting at that time. Find a formula which is satisfied by the probability generating function of the stationary distribution of queue length at the times of departures, and evaluate this generating function explicitly in the case when m and service times are exponentially distributed.
=2
5. Consider M(J... )/M(J.L)/l where J... < J.L. Find the moment generating function of the length B of a typical busy period, and show that E(B) (J.L - J... ) - 1 and var(B) (J... + J.L)/(J.L - J... ) 3 . Show that the density function of B is
=
=
where It is a modified Bessel function. 6. Consider M(J... )/G/I in equilibrium. Obtain an expression for the mean queue length at departure times. Show that the mean waiting time in equilibrium of an arriving customer is ! J... E (S2 ) / ( l - p) where S is a typical service time and p J... E (S) .
=
Amongst all possible service-time distributions with given mean, find the one for which the mean waiting time is a minimum. 7. Let Wt be the time which a customer would have to wait in a M(J... )/G/1 queue if he were to arrive at time Show that the distribution function lP'(Wt :s satisfies
F(x ; t) =
t.
ofot ofox F(x , t) x)H(x)x 0, x t 00, (0, 00) .
- = - - J... F + J... lP' (Wt +
x) S x) :s
where S is a typical service time, independent of Wt . Suppose that
-+
for all
0 = h - J... H + J...lP' (U + S :s for and h is the density function of H on
>
satisfies
as -+ where H is a distribution function satisfying where U is independent of S with distribution function H, Show that the moment generating function Mu of U _
Mu (0 ) -
(1 - p)O J... + 0 - J... Ms (O )
where p is the traffic intensity. You may assume that lP'(S
1 15
= 0) = O.
[11.8.8]-[11.8.14]
Queues
Exercises
8. Consider a G/GIl queue in which the service times are constantly equal to 2, whilst the interarrival times take either of the values 1 and 4 with equal probability Find the limiting waiting distribution.
�.
time
9. Consider an extremely idealized model of a telephone exchange having infinitely many channels available. Calls arrive in the manner of a Poisson process with intensity A, and each requires one channel for a length of time having the exponential distribution with parameter J1" independently of the arrival process and of the duration of other calls. Let be the number of calls being handled at time and suppose that (0) =
t,
Q
Q(t)
I.
Determine the probability generating function of the limiting distribution of Q(t) as t
� 00.
Q(t), and deduce lE(Q(t)), lP'(Q(t)
Assuming the queue is in equilibrium, find the proportion of time that no channels and the mean length of an idle period. Deduce that the mean length of a busy period is
=
0), and
are occupied,
(eA liL - 1)/,1,..
10. Customers arrive in a shop in the manner of a Poisson process with intensity A, where 0 < A < 1. They are served one by one in the order of their arrival, and each requires a service time of unit length. Let Q(t) be the number in the queue at time t. By comparing Q(t) with Q(t + 1 ) , determine the limiting distribution of Q(t) as t (you may assume that the quantities in question converge). Hence show that the mean queue length in eqUilibrium is ,1,. ( 1 - i A)/(1 - A).
� 00
Let W be the waiting time of a newly arrived customer when the queue is in equilibrium. Deduce from the results above that lE(W) = � A / ( 1 - A). 11. Consider M(A)ID(l)/l , and suppose that the queue is empty at time O. Let T be the earliest time at which a customer departs leaving the queue empty. Show that the moment generating function of T satisfies = - ,1,.) ( 1 log 1 + log
Mr
( f)
Mr (s)
Mr (s)) ,
(s
and deduce the mean value of T, distinguishing between the cases A < 1 and A :::: 1 . 12. Suppose A < J1" and consider a M(A)/M(J1,)11 queue
(a) Show that
Q is a reversible Markov chain.
Q in eqUilibrium.
(b) Deduce the eqUilibrium distributions of queue length and waiting time.
(c) Show that the times of departures of customers form a Poisson process, and that Q(t) is indepen dent of the times of departures prior to t.
K
(d) Consider a sequence of single-server queues such that customers arrive at the first in the manner of a Poisson process, and (for each j) on completing service in the jth queue each customer moves to the (j + l )th. Service times in the jth queue are exponentially distributed with parameter J1,j ' with as much independence as usual. Determine the Ooint) equilibrium distribution of the queue lengths, when A < J1,j for all j . 13. Consider the queue M(A)/M(J1,)lk, where k :::: 1 . Show that a stationary distribution and only if A < kJ1" and calculate it in this case.
Jr
exists if
Suppose that the cost of operating this system in equilibrium is Ak + B
00
l:: (n - k + 1)JZ'n ,
n =k
the positive constants A and B representing respectively the costs of employing a server and of the dissatisfaction of delayed customers. Show that, for fixed J1" there is a unique value ,1,. * in the interval (0, J1,) such that it is cheaper to have k = 1 than k = 2 if and only ir A < ,1,. * . 14. Customers arrive in a shop in the manner of a Poisson process with intensity A. They form a single queue. There are two servers, labelled 1 and 2, server i requiring an exponentially distributed
1 16
Problems
Exercises
[11.8.15]-[11.8.19]
time with parameter J.Li to serve any given customer. The customer at the head of the queue is served by the first idle server; when both are idle, an arriving customer is equally likely to choose either. (a) Show that the queue length settles into equilibrium if and only if A < J.L l + J.L 2 . (b) Show that, when in equilibrium, the queue length is a time-reversible Markov chain. (c) Deduce the equilibrium distribution of queue length. (d) Generalize your conclusions to queues with many servers. 15. Consider the D(I )/M(J.L)/1 queue where J.L > 1 , and let Qn be the number of people in the queue just before the nth arrival. Let Qf.L be a random variable having as distribution the stationary l distribution of the Markov chain { Q n }. Show that ( 1 - J.L- ) Q f.L converges in distribution as J.L ,!.. 1 , the limit distribution being exponential with parameter 2.
1:,
16. Taxis arrive at a stand in the manner of a Poisson process with intensity and passengers arrive in the manner of an (independent) Poisson process with intensity 7r . IT there are no waiting passengers, the taxis wait until passengers arrive, and then move off with the passengers, one to each taxi. IT there is no taxi, passengers wait until they arrive. Suppose that initially there are neither taxis nor passengers at the stand. Show that the probability that n passengers are waiting at time t is 1
n
(7r 11:p e- (7r + r )t In (2t ..jWi), where In (x) is the modified Bessel function, i.e., the coefficient of z n in the power series expansion of exp{ � x (z + z }.
-1)
17. Machines arrive for repair as a Poisson process with intensity A . Each repair involves two stages, the ith machine to arrive being under repair for a time X i + Yi , where the pairs (Xi , Yi ) , i = 1 , 2, . . . , are independent with a common joint distribution. Let U (t) and V (t) be the numbers of machines in the -stage and Y -stage of repair at time t. Show that U (t) and V (t) are independent Poisson random variables.
X
18. Ruin. An insurance company pays independent and identically distributed claims { Kn : n :::: I } at the instants of a Poisson process with intensity A , where AlE(Kl ) < 1 . Premiums are received at constant rate 1 . Show that the maximum deficit the company will ever accumulate has moment generating function (1 . lE(eO M ) = A + _ AlE(eO K )
M p)8 8
19. (a) Erlang's loss formula. Consider M(A)/M(J.L)/s with baulking, in which a customer departs immediately if, on arrival, he sees all the servers occupied ahead of him. Show that, in eqUilibrium, the probability that all servers are occupied is
7rs =
"'� L.. ={} P i lJ' ".
where P = AI J.L .
J
(b) Consider an M(A)/M(J.L)/oo queue with channels (servers) numbered 1 , 2, . . . . O n arrival, a customer will choose the lowest numbered channel that is free, and be served by that channel. Show in the notation of part (a) that the fraction Pc of time that channel c is busy is Pc = P (7rc - l - 7rc) for c :::: 2, and pl = 7rl ·
1 17
12 Martingales
12.1 Exercises. Introduction
(Y, !F) is a martingale, show that JE(Yn) = JE(Yo) for all n . (Y, !F) i s a submartingale (respectively supermartingale) with finite means, show that JE(Yn ) � JE(Yo) (respectively JE(Yn) � JE(Yo». � o. 2. Let (Y, !F) be a martingale, and show that JE(Yn +m I :Fn ) = Yn for all n, 3. Let Zn be the size of the nth generation of a branching process with Zo = 1, having mean family size JL and extinction probability 7] . Show that Zn JL - n and 7]zn define martingales. 4. Let {Sn : n � o } be a simple symmetric random walk on the integers with So = k. Show that Sn and S; n are martingales. Making assumptions similar to those of de Moivre (see Example ( 1 2. 1 .4» , 1.
(i) If
(ii) If
m
-
find the probability of ruin and the expected duration of the game for the gambler's ruin problem.
S. Let (Y, !F) be a martingale with the property that JE(Y;) < 00 for all n. Show that, for i � j � k, JE{(Yk - lj )Yj } = 0, and JE{(Yk Yj ) 2 I .ft } = JE(yl l .ft) - JE(Y] I .ft). Suppose there exists K such that JE(Y;) � K for all n . Show that the sequence {Yn } converges in mean square as n -+ 00. 6. Let Y be a martingale and let u be a convex function mapping JR to JR. Show that (u(Yn ) : n � o } is a submartingale provided that JE(u(Yn) + ) < 00 for all n . Show that I Yn l, Y;, and Y;i constitute submartingales whenever th e appropriate moment condi -
tions are satisfied.
Y be a submartingale and let u be a convex non-decreasing function mapping JR to JR. Show (u(Yn) : n � o} is a submartingale provided that JE(u(Yn) + ) < 00 for all n . Show that (subject to a moment condition) Y;i constitutes a submartingale, but that IYn I and Y;
7. Let that
need not constitute submartingales.
X
S p. S JR L,jES Pij 1/l (j) i E S. Show that J... -n 1/l(Xn ) constitutes a supermartingale. 9. Let G n (s) be the probability generating function of the size Zn of the nth generation of a branching process, where Zo = 1 and var(Zl ) O. Let Hn be the inverse function of the function G n , viewed as a function on the interval [0, 1 ] , and show that Mn = {Hn (s )}zn defines a martingale with respect to the sequence Z.
8. Let be a discrete-time Markov chain with countable state space and transition matrix Suppose that 1/1 : -+ is bounded and satisfies � J... 1/I (i) for some J... > 0 and all
>
1 18
Crossings and convergence
Exercises
[12.2.1]-[12.3.4]
12.2 Exercises. Martingale differences and Hoeffding's inequality
1. Knapsack problem. It is required to pack a knapsack to maximum benefit. S\1ppose you have n where Vl , V2 , . . . , Vn , Wl , W2 , . . . , Wn are objects, the i th object having volume and worth independent non-negative random variables with finite means, and ::: M for all i and some fixed M. Your knapsack has volume c, and you wish to maximize the total worth of the objects packed in it. That is, you wish to find the vector Z l , Z2 , . . . , Zn of O's and l 's such that ::: c and which maximizes Let Z be the maximal possible worth of the knapsack's contents, and show that lP'(I Z - EZ I � x) ::: 2 exp{-x 2 / (2n M2 ) } for x > o.
Wi ,
Vi
Wi
�'{ Z i Vi
�'{ Zi Wi .
2. Graph colouring. Given n vertices V l , V2 , . . . , Vn , for each 1 ::: i < j ::: n we place an edge between Vi and Vj with probability p; different pairs are joined independently of each other. We call Vi and V neighbours if they are joined by an edge. The chromatic number X of the ensuing graph is the minimal number of pencils of different colours which are required in order that each vertex may be coloured differently from each of its neighbours. Show that - EX 1 � x) ::: 2 exp{ - i x 2 / n } for x > O.
j
lP'(l x
12.3 Exercises. Crossings and convergence
1. Give a reasonable definition of a downcrossing of the interval Yo , Yl , . . . .
[a, b)
by the r3l)dom sequence
(a) Show that the number of downcrossings differs from the number of upcrossings by at most 1 . (b) If (Y, fF) is a submartingale, show that the number Dn (a, b ; Y ) of downcrossings of [a, b ) by Y up to time n satisfies E{(f: - b) + } EDn (a , b; Y) ::: .
;- a
2. Let (Y, fF) be a supermartingale with finite means, and let Un (a, ings of the interval [a, b) up to time n. Show that
b; Y) be the number of upcross
b; Y) ::: a / (b - a) if Y is non-negative and a � o . 3. Let X be a Markov chain with countable state space S and transition matrix P. Suppose that X is irreducible and persistent, and that 1/1 S � S is a bounded function satisfying �j ES Pij 1/I (j) ::: 1/I (i ) for i E S. Show that 1/1 i s a constant function. 4. Let Z l , Z2 , . . . be independent random variables such that: Deduce that EUn (a ,
:
with probability
� n -2 ,
with probability 1 - n - 2 , with probability i n - 2 , where a l = 2 and an = 4 aj . Show that Yn = j l Zj defines a martingale. Show that Y = lim Yn exists almost surely, but that there exists no M such that E I Yn 1 ::: M for all n .
�7:t
�=
1 19
[12.4.1]-[12.5.5]
Martingales
Exercises
12.4 Exercises. Stopping times 1. If TI and T2 are stopping times with respect to a filtration :F, show that TI + T2 , max{TI , T2 }, and min { TI , T2 } are stopping times also.
2.
=
Let X l , X2 , ' " be a sequence of non-negative independent random variables and let N (t) max {n : X l + X2 + . . . + X ::: t } . Show that N (t) + is a stopping time with respect to a suitable filtration to be specified.
3.
n
I
(Y, :F) be a submartingale and
Let
max Ym �
x
)
:::
(Y, :F) be a non-negative supermartingale and
Let
lP'
5.
> O. Show that
(O::;:m::;:n
lP'
4.
x
(Y,
(O::;:m::;:n
max Ym �
x
)
x
:::
.!. lE(Y,i ) . x
> O. Show that
.!. lE( Yo ) . x
Let :F) be a submartingale and let S and T be stopping times satisfying 0 ::: S ::: T ::: N for some deterministic N. Show that lEYo ::: lEYs ::: . lE
lEYT ::: YN Let { Sn } be a simple random walk with So = 0 such that 0
6. Moivre's martingale to show that lE(suPm Sm ) ::: be replaced by an equality.
p/(l - 2p).
1) !. Use de lP'(SI Show further that this inequality may
0 and all n . Show that (YT An, :Fn) is a uniformly integrable martingale for any
Let :F) be a martingale. finite stopping time T such that either: (a)
< (0) =
lE(I Yn II{T> n j)
....
0 as n
....
00,
or
(b) { Yn } is uniformly integrable.
3.
(Y,
Let :F) be a uniformly integrable martingale, and let S and T be finite stopping times satisfying S ::: T. Prove that lE(Yoo :F ) and that s :Fs) , where Yoo is the almost sure limit as n .... of
00
Yn . :n�
YT =
I T
Y = lE(YT I
4. Let { Sn O} be a simple symmetric random walk with 0 So N and with absorbing barriers at 0 and N. Use the optional stopping theorem to show that the mean time until absorption is lE{So (N - So ) } · 5.
Let { Sn
<
U Ai 1
5.
We have that
p :::: n
7.
1 - 3JP>(AI ) + 3JP>(AI n A2) - JP>(AI n A2 n A 3)
=
1 - 3(�)6 + 3(�)6 (�)6. _
By the continuity of JP>, Exercise ( 1 .2. 1), and Problem
(
-
I , and �
1
=
JP>
(U Ar ) 1
n(n - l)q np =
Since at least one of the
1 = JP>
1
=
)
=
L JP>(Ar) - L JP>(Ar n As ) r r<s - 1 ::::: - 1.
Ar occurs,
(U Ar )
(
)
=
np - i n(n
- 1 )q . Hence
n
L JP>(Ar ) - L JP>(A r n As ) + L JP>(Ar n As n At ) r<s r<s U (Ar n As » r<s
(1.8. 1 1),
n � JP>(A�) = 1 . 1 - nlim JP> U A� :::: 1 - nlim L-.... oo .... oo r=I r= I
=
6.
=
=
L JP>(Ar n As ) - � r<s
i, L JP>(Ar n As n A t n A u ) + . . . r<s t(A) JP>(A) B) JP>(A I B) - JP>(A n - JP>(BJP>(A)A) JP>(B) - JP>(B I A) JP>(B) JP>(B) _
_
1 37
4/n.
.
[1.4.2]-[1.4.5] if lP' (A) lP' (B) =f:
Events and their probabilities
Solutions
O. Hence
lP'(A
I B)
lP'(B
lP'(A)
I A) ,
lP'(B)
whence the last part is immediate.
2.
Set Ao = n for notational convenience. Expand each term on the right-hand side to obtain
t
3.
R
Let M be the event that the first coin is double-headed, the event that it is double-tailed, and N the event that it is normal. Let H be the event that the lower face is a head on the ith toss, T� the event that the upper face is a tail on the i th toss, and so on. Then, using conditional probability we find:
nauseam, (i)
h � / I + ! / I R) + � /n J I J / / I J I J +� ?I J I J) I J + 1 (1 =
lP'(H
(ii)
lP' ( H,1
(iii)
lP'(H
(iv) 2 lP'(HI
I
M)
lP'(H
ad
lP'(Hll
lP'(H
� ��
I N) = + 0 +
lP'(H H ) lP'(M) I 2 3 2 -- Hu ) ) lP'(H lP'(H ) - 5" 5" - 3 · H ) = l · lP'(M H ) i = lP'(Hl H )
I HuI n Hu2 ) --
lP' ( HI2
-
n Hul n Hu2 ) lP'(H n H )
!
H ) I lP'(H1 H ) =
lP'(N
J J
-
.
=
l
j + � . 1 = �.
lP'(M)
1 . lP'(M) + ! . lP'(N)
-
(v) From (iv), the probability that he discards a double-headed coin is
�
� + Ih "
--
4 - 5· -
�, the probability that he
discards a normal coin is . (There is of course no chance of it being double-tailed.) Hence, by conditioning on the discard,
j
4. The final calculation of refers not to a draw of one ball from an urn containing three , but rather to a composite experiment comprising more than one stage (in this case, two stages). While it is true that {two black, one white} is the only fixed collection of balls for which a random choice is black with probability the composition of the urn is prior to the final draw. After all, if Carroll ' s argument were correct then it would apply also in the situation when the urn
single
j,
not determined
originally contains just one ball, either black or white . The final probability is now i , implying that the original ball was one half black and one half white ! Carroll was himself aware of the fallacy in this argument.
5.
(a) One cannot compute probabilities without knowing the rules governing the conditional prob abilities. If the first door chosen conceals a goat, then the presenter has no choice in the door to be opened, since exactly one of the remaining doors conceals a goat . If the first door conceals the car, then a choice is necessary, and this is governed by the protocol of the presenter. Consider two 'extremal ' protocols for this latter situation . (i) The presenter opens a door chosen at random from the two available. (ii) There is some ordering of the doors (left to right, perhaps) and the presenter opens the earlier door in this ordering which conceals a goat . Analysis of the two situations yields
p
=
� under (i), and 138
p
=
� under (ii).
Independence
Solutions
[1.4.6]-[1.5.3]
[�, �
Let ex E 1, and suppose the presenter possesses a coin which falls with heads upwards with probability = He flips the coin before the show, and adopts strategy (i) if and only if the coin shows heads. The probability in question is now + = ex . You never lose by swapping, but whether you gain depends on the presenter ' s protocol.
13
6a - 3.
� 13 � (1 - 13)
D denote the first door chosen, and consider the following protocols: (iii) If D conceals a goat, open it. Otherwise open one of the other two doors at random. In this case = O. (iv) If D conceals the car, open it. Otherwise open the unique remaining door which conceals a goat. In this case = 1.
(b) Let
p
p
As in part (a), a randomized algorithm provides the protocol necessary for the last part.
6. 7.
This is immediate by the definition of conditional probability. Let Cj be the colour of the i th ball picked, and use the obvious notation.
(a) Since each be any of the
urn contains the same number
n(n - 1)
n-1
of balls, the second ball picked is equally likely to available. One half of these balls are magenta, whence ( C = =
lP'
(b) By conditioning on the choice of urn,
lP'(C2 = M I C l = M) = lP'(ClP'(lC, IC=2 =M)M) =
2
tr=1 (nn(n- r)(n- l)(n --2)1) /�2 - r
M)
=
�.
�
3.
1.5 Solutions. Independence
1.
Clearly
lP'(AC n B) = lP'(B \ { A n B}) = lP'(B) - lP'(A n B) = lP'(B) - lP'(A)lP'(B) = lP'(AC)lP'(B). For the final part, apply the first part to the pair B, AC 2. Suppose i j and m n. If j m, then Aij and Amn are determined by distinct independent •
lP' (A)
= lP' (A
> lP'(B I C)
+ lP'(A I CC ) lP'(C C ) > lP'(B I C)lP'(C) + lP'(B I C C )lP'(C c ) = lP'(B) . I C)lP'(C)
The other author is more suspicious of the question, and points out that there is a difficulty arising from the use of the word 'you ' . In Example Simpson ' s paradox, whilst drug I is preferable to drug II for both males and females, it is drug II that wins overall.
(1.7. 10),
5.
L k be the label of the kth card. Then, using symmetry, k= lP'(L k = m i L k > L r for 1 :s < ) = lP'(L >lP'(L k L r £or 1 :s
Let
m)
r k
1
=-
/ -1 = ki m.
r k m k
(exactly two heads) =
2 heads and (�)2 - n .
(�)
n-2
tails. Each sequence has probability 2 -n ,
(d) Clearly Jl>(at least
2 heads) = 1
-
Jl>(no heads) - Jl>(exactly one head) =
(1 .2.1)): ni A i
1
- 2-n - (�) 2-n •
A 3. (a) Recall De Morgan ' s Law (Exercise = the complement of a countable union of complements of sets in :F.
(Ui �t , which lies in J=' since it is
(b) Je is a a -field because: (i) 0 E J=' and 0 E
9.; therefore 0
E Je .
(ii) I f A I , A 2 , . . . i s a sequence o f sets belonging to both J='and 9., then their union lies i n both J='and 9., which is to say that Je is closed under the operation of taking countable unions. (iii) Likewise A C is in Je if A is in both J=' and 9.. (c) We display a n example. Let Q=
{a,9.c}
Then Je = J='U but the union
4.
{a, b, c},
{ {a}, {b, c}, } 9. { {a, b}, {c}, Ha}, {c}, {a, b}, {b, c}, } {a}
J=' =
0, Q ,
=
}
0, Q .
is given by Je = 0, Q . Note that is not in Je, which is therefore not a a-field.
E Je and
{c}
E Je,
In each case J='may be taken to be the set of all subsets of Q, and the probability of any member of J=' is the sum of the probabilities of the elements therein. (a) Q = {H, T} 3 , the set of all triples of heads (H) and tails (T). With the usual assumption of
h
t 3 -h
ph (1 p)t
_ independence, the probability of any given triple containing heads and = tails is , where is the probability of heads on each throw. (b) In the obvious notation, Q = {U, V} 2 = {UU, VV, UV, VU} . Also Jl>(UU) = Jl>(VV) = . and Jl>(UV) = Jl>(VU) = . �. (c) Q is the set of finite sequences of tails followed by a head, {Tn H : together with the infinite sequence TOO of tails. Now, Jl>(Tn H) = and Jl>(Too ) = limn .... = if #-
p
� �
�
(1
- p)n p,
n � O}, - p)n oo {1
5.
As usual, Jl>(A /::,. B) = Jl> ( A U B) \ Jl>(A n B) = Jl>(A U B) - Jl>(A n B ) .
6.
Clearly, by Exercise
)
0
p
O.
(1.4.2),
Jl>(A U B U C) = Jl> ( A C n B C n C C ) C = Jl>(AC n B c n C C ) = Jl>(A c I B C n C C )Jl>(B c I C C )Jl>(C c ) .
1
7.
)
-
1
-
0
0
(b) If Jl>(A) = then = Jl>(A n B ) = Jl>(A)Jl>(B) for all B . If Jl>(A) = that Jl>(A n B) = Jl>(A)Jl>(B ) .
8.
0 or 1. 1 then Jl>(A n B ) = Jl>(B), so
(a) If A i s independent of itself, then Jl>(A) = Jl>(A n A ) = Jl>(A) 2 , s o that Jl>(A) =
Q U 0 = Q and Q n 0 = 0, and therefore that Jl>(0) =
O.
1 = Jl> ( Q U 0) = Jl>(Q) + Jl>(0) = 1 + Jl>(0) , implying
O.
9.
i I} 1 .
(i) Q (0) = Jl>(0 I B) = Also Q (Q) = Jl>(Q I B) = Jl>(B)/Jl>(B) = are disjoint members of :F, (ii) Let A I , A 2 , be disjoint members of :F. Then { A i n B implying that
: �
• . .
142
Problems
Solutions
[1.8.10]-[1.8.13]
Q is a probability measure, n C) = P(A n C I B) = P(A n B n C) = P(A I B n C) . Q(A I C) = Q(AQ(C) P (B n C) P (C I B) The order of the conditioning (C before B, or vice versa) is thus irrelevant. Finally, since
10.
As usual,
11.
The first inequality is trivially true if
n :::; m. Then
:::; P
n = 1. Let m :::: 1 and assume that the inequality holds for
(U1 Ai ) + P(Am+1 ) :::; 'I:1 P(Ai) ,
by the hypothesis. The result follows by induction. Secondly, by the first part,
12.
P
We have that
(0 Ai ) = p ( (Y Afr) = 1 - P (Y Af)
(n1 Af) n) + ' " U Aj) _ ( (Ai = 1 - n + L. P (Ai) + (n2 ) - L P 3 .. + (-It (: ) - (-l) n p (Y Ai ) = (1 - 1) n + � P(Ai) - . . . - (_1) n p (Y Ai ) = 1 - � P(AD + � P(Af n Ai ) - . . . + (-l) n p I
by Exercise
(1.3.4)
I <J
I <J
I
using De Morgan ' s laws again
by the binomial theorem.
13.
(
Clearly,
)
L p n Ai n Ai . SS;; ( S1 , 2, k... , n j i eS us I I= For any such given S, we write A s = n i eS Ai . Then p n Ai n Ai = P(A s ) - L P (A su{j } ) + L P(A su{j ,k} ) - . . . j lP(B) . lP(A I BC) = lP(A n BC)/lP(BC) = {lP(A) - lP(A n B)}/lP(BC) < lP(A). (c) No. Consider the case A n C = 0.
29. (b)
30.
365m ;
The number of possible combinations of birthdays of m people is the number of combina tions of different birthdays is m) ! . Use your calculator for the final part.
32.
365! / (365 -
In the obvious notation, lP(wS, xH, yD, zC) =
(!;) (�) (�) (�3) / (W . Now use your calcula
tor. Thrning to the 'shape vector ' (w , x , y , z) with w ::: x ::: y ::: z, lP(w, x , y , z)
=
{
41P(wS, xH, yD, zC)
if w =j:. x
1 21P(wS, xH, yD, zC)
if w
on counting the disjoint ways of obtaining the shapes in question.
33.
Use your calculator, and divide each of the following by
148
(552) .
= y = z,
= x =j:. y =j:. z,
Problems 34.
Solutions
[1.8.34]-[1.8.36]
Divide each of the following by 65 . 6! 5 ! 3 ! (2 !) 2 ' 6!, 6! 5 ! (5 !) 2
6! 5 ! ! 3 ( 2! ) 3 ' 6! 5 !
6! 5 ! 2 ! (3 !) 2 ' 6! 5 ! (4 !) 2 '
·
8r
35. Let denote the event that you receive r similar answers, and T the event that they Denote the event that your interlocutor is a tourist by V. Then T n Vc 0, and
=
are correct.
n V n 8r ) = lP'(T n 8r I V)lP'(V) . lP'(T I 8r ) = lP'(T lP'(8 lP'(8r ) r) Hence:
81 ) = i V I = ! . 82) = ( i )2 . V [{ In the latter case, passengers are correctly seated. We obtain thus that �k
Therefore
for
�k
y and 1 if b ::; y .
Assume that any specified sequence o f heads and tails with length n has probability are exactly such sequences with k heads.
(k)
p -
2 -n . There
If heads occurs with probability then, assuming the independence of outcomes, the probability of any given sequence of k heads and n k tails is ( 1 (1The answer is therefore
p) n -k . (k) pk p ) n -k . 4. Write H = )"F + (1 - )")G. Then 1imx--* - oo H(x) = 0, limx--* oo H(x) = 1 , and clearly H is non-decreasing and right-continuous. Therefore H is a distribution function. S. The function g(F(x)) is a distribution function whenever g is continuous and non-decreasing on [0, 1], with g(O) = 0, g(l) = 1 . This is easy to check in each special case. pk
151
[2.2.1]-[2.4.1]
Random variables and their distributions
Solutions
2.2 Solutions.
The law of averages
thde potenti allyyanswer embarrassed fractith probabi on of thelitypopulp inadependent tion, and suppose thatothereachindisampl esd. iIn1.ndithvLeteidmodi ualp bewoul trut h ful "yes" wi l y of al l v i d ual that someone says yes is p + � (1 - p) = � (1 + p). If the proportiTheoadvantage n offieyes'd procedure, s iofs nowthe gi¢,tvhtenehenchance 2¢ - 1 is a decent estimate of p. that it allonwsg propert individyual. s to answer ' yes'' without their being identified with certainty asprocedure having theisembarrassi 2. Clearly Hn + Tn = n, so that (Hn - Tn)/n = (2 Hn/ n ) - 1. Therefore lP' (2P - 1 - E � (Hn - Tn) 2p - 1 + E) = lP' ( I � Hn - p i n I as n by the law oflarge numbers (2.2.1). 3. LetIn(x )betheindicatorfunctionoftheevent{Xn x). Bythelawofaverages,n - l L�=l Ir(x) converges in the sense of (2.2. 1) and (2.2.6) lP'(Xn x) = F(x). �
�
�
--+
--+ 00,
�
to
�
2.3 Solutions. Discrete and continuous variables
With 8 = supm l am - am - I I , we have that I F (x ) - G(x)1 IF(am ) - F(am - dl I F (x + 8) - F(x -8)1 for x am- I am ). Hence G(x ) approaches F(x) for anyl x at which F is continuous. 2. For y lying in the range of g, {Y y} = {X g - (y)} 3. Certainly Y is a random variable, using the result of the previous exercise (2). Also lP'(Y y) = lP' (F - I (X) y) = lP' (X F(y)) = F(y) red. If F is ndigsconti nuousnuous, then butF- Inot(x)striis cnottly defiincreasi ned forng, althlenx,F-so tIh(x)at Yis inots notalwwelaysl defidefinned.ed Ifunias Frequi iquels non-decreasi and conti y. Such difficulties may be circumvented by defining F - I (x) = inf{y : F(y) x}. 4. The function AI + ( 1 - A)g is non-negative and integrable over lIUo 1. Finally, Ig is not necessarily a density, though it mayoobe: e.g., if 1 = g = 1, 0 x 1 then I(x)g(x) = 1, 0 x 1. 5. (a) d 1(, thenl ) fl cx -d dx = c/(d - 1). Therefore I is a density function if c = d - 1, and F(x)t=y functi 1 -x- d - when this holds. If d 1, then I has infinite integral and cannot therefore be a densi (b) By differenton.iating F(x ) =. eX /( 1 + eX ), we see that F is the distribution function, and c = 1. 1.
�
E [
�
,
�
E
�
�
�
y::
�
�
�
If
�
�
>
�
2.4 Solutions.
Worked examples
(a) If y � 0, lP'(X2 y) = lP'(X -lP'(X (b) We must assume that X � O. If y 0, 1.
�
� ..jY)
0, s not a joint distribution function. (b)so thatIn thiFsicase 88x8y2 F = { e-Y if 0 � x � y, i f O � y � x, 0 and in addition 1000 1000 82 F dx dy = 1 . 8x8y Hence F is a joint distribution function, and easy substitutions reveal the marginals: FX (x) = yl-*imoo F(x , y) = 1 - e-x , x :::: 0, Fy (y) = xl-*imoo F(x , y) = 1 - e-Y - ye-Y , y :::: O. E
E
E
E
k
--
o
0
--
156
Problems
Solutions
[2.7.15]-[2.7.20]
15. Suppose that, for some weof BjhaveandPi consi < Pj and Bi is to the left of Bj. Write m for the posi t i o n of Bi and r for t h e posi t i o n der the effect of interchanging Bi and Bj. For k ::; m and k r, JP(T � k) is unchanged by' the move. For m < k ::; r, JP(T � k) is decreased by an since this ofis tBihe iandncreased probabirablleit.y that the search is successful at the mth position. amount Pj the- Pii,nterchange Therefore Bj is desi Ith tfolhelobooks ws thatappear the onlinydecreasi orderinnggiorder n whicofh probabi JP(T lk)ity.canIn bethereduced forties,noitkisisofthatno iorderi nagncein whi c event of m port how the tied books are placed. 16. Intuitively, it may seem better to go first since the first person has greater choice. This conclusion ichooses s in fact falClse.thenDenote thechoosecoinsCby3 , Cbecause andCl ) suppose you go second. If your opponent l , C2 , CJP3(Cin3order, you beats = � � � = � i . Likewise JP(ChaveI beata bets Cte2r) = JP(Cevens 2 beatschanceC3 ) of= wi� nnin�.g. Whichever coin your opponent picks, you can arrange 17. Various difficulties arise in sequential decision theory, even in simple problems such as this one. The follo"ahead" wing simandple bargument yieldssearches the optim"behial polnd"icy.(ifSuppose thatsearches you havewere madesuccessful unsuccessful searches unsuccessful any of these there is no further problem). Let A be the event that the correct direction is ahead. Then , then knowledge I A)lP(A) JP(A I current know edge - JP(current JP(current knowa edge) (1 - p) a i =1= j ,
>
�
.
+
to
>
>
than
a
I
)
_
I
whi(1 -chp)exceeds i if and only if (1 - p)a a (1 - p) b (1 The opti m al pol i c y i s t o compare a a with (1 - p) b (1 the event of a tie, do either. You search ahead if the former is larger and behind otherwise; in 18. (a) There are (�) possible layouts, of which 8+8+2 are linear. The answer is 18/ ( �) . (b) EachHavirowng chosen and coluwhimnchmustof these contaiisn occupi exactlyed,onetherepawn.are There are 8 possible positions in the first row. 7 positions in the second row which are admissible, 6 in the third, and so one. The answer is 8!1 (�) . 19. (a) The density function is f(x) = F ' (x) = 2xe -x 2 , x � O. The densi t y functi o n i s f(x) = F' (x) = x 2e- 1 /x , x 2O. ' (x) = 2(eX e-x ) - , x (c)(d) TheThisdensi '(1) < O. is nottyafuncti distribonutiios nf(x)functi=oFn because F 20. We have that JP(U = V) = }({(u. v) :u=v) fu. v (u, v) du dv = O. The functrandom ion f vari1]a2bles X,suchY arethatcontinuous but not jointly continuous: there exists no integrable >
a).
- a).
>
(b)
+
:
[0,
--+
IR
E R.
JP(X ::; x, Y ::; y) = }u=O {X }{Yv =O f(u, v)du dv, O ::; x, y ::; l.
157
3 Discrete random variables
3.1 Solutions. Probability mass functions
(a)- 1C - 1 Ef-k 2-k 1. (b) C 1 Ef 2-2 /k 2log 2. (c) C-- 1 Ef kk 1f /6.2 (d) C Ef 2 /k! e - 1. 2. (i) ! ; 1 - (2 10g2) - 1 ; 1 - 61f - 2 ; (e2 - 3)/(e2 - I). (i(i(aii)))lI;It, (I;isb)t1;h1e1-caseand(log3)/( t2.hat JP(Xlog4),even)(c) !.Ef:: (d) Wel JP(Xhave th2k),at and the answers are therefore i (_2)i 1 1 (e2 e-2) - 1 2 22k t:l (2k)! 6J 2(i!) so the answer is ! O - e-2). 3. The number X of heads on the second round is the same as if we toss all the coins twice and count 2, tsoheJP(Xnumberk)whic(Z)p h show2k (1heads on bot h occasi o ns. Each coi n shows heads twi c e wi t h probabi l i t y p n 2 k _p ) - . 4. Let Dk be the number of digits (to base 10) in the integer k. Then 1.
=
=
=
=
=
=
=
=
=
00
�
=
=
00
-
� -
+
-
- :2 -
+
'
=
5. (a) The assertion follows for the binomial distribution because k(n - k) (n - k 1)(k 1). The Poi(b)ssonThiscasefollisowstrifrom vial. the fact that kg ::: (k2 - 1)4 . (c) The geometric mass function f(k) qpk , k ::: o. �
=
1.
3.2 Solutions.
We have that
JP(X 1, Z 1) JP(X 1, =
=
=
=
Independence
Y=
158
1)
=
! = JP(X = I)JP(Z = 1).
+
+
Independence
Solutions
[3.2.2]-[3.2.3]
Thiare sin, dependent. together witHowever h three similar equations, shows that X and Z are independent. Likewise, Y and Z JP(X = 1 , Y = 1 , Z = - I) = 0 =I- � = JP(X = I)JP(Y = I)JP(Z = - 1), X , Y, Z are If x 2: 1, JP ( rnin{X, Y} � x) = I - JP(X x , Y x ) = 1 - JP(X x )JP(Y x)
so that and not independent. 2. (a)
>
>
= 1 - 2 -x · 2 -x = 1 - 4 -x .
>
>
(b) JP(Y X) = JP(Y < X) by symmetry. Also JP(Y X) JP(Y < X) JP(Y = X) = 1 . Since JP(Y = X) = 2: JP(Y = X = x) = 2: 2-x . TX = ! , x we have that JP(Y X) = ! . x (c) ! by part (b). (d) JP(X kY) = 2: JP ( X kY, Y = y ) y =1 >
>
+
+
>
00
2:
2:
00
00
= 2: JP ( X 2: ky , Y = y ) = 2: JP(X 2: ky )JP(Y = y ) y =1 y =1 00 00 '" 2 -ky -x 2 - y = k 21 . = '" �� 2+ _1 y =l x =O
(e)
JP(X
(t) Let 3.
r
= min
divides Y) = k=12: JP(Y = kX) = k=l2: x2:=1 JP(Y = kx , X = x) 00
00 00
�� � T kx T X = � 1 . =� � 2k + 1 _ 1 k=1 x =1 k=1
where m and n are coprime. Then 00
00
k=1
k=1
JP(X = rY) = 2: JP(X = km, Y = kn) = 2: 2 -km Tkn = 2m +n1 _ I ·
(a) We have that
JP(X I < X2 < X3 ) =
2:
(1 - PI )(I - P2 )(1 - P3 ) P�- 1 p� - I p� - I
i<jJ
so the required mean is
n ( I - �) 2n - l . 2n ( 1 - �) 4. Any given red ball is in urn R after stage k if and only if it has been selected an even number of times. The probability of this is �n (�) ( � r ( 1 - � t-m = � { [ ( 1 - � ) + � r + [ ( 1 - � ) - � r }
m
=
� { I + (I - �t } ,
and the mean number of such red balls is n times this probability. 5. Label the edges and vertices as in Figure 3. 1 . The structure function is � (X) = Xs + (1 - Xs ) { (1 - X l )X4 [X3 + (1 - X3 )X2 X6] +X l [X2 + (1 - X2 ) (X3 ( X6 + X4(1 - X6) ) )J ) .
163
[3.4.6]-[3.4.9]
Discrete
Solutions
random variables
2
1
3
s Figure 3 . 1 . The network with source s and sink
t.
For the reliability, see Problem ( 1 .8.19a). 6. The structure function is I{S� k} ' the indicator function of {S :::: k} where S = E�=l reliability is therefore E t=k (7) p i (1 _ p) n - i .
7.
!
Xc. The
Independently colour each vertex livid or bronze with probability each, and let L be the random set of livid vertices. Then L = 1 E There must exist one or more possible values of L which are at least as large as its mean.
EN
!
I.
N
X
S. Let Ir be the indicator function that the rth pair have opposite polarity, so that = 1 + E��t Ir . We have that lP'(lr = 1 ) = and lP'(lr = Ir +l = 1 ) = ! , whence EX = ! (n + 1 ) and var = ! (n - 1 ) .
!
9.
(a) Let A i b e the event that the integer i remains i n the i th position. Then
()
1 n 1 l 1 =n·-+ · · · + (- 1) n - - . n 2 n (n - 1 ) n! Therefore the number M of matches satisfies l I 1 lP'(M = 0) = - - - + . . . + (- 1 ) n - . 2! 3! n! Now lP'(M = r) = = (b)
(; )
lP' (r given numbers match, and the remaining n - r are deranged)
n! (n - r) ! r ! (n - r) ! n!
( � _ � + . . . + ( _ 1 ) n -r 2!
3!
n+l
)
1 _ . (n - r) !
_ _
E #{derangements with 1 in the rth place} r=2 = n {#{derangements which swap 1 with 2} + #{derangements in which 1 is in the 2nd place and 2 is not in the 1 st place } } = ndn- l + ndn ,
dn+ l =
164
X
Dependence
Solutions
[3.5.1]-(3.6.2]
where Set un #A= dndenotes n Un = (_ 1)dnn+, n1 - (n2,+andl )dnhence= - (dn -ndn- l ). and notnalietythofattheU2 set= 1,A. Byobtairearrangement, - ndnthe- l cardi n! dn = -n!2! - -n!3! + . . . + (-1)n -. n! Now divide by n! to obtain the results above. to
�
3.5 Solutions. Examples of discrete variables
1. There are n!/(n l ! n 2 ! ' " nt! ) sequences of outcomes in which the ith possible outcome occurs nj times for each i. The probability of any such sequence is P� p�2 . . . p�t , and the result follows. 2. The total number H of heads satisfies 'An -J.. II"(H = x) = � II"(H = x I N = n)II"(N = n) = � (: ) pX (1 _ p)n -x +, = ('Ap)x!Xe-J.p. n:E=x ('A (I - p)}(n n--xx)!e-J..(1 -p) . The lastersummati paramet 'A(1 - p).on equals 1, since it is the sum of the values of the Poisson mass function with 3. dpn/d'A = Pn - l - Pn where P - l = O. Hence (d/d'A)II"(X ::: n) = Pn('A). 4. The probability of a marked animal in the nth place is a /b . Conditional on this event, the chance of n - 1 preceding places containing m 1 marked and n -m unmarked animals is 1
00
00
00
(am -- Il ) (bn -m -a) /(bn -- l)l , t Xj be Bythe symmetr number yof, EXunmarked mals between the j - Ith and jth marked anim(as requi mb +als1)/(a ,reifd.alNow l+were1).lecaught. =j (b -ania)/(a + 1), whence EX = m (EX l + 1) = 3 . 6 Solutions. Dependence
1. Remembering Problem (2.7 . 3b), it suffices to show that var(aX + bY) < if a , b and var(X), var(Y) < Now, var(aX + bY) = E ( { aX + b Y - E(a X + bY) } 2) = a22 var(X) + 2ab cov(X, Y) + b2 var(Y)2 ::: a var(X) + 2ab vvar X) var(Y) + b var(Y) 2 = (a vvar(X) + b vvar(Y) ) where we have used the Cauchy-Schwarz inequality (3.6.9) applied to X -E(X), Y -E(Y). Ni beersthne andnumberi. of times the ith outcome occurs. Then Ni has the binomial distribution wi2. thLetparamet P 00
00.
(
165
E lR
[3.6.3]-[3.6.8]
Discrete random variables
Solutions
For x = 1, 2, ... , IP'(X = x) = L IP'(X = X, Y = y) = :;OO 2"C { (x + y - Il)(x + y) - (x + y)(xI + y + 1) } = 2x(xC+ 1) = 2"C ( �1 - x +1 1 ) ' andthe covari henceaCnce=does2. Clnotearlexiy Yst.has the same mass function. Finally E(X) = E� 1 (x + 1) - 1 = ' so 4. Max{u, } = 1(u + + 1 1 u and therefore E ( max{X2 , y2 }) = 1 E(X2 + y2) + 1E I (X - Y)(X + Y) I 1 + 1 VE ( X - y)2 ) E ( X + y)2 ) = 1 + h i(2 - 2p)(2 + 2p) = 1 + VI - p2 , where we have used the Cauchy-Schwarz inequality. 5. (a) log y y - 1 with equality if and only if y = 1. Therefore, fy (X» ) E [ fY (X) 1] = 0, E ( g fx(X) - fx(X) wi(b)thThiequals holitydsifliandkewionlse,ywiiftfhyequal = fx. say that X and Y are independent. ity if and only if f (x, y) = fx (x) fy (y) for all x, y, which is to (a)ed, for+ exampl + = E{l(x >Y j + I( Y>Z j + l( z> X j } = 2, whence min {a, } j. Equality is attai n e, if the vector (X, Y, Z) takes only values with probabilities f(2, 1, 3) = f(3, 2, 1) = f(l, 3, 2) = 1 . (b) IP'(X (c) We haveY)that= IP'=(Y =X),p andetc. = 1 - p2 . Also supmin{p, (1 - p2)} = 1(.J5 - 1). 7. We have for 1 x 9 that fx(x) t log ( 1 + 10x1+ y ) = log IT ( 1 + 10x1+ y ) = log ( 1 + x� ) . By calculation, EX 3.44. { . j + �j } = J. + ) j . 8. (i) fX (J. ) = J. . . J. J. (ii) 1 = Lj fx(i) = 2 2 , whence = j(2a). (ii) fx+y(r) = L J. . ( - J.) ., = 1. 3.
00
y= 1
00,
v
v)
- vi,
�
�
10
6.
a
b
0,
0
g.
88).
are
Now,
var(Y) = lE({Y _ lEy}2) = lE [lE( { Y - lE(Y I X) lE(Y I X) - lEY}2 1 X) ] = lE(var(Y I X)) var (lE(Y I X)) since the mean of lE(Y I X) is lEY, and the cross product is, by Exercise (Ie), 2lE [lE ( {Y - lE(Y I X) } {lE(Y I X) - lEy} 1 X) ] = 2lE [ {lE(Y I X) - lEY}lE{Y - lE(Y I X) 1 X} ] = +
+
167
0
[3.7.5]-[3.7.10]
Discrete random variables
Solutions
JE Y - JE Y I X) I X} = JE(Y I X) -JE(Y I X) = o.
since { ( 5. We have that
(a) (b)
JE(T - T > = E= IP'(T > r I T > = E= IP'(TIP'(T> > r O r O N-t N - - r JE(T - T > = E= N - = �(N - I) . r O 2- (t+r ) JE(T - T > = E= � = 2. t i
t)
t i
t)
t i
t)
00
00
t)
t+
t
t + r) t)
.
t+
t
00
r O
6. Clearly
JE(S I N = n) = JE (t= Xi ) = /Ln, 1 and hence JE(S I N) = /LN. It follows that JE(S) = JE{JE(S I N)} = JE(/LN). 7. A robot passed is in fact faulty with probability = (q,(1 - 8)}/(1 -q,8). Thus the number of faulty passed robots, given Y, is bin(n - Y, with mean (n - Y){q,(1 - 8)}/(1 -q,8). Hence JE(X I Y) = Y (n - 1y)A.(1 - q,8 -8) 1
7r
7r ) ,
+
'I'
.
G, B of girlsq, and boys satisfy m B = E /Lr , JE(G) = � = JE(B). r= 1 r= 1
8. (a) Let m be the family size, r the indicator that the rth child is female, and /Lr the indicator of a male . The numbers
m
(It will be shown later that the result remains true for random m under reasonable conditions . ) We have not used the property of independence. (b) With M the event that the selected child is male,
ml JE(G 1 M) = JE ( E=- q,r ) = � (m - I) = JE(B). r 1
The independence is necessary for this argument. 9. Conditional expectation is defined in terms of the conditional distribution, so the first step is not justified. Even if this step were accepted, the second equality is generally false. 10. By conditioning on
Xn - l ,
where Xn has the same distribution as Xn. Hence JEXn = (1 JEXn - l )/ P . Solve this subject to JEX 1 = P - 1 . +
168
Sums of random variables
Solutions
3.8 Solutions.
[3.8.1]-[3.8.5)
Sums of random variables
1. By the convolution theorem,
JI»(X + Y = z) = L JI»(X = k)JI»( Y = z - k) k k+l (m + 1)(n + 1) if ° :::: k :::: m n, (m n) + 1 if m n < k < m n, (m + l)(n + 1) m+n+ l -k (m + 1)(n + 1) if m n :::: k :::: m + n, where m n = min{m, n} and m n = max{m, n}. 2. If z 2, JI»(X + Y = z) = L JI»(X = k, Y = z - k) = z(z c+ 1) k=l Also, if z 0, JI»(X - Y = z) = L JI»(X = k + z, Y = k) k=l - C L=l (2k + z - 1)(2k 1+ z)(2k + z + 1) k = � {;OO { (2k + z - I1)(2k + z) - (2k + z)(2kI + z + 1) } = I � (r + (-z)(rIY++z1 + 1) . By symmetry, if z :::: 0, JI»(X - Y = z) = JI» ( X - Y = -z) = JI»(X - Y = I z l } . z-l afJ (1 fJ)Z - 1 - (1 - a)Z - I } . 3. L a(1 - a) r - l fJ(1 - fJ) z -r -1 = { a - fJ r� 4. Repeatedly flip a coin that shows heads with probability p. Let Xr be the number of flips after the r - Ith head up to, and including, the rth. Then Xr is geometric with parameter p. The number of flips Z to obtain n heads is negative binomial, and Z = E�=l Xr by construction. 5. Sam. Let Xn be the number of sixes shown by 6n dice, so that Xn+l = Xn + Y where Y has the same distribution as X I and is independent of Xn. Then, 1\
1\
1\
V
V
1\
v
�
00
.
�
00
00
-
------
C
00
�C
-
JI»(Xn+ 1 n + 1) = L JI»(Xn n + 1 - r)JI»(Y = r) r=O �
6
�
= JI»(Xn
n) + L [JI»(Xn n + 1 - r) - JI»(Xn n)]JI»(Y = r). r=O We set g(k) = JI»(Xn = k) and use the fact, easily proved, that g(n) g(n - 1) g(n - 5) to find that the last sum is no bigger than �
6
�
�
�
g(n) L (r - 1)JI»( Y = r) g(n) (lE(Y) - 1). r=O 6
=
169
� ... �
[3.8.6]-[3.9.2]
Discrete random variables
Solutions
lE(Y) = 1. 00 = n=O ng(n)e -A).,n n ! = )., nL g(n(n)e- I-)A! ).,n - l = RHS. =l (ii) Conditioning on N and XN , LHS lE(lE(Sg(S) I N)) lE { NlE(XN g(S) I N) } ( (n l )) = Ln (:--A).,In) ! J xlE g L- Xr r=l RHS. = )., J xlE(g(S )
The claim follows since 00 6. (i) LHS L
I
=
=
d F (x)
+x
+ x) d F (x) =
3.9 Solutions.
Simple random walk
1. (a) Consider an infinite sequence of tosses of a coin, any one of which turns up heads with probability With probability one there will appear a run of heads sooner or later. If the coin tosses are 'driving' the random walk, then absorption occurs no later than this run, so that ultimate absorption is (almost surely) certain. Let be the number of tosses before the first of heads . Certainly > r since Nr tosses may be divided into r blocks of tosses, each of which is such a run with probability Hence s) Ls / J , and in particular < 00 for all By the above argument, < 00 also. 2. If then the first step satisfies
p.
N
S run N N JP(S N ) :::: (1 - pNt, pN. N :::: (1 pN) JP(S = lE(Sk) k :::: 1. lE(Tk) So = k XI I JP(XI = 1 I W) = JP(XI = l)JP(JP(W)W I X l = 1) = PPH Pk . Let T be the duration of the walk. Then h = lE(T I So = k, W) = lE(T I So k, W, X l = l)JP(X I = 1 I So = k, W) lE(T I So = k, W, X l = -1)JP(X I -1) I So = k, W) ( I P Pk Pk + = (1 h+ l ) Pk (1 Jk- l ) 1 Pk+ I P ) Jk- l , = 1 PPHIPkh+ l (Pk - PPHI Pk as required. Certainly Jo = O. If P = i then Pk = 1 - (k i N), so the difference equation becomes (N - k l)JHI - 2(N - k) h (N - k l ) Jk 1 = 2(k - N) for 1 :::: k :::: N - 1. Setting U k = (N - k)h , we obtain UHI - 2Uk uk- l = 2(k - N), with general solution U k = A Bk - 1(N - k) 3 for constants A and B. Now Uo = U N = 0, and therefore A = 1N 3 , B -1N2 , implying that Jk = 1{N 2 - (N - k) 2 }, 0 :::: k < N. +
=
=
+
+
+
+
_
+
+
-
+
=
+
170
+
-
Random walk: counting sample paths
Solutions
3. The recurrence relation may be established as in Exercise (2) . Set U k the fact that Pk = (p k - p N )/(I _ p N ) where p = q / p , to obtain p U k+ 1 - (1 - r ) u k + q Uk - 1 = p N _ p k . The solution is
Uk - A + _
B.
Bk P
+
[3.9.3]-[3.10.3]
= (pk - p N ) Jk and use
k (pk + p N ) , p-q
for constants A and The boundary conditions, Uo = U N = 0, yield the answer. 4. Conditioning in the obvious way on the result of the first toss, we obtain
= PPm- I,n
+
(1 - P) Pm,n - l , The boundary conditions are Pm O = 0, POn = 1 , if m, n :::: 1. Pmn
if m, n :::: 1 .
Let Y be the number of negative steps of the walk up to absorption. Then lE(X + Y ) walk is absorbed at N, X - Y = N-k- k ifif the the walk is absorbed at 0. Hence lE(X - Y) = (N - k)(1 - Pk ) - kpk o and solving for lEX gives the result.
5.
=
Dk and
{
3.10 Solutions. Random walk: counting sample paths
1. Conditioning on the first step X I ,
lP'(T = 2n) = 1 lP'(T = 2n I X l = 1) + ! lP'(T = 2n = 1 f- I (2n - 1) + ! fI (2n - 1)
I X l = - 1)
b
where fb (m) is the probability that the first passage to of a symmetric walk, starting from 0, takes place at time m . From the hitting time theorem (3.10. 14),
fI (2n - 1) = f-I (2n - 1) =
1 _1_ (2n - 1 ) n). Hence G(k + 1) = and X is geometric. 8. Clearly,
G(l)k+ 1
k j, j) j=O = t (k : . ) pk-j q m -k+j (�) pj q n - j J =v = pk q m+n- k � (k m- ' ) (n ) = k q m +n -k (m k+ n ) j=O
lP'(X + Y = k) = L lP'(X = k - Y = ,
---'\
J
J
�
J
173
.
J
P
[3.11.9]-[3.11.13]
Discrete random variables
Solutions
p) .
which is bin(m + n, 9. Turning immediately to the second request, by the binomial theorem,
as required. Now,
k n -k : ( )p k= Heven(p + 1 - p)(1n-+p)(1 -p - p)n } = � {I + (I - 2p)n } (1. 8.20).
Yf(N even) = L in agreement with Problem
10. There are (�) ways of choosing k blue balls, and (�=k) ways of choosing n - k red balls. The total number of ways of choosing n balls is (�) , and the claim follows. Finally,
()
Yf(B = k) = nk (b -b! k)! · (N -(Nb -- nb)!+ k)! · (NN!- n)! { ! . b ; I . . . b -�+ I } = { N; b . . . N - b -; + k + {Z . . . N -; +
(: )
1}
x
�
(:)pk (1 _ p)n-k (3.11.8),
1 }-I
as N � oo.
11. Using the result of Problem
12. (a) JE(X) = c + d, JE ( Y ) = b + d, and JE (X Y ) = d, so cov(X, Y) = d - (c + d)(b + d), and X and Y are uncorrelated if and only if this equals O. (b) For independence, we require f(i, = Yf(X = i)Yf(Y = for all i, which is to say that
j)
j)
j,
a = (a + b)(a + c), b = (a + b)(b + d), c = (c + d)(a + c), d = (b + d)(c + d). Now a + b + c + d = I, and with a little work one sees that any one of these relations implies the other three. Therefore X and Y are independent if and only if d = (b + d)(c + d), the same condition as for uncorrelatedness. 13. (a) We have that 00
mL- I Yf(X = = L L Yf(X = = L Yf(X n=O m=n+ n=O m=O n=O 00
JE(X) = L mYf(X = m ) = L
m =O
m)
174
00
00
00
m)
1
>
n).
Problems (b) First method. Let N be the number of balls drawn. Then, by (a), lE (N)
= = =
r
L JJ»(N Lr
n=O
r
n=O
>
n) =
�
t (n
(b + r) !
n=O
[3.11.14H3.11.14J
r
L JJ»(first n balls are red) r ... -1 -n 1 L n=O
r- l
b+rb+r
Solutions
r-n+l
+b b
)
b+r
=
=
+
b+r + l b+l
r!
n=O
(b + r - n) ! (r - n) !
(b + r) !
'
( ) (rt!t l ) .
b where we have used the combinatorial identity E� =o n t = To see this, either use (�) the simple identity + = repeatedly, or argue as follows. Changing the order of summation, we find that
(r': l ) (x ;: l ) � xr � (n ; ) 1 � x � x (n ; ) ( 1 1) (1 - x ) - b 2 "f x r by the (negative) binomial theorem. Equating coefficients of x r, we obtain the required identity. b
=
b
n
( + )
=
=
b+r+
r =O
b+
Second method. Writing m (b , r) for the mean in question, and conditioning on the colour of the first ball, we find that b r m (b , r) = -- + {I + m (b , r - 1) } -- '
b+r
b+r
With appropriate boundary conditions and a little effort, one may obtain the result. Third method. Withdraw all the balls, and let Ni be the number of red balls drawn between the i th and (i + l)th blue ball (No = N, and Nb is defined analogously). Think: of a possible 'colour sequence' as comprising r reds, split by b blues into b + 1 red sequences. There is a one-one correspondence between the set of such sequences with No = i, Nm = j (for given i, j, m) and the set of such sequences with No = j, Nm = i ; just interchange the 'Oth' red run with the mth red run. In particular lE (No) = lE (Nm ) for all m. Now No + N + . . . + Nb = r , so that lE (Nm ) = r /(b + 1), whence the claim is immediate. (c) We use the notation just introduced. In addition, let be the number of blue balls remaining after the removal of the last red ball. The length of the last 'colour run' is Nb + only one of which is non-zero. The answer is therefore r/(b + 1) + b/(r + 1), by the argument of the third solution to part (b). 14. (a) We have that lE (Xk ) = Pk and var(Xk ) = Pk (1 - Pk ), and the claims follow in the usual way, the first by the linearity of lE and the second by the independence of the Xi ; see Theorems (3.3.8) and (3.3 . 1 1). (b) Let s = Ek Pb and let Z be a random variable taking each of the values PI , P2 , . . . , Pn with equal probability n- 1 . Now lE (Z2 ) _ lE (Z) 2 = var(Z) 2:: 0, so that 2 s 2:: .!. Pk = n k n k n with equality if and only if Z is (almost surely) constant, which is to say that P I = P2 = . = Pn . Hence
l
Br
L .!. pf (L ) �
175
Br ,
..
[3.11.15]-[3.11.18]
Discrete random variables
Solutions
with equality if and only if P I = P2 = . . . = Pn . Essentially the same route may be followed using a Lagrange multiplier. (c) This conclusion is not contrary to informed intuition, but experience shows it to be contrary to much uninformed intuition. 15. matrix V has zero determinant if and only if it is singular, that is to say if and only if there is a non-zero vector x such that xVx' = O. However,
A
Hence, by the result of Problem (3. 1 1 .2), Ek Xk( Xk - lEXk) is constant with probability one, and the result follows. 16. The random variables X + Y and I X - Y I are uncorrelated since cov { X + Y,
IX - YI ) = lE{ (X + Y) I X - YI } - lE(X + Y)lE(IX - YD = ! + ! - 1 . ! = O.
However,
! = lP(X + Y = 0, IX - YI = 0)
# lP(X
+
Y = O)lP(I X - Y I
=
0)
= ! . ! = l,
so that X + Y and I X - Y I are dependent. 17. Let h be the indicator function of the event that there is a match in the kth place. Then lP(h 1) = n - I , and for k
# j,
lP(h
=
1 , Ij
=
= 1) = lP(/j = 1 I h = 1)lP(h = 1) = n(-'-n-1--l-:-)
= lE (X2 ) - (lEX) 2 = lE (E h ) - 1 2
var(X)
1
()
= E lE(h) 2 + L lE(/j h ) - I = I + 2 n2 n(n 1- 1) - 1 = 1 . j# 1 We have by the usual (mis)matching argument of Example (3.4.3) that n 1 - r ( - 1) j o � r � n - 2, lP(X = r) = -, '"' . , r . jL..J I . =O -
'
which tends to e - I Ir ! as n -+ 00 . 18. (a) Let YI , Y2 , . . . , Yn be Bemoulli with parameter P2 , and Z I , Z2 , . . . , Zn Bernoulli withparam eter P I I P2 , and suppose the usual independence. Define Aj = Yj Zj , a Bernoulli random variable that has parameter lP(Aj = 1) = lP(Yj = 1)lP(Zj = 1) = Pl . Now (A I , A 2 , . . . , An) � (YI , Y2 , . . . , Yn) so that f (A) � f (Y) . Hence e (P I ) = lE( f (A)) � lE( f (Y)) = e ( p2 ) ' (b) Suppose first that n = 1 , and let X and X' be independent Bernoulli variables with parameter p. We claim that { J (X) - f(X') } { g (X) - g (X' ) } 2: 0; 1 76
Problems
Solutions
to see this consider the three cases X = X', X are increasing. Taking expectations, we obtain
[3.11.19]-(3.11.19]
< X', X > X' separately, using the fact that I and g
lE ( {f(X) - I(X') }{g (X) - g (X') })
:::
0,
which may be expanded to find that
o :::; lE (J (X) g (X ») - lE (J (X') g (X ») - lE (J (X) g (X ' ») + lE (J (X') g (X' ») =
2 { lE (J (X) g (X») - lE (f(X » lE(g (X »
}
by the properties of X and X'. Suppose that the result is valid for all n satisfying n lE (J (X) g (X»)
=
< k where k ::: 2. Now
lE {lE (J (X) g (X) I Xl , X2 , . . . , Xk - t ) } ;
here, the conditional expectation given X I , X2 , . . . , Xk - I is defined in very much the same way as in Definition (3 .7.3), with broadly similar properties, in particular Theorem (3 . 7.4); see also Exercises (3.7.1, 3). If Xl , X2 , " " Xk - I are given, then I(X) and g (X) may be thought of as increasing functions of the single remaining variable Xb and therefore
by the induction hypothesis. Furthermore
are increasing functions of the k - 1 variables X I , X2 , . . . , Xk - I , implying by the induction hypothesis that lE (f'(X) g' (X» ::: lE (f' (X» lE (g ' (X» . We substitute this into (*) to obtain
lE (J (X) g (X») ::: lE (f' (X» lE (g ' (X» by the definition of I' and g '. 19. Certainly R(p) = lE (/A) Differentiating, we obtain
=
=
La> IA (w)lP'(w) and lP'(w)
lE (f(X» lE(g (X» =
p N (a» q m- N (a» where p + q
=
1.
Applying the Cauchy-Schwarz inequality (3 . 6.9) to the latter covariance, we find that R' (p) :::; (pq) - I '/var(/A) var(N) . However IA is Bernoulli with parameter R(p), so that var(/A) = R(p)(1 R(p» , and finally N is bin(m, p) so that var(N) = mp(1 - p), whence the upper bound for R' (p) follows. As for the lower bound, use the general fact that cov(X + Y, Z) = cov(X, Z) + cov(Y, Z) to deduce that COV(/A , N) = COV(/A , IA) + COV(/A , N - IA). Now IA and N - IA are increasing functions of w, in the sense of Problem (3. 1 1 . 1 8); you should check this. Hence cov(/A , N) ::: var(/A) + 0 by the result of that problem. The lower bound for R ' (p) follows. 177
[3.11.20]-[3.11.21]
Discrete random variables
Solutions
20. (a) Let each edge be blue with probability P I and yellow with probability P2 ; assume these two events heredge and ofis tgreen he colwiourithnprobabi gs of alllitoty hPerI P2edges. Call an edge green if it . IT there is a working green isconnecti both areblouneinfrom anddependent yelsource low,oftsooeach tsihnatk,oteach then there is also a blue connection and a yellow connection. Thus lP'(green connection) ::: lP'(blue connection, and yellow connection) lP'(blue connection)lP'(yellow connection) I P2 ) ::: R (p } )R (P2 ) . so(b)thatThisRis(Psomewhat maybbele proved byeldisnducti onthoner Rth(epnumber n of edges of G. IT n 1 talhlenp.aInconsieitherderaticaseontofhharder, te requi he twandroedpossi cases yi t h at ei ) 1 for all P, or R (p) P for i n equal i t y hol d s. Suppose then that the inequality is valid whenever n k where k ::: 2, and consider the case when G has k edges. Let e be an edge of G and write w(e) for the state of e; w(e) 1 if e is working, andsink,w(e)we have0tothatherwise. Writing A for the event that there is a working connection from source R (p Y ) lP'p Y (A I w(e) l)p Y + lP'p Y (A I w(e) 0)(1 - p Y ) ::: lP'p (A I w(e) I) Y p Y + lP'p (A I w(e) O) Y (1 - p Y ) where theisappropri antce,e probabi lity measure when each edge is working with probability The iedge; nequalthlP'iet0!yiisnhereducti val i d si i f w (e) is given, then the network G is effectively reduced in size by one on hypothesis is then utilized for the case n k - 1. It is a minor chore to check that x Y p Y + y Y (1 - p) Y ::: {xp + y(l - p)} Y if x ::: y ::: 0; trespect o see thitos,xcheck that equal ity holds whenng derix vatyiv:::e 0ofandthethriatght-hand the derivsiatdievewhen of thx,e leyft-handO. Appl sideywiththe is at most t h e correspondi lat er inequality with x lP'p (A I w(e) 1) and y lP'p (A I w(e) 0) obtain R (p Y ) ::: { lP'p (A I w( e) l)p + lP'p (A I w(e) 0)(1 - p) Y R(p) Y . 21. (a) The number X of such extraordinary individuals has the bin (107 , 10-7 ) distribution. Hence lEX 1 and lP'(X 1 I X - 1) lP'lP'((XX 0)1) 1 -lP'(1X-lP'0)(X- lP'0)(X 1) 1 - (1 _ 10-7) 107 _ 107 . 107-170(17 _ 10-7 ) 107 - 1 1 - (1 - 10- ) 1 - 1l -2- e-e 1 0.4. (Shades of (3. 5 . 4 ) here: X is approxi m atel y Poi s son di s tri b uted wi t h paramet e r 1. ) (b) Likewise lP'(X 2 I X ::: 2) 1 -21 e-- 12e--�l e- 1 0.3. (c) Provided m N 107 , lP'(X m) N ! ( 1 )m ( 1 - 1 ) N-m e- 1 =
=
=
> >
-
>
«
=
=
:::
to
=
=
=
>
=
=
=
=
�
�
�
=
=
=
m ! (N - m) ! N
-
178
-
N
� -,
m!
=
Problems
Solutions
[3.11.22]-[3.11.23]
then) :::Poir sforsonsome distrisuibutitablon.e smal Assume that "reasonabl y confident that n is all" means that IP'(X n I X l number r. Assuming the Poisson approximation, IP'(X n) ::: r IP'(X n) if and only if 00 00 e - 1 '"' 1 - re - 1 '"' -1 . k=nL...J+ l k! k=nL...J k! Forveryanyroughlgiveny nr, thl/re smalwillestdoacceptabl e value of n may be determined numerically. If r is small, then (e. g . , i f r 0.05 then n 20). probabi lity is sufficiently small for one to be sure that the person is specified uni(d)Therefore, qNouelyle. velIf ppinofthi10ims-case, 7 then X is bin (107 , 10-7 which is approximately Poisson with parameter say. IP'(X 1 I X 1) 1 - e1--a e- aae-a Anp acceptabl e value of for a very petty offence might be 0.05, in which case 0. 1 and so 10- 8 might be an acceptable level of improbability. For a capital offence, one would normally requi a muchlinessmalin alecourt r valuofe oflaw iWen thenoteUnithtedat tKihenrulgdom.es of evidence do not allow an overt discussion alongrethese 22. The number G of girls has the binomial distribution bin (2n, p) . Hence IP'(G 2n - G) IP'(G n) k=nL2n (2;) pk q2n-k ( ) � pkq2n-k (2n) pnqn --q , ::: 2n n q-p n k=nL...J where we have used the fact that e:) (�) for all k. With p 0.485 and n 104, we have using Stirling's formula (Exercise (3. 10. 1)) that (2n) pn qn _q � { (1 0.03)(1 + 0.03) } n 0.5 1 5 0 03 n q-p 1 04 0.5 15 ( 9 1 - ) ::: 1 .23 10 - 5 . 3.,fii 104 It10fol-5)8lows2 th0.at9t9899. he probability that boys outnumber girls for 82 successive years is at least (1 - 1 .23 of suchlityvi1 s-itkNs. If-k1 by0,equati thenoMn (1.7.7).1 if andHavionlngyhiiftthe0, thpartie chance cle hitofs 0anotbeforeher ivit hisittLetstoN,0Mbefore anbeeventthhie twinumber tinhg probabi l ce the particle at 0 moves immediately to 1 whence there is probability 1 - N- 1 ofNanotis h1 er- viNs-it t,osi0nbefore visiting N. Hence IP'(M r I So k) ( 1 - Nk ) ( 1 - Nl ) r- l ' r 1 , so that IP'(M I So k) IP'(M I So k) jIP'- lM + 1 I So 0) ( 1 � ) ( 1 � ) �, 1. >
>
� �
_
�
p
=
= p,
_
p
�
a
�
p.
=
�
=
�
=
=
:::
=
_ :::
'V
(mr )
_
.
=
23.
x
x
�
=F
=
�
= j
=
=
�
=
=
�
=
�j
_
_
179
- (
=
�j
j �
[3.11.24]-[3.11.27]
Discrete random variables
Solutions
Eidiffthererence readequati the solouns.tion to Exercise (3.9.4), or the following two related solutions neither of which uses First method. Let Tk be the event that A wins and exactly k tail+s appear. Then k n so that However IP'(Tk ) is the probability that m k tosses yield m heads, k tails, IP'(Tk ).Hence andIP'(Atwihenlas)st toss2:k:�is heads. 24.
' = -x ifJ . Using this identity and integrating by parts repeatedly, ifJ'u(u) du = ifJx(x ) 1x00 ifJ'u(u)3 du 1 - (x) = 1x00 ifJ (u)du = - 1x00 -3ifJ' (u) du = ifJx(x ) ifJx(x3 ) 3ifJx5(x ) i 15uifJ6(u) du o = ifJ x(x ) ifJ (x3 ) x ix u5 x 7.
+
+
+
-
_
_
r oo
+
_
4.5 Solutions.
--
_
r oo
Dependence
(i) As the product of non-negative continuous functions, f is non-negative and continuous. Also 00 1 -2 e- :Zx dy = �e- ix i g(x) = �e - i x i 1 - 00 27r x igf ixs dis0,contisincenuous,the inwhitegrand i s t h e N(O, x -2 ) density function. It is easily seen that g(O) = 0, so that le 100 00 g(x) dx = 1 � e - I x l dx = 1. - 00 - 00 (ii) Clearly fQ and 100 100 fQ (x, y) dx dy = ?;00 (�r · 1 = 1. - 00 - 00 2 of the form [- M, M] JR.; Alhenceso ffQQisisthconti e uninfuous. orm limHence it of conti n uous functi o ns on any subset of JR. fQ is a continuous density function. On the other hand 100 fQ (x, y) dY = L00 ( l )n g (x - n ), - 00 n= l where (ii ) Takeg isQditsoconti be thenuousset ofat the0. rationals, in some order. We mayanglassume thatthetherodcentandre ofa ltihnee rodof thiseunifirstformly posiunitfiorm onedonin [a0square ofIf tsihezelaatterb,anglwhie lies the2.thacute e between gri d is , � rr]. wiwitthhithen aaicertd ofaina idinneragram,rectangl one efinofdssitzheat(ath-rere iscosno intersecti onsinif andHence only ifththee probabi centre lofitythofe rodan inen,liteesrsect (b -r ion is 2 rr2ab 101C/ {ab - (a -rcosO)(b - r sin O) } dO = -(a rr2rab b - � r). 3. the(i) Letsecondbeneedl the inediicnator of thaelevent thatthteheresulfirstt needl e intersects a),linlE(l)e, and=llE(J) et J be=th2/rr; e indihence cator that tersects i n e. By of Exerci s e ( 4 . 5 . 2 Z = J satisfies lE( � Z) = 2/rr. 1.
1
2 2
Y
J
I-
::: °
x
q
2"
x
o
0)
x
0).
+
0
I
I+
191
[4.5.4]-[4.5.6]
Continuous random variables
Solutions
(ii) We have that
on of (4.5.8), if < 8 < � rr , then two intersections occur if z < � min{sin 8, cos8} or 1 In- zth<e notati i min {sin 8, cos 8}. With a similar component when � rr 8 < rr, we find that dz d8 lE(l J) lP(two intersections) � II i 0
:5
=
=
(z ,9) : O< z < min {sin 9, cos 9 }
� 4 11' 11' -rr4 100 /2 � min {sin8,cos8}d8 -rr4 100 / sin 8d8 -rr4 ( 1 - '\121 ) and hence 3 -rr-../2 - rr4 · var( z1 Z) ;1 - rr42 ;1 (2 - '\12) 2 (ivar(i ) Fori Z).Buffon' s needl e , t h e vari a nce of t h e number of i n tersecti o ns i s (2/ rr) -(2/rr)2 which exceeds You should therefore use Buffon's cross. O
G o (v )
1
v E
G n - 1 (v)
v
1],
>
n==O
n
n
G N (S )
n= l
n= l
194
n- 1
n
1,
Functions of random variables
[4.6.7]-[4.7.1]
Solutions
inequalWeitymay, assume without loss of generality that lEX lEY O. By the Cauchy-Schwarz Hence, lE(var(Y I X)) lE(y2) -lE(lE(Y I X)2) -< lEy2 - lE(XlE(Xy2))2 (1 -p2) var(Y). 8. One way is to evaluate =
7.
=
=
=
Anot h<ermiwayn{Y,is Z})to observe)../ ().th. +at/1-mi+n{Y,v). Z}Simisilexponenti a
=
=
=
=
>
=
m)
=
=
=
=
=
=
=
=
2:
=
=
=
=
2: . . . 2:
2:
4.7 Solutions.
2: . . . 2:
2:
=
m!
-
(m
I) !
Functions of random variables
'
We observe that, if 0 ::::: u ::::: 1, JP(XY ::::: u) JP(XY ::::1 : u, Y ::::: u) + JP(XY ::::: u, Y u) JP(Y ::::: u) + JP(X ::::: u/Y, Y u) u + 1 yu dy u( -logu). By the independence of XY and Z, JP(XY ::::: u, Z2 ::::: v) JP(XY ::::: u)JP(Z ::::: ../V) u../V(1 -logu), 0 < u, v < 1. 1.
=
=
>
u
=
-
I
=
=
1 95
=
>
[4.7.2H4.7.5]
Continuous random variables
Solutions
Differentiate to obtain the joint density function 1 g(1 /u) u, v :::: 1. g(u, v) = 2....(V , Hence IP'(XY :::: Z2) = If 10;%U) du dv = a . O:;: u :;: v :;: l Arguing more directly, lP'(XY :::: Z2) = fff dx dy dz = �. 0
0 ::::
O:;:x.y.z:;: l xy:;:z2
The transformation x = uv, y = u - uv has Jacobian J = 1 1 -v v -uu I = -u. Hence I J I = andlu i , andfv (v)therefore independent, = I onfu.[0,v1](uas, v)requi= ured.e-u , for 0 :::: u < 00,0 :::: v :::: 1. Hence U and V 3. Arguing directly, lP'(sin X :::: y) = lP'(X :::: sin- 1 y) = �rr sin- 1 y, 0 :::: y :::: 1, sovarithaatblefys. (y) = 2/ (rr 0-?), for 0 :::: y :::: 1. Alternatively, make a one-dimensional change of 4. (a) lP' (sin - 1 X :::: y) = lP'(X :::: sin y) = sin y, for 0 :::: y :::: !rr . Hence fy (y ) = cosy, for O :::: y :::: irr. (b)- !rrSim X iVIf and- onlp yV,ifwe(1 have - p) U < VI - p 2 V. Turning to polar coordinates, JE:(max{X, Y}) = 1000 � [£Vt+7r { pr cose + r H sin e } de + £�7r r cosede] dr where 1fr = (1 - p) / (1 + p). Some algebra yields the result. For the second part, JE:(max{X, y}2) = JE:(X2 I{x >Y}) + JE:(y2 I{y> X}) = JE:(X2 I{x < y}) + JE:(y2 I{y <x}), 2) = JE:(X2 ) + byJE:(yth2e) =symmetry of t h e margi n al s of X and Y. Addi n g, we obtai n 2JE: ( max{X, y} 2. 7. We have that A e- (A+/L)z = lP'(X < Y)lP'(Z > z). lP'(X < Y, Z > z) lP'(z < X < Y) = A +JL A (a)lP'(X = Z) = lP'(X < Y) = A +JL (b) By conditioning on Y, lP' ( X - Y)+ = 0) = lP'(X Y) = A +A JL ' lP' ( X - Y) + > ) = �e A +JL -AW for > O. By conditioning on X, lP'(V > v) = lP'(I X - YI > v) = 1000 lP'(Y > v + x) fx(x) dx + 100 lP'(Y < x - v) fx(x) dx v > o. (c) By conditioning on X, the required probability is found to be a
=1=
tan
.J
=
__
-- .
:s
w
w
find tthheatJacobian, or argue directly. With the convention t8.hat EiVtrh2er-umake2 =a0change when rof2 -variu2abl<e0,s andwe have F(r, x) = lP'(R r, X x) = -7r2 jX Vr2 - u2 du, 82 F = r-;---;:; 2r2 2 Ix l < r < 1 . f (r, x) = 8r8x 7r V r -x 9. As in the previous exercise, lP'(R r, Z z) = 37r jZ 7r(r2 - 2) d . :s
:s
-r
'
-
:s
:s
4
1 97
-r
w
w
[4.7.10H4.7.14]
Continuous random variables
Solutions
f(r, z) = �r for Izl < r < 1 . This question may be solved in spherical polars also. 10. The transformation s x + y, r = xl(x + y), has inverse x = rs, y = (1 - r)s and Jacobian = s. Therefore, 00 fR (r) = (X) fR S (r, s) ds = [ fx y (rs, (1 - r)s ) s ds Jo ' Jo ' O � r � 1.
Hence
J
11.
=
We have that
whence
fy (y) = 2fx (- v(aly) - 1) = 12.
Using the result of Example
IP(X > a, Y > b) =
(4.6.7),
1 rr Jy(a _ y) '
o � y � a.
and integrating by parts, we obtain
100 ifJ (x) { 1 - ( n ) } dx
00 = [1 - (a)][1 - (c)] + [ [1 - (x)]ifJ Since
Ja
dx. (�) h 1 - p2 1 - p2
[1 - (x)]/ifJ (x) is decreasing, the last term on the right is no greater than b px 1 - (a) [ 00 P dx, ifJ(x )ifJ ifJ(a) VI _ p 2 VI p 2
( - )
Ja
_
which yields the upper bound after an integration.
13.
The random variable
Y is symmetric and, for a > 0,
IP(Y > a) = IP(O < X < a - I ) = by the transformation
J14.= w,
are
la
{
w = x + y, z = x I (x + y) has inverse x = wz, y = (1 - z)w, and Jacobian
f(w, z) = W ·
Z
o
_ v -2 dv = rr(1 + u 2 ) 00 rr(1 + v- 2 ) ' du
v = l /u . For another example, consider the density function l x - 2 if x > 1 ' f(x) = 2 if O � x � 1 . �
The transformation whence
Hence W and
Ioa - I
A(Awz)a - I e -J.. wz A(A(1 - z)w) fJ- 1 e - J.. ( l -z ) w · r({3) r(a)
independent, and
Z
w > 0, 0 < z < 1 . is beta distributed with parameters a and {3.
1 98
Sums of random variables 4.8 Solutions.
1.
By the convolution formula (4.8.2),
fz(z) if A =j:
2.
=
z
Z
Sums of random variables =
X + Y has density function
r A/Le -J...x e -/L (z -x ) dx
10
/L. What happens if A
=
/L?
(Z
=
=
(
� e -J...z
/L - A
_
e -/Lz ) ,
z � 0,
has a gamma distribution in this case.)
Using the convolution formula (4.8.2), W
fw (w)
[4.8.1H4.8.4]
Solutions
otX + {3Y has density function 1 1 . dx, not (1 + (x/ot) 2 ) n{3 (1 + {(w - x)/{3} 2 )
100 -00
=
which equals the limit of a complex integral:
{ ot{3n 2 . z2 +1 _ot2 . (z - w)12 + {32 dz
lim
__
R -+ oo 1D
where D is the semicircle in the upper complex plane with diameter ating the residues at z iot and z w + i{3 yields
{
=
fw (w)
=
=
[- R, R]
on the real axis. Evalu
=
ot{32n i 1 1 I I . -n-2- 2i-{3 -(w-+-j-{3')2'+-ot"2 2i-ot ' (iot - w) 2 + {3 2 + 1 1 . -00 < w < 00 n(ot + {3) 1 + {w/(ot + {3)} 2 '
}
after some manipulation. Hence W has a Cauchy distribution also.
3.
Using the convolution formula (4.8.2),
fz (z) 4.
=
z
1 e -z d r 2"z y
10
=
1 2e -z , 2"Z
z � o.
Let fn be the density function of Sn . By convolution,
This leads to the guess that
n � 2, which may be proved by induction as follows. Assume that (*) holds for n
1 99
:::
N.
Then
[4.8.5]-[4.8.8]
Continuous
Solutions
for some constant
random variables
A. We integrate over x to find that N N+l
I
As A , + = E IT r = 1 s = 1 As - A r A N+l si'r
= N + 1 on solving for A. 5. The density function of X + Y is, by convolution, and (*) follows with n
{X
h (x) = 2 - x Therefore, for 1 �
!3 (x) = Likewise,
if O � x � l , if 1 � x � 2.
x � 2,
10 1 h (x - y) dy = 1 1
x-I
o
(x - y) dy +
A simple induction yields the last part. The covariance satisfies cov(U, V) = JE(X 2 random variables taking values ±1, then
6.
JP>(U
= 2,
V
= 2) = °
If X and Y are independent N(O, 1) variables, a function of u multiplied by a function of v.
Io - (2 - x + y) dY = i - (x _ �)2 . x 1
0
- y 2 ) = 0, as required.
but JP>(U
If X and
= 2) JP>(V = 2) > 0.
fu. v (u , v) = (4rr) - l e - ! (u2 + v2 ) , which factorizes as
7.
From the representation X = apU + a V I - p 2 V, Y = r:U, where U and V N(O, 1), we learn that apy JE(X I Y = y) = JE(ap U I U = y/r:) = - . r: Similarly,
whence var(X I Y) 0' 2 + pa r:, var(X +
Y are symmetric
= 0' 2 (1 - p 2 ) . For parts (c) and (d), Y) = 0' 2 + 2pa r: + r: 2 , and
are independent
simply calculate that cov(X, X
+ Y) =
8.
First recall that JP>(I X I � y) = 2(y) - 1 . We shall use the fact that U = (X + Y)/..[2, = (X - Y)/..[2 are independent and N(O, 1) distributed. Let 8 be the triangle of lR2 with vertices (0, 0), (0, Z), (Z, 0). Then V
JP>(Z �
z I X > 0, Y > 0) = 4JP> ( X, Y) E 8) = JP> ( l U I � z/ ...fi, I V I � z/ ...fi) = 2{2 (z/ ...fi) _ 1 } 2 ,
200
by symmetry
Multivariate normal distribution
Solutions
[4.9.1]-[4.9.5]
whence the conditional density function is
f(z) = 2�{ 2 (Z/�) - 1 }cf>(Z/�) . Finally, 1&(Z I X >
0,
Y>
0) = 21&(X I X > 0,
Y >
0)
4.9 Solutions. Multivariate normal distribution
1. Since V is symmetric, there exists a non-singular matrix M such that M' = M - l and V = MAM- 1 , where A is the diagonal matrix with diagonal entries the eigenvalues A 1 , A2 , , An of V. 1 1 Let A ! be the diagonal matrix with diagonal entries $t, ..;>:2, . . . , $n; A ! is well defined since 1 V is non-negative definite. Writing = MA ! M', we have that = and also • . .
W W'
W
1
as required. Clearly is non-singular if and only if A ! is non-singular. This happens if and only if Ai > ° for all i , which is to say that V is positive definite.
W
2. By Theorem (4.9.6), Y has the multivariate normal distribution with mean vector 0 and covariance matrix Clearly Y = (X - p.)a' + p.a' where a = (a l ' a2 , . . . , an ) . Using Theorem (4.9.6) as in the previous solution, (X - p. )a' is univariate normal with mean 0 and variance aVa' . Hence Y is normal with mean p.a' and variance aVa' .
3.
4 . M ake the transformation u = x + y , v = x - y , with inverse x = that I J I = i . The exponent of the bivariate normal density function is
i (u + v), y = i (u - v), so
and therefore U = X + Y, V = X - Y have joint density
f (u, v ) =
{
v2 1 u2 exp - 4(1 _ p) + 4(1 p) 2 4n VI _ p
}
,
are independent with respective distributions N(O, 2(1 + p» and N(O, 2(1 - p» . That Y is N(O, 1) follows by showing that JP'(Y � y ) = JP'(X � y ) for each of the cases y � -a,
whence U and V 5.
Iy l
(X) dx - j-a -a -00
x
2 cf> (X) dx 201
1a00
x
2 cf>(X) dx = 1 - 4
1a00
x
2 cf>( ) d . x
x
[4.9.6]-[4.10.1]
Continuous random variables
Solutions
The answer to the final part is no; X and Y are N(O, 1) variables, but the pair (X, Y) is not bivariate normal. One way of seeing this is as follows. There exists a root a of the equation p(a) = O. With this value of a, if the pair Y is bivariate normal, then X and Y are independent. This conclusion is manifestly false: in particular, we have that JP'(X > a, Y > a) ¥- JP'(X > a)JP'(Y > a).
X,
6.
Recall from Exercise
(4. 8 .7) that for any pair of centred normal random variables
JE(X I Y) =
cov(X, Y) Y, var Y
var(X I Y) = { I -
p(X, y ) 2 } var X.
The first claim follows immediately. Likewise,
As in the above exercise, we calculate a = JE ( X I �i Xr ) and b = var(X I �i Xr ) using the facts that var X I = V l l , var ( �i Xi ) = � ij Vij , and COV ( X l �i Xr ) = � r V Ir ·
7.
8.
Let p = JP'(X >
0,
Y>
0,
Z>
0) = JP'(X < 0,
1
1
'
Y
< 0,
Z
< 0). Then
1 - p = JP' ( {X > O} U { Y > O} U {Z > On = JP'(X > 0) + JP'(Y > 0) + JP'( Z > 0) + p
- �X > � Y > � - �Y > � Z > � - �X > � Z > � 1 . = 2: + p - 4 + { sm PI + sm P2 + sm P3 } .
3
[3
27r
-1 . -1 . -1 ]
U, V, W be independent N(O, 1) variables, and represent X, Y, Z as X = U, J1 - p ? V, P - PIP3 V + 1 - P 2I - P22 - P32 + 2PIP2 P3 W. Z = P3 U + 2 (1 - P 2I ) J1 - p?
9.
Let
We have that U = X, conditional variance.
V=
(Y
- PI X) / J1 - p ? and JE(Z I
Y=
X, Y) follows immediately, as does the
4.10 Solutions. Distributions arising from the normal distribution
1.
First method. We have from (4.4.6) that the x 2 (m) density function is
1
1 1 1 J m (x) = r(m/2) 2 -m/2x 'l m - e - 'l x , �
___
x 2: O.
The density function of Z = X I + X2 is, by the convolution formula,
10
g(z) = c Z A m - l e - ! x (z - x) ! n - l e - ! (z -x) dx = cz � (m +n ) - l e - ! Z
10 1 u !m - l (1 _ u) ! n- l du
202
PI U +
Distributions arisingfrom the normal distribution
Solutions
(4.10.2]-[4.10.6J
c is a constant. Hence g(z) c'z ! (m+n ) - l e - ! z for z 0, for byan appropri the substiatteuticonstant on c',lzas, where required. Second method. m and n integral, the following argument is neat. Let Z l , Z2 , . . . , Zm+n be distribution as Zi Z� Z;, and X2 ttheinhdependent e same same didisstritribbutiutioo1)nn vari asas Z;Ziabl+1es. ThenZ;Z;+X2 l has' i.eth.,etZ;hsame +n (see Problem (4.14.12» . Hence X l X2 has e 2 (m n) distribution. +n (i) The t (r) distribution is symmetric with finixte2mean, and hence thi s mean i s 0. 2 (ii) Here is one way. Let U and V be independent (r) and X (s ) variables (respectively). Then ;::
=
u = x
are
If
+...+
+
N(O,
+ +...+
+...+
x
+
+
2.
( )
lE Ulr Vis
=
-l) �lE(U)lE(V r
by independence. Now U is r(-�, !r) and V is r(-� , i s), so that lE(U) r and =
if s 2, since the integrand is a density function. Hence s if s 2. lE ( Ulr ) s-2 Vis (ii ) If s :::: 2 then lE( V - l ) 3. Substitute r 1 into the t (r) density function. First method. Find the density function of XI Y, using a change of variables. The answer is F(2,2). Second method. X and Y are independent X 2 (2) variables (just check the density functions), and hence XI Y is F(2, 2). Thebutiovectn. Weor (X,haveXasl -X, X2 -X, . . . , Xn -X) has, by Theorem (4 . 9 . 6 ), a multivariate normal diis sitrindependent i n Exerci se (4.5.7) that cov(X, Xr - X) ° for all r, which implies that X of each Xr . Using the form of the multivariate normal density function, it follows that XNowis iSndependent of the family {Xr - X : 1 :::: r :::: n}, and hence of any function of these variables. 2 (n - 1) - 1 Er (Xr - X) 2 is such a function. The choi ceweof tfihxerefore ed vectortakeisthiims vector materiatol, besin(0,ce 0,the. .joi. ,n0,t di1).striWebutimake on ofthtehechange Xj is spherically symmetri c , and of variables QI Xn , where Q 2 E�:t X; and Q 0. Since Q has the 2 (n - 1) U2 Q 2 X�, tan distribution, and is independent of Xn, the pair Q, Xn has joint density function >
>
=
= 00 .
=
4.
5.
=
=
6.
=
+
IJI
=
x
;::
=
X
E
JR,
q>
0.
Thehe joithneoryt densiis now slightlony!Ueasi, IV(U,er th1/1an) tofhe practicWee. Wenowsolinvtegrate e for U,over findandthechoose Jacobitahn,e andconstant deduceso tthat t y functi the total integral is 1. u,
IJI,
IJI.
u,
203
[4.11.1]-[4.11.7]
Continuous random variables
Solutions
4. 1 1 Solutions.
Sampling from a distribution
. . . , n }. 2. The result holds trivially when n = 2, and more generally by induction on n. 3. We may assume without loss of generality that A = 1 (since ZI>" is r (>" , t) if Z is r(1, t)). Let U, V be independent random variables which are uniformly distributed on [0, 1]. We set X = -t log V and note that X has the exponential distribution with parameter 1 I t . It is easy to check that 1.
Uniform on the set { I , 2,
for x > where e
=
t t e - t+ l I r (t ) .
0,
Also, conditional on the event A that U
X t - l e - t te - Xl t , < r (t )
X has the required gamma distribution. This observation may be used as a basis for sampling using the rejection method. We note that A = { log U ::: (n - 1 ) ( log( X l n ) - (Xln) + I ) } . We have that lP(A) = l i e , and therefore there is a mean number e of attempts before a sample of size 1 is obtained.
Use your answer to Exercise (4. 1 1 .3) to sample X from r ( l , a) and Y from r(l, fJ) . By Exercise (4.7. 14), Z = X/(X + Y) has the required distribution. 5. (a) This is the beta distribution with parameters 2, 2. Use the result of Exercise (4). (b) The required r (1 , 2) variables may be more easily obtained and used by forming X = - log( UI U2 ) and Y - log(U3 U4 ) where { Ui 1 ::: i ::: 4 } are independent and uniform on [0, 1]. 4.
:
(c) Let UI , U2 , U3 be as in (b) above, and let Z be the second order statistic U(2) . That is, Z is the middle of the three values taken by the Uj ; see Problem (4. 14.21). The random variable Z has the required distribution. (d) As a slight variant, take Z = max{UI , U2 } conditional on the event {Z ::: U3 }. (e) Finally, let X = ../Ui/ (../Ui + ../U2) , Y = ../Ui + ../Ui.. The distribution of X, conditional on the event { Y ::: I }, is as required.
3
6. We use induction. The result is obvious when n = 2. Let n :::: and let p = (P I , P2 , . . . , Pn ) be a probability vector. Since p sums to 1, its minimum entry P( l ) and maximum entry P(n ) must satisfy
1
1 n-l
P(l) ::: - < -- , n
P(l)
+ P(n)
:::: P(l)
We relabel the entries of the vector p such that P I (n - 1 ) P I , 0, . . . , 0) . Then p
=
1 n-2 -n - 1 V I + n-- 1 Pn - l
where Pn - l
1+ n -P(l) 1
=
P(l) and P2
= =
1 + (n - 2) P(l) 1 :::: n - 1 n-1 P(n) , and set V I
(
--
=
--
( n - l)pl o 1 -
1 n-l 0, P I + P2 - n , P3 , . . . n-2 -l
= --
)
, Pn ,
is a probability vector with at most n - 1 non-zero entries. The induction step is complete. It is a consequence that sampling from a discrete distribution may be achieved by sampling from a collection of Bernoulli random variables. 7. It is an elementary exercise to show that lP(R 2 ::: 1) = i n, and that, conditional on this event, the vector (TI , T2 ) is uniformly distributed on the unit disk. Assume henceforth that R 2 ::: 1, and write (R, e) for the point (TI , T2 ) expressed in polar coordinates. We have that R and e are independent with joint density function fR,s(r, = rln, 0 ::: r ::: 1, ° ::: < 2n . Let ( Q , \II ) be the polar
e)
e
204
Coupling and Poisson approximLltion
Solutions
[4.11.8]-[4.12.1]
Y), and note that IJI = 8 and e - 1 Q2 = R2 . The random variables2 Q and IJI are independent, and, by a change of variables, Q has density function fQ (q ) = qe - '1.Q , q > O. We recognize the distribution of (Q, IJI) as that of the polar coordinates of (X, Y) where X and Y are independent N(O, 1) variables. [Alternatively, the last step may be achieved by a two-dimensional coordinates of (X,
I
change of variables.]
8.
We have that
9.
The polar coordinates
fR .
(R, 8) of (X, 2r , 9 (r, 0) = 7r
Make a change of variables to find that
10.
By the definition of Z, JP(Z
Y) have joint density function
Y/ X = tan 8 has the Cauchy distribution.
m- l
= m) = h (m) II (1 - h (r)) = JP(X > O)JP(X >
1 I X > 0) . . · JP(X
= m I X > m - 1) = JP(X = m).
11. Suppose g is increasing, so that h(x) = -g(1 - x) is increasing also. By the FKG inequality of Problem (3. l 1 . 1 8b), " = cov(g (U) , -g(l - U)) � 0, yielding the result. Estimating I by the average (2n ) - 1 �� 1 g (Ur ) of 2n random vectors Ur requires a sample of size 2n and yields an estimate having some variance 2nu 2 . If we estimate I by the average (2n ) - 1 {��= 1 g (Ur ) + g(l - Ur ) } , we require a sample of size only n, and we obtain an estimate with the smaller variance 2n(u 2 - ,, ).
12.
(a) By the law of the unconscious statistician, JE
g(Y )fx(Y ) [ g(y)fx(Y) fy (Y) ] = ! fy (y) fY (Y ) dY = I .
(b) This is immediate from the fact that the variance of a sum of independent variables is the sum of their variances; see Theorem (3.3. l lb). (c) This is an application of the strong law of large numbers, Theorem (7.5. 1 ).
13. (a) If U is uniform on [0, 1], then X = sin( � 7r U) has the required distribution. This is an example of the inverse transform method. 1 (b) If U is uniform on [0, 1], then 1 - U 2 has density function g(x) = { 2 .Jf=X} - , 0 � x � 1 . Now g(x) � (7r /4) (x ), which fact may be used as a basis for the rejection method.
f
4.12 Solutions.
1.
Coupling and Poisson approximation
Suppose that JE(u (X)) � JE(u(Y)) for all increasing functions u . Let C E IR and set u Ie (x )
_
{I
- o 205
if x > if x �
c,
c,
= Ie where
[4.12.2]-[4.13.1]
Continuous random variables
Solutions
to find that lP(X > ) = lE(Ic(X)) � lE(Ic(Y)) = lP(Y > Conversely, suppose that X � st Y. We may assume by Theorem (4. 12.3) that X and Y are defined on the same sample space, and that lP(X � Y) = 1 . Let u be an increasing function. Then lP(u(X) � u(Y)) � lP(X � Y) = 1 , whence lE(u(X) - u(Y)) � 0 whenever this expectation exists.
c
c) .
2. Let a = 11- / >" , and let {lr : r � I } be independent Bernoulli random variables with parameter a. Then Z = 2:;=1 Ir has the Poisson distribution with parameter >..a = 11-, and Z .::: X. 3.
Use the argument in the solution to Problem (2.7. 1 3).
4.
For any
A � JR,
lP(X #- Y) � lP(X E A, Y E AC) = lP(X E A) - lP(X E A, Y E A) � lP(X E A) - lP(Y E A), and similarly with X and
Y interchanged.
lP(X #- Y) � 5. For any positive x and follows that
sup
A�1R
Hence,
IlP(X E A) - lP(Y E A) I
y, we have that (y - x) + + x /\ Y
�) fX (k) - fy (k)} +
k
=
L{fy (k) - fx (k)} +
k
=
=
1dTV(X, Y).
=
1
y, where x /\ y
=
min{x, y}.
It
k
- L fx(k) /\ fy (k),
and by the definition of dTV (X, Y) that the common value in this display equals 1dTV(X, Y) = 8. Let U be a Bernoulli variable with parameter 1 - 8, and let V, W, Z be independent integer-valued variables with
lP(V = k) = (fx (k) - fy (k)} + /8, lP(W = k) = (fy (k) - fx (k)} + /8, lP(Z = k) = fx(k) /\ fy (k)/(1 - 8). X' = U Z + (1 - U) V and Y' = U Z + (1 - U) W have the required marginals, and lP(X' = Y') = lP(U = 1) = 1 - 8. See also Problem (7. 1 1 . 16d). 6. Evidently dTV(X, Y) = Ip - q l , and we may assume without loss of generality that p � q . We have from Exercise (4. 12.4) that lP(X = Y) .::: 1 - (p - q ). Let U and Z be independent Bernoulli variables with respective parameters 1 - p+ q and q /( 1 - p + q ) . The pair X' = U (Z- I)+ 1 , Y' = UZ has the same marginal distributions as the pair X, Y, and lP(X' = Y') = lP(U = 1 ) = 1 - P + q as Then
required. To achieve the minimum, we set X"
Y') = P - q .
=
4.13 Solutions.
1
- X' and Y" = Y', so that lP(X" = Y") = 1 - lP(X' = Geometrical probability
2 (0" 1/1 (1/1, 0" ) 1 ( ) -0" . 1/1),
1. The angular coordinates \II and 1: of A and B have joint density f by p = cos{ 1 - )}, () = the change of variables from (p, () ) 1-+ inverse I I I I = () - '!7Z' + cos - p, = () - '!7Z' - cos - p,
J
and Jacobian I I
=
1/1
2/ Vf=P2.
(1/1, 0" )
0"
206
=
27Z' (7Z' +
+
We make with
Geometrical probability
Solutions
[4.13.2]-[4.13.6]
left shaded randomLetlAine,bebytheExampl e region and B the right shaded region in the figure. Writing >.. for the lP'(>.. meets both Sl and S2) = lP'(>.. meets both A and B) = lP'(>.. meets A) + lP'(>.. meets B) -lP'b(H), (>.. meets either A or B) b(A) + b(B) - b(H) = b(X) whenceThelP'case(>.. meetwhens SS2 I >.. meet s Sl ) = [b(X) - b(H)]/b(Sl ).When S n S and S S S l is treated in Example l 2 l 2 2 the argument above shows the answer to be [b(Sl ) + b(S2) - b(H)]fb(Sd. th I I I thneedl e lengte calh ofcultahtieoinntercept I ofThe>" 1requiwithredS2probabi , we havelitythisat lP'(>"2 meets I) = 2 1 / 1 /b(Sl ), by thWie Buffon 21f I S2 1 = 21l'I S2 1 · 1 1 r 21r 1 r . = 2 Jo b(S l ) b(S l ) Jo b(S l ) 2 b(S l ) 2 If the two points are denoted P = (X l , Yl ), and Q = (X2 , Y2), then 2.
(4. 1 3 . 2),
ex
�
=I-
(4. 1 3 . 2).
0
/::,.
=I-
0,
3.
(4. 1 3.2).
1
00
21
dp dB
dB
-00
4.
WeannuluseusCroft orderx)to=cal1&c(ZulaI teP,1&Q(Z).E D),Consiandderfinad dithsatc D of radius x surrounded by an A ofowin'dsthmethod h. WeinsetA( >.. (x +h) = >.. (x) ( � -O(h)) + 21&(Z I P E D, Q E A) c: + O(h)) . Now 1o !1f 102x cosO +0(1) = -, 1&(Z I P E D, Q E A) = 20 0 9 X 1l' 1l' whence - = --x + -, which is easily integrated subject to = to give the91l'result. (i) Wety may assume without loss of generalityThethattrithangle sphere hasdesradian obtuse us Theanglleengthif B Xlie=s eiI AtherOI hasin thdensi functi o n f (x) = 3x 2 for x e i n cl u or in tchule asphere th centre ! X and radius ! X, or in the segment cut offe hemi by thsephere planeopposi throughte tAo A,perpendi r to AO.wiHence, lP'(obtuse) = i +1&( i X)3 ) + (1 1l' ) - 1 1& (I: 1l' (I - ) = i + rt; + ( 1) - 1 1&(� - X + 1X3) = i· (ii) In the case of the circle, X has density function for :::: x :::: I, and similar calculations yield lP'(obtuse) = 2 + "8 + ; 1& (cos- l X - X v - X2 ) = 4 · Choose the x-axis along AB. With P = (X, Y) and G = (Yl , Y2), 1&IABPI = iI ABI1&(Y) = !IABI Y2 = I ABGI · 1 -
2
32x
2 r dr dO
4>..
d>"
1 28
dx >" (0) 0
S.
0 ::::
1.
:::: 1 .
2 y ) dY
2x
1
1
1
0
r;--;;;; 1
6.
207
3
[4.13.7]-[4.13.11] 7.
Continuous random variables
Solutions
We use Exercise (4 . 1 3 . 6). First fix
P,
and then Q, to find that
With b IAB I and h the height of the triangle ABC on the base AB, we have that IG I G2 1 the height of the triangle AG I G2 is h. Hence, =
�
=
1 b and
8. Let the scale factor for the random triangle be X, where X E (0, 1 ) . For a triangle with scale factor x, any given vertex can lie anywhere in a certain triangle having area (1 - x) 2 IABCj. Picking one at random from all possible such triangles amounts to supposing that X has density function f (x) 3 ( 1 - x) 2 , 0 ::::: x ::::: 1 . Hence the mean area is =
9.
We have by conditioning that, for 0 ::::: z ::::: a , F (z , a + da)
=
F (z , a)
=
F (z , a)
( (
)
a --a + da 2 da 1 a
2 + IP'(X ::: a - z) .
2da - ) + -az . + o(da), a
2a da
(a + da) 2
+ o(da)
and the equation follows by taking the limit as da ..t- O. The boundary condition may be taken to be 1 , and we deduce that F (a , a) =
F (z , a)
(Z )2 ,
2z
=
-;; - -;;
0 ::::: z ::::: a .
Likewise, by use of conditional expectation,
lE(
-
Now, (a Xn and therefore
=
a r / (r + 1 ) , yielding the required equation. The boundary condition is m r (0) m r (a)
2a r =
(r + l ) (r + 2)
=
0,
.
10. If n great circles meet each other, not more than two at any given point, then there are 2 (�) intersections. It follows that there are 4 (�) segments between vertices, and Euler's formula gives the number of regions as n (n - 1 ) + 2 . We may think of the plane as obtained by taking the limit as R -+ 00 and 'stretching out' the sphere. Each segment is a side of two polygons, so the average number of sides satisfies 4n (n - 1 )
--'-: -+ -::--'-:
2 + n (n - 1 )
4
P
as n
-+
00.
1 1 . By making an affine transformation, w e may without loss o f generality assume the triangle has vertices A (0, 1 ) , B (0, 0) , C ( 1 , 0) . With (X, Y), we have that =
C: ) =
L
=
Y'
0
=
,
M
=
( � X
=
Y' X
208
: ) Y
,
N
=
(
0,
1
�X
)
.
Problems
Solutions
Hence, JEIBLNI = 2
1
ABC
xy
2(1 - x)(1 - y)
and likewise JEICLMI = JEIANMI = rr 2 ) I AB C I .
dx dy =
irr 2 - � .
101 ( 0
x
-x -
1 -x
--
I g
o x
)
It follows that JEILMNI =
[4.13.12]-[4.14.1]
rr 2
3 2
dx = - - - , 6
� (10 - rr 2 )
=
(10 -
12. Let the points b e P, Q , R , S. B y Example (4. 13.6),
1P'(one lies inside the triangle formed by the other three)
=
41P'(S E PQR) = 4 . -b. .
13. We use Crofton's method. Let m(a) be the mean area, and condition on whether points do or do not fall in the annulus with internal and external radii a, a + h. Then
m(a + h) = m(a)
C : h r + [� + O(h)] m(a),
where m(a) is the mean area of a triangle having one vertex P on the boundary of the circle. Using polar coordinates with P as origin,
Letting h ..J..
0 above, we obtain
whence m(a) =
dm da
(35a2 )/(48rr).
6m a
--
6 35a 2 a 36rr ,
= -- + - '
14. Let a be th e radius of C , and let R b e the distance o f A from the centre. Conditional on R, the required probability is (a - R) 2 /a 2 , whence the answer is JE « a - R) 2 /a 2 ) = JJ (1 - r) 2 2r dr = i .
15. Let a be the radius of C , and let R be the distance of A from the centre. As in Exercise (4.13. 14), the answer is JE « a - R) 3 /a 3 ) = JJ (1 - r) 3 3r 2 dr = 10. 4.14 Solutions to problems
1.
(a) We have that
Secondly, f �
I-L)/(u..ti).
0,
and it is easily seen that
1
-1 1 1 22d
(b) The mean is J�oo x (2rr) - � e � x
-
The variance is J�oo x 2 (2rr) � e - � x
J�oo f (x) dx
=
1
using the substitution y = (x
1 2
-
x , which equals 0 since x e - � x is an odd integrable function.
dx , easily integrated by parts to obtain 1 . 209
[4.14.2]-[4.14.5]
Continuous random variables
Solutions
(c) Note that
1 2 and also 1 - 3y - 4 < 1 < 1 + y - 2 . Multiply throughout these inequalities by e - � Y 1../2ii, and integrate over [x , to obtain the required inequalities. More extensive inequalities may be found in Exercise (4.4.8). (d) The required probability is a (x) = [ 1 - ct> (x + alx)]/[1 - ct> (x)] . By (c),
(0),
as x -+
2.
Clearly f
� 0
if and only if 0 ::5 a C -1 =
lfJ (X
00.
< /3 ::5 1 . Also
2 - x ) dx =
! (/3 2 a 2 ) _
_
� (/3 3 a 3 ) _
.
3. The A i partition the sample space, and i - I ::5 X (w) < i if w E A i . Taking expectations and using the fact that lE(Ii ) = lP(Ai ), we find that S ::5 lE(X) ::5 1 + S where 00
00
i- I
00
00
S = L )i - l )lP(Ai ) = L L 1 · lP (Ai ) = L L i=2
4.
i=2 j=1
j=1 i=j+l
00
lP(A i ) =
L lP(X � j).
j =1
(a) (i) Let F - 1 (y) = sup{x : F (x) = y}, so that 1 1 lP (F (X) ::5 y) = lP(X ::5 F - (y» = F (F - (y» = y ,
(ii) lP ( - log F (X) ::5 y) = lP (F (X) (b) Draw a picture. With = PR,
D
� e -Y ) =
1
-
O ::5 y ::5 1 .
e- Y if y � O.
Differentiate to obtain the result. 5.
Clearly lP (X > s + x I X > s) =
lP(X > s + x) lP (X > s)
=
e- A(s +x ) = e - Ax e- AS
if x , s � 0, where A is the parameter of the distribution. Suppose that the non-negative random variable X has the lack-of-memory property. Then G (x) = lP(X > x) is monotone and satisfies G (O) = 1 and G (s + x) = G (s ) G (x ) for s , x � O. Hence G (s ) = e- AS for some A; certainly A > 0 since G (s) ::5 1 for all s . Let us prove the hint. Suppose that g is monotone with g (O) = 1 and g (s + t) = g (s)g(t) for s , t � O. For an integer m , g (m) = g ( 1 ) g (m - 1) = . . . = g ( 1 )m . For rational x = min, g (x) n = g (m) = g ( 1 )m so that g (x ) = g ( I )X ; all such powers are interpreted as exp{x 10g g (1)}. Finally, if x is irrational, and g is monotone non-increasing (say), then g (u) ::5 g (x) ::5 g (v) for all
210
Problems
Solutions
[4.14.6]-[4.14.9]
rationals u and v satisfying v :::: x :::: u. Hence g ( 1 ) U :::: g(x) :::: g ( 1 ) v . Take the limits as u .,!.. v t x through the rationals to obtain g (x) eiLX where J.L log g ( 1 ) . =
6.
=
fx (x)
=
and 1
=
i: h (y) dy,
g(x)
fx
=
and
h
fy (y) h (y) =
fy .
=
Suppose conversely that
i: g(x) dx
i: fy (y) dy i: g(x) dx i: h (y) dy. =
f (x, y), so that X and Y are independent. They are not independent since lP' (Y < 1 , X > 1 ) O whereas lP' (Y < 1 ) > O and lP' (X >
Hence fx(x)fy(y) 7.
g
If X and Y are independent, w e may take
f (x, y) g(x)h(y). Then
=
x and
=
g(x)h (y)
=
=
100 2e-X -y dy
1) > O.
As for the marginals,
fx (x)
=
=
2e - 2x ,
for x, y ::: O . Finally,
JE(XY) and JE(X)
8.
!, JE(Y)
=
=
=
fy (y)
=
loy 2e-X -Y dx
foo foo xy2e -x -y dx dy
ix =o iy =x
� , implying that cov(X, Y)
=
=
2e -Y (1 - e -Y ),
1
i.
=
P
As in Example (4. 1 3. 1), the desired property holds if and only if the length X of the chord satisfies X :::: .../3 . Writing R for the distance from to the centre 0, and e for the acute angle between the chord and the line we have that X 2";1 - R 2 sin2 e, and therefore IP'(X :::: ...(3) IP'(R sin e ::: !). The answer i s therefore
OP,
=
(
IP' R ::: which equals 9.
=
1
_. _ 2 sm e
)
�
=
n
f !1f IP' ( R ::: -�dB, 2 sm B )
io
� - .../3/ (2n) in case (a) and § + n - 1 10g tan(n 112) in case (b).
Evidently,
JE(U)
=
JE(V)
=
JE(W)
=
IP'(Y :::: g (X »
JE(g (X »
!
2 JE(U )
=
2 JE(W )
=
JE(U)
1
=
1
10 {g(x)
Secondly,
=
J,
11O::o;x , y ::O;l
1
dx dy
=
y ::O;g (x)
10 g(x) dx, + g(1
- x) } dx
2 JE(V )
10 g(x) dx, 0
1
=
10 g(x) dx.
10 g x dx :::: J 1
( )
2
{ 10 g(x)2 dx 2 10 g(x)g(l - x) dX } =
=
since g :::: 1 ,
i 2
1
=
JE(V 2 )
-!
=
JE(V 2 )
l - i io {g(x) - g(1 - x) } 2 dx :::: JE(V 2 ) .
1
+
1
10 g(x) {g(x) - g(1 - x) } dx f
21 1
[4.14.10J-[4.14.11J
Continuous
Solutions
=
random variables
Hence var(W) :::: var(V) :::: var(U).
10.
=
is
Clearly the claim is true for n 1 , since the r (A , 1 ) distribution the exponential distribution. Suppose it is true for n :::: k where k ::: 1 , and consider the case n k + 1 . Writing in for the density function of Sn , we have by the convolution formula (4 .8. 2 ) that
which is easily seen to be the r (A , k + 1) density function.
=
=
11. (a) Let Z l , Z2 , . . . , Zm+n be independent exponential variables with parameter A. Then, by Problem (4. 14. 1 0), X ' Z l + . . . + Zm is r (A , m), yl Zm+ l + . . . + Zm+n is r (A , n) , and X ' + yl is r (A , m + n) . The pair (X, Y) has the same joint distribution as the pair (X ' , yl), and therefore X + Y has the same distribution as X , + yl, i.e., r (A , m + n) . (b) Using the transformation u x + y, v x/(x + y), with inverse x u v , y u ( 1 - v), and Jacobian w e find that U
=
X + y, V
=
=
J=I
=
v 1 -v
u -u
1=
=
-u,
=
X / ( X + Y) have joint density function
for u ::: 0, 0 :::: v :::: 1 . Hence U and V are independent, U being r (A , m + n), and V having the beta distribution with parameters m and n . (c) Integrating by parts, lP'(X
> t)
= 1t 00 (m -m I) ! xm - 1 e-Ax dx 00 = [ -(m-, m_--l)l -!Xm - l e-Ax] t 1t 00 (m, m_-2)l ! Xm -2e-Ax dx = e -At (m(M)-m -I)l! > A
I\.
+ lP'(X '
where X , is r (A , m
I\.
+
t)
- 1 ) . Hence, by induction, m- l (At i lP'(X > t) l L e -J... t k.
= k =O
= lP'(Z < m).
=
(d) This may be achieved by the usual change of variables technique. Alternatively, reflect that, using the notation and result of part (b), the invertible mapping u = x + y, v x / (x + y) maps a pair X, Y of independent (r (A , m) and r (A , n» variables to a pair U, V of independent (r (A , m + n) and B(m, n» variables. Now U V X, so that (figuratively)
=
"r (A , m + n)
Replace n by n
-
x
B (m , n )
m to obtain the required conclusion. 212
=
r (A , m)" .
Problems
12. (a) Z
Solutions =
[4.14.12H4.14.15]
r
X satisfies
z ::: 0,
the
r(1, 1) or x 2 ( 1 ) density function.
(b) H z ::: 0, Z
=
r
�
X + X satisfies
the X 2 (2) distribution function. (c) One way is to work in n-dimensional polar coordinates ! Alternatively use induction. It suffices to show that if X and Y independent, X being x 2 (n ) and Y being X 2 (2) where n ::: , then 2 Z X + Y is x (n + 2) . However, by the convolution formula (4.8 .2),
I
are
=
z ::: 0,
for some constant c . This is the x 2 (n + 2) density function as required. 13. Concentrate on where x occurs in fx I y (x I y ) ; any multiplicative constant can be sorted out later: fX I Y (x I y )
=
fx , y (x , y) fy (y)
by Example (4.5 .9), where
=
( ) { I ( -r - --r -
CI Y
exp -
2(1 - p 2 )
x2
2xIL I
2px (y - IL 2 )
a
a
0'1 0'2
)}
C} (y) depends on y only. Hence
x E
JR,
for some C2 (y). This is the normal density function with mean IL l + pal (y - IL2 )/a2 and variance ar ( 1 - p 2 ) . See also Exercise (4.8.7). 14. Set u x , with inverse x y/x , v u v , and I J I v, y fx , y (v , u v) l v l for -00 V X has joint density fu, v (u , v) =
=
=
J�oo f( v , u v) l v l d v.
=
=
=
15. By the result of Problem (4. 14. 14), U fu (u)
=
=
=
2(v - x)) fx (x) dx = lo
v
e- 2 ( v-x ) e -x
= e- 2 V (e V - 1) + e- V = 1 - (1 - e-v ) 2 = lP'(V > 214
The
v) .
dx +
100
e -X
dx
The
Problems
Solutions
Therefore E(V)
=
E(X) + � E(Y)
� , and var(V)
=
var(X) +
=
18. (a) We have that
(b) Clearly, for
w
i var(Y)
[4.14.18]-[4.14.19]
i by independence.
=
> 0,
JP(U :::: u ,
W
>
JP(U :::: u ,
W
>
Now
w) w,
JP(U :::: u ,
=
X :::: Y)
=
=
and similarly < JP (U -
Hence, for 0 :::: u :::: u + < JP( U -
U,
w
U,
w >
w
W
w,
>
X :::: Y) + JP(U :::: u ,
)
=
=
>
w,
X > Y).
Ae - AX e - JL(x + w ) dx
_A_ e - JL W ( I _ e - (A+JL)U ) A + JL
, X > Y)
=
� e -A W ( 1 A + JL
00,
w
w) iou
JP(X :::: u , Y > X +
W
( 1 - e - (A+JL)U )
(
_
e - (A+JL) U ) .
_A_ e - JL W + � e -A W A + JL A + JL
)
an expression which factorizes into the product of a function of u with a function of are independent.
W
19. U
=
X + Y, V
=
'
w.
Hence U and
X have joint density function fy (u - v) fx (v) , 0 :::: v :::: u . Hence
fv I u (v I u) (a) We are given that fv l U ( v I u)
=
=
fu , v (u , v) fu (u)
=
fy (u - v ) fx (v)
JoU fy (u - y ) fx (y ) dy
.
l u - for 0 :::: v :::: u ; then
fy (u - v) fx ( v)
1 =
-
u
io0u
fy (u - y ) fx ( y ) dy
is a function of u alone, implying that fy (u - v) fx (v)
=
=
fy (u) fx (O) fy (O) fx (u)
by setting v by setting v
=
0
=
u.
In particular fy(u) and fx (u) differ only by a multiplicative constant; they are both density functions, implying that this constant is 1 , and fx fy. Substituting this throughout the above display, we find that g (x) fx(x)lfx (O) satisfies g (O) 1 , g is continuous, and g (u - v ) g ( v ) g (u) for e- AX for x 2: 0 for some A > 0 (remember that g is integrable). 0 :::: v :::: u . From the hint, g (x) (b) Arguing similarly, we find that =
=
=
=
=
fy (u - v) fx ( v )
=
C
u a+.8 - 1
l va - eu - v) .8 - 1
215
f"
fy (u - y ) fx (y ) dy Jo
[4.14.20]-[4.14.22]
Continuous random variables
Solutions
for 0 :::: v :::: u and some constant c. Setting fx ( v ) X (v)v a- 1 , fy (y) 7J (y)y fJ - 1 , we obtain 7J (u - v)X (v) h (u) for 0 :::: v :::: u , and for some function h . Arguing as before, we find that 7J and X are proportional to negative exponential functions, so that X and Y have gamma distributions. =
=
=
20. We are given that U is uniform on [0, 1], so that 0 ::::
E
=
X, Y :::: 1 almost surely. For 0 < E < !,
lP(X + Y < E) :::: lP(X < E, Y < E)
=
lP(X < E) 2 ,
and similarly
E
lP(X + Y > 1 - E) :::: lP(X
=
implying that lP(X
2E
=
=
< E) :::: ./E and lP(X
>
1 - E, Y > ! - E)
>
1 - E) :::: ./E.
=
lP(X > 1 - E ) 2 ,
Now
lP(! - E < X + Y < 1 + E) :::: lP(X > 1 - E, Y < E) + lP(X < E, Y > 1 - E) 2lP(X > 1 - E)lP(X < E) :::: 2( ./E)2 .
Therefore all the above inequalities are in fact equalities, implying that lP(X ./E if 0 < E < ! . Hence a contradiction:
< E) lP(X > 1 - E) =
=
21. Evidently lP(X ( l ) :::: Y l , · · · , X (n ) :::: Yn )
=
L lP (XlI'\
:::: Y l o · · · ,
11'
where the sum is over all permutations :n: (:n: l , :n:2 , term in the sum is equal, whence the sum equals =
. • •
XlI'n
:::: Yn ,
XlI'\ < . . . < XlI'n )
, :n:n ) of ( 1 , 2, . . . , n) . By symmetry, each
The integral form is then immediate. The joint density function is, by its definition, the integrand. 22. (a) In the notation of Problem (4. 14.2 1 ), the joint density function of X (2) , . . . , X (n) is g 2 ( Y2 , · · · , Yn )
=
=
J-oo (Y2
g (Y l , · · · , Yn ) dY l
n ! L ( Y2 , . . . , Yn ) F ( Y2 ) f ( Y2 ) f ( Y3 ) · · · f ( Yn )
where F is the common distribution function of the
Xj .
Similarly
X (3 ) ,
• • •
,
X (n) have joint density
and by iteration, X (k) , . . . , X (n ) have joint density
We now integrate over Yn , Yn l o . . . , Yk+ l in turn , arriving at fX(k) ( Yk )
=
(k
n! _
I ) ! (n
_
k) !
l F ( Yk ) k - { I - F ( Yk ) }n -k f ( Yk ) ·
216
Problems
Solutions
[4.14.23]-[4.14.25]
ly. FiSx ix,s diandstrilbeuted t Ir beastbiheni(n,ndicF(atorx)),functi ... +diIn.rectThen l(ebt) S is neat It +er12 +argue and on of the event {Xr ::: xl, and lP'(X(k) ::: x) lP'(S :::: k) t=k (;) F(x)l - F(x))n-l. l Differentiate to obtain, with F F(x), It
to
=
=
=
(1
=
by successive cancellation of the terms in the series. Usi n g the resul t of Probl e m t h e joi n t densi t y functi o n is g(y) n! L(y)T -n for ::: Yi ::: T, ::: i ::: n, where y (YI , Y2 , ... , Yn ). on of X( is (x) k (�)x(ak)-Wel make -x)n-usek forof Probl ::: xe:::ms so that the density functiTheondensiof nXty (functi k) is k) fk -fn k (x ln) k!k n(n - ...nk(n -k + xk- l ( xn ) n-k --(k - l )! xk- l e as(b)nFor an increasi The linmgitsequenceX( is the k)), X(densi) , t. y. .functi oinn. we define the sequence U -n 10gX( ), , X( ) n ng has inverse n n Uk -klog(X(k)lx(k+I )) for :::I k .
=
(4. 14.22)-(4. 14.23).
°
(1
1)
1 -
-
1
--+-
r(1,
=
[0, 1 ] ,
=
1
=
=
-x
I
J=
0, 1 1.
=
=
=
=
1
1,
=
=
25.
three
1 -3
=
(0, 0, 0) , ( 1 , 0, 1 ) , (0, 1 , 1 ) , (0, 0, 1 ) ,
1
(0, 0, . . . , 0) , ( 1 , 0, . . . , 0) , ( 1 , 1 , 0, . . . , 0) , ( 1 , 0, 1 , 0, . . . , 0) , . . . , ( 1 , 0, . . . , 0, 1 ) .
217
[4.14.26]-[4.14.27]
Continuous random variables
Solutions
Mapping Xl 1-+ 1 - X l , we see that this has the same volume Vn as has the convex hull of the origin o together with the n unit vectors el , e2 , . . . , en . Clearly V2 = �, and we claim that Vn = l i n ! . Suppose this holds for n < k, and consider the case n = k. Then
where Vk - l (0, x l e2 , . . . , xl ek ) is the (k - I )-dimensional volume of the convex hull of 0, x l e2 , . . . , x l ek . Now
so that v:k -
10 1 0
k l xl 1 --- dx - - '
(k - I ) ! = 1
The required probability is therefore 1 - n l (n !)
1-
k!
l
- { (n _ l ) ! } - .
26. (i) The lengths of the pieces are U = min{X l , X2 } , V = I X I - X2 1 , = 1 - U - V, and we require that U < V + etc, as in the solution to Problem (4. 14.25). In terms of the Xi we require
W
W,
either :
I X I - X2 1
or :
I X l - X2 1
< �, < �,
1 - X2 l - Xl
< �, < �'
Plot the corresponding region of R2 . One then sees that the area of the region is i , which is therefore the probability in question. (ii) The pieces may form a polygon if no piece is as long as the sum of the lengths of the others. Since the total length is I , this requires that each piece has length less than �. Neglecting certain null events, this fails to occur if and only if the disjoint union of events AO U A l U . . . U A n occurs, where AO
=
{ no break: in (0, � ] } ,
Ak
=
{ no break: in (Xk o Xk + �l) for 1
remember that there i s a permanent break: at 1 . Now JP(AO)
Hence JP(AO U A l U . . . U A n )
= (n +
=
� k � n;
(�) n , and for k 2: I ,
I ) 2-n whence the required probability i s 1 - (n + I ) 2 -n .
27. (a) The function g (t ) = (tP I p) + (t - q Iq ), for t > 0, has a unique minimum at t l l g (t ) 2: g ( l ) = 1 for t > O. Substitute t = x / q y - / p where
(we may as well suppose that JP( X Y IXjP plE l X P I
= 0)
+
=1= 1 ) to find that
I Y lq qlEl yq l
2:
IXYI
{lE I X P l } l / p {lE l yq l } l / q
HOlder's inequality follows by taking expectations.
218
'
=
I , and hence
Problems
Solutions =
[4.14.28]-[4.14.31]
Z I X YI . lE(ZP) lE(Z . ZpI - l ) ::: lE(I XI I ZP- I ) lE(IIY I ZP- I ) I ::: {lEI XPI } /p{lE(ZP)} /q {lEl yPI } /p{lE(ZP)} /q - l q- l {lE(ZP)} I /q -a) I Z I �(b+a) , ::: ::: �( b I Z I 2 + a -a ::: {lEI ZbI } ::: lEI Zb llElI zb l . M) xx < ::: x, X, Y, Z f. f(x , y, z) dx. lE(X I Y y, Z z) jXfXI Y, Z (X I y, z) x jx !y.z (y, z)
(b) We have, with
+
=
+
+
by Holder's inequality, where p
= 1.
+
Divide by
to get the result.
28. Apply the Cauchy-Schwarz inequality to and where 0 a b, to obtain Now take logarithms: 2g (b ) g (b - a) + g (b + a) for O ::: a ::: b. Also g (p) --+ g (O) = as p .j.. 0 (by dominated convergence). These two properties of g imply that g is convex on intervals of the form [0, if it is finite on this interval. The reference to dominated convergence may be avoided by using Holder instead of Cauchy-Schwarz. By convexity, g ( ) / is non-decreasing in and therefore g (r ) / r ;:: g (s ) /s if 0 s r. 29. Assume that
are jointly continuously distributed with joint density function =
=
=
Then
d =
Hence
lE{lE(X I Y, Z) I Y
=
y } j lE(X I Y y, Z z)fzI Y (z I y) dz {[ x f(x , y, z) !y.z (y, z) dx dz }} !y.z (y, z) fy(y) �r x f(x , y, z) dx dz lE(X I Y y) . fy(y) =
=
=
= =
=
=
Alternatively, use the general properties of conditional expectation as laid down in Section 4 . 6 .
x
x I
30. The first car to arrive in a car park of length + effectively divides it into two disjoint parks of length and where is the position of the car's leftmost point. Now is uniform on [0, and the formula follows by conditioning on Laplace transforms are the key to exploring the asymptotic behaviour of as --+ 00.
x) ,
Y
-
Y,
Y
Y.
m (x) /x x
Y
31. (a) If the needle were of length d, the answer would be 2/rr as before. Think about the new needle as being obtained from a needle of length d by dividing it into two parts, an 'active' part of length L , and a 'passive' part of length d - L , and then counting only intersections involving the active part. The chance of an 'active intersection' is now (2/rr ) (L /d) = 2L / (rr d) . (b) A s in part (a), th e angle between th e line and th e needle i s independent of the distance between the line and the needle's centre, each having the same distribution as before. The answer is therefore unchanged. (c) The following argument lacks a little rigour, which may be supplied as a consequence of the statement that S has finite length. For E > 0, let . . . , Xn be points on S, taken in order along S, such that and Xn are the endpoints of S, and < E for 0 < n; - y l denotes the Euclidean distance from to Let be the straight line segment joining to and let lj be the indicator function of n A =;f If E is sufficiently small, the total number of intersections between U It U · · · U In - l and S has mean
xo
Jo
{Jx y . i
0}J.i
Xl I,XXi+2 ,1 -Xi I
219
::: i Xi IXxi+ !,
[4.14.321-£4.14.34]
Continuous random variables
Solutions
by part (b). In the limit as E � 0, we have that
E i lE(li) approaches the required mean, while
n-l rrd L I Xi + l 2
i =O
-xi i
�
2L (S)
-- .
rrd
32. (i) Fix Cartesian axes within the gut in question. Taking one end of the needle as the origin, the other end is uniformly distributed on the unit sphere of R3 . With the X-ray plate parallel to the y)-plane, the projected length V of the needle satisfies V :::: v if and only if l Z I � �, where Z is the (random) z-coordinate of the 'loose' end of the needle. Hence, for ° � v �
(x ,
I,
since 4rr � is the surface area of that part of the unit sphere satisfying Iz l � � (use Archimedes's theorem of the circumscribing cylinder, or calculus). Therefore V has density function fv ( v ) = v /� for O � v � 1 . (ii) Draw a picture, if you can. The lengths of the projections are determined by the angle e between the plane of the cross and the X-ray plate, together with the angle \11 of rotation of the cross about an axis normal to its arms Assume that e and \11 are independent and uniform on [0, �rr]. If the axis system has been chosen with enough care, we find that the lengths of the projections of the arms are given by .
A, B
with inverse \11 =
tan
_1 � -I -- A22 - . B
Some slog is now required to calculate the Jacobian J of this mapping, and the answer will be fA , B (a , b) = 4 1 J l rr - for 0 < a , b < 1 , a + b > 1 .
2
33. The order statistics of the
2 2
Xi have joint density function
on the set I of increasing sequences of positive reals. Define the one-one mapping from I onto (0, oo) n by Yr = (n + 1 r) (xr - Xr - l ) for 1 < r � n , Y l = nX l , with inverse Xr function of Yl ,
-
= E k = l Yk / (n - k + 1 ) for r :::: 1 . Y2 , . . . , Yn is
The Jacobian is (n !) - l , whence the joint density
34. Recall Problem (4 . 14 . 4). First, = 1 � i � n, is a sequence of independent variables with the uniform distribution on [0, 1 ] . Secondly, a variable U has the latter distribution if and only if - log U has the exponential distribution with parameter 1 .
Zi F(Xi),
220
Problems
[4.14.35]-[4.14.37]
Solutions
Itfol ltriowsthatLj = -log F(Xj), 1 ::: i ::: n,is a sequence of independent variables with the expo nent i a ldi s b ut i o n. Theorderstati sticsL(ncreasi l ), ...ng., LAppl (n) in order -log F(X (n) ), . . . , -log F(X( I )), si-nncelogthF(X e functi(n )oandn -log FO is non-i ying the result of Problem (4. 1 4.33), El ) Er = -(n 1 -r){log F(X(n+ l -r) -log F(X(n+2-r) ) } , 1 < r ::: n, withinthedependent uniformwidithstrithbeutiexponenti on. al distribution. Therefore exp(-Er), 1 ::: r ::: n, independent One may be saiThere d beareintwstoatpossi e j ifbthele decifirstsijons- at1 prithizsesstage: have accept been rejected andprizteheifjthit isprithzeebesthas justso farbeen vi e wed. the jth s no pois thent probabi in acceptilityngjIitnitfhiatt itshnot), orprireject iteandbestconti nue.andThethemean return ofof tthhee fisecond rst deci(there sis itohneiequal e jt h z e i s t h so far, mean return Thus the maximaxi mal mmeanal probabi returnliVty(j)f (j)in stthaatteonej satimaysfieobts ain the best prize having rejected the first j. V(j) = max{jln, f(j)}. jalIsno).increases withthj,ereandexifsts decreases wijInth j:::(sifn(j)ce aifpossiandbonlle stategy is to reject the (jrmslt)thhe priNow z e Therefore such t h at Thi s confi y i f j ::: optsubsequent imal statpriegyzeaswhihavichnigs thethe best folloofwinthgoseform:viewedrejectso tfar.he firsttherepriiszesno outsuchofprihand,ze, weandpicaccept tahste k t h e l prizeLetpresented. best prithzate theby folprilzoewipincgkedtheisabove gy. Let Ak be the eventfIthatJ beyouthepiprobabi ck the lity priof zachie, andevinBg tthhee event the best.strateThen, n (k) ( n n 1 . -1 ) = - L TI J = L IP' (B I Ak )IP'(Ak ) = L k= J + l n k - 1 k n k= J + l k - 1 , k=J + l and weWhenchoosen is thelarge,intewegerhavewhithechasymptoti maximizesc relthiastiexpresi on. In) log(nl The maximum of the o n TI J functieomn hwasn (x)posed= (xln) occursOurat xsol=unltion, isandduewetodeduce at ]nl . [A version of this probl by Cayllog(nlx) ey in 1875. Lindleyth(1961). The joint density function of (X, Y, Z) is 1 exp{ -1(r2 - 2Ax - 2JLY - 2vz 1..2 JL2 v2) } f(x, Y, z) = (2nD" 2 ) 3 /2 z2 . Thechoosicondingtspheri whereexp{rA2x = xJL2Y yvz}.2 Now ional densi tyaofr coordi X, Y,nZatesgivwienth axi= srinisttherefore proporti onal c al pol h e di r ecti o n (A , JL , v), weconstobtantainis achosen densityinfuncti such oanwayproport that itheonaltotalto exp(a probabicosO) lity isis unin O,ty.where a = rV1..2 JL2 v2. The (a) ¢'(x) = -x¢ (x), so HI (x) = x. Differentiate the equation for Hn to obtain H +l (x) = xHn (x) - H� (x), and use induction to deduce that Hn is a polynomial of degree n as required. Integrating by parts gives, when ::: n, are
+
are
are
to
35.
(j)
+
J
J.
J
If
kth
J
J
J
e
�
(J
J).
J
�
e
36.
+
to
+
+ +
+
+
R
+
+
+
n
37.
m
f: Hm (x)Hn (x)¢ (x) dx = (_ I)n f: Hm (x)¢ (n) (x) dx = ( _ l) n - l f: H:n (x)¢ (n - l ) (x) dx
oo = . . . = ( _ l) n -m r H�m) (x)¢ (n -m) (x) dx, L oo 22 1
[4.14.38]-[4.14.39]
Continuous random variables
Solutions
and the claim follows by the fact that H�m) (x) = m I . (b) �y Taylor's theorem and the first part,
tn ( t) n (n ) ¢ (x) E -! Hn (x) = E0 �¢ (x) = ¢ (x - t), n=O n n= n ! 00
00
whence
are
38. The polynomials of Problem (4. 1 4.37) are orthogonal, and there unique expansions (subject to mild conditions) of the form u(x) = E�o ar Hr (x) and v(x) = E�o br Hr (x). Without loss of generality, we may assume that JE(U) = JE(V) = 0, whence, by Problem (4. 14.37a), ao = bo = O. By (4. 14.37a) again, var(U) =
00
JE (u (X ) 2 ) = E a; r ! , r= 1
var(V) =
00
E b; r ! . r= 1
By (4. 14.37b),
JE
(� Hm�)Sm � Hn ��)tn )
= JE (exp {s X
_
� s 2 + tY �t 2 }) = estP . _
By considering the coefficient of s m t n ,
and so
where we have used the Cauchy-Schwarz inequality at the last stage. 39. (a) Let Yr = X (r ) - X (r - l ) with the convention that X (O) = 0 and X (n + l ) = 1 . By Problem (4. 14.21) and a change of variables, we may see that Yl , Y2 , . . . , Yn + l have the distribution of a point chosen uniformly at random in the simplex of non-negative vectors y = (Y l , Y2 , . . . , Yn + 1 ) with sum 1 . [This may also be derived using a Poisson process representation and Theorem (6. 1 2.7).] Conse quently, the Yj are identically distributed, and their joint distribution is invariant under permutations of the indices of the Yj . Now E�if Yr = 1 and, by taking expectations, (n + I)JE(Yl ) = 1 , whence JE(X (r» ) = r JE ( Yl ) = r/ (n + 1 ) . (b) We have that
10 1
JE(Yf ) = o 1 =
JE
x2n ( 1
- x) n - l dx =
2 ) Yr (I: [ r=1 ] =
(n +
222
2 (n + l ) (n + 2) '
I)JE(Yf ) + n (n + I)JE(YI Y2 ),
Problems
Solutions
implying that
1 , (n + 1)(n + 2)
JE(YI Y2 ) = and also
[4.14.40]-[4.14.42]
JE(X (r) X (s » = r JE(YI2 ) + r es - I)JE(YI Y2 ) =
1) 1)(n + 2)
r es + (n +
The required covariance follows. 40.
(a) By paragraph (4.4 . 6),
(4.7.14).
X 2 is r(�, �) and y 2 + Z2 is r(1 , 1). Now use the results of Exercise
(b) Since the distribution of X 2 / R 2 is independent of the value of R 2 = X 2 + y 2 + also if the three points are picked independently and uniformly within the sphere. 41. (a) Immediate, because the N(O, (b) We have that
Z2 , it is valid
1) distribution is symmetric.
i: 2¢ (x) (x)[1 - r) dr = 10 1 -dr = 2 10g 2 - 1 , 1 0 +r 1 - r) dr = 3 - 4 10g 2, 1 lE(R2 ) = 10 2rlP'(R > r) dr = 10 2r(1 1 +r
lE(R) =
o o
and var( R )
= 2 - (2 log 2) 2 .
0
55. With an obvious notation,
By a natural re-scaling, we may assume that a = 1 . Now, X l - X2 and YI - Y2 have the same triangular density symmetric on (- 1 , 1), whence (X l - X2 ) 2 and (Yl - Y2 ) 2 have distribution
226
Problems
Solutions
{
function F(z) = 2...;z - Z and density function the density f given by
f (r) = The claim follows since
lb --a ...;z � l
-
i
1
fz (z) = z - !
- 1 , for 0 ::: Z ::: 1 . Therefore
) dz ( ) (_ 10 ...;z � 1 ) dz 1r - l ( ...;z ) ( _ � r _ 1
_
1
1
1
1 _-1
dz =
1
_
( �r
. -1 2 sm
if 0 ::: if 1
-1
. 1 - - sm -
[4.14.56]-[4.14.59]
l)-r
:::
r :::
R 2 has
1,
r ::: 2.
for O ::: a ::: b ::: 1 .
56. We use an argument similar to that used for Buffon's needle. Dropping the paper at random amounts to dropping the lattice at random on the paper. The mean number of points of the lattice in a small element of area dA is dA . By the additivity of expectations, the mean number of points on the paper is A. There must therefore exist a position for the paper in which it covers at least r A 1 points.
57. Consider a small element of surface d S . Positioning the rock at random amounts to shining light at this element from a randomly chosen direction. On averaging over all possible directions, we see that the mean area of the shadow cast by d S is proportional to the area of d S . We now integrate over the surface of the rock, and use the additivity of expectation, to deduce that the area A of the random shadow satisfies lE(A) = c S for some constant C which is independent of the shape of the rock. By considering the special case of the sphere, we find C = ! . It follows that at least one orientation of the rock gives a shadow of area at least ! S.
Yr = Xr/(X l + . . . + Xr) is independent of X l + Xr+ l , Xr +2 , ' " , Xk+ 1 ' X l + . . . + Xk+ 1 ' Therefore Yr is
58. (a) We have from Problem (4. 14. 1 1b) that ...+ and therefore of the variables
Xr,
independent of {Yr+s : s � I }, and the claim follows. (b) Let S = X l + . . . + Xk+ l ' The inverse transformation xk+ 1 = S - Z l S - Z2S - . . . - ZkS has Jacobian
S
0
S
0
Xl
Z l S, X2
=
Zl Z2
0 0
0
0
0
-s
-s
-s
{IT (zrs)fJr -l e-'Azrs } r (fJr)
=
ZkS,
=s .
The joint density function of X l , X2 ,
r=l
Z2 S, . . . , Xk
k
J =
j.}r
=
.
Zk
1 - Z l - . . . - Zk
. . . , Xko S is therefore (with
C1
=
E�;!;� fJr),
j.}k+1 ( s ( 1 - Z l - . . . - Zk ) } fJk+l - l e-'As( 1 -z 1 -"'-Zk )
=
f('A, {j, s)
r (fJk+ d k zfr - l r=l
(IT
)
l (1 - Z l - . . . - Zk ) fJk+l - ,
f is a function of the given variables. The result follows by integrating over s. 59. Let C = (crs ) be an orthogonal n n matrix with Cn i = 1/.;ri for 1 ::: i ::: n. Let Yir = E�=l XisCrs, and note that the vectors Yr = (Ylr > Y2r > . . . , Ynr), 1 ::: r ::: n, are multivariate normal. Clearly lEYir = 0, and where
x
lE(Yir Yjs) = L CrtCsu lE(Xi t Xj u ) = L Cr t csu8tu Vij t,u t,u 227
=
L Crt Cs t Vjj t
=
8rs vjj ,
[4.14.60]-[4.14.60]
Continuous random variables
Solutions
where 8t u is the Kronecker delta, since C is orthogonal. It follows that the set of vectors Yr has the same joint distribution as the set of Xr . Since C is orthogonal, X i r = E�= 1 Csr Yis ' and therefore
Sij
1 1 1 E CsrCt r Yis Yj t - -n E Xir E Xjr = E 8s t Yis Yj t - n E Xir n E Xjr r r r r r, s ,t "I "I s ,t n-l = E Yi s Yj s - Yi n Yjn = E YiS Yj S ' s s=1 =
r.::
r.::
This has the same distribution as
1ij
because the Yr and the Xr are identically distributed.
60. We sketch this. Let lE/PQR/ = m (a), and use Crofton's method. A point randomly dropped in a + da) lies in S (a) with probability
S(
(
a -a + da
Hence
)2
=
16
2da a
-
dm m - = -- + da a
+ o(da) .
6mb a
--
,
where mb (a) is the conditional mean of /PQR/ given that P is constrained to lie on the boundary of S (a). Let b (x ) be the conditional mean of /PQR/ given that P lies a distance x down one vertical edge. x
P
1------;
R2 By conditioning on whether Q and R lie above or beneath P we find, in an obvious notation, that
(4.13.6)
(4. 13.7» ,
! !
l
!
By Exercise (see also Exercise m R " R2 = ( a) ( a) = a . In order to find m R " we condition on whether Q and R lie in the triangles or T2 , and use an obvious notation. Recalling Example that example,
Tl
iT . ! ax . 1 - 4I ax - gI ax } . T2 = :11 . 4lJ { ax - 4ax
(4. 13.6), we have that m T, m T"
= m T2 =
2
Next, arguing as we did in
Hence, by conditional expectation,
1
1
4 . :1 ax + m R , = 4 . 'l"i
4 . 41 . 'l"i
1
1
13
3
4 :1 ax + :1 . lJ . gax = TIlir ax .
We replace x by a - x to find m R2 ' whence in total _
b (x) -
(:,) 2 a
1 3ax l OS
+
( - X)2 a
a
1 3a (a - x) l OS
+
228
2x (a - x) a _ � . a a2 S - l OS
2
2
_
1 2ax l OS
+
1 2x
2
l OS
.
Problems
Solutions
[4.14.61H4.14.63]
Since the height of P is uniformly distributed on [0, a], we have that
We substitute this into the differential equation to obtain the solution m (a) = tka 2 . Turning to the last part, by making an affine transfonnation, we may without loss of generality take the parallelogram to be a square. The points form a convex quadrilateral when no point lies inside the triangle formed by the other three , and the required probability is therefore 1 - 4m (a) I a 2 =
1-
& = �.
61. Choose four points P, Q, R, S uniformly at random inside C, and let T be the event that their convex hull is a triangle. By considering which of the four points lies in the interior of the convex hull of the other three, we see that lP'(T) = 4lP' ( S E PQR) = 4lEIPQRI/ I C I . Having chosen P, Q, R, the four points form a triangle if and only if S lies in either the triangle PQR or the shaded region A. Thus, lP'(T) = {IAI + lEIPQRI }/ICI, and the claim follows on solving for lP'(T). 62. Since X has zero means and covariance matrix I, we have that lE(Z) = covariance matrix of Z is lE(L'X'XL) = L'IL = V. 63. Let D
/L +
lE(X)L = /L, and the
= (dij ) = AD - C. The claim is trivial if D = 0, and so we assume the converse. Choose
i, k such that dik i= 0, and write Yi = 'L,}=l dij Xj = S + dikXk . Now lP' (Yi = 0) = lE (lP' (Xk = -S Idik I S») . For any given S, there is probability at least � that Xk i= -S Idik' and the second claim
follows. Let x l , X2 , . . . , Xm be independent random vectors with the given distribution. If D i= 0, the probability that Dxs = ° for 1 .::: s .::: m is at most (�) m , which may be made as small as required by choosing m sufficiently large.
229
5 Generating functions and their applications
5.1 Solutions. Generating functions
1.
ls i < (1 _ p) - l , G(s ) = [; sm (n+ : - 1 ) pn (1 _ p)m = { 1 - S � _ p) } n . 2 = n(1 -p)lp2 . G'(1) = n(1 p)lp. G"(1) +G'(1)-G'(1) ls i < 1, G(s ) = mf= l sm ( +1 _1 ) = 1 + ( 1 -s S ) g(1 -s) 1 p < G'l s i (1)< p=-00,l , is } . G(s) = m=f-oo sm ( 11 + p ) p lm l = 11 +- p { I + 1 -sp + 1 -p(pis) G'(1) = G"(1) = 2p(1 -p) -2 . fA T(s ) = ?;00 sn ( n) = (?;00 sn f{n <XJ ) = (X?;- l sn) = ( 11-_ssX ) = I -1 _G(sS) . (a) If
Therefore the mean is (b) If
The variance is
� m
Therefore (c) If
_
lo
_
m
and there exist no moments of order or greater.
-P
The mean is 2.
P
�
0, and the variance is
(i) Either hack it out, or use indicator functions p
(ii) It follows that
.
X >
thus:
lE
lE
lE
T(1) = st l { 1 -1 -sG(S) } = st l G'1(s) = G'(1) = lim
lim
lE(X)
by L'Hopital's rule. Also,
T'(1) = st l { -(1 -s)G'((1 -s)s ) +2 1 - G(S ) } = �G"(I) = H - G'(1) + G'(1)2 } lim
var(X)
230
Generating functions
Solutions
whence the claim is immediate. (i) We have that Gx, Y (s, t) (ii) If lElXYI < 00 then
3.
[5.1.3]-[5.1.5]
= lE(s x t y ), whence G x ,Y (s, 1) = G x (s) and G X, y(1 , t) = Gy(t).
-
a2 . lE(XY) = lE ( XY X - 1 t Y 1 ) I =t =1 = -- G x ,Y (s, t) as at 8 = =1 t 8 S
4.
(a)
I
We write G (s, t) for the joint generating function.
G(s, t) =
i :�::> i t k (1 - a)(f3 - a)a i f3 k - j - 1 :E j=O k=O 00
f= ( as ) j
- a)(f3 - a) . 1 - (f3t) i + 1 if f3 ltl < 1 f3 1 - f3t j=O f3 f3t - a) 1 = ( 1 -( 1 a)(f3 1 - ast - f3t)f3 1 - (as/f3) (1 - a)(f3 - a) (1 - ast)(f3 - as) (the condition alstl < 1 is implied by the other two conditions on s and t). The marginal generating functions are 1 -a (1 - a) G(1, t) = G(s, 1) = (1 --asa)(f3 -, 1 - at )(f3 - as ) ' and the covariance is easily calculated by the conclusion of Exercise (5. 1 .3) as a(1 - a) - 2 . (b) Arguing similarly, we obtain G(s, t) = (e - l)/{e(1 - te8- 2 )} if Itle8- 2 < 1 , with marginals 1 - e- 1 1 - e- 1 G(s, 1) = 1 - e8-2 ' G(1 , t) = 1 - te- l ' and covariance e(e - 1) - 2 .
=
(1
}
{
(c) Once again,
G(s, t) =
log { l - tp ( 1 - P + sp) } log( 1 p) _
if Itp(1
- p + sp) 1
n)p 2 q = JP(X = n + l)pq + JP(X = n + 3). We multiply by s n +3 and sum over n to obtain
x 1 - JE(s_ _----'- ) p2 qs 3 = pqs 2JE(s X ) + JE(s x ), l -s
which may be solved as required. Let Z be the time at which THT first appears, so By a similar argument,
Y = min {X , Z}.
JP(Y > n)p2 q = JP(X = Y = n + l)pq + JP(X = Y = n + 3) + JP(Z = Y = n + 2)p, JP(Y > n)q 2 p = JP(Z = Y = n + l)pq + JP(Z = Y = n + 3) + JP(X = Y = n + 2)q. We multipy by s n + l , sum over n, and use the fact that JP(Y
=
n) = JP(X = Y = n) + JP(Z = Y = n).
Suppose there are n + 1 matching letter/envelope pairs, numbered accordingly. Imagine the envelopes lined up in order, and the letters dropped at random onto these envelopes. Assume that exactly j + 1 letters land on their correct envelopes. The removal of any one of these j + 1 letters, together with the corresponding envelope, results after re-ordering in a sequence of length n in which exactly j letters are correctly placed. It is not difficult to see that, for each resulting sequence of length n, there are exactly j + 1 originating sequences of length n + 1 . The first result follows. We multiply by si and sum over j to obtain the second. It is evident that G 1 (s) = s. Either use induction, or integrate repeatedly, to find that G n (s) = E�=o (s - or Ir !. 7.
8.
We have for
Is I < JL + 1 that
f(
)
JL _S_ k JE(S X ) = JE{JE(s x I A )} = JE(e A(s - l ) = JL - (s - 1) = � JL + 1 k=O JL + 1 9.
Since the waiting times for new objects are geometric and independent,
( ) ( ---s s ) ( 4 -s s )
3S JE(s T ) - s -4-s Using partial fractions, the coefficient of s k is
2
--
3
.
fz { ! (!)k-4 - 4(!) k-4 + i (�)k-4}, for k ::: 4.
5.3 Solutions. Random walk
1.
Let Ak be the event that the walk: ever reaches the point k . Then Ak
2
r- l JP(M ::: r) = JP (Ar ) = JP(Ao) II JP(Ak+ 1 I Ak ) = (plq) r , k=O 234
Ak + l if k ::: 0, so that
r ::: 0,
Random walk since IP(Ak+1
2.
Solutions
[5.3.2]-[5.3.4]
I Ak ) = IP(A I l Ao ) = p/q for k � 0 by Corollary (5.3 .6).
(a) We have by Theorem (5.3 . 1c) that 00
", s 2k 2kfo (2k) = s Fo (s) = k=l L...J
I
s2 V I - s-
�
=
00
k L...J s 2 po(2k - 2), s 2 IPo (s ) = ", k=l
and the claim follows by equating the coefficients of s 2k . (b) It is the case that an = IP(SI S2 ' . . S2n =1= 0) satisfies 00
an = L fo(k) , k=2n+2 k even with the convention that ao = 1 . We have used the fact that ultimate return to 0 occurs with probability 1 . This sequence has generating function given by
!
k- l 00 00 n 2 s o(k) = L L f L fo(k) L s 2n n=O n=O k=2n+2 2 kk= even k even 1 _ 1 - Fo(s) _ - l - s2 - � 00
by Theorem (5.3. 1c)
00
=
Now equate the coefficients of function of the an directly.)
s 2n .
IPo(s) = L s 2n IP(S2n = 0) . n=O
(Alternatively, use Exercise (5. 1 .2) to obtain the generating
3.
Draw a diagram of the square with the letters ABCD in clockwise order. Clearly PAA (m) = 0 if is odd. The walk is at A after 2n steps if and only if the numbers of leftward and rightward steps are equal and the numbers of upward and downward steps are equal. The number of ways of choosing 2k horizontal steps out of 2n is (�) . Hence m
with generating function
Writing FA (s) for the probability generating function of the time T of first return, we use the argument which leads to Theorem (5.3 . 1 a) to find that GA(S) = 1 + FA (S)GA(S), and therefore FA (S) = 1 - GA(S) - l .
Write (Xn , Yn ) for the position of the particle at time n. It is an elementary calculation to show that the relations Un = Xn + Yn , Vn = Xn - Yn define independent simple symmetric random walks U and V. Now T = min{n : Un = m}, and therefore G T (S) = {s - l ( 1 - �) } m for ls i ::: 1 by Theorem (5.3.5).
4.
235
[S.3.SHS.3.6]
Now X
Generating functions and their applications
Solutions
- Y = VT , so that
where we have used the independence of U and V. This converges if and only if I ! (s + s - 1 ) I � 1, which is to say that s = ± l . Note that G T (S) converges in a non-trivial region of the complex plane.
S
T
O.
S. Let be the time of the first return of the walk to its starting point During the time-interval (0, the walk is equally likely to be to the left or to the right of 0, and therefore
T),
L 2n =
{ T R + L' 2n R
if T � if T >
2n, 2n,
where R is Bernoulli with parameter ! , L' has the distribution of L 2n - T , and R and L' are independent. It follows that G 2n (S) = JE(s L2n ) satisfies
n G 2n (S) = L ! (1 + s 2k )G 2n _ 2k (s)f(2k) + L ! (1 + s 2n )f(2k) k=1 k >n where f(2k) = lP'(T = 2k). (Remember that L 2n and T are even numbers.) Let H(s, t) I:�o t 2n G 2n (S). Multiply through the above equation by t2n and sum over n, to find that H(s, t) = ! H(s, t) (F(t) + F(st}} + ! ( J (t) + J (st}} where F(x) = I:�o x 2k f(2k) and 00 1 ' J (x) = L x 2n L f(2k) = � 1 x n=O k > n
Ix l
2m) + L 2kfo (2k) = 2mlP'(S2m = 0) + L lP'(S2k -2 = 0) . k=1 k=1 236
Random walk Hence,
Solutions
00
[5.3.7]-[5.3.8]
18 2 s 2 Po(s) (s) + = s P L s 2m lE( T 1\ 2m) = O 1 - s2 (1 - s 2 ) 3/2 · m =O Finally, using the hitting time theorem (3. 10. 14), (5.3.7), and some algebra at the last stage, I
Let In be the indicator of the event {Sn = OJ, so that Sn +l = Sn + Xn +l + In. In equilibrium, lE(So) = lE(So) + lE(Xl) + lE(lo), which implies that lP'(So = 0) = lE(lo) = -lE(Xl) and entails lE(Xl) :::; O. Furthermore, it is impossible that lP'(So = 0) = 0 since this entails lP'(So = a) = 0 for all a < 00. Hence lE(X 1 ) < 0 if S is in equilibrium. Next, in equilibrium, 7.
Now,
lE{ zSn +Xn + ! +In (1 - In)} = lE(zSn I Sn > O) lE(zX! )lP'(Sn > 0) lE(z Sn + Xn + ! +In In ) = z lE(zX! )lP'( Sn = 0). Hence
]
lE(Z SO) = lE(zX! ) [{lE(zSO) - lP'(So = O)} + z lP'(So = 0) which yields the appropriate choice for lE(zSO ) . 8. The hitting time theorem (3. 10. 14), (5.3.7), states that lP'(TOb = n) = (lbljn)lP'(Sn = b), whence
lE(TOb I TOb
q that lP'(TOb < ) = 1 , and lE(TOl) = (p - q) - . The result follows from the fact that lE(TOb ) = blE(Tol) . where
(0)
(0
237
[5.4.1]-[5.4.4]
Generating functions and their applications
Solutions
5.4 Solutions. Branching processes
Clearly 1&(Zn I Zm ) = ZmtLn - m since, given Zm , Zn is the sum of the numbers of (n - m)th generation descendants of Zm progenitors. Hence 1&(Zm Zn I Zm) = Z� tLn -m and 1&(Zm Zn) = 1&{1&(Zm Zn I Zm)} = 1&(Z� )tLn -m . Hence cov(Zm , Zn) = tLn -m 1&(Z� ) - 1&(Zm)1&(Zn) = tLn -m var(Zm ), 1.
and, by Lemma (5.4.2),
Suppose 0 .:::: r .:::: n, and that everything is known about the process up to time r. Conditional on this information, and using a symmetry argument, a randomly chosen individual in the nth generation has probability 1/Zr of having as rth generation ancestor any given member of the rth generation. The chance that two individuals from the nth generation, chosen randomly and independently of each other, have the same rth generation ancestor is therefore 1/Zr • Therefore IP(L < r) = 1&{IP(L < r I Zr)} = 1&(1 - Z; l ) 2.
and so O ':::: r < n. If 0 < IP(ZI = 0) < 1, then almost the same argument proves that IP(L TJr - TJr+ l for 0 .:::: r < n, where TJr = 1&(Z; l I Zn > 0). 3. The number Zn of nth generation decendants satisfies
=
r I Zn > 0)
=
if p = q,
whence, for n � 1, IP(T = n) = IP(Zn
= 0)
- IP(Zn - l
= 0) =
{
if p #- q , 1
n(n + 1) pn - l q n (p - q) 2
if p = q, if p #- q .
It
follows that 1&(T) < 00 if and only if p < q. 4. (a) As usual, This suggests that G n (s) = 1 - a 1 +p+ ·· +p n - l (1 - s ) p n for n easily by induction, using the fact that G n (s) = G(G n - l (s)). (b) As in the above part (a),
238
�
1 ; this formula may be proved
Age-dependent branching processes
Solutions
[5.4.5]-[5.5.1]
where P2 (s) = P(P(s )) . Similarly G n (s ) = j - I (Pn ( f (s ))) for n 2: I, where Pn (s) = P(Pn - I (S )). (c) With P(s ) = as/{I - (I - a)s} where a = y - l , it is an easy exercise to prove, by induction, that Pn (s ) = a n s/ { I - (1 - a n ) s } for n 2: 1, implying that
Let Zn be the number of members of the nth generation. The (n + I)th generation has size Cn+ l + In+ l where Cn+ l is the number of natural offspring of the previous generation, and In+ l is the number of immigrants. Therefore by the independence, 5.
whence 6.
G n+ l (s) = JE(s Zn +l ) = JE{G (s) Zn }H(s) = G n (G(s))H(s).
By Example (5.4.3 ),
1 n - (n - I)s n - 1 + JE(s Zn ) = n + 1 - ns = -2 n n (1 + n - I - s) ' Differentiate and set s = 0 to find that
00
n 2: o.
00
1 JE(VI ) = E IP'(Zn = 1) = E (n + 1) 2 = ! n 2 • n=O n=O Similarly,
00
00
00
00
1 1 1 n I 2 JE ( V2 ) = E (n + 1)3 = E (n + 1) 2 - E (n + 1)3 = (i n - E (n + 1)3 ' n=O n=O n=O n=O 2 2 n _ (n + 1) - 2 (n + 1) + 1 _ (iI 2 + 9QI 4 - 2 � 1 . _ JE(V3 ) - � (n + 1)4 - � n - n L....(n + 1)4 L....- (n + 1)3 L... .n=O n=O n=O The conclusion is obtained by eliminating L:n (n + 1) -3 . 5.5 Solutions. 1.
Age-dependent branching processes
(i) The usual renewal argument shows as in Theorem (5.5.1) that
G t (s) =
l G (G t -u (s )) fr (u ) du + 100 sfr (u) du.
Differentiate with respect to t, to obtain
a r a at G t (s ) = G(Go(s))fr (t) + 10 at (G(G t - u (s))} fr (u) du - sfr (t) · Now Go(s) = s, and a a at (G(G t - u (s))} = - au (G(G t - u (s))} , 239
[5.5.2]-[5.5.2]
Generating functions and their applications
Solutions
so that, using the fact that h (u) = 'Ae - )...u if u � 0,
t
at {G (G t - u (s))} h (u) du = - [G(G t - u (s» h (u) ] 0 - 'A Jor G(G t - u (s»fT (u) du, Jor � having integrated by parts. Hence
:t Gt (s)
{ - [X) sfT (u) dU } - s'Ae-At
G(s)'Ae -)...t + { - G (s)'Ae -At + 'AG(G t (s» } - 'A G t (s) = 'A {G(G t (s» - G t (s}} . =
(ii) Substitute G(s) = s 2 into the last equation to obtain
aG t at
=
'A(G t2 G t ) _
with boundary condition Go(s) = s. Integrate to obtain At + c(s) = log{ 1 - Gr l } for some function c(s) . Using the boundary condition at t = 0, we find that c(s) = log{1 - G0 1 } = log{1 - s - l }, and hence G t (s) = se -A t /{1 - s(1 - e - )...t }}. Expand in powers of s to find that Z(t) has the geometric distribution IP(Z(t) = k) = (1 - e - )...t ) k - I e -At for k � 1 . 2. The equation becomes with boundary condition Go(s) = s. This differential equation is easily solved with the result
2s + t - s) G t (s) = 2 + t (1(1 - s)
=
We pick out the coefficient of s n to obtain
4/t 2 + t(1 - s)
2-t
( )
4 t n n � 1, IP ( Z (t ) - n ) - ---t(2 + t) 2 + t ' and therefore
( ( ) - ) - n=kL00 t (2 4+ t) ( 2 +t t ) n - -2t ( 2 +t t ) k '
IP Z t > k -
---
--
--
It follows that, for x > 0 and in the limit as t -+ 00,
> xt) IP(Z(t) � xt I Z(t) > 0) = IP(Z(t) IP(Z(t) ; 1)
=
( 2 +t t ) LxtJ - I ( + 2 ) I - LxtJ
240
=
1
t
-+
e -2x •
Characteristic functions
Solutions
[5.6.1]-[5.7.2]
5.6 Solutions. Expectation revisited
1. Set a = E(X) to find that u(X) 2: u(E X ) + A(X - EX) for some fixed A. Take expectations to obtain the result. 2. Certainly Zn = E7=1 Xi and Z = E �l IXi I are such that IZn I :::: Z, and the result follows by dominated convergence. 3. Apply Fatou's lemma to the sequence {- Xn n 2: I} to find that
(
E lim sup Xn n �oo 4.
)
=
(
-E lim inf -Xn n �oo
)
:
2:
- lim inf E(-Xn) = lim sup E(Xn). n �oo n �oo
Suppose that EIXr I < 00 where r > O. We have that, if x > 0,
u r dF(u) --+ 0 as x --+ 00, hx ,oo) where F is the distribution function of IXI. Conversely suppose that x r JP'(IXI 2: x) --+ 0 where r 2: 0 , and let 0 :::: s < r. Now E IXs l limM-+oo foM US dF(u) and, by integration by parts, x r JP'(IXI 2: x) ::::
f
=
The first term on the right-hand side is negative. The integrand in the second term satisfies sus - 1 JP'( 1 X I > u) :::: su s - I . u -r for all large u. Therefore the integral is bounded uniformly in as required. 5. Suppose first that, for all E > 0, there exists 8 = 8 (E) > 0, such that E(IXIIA) < E for all A satisfying JP'(A) < 8. Fix E > 0, and find x (> 0) such that JP'(I XI > x) < 8(E). Then, for y > x,
M,
/:y lu i dFx (u) /:x lui dFx (u) + E (IXII£I X I >x }) :::: i: lu i dFx(u) + E. ::::
Hence f!.y lui dFx (u) converges as y --+ 00, whence EIXI < 00. Conversely suppose that EIXI < 00. It follows that E (IXI1n X I > y }) --+ 0 as y --+ 00. Let E > 0 , and find y such that E (IXII£IXI> y }) E. For any event A, IA :::: IAn B C + IB where B = {IXI > y }. Hence E(IXIIA) :::: E (IXIIAn Bc ) + E (IXIIB) :::: yJP'(A) + E. Writing 8 = E/(2y), we have that E(IXIIA) < E if JP'(A) < 8.
, then X - Y has characteristic function 1fr (t ) = JE (e i tX ) JE (e - i t Y ) = cJ> (t)cJ> (-t) = cJ> (t ) cJ> (t ) = 1 cJ> (t ) 1 2 . Apply the result of part (i) to the function 1fr to obtain that 1 - 1 cJ> (2t) 1 2 :::: 4(1 - 1 cJ> (t) 1 2 ). However 1 cJ> (t) 1 :::: 1, so that 1 - 1cJ>(2t) 1 :::: 1 - 1 cJ> (2t) 1 2 :::: 4(1 - 1 cJ> (t ) 1 2 ) :::: 8(1 - 1 cJ> (t ) l ). 3.
(a) With m k = JE(Xk ), we have that
say, and therefore, for sufficiently small values of e, 00 ( + l S (e l . Kx (e) = E - W r=l r Expand S (e? in powers of e, and equate the coefficients of e, e 2 , e 3 , in turn, to find that kl (X) = m I , k2 (X) = m 2 - mf, k3 (X) = m 3 - 3m l m 2 + 2m I (b) If X and Y are independent, Kx+ y (e) = 10g{JE(eo x )JE(eO Y )} = Kx (e) + Ky (e), whence the claim is immediate. x ' 02 4. The N(O, 1) variable X has moment generating function JE (e o ) = e 2 , so that Kx (e) = �e 2 . 5. (a) Suppose X takes values in L (a, b). Then '
lcJ>x (2n/b) I =
1 :�:>21TiX/blP'(X I
= X) =
le21T ia /b l
l � >21Tim lP'(X
=
a + bm)
1
=
1
since only numbers of the form x = a + bm make non-zero contributions to the sum. Suppose in addition that X has span b, and that IcJ>X (T) I = 1 for some T E (O, 2n/b). Then cJ>x ( T ) = e i c for some c E R . Now JE (cos ( T X - c» ) = �JE(i T X -i c + e - i T X+ic ) = 1 , using the fact that JE (e - i T X ) = cJ>x ( T ) = e - ic . However cos x :::: 1 for all x, with equality if and only if x is a multiple of 2n . It follows that T X - c is a multiple of 2n, with probability 1, and hence that X takes values in the set L(c/ T, 2n/ T). However 2n/ T > b, which contradicts the maximality of the span b. We deduce that no such T exists. (b) This follows by the argument above. 6. This is a form of the 'Riemann-Lebesgue lemma'. It is a standard result of analysis that, for E > 0, there exists a step function gE such that f�oo If(x) - gE (x ) 1 dx < E. Let cJ>E (t ) = f�oo eitx gE (X ) dx. Then
If we can prove that, for each E, IcJ>E (t) I large t, and the claim then follows.
-+ °
as t -+ 242
±oo,
then it will follow that lcJ>x (t) I < 2E for all
Characteristic junctions
[5.7.7]-[5.7.9]
Solutions
Now g£ (x) is a finite linear combination of functions of the form CIA (x) for reals c and intervals A, that is g£ (x) = '2:[= 1 Ck IAk (x); elementary integration yields
where ak and bk are the endpoints of Ak . Therefore
2 K 1¢£ (t) l :::: -t L Ck -+ 0, k= 1 7.
as t -+ ±oo.
If X is N(/-L, 1), then the moment generating function of X2 is
X X2 (S ) = E(eS
M
2
)=
100 eSx2 --1 e- :Z (X-/L)2 dx 1
- oo
...fiii
=
( -- )
1 exp /-L 2 s , .J 1 - 2s 1 - 2s
if s < !, by completing the square in the exponent. It follows that
My(s) =
1 exp ( /-Lh ) } Jln { ..Jf=2s 1 � 2s
(
1 s() (1 2s) n /2 exp 1 - 2s .
=
_
)
It is tempting to substitute s = j t to obtain the answer. This procedure may be justified in this case using the theory of analytic continuation. S. (a) T2 = X2 I(Y n) , where X2 is X 2 (1; /-L2 ) by Exercise (5.7.7), and Y is x 2 (n) Hence T 2 is
.
I
F(1, n; /-L2 ). (b) F has the same distribution function as
+ :-:- B)/m (A 2--: - "" -'"":' Z = -'--Vln
where A,
B, V are independent, A being N(JO, 1), B being x 2 (m - 1), and V being x 2 (n). Therefore E(Z) � { E(A2 )E (�) + (m - 1)E ( B I 0� 1» ) } =
=
�
{
n n m (1 + () _ n -_2 + (m _ 1) _ n -_2
}
=
n(m + () , m(n - 2)
where we have used the fact (see Exercise (4. 1 0.2» that the F(r, s) distribution has mean s I(s - 2) if s > 2. 9. Let X be independent of X with the same distribution. Then 1¢1 2 is the characteristic function of X - X and, by the inversion theorem,
1 2n
-
100 1¢ (t) 1 2 e-l.tx dt - 00
=
fx _ x (x) =
100 f(y)f(x + y) dy. -00
Now set x = O. We require that the density function of X - X be differentiable at O. 243
[5.7.10]-[5.8.2] 10.
Generating functions and their applications
Solutions
By definition,
e - i ty ¢x (y) =
1
00
-00
eiy (x - t ) fx (x) dx .
Now multiply by fy (y), integrate over y E R, and change the order of integration with an appeal to theorem. 1 1 . (a) We adopt the usual convention that integrals of the form J: g(y) dF(y) include any atom of the distribution function F at the upper endpoint but not at the lower endpoint It is a consequence that Fr: is right-continuous, and it is immediate that Fr: increases from 0 to 1 . Therefore Fr: is a distribution function. The corresponding moment generating function is Fubini 's
v
Mr: (t) =
1
00
-00
e tx dFr: (x) =
u.
1 M(t)
1_ _
00
-00
etx + r:x dF(x) = M(t + . ) . M(t)
(b) The required moment generating function is Mx + y (t + .) Mx + y(t)
=
Mx (t + .)My (t + . ) Mx (t)My (t)
the product of the moment generating functions of the individual tilted distributions. 5.8 Solutions. Examples of characteristic functions
(i) We have that if)(t) = JE(eitX ) = JE(e -itX ) = ¢- x (t) . (ii) If X I and X2 are independent random variables with common characteristic function ¢ , then 1.
¢X , +X2 (t) = ¢x , (t)¢X2 (t) = ¢ (t) 2 . (iii) Similarly, ¢X , - X2 (t) = ¢x , (t)¢- X2 (t) = ¢(t)¢ (t) = 1¢ (t) 1 2 . (iv) Let X have characteristic function ¢ , and let Z be equal to X with probability � otherwise. The characteristic function of Z is given by
and to -X
where we have used the argument of part (i) above. (v) If X is Bernoulli with parameter 1, then its characteristic function is ¢ (t) = � + 1eit . Suppose Y is a random variable with characteristic function 1/f(t) = 1¢ (t)l. Then 1/f(t) 2 = ¢ (t)¢(-t). Written in terms of random variables this asserts that YI + Y2 has the same distribution as X l - X2, where the Yi are independent with characteristic function 1/f, and the Xi are independent with characteristic function ¢. Now Xj E to, I }, so that X l - X2 E { - 1 , 0, I }, and therefore Y.i E { - � , � } . Write a = lJ»(Yj = 2I ) . Then lJ»(YI + Y2 = 1 ) = a 2 = lJ»(X I - X2 = 1 ) = � , lJ»(YI + Y2 = - 1 ) = ( 1 - a) 2 = lJ»(XI - X2 = - 1 ) = � ,
implying that a 2 = ( 1 - a) 2 so that a = J;, contradicting the fact that a2 = �. We deduce that no such variable Y exists. 2. For t � 0, Now minimize over t � O. 244
Examples of characteristic junctions 3.
Solutions
The moment generating function of Z is
[5.8.3]-(5.8.5]
{ C. �tY ) m }
}
MZ (t) = JE { JE(etX Y I Y) = JE{Mx (ty)} = JE m y n - l (l _ y) m -n - l 1 A dy. = A - ty B(n, m - n) o Substitute v = 1/y and integrate by parts to obtain that OO (v _ 1) m -n - l Imn = (AV - t) m dv 1
( 10 -- )
satisfies
i
[
]
_ m -n - l OO m - n - l Imn = - A(m 1- 1) (V 1) m - l + _ Im - l ,n = c (m , n , A) Im - l ,n (AV - t) 1 A(m 1) for some c(m, n, A). We iterate this to obtain
,
Imn = c In+ l ' n = c
' i1 oo (AV -dvt)n+ 1
c'
for somec' depending onm, n, A. Thereforen Mz(t) = C"(A - t) -n for somec" depending on m, n, A. However Mz(O) = 1, and hence c" = A , giving that Z is r(A, n). Throughout these calculations we have assumed that t is sufficiently small and positive. Alternatively, we could have set t = is and used characteristic functions. See also Problem (4.14.12). 4. We have that
(x )x ( [x . t ) exp ( ). exp (
100 eitx2 1 exp -00 ", 27r(l· 00 1 exp = 1 ---00 ../27r(l 2
.,.1Il'(eitX2 ) =
=
�
1 J 1 - 2a 2 it
- p,) 2 d - 2a 2 - p,(l - 2a 2 it) - l 2(1 2 (1 - 2a 2 , t) - 1
itp,2 1 - 2a 2 it
itP,2 1 - 2a 2 a.
) dx
The integral is evaluated by using Cauchy's theorem when integrating around a sector in the complex plane. It is highly suggestive to observe that the integrand differs only by a multiplicative constant from a hypothetical normal density function with (complex) mean JL(1 - 2a 2 it)- 1 and (complex) variance (1 2 (1 - 2(1 2 it) - 1 . 1 5. (a) Use the result of Exercise (5.8.4) with p, = 0 and (1 2 = 1 : i Jn} is sufficiently unlikely that its occurrence casts doubt on the original supposition that babies look the same. A statistician would level a good many objections at drawing such a clear cut decision from such murky data, but this is beyond our scope to elaborate. 3. Clearly -
I _. S it X) } = lE{exp (X(eit - I») } = ( 1 - (e�l t - 1) ) S = ( _ 2 - e't ) It follows that lE(Y) = -:-I ¢Y, (0) = s, whence var(Y) = 2s . Therefore the characteristic function of the normalized variable Z = (Y lEY)/ .Jvar(Y) is ¢y (t ) = lE { lE(e Y I
I
Now,
-s log(2 - e it /$s) s(eit /$s - 1 ) + � s(e it /$s _ 1 ) 2 + 0(1)
= 10g {¢y (t/.J2S) } = = it # - ! t 2 - ! t 2 + 0(1),
where the 0(1) terms are as s --+ 00. Hence log {¢z (t ) } --+ - � t2 as s --+ 00, and the result follows by the continuity theorem (5.9.5). Let PI , P2 , . . . be an infinite sequence of independent Poisson variables with parameter 1. Then Sn = PI + P2 + . . . + Pn is Poisson with parameter n. Now Y has the Poisson distribution with parameter X, and so Y is distributed as Sx. Also, X has the same distribution as the sum of s independent exponential variables, implying that X --+ 00 as s --+ 00, with probability 1. This suggests by the central limit theorem that Sx (and hence Y also) is approximately normal in the limit as s --+ 00. We have neglected the facts that s and X are not generally integer valued. 4. Since Xl is non-arithmetic, there exist integers n 1 , n 2 , . . . , nk with greatest common divisor 1 and such that lP'(X 1 = n ) > 0 for 1 .::; i .::; k. There exists N such that, for all n � N, there exist non negative integers a I , a2 i, . . . , ak such that n = a 1 n 1 + . . . + aknk . If x is a non-negative integer, write
250
1Wo limit theorems
Solutions
[5.10.5]-[5.10.5]
N = f h n l + . . . + f3k n k , N +x = Ylnl + . . . + Ykn k for non-negative integers f3 l , . . . , 13k. YI . . . , Yk. Sn = X l + . . . + X n i s such that k lI"(SB = N) � lI" ( Xj = nj for Bj - l < j � Bj , 1 � i � k) = II lI"(X I = nj)fJj > 0 j=l where Bo = 0, Bj = f3 1 + 132 + . . . + f3i , B = Bk . Similarly lI"(SG = N + x) > 0 where G = YI + Y2. + . . . + Yk Therefore lI" (SG - SG, B +G = x) � lI"(SG = N + X) lI"(SB = N) > 0 Now
·
where SG, B +G =
�f=+J'+1 Xi ·
Also, lI"(SB - SB , B +G =
-x) > 0 as required.
X2 , . . . be independent integer-valued random variables with mean 0, variance 1 , span 1, Jnll"(Un = x) � e - 2 x /$ n � 00 1 + . . . + Xn Sn = X l + X2In Un = Jn and x is any number of the form k/ In for integral k. The case of general JL and 0' 2 is easily derived
S.
Let X l ,
1
and common characteristic function 1/>. We are required to prove that as where
from this.
By the result of Exercise (5.9.4), for any such
lI"(Un = x) = since
Un is arithmetic.
x,
1 j 2 n r.;
1I:",
2
rr
.jTi
-rr
e - r tx l/>un (t) dt, .
.jTi
Arguing as in the proof of the local limit theorem
(6),
211: I Jnll"(Un = x) - f(x) 1 � In + In where
f is the N(O, 1) density function, and
In 2$ (1 - cI> (11: In) ) � 0 as n � 00, where cI> is the N(O, 1) distribution function. In, 8 E (0, 11:). Then
Now = for pick
(t) 1 ').. if
By Exercise (5 .7.5a), there exists ').. E
and it remains only to show that as
25 1
n � 00 .
As
(0, 1)
[5.10.6]-[5.10.7]
Generating functions and their applications
Solutions
of this is considerably simpler if we make the extra (though unnecessary) assumption tThehat mproof 3 = JEI X il < 00, and we assume this henceforth. It is a consequence of Taylor' s theorem (see Theorem (5. 7 . 4 )) t h atq,(t) = 1-� t 2 - iit 3 m 3 +0(t 3 ) as t � It follows thatq,(t) = e - !t 2 +t38(t) for some finite 9(t). Now l ex - 11 Ix l e 1 l , and therefore o.
x
�
Letand K88K8=<sup{1:1 . For9 (u)1It I :
�
+...+
7.
x
�
>
2x )
252
[5.10.8]-[5.11.3] Large deviations byfolltohwse resul by consul of integrals. by analt ofyProbl tic contiemn(5.uati1 2.on1 8c),in theorupper half-pltinagne.a tablSeeeMoran 1968, p.The271.n required conclusion 8. (a) The sum Sn = E�= l Xr has characteristic function JE(ei t Sn) = ¢(t) = ¢(tn2), whence Un = Sn/n has characteristic function ¢(tn) = JE(eitnXl). Therefore, lP'(Sn < c = lP'(nXI < c = lP' (X l < �) 0 as n 00. (b) JE(eit Tn) = ¢(t) = JE(eitXI). (a) Yes,variabecause Xn is the sum of independent identically distributed random variables with non-zero nce. ((n2b) - cannot in general obey-whatJE(8))weandhaven var(XI) called the=central lim-it JE(8)) theorem,arebecause var(Xn) = n) var(8) nJE(8)(I nJE(8)(I di ff erent whenever var(8)var(8) =1= O. Indeed the right 'normalization' involves dividing by n rather than...;n. It may be shown when =1= 0 that the distribution of Xn/n converges to that of the random variable 8. Solutions
-+
)
)
-+
9.
It
+
5.11 Solutions. Large deviations
We may wri t e Sn = E7 Xi where the Xi have moment generating function M (t) = i (et e -t). Appl y i n g t h e l a rge devi a ti o n t h eorem (5.11. 4 ), we obtai n t h at , for 0 < a < 1, lP'(Sn > an) l/n iint takes ft >o {g(t)} e -at M(t). the valwhere ue 1/ Jg(t)(1 = a)1+a (1 - a)Nowl -agashasrequia minired.mIfuma when1, theten=lP'(J(Sn l> an)a)/(=l 0-fora),alwhere l n. the binvariomiaabll ediSnstriibnutiExerci on wisteh(5.paramet rs n and i i. fThen 2Yn - n has the same distri(ib)utiLeton Ynas thehaverandom 1 1.1). eTherefore, 0 < a < 1, 1.
+
+
-+
�
+
2.
and similarly for lP'(Yn - i n < - i an), by symmetry. Hence (iTni) Thi= esntilP'm(Sne le>t Snn(I= Xa)). TheXn,moment the sumgenerati of independent Poin ofssonX vari- 1aibls eM(t) s with=paramet e-r 1.1 Then n g functi o exp(et - t), l andg'(t)th=e l(etarge-devia -atiI)oexp(et n theorem- atgi-vest -thatI) Twhence l / n einft > o {g(t)} where g(t) = e -at M(t). Now Tl /n eg(log(I a)) = {e/(a I)}a+ l . g has a minimum at t = log(a 1). Therefore Suppose hat> M(t) JE(etX) is Sifinmitielaonrlyt,hlP'e(iXnte 0, M(8) e8alP'(X > a),3. soSuppose that lP'(Xtconversel a) .:::y: =M(8)e-8a. that such A, exist. Then M(t) .:::: JE(e 1tX1 ) = irro, oo) e 1t 1x dF(x) where F is the distribution function of I X I . Integrate by parts to obtain M(t) .:::: 1 [- e 1t 1x [ 1 - F(x )]]� 1000 I t l e 1 t 1x [ 1 - F(x)] dx 1 +' . .+
+
-+
-+
+
+
+
�
/-L
+
+
253
[5.11.4]-[5.12.2]
Generating functions and their applications
Solutions
'I '
(the term takes care of possible atoms at is sufficiently small.
ItI
4.
0). However 1 - F (x ) � JLe -Ax , so that M(t) < 00 if
{e - 1 t / n l } n = e - 1 t l , and hence Sn / n is Cauchy. Hence dx = 2. ( :: tan - I a ) lP'(Sn > an) = t"JO . Ja n ( 1 + x 2 ) n 2
The characteristic function of Sn / n is
S.12 Solutions to problems
1.
{ � t s j rO = s) lO { 1 -=.s 6 fO = s ) 1O (I - lOS 6 + . . . )(I + lOS + . . . ) G 1 s G
The probability generating function of the sum is
The coefficient of
.
s 27 is
2.
(a) The initial sequences T, HT, HHT, HHH induce a partition of the sample space. By conditioning on this initial sequence, we obtain - 3) for 3, - 2) where (2) In principle, this difference equation Also may be solved in the usual way (see Appendix An alternative is to use generating functions. k Set E� 1 multiply throughout the difference equation by s and sum, to find that
p + q = 1.
k
+ p2 qf(k
f(k) = qf(k - 1) + pqf(k f(1) = f = 0, f(3) = p 3 .
G(s) = s f (k), G(s) = p 3 s 3 /{l - qs - pqs 2 - p2 qs 3 }.
I).
To find the coefficient of expand in partial fractions, and use the binomial series.
f(k)
Another equation for is obtained by observing that X last four tosses were THHH. Hence
( k-4 )
f(k) = qp 3 1 - � f(i) 1= 1
,
s
k
k>
, factorize the denominator,
= k if and only if X > k - 4 and the k > 3.
Applying the first argument to the mean, w e find that JL = JE(X) satisfies JL = q(1 + JL) + pq(2 + JL) + p2 q(3 + JL) + 3p 3 and hence JL = (I + p + p2 )/p 3 . As for HTH, consider the event that HTH does not occur in n tosses, and in addition the next three tosses give HTH. The number Y until the first occurrence of HTH satisfies
> n)p2 q = lP'(Y = n + I)pq + lP'(Y = n + 3), n � 2. Sum over n to obtain JE(Y) = (pq + 1)/(p 2 q). n (b) G N (S) = (q + ps ) , in the obvious notation. (i) lP'(2 divides N) = i {G N (1) + G N (- 1)}, since only the coefficients of the even powers of s lP'(Y
contribute to this probability.
(ii) Let (J) be a complex cube root of unity. Then the coefficient of lP'(X
G N ({J)2 ) } is
j { l + {J)3 + {J)6 } = I, j { l + + {J)2 } = 0, j { l + {J)2 + {J)4 } = 0, (J)
254
k = 3 r, if k = 3 r + I, if k = 3 r + 2, if
= k) in j {G N (1) + G N ({J)) +
Problems
Solutions
[5.12.3]-[5.12.6]
§- {G N (1) G N (W) G N (W2 )} = E ;!�j ff>(N = 3 r ) , the probability that N
+ for integers r . Hence + is a multiple of 3 . Generalize this conclusion.
T=k n n ff>(T = k) = ff>(T sk ff>(T = n) = pn . G T:
k n k n - l)qpn
3. We have that if no run of heads appears in the first - - 1 throws, then there is a tail, and then a run of heads. Therefore > for � + 1 where and sum to obtain a formula for the probability Multiply by + 1 . Finally generating function of
p q=
k n
00 00 � G(s) - pn s n = qpn L s k L ff>(T = j) = qpn L ff>(T = j) L s k j=1 k=n+l j>k -n - l k=n+l 00 n n+l n n+l '"' ff>(T = j)(1 - s j ) = qp s (1 - G(s» . = qp s LJ l -s l - s j=1
Therefore
4.
The required generating function is
( )
)
(
� s k k - 1 pr (1 _ pi - r = � r G(s) = LJ l - qs k=r r - 1 where p + q = 1 . The mean is G' (1) = r / p and the variance is Gil (1) + G' (1) - {G' (1)} 2 = r q / p 2 . n 5. It is standard (5 .3 .3) that po(2n) = (�) (pq) . Using Stirling's formula, po(2n) The generating function
Po(s) = En s 2n Po (2n).
�
(2n) 2n + ! e -2n $ n = (4pq) n (pq) '\I nn {n n+ 2 e -n $} 2 '-=
1
Fo(s) = 1 - PO (s) - 1 where Fo(1) = 1 - A. - 1 where, by
Fo(s)
for the first return time is given by Therefore the probability of ultimate return is
Abel's theorem,
{ - OO
A. = L Po(2n) < 00 n Hence Fo(1) = 1 if and only if p = �. 6. (a) Rn = X� + Y; satisfies
Hence
Rn = n + Ro = n.
P = q = 21 ' if p =/= q. if
Un = Xn + Yn, Vn = Xn - Yn.
(b) The quick way i s to argue as i n the solution to Exercise (5.3 .4). Let Then and simple symmetric random walks, and furthermore they
U
'
V are
PO (2n) = ff>(U2n = 0, V2n = 0) = ff>(U2n = 0)ff>(V2n = 0) =
255
are independent. Therefore
{ (D e:) } , 2n
2
[5.12.71-[5.12.8]
Generating functions and their applications
Solutions
po(2n) 1.
by (5 .3.3). Using Stirling's formula, that the chance of eventual return is
�
(mr)- I , and therefore 2:n po(2n) = 00, implying
0
A longer method is as follows. The walk is at the origin at time if and only if it has taken equal numbers of leftward and rightward steps, and also equal numbers of upward and downward steps. Therefore
()
1 2n n o(2n) = P 4
E
()
()
1 4n 2n 2 (2n) ! = 2" n (m !) 2 {(n - m) !} 2
7. Let eij be the probability the walk ever reaches j having started from i . Clearly ea o = ea,a - l ea - l,a - 2 · · · e lO , since a passage to 0 from a requires a passage to a - 1, then a passage to a - 2, and so on. By homogeneity, ea o = (e lO ) a . By conditioning on the value of the first step, we find that e lO = pe3 0 + qeOO = pero + q. The cubic equation x = px 3 + q has roots x = 1, c, d, where c=
- p - Jp2 + 4pq 2p
--"-----'--=----=----=-
d=
-p + Jp2 + 4pq . 2p
> 1, and Id l � 1 if and only if p2 + 4pq � 9p2 which is to say that p .:::: j . It follows that e lO = 1 if p .:::: j , so that ea o = 1 if p .:::: i . When p > 1, we have that d < 1, and it is actually the case that e lO = d, and hence Now
Ic l
1
'f
p > 31 '
ea o < 1 for all large a; this is a minor but necessary Xj is the value of the i th step. Then
In order to prove this, it suffices to prove that chore. Write where
Tn = Sn - So = 2:1=1 Xj ,
eaO = lP(Tn .:::: -a for some n � 1) = lP(n/-L - Tn � n/-L + a for some n � 1) 00
.:::: L lP(n/-L - Tn � n/-L + a) n =1 where /-L = lE(X l ) = 2p - q > O. As in the theory of large deviations, for t > 0, X
lE(e t(tL - X) ) = 1 + o(t) as t t 0, and therefore we may pick t > 0 e(t) = e - t tLlE(e t(tL - X) ) < 1. It follows that ea o .:::: 2:� 1 e - ta e(t) n which is less than 1 a,
where is a typical step. Now such that for all large as required. 8.
We have that
p + q = 1 . Hence G x,y (s, t) = G(ps + qt) where G is the probability generating function X+ X and Y are independent, so that
where of Y. Now
G(ps + qt) = G x (s)G y (t) = G x,y (s, 1)G x,y (1, t) = G(ps + q)G(p + qt). 256
Problems
Solutions
[5.12.9]-[5.12.11]
f(u) = G(1 + u), x = s - 1, y = t - 1, f(px + qy) = f(px)f(qy), < x, y O. f f(O) = 1 ; (4.14.5» f(x) = e AX I s ) A( A, G(s) = f s - 1) = e . X+Y A. G x (s) = G(ps + q ) = e Ap (s - I ) , X Y Aq. 9. I n the usual notation, G n +1 (s) = G n ( G (s » . It follows that G�+I (1) = G � ( 1 ) G' ( 1 ) 2 + G� ( 1 ) G (1) so that, after some work, var( Zn+l ) = JL2 var(Zn) + JLn a 2 . Iterate to obtain a 2 JLn (1 - JLn+l ) , n � 0, var(Zn+1 ) = a 2 (JLn + JLn +1 + . . . + JL 2n ) = 1 - JL for the case JL =1= 1. If JL = 1, then var(Zn+l ) = a 2 (n + 1).
Write a functional to obtain equation valid at least when -2 � Now is continuous within its disc of convergence, and also the usual argument (see Problem implies that for some and therefore ( Therefore has the Poisson distribution with parameter Furthermore, whence has the Poisson distribution with parameter Ap. Similarly has the Poisson distribution with parameter "
D
10. (a) Since the coin i s unbiased, we may assume that each player, having won a round, continues to back the same face (heads or tails) until losing. The duration of the game equals if and only if is the first time at which there has been either a run of r heads or a run of r tails; the probability of this may be evaluated in a routine way. Alternatively, argue as follows. We record S (for 'same' ) each time a coin shows the same face as its predecessor, and we record C (for 'change' ) otherwise; start with a C . It is easy to see that each symbol in the resulting sequence is independent of earlier symbols and is equally likely to be S or C . Now if and only if the first run of r - 2 It is immediate from the result of Problem (5 . 1 2.3) that S 's is completed at time
k
k -1
-1
D=k
k.
G D (S) = (b) The probability that
( 2l s/ -2 (1 _ 2I s) . 1 - s + (�sy - l
Ak wins is 00
7rk = E n=l
lP'(D =
n (r
- 1) + k - 1) .
Let W be a complex (r - 1 )th root of unity, and set
{
Wk (S) = r _1 1 G D (S) + wk1- l G D (WS ) + 2 (k1 - l ) G D (W2 s) w
1 + . . . + w (r -2)(k - l ) G D (Wr -2 s)
51 »
si
(c) The pool contains
£D
0
lP'(D
.
Wk (S) is = i ) if i is of the lP'(Ak wins) = Wk (1).
It may be seen (as for Problem ( . 2.2 that the coefficient of in form n (r for some n , and is otherwise. Therefore
- 1) + (k - 1)
}
when it is won. The required mean is therefore
E(D I Ak
wins)
=
E(DI
W'k_ (1) { A k WI ns} ) _ lP'(Ak wins) = Wk (1) . .
lP'(D
> k), k � 0, is T(s) = (1 - G D s » 1 - s). s n T(s). 11. Let Tn be the total number of people in the first n generations. B y considering the size Z l o f the (d) Using the result of Exercise (5 . 1 .2), the generating function of the sequence ( /( The required probability is the coefficient of in
first generation, we see that
Zl
Tn = 1 + E Tn - l (i) i=l 257
[5.12.12]-[5.12.14]
Tn - l
Generating functions and their applications
Solutions
Tn - l
where (2) , . . . are independent random variables, each being distributed as (1), the compounding formula (5 . 1 .25),
Hn (s) = sG(Hn - l (s».
12. We have that
Tn - I . Using
JP(Zn > N, Zm = 0) JP(Zm = 0) = N + r )lP (Zn = N + r) = JP(Zm = 0 I ZnJP(Z m = 0) r=1 o) N+r JP(Zn = N + r) = JP(Zm -n = JP(Z m = 0) r=1 JP(Zm = O) N+l � L JP(Zn = N + r) � JP(Zm = 0) N = G m (O) N JP(Zm = 0) r=1
JP(Zn > N I Zm = 0) =
"f "f
00
.
G w (s) = G N (G(S» = e A( G (s)- l ) . Also, Gw (s) l / n = e A«G (s)- I ) / n , the G w but with A replaced by Aln. (b) We can suppose that H (0) < 1 , since i f H (0) = 1 then H (s) = 1 for all s, and w e may take A = 0 and G (s) = 1 . We may suppose also that H (0) > O. To see this, suppose instead that H (0) = 0 so that H(s) = s r "L-i=o s j hj+r for some sequence (h k ) and some r ::: 1 such that h r > O. Find a n positive integer n such that r In is non-integral; then H(s) l / is not a power series, which contradicts the assumption that H is infinitely divisible. Thus we take 0 < H(O) < 1 , and so 0 < 1 - H(s) < 1 for 0 � s < 1 . Therefore 13. (a) We have that same probability generating function as
10g H(s) = 10g ( 1 - { l - H(s)J) = A ( - 1 + A (s»)
00
A = - log H(O) and A(s) is a power series with A (O) = "L-j=1 aj sJ , we have that
where
•
0, A(I)
1 . Writing
A(s) =
H(s) l / n is a probability generating function, so that each such expression is non aj � 0 for all j , implying that A(s) is a probability generating function, as
as -+ Now negative. Therefore required.
n
00.
14. It is clear from the definition of infinite divisibility that a distribution has this property if and only such that t/> if, for each there exists a characteristic function for all
n,
(t) = 1frn (t) n
1frn
t.
(a) The characteristic functions in question are
N(/-L, a 2 ) : Poisson (A) : r(A , /-L) :
.
1
2 2
(t) = e i t lL- '1 U t t/> (t) = e A(e l - 1 ) A t/> (t) = A - I. t t/>
·t
( _ )IL .
1frn
In these respective cases, the 'nth root' of t/> is the characteristic function of the Poisson and distributions.
(Aln),
r(A, /-LIn)
258
N(/-Lln, a 2 In),
Problems
Solutions
¢
[5.12.15]-(5.12.16]
1/In that¢(t) = 1/In (t)n . 1¢ (t)1 1 t, 1 1/1n (t)1 = 1¢ (t)1 1 /n { 0I 11 ¢¢ (t)1(t)1 =1= 0, t e ¢(t) =1= 0,0 < e < 2n 1/In (t)1/I (t)1 nei(} 00. n 1/In (t)n =1= 0, 1/I(t) = n-+oo 1/In (t) = { 0I ¢(t) ¢(t) = O. ¢(t) =1= 0 1/1 1/I(t) = 1 ¢ 1/1 1/I(t) = 1 t, 1/1 ¢(t) =1= 0 t. lP'(N = n I S = N) = ElP'(Sk lP'(S= =n kI Ni N= =n)lP'k)(lP'N(N= =n)k) E�pnllP'p(NklP'(N= n)= k) . E(x N I S = N) = G(px)/G(p). If N e A ( px - l ) E(x N I S = N) = eA (p - l ) = eAp (x - l ) = G(x) P . E(x N I S = N) G(x)P. G(px) G(p)G(x)P, Ix l 1, 0 < P < 1. f(px) = f(p) + pf (x), f(x) = G(x ) f 0 < x 1 j. pi x , f(x) = - >"(1 - x) f, N Gx(s ) = Gx, Y (s , 1) = ( 11 --P(PI2 -+PIP2s» ) n ' Gy(t) = Gx' y O, t) = ( 11 -- (PIPI -+PP22t» ) n ' Gx+y(s ) = Gx, Y (s , s) = ( 11 _- ((PIPI ++PP22))S ) n ' X, Y, X + Y lP'(X = k) = (n + : - l )ak O _ a) n , lP'(Y = k) = (n + : - l ) fJ k (l _fJ ) n , lP'(X + Y = k) = (n + : - 1 )y k ( l _ y) n , k 0, a = PI /(1 -P2), fJ = P2 /(1 - PI ), y = PI + P2 · (b) Suppose that is the characteristic function of an infinitely divisible distribution, and let characteristic function such Now � for all so that -+
if
if
be a
= o.
For any value of such that it is the case that -+ as -+ To see this, suppose such that instead that there exists satisfying along some subsequence. -+ Then does not converge along this subsequence, a contradiction. It follows that �
lim
if
Now
i s a characteristic function, s o that o n some neighbourhood of th e origin. Hence on some neighbourhood of the origin, so that is continuous at the origin. Applying the continuity theorem (5.9.5), we deduce that is itself a characteristic function. In particular, is continuous, and hence for all by (*). We deduce that for all 15. We have that
=
Hence
is Poisson with parameter >.. , then
Conversely, suppose that Then = Therefore 10g satisfies power series expansion which is convergent at least for � above functional equation for and equating coefficients of for some >.. � o. I t follows that has a Poisson distribution.
valid for � and in addition has a Substituting this expansion into the we obtain that =
16. Certainly
giving that specifically,
for
�
and
have distributions similar to the negative binomial distribution. More
where
259
[5.12.17]-[5.12.19]
Generating functions and their applications
Solutions
Now
JE(S X J y =y } ) A ( B JP(Y = y) where A is the coefficient of tY in G X, y (s, t) and B is the coefficient of tY in G y (t ) . Therefore JE(s X I Y = y) =
17.
As in the previous solution,
18.
(a) Substitute
u = y la to obtain
(b) Differentiating through the integral sign,
1000 { - 2b2 x -a2 u2 - b2 u -2 ) } du u 00 10 2 exp(-a2 b2 y -2 - y2) dy = -2J (I, ab),
aJ = ab 0 =-
-
e p(
u = b I y. aJ lab = -2aJ, whence J = c (a ) e 2ab where
by the substitution (c) Hence
-
c(a) = J (a, O) = (d) We have that
by the substitution x
_22 .fii . Jro OO e a u du = �
= y2 .
(e) Similarly
by substituting
19.
x
= y -2 .
(a) We have that
260
Problems
Solutions
in the notation of Problem (5 . 1 2 . 1 8). Hence
[5.12.20]-[5.12.22]
U has the Cauchy distribution.
(b) Similarly
for
t > O. Using the result of Problem (5. 1 2. 1 8e), V has density function 1 - I / (2x ), x > O. f(x) = __e .J271:x 3
(c) We have that
t > O.
W -2 = X -2 + y - 2 + Z - 2 . Therefore, using (b), W -2
for It follows that has the same distribution as distribution as Therefore, using the fact that both and
� X2 .
has the same distribution as
20.
X
1 x, that is N(O, �).
. j 1 271:
9V = 9X - 2 , and so W 2 has the same W are symmetric random variables, W
It follows from the inversion theorem that
N 1 h) _ 1 ----,--- - - 1m N -+ oo -N h
F(x + - F(x)
I¢I
- e - i t h e - i tx A. (t ) d t. it 'I'
Since is integrable, we may use the dominated convergence theorem to take the limit as h + within the integral:
¢
f (x) = � 271: Nlim -+ oo
j
N
-N
0
e - i tx ¢ (t) dt.
The condition that be absolutely integrable is stronger than necessary; note that the characteristic function of the exponential distribution fails this condition, in reflection of the fact that its density function has a discontinuity at the origin.
21. Let Gn denote the probability generating function of Zn . The (conditional) characteristic function n of Zn /JL is It is a standard exercise (or see Example (5 .4.3)) that
whence by an elementary calculation as
n --+ 00,
the characteristic function o f the exponential distribution with parameter 1 - JL
22.
The imaginary part of
-I .
¢x (t) satisfies
H ¢x(t) - ¢x (t) } = H ¢x (t) - ¢x ( -t)} = HlE(e i tX ) - lE(e - i tX ) } = 0 261
[5.12.23]-[5.12.24]
Generating junctions and their applications
Solutions
t,
for all if and only if distribution.
X and -X have the same characteristic function,
or equivalently the same
U
23. (a) = X + Y and V = X - Y are independent, so that ¢u + V = ¢u¢ v , which is to say that ¢2 x = ¢x +Y ¢X - y , or ¢ (2t) = {¢ (t) 2 } {¢ (t) ¢(-t) } = ¢ (t) 3 ¢ (-t). Write
1/I(t) = ¢(t)/¢( -t). Then (2t) = ¢ (t) 3 ¢ ( -t) = 1/1 (t) 2 1/I(2t) = ¢( -2t) . ¢ ¢( -t) 3 ¢ (t)
Therefore However, as
1/I(t) = 1/I(�t) 2 = 1/I (it)4 = . . . = 1/I(t/2n ) 2n h -+ 0,
for
n :::: 0.
¢(h) _ I - :21 h 2 + 0(h 2 ) 1 + o(h 2 ) , .I·(h) - -¢ (-h) - 1 - �h 2 + 0(h 2 ) n 1 for all t, so that 1/I(t) = { I + 0(t 2 /22n ) } 2 -+ 1 as n -+ 00, whence 1/I(t) ¢ ( -t) = ¢ (t). It follows that ¢ (t) = ¢ (�t) 3 ¢( _ �t) = ¢(�t) 4 = ¢ (t/2n ) 22n for n :::: 1 2 2n n !t 2 a s n -+ 00, 2 2 . � ! 1 -+ e = - 2 22n + 0(t /2 ) 'I'
{
so that
}
X and Y are N(O, 1). = X + Y and V = X - Y, w e have that 1/I (s, t) = JE(e isU +i t V) satisfies 1/I(s, t) = JE(e i (s + t)X +i (s - t) Y ) = ¢ (s + t)¢(s - t).
(b) With
U
Using what is given,
However, by (*),
/
a 2 t = 2 {¢ "(s) ¢(s) - ¢ ' (s) 2 }, at t =O
yielding the required differential equation, which may be written as
d , d/¢ /¢) = - 1 . 1 2 ¢ (s ) = a + bs - �s 2 for constants a, b, whence ¢ (s) = e - Zs . 24. (a) Using characteristic functions, ¢z(t) = ¢x (t/n) n = e - i t i. (b) JEIXd = 00 .
Hence log
262
giving that
Problems
Solutions
[5.12.25]-[5.12.27]
25.
(a) See the solution to Problem (5 . 1 2.24). (b) This is much longer. Having established the hint, the rest follows thus: fx +Y (Y ) = =
L: f(x)f(y - x) dx I
n (4 + y2 )
1
00
2 x) + = (x) f(y dx + g(y) J { } j n (4 + y 2 ) + J g(y) - 00
where J= =
Finally,
L: {xf (x) + (y - x)f(y - x) } dx
[
N
lim � { IOg(l + X 2 ) - log(1 + (y X ) 2 ) } ] - = O. M. N � oo 2 n M _
fz (z) = 2fx + y (2z) =
1
n (l + Z2 ) '
(a) X l + X2 + " , + Xn. (b) X I - Xl ' where X I and Xl are independent and identically distributed. (c) XN , where N is a random variable with lP'(N = j) = Pj for 1 ::: j ::: n, independent of
26.
X l , X2 , · · " Xn.
(d) I:j'!, 1 Zj where Z I , Z2 , are independent and distributed as X I , and M is independent of the Zj with lP'(M = m) = (i)m+I for m � O. (e) Y X I , where Y is independent of X I with the exponential distribution parameter I . 27. (a) We require . . .
¢ (t) = First method.
1
Consider the contour integral IK =
00
2 i
e tx dx. - 00 e 1fX + e 1fX -
1 e1rZ eitz 1rZ dz 2
C
+e
where C is a rectangular1 contour with vertices at ±K, ±K + i . The integrand has a simple pole at z = 1 i , with residue e - 2 t l(in). Hence, by Cauchy's theorem, as K -+ 00. Second method.
Expand the denominator to obtain 1 ---,--,--,-- =
cosh(nx)
00
L ( - I ) k exp { - (2k + l)n lxl } .
k =O
Multiply by ei tx and integrate term by term. 263
[5.12.28]-[5.12.30]
Generating junctions and their applications
Solutions
(b) Define ¢ (t) = 1 - It I for It I ::s 1, and ¢ (t) = 0 otherwise. Then 00 � 1 e - i t x ¢ (t) dt = � 1 1 e - i tx (1 - ltD dt 2]"( - 1 2]"( - 00 1 (1 - cos x). = -1 10 1 (1 - t) cos(tx) dt = ]"( 0 ]"( x 2 Using the inversion theorem, ¢ is the required characteristic function. (c) In this case, 1-0000 eitx e -x-e -X dx = Jrooo y -it e -y dy = r(1 - it) where r is the gamma function. (d) Similarly, L: �eitx e -Ix l dx = i { foOO eitx e -x dx + 1000 e-itx e-x dX 1 I 1 I = 2" { 1 - it + 1 + it = 1 + t 2 . (e) We have that lE(X) = -i ¢ '(O) = - r' ( I ) . Now, Euler's product for the gamma function states that n '_ nZ --_. r(z ) = n-+oo lim z(z + n) + 1) · . . (z-where the convergence is uniform on a neighbourhood of the point z = 1. By differentiation, 1_) n _ (I y lim { _ 1- ! ..._ _ r'(1) = n-+oo n+1 = n + 1 gn - 2 28. (a) See Problem (5.12.27b). (b) Suppose ¢ is the characteristic function of X. Since ¢ ' (0) = ¢" (0) = ¢'" (0) = 0, we have that lE(X) = var(X) = 0, so that JI>(X = 0) = 1, and hence ¢ (t) = 1, a contradiction. Hence ¢ is not a characteristic function. (c) As for (b). (d) We have that cos t = i (ei t + e - it ), whence ¢ is the characteristic function of a random variable taking values ± 1 each with probability i . (e) By the same working as in the solution to Problem (5. 12.27b), ¢ is the characteristic function of the density function if I x l � 1, f(x) = { o1 - I x l otherwIse. 29. We have that 1 1 - ¢ (t ) 1 ::s lEl l - e itX I = lE J(1 - e itX )(1 - e- itX ) = lE v'2{1 - cos(tX)} ::s lEltXI since 2(1 - cos x) ::s x 2 for all x. 30. This is a consequence of Taylor's theorem for functions of two variables:
}
}
O
_
264
}
_
o
Problems
Solutions
[5.12.31]-[5.12.33]
where ¢mn is the derivative of ¢ in question, and RMN is the remainder. However, subject to appro priate conditions, whence the claim follows. 31. (a) We have that
x-2 < x-2 - x4 < 1 - cos x 4! 3 - 2! -
if I x l :s I, and hence
1 i:
{ 2 It -t-I ,t-I] (tx) dF(x) :s [ - t-I ,t-I] 3{1 - cos(tx)} dF(x) :s 3 {I - cos(tx)} dF(x) = 3{1 - Re ¢(t)}.
(b) Using Fubini's theorem, !
r r r t 10 {l - Re ¢(v)} dv 1xoo= - 00 !t 1v=0 {I - cos(vx )} dv dF(x) 00 ( ) dF(x) = 1 1 - Sin(tX) tx - 00 ( Sin(tX) ) dF(x) { � 1l txlx:� 1 1 - tx since 1 - (tx) - I sin(tx) � 0 if Itxl < 1. Also, sin(tx) :s (tx) sin 1 for Itxl � I, whence the last integral is at least { I 1 x : (1 - sin l) dF(x) � � JI>(IXI � t - ). =
l tx l � 1
32.
It is easily seen that, if y > 0 and n is large,
(a) The characteristic function of Y.. is i t/I.. (t) = lE { exp (it (X - I.. ) / JA) } = exp { A (e t / .JI - 1) - itJA} = exp { _ ! t 2 + 0(1) } as A -+ 00. Now use the continuity theorem. (b) In this case,
33.
so that, as A -+ 00,
(-
)
t 2 + 0(1.. - 1 ) -+ - 1 t2 . It. ) = -itJA + A It. - log t/I.. (t) = -itJA - Hog ( 1 - z v'I v'I 21.. 265
[5.12.34]-[5.12.36]
Generating functions and their applications
Solutions
(c) Let Zn be Poisson with parameter n. By part (a), 11"
( Z'Jn n 0) -HI>(O) = i ::s
where 4> is the N(O, 1) distribution function. The left hand side equals lI" (Zn ::s n) = "E,'k=O e -n nk /kL 34. If you are in possession of r - 1 different types, the waiting time for the acquisition of the next new type is geometric with probability generating function G r (s ) = (n - r + l)s .
n - (r - l)s Therefore the characteristic function of Un = (Tn - n log n)/n is n n it / n - it � , 1/In (t) = e - i t log n II G r (e it / n ) = n -it II n - r + 1) el. t / n = - 1 n t . n . rrr=O (ne -l / - r) r= l n - (r - 1)e r= l
{(
}
The denominator satisfies n- l n- l n II (ne - it / - r) = (1 + 0(1» II (n - it - r) r=O r=O
as n -+ 00, by expanding the exponential function, and hence nlim ---+ oo 1/In (t) = nlim ---+ oo
n - it n
. 1 o rr�,: (n - it - r) = r(1 - it), '
where we have used Euler's product for the gamma function: n ! nZ r=O (z + r)
rrn
-+
r(z)
as n -+ 00
the convergence being uniform on allY region of the complex plane containing no singularity of r. The claim now follows by the result of Problem (5. 12. 27c) . 35. Let Xn be uniform on [-n, n], with characteristic function if t :;6 0, if t = O. It follows that, as n -+ 00, ¢n (t) -+ OOt , the Kronecker delta. The limit function is discontinuous at t = 0 and is therefore not itself a characteristic function. 36. Let G (s) be the probability generating function of the number shown by the i th die, and suppose that 12 j
'"" 1 S k = s 2 (1 - s 1 1 ) , G l (S)G 2 (S) = f=5. IT 1 1(1 - s)
so that 1 - s l 1 = 1 1(1 - S)Hl (S)H2 (S) where Hi (S) = s - I Gj (s) is a real polynomial of degree 5. However 5 1 - s l 1 = (1 - s) II (Wk - S)(Wk - s) k= l 266
Problems
Solutions
[5.12.37]-[5.12.38]
where wI , wI , . . . , w5 , W5 .are the ten complex eleventh roots of unity. The wk come in conjugate pairs, and therefore no five of the ten tenns in rr�= l (Wk - S)(Wk - s) have a product which is a real polynomial. This is a contradiction. 37. (a) Let H and T be the numbers of heads and tails. The joint probability generating function of H and T is where p
=
I -q
is the probability of heads on each throw. Hence G H,T (S, t) = G N (qt + ps) = exp { A (qt + ps - I)} .
It follows that so that GH,T (S, t) = GH(S)G T (t), whence H and T are independent. (b) Suppose conversely that H and T are independent, and write G for the probability generating function of N. From the above calculation, G H,T (S, t) = G(qt + ps), whence G H (S) G(q + ps) and G T (t) = G(qt + p), so that G(qt + ps) = G(q + ps)G(qt + p) for all appropriate s , t. Write f(x) = G( 1 - x) to obtain f(x + y) = f(x)f(y), valid at least for all 0 ::: x, y ::: min{p, q} The only continuous solutions to this functional equation which satisfy f (0) = 1 are of the form f(x) = eJ1-X for some {.t, whence it is immediate that G (x) = eA(x- l ) where A = - {.t . 38. The number of such paths rr containing exactly n nodes is 2n - l , and each such rr satisfies lP(B(rr) � k) = lP(Sn � k) where Sn = YI + Y2 + . . . + Yn is the sum of n independent Bernoulli variables having parameter p (= 1 - q). Therefore lE{Xn (k)} = 2n -1 lP(Sn � k). We set k = nf3, and need to estimate lP(Sn � n(3). It is a consequence of the large deviation theorem (5.11.4) that, if p ::: f3 < I , lP(Sn � n(3) I / n -+ inf { e -tf3 M(t) } t >O where M(t) = lE(et Y1 ) = (q + pet ) . With the aid of a little calculus, we find that =
.
( )f3 (1 = � ) 1-f3 , P ::: f3 < 1 . � 1 Hence lE{Xn (f3n)} { O �f y(f3) < 1, f y (f3) 1, where -f3 y(f3) 2 � (�r C = r is a decreasing function of f3. If p < !, there is a unique f3c [p, 1) such that y (f3c) 1; if p then y(f3) 1 for all f3 [p, 1) so that we may take f3c = 1. lP(Sn � n(3) I / n -+
-+
00
I
>
=
>
E
E
=
Turning to the final part,
if f3 > f3c. As for the other case, we shall make use of the inequality 2 lP(N =1= 0) >- lE(N) lE(N2 )
-
267
�
1
[5.12.38]-[5.12.38]
Generating junctions and their applications
Solutions
for any N taking values in the non-negative integers. This is easily proved: certainly var(N I N =1= 0) = lE(N2 I N =1= 0) - lE(N I N =1= 0) 2 2:: 0, whence lE (N2 ) 2:: lE (N) 2 lP'(N =1= 0) lP'(N =1= 0) 2 . We have that lE{Xn CBn) 2 } = Err, p lE ( Irr lp ) where the sum is over all such paths Jr, and Irr is the indicator function of the event {B(Jr) 2:: fJn}. Hence lE{Xn (fJn) 2 } = L lE ( Irr) + L lE ( I lp ) = lE{Xn (fJn)} + 2n - 1 L lE(hlp) p,
rr
rr #p
rr
p# L
where L is the path which always takes the left fork (there 2n - 1 choices for Jr, and by symmetry each provides the same contribution to the sum). We divide up the last sum according to the number of nodes in common to and L, obtaining E�-:;\ 2n -m - 1 lE(hIM) where M is a path having exactly m nodes in common with L. Now are
p
where Tn -m has the bin(n - m, p) distribution (the 'most value' to 1M of the event {h = I} is obtained when all m nodes in L n M are black). However so that lE(hIM) � p -m lE(h)lE(IM). It follows that N = Xn (fJn) satisfies
whence, by (*), lP'(N =1= 0) ->
1 . 1 lE(N) - + � E�-:'\ (2p) -m
If fJ < fJe then lE(N) 00 as n 00. It is immediately evident that lP'(N =1= 0) 1 if p � � . Suppose finally that p > � and fJ < fJe. By the above inequality, lP'(Xn (fJn) > 0) 2:: c(fJ) for all n where c(fJ) is some positivemconstant. Find E > 0 such that fJ + E < fJe. Fix a positive integer m, and let :Pm be a collection of 2 disjoint paths each of length n - m starting from depth m in the tree. Now lP'(Xn (fJn) = 0) � lP'(B(v) < fJn for all v E :Pm ) = lP'(B(v) < fJn ) 2m where v E :Pm . However lP'(B(v) < fJn) � lP'( B(v) < (fJ + E ) (n - m) ) if fJn < (fJ + E ) (n - m), which is to say that n 2:: (fJ + E)m/E. Hence, for all large n, -+
-+
-+
268
Problems
Solutions
[5.12.39]-[5.12.42]
by (**); we let n --+ 00 and m --+ 00 in that order, to obtain JI>(Xn (fJn) = 0) --+ ° as n --+ 00. 39. (a) The characteristic function of Xn satisfies the characteristic function of the Poisson distribution. (b) Similarly, it/n A lE(e i t Yn/ n ) = I (1p e- p)e 1t. / n --+ A it as n --+ 00, the limit being the characteristic function of the exponential distribution. 40. If you cannot follow the hints, take a look at one or more of the following: Moran 1968 (p. 389), Breiman 1968 (p. 186), Loeve 1977 (p. 287), Laha and Rohatgi 1979 (p. 288). 41. With Yk = kXb we have that lE(Yk) = 0, var(Yk) = k2 , lEIY,? 1 = k3 . Note that Sn = YI + Y2 + . . . + Yn is such that
-
-
__
1 n4 � -{v-ar-(S-n"--'-)}"3 /"'2 f:J. lEI Yk3 I '" c n 9/2 --+ °
as n --+ 00, where c is a positive constant. Applying the central limit theorem « 5. 10.5) or Problem (5.12.40» , we find that Sn �D N(O, 1), as n --+ 00, r.;;;:c;",var Sn where var Sn = 2:Z= 1 k2 '" 1 n 3 as n --+ 00. 42. We may suppose that J-L = ° and (J = 1; if this is not so, then replace Xi by Yi = (Xi J-L)/(J . Let t = (to, tl , t2 , . . . , tn) E IRn+l , and set I = n - I 2:7= 1 tj . The joint characteristic function of the n + 1 variables X, Z } , Z2 , . . . , Zn is
-
¢ (t)
itj Zj ) } = lE { fI exp (i [� + tj - ] Xj ) } t =I J J =I = fl exp (-� [ � + tj f )
{ (
I
= lE exp itoX +
by independence. flence ¢ (t)
= exp
-I
(- -2I j=E1 [ tn + (tj - 7)] 2 ) = exp { _ ...Q.2nt2 . - -21 j=E1 (tj _ 7)2 } n
n
J1.
where we have used the fact that 2:7= 1 (tj - 7) = 0. Therefore
whence X is independent of the collection Z I , Z2 , . . . , Zn. It follows that X is independent of S2 = (n - 1) - 1 2:J= 1 ZJ . Compare with Exercise (4.10.5). 269
[5.12.43]-[5.12.47]
Generating functions and their applications
Solutions
(i) Clearly, lP(Y :s y) = lP(X :s log y ) = (log y) for y > 0, where is the N(O, 1) distribution function. The density function of Y follows by differentiating. (ii) We have that fa (x) � 0 if lal :s 1, and
43.
1
1 2 1 e - z1 (Iog x) 2 dx = 00 1 a sin (2ny)e - zY dy = 0 a sin (2n log x) r,:o= o -00 ...; 2n x...; 2n since sine is an odd function. Therefore J�oo fa (x) dx = 1, so that each such fa is a density function. For any positive integer k, the kth moment of fa is J�oo x k f (x) dx + la (k) where 00 1 k 1 2 la Ck) = r,:o= a sin(2ny )e Y-zY dy = 0 -00 ...; 2n since the integrand is an odd function of y - k. It follows that each fa has the same moments as f. 44. Here is one way of proving this. Let X l , X2 , . . . be the steps of the walk, and let Sn be the position of the walk after the nth step. Suppose J1, = lE(X I ) satisfies J1, < 0, and let em = lP(Sn = o for some n � 1 I So = -m ) where m > O. Then em :s E� l lP(Tn > m) where Tn = X l + X2 + . . . + Xn = Sn - SO . Now, for t > 0, lP(Tn > m ) = lP(Tn - nJ1, > m - nJ1,) :s e - t(m - ntl-) lE(e t( Tn-ntl-» ) = e - tm { e t tl- M(t) r
10
00
r,:o=
1
where M(t) = lE(e t(X I -tl-» ). Now M(t) = 1 + 0(t 2 ) as t -+ 0, and therefore there existsmt ( > 0)n such that B(t) = e t tl-M(t) < 1 (remember that J1, < 0). With this choice of t, em :s E� l e - t B(t) -+ 0 as m -+ whence there exists K such that em < i for m � K. Finally, there exist 8, E > 0 such that lP(X I < -8) > E, implying that lP(SN < -K I So = 0) > E N where N = r K / 81 , and therefore lP(Sn ;6 0 for all n � 1 I So = 0) � ( l - e K )E N � iE N ; therefore the walk is transient. This proof may be shortened by using the Borel-Cantelli lemma. 45. Obviously, { a if X I > a, L= X I + L if X I :s a, where L has the same distribution as L. Therefore, 00,
lE(s L ) = s a lP(X I 46.
We have that
>
{ Wn- I + 1
a a) + L s r lE(s L )lP(X I = r). r= l
with probability p, Wn - l + 1 + Wn with probability q, where Wn is independent of Wn _ 1 and has the same distribution as Wn . Hence Gn (s) = psGn _ I (S) + qsG n - 1 (s)Gn (s). Now Go(s) = 1, and the recurrence relation may be solved by induction. (Alter natively use Problem (5.12.45) with appropriate Xi .) 47. Let Wr be the number of flips until you first see r consecutive heads, so that lP(L n < r ) = lP(Wr > n). Hence, Wn =
-
270
Problems
Solutions
[5.12.48]-[5.12.52]
where lEes wr ) = G r (s ) is given in Problem (5.12.46). 48. We have that with probability � , iXn Xn+l = 1 Z Xn + Yn with probability � . Hence the characteristic functions satisfy
{
_
1 n ( z1 t) + z4> 1 n( z1 t) A 4>n+ l (t). = lE(e i t Xn +J ) = z4> A it -n ,\ it iit A A = 4>n ( z1 t) A - it = 4>n - l ( 41 t) A - it = . . · = 4> I (t2-n ) A -A -it2it -+ A -A it as n -+ 00 . The limiting distribution is exponential with parameter A. 49. We have that
(a) (l- e - A )/A , (b) -(p/q 2 ) (q +1og p), (c) (l _ q n+l )/ [ (n+1) p] , (d) - [ 1+(p/q ) log p]/ log p . (e) Not if lP(X + 1 > 0) = 1, by Jensen's inequality (see Exercise (5.6.1)) and the strict concavity of the function f(x) = l /x . If X + 1 is permitted to be negative, consider the case when lP(X + 1 = -1) = lP(X + 1 = 1) = i. 50. By compounding, as in Theorem (5.1 .25), the sum has characteristic function G N (4)X (
t )) - 1 p4>x (t)(t) -- � AP - it ' - q 4>x _
whence the sum is exponentially distributed with parameter Ap . 51. Consider the function G (x ) = (lE(X 2 )} - 1 J� oo y 2 dF(y). This function is right-continuous and increases from 0 to 1, and is therefore a distribution function. Its characteristic function is
By integration, fx(x ) = fy (y) = � , Ix l < 1, Iyl < 1. Since f(x, y) =1= fx (x) fy (y), X and Y are not independent. Now, 1 ,\ (z + 2) if - 2 < z < 0, fx+ Y (z) = 1 f(x , z - x) dx = 1 4 (2 - z) if O < z < 2 , -1 the 'triangular' density function on (-2, 2). This is the density function of the sum of two independent random variables uniform on (-1, 1). 52.
{
27 1
6 Markov chains
6.1 Solutions.
Markov Processes
1.
The sequence X l , X2 , . . . of independent random variables satisfies lP'(Xn+l = j I X l = il , · · · , Xn = in) = lP'(Xn+l = j), whence the sequence is a Markov chain. The chain is homogeneous ifthe Xi are identically distributed. 2. (a) With Yn the outcome of the nth throw, Xn+l = max{Xn, Yn+l }, so that if j < i . . 1 · Pij = 6 ' If J = , i if j > i, for 1 ::: i, j ::: 6. Similarly, if j < i Pij (n) = ( -1 ,. ) n I. f . = l. . J 6 If j > i, then Pij (n) = lP'(Zn = j), where Zn = max{Yl , Y2 , . . . , Yn}, and an elementary calculation yields ( . ) n - ( J. l ) n , i < j ::: (n) =
{o
.
{o
Pij
=k
{
6.
�
(b) Nn+l - Nn is independent of Nl , N2 , . . . , Nn, so that N is Markovian with 1 I· f J· = ,. + 1 , 6 Pij = � if j i, otherwise. (c) The evolution of C is given by if the die shows 6, Cr+ l = { + 1 otherwise, �r whence C is Markovian with 1 o j = 0, Pij = � j = i + I , otherwise.
o
{
=
o
272
Markov processes
Solutions
{
(d) This time,
Br+ 1 - Br - 1 Yr
[6.1.3]-[6.1.4]
if Br > 0, ·f B - 0, where Yr is a geometrically distributed random variable with parameter t, independent of the sequence Bo, B2 , . . . , Br . Hence B is Markovian with if j = i - I ;:: 0, I Pij = j l ( �o ) - �o l· f /· = 0 , j. ;:: 1 . 3. (i) If Xn = i, then Xn + 1 E {i - 1, i + I}. Now, for i ;:: 1, 1
r -
{
(* )
lP(Xn + l = i + 1 I Xn = i, B) = lP(Xn + l = i + 1 I Sn = i, B)lP(Sn = i I Xn = i, B) + lP(Xn+ l = i + 1 I Sn = -i, B)lP(Sn = -i I Xn = i, B) where B = {Xr = ir for 0 ::: r < n} and io, i l , . . . , in - l are integers. Clearly lP(Xn+l = i + 1 I Sn = i, B) = p , lP(Xn+l = i + 1 I Sn = -i, B) = q, where p (= 1 q) is the chance of a rightward step. Let 1 be the time of the last visit to 0 prior to the time n, 1 = max{r i r = OJ. During the time-interval (1, n], the path lies entirely in either the -
:
positive integers or the negative integers. If the former, it is required to follow the route prescribed by the event B n {Sn = i }, and if the latter by the event B n {Sn = -i}. The absolute probabilities of these two routes are whence lP(Sn = i I Xn = i, B) =
Jl"1 Jl"1 + Jl"2
=
L = 1 - lP(Sn pi + q l
=
-i I Xn = i, B).
Substitute into (*) to obtain HI Hl lP(Xn+l = . + 1 I Xn = . B) = p pi. + q l. = 1 - lP(Xn + l = i - I I Xn i, B). +q Finally lP(Xn+l = 1 I Xn = 0, B) = 1. (ii) If Yn > 0, then Yn - Yn+ 1 equals the (n + l)th step, a random variable which is independent of the past history of the process. If Yn = 0 then Sn = Mn , so that Yn+ 1 takes the values 0 and 1 with respective probabilities p and q, independently of the past history. Therefore Y is a Markov chain with transition probabilities { p if j = O for i > 0, Pij = { qp if·f j· == i· +- 11 POj = q if j = 1. j The sequence Y is a random walk with a retaining barrier at O. 4. For any sequence io, i I , . . . of states, s = i s for 0 -< s -< k + 1) lP(Yk+ 1 = ik+l I Yr = i r for 0 ::: r ::: k) = lP(Xn lP(Xn = . for 0 ::: s ::: k) I
I,
=
1
I
,
s
s TI�=o Pis , is+l (n s + 1 - n s ) = k l TIs =-o Pis , is+ ! (n s + 1 - n s ) = Pik , ik+ ! (n k+ 1 - n k ) = lP(Yk+l 273
I
=
ik+l I Yk = i k ),
[6.1.5]-[6.1.9]
Marlwv chains
Solutions
where Pij (n) denotes the appropriate n-step transition probability of X. (a) With the usual notation, the transition matrix of Y is p 2 if j = i + 2, TCij = 2 pq if j = i, q 2 if j = i 2.
{
-
(b) With the usual notation, the transition probability TCij is the coefficient of s j in G(G(s)i . 5. Writing X = (X l , X2 , . . . , Xn), we have that J (X) = I , Xn = i ) IP' ( F I J (X) = I , Xn = 1 ) = IP'(F, IP' ( J (X) = 1 , Xn = . ) .
I
where F is any event defined in terms of Xn, Xn+l , . . . Let A be the set of all sequences x = (X l , x2 , . . . , xn - l , i) of states such that J (x) = 1. Then .
xeA
xeA
by the Markov property. Divide through by the final summation to obtain IP'(F I J (X) = 1, Xn =
i) = IP'(F I Xn = i). 6 . Let Hn = {Xk = Xk for 0 :::: k < n, Xn = i }. The required probability may be written as
IP'(X T+m = j, HT ) = �������----� En IP'(XT +m = j, HT , T = n) IP'(HT ) IP'(HT ) Now IP'(XT+m = j I HT , T = n) = IP'(Xn +m = j I Hn, T = n). Let J be the indicator function of the event Hn n {T = n}, an event which depends only upon the values of X l , X2 , . . . , Xn. Using the result of Exercise (6.1.5), IP'(Xn +m = j I Hn , T = n) = IP'(Xn +m = j I Xn = i) = Pij (m) .
Hence 7.
Clearly
IP'(Yn+l = j I Yr = i r for 0 :::: r :::: n) = IP'(Xn + l = b I Xr = ar for 0 :::: r :::: n) where b = h - l (j), ar = h - l (ir ); the claim follows by the Markov property of X. It is easy to find an example in which h is not one-one, for which X is a Markov chain but Y is not. The first part of Exercise (6. 1.3) describes such a case if So =1= o. 8. Not necessarily! Take as example the chains S and Y of Exercise (6. 1.3). The sum is Sn + Yn = Mn, which is not a Markov chain. 9. All of them. (a) Using the Markov property of X, IP'(Xm+r = k I Xm = i m , . . . , Xm+r - l
=
i m+r - l ) = IP'(Xm+r = k I Xm+r - l = im+r - l ) · 274
Classification of states
Solutions
(b) Let {even} = {X2r = i2r for 0 ::::: Then,
r :::::
[6.1.10]-[6.2.1]
m} and {odd} { X2r+ l = i2r+ l for 0 ::::: =
r :::::
m - l}.
, IP'(X2m+2 = k, X2m+ l = i 2m+ l ' even, odd) w IP'(X2m+2 = k I even) = � lP'(even) X k, IP'(X = m+ 2m+l = i 2m+ l I X2m = i2m )lP'(even, odd) 2 2 = L, =
IP'(X2m+2 = k I X2m = i 2m ),
lP'(even)
where the sum is taken over all possible values of is for odd s. (c) With Yn = (Xn , Xn+l ), IP' ( Yn+ 1 = (k, I) I YO = (io, i l ), · · · , Yn = (in, k))
=
=
IP' ( Yn+l = (k, I) I Xn+l = k) IP' ( Yn+l = (k, I) I Yn = (i n, k)) ,
by the Markov property of X. 10. We have by Lemma (6.1 .8) that, with /.Ly> = IP'(Xj = j ) , (1)
LHS - /.Lx I P(XIX2 1) _
. . •
PXr_l ,k Pk, xr+I
. • •
PXn -lXn
/.Lx I · · · PXr-IXr+ 1 (2) · · · PXn -lXn
=
( 1)
/.Lx I PXr_l ,k Pk, xr+ 1 ( 1) /.Lx I PXr-IXr+l (2)
=
RHS.
11. (a) Since Sn+ 1 = Sn + Xn+ 1 , a sum of independent random variables, S is a Markov chain. (b) We have that IP'(Yn+ l = k I Yj = Xi + Xi - l for 1 ::::: i ::::: n) = IP'(Yn + l = k I Xn = xn) by the Markov property of X. However, conditioning on Xn is not generally equivalent to conditioning on Yn = Xn + Xn - l o so Y does not generally constitute a Markov chain. (c) Zn = nX 1 + (n - 1)X2 + . . . + Xn, so Zn+ l is the sum of Xn+l and a certain linear combination of Z I , Z2 , . . . , Zn, and so cannot be Markovian. (d) Since Sn+l = Sn + Xn+l ' Zn+l Zn + Sn + Xn+l o and Xn+l is independent of X l , · · · , Xn, this is a Markov chain. 12. With 1 a row vector of 1 's, a matrix P is stochastic (respectively, doubly stochastic, sub-stochastic) interpreted coordinatewise). By recursion, if PI' = 1 (respectively, IP 1, PI' ::::: 1, with inequalities n P satisfies any of these equations if and only if p satisfies the same equation. =
=
6.2 Solutions. Classification of states
1. Let Ak be the event that the last visit to i, prior to n, took place at time k. Suppose that Xo = i, so that Ao, A I , . . . , An - l form a partition of the sample space. It follows, by conditioning on the Ai , that n l
-
Pij (n) = L Pii (k)lij (n - k) k=O
for i =f. j . Multiply by s n and sum over n (;::: 1) to obtain Pij (s) = Pii (s) L ij (s) for i =f. j . Now Pij (s) = Fij (s)Pjj (s) if i =f. j, so that Fij (s) = Lij (s) whenever Pii (s) = Pjj (s). As examples of chains for which Pii (s) does not depend on i, consider a simple random walk on the integers, or a symmetric random walk on a complete graph. 275
[6.2.2]-[6.3.1]
Markov chains
Solutions
LeU (¥= s ) be a state of the chain, and define n i = min{n : Pis (n ) > O}. If Xo = i and Xnj = s then, with probability one, X makes no visit to i during the intervening period [1, n i - I]; this follows from the minimality of n i . Now s is absorbing, and hence JI>(no return to i I Xo = i) � JI>(Xnj = s I Xo = i) > O.
2.
Let h be the indicator function of the event {Xk = i }, so that N = l:�o h is the number of visits to i. Then 00 00 lE(N) = :L lE(h ) = :L Pii (k) 3.
k=O
k=O
which diverges if and only if i is persistent. There is another argument which we shall encounter in some detail when solving Problem (6.15.5). 4. We write Jl>i O = JI>( . I Xo = i). One way is as follows, another is via the calculation of Problem (6. 15.5). Note that Jl>j (Vj � 1) = JI>j (Ij < 00). (a) We have that by the strong Markov property (Exercise (6. 1 .6)) applied at the stopping time 11. By iteration, Jl>j (Vj � n) = JI>j (Vj � 1 ) n , and allowing n -+ 00 gives the result. (b) Suppose i =1= j. For m � 1, Jl>j (Vj � m ) = Jl>j ("} � m I Ij < oo)JI>j (Ij < 00 ) = Jl>j ("} � m - l)JI>j (Ij < 00)
by the strong Markov property. Now let m -+ 00, and use the result of (a). 5. Let e = JI>(Ij < 11 I Xo = i) = JI>(Tj < Ij I Xo = j), and let N be the number of visits to j before visiting i . Then JI>(N � 1 I Xo = i) = JI>(Ij < 11 I Xo = i) = e. Likewise, JI>(N � k I Xo = i) = e(1 - e) k - 1 for k � 1, whence 00
lE(N I Xo = i) = :L e(1 - e) k - 1 = 1 . k= 1
6.3 Solutions.
Classification of chains
1. If r = 1, then state i is absorbing for i � 1; also, 0 is transient unless ao = 1. Assume r < 1 and let J = supU : aj > O}. The states 0, 1 , . . . , J form an irreducible persistent class; they are aperiodic if r > O. All other states are transient. For 0 :s i :s J, the recurrence time Tj of i satisfies JI>(1I = 1) = r. If 11 > 1 then Tj may be expressed as the sum of 11( 1 ) := time to reach 0, given that the first step is leftwards, 11(2) := time spent in excursions from 0 not reaching i, TP ) := time taken to reach i in final excursion. 276
Classification of chains
Solutions
[6.3.2]-[6.3.3]
It is easy to see that E(TP ) ) = 1 + (i - 1)/0 - r) if i � I , since the waiting time at each intermediate point has mean (1 - r ) - I . The number N of such 'small' excursions has mass function JJ!'(N = n) = aiO -ai) n , n � 0, where aj = L�i aj ; hence E(N) = O -aj)/aj. Each such small excursion has mean duration i- I
( _J_. + 1 ) �
j; l - r
i-I
Ja. j
l - aj - j; o - aj)o - r) _
1+
and therefore
(
By a similar argument,
oo 1 + j - i ) a · . E ( TI.(3) ) = a.1 L 1 r J I j =j Combining this information, we obtain that -
_
and a similar argument yields E(To) = 1 + Lj jaj /O -r ) . The apparent simplicity of these formulae suggests the possibility of an easier derivation; see Exercise (6.4.2). Clearly E(Tj) < for i :s J whenever Lj jaj < a condition which certainly holds if J < 2. Assume that 0 < P < 1 . The mean jump-size is 3 P - I, whence the chain is persistent if and only if P = 1 ; see Theorem (5.lD.17). 3. (a) All states are absorbing if P = O. Assume henceforth that # O. Diagonalize P to obtain l P = BAB - where 00
00,
00 .
p
0
-1
A=
P"
� BA"B - 1 � B
P 1 1 (n) = 1 +
1 1
_
:2
if
0
1 2
(I
! O - 2p)n + 1 0 - 4p)n ,
and P33 (n) = P 1 1 (n) by symmetry. 277
o o
1 4
whence jj (n) In particular,
p
if
B- 1 -
0
Therefore
1
I
G �} C ) -l -p -p G ) gp>" 4p)n G is easily found. B�
1
:2 0
if
)
.
'
o o
B-1
_
1 2
(1 -
! ! O - 4p)n ,
P22 (n) = +
[6.3.4]-[6.3.5]
Solutions
Markov
chains
Now Fjj (s) = 1 - Pjj (s) - I , where 1 1 1 Pl 1 (s) = P33 (S) = 4(1 s) + 2{1 - s(1 - 2 )} + 4{1 - s(1 - 4p)} , p 1 1 P22 (S) = 2(1 - s) + 2{1 - s(1 - 4p)} . _
After a little work one obtains the mean recurrence times l1-i = F[i (1): 11- 1 = 11-3 = 4, 11-2 = 2. (b) The chain has period 2 (if p =j:. 0), and all states are non-null and persistent. By symmetry, the mean recurrence times l1-i are equal. One way of calculating their common value (we shall encounter an easier way in Section 6.4) is to observe that the sequence of visits to any given state j is a renewal process (see Example (5.2.15)). Suppose for simplicity that p =j:. O. The times between successive visits to j must be even, and therefore we work on a new time-scale in which one new unit equals two old units. Using the renewal theorem (5.2.24), we obtain 2 if i j 2 if i j - il is odd; ( 2n + 1) -+ i I is even, Pij Pij (2n ) -+ . I1I1-
j j note that the mean recurrence time of j in the new time-scale is � l1-j . Now Ej Pij (m) = 1 for all m, and so, letting m = 2n -+ 00, we find that 4/ 11- = 1 where 11- is a typical mean recurrence time. There is insufficient space here to calculate Pij (n). One way is to diagonalize the transition matrix. Another is to write down a family of difference equations of the form P 12 (n) = P . P22 (n 1) + (1 - p ) . P42 (n - 1), and solve them. 4. (a) By symmetry, all states have the same mean-recurrence time. Using the renewal-process argument of the last solution, the common value equals 8, being the number of vertices of the cube. Hence I1- v = 8. Alternatively, let s be a neighbour of and let t be a neighbour of s other than In the obvious notation, by symmetry, v,
v.
- 1 + 43 l1-sv , 1 , I1-t v = 1 + 2,1 l1-s v + 41 11-t v + 4l1-wv I1- v
1 + 41 11-sv + 2,1 l1-t v , I1- wv = 1 + 41 11- wv + 43 l1- t v , I1-sv =
a system of equations which may be solved to obtain I1- v = 8 . (b) Using the above equations, I1- wv = �o , whence I1- vw = �o by symmetry. (c) The required number X satisfies lP(X = n ) = n - 1 ( 1 - ) 2 for n � 1, where is the probability that the first return of the walk to its starting point precedes its first visit to the diametrically opposed vertex. Therefore 00 n JE(X) = L n - 1 ( 1 - ) 2 = 1 .
e
n= 1
5.
e
e
e
e
(a) Let lPi ( ) = lP(· I Xo = i). Since i is persistent, .
1 = lPi (Vi = 00 ) = lPj (Vj = 0, Vi = 00) + lPi (Vj > 0, Vi = 00) ::: lPi (Vj = 0) + lPi ('1j < 00, Vi = 00) .::: 1 - lPi ('1j < 00 ) + lPi ('1j < oo)lPj (Vi = 00),
by the strong Markov property. Since i -+ j, we have that lPj (V = 00) � 1, which implies TJji = 1. Also, lPi ('1j < 00) = 1, and hence j -+ i and j is persistent. Thisi implies TJij = 1. (b) This is an immediate consequence of Exercise (6.2.4b). 278
Classification of chains
Solutions
[6.3.6]-[6.3.9]
6. Let lP'i (- ) = lP'( . I Xo = i). It is trivial that TJj = 1 for j E A. For j ¥ A, condition on the first step and use the Markov property to obtain
k) = L Pj k TJk· k If x = (Xj : j E S) is any non-negative solution of these equations, then Xj = 1 ;::: TJj for j E A. For j ¥ A, TJj = L Pjk lP'(TA < ke S
00 I Xl
=
Xj = L Pj k Xk = L Pjk + L Pj k Xk = lP'j (TA = 1) + L Pj k Xk ke A keS k�A k�A = lP'j (TA = 1) + L Pjk L Pki + L Pki Xi = lP'j (TA � 2) + L Pj k L Pki Xi . k�A i�A k�A i�A i eA
}
{
We obtain by iteration that, for j if. A,
where the sum is over all kl ' k2' . . . , kn if. A. We let n -+ to find that Xj ;::: lP'j (TA < = TJj . 7. The first part follows as in Exercise (6.3.6). Suppose x = (Xj : j E S) is a non-negative solution to the equations. As above, for j ¥ A,
00
00)
)
(
Xj = 1 + L Pjk Xk = lP'j (TA ;::: 1) + L pjk l + L Pki Xi k k� i� = lP'j (TA ;::: 1) + lP'j (TA ;::: 2) + . . . + lP'j (TA ;::: n) + L Pjk l Pk l k2 . . . Pkk_l kn Xkn n ;::: L lP'(TA ;::: m), m= l
where the penultimate sum is over all paths of length n that do not visit A. We let n -+ to obtain that xj ;::: Ej (TA ) = Pj . S. Yes, because the Sr and Tr are stopping times whenever they are finite. Whether or not the exit times are stopping times depends on their exact definition. The times Ur min{k > Ur- l : X Ur E A, XUr+l if. A} are not stopping times, but the times Ur + 1 are stopping times. 9. (a) Using the aperiodicity of j, there exist integers r l , r2 , . . . , rs having highest common factor 1 and such that Pjj (rk ) > 0 for 1 � k � s. There exists a positive integer M such that, if r ;::: M, then r = 2:�= 1 ak rk for some sequence a I , a2 , . . . , as of non-negative integers. Now, by the Chapman Ko1mogorov equations,
00
=
s Pjj (r) ;::: II Pjj (rk ) ak
> 0,
k=l
so that Pjj (r) > 0 for all r ;::: M. Finally, find m such that Pij (m) > O. Then if r ;::: M. (b) Since there are only finitely many pairs i, j, the maximum R(P) = max{N(i, j) : i , j E S} is finite. Now R(P) depends only on the positions of the non-negative entries in the transition matrix P. 279
[6.3.10]-[6.3.10]
Markov chains
Solutions
There are only finitely many subsets of entries of P, and so there exists f(n) such that R(P) .:s f(n) for all relevant n n transition matrices P . (c) Consider the two chains with diagrams in the figure beneath. In the case on the left, we have that PII (5) = 0, and in the case on the right, we may apply the postage stamp lemma with a = n and b = n - 1. x
3
4 2
4
2
10. Let Xn be the number of green balls after n steps. Let ej be the probability that Xn is ever zero when Xo = j. By conditioning on the first removal,
ej = j + 2 ej+ 1 + 2(j j+ 1) ej - I , j � 1, 2(j + 1)
with eo = 1. Solving recursively gives
{
}
q i + . . . + q I q2 · · · qj - I , ej = 1 - (1 - e l ) 1 + PI P2 · · · Pj - I PI
(*)
where
j+2 Pj = 2(j + 1) '
It is easy to see that
j-I q q . . · q I 2 J·- I = 2 - 2 -+ 2 as J. -+ 00 . j+1 PI r=O P2 · · · Pj - I By the result of Exercise (6.3.6), we seek the minimal non-negative solution (ej ) to (*) , which is attained when 2(1 - e l ) = 1, that is, e l = 1 . Hence j-I q q · · · q · I _1 = 1 I 2 ej = 1 · . . . PI P j 2: r=O 2 ; -I j + 1 For the second part, let dj be the expected time until j - 1 green balls remain, having started with j green balls and j + 2 red. We condition as above to obtain dj = 1 + j {dj+l + dj }. 2(j + 1) We set ej = dj - (2j + 1) to find that (j + 2)ej = jej+ b whence ej = 1 j (j + 1)e l . The expected
L
--
L
time to remove all the green balls is n
n
n
L j=L1 { ej + 2(j - 1) } = n(n + 2) + e l j=L1 1 j (j + 1). j= 1 The minimal non-negative solution is found by setting e l = 0, and the conclusion follows by Exercise (6.3.7). dj =
280
Stationary distributions and the limit theorem
Solutions
[6.4.1]-[6.4.2]
6.4 Solutions. Stationary distributions and the limit theorem 1. Let Yn be the number of new errors introduced at the nth stage, and let G be the common probability generating function of the Yn. Now Xn+ 1 = Sn + Yn+ 1 where Sn has the binomial distribution with parameters Xn and q (= 1 - p). Thus the probability generating function G n of Xn satisfies
G n+I (S) = G(s) JE. (s Sn ) = G(s) JE. {JE. (s Sn I Xn)} = G(s) JE. {(p + qs) Xn } = G(s)G n (p + qs) = G(s)G n (1 - q(1 - s) ) . Therefore, for s < 1, Gn (s) = G(s)G (1 - q(1 - s) ) G n - 2 (1 - q 2 (1 - s)) = n- l 00 n r r = Go (l - q (1 - s)) II G ( 1 - q (1 - s) ) � II G ( 1 - q (1 - s)) r=O r=O as n � 00, assuming q < 1. This infinite product is therefore the probability generating function of the stationary distribution whenever this exists. If G(s) = eA(s - l ) , then . . .
G ( 1 - q r (1 - s)) IT r=O
=
exp { J.. (s - 1) f qr
r=O so that the stationary distribution is Poisson with parameter J.. / p. 2. (6.3.1): Assume for simplicity that sup{j : aj > O} = 00 .
}
=
eA(s - I ) /p ,
The chain is irreducible if r < 1 . Look for a stationary distribution with probability generating function G. We have that 1C
Hence
sG(s) = 1CosA(s) + rs (G(s) - 1Co) + (1 - r)(G(s) - 1CO ) where A(s) = 'L.1=o aj s j , and therefore
(
)
- (1 - r + sr) . G(s) = 1Co SA(S) (1 - r)(s - 1)
Taking the limit as s t 1, we obtain by L'Hopital ' s rule that G(1) = 1Co
( A'(1)l -+ r1 - r ) .
There exists a stationary distribution if and only if r < 1 and A' (1)
0 . 2p 2p 0 > > Note that < f3 < 1 if p 1, while f3 1 if p < 1 . For p > 1, set A + Bf3 i if i � 1 , Yi = C + Da i if i :::; - 1 , the constants A, B, C, D being chosen in such a manner as to 'patch over' the omission of 0 in the
{
equations (*):
Y -z = q Y- 3 , Y- l = qy-Z + PYl , Yl = PY3 · The result is a bounded non-zero solution { Yj } to (*), and it follows that the chain is transient. For p < 1, follow the same route with 0 if i � 1 , = Yi C + Da i + Ef3 i if i :::; - 1, the constants being chosen such that Y-Z = qY- 3 , Y- l = qy-z · Finally suppose that p = 1, so that a = -2 and f3 = 1. The general solution to (*) is A + Bi + Ca i if i � 1 , YI - D + Ei + Fa i if i :::; - I , subject to (**). Any bounded solution has B = E = C = 0 , and (**) implies that A = D = F =
{
.
_
{
o.
Therefore the only bounded solution to (*) is the zero solution, whence the chain is persistent. The equation x = xP is satisfied by the vector x of 1 's; by an appeal to (6.4.6), the walk is null. (6.3.3): (a) Solve the equation n = nP to find a stationary distribution n = (! , �, ! ) when p oF O. Hence the chain is non-null and persistent, with /.L l = ni l = 4, and similarly /.Lz = 2, /.L3 = 4. (b) Similarly, n = (! , ! , ! , ! ) is a stationary distribution, and /.Li = nj- l = 4. (6.3.4): (a) The stationary distribution may be found to be nj = � for all i, so that /.L v = 8. 3. The quantities X 1 . Xz ' . . . , Xn depend only on the initial contents of the reservoir and the rainfalls Yo, Yl , . . . , Yn - l . The contents on day n + 1 depend only on the value Xn of the previous contents and the rainfall Yn . Since Yn is independent of all earlier rainfalls, the process X is a Markov chain. Its state space is S = {O , 1 , 2, . . . , K - I } and it has transition matrix gO + g l gZ g3 . . . g K- l G K g0O g l gZ . . . g K-Z G K- l gO g l . . . g K- 3 G K-Z = ·· · ". , . . . ... ·· 0 0
[IP'
o ·
.
.
1
Gl gO where gj = IP'(YI = i) and Gj = 'L-1=j gj . The equation n = n P is as follows: Jro = Jro(gO + g l ) + Jr l gO , 0 < r < K - 1, Jrr = JrO gr+ l + Jr l gr + . . . + Jrr+ l gO , . . . JrK-l = JrO G K + Jr l G K- l + + JrK-I G l · 282
Stationary distributions and the limit theorem
SolutioDS
[6.4.4]-[6.4.5]
The final equation is a consequence of the previous ones, since ��O I 1T:i = 1 . Suppose then that = ( V I , V2 , . . . ) is an infinite vector satisfying Vo = vo(gO + g l ) + v I gO , Vr = vO gr+ 1 + v I gr + . . . + vr+ l gO for r > O. Multiply through the equation for Vr by s r+ l , and sum over r to find (after a little work) that v
00
00
N(s) = L VjS i , i=O
G(s) = L gi s i i=O
satisfy sN(s) = N(s)G(s) + vOgo(s - 1), and hence 1 go(s - 1) . -N(s) = s - G(s) Vo The probability generating function of the 1T:j is therefore a constant mUltiplied by the coefficients of s o , s l , . . . , s K - I in go(s - l)/(s - G(s» , the constant being chosen in such a way that ��O I 1T:i = 1 . When G(s) = p(1 - qs) - I , then gO = P and go(s - 1) p(1 - qs) = p + ---,--:-:q ----'----:-:---:---:- = s - G(s) l - (qs/p) p - qs The coefficient of siO is 1, and of s i is q i+ 1 / p i if i � 1 . The stationary distribution is therefore given by 1T:j = q1T:O (q/p) for i � 1, where 1 p-q 1T:o = 1 + �f - I q(q/p) 1. P - q + q 2 (1 - (q/p) K - I ) _ -
if p # q, and 1T:O = 2/(K + 1) if p = q = i. 4. The transition matrices
have respective stationary distributions I = (p, 1 - p) and 2 = (�p, � p, � (1 - p), � (1 - p») for any 0 � p � 1. 5. (a) Set i = 1, and find an increasing sequence n l (1), n l (2), . . . along which x I (n) converges. Now set i = 2, and find a subsequence of (n l (j) : j � 1) along which x2 (n) converges; denote this subsequence by n 2 (1), n 2 (2), . . . . Continue inductively to obtain, for each i, a sequence Di = (n i (j) : j � 1), noting that: (i) Di+ I is a subsequence of Dj , and (ii) limr-+oo Xj (n i (r» exists for all i . Finally, define m k = n k (k). For each i � 1, the sequence mj , m i+ ! , . . . is a subsequence of Di , and therefore limr -+oo Xi (m r ) exists. (b) Let S be the state space of the irreducible Markov chain X . There are countably many pairs i, j of states, and part (a) may be applied to show that there exists a sequence (n r : r � 1) and a family (Olij : i, j E S), not all zero, such that pjj (n r ) -+ Ol ij as r -+ 1T:
1T:
00 .
283
[6.4.6]-[6.4.10]
Markov chains
Solutions
Now X is persistent, since otherwise Pij (n) � 0 for all i, j. The coupling argument in the proof of the ergodic theorem (6.4.17) is valid, so that Paj (n) - Pbj (n) � 0 as n � implying that fXaj = fXbj for all a, b, j. 6. Just check that 1r satisfies 1r = 1rP and E = 1. 7. Let Xn be the Markov chain which takes the value r if the walk is at any of the 2 r nodes at level r. Then Xn executes a simple random walk with retaining barrier having P = 1 - q = i, and it is thus transient by Example (6.4.15). 8. Assume that Xn includes particles present just after the entry of the fresh batch Yn . We may write 00,
v TC v
Xn Xn+l = L Bi,n + Yn i=1
where the Bi,n are independent Bernoulli variables with parameter 1 - p. Therefore X is a Markov chain. It follows also that G n+l (s) = lE(s Xn+ l ) = G n ( p + qs)e A (s - I ) . In equilibrium, G n+ ! = G n = G, where G(s) = G(p + qs)eA (s - I ) . There is a unique stationary distribution, and it is easy to see that G (s) = eA (s -1 ) /P must therefore be the solution. The answer is the Poisson distribution with parameter Alp. 9. The Markov chain X has a uniform transition distribution Pjk = l/(j + 2), 0 � k � j + 1 . Therefore, lE(Xn) = lE (lE(Xn I Xn - l ») = � ( 1 + lE(Xn - l ») = . . . = 1 - (�) n + (�) n Xo. The equilibrium probability generating function satisfies
{
}
Xn + 2 G(s) = lE(s xn ) = lE(lE(s Xn I Xn_d ) = lE (1 -1 -s )s(Xn 2) ' +
whence
d ds { (1 - s)G(s) } = -sG(s), subject to G(l ) = 1 . The solution is G(s) = es -1 , which is the probability generating function of the Poisson distribution with parameter 1 . 10. This is the claim of Theorem (6.4.1 3 ). Without loss of generality we may take s = 0 and the Yj to be non-negative (since if the Yj solve the equations, then so do Yj + c for any constant c). Let T be the matrix obtained from P by deleting the row and column labelled 0, and write Tn = (tij (n) : i, j 1= 0). Then Tn includes all the n-step probabilities of paths that never visit zero. We claim first that, for all i, j it is the case that tij (n) � 0 as n � 00. The quantity tij (n) may be thought of as the n-step transition probability from i to j in an altered chain in which s has been made absorbing. Since the original chain is assumed irreducible, all states communicate with s, and therefore all states other than s are transient in the altered chain, implying by the summability of tij (n) (Corollary (6.2.4» that tij (n) � 0 as required. Iterating the inequality y ;:: Ty yields y ;:: Tn y, which is to say that 00
00
} L tij (n), Yi ;:: L tij (n) Yj ;:: min{ s :::: 1 Yr+s j=r+l j= 1 284
i ;:: 1.
Stationary distributions and the limit theorem
Solutions
[6.4.11]-[6.4.12]
Let An = {Xk :;f O for k � n } . For i ;:: 1 , IP'(A oo
{.r; i ( · 11m r
-< n -+ oo
Let
E
00
tij (n) I Xo = i) = nlim L...J -+ oo IP'(An I Xo = i) = �
> 0, and pick R such that
j=1
t J· n ) + mt.
Yi ----,-< mins;::: d Y R +s }
}
Yi ns;::: d Yr+s }
.
E.
Take r = R and let n -+ implying that IP'(A oo I Xo = i) = o. It follows that 0 is persistent, and by irreducibility that all states are persistent. 11. By Exercise (6.4.6), the stationary distribution is 7r:A = 7r:B = 7rD = 7r:E = j; , 7r:C = 1 · (a) By Theorem (6.4.3), the answer is J1-A = 1 /7r:A = 6. (b) By the argument around Lemma (6.4.5), the answer is PD (A) = 7rDJ1-A = 7r:D/7r:A = 1 . (c) Using the same argument, the answer is pc(A) = 7r:C /7r:A = 2. (d)LetlP'i( · ) = IP'( . I Xo = i), let 1j be the time ofthe first passage to state j, andlet vi = lP'i(h < TE). By conditioning on the first step, 00,
. 3 Vc = 21 ' v = 1 ·th so1utlon VA = 85 ' VB = 'I> n 4· A typical conditional transition probability Lij = lP'i(X 1 = j I TA < TE) is calculated as follows:
WI
and similarly, LAC = S2 ' LBA = 32 ' iBC =
1 LCA = 21 ' LCB = "8"3 ' LCD = 81 ' LDC = 1
3'
.
We now compute the conditional expectations Tii = lEi( TA I TA < TE) by conditioning on the first step of the conditioned process. This yields equations of the form TiA = 1 + TiB + Tic, whose solution gives TiA = 154 • (e) Either use the stationary distribution of the conditional transition matrix or condition on the first step as follows. With N the number of visits to D, and TJ i = lEi (N I TA < TE) , we obtain 1 TJ A = S3 TJ B + s2 TJC , TJ B - 0 + 3 TJC , TJD = TJc ,
� �
T,
whence in particular TJA = lo · 12. By Exercise (6.4.6), the stationary distribution has 7r:A = �, 7r:B around Lemma (6.4.5), the answer is PB (A) = 7r:B J1-A = 7r:B /7r:A = 2. 285
t . Using the argument
[6.5.1]-[6.5.6]
Markov chains
Solutions
6.5 Solutions.
Reversibility
1. Look for a solution to the equations 1T:i Pij = 1T:j Pji . The only non-trivial cases of interest are those with j = i + 1, and therefore A i1T:i = tL i +l1T:i+ l for 0 :::; i < b, with solution
0 :::; i :::; b , an empty product being interpreted as 1. The constant 1T:o is chosen in order that the 1T:i sum to 1, and the chain is therefore time-reversible. 2. Let1T: be the stationary distribution of X, and suppose X is reversible. We have that 1T:i Pij = Pji1T:j for all i, j , and furthermore 1T:i > 0 for all i . Hence 1T:iPij PjkPki
=
Pji1T:j PjkPki = Pj i Pkj1T:kPki
=
PjiPkj Pik1T:i
as required when n = 3. A similar calculation is valid when n > 3. Suppose conversely that the given display holds for all finite sequences of states. Sum over all values of the subsequence h, . . . , jn - l to deduce that Pij (n - 1) Pji = Pij Pji (n - 1), where i = iI , j = jn . Take the limit as n -+ to obtain 1T:j Pji = Pij 1T:i as required for time-reversibility. 3. With 1T: the stationary distribution of X, look for a stationary distribution of Y of the form if i fl C, c Vi = { {31T:i if i E C. C1T:i There are four cases. (a) i E C, j fI C : Viqjj = C1T:i{3Pij = c{31T:jPji = Vj qji , (b) i, j E C : Vi qjj = C1T:iPij = C1T:j Pji = Vj qji , (c) i fI c , j E C : Vi qjj = c{31T:i Pij = c{31T:j Pj i = Vj qji , (d) i, j fI C : viqij = c{31T:i Pij = c{31T:j Pji = Vj qji · Hence the modified chain is reversible in equilibrium with stationary distribution when 00
v
v,
In the limit as {3 + 0, the chain Y never leaves the set C once it has arrived in it. 4. Only if the period is 2, because of the detailed balance equations. 5. With Yn = Xn - im,
Now iterate. 6. (a) The distribution 1T:l = f3! (a + {3 ), 1T:2 = a / (a + {3 ) satisfies the detailed balance equations, so this chain is reversible. (b) By symmetry, the stationary distribution is = ( 1, 1, 1), which satisfies the detailed balance equations if and only if P = i . (c) This chain is reversible if and only if P = i . Jr:
286
Chains with finitely many states
Solutions
[6.5.7]-[6.6.2]
7. A simple random walk which moves rightwards with probability P has a stationary measure 7rn = A(p/q) n , in the sense that is a vector satisfying = 1r:P. It is not necessarily the case that this has finite sum. It may then be checked that the recipe given in the solution to Exercise (6.5.3) yields 7r(i, j) = p{p4 ; L: (r,s) ec PI P� as stationary distribution for the given process, where C is the relevant region of the plane, and Pi = Pi /qi and Pi (= 1 - qi) is the chance that the ith walk moves rightwards on any given step. S. Since the chain is irreducible with a finite state space, we have that tri > 0 for all i . Assume the chain is reversible. The balance equations tri Pij = 7rj Pj i give Pij = 7rj Pj i/7ri. Let D be the matrix with entries 1/7ri on the diagonal, and S the matrix (7rj Pji ), and check that P = DS. Conversely, if P = DS, then dj- l Pij = dT I Pj i , whence tri = di- l / L: k d;; I satisfies the detailed balance equations. Note that Pij = � Pij .j1fj. 1r:
1r:
1r:
�
If the chain is reversible in equilibrium, the matrix M = (J7ri/7rj Pij ) is symmetric, and therefore M, and, by the above, P, has real eigenvalues. An example of the failure of the converse is the transition matrix p=
( i1 0� i)0 ,
which has real eigenvalues 1, and - i (twice), and stationary distribution = ( � , � , �). However, 7r I P1 3 = 0 i= � = 7r3 P3 l > so that such a chain is not reversible. 9. Simply check the detailed balance equations tri Pij = 7rj Pj i. 6.6 Solutions.
1r:
Chains with finitely many states
Let P = (Pij I � i, j � n) be a stochastic matrix and let C be the subset of JRn containing all vectors x = (Xl , x2 , . . . , xn) satisfying Xi ;::: 0 for nall i and L:i l Xi = 1; for x E C, let I l x ll = maxj {Xj }. Define the linear mapping T C --+ JR by T (x) = xP. Let us check that T is a continuous function from C into C. First, II T (x) 11 = jX { 2; Xi Pij � a ll x ll 1.
:
=
:
I
where
}
hence II T(x) - T (y) 11 � a ll x - Y II . Secondly, T(x)j ;::: 0 for all j, and L T (x)j = L L Xi Pij = L Xi L Pij = 1. i j j j Applying the given theorem, there exists a point 7r in C such that T (7r ) = 7r , which is to say that
7r = nP. 2. Let P be a stochastic m m matrix and let T be the m (m + I) matrix with (i, j)th entry tI.J. -- PiJ· - 8iJ· if j � m, I if j = m + 1, x
x
{
287
[6.6.3]-[6.6.4]
Markov cMins
Solutions
where 8ij is the Kronecker delta. Let v = (0, 0, . . . , 0, 1) E jRm + 1 . If statement (ii) of the question is valid, there exists y = (YI , Y2 , . . . , Ym + I ) such that Ym + I < 0,
m � (pjj - 8ij ) Yj + Ym + I � ° for 1 � i � m ; j=I
this implies that
m � Pij Yj � Yi - Ym + I > Yi for all i, j=I and hence the impossibility that L:j=I Pij Yj > maxi {yj }. It follows that statement (i) holds, which is to say that there exists a non-negative vector x = (X l , x2 , . . . , xm ) such that x(P - I) = 0 and L:i=I Xi = 1 ; such an x is the required eigenvector. 3. Thinking of xn + 1 as the amount you may be sure of winning, you seek a betting scheme x such that xn +1 is maximized subject to the inequalities n xn + I � � xi tij for 1 � j � m . i=I Writing aij = -tij for 1 � i � n and an + I,j = 1, we obtain the linear program: n+ I maximize xn + I subject to � Xi aij � ° for 1 � j � m . i=I
The dual linear program is: minimize
°
m
subject to � aij Yj = ° for 1 � i � n, j=I m � an +1 ,jYj = 1 , Yj � ° j=I
for 1 � j � m .
Re-expressing the aij in terms of the tij as above, the dual program takes the form: minimize
°
m
subject to � tij Pj = ° for 1 � i � n, j=I m � Pj = 1 , Pj � ° j=I
for 1 � j � m .
The vector x = 0 is a feasible solution of the primal program. The dual program has a feasible solution if and only if statement (a) holds. Therefore, if (a) holds, the dual program has minimal value 0, whence by the duality theorem of linear programming, the maximal value of the primal program is 0, in contradiction of statement (b). On the other hand, if (a) does not hold, the dual has no feasible solution, and therefore the primal program has no optimal solution. That is, the objective function of the primal is unbounded, and therefore (b) holds. [This was proved by De Finetti in 1937.] 4. Use induction, the claim being evidently true when n = 1. Suppose it is true for n = m. Certainly pm + I is of the correct form, and the equation pm+l x' = p(pm x') with x = (1 , W , w2 ) yields in its first row 288
Branching processes revisited
[6.6.5]-[6.7.2]
Solutions
as required. 5. The first part follows from the fact that K i f = 1 if and only if KU = 1 . The second part follows from the fact that 1{j > 0 for all i if P is finite and irreducible, since this implies the invertibility of
J - P + U.
6. The chessboard corresponds to a graph with 8 8 = 64 vertices, pairs of which are connected by edges when the corresponding move is legitimate for the piece in question. By Exercises (6.4.6), (6.5.9), we need only check that the graph is connected, and to calculate the degree of a comer vertex. (a) For the king there are 4 vertices of degree 3, 24 of degree 5, 36 of degree 8. Hence, the number of edges is 210 and the degree of a comer is 3. Therefore JL(king) = 420/ 3 = 140. (b) JL(queen) (28 21 + 20 23 + 12 25 + 4 27)/21 = 208 /3. (c) We restrict ourselves to the set of 32 vertices accessible from a given comer. Then JL(bishop) (1 4 7 + 1 0 9 + 6 1 1 + 2 13)/7 = 40. (d) JL(knight) = (4 2 + 8 3 + 20 4 + 16 6 + 16 8)/2 = 168. (e) JL(rook) = 64 14/14 64. 7. They are walking on a product space of 8 16 vertices. Of these, 6 16 have degree 6 3 and 16 2 have degree 6 5. Hence JL( C ) = (6 16 6 3 + 16 2 6 5)/18 = 448/3. x
x
=
x
x
x
=
x
x
x
x
x
x
x
x
x
IP
x
x
x
x
8.
x
=
x
x
x
x
x
x
x
( 11 11
- Al l = (A - l)(A + �)(A + j;). Tedious computation yields the eigenvectors, and thus
pn = 1
1 1
6.7 Solutions.
Branching processes revisited
1. We have (using Theorem (5.4.3), or the fact that G n + 1 (s) = G(Gn (s))) that the probability generating function of Zn is l)s , Gn(s) = nn-+(n1 -- ns so that ( n ) k+1 - ( n - 1 ) ( n ) k- l = nk- 1k+l P(Zn = k) = n + 1 n+1 n+1 (n + l) for k ::: 1. Therefore, for y > 0, as n --+ 00 ,
2y n J n k - 1 1 L"'" ( 1 ) - L2 y n J = 1 1 + P(Zn 0) - 1 Gn(O) n tI (n + l)k+ l -
2.
--+
1 - e - 2y .
Using the independence of different lines of descent, 1 (S 1/ ) , . . = L...J � s k p(Zn = k)r / = -Gn � s k p(Zn = k,. extinction) L...J = E(s zn I extinction) . (extinction) 1/ 1/ P k=O k=O where G n is the probability generating function of Zn. 289
[6.7.3]-[6.8.1] 3.
Markov chains
Solutions
We have that TJ = G(TJ). In this case G(s ) = q ( 1 - ps ) - 1 , and therefore TJ = q / p. Hence
which is Gn (s) with p and q interchanged. 4. (a) Using the fact that var(X I X > 0) � 0, E(X 2 ) = E(X 2 I X > O)IP'(X > 0) � E(X I X > 0) 2 1P'(X > 0) = E(X)E(X I X > 0).
(b) Hence
E(Zn/ /-Ln I Zn > 0) � /-LE(Z�) n E(Zn)
E(Wn2 ) where Wn = Zn /E(Zn ). By an easy calculation (see Lemma (5.4.2)), =
where a 2 = var(Z I ) = p/q 2 . (c) Doing the calculation exactly, n) 1 1 E ( Zn//-Ln I Zn > 0) - E(Zn//-L -+ 1 1 - TJ Gn(O) IP'(Zn > 0) where TJ = lP'(ultimate extinction) = q/p.
_
_
6.8 Solutions.
Birth processes and the Poisson process
1. Let F and W bethe incomingPoissonprocesses, and let N(t) = F(t)+ W(t). Certainly N(O) = 0 and N is non-decreasing. Arrivals of flies during [0, s] are independent of arrivals during (s, t], if s < t; similarly for wasps. Therefore the aggregated arrival process during [0, s] is independent of the aggregated process during (s, t]. Now IP' ( N(t + h) = n + 1 1 N(t) = n) = IP'(A f:. B) where A = { one fly arrives during (t, t + hJ ) , B = { one wasp arrives during (t, t + hJ ) . We have that IP'(A f:. B) = IP'(A) + IP'(B) - IP'(A n B) = Ah + /-Lh - (Ah)(/-Lh) + o(h) = (A + /-L)h + o(h). Finally IP' ( N(t + h) > n + 1 1 N(t) = n) � IP'(A n B) + IP'(C U D) ,
290
Birth processes and the Poisson process
Solutions , [6.8.2]-[6.8.4]
where C = {two or more flies arrive in (t, t + h]) and D = {two or more wasps arrive in (t, t + h]). This probability is no greater than (Ah)(f.Lh) + o(h) = o (h) . 2. Let 1 be the incoming Poisson process, and let G be the process of arrivals of green insects. Matters of independence are dealt with as above. Finally, lP' ( G(t + h) = n + 1 1 G(t) = n ) = plP' ( I (t + h) = n + 1 1 l (t) = n ) + o(h) = pAh + o(h), lP' ( G(t + h) > n + 1 1 G(t) = n ) � lP' ( I (t + h) > n + 1 1 I (t) = n ) = o (h) . 3.
Conditioning on Tl and using the time-homogeneity of the process, if u � t, if t < u � t + x, if u > t + x, (draw a diagram to help you see this). Therefore lP' ( E(t) > x ) = =
1000 lP' (E(t) > x I Tl u ) Ae -AU du Jto lP' ( E(t - u) > x ) Ae -AU du + 1t +00x Ae -AU duo =
You may solve the integral equation using Laplace transforms. Alternately you may guess the answer and then check that it works. The answer is lP'(E(t) � x) = 1 - e -AX , the exponential distribution. Actually this answer is obvious since E(t) > x if and only if there is no arrival in [t, t + x], an event having probability e -Ax . 4. The forward equation is with boundary conditions Pij (O) = 8ij ' the Kronecker delta. We write G i (S, t) = Lj s j Pij (t), the probability generating function of B(t) conditional on B(O) = i . Multiply through the differential equation by s j and sum over j: a partial differential equation with boundary condition G i (s, 0) = s i . This may be solved in the usual 1 At way to obtain G i (s,l t) =i g (e (l - s )) for somei function g. Using the boundary condition, we find that g(1 - s - ) = s and so g(u) = (1 - u) - , yielding The coefficient of s j is, by the binomial series, j ?:. i, as required. 29 1
[6.8.5]-[6.8.6]
Markov chains
Solutions
Alternatively use induction. Set j = i to obtain Pli (t) = -A i PH (t) (remember Pi,i - l (t) = 0), and therefore PH (t) = e - A i t . Rewrite the differential equation as d (t)eAJ· t ) = A(j - l) Pi,j - l (t)eAJ· t dt (Pij Set j = i + 1 and solve to obtain Pi,i+ 1 (t) = i e - A i t ( 1 - e - At ) . Hence (*) holds, by induction. The mean is a = IeAt , E(B(t)) = -G[ a s (s, t) s =1 by an easy calculation. Similarly var(B(t)) = A + E(B(t)) - E(B(t)) 2 where •
I
Alternatively, note that B(t) has the negative binomial distribution with parameters e - At and I . 5. The forward equations are n ?: 0, p� (t) = An - 1 Pn - l (t) - An Pn(t), where A i = iA + v . The process is honest, and therefore m(t) = En npn (t) satisfies 00
00
m'(t) = 2: n [(n - l)A + v 1 Pn - l (t) - 2: n(nA + V) Pn (t) n =1 n =O 00
=
2: { A[(n + l)n - n 2 ] + v [(n + 1) - n1 } Pn(t)
n =O 00
=
2: (An + V) Pn (t) = Am(t) + v.
n =O
Solve subject to m(O) = 0 to obtain m(t) = v (eA t - l)/A. 6. Using the fact that the time to the nth arrival is the sum of exponential interarrival times (or using equation (6.8. 15)), we have that is given by
_ � IIn
Pn (() ) - An � i =O A I. + () which may be expressed, using partial fractions, as �
where
n
A:l . aI· -- II j =o A:l· - A I. j #. i 292
Continuous-time Markov chains
Solutions
[6.8.7]-[6.9.1]
so long as Ai #- Aj whenever i #- j. The Laplace transform Pn may now be inverted as
See also Exercise (4.8.4). 7 . Let Tn be the time of the nth arrival, and let T = limn--* oo Tn = sup{t : N (t) < oo}. Now, as in Exercise (6.8.6), n AnPn (e ) = II A A+I' -e = E(e -B Tn ) i=O i since Tn = Xo + X I + . . . + Xn where Xk is the (k + l)th interarrival time, a random variable which is exponentially distributed with parameter Ak . Using the continuity theorem, E(e -B Tn ) -+ E(e - B T ) as n -+ 00, whence An Pn(e ) -+ E(e - B T ) as n -+ 00 , which may be inverted to obtain An Pn (t) -+ f(t) as n -+ 00 where f is the density function of T. Now __
which converges or diverges according to whether or not En npn (t) converges. However Pn (t) A;:;- 1 f(t) as n -+ 00, so that En npn (t) < 00 if and only if En nA;:;- 1 < 00 . When An = (n + �) 2 , we have that
{
�
}
00 -1 = sech (rrv'e) . E(e -B T ) = II 1 + e 1 2 (n + 2 ) n =O Inverting the Laplace transform (or consulting a table of such transforms) we find that
where e l is the first Jacobi theta function. 6.9 Solutions. Continuous-time Markov chains 1.
(a) We have that
where P 1 2 = 1 - P 11 , P2 1 = 1 - P22 . Solve these subject to Pij (t) = 8ij , the Kronecker delta, to obtain that the matrix Pt = (Pij (t)) is given by -(J... +f.J-)t /-L - /-Le - (J... +f.J-)t Pt = A +1 /-L AA +- /-Le Ae -(A +f.J-)t /-L + Ae -(A +f.J-)t .
)
(
(b) There are many ways of calculating Gn ; let us use generating functions . Note first that GO the identity matrix. Write n � O, 293
=
I,
[6.9.2]-[6.9.4]
Markov chains
Solutions
and use the equation Gn+1 = G . Gn to find that
Hence an+1 = - (I1P,, ) C + l for n � 0, and the first difference equation becomes an+l = -(A+tL)an, n � 1, which, subject to a l = -tL, has solution an = ( _ l) n tL(A + tL) n - l , n � 1. Therefore for Cn = ( _ l ) n +1 A(A + tL) n - l for n � 1, and one may see similarly that bn = -an, ndn = -Cn n � 1. Using the facts that ao = do = 1 and bo = Co = 0, we deduce that I:�o (t In ! ) Gn = Pt where Pt is given in part (a). (c) With 1r = (Jr l , Jr2 ), we have that - tLJr l + AJr2 = 0 and tLJr l - AJr2 = 0, whence Jr l = (AI tL)Jr2 . In addition, Jr l + Jr2 = 1 if Jr l = A/(A + tL) = 1 - Jr2 . 2. (a) The required probability is n
IP' ( X(t) = 2, X(3t) = 1 I X(O) = 1) IP' ( X(3t) = 1 I X(O) = 1)
p dt) P21 (2t) Pl 1 (3t)
using the Markov property and the homogeneity of the process. (b) Likewise, the required probability is P 12 (t) P21 (2t) Pl 1 (t) Pl 1 (3t) Pl 1 (t) the same as in part (a). 3. The interarrival times and runtimes are independent and exponentially distributed. It is the lack of-memory property which guarantees that X has the Markov property. The state space is S = {O , 1 , 2, . . . } and the generator is
(
0 -A A 0 0 ... A tL -(A + tL) G= 0 -(A + tL) A tL · · ·
" "
. . .
Solutions of the equation 1rG = 0 satisfy
)
-Ano + tLJr l = 0, AJrj - l - (A + tL)Jrj + tLJrj+1 = 0 for j � 1 , with solution Jri = Jro(AltL) i . We have in addition that I: i Jri = 1 in < tL and Jro = {1 - (AltL)} - l . 4. One may use the strong Markov property. Alternatively, by the Markov property,
IP'(Yn+l = j I Yn = i, Tn = t, B) = IP'(Yn+ 1 = j I Yn = i, Tn = t) for any event B defined in terms of {X(s ) : s � Tn }. Hence
1000
IP'(Yn+l = j I Yn = i, B) = IP'(Yn+ 1 = j I Yn = i, Tn = t) !Tn (t) dt = IP'(Yn+ 1 = j I Yn = i),
so that Y is a Markov chain. Now qij = IP'(Yn+l = j I Yn = i) is given by
294
Continuous-time Markov chains
Solutions
[6.9.5]-[6.9.8]
by conditioning on the (n + l) th interarrival time of N; here, as usual, Pij (t) is a transition probability of X. Now
5. The jump chain Z = {Zn : n � O} has transition probabilities h ij = gij / gi , i i= j . The chance that Z ever reaches A from j is also TJj ' and TJj = Ek hjk TJk for j ¢. A, by Exercise (6.3.6). Hence -gj TJj = Ek gjk TJb as required. 6. Let Tl = inf{t : X(t) i= X (O) }, and more generally let Tm be the time of the mth change in value of X. For j ¢. A, /-Lj = Ej (Tl ) + L hj k /-Lb kt.j where Ej denotes expectation conditional on Xo = j . Now Ej (Tl ) = (; 1 , and the given equations follow. Suppose next that (ak : k E S) is another non-negative solution of these equations. With Ui = Ti +1 - Ii and R = min {n � I Zn E A}, we have for j ¢. A that :
where 1; is a sum of non-negative terms. It follows that aj � Ej (Uo) + Ej (UI I{ R >l} ) + . . . + Ej (Un I{ R >n} ) =
Ej
(r=Ot Ur I{R>r} )
=
Ej (min{Tn , HA l) -+ Ej (HA )
as n -+ 00, by monotone convergence. Therefore, JL is the minimal non-negative solution. First note that i is persistent if and only if it is also a persistent state in the jump chain Z. The integrand being positive, we can write
7.
where {Tn : n � I} are the times of the jumps of X. The right side equals
00
00
I L hii (n) L E(TI I X(O) = i)hii (n) = -: g l n=O n=O where H = (h ij ) is the transition matrix of Z. The sum diverges if and only if i is persistent for Z. 8. Since the imbedded jump walk is persistent, so is X. The probability of visiting m during an excursion is a = (2m) - 1 , since such a visit requires an initial step to the right, followed by a visit to m before 0, cf. Example (3.9.6). Having arrived at m, the chance of returning to m before visiting 0 is I - a, by the same argument with 0 and m interchanged. In this way one sees that the number N of visits to m during an excursion from 0 has distribution given by lP'(N � k) = a(1 - a) k - l , k � l . The 'total jump rate ' from any state is A, whence T may be expressed as E�o Vi where the Vi are exponential with parameter A. Therefore,
295
[6.9.9]-[6.9.11]
Markov chains
Solutions
The distribution of T is a mixture of an atom at 0 and the exponential distribution with parameter IXA. k 9. The number N of sojourns in i has a geometric distribution lP'(N = k) = f - l (1 - f), k � 1, for some f < 1 . The length of each of these sojourns has the exponential distribution with some parameter gi . By the independence of these lengths, the total time T in state i has moment generating function k (1 E(e e T ) = � f k - I (1 - f) � = gi - f) . gi (1 - f) - () gi - () f=r. The distribution of T is exponential with parameter gi (1 - f). 10. The jump chain is the simple random walk with probabilities A/(A + JL) and JLI(A + JL), and with PO I = 1 . By Corollary (5.3.6), the chance of ever hitting 0 having started at 1 is JLIA, whence the probability of returning to 0 having started there is f = JLIA. By the result of Exercise (6.9.9),
(
)
as required. Having started at 0, the walk visits the state r � 1 with probability 1 . The probability of returning to r having started there is and each sojourn is exponentially distributed with parameter gr = A + JL. Now gr (I - fr ) = A - JL, whence, as above, E(e e fT ) = A - JL A - JL - () The probability of ever reaching 0 from X(O) is (JLIA) X (O) , and the time spent there subsequently is exponential with parameter A - JL. Therefore, the mean total time spent at 0 is rr
11.
-----'
(a) The imbedded chain has transition probabilities
where gk = -gkk. Therefore, for any state j ,
where we have used the fact thatKG = O . Also nk � O and Ek nk = 1, and therefore n is a stationary distribution of Y. Clearly nk = 1{k for all k if and only if gk = E i 1{i gi for all k, which is to say that gi = gk for all pairs i, k. This requires that the 'holding times ' have the same distribution. (b) Let Tn be the time of the nth change of value of X, with To = 0, and let Un = Tn + I - Tn. Fix a state k, and let H = min{n � 1 Zn = k}. Let Yi (k) be the mean time spent in state i between two consecutive visits to k, and let )li (k) be the mean number of visits to i by the jump chain in between two :
296
Birth-death processes and imbedding
Solutions
visits to k (so that, in particular, Yk (k) = gk" l and n (k) and probability conditional on X (0) = j, we have that Yi (k) = Ek =
00
=
1). With Ej and IP'j denoting expectation
(nf=O Un I{Zn=i, H>nj ) nf=O Ek(Un I I{Zn=ij )lP'k(Zn =
1 1 = i , H > n ) = - Yi (k). L.:: -lP' gi k(Zn gi
n =O The vector y(k) = (Yj (k) : i E S) satisfies y(k)H transition matrix of the jump chain Z. That is to say,
=
[6.9.12]-[6.11.2]
=
i, H > n)
y(k), by Lemma (6.4.5), where H is the for j E S ,
whence � i n (k)gij = o for all j E S. If ILk = � i n (k) < 00, the vector (n (k)/ ILk ) is a stationary distribution for X, whence nj = Yi (k)/ lLk for all i . Setting i = k we deduce that nk = 1/(gkILk ). Finally, if � j ni gj < 00, then 1 = � i nigi = " n ILk L.J igi . ILk = -;;: nk :nkgk i �
12. Define the generator G by gii = -Vi , gij = Vjhij , so that the imbedded chain has transition matrix H. A root of the equation 1I'G = 0 satisfies
0 = L.:: nigij = - nj vj + L.:: (nj Vj)hij i i : i #.j whence the vector t = (nj vj : j E S) satisfies t = tH. Therefore t = av, which is to say that nj vj = aVj , for some constant a. Now Vj > o for all j, so that nj = a, which implies that �j nj #- 1 . Therefore the continuous-time chain X with generator G has no stationary distribution. 6.11 Solutions. Birth-death processes and imbedding
1. The jump chain is a walk {Zn } on the set S = {O, 1 , 2, . . . } satisfying, for i ::: 1 , if j = i + 1 , IP'(Zn +1 = j I Zn = i) = Pi 1 - Pi if j = i - I , where Pi = A i!(').i + ILj ) · Also IP'(Zn +1 = 1 I Zn = 0) = 1 . 2. The transition matrix H = (hij ) of Z is given by ilL 1' f J' = 1. - 1 , A + iIL h IJ A --.- if j = i + 1 . A + I IL To find the stationary distribution of Y, either solve the equation = 1I' Q, or look for a solution of the detailed balance equations njh i,i+1 = ni +l hi +l, i . Following the latter route, we have that nj = no h Ol h 1 2 ' " hi - l ,i , i ::: 1 , hi,i - l " · h 2 l h lO
{
{
.. - -_
11'
297
[6.11.3]-[6.1 1.4]
Markov chains
Solutions
whence ni = nop i (l + i / p)/ i ! for i ::: 1. Choosing no accordingly, we obtain the result. It is a standard calculation that X has stationary distribution given by Vi = p i e - P / i ! for i ::: O. The difference between and arises from the fact that the holding-times of X have distributions which depend on the current state . 3. We have, by conditioning on X(h), that n
v
v
TJ(t + h) = E{IP'(X(t + h) = 0 I X(h») } = /-Lh . 1 + (1 - Ah - /-Lh)TJ(t) + Ah�(t) + o(h)
where �(t) = IP'(X (t) = 0 I X (0) = 2). The process X may be thought of as a collection of particles each of which dies at rate /-L and divides at rate A, different particles enjoying a certain independence; this is a consequence of the linearity of An and /-Ln. Hence �(t) = TJ(t) 2 , since each of the initial pair is required to have no descendants at time t. Therefore subject to TJ (0) = O. Rewrite the equation as
TJ ' --'-(1 --TJ)(/-L- ATJ) and solve using partial fractions to obtain
-
-
=
1 if A = /-L,
Finally, if 0 < t < u,
IP'(X(t) = 0 I X(u) = 0) 4.
=
IP'(X(t) = 0) IP'(X(u) = 0 I X(t) = 0) IP'(X(u) = 0)
=
The random variable X (t) has generating function
- s) - (/-L - As)e - t(J.. - /L) G(s, t) = -/-L(l A-(l---S)---(/-L-- A-s)-e--....,t(-:--J.. --/L"""'") "" as usual. The generating function of X(t), conditional on { X(t)
� n IP'(X(t) = n) IP'(X(t) > 0)
�s
=
>
G(s, t) - G(O, t) . 1 - G(O, t)
Substitute for G and take the limit as t -+ 00 to obtain as limit
H(s) = (/-L - A)S /-L - AS
=
00
2: s n Pn n=l
where, with p = A//-L, we have that Pn = p n - l (l - p) for n ::: 1 . 298
O}, is therefore
TJ(t) TJ(u)
- .
Special processes 5.
Solutions
[6.11.5]-[6.12.1]
Extinction is certain if A < /-L, and in this case, by Theorem (6.1 1 .10),
E( T ) = =
1000 IP'(T > t) dt 1000 { 1 - E(s X(t) ) l s =O } dt (J.. - /L)t 1 ( /-L ) rJooo (/-L/-L -- A)e Ae (J.. - /L)t dt i log /-L - A . =
=
If A > /-L then IP'(T < (0) = /-L/A, so
roo { I
_��:
(
)
(A - /-L )� (/L t dt = .!.. log _A_ . � E(s X(t) ) 1 - dt = o s /-L A - /-L /-L Jo A - /-Le /L In the case A = /-L, IP'(T < (0) = 1 and E(T) = 6. By considering the imbedded random walk, we find that the probability of ever returning to 1 is max{A , /-L}/(A + /-L), so that the number of visits is geometric with parameter min {A , /-L}/(A + /-L). Each visit has an exponentially distributed duration with parameter A + /-L, and a short calculation using moment generating functions shows that VI is exponential with parameter min {A , /-L}. Next, by a change of variables, Theorem (6. 1 1 .10), and some calculation,
E(T I T < (0) = Jo
r oo
}
_
00. (00)
� s rE(Vr (t)) E (� l s r I{X(u) =r) dU) E (l s X(u) dU) lot E(s X(U) du /-Lt - -1 log { A (1 - s) - (/-L - As)e - (J.. -/L)t } =
=
)
-
A A A - /-L p t 1 As (e - 1) terms not involving s, = - - log 1 A /-LePt - A + where P = /-L - A. We take the limit as t -+ and we pick out the coefficient of s r . 7. If A = /-L then, by Theorem (6.1 1 .10), M(1 - s) + s = 1 - --,--l --.,s -----:E(s X(t) ) = M(1 - s) + 1 At(1 - s) + 1 and =
o
=
{
}
00
r E(s X(u » ) du = t - ! 1 g{A t(1 - s) + I } A 1 A tS · s. . Ivmg = - - Iog { I - -- } + terms not mvo 1 + At A and picking out the coefficient of s r gives E( Vr ( )) = (rA) - I . An alternative
Jo
0
Letting t -+ method utilizes the imbedded simple random walk and the exponentiality of the sojourn times.
00
oo
6.12 Solutions.
Special processes
1. The jump chain is simple random walk with step probabilities A / (A + /-L) and /-L / (A + /-L). The expected time /-L 1O to pass from 1 to 0 satisfies
299
[6.12.2]-[6.12.4]
Markov chains
Solutions
whence J-L 1 O = (J-L + A)/(J-L - A). Since each sojourn is exponentially distributed with parameter J-L + A, the result follows by an easy calculation. See also Theorem (1 1 .3.17). 2. We apply the method of Theorem (6. 12. 1 1) with
the probability generating function of the population size at time u in a simple birth process. In the absence of disasters, the probability generating function of the ensuing population size at time is v
The individuals alive at time t arose subsequent to the most recent disaster at time t - D, where D has density function 8e-8x , x > 0. Therefore,
The mean number of descendants after time t of a single progenitor at time ° is e(Je-f,J-)t. The expected number due to the arrival of a single individual at a uniformly distributed time in the interval on [0, x] is therefore e(Je-f,J-)x - 1 -1 e (Je- f,J-) u du = (A - J-L) x x 0 The aggregate effect at time x of N earlier arrivals is the same, by Theorem (6. 12.7), as that of N arrivals at independent times which are uniformly distributed on [0, x]. Since E(N) = v x , the mean population size at time x is v [e(Je-f,J-) x - 11/(A - J-L). The most recent disaster occurred at time t - D, where D has density function 8e - h , x > 0, and it follows that 3.
loX
.
This is bounded as t --+ 00 if and only if 8 > A - J-L. 4. Let N be the number of clients who arrive during the interval [0 , t]. Conditional on the event { N = n}, the arrival times have, by Theorem (6. 12.7), the same joint distribution as n independent variables chosen uniformly from [0, t]. The probability that an arrival at a uniform time in [0, t] is still in service at time t is f3 = fri [ 1 - G(t - x ) ] t - 1 dx, whence, conditional on { N = n}, the total number M still in service is bin(n, (3). Therefore,
whence M has the Poisson distribution with parameter Af3t parameter approaches AE(S) as t --+ 00 .
300
=
A fri[1 - G(x)] dx. Note that this
Solutions
Spatial Poisson processes
[6.13.1]-[6.13.3]
6.13 Solutions. Spatial Poisson processes 1. It is easy to check from the axioms that the combined process N(t) = B(t) + G(t) is a Poisson process with intensity f3 + y. (a) The time S (respectively, T ) until the arrival of the first brown (respectively, grizzly) bear is exponentially distributed with parameter f3 (respectively, y), and these times independent. Now,
r oo f3e -fJs e - Ys ds 10
lP'(S < T) =
=
_f3_ . f3 + y
are
(b) Using (a), and the lack-of-memory of the process, the required probability is y n f3 f3 + y f3 + y ' (c) Using Theorem (6.12.7),
.
E (mm{
)
(
S, T} I B(1) = 1)
=
�
{G
1
} (I) + 2
Let Br be the ball with centre 0 and radius r , and let Nr (6.13. 1 1) that Sr = L: xennBr g (x) satisfies
2.
-
=
y 1 + e-Y . y2
=
l IT n Br l. We have by Theorem
),, (u) Sr I Nr ) = Nr ( g (u) A(B du, r) lBr where A(B) = IyeB ),, (y) dy. Therefore, E (Sr ) = IBr g (u) ),, (u) du, implying by monotone conver gence that E ( S) = I g (u) ),, (u) du. Similarly, E(
d R
E(
whence
S; I Nr ) = E
(l (
=
E
=
Nr
]
L g (X ) 2 ennBr
L g (x)2) + E
xennBr
h
Br
E(
(
L g (x ) g (y )
- �'iu
x iy x,ye nnBr
),, (u) -- du + Nr (Nr 2 (u) g (B )
S; ) =
By monotone convergence,
are
)
)
A r
1)
)
),, (u)),, (v) v du dv, (u) ( ) u,veBr g g A(Br ) 2
�r g (u)2 ),, (u) du + (�r g (u)),, (u) dU) 2
and the formula for the variance follows. 3. If B l , B2 , . . . , Bn are disjoint regions of the disc, then the numbers of projected points therein Poisson-distributed and independent, since they originate from disjoint regions of the sphere. By 301
[6.13.4]-[6.13.8]
Markov chains
Solutions
elementary coordinate geometry, the intensity function in plane polar coordinates is 2'A/ �, o ::::: r ::::: 1, 0 ::::: 0 < 2n . 4. The same argument is valid with resulting intensity function 2A � . 5. The Mercator projection represents the spherical coordinates (0 , ¢ ) as Cartesian coordinates in the range 0 ::::: ¢ < 2n , 0 ::::: 0 ::::: n . (Recall that 0 is the angle made with the axis through the north pole.) Therefore a uniform intensity on the globe corresponds to an intensity .function 'A sin 0 on the map. Likewise, a uniform intensity on the map corresponds to an intensity 'AI sin 0 on the globe. 6. Let the X r have characteristic function ¢ . Conditional on the value of N(t), the corresponding arrival times have the same distribution as N(t) independent variables with the uniform distribution, whence
lE(e i8S (t » ) = lE{lE(e i8S (t) I N(t))} = lE{lE(e i 8 x e -a u ) N (t) } t i -U = exp { A t (lE(e 8 Xe a ) - I ) } = exp 'A {¢ (Oe -a u )
{ fo
- } dU } 1
,
where U is uniformly distributed on [0, t]. By differentiation,
'A lE(S(t)) = - i ¢� (t /O) = -lE(X)(1 - e -a t ), a 'A lE(X2 lE(S(t) 2 ) = ¢� (t) (O) = lE(S(t)) 2 + 2a ) (1 e - 2a t ) .
-- -
-
Now, for s < t, Set) = S(s)e -a (t-s) + Set - s) where Set - s) is independent of S(s) with the same distribution as Set - s). Hence, for s < t,
cov(S(s), Set)) =
�(t-S)
=
'AlE(X2 ) 'AlE(X 2 ) e -a v -2as )e -a (t-s) --+ � e (1 �
as s --+ 00 with v = t - s fixed. Therefore, p(S(s), S(s + v)) --+ e -a v as s --+ 00. 7. The first two arrival times Tl , T2 satisfy
Differentiate with respect to x and y to obtain the joint density function 'A(x)'A(x + y)e - A (x +y ) , x , y � O. Since this does not generally factorize as the product of a function of x and a function of y, Tl and T2 are dependent in general. 8. Let X i be the time of the first arrival in the process Ni . Then
302
Markov chain Monte Carlo
Solutions
[6.14.1]-[6.14.4]
6.14 Solutions. Markov chain Monte Carlo 1.
If P is reversible then
RHS = � I
(:�;:>ij Xj ) Yi:7Ti = � :7TiPij XjYi = � :7Tj PjiYi Xj = � :7Tj Xj (� PjiYi ) = LHS. )
I,}
I,}
}
I
Suppose conversely that (x, Py) = (Px, y) for all x, y E 1 2 (:7T). Choose x, y to be unit vectors with 1 in the i th and j th place respectively, to obtain the detailed balance equations :7Ti Pij = :7Tj Pji. 2. Just check that 0 :::: bij :::: 1 and that the Pij = g ij bij satisfy the detailed balance equations (6.14.3). 3. It is immediate that Pjk = IAj k l, the Lebesgue measure of Ajk . This is a method for simulating a Markov chain with a given transition matrix. 4. (a) Note first from equation (4.12.7) that d(U) = � sUP i;6j dTV(Ui. , Uj.), where Ui. is the mass function Ui t , t E T . The required inequality may be hacked out, but instead we will use the maximal coupling of Exercises (4.12.4, 5); see also Problem (7. 1 1.16). Thus requires a little notation. For i, j E S, i =1= j, we find a pair (Xi , Xj) of random variables taking values in T according to the marginal mass functions Ui . , Uj . , and such that IP'(Xi =1= Xj) = ! dTV(Ui. , Uj.). The existence of such a pair was proved in Exercise (4.12.5). Note that the value of Xi depends on j, but this fact has been suppressed from the notation for ease of reading. Having found (Xi , Xj ), we find a pair (Y(Xi ) ' Y(Xj )) taking values in U according to the marginal mass functions VXi" vXj" and such that IP'(Y( Xi ) =1= y(Xj) I Xi , Xj ) = ! dTV( VXi " vXF )' Now, taking a further liberty with the notation,
�
IP' ( Y(Xi) =1= Y(Xj)' = L IP'(Xi = r, Xj = s)IP' ( Y(r) =1= Yes) ) ,SES #s = L IP'(Xi = r, Xj = s) ! dTV( Vr. , Vs·) r,sES r is :::: g SUp dTV (Vr. , Vs . ) } IP'(Xi =1= Xj), r is whence
d(UV) = suP IP' ( Y (Xi ) =1= Y(Xj ) ) :::: g sup dTV( Vr . , vs.) } { suP IP' (Xi =1= Xj) } i ;6j r #S i,j and the claim follows. (b) Write S = { I , 2, . . . , m }, and take
(
= 1) IP'(Xo = 2) U = IP'(Xo IP'(Yo = 1) IP'(Yo = 2)
)
IP'(Xo = m) . IP'(Yo = m)
The claim follows by repeated application of the result of part (a). It may be shown with the aid of a little matrix theory that the second largest eigenvalue of a finite stochastic matrix P is no larger in modulus that d(P); cf. the equation prior to Theorem (6.14.9).
303
[6.15.1]-[6.15.6]
Markov chains
Solutions
6.15 Solutions to problems 1. (a) The state 4 is absorbing. The state 3 communicates with 4, and is therefore transient. The set { I , 2} is finite, closed, and aperiodic, and hence ergodic. We have that !34 (n) = ( i ) n - 1 t, so that 134 = En !34 (n) = � . (b) The chain is irreducible with period 2. All states are non-null persistent. Solve the equation = P to find the stationary distribution = (�, {6 ' 156 ' k) whence the mean recurrence times are 16 ' 8 , m' order. 8 16 ' 5 "3 ' 3 2. (a) Let P be the transition matrix, assumed to be doubly stochastic. Then
1r
1r
1r
(
)
L: Pij (n) = L: L: Pik (n - I) Pkj = L: L: Pik (n - I) Pkj
i i k k i whence, by induction, the n-step transition matrix pn is doubly stochastic for all n � 1. If j is not non-null persistent, then Pij (n) � 0 as n � 00, for all i , implying that E i Pij (n) � 0, a contradiction. Therefore all states are non-null persistent. If in addition the chain is irreducible and aperiodic then Pij (n) � 1fj ' where is the unique stationary distribution. However, it is easy to check that = (N - 1 , N - 1 , . . . , N - 1 ) is a stationary distribution if P is doubly stochastic. (b) Suppose the chain is persistent. In this case there exists a positive root of the equation x = xP , this root being unique up to a multiplicative constant (see Theorem (6.4.6) and the forthcoming Problem (7)). Since the transition matrix is doubly stochastic, we may take x = 1, vector of 1 ' so By the above uniqueness of x, there can exist no stationary distribution, and there£ the chain is null. We deduce that the chain cannot be non-null persistent. 3. By the Chapman-Kolmogorov equations, m, r, n � O. Choose two states i and j , and pick m and n such that a = Pij (m)pji (n) > O. Then 1r
1r
�
Pii (m + r + n) � apjj (r). Set r = 0 to find that Pii (m + n) > 0, and so d(i) I (m + n). If d(i) f r then Pii (m + r + n) = 0, so that Pjj (r) = 0; therefore d(i) I d(j). Similarly d(j) I d(i) , giving that d(i ) = d(j). 4. (a) See the solution to Exercise (6.3.9a). (b) Let i, j, r, s E S, and choose N(i, r) and N(j, s) according to part (a). Then
lP'(Zn = (r, s ) I Zo = (i, j) ) = Pi r (n) Pj s (n) > 0 if n � max{ N (i , r), N(j, s )}, so that the chain is irreducible and aperiodic. (c) Suppose S = { I , 2} and P=
( � �) .
In this case { { I , I}, {2, 2} } and { { I , 2}, {2, I } } are closed sets of states for the bivariate chain. 5. Clearly lP'(N = 0) = 1 - iij , while, by conditioning on the time of the nth visit to j , we have that lP'(N � n + 1 I N � n) = Ijj for n � 1, whence the answer is immediate. Now lP'(N = = 1 - E�o lP'(N = n) which equals 1 if and only if iij = ijj = 1. 6. Fix i =1= j and let m = min{n : Pij (n) > OJ. If Xo = i and Xm = j then there can be no intermediate visit to i (with probability one), since such a visit would contradict the minimality of m.
(0)
304
Solutions
Problems
[6.15.71-[6.15.81
Suppose Xo = i , and note that (1 - fii) Pij (m) � 1 - iii , since if the chain visits j at time m and subsequently does not return to i , then no return to i takes place at all. However iii = 1 if i is persistent, so that fii = 1 . 7. (a) We may take S = {O, 1 , 2, . . . } . Note that qij (n) :::: 0 , and 00
L qu (n) = 1 , qu (n + 1) = L qi l (l)qu (n), j [=0 whence Q = (qu (1)) is the transition matrix of a Markov chain, and QTI = (qu (n)). This chain is persistent since for all i, n n and irreducible since i communicates with j in the new chain whenever j communicates with i in the original chain. That
i =/= j, n :::: l , is evident when n = 1 since both sides are qij (1). Suppose it is true for n = m where m :::: 1 . Now
Ij i (m + 1) = L Ijk (m) Pkj , k:kij so that
\
X ' l + l) = ""' (m)qi 1), i (m � L.J gkj d Xi j k:kfj
i =/= j, i =/= j,
which equals gij (m + 1) as required. (b) Sum (*) over n to obtain that
i =/= j, where Pi (j) is the mean number of visits to i between two visits to j; we have used the fact that L:n gij (n) = 1, since the chain is persistent (see Problem (6.15.6)). It follows that Xi = XO Pi (0) for all i , and therefore x is unique up to a multiplicative constant. (c) The claim is trivial when i = j, and we assume therefore that i =/= j. Let Ni (j) be the number of visits to i before reaching j for the first time, and write lP'k and Ek for probability and expectation conditional on Xo = k . Clearly, lP'j (Ni (j) :::: r) = hj i (1 - hij y - l for r :::: 1, whence
The claim follows by (**) . 8. (a) If such a Markov chain exists, then n Un = L iiUn - i , i =l
305
n :::: 1 ,
[6.15.9]-[6.15.9]
Markov chains
Solutions
where ii is the probability that the first return of X to its persistent starting point s takes place at time i . Certainly Uo = 1 . Conversely, suppose u is a renewal sequence with respect to the collection (fm : m :::: 1). Let X be a Markov chain on the state space S = {O, 1 , 2, . . . } with transition matrix
-{1
:::: i + 2 I T :::: i + 1) if j = i + 1 , :::: i + 2 I T :::: i + 1) if j = 0, where T is a random variable having mass function im = IP'(T = m). With Xo = 0, the chance that the first return to 0 takes place at time n is PI}
. .
(
1P' Xn = 0,
_
IP'(T _
IP'(T
IT1 Xi =f. 0 I Xo = 0) = POI P1 2 ' " Pn-2, n- 1 Pn- l ,O G(i + 1) ( 1 G(nG(n)+ 1) ) IT i =l G(i) =
_
= G(n) - G(n + 1) = in
where G (m) = IP'(T :::: m ) = L:�m in . Now Vn = IP'(Xn = 0 I Xo = 0) satisfies n Vn = L ii Vn - i for n :::: 1 , Vo = 1 , i=l whence Vn = U n for all n . (b) Let X and Y be the two Markov chains which are associated (respectively) with u and v in the above sense. We shall assume that X and Y are independent. The product (un vn : n :::: 1) is now the renewal sequence associated with the bivariate Markov chain (Xn , Yn ). 9 . Of the first 2n steps, let there be i rightwards, j upwards, and k inwards. Now X2n = 0 if and only if there are also i leftwards, j downwards, and k outwards. The number of such possible combinations is (2n) !/{(i ! j ! k! ) 2 } , and each such combination has probability ( ! ) 2 (i+j+k) = ( ! ) 2n . The first equality follows, and the second is immediate. Now 1 2n 2n n! 1P'(X2n = 0) � 2: n M i+j+k=n 3n I' .1 J' .1 k .1 where
()
{
()
L
}
M = max 3n . 7 !' 1 k 1 : i, j, k :::: O, i + j + k = n . J. . It is not difficult to see that the maximum M is attained when i , j, and k are all closest to j n, so that l .
Furthermore the summation in (*) equals 1 , since the summand is the probability that, in allocating n balls randomly to three urns, the urns contain respectively i , j, and k balls. It follows tllat
-
(2n) ! IP'(X2n = 0) < n 12 n ! n j- nJ !) 3 306
Problems
Solutions
[6.15.10]-[6.15.13]
3
which, by an application of Stirling's formula, is no bigger than Cn - 2 for some constant C. Hence :En 1P'(X2n = 0) < 00, so that the origin is transient. 10. No. The line of ancestors of any cell-state is a random walk in three dimensions. The difference between two such lines of ancestors is also a type of random walk, which in three dimensions is transient. 11. There are one or two absorbing states according as whether one or both of a and {3 equal zero. If a{3 =f. 0, the chain is irreducible and persistent. It is periodic if and only if a = {3 = 1, in which case it has period If 0 < a{3 < 1 then
2.
(! :)
3r = a {3 ' a {3
is the stationary distribution. There are various ways of calculating example. In this case the answer is given by
pn ; see Exercise (6.3.3) for
proof by induction. Hence
as n -+ 00. The chain is reversible in equilibrium if and only if 7r l Pl 2 = 7r2 P2 l , which is to say that a{3 = {3a ! 12. The transition matrix is given by
Pij =
::: i ::: N.
(N;i r -i)2 -1 - ( -i )2 - (N N N (� r
if j if j if j
i = i,
= + 1,
-
= i 1,
for 0 This process is a birth-death process in discrete time, and by Exercise (6.5. 1) is reversible in equilibrium. Its stationary distribution satisfies the detailed balance equation 7ri Pi , i +l = 7ri+l Pi +1 , i for O ::: < whenCe 7ri = 7rO ( �) 2 for O ::: where
i N,
i ::: N, 1 = N (N ) 2 = (2N ) N. 7rO � i
13. (a) The chain X is irreducible; all states are therefore of the same type. The state 0 is aperiodic, and so therefore is every other state. Suppose that Xo = 0, and let T be the time of the first return to O. Then IP'(T > n) = aO a l . . . an - l = bn for n :::: 1 , so that 0 is persistent if and only if bn -+ 0 as
n -+ 00.
(b) The mean of T is
00
00
E(T) = L IP'(T > n) = L bn . 307
[6.15.14]-[6.15.14]
Markov clulins
Solutions
The stationary distribution 'Ir satisfies 00
'irQ = L 'lrk (l - ak ), 1rn = 1rn - l an - l for n ::: 1. k=O Hence 1rn = 1rQbn and 1rO l = L:�Q bn if this sum is finite. (c) Suppose ai has the stated form for i ::: I. Then n-l bn = bI II (1 - Ai - f3 ), n ::: I. i =I Hence bn --+ 0 if and only if L: i Ai - f3 = 00, which is to say that f3 persistent if and only if f3 � 1 . (d) We have that 1 - x � e -x for x ::: 0 , and therefore
{
n- l 00 00 L bn � bI L exp -A L i - f3 i= I n =I n =I (e) If f3
}
00
{
�
1 . The chain is therefore
}
bI L exp -An . n - f3 < 00 if f3 < 1 . n= I
�
= 1 and A > 1 , there is a constant CI such that n- l I 00 00 00 00 bn exp -A � b exp {-A log n} -;� C = CI L n - A < 00, L I I n =I i= I I n= I n= I n= I
L
{
giving that the chain is non-null. (!) If f3 = 1 and A � 1 ,
L
}
L
( A)
( )
( )
n-l . n-l bn = b I II 1 - --;- ::: bI II �1 = bI nI = 11 . i =I I i =I I Therefore L:n bn = 00, and the chain is null. 14. Using the Chapman-Kolmogorov equations,
/
/ (
I Pij (t + h) - Pij (t) 1 = L (Pi k (h) - 8i k ) Pkj (t) � 1 - Pii (h)) pij (t) + L Pik (h) k koli � 1 - Pii (h)) + 1 - Pii (h)) --+ 0
(
(
as h .J, 0, if the semigroup is standard. Now log x is continuous for 0 < x � 1 , and therefore g is continuous. Certainly g(O) addition Pii (S + t) ::: Pii (S) Pii (t) for s, t ::: 0, whence g(s + t) � g(s) + g(t), s, t ::: O. For the last part 1 get) ii (t) - 1 -t ( Pi i (t) - 1 = t - log{ 1P- (1 - ii (t))) --+ -A P
)
as t .J, 0, since x /log(1 - x)
-
.
--+ -1 as x .J, O. 308
= O. In
Problems
Solutions
[6.15.15]-[6.15.16]
15. Let i and j be distinct states, and suppose that Pij (t) > 0 for some t. Now 00 1 t n (G n ) , � ij Pij (t) = L...J , n n=O . implying that (G n )ij > 0 for some n, which is to say that
for some sequence k l ' k2 , . . . , kn of states. Suppose conversely that (*) holds for some sequence of states, and choose the minimal such value of n. Then i, k l ' k2 , " " kn , j are distinct states, since otherwise n is not minimal. It follows that (Gn )ij > 0, while (Gm ) ij = 0 for 0 � m < n. Therefore 00 1 t k (G k )i � j Pij (t) = tn L...J k=n k '. is strictly positive for all sufficiently small positive values of t. Therefore i communicates with j. 16. (a) Suppose X is reversible, and let i and j be distinct states. Now IP'(X(O) = i, X(t) = j) = IP'(X(t) = i, X(O) = j) , which is to say that 11:iPij (t) = 11:j Pji (t). Divide by t and take the limit as t .J, 0 to obtain that 11:i gij = 11:j gji ' Suppose now that the chain is uniform, and X (0) has distribution If t > 0, then 1r .
so that X(t) has distribution 1r also. Now let t < 0, and suppose that X(t) has distribution IL. The distribution of X (s) for s � 0 is ILPs - t = 1r , a polynomial identity in the variable s - t, valid for all s � O . Such an identity must be valid for all s , and particularly for s = t, implying that IL = 1r . Suppose in addition that 11:i gij = 11:j gj i for all i , j. For any sequence k l , k2' . . . , kn of states,
11:igi , k l gk l , k2 " ' gkn , j = gk l , i 11:k l gk l , k2 " ' gkn , j = . . . = gk l , i gk2 , k l . . · gj , kn 11:j · Sum this expression over all sequences k l , k2' . . . , kn of length n, to obtain n � O. 11:i (G n + 1 )ij = 11:j (Gn + 1 )ji '
It follows, by the fact that Pt
for all i,
= e tG , that
j, t . For tl < t2 < . . , < tn, IP'(X(t l ) = ii , X (t2 ) = i 2 , " " X(tn) = in) = 11:i l Pi l . i2 (t2 - t l ) . . . Pin - l , in (tn - tn - d = Pi2 , i l (t2 - tl ) 11:i2 Pi2 , i3 (t3 - t2 ) . . . Pin - l , in (tn - tn - d = . . . = Pi2 , i 1 (t2 - tl ) " . Pin , in - l (tn - tn - d11:in = IP'(Y(td = i i , Y(t2 ) = i 2 , " " Y(tn) = in) , 309
[6.15.17]-[6.15.19]
Markov chains
Solutions
giving that the chain is reversible. (b) Let S = { I , 2} and
(
a G = -a f3 -f3
)
where af3 > O. The chain is unifonn with stationary distribution
(! :)
7r = a f3 ' a f3 ' and therefore 7r l g 1 2 = 7r2 g2 1 · (c) Let X be a birth-death process with birth rates A i and death rates P, i . The stationary distribution 7r satisfies Therefore irk+ 1 P,k+ 1 = irk A k for k ;::: 0, the conditions for reversibility. 17. Consider the continuous-time chain with generator
)
(
f3 . G = -f3 y -y It i s a standard calculation (Exercise (6.9. 1)) that the associated semigroup satisfies
f3( 1 - h(t)) (f3 + y )Pt = yY(1+-f3h(t) h(t)) f3 + yh(t)
(
)
where h(t) = e - t({J + y ) . Now Pi = P if and only if y + f3h(1) = f3 + yh(1) to say that f3 = y = - � 10g(2a - 1), a solution which requires that a > � . 18. The forward equations for Pn (t) = lP'(X(t) = n ) are
po (t) = P,Pl - APO , p� (t) = APn - l - (A + np,) Pn + p,(n + I) Pn+l , In the usual way,
= a(f3 + y), which is
n ;::: l .
)
aG = (s 1) AG p, aG at as I with boundary condition G(s, 0) = s . The characteristics are given by dG dt = p, ds 1) (s A(S - I )G ' and therefore G = e P(s - 1 ) f ( (s - 1)e - ILt ) , for some function f, determined by the boundary condition to satisfy e P(s - l ) f(s - 1) = s I . The claim follows. As t -+ 00, G(s, t) -+ e P (s - l ) , the generating function of the Poisson distribution, parameter p . _
19.
(
_
(a) The forward equations are
a at Pii (s, t) = -A(t) Pii (S, t), a (t), at Pij (s, t) = -A(t) Pij (s, t) + A(t) Pi, j - 1 310
i
< j.
Problems Assume N(s)
Solutions
[6.15.201-16.15.20]
= i and s < t. In the usual way, G (s, t; x)
satisfies
00
= L x j JP'( N (t) = j I N(s) = i) j=i aG at
= A (t) (X - l) G.
We integrate subject to the boundary condition to obtain G(s, t; x)
{
= x i exp (x - 1 )
whence Pij (t) is found to be the probability that A parameter f: A (U) duo The backward equations are
it A(U) du } ,
= j - i where A has the Poisson distribution with
a as Pij (s, t) = A(S) Pi+l , j (S, t) - A(S) Pij (s, t); using the fact that Pi+l , j (t) = Pi , j - l (t) , we are led to aG -= A(S)(X - l) G. as
The solution is the same as above . (b) We have that JP'(T
so that fT (t)
In the case A (t)
{ fot A(U) du },
> t) = poo (t) = exp =
= c/(l + t), E(T)
A(t)exp { - l A (U) dU } ,
looo
= o JP'(T > t) dt =
1000 0
t � O.
du
(1
+ u)C
which is finite if and only if c > 1 . 20. Let S > O . Each offer has probability 1 - F (s) of exceeding s , and therefore the first offer exceeding S is the Mth offer overall, where JP'(M = m) = F(s) m - l [1 - F(s)], m � 1 . Conditional on {M = m}, the value of XM is independent of the values of X l , X2 , " " XM - l , with JP'(XM
> U I M = m) = 11 -- F(u) , F(s)
0 < s ::: u,
and X l , X2 , . . . , X M - 1 have shared (conditional) distribution function G (u I s)
= F(u) , F(s) 311
0 ::: U ::: S.
[6.15.211-16.15.21]
Markov chains
Solutions
For any event B defined in terms of X l , X2 , " 00
" XM - I , we have that
lP'(XM u, B) = .2: lP'(XM u, B I M = m ) lP'( M = m ) m= l
>
> .2: lP'(XM > U I M ) lP'(B I M ) lP'(M ) m= l lP'(XM > u) .2: lP'(B I M ) lP'( M ) m=l 0 < s ::S u, lP'(XM > u)lP'(B), where we have used the fact that lP'(XM > u I M ) is independent of It follows that the first record value exceeding s is independent of all record values not exceeding By a similar argument =
00
=m
= m
00
=
= m
= m
=m
=
= m
(or an iteration of the above) all record values exceeding s exceeding s . The chance of a record value in (s, s + h] is
- F(s ) lP'(s < XM ::s s + h) = F(s 1+_h)F(s )
m.
s.
are
=
independent of all record values not
1
f(s )h + o(h). F(s ) _
A very similar argument works for the runners-up. Let XMl ' XM2 ' . . . be the values, in order, of offers exceeding s . It may be seen that this sequence is independent of the sequence of offers not exceeding s , whence it follows that the sequence of runners-up is a non-homogeneous Poisson process. There is a runner-up in (s, s + h] if (neglecting terms of order o(h)) the first offer exceeding s is larger than s + h, and the second is in (s, s + h]. The probability of this is
( 1 - F(s + h) ) ( F(S + h) - F(S) ) + o(h) 1
- F(s)
1
- F(s)
=
1
f(s )h + o(h) . - F(s)
21. Let Ft (x) = lP'(N * (t) ::s x), and let A be the event that N has a arrival during (t, t + h). Then Ft +h (X) = AhlP' (N * (t + h) ::s x I A) + (1 - Ah)Ft (x) + o(h) where Hence
L: Ft (x - y)f(y) dy. -AFt (x) + A 1 Ft (x - y)f(y) dy.
lP'(N * (t + h) ::s x I A) =
00
� Ft (x) = at - 00 i N Take Fourier transforms to find that ¢t (8) = lE(e O * (t) ) satisfies a¢ att = -A ¢t + A ¢t ¢ , an equation which may be solved subject to 4>0(8) Alternatively, using conditional expectation,
=
1 to obtain ¢t (8)
=
eAt(t/J (O )-i) .
¢t (8) = lE{lE(e i O N* (t) I N(t)) } = lE {¢ (8) N (t) } where N (t) is Poisson with parameter At. 312
e ms
Probl
Solutions
[6.15.22H6.15.25]
22. We have that
E(S N (t » = E { E(s N (t) I A )} = 1 {eAj t(s - l ) + eA 2 t(s- I ) }, whence E(N(t» = � (A, 1 + A,2 )t and var(N(t» = 1 (A, 1 + A,2 )t + £ (A, I - A,2 ) 2 t 2 . 23. Conditional on {X (t) = i}, the next arrival in the birth process takes place at rate Ai . 24. The forward equations for Pn (t) = jp(X (t) = n) are 1 + p,n 1 + p,(n - 1) n :::: 0, n (t), n (t) Pn, (t) = 1 + p,t P - l 1 + p,t P with the convention that l (t) = O. Multiply by s n and sum to deduce that
P-
aG
aG as
aG as
(I + p,t) = s G + p,s 2 - - G - p,s at as required. Differentiate with respect to s and take the limit as s t 1. If E(X (t) 2 )
< 00, then
I
aG m (t) = E (X (t» = as s =1 satisfies (1 + p,t)m ' (t) = 1 + p,m (t) subject to m (O) = I. Solving this in the usual way, we obtain m (t) = I + (1 + p,I)t. Differentiate again to find that n (t) = E ( X (t) (X (t) - 1) ) =
a2 G as 2 s =1
-
I
satisfies (1 + p,t)n ' (t) = 2 (m (t) + p,m (t) + p,n (t» ) subject to n (O) = 1 (/ - 1). The solution is n (t) = 1 (/ - 1) + 2 1 (1 + p,I)t + (1 + p,I) ( 1 + p, + p,I)t 2 . The variance of X (t) is n (t) + m (t) - m (t) 2 . 25. (a) Condition on the value of the first step: j :::: 1 , as required. Set Xi =
'1i + l - '1i to obtain A,j Xj = P,j Xj - l for j :::: 1 , so that j :::: 1 .
It follows that
j j '1j + l = '10 + L Xk = 1 + ('1 1 - 1) L ek· k=O k=O The '1j are probabilities, and lie in [0, 1]. If Z=f ek = 00 then we must have '1 1 = which implies that '1j = 1 for all j .
1,
313
[6.15.26]-[6.15.28]
Markov chains
Solutions
(b) By conditioning on the first step, the probability TJj , of visiting 0 having started from j, satisfies
(j + 1) 2 TJj +l + j 2 TJj_ 1 TJj = j 2 + (j + 1) 2 Hence, (j + 1) 2 (TJj+l - TJj ) = f (TJj - TJj - l ), giving (j + 1) 2 (TJj+1 - TJj) = TJ l - TJo. Therefore, as j
-+ 00 .
1(6/rr 2 ). 26. We may suppose that X (O) = O . Let Tn = inf{t : X(t) = n}. Suppose Tn = T, and let Y = Tn +l - T. Condition on all possible occurrences during the interval (T, T + h) to find that JE(Y) = O nh)h + /1-nh(h + JE(Y' )) + (1 - Anh - /1-nh)(h + JE(Y)) + o(h), By Exercise (6.3.6), we seek the minimal non-negative solution, which is achieved when TJ l =
..
where Y' is the mean time which elapses before reaching n + 1 from n - I . Set m n = JE(Tn + 1 to obtain that mn = /1-nh(m n - l + mn) + mn + h{l - (An + /1-n)mn } + o(h). Divide by h and take the limit as h .,I. 0 to find that An m n = 1 + /1-nm n - l , n :::: 1. Therefore
- Tn)
/1-n m = . . . = 1 + /1-n + . . . + /1-n/1-n - l " . /1-1 , mn = An1 + An An - l . . . 1.. 0 An n - l An AnAn - l since mo = 1.. 0 1 . The process is dishonest if z=� o mn < 00, since in this case Too = lim Tn has finite mean, so that JP'(Too < (0) = 1. On the other hand, the process grows no faster than a birth process with birth rates A i , which is honest if z=� o l/An = 00. Can you find a better condition? 27. We know that, conditional on X (O) = I, X(t) has generating function -
-
(
---
)
G(s, t) = At(1 - S) + S 1 ' At (1 s) + 1 _
so that
JP'(T ::s x I X (0) = I) = JP'(X (x) = 0 I X (0) = I) = G(O, x) = It follows that, in the limit as x
-+ 00,
(
(
)
Cx� 1 ) 1 )
AX 1 JP'(X(O) = I) = GX(O) AX -+ 1 . L..JP'(T ::s x) = � Ax + 1 AX +1 1 =0 For the final part, the required probability is {xI/(xI + 1)}1 = { I + (xI) - 1 } - I , which tends to 1 / e - x as I -+ 00 . 28. Let Y be an immigration-
It follows that IP'( R r) = IP'( V ball in n dimensions. Therefore
> crn ) = e-J...crn for r � 0, where c is the volume of the unit r � O.
Finally, lE ( R )
= Jooo e-J...crn dr , and we set = Acr n . v
32. The time between the kth and (k + 1)th infection has mean Ak 1 , whence N
1 lE(T) = :E . A k=l k Now N
t1
1 k(N + 1 - k)
= N1 1 +
{N 1+N 1 } t1 k t1 N + 1 - k N
2 1 2 = N + 1 :E - = -{ log N + y + O(N - 1 ) } . 1 N + k k=l --
It may be shown with more work (as in the solution to Problem (5. 12.34)) that the moment generating function of A(N + 1) T - 2 log N converges as N -+ 00, the limit being (r(1 - B)f.
33. (a) The forward equations for Pn (t) = IP'( V (t) = n + !) are
n � 0, with the convention that P - l (t)
= O. It follows as usual that
aG aG ( aG ) ( aG G) as at as as
- = - - 2s - + G + s 2 - + s as required. The general solution is
G (s, t)
(
1 _ = _1_ 1 t + _ 1 -s 1 -s
for some function f . The boundary condition is G (s, 0) (b) Clearly
mn(T) = 1E
)
= 1 , and the solution is as given.
(loT Int dt) = loT lE(lnt) dt
by Fubini's theorem, where Int is the indicator function of the event that V ( t ) As for the second part, 00
'"' s n mn (T) n=O so that, in the limit as T -+ 00, L..J
� s n (mn(T) - log T) = n=O L..J
= n + !.
- 10o T G(s, t) dt - log[1 +1 -(1 s- s)T] , _
( 1 + (1 - S)T ) -+ log(l - s) = - :Eoo snan 1 -log 1 -s 1 -s T n=l 3 16
Problems
[6.15.34]-{6.15.35]
Solutions
if l s i < 1 , where an = 2:7=1 i- I , as required. (c) The mean velocity at time is
t
�'-------'- -
---.
(2, 1)
�'----I__--'o=--_�_----'-
(0, 1)
�--....--,.=--...-�
( 1 , 0)
_
-
_
_
_
.
- - - .
""-:::,---.---,.. - - - - .
34. It is clear that Y is a Markov chain, and its possible transitions are illustrated in the above diagram. Let x and y be the probabilities of ever reaching ( 1 , 1) from ( 1 , 2) and (2, 1), respectively. By conditioning on the first step and using the translational symmetry, we see that x = i y + i x 2 and y = i + ixy. Hence x 3 - 4x 2 + 4x - 1 = 0, an equation with roots x = 1 , i (3 ± .J5). Since x is a probability, it must be that either x = l or x = i (3 - .J5), with the corresponding values of y = 1 and y = i (.J5 - 1). Starting from any state to the right of ( 1 , 1) in the above diagram, we see by recursion that the chance of ever visiting ( 1 , 1) is of the form XIX yf3 for some non-negative integers a, p. The minimal non-negative solution is therefore achieved when x = i (3 - .J5) and y = i (.J5 - 1). Since x < 1 , the chain is transient. 35. We write A, 1 , 2, 3, 4, 5 for the vertices of the hexagon in clockwise order. Let Ti = min{n :::: 1 Xn = i} and JlDi ( - ) = JID( . I Xo = i). (a) By symmetry, the probabilities Pi = JlDi (TA < Tc) satisfy :
PA = "32 P1 , P I whence PA =
1 + "31 P , P = "31 P1 + 31 P , P = "32 P , 2 2 2 3 3
= "3
17 '
(b) By Exercise (6.4.6), the stationary distribution is nc
lCA- 1 = 8 .
=
1, lCi
=
� for i =f:.
C, whence
/LA
=
(c) By the argument leading to Lemma (6.4.5 ) , this equals /LA lCC = 2. (d) We condition on the event E = {TA < Tc} as in the solution to Exercise (6.2.7). The probabilities bi = JlDi (E) satisfy 317
[6.15.36]-[6.15.37]
Markov clulins
Solutions
yielding h 1 = ft' h2 = equations of the form
i , h3 = �
0
The transition probabilities conditional on E are now found by
12
lE (x A(t ) y B (t ) ) = lE { lE (x A(t ) yB(t) I N (t)) } = 1E { (lrX + py + 1 - lr _ p ) N (t ) } = e A1r(x- l ) eAp (y - l ) . Thus A(t) and B(t) are independent Poisson-distributed random variables. If the ants arrive in pairs and then separate,
3 19
[6.15.42]-[6.15.45]
Markov chains
Solutions
where y = 1 - 7r - p . Hence,
whence A(t) and B(t) are not independent in this case. 42. The sequence {Xr } generates a Poisson process N(t) = max{n : Sn t} . The statement that Sn = t is equivalent to saying that there are n - 1 arrivals in (0, t), and in addition an arrival at t . By Theorem (6. 12.7) or Theorem (6. 13.1 1), the first n - 1 arrival times have the required distribution. Part (b) follows similarly, on noting that .fu (u) depends on u = (U I , U 2 , . . . , un) only through the constraints on the U r . 43. Let Y be a Markov chain independent of X, having the same transition matrix and such that Yo has the stationary distribution 7r . Let T = min{n � 1 : Xn = Yn } and suppose Xo = i . As in the proof of Theorem (6.4. 17),
:::
I Pij (n) - 7rj I = Now,
I I:k 7rk (Pij (n) - Pkj (n) ) I ::: I:k 7rklP'(T > n) = IP'(T > n).
IP'(T > r + 1 I T > r) ::: 1 - € 2
for r � 0, 2 where € = min ij { Pij } > O. The claim follows with A = 1 - € . 44. Let /k (n) be the indicator function ofa visitn to k at time n, so that E(h (n)) = IP'(Xn = k) = ak (n), say. By Problem (6. 15.43), lak (n) - 7rk I ::: A . Now,
Let S = minIm , r } and t = 1m - r l . The last summation equals
=
� I: I: { (aj (s) - 7rj ) (Pii (t) - 7rj ) + 7rj (Pii (t) - 7rj )
n
r
m
+ 7rj (a j (s) - 7rj ) - 7rj (aj (r) - 7rj ) - 7rj (aj (m) - 7rj ) }
as n -+
00 ,
where 0 < A < 00 . For the last part, use the fact that :E�;:J f ( Xr ) = :E j E S f (i ) Vj (n). The result is obtained by Minkowski's inequality (Problem (4. 14.27b)) and the first part. 45. We have by the Markov property that f(Xn+ I I Xn, Xn - I , . . . , Xo) = f(Xn+ 1 I Xn), whence E (log f(Xn+ 1 I
Xn , Xn - I , · · · , Xo) I Xn , . . . , Xo) = E (log f(Xn+I I Xn) I Xn) . 320
Problems
Solutions
[6.15.46]-[6.15.48]
Taking the expectation of each side gives the result. Furthermore, H (Xn +1 I Xn) =
- L (Pij log Pij ) 1P' (Xn = i). i,j
Now X has a unique stationary distribution 1C , so that IP'(Xn = i) --+ 7ri as n --+ 00 . The state space is finite, and the claim follows. 46. Let T = inf{t : Xt = Yd. Since X and Y are persistent, and since each process moves by distance 1 at continuously distributed times, it is the case that IP' (T < (0) = 1. We define Zt =
{ Xt
if t < T, Yt if t � T,
noting that the processes X and Z have the same distributions. (a) By the above remarks, I IP'(Xt = k) - 1P'(Yt = k) 1 = I IP'(Zt = k) - 1P'(Yt = k) 1 ::::: \1P' (Zt = k, T ::::: t ) + IP'(Zt = k , T t ) - 1P'(Yt = k , T ::::: t ) - 1P'(Yt = k , T ::::: IP'(Xt = k, T t) + IP'(Yt = k, T t).
>
> >
> t) \
We sum over k E A, and let t --+ 00 . (b) We have in this case that Zt ::::: Yt for all t . The claim follows from the fact that X and Z are processes with the same distributions. 47. We reformulate the problem in the following way. Suppose there are two containers, W and N, containing n particles in all. During the time interval (t, t + dt), any particle in W moves to N with probability f.Ldt + o(dt), and any particle in N moves to W with probability Adt + o(dt). The particles move independently of one another. The number Z (t) of particles in W has the same rules of evolution as the process X in the original problem. Now, Z(t) may be expressed as the sum of two independent random variables U and V, where U is bin(r, (}t), V is bin(n - r, 1/It), and (}t is the probability that a particle starting in W is in W at time t, 1/It is the probability that a particle starting in N at 0 is in W at t. By considering the two-state Markov chain of Exercise (6.9. 1),
1/It =
A - A e -(A + /-t) t A + f.L
and therefore
JE(S X(t » ) = JE(s U )JE(s v ) = (s(}t + 1 - s/{s1/lt + 1 s) n - r . Also, JE(X (t)) = r (}t + (n - r)1/It and var(X (t)) = r(}t(1 - (}t) + (n - r)1/It (1 - 1/It). In the limit as n --+ the distribution of X (t) approaches the bin(n , A/(A + f.L)) distribution. 48. Solving the equations _
00,
gives the first claim. We have that y = L: i (Pi - qi )7ri , and the formula for y follows. Considering the three walks in order, we have that: A. 7ri = � for each i , and YA = -2a < o. B. Substitution in the formula for YB gives the numerator as 3{ - !g a + o(a) } , which is negative for small a whereas the denominator is positive. 321
[6.15.49]-[6.15.51]
Markov chains
Solutions
C. The transition probabilities are the averages of those for A and B, namely, Po = a) + - a) = fu - a, and so on. The numerator in the formula for YC equals � + 0(1), which is positive for small a. 49. Call a car green if it satisfies the given condition. The chance that a green car arrives on the scene during the time interval (u, u + h) is >" hJP( V < x/(t - u)) for u < t. Therefore, the arrival process of green cars is an inhomogeneous Poisson process with rate function
� ( -Ib
� (�
>.. ( u) =
{ U (V < x/(t - u)) o
if u < t, if u ::: t.
Hence the required number has the Poisson distribution with mean >..
fot JP ( V
.. fo JP ( V < �) du t � u) = >..
fot
E(l{vu<x} ) du = >..E (V - 1 min{x, Vt} ) .
50. The answer is the probability of exactly one arrival in the interval (s, t), which equals g(s) = >.. ( t - s)e - J.. Ct -s) . By differentiation, g has its maximum at s = max{O, t - >.. - 1 }, and g(S) = e - 1 when t ::: >.. - 1 . 51. We measure money in millions and time in hours. The number of available houses has the Poisson distribution with parameter 30>", whence the number A of affordable houses has the Poisson distribution with parameter · 30>" = 5>.. (cf. Exercise (3.5.2)). Since each viewing time T has moment generating function E(e B T ) = (e 2B - e B )/(J, the answer is
i
322
7
Convergence of random variables
7.1 Solutions. Introduction
(a) EI(cXY I = Icl r . {IIXli r V . (b) This is Minkowski's inequality. (c) Let O. Certainly I X I � h where h is the indicator function of the event { I X I } . Hence implying that IP'(IXI = 0 for all O. The converse is trivial. EIXr I � Ell; I = IP'(IXI 2. (a) E({aX + bY}Z) = aE(XZ) + bE(YZ) . (b) E({X + Y} 2 ) + E({X - y} 2 ) = 2E(X 2 ) + 2E(y 2 ) . (c) Clearly 1.
E>
> E),
3. Let f(u) = dE (f, h) = 1.
> E)
�E , g(u) = 0, h(u) = -�E , for all u .
E>
Then dE (f, g)
>E
+ dE (g, h) = 0 whereas
4. Either argue directly, or as follows. With any distribution function F, we may associate a graph
F obtained by adding to the graph of F vertical line segments connecting the two endpoints at each discontinuity of F . By drawing a picture, you may see that .J2 d (F, G) equals the maximum distance between F and G measured along lines of slope - 1 . It is now clear that d(F, G) = 0 if and only if F = G, and that d (F, G) = d (G, F) . Finally, by the triangle inequality for real numbers, we have
that d(F, H) � d(F, G) + d(G, H) . 5. Take X to be any random variable satisfying E(X 2 ) = 00, and define Xn = X for all n. 7.2 Solutions. Modes of convergence 1.
(a) By Minkowski's inequality,
let n --+
00
to obtain lim infn� oo EIX� I
�
EIX r l. By another application of Minkowski's inequality,
323
[7.2.2]-[7.2.4]
Corwergence of random variables
Solutions
(b) We have that
IlE(Xn) - lE(X) 1 = IlE(Xn - X) I � lElXn XI -+ 0 as n -+ 00. The converse is clearly false. If each Xn takes the values ± I , each with probability i, then lE(Xn ) = 0 , but lElXn - 0 1 = 1 . (c) By part (a), lE(X;) -+ lE(X 2 ) . Now Xn � X by Theorem (7.2.3), and therefore lE(Xn ) -+ lE(X) by part (b). Therefore var(Xn) = lE(X;) - lE(Xn) 2 -+ var (X) . -
2. Assume that Xn � X . Since IXn l � Z for all n, it is the case that IXI � Z a.s. Therefore
>
Zn = IXn - XI satisfies Zn � 2 Z a.s. In addition, if E 0,
>
As n -+ 00, JP (I Zn I E) -+ 0, and therefore the last term tends to 0; to see this, use the fact that lE(Z) < 00, together with the result of Exercise (5.6.5). Now let E ..I- 0 to obtain that lElZn l -+ 0 as n -+ 00 . 3. We have that X - n l � Xn � X, so that lE(Xn ) -+ lE(X), and similarly lE(Yn) -+ lE(Y) . By the independence of Xn and Yn , -
lE(Xn Yn) = lE(Xn)lE(Yn) -+ lE(X)lE(Y) . Finally,
(X - n l )( y n l ) � Xn Yn � XY, and -
lE as n
-
-
{ (X - ;;I ) (Y - ;;I ) } = lE(XY)
-
lE(X) + lE(Y) + I n n2
-+
lE(XY)
so that lE(Xn Yn) -+ lE(XY). 4. Let Fl , F2 , ' " be distribution functions. As in Section 5.9, we write Fn -+ F if Fn (x) -+ F (x) for all x at which F is continuous. We are required to prove that Fn -+ F if and only if d (Fn , F) -+ O. Suppose that d (Fn , F) -+ O. Then, for E 0, there exists N such that -+ 00 ,
>
F(x - E) - E � Fn (x) � F(x + E) + E
for all x.
Take the limits as n -+ 00 and E -+ 0 in that order, to find that Fn (x) -+ F(x) whenever F is continuous at x. Suppose that Fn -+ F. Let E 0 , and find real numbers a = X l < X2 < . . . < Xn = b, each being points of continuity of F, such that (i) Fj (a) < E for all i, F (b) 1 - E, (ii) IXj +! - xi i < E for I � i < n . In order to pick a such that Fj (a) < E for all i, first choose a/ such that F (a ' ) < i E and F is continuous at a/ , then find M such that I Fm (a ' ) F (a ' ) 1 < i E for m ::: M, and lastly find a continuity point a of F such that a � a/ and Fm (a) < E for I � m < M. There are finitely many points Xj , and therefore there exists N such that IFm (xj) - F(xj)1 < E for all i and m ::: N. Now, if m ::: N and Xj � x < xi+l ,
>
>
-
and similarly
>
Fm (x) ::: Fm (xj) F(xj) - E ::: F(x - E) - E. Similar inequalities hold if x � a or x ::: b, and it follows that d( Fm , F) < E if m ::: N. Therefore d( Fm , F) -+ 0 as m -+ 00 . 324
A1odes of convergence
Solutions
5. (a) Suppose C > 0 and pick li such that 0
n :::: N. Now, for x :::: 0, IP'(Xn Yn
x ) ::::; IP' ( Xn Yn > x , I Yn
of continuity of the distribution function of X. A similar argument holds if x < 0, and we conclude that Xn Yn .s c X if c > O. No extra difficulty arises if c < 0, and the case c = 0 is similar. For the second part, it suffices to prove that yn- 1 � c - 1 if Yn � c (# 0). This is immediate from the fact that Iy,;- l - c - 1 1 < E/{lcl(lcl - E)} if I Yn - cl < E « Icl) . (b) Let E > O. There exists N such that
E)
E, 1P' ( I Yn - Y I > E)
E, if n :::: N, and in addition IP'(I Y I > N) < E. By an elementary argument, g is uniformly continuous at points of the form (0, y) for I y l ::::; N. Therefore there exists li (> 0) such that Ig(x / , y /) - g(O, y) 1 < E if l x / I ::::; li, Iy ' - y l ::::; li . If I Xn l ::::; li, I Yn - Y I ::::; li , and I Y I ::::; N, then I g(Xn , Yn) - g(O, Y) I < E, so that 1P' ( lg(Xn , Yn) - g(O, Y) I :::: E) ::::; 1P' ( I Xn l > li ) + 1P' ( I Yn - Y I > li ) + IP'( I Y I > N) ::::; 3E, 1P' ( I Xn l >
iE) + 1P' ( lcn - cl I X I iE) E E + IP' I X I > ::::; IP' I Xn - X I > 21cn - cl 2(c + E) lE ( lcn Xn
-
�0
(
as n �
00 .
325
) (
· >
)
[7.2.8]-[7.3.1]
Convergence of random variables
Solutions
(d) A neat way is to use the Skorokhod representation (7.2. 14). If Xn S X, find random variables Yn , Y with the same distributions such that Yn � Y. Then cn Yn � c Y, so that cn Yn S c Y, implying the same conclusion for the X's.
> 2E , > C + E) > 2E . Since Xn � X, there exists N such that JP(Xn < c) > E, JP(Xn > C + E) > E, if n 2::: N. Also, by the triangle inequality, I Xr - Xs i :::: IXr - X I + I Xs - X I ; therefore there exists M such that JP(IXr - Xs i > E) < E 3 for r, s 2::: M. Assume now that the Xn are independent. Then, for r, s 2::: max{M, N}, r =1= s, E 3 > JP(I Xr - Xs i > E) 2::: JP( Xr < C, XS > C + E) = JP (Xr < c) JP (Xs > C + E) > E 2 , 8. If X is not a.s. constant, there exist real numbers C and E such that 0 < E < � and JP(X < c) JP(X
a contradiction. 9. Either use the fact (Exercise (4. 12.3)) that convergence in total variation implies convergence in distribution, together with Theorem (7.2. 1 9), or argue directly thus. Since l uOI :::: K < 00,
1
Ik
I lE( u (Xn)) - lE( u (X)) 1 = I: u (k) { fn (k) - f (k) } :::: K I: I fn (k) - f (k) I --+
k
o.
10. The partial sum Sn = E�=l Xr is Poisson-distributed with parameter an = E�=l Ar. For fixed x,non-negative the event {Sn :::: x } is decreasing in n, whence by Lemma (1 .3.5), if an --+ a < 00 and x is a integer,
Hence if a < 00, E�l Xr converges to a Poisson random variable. On the other hand, if an --+ 00, then e - Cfn EJ=o a1 / j ! --+ 0, giving that JP(E�l Xr ) = 1 for all and therefore the sum diverges with probability 1 , as required.
x,
>x
7.3 Solutions.
Some ancillary results
> E then either I Xn - XI > �E or IXm - XI > �E, so that JP(IXn - Xm l > E) :::: JP(IXn - X I > �E) + JP(IXm - X I > � E) --+ 0 as n, m --+ 00, for E > O. 1. (a) If I Xn - Xm I
Conversely, suppose that {Xn } is Cauchy convergent in probability. For each positive integer k, there exists n k such that for n , m
2:::
nk
·
The sequence (n k ) may not be increasing, and we work instead with the sequence defined by Nl = n t. Nk+ l = max{Nk + 1 , n k+ i l. We have that
I:JP( IXNk+l - XNk l 2::: Tk ) < 00, k 326
Some ancillary results
Solutions
[7.3.2]-[7.3.3]
whence, by the first Borel-Cantelli lemma, a.s. only finitely many of the events {I X N k+ ! 2 - k } occur. Therefore, the expression
- XNk I �
00
X XNl + L (XNk+! - XNk ) =
k= l
converges absolutely on an event C having probability one. Define X (w) accordingly for w E C, and X (w) = 0 for w rf. C. We have, by the definition of X, that XNk � X as k -+ 00. Finally, we 'fill in the gaps' . As before, for 0,
E>
as n, k -+ 00, where we are using the assumption that {Xn } is Cauchy convergent in probability. (b) Since Xn �
X, the sequence {Xn } is Cauchy convergent in probability. Hence JP ( I Yn - Ym l > E ) JP (I Xn - Xm l > E ) -+ 0 as n, m -+ 00, =
E>
for O. Therefore {Yn } is Cauchy convergent also, and the sequence converges in probability to some limit Y. Finally, Xn .!; X and Yn .!; X, so that X and Y have the same distribution. 2. Since An � U:;;'=n Am , we have that
oo
Un Am ) .... oo m= oo ( U ) JP (nlim
lim sup JP(An ) ::: nlim .... JP m=n Am n ....
=
=
JP(An Lo.) ,
where we have used the continuity of JP. Alternatively, apply Fatou's lemma to the sequence IA� of indicator functions. 3. (a) Suppose X2n = 1, X2n+l = - 1, for n � 1. Then {Sn = 0 Lo.} occurs if X l = - 1, and not if X I = 1 . The event is therefore not in the tail a-field of the X's. (b) Here is a way. As usual, JP(S2n = 0) = (� ) {p(1 - p) } n , so that
L JP(S2n = 0) n
< 00 if p #
i,
implying by the first Borel-Cantelli lemma that JP(Sn = 0 i.o.) = O. (c) Changing the values of any finite collection of the steps has no effect on I = lim inf Tn and J = lim sup Tn , since such changes are extinguished in the limit by the denominator ' ..;n' . Hence I and J are tail functions, and are measurable with respect to the tail a -field. In particular, {l ::: -x} n {J � x } lies in the a -field. Take x = 1, say. Then, JP(l ::: - 1) = JP(J � 1) by symmetry; using Exercise (7.3.2) and the central limit theorem,
oo
1) � JP(Sn � ..;n Lo.) � lim sup JP(Sn � ../ii) = 1 - (1) 0, n .... where is the N(O, 1) distribution function. Since {J � I } is a tail event of an independent sequence, it has probability either 0 or 1, and therefore JP(l ::: - 1) = JP(J � 1) = 1, and also JP(l ::: - 1, J � 1) = 1 . That is, on an event having probability one, each visit of the walk to the left of -..;n is JP(J
�
followed by a visit of the walk to the right of ..;n, and vice versa. infinitely often, with probability one. 327
>
It
follows that the walk visits 0
[7.3.4]-[7.3.8]
Convergence of random variables
Solutions
4. Let A be exchangeable. Since A is defined in tenns of the Xi , it follows by a standard result of measure theory that, foreach n, there exists an event An E a (X l , X2 , . . . , Xn), such thatJP>(ALlAn ) -+ o as n -+ 00. We may express An and A in the fonn An = {Xn E Bn },
A = {X E B},
where Xn = (X l , X2 , . . . , Xn ), and Bn and B are appropriate subsets of Rn and R oo . Let A� = {X� E Bn },
A ' = {X ' E B},
where X� = (Xn+ l , Xn+2 , . . . , X2n ) and X' = (Xn +l , Xn+2 , . . . , X2n , X l , X2 , · · · , Xn , X2n+l , X2n+2 , · · . ) . Now JP>(An n A�) = JP>(An )lP'(A� ), by independence. Also, JP>(An) = JP>(A�), and therefore
By the exchangeability of A, we have that JP>(A Ll A�) = JP>(A ' Ll A� ), which in turn equals JP>(A Ll An), using the fact that the Xi are independent and identically distributed. Therefore, 1JP>(An n A�) - JP>(A) I :::: JP>(A Ll An) + JP>(A Ll A�)
-+ 0 as n -+ 00.
Combining this with (*), we obtain that JP>(A) = JP>(A) 2 , and hence JP>(A) equals 0 or 1. 5. The value of Sn does not depend on the order of the first n steps, but only on their sum. If Sn = 0 i.o., then S� = 0 i.o. for all walks {S� } obtained from {Sn } by pennutations of finitely many steps. 6. Since f is continuous on a closed interval, it is bounded: If (y ) I :::: c for all y E [0, I] for some c . Furthennore f is uniformly continuous on [0, 1], which is to say that, if E > 0, there exists 8 (> 0), such that I f(y ) - f (z ) 1 < E if I y - zl :::: 8 . With this choice of E, 8, we have that 1JE:(Z/AC) I < E, and x ( 1 x) 1JE:(ZIA) I :::: 2cJP>(A) :::: 2c · n8-2 by Chebyshov's inequality. Therefore
1JE:(Z) I < E + n82c2 ' which is less than 2E for values of n exceeding 2C/(E8 2 ). 7. If {Xn } converges completely to X then, by the first Borel-Cantelli lemma, I Xn - X I > E only finitely often with probability one, for all E > O. This implies that Xn � X; see Theorem (7.2 .4c) . Suppose conversely that {Xn } is a sequence of independent variables which converges almost surely to X. By Exercise (7.2.8), X is almost surely constant, and we may therefore suppose that Xn � c where c E R. It follows that, for E > 0, only finitely many of the (independent) events { I Xn - cl > E } occur, with probability one. Using the second Borel-Cantelli lemma,
L JP> ( I Xn - cl > E ) < 00 . n
8.
( ) -1
Of the various ways of doing this, here is one. We have that n
2
(
)
n n 2 1 1 n X . Xi L Xi Xj = _ L L _ n - 1 n i=l n(n - 1) i = l f l ::;: i <j::;: n 328
Some ancillary results
n- 1
Solutions
n- 1
[7.3.9]-[7.3.12]
Now �1 X � fJ" by the law of large numbers (5. 10.2); hence �1 X � fJ, (use Theorem (7.2.4a» . It follows that �1 X ) 2 � fJ,2 ; to see this, either argue directly or use Problem (7. 1 1 .3). Now use Exercise (7.2.7) to find that
i
(n- 1
i n ) 2 --+ fJ,2 . n ( 1 X i L -n 1 n i=1
i
p
-
Arguing similarly,
n
1 . " Xf � O --....". 1)
n(n - 1�= 1 1
'
and the result follows by the fact (Theorem (7.3.9» that the sum of these two expressions converges in probability to the sum of their limits. 9. Evidently, Xn 1 lP -for < 1 . log n By the Borel-Cantelli lemmas, the events An = {Xn/ 1 occur a.s. infinitely often for O. - 1 < � 0, and a.s. only finitely often for 10. (a) Mills's ratio (Exercise (4.4.8) or Problem (4. 14. 1c» informs us that 1 as -+ 00. Therefore,
( � I + E) = -r-+ ' n €
E
lEI 10gn � + E}
E>
- 1P(x) "" x- 1 cp(x)
x
� .j210gn(1 + E» ) "" J27f 10gn(11+ E)n( 1 +€) The sum over n of these terms converges if and only if E > 0, and the Borel-Cantelli lemmas imply the claim. lP(IXn l
2 .
(b) This is an easy implication of the Borel-Cantelli lemmas. 11. Let X be uniformly distributed on the interval [- 1 , 1], and define Xn I{ X::; ( - I N n j . The dis tribution of Xn approaches the Bernoulli distribution which takes the values ± 1 with equal probability !. The median of Xn is 1 if is even and - 1 if is odd. 12. (i) We have that
n
n
=
x>
for o. The result follows by the second Borel-Cantelli lemma. (ii) (a) The stationary distribution 1C is found in the usual way to satisfy
k-l . 2 k � 2. = -k + 1 7fk - l = . . = k(k + 1) 7fl , Hence tfk = {k(k + 1) }- 1 for k � 1, a distribution with mean �� 1 (k + 1) - 1 = 00. (b) By construction, lP(Xn � Xo + n) = 1 for all n, whence ( Xn � 1 ) = 1 . lP lim sup n--* oo n It may in fact be shown that lP(lim sUPn--* oo Xn /n = 0) = 1 . 7fk
329
[7.3.13]-[7.4.3]
Convergence of random variables
Solutions
13. We divide the numerator and denominator by Jna . By the central limit theorem, the former converges in distribution to the N(O, 1) distribution. We expand the new denominator, squared, as n l I: (Xr na2 r=1
2 I: 1 - 11-) 2 - (X - 11-) 2 . a2 n a 2 (X - 11-) (Xr - 11-) + n
r=1
By the weak law of large numbers (Theorem (5. 10.2), combined with Theorem (7.2.3)), the first term converges in probability to 1 , and the other terms to o. Their sum converges to 1 , by Theorem (7.3.9), and the result follows by Slutsky's theorem, Exercise (7.2.5).
7.4 Solutions. Laws of large numbers
1. Let Sn = X l + X2 + · · · + Xn . Then JE(Sn2 )
n . n2 ", z < = L.J i =2 10g i log n -
and therefore Sn/n � O. On the other hand, � i JP(I Xi i 2: i) = 1 , so that I Xi i 2: i i.o., with probability one, by the second Borel-Cantelli lemma. For such a value of i , we have that l Si - Si - l l 2: i , implying that Sn /n does not converge, with probability one. 2.
Let the Xn satisfy JP(Xn
= -n) = 1 - 21 ' n
JP(Xn
= n 3 - n) = 21 ' n
whence they have zero mean. However,
implying by the first Borel-Cantelli lemma that JP(Xn /n -+ - 1) = 1 . It is an elementary result of real analysis that n - 1 � �= 1 xn -+ - 1 if Xn -+ - 1 , and the claim follows.
3. The random variable N(S) has mean and variance A I S I = cr d , where c is a constant depending only on d. By Chebyshov's ineqUality, JP
(I N(S) - A E) � 2A = ( A ) 2 1d · E 1S1 € cr lSi I 2:
By the first Borel-Cantelli lemma, i l Sk l - 1 N(Sk ) - A I 2: E for only finitely many integers k, a.s., where Sk is the sphere of radius k. It follows that N(Sk ) / I Sk l � A as k -+ 00. The same conclusion holds as k -+ 00 through the reals, since N(S) is non-decreasing in the radius of S.
330
Martingales
Solutions
[7.5.1]-[7.7.1]
7.5 Solutions. The strong law
1. Let Iij be the indicator function of the event that Xj lies in the i th interval. Then
where, for 1 ::::: j ::::: variables with mean
m,
n
n
m
m
L Zm (i ) log Pi = L L Iij log Pi = L Yj j =1 i =1 j =1 i =1 Yj = L:7=1 Iij log Pi is the sum of independent identically distributed n lE ( Yj ) = L Pi log Pi = -h. i =1
10g Rm
=
By the strong law, m - 1 log Rm � - h . 2. The following two observations are clear: (a) N(t) < n if and only if T t, (b) TN (t) ::::: t < TN (t) + l · If ( X 1 ) < 00, then ( T ) < 00, so that
n> lE n lP'(Tn > t) -+ 0 as t -+ 00. Therefore, by (a), lP'(N (t) < n) lP'(Tn > t) -+ 0 as t -+ 00,
lE
=
implying that N(t) � 00 as t -+ 00. Secondly, by (b), Take the limit as t -+ 00,
t/N(t) � lE (X l ).
3. By the strong law, often.
TN (t) < _t_ < TN (t) +1 . ( 1 + N(t) - I ). N(t) - N(t) N(t) + 1 � ( X l ) by the strong law, to deduce that using the fact that
Tn/n
lE
Sn /n � lE (X l ) =1= O. In particular, with probability 1 , Sn
=
0 only finitely
7.6 Solution. The law of the iterated logarithm
1. The sum
Sn is approximately N(O, n), so that -�a lP'(Sn > van log n ) 1 - ( va log n ) < � alogn =
for all large n, by the tail estimate of Exercise (4.4.8) or Problem (4. l4. 1c) for the normal distribution. This is summable if a > 2, and the claim follows by an application of the first Borel-Cantelli lemma. 7.7 Solutions. Martingales
1. Suppose i < j. Then
lE (XjXi) = lE { lE [(Sj - Sj - l ) Xi I So, SI , · · · , Sj-d } = lE { X i [lE (Sj I SO , s} , . . . , Sj-l) - Sj-d } = 0 33 1
[7.7.2]-[7.8.2]
Convergence of random variables
Solutions
by the martingale property. 2. Clearly lE l Sn l < 00 for all n. Also, for n
:::
{ {
0,
( )}
1 - /-tn + l 1 lE(Sn +1 I Zo, Zl , . . . , Zn) = /-tn + l lE(Zn +1 I ZO , . . . , Zn) - m 1 /-t 1 1 - /-tn + l = Sn . = n l m + /-tZn - m 1 - /-t /-t +
(
_
)}
3. Certainly lE l Sn l < 00 for all n. Secondly, for n ::: 1 ,
lE(Sn + l I Xo, Xl , . · · , Xn) = alE(Xn + l I Xo, · . · , Xn) + Xn = (aa + I)Xn + abXn - l , which equals Sn if a = (1 - a) - I . 4. The gambler stakes Zi = fi - l (X l , . . . , X i - } ) on the ith play, at a return of X i per unit. Therefore Si = Si - l + Xi Zi for i ::: 2, with Sl = Xl Y. Secondly, where we have used the fact that Zn + l depends only on X l , X2 , . . . , Xn . 7.8 Solutions.
Martingale convergence theorem
1. It is easily checked that Sn defines a martingale with respect to itself, and the claim follows from the Doob-Kolmogorov inequality, using the fact that
.
n lE(S; ) = L var(Xj) . j=l
2. It would be easy but somewhat perverse to use the martingale convergence theorem, and so we give a direct proof based on Kolmogorov's inequality of Exercise (7.8. 1). Applying this inequality to the sequence Zm , Zm+l , . . where Zi = (Xi - lEXi)/ i, we obtain that Sn = Zl + Z2 + . . . + Zn satisfies, for 0,
E>
We take the limit as r
-+
(
> E) � E; n t l var(Zn).
(n � m
> E) � E1 n L00 l n1 var(Xn).
JP m
E > 0,
o.
Prediction and conditional expectation
Solutions
[7.8.3]-[7.9.4]
is Cauchy convergent, and hence convergent. It follows that Sn converges a.s. to some limit Y. The last part is an immediate consequence, using Kronecker's lemma.
3. By the martingale convergence theorem, S = limn-+ oo Sn exists a.s., and Sn � S. Using Exercise (7.2. lc), var(Sn) -+ var(S), and therefore var(S) = O. 7.9 Solutions. Prediction and conditional expectation
(a) Clearly the best predictors are JE( X I Y) = y 2 , JE(Y I (b) We have, after expansion, that 1.
X) = o.
j,
since JE(Y) = JE(y 3 ) = O. This is a minimum when b = JE(y 2 ) = and a = o. The best linear predictor of X given Y is therefore t . Note that JE(Y I X) = 0 is a linear function of X ; it is therefore the best linear predictor of Y given X. 2. By the result of Problem (4. 14.13), JE(Y I X) = 1-t2 + P CT2 ( X - I-t l )/CTl , in the natural notation. 3. Write n g (a) =
and
�::>i Xi = aX' , i =l
v (a) = JE { (Y - g (a)) 2 } = JE(y 2 ) - 2aJE(YX') + aVa' .
Let Ii be a vector satisfying va' = JE(YX' ) . Then
v (a) - v OO = aVa' - 2alE(YX') + 23JE(YX') - ava' = aVa' - 2ava' + ava' = (a - 3)V(a - 3)' ::: 0, since V is non-negative definite. Hence v(a) is a minimum when a = Ii, and the answer is g OO . If V is non-singular, Ii = JE(YX)V- 1 . 4. Recall that Z = JE(Y I g,) is the ('almost') unique g,-measurable random variable with finite mean and satisfying JE{(Y - Z)IG} = 0 for all E g,. (i) n E g" and hence JE{JE(Y I g,)In} = JE( Z ln ) = JE(Y In). (ii) U = aJE(Y I g,) + ,B JE( Z I g,) satisfies
G
JE(
U IG ) = aJE { JE(Y I g,)IG } + ,B JE { JE( Z I g,)IG }
= aJE(Y IG) + ,BJE(ZIG ) = JE { (a Y + ,B Z)IG } ,
G
Also, U is g,-measurable. (iii) Suppose there exists ( 0) such that = {JE(Y I g,) < - } has strictly positive probability. Then E g" and so JE(Y IG) = JE{JE(Y I g,)IG }. However Y IG ::: 0, whereas JE(Y I g,)IG < We obtain a contradiction on taking expectations. (iv) Just check the definition of conditional expectation. (v) If Y is independent of g" then JE(Y IG ) = JE(Y)lP'(G) for E g,. Hence JE{(Y - JE(Y))IG } = 0 for E g" as required.
G
G
m>
m
G
333
-m.
[7.9.5]-[7.10.1]
Convergence of random variables
Solutions
(vi) If g is convex then, for all a
E JR, there exists J.. (a) such that g(y) :::: g(a) + (y - a ) J.. (a ) ;
furthermore J.. may be chosen to be a measurable function of a. Set a = lE(Y I 9,) and y = Y, to obtain g(Y) :::: g{lE(Y I 9,)} + { Y - lE(Y I 9,) } J.. { lE(Y I 9,)} .
Take expectations conditional on 9" and use the fact that lE(Y I 9, ) i s 9,-measurable. (vii) We have that IlE(Yn I 9,) - lE(Y I 9,) 1 � lE{ l Yn - YI I 9, } � Vn
where Vn = lE { suPm�n I Ym - Y I I 9, } . Now { Vn : n :::: I } is non-increasing and bounded below. Hence V = limn-+ oo Vn exists and satisfies V :::: O. Also
}
{
sup I Ym - YI , lE ( V ) � lE ( Vn ) = lE m�n which tends to 0 as m -+ 00, by the dominated convergence theorem. Therefore lE ( V ) = 0, and hence lP'(V = 0) = 1 . The claim follows.
5. lE(Y I X) = X. 6. (a) Let {Xn n :::: I } be a sequence of members of H which is Cauchy convergent in mean square, that is, lE{IXn - Xm 1 2 } -+ 0 as n -+ 00. By Chebyshov's inequality, {Xn n :::: I } is Cauchy convergent in probability, and therefore converges in probability to some limit X (see Exercise (7.3. 1)). It follows that there exists a subsequence {Xnk k :::: I } which converges to X almost surely. Since each Xnk is 9,-measurable, we may assume that X is 9,-measurable. Now, as n -+ 00, :
m,
:
:
where we have used Fatou's lemma and Cauchy convergence in mean square. Therefore Xn � X. That lE(X 2 ) < 00 is a consequence of Exercise (7.2. la). (b) That (i) implies (ii) is obvious, since IG E H . Suppose that (ii) holds. Any Z (E H) may be written as the limit, as n -+ 00, of random variables of the form
m (n ) Zn = L aj (n)IG j (n) j= l for reals aj (n) and events Gj (n) in 9,; furthermore we may assume that IZn l � IZI . It is easy to see that lE { (Y - M)Zn } = 0 for all n. By dominated convergence, lE{(Y - M)Zn } -+ lE{(Y - M)Z}, and the claim follows. 7.10 Solutions.
Uniform integrability
1. It is easily checked by considering whether Ix I � a or I y I � a that, for a Now substitute x = Xn and y = Yn, and take expectations. 334
> 0,
Uniform integrability
Solutions
[7.10.2]-[7.10.6]
2. (a) Let E > O. There exists N such that lE(I Xn - X n < E if n > N. Now lE l xr l < 00, by Exercise (7.2. 1a), and therefore there exists 8 (> 0) such that lE(I X ( fA) < E, lE(I Xn ( fA) < E for 1 :s n :s N, for all events A such that IP'(A) < 8. By Minkowski's inequality, l l l if n > N {lE ( I Xn l r fA) } / r :s { lE(I Xn - X l r fA) } / r + { lE(I X l r fA) } / r :s 2E 1 / r if lP'(A) < 8. Therefore { I Xn l r : n � I } is unifonnly integrable. If r is an integer then {X� : n � I } is unifonnly integrable also. Also X� � Xr since Xn � X (use the result of Problem (7. 1 1 .3». Therefore lE(X�) -+ lE(xr) as required.
(b) Suppose now that the collection { I Xn l r : n � I } is uniformly integrable and Xn � X. We show first that lEl Xr l < 00, as follows. There exists a subsequence {Xn k : k � I } which converges to X almost surely. By Fatou's lemma, lElX r l
=
( k--+oo
lE lim inf I Xnk l r
)
:s
lim inf lEI X�k l :S sup lEl X� 1 n k--+oo
< 00.
> 0, there exists 8 (> 0) such that for all n, lE(I X r l fA ) < E, lE(I X� l fA ) < E whenever A is such that IP'(A) < 8. There exists N such that Bn (E) = { I Xn - X I > E} satisfies IP'(Bn (E» < 8 for n > N. Consequently n > N, lE(I Xn - X n :s E r + lE ( I Xn - X l r fBn(E» ) , If E
of which the final term satisfies l {lE (IXn - X l r fBn (E») } / r
:s
l {lE (I X� I fBn(E» ) } / r
+ {lE (I X r I fBn (E») } l / r :s 2E 1 / r .
Therefore, Xn � X. 3. Fix E > 0, and find a real number a such that g(x) > X/E if x > a. If b � a, lE (I Xn l f{lXn l >b J ) < ElE{g(I Xn D } :s E sup lE{g( I Xn D } ' n whence the left side approaches 0, unifonnly in n, as b -+ 00. 4.
Here is a quick way. Extinction is (almost) certain for such a branching process, so that Zn � 0, and hence Zn � O. If {Zn : n � O} were unifonnly integrable, it would follow that lE(Zn) -+ 0 as n -+ 00; however lE(Zn) = 1 for all n. 5. We may suppose that Xn , Yn , and Zn have finite means, for all n. We have that 0 :s Yn - Xn :s p Zn - Xn where, by Theorem (7.3.9c), Zn - Xn --+ Z - X. Also lE l Zn - Xn l = lE(Zn - Xn) -+ lE(Z - X) = lEl Z - XI , so that {Zn - Xn : n :::: I } is unifonnly integrable, by Theorem (7. 1 0.3). It follows that {Yn - Xn : n � I } is unifonnly integrable. Also Yn - Xn � Y - X, and therefore by Theorem (7. 10.3), lEl Yn - Xn I -+ lE l Y - X I , which is to say that lE(Yn ) - lE(Xn) -+ lE(Y) - lE(X) ; hence lE(Yn ) -+ lE(Y) . It i s not necessary to use uniform integrability; try doing it using the 'more primitive' Fatou's lemma. 6. For any event A, lE(IXn lfA) :s lE(ZfA) where Z = sUPn I Xn l . The uniform integrability follows by the assumption that lE(Z) < 00.
335
[7.11.1]-[7.11.2]
Convergence of random variables
Solutions
7. 1 1 Solutions to problems
1. lEIX� I = 00 for r 2: 1 , so there is no convergence in any mean. However, if E > 0,
lP'(IXn l > E ) = 1 - -2 tan - 1 (n E ) --+ 0 11:
as n
--+ 00,
p
so that Xn -r O. You have insufficient information to decide whether or not Xn converges almost surely: (a) Let X be Cauchy, and let Xn = X no Then Xn has the required density function, and Xn (b) Let the Xn be independent with the specified density functions. For > 0,
/
E
� O.
E
E)
so that L:n lP'(IXn l > = 00. By the second Borel-Cantelli lemma, IXn l > i.o. with probability one, implying that Xn does not converge a.s. to O. 2. (i) Assume all the random variables are defined on the same probability space; otherwise it is meaningless to add them together. (a) Clearly Xn (w) + Yn (w) --+ X(w) + Y(w) whenever Xn (w) --+ X(w) and Yn (w) --+ Y(w). Therefore { Xn + Yn ,.. X + Y} � {Xn ,.. X} U {Yn ,.. Y}, a union of events having zero probability. (b) Use Minkowski's inequality to obtain that
{lE (IXn + Yn - X - y n } l / r .::: { lE(IXn - X n } l / r + {lE(I Yn - y n } l / r . (c) If
E > 0, we have that { IXn + Yn - X - YI > E }
�
{ IXn - XI > iE} U { I Yn - YI > iE} ,
and the probability of the right side tends to 0 as n
--+ 00. (d) If Xn S X and the Xn are symmetric, then -Xn S X. However Xn + (-Xn) S 0, which generally differs from 2X in distribution.
(ii) (e) Almost-sure convergence follows as in (a) above. The corresponding statement for convergence in rth mean is false in general. Find a random variable Z such that lElZ r l < 00 but lEIZ 2r l = 00, and define Xn = Yn = Z for all n. (t)
(g) Suppose Xn
p p -r X and Yn -r Y. Let E > O. Then
lP'(IXn Yn - XYI > E ) = lP' ( I (Xn - X)(Yn - Y) + (Xn - X)Y + X(Yn - Y) I > E ) .::: lP'(IXn - XI · I Yn - YI > 1 E ) + lP'(IXn - XI · IYI > 1 E ) + lP'(IXI · I Yn - YI > j E ) . Now, for
8 > 0,
lP'(IXn - XI · IYI >
1 E) .::: lP'(IXn - XI > E/ (38») + lP'(I YI > 8), 336
Problems
Solutions
[7.11.3]-[7.11.5]
which tends to 0 in the limit as n -+ 00 and 8 -+ 00 in that order. Together with two similar facts, we obtain that Xn Yn � XY. (h) The example of (d) above indicates that the corresponding statement is false for convergence in distribution. 3. Let E O. We may pick M such that 1P' ( I XI � M) :::: E. The continuous function g is uniformly continuous on the bounded interval [- M, M]. There exists 8 0 such that >
>
If I g(Xn) - g(X) 1
Ig(x) - g(y) 1 :::: E if Ix - y l :::: 8 and I x l :::: M.
>
E, then either I Xn - X I
1P'(l g(Xn) - g(X) 1
in the limit as n -+ 4. Clearly
00.
>
>
E ) :::: 1P'(I Xn - XI
8 or IX I >
�
M. Therefore
8) + 1P'(I X I � M )
-+
1P'(I X I
�
M) :::: E,
It follows that g(Xn) � g(X).
lE(e i t Xn )
=
n
II lE(e i t Yj /IOJ. )
J=l .
1 - eit
=
n
II
J =l .
1
{ -I .
j
1 - e i t / l O -1 . 10 1 e l. t / l OJ
eit
_
}
- I On ( l - e i t / 10" ) it as n -+ 00. The limit is the characteristic function of the uniform distribution on [0, 1 ] . Now Xn :::: Xn + l :::: 1 for all n, so that Y(w) Iimn� oo Xn (w) exists for all w. Therefore Xn � Y; hence Xn S Y, whence Y has the uniform distribution. 5. (a) Suppose s < t. Then -
-+
_
----
=
lE(N(s)N(t)) = lE(N(s f ) + lE{ N (s)(N(t) - N (s)) } = lE(N(s) 2 ) + lE(N(s))lE(N(t) - N (s) ) ,
since N has independent increments. Therefore cov(N (s), N (t))
=
=
lE (N(s)N (t) ) - lE(N(s))lE(N(t)) (As) 2 + AS + AS{A(t - s)} - (AS)(>.. t ) = AS.
In general, cov(N (s), N (t)) = A min{s, t}. (b) N (t + h) - N(t) has the same distribution as N (h), if h O. Hence >
o.
which tends to 0 as h -+ (c) By Markov's inequality, 1P'(IN (t + h) - N(t) 1 which tends to 0 as h -+ 0, if E O. (d) Let E O. For O < h < E - 1 , + IP' N(t h - N(t)
>
>
>
(
I
�
I E) >
E ) :::: 1 lE ( {N(t + h) - N (t) } 2 ) , E2
=
IP'(N(t + h) - N(t ) 337
�
1 ) = Ah + o(h),
[7.11.6]-[7.11.10]
which tends to ° as h -+ 0. On the other hand, JE
which tends to 00 as h + 0.
6.
Convergence of random variables
Solutions
( { N(t + �
By Markov ' s inequality,
-
N (t)
f) = h12 { (Ah)2 + Ah }
Sn = 2:i= l Xj satisfies
for £
> 0. Using the properties of the X ' s,
since
JE
(Xj ) = ° for all i . Therefore there exists a constant C such that
implying (via the first Borel-Cantelli lemma) that n - 1 Sn � 0. 7. We have by Markov ' s inequality that
< 00 for £
> 0, so that Xn � X (via the first Borel-Cantelli lemma).
8. Either use the Skorokhod representation or characteristic functions. Following the latter route, the characteristic function of a Xn + b is
where 0, £ 1+ IXn l IXn JP'(I Xn l > £) = JP' 1 + IXnl > 1 + £ � £ £ . JE 1 + I Xn l l
Set c
=
)
(
338
-- (
)
Problems
[7.11.11]-[7.11.14]
Solutions
by Markov ' s inequality. If this expectation tends to 0 then Xn � O. Suppose conversely that Xn
lE as n
(
I Xn l 1 + I Xn l
)
p
-+
O. Then
E
1 +E
::::
--
. JP>( I Xn l ::::
E) + 1 · JP> ( I Xn l > E)
-+
E
1 +E
--
-+ 00, for E > O. However E is arbitrary, and hence the expectation has limit O.
11. (i) The argument of the solution to Exercise (7.9.6a) shows that {Xn } converges in mean square if it is mean-square Cauchy convergent. Conversely, suppose that Xn � X. By Minkowski ' s inequality,
as m, n -+ 00, so that {Xn } is mean-square Cauchy convergent. (ii) The corresponding result is valid for convergence almost surely, in rth mean, and in probability. For a.s. convergence, it is self-evident by the properties of Cauchy-convergent sequences of real numbers. For convergence in probability, see Exercise (7.3. 1). For convergence in rth mean (r ::: 1), just adapt the argument of (i) above. 12. If var(Xi ) :::: M for all i , the variance of n - 1 2:: 7= 1 Xi is n M 1 2" L var(Xi ) :::: -+ 0 as n -+ 00. n n i =l -
13. (a) We have that
If x :::: 0 then F (anx) n -+ 0, so that H (x) - log H (x)
=
=
O. Suppose that x
- nlim n log [ l - ( 1 - F (anx)) ] �oo
{
}
> =
o.
Then
nlim �oo { n ( l - F (anx)) }
since -y - 1 log(1 - y) -+ 1 as y .J, O. Setting x = 1 , we obtain n ( 1 - F (an)) -+ - log H(I), and the second limit follows. (b) This is immediate from the fact that it is valid for all sequences {an } . (c) We have that 1 - F(tex +Y ) 1 - F(t)
----- =
1 - F(tex +y ) . 1 - F (teX ) log H (eY) . log H (eX ) --=---+ 1 - F(te X ) 1 - F(t) log H ( 1 ) log H ( I )
a s t -+ 00. Therefore g (x + y) = g (x)g(y) . Now g i s non-increasing with g (O) = 1 . Therefore g(x) = e -fJ x for some ,8 , and hence H (u) = exp(-au -fJ ) for u > 0, where a = - log H ( 1 ) .
14.
Either use the result of Problem (7. 1 1 . 1 3) or d o the calculations directly thus. We have that JP> ( Mn :::: xn/rr ) =
if x
{� � +
> 0, by elementary trigonometry. JP> ( Mn :::: xn/rr )
=
(
tan -
I
c;) r
Now tan - I y
=
{ � 1-
tan - l
(:J r
y + o(y) as y -+ 0, and therefore n 1 -+ e - 1 /x 1as n -+ 00. + 0(n - 1 ) xn
339
=
)
[7.11.15]-[7.11.16] 15.
Convergence of random variables
Solutions
The characteristic function of the average satisfies
as n � By the continuity theorem, the average converges in distribution to the probability also.
16.
(a) With Un
00.
constant IL ,
and hence in
= u (xn ) , we have that
I E(u(X)) - E(u (Y)) 1
:s
L IUn l · l fn - gn l n
if lI u li oo :s 1 . There is equality if Un equals the sign of fn Problem (2.7. 1 3) and Exercise (4. 1 2.3). (b) Similarly, if lI u li oo :s 1 ,
I E(u(X)) - E(u(Y)) 1
:s
i:
with equality if u (x) is the sign of f(x)
l u (x) 1
:s
- gn.
. If(x) - g(x) 1 dx :s
- g(x).
u (x)
o.
=
The second equality holds as in
i: If(x) - g(x) 1 dx
Secondly, we have that
IIP'(X E A) - 1P'(Y E A) I = � I E(u(X)) - E(u(Y)) 1 where
L I fn - gn l n
{I
-1
:s
�dTV(X, Y),
if x E A, if x ¢ A .
Equality holds when A = {x E JR : f(x) � g(x)}. (c) Suppose dTV(Xn , X) � Fix a E JR, and let u b e the indicator function o f the interval (-00, a]. Then IE(u(Xn)) - E(u(X)) 1 = IIP'(Xn :s a) - 1P'( X :s a) l , and the claim follows.
On the other hand, if Xn = n - 1 with probability one, then Xn .s O. However, by part (a), dTV(Xn , O) = 2 for all n. (d) This is tricky without a knowledge of Radon-Nikodym derivatives, and we therefore restrict ourselves to the case when X and Y are discrete. (The continuous case is analogous.) As in the solution to Exercise (4. 1 2.4), IP'(X '# Y) � �dTV(X, Y). That equality is possible was proved for Exercise (4. 1 2.5), and we rephrase that solution here. Let ILn = min{fn , gn} and IL = I:n ILn , and note that dTV(X, Y) = L I fn - gn l = L{fn + gn - 21Ln } = 2 ( 1 - 1L) · It is easy to see that
n
n
1 '1dTV(X, Y) = IP'(X '# Y) =
U,
{I
if IL 0 if IL
= 0, = 1,
IL < 1 . Let V, W be random variables with mass functions max{fn - gn , O} , - min{fn - gn , O} , IP'( U - Xn ) - ILn , lP'(V - Xn ) IP'(W - Xn ) l - IL l - IL IL and let Z be a Bernoulli variable with parameter IL , independent of (U, V, W). We now choose the pair X', Y ' by if Z = 1 , (X' , Y ' ) = (V, W) if Z = O. and therefore we may assume that 0 < _
_
{ (U, U) 340
_
_
Problems
Solutions
It
[7.11.17]-[7.11.19]
may be checked that X ' and Y ' have the same distributions as X and Y, and furthermore, lI"(X ' =I Y ' ) = lI"(Z = 0) = 1 - J1, = dTV (X, Y). (e) By part (d), we may find independent pairs (X , Y ) , 1 :::: i :::: n, having the same marginals as (Xi , Yi ), respectively, and such that ll"(X =I- Y ) = � dTV (Xi ' Yi ) . Now,
i
�
[
� [
17. If X l , X2 , . . . are independent variables having the Poisson distribution with parameter A, then Sn = X l + X2 + . . . + Xn has the Poisson distribution with parameter An. Now n - I Sn � A, so that E(g(n - l Sn)) --+ g(A) for all bounded continuous g. The result follows. 18. The characteristic function o/mn of - (Ym - m) Umn = (Xn - n) r.:::;-;-; v m + n; satisfies log o/m n (t )
=
n (e i t f./m+n - 1 ) + m (e - i t f./m+n - 1 ) + (m - n)it --+ _ l2 t 2 ../m + n
as m, n --+ 00, implying by the continuity theorem that distributed with parameter m + n, and therefore V.
mn -
J
Xn + Ym m +n
P �
Umn � N(O, 1 ) . Now Xn + Ym is Poisson as m, n
1
--+ 00
by the law oflarge numbers and Problem (3). It follows by Slutsky ' s theorem (7.2.5a) that Umn / Vmn N(O, 1) as required.
19.
(a) The characteristic function of Xn is ¢n (t )
=
exp{i J1,n t
- i O'; t 2 }
D ---+
where J1,n and a; are 1
2
the mean and variance of Xn . Now, limn-+ oo ¢n ( 1 ) exists. However ¢n ( 1 ) has modulus e - 20'n , and therefore 0' 2 = limn-+ oo a; exists. The remaining component e i J.Ln t of ¢n (t) converges as n --+ 00, say e i J.Ln t --+ O(t) as n --+ 00 where O (t) lies on the unit circle of the complex plane. Now 1
2 2
¢n (t ) --+ 0 (t)e - 20' t , which is required to be a characteristic function; therefore 0 is a continuous function of t. Of the various ways of showing that O(t) = e i J.Lt for some J1" here is one. The sequence o/n (t) = e i J.Ln t is a sequence of characteristic functions whose limit O(t) is continuous at t = 0. Therefore 0 is a characteristic function. However 0/n is the characteristic function of the constant J1,n , which must converge in distribution as n --+ 00; it follows that the real sequence {J1,n } converges to some limit J1" and O(t) = e i J.Lt as required . This proves that ¢n (t ) --+ exp{ i J1,t - � O' 2 t 2 }, and therefore the limit X is N(J1" 0' 2 ). (b) Each linear combination s Xn + t Yn converges in probability, and hence in distribution, to s X + t Y . Now s Xn + t Yn has a normal distribution, implying by part (a) that s X + t Y i s normal. Therefore the joint characteristic function of X and Y satisfies
¢X, Y (s, t) = ¢s x+ t y ( 1 ) = exp { i E (s X + tY) - � var(s X + tY) } 2 2 = exp { i (s J1, X + t J1, y ) - � (s O'i + 2stPxyO'xO'y + t O' ¥ ) } 341
[7.11.20]-[7.11.21]
Convergence of random variables
Solutions
in the natural notation. Viewed as a function of s and t, this is the joint characteristic function of a bivariate normal distribution. When working in such a context, the technique of using linear combinations of Xn and Yn is sometimes called the 'Cramer-Wold device ' . 20. (i) Write Yj = Xi - lE(Xj ) and Tn = 2: 1= 1 Yi . It suffices to show that n - 1 Tn � O . Now, as n --+ 00, n 1 2 nc lE(Tn2 In 2 ) = 2" L var(Xi ) + 2" L COV(Xi , Xj ) :5 2" --+ o. n i= l n 1::5.i <j::s.n n
E if I i - j l
(ii) Lett:" > O. There exists I such that I p (Xj , Xj ) 1 :5 n
L
i ,j= l
cov(Xj , Xj ) :5
L
l i - j l::s.! 1 ::s. j, j 5 n
cov(Xj , Xj ) +
L
l i -j l > ! 1 ::s. i ,j5 n
� I . Now
COV(Xi , Xj ) :5
E
2nlc + n 2 c ,
since COV(Xi , Xj ) :5 I p (Xi ' Xj ) l ..jvar(Xi ) · var(Xj ) . Therefore,
2Ic n
lE(Tn2 In 2 ) :5 - + This is valid for all positive
21.
The integral
EC --+ EC
as n
--+ 00.
E, and the result follows. r oo
J2
C _ dx x log Ix l
__
diverges, and therefore lE(X 1 ) does not exist. The characteristic function ¢ of X 1 may be expressed as A.
'f' (t ) whence
¢ (t) - ¢ (O)
2c
Now 0
:5
Now
and
Therefore
=
2C
hoo 2
oo _ r
=
1 - cos e :5 min{2, e 2 }, and therefore
I
¢ (t) - ¢ (O)
2c
100 U
I 0, n � 1, 1P' ( I X Nk - X I > E ) :::: IP'(Nk :::: :::: IP'(Nk ::::
n) + 1P' ( I XNk - X I > E, Nk > n ) n) + 1P'(Yn > E) � IP'(Yn > E)
Now take the limit as n � 00 and use the fact that Yn � 0. 26. (a) We have that
k . k . a (n - k , n ) = II < exp - L !... 1 !... n a(n + 1 , n) i=O i =O n
( ) 344
-
(
)
.
as k � 00.
Problems
Solutions
[7.11.27]-[7.11.28]
( )
(b) The expectation is
i n En = L g j"I�nn n �� J. . J where the sum is over all j satisfying n - M .;n :::: j :::: n . For such a value of j , e -n n i H _ n i j - n n i e -n = g j! (j - I)! ' .;n .;n j ! whence En has the form given. (c) Now g is continuous on the interval [- M, 0], and it follows by the central limit theorem that
)
(
( )
Also,
e -n e -n e -n -k2 /(2n) a(n + 1 , n) a(n + 1 , n) :::: En + a(n - k, n) :::: En + .;n .;n .;n where k = L M .;nJ . Take the limits as n � 00 and M � 00 in that order to obtain 1 < lim n n+ ! e -n < -1 -- $. $ - n � oo n! En ::::
{ I }
27. Clearly
Rn E(RnH I Ro, R l , " " Rn) = Rn + - n+2 since a red ball is added with probability Rn/(n + 2) . Hence E(SnH I Ro, R l , " " Rn) = Sn, and also 0 :::: Sn :::: 1 . Using the martingale convergence theorem, S = limn � oo Sn exists almost surely and in mean square.
28. Let 0
< E < 1, and let
k(t) = Let J , met) =
ro
- E 3 )k(t) 1 , net) = L (1 + E 3 )k(t)J
and let Imn (t) be the indicator function of the event {m(t) :::: M(t) < net)}. Since M(t)/t � e, we may find T such that E(lmn (t)) 1 - E for t � T. We may approximate SM (t) b y the random variable Sk (t) a s follows. With Ai = { lSi - Sk (t) I
>
>
E..jk(t) },
lP' ( AM (t) ) :::: lP' (AM (t) , Imn (t) = 1 ) + lP'(AM (t) , Imn(t) = 0) n (t) - l k (t) - l :::: lP' U Ai + lP' U Ai + lP'(Imn (t) = O) j=m (t) i=k (t) 2 {n(t) - k(t) }0' 2 {k(t) m(t) }0' +E + -< E 2 k(t) E 2 k(t) :::: E(1 + 20' 2 ), if t � T,
(
) ( 345
)
[7.11.29]-[7.11.32]
Convergence of random variables
Solutions
by Kolmogorov ' s inequality (Exercise (7.8. 1) and Problem (7. 1 1 .29» . Send t --+ 00 to find that
Dt Now Sk (t ) / Jk(t) S N(O,
= SM(t ) - Sk (t ) � 0 Jk(t)
as t --+ 00.
(J' 2 ) as t --+ 00, by the usual central limit theorem. Therefore
which implies the first claim, since k(t)/«()t) --+ 1 (see Exercise (7.2.7» . The second part follows by Slutsky ' s theorem (7.2.5a). 29. We have that
Sn = Sk + (Sn - Sk ), and so, for n ::: k,
Now St JA k ::: c2 IA k ; the second term on the right side is 0, by the independence of the X ' s, and the third term is non-negative. The first inequality of the question follows. Summing over k, we obtain lE(S� ) ::: c2 1P'(Mn c) as required. 30. (i) With
>
Sn = LI=1 Xi , we have by Kolmogorov ' s inequality that IP'
E>
(
l �k � n ISm +k - Sm l max
)
m +n
> E � E1 k=m 2: lE(X� ) 2"
for O. Take the limit as m, n --+ 00 to obtain in the usual way that {Sr : r ::: O} is a.s. Cauchy convergent, and therefore a.s. convergent, if Lj'" lE(X� ) < 00. It is shorter to use the martingale convergence theorem, noting that Sn is a martingale with uniformly bounded second moments. (ii) Apply part (i) to the sequence Yk = Xk /bk to deduce that L� 1 Xk /bk converges a.s. The claim now follows by Kronecker ' s lemma (see Exercise (7.8.2» . 31. (a) This is immediate by the observation that
eA (P) - fXo
IIi, j PijNij '
/
Lj Pij = 1 for each i , and we introduce Lagrange multipliers { ILi i E S} and write V = i,.(P) + Li ILi Lj Pij ' Differentiating V with respect to each Pij yields a stationary (maximum) value when (Nij / Pij ) + ILi = O. Hence Lk Nik = - ILk > and :
(b) Clearly
- --
pI.J. - - Nij - LNij . ILi k Nik �
E Nk
(c) We have that Nij = L \ i Ir where lr is the indicator function of the event that the rth transition out of i is to j . By the Markov property, the Ir are independent with constant mean Pij . Using the strong law of large numbers and the fact that Lk Nik � 00 as n --+ 00,
Pi) � lE(h) = Pij . I 32. (a) If X is transient then Vi (n ) < 00 a.s., and ILi = 00, whence Vi (n)/n � 0 = ILI . If X is persistent, then without loss of generality we may assume X0 = i . Let T (r) be the duration of the rth 346
Problems
Solutions
[7.11.33]-[7.11.34]
excursion from i . By the strong Markov property, the T (r) are independent and identically distributed with mean iJ-i . Furthermore,
By the strong law of large numbers and the fact that Vi (n) � 00 as n --+ 00, the two outer terms sandwich the central term, and the result follows. (b) Note that L�:J ! (Xr ) = L i eS ! (i ) Vi (n). With Q a finite subset of S, and 1!i = iJ-i 1 , the unique stationary distribution,
1:
---" r=v
!(Xr ) n
-
I: . I
�
!( ) iJ-1
I: ( I:::: {I:e I
=
. I
i Q
r;. Q
-
Vj (n) n
-
S}. The sum over i
where Il f ll oo = sup{ I ! (i ) 1 : i E The other sum satisfies
I:i (
Vi (n) n
�) = 2
Vi (n) + n iJ-1
�) �I iJ-1
which approaches 0 a.s., in the limits as n --+ 00 and
+
i r;. Q
� ) } II ! l I oo ,
Vi (n) + n iJ- 1
Q converges a.s. to 0 as n --+
E -
1 I: (
! (i )
iJ-1
I:i e ( Q
Vi (n) + 1!i n
Q t S.
00, by part (a).
)
33. (a) Since the chain is persistent, we may assume without loss of generality that Xo = j . Define the times R l , R2 , . . . of return to j, the sojourn lengths S I , S2 , . . . in j , and the times VI , V2 , . . . between visits to j . By the Markov property and the strong law of large numbers,
= -n � -Rn L...J Vr n 1
1
r= 1
Also,
Rnl Rn + l
� 1 , since iJ-j
�.
Rn + 1
= lE(R l )
L�= 1 Sr n V; � L."r= 1 r
< -
� t
0) = IP'(N(x) > 0) + IP'({N(x) = OJ n {N(x + y) - N(x) > OJ) ::::: g(x) + g(y) for x, y � O. 349
[8.3.1]-[8.3.4]
Random processes
Solutions
Such a function g is called subadditive, and the existence of A follows by the subadditive limit theorem discussed in Problem (6. 15. 14). Note that A = 00 is a possibility. Next, we partition the interval (0, 1 ] into n equal sub-intervals, and let In (r) be the indicator function of the event that at least one arrival lies in ( r - l)ln, r In , 1 :s r :s n . Then E�=l In(r) t N(I) as n -+ 00, with probability 1 . By stationarity and monotone convergence,
]
)
(
)
(
� In (r) = lim E � In(r) = lim ng(n - 1 ) = A . E (N ( I )) = E nlim n -> oo r=l n -> oo L.J L.J -> oo r=l 8.3 Solutions.
Renewal processes
1.
See Problem (6. 1 5.8).
2.
With X a certain inter-event time, independent of the chain so far,
{
Bn + l = X - 1
Bn - 1
if Bn if Bn
= 0,
> 0.
>
Therefore, B is a Markov chain with transition probabilities Pi , i - l = 1 for i 0, and POj = /j + l for j ::: 0, where In = IP'(X = n) . The stationary distribution satisfies IT} = JIj + l + 7rO /j + l , j ::: 0, with solution 7rj = IP'(X j)/E(X), provided E(X) is finite. The transition probabilities of B when reversed in equilibrium are
>
Pi ,H l _
=
>
7rH l IP'(X i + 1 ) , - = IH I , PiO IP'(X i) ---;r;- = IP'(X i)
>
>
for i ::: 0.
These are the transition probabilities of the chain U of Exercise (8.3 . 1 ) with the /j as given.
We have that p n u n = E �=l p n -k u n _ k p k /k , whence Vn = p n un defines a renewal sequence provided p ° and E n p n In = 1 . By Exercise (8.3. 1 ), there exists a Markov chain U and a state s such that Vn = IP' (Un = s) -+ 7rs , as n -+ 00, as required.
3. 4.
>
Noting that N (0)
= 0,
00 00 r 00 00 L E ( N (r )) s r = L L u k S r = L U k L s r r=O r=l k=l k=l r=k k U k S = U (s) - 1 = F (s ) U (s ) . =� L.J l l -s l -s k=l - s Let Sm = EZ'= l Xk and So = 0 . Then IP'(N(r) = n) = IP'(Sn :s r) - 1P'(Sn + l :s r), and
� SIE [(N(t)k + k)] = � SI � (n +k k) (IP'(Sn 00
00
00
00
[
= L/ 1= 0
1+
00
L n=l
Now,
350
(
n +k k_
:s
�1
t) - 1P'(Sn + l :s t))
) IP'(Sn t)] :s
.
Queues
Solutions
[8.3.5]-[8.4.4]
[ (N(t) ) ] = (1 - s)(II- F(s))k = U(S)l - sk ·
whence, by the negative binomial theorem,
00
�
5.
t
S JE
+k
k
This is an immediate consequence of the fact that the interarrival times of a Poisson process are exponentially distributed, since this specifies the distribution of the process. 8.4 Solutions.
Queues
1. We use the lack-of-memory property repeatedly, together with the fact that, if X and Y are independent exponential variables with respective parameters "A and /-t, then JP'(X < Y) = "A/("A + /-t) . (a) In this case,
I P=2 (b) If "A
{
"A /-t /-t . + "A + /-t "A + /-t "A + /-t
} I{ +
/-t
.
"A
2 "A + /-t "A + /-t
< /-t, and you pick the quicker server, p = I
_
+
(_/-t_ ) 2 .
"A "A + /-t
"A + /-t 2"A/-t (c) And finally, p = . ("A + /-t) 2. The given event occurs if the time X to the next arrival is less than Y of service of the customer present. Now,
2
3.
t,
}=I
2
+
2"A/-t ("A + /-t)
2.
and also less than the time
By conditioning on the time of passage of the first vehicle,
JE(T) =
loa (x + JE T) ) e -AX dx + ae -Aa , "A
(
and the result follows. If it takes a time b to cross the other lane, and so a + b to cross both, then, with an obvious notation, (a) (b)
eaA I + e b/L - I , JE ( Ta ) + JE ( Tb ) = -/-t "A A + ( ) )( + b /L a e -I JE(Ta+b ) = "A + /-t _
--
----
The latter must be the greater, by a consideration of the problem, or by turgid calculation.
4.
Look for a solution of the detailed balance equations /-trrn+ l
+ 1) = "A (n rrn , n+2
n ::: O.
to find that rrn = p n rro/(n + l ) is a stationary distribution ifp < I , in which case rro = -pj log(l - p) . Hence L:: n nrrn = "Arro/(/-t - "A ) , and b y the lack-of-memory property the mean time spent waiting for service is prro/(/-t - "A). An arriving customer joins the queue with probability
351
[8.4.5]-[8.5.5]
Random processes
Solutions
5. By considering possible transitions during the interval (t, t + h), the probability Pi (t) that exactly i demonstrators are busy at time t satisfies: P2 (t + h) = PI (t)2h + P2 (t)( 1 - 2h) + o(h), PI (t + h) = po(t)2h + PI (t)(1 - h)(1 - 2h) + P2 (t)2h + o(h), Po(t + h) = po(t)( 1 - 2h) + PI (t)h + o(h).
Hence,
p� (t) = 2Pl (t) - 2P2 (t), P I (t) = 2po(t) - 3 Pl (t) + 2p2 (t), po et) = -2po (t) + PI (t), and therefore P2 ( t ) = a + be - 2 t + ce - 5 t for some constants a, b, c. By considering the values of P2 and its derivatives at t = 0, the boundary conditions are found to be a + b + c = 0, -2b - 5c = 0, 4b + 25c = 4, and the claim follows. 8.5 Solutions. The Wiener process
We might as well assume that W is standard, in that u 2 = 1 . Because the joint distribution is multivariate normal, we may use Exercise (4.7.5) for the first part, and Exercise (4.9. 8 ) for the second, giving the answer 1 1 . . -1 . -1 - + sm - . - + - sm - 1 - + sm 8 4n t u u 1.
{ V;
Jr
Jr}
Writing W(s) = .fix, Wet) = ..jiz, and W(u) = ,J'UY, we obtain random variables X, Y, Z with the standard trivariate normal distribution, with correlations PI = ...jSfU, P2 = .filli, P3 = .fiTi. By the solution to Exercise (4.9.9),
2.
var(Z I X, y)
- s) , = (u - t)(t t(u - s)
W(s), W(u)) as required. Also, (u - t)W(s) + (t - S)W(U) W(u) W(s), W(v) , lE ( W(t)W(u) I W(s), W(v)) = lE u s
yielding var( W (t) I
{[
_
which yields the conditional correlation after some algebra.
3. 4.
5.
Whenever a 2 + b 2
]
I
= 1. Let Llj (n) = W«j + l)t/n) - W(jt/n). By the independence of these increments,
They all have mean zero and variance ments.
}
t, but only (a) has independent normally distributed incre
352
Problems
Solutions
[8.7.1]-[8.7.3]
8.7 Solutions to problems
1. lE(Yn ) = 0, and cov(Ym , Ym+n ) = Ei=o ajan+j for m , n
:::: 0, with the convention that ak = 0 > r. The covariance does not depend on m, and therefore the sequence is stationary. m+ 1 Yn m l where Sn (m) = Ej=o a j Zn j . There 2. We have, by iteration, that Yn = Sn(m) + a - are various ways of showing that the sequence {Sn (m) : m :::: I } converges in mean square and almost m+! surely, and the shortest is as follows. We have that a Yn -m - l --+ 0 in m.s. and a.s. as m --+ 00; to see this, use the facts that var(a m+! Yn - m - l ) = a 2 (m+ 1 ) var(Yo), and
for k
E > O. It follows that Sn (m) = Yn - a m+ 1 Yn -m - l converges in m.s. and a.s. as m --+ 00. A longer route to the same conclusion is as follows. For r < s,
whence {Sn(m) : m :::: I } i s Cauchy convergent in mean square, and therefore converges in mean square. In order to show the almost sure convergence of Sn (m), one may argue as follows. Certainly
whence E�o a j Zn - j is a.s. absolutely convergent, and therefore a.s. convergent also. We may express limm ....Hlo Sn (m) as E�o a j zn - j . Also, a m+! Yn _m _ l --+ 0 in mean square and a.s. as m --+ 00, and we may therefore express Yn as 00
Yn = E a j Zn - j j=o It follows that lE(Yn) is given by
a.s.
= limm �oo lE(Sn (m)) = O. Finally, for r > 0, the autocovariance function
c(r) = cov(Yn , Yn -r ) = lE{ (aYn - l + Zn)Yn - r } = ac(r - 1),
whence
since c(O)
= var(Yn).
a lr l 2 ' c(r) = a l r l c(O) = -I -a
r=...
,
-1 0 ,
,
1, . . . ,
3. Ift is a non-negative integer, N(t) is the number of O ' s and 1 ' s preceding the (t + l )th 1 . Therefore N(t) + 1 has the negative binomial distribution with mass function
k :::: t + 1 . If t is not an integer, then N(t)
= N(ltJ ). 353
[8.7.4]-[8.7.6] 4.
Random processes
Solutions
We have that lP ( Q (t + h)
= j I Q (t) = i ) =
{
if j A h + o (h ) if j /1-ih + o(h) 1 - (A + /1- i)h + o(h) if j
= i + 1, = i - I, = i,
an immigration-death process with constant birth rate A and death rates /1- i = i /1-. Either calculate the stationary distribution in the usual way, or use the fact that birth-death processes are reversible in equilibrium. Hence Ani = /1- (i + l)ni + l for i ::: 0, whence
-( )
i n.I = i1! -A e -J.. //L ' /1-
i
::: 0.
We have that X (t) = R cos (Ill ) cos(Ot) - R sin(llI) sin(Ot). Consider the transformation u = r cos 1{1, v = -r sin 1{1, which maps [0, (0) [0, 2n) to R2 . The Jacobian is au au ar a1{l av av = -r, ar a1{l whence U = R cos IV , V = -R sin III have joint density function satisfying 5.
x
rfu, v (r cos 1{1, -r sin 1{1) = fR , IJ/ (r, 1{1 ) . Substitute
1 ( 2 2) fu , v (u , v) = e - Z + V /(2n), to obtain U
r Thus
> 0, ° � 1{1 < 2n.
R and IV are independent, the latter being uniform on [0, 2n).
A customer arriving at time u i s designated green i f he i s i n state A at time t, an event having probability p(u, t - u). By the colouring theorem (6. 1 3 . 14), the arrival times of green customers form a non-homogeneous Poisson process with intensity function A (U)p(U , t - u), and the claim follows. 6.
354
9 Stationary processes
9.1 Solutions. Introduction 1.
We examine sequences Wn of the form Wn
for the real sequence {ak : k � with solution
=
00
L:: ak Zn -k
k=O
OJ. Substitute, to obtain ao = 1, a l = a, ar = aar - l + f3ar -2 , if a 2 + 4f3
=
r
� 2,
0,
otherwise, where )'1 and A 2 are the (possibly complex) roots of the quadratic x 2 - ax - f3 = 0 (these roots are distinct if and only if a 2 + 4f3 #- 0). Using the method in the solution to Problem (8.7.2), the sum in (*) converges in mean square and almost surely if IA l l < 1 and IA 2 1 < 1 . Assuming this holds, we have from (*) that E(Wn ) = 0 and the autocovariance function is c(m)
=
E(Wn Wn - m )
=
ac(m -
1) + f3c(m - 2) ,
m�
1,
by the independence of the Zn . Therefore W is weakly stationary, and the autocovariance function may be expressed in terms of a and f3 . 2.
We adopt the convention that, if the binary expansion of U i s non-unique, then w e take the (unique) non-terminating such expansion. It is clear that Xi takes values in {O, I } , and lP' ( Xn+ l
=
1 1 Xi = Xi for 1 � i � n) = �
for all X l , x2 , . . . , Xn ; therefore the X ' s are independent Bernoulli random variables. For any se quence k l < k2 < . . . < kr , the joint distribution of Vkj ' Vk2 ' . . . , Vkr depends only on that of Xkj + l ' Xkj +2 , . . . . Since this distribution is the same as the distribution of X l , X2 , . . . , we have that (Vkj , Vk2 ' . . . , Vkr ) has the same distribution as ( Vo , Vk2 - kj , . . . , Vkr -k j ) ' Therefore V is strongly stationary. Clearly E( Vn) = E( Vo) = i, and, by the independence of the Xi , 00
cov( Vo , Vn )
= L:: r 2i - n var(Xi ) = -fi ( i ) n . i= l
355
[9.1.3]-[9.2.1]
Stationary processes
Solutions
3. (i) For mean-square convergence, we show that vergent as k --+ 00. We have that, for r < s,
Sk = L:� =o anXn is mean-square Cauchy con
since Ic(m) I :5 c(O) for all m, by the Cauchy-Schwarz inequality. The last sum tends to 0 as r, s --+ 00 if L: i l ai I < 00. Hence Sk converges in mean square as k --+ 00. Secondly,
E
lak Xk l ) t lak l · EIXk l (t k= l k= l :5
:5
VE(X5) t l ak l k= l
L:k= l lak Xk l
which converges as n --+ 00 if the lak l are summable. It follows that absolutely (almost surely), and hence L:k= l ak Xk converges a.s. (ii) Each sum converges a.s. and in mean square, by part (i). Now
converges
00
c y (m) = L aj ak c(m + k - j) j,k=O whence
Clearly Xn has distribution 1C for all n, so that {f(Xn) : n :::: on the value of m. Therefore the sequence is strongly stationary.
4.
9.2 Solutions.
1.
Linear prediction
(i) We have that
which is minimized by setting a (ii) Similarly ( ** )
m} has fdds which do not depend
=
c(l)/c(O). Hence Xn+! = c(1)Xn /c(O).
E{ (Xn+! - f3Xn - y Xn _ t > 2 } = (1 + f3 2 + y 2 )c(O) + 2f3(y - l)c(l) - 2yc(2),
an expression which is minimized by the choice
f3 =
c(l) (c(O) - c(2» ) c(O) 2 c(1) 2 _
- c(1) 2 . y - c(O)c(2) 2 c(O) - c(1) 2
'
'
Xn+l is given accordingly. (iii) Substitute a, f3, y into (*) and (**), and subtract to obtain, after some manipulation,
--
-
2 2 D - {c(1) - c(O)c(2)} 2 2 1 c(O){c(O) - c( ) } . ----,.--
356
--..,.-
Autocovariances and spectra
Solutions
1 and c(1) = c (2) = 0. Therefore Xn+ 1 = Xn+ 1 = 0, and D (a) In this case c(O) = 2;' (b) In this case D = ° also. In both (a) and (b), little of substance is gained by using Xn+ 1 in place of X n+ 1 . -
�
[9.2.2]-[9.3.2] = 0.
Let {Zn : n = . . . , - 1 , 0, 1 , . . . } be independent random variables with zero means and unit variances, and define the moving-average process
2.
Xn = Zn + aZn2- 1 . Vi + a
It is easily checked that X has the required autocovariance function. By the projection theorem, Xn - Xn is orthogonal to the collection E{(Xn - Xn)Xn - r } = 0, r 1 . Set Xn = 2:s00= 1 bs Xn -s to obtam that
�
�
�
•
for s
a = a/(1 + a 2 ). ( _ 1)s + 1 a s , and therefore
where
{Xn - r : r
> I } , so that
� 2,
The unique bounded solution to the above difference equation is 00
Xn = L ( _ 1 ) s + 1 a s Xn - s · s= 1
The mean squared error of prediction is
Clearly E( Xn )
= ° and 00
cov(Xn , Xn -m ) = L br bs c (m + r - s), r, s = 1
m
� O,
so that X is weakly stationary.
9.3 Solutions. 1.
It is clear that E(Xn )
Autocovariances and spectra
= ° and var(Xn ) = 1.
Also
cov(Xm , Xm+n) = cos(mA) cos{ (m + n )A} + sin(mA) sin{ (m + n)A } = cos(nA) , so that X is stationary, and the spectrum of X is the singleton {A } .
2.
= (e i trr - e - i t rr )/(27rit), so that E(Xn ) = ¢u (1) ¢v (n) = 0. Also cov(Xm , Xm+n ) = E(Xm Xm+n ) = E (e i { U -V m - U + V(m+n ) } ) = ¢v (n),
Certainly ¢u (t)
whence X i s stationary. Finally, the autocovariance function is
J
c(n) = ¢v (n) = e i nJ... dF(A), whence
F is the spectral distribution function.
357
bs
[9.3.3]-[9.3.4] 3.
Stationary processes
Solutions
The characteristic functions of these distributions are 1 2
pet) = pet) = 2"1 1 1 it + 1 +1 it = 1 +1 t 2 .
(
e - '1.t ,
(i) (ii) 4.
)
-
(i) We have that var
( -I Ln Xj ) = 1 Ln cov(Xj , Xk ) = c (O) j n l n k l n2 2"
j=
(-11',11']
j, =
( Ln i. . -k A ) dF()"). j, k= l
(j
)
The integrand is
/ Ln eij A / 2 = ( ei�A _ l) ( e -i�A _ l ) = l - cos(n)..) , elA - 1 e-1 A - 1 1 - cos ).. j= l whence var
( -1 � Xj ) = c (O) j n j= l L..J
It is easily seen that I sin 8 I
(-11',11']
(
)
Sin(n)"/2» 2 d F ()"). n sin()"/2)
� 18 I , and therefore the integrand is no larger than
(
)
2 ),, / 2 sin()"/2)
1 2 � (�n) .
As n � 00, the integrand converges to the function which is zero everywhere except at the origin, where (by continuity) we may assign it the value 1. It may be seen, using the dominated convergence theorem, that the integral converges to F (0) - F (0-), the size of the discontinuity of F at the origin, and therefore the variance tends to 0 if and only if F (0) - F (0-) = O. Using a similar argument,
nl (0) j -n1 L- c(j) = � n
(-11',11']
j=O
(nL- l eij A ) d F ()..) = c (O) j
(-11',11']
j=O
{
where
l gn ()..) = einA - 1 n(eiA - 1)
if)..
gn ()..) dF()")
= 0,
if).. :;6 0,
is a bounded sequence of functions which converges as before to the Kronecker delta function Therefore
n- l
1 -n L c(j) � c(O) ( F (O) - F (O-» ) j=O
358
as n
� 00.
8Ao.
The ergodic theorem
Solutions
[9.4.1]-[9.5.2]
9.4 Solutions. Stochastic integration and the spectral representation
1. Let Hx be the space of all linear combinations of the Xi , and let H X be the closure of this space, that is, Hx together with the limits of all mean-square Cauchy-convergent sequences in Hx . All members of Hx have zero mean, and therefore all members of H X also. Now S(A) E H X for all A, whence lE(S(A) - S(p,)) = 0 for all A and J-L .
First, each Ym lies in the space H X containing all linear combinations of the Xn and all limits of mean-square Cauchy-convergent sequences of the same form. As in the solution to Exercise (9.4. 1), all members of H X have zero mean, and therefore lE ( Ym ) = 0 for all m. Secondly,
2.
As for the last part,
This proves that such a sequence sequence.
Xn
may be expressed as a moving average of an orthonormal
3. Let H X be the space of all linear combinations of the Xn , together with all limits of (mean square) Cauchy-convergent sequences of such combinations. Using the result of Problem (7. 1 1 . 1 9), all elements in H X are normally distributed. In particular, all increments of the spectral process are normal. Similarly, all pairs in H X are jointly normally distributed, and therefore two members of H X are independent if and only if they are uncorrelated. Increments of the spectral process have zero means (by Exercise (9.4. 1 )) and are orthogonal. Therefore they are uncorre1ated, and hence independent.
9.5 Solutions.
The ergodic theorem
1. With the usual shift operator . , it is obvious that .- 1 0 = 0, so that 0 E 1 . Secondly, if A E 1 , then .- 1 (AC ) = (.- 1 A)C = AC , whence A C E 1 . Thirdly, suppose A I , A 2 , . . . E 1 . Then
( 00 ) 00
00
.- 1 U Ai = U . - 1 Ai = U Ai ' i =1 i =1 i=1 so that Uf A i E 1 .
The left-hand side is the sum of covariances, c(O) appearing n times, and c(i) appearing 2(n - i) times for 0 < i < n, in agreement with the right-hand side. Let E > O . If e(j) = rl 'E{�� c(i) -+ (1 2 as j -+ 00, there exists J such that le(j) - (1 2 1 < E when j � J . Now
2.
{
2 J je(j) + n j «(1 2 + E ) 2 n jC(j) :::; "2 L L L n j=1 n j=1 j=J + 1
"2
}
-+ (1 2 + E
as n -+ 00. A related lower bound is proved similarly, and the claim follows since E ( > 0) is arbitrary.
359
[9.5.3H9.6.3]
Stationary processes
Solutions
3. It is easily seen that Sm = 2::r=0 ai Xn +i constitutes a martingale with respect to the X ' s, and m E( S; ) = l: a[E(X;+i ) � l: a[, i =O i =O whence Sm converges a.s. and in mean square as 00. Since the Xn are independent and identically distributed, the sequence Yn is strongly stationary; also E(Yn) = 0, and so n - 1 2::7=1 Yi Z a.s. and in mean, for some random variable Z with zero. For any fixed (� 1), the contribution of X I , X2 , . . . , Xm towards 2:: 7=1 Yi is, for large n, no larger than Cm = (ao + al + . . . + aj_l)Xj =1 J Now n - 1 Cm 0 as n 00, so that Z is defined in terms of the subsequence Xm+l ' Xm +2 , . . . for all which is to say that Z is a tail function of a sequence of independent random variables. Therefore Z is a.s. constant, and so Z = 0 a.s.
00
m --+
--+
m
m, --+
mean
It
--+
I
9.6 Solutions.
·
Gaussian processes
1. The quick way is to observe that c is the autocovariance function of a Poisson process with intensity 1. Alternatively, argue as follows. The sum is unchanged by taking complex conjugates, and hence is real. Therefore it equals
where to = O. 2. For s, t � 0, X(s) and X(s + t) have a bivariate normal distribution with zero means, unit variances, and covariance c(t). It is standard (see Problem (4. 14.l3)) that E(X(s + t) I X(s)) = c(t)X(s). Now c(s + t) = E (X(O)X(s + t)) = E E ( X(O)X(s + t) I X(O), X(s)) } = E ( X(O)c(t)X(s)) = c(s)c(t)
{
by the Markov property. Therefore c satisfies c(s + t) = c(s)c(t), c(O) = 1, whence c(s) = c ( 1 ) l s l = pi s i . Using the inversion formula, the spectral density function is
00
2 c(s)e - I S )" = I - p l )" ' f().. ) = 21 " L 27r 1 1 - pe I 2 7r s = - oo Note that X has the same autocovariance function as a certain autoregressive process. Indeed, stationary Gaussian Markov processes have such a representation. 3. If X is Gaussian and strongly stationary, then it is weakly stationary since it has a finite variance. Conversely suppose X is Gaussian and weakly stationary. Then c(s, t) = cov(X(s), X(t)) depends -
.
.
360
Problems
Solutions
[9.6.4]-[9.7.2]
on t - s only. The joint distribution of X(tI ), X(t2 ), . . " X(tn) depends only on the common mean and the covariances C(ti ' tj ). Now C(ti ' tj ) depends on tj - ti only, whence X(t I ), X(t2 ), . . , X (tn ) have the same joint distribution as Xes + t I ), xes + t2 ), . . . , xes + tn ) . Therefore X is strongly stationary. .
4.
(a) If s, t > 0, we have from Problem (4. 14. l3) that
whence
COV (X(S) 2 , Xes + t) 2 ) = E(X(s) 2 xes + t) 2 ) - 1 = E { X(s) 2 E (X(s + t) 2 1 Xes)) - 1 = c(t) 2 E(X(s) 4 ) + (1 - c(t) 2 )E(X(s) 2 ) - 1 = 2c(t) 2
}
by an elementary calculation. (b) Likewise cov(X(s) 3 , xes + t) 3 )
= 3 (3 + 2c(t) 2 )c(t).
9.7 Solutions to problems
{
Xn + (a - P)Xn - I + P Yn - I , whence the autocovariance function c 1 a2 _ p2 'f k 0 __+1___-, p 2.--_ c(k) = 2 p l k l - I a(1 + ap -2 p ) if k :;6 O. 1-p
1. It is easily seen that Yn of Y is given by
=
{
}
1
-
,
Set Yn+I = I:�o ai Yn - i and find the ai for which it is the case that E{(Yn+ I - Yn+I ) Yn - k } for k � O. These equations yield 00
c(k + 1) = l: ai c(k - i), i=O = a(p - ai for i � O. 2. The autocorrelation functions of X and Y satisfy which have solution ai
r
O'ipx (n ) = O'� l: aj ak Py(n + k - j).
j,k=O
Therefore
0' 2
r
00 l: e - inJ.. l: aj ak Py (n + k - j) n= - oo j,k=O 00 O'y2 "" a ' a ei (k - j )J.. "" e - i (n+k - j ) J.. p y (n + k _ j) k L..J J L..J 2n j,k=O n= - oo ' J.. ) 2 2 = O'y l Ga (ei 1 fy ().. ) ·
O'ifx O. ) =
2;
r
=
361
=0
[9.7.3]-[9.7.5]
Stationary processes
Solutions
In the case of exponential smoothing,
G a (e iA ) = ( 1 - j.t)/ ( 1 - j.te iA ), so that
j.t) 2 fy().. ) , fxO.. ) = 1 c-( l2j.t- cos ).. + j.t 2 where c 3.
1).. 1
< ](,
= a� /ai is a constant chosen to make this a density function.
Consider the sequence {Xn } defined by
Xn = Yn - Yn = Yn - a Yn - l - {3Yn -2 ·
Now Xn is orthogonal to { Yn - k : k � I }, so that the Xn are uncorrelated random variables with spectral density function fx ().. ) = (2]( ) - 1 , ).. E ( -](, ]( ) . By the result of Problem (9.7.2),
ai fx ().. ) = a� l l - ae i A - {3 e2iA 1 2 fy().. ) ,
whence
a2 2 fy().. ) = 2]( 1 1 - aexl'A/a-y enI I 2 ' {3
-]( < ).. < ](.
Let {X� : n � I} be the interarrival times of such a process counted from a time at which a meteorite falls. Then Xl ' Xz, ' " are independent and distributed as X2 . Let Y� be the indicator function of the event {X:n = n for some m }. Then
4.
E(Ym Ym+n)
= lP'(Ym = 1 , Ym+n = 1 ) = lP'(Ym+n = 1 I Ym = l )lP'(Ym = 1 ) = lP'(Y� = l)a
where a = lP'(Ym = 1). The autocovariance function of Y is therefore c (n) n � 0, and Y is stationary. The spectral density function of Y satisfies 1 � - I. nA c(n) e a ( 1 - a) = Re fy ().. ) = 2]( L n = - oo
Now
00
{
= a {lP'(Y� =
� el.nA c (n) - I
I
]( a ( 1 - a) nL =O
2](
00
L einA y� = nL=O eiA T� n =O
where Th
= Xl + Xz + . . . + X� ; just check the non-zero terms. Therefore
when e iA :f. 1 , where ¢ is the characteristic function of X2 . It follows that
fy().. ) = ]( ( 1 1- a) Re 5.
We have that E (cos(n U) )
=
1
:n;
-:n;
1 2](
{
I
a . - -1 - ¢ ().. ) 1 - e lA
- cos(nu) du
= 0, 362
1 , }-2](
1 ).. 1
< ](.
}
.
1) - a},
Problems for n
�
Solutions
[9.7.6]-[9.7.7]
1 . Also lE (cos(m U) cos(n U) )
= lE U (cos[(m + n)U] + cos[(m - n)Ul ) = 0
}
=1=
if m n . Hence X is stationary with autocorrelation function function f(A) = (2Jr ) 1 for IAI < Jr. Finally
p(k) =
8kO , and spectral density
-
lE { cos(m U) cos(n U) cos(r U) }
i lE { (cos[(m + n)U] + cos[(m - n)Ul ) cos(r U) = Hp(m + n - r) + p(m - n - r) } =
which takes different values in the two cases (m , n, r)
= ( 1 , 2, 3),
}
(2, 3, 4) .
{(Ui, Vi)
6. (a) The increments of N during any collection of intervals : 1 :::: i :::: n } have the same fdds if all the intervals are shifted by the same constant. Therefore X is strongly stationary. Certainly lE(X (t)) = Aa for all t, and the autocovariance function is
c (t)
{0
if t > a, if O :::: t :::: a.
= cov ( X (O) , X (t) ) = A(a - t)
Therefore the autocorrelation function is
pet) =
{0 1
- It/a l
if It I > a, if It I :::: a,
which we recognize as the characteristic function of the spectral density f (A) = { 1 -cos(aA) see Problems (S. 12.27b, 28a). (b) We have that lE(X (t)) = 0; furthermore, for s :::: t, the correlation of X (s ) and X (t) is 1 Z- cov ( X (s), X (t) )
(J
} / (aJr A2 );
1 cov ( W (s) - W (s - 1 ) , W et) - W et - 1 ) ) = Z(J = s - min{s, t - - (s - 1 ) + (s - 1 ) if s :::: t - 1 , = s - t + l if t - l < s :::: t .
I}
{I
This depends o n t - s only, and therefore X i s stationary; X i s Gaussian and therefore strongly stationary also. The autocorrelation function is
p (h ) _
{0 1
- Ih l
if Ih l � 1 , if Ih l < 1 ,
which we recognize as the characteristic function of the density function 7.
f(A) = ( 1 - cos A)/(JrA2 ).
We have from Problem (8.7 . 1 ) that the general moving-average process o f part (b) i s stationary with autocovariance function c(k) = 'L.J =o aj ak + j ' k � 0, with the convention that as = 0 if s < 0 or s > r . (a) In this case, the autocorrelation function is
p(k) =
{ o� 1
+ a2 363
if k
=
if Ikl
0,
= 1,
if Ikl > 1 ,
[9.7.8]-[9.7.13]
Stationary processes
Solutions
whence the spectral density function is
(
)
) 1 1 + -2a COS A 1 ( l'A -I' A n. f(A) = 2n p(O) + e p(l) + e p(- l) = � l + a 2 ' IAI
E. Let Xi = El{ Xk ?: E}, and denote by N' the related renewal process. Now N(t) � N'(t), so that E(e(l N (t) � E(e(l N' (t) , for 0 > O. Let Zm be the number of renewals (in N') between the times at which N' reaches the values (m - I)E and mE. The Z ' s are independent with
1.
€e(l E(e(lZm ) = 1 - (1 - E)e , if (1 - E)e(l < 1, (l E whence E(e(l N ' (t) � (E e(l { I - (1 - E)e(l } ) t / for sufficiently small positive O. 2. Let X b e th e time of the first arrival. IT X l > then W = On the other hand if X l
y) dy, implying by (*) that lE(E(tn = lE ({(X I - t) + n + ry r - 1 1P' (X I > t + y - x) dm(x) dy, 0 whence the given integral equation is valid with oo r - 1 ry 1P' (X I > u + y) dy = lE ( {(X I - u) + n . h (u) =
lo
lo
r
......
lo
lo
o
i:o 1�
lo
Now h satisfies the conditions of the key renewal theorem, whence oo .!.. r h(u) du = .!.. = lim lE(E(tn y r dF(u + y) du t oo fJ, JJOt}) � 0 as t � 00. where
�
as t
(N = n) = p ( 1 - p) n - l for n � 1, where p = JJ> ( 1j < Tk I Xo = k). Since E(N) = p - l , we deduce as required that 1 / JJ> ( 1j < Tk I Xo = k) nk = E--( T----�-------------k I Xo = j) + E(1j I Xo = k) 10.6 Solutions to problems
--+
--+ 00.
1. (a) For any n, JJ> (N (t ) < n) � JJ>(Tn > t) 0 as t (b) Either use Exercise (10. 1 . 1), or argue as follows. Since JJ>(X I > E) > O. For all n,
/.l
>
0, there exists E (> 0) such that
JJ>(Tn � nE) = 1 - JJ>(Tn > nE) :::: 1 - JJ>(X I > E ) n < 1, s o that, i f t > 0, there exists n = net) such that JJ>(Tn � t) < 1. Fix t and let n be chosen accordingly. Any positive integer k may be expressed in the form k = an + fJ where 0 � fJ < n. Now JJ>(Tk :::: t) � JJ>(Tn :::: t ) a for an � k < (a + l)n, and hence 00
00
met ) = L JJ>(Tk :::: t) � L nJJ>(Tn � t ) a < k= l a =O
00.
(c) It is easiest to use Exercise (10. 1. 1), which implies the stronger conclusion that the moment generating function of N(t) is finite in a neighbourhood of the origin.
2.
(i) Condition on X I to obtain
vet ) = t E{ ( N (t - u) + 1) 2 } dF(u) = io
rt {v(t - u) + 2m(t - u) + I } dF(u).
io
Take Laplace-Stieltjes transforms to find that v* = (v* + 2m * + 1) F*, where m * = F * + m * F * as usual. Therefore v* = m* ( 1 + 2m*), which may be inverted to obtain the required integral equation. (ii) If N is a Poisson process with intensity A, then met ) = A t, and therefore vet ) = ()..t ) 2 + At.
3.
Fix x E R. Then
376
P ble s ro
m
Solutions
[10.6.4]-[10.6.7]
where a(t) = l(tIJL) + x Vtu2/JL3 J . Now, lP' (Ta(t) ::::: t )
However aCt)
�
t I JL as t
= lP'
( Ta(t) - JLa(t) ::::: t - JLa(t) ) . u JaCt)
--+ 00, and therefore t - JLa(t) --+ -x u Ja(t) --==-
implying by the usual central limit theorem that
(
) --+
u
as t
JaCt)
--+ 00,
- (tIJL) � x (-x) as t 00 lP' N(t) Vtu2/JL 3 where is the N (O, 1) distribution function. An alternative proof makes use of Anscombe's theorem (7.1 1 . 28). 4. We have that, for y ::::: t, as t lim lP'(E(u) > y) lP'(C(t) � y) = lP'(E(t - y) > y) u-+oo
--+
=
ryoo .!..JL [1 - F(x)] dx
--+
--+ 00
J
by Exercise ( l 0.3.3a). The current and excess lifetimes have the same asymptotic distributions. 5. Using the lack-of-memory property of the Poisson process, the current lifetime C (t) is independent of the excess lifetime E(t), the latter being exponentially distributed with parameter A. To derive the density function of C(t) either solve (without difficulty in this case) the relevant integral equation, or argue as follows. Looking backwards in time from t, the arrival process looks like a Poisson process up to distance t (at the origin) where it stops. Therefore C(t) may be expressed as min{Z, t} where Z is exponential with parameter A; hence Ae -J..s if s t, and lP'(C(t) = t) = e -At . Now D(t) = C(t) + E(t), whose distribution is easily found (by the convolution formula) to be as given. 6. The i th interarrival time may be expressed in the form T + Zj where Zj is exponential with parameter A. In addition, Z 1 , Z2 , . . . are independent, by the lack-of-memory property. Now 1 - F (x) = lP'(T + Zl > x) = lP'(Zl > x - T) = e - J.. (x- T ) , x � T.
{
Taking into account the (conventional) dead period beginning at time 0, we have that
(
k
)
t � kT, lP'(N(t) � k) = lP' kT + � Zj ::::: t = lP'(N(t - kT) � k) , 1 =1 where N is a Poisson process. 7. We have that Xl = L + E(L) where L is the length of the dead period beginning at 0, and E(L) is the excess lifetime at L. Therefore, conditioning on L,
377
[10.6.8]-[10.6.10]
Rene als w
Solutions
We have that
lP'(E(t ) ::: y) = F(t + y) By the renewal equation,
lot { 1 - F(t + y - x) } dm(x). Jo
+y m(t + y) = F(t + y) + t F(t + Y - x) dm(x), whence, by subtraction,
lP'(E(t ) ::: y) = It follows that
It t+y { 1 - F(t + y - x) } dm(x).
l�o i:l { I - F(x - y) } dm(y) dFL (l) X X = [ FL (l) I { 1 - F(x - Y ) } dm(y)] o + r FL (l) { I - F(x - l) } dm(l) � k
lP'(X l ::: x) =
I
using integration by parts. The term in square brackets equals O.
8. (a) Each interarrival time has the same distribution as the sum of two independent random variables with the exponential distribution . Therefore N(t) has the same distribution as l i M (t )J where M is a Poisson process with intensity A. Therefore m(t) = � E(M(t)) - � lP'( M (t ) is odd) . Now E(M(t)) = At , and
00
1
(M) 2n+l e - At = e -At (e At e - At ) '"' lP'(M (t ) is odd) = � . :2 n=O (2n + I) ! _
With more work, one may establish the probability generating function of N(t) . (b) Doing part (a) as above, one may see that iii ( t) = m(t). 9. Clearly C (t) and E (t) are independent if the process N is a Poisson process. Conversely, suppose that C(t) and E(t) are independent, for each fixed choice of t. The event {C(t) � y} n {E(t) � x} occurs i f and only if E (t - y) � x + y. Therefore
lP'(C(t ) � y)lP'(E(t ) � x) = lP'(E(t - y) � x + y) .
--+
t 00, remembering G(y)G(x) = G(x + y) if x, y � 0, where
Take the limit as
G (u)
=
Exercise (10.3.3) and Problem
(10.6.4),
to obtain that
100 -1 [1 - F(v)] dv. u
JL
Now 1 - G is a distribution function, and hence has the lack-of-memory property (Problem (4.14.5)), implying that G (u) = e - J.. u for some A. This implies in tum that [1 - F (u)]jJL = -G ' (u) = Ae - J.. u, whence JL = I/A and F (u) = 1 - e - J.. u .
10. Clearly N is a renewal process if N2 is Poisson. Suppose that N is a renewal process, and write A for the intensity of Nl , and F2 for the interarrival time distribution of N2 . By considering the time X to the first arrival of N,
1
378
Problems Writing
Solutions
[10.6.11]-[10.6.12]
E, Ej for the excess lifetimes of N, Nj , we have that lP'(ECt) > x ) = lP'(E I Ct) > x , E2 (t) > x ) = e -Ax lP'(E2 (t)
> x) .
Take the limit as t --+ 00, using Exercise (10.3.3), to find that
100 -1 [1 - F(u)] du x /.l
=
e -AX
100 -1 [ 1 - F2 (U)] du, x /.l2
where /.l2 is the mean of F2 . Differentiate, and use (*), to obtain
100
1 1 - Ax e -AX -e [1 - F2 (X)] = Ae - AX - [1 - F2 (U)] du + -- [1 - F2 (X)], /.l2 /.l x /.l 2 which simplifies to give 1 - F2 (X) = C J:O [1 - F2(U)] du where c = A/.l/(/.l 2 - /.l); this integral equation has solution F2 (X) = 1 - e - cx . 11. (i) Taking transforms of the renewal equation in the usual way, we find that
* m * (() ) = 1 F «() - F* «() where
F * «()
=
=
1
1
-
F*«()
_
1
JE(e -O X t ) = 1 - () /.l + �() 2 (/.l2 + (]' 2 ) + 0 «() 2 )
as () --+ O. Substitute this into the above expression to obtain
and expand to obtain the given expression. A formal inversion yields the expression for m. (ii) The transform of the right-hand side of the integral equation is + * m «() - FE «() m * «() . /.l() - FE «() By Exercise (10.3.3), FE «() = [ 1 - F * «() ]f(/.l() , and (* ) simplifies to m*«() - (m * - m * F* F*)/(/.l() , which equals m*«() since the quotient is 0 (by the renewal equation). Using the key renewal theorem, as t --+ 00, 1 JE(X 21 ) = (]' 2 + /.l2 [1 FE Ct - x)] dm(x) --+ [1 - FE (u)] du = -2o 2/.l 2/.l 2 /.l 0
�
lot
-
1000
by Exercise (lO.3.3b). Therefore,
12. (i) Conditioning on X I , we obtain
379
[10.6.13]-[10.6.16] Therefore m d* =
Renewals
Solutions
F d* + m * Fd* . Also m* = F* + m* F *, so that
whence m d* = F d* + m d* F *, the transform of the given integral equation. (ii) Arguing as in Problem (10.6.2), v d* = F d* +2m * F d* +v * F d* where v* = F * (1 +2m * )/(1 -F*) is the corresponding object in the ordinary renewal process. We eliminate v* to find that
by (*). Now invert. 13. Taking into account the structure of the process, it suffices to deal with the case I = 1. Refer to Example (10.4.22) for the basic notation and analysis. It is easily seen that f3 = (v - 1)1.. . Now F(t) = 1 - e -v M . Solve the renewal equation (10.4.24) to obtain
V AX
g(t) = h(t) +
lot h (t - x) diii (x)
where iii (x) = is the renewal function associated with the interarrival time distribution F. Therefore g(t) = 1, and m(t) = ef3 t . 14. We have from Lemma (10.4.5) that p* = 1 - Fz + p* F*, where F* = FYFz. Solve to obtain
P
* = 1 - Fz 1 - F*F Y Z* '
15. The first locked period begins at the time of arrival of the first particle. Since all future events may be timed relative to this arrival time, we may take this time to be O. We shall therefore assume that a particle arrives at 0; call this the Oth particle, with locking time Yo. We shall condition on the time X I of the arrival of the next particle. Now
{
if u > t, JP' (L > t I X l = u ) = JP'(Yo > t) JP'(Yo > u)JP'(L' > t - u) if u .::: t, where L' has the same distribution as L ; the second part is a consequence of the fact that the process 'restarts ' at each arrival. Therefore
JP'(L > t) = (1 - G(t»)JP'(X I
>
t) +
lot JP'(L > t - u) ( I - G(u») fX j (u) du,
the required integral equation. If G(x) = 1 - e - /LX , the solution is JP'(L > t) = e - /Lt , so that L has the same distribution as the locking times of individual particles. This striking fact may be attributed to the lack-of-memory property of the exponential distribution. 16. (a) It is clear that M(tp) is a renewal process whose interarrival times are distributed as X l + X2 + . . . + XR where JP'(R = r) = pq r - l for r � 1. It follows that M(t) is a renewal process whose first interarrival time
X(p) = inf{t : M(t) = I } = p inf{t : M(tp) = I} 380
Problems
Solutions
has distribution function lP' ( X (p) :::; x ) =
[10.6.17]-[10.6.19]
00
L lP'(R = r)Fr (x/p) . r= 1
00 pq r- 1 1-00 e 1.xt dF (t/p) = L00 pq r- 1 ¢ (pt/ = p¢ (pt) L ---1 - q¢ (pt) 00 r r=1 r=1
(b) The characteristic function ¢p of Fp is given by ¢p (t) =
where ¢ is the characteristic function of F. Now ¢ (pt) = 1 + i ILPt + o(p) as p + 0, so that l + i ILpt + O(P) 1 + 0(1) ¢p (t) = = -,---- 1 - iILt + 0(1) 1 - iILt as p + O. The limit is the characteristic function of the exponential distribution with mean IL, and the continuity theorem tells us that the process M converges in distribution as p + 0 to a Poisson process with intensity 1 / IL (in the sense that the interarrival time distribution converges to the appropriate limit). (c) If M and N have the same fdds, then ¢p (t) = ¢ (t), which implies that ¢ (pt) = ¢ (t)/(p + q¢ (t)) . Hence 1/I (t) = ¢ (t) - 1 satisfies 1/I (pt) = q + p1/l (t) for t E R. Now 1/1 is continuous, and it follows as in the solution to Problem (5. 1 2. 1 5) that 1/1 has the form 1/I (t) = l + t.lt, implying that ¢ (t) = (1 +,Bt) - 1 for some ,B E C. The only characteristic function of this form is that of an exponential distribution, and the claim follows. 17. (a) Let N (t) be the number of times the sequence has been typed up to the tth keystroke. Then N is a renewal process whose interarrival times have the required mean IL; we have that lE(N(t))/t --+ IL - 1 as t --+ 00. Now each epoch of time marks the completion of such a sequence with probability ( nk ) 14 , so that 14 1 14 _1_ ! lE(N (t)) = ! --+ __ as t --+ 00, t t n=1 100 100
t( )
( )
4
implying that IL = 1028 . The problem with 'omo ' is 'omomo ' (Le., appearances may overlap). Let us call an epoch of time a 'renewal point' if it marks the completion of the word 'omo ' , disjoint from the words completed at previous renewal points. In each appearance of 'omo ' , either the first '0 ' or the second '0 ' (but not both) is a renewal point. Therefore the probability Un, that n is a renewal point, satisfies ( nk ) 3 = Un + U n - 2 ( nk ) 2 . Average this over n to obtain 1 )3 1 2 1 2 (_ = nlim ! t { un + U n - 2 ( _ ) } = .!. + .!. ( _ ) , --+ oo t 100 IL IL 100 100
n =1
and therefore IL = 106 + 102 . (b) (i) Arguing as for 'omo ' , we obtain p 3 = un + P U n - 1 + P 2 U n _ 2' whence p 3 = (1 + p + p 2 )/IL. (it) Similarly, p 2 q = Un + pqu n - 2 , so that IL = (1 + pq)/(p 2 q). 18. The fdds of {N(u) - N(t) : u � t} depend on the distributions of E (t) and of the interarrival times. In a stationary renewal process, the distribution of E (t) does not depend on the value of t, whence {N(u) - N(t) : u � t} has the same fdds as {N(u ) : u � OJ, implying that X is strongly stationary. 19. We use the renewal-reward theorem. The mean time between expeditions is B IL, and this is the mean length of a cycle of the process. The mean cost of keeping the bears during a cycle is i B(B - l)cIL, whence the long-run average cost is {d + B(B - 1)cIL/2}/(BIL). 381
11 Queues
1 1.2 Solutions. MIMIl
1. The stationary distribution satisfies 1r equations
= P when it exists, where P is the transition matrix. The 1r
- 1 1rn +1 > 2, n1rn = P1rn 1 +p + 1 +p with L:�o 1rj = 1, have the given solution. If p � 1 , no such solution exists. It is slightly shorter to --
--
+" l or
use the fact that such a walk is reversible in equilibrium, from which it follows that 1r satisfies
1r 1 1rO = -1 +p,
P1rn 1 +p
1rn +1 1 +p
for n �
1.
2. (i) This continuous-time walk is a Markov chain with generator given by gOI = eo, gn , n+l enP/(l + p) and gn , n - l = en/(1 + p) for n � 1, other off-diagonal terms being O. Such a process is reversible in equilibrium (see Problem (6. 15. 16», and its stationary distribution must satisfy vngn , n+l = vn+l gn+l,n ' These equations may be written as =
v
for n �
1.
These are identical to the equations labelled (*) in the previous solution, with 1rn replaced by vnen. follows that Vn = C 1rn /en for some positive constant C. (ii) If eo = A, en = A + JL for n � 1 , we have that
'"
{
It
1r0 + 1 - 1r0 = C 1 = � vn = C J: 2A ' JL + A
}
whence C = 2A and the result follows. 3. Let Q be the number of people ahead of the arriving customer at the time of his arrival. Using the lack-of-memory property of the exponential distribution, the customer in service has residual service time with the exponential distribution, parameter JL, whence W may be expressed as S I + S2 + . . . + S Q , the sum of independent exponential variables, parameter JL. The characteristic function of W is
382
MIMI]
[11.2.4]-[11.2.7]
Solutions
This is the characteristic function of the given distribution. The atom at 0 corresponds to the possibility that Q = O. 4. We prove this by induction on the value of i + j. If i + j = 0 then i = j = 0, and it is easy to check that Jl'(0; 0, 0) = 1 and A(O; 0, 0) = 1, A(n; 0, 0) = 0 for n ::: 1 . Suppose then that K ::: 1 , and that the claim is valid for all pairs (i, j ) satisfying i + j = K. Let i and j satisfy i + j = K + 1. The last ball picked has probability i / (i + j) of being red; conditioning on the colour of the last ball, we have that
j Jl'(n; i, j) = +i Jl'(n l ; i l , j) + + Jl'(n + l ; i, j l ) J J Now (i - 1) + j = K = i + (j - 1). Applying the induction hypothesis, we find that Jl' (n; i, j) = i j A(n - 1 ; i - I , j) - A(n; i - I , j) } .
-
.
I
�{
+
-
�
.
I
-
.
.
i j { A(n + 1 ; i, j - 1) A(n + 2; i, j - 1) } . -
Substitute to obtain the required answer, after a little cancellation and collection of tenns. Can you see a more natural way? 5. Let A and B be independent Poisson process with intensities ).. and /L respectively. These processes generate a queue-process as follows. At each arrival time of A, a customer arrives in the shop. At each arrival-time of B, the customer being served completes his service and leaves; if the queue is empty at this moment, then nothing happens. It is not difficult to see that this queue-process is M()..) /M(/L)/1. Suppose that A(t) = i and B(t) = j. During the time-interval [0, t], the order of arrivals and departures follows the schedule of Exercise ( 1 1 .2.4), arrivals being marked as red balls and departures as lemon balls. The imbedded chain has the same distributions as the random walk of that exercise, and it follows that JP> ( Q(t) = n I A(t) = i, B(t) = j) = Jl'(n; i, j). Therefore
Pn (t) = E i, i Jl'(n; i, j)JP>(A(t) = i)JP>(B(t) = j). 6. With p = ).. / /L, the stationary distribution of the imbedded chain is, as in Exercise (1 1.2. 1), if n = 0, (1 - p) Jl'n = i(1 p 2 )p n - l if n ::: 1 . In the usual notation of continuous-time Markov chains, gO = ).. and gn = ).. + /L for n ::: 1, whence, by Exercise (6.10. 1 1), there exists a constant c such that (1 - p), Jl'n = c (1 - p 2 )p n - l for n > 1 . Jl'o = � 2)" 2()" + /L) Now Ei Jl'i = 1, and therefore c = 2)" and Jl'n = (1 - p)p n as required. The working is reversible. 7. (a) Let Q i (t) be the number of people in the ith queue at time t, including any currently in service. The process Q 1 is reversible in equilibrium, and departures in the original process correspond to arrivals �
{i
_
-
in the reversed process. It follows that the departure process of the first queue is a Poisson process with intensity ).. , and that the departure process of Q 1 is independent of the current value of Q 1 . (b) We have from part (a) that, for any given t, the random variables Q l (t), Q 2 (t) are independent. Consider an arriving customer when the queues are in equilibrium, and let Wi be his waiting time (before service) in the i th queue. With T the time of arrival, and recalling Exercise (1 1.2.3), JP>(WI = 0, W2 = 0) > JP>(Qi (T) = 0 for i = 1 , 2) = JP>(Q I (T) = 0)JP>(Q 2 (T) = 0)
= (1 - Pl )(1 - pz ) = JP>(WI = 0)JP>(W2 = 0).
Therefore WI and W2 are not independent. There is a slight complication arising from the fact that T is a random variable. However, T is independent of everybody who has gone before, and in particular of the earlier values of the queue processes Q i .
383
[11.3.1]-[11.4.1]
Queues
Solutions
11.3 Solutions. MlG/!
1. In equilibrium, the queue-length Q n just after the nth departure satisfies where A n is the number of arrivals during the (n + l)th service period, and h (m) Qn and Q n+ 1 have the same distribution. Take expectations to obtain
where E(An) expectations:
o
=
o
=
=
1
- 8mo. 'Now
E(An) - J1D(Qn > 0),
Ad, the mean number of arrivals in an interval of length d. Next, square (*) and take
Use the facts that An is independent of Q n , and that Q n h(Q n ) =
=
-
Q n , to find that
{(Ad) 2 + Ad} + J1D(Qn > 0) + 2{ (Ad - l)E(Qn) AdJID (Qn > 0) }
and therefore, by (**),
2. From the standard theory, MB satisfies MB (S) = Ms(s - A + AMB (S)), where Ms(O) JL/(JL - 0 ) . Substitute to find that x = MB (S) is a root of the quadratic Ax 2 - X(A + JL - s ) + JL = O. For some small positive s, MB (s) is smooth and non-decreasing. Therefore MB (s) is the root given. 3. Let Tn be the instant of time at which the server is freed for the nth time. By the lack-of-memory property of the exponential distribution, the time of the first arrival after Tn is independent of all arrivals prior to Tn, whence Tn is a 'regeneration point ' of the queue (so to say). It follows that the
times which elapse between such regeneration points are independent, and it is easily seen that they have the same distribution. 11.4 Solutions.
GIM/!
)
1. The transition matrix of the imbedded chain obtained by observing queue-lengths just before arrivals is ... ... The equation 1r
=
1r
PA may be written as 00
1rn = L ai1rn +i - l for n � 1 .
;=0
It is easily seen, by adding, that the first equation is a consequence of the remaining equations, taken in conjunction with 2:8" 1ri = 1 . Therefore 1r is specified by the equation for 1rn, n � 1 . 384
GIGI]
Solutions
[11.4.2]-[11.5.2]
The indicated substitution gives 00
e n = e n - 1 L ai e i i=O which is satisfied whenever e satisfies
It is easily seen that A(e) = MX (JL(e - 1» is convex and non-decreasing on [0, 1], and satisfies A (O) > 0, A(I) = 1. Now A'(I) = JLE(X) = p - l > 1, implying that there is a unique 7j E (0, 1 ) such that A(7j) = 7j. With this value of 7j, the vector 1r given by 1tj = (1 - 7j ) 7j j , j � 0, is a stationary distribution of the imbedded chain. This 1r is the unique such distribution because the chain is irreducible. 2. (i) The equilibrium distribution is 1tn = (1 _ 7j)7jn for n � 0, with mean I:�o n1tn = 7j/(I - 7j). (ii) Using the lack-of-memory property of the service time in progress at the time of the arrival, we see that the waiting time may be expressed as W = S l + S2 + . . . + S Q where Q has distribution 1r , given above, and the Sn are service times independent of Q. Therefore
E ( W ) = E(S l )E(Q) = (17j/-JL7j) . --
3. We have that Q (n+) = 1 + Q (n - ) a.s. for each integer n, whence limt --+oo IP' ( Q (t ) = m) cannot exist. Since the traffic intensity is less than 1, the imbedded chain is ergodic with stationary distribution as in Exercise (1 1.4.1). 11.5 Solutions.
G/G/!
1. Let Tn be the starting time of the nth busy period. Then Tn is an arrival time, and also the beginning of a service period. Conditional on the value of Tn , the future evolution of the queue is independent of the past, whence the random variables {Tn+ 1 - Tn : n � I } are independent. It is easily seen that they are identically distributed. 2. If the server is freed at time T, the time I until the next arrival has the exponential distribution with parameter JL (since arrivals form a Poisson process). By the duality theory of queues, the waiting time in question has moment generating function Mw (s) = (1 - s)/(1 - sM/ (s» where M/ (s) = JL / (JL - s) and s = IP'(W > 0). Therefore, I - S) + (1 - S), MW (s) = JL(s JL( 1 - S) - s the moment generating function of a mixture of an atom at ° and an exponential distribution with parameter JL(1 - S}If G is the probability generating function of the equilibrium queue-length, then, using the lack of-memory property of the exponential distribution, we have that MW (s) = G (JL / (JL - s», since W is the sum of the (residual) service times of the customers already present. Set u = JL / (JL - s) to find that G(u) = (1 - S)/(1 - su), the generating function of the mass function f(k) = (1 - s > s k for k � 0. 385
[11.5.3]-[11.7.2]
Queue
s
Solutions
It may of course be shown that s is the smallest positive root of the equation x where X is a typical interarrival time. 3. We have that 1-
G(y) = J1D(S - X > y) =
fooo J1D(S > u + y) dFx (u),
y
=
MX (t-t (x - 1) ) ,
ER,
where S and X are typical (independent) service and interarrival times. Hence, formally,
100
oo dG(y) = - r dJID(S > u + y) dFx (u) = dy t-te -/L (U +Y) dFx (u), Jo -Y since fs (u + y) = e - /L (u +y ) if u > -y, and is 0 otherwise. With F as given, x F(x - y) dG(y) = { 1 - 17 e -/L ( I -I1)(x- Y) }t-te -/L (u +y ) dFx (u) dy. JJ- oo < y �x - Y < U < OO First integrate over y, then over u (noting that FX (u) = 0 for u < 0), and the double integral collapses to F(x), when x � o.
I-00
rr
1 1.6 Solution.
1. Q p has characteristic function
Heavy traffic
00
1-P in n 'l'p (t ) = ""' . L...J e t p ( 1 - p ) = 1 - pe l· t n=O Therefore the characteristic function of (1 - p) Q p satisfies 1 1-p ¢p ( ( 1 - p) t ) = 1 pe i ( 1 - p)t --+ 1 - i t as p t l . A.
_
The limit characteristic function is that of the exponential distribution, and the result follows by the continuity theorem. 1 1.7 Solutions.
Networks of queues
1. The first observation follows as in Example ( 1 1 .7.4). The equilibrium distribution is given as in Theorem ( 1 1 .7. 14) by :7r
(0)
n i e -ClI · II --,1,--i =1 n I· '· c
=
0/ .
I
for 0 =
(n l , n 2 , . . . , nc)
E
71/ ,
the product of Poisson distributions. This is related to Bartlett ' s theorem (see Problem (8.7.6)) by defining the state A as 'being in station i at some given time ' . 2. The number of customers in the queue is a birth-death process, and is therefore reversible in eqUilibrium. The claims follow in the same manner as was argued in the solution to Exercise ( 1 1 .2.7). 386
Problems
Solutions
[11.7.31-111.8.1]
3. (a) We may take as state space the set {O, 1', 1", 2, 3, . . . }, where i E {O, 2, 3, . . . } is the state of having i people in the system including any currently in service, and l' (respectively I") is the state of having exactly one person in the system, this person being served by the first (respectively second) server. It is straightforward to check that this process is reversible in equilibrium, whence the departure process is as stated, by the argument used in Exercise (1 1 .2.7). (b) This time, we take as state space the set {O', 0", 1', 1", 2, 3, . . . } having the same states as in part (a) with the difference that 0' (respectively 0" ) is the state in which there are no customers present and the first (respectively second) server has been free for the shorter time. It is easily seen that transition from 0' to I" has strictly positive probability whereas transition from I" to 0' has zero probability, implying that the process is not reversible. By drawing a diagram of the state space, or otherwise, it may be seen that the time-reversal of the process has the same structure as the original, with the unique change that states 0' are 0" are interchanged. Since departures in the original process correspond to arrivals in the time-reversal, the required properties follow in the same manner as in Exercise (1 1 .2.7). 4. The total time spent by a given customer in service may be expressed as the sum of geometrically distributed number of exponential random variables, and this is easily shown to be exponential with parameter 8 J.L . The queue is therefore in effect a M(A )/M(8 J.L )/1 system, and the stationary distribution is the geometric distribution with parameter p = A I (8 J.L ), provided p < 1 . As in Exercise (1 1 .2.7), the process of departures is Poisson. Assume that rejoining customers go to the end of the queue, and note that the number of customers present constitutes a Markov chain. However, the composite process of arrivals is not Poisson, since increments are no longer independent. This may be seen as follows. In equilibrium, the probability of an arrival of either kind during the time interval (t, t + h) is Ah + p J.L (1 - 8)h + o(h) = (AI8)h + o(h). If there were an arrival of either kind during (t - h , t), then (with conditional probability 1 O(h)) the queue is non-empty at time t, whence the conditional probability of an arrival of either kind during (t, t + h) is "Ah + J.L(1 - 8)h + o(h); this is of a larger order of magnitude than the earlier probability (AI8)h + o(h). 5 . For stations r , we write r -+ if an individual at r visits at a later time with a strictly positive probability. Let C comprise the station j together with all stations i such that i -+ j . The process restricted to C is an open migration process in equilibrium. By Theorem (1 1 .7.19), the restricted process is reversible, whence the process of departures from C via j is a Poisson process with some intensity s . Individuals departing C via j proceed directly to k with probability -
s,
s
s
Aj k Ajk¢j (nj ) = J.Lj¢j (nj ) + L.rfJC Ajr¢j (nj ) J.Lj + L.rfJC Ajr ' independently of the number nj of individuals currently at j . Such a thinned Poisson process is a Poisson process also (cf. Exercise (6.8.2)), and the claim follows.
1 1.8 Solutions to problems
1. Although the two cases may be done together, we choose to do them separately. When k the equilibrium distribution 'Jf satisfies: J.L 1l'1 - A 1l'O = 0, l ::; n < N, J.L1l'n +1 - (A + J.L ) 1l'n + A 1l'n - l = 0, - J.L1l'N + A 1l'N - l = 0, a system of equations with solution 1l'n = 1l'o (AI J.L ) n for 0 ::; n ::; N, where (if A "I J.L ) N N n 1 - (AI J.L ) +l 1 1l'o- = '"' (AI J.L ) = 1 - (AI ) J.L
�
387
=
1,
[11.8.2]-[11.8.3] Now let k
Queues
Solutions
= 2.
The queue is a birth-death process with rates A· -
1-
{
A if i < N, 0 if i ?:. N,
JLi
=
{
JL if i = 1, 2JL If I ?:. 2. .
.
It is reversible in equilibrium, and its stationary distribution satisfies A(lfi that lfi = 2p i lfO for 1 � i � N, where p = A/(2JL) and 1rO
I
= JLi+ l lfi+ l . We deduce
N
= 1 + L 2p i . i= 1
2.
The answer is obtainable in either case by following the usual method. It is shorter to use the fact that such processes are reversible in equilibrium. (a) The stationary distribution 1C satisfies 1rnAp(n) = 1rn+ l JL for n ?:. 0, whence 1rn = 1rop n / n ! where n p P = A/JL. Therefore 1rn = p e - I n ! . (b) Similarly, 1rn
n- l = 1rO p n II p(m ) = 1rO p n 2 - � n (n - 1 ) ,
n ?:. 0,
m =O
where 1rO
I
00 = L p n ( �) 2; n (n - l ) . I
n =O
At the instant of arrival of a potential customer, the probability obtained by conditioning on its length: q
q
that she joins the queue is
00 00 00 = L p(n)1rn = 1ro L p n 2 -n - z n (n - l ) = 1ro L p n 2 - z n (n + l ) = 1rO -p1 { 1ro l - I } . I
I
n =O
n =O
n =O
3. First method. Let ( Q l , Q 2 ) be the queue-lengths, and suppose they are in equilibrium. Since Q 1 is a birth-death process, it is reversible, and we write c h (t) = Q 1 ( -t). The sample paths of Q l have increasing jumps of size 1 at times of a Poisson process with intensity A; these jumps mark
arrivals at the cash desk. By reversibility, 0 1 has the same property; such increasing jumps for 0 1 are decreasing jumps for Q 1 , and therefore the times of departures from the cash desk form a Poisson process with intensity A. Using the same argument, the quantity Q I (t) together with the departures prior to t have the same joint distribution as the quantity 0 1 (-t) together with all arrivals after -t o However 0 1 (-t) is independent of its subsequent arrivals, and therefore Q l (t) is independent of its earlier departures. It follows that arrivals at the second desk are in the manner of a Poisson process with intensity A, and that Q 2 ( t ) is independent of Q l (t). Departures from the second desk form a Poisson process also. Hence, in equilibrium, Q I is M(A)IM(JL I )11 and Q 2 is M(A)IM(JL2 )11 , and they are independent at any given time. Therefore their joint stationary distribution is 1rm n
= IP'(Q l (t ) = m, Q 2 (t) = n ) = ( 1 - P l ) ( l - P2 ) P i" P2
where Pi = A/ JLi . Second method. The pair ( Q l (t), Q 2 (t )) (1rm n m, n ?:. 0) satisfies :
is a bivariate Markov chain. A stationary distribution m , n ?:. l ,
388
Problems
Solutions
[11.8.4]-[11.8.5]
together with other equations when m = 0 or n = O. It is easily checked that these equations have the solution given above, when Pi < 1 for i = 1 , 2. 4. Let Dn be the time of the nth departure, and let Q n = Q (Dn + ) be the number of waiting customers immediately after Dn. We have in the usual way that Q n+ 1 = An + Qn - h(Qn), where An is the number of arrivals during the (n + l)th service time, and h (x) = min{x, m } . Let G(s) = L:�o 1l'i S i be the equilibrium probability generating function of the Q n . Then, since Q n is independent of An ,
where
E(S An ) =
lX> eAu (s - l ) fS (u) du = MS (J... (s - 1)) ,
Ms being the moment generating function of a service time, and
Combining these relations, we obtain that G satisfies
{
s m G(s) = Ms (J... (s - 1)) G(S) + whenever it exists. Finally suppose that m
(S m - S i )1l'i } ' t 1 =0
= 2 and Ms«(}) = p,/(p, - (}). In this case,
+ 1) + 1l' I S} G(s) = p,{1l'o(s p,(s + 1) - J... s 2 Now G(I) = 1, whence p,(21l'0 + 1l' 1 ) = 2p, - J... ; this implies in particular that 2p, - J... > O. Also G(s) converges for Is I :::: 1. Therefore any zero of the denominator in the interval [- 1 , 1] is also a
zero of the numerator. There exists exactly one such zero, since the denominator is a quadratic which takes the value -J... at s = - 1 and the value 2p, - J... at s = 1 . The zero in question is at
and it follows that 1l'0 + (1l'0 + 1l' 1 )so
= O. Solving for 1l'0 and 1l' 1 , we obtain I -a , G(s) = -1 - as
where a = 2J... / {p, + Vp, 2 + 4J... p, }. 5. Recalling standard MlG/1 theory, the moment generating function MB satisfies
whence MB (s) is one of
(J... + p, - s) ± V(J... + p, - s) 2 - 4J... p, 2J... 389
[11.8.6]-[11.8.7]
Queues
Solutions
Now MB (s) is non-decreasing in s, and therefore it is the value with the minus sign. The density function of B may be found by inverting the moment generating function; see Feller (197 1 , p. 482), who has also an alternative derivation of MB . As for the mean and variance, either differentiate MB , or differentiate (*). Following the latter route, we obtain the following relations involving M (= MB ):
2AM M' + M + (s - A - I-L)M' = 0, 2AM M" + 2A(M, ) 2 + 2M ' + (s - A - I-L ) M" = O. Set s = O to obtain M'(O) = (I-L -A) - l and M"(O) = 21-L(I-L -A) - 3 , whence the claims are immediate. 6. (i) This question is closely related to Exercise (1 1 .3. 1). With the same notation as in that solution,
we have that
where h (x)
= min{ l , x}. Taking expectations, we obtain lP'( Qn > 0) = lE(An) where lE ( An ) =
lX> lE(An
I
S = s) dFs(s) = AlE(S) = p ,
and S is a typical service time. Square (*) and take expectations to obtain
lE(Qn) =
p (1 - 2p ) + lE(A� +1 ) , 2 (1 p ) _
where lE(A� ) is found (as above) to equal p + A 2 lE(S 2 ). (ii) If a customer waits for time W and is served for time S, he leaves behind him a queue-length which is Poisson with parameter A ( W + S). In equilibrium, its mean satisfies AlE(W + S) = lE(Q n), whence lE(W) is given as claimed. (iii) lE(W) is a minimum when lE(S 2 ) is minimized, which occurs when S is concentrated at its mean. Deterministic service times minimize mean waiting time. 7. Condition on arrivals in (t, t+h). If there are no arrivals, then Wt +h ::: x if and only if Wt ::: x +h. If there is an arrival, and his service time is S, then Wt +h ::: x if and only if Wt ::: x + h - S. Therefore F(x ; t + h)
= (1 - Ah )F(x + h ; t) +
Ah Jrox +h F(x + h - s; t) dFs(s) + o(h).
Subtract F(x; t ), divide by h, and take the limit as h -!- 0, to obtain the differential equation. We take Laplace-Stieltjes transforms. Integrating by parts, for () ::: 0,
r
/Jx dh (x) = - h (O) - (}{Mu «(}) - H(O)},
r
/Jx dH(x) = Mu «(}) - H(O) ,
J(O, oo) r
J(O, oo)
J(O, oo)
eO x dlP'(U + S ::: x) = Mu «(})Ms«(}),
and therefore
0 = - h (O) - (} {Mu «(}) - H (O) } + A H(O) + AMU «(}){Ms«(}) - I } . 390
Problems
Solutions
[11.8.8]-[11.8.10]
Set () = 0 to obtain that h (O) = AH (O), and therefore
Take the limit as () -+ 0, using L' Hopital ' s rule, to obtain H(O) = 1 - AE( S ) = I - p . The moment generating function of U is given accordingly. Note that Mu is the same as the moment generating function of the equilibrium distribution of actual waiting time. That is to say, virtual and actual waiting times have the same equilibrium distributions in this case.
8. In this case U takes the values I and -2 each with probability 1 (as usual, U = S - X where S and X are typical (independent) service and interarrival times). The integral equation for the limiting waiting time distribution function F becomes F(O) = 1 F(2) ,
F(x) =
1 { F(x - 1) + F(x + 2) }
for x = I , 2, . . . .
The auxiliary equation is (} 3 - 2() + 1 = 0, with roots 1 and - � (1 ± .J5). Only roots lying in [- 1 , 1 ] can contribute, whence - 1 .J5 F(x) = A + B
( ; r
for some constants A and B . Now F(x) -+ 1 as x -+ 00, since the queue is stable, and therefore A = 1 . Using the equation for F(O), we find that B = 1 (1 - .J5). 9. Q is a M(A)IM(JL)/oo queue, otherwise known as an imrnigration-death process (see Exercise (6. 1 1 .3) and Problem (6. 15.18)). As found in (6. 15.18), Q (t) has probability generating function
where p = A/ JL. Hence E(Q(t)) = I e - JL t + p (1 - e - JL t ), IP'(Q (t) = 0) = (1 - e - JL t ) 1 exp { -p(1 - e - JLt ) } , 1 as t -+ 00. IP'( Q (t) = n) -+ _p n e P
n!
_
If E(l) and E(B) denote the mean lengths of an idle period and a busy period in equilibrium, we have that the proportion of time spent idle is E(l)/{E(l) + E(B)}. This equals limt -HlO IP'( Q (t) = 0) = e - p . Now E(l) = A - I , by the lack-of-memory property of the arrival process, so that E(B) = (eP - 1)/A. 10. We have in the usual way that Q (t + 1) = A t + Q (t) - min{ l , Q (t)} where A t has the Poisson distribution with parameter A. When the queue is in equilibrium, E(Q(t)) = E(Q(t + 1)), and hence IP' (Q(t) > 0) = E (min{ l , Q (t)} ) = E(A t ) = A . We have from (*) that the probability generating function G (s) of the equilibrium distribution of Q (t) (= Q) is G(s) = E(s A t )E(s Q - min { I , Q } ) = e A (s - I ) { E(s Q - l I{ Q � l } ) + IP'(Q = O) } . 391
[11.8.11]-[11.8.13] Also,
Queues
Solutions
G(s) = lE(s Q I{Q ::: l } ) + IP'(Q = 0) ,
and hence
{ � G(S) ( � )
G(s) = eA(s - l )
+ 1-
whence
(1 - A)
}
- A) G(s) = (1 - s)(1 A 1) . ( 1 - se The mean queue length is G'(I) = i A(2 - A)/(I - A). Since service times are of unit length,
s-
and arrivals form a Poisson process, the mean residual service time of the customer in service at an arrival time is i, so long as the queue is non-empty. Hence
lE(W) = lE(Q) - ilP'(Q > 0) = 2(1 A- A) 11. The length B of a typical busy period has moment generating function satisfying MB (s) = exp{s - A + AMB (S)}; this fact may be deduced from the standard theory of M/G/l, or alternatively by a random-walk approach. Now T may be expressed as T = I + B where I is the length of the first idle period, a random variable with the exponential distribution, p eter A. It follows that MT (S) = AMB (S)/(A - s). Therefore, as required, (A - S)MT (S)
=
A exp { s - A + (A
aram
- s)MT (S) } .
If A � 1, the queue ..length at moments of departure is either null persistent or transient, and it follows that lE(T) = 00. If A < 1, we differentiate (*) and set s = 0 to obtain AlE(T) - 1 = A 2 lE(T), whence lE(T) = {A( I - A)} - I . 12. (a) Q is a birth-death process with parameters A i = A, p'i = p" and is therefore reversible in equilibrium; see Problems (6. 15.16) and (1 1 .8.3). (b) The equilibrium distribution satisfies A7Ci = W'i + l for i � 0, whence 7Ci = (1 - p)p i where p = A/ p,. A typical waiting time W is the sum of Q independent service times, so that
Mw(s) = G Q (Ms(s))
=
1 -P 1 - pp,/(p,
- s)
=
-
(1 - p) (p, - s) . p,(1 p) - s
(c) See the solution to Problem ( 1 1 .8.3). (d) Follow the solution to Problem ( 1 1 .8.3) (either method) to find that, at any time t in equilibrium, the queue lengths independent, the jth having the eqUilibrium distribution of M(A)/M(p,j )/1 . The joint mass function is therefore
are
where Pj = A/p,j . 13. The size of the queue is a birth-death process with rates A i = A, P, i = P, min{i, k}. Either solve the equilibrium equations in order to find a stationary distribution 7C , or argue as follows. The process is reversible in eqUilibrium (see Problem (6. 15.16)), and therefore A i 7Ci = P, i+I 7Ci+l for all i. These 'balance equations ' become if 0 ::::: i < k, if i � k. 392
Problems
Solutions
[11.8.14]-[11.8.14]
where = AI J.t . Therefore there exists a stationary distribution if and only if A accordingly, with
< kJ.t , and it is given
j7r = { 7roaJ li! j k if O :s i :s k, 7ro(a/ k) k I k! if i ?:. k
These are easily solved iteratively to obtain
a
k
The cost of having servers is
7ro(k). One finds, after a little computation, that Ba 2Ba 2 C = 2A + -- ' C l = A + -, 2 I -a 4 - a2 3 (A - B) + a 2 (2B - A) - 4a(A + B) + 4A Therefore a C2 - C l = (1 a)(4 ( 2 ) Viewed as a function of a, the numerator is a cubic taking the value 4A at a ° and the value -3B at a = 1 . This cubic has a unique zero at some a* (0, 1), and C l < C if and only if ° < a < a*. where 7r0 =
_
_
=
E
2 14. The state of the system is the number Q (t) of customers within it at time t. The state 1 may be
i
divided into two sub-states, being a l and a2 , where aj is the state in which server is occupied but the other server is not. The state space is therefore S = {O, ai , a2 , 2, 3, . . . }. The usual way of finding the stationary distribution, when it exists, is to solve the equilibrium equations. An alternative is to argue as follows. If there exists a stationary distribution, then the process is reversible in equilibrium if and only if
i2, . . . , ik
for all sequences i i , of states, where G = (g u v ) u. ve S is the generator of the process (this may be shown in very much the same way as was the corresponding claim for discrete-time chains in Exercise (6.5.3); see also Problem (6. 15.16» . It is clear that (*) is satisfied by this process for all sequences of states which do not include both a l and a2 ; this holds since the terms gu v are exactly those of a birth-death process in such a case. In order to see that (*) holds for a sequence containing both al and a2 , it suffices to perform the following calculation:
gO, uI gUI , 2 g2 , u2gu2. 0 = ( 1 A)A J.t2 J.t l = gO, u2gu2, 2 g2 , uI gUI , 0 ' Since the process is reversible in eqUilibrium, the stationary distribution 7r satisfies v E S, U =1= v. Therefore
7rvgv u for all u,
)
7r
7ru A = u + l (J.t l + J.t2 , and hence 7ru =
u ?:. 2,
(
A2 A 2J.t l J.t2 J.t l + J.t2
393
7ru gu v
) U-2 7r0
for u ?:. 2.
[11.8.15]-[11.8.17]
Queues
Solutions
This gives a stationary distribution if and only if A < P, l + P,2 , under which assumption Jro is easily calculated. A similar analysis is valid if there are s servers and an arriving customer is equally likely to go to any free server, otherwise waiting in turn. This process also is reversible in equilibrium, and the stationary distribution is similar to that given above. 15. We have from the standard theory that Q /L has as mass function Jrj = (I 11)l1 j , j :::: 0, where 11 is the smallest positive root of the equation x = e/L (x- l ) . The moment generating function of (1 - p, - l ) Q /L is 1 - 11 M/L (O ) = E (exp { O (l - p, - l ) Q /L }) = -I . I - l1eO ( l -/L ) Writing p, = 1 + f, we have by expanding e/L ( t} - l ) as a Taylor series that 11 = l1 (f) = 1 - 2f + O(f) as f -I- 0. This gives -
M/L (0) =
2f + O (f ) 2f + O (f ) = 1 - (1 - 2f) ( I + O f ) + O (f ) (2 - O ) f + O (f )
�
2_ 2-0
_
as f -I- 0, implying the result, by the continuity theorem. 16. The numbers P (of passengers) and T (of taxis) up to time t have the Poisson distribution with respective parameters Jrt and r:t. The required probabilities Pn = lP'(P = T + n) have generating function
n= - oo
00 00 L L lP'(P = m + n)lP'(T = m)z n n= - oo m=O 00 = L lP'(T = m)z - m G p (z) m=O 1 = GT (z - l ) G p (z) = e -(1T + T )t e (1TZ + TZ - )t ,
in which the coefficient of z n is easily found to be that given. 17. Let N(t) be the number of machines which have arrived by time t. Given that N(t) = n, the times Tl , T2 , . . . , Tn of their arrivals may be thought of as the order statistics of a family of independent uniform variables on [0, t], say VI , V2 , . . . , Vn ; see Theorem (6. 12.7). The machine which arrived at time Vj is, at time t, in the x-stage in the Y -stage repaired
}
with probability
{
a(t) f3(t) 1 - a(t) - f3(t)
>
where a(t) = lP'(V + X t) and f3(t) = lP'(V + X � t < V + X + y) , where V is uniform on [0, t], and (X, Y) is a typical repair pair, independent of V. Therefore lP' ( V( t ) = ),. V(t) = k I N(t)
=
n) =
n ! a(t) j f3(t) k (I - a(t) - f3(t)) n - k - j ' )· '. k .' (n - )· - k) .'
implying that lP' ( V( t ) = j,
00 - J..t n V(t) = k) = L e n(At) ., lP' ( V(t ) = j, V(t) = k I N(t) = n ) n=O ). ., 394
k!
Problems
Solutions
[11.8.18]-[11.8.19]
18. The maximum deficit Mn seen up to and including the time of the nth claim satisfies
{ Jt(= l Kj - Xj)}
Mn = max Mn - l ,
Xj
=
max { O, Ul , Ul + U2 , · · · , U l + U2 + . . . + Un } ,
Kj - Xj.
where the are the inter-claim times, and Uj = We have as in the analysis of GIGll that Mn has the same distribution as Vn = max{O, Un , Un + Un - l , . . . , Un + Un - l + . . . + Ul l, whence Mn has the same distribution as the (n + l)th waiting time in a M(A)/G/l queue with service times and interarrival times The result follows by Theorem ( 1 1 .3.16). 19. (a) Look for a solution to the detailed balance equations A7fi = (i + l)JL7fH l , ° ::::: i < to find that the stationary distribution is given by 7fi = (p i / i !)7fO . (b) Let Pc be the required fraction. We have by Little ' s theorem (10.5 . 1 8) that
Kj
Xj.
Pc =
s,
A(7fc - l - 7fc ) = P (7fc - l - 7fe >, JL
c :::
2,
and P l = 7f l , where tfs is the probability that channels 1 , 2, are busy in a queue the property that further calls are lost when servers are occupied.
...,s
all s
395
MlM/s having
12 Martingales
12.1 Solutions. Introduction
1. (i) We have that E(Ym ) = E{E(Ym+l I J='m )} = E(Ym+ l ), and the result follows by induction. (ii) For a submartingale, E(Ym ) � E{E(Ym+ l I J='m )} = E(Ym+l ), and the result for supermartingales follows similarly. 2. We have that :::
I, since J='n � J='n+m - l . Iterate to obtain E(Yn+m I J='n ) = E(Yn I J='n ) = Yn · 3. (i) ZnJ-L - n has mean I, and .""" ( Zn+l J-L -(n+l ) I .rn ) = J-L -(n+l ) ."1L'(Zn+l I .rn ) = J-L -n Zn , if m
(J""
(J""
where J='n = cr (Z l , Z2 , . . . , Zn). (ii) Certainly T/ Zn � 1 , and therefore it has finite mean. Also,
where the X i are independent family sizes with probability generating function G. Now G (T/) and the claim follows. 4. (i) With Xn denoting the size of the nth jump,
where J='n = cr(X l , X2 , . . . , Xn). Also EISnl (ii) Similarly E(S�) = var( Sn ) = n, and
�
=
T/,
n, so that {Sn } is a martingale.
(iii) Suppose the walk starts at k, and there are absorbing barriers at 0 and N ( ::: k). Let T be the time at which the walk is absorbed, and make the assumptions that E(S T ) = So, E(Si- - T) = S5 . Then the probability Pk of ultimate ruin satisfies
o
. Pk + N . (1
- Pk ) = k, 396
Introduction
Solutions
and therefore Pk = 1 - (k/ N) and lE(T) 5. (i) By Exercise (12.1 .2), for r ::: i ,
[12.1.5]-[12.1.9]
= keN - k) .
lE(Yr Yj) = lE{lE(Yr Yj I :J=i ) } = lE{YjlE(Yr I :J=i ) } = lE(yh , an answer which is independent of r . Therefore if i
:::::
j
::::: k.
(li) We have that
lE{ (Yk - Yj ) 2 1 :J=i } = lE(Yf l :J=i ) - 2lE(Yk Yj I :J=i ) + lE(Y/ 1 :Ft ) . Now lE(Yk Yj 1 :Ft) = lE{lE(Yk Yj 1 Jj) � J='j } = lE(YJ I J='j), and the c1aim follows. (iii) Taking expectations of the last conclusion, j ::::: k.
{ lE(Y;) : n ::: 1 } is non-decreasing and bounded, and therefore converges. Therefore, by (*), { Yn : n ::: 1 } is Cauchy convergent in mean square, and therefore convergent in mean square, by
Now
Problem (7. 1 1 . 1 1). 6. (i) Using Jensen ' s inequality (Exercise (7.9.4)),
lE(U(Yn+l ) I J='n ) ::: u (lE(Yn +l I J='n )) = u(Yn). (li) It suffices to note that lx i , x 2 , and x + are convex functions of x; draw pictures if you are in doubt
about these functions. 7. (i) This follows just as in Exercise (12.1 .6), using the fact that u{lE(Yn+l I J='n ) } ::: u (Yn ) in this case. (ii) The function x + is convex and non-decreasing. Finally, let {Sn : n ::: O} be a simple random walk whose steps are + 1 with probability P (= 1 - q > !) and - 1 otherwise. If Sn < 0, then
lE (ISn+l l l J='n ) = P (ISn l - 1 ) + q(ISn l + 1) = ISn l - (p - q) < ISn l; note that IP'(Sn < 0) > 0 if n ::: 1 . The same example suffices in the remaining case. n n S. Clearly lEli.. - 1/! (Xn) 1 ::::: i.. - sup{I 1/! (j) 1 : j E S }. Also, lE( 1/! (Xn+} ) I J='n ) = L PXn ,j 1/! (j) ::::: i.. 1/! (Xn ) jeS where J='n = cr(X 1 , X 2 , . . . , Xn ). Divide by i..n+ 1 to obtain that the given sequence is a supermartin gale.
9. Since var(Z l ) > 0, the function G, and hence also Gn, is a strictly increasing function on [0, 1].
= G n+l (Hn+l (s)) = G n(G(Hn +l (s))) and Gn(Hn (s)) = 1 , we have that G(Hn+l (s)) = Hn (s). With J='m = cr (Zk : 0 ::::: k ::::: m), lE (Hn+l (s) Zn+ l I J='n ) = G(Hn+l (S)) Zn = Hn (s) Zn .
Since 1
397
[12.2.1]-[12.3.2]
Martingales
Solutions
12.2 Solutions. Martingale differences and Hoeffding's inequality
1. Let J'f = a ( { Vj ' Wj : I ::: j ::: i }) and Yi without using the jth object, we have that lE (Z(j) I :Fj )
=
=
lE(Z I J'f). With Z(j) the maximal worth attainable
lE ( Z(j) I :Fj - d ,
Z(j) ::: Z ::: Z(j) + M.
Take conditional expectations of the second inequality, given :Fj and given :Fj - 1 , and deduce that IYj - Yj - l l ::: M. Therefore Y is a martingale with bounded differences, and Hoeffding ' s inequality yields the result. 2. Let J'f be the a-field generated by the (random) edges joining pairs (va , Vb) with I ::: a, b ::: i, and let Xi = lE(X I J'f ) . We write X (j) for the minimal number of colours required in order to colour each vertex in the graph obtained by deleting Vj . The argument now follows that of the last exercise, using the fact that X (j) ::: X ::: X (j) + 1 . 12.3 Solutions.
1. Let Tl
=
Crossings and convergence
>
rnin{n : Yn � b} , T2 = rnin{n Tl : Yn ::: a}, and define Tk inductively by T2k - l = rnin{n T2k- 2 : Yn � b} , T2k = min{n T2k- l : Yn ::: a}.
>
>
The number of downcrossings by time n is Dn (a, b; Y) = max {k : T2k ::: n}. (a) Between each pair of upcrossings of [a, b], there must be a downcrossing, and vice versa. Hence IDn (a, b; Y) - Un (a, b; Y) I ::: 1 . (b) Let Ii b e the indicator function of the event that i E (T2k - l ' T2k ] for some k, and let Zn It is easily seen that
=
n L Ii (Yj - Yi - l ) , i= 1
n � O.
Zn ::: -(b - a)Dn (a, b; Y) + (Yn - b) + ,
whence Now Ii is J'f_ l -measurable, since
{Ii
=
I} = U ({T2k - l ::: i - I} \ {T2k ::: i - l n · k
Therefore, since In � 0 and Y is a submartingale. It follows that lE(Zn ) � lE(Zn - l ) � . . . � lE(Zo) = 0, and the final inequality follows from (*). 2. If Y is a supermartingale, then - Y is a submartingale. Upcrossings of [a, b] by Y correspond to downcrossings of [-b, -a] by -Y, so that
+ a) + } lEUn(a, b; Y) = lE Dn (- b , -a ; -Y) ::: lE{(-Yn b-a 398
=
lE{(Yn - a) - } b-a '
Stopping times
Solutions
[12.3.3]-[12.4.5]
by Exercise (12.3. 1). If a, Yn :::: 0 then (Yn - a) - ::s a. 3. The random sequence {1/I (Xn ) : n :::: 1 } is a bounded supermartingale, which converges a.s. to some limit Y. The chain is irreducible and persistent, so that each state is visited infinitely often a.s.; it follows that limn-+oo (Xn) cannot exist (a.s.) unless is a constant function. 4. Y is a martingale since Yn is the sum of independent variables with zero means. Also r:l'" lP'( Zn i= 0) = r:l'" n - 2 < 00, implying by the Borel-Cantelli lemma that Zn = 0 except for finitely many values of n (a.s.); therefore the partial sum Yn converges a.s. as n -+ 00 to some finite limit. It is easily seen that an = San - l and therefore an = 8 · s n - 2 , if n :::: 3. It follows that I Yn I :::: i an if and only if I Zn I = an . Therefore
1/1
1/1
which tends to infinity as n
-+ 00. 12.4 Solutions. Stopping times
1. We have that n
{TI + T2
{ max{TI ' T2 } { min{TI ' T2 }
= n} = U (fTl = k} n { T2 = n - k} ) , ::s
k=O
n } = {TI ::s n} n {T2 ::s n} , ::s n } = {TI ::s n} U {T2 ::s n} .
Each event on the right-hand side lies in :Fn . 2. Let :Fn = a (XI , X2 , . . . , Xn) and Sn = X l + X2 + . . . + Xn . Now {N(t ) + 1
= n} = {Sn - l
::s
3. (Y+ , s:) is a submartingale, and T = min{k : Yk so that lE(Yci ) ::s lE(Yil\n ) ::s lE(Y,t), whence
t} n {Sn ::::
> t} E :Fn .
x} is a stopping time. Now 0 ::s T /\ n ::s n,
4. We may suppose that lE(Yo) < 00. With the notation of the previous solution, we have that 5. It suffices to prove that lEYs ::s lEYT , since the other inequalities are of the same form but with different choices of pairs of stopping times. Let lm be the indicator function of the event {S < m ::s T}, and define n Zn
= L Im (Ym - Ym - l ) ,
m=l Note that 1m is :Fm _ I -measurable, so that
399
O ::S n ::s N.
[12.4.6]-[12.5.2]
Martingales
Solutions
since Y is a submartingale. Therefore E(ZN) 2: E(ZN - I ) 2: . . . 2: E(Zo) = O. On the other hand, ZN = YT - Ys, and therefore E(YT ) 2: E(Ys) · 6. De Moivre ' s martingale is Yn = (q j p ) Sn , where q = 1 - p . Now Yn 2: 0, and E(Yo) = 1, and the maximal inequality gives that
(
) (
)
max Sm 2: x = lP' max Ym 2: (qj P ) x ::: (p jq) x . lP' O�m�n O�m�n Take the limit as n -+ 00 to find that Soo = s Pm Sm satisfies E(Soo)
00
U
= L lP' ( Soo 2: x) ::: -p- . q-P x =I
We can calculate E(Soo) exactly as follows. It is the case that Soo 2: x if and only if the walk ever visits the point x, an event with probability fX for x 2: 0, where f = p jq (see Exercise (5.3. 1» . The inequality of (*) may be replaced by equality. 7. (a) First, 0 n {T ::: n} = 0 E :Fn. Secondly, if A n {T ::: n} E :Fn then
AC n {T ::: n} = {T ::: n} \ (A n {T ::: n}) E :Fn. Thirdly, if A I , A 2 , ' " satisfy A i n {T ::: n} E :Fn for each i , then
(l) Ai ) n {T ::: n} = l) (Ai n {T ::: n}) I
I
Therefore :FT is a a-field. For each integer m, it is the case that
E
:Fn ·
{ {T -< n}
if m > n, {T ::: m} if m ::: n, an event lying in :Fn. Therefore {T ::: m} E :FT for all m. (b) Let A E :Fs . Then, for any n, n (A n {S ::: T}) n {T ::: n} = U (A n {S ::: m}) n {T = m}, m=O the union of events in :Fn , which therefore lies in :Fn . Hence A n {S ::: T} E :FT . (c) We have {S ::: T} = and (b) implies that A E :FT whenever A E :Fs.
{T -< m} n {T -< n} =
n,
12.5 Solutions.
Optional stopping
1. Under the conditions of (a) or (b), the family {YT An n 2: O} is uniformly integrable. Now T /\ n -+ T as n -+ 00, so that YT An -+ YT a.s. Using uniform integrability, E(YT An) -+ E(YT ), and the claim follows by the fact that E(YT An ) = E(Yo). 2. It suffices to prove that {YT An n 2: O} is uniformly integrable. Recall that {Xn : n 2: O} is :
uniformly integrable if
:
400
Optional stopping
Solutions
[12.5.3]-[12.5.5]
(a) Now, E ( IYTAn II{ l YTAn I � a}) = E ( IYT II{T ::: n , I YT I � a}) + E (IYn II{T>n, I Yn l � a}) :::: E (I YT II{ I YT I � a} ) + E (IYn II{T>n}) = g(a) + hen), say. We have that g(a) � 0 as a � 00 , since EIYT I < 00 . Also hen) � 0 as n � 00 , so that sUPn> N hen) may be made arbitrarily small by suitable choice of N. On the other hand, E (IYn II{ I Yn l � a}) � 0 as a � 00 uniformly in n E {O, 1 , . . . , N}, and the claim follows. (b) Since Y;i defines a submartingale, we have that supn E(YiAn ) :::: sUPn E(Y;i ) < 00 , the second inequality following by the uniform integrability of {Yn } . Using the martingale convergence theorem, YT lin � YT a.s. where E I YT 1 < 00. Now Also lP'(T > n) � 0 as n � 00, so that the final two terms tend to 0 (by the uniform integrability of the Yi and the finiteness of EI YT 1 respectively). Therefore YT An � YT , and the claim follows by the standard theorem (7. 10.3). 3. By uniform integrability, Y00 = limn--+oo Yn exists a.s. and in mean, and Yn = E(Y00 1 .'Fn ). (a) On the event {T = n} it is the case that YT = Yn and E(Y00 1 .'FT ) = E(Y00 1 .'Fn ) ; for the latter statement, use the definition of conditional expectation. It follows that YT = E(Y00 1 .'FT ), irrespective of the value of T. (b) We have from Exercise (12.4.7) that .'Fs � .'FT . Now Ys = E(Yoo 1 .'Fs) = E{E(Yoo 1 .'FT ) 1 .'Fs) = E(YT 1 .'Fs) · 4. Let T be the time until absorption, and note that {Sn } is a bounded, and therefore uniformly integrable, martingale. Also JP'(T < 00 ) = 1 since T is no larger than the waiting time for N consecutive steps in the same direction. It follows that E(So) = E(ST ) = NJP'(ST = N), so that JP'(ST = N) = E(So)/ N. Secondly, {S� - n : n 2': O} is a martingale (see Exercise (12. 1 .4» , and the optional stopping theorem (if it may be applied) gives that and hence E(T) = NE(So) - E(S5 ) as required. It remains to check the conditions of the optional stopping theorem. Certainly JP'(T < 00 ) = 1 , and in addition E(T 2 ) < 00 by the argument above. We have that E I S i- - TI :::: N 2 + E(T) < 00 . Finally, E { (S� - n)I{T>n) } :::: (N 2 + n)JP'(T > n) � 0
as n � 00, since E(T 2 ) < 00 . 5. Let .'Fn = O' (S l , S2 , . . . , Sn). It is immediate from the identity cos(A + A) + cos(A - A) = 2 cos A cos A that E(Yn + 1 1 .'Fn ) =
COS[A(Sn + 1
- ! (b - a))] + COS[A(Sn - 1 - i (b - a))] 2(COS A) n + 1
and therefore Y is a martingale (it is easy to see that E I Yn 1 < 00 for all n). Suppose that 0 < A < rr/(a + b), and note that 0 :::: IA{Sn - i (b - a)}1 < n :::: T. Now YT An constitutes a martingale which satisfies
40 1
= Yn ,
iA(a + b)
(T > n (a + b)) ::::: { I - ( � ) a+ b }n --+ 0 as n --+ 00. Therefore, lE l si - T I ::::: lE(S i ) + lE(T) ::::: (a + b) 2 + lE(T) and
< 00, as n --+ 00.
402
Problems
Solutions
[12.7.1]-[12.9.2]
12.7 Solutions. Backward martingales and continuous-time martingales
1. Let s ::: t. We have that lE(l1 (X (t)) I :Fs , Xs = i) = l:.j Pij (t - s)l1(j). Hence � lE ( l1 (X (t)) I :Fs , X = i) = (P G rl') = 0, s t-s l· dt so that lE(l1 (X (t)) I :Fs , Xs = i) = l1 (i), which is to say that lE(l1 (X (t)) I :Fs ) = l1 (X (S)). 2. Let Wet) = exp{- O N(t) + At (l - e -li)} where 0 2: 0. It may be seen that W et /\ Ta ), t 2: 0, constitutes a martingale. Furthermore
I W (t /\ Ta ) 1 ::: exp { A (t /\ Ta ) ( 1
- e -li ) } t exp { A Ta O - e -li ) } as t � 00 ,
where, by assumption, the limit has finite expectation for sufficiently small positive 0 (this fact may be checked easily). In this case, ( W et /\ Ta ) : t 2: o} is uniformly integrable. Now W et /\ Ta ) � W ( Ta ) a.s. as t � 00, and it follows by the optional stopping theorem that 1
=
lE(W(O))
=
lE (W (t 1\ Ta ) ) � lE(W( Ta ))
=
e -li a lE{e A Ta ( l -e -O ) }.
Write s = e -Ii to obtain s - a = lE{e A Ta ( l -S) }. Differentiate at s = 1 to find that a = AlE( Ta ) and a (a + 1) = A 2 lE( T; ), whence the claim is immediate. 3. Let 9. m be the a -field generated by the two sequences of random variables Sm , Sm+ I . . . , Sn and Um+ l , Um+2 , . . . , Un . It is a straightforward exercise in conditional density functions to see that
m-1 IoUm+2 (m + l)xm+l dx
- I I 9. m+l ) = lE(Um+1 o
( Um+
2)
=
m+1
, m Um+-2
whence lE(Rm I 9. m+ l ) = Rm+1 as required. [The integrability condition is elementary.] Let T = maxIm : Rm 2: 1 } with the convention that T = 1 if Rm < 1 for all m . As in the closely related Example (12.7.6), T is a stopping time. We apply the optional stopping theorem 02.7.5) to the backward martingale R to obtain that lE(R T I 9. n ) = Rn = Snit. Now, R T 2: 1 on the event {Rm 2: 1 for some m ::: n}, whence = lE(R T I Sn = y) 2: lP' (Rm 2: 1 for some m ::: n I Sn = y) . t [Equality may be shown to hold. See Karlin and Taylor 198 1 , pages 1 10-1 13, and Example 02.7.6).]
�
12.9 Solutions to problems
1. Clearly lE(Zn ) ::: (fJ, + m)n , and hence lElYn I < 00. Secondly, Zn+ I may be expressed as l:.f� l Xi + A, where X l , X2 , . · . are the family sizes of the members of the nth generation, and A is the number of immigrants to the (n + l)th generation. Therefore lE(Zn +1 I Zn ) = fJ,Zn + m , whence 1 - fJ,n + l 1 = Yn · lE( Yn +1 I Zn) = n +1 fJ,Zn + m 1 1 _ fJ, fJ,
(
{
)
}
2. Each birth in the (n + l)th generation is to an individual, say the sth, in the nth generation. Hence, for each r, B (n+I ) ,r may be expressed in the form B (n +1 ) ,r = Bn, s + Bj (s), where Bj (s) is the age
of the parent when its jth child is born. Therefore
{
I }
lE L e -li B (n+ l ) , r :Fn r
=
lE
{ �S,] e-li (Bn,s+Bj (s)) I :Fn } 403
=
L e -li Bn , s M I (0 ) , s
[12.9.3]-[12.9.5]
Martingales
Solutions
which gives that E(Yn +1 I :Fn) = Yn . Finally, E(YI (0)) = 1 , and hence E(Yn (O)) = 1 . 3. If x, C > 0 , then
Now (Yk + c) 2 is a convex function of Yk . and therefore defines a submartingale (Exercise (12. 1 .7)). Applying the maximal inequality to this submartingale, we obtain an upper bound ofE{ (Yn + c) 2 } / (x + c) 2 for the right-hand side of (*). We set c = E(Y; ) /x to obtain the result. 4. (a) Note that Zn = Zn - l + Cn {Xn - E(Xn I :Fn - l)}, so that (Z, F) is a martingale. Let T be the stopping time T = rnin{k : q Yk � x}. Then E(ZT/\n) = E(Zo) = 0, so that
since the final term in the defiuition of Zn is non-negative. Therefore n xJP'(T ::::: n) ::::: E{CT/\n YT/\n } ::::: I'>k E{E(Xk I :Fk -d } , k= l
where we have used the facts that Yn � ° and E(Xk I :Fk - l ) � 0. The claim follows. (b) Let Xl , X2 , . . . be independent random variables, with zero means and fiuite variances, and let Yj = � {= l Xi · Then Y} defines a non-negative submartingale, whence
5. The function h (u) = lu l r is convex, and therefore Yi ( ) = l Si - Sm l r , i � defines a submartingale with respect to the filtration :Fi = a ( { Xj 1 ::::: j ::::: i}) . Apply the HRC inequality of Problem (12.9.4), with q = 1 , to obtain the required inequality. If r = 1 , we have that m+n E (I Sm +n - Sm l) ::::: L E I Zk l k =m+ l :
m
m,
by the triangle iuequality. Let n -+ 00 to find, in the usual way, that the sequence {Sn} converges a.s.; Kronecker ' s lemma (see Exercise (7.8.2)) then yields the final claim. Suppose 1 < r ::::: 2, in which case a little more work is required. The function h is differentiable, and therefore h (v) - h (u) = (v - u)h ' (u) + {h ' (u + x) - h ' (u) } dx. Now h'(y) = r l y l r- l sign(y) has a derivative decreasing in Iy l . It follows (draw a picture) that h' (u + x) - h'(u) ::::: 2h' ( 1 x) if x � 0, and therefore the above integral is no larger than 2h(�(v - u)). Apply this with v = Sm +k+ l - Sm and u = Sm+k - Sm, to obtaiu m,
fov -u
404
Problems
Solutions
[12.9.6]-[12.9.10]
Sum over k and use the fact that to
deduce that
m+n E(ISm+n - Sm n :::; 22 - r L E(IZk n · k=m+ l
The argument is completed as after (*). 6. With It = I{ Yk =O} , we have that
E(Yn I J="'n - l ) = E ( XnIn - l + nYn - l IXn l (1 - In - I ) I J="'n - l ) = In l E(Xn) + n Yn - l (1 - In - I )EIXn l = Yn - l since E(Xn) = 0, EIXn l = n - l . Also EIYn l :::; E{ IXn l (1 + n lYn - l D } and ElYl l < 00, whence EIYn l < 00. Therefore (Y, !F) is a martingale. Now Yn = 0 if and only if Xn = O. Therefore lP'(Yn = 0) = lP'(Xn = 0) = 1 - n - 1 � 1 as n � 00, implying that Yn � O. On the other hand, Ln lP'(Xn =1= 0) = 00, and therefore lP'(Yn =1= 0 i.o.) = 1 by the second Borel-Cantelli lemma. However, Yn takes only integer values, and therefore Yn does not converge to 0 a.s. The martingale convergence theorem is inapplicable since sUPn EIYn l = 00. 7. Assume that t > 0 and M(t) = 1. Then Yn = e t Sn defines a positive martingale (with mean 1) with respect to J="'n = a (X 1 , X2 , . . . , Xn). By the maximal inequality,
and the result follows by taking the limit as n � 00. S . The sequence Yn = ;Zn defines a martingale; this may be seen easily, as in the solution to Exercise (12.1.15). Now {Yn } is uniformly bounded, and therefore Yoo = limn -+ oo Yn exists a.s. and satisfies E(Y = E(Yo) = ; . Suppose 0 < ; < 1 . In this case Z l is not a.s. zero, so that Zn cannot converge a.s. to a constant c unless C E { O, oo}. Therefore the a.s. convergence of Yn entails the a.s. convergence of Zn to a limit random variable taking values 0 and 00. In this case, E(Yoo ) = 1 · lP'(Zn � 0) + 0 · lP'(Zn � (0), implying that lP'(Zn � 0) = ; , and therefore lP'(Zn � (0) = 1 - ;.
00 )
9. It is a consequence of the maximal inequality that lP'(Y; � x) :::; x - 1 E(Yn I{ Y,i 2:x } ) for x > O. Therefore
1000 lP'(Y; � x) dx :::; 1 + 100 lP'(Y; � x) dx :::; 1 + E { Yn 1 00 x - I 1( 1, Y,i j (x) dX }
E(Y; ) =
=
1 + E(Yn log+ Y;) :::; 1 + E(Yn log+ Yn) + E(Y;) / e .
10. (a) We have, as in Exercise (12.7.1), that
E(h (X(t)) I B, X(s) = i) = L Pij (t)h(j) for s < t, j
405
[12.9.11]-[12.9.13]
Martingales
Solutions
for any event B defined in terms of (X(u) : u ::: s}. The derivative of this expression, with respect to t, is (Pt Gh/)j , where Pt is the transition semigroup, G is the generator, and h = (h (j) : j � 0). In this case,
(Gh/ )j
=
L gj k h (k) = Aj {h(j + 1) - h (j) } - J-Lj {h(j) - h (j - I ) } = 0 k
for all j . Therefore the left side of (*) is constant for t � s, and is equal to its value at time s, i.e. X(s). Hence h (X(t)) defines a martingale. (b) We apply the optional stopping theorem with T = min{t : X(t) E to, n}} to obtain lE(h (X(T))) = lE(h(X(O))), and therefore ( I - :n: (m))h(n ) = h (m) as required. It is necessary but not difficult to check the conditions of the optional stopping theorem. 11. (a) Since Y is a submartingale, so is y + (see Exercise (12. 1 .6)). Now Therefore (lE(Y,i+m I :Fn) : m � O} is (a.s.) non-decreasing, and therefore converges (a.s.) to a limit Mn . Also, by monotone convergence of conditional expectation, and furthermore lE(Mn) = limm -+ oo lE(Y';;+n ) ::: M. It is the case that Mn is :Fn-measurable, and therefore it is a martingale. (b) We have that Zn = Mn - Yn is the difference of a martingale and a submartingale, and is therefore a supeI1lUllj:ingale. Also Mn � Y;t � 0, and the decomposition for Yn follows. (c) In thi; case Zn is a martingale, being the difference oftwo martingales. Also Mn � lE(Y;t I :Fn) = Y;t � Yn a.s., and the claim follows. 12. We may as well assume that J-L < P since the inequality is trivial otherwise. The moment generating function of P - C l is M(t) = /(P - /L) + !u2 t 2 , and we choose t such that M(t) = 1 , i.e., t = -2(P - J-L)/a 2 . Now define Zn = min{ e t Yn , I } and :Fn = a (C l , C2 , . . . , Cn). Certainly lEl Zn l < 00; also lE(Zn + l I :Fn) ::: lE(e t Yn 1 I :Fn) = e t Yn M(t) = e t Yn
+
and lE(Zn +1 I :Fn) ::: 1 , implying that lE(Zn + l I :Fn) ::: Zn . Therefore (Zn , :Fn) is a positive supermartingale. Let T = inf{n : Yn ::: O } = inf{n : Zn = I}. Then T /\ m is a bounded stopping time, whence lE(Zo) � lE(ZT Am ) � lP'(T ::: m). Let m --+ 00 to obtain the result. 13. Let :Fn = a (R l , R2 , . . . , Rn). (a) 0 ::: Yn ::: 1 , and Yn is :Fn-measurable. Also
whence Yn satisfies lE(Yn +1 I :Fn) = Yn . Therefore {Yn : n � O} is a uniformly integrable martingale, and therefore converges a.s. and in mean. (b) In order to apply the optional stopping theorem, it suffices that lP'(T < 00) = 1 (since Y is uniformly integrable). However lP'(T n) = � . � . . . n� l = (n + 1) - 1 --+ O. Using that theorem, lE(YT) = lE(Yo), which is to say that lE{T/(T + 2)} = � , and the result follows. (c) Apply the maximal inequality.
>
406
Problems
Solutions
[12.9.14]-[12.9.17]
14. As in the previous solution, with 9.n the a-field generated by A I , A 2 , . . . and :Fn, Rn + An Bn Rn Rn lE( Y.n+1 I 9. n ) + Rn + Bn + An Rn + Bn Rn + Bn + An Rn + Bn Rn Rn + Bn - Y.n , _
)(
) (
)(
(
)
_
so that lE(Yn +1 I :Fn) = lE { lE(Yn +1 I 9.n) I :Fn } = Yn · Also I Yn I ::: 1 , and therefore Yn is a martingale. We need to show that lP'(T < (0) = 1 . Let In be the indicator function of the event { T > n}. We have by conditioning on the A n that
) 00 (
(
n- I 1 1 lE(ln I A) = II 1 - ---+ II 1 - -2 + Sj 2 + S j j =O j =O
)
as n --+ 00, where Sj = 'E, {= I A i . The infinite product equals 0 a.s. if and only if 'E,j (2 + Sj ) - 1 = 00 a.s. By monotone convergence, lP'(T < (0) = 1 under this condition. If this holds, we may apply the optional stopping theorem to obtain that lE(YT ) = lE(Yo), which is to say that
(
lE 1
_
1 + AT 2 + ST
)
=
�. 2
15. At each stage k, let Lk be the length of the sequence 'in play ' , and let Yk be the sum of its entries, so that Lo = n, Yo = 'E,?=I xi . If you lose the (k + l )th gamble, then Lk+1 = Lk + 1 and Yk+1 = Yk + Zk where Zk is the stake on that play, whereas if you win, then Lk +1 = Lk - 2 and Yk+1 = Yk - Zk ; we have assumed that Lk � 2, similar relations being valid if Lk = 1 . Note that Lk is a random walk with mean step size - 1 , implying that the first-passage time T to 0 is a.s. finite, and has all moments finite. Your profits at time k amount to Yo - Yk , whence your profit at time T is Yo, since YT = O. Since the games are fair, Yk constitutes a martingale. Therefore lE(YT Am ) = lE(YO ) =I- 0 for all m . However T 1\ m --+ T a.s. as m --+ 00, so that YT Am --+ YT a.s. Now lE(YT ) = 0 =I limm --+oo lE(YT Am ), and it follows that {YT A m : m � I } is not uniformly integrable. Therefore lE(suPm YT Am ) = 00; see Exercise (7. 10.6). 16. Since the game is fair, lE(Sn +1 I Sn ) = Sn . Also ISn l ::: 1 + 2 + . . . + n < 00 . Therefore Sn is a martingale. The occurrence of the event {N = n} depends only on the outcomes of the coin-tosses up to and including the nth; therefore N is a stopping time. A tail appeared at time N - 3, followed by three heads. Therefore the gamblers G I , G 2 , . . . , G N - 3 have forfeited their initial capital by time N, while G N - i has had i + 1 successful rounds for 0 ::: i ::: 2. Therefore SN = N - (p - I + p -2 + p - 3 ), after a little calculation. It is easy to check that N satisfies the conditions of the optional stopping theorem, and it follows that lE(SN ) = lE(So) = 0, which is to say that lE(N) = p - I + p -2 + p - 3 . In order to deal with HTH, the gamblers are re-programmed to act as follows. If they win on their first bet, they bet their current fortune on tails, returning to heads thereafter. In this case, SN = N - (p - I + p -2 q - l ) where q = 1 - p (remember that the game is fair), and therefore lE(N) = p -I + p -2 q - l . 17. Let :Fn = a ({ Xi , Yi : 1 ::: i ::: n} ) , and note that T is a stopping time with respect to this filtration. Furthermore lP'(T < (0) = 1 since T is no larger than the first-passage time to 0 of either of the two single-coordinate random walks, each of which has mean 0 and is therefore persistent. Let al = var(XI ) and ai = var(YI ) · We have that Un - Uo and Vn - Vo are sums of independent summands with means 0 and variances al and ai respectively. It follows by considering 407
[12.9.18]-[12.9.19]
Martingales
Solutions
the martingales (Un - UO ) 2 -na'f and (Vn - VO ) 2 -na:j (see equation (12.5.14) and Exercise (10.2.2)) that E{(UT - UO ) 2 } = a'f E(T ), E{(VT - VO ) 2 } = a:j E(T ). Applying the same argument to (Un + Vn) - (Uo + Yo), we obtain
E{ (UT + VT - Uo - VO ) 2 } = E(T)E{ (X 1 + Y1 ) 2 } = E(T ) (a'f + 2c + a:j ).
Subtract the two earlier equations to obtain
E{ (UT - UO )(VT - Vo) } = cE(T) if E(T) < 00. Now UT VT = 0, and in addition E(UT ) = Uo, E(VT ) = Yo, by Wald ' s equation and the fact that E(X 1 ) = E( Y1 ) = O. It follows that -E(Uo yo) = cE(T) if E(T) < 00, in which case c < o. Suppose conversely that c < O. Then (*) is valid with T replaced throughout by the bounded stopping time T /\ m, and hence o � E(UT Am VT Am ) = E(UO yO ) + cE(T /\ m).
Therefore E(T /\ m) � E(Uo Vo)/(2Icl) for all m, implying that E(T) = limm-+ oo E(T /\ m) < 00, and so E(T) = -E(Uo Vo)/c as before. 18. Certainly 0 � Xn � 1, and in addition Xn is measurable with respect to the a-field :Fn = a (R 1 , R2 , . . . , Rn). Also E(Rn+1 I Rn) = Rn - Rn/(52 - n), . whence E(Xn + 1 I :Fn) = Xn · Therefore Xn is a martingale. A strategy corresponds to a stopping time. If the player decides to call at the stopping time T, he wins with (conditional) probability XT, and therefore lP'(wins) = E(X T ), which equals E(Xo) (= �) by the optional stopping theorem. Here is a trivial solution to the problem. It may be seen that the chance of winning is the same for a player who, after calling "Red Now", picks the card placed at the bottom of the pack rather than that at the top. The bottom card is red with probability ! , irrespective of the strategy of the player. 19. (a) A sum of money in week t is equivalentto a sum / (1 +a) t in week 0, since the latter sum may be invested now to yield in week t. If he sells in week t, his discounted costs are E� = l c/(1 + a) n and his discounted profit is Xt /(1 + a) t . He wishes to find a stopping time for which his mean discounted gain is a maximum. Now T - L (1 + a) -n c = a� {(1 + a) - T - I } , n= l T so that JL(T) = E{ (1 + a) - Z T } - (c/a). (b) The function h (y) = ay - J; lP'(Zn > y) dy is continuous and strictly increasing on [0, 60), with h (O) = -E(Zn) < 0 and h (y) � 00 as y � 00. Therefore there exists a unique y (> 0) such that h (y) = 0, and we choose y accordingly. (c) Let :Fn = a (Zl , Z2 , . . . , Zn). We have that oo E (max { Zn , y } ) = y + [1 - G(y)] dy = (1 + a)y s
s
s
l
where G(y) = lP'(Zn � y). Therefore E(Vn + 1 I :Fn) non-negative supermartingale. 408
=
(1 + a) -n y
�
Vn, so that (Vn , :Fn) is a
[12.9.20]-[12.9.21] Let fL(-e) be the mean gain of following the strategy 'acceptn the first offer exceeding -e - (cia) , . The corresponding stopping time T satisfies lP'(T = n) = G (-e) (1 - G (-e)), and therefore
Problems
Solutions
fL(-e) + (cia) = L E{ (1 + a) - T ZT I{ T =n J } n =O 00 n n = L (1 + a) - G(-e) (1 - G (-e))E(Z I I Z I > -e) n =O 1 +a (1 - G(y)) dy . = -e ( 1 G (-e)) + 1 + a - G(-e) J, Differentiate with care to find that the only value of -e lying in the support of Z 1 such that fL' ( -e) = 0 is the value -e = y. Furthermore this value gives a maximum for fL (-e). Therefore, amongst strategies of the above sort, the best is that with -e = y . Note that fL( Y ) = y (1 + a) - (cia). Consider now a general strategy with corresponding stopping time T, where lP'(T < 00) = 1 . For any positive integerm, T I\m is abounded stopping time, whence E(VTAm ) ::::; E(Vo) = y (1 +a). Now I VT Am I ::::; I:�o l Vi I , and I:�o El Vi i < 00. Therefore {VT Am : m � O} is uniformly integrable. Also VTAm � VT a.s. as m � 00, and it follows that E(VT Am ) � E(VT). We conclude that fL(T) = E(VT) - (cia) ::::; y(1 + a) - (cia) = fL( Y ). Therefore the strategy given above is optimal. (d) In the special case, lP'(Zl > y) = (y - 1) - 2 for y � 2, whence Y = 10. The target price is therefore 9, and the mean number of weeks before selling is G(y)/(1 - G(y)) = 80. 20. Since G is convex on [0, 00) wherever it is finite, and since G(I) = 1 and G'(1) < 1, there exists a unique value of TJ (> 1) such that G(TJ) = TJ. Furthermore, Yn = TJZn defines a martingale with mean E(Yo) = TJ. Using the maximal inequality (12.6.6), 00
{
lP' ( sup n Zn � k)
for positive integers k. Therefore
=
_
roo
}
k lP'( Sup n Yn � TJ ) ::::; TJk - l 1
00
1 -. E ( suP n Zn) ::::; kL TJ 1 =1
21. Let Mn be the number present after the nth round, so Mo = K, and Mn + l = Mn - Xn + l , n � 1 , where Xn is the number of matches in the nth round. By the result of Problem (3. 1 1 . 17), EXn = 1 for all n, whence E(Mn + l + n + 1 1 :Fn) = Mn + n , where :Fn is the a-field generated by Mo, Ml , . . . , Mn . Thus the sequence {Mn + n} is a martingale. Now, N is clearly a stopping time, and therefore K = Mo + 0 = E(MN + N ) = EN . We have that E{ (Mn +1 + n + 1) 2 + Mn +1 1 :Fn } = (Mn + n) 2 - 2(Mn + n)E(Xn + l - 1) + Mn + E{ (Xn + l - 1) 2 - Xn + l I :Fn } ::::; (Mn + n) 2 + Mn , where we have used the fact that
var(Xn + l I :Fn) = I if Mn > 1 , 0 if Mn = 1 .
{
409
[12.9.22]-[12.9.24]
Martingales
Solutions
Hence the sequence {(Mn + n) 2 + Mn } is a supermartingale. By an optional stopping theorem for supermartingales,
and therefore var(N) ::::: K. 22. In the usual notation,
lE ( M(s + t) I J=S )
=
=
lE (IoS W(u) du + l
s+t
W(u) du - H W(s + t) - W(s) + W(s) } 3 1 J=S
M(s ) + t W(s ) - W(s)lE ( [W(s + t) - W(s)] 2 1 .1"'s )
=
M(s)
)
as required. We apply the optional stopping theorem (12.7. 12) with the stopping time T = inf { u : W(u) E {a, b} } . The hypotheses of the theorem follow easily from the boundedness of the process for t E [0, T], and it follows that
, Hence the required area A has mean
[We have used the optional stopping theorem twice actually, in that lE(W(T))
lP'(W(T) = a) = -b/ (a - b) .] 23. With .1"'s = a(W(u) : 0 ::::: u ::::: s ), we have for s < t that
lE(R(t) 2 I J=S ) = lE (I W(s ) 1 2 + I W(t) - W(s ) 1 2 + 2W(s ) · (W(t) - W(s)) l .1"'s )
=
=
0 and therefore R(s) 2 + (t - s),
and the first claim follows. We apply the optional stopping theorem (12.7. 12) with T = inf{u : I W(u) 1 = a } , as in Problem (12.9.22), to find that 0 = lE(R(T) 2 - T) = a 2 - lE(T). 24. We apply the optional stopping theorem to the martingale W(t) with the stopping time T to find that lE(W(T)) = -a (1 - Pb) + bPb = 0, where Pb = lP'(W(T) = b). By Example (12.7. 10), W(t) 2 - t is a martingale, and therefore, by the optional stopping theorem again, whence lE(T) = a b o For the final part, we take a martingale exp[e W(t) - � e2 t] to obtain
=
b and apply the optional stopping theorem to the
on noting that the conditional distribution of T given W (T) = b is the same as that given W (T) 1 02 Therefore, lE(e - :Z T ) = 1/ cosh (be) , and the answer follows by substituting s = !e 2 .
410
=
-b.
13 Diffusion processes
13.3 Solutions. Diffusion processes
1. It is easily seen that lE{ X(t + h) - X(t) I X(t) } = (A - fJ,)X(t)h + o(h), lE ( {X(t + h) - X(t) } 2 1 X(t)) = (A + fJ,)X(t)h + o (h ) , which suggest a diffusion approximation with instantaneous mean a (t, x) = (A - fJ,)x and instanta neous variance bet, x) = (A + fJ,)x. 2. The following method is not entirely rigorous (it is an argument of the following well-known type: it is valid when it works, and not otherwise). We have that
by using the forward equation and integrating by parts. Assume that a (t, y) l:n f3n (t)y n . The required expression follows from the 'fact ' that
100 /ly yn f dy - 00
an ao n
= -
100 /ly f dy - 00
=
=
l:n an (t)y n , b(t, y) =
an M ao n
-- .
3. Using Exercise (13.3.2) or otherwise, we obtain the equation aM = + 102M
OmM
at
2
with boundary condition M(O, 0) = 1 . The solution is M(t) = exp{ � + O)t}. 4. Using Exercise (13.3.2) or otherwise, we obtain the equation aM = -0 aM + 1 2 M at ao with boundary condition M(O, 0) = 1 . The characteristics of the equation are given by dt dO = 2 dM
O(2m
zO
-
T
=
e
02M '
o2 g(Oe-t ) where g is a function satisfying 1 e ! o 2 g(O). Therefore e e- t
with solution M(t, 0) = ! M = exp{ A 0 2 (1 - 2 ) } .
=
41 1
[13.3.5]-[13.3.10]
Diffusion processes
Solutions
5. Fix t > O. Suppose we are given WI (s), W2 (S), W3 (S), for O � s � t. By Pythagoras ' s theorem, R(t + u) 2 = Xr + X� + X� where the Xi are independent N (Wi (t), u) variables. Using the result of Exercise (5.7.7), the conditional distribution of R(t + u) 2 (and hence of R(t + u) also) depends only on the value of the non-centrality parameter e = R(t) 2 of the relevant non-central X 2 distribution. It follows that R satisfies the Markov property. This argument is valid for the n-dimensional Bessel process. 6. By the spherical symmetry of the process, the conditional distribution of R(s +a) given R(s) = x is the same as that given W(s) = (x, 0, 0) . Therefore, recalling the solution to Exercise (13.3.5), lP'(R(s + a) � y I R(s) = x) i - x) 2 + v 2 + w 2 du dv dw = , v , w ) : (2rra ) 3 /2 exp ( u 2a u2 + v2 +w2 �y2 Y 2rr rr 2 I e 2 exp _ p - 2px cos + x p 2 sin e de d 0, t+u t +u W(s) ds :Ft = (t + u) W(t) W(s) ds _ W(t) ds E (t + u) W(t + u) -
{ (U { ) (
J
{ (
{
Io
1 }
= tW(t) -
}
)}
}
1
-l
lot W(s) ds,
whence this is a martingale. [The integrability condition is easily verified.] 9. (a) With s < t, S(t) = S(s) exp{a(t - s) + b(W(t) - W(s))}. Now W(t) - W(s) is independent of {W(u) : 0 � u � s}, and the claim follows. (b) S(t) is clearly integrable and adapted to the filtration :F= (:Ft) so that, for s < t, E ( S(t) I Fs ) = S(s)E (exp{a(t - s) + b(W(t) - W(s))} I Fs) = S(s) exp{a(t - s) + 1 b2 (t - s)}, which equals S(s) if and only if a + 1 b2 = O. In this case, E(S(t)) = E(S(O)) = 1. 10. Either find the instantaneous mean and variance, and solve the forward equation, or argue directly as follows. With s < t, lP'(S(t) � y I S(s) = x ) = lP'(bW(t) � -at + log y I bW(s) = -as + log x ) . Now b(W(t) - W(s)) is independent of W(s) and is distributed as N (O, b2 (t - s)), and we obtain on differentiating with respect to y that 2 1 exp - (lOg(Ylx ) - a(t - s)) ' x, y > O. f( t, Y I s, x ) = 2 2b2 (t - s) y v2rrb (t - s)
(
)
412
Excursions and the Brownian bridge
Solutions
[13.4.1]-[13.6.1]
13.4 Solutions. First passage times
1. Certainly X has continuous sample paths, and in addition E I X (t ) 1 < 00. Also, if s < t, 1 2 1 2 . 1 2 E( X (t ) l .1's ) = X (s ) e 2 (J (t -S) E (el(J { W (t) -W (s» ) I Fs ) = X (s ) e :Z (J (t -s) e - :Z (J (t -s) = X es ) as required, where we have used the fact that W(t ) - W(s ) is N(O, t - s) and is independent of .1's .
2. Apply the optional stopping theorem to the martingale X of Exercise (13.4. 1), with the stopping time T, to obtain E (X (T)) = 1 . Now WeT) = a T + b, and therefore E(e VtT + i Bh ) = 1 where 1{f = ia() + �()2. Solve to find that E(eVtT ) = e- i (J h = exp -b ( Ja 2 - 21{f + a)
{
}
is the solution which gives a moment generating function. 3. We have that T ::: u if and only if there is no zero in (u, t], an event with probability 1 (2j 1r) cos - 1 { .JUTt}, and the claim follows on drawing a triangle. 13.5 Solution. Barriers
1. Solving the forward equation subject to the appropriate boundary conditions, we obtain as usual
that
P (t, y) = g(t, y I d) + e-2md g(t, y I -d) 1
- i: 2m e2mx g(t, y I x) dx
where get, y I x ) = (21rt) - :Z exp { -(y - x - mt) 2 j(2t) } . The first two terms tend to 0 as t � 00, regardless of the sign of m. As for the integral, make the substitution u = (x - y - mt) j.../i to obtain, as t � 00,
1 2 j e- I I Y - (d +y+m t ) /.ji 2mY e - :Z u 2me -- du � { 2 I m l 2 m -00 v'2rr 0
if m if m
< 0, � O.
13.6 Solutions. Excursions and the Brownian bridge
1. Let f(t, x) = (21rt) :Z e-x 2 / (2t ) . It may be seen that _ I
IP' ( W(t ) > x I z, W(O)
=
lim IP' ( W(t ) > x I z, W(O) = w ) 0) = w.j.O
where Z = {no zeros in (0, t]); the small missing step here may be filled by conditioning instead on the event {WeE) = w , no zeros in (E, t]), and taking the limit as E + O. Now, if W > 0,
IP' ( W(t) > x, Z I W(O) = w ) = by the reflection principle, and
IP'(Z I W(O)
=
w)
=
1-2
100 { J (t,
Y
- w) - f(t, y + w ) } dy
LOO f(t, y ) dy i: f(t, y ) dy =
413
[13.6.2]-[13.6.5]
Diffusion processes
Solutions
by a consideration of the minimum value of W on (0, t]. It follows that the density function of W(t), conditional on Z n { W (O) = }, where > 0, is f(t, x - - f(t, x + h w (x ) ' x > O. f-Ww f (t, y) dy Divide top and bottom by and take the limit as -l- 0: 1 af x x 2 /(2t) lim h (x) = = -e x > O. , w ,!, o w f (t, O) ax t
w
w
_
w)
w)
2w,
w
- --
-
2. It is a standard exercise that, for a Wiener process W, E { W(t) I W(s )
= a,
W(1)
=
o}
E { W (s ) 2 1 W(O) = W(1) = o}
= a
( I -t) , -
l-s = s (1 - s),
if 0 ::: s ::: t ::: 1 . Therefore the Brownian bridge B satisfies, for 0 � s ::: t ::: 1 , 1 -t E ( B (s ) B (t) ) = E { B (s )E ( B(t) I B (s ) ) } = E(B(s ) 2 ) = s (1 - t) l -s as required. Certainly E(B(s )) = 0 for all s, by symmetry.
3. W is a zero-mean Gaussian process on [0, 1] with continuous sample paths, and also W(O) = W ( I ) = O. Therefore W is a Brownian bridge if it has the same autocovariance function as the
Brownian bridge, that is, c(s, t) cov (W (s ) , W (t) )
=
since cov(W(u), W(v))
=
=
min{s, t} - st. For s < t,
cov ( W(s) - s W ( I ) , W(t) - tW(1) )
=
s - ts - st + s t = s - st
min{u, v}. The claim follows.
4. Either calculate the instantaneous mean and variance of W, or repeat the argument in �e solution to Exercise (13.6.3). The only complication in this case is the necessity to show that W(t) is a.s. continuous at t = 1, Le., that u - 1 W(u - 1) --+ 0 a.s. as u --+ 00. There are various ways to show this. Certainly it is true in the limit as u --+ 00 through the integers, since, for integral u, W (u - 1) may be expressed as the sum of u - l independent N(O, 1) variables (use the strong law). It remains to fill in the gaps. Let n be a positive integer, let x > 0, and write Mn = max { I W(u) - W(n ) 1 : n ::: u ::: n + I } . We have by the stationarity of the increments that 00
00
E(M1 ) < 00, L lP' (Mn � n x) = L lP'(Ml � n x) ::: 1 + -X n=O n=O implying by the Borel-Cantelli lemma that n - 1 Mn ::: x for all but finitely many values of n, a.s. Therefore n- 1 Mn --+ 0 a.s. as n --+ 00, implying that 1 __ I W(u) l ::: lim � { I W(n ) 1 + Mn } --+ 0 lim a.s. -+oo n-+oo n u + 1 u 5. In the notation of Exercise (13.6.4), we are asked to calculate the probability that W has no zeros in the time interval between s / (1 - s) and t /(1 - t). By Theorem (13.4.8), this equals s (1 - y) = ; - 1 �. 1 � cos - 1 t ( 1 - s) V� _
2
7r
414
COS
Stochastic calculus
Solutions
[13.7.1]-[13.7.5]
13.7 Solutions. Stochastic calculus
1. Let :Fs = a (Wu : 0 ::::: u ::::: s). Fix n � 1 and define Xn (k) = I Wkt /zn l for 0 ::::: k ::::: 2n . By Jensen ' s inequality, the sequence {Xn (k) : 0 ::::: k ::::: 2n } is a non-negative submartingale with respect to the filtration :Fkt /Zn , with finite variance. Hence, by Exercise (4.3.3) and equation (12 . 6.2), X� = max {Xn (k) : 0 ::::: k ::::: 2n } satisfies
E(X�z ) = 2 =
l)() xlP'(X� > x) dx ::::: 2 1000 E(W/I{X�:::x} ) dx 2E { W/ IoX� d } X
=
2E(W/ X�) ::::: 2 ..jE(whE(X'hZ )
by the Cauchy-Schwarz inequality.
Hence E(X� z ) ::::: 4E( Wh . Now X� z is monotone increasing in n, and W has continuous sample paths. By monotone convergence,
2. See the solution to Exercise (8.5.4). 3. (a) We have that h en)
=
{j=O
}
n- l n- l � L ( �?+1 - v/ ) - L ( VJ + l - VJ ) z . j =O
The first summation equals wl , by successive concellation, and the mean-square limit of the second summation is t, by Exercise (8.5.4). Hence limn-+oo h (n) = � wl - �t in mean square. Likewise, we obtain the mean-square limits: lim h en) = � wl + �t , n-+oo
lim h en) = n-+oo lim 14 (n) = � W? n-+oo
4. Clearly E(U(t)) = O. The process U is Gaussian with autocovariance function Thus U is a stationary Gaussian Markov process, namely the Ornstein-Uhlenbeck process. [See Example (9 . 6.10) . ] 5. Clearly E(Ut ) = O. For s < t,
(i
)
s r e-.8 (S-U ) e-.8 (t - v ) wv du dV Wu E( Us Us +t ) = E(Ws Wt ) + {P E u=O }v =O t s - E Wt ,8 e-.8 (s - u ) wu dU - E Ws e-.8 (t - v ) wv dV s r e.8 (u+v ) min{u, v} du dv = s + ,8 Z e -.8 (s + t ) u=o }v =o t s - ,8 e-.8 (s- u ) min{u, t} du - e-.8 (t - v ) min{s, v} dv e Z.8s - 1 e -.8 (s +t)
( lo
2,8
lo
) ( lo
i
lo
415
)
Diffusion processes after prolonged integration. By the linearity of the definition of it is a Gaussian process. From the calculation above, it has autocovariance function c(s , s +t) (e - fJ (t - s ) e- fJ (t +s » )/ 2{J) . From this we may calculate the instantaneous mean and variance, and thus we recognize an Omstein-Uhlenbeck [13.8.1]-[13.8.1]
Solutions
U,
.
=
(
-
process. See also Exercise ( 1 3.3 .4) and Problem ( 1 3 . 12 4).
13.8 Solutions. The Ito integral
1. (a) Fix t > 0 and let n � 1 and /) = tin. We write tj = jtln and correlation of Wiener increments, and the Cauchy-Schwarz inequality,
Vj
=
Wtj ' By the absence of
Therefore,
(b) As n
� 00 ,
n- l n- l = ( ) � L V/ Vj+ l Vj L U�/+l - V] - 3 Vj ( Vj+1 - Vj ) 2 - ( V)+ l - Vj ) 3 } j=o j=O n- l n- l = � wi L [Vj (t)+ l - tj ) Vj { ( Vj+ l - Vj ) 2 - (t)+ l - tj ) } ] - i L ( V)+1 - Vj ) 3 j� j� � i w? -
+ fot W(s )ds +O+O.
The fact that the last two terms tend to 0 in mean square may be verified in the usual way. For example,
lE
( [}�=o ( Vj+1 - Vj ) 3 [ 2) }�=o lE [( V)+ l - Vj ) 6] =
n-l
=
6 L Ct)+ l
j=O
U')
n- l 3 - tj ) 3 = 6 L n � 0 j=O
416
as n � 00 .
Ito's jormula
Solutions
(c) It was shown in Exercise (13.7.3a) that fci Ws dWs
[13.8.2]-[13.9.1]
� wl - � t . Hence,
=
and the result follows because lE(Wl) = t. 2. Fix t ° and n � 1 , and let 8 = tin. We set Vj = Wj t / n . It is the case that Xt = limn�oo I:j Vj (tj+1 - tj ). Each term in the sum is normally distributed, and all partial sums are multivariate normal for all 8 0, and hence also in the limit as 8 -+ 0. Obviously lE(X t ) = 0. For
>
>
s � t,
lE(Xs Xt ) = =
Hence var(X t )
=
lot los lE(Wu Wv ) du dv lot los v} du dv los � u2 du los u (t - u) du s2 G �) . min{u,
=
+
=
-
j- t 3 , and the autocovariance function is
3. By the Cauchy-Schwarz inequality, as n -+ 00,
4. We square the equation 11 1 (1/11 + 1/12 ) 11 2 = 11 1/11 + 1/12 11 and use the fact that II I ( 1/Ii ) 11 2 = 11 1/Ii II for i = 1 , 2, to deduce the result. S. The question permits us to use the integrating factor ef3 t to give, formally,
lo
lo
t t dW ef3 t Xt = o ef3s dss ds = ef3 t Wt - f3 0 ef3s Ws ds __
on integrating by parts. This is the required result, and substitution verifies that it satisfies the given equation.
6. Find a sequence '" = (¢ (n» ) of predictable step functions such that 11 ¢ (n) - 1/1 11 -+ ° as n -+ 00 . By the argument before equation (13.8.9), I (¢ (n » ) � 1 (1/1) as n -+ 00 . By Lemma (13.8.4), II I (¢ (n» ) 11 2 = 11 ¢ (n ) II , and the claim follows. 13.9 Solutions. Ito's formula
1. The process Z is continuous and adapted with Zo lE(Zt - Zs I :Fs ) = 0, and by Exercise (13.8.6) that
=
0. We have by Theorem (13.8. 1 1) that
The first claim follows by the Levy characterization of a Wiener process ( 12.7. 10). 417
[13.9.2]-[13.10.2] Zt
Diffusion processes
Solutions
We have in n dimensions that R 2 = XI + X� + . . . + X�, and the same argument yields that = L: i fci (X i / R) dX i is a Wiener process. By Example (13.9.7) and the above,
n
n
X . .. dX + n dt = 2R dW + n dt. d(R 2 ) = 2 L Xi dXi + n dt = 2R L ---.!. i i=l i=l R 2. Applying Ito ' s formula (13.9.4) to Yt = wt4 we obtain dYt = 4 W? dWt + 6W? dt. Hence,
3. Apply Ito ' s formula (13.9.4) to obtain dYt = Wt dt + t dWt . Cf. Exercise (13.8. 1). 4. Note that X l = cos W and X2 = sin W . By Ito ' s formula (13.9.4), dY = d(X I + i X2) = dX I + i dX2 = d(cos W) + i d(sin W) = - sin W dW - � cos W dt + i cos W dW 5. We apply Ito ' s formula to obtain: (a) (1 + t) dX = -X dt + dW, (b) dX = - � X dt + J l - X 2 dW, (c)
d
-�
sin W dt.
(�) -� (�) dt + ( b� -�b ) (�) dW. =
a
13.10 Solutions.
Option pricing
1. (a) We have that E ( (a e z
_
K) + )
=
=
['Xl (aeZ Jlog(K/a)
1
ct
00
_
K)
(aeY +t" Y - K)
�2 exp
v
2 1 e - Z Y2
7n
-dy ../iii
(
_
where y
=
00 e - � (Y- t" ) 2 1 2 = aeY + z t" dy - K 4> (-a) ../iii 1 2 = aeY + Zt" 4> (. - a) - K 4> (-a).
1
)
(z - ; ) 2 dz 2.
-Y, a -Z
•
=
10g(K/a)
-y
-'--=-'-----'•
ct
(b) We have that ST = ae z where a = St and, under the relevant conditional lQi-distribution, Z is normal with mean y = (r - �( 2 )(T - t) and variance . 2 = a 2 ( T - t) . The claim now follows by the result of part (a). 2. (a) Set � (t, S) = � (t , St ) and 1frCt, S) = 1fr(t, St ), in the natural notation. By Theorem (13. 1O. 15), we have 1frx = 1frt = 0, whence 1fr(t, x) = c for all t, x, and some constant c. (b) Recall that d S = fJ,S dt + as dW. Now,
d (� S + 1fre rt ) = d (S 2 + 1fre rt ) = (a S) 2 dt + 2S d S + e rt d1fr + 1frre rt dt, 418
Option pricing
Solutions
[13.10.3]-[13.10.5]
by Example (13.9.7). By equation (13. 10.4), the portfolio is self-financing if this equals 1/I r er t dt, and thus we arrive at the SDE e r t d1/l = -S dS - a 2 S 2 dt, whence
(c) Note first that Zt
sl dt, whence
=
fci Su du satisfies dZt
=
St dt. By Example (13.9.8), d(St Zt )
=
S dS +
Zt dSt +
d(� S + 1/Ie r t ) = Zt dSt + sf dt + e r t d1/l + r e r t dt. Using equation (13.10.4), the portfolio is self-financing if this equals Zt dSt + 1/I r e r t dt, and thus we require that e r t d1/l = -sl dt, which is to say that
3. We need to check equation (13. 10.4) remembering that dMt self-financing. (a) This case is obvious. (b) d@S + = d(2S 2 - S 2 - t) = 2S dS + dt - dt = � dS. (c) d(�S + = -S - t dS + S = � dS. (d) Recalling Example (13.9.8), we have that
=
O. Each of these portfolios is
1/1)1/1)
4. The time of exercise of an American call option must be a stopping time for the filtration (Ft ) .
The value of the option, if exercised at the stopping time ., i s V� = (S� K) + , and it follows by the usual argument that the value at time 0 of the option exercised at . is lElQ! (e - r � V�) . Thus the value at time 0 of the American option is sup � {lElQ! (e -r � V� )}, where the supremum is taken over all stopping times . satisfying ]P(. ::::: T) = 1 . Under the probability measure Q, the process e -r t Vt is a martingale, whence, by the optional stopping theorem, lElQ! (e-n V� ) = Vo for all stopping times • . The claim follows. 5. We rewrite the value at time 0 of the European call option, possibly with the aid of Exercise (13. 10. 1), as -
where N is an N(O, 1) random variable. It is immediate that this is increasing in So and r and is decreasing in K. To show monotonicity in T, we argue as follows. Let Tr < T2 and consider the European option with exercise date T2 . In the corresponding American option we are allowed to exercise the option at the earlier time Tr . By Exercise (13.10.4), it is never better to stop earlier than T2 , and the claim follows. Monotonicity in a may be shown by differentiation.
419
[13.11.1]-[13.12.2]
Diffusion processes
Solutions
13. 1 1 Solutions.
Passage probabilities and potentials
1. Let H be a closed sphere with radius R (> I w i ), and define P R (r) Then P R satisfies Laplace ' s equation in Rd , and hence !!.-
(
d - 1 dP R dr r dr
)
=
=
lP' ( G before H I I W (0) I
_
r) .
0
since P R is spherically symmetric. Solve subject to the boundary equations P R (E) to obtain r 2-d - R 2-d P R (r) = 2-d 2-d � (E/r) d -2 as R � 00.
E
=
=
1, P R ( R )
=
0,
R
2. The electrical resistance Rn between 0 and the set t:.n is no smaller than the resistance obtained by, for every i = 1 , 2, . . . , 'shorting out ' all vertices in the set t:. i . This new network amounts to a linear chain of resistances in series, points labelled t:. j and t:. i + 1 being joined by a resistance if size Ni- 1 . It follows that 00 1 R(G) = n-+oo lim Rn � L - . i=O Ni By Theorem (13 . 1 1 . 18), the walk is persistent if Lj Nj- 1 = 00. 3. Thinking of G as an electrical network, one may obtain the network H by replacing the resistance of every edge e lying in G but not in H by 00. Let 0 be a vertex of H . By a well known fact in the theory of electrical networks, R(H) � R ( G) , and the result follows by Theorem (13. 1 1 . 19). 13.12 Solutions to problems
1. (a) T(t) = aW(t /( 2 ) has continuous sample paths with stationary independent increments, since
W has these properties. Also T (t)/a is N(O, t/( 2 ) , whence T(t) is N(O, t). (b) As for part (a). (c) Certainly V has continuous sample paths on (0, (0) . For continuity at 0 it suffices to prove that tW(t - l ) � 0 a.s. as t .J, 0; this was done in the solution to Exercise (13.6.4). If (u, v), (s, t) are disjoint time-intervals, then so are (v - I , u - 1 ), (t - l , s - l ); since W has independent increments, so has V. Finally,
is N(O, fJ) if s, t > 0, where fJ
=
(�
2 s� +t +s s
1
_
s +_t _
)
=
t.
2. Certainly W is Gaussian with continuous sample paths and zero means, and it is therefore sufficient to prove that cov(W(s) , Wet)) = min{s, t}. Now, if s :::: t,
as required. 420
Solutions
Problems
[13.12.3]-[13.12.4]
If u(S) = s, v(t) = 1 - t, then r (t ) = t/(l - t), and r - 1 (w) = w/(l + w) for 0 � this case X(t) = (1 - t) W(t/(1 - t)). 3. Certainly U is Gaussian with zero means, and U(O) = O. Now, with St = e2fJ t - 1 ,
w < 00. In
lE{ U(t + h) I U(t) = u } = e - fJ (t +h ) lE{ W(St +h ) I W(St ) = uefJ t } +h ) efJ t = u - f3u h + o(h) , = ue - fJ (t whence the instantaneous mean of U is a(t, u) therefore
=
- f3u . Secondly, St +h
=
St + 2f3e2fJ t h + o(h), and
lE{ U(t + h) 2 1 U(t) = u } = e -2fJ (t +h ) lE{ W(St +h ) 2 1 W(St ) = uefJ t } 2 +h ) ( u 2 e2fJ t + 2f3e2fJ t h + o(h) ) = e - fJ (t 2 2 = u - 2f3h (u - 1) + o(h). It follows that
lE { I U(t + h) - U (t ) 1 2 1 U(t) = u } = u 2 - 2f3 h (u 2 - 1) - 2u (u - f3u h) + u 2 + o(h) = 2f3h + o(h), and the instantaneous variance is b(t, u) = 2f3. 4. Bartlett ' s equation (see Exercise (13.3.4)) for M(t, e)
aM at
- =
with boundary condition M(e, 0)
=
-{3e
=
lE(e O V (t » ) is
aM + za 1 2e2 M ae
-
iJu . Solve this equation (as in the exercise given) to obtain
the moment generating function of the given normal distribution. Now M(t, e) --+ exp{ ie 2 a 2 /(2f3)} as t --+ 00, whence by the continuity theorem V(t) converges in distribution to the N(O, ia 2 /f3) distribution. If V (0) has this limit distribution, then so does V(t) for all t. Therefore the sequence (V(t 1 ), . . . , V(tn )) has the same joint distribution as (V(t 1 + h), . . . , V(tn + h)) for all h , t1 ( . . . , tn , whenever V (0) has this normal distribution. In the stationary case, lE ( V ( t )) = 0 and, for S � t, cov ( V (S ),
V(t))
=
lE{ V(s)lE (V(t) I V (s ) ) } = lE { V(s) 2 e - fJ (t-s) } = c(O)e - fJ l t-s l
where c(O) = var(V (s)); we have used the first part here. This is the autocovariance function of a stationary Gaussian Markov process (see Example (9.6. 10)). Since all such processes have autocovariance functions of this form (i.e. , for some choice of f3), all such processes are stationary Ornstein-Uhlenbeck processes. The autocorrelation function is p(s) = e - fJ1 s 1, which is the characteristic function of the Cauchy density function
1 f(x) = f3 ;rr { 1 + (x/f3) 2 } ' 42 1
xER
[13.12.5]-[13.12.7]
Diffusion processes
Solutions
5. Bartlett ' s equation (see Exercise ( 13.3.2)) for M is aM = a ° aM + 1 fJO 2 aM at ai l. ai ' = iJd . The characteristics satisfy 2 dO dM d t
subject to M(O, 0 )
°
The solution is M = g(Oea t /(a + The solution follows as given. By elementary calculations,
whence var(D(t))
1
ifJO)) where g is a function satisfying g(O/(a + ifJO)) = iJd .
= (fJd/a)ea t (ea t - 1). Finally lP'(D(t)
e-+-oo
= 0) = lim M(t, 0) = exp
{
2adea t fJ (1 ea t ) -
}
which converges to e - 2a d/ fJ as t 6. The equilibrium density function g(y) satisfies the (reduced) forward equation
--+ 00.
d (ag) + 1 d2 --2 (bg) = ° -2 dy dy where a(y) are
= -fJy and b(y) = a 2 are the instantaneous mean and variance. The boundary conditions y=
Integrate (*) from
-c
-
c,
d.
to y, using the boundary conditions, to obtain
dg = 0, fJyg + l.1 a 2 dy -
-c ::;
y ::; d.
2 2
Integrate again to obtain g(y) = Ae - fJ y /(1 . The constant A is given by the fact that J�c g(y) dy = 1 . 7. First we show that the series converges uniformly (along a subsequence), implying that the limit exists and is a continuous function of t. Set
n - 1 sm. (kt) Xk Zmn (t) = � L..J k=m k --
We have that
2 Mmn
Mmn = sup { IZmn (t) I ; ° ::; t ::; rr } .
J J +l 1
n - 1 e i kt 2 n - 1 X 2 n - m - 1 n - I - 1 X · X · � Xk < � L..J L..J --.Ii + 2 � L..J � L..J . 0 9 �1r k=m k k=m k2 1=1 j=m J (J + I) .
< suP
-
1
o
I
-
-
422
I
"
Solutions
Problems
[13.12.8]-[13.12.8]
The mean value of the final term is, by the Cauchy-Schwarz inequality, no larger than 2
(I � �j �i+1 12)
n -m-1 n 1 E lE 1) =1 j =m J { j +
1
=2
n
f1
n -I-1 '" L
j =m
1=1
I < 2(n p (j + 1 )2 -
Combine this with (*) to obtain
-m m) � m4 • -
-
, ",m 00
3 lE(Mm , 2m ) 2 � lE(Mm2 2m ) � r.:;; '
It follows that
00 ( lE M
)
6 E 2n -l , 2n � E 2n /2 < 00, n=1 n=1 < 00 a.s. Therefore the series which defines W converges uniformly
implying that 2:� 1 M2n -l 2n with probability 1 (along a subsequence), and hence W has (a.s.) continuous sample paths. Certainly W is a Gaussian process since Wet) is the sum of normal variables (see Problem (7. 1 1 . 19» . Furthermore lE(W(t» = 0, and
f
;
sin(ks) in(kt) cov ( W(s), Wet») = � + � rr rr k=1 k since the Xi are independent with zero means and unit variances. It is an exercise in Fourier analysis to deduce that cov(W(s), Wet»� = min{s , t}. 8. We wish to find a solution get, y) to the equation Iy l
< b,
satisfying the boundary conditions g(O, y) = 8 yo if Iy l � b, Let get, y I 'source ' d. Let
00
g(t, y) = o if l y l = b.
d) be the N(d, t) density function, and note that g( . " I d) satisfies g(t, y) =
E
(- I ) k g(t, y I 2kb) ,
(*)
for any
k= - oo a series which converges absolutely and is differentiable term by term. Since each summand satisfies (*), so does the sum. Now g(O, y) is a combination of Dirac delta functions, one at each multiple of 2b. Only one such multiple lies in [-b, b], and hence g(y, 0) = 8dO . Also, setting y = b, the contributions from the sources at -2(k - l)b and 2kb cancel, so that get, b) = 0. Similarly get, -b) = 0, and therefore g is the required solution. Here is an alternative method. Look for the solution to (*) of the form e -Ant sin{ � nrr(y + b)/b}; such a sine function vanishes when Iy l = b. Substitute into (*) to obtain An = n 2 rr 2 /(8b2 ). A linear combination of such functions has the form get, y) =
00
E an e -Ant sm. n=1 423
( nrr(y + b) 2b
).
[13.12.9]-[13.12.11]
Diffusion processes
Solutions
We choose the constants an such that g(O, y) = 8yO for I y l < b. With the aid of a little Fourier analysis, one finds that an = b - 1 sin( � n:7r). Finally, the required probability equals the probability that Wa has been absorbed by time t, a probability expressible as 1 - J�b r (t , y) dy. Using the second expression for r, this yields
9. Recall that U (t) = e -2mD (t) is a martingale. Let T be the time of absorption, and assume that the conditions of the optional stopping theorem are satisfied. Then lE(U (0)) = lE(U (T)), which is to say that 1 = e 2ma Pa + e - 2m b (1 - Pa). 10. (a) We may assume that a, b > O. With pt (b) = IP'(W(t) > b, F(O, t) / W(O) = a ) , we have by the reflection principle that
pt (b) = IP'(W(t) > b / W(O) = a) - 1P'(W(t) < -b / W(O) = a) = 1P'(b - a < W(t) < b + a / W(O) = 0) , giving that
8pt (b ) � = I(t, b + a) - I(t, b - a)
where I(t, x) is the N(O, t) density function. Now, using conditional probabilities,
1 8pt (b ) = 1 _ e -2a b / t . IP'(F(O, t) / W(O) = a, W(t) = b) = I(t, b - a) 8b (b) We know that
2 -1 = -2 sin - 1 { VS7t} IP'(F(s, t)) = 1 - 7r cos { VS7t} 7r
if 0 < s < t. The claim follows since F(to, t2 ) S; F(t l , t2 ) . (c) Remember that sin x = x + o(x) as x .J, o. Take the limit in part (b) as to .J, 0 to obtain ..filTi2. 11. Let M(t) = sup{W(s) : 0 ::: s ::: t} and recall that M(t) has the same distribution as I W(t) I . By symmetry, IP' ( sup I W(s)l ::: ::: 21P'(M(t) ::: ) = 21P'(I W(t) 1 ::: )
09 ::9
w)
w
w.
By Chebyshov ' s inequality,
Fix E
> 0, and let An (E) = { I W(s) l/s > E for some s satisfying 2n - 1 < s ::: 2n } .
Note that
424
Solutions
Problems
for all large n , and also
(
00
[13.12.12]-[13.12.13]
) 00
2n+l L IP' sup I W(s) I � 22n / 3 � L 24n /3 < 00. n=1 n=1 O:;:: s :;:: 2n Therefore l:n IP'(A n (E)) < 00, implying by the Borel-Cantelli lemma that (a.s.) only finitely many of the An (E) occur. Therefore t - 1 W ( t ) � 0 a.s. as t � 00. Compare with the solution to the relevant
part of Exercise (13.6.4). 12. We require the solution to Laplace ' s equation \7 2 p
p(w) =
{O
=
0, subject to the boundary condition
if w E H, 1 if w E G.
Look for a solution in polar coordinates of the form
00
p(r, e) = L r n {an sin(ne) + bn cos(ne) } . n=O Certainly combinations having this form satisfy Laplace ' s equation, and the boundary condition gives that H(e)
=
00
bo + L {an sin(ne) + bn cos(ne) } , n=1
where
le i < n,
{0
if - n < e < 0, 1 if O < e < n. The collection {sin(me), cos(me) : m � O} are orthogonal over ( n n). Multiply through (*) by sin(me) and integrate over (-n, n) to obtain n am = { I - cos(nm)}lm , and similarly bo = � and bm = O for m � l . 13. The joint density function of two independent N(O, t) random variables is (2n t ) - 1 exp{ _ (x2 + y 2 )/(2t )}. Since this function is unchanged by rotations of the plane, it follows that the two coordi nates of the particle ' s position are independent Wiener processes, regardless of the orientation of the coordinate system. We may thus assume that I is the line x = d for some fixed positive d. The particle is bound to visit the line I sooner or later, since IP' ( WI (t) < d for all t) = O. The first-passage time T has density function H(e)
=
-
t }, D = W2 (T) is N(O, t). Therefore the density function of D is d e -(U +d )/(2t ) dt = d t) , U E R, o fD I T (U I fT (t) dt = 0 2nt2 n (u 2 + d2 )
Conditional on {T
fD (U) =
1000
=
,
1000
2 2
__
giving that Did has the Cauchy distribution. The angle E> = :P6R satisfies e = tan - 1 (D Id), whence IP'(E> � e)
=
IP' (D � d tan e)
=
425
1 2"
+ ne '
le i < � n.
[13.12.14]-[13.12.18] Solutions
Diffusion processes
14. By an extension of Ito ' s formula to functions of two Wiener processes, U V = V(WI , W2 ) satisfy
=
U(WI , W2 ) and
d U = Ux dWI + Uy dW2 + � (uxx + Uyy) dt, dV = Vx d WI + Vy d W2 + � (vxx + Vyy) dt, where ux , Vyy , etc, denote partial derivatives of U and v. Since ¢ is analytic, U and v satisfy the Cauchy-Riemann equations Ux = Vy , Uy = -Vx , whence u and v are harmonic in that Uxx + Uyy = Vxx + Vyy O. Therefore,
= (
dU
)
= Ux dWI + uy d W2 ,
dV = -uy d WI + Ux dW2 .
The matrix ux uy is an orthogonal rotation ofR2 when u; +u � = 1 . Since thejoint distribution -Uy Ux of the pair (WI , W2 ) is invariant under such rotations, the claim follows. 15. One method of solution uses the fact that the reversed Wiener process {W(t -s) - W(t) : 0 :::: s :::: t} has the same distribution as {W(s) : 0 :::: s :::: t}. Thus M(t) - W(t) = maxo < s < d W (s ) - W(t)} has the same distribution as maxo::; u ::;d W(u) - W (O) } = M(t). Alternatively, by fuereflection principle,
lP' ( M(t) � x , W(t) :::: y ) = IP'(W(t) � 2x - y) for x � max{O, y}. By differentiation, the pair M(t), W(t) has joint density function -2¢'(2x - y) for y :::: x, x � 0, where ¢ is the density function of the N(O, t) distribution. Hence M(t) and M(t) - W(t) have the joint density function - 2¢' (x + y). Since this function is symmetric in its arguments, M (t) and M(t) - W(t) have the same marginal distribution. 16. The Lebesgue measure A (Z) is given by A(Z)
=
1000 I{W (t)=u } du ,
whence by Fubini ' s theorem (cf. equation (5.6. 13)), lE(A (Z))
=
1000 IP' ( W(t)
=
u) dt
=
O.
17. Let 0 < a < b < c < d, and let M(x , y) = maxx:::s ::; y W(s). Then M(c, d) - M(a, b) = c::;s::;d max { W(s) - W(c) } + { W(c) - W(b) } - maxb { W(s) - W(b) } . a ::;s::; Since the three terms on the right are independent and continuous random variables, it follows that lP'( (M(c, d) = M(a, b)) = O. Since there are only countably many rationals, we deduce that lP'( M (c , d) = M(a, b) for all rationals a < b < c < d) = 1 , and the result follows. 18. The result is easily seen by exhaustion to be true when n = 1. Suppose it is true for all m :::: n - 1 where n 2: 2. (i) If S n :::: 0, then (whatever the final term of the permutation) the number of positive partial sums and the position of the first maximum depend only on the remaining n - 1 terms. Equality follows by the induction hypothesis. (ii) If Sn 0, then
>
n
Ar = L Ar - I (k), k=1 426
Solutions
Problems
[13.12.19]-[13.12.20]
where Ar- l (k) is the number of permutations with Xk in the final place, for which exactly r - 1 of the first n - 1 terms are strictly positive. Consider a permutation n = (Xi I ' Xi2 ' . . . , Xin _ 1 ' Xk) with Xk in the final place, and move the position ofxk to obtain the new permutation n' = (Xb Xi I ' Xi2 ' . . . , Xin _ I ) . The first appearance of the maximum in n' is at its rth place if and only if the first maximum of the reduced permutation (Xi I ' Xi2 ' . . . , Xin _ l ) is at its (r - l)th place. [Note that r = 0 is impossible since Sn > 0.] It follows that
n Br = L Br - l (k), k= l
where Br - l (k) is the number of permutations with Xk in the final place, for which the first appearance of the maximum is at the (r - l)th place. By the induction hypothesis, A r - l (k) = Br - l (k) , since these quantities depend on the n - 1 terms excluding Xk . The result follows. 19. Suppose that Sm = L:j= l Xj , 0 ::::: m ::::: n, are the partial sums of n independent identically distributed random variables Xj . Let An be the number of strictly positive partial sums, and Rn the index of the first appearance of the value of the maximal partial sum. Each of the n ! permutations of (X l , X2 , . . . , Xn) has the same joint distribution. Consider the kth permutation, and let h be the indicator function of the event that exactly r partial sums are positive, and let h be the indicator function that the first appearance of the maximum is at the rth place. Then, using Problem ( 1 3 . 1 2. 1 8), 1 n! 1 nl IP'(An = r) = ,. lE(h) = ,. L lE ( h ) = IP' ( Rn = r). L n . k= l n . k= l We apply this with Xj = W(jt/n) - W«j - 1)t/n), so that Sm = W(mt/n). Thus A n L:j I{W (jt / n » O } has the same distribution as Rn = min { k 2': 0 : W(kt/n) = O::s�xn W(jt/n) } . } ::s By Problem (13. 12. 17), Rn � R as n --+ 00 . By Problem (13. 12. 16), the time spent by W at zero is a.s. a null set, whence An � A. Hence A and R have the same distribution. We argue as follows to obtain that that L and R have the same distribution. Making repeated use of Theorem (13.4.6) and the symmetry of W, inf W(s) > 0) IP'(L < x) = IP' ( sup W(s) < 0) + IP' (x::ss::st x::ss::st = 21P' ( sup {W(s) - W(x)} < - W(x) ) = 21P'(IW(t) - W(x) 1 < W(x)) x::ss::st = 1P'(I W(t) - W(x) 1 < I W(x) l ) = IP' ( sup {W(s) - W(x)} < sup {W(s) - W (X )} ) = IP'(R ::::: x). x::ss::st O::ss::sx Finally, by Problem (13.12. 15) and the circular symmetry of the joint density distribution of two independent N(O, 1) variables U, V,
(
V2 1P'(IW(t) - W(x) 1 < I W(x) l) = 1P'(t - x) V 2 -< x U 2 ) = IP' U 2 + V 2 ::::: ::t
20. Let
{
{t ::::: 1 : W (t) = x } if this set i s non-empty, Tx = inf 1 otherwise,
427
)
=
� sin - l n
�. Vt
[13.12.21]-[13.12.24] Solutions
Diffusion processes
and similarly = sup{t .:::: 1 : = } , with for all E = 1 if have an arc sine distribution as in Problem (13. 12. 19). On the event { and write (using the re-scaling property of
Uo
Vo
Vx
x
Wet)
W(t) =f:. x
Vx
t [0, 1]. Recall that U < I }, we may
W)
X
=
Ux = Tx + (1 - Tx ) Uo, Vx Tx + (1 - Tx ) Yo, where Uo and Yo are independent of Ux and Vx , and have the above arc sine distribution. Hence Ux and Vx have the same distribution. Now Tx has the first passage distribution of Theorem (13.4.5),
whence
Therefore,
and
fux (u) =
iou irx , ux (t, u) du
=
7r
�
.JU( _ u)
exp
(-�:) ,
0
< x < l.
t and let 1/Is = sign(Ws ), 0 .:::: s ::'S t. lE(Vh = II I (1/I) II� = t. By a similar calculation, lE(V? 1 :Fs ) = V} + t - s for 0 ::'S s .:::: t. That is to say, V? - t defines a martingale, and
21.
V
Note that is a martingale, by Theorem (13.8. 1 1). Fix We have that 11 1/1 11 = ..,fi, implying by Exercise (13.8.6) that
the result follows by the Levy characterization theorem of Example
22.
(12.7. 10).
The mean cost per unit time is
Differentiate to obtain that /-L
R = 2C
'
(T) = 0 if
{ loT (a/.Ji) dt - T (a/.JT) } a loT t- 1 f/> (a/../t) dt, =
C
where we have integrated by parts .
23. Consider the portfolio with �(t, St ) units of stock and 1/I(t, Sr) units of bond, having total value w(t, St ) = x�(t, x) + e r t 1/l(t, St ). By assumption, (*) (1 - y)xW, x) = ye r t 1/I(t, x). Differentiate this equation with respect to x and substitute from equation (13. 10. 16) to obtain the differential equation ( 1 - y)� + x �x = 0, with solution �(t, x) = h(t)xy - 1 , for some function h(t).
We substitute this, together with (*), into equation
(13. 10. 17) to obtain that
h ' - h(1 - y)(iya 2 + r) = O. It follows that h(t) = A exp { ( 1 - y)(! ya 2 + r ) t } , where A is an absolute constant to be determined according to the size of the initial investment. Finally, wet, x) = y - l x�(t, x) = y - l h ( t ) x Y . 24. Using Ito's formula (13.9.4), the drift term in the SDE for Ut is (-U l (T - t, W) + !U 22 (T - t, W») dt, where U l and U 22 denote partial derivatives of u. The drift function is identically zero if and only if U l = iU 22 ' 428
Bibliography
A man will tum over half a library to make one book.
Samuel Johnson
Abramowitz, M. and Stegun, I. A. ( 1 965). Handbook of mathematicalfunctions with formulas, graphs and mathematical tables. Dover, New York. Billingsley, P. ( 1 995). Probability and measure (3rd edn). Wiley, New York. Breiman, L. ( 1 968). Probability. Addison-Wesley, Reading, MA, reprinted by SIAM, 1 992. Chung, K. L. ( 1 974). A course in probability theory (2nd edn) . Academic Press, New York. Cox, D. R. and Miller, H. D. ( 1 965). The theory of stochastic processes. Chapman and Hall, London. Doob, J. L. ( 1 953). Stochastic processes. Wiley, New York. Feller, W. ( 1 968). An introduction to probability theory and its applications, Vol. 1 (3rd edn). Wiley, New York. Feller, W. ( 1 97 1 ) . An introduction to probability theory and its applications, Vol. 2 (2nd edn). Wiley, New York. Grimmett, G. R. and Stirzaker, D. R. (200 1 ) . Probability and random processes, (3rd edn). Oxford University Press, Oxford. Grimmett, G. R. and Welsh, D. J. A. ( 1 986). Probability, an introduction. Clarendon Press, Oxford. Hall, M. ( 1 983). Combinatorial theory (2nd edn) . Wiley, New York. Harris, T. E. ( 1 963). The theory of branching processes. Springer, Berlin. Karlin, S. and Taylor, H. M. ( 1 975). Afirst course in stochastic processes (2nd edn). Academic Press, New York. Karlin, S . and Taylor, H. M. ( 1 9 8 1 ) . A second course in stochastic processes. Academic Press, New York. Laha, R. G. and Rohatgi, V. K. ( 1 979) . Probability theory. Wiley, New York. Loeve, M. ( 1 977) . Probability theory, Vol. 1 (4th edn) . Springer, Berlin. Loeve, M. ( 1 978). Probability theory, Vol. 2 (4th edn). Springer, Berlin. Moran, P. A. P. ( 1 968). An introduction to probability theo ry. Clarendon Press, Oxford. Stirzaker, D. R. ( 1 994). Elementary probability. Cambridge University Press, Cambridge. Stirzaker, D. R. ( 1 999). Probability and random variables. Cambridge University Press, Cambridge. Williams, D. ( 1 99 1 ) . Probability with martingales. Cambridge University Press, Cambridge.
429
Index
Abbreviations used in this index: c.f. characteristic function; distn distribution; eqn equation; fn function; m.g.f. moment generating function; p.g.f. probability generating function; pro process; r.v. random variable; r.w. random walk; s.r.w. simple random walk; thm theorem.
A
absolute value of s.r. W. 6. 1.3
absorbing barriers: s.r.w. 1.7.3, 3.9. 1, 5, 3. 1 1.23, 25-26, 12.5 .4-5, 7; Wiener pro 12.9.22-3, 13. 12.8-9 absorbing state 6 .2 . 1 adapted process 13.8.6 affine transformation 4. 13. 1 1 ; 4. 14.60 age-dependent branching pro 5 .5 . 1-2; conditional 5 . 1.2; honest martingale 12.9.2; mean 10.6. 13 age, see current life airlines 1.8.39, 2.7.7
arc sine laws for Wiener pro 13.4.3, 13. 12. 10, 13. 12.19 Archimedes's theorem 4. 1 1.32 arithmetic r.v. 5.9.4 attraction 1.8.29 autocorrelation function 9.3.3, 9.7.5, 8 autocovariance function 9.3.3, 9.5 .2, 9.7.6, 19-20, 22 autoregressive sequence 8.7.2, 9 . 1 . 1, 9.2. 1, 9.7.3 average, see moving average
babies 5 . 10.2
B
alarm clock 6.15.21
backward martingale 12.7.3
algorithm 3 . 1 1 .33, 4. 14.63, 6 . 14.2
bagged balls 7. 1 1.27, 12.9. 13-14
aliasing method 4. 1 1.6
balance equations 1 1 .7. 13
alternating renewal pro 10.5.2, 10.6 . 14
B andrika 1.8.35-36, 4.2.3
American call option 13. 10.4
B arker's algorithm 6. 14.2
analytic fn 13. 12. 14
barriers: absorbing/retaining in r.w. 1.7.3, 3.9. 1-2, 3. 1 1.23, 25-26; hitting by Wiener pro 13.4.2
ancestors in common 5 .4.2 anomalous numbers 3.6.7 Anscombe's theorem 7. 1 1 .28 antithetic variable 4. 1 1. 1 1 ants 6. 15.41 arbitrage 3.3.7, 6.6.3
Arbuthnot, I. 3. 1 1.22
arc sine distn 4. 1 1 . 13; sample from 4. 1 1 . 13
bankruptcy, see gambler's ruin
Bartlett: equation 13.3.2-4; theorem 8.7.6, 1 1.7. 1 batch service 1 1 .8.4 baulking 8.4.4, 1 1 .8.2, 19 Bayes 's formula 1.8. 14, 1.8.36 bears 6 . 13. 1, 10.6. 19
arc sine law density 4. 1. 1
Benford's distn 3.6.7
arc sine laws for r.w. : maxima 3. 1 1 .28; sojourns 5 .3.5 ; visits 3. 10.3
Berkson's fallacy 3 . 1 1.37 Bernoulli: Daniel 3.3.4, 3.4.3-4; Nicholas 3.3.4
430
Bernoulli: mode1 6. 15 .36; renewal 8.7.3; shift 9. 17. 14; sum of r.v.s 3. 1 1 . 14, 35 Bertrand's paradox 4. 14.8 Bessel: function 5 .8.5, 1 1 .8.5, 1 1.8. 16; B. pro 12.9.23, 13.3.5-6, 13.9. 1 best predictor 7.9 . 1 ; linear 7.9.3, 9.2. 1-2, 9.7. 1, 3 beta fn 4.4.2, 4. 10.6 beta distn: b.-binomial 4.6.5; sample from 4. 11.4-5 betting scheme 6.6.3 binary: expansion 9. 1.2; fission 5 .5 . 1 binary tree 5 . 12.38; r.w. on 6.4.7 binomial r.v. 2 . 1.3; sum of 3 . 1 1 .8, 11 birth process 6.8.6; dishonest 6.8.7; forward eqns 6.8.4; divergent 6.8.7; with immigration 6.8.5; non-homogeneous 6 . 15.24; see also simple birth birth-death process: coupled 6. 15 .46; extinction 6 . 1 1 .3, 5, 6 . 15 .25, 12.9. 10; honest 6. 15.26; immigration-death 6 . 1 1 .3, 6. 13. 18, 28; jump chain 6 . 1 1 . 1 ; martingale 12.9. 10; queue 8.7.4; reversible 6.5 . 1, 6 . 15 . 16; symmetric 6 . 15 .27; see also simple birth-death birthdays 1.8.30 bivariate: Markov chain 6. 15.4; negative binomial distn 5. 12. 16; p.g.f. 5 . 1.3 bivariate normal distn 4.7.5-6, 12, 4.9.4-5, 4. 14. 13, 16, 7.9.2,
Index 7. 1 1 . 19 ; c.f. 5 .8. 1 1 ; positive part 4.7.5, 4.8.8, 5.9.8 Black-Scholes: model 13. 12.23; value 13. 10.5 Bonferroni's inequality 1.8.37 books 2.7. 15 Boo1e's inequalities 1.8. 1 1 Borel: normal number theorem 9.7. 14; paradox 4.6 . 1 Borel-Cantelli lemmas 7.6. 1, 13. 12. 1 1 bounded convergence 12.1.5 bow tie 6.4. 1 1 Box-Muller normals 4. 1 1.7 branching process: age-dependent 5.5. 1-2, 10.6. 13; ancestors 5.4.2; conditioned 5 . 12.21, 6.7. 1-4; convergence, 12.9.8; correlation 5 .4. 1; critical 7. 10. 1; extinction 5 .4.3; geometric 5.4.3, 5 .4.6; imbedded in queue 1 1 .3.2, 11.7.5, 11; with immigration 5 .4.5, 7.7.2; inequality 5 . 12. 12; martingale 12. 1 .3, 9, 12.9. 1-2, 8; maximum of 12.9.20; moments 5 .4. 1; p.g.f. 5 .4.4; supercritical 6.7.2; total population 5 . 12 . 1 1 ; variance 5 . 12.9; visits 5 .4.6 bridge 1.8.32 Brownian bridge 9.7.22, 13.6.2-5 ; autocovariance 9.7.22, 13.6.2; zeros of 13.6.5 Brownian motion; geometric 13.3.9; tied-down, see Brownian bridge Buffon: cross 4.5.3; needle 4.5 .2, 4. 14.31-32; noodle 4. 14.3 1 busy period 6. 12. 1; in GIGII 1 1.5 . 1 ; in MlGll 1 1 .3.3; in MIMIl 1 1 .3.2, 1 1 .8.5; in MlMloo 1 1 .8.9
C
cake, hot 3.1 1.32 call option: American 13. 10.4; European 13. 10.4-5 Campbell-Hardy theorem 6 . 13.2 capture-recapture 3.5.4 car, parking 4. 14.30 cards 1.7.2, 5, 1.8.33 Carroll, Lewis 1.4.4 casino 3.9.6, 7.7.4, 12.9. 16 Cauchy convergence 7.3. 1; in m.s. 7. 1 1 . 1 1 Cauchy distn 4.4.4; maximum 7. 1 1 . 14; moments 4.4.4;
sample from 4. 1 1.9; sum 4.8.2, 5 . 1 1.4, 5 . 12.24-25 Cauchy-Schwarz inequality 4. 14.27 central limit theorem 5 . 10. 1, 3, 9, 5 . 12.33, 40, 7. 1 1 .26, 10.6.3 characteristic function 5 . 12.26-3 1 ; bivariate normal 5 .8. 1 1 ; continuity theorem 5 . 12.39; exponential distn 5 .8.8; extreme-value distn 5 . 12.27; first passage distn 5 . 10.7-8; joint 5 . 12.30; law of large numbers 7. 1 1 . 15 ; multinormal distn 5 .8.6; tails 5 .7.6; weak law 7. 1 1. 15 Chebyshov's inequality, one-sided 7. 1 1.9 cherries 1.8.22 chess 6.6.6-7 chimeras 3. 1 1.36 chi-squared distn: non-central 5 .7.7; sum 4. 10. 1, 4. 14.12 Cholesky decomposition 4. 14.62 chromatic number 12.2.2 coins: double 1.4.3; fair 1.3.2; first head 1.3.2, 1 .8.2, 2 .7. 1; patterns 1.3.2, 5 .2.6, 5 . 12.2, 10.6. 17, 12.9 . 16; transitive 2.7. 16; see Poisson flips colouring: graph 12.2.2; sphere 1.8.28; theorem 6.15.39 competition lemma 6. 13.8 complete convergence 7.3.7 complex-valued process 9.7.8 compound: Poisson pro 6. 15 .21; Poisson distn 3.8.6, 5 . 12 . 13 compounding 5 .2.3, 5.2.8 computer queue 6.9.3
continuous r.v.: independence 4.5.5, 4. 14.6; limits of discrete r.v.s 2.3. 1 convergence: bounded 12. 1.5; Cauchy 7.3. 1, 7. 1 1 . 1 1 ; complete 7.3.7; conditional 13.8.3; in distn 7.2.4, 7. 1 1.8, 16, 24; dominated 5.6.3, 7.2.2; event of 7.2.6; martingale 7.8.3, 12. 1.5, 12.9.6; Poisson pro 7. 1 1 . 5 ; in probability 7.2.8, 7. 1 1 . 15 ; subsequence 7. 1 1.25; in total variation 7.2.9 convex: fn 5 . 6 . 1, 12. 1.6-7; rock 4. 14.47; shape 4. 13.2-3, 4. 14. 6 1 corkscrew 8.4.5 Com Flakes 1.3.4, 1.8. 13 countable additivity 1.8. 18 counters 10.6.6-8, 15 coupling: birth-death pro 6.15.46; maximal 4. 12.4-6, 7. 1 1 . 16 coupons 3.3.2, 5 .2.9, 5 . 12.34 covariance: matrix 3. 1 1 . 15, 7.9.3; of Poisson pro 7. 1 1.5 Cox process 6. 15.22 Cp inequality Cramer-Wold device 7. 1 1 . 19, 5 .8. 1 1 criterion: irreducibility 6 . 1 5 . 1 5 ; Kolmogorov's 6 . 5 . 2 ; for persistence 6.4. 10 Crofton's method 4. 13.9 crudely stationary 8.2.3 cube: point in 7. 1 1 .22; r.w. on 6.3.4 cumulants 5 .7.3-4 cups and saucers 1.3.3 current life 10.5 .4; and excess 10.6.9; limit 10.6.4; Markov 10.3.2; Poisson 10.5 .4
concave fn 6 . 15 .37 conditional: birth-death pro 6. 1 1.4-5; branching pro 5 . 12.21, 6.7. 1-4; convergence 13.8.3; correlation 9.7.21; entropy 6.15.45 ; expectation 3.7.2-3, 4.6.2, 4. 14. 13, 7.9.4; independence 1.5.5; probability 1 .8.9 ; s.r.w. 3.9.2-3; variance 3.7.4, 4.6.7; Wiener pro 8.5.2, 9.7. 2 1, 13.6.1; see also regression continuity of: distn fns 2.7. 10; marginals 4.5 . 1; probability measures 1.8. 16, 1.8. 18; transition probabilities 6 . 15 . 14 continuity theorem 5 . 12.35, 39
43 1
D
dam 6.4.3 dead period 10.6.6-7 death-immigration pro 6 . 1 1.3 decay 5 . 12.48, 6.4.8 decimal expansion 3.1.4, 7. 1 1.4 decomposition: Cholesky 4. 14.62; Krickeberg 12.9 . 1 1 degrees o f freedom 5 .7.7-8 delayed renewal pro 10.6. 12 de Moivre: martingale 12. 1.4, 12.4.6; trial 3.5 . 1 D e Morgan laws 1.2 . 1 density: arc sine 4. 1 1 . 13; arc sine law 4. 1 . 1 ; betsa 4. 1 1 .4,
Index 4. 14. 1 1, 19, 5 .8.3; bivariate normal 4.7.5-6, 12, 4.9.4-5, 4. 14. 13, 16, 7.9.2, 7. 1 1 . 19; Cauchy 4.4.4, 4.8.2, 4. 10.3, 4. 14.4, 16, 5 .7. 1, 5 . 1 1.4, 5 . 12. 19, 24-25, 7. 1 1 . 14; chi-squared 4. 10. 1, 4. 14. 12, 5 .7.7; Dirichlet 4. 14.58; exponential 4.4.3, 4.5.5, 4.7.2, 4.8. 1, 4. 10.4, 4. 14.4-5, 17-19, 24, 33, 5 . 12.32, 39, 6.7. 1 ; extreme-value 4. 1 . 1, 4. 14.46, 7. 1 1 . 13; F (r, s) 4. 10.2, 4, 5 .7.8; first passage 5 . 10.7-8, 5 . 12. 18-19; Fisher's spherical 4. 14.36; gamma 4. 14. 10- 12, 5 .8.3, 5.9.3, 5 . 10.3, 5 . 12 . 14, 33; hypoexponential 4.8.4; log-normal 4.4.5, 5 . 12.43; multinormal 4.9.2, 5 .8.6; normal 4.9.3, 5, 4. 14. 1, 5.8.4-6, 5 . 12.23, 42, 7. 1 1 . 19; spectral 9.3.3; standard normal 4.7.5; Student's t 4. 10.2-3, 5 .7.8; uniform 4.4.3, 4.5 .4, 4.6.6, 4.7. 1, 3, 4, 4.8.5, 4. 1 1 . 1, 8, 4. 14.4, 15, 19, 20, 23-26, 5 . 12.32, 7. 1 1 .4, 9. 1.2, 9.7.5; WeibulI 4.4.7, 7. 1 1 . 13 departure pro 1 1.2.7, 1 1.7.2-4, 1 1 .8. 12 derangement 3.4.9 diagonal selection 6.4.5 dice 1.5 .2, 3.2.4, 3.3.3, 6. 1.2; weightedlloaded 2.7. 12, 5 . 12.36 difference eqns 1.8.20, 3.4.9, 5.2.5 difficult customers 1 1 . 7.4 diffusion: absorbing barrier 13. 12.8-9; Bessel pro 12.9.23, 13.3.5-6, 13. 9 . 1 ; Ehrenfest model 6.5 .5, 36; first passage 13.4.2 ; Ito pro 13.9.3; models 3.4.4, 6.5.5, 6.15. 12, 36; Ornstein-Uhlenbeck pro 13.3.4, 13.7.4-5, 13. 12.3-4, 6; osmosis 6.15.36; reflecting barrier 13. 5 . 1, 13. 12.6; Wiener pro 12.7.22-23; zeros 13.4. 1, 13. 12. 10; Chapter 13 passim
diffusion approximation to birth-death pro 13.3 . 1
distribution: see also density; arc sine 4. 1 1 . 13; arithmetic 5.9.4; Benford 3.6.7; Bernoulli 3. 1 1 . 14, 35 ; beta 4. 1 1 .4; beta-binomial 4.6.5; binomial 2 . 1 .3, 3. 1 1.8, 1 1, 5 . 12.39; bivariate normal 4.7.5-6, 12; Cauchy 4.4.4; chi-squared 4. 10. 1; compound 5.2.3; compound Poisson 5 . 12 . 13; convergence 7.2.4; Dirichlet 3. 1 1.31, 4. 14.58; empirical 9.7.22; exponential 4.4.3, 5 . 12.39; F (r, s) 4. 10.2-4; extreme-value 4. 1 . 1, 4. 14.46, 7. 1 1 . 13; first passage 5 . 10.7-8. 5. 12. 18-19; gamma 4. 14. 10-12; Gaussian, see normal; geometric 3. 1 . 1, 3.2.2, 3.7.5, 3. 1 1 .7, 5 . 12.34, 39, 6 . 1 1.4; hypergeometric 3 . 1 1 . 1 0-1 1 ; hypoexponential 4.8.4; infinitely divisible 5 . 12. 13-14; inverse square 3 . 1 . 1 ; joint 2.5.4; lattice 5 .7.5 ; logarithmic 3 . 1 . 1, 5 .2.3; log-normal 4.4.5, 5 . 12.43; maximum 4.2.2, 4. 14. 17; median 2.7. 1 1, 4.3.4, 7.3. 1 1 ; mixed 2 . 1 .4, 2.3.4, 4. 1.3; modified Poisson 3 . 1 . 1 ; moments 5 . 1 1 .3; multinomial 3.5. 1, 3.6.2; multinormal 4.9.2; negative binomial 3.8.4, 5.2.3, 5 . 12.4, 16; negative hypergeometric 3.5.4; non-central 5 .7.7-8; normal 4.4.6, 8, 4.9.3-5; Poisson 3. 1 . 1, 3.5.2-3, 3. 1 1.6, 4. 14. 1 1, 5.2.3, 5 . 10.3, 5 . 12.8, 14, 17, 33, 37, 39, 7. 1 1 . 18; spectral 9.3.2, 4; standard normal 4.7.5; stationary 6.9. 1 1 ; Student's t 4. 10.2-3, 5 .7.8; symmetric 3.2.5; tails 5 . 1 .2, 5.6.4, 5 . 1 1.3; tilted 5 .7. 1 1 ; trapezoidal 3.8. 1; trinormal 4.9.8-9; uniform 2.7.20, 3.7.5, 3.8. 1, 5 . 1.6, 9.7.5 ; Weibu1l 4.4.7, 7. 1 1 . 13; zeta or Zipf 3. 1 1.5 divergent birth pro 6.8.7 divine providence 3 . 1 1.22 Dobrushin's bound and ergodic coefficient 6. 14.4
dimer problem 3 . 1 1 .34
dog-flea model 6.5.5, 6 . 15 .36
Dirichlet: density 4. 14.58; distn 3. 1 1.31
dominated convergence 5.6.3, 7.2.2
disasters 6. 12.2-3, 6. 15 .28
Doob's L2 inequality 13.7. 1
discontinuous marginal 4.5 . 1
Doob-Kolmogorov inequality 7.8. 1-2
dishonest birth pro 6.8.7
432
doubly stochastic: matrix 6 . 1 . 12, 6. 15.2; Poisson pro 6.15.22-23 downcrossings inequality 12.3. 1 drift 13.3.3, 13.5 . 1, 13.8.9, 13. 12.9 dual queue 1 1.5.2 duration of play 12. 1.4 E Eddington's controversy 1.8.27 editors 6.4. 1 eggs 5 . 12. 13 Ehrenfest model 6.5 .5, 6 . 15 .36 eigenvector 6.6. 1-2, 6. 15.7 embarrassment 2.2.1 empires 6. 15. 10 empirical distn 9.7.22 entrance fee 3.3.4 entropy 7.5 . 1 ; conditional 6. 15.45 ; mutual 3.6.5 epidemic 6. 15 .32 equilibrium, see stationary equivalence class 7. 1. 1 ergodic: coefficient 6. 14.4; measure 9.7. 1 1 ; stationary measure 9.7. 1 1 ergodic theorem: Markov chain 6 . 15 .44, 7. 1 1 .32; Markov pro 7. 1 1 .33, 10.5 . 1 ; stationary pro 9.7. 10- 1 1, 13 Erlang's loss formula 1 1 .8.19 error 3.7.9; of prediction 9.2.2 estimation 2.2.3, 4.5.3, 4. 14.9, 7. 1 1.3 1 Euler: constant 5 . 12.27, 6.15.32; product 5 . 12.34 European call option 13. 10.4-5 event: of convergence 7.2.6; exchangeable 7.3.4-5; invariant 9.5.1; sequence 1.8.16; tail 7.3.3 excess life 10.5 .4; conditional 10.3.4; and current 10.6.9; limit 10.3.3; Markov 8.3.2, 10.3.2; moments 10.3.3; Poisson 6.8.3, 10.3. 1, 10.6.9; reversed 8.3.2; stationary 10.3.3 exchangeability 7.3.4-5 expectation: conditional 3.7.2-3, 4.6.2, 4. 14. 12, 7.9.4; independent r.v.s 7.2.3; linearity 5.6.2; tail integral 4.3.3, 5; tail sum 3 . 1 1 . 13, 4. 14.3 exponential distn: c.f. 5.8.8; holding time 1 1.2.2; in Poisson pro 6.8.3; lack-of-memory
Index property 4. 14.5 ; limit in branching pro 5 .6.2, 5 . 12.21,
gambling: advice 3.9.4; systems
6.7 . 1 ; limit of geometric distn 5. 12.39 ; heavy traffi c limit 1 1 .6.1; distn of maximum 4. 14. 18; in Markov pro 6.8.3, 6.9.9; order statistics 4. 14.33; sample from 4. 14.48; sum 4.8. 1, 4, 4. 14. 10, 5 . 12.50, 6.15.42 exponential martingale 13.3.9 exponential smoothing 9 .7.2
gamma distn 4. 14. 10-12, 5 .8.3,
extinction: of birth-death pro
6 . 1 1 .3, 6 . 15 .25, 27, 12.9. 10; of branching pro 6.7.2-3 extreme-value distn 4. 1 . 1, 4. 14.46, 5 . 12.34, 7. 1 1 . 13; c.f. and mean 5 . 12.27
F(r, s)
F
distn 4. 10.2, 4; non-central 5 .7.8
fair fee 3.3.4 families 1.5 .7, 3.7.8 family, planning 3. 1 1 .30
7.7.4 5.9 .3, 5 . 10.3, 5 . 12 . 14, 33; g. and Poisson 4. 14. 1 1 ; sample from 4. 1 1.3; sum 4. 14. 1 1 gamma fn 4.4. 1, 5 . 12.34 gaps: Poisson 8.4.3, 10. 1.2; recurrent events 5 . 12.45 ; renewal 10. 1.2 Gaussian distn, see normal distn Gaussian pro 9.6.2-4, 13. 12.2; Markov 9.6.2; stationary 9.4.3; white noise 13.8.5 generator 6.9 . 1
filter 9.7.2 fingerprinting 3.11.21 finite: Markov chain 6.5 .8, 6.6.5, 6 . 15 .43-44; stopping time
12.4.5 ; waiting room 1 1.8. 1 first passage: c.f. 5. 10.7-8; diffusion pro 13.4.2; distn
5 . 10.7-8, 5 . 12 . 18-19; Markov chain 6.2. 1, 6.3.6; Markov pro 6.9.5-6; mean 6.3.7; m.g.f. 5 . 12 . 18; s.r.w. 5 .3.8; Wiener pro 13.4.2 first visit by s.r.w. 3. 10. 1, 3 Fisher: spherical distn 4. 14.36; F.-Tippett-Gumbel distn
4. 14.46 FKG inequality 3 . 1 1 . 18, 4. 1 1 . 1 1 flip-flop 8.2. 1
geometric Brownian motion, Wiener pro 13.3.9 geometric distn 3 . 1 . 1, 3.2 .2,
3.7.5, 3 . 1 1 .7, 5 . 12.34, 39; lack-of-memory property
3 . 1 1 .7; as limit 6 . 1 1.4; sample from 4. 1 1 .8; sum 3.8.3-4 goat 1.4.5 graph: colouring 12.2.2; r.w. 6.4.6, 9, 13. 1 1 . 2-3 H
Hlijek-Renyi-Chow inequality
12.9.4-5 Hall, Monty 1.4.5 Hastings algorithm 6. 14.2 Hawaii 2.7. 17 hazard rate 4. 1.4, 4.4.7; technique
4. 1 1 . 10 Heathrow 10.2 . 1 heavy traffic 1 1 .6. 1, 1 1 .7. 16 hen, see eggs Hewitt-Savage zero-one law
7.3.4-5 hitting time 6.9 .5-6; theorem
3.10.1, 5 .3.8 Hoeffding's inequality 12.2. 1-2 Holder's inequality 4. 14.27
Fourier: inversion thm 5.9.5;
homogeneous Markov chain 6. 1 . 1
fourth moment strong law 7. 1 1.6
house 4.2. 1, 6. 15 .20, 5 1
series 9.7. 15, 13. 12.7
fractional moments 3.3.5, 4.3. 1 function, upper-class 7.6. 1 functional eqn 4. 14.5, 19
G
Galton's paradox 1.5.8 gambler's ruin 3 . 1 1 .25-26, 12. 1.4,
12.5.8
1.3.4, 1.8. 12 increasing sequence: of events 1.8. 16; of r.v.s 2.7.2 increments: independent 9.7.6,
16- 17; orthogonal 7.7. 1 ; 9.7. 17; of Wiener pro 9.7.6 independence and symmetry 1.5.3 independent: conditionally 1.5.5; continuous r.v.s 4.5.5, 4. 14.6; current and excess life 10.6.9; customers 1 1 .7. 1 ; discrete r.v.s 3 . 1 1 . 1, 3; events 1.5 . 1 ; increments 9 .7. 17; mean, variance of normal sample
4. 10.5, 5 . 12.42; normal distn 4.7.5; pairwise 1.5 .2, 3.2. 1, 5 . 1 .7; set 3 . 1 1.40; triplewise 5 . 1 .7 indicators and matching 3. 1 1 . 17 inequality: bivariate normal
4.7. 12; Bonferroni 1.8.37; Boole 1.8. 1 1 ; Cauchy-Schwarz 4. 14.27; Chebyshov 7. 1 1 .9; Dobrushin 6. 14.4; Doob-Kolmogorov
heat eqn 13. 12.24
holding time 1 1.2.2
forest 6. 15.30
inclusion-exclusion principle
spectral 9.4. 1, 3; stationary
Farkas's theorem 6.6.2 filtration 12.4. 1-2, 7
1 1 .4. 1, 3; MlGIl 1 1 .3 . 1, 1 1 .7.4; unsuccessful 6 . 15 . 17 immigration: birth-i. 6.8.5 ; branching 5 .4.5, 7.7.2; i.-death 6 . 1 1 .2, 6 . 15 . 18; with disasters 6. 12.2-3, 6. 15 .28 immoral stimulus 1.2 . 1 importance sampling 4. 1 1 . 12
honest birth-death pro 6. 15 .26 Hotelling's theorem 4. 14.59 hypergeometric distn 3. 1 1 . 10-1 1 hypoexponential distn 4.8.4
I
idle period 1 1 .5 .2, 1 1 .8.9 imbedding: jump chain 6.9 . 1 1 ; Markov chain 6.9. 12, 6 . 15 . 17; queues: DIMJ I 1 1 .7. 16; GIMJI
433
7.8. 1; Doob L 2 13.7. 1 ; downcrossings 12.3 . 1 ; FKG 3. 1 1 . 18; Hlij ek-Renyi-Chow 12.9.4-5; Hoeffding 12.2. 1-2; Holder 4. 14.27; Jensen 5 .6. 1, 7.9.4; Kolmogorov 7.8. 1-2, 7. 1 1 .29-30; Kounias 1.8.38; Lyapunov 4. 14.28; maximal 12.4.3-4, 12.9.3, 5, 9; m.g.f. 5.8.2, 12.9.7; Minkowski 4. 14.27; triangle 7. 1 . 1, 3; upcrossings 12.3.2 infinite divisibility 5 . 12. 13-14 inner product 6. 14. 1, 7. 1.2 inspection paradox 10.6.5 insurance 1 1.8. 18, 12.9. 12 integral: Monte Carlo 4. 14.9; normal 4. 14. 1; stochastic 9.7. 19, 13.8. 1-2 invariant event 9 . 5 . 1 inverse square distn 3 . 1 . 1 inverse transform technique 2.3.3 inversion theorem 5.9.5; c.f. 5 . 12.20
Index irreducible Markov pro 6. 15. 15 iterated logarithm 7.6. 1 Ita: formula 13.9.2; process
13.9.3
Jaguar 3. 1 1.25
J
Jensen's inequality 5.6. 1, 7.9.4 joint: c.f. 5. 12.30; density 2.7.20; distn 2.5.4; mass fn 2.5.5; moments 5. 12.30; p.g.f. 5. 1.3-5 jump chain: of MIMI1 1 1.2.6
K
key renewal theorem 10.3.3, 5,
10.6. 1 1
Keynes, J . M. 3.9.6
knapsack problem 12.2. 1 Kolmogorov: criterion 6.5.2; inequality 7.8.1-2, 7. 1 1.29-30 Koro1yuk-Khinchin theorem 8.2.3 Kounias's inequality 1.8.38
5. 12.33, 40, 7. 1 1 .26, 10.6.3; diffusion 13.3. 1; distns 2.3. 1; events 1.8. 16; gamma 5.9.3; geometric-exponential 5. 12.39; 1im inf 1.8. 16; lim sup 1.8. 16, 5.6.3, 7.3.2, 9- 10, 12; local 5.9.2, 5. 10.5-6; martingale 7.8.3; normal 5. 12.41, 7. 1 1 . 19; Poisson 3. 1 1. 17; probability 1.8. 16-18; r.v. 2.7.2; uniform 5.9. 1 linear dependence 3. 1 1 . 15 linear fn of normal r.v. 4.9.3-4 linear prediction 9.7. 1, 3 local central limit theorem 5.9.2, 5 . 10.5-6 logarithmic distn 3. 1. 1, 5.2.3 log-convex 3. 1.5 log-likelihood 7. 1 1.31 log-normal r.v. 4.4.5, 5. 12.43 lottery 1.8.3 1 Lyapunov's inequality 4.14.28 M
Krickeberg decomposition 12.9. 1 1 Kronecker's lemma 7.8.2,
7. 1 1 .30, 12.9.5 kurtosis 4. 14.45
machine 1 1.8. 17 magnets 3.4.8
marginal: discontinuous 4.5 . 1 ;
L
L 2 inequality 13.7. 1 Labouchere system 12.9 . 15 lack of anticipation 6.9.4 1ack-of-memory property: exponential distn 4. 14. 5 ; geometric distn 3. 1 1.7 ladders, see records Lancaster's theorem 4. 14.38 large deviations 5 . 1 1. 1-3, 12.9.7 last exits 6.2. 1, 6. 15.7 lattice distn 5.7.5 law: anomalous numbers 3.6.7; arc sine 3. 10.3, 3. 1 1.28, 5.3.5; De Morgan 1.2. 1; iterated logarithm 7.6. 1; large numbers
2.2.2; strong 7.4.1, 7.8.2, 7. 1 1.6, 9.7. 10; unconscious statistician 3. 1 1.3; weak 7.4. 1, 7. 1 1. 15, 20-2 1 ; zero-one 7.3.4-5 Lebesgue measure 6. 15.29, 13. 12.16 left-continuous r.w. 5.3.7, 5. 12.7 level sets of Wiener pro 13. 12. 16 Levy metric 2.7. 13, 7.1.4, 7.2.4 limit: binomial 3. 1 1. 10; binomial-Poisson 5. 12.39 ; branching 12.9.8; central limit theorem 5. 10. 1, 3, 9,
multinomial 3.6.2; ord�r statistics 4. 14.22 Markov chain in continuous time: ergodic theorem 7. 1 1.33,
10.5 . 1 ; first passage 6.9.5-6; irreducible 6. 15. 15 ; jump chain 6.9. 1 1 ; martingale 12.7. 1 ; mean first passage 6.9.6; mean recurrence time 6.9. 1 1 ; renewal pro 8.3.5; reversible 6. 15. 16, 38; stationary distn 6.9. 1 1; two-state 6.9. 1-2, 6. 15. 17; visits 6.9.9 Markov chain in discrete time: absorbing state 6.2.2; bivariate
6. 15.4; convergence 6.15.43; dice 6. 1.2; ergodic theorem
7. 1 1.32 ; finite 6.5.8, 6.15.43; first passages 6.2. 1, 6.3.6; homogeneous 6. 1.1; imbedded 6.9. 1 1, 6. 15. 17, 1 1.4. 1; last exits 6.2. 1, 6. 15.7; martingale 12. 1.8, 12.3.3; mean first passage 6.3.7; mean recurrence time 6.9. 1 1 ; persistent 6.4. 10; renewal 10.3.2; reversible 6. 14. 1; sampled 6. 1.4, 6.3.8; simulation of 6. 14.3; stationary distn 6.9. 1 1 ; sum 6. 1.8; two-state 6.15. 1 1, 17, 8.2. 1; visits 6.2.3-5, 6.3.5, 6. 15.5, 44 Markov-Kakntani theorem 6.6. 1
434
Markov process: Gaussian 9.6.2; reversible 6. 15. 16 Markov property 6. 1.5, 10; strong
6.1.6 Markov renewal, see Poisson pro Markov time, see stopping time Markovian queue, see MIMI1 marriage problem 3.4.3, 4. 14.35 martingale: backward 12.7.3; birth-death 12.9. 10; branching pro 12. 1.3, 9,
12.9. 1-2, 8, 20; casino 7.7.4; continuous parameter 12.7. 1-2; convergence 7.8.3, 12.1.5, 12.9.6; de Moivre 12. 1.4, 12.4.6; exponential 13.3.9; finite stopping time 12.4.5 ; gambling 12. 1.4, 12.5.8; Markov chain 12. 1.8, 12.3.3, 12.7. 1; optional stopping 12.5. 1-8; orthogonal increments 7.7. 1; partial sum 12.7.3; patterns 12.9.16; Poisson pro 12.7.2; reversed 12.7.3; simple r.w. 12. 1.4, 12.4.6, 12.5.4-7; stopping time 12.4. 1, 5, 7; urn 7. 1 1.27, 12.9. 13-14; Wiener pro 12.9.22-23 mass function, joint 2.5.5 matching 3.4.9, 3. 1 1. 17, 5.2.7,
12.9.2 1 matrix: covariance 3. 11.15; definite 4.9. 1; doubly stochastic 6.1. 12, 6. 15.2; multiplication 4.14.63; square root 4.9. 1; stochastic 6.1. 12, 6. 14. 1 ; sub-stochastic 6.1. 12; transition 7. 1 1.3 1; tridiagonal
6.5. 1, 6. 15. 16 maximal: coupling 4. 12.4-6,
7. 1 1. 16; inequality 12.4.3-4, 12.9.3, 5, 9 maximum of: branching pro
12.9.20; mu1tinormal 5.9.7; r.w. 3. 10.2, 3. 1 1.28, 5.3. 1, 6. 1.3, 12.4.6; uniforms
5. 12.32; Wiener pro 13. 12.8, 1 1, 15, 17 maximum r.v. 4.2.2, 4.5.4,
4.14. 17-18, 5. 12.32, 7. 1 1 . 14 mean: extreme-value 5. 12.27; first passage 6.3.7, 6.9.6; negative binomial 5. 12.4; normal 4.4.6; recurrence time
6.9. 1 1 ; waiting time 1 1.4.2, 1 1.8.6, 10 measure: ergodic 9.7. 1 1-12; Lebesgue 6. 15.29, 13. 12. 16;
Index stationary 9.7. 1 1-12; strongly mixing 9.7.12 median 2.7. 1 1, 4.3.4, 7.3. 1 1
norm 7. 1. 1, 7.9.6; equivalence class 7. 1 . 1 ; mean square 7.9.6; rth mean 7. 1 . 1 normal distn 4.4.6; bivariate
menages 1.8.23
modified renewal 10.6. 12
4.7.5-6, 12; central limit theory 5. 10. 1, 3, 9, 5 . 12.23, 40; characterization of 5 . 12.23; cumulants 5 .7.4; limit 7. 1 1 . 19 ; linear transfomations 4.9.3-4; Mills's ratio 4.4.8, 4. 14. 1 ; moments 4.4.6, 4. 14. 1 ; multivariate 4.9.2, 5 .8.6; regression 4.8.7, 4.9 .6, 4. 14. 13, 7.9.2; sample 4. 10.5, 5 . 12.42; simulation of 4. 1 1.7, 4. 14.49; square 4. 14. 12; standard 4.7.5; sum 4.9.3; sum of squares 4. 14. 12; trivariate 4.9 .8-9; uncorrelated 4.8.6 normal integral 4. 14. 1 normal number theorem 9.7. 14
moments: branching pro 5 .4. 1 ;
now 6. 15.50
Mercator projection 6. 13.5 meteorites 9.7.4 metric 2.7. 13, 7. 1.4; Levy 2.7. 13,
7. 1.4, 7.2.4; total variation 2.7. 13 m.g.f. inequality 5 .8.2, 12.9.7 migration pr., open 1 1 .7. 1, 5 millionaires 3.9.4 Mills's ratio 4.4.8, 4. 14. 1 minimal, solution 6.3.6--7, 6.9.6 Minkowski's inequality 4. 14.27 misprints 6.4. 1 mixing, strong 9.7. 12 mixture 2.1.4, 2.3.4, 4. 1 .3, 5 . 1.9
fractional 3.3.5, 4.3. 1 ; generating fn 5 . 1.8, 5 .8.2, 5 . 1 1.3; joint 5 . 12.30; problem
5. 12.43; renewal pro 10. 1 . 1 ; tail integral 4.3.3, 5 . 1 1.3 Monte Carlo 4. 14.9 Monty Hall 1.4.5 Moscow 1 1 .8.3 moving average 8.7. 1, 9 . 1 .3,
9.4.2, 9.5.3, 9.7. 1-2, 7; spectral density 9.7.7 multinomial distn 3.5 . 1 ; marginals 3.6.2; p.g.f. 5 . 1.5 multinormal distn 4.9.2; c.f. 5 .8.6; conditioned 4.9.6--7; covariance matrix 4.9.2; maximum 5.9.7; sampling from 4. 14.62; standard 4.9.2; transformed 4.9.3, 4. 14.62 Murphy's law 1.3.2 mutual information 3.6.5
N
needle, Buffon's 4.5 .2,
o 13. 12.20 open migration 1 1 .7. 1, 5 optimal: packing 12.2. 1; price 12.9 . 19 ; reset time 13. 12.22; serving 1 1 .8. 13 optimal stopping: dice 3.3.8-9; marriage 4. 14.35 optional stopping 12.5. 1-8, 12.9. 19; diffusion 13.4.2; Poisson 12.7.2 order statistics 4. 14.21; exponential 4. 14.33; general
4. 14.21; marginals 4. 14.22; uniform 4. 14.23-24, 39, 6.15.42, 12.7.3 Ornstein-Uhlenbeck pro 9.7. 19, 13.3.4, 13.7.4--5 , 13. 12.3-4, 6; reflected 13. 12.6 orthogonal: increments 7.7. 1 ; polynomials 4. 14.37 osmosis 6.15.36
4. 14.3 1-32 bivariate 5 . 12.16; moments
5 . 12.4; p.g.f. 5 . 1 . 1, 5 . 12.4 negative hypergeometric distn
3.5.4 Newton, I. 3.8.5 non-central distn 5 .7.7-8 non-homogeneous : birth pro
6 . 15 .24; Poisson pro 6. 13.7, 6.15. 19-20 noodle, Buffon's 4. 14.3 1
particle 6 . 15 .33 partition of sample space 1.8.10 Pasta property 6.9.4 patterns 1.3.2, 5 .2.6, 5 . 12.2,
10.6. 17, 12.9 . 16 pawns 2.7. 18 Pepys's problem 3.8.5 periodic state 6.5 .4, 6. 15 .3 persistent: chain 6.4. 10, 6. 15.6; r.w. 5 . 12.5-6, 6.3.2, 6.9.8, 7.3.3, 13. 1 1.2; state 6.2.3-4, 6.9.7 Petersburg, see St Petersburg pig 1.8.22 points, problem of 3.9.4, 3 . 1 1.24 Poisson: approximation 3. 1 1.35; coupling 4. 12.2; flips 3.5 .2, 5 . 12.37; sampling 6.9.4 Poisson distn 3.5.3; characterization of 5 . 12.8, 15 ;
occupation time for Wiener pro
p
negative binomial distn 3.8.4;
parking 4. 14.30 Parrando's paradox 6 . 15 .48
pairwise independent: events
1.5.2; r.v.s 3.2. 1, 3.3.3, 5 . 1.7 paradox: Bertrand 4. 14.8; Borel 4.6. 1 ; Carroll 1.4.4; Galton 1.5 .8; inspection 10.6.5; Parrando 6 . 15 .48; St Petersburg 3.3.4; voter 3.6.6 parallel lines 4. 14.52 parallelogram 4. 14.60; property
7. 1.2;
435
compound 3.8.6, 5 . 12. 13; and gamma distn 4. 14. 1 1 ; limit of binomial 5 . 12.39; modified 3. 1 . 1 ; sum 3 . 1 1.6, 7.2. 10 Poisson pro 6. 15.29; age 10.5 .4; arrivals 8.7.4, 10.6.8; autocovariance 7. 1 1.5, 9 .6. 1; characterization 6. 15 .29, 9.7. 16; colouring theorem 6. 15.39; compound 6.15.21; conditional property 6. 13.6; continuity in m.s. 7. 1 1 . 5 ; covariance 7. 1 1 . 5 ; differentiability 7. 1 1.5; doubly stochastic 6. 15 .22-23; excess life 6.8.3, 10.3. 1 ; forest 6 . 15 .30; gaps 10. 1.2; Markov renewal pro 8.3.5, 10.6.9; martingales 12.7.2; non-homogeneous 6. 13.7, 6.15. 19-20; optional stopping 12.7.2; perturbed 6. 15 .39-40; renewal 8.3.5, 10.6.9-10; Renyi's theorem 6. 15.39; repairs 1 1 .7. 18; sampling 6.9.4; spatial 6 . 15 .30-31, 7.4.3; spectral density 9.7.6; sphere 6. 13.3-4; stationary increments 9.7.6, 16; superposed 6.8. 1 ; thinned 6.8.2; total life 10.6.5; traffi c 6. 15 .40, 49, 8.4.3 poker 1.8.33; dice 1.8.34 P6lya's urn 12.9. 13-14 portfolio 13. 12.23; self-financing 13. 10.2-3 positive definite 9.6. 1, 4.9 . 1
Index positive state, see non-null postage stamp lemma 6.3.9 potential theory 13. 1 1 . 1-3 power series approximation
7. 1 1 . 17 Pratt's lemma 7. 10.5 predictable step fn 13.8.4 predictor: best 7.9. 1 ; linear 7.9.3,
9.2. 1-2, 9 .7. 1, 3 probabilistic method 1.8.28, 3.4.7
exchange 1 1 .8.9; two servers 8.4. 1, 5, 1 1.7.3, 1 1.8. 14; virtual waiting 1 1 .8.7; waiting time 1 1 .2.3, 1 1.5.2-3, 1 1 .8.6, 8, 10 quotient 3.3. 1, 4.7.2, 10, 13-14,
4. 10.4, 4. 1 1 . 10, 4. 14. 1 1, 14, 16, 40, 5.2.4, 5 . 12.49, 6.15.42 R
probability: continuity 1.8. 16,
radioactivity 10.6.6-8
1.8. 18; p.g.f. 5 . 12.4, 13; vector 4. 1 1 .6 problem: matching 3.4.9, 3 . 1 1 . 17, 5.2.7, 12.9.21; menages 1.8.23; Pepys 3.8.5; of points 3.9.4, 3 . 1 1 .24; Waldegrave 5 . 12 . 10
random: bias 5 . 10.9; binomial
program, dual, linear, and primal
6.6.3 projected r.w. 5 . 12.6 projection theorem 9.2. 10 proof-reading 6.4. 1 proportion, see empirical ratio proportional investor 13. 12.23 prosecutor's fallacy 1.4.6 protocol 1.4.5, 1.8.26 pull-through property 3.7. 1
Q
quadratic variation 8.5 .4, 13.7.2 queue: batch 1 1 .8.4; baulking
8.4.4, 1 1 .8.2, 19; busy period 6. 12. 1, 1 1.3.2-3, 1 1 . 5 . 1, 1 1 .8.5, 9 ; costs 1 1 .8. 13; departure pro 1 1.2.7, 1 1.7.2-4, 1 1 .8. 12; difficult customer 1 1.7.4; DIM/I 1 1 .4.3, 1 1.8. 15; dual 1 1.5.2; Erlang's loss fn 1 1.8. 19; finite waiting room 1 1 .8. 1; G/GIl 1 1 . 5 . 1, 1 1.8.8; GIM/l 1 1 .4. 1-2, 1 1.5.2-3; heavy traffi c 1 1 . 6 . 1, 1 1.8. 15 ; idle period 1 1 .5 .2, 1 1.8.9; imbedded branching 1 1 .3.2, 1 1 .8.5, 1 1 ; imbedded Markov pro 1 1 .2.6, 1 1 .4. 1, 3, 1 1 .4. 1 ; imbedded renewal 1 1 .3.3, 1 1 . 5 . 1 ; imbedded r.w. 1 1 .2.2, 5 ; Markov, see MIMIl ; MID/I 1 1.3. 1, 1 1.8. 10- 1 1 ; MIG/I 1 1 .3.3, 1 1 .8.6-7; MIG/co 6. 12.4, 1 1.8.9; migration system 1 1 .7. 1, 5; MIMIl 6.9.3, 6. 12. 1, 1 1.2.2-3, 5-6, 1 1 .3.2, 1 1 . 6 . 1, 1 1 .8.5, 12; MIM/k 1 1.7.2, 1 1.8. 13; M/M/co 8.7.4; series 1 1 .8.3, 12; supermarket 1 1 .8.3; tandem 1 1.2.7, 1 1 .8.3; taxicabs 1 1.8. 16; telephone
coefficient 5.2.1; chord 4. 13. 1; dead period 10.6.7; harmonic series 7. 1 1 .37; integers 6.15.34; line 4. 13. 1-3, 4. 14.52; paper 4. 14.56; parameter 4.6.5, 5 . 1.6, 5 .2.3, 5.2.8; particles 6.4.8; pebbles 4. 14.5 1; permutation 4. 1 1.2; perpendicular 4. 14.50; polygons 4. 13. 10, 6.4.9; rock 4. 14.57; rods 4. 14.25-26, 53-54; sample 4. 14.21; subsequence 7. 1 1.25; sum 3.7.6, 3.8.6, 5.2.3, 5 . 12.50, 10.2.2; telegraph 8.2.2; triangle 4.5.6, 4. 13.6-8, 1 1, 13; velocity 6 . 15 .33, 40 random sample: normal 4. 10.5 ; ordered 4. 12.2 1 random variable: see also density and distribution; arc sine
4. 1 1 . 13; arithmetic 5.9.4; B ernoulli 3. 1 1 . 14, 35; beta 4. 1 1.4; beta-binomial 4.6.5; binomial 2 . 1 .3, 3. 1 1.8, 1 1, 5 . 12.39; bivariate normal 4.7.5-6, 12; Cauchy 4.4.4; c.f. 5 . 12.26-3 1 ; chi-squared 4. 10. 1; compounding 5.2.3; continuous 2.3. 1 ; Dirichlet 3 . 1 1.31, 4. 14.58; expectation 5.6.2, 7.2.3; exponential 4.4.3, 5 . 12.39 ; extreme-value 4. 1 . 1, 4. 14.46; F (r, s) 4. 10.2, 4, 5 .7.8; gamma 4. 1 1 .3, 4. 14. 10-12; geometric 3.1. 1, 3 . 1 1 .7; hypergeometric 3. 1 1 . 10- 1 1 ; independent 3. 1 1 . 1, 3, 4.5.5, 7.2.3; indicator 3 . 1 1 . 17; infinitely divisible 5 . 12. 13-14; logarithmic 3 . 1 . 1, 5.2.3; log-normal 4.4.5 ; median 2.7. 1 1, 4.3.4; m.gJ. 5 . 1 .8, 5 .8.2, 5 . 1 1.3; multinomial 3.5 . 1 ; multinormal 4.9.2; negative binomial 3.8.4; normal 4.4.6, 4.7.5-6, 12;
436
orthogonal 7.7. 1; p.gJ. 5 . 12.4, 13; Poisson 3.5.3; standard
normal 4.7.5; Student's t 4. 10.2-3, 5 .7.8; symmetric 3.2.5, 4. 1.2, 5 . 12.22; tails 3. 1 1 . 13, 4.3.3, 5, 4. 14.3, 5 . 1 .2, 5.6.4, 5 . 1 1.3; tilted 5 . 1.9, 5 .7. 1 1 ; trivial 3. 1 1.2; truncated 2.4.2; uncorrelated 3.11. 12, 16, 4.5.7-8, 4.8.6; uniform 3.8. 1, 4.8.4, 4. 1 1 . 1, 9.7.5 ; waiting time, see queue; Weibull 4.4.7; zeta or Zipf 3. 1 1.5 random walk: absorbed 3 . 1 1 .39, 12.5 .4-5; arc sine laws 3. 10.3, 3 . 1 1 .28, 5 .3.5; on binary tree 6.4.7; conditional 3.9 .2-3; on cube 6.3.4; first passage 5 .3.8; first visit 3. 10.3; on graph 6.4.6, 9, 13. 1 1.2-3; on hexagon 6. 15.35; imbedded in queue 1 1 .2.2, 5; left-continuous 5 .3.7, 5 . 12.7; martingale 12. 1 .4, 12.4.6, 12.5.4-5; maximum 3. 10.2, 3 . 1 1.28, 5 .3.1; persistent 5 . 12.5-6, 6.3.2; potentials 13. 1 1.2-3; projected 5. 12.6; range of 3 . 1 1.27; reflected 11.2.1-2; retaining barrier 1 1 .2.4; returns to origin 3.10.1, 5 .3.2; reversible 6.5 . 1 ; simple 3.9. 1-3, 5 , 3. 10. 1-3; on square 5 .3.3; symmetric 1.7.3, 3. 1 1.23; three dimensional 6.15 .9-10; transient 5 . 12.44, 6. 15 .9, 7.5.3; truncated 6.5 .7; two dimensional 5 .3.4, 5 . 12.6, 12.9. 17; visits 3 . 1 1 .23, 6.9.8, 10; zero mean 7.5.3; zeros of 3. 10. 1, 5 .3.2, 5 . 12.5-6 range of r.w. 3 . 1 1.27 rate of convergence 6.15.43 ratios 4.3.2; Mills's 4.4.8; sex 3. 1 1.22 record: times 4.2. 1, 4, 4.6.6, 10; values 6. 15 .20, 7. 1 1 .36 recurrence, see difference recurrence time 6.9 . 1 1 recurrent: event 5. 12.45, 7.5.2, 9.7.4; see persistent red now 12.9 . 18 reflecting barrier: s.r.w. 1 1.2. 1-2; drifting Wiener pro 13.5 . 1; Ornstein-Uhlenbeck pro
13. 12.6 regeneration 1 1 .3.3 regression 4.8.7, 4.9.6, 4. 14.13,
7.9.2 rejection method 4. 1 1.3-4, 13
Index reliability 3.4.5-6, 3. 1 1. 18-20, renewal: age, see current life; alternating 10.5.2, 10.6. 14; asymptotics 10.6. 1 1 ; Bernoulli 8.7.3; central limit theorem 10.6.3; counters 10.6.6-8, 15; current life 10.3.2, 10.5.4, 10.6.4; delayed 10.6. 12; excess life 8.3.2, 10.3. 1-4, 10.5.4; r. function 1 0.6. 1 1 ; gaps 1 0. 1 .2; key r. theorem 10.3.3, 5, 10.6 . 1 1 ; Markov 8.3.5; m.gJ. 10. 1. 1; moments 10. 1.1; Poisson 8.3.5, 10.6.9-10; r. process 8.3.4; r.-reward 10.5. 1-4; r. sequence 6. 15.8, 8.3. 1, 3; stationary 10.6. 18; stopping time 12.4.2; sum/superposed 10.6. 10; thinning 10.6. 16 Renyi's theorem 6. 15.39 repairman 1 1.7. 18 repulsion 1.8.29 reservoir 6.4.3 resources 6. 15.47 retaining barrier 1 1.2.4 reversible: birth-death pro 6.15. 16; chain 6. 14. 1; Markov pro 6. 15. 16, 38; queue 1 1.7.2-3, 1 1.8. 12, 14; r.w. 6.5. 1 Riemann-Lebesgue lemma 5.7.6 robots 3.7.7 rods 4. 14.25-26, ruin 1 1.8. 18, 12.9. 12; see also gambler's ruin runs 1.8.21, 3.4. 1, 3.7.10, 5. 12.3, 46-47
S
a -field 1.2.2, 4, 1.8.3, 9.5. 1, 9.7. 13; increasing sequence of 12.4.7 St John's College 4. 14.5 1 St Petersburg paradox 3.3.4 sample: normal 4. 10.5, 5. 12.42; ordered 4. 12.21 sampling 3. 1 1.36; Poisson 6.9.4; with and without replacement 3. 11.10 sampling from distn: arc sine 4. 11.13; beta 4. 1 1.4-5; Cauchy 4. 1 1.9; exponential 4. 14.48; gamma 4. 1 1.3; geometric 4. 1 1.8; Markov chain 6. 14.3; multinormal 4. 14.62; normal 4. 1 1.7, 4.14.49; s.r.w. 4. 1 1.6; uniform 4. 1 1 . 1
secretary problem 3. 1 1 . 17, 4. 14.35 self-financing portfolio 13. 10.2-3 semi-invariant 5.7.3-4 sequence: of c.f.s 5. 12.35 ; of distns 2.3. 1; of events 1.8. 16; of heads and tails, see pattern; renewal 6. 15.8, 8.3. 1, 3; of r.v.s 2.7.2 series of queues 1 1.8.3, 12 shift operator 9.7. 1 1-12 shocks 6. 13.6 shorting 13. 1 1.2 simple birth pro 6.8.4-5, 6. 15.23 simple birth-death pr.: conditioned 6. 1 1.4-5; diffusion approximation 13.3. 1 ; extinction 6. 1 1.3, 6. 15.27; visits 6. 1 1.6-7 simple immigration-death pro 6. 1 1.2, 6. 15. 18 simple: process 8.2.3; r.w. 3.9. 1-3, 5, 3. 10. 1-3, 3. 1 1.23, 27-29, 1 1.2. 1, 12. 1.4, 12.4.6, 12.5.4-7 simplex 6. 15.42; algorithm 3. 1 1.33 simulation, see sampling sixes 3.2.4 skewness 4. 14.44 sleuth 3. 1 1.21 Slutsky's theorem 7.2.5 smoothing 9.7.2 snow 1.7. 1 space, vector 2.7.3, 3.6. 1 span of r.v. 5 .7.5, 5.9.4 Sparre Andersen theorem 13. 12. 18 spectral: density 9.3.3; distribution 9.3.2, 4, 9.7.2-7; increments 9.4. 1, 3 spectrum 9.3. 1 sphere 1.8.28, 4.6. 1, 6. 13.3-4, 12.9.23, 13. 1 1 . 1 ; empty 6. 15.31 squeezing 4. 14.47 standard: bivariate normal 4.7.5; multinormal 4.9.2; normal 4.7.5; Wiener pro 9.6. 1, 9.7. 18-2 1, 13. 12. 1-3 state: absorbing 6.2.2; persistent 6.2.3-4; symmetric 6.2.5; transient 6.2.4 stationary distn 6.9. 1, 3-4, 1 1-12; 6. 1 1.2; birth-death pro 6. 1 1.4; current life 10.6.4; excess life 10.3.3; Markov chain 9. 1.4; open migration 1 1.7. 1,
437
5 ; queue length 8.4.4, 8.7.4, 1 1.2. 1-2, 6, 1 1.4. 1, 1 1.5 .2, and Section 1 1.8 passim; r.w. 1 1.2. 1-2; waiting time 1 1.2.3, 1 1.5.2-3, 1 1.8.8 stationary excess life 10.3.3 stationary increments 9.7. 17 stationary measure 9.7. 1 1-12 stationary renewal pro 10.6. 18, Stirling's formula 3. 10. 1, 3. 1 1.22, 5.9.6, 5. 12.5, 6. 15.9, 7. 1 1.26 stochastic: integral 9.7. 19, 13.8. 1-2; matrix 6.1. 12, 6. 14. 1, 6. 15.2; ordering 4. 12. 1-2 stopping time 6. 1.6, 10.2.2, 12.4. 1-2, 5, 7; for renewal pro 12.4.2 strategy 3.3.8-9, 3. 1 1.25, 4. 14.35, 6.15 .50, 12.9. 19, 13. 12.22 strong law of large numbers 7.4. 1, 7.5. 1-3, 7.8.2, 7. 1 1.6, 9.7. 10 strong Markov property 6.1.6 strong mixing 9.7. 12 Student's t distn 4. 10.2-3; non-central 5.7.8 subadditive fn 6.15. 14, 8.3.3 subgraph 13. 1 1.3 sum of independent r.v.s: Bernoulli 3. 1 1 . 14, 35; binomial 3. 1 1.8, 1 1 ; Cauchy 4.8.2, 5 . 1 1.4, 5. 12.24-25; chi-squared 4. 10. 1, 4. 14. 12; exponential 4.8. 1, 4, 4. 14. 10, 5. 12.50, 6. 15.42; gamma 4. 14. 1 1 ; geometric 3.8.3-4; normal 4.9.3; p.g.f. 5 . 12.1; Poisson 3. 1 1.6, 7.2. 10; random 3.7.6, 3.8.6, 5.2.3, 5. 12.50, 10.2.2; renewals 10.6. 10; uniform 3.8. 1, 4.8.5 sum of Markov chains 6. 1.8 supercritical branching 6.7.2 supermartingale 12. 1.8 superposed: Poisson pro 6.8. 1 ; renewal pro 10.6. 10 sure thing principle 1.7.4 survival 3.4.3, 4. 1.4 Sylvester's problem 4. 13. 12, 4. 14.60 symmetric: r.v. 3.2.5, 4. 1.2, 5. 12.22; r.w. 1.7.3; state 6.2.5 symmetry and independence 1.5.3 system 7.7.4; Labouchere 12.9. 15
Index t,
T
Student's 4. 10.2-3; non-central
5 .7.8
tail: c.f. 5.7.6; equivalent 7. 1 1 .34; event 7.3.3, 5; function 9.5.3; integral 4.3.3, 5 ; sum 3. 1 1 . 13,
4. 14.3 tail of distu: and moments 5.6.4, 5 . 1 1.3; p.g.f. 5 . 1.2 tandem queue 11.2.7, 1 1 .8.3
two-dimensional: r.w. 5.3.4, 5 . 12.6, 12.9. 17; Wiener pro
13. 12. 12-14 two server queue 8.4. 1, 5, 1 1.7.3,
1 1 .8. 14
telephone: exchange 1 1 .8.9; sales
3 . 1 1 .38 testimony 1.8.27 thinning 6.8.2; renewal 10.6.16 three-dimensional r.w. 6 . 15 . 10; transience of 6. 15.9, three-dimensional Wiener pro
13. 1 1 . 1 three series theorem 7. 1 1.35 tied-down Wiener pro 9 .7.2 1-22,
13.6.2-5 tilted distu 5 . 1.9, 5.7. 1 1 time-reversibility 6.5. 1-3, 6. 15 . .1 6 total life 10.6.5 total variation distance 4. 12.3-4,
7.2.9, 7. 1 1 . 16 tower property 3.7. 1, 4. 14.29 traffic: gaps 5. 12.45, 8.4.3; heavy 1 1.6. 1, 1 1.8. 15; Poisson
6.15.40, 49, 8.4.3 transform, inverse 2.3.3 transient: r.w. 5 . 12.44, 7.5 .3; Wiener pro 13. 1 1 . 1 transition matrix 7. 1 1.3 1
17, 8.2. 1; Markov pro 6.9. 1-2, 6. 15 . 16-17 Type: T. one counter 10.6.6-7; T. two counter 10.6. 15
U Mys8ka 8.7.7
u
unconscious statistician 3. 1 1.3 uncorrelated r.v. 3 . 1 1 . 12, 16,
4.5 .7-8, 4.8.6 uniform integrability: Section
7. 10 passim, 10.2.4, 12.5. 1-2 uniform distn 3.8. 1, 4.8.4, 4. 1 1 . 1, 9.7.5; maximum 5. 12.32; order statistics 4. 14.23-24, 39, 6.15.42; sample from 4. 1 1 . 1 ; sum 3.8. 1, 4.8.4 uniqueness of conditional expectation 3.7.2 upcrossings inequality 12.3.2 upper class fn 7.6 . 1 urns 1.4.4, 1.8.24--2 5, 3.4.2,
4, 6.3. 10, 6 . 1 5 . 12; P6lya's 12.9. 13-14
V
value function 13. 10.5 variance: branching pro 5 . 12.9; conditional 3.7.4, 4.6.7; normal
4.4.6 vector space 2.7.3, 3.6. 1 Vice-Chancellor 1.3.4, 1.8. 13 virtual waiting 1 1 .8.7
trapezoidal distn 3.8. 1
visits: birth-death pro 6.1 1.6-7;
trial, de Moivre 3.5 . 1
Markov chain 6.2.3-5, 6.3.5, 6.9.9, 6. 15.5, 44; r.w. 3. 1 1.23, 29, 6.9.8, 10 voter paradox 3.6.6
Trinity College 12.9. 15 triplewise independent 5 . 1.7 trivariate normal distn 4.9.8-9 trivial r.v. 3 . 1 1.2 truncated: r.v. 2.4.2; r.w. 6.5.7 Tunm's theorem 3 . 1 1.40 turning time 4.6. 10
Waldegrave's problem 5 . 12. 10 Waring's theorem 1.8. 13, 5.2. 1 weak law of large numbers 7.4. 1,
7. 1 1 . 15, 20-21 Weibull distn 4.4.7, 7. 1 1 . 13
transitive coins 2.7. 16
triangle inequality 7. 1 . 1, 3
1 1 .2.3; stationary distn 1 1 .2.3, 11.5.2-3; virtual 1 1 .8.7 Wald's eqn 10.2.2-3
two-state Markov chain 6.15. 1 1,
taxis 1 1 .8.16 telekinesis 2.7.8
1 1 .4.2; in MlDIl 1 1.8. 10; in MlG/I 1 1 .8.6; in MIMII
W
Weierstrass's theorem 7.3.6 white noise, Gaussian 13.8.5 Wiener process : absorbing barriers 13. 13.8, 9; arc sine laws 13.4.3, 13. 12. 10,
13. 12. 19; area 12.9.22; Bartlett eqn 13.3.3; Bessel pro 13.3.5; on circle 13.9.4; conditional 8.5 .2, 9.7.21, 13.6. 1; constructed 13. 12.7;
d-dimensional 13.7. 1; drift
13.3.3, 13.5.1; on ellipse 13.9.5; expansion 13. 12.7; first passage 13.4.2; geometric 13.3.9, 13.4. 1; hits sphere 13. 1 1 . 1 ; hitting barrier 13.4.2; integrated 9.7.20, 12.9.22, 13.3.8, 13.8. 1-2; level sets 13. 12. 16; martingales 12.9 .22-23, 13.3.8-9; maximum 13. 12.8, 1 1, 15, 17; occupation time 13. 12.20; quadratic variation 8.5.4, 13.7.2; reflected 13.5 . 1; sign 13. 12.21; standard 9.7. 18-2 1; three-dimensional 13. 1 1 . 1 ; tied-down, see Brownian bridge; transformed 9.7. 18,
13. 12. 1, 3; two-dimensional 13. 12. 12-14; zeros of 13.4.3, 13. 12. 10 Wiener-Hopf eqn 1 1.5.3, 1 1 .8.8
X-ray 4. 14.32
waiting room 1 1 .8. 1, 15 waiting time: dependent 1 1.2.7; for a gap 10. 1.2; in G/G/I
1 1.5.2, 1 1 .8.8; in GIMlI
438
x Z
zero-one law, Hewitt-Savage
7.3.4-5
FROM A REVI EW OF PROBABILITY ANO RANDOM PROCESSES: PROBLEMS AND SOLUTIONS
