This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
cm
X>O, (21.17)
where
H(z,b)= e  (l+zP) I / @
in the domain 1
(21.18)
+ zp > 0,includes all the cdf’s
as special cases. Equation (21.17) defines the generalized extreme value distributions for maxima, while H ( z ,p) in (21.18) correspond to its standard form. Similarly,
G(z, P, a, A)
=
1 H (  T
P, a , A)
(21.19)
EXTREME VALUE DISTRIBUTIONS
194
defines the generalized extreme value distributions for minima, and (212 0 )
G ( x , P ) = 1 H (  x , P )
correspond to its standard form.
21.6
Moments
Making use of the representation in Exercise 21.1, we can express moments of the extreme value distributions in terms of moments of the standard exponential distribution. Let random variables Y , W , and V have cdf's H I , ~ ( X ) , respectively. Then, from Exercise 21.1, we have the following H Z . ~ ( XH3(x), ), relations:
y
d
=xl/a,
wd
XI/",
v =d

logX,
(21.21)
where X has the standard exponential distribution. Hence, we have
EYk
=
EX'/"
1
cc xk/a
e I dx,
(21.22)
and (21.24) It readily follows from (21.22) that moments E Y k exist if k
k).
E Y k =I (1
< Q and that (21.25)
Relation (21.23) reveals that moments EW' exist for any 1 , 2 , . . . , and
Q
> 0 and k
=
(21.26) From (21.25) and (21.26), we also obtain (21.27)
Var for any a
> 0.
w = r (1 + :)

{r (1 +
:)}
2
(21.28)
195
MOMENTS It is known that Euler's constant y = 0.57722.. . is defined as lim
=
n+cc
(2

log n )
(21.29)
k=l
rw
=

Jo
logx e" dx.
(21.30)
Comparing (21.24) and (21.30), we immediately see that
EV
= y = 0.57722 ....
(21.31)
Another way to obtain (21.31) is through the characteristic function fv(t) of the random variable V . We have
fv(t)
EeitV  ~ ~  i t l o g X EXit
lee
=
=
xit
e x dx
= r ( lit).
(21.32)
From (21.32) and the relation
f'"(0)
= i k E V k,
we readily find that
k = 1,2,... .
E V ~= (  i ) k r ( k ) ( i ) ,
(21.33)
The following useful identity, which is valid for positive z , helps us to find the necessary derivatives of the gamma function:
(21.34) Since
we obtain from (21.33) and (21.34) that
EV
=
 q i ) = +(I) = 7,
(21.36)
(21.37)
It follows now from (21.36) and (21.37) that n
Var V
7r4
=
6
(21.38)
EXTREME VALUE DISTRIBUTIONS
196
), Now, let, the random varia.bles Yl, W1, and V1 have cdf's G I , ~ ( XGz,a(x), and G3 ( x ) , respectively. Since Yl
d
=
Y,
d
WI =
w,v1 = v, d
(21.39)
we immediately obtain EY:
5)'
=
Q
k
(21.40) (21.41)
for cv > 2, (21.42) Var Wl
=
Var W =I'
(
l +
(21.43) (21.44)
and ?I2
Var V1 = Var V = 6
.
(21.45)
It is important t o mention here that extreme value distributions discussed in this chapter have assumed a very important role in lifetesting and reliability probltnis besides being used as probabilistic models in a variety of other problems.
CHAPTER 22
L 0GISTIC DISTRIBUTION 22.1
Introduction
Let Vl and V2 be i.i.d. random variables having the extreme value distribution with cdf [see (21.7)] &(2)
Let V
= V1 
= ee",
02
< 2 < 00.
V2. Then, the cdf Fv(x)of V is obtained as roo
J00
This distribution is a particular case of the logistic distribution, which has been known since the pioneering work of Verhulst (1838, 1845) on demography. A booklength account of logistic distributions, discussing in great detail their various properties and applications, is available [Balakrishnan (1992)].
22.2
Notations
A random variable X is said to have a logistic distribution if its pdf is given by
The corresponding cdf is given by
197
198
LOGISTIC DISTRIBUTION
We will use X L o ( p , a 2 ) to denote the random variable X which has the logistic distribution with pdf and cdf as in (22.2) and (22.3), respectively. It is evident that p (03 < p < cc) is the location parameter while u ( u > 0) is the scale parameter. Shortly, we will show that p and a2 are, in fact, the mean and variance of this logistic distribution. The standard logistic random variable, denoted by Y Lo(0, l), has its pdf and cdf as N
N
7r
eXX/&
PY(Z)= 
03<2<03
2’
(22.4)
and
cc < 2 < 03,
(22.5)
respectively. Although we will see later that this is the standardized form of the distribution (i.e., having zero mean and unit variance), yet it is often convenient to work with the random variable V having the logistic Lo (0, 7r2/3) distribution as it possesses the following simple expressions for the pdf and cdf: (22.6) and FV(2) =
1
1+>m
< 2 < 00.
(22.7)
Since the logistic density function in (22.2) can be rewritten as
P x ( z )=
a { 7r
sech2
7 r ( 2
2ud3
},
m
< 2 < oc,
(22.8)
the logistic distribution is also sometimes referred to as the sechsquared distribution. If X Lo(p,a2),then we note from (22.2) and (22.3) that the cdf F x ( 2 ) and the pdf p x ( 2 ) satisfy the relationship N
7r
p x ( z ) = Fx(2)
06
(1  F x ( 2 ) } ,
Of course, in the special case when V reduces to
N

03
< 2 < 00.
(22.9)
Lo (0, 7r2/3),the relationship in (22.9)
p v ( z ) = Fv(2)(1  Fv(2)},
cc < 2 < cc.
(22.10)
When X Lo(p,a2),we may observe from (22.3) that the cdf of X has exponentially decreasing tails. Specifically, we find that
Fx(z) as z + 00.
N
e.rr(X+P)/(u&)
and
1  Fx(2)
eX(zP)/(u&)
199
MOMENTS
22.3
Moments
Let V Lo (0, 7r2/3). The simple form of its pdf p v ( x ) in (22.6) enables us to obtain the moments of V. Then, upon exploiting the obvious relationship N
x=%IT+ p ,
(22.11)
we can readily obtain the moments of X Lo ( p ,0 ' ) . Of course, the exponential rate of decrease of the pdf entails the existence of all the moments of the logistic distribution. N

Moments about zero: Let v LO (0,7 r 2 / 3 ) . Since
all the oddorder moments of V are zero. Hence, we need to determine only the evenorder moments of V. Now, making use of the identity that
C 00
(1+ x )  2 =
00
=
(;)xn
n=O
C(l)n(n+ n=O
(22.12)
l)Zn,
we derive for m = 0 , 1 , 2 , . . . ,
1, 00
EVZ"
=
=
2Jd
x2m(1
00
e" +
ez)2
dx
e"
x2m(1 + e  q 2 dx
n=O
=
{
2(2m)! C ( 2 k

l)2m  C(2k)2m
}
(22.13)
200
LOGISTIC DISTRIBUTION
where ((s) is the Riernann zeta function defined by
c
k".
C(s) =
(22.14)
k=l
Using now the wellknown facts that ((2) readily have E V 2 = Var V
=2
=
7r2/6 and ((4) =: 7r4/90, we
7r2
((2)
(22.15)
=
3
and
E V 4 = 42 ((4) In the general case when X we readily have
N
77r4 15
(22.16)
= __ .
Lo ( p , a 2 ) ,using the relationship in (22.11),
In particulax, we find from (22.17) that
EX
=p+
*EV IT
(22.18)
=p
and 3a2 + EV2 IT2
E X = p2 + 2pEV 2
V5U
7r
= p2
+ a2.
(22.19)
Central moments: From (22.11), we find for n = 0 , 1 , 2 , . . . ,
=
In pa.rticular, we have
($)
n
EV".
(22.20)
CHARACTERISTIC FUNCTION
201
Also, due to the symmetry of the distribution of X (about p ) , we have /32n1
E (X

E ( X  P ) ~ ~ = 'O
E X )2n1
for n==1 , 2, . . . .
22.4
(22.23)
Shape Characteristics
Let X be a logistic Lo(p, 0 2 )random variable. As noted earlier, the distribution of X is symmetric about p and, consequently, the Pearson coefficient of skewness of X is [see also (22.23)] y1 = 0. In addition, we find from (22.21) and (22.22) the Pearson coefficient of kurtosis of X to be (22.24)
Thus, the logistic distribution is a symmetric leptokurtic distribution. Plots of logistic density function presented in Figure 22.1 (for p = 0 a.nd different values of 0 ) reveal these properties.
22.5
Characteristic Function

For simplicity, let us consider V Lo (0,7r2/3) with pdf and cdf as given in (22.6) and (22.7), respectively. Then, the characteristic function of V is determined to be
f"(t)
= EeitV
dx
=
= = =
1
uit(l
d7~
B(1+ it, 1  i t ) r(i +it) r(i  i t ) .
(22.25)
Comparing the characteristic function of V Lo (0, 7r2/3) in (22.25) with the clmracteristic function r(l  it) of a random variable Vl which has the extreme value distribution H 3 ( 5 ) in (21.7) [see, for example, (21.32)], we readily observe the fact that the logistic Lo (0, 7r2/3) random variable V has the same distribution as the difference V,  V2, where Vl and Vz are i.i.d. random variables with the extreme value distribution H 3 ( 2 ) in (21.7). Note that this result was derived in Section 22.1 using the convolution formula. N
LOGISTIC DISTRIBUTION
202
5
0
5
X Figure 22.1. Plots of logistic density function when p
=0
RELATIONSHIPS WITH OTHER DISTRIBUTIONS
as
203
From (22.25), we readily find the characteristic function of X
N
Lo(p, 02)
Now, making use of the facts that r(Z
+ 1) = z r(z)
and
r(Z)r(lZ) =
n sin(nz)
~
'
we can rewrite the characteristic functions in (22.25) and (22.26) as fv(t)
= 
r(i+ it)r(i
it) = it r(it)r(i it) 7rt sinh(nt) 
nit sin(7rit)

( 22.27)
and (22.28)
Exercise 22.1 From the expression of the characteristic function in (22.28), show that the mean and variance of X are p and a2,respectively.
22.6
Relationships with Other Distributions
For simplicity, let us once again consider V Lo (0, n2/3) with pdf and cdf as given in (22.6) and (22.7), respectively. Then, using the probability integral transformation, we readily observe the relationship N
v 2 log (")1  u
,
(22.29)
where U has the standard uniform U ( 0 , l ) distribution. Since the distribution of V is symmetric about zero, we may consider the folded form of this logistic distribution, termed the half logistic distribution [see, for example, Balakrishnan (1992)l. Specifically, the random variable IVI has the half logistic distribution with pdf and cdf
LOGISTIC DISTRIBUTION
204
Exercise 22.2 For the half logistic distribution defined in (22.30), derive the mean a.nd variance.

Realizing that the half logistic distribution in (22.30) is simply the lefttruncated (a.t zero) distribution of V Lo (0, 7r2/3), we can introduce a general truncated logistic distribution with pdf (22.31) where a and b are the lower and upper points of truncation of the distribution of V , and A = 1/(1 ea) and B = 1/(1 e d ) . Note that the half logistic density function in (22.30) is a special case o f t h e pdf in (22.31) when u = 0 and b = cx) so that A = and B = 1.
+
22.7
+
Decompositions

As already observed, the logistic random variable V Lo (0, 7r2/3) can be represented as the difference Vl  14 of two i.i.d. random variables having the extreme value distribution H ~ ( xin) (21.7). This readily reveals that the logistic random variable is decomposable. Using Euler’s formula on the gamma function given by
S,
00
=
,t
t
1
dt
71!
=
z(z+ l)(z+2)...(z+n)
we have
r(l + it) r(l

it)
=
limnim
=
n 03
+
j=1
+
(1 P ) ( 4 1
14( t / j ) 2 ‘
n”, (22.32)
+
{ ( n 1)!j2
+ t 2 ) . ’ . { ( n+ 1 ) 2 + t 2 } (22.33)
Recalling that 1/(1 t 2 )is the characteristic function of the standard Laplace L ( 0 , l ) distribution [see, for example, (19.5)], we obtain the following decorriposition result from (22.33): (2 2.34)

where V Lo ( 0 , x 2 / 3 ) and Yl,Y2,.. . are i.i.d. standard Laplace L ( 0 ;1) random variables with density function
205
ORDER STATISTICS
22.8
Order Statistics
~ Let Vl,V2,.. . ,V, be i.i.d. Lo (0,7r2/3) random variables, and let V I , < V2,n< . . . < Vn,, denote the corresponding order statistics. Then, the density function of Vr,%(for 1 T n) is given by
< <
pvJz)
=
n! (.  I)! ( n  T ) !
(6) + n r
1
(=>'
m
e"
(1
< 5 < m.
ez)2 '
(22.35)
From (22.35), we derive the characteristic function of Vr,nas
fv,,,(t)
=
EeitKsn
n!



eitx
n! ( T  I)! ( n  T ) ! r(T+ i t ) r ( n 
r(n
r(T)
s',+ T
nr+l
(e") (1+ ex)n+l
ur+zt 1
dx
(1 4nr+if du
1  it) 1)
+
(22.36)
Note that we have fv,,,(t) = f v n ~ y + l , n due (  t )to the symmetry of the logistic distribution and, consequently,
EV& for 1 5 T 5 n and k
=
=
(22.37)
(l)kEvL.+l,n
1,2,. .. .
Exercise 22.3 From the characteristic function of Vr,nin (22.36), derive expressions for the mean and variance of Vr,, for 1 T n.
< <
22.9
Generalized Logistic Distributions
Due to the simple form of the logistic distribution, several generalizations have been proposed in the literature. Four prominent types of generalized logistic densities are as follows: The Type I generalized logistic density function is (22.38)
The Type 11 generalized logistic density function is (22.39)
LOGISTIC DISTRIBUTION
206
T h e Type 111 generalized logistic density f u n c t i o n is
and the Type IV generalized logistic d e n s i t y f u n c t i o n is IV PV
ecbX
qa+b)
=
r(a)r(b)(1 + ez)a+b’ W
< 2 < 0 0 ~ > 0 , b > 0.
(22.41)
It should be mentioned that all these forms are special cases of a very general family proposed by Perks (1932). Exercise 22.4 Show that the characteristic functions of these four generalized logistic distributions are
r(i

i t ) r ( a+ it)
r(i + i t ) r ( a+ i t )
1
J3a)
1
respectively. Exercise 22.5 From the characteristic functions in (22.42), derive the expressions of the moments and discuss the shape characteristics. Exercise 22.6 If V has the Type I generalized logistic density function in (22.381, then prove the following: The distribution is negatively skewed for 0 < a < 1 and positively skewed for a > 1; The distribution of aV behaves like standard exponential E ( 1) when a + 0; The distribution of V  log a behaves like extreme value distribution H s ( 2 ) in (21.7) when a 400;
(d) V has the Type I1 generalized logistic density function in (22.39). Exercise 22.7 Let V have the Type I generalized logistic density function in (22.38). Let Y, given T , have the extreme value density function
re
.x
e
Tez
where r has a gamma distribution with density function
Then, show that the marginal density function of Y is the same as that of V .
GENERALIZED LOGISTIC DISTRIBUTIONS
207
Exercise 22.8 If V has the Type I11 generalized logistic density function in (22.40), then prove the following: (a) The distribution of @V behaves like standard normal distribution when a + 00; (b) If Y1 and Yz are i.i.d. random variables with loggamma density function
then the difference Y1  Y2 is distributed as V; and (c) Let Y1 and Y2 be i.i.d. random variables with the distribution of  log 2,where 2 given T is distributed as gamma I?(&, a b) and T is distributed as beta(a, b). Then, the difference Y1  Y2 has the same distribution as V.
+
Exercise 22.9 If V has the Type IV generalized logistic density function in (22.41), then prove the following: (a) The distribution is negatively skewed for a < b, positively skewed for a > b, and is symmetric for a = b (in this case, it simply becomes Type I11 generalized logistic density function); (b) If V is distributed as beta(a, b ) , then log ( Y / ( l Y ) )is distributed as
and
v;
(c)
V has the Type IV generalized logistic density function in (22.41) with parameters a and b interchanged.
This Page Intentionally Left Blank
CHAPTER 23
NORMAL DISTRIBUTION 23.1
Introduction
In Chapter 5 [see Eq. (5.39)] we considered a sequence of random variables
x,

np
wn=&$Tq,
n = 1,2,...]
where X , has binomial B ( n ,p ) distribution and showed that the distribution of W, converges, as n + 03, to a limiting distribution with characteristic function (23.1)
f ( t )= et2/2.
Similar result was also obtained for some sequences of negative binomial, Poisson, and ga.mma distributions [see Eqs. (7.30), (9.33), and (20.27)]. Since M
s,
If(t)l d t
< 00,
the corresponding limiting distribution has a density function p(x), which is given by the inverse Fourier transform 1 cp(x) = 2.ir
Lm "
eCitZ"ft) d t .
(23.2)
Since f ( t ) in (23.1) is an even function, we can express p(x) in (23.2) as (23.3) Note that p(x) is differentiable a.nd its first derivative is given by
=
x p(x). 209
(23.4)
210
NORMAL DISTRIBUTION
We also have from (23.3) that
(23.5) since r ($) = f i . Upon solving the differential equation in (23.4) using (23.5), we readily obtain
as the pdf corresponding to the characteristic function f(t) = e p f 2 / 2 ,i.e.,
Lm 00
f ( t ) = et2/2 =
eit"cp(x) dx
A booklength account of normal distributions, discussing in great detail
their various properties and applications, is available [Patel and Read (1997)].
23.2
Notations
We say tha,t a random variable X has the standard n o r m a l distribution if its pdf is as given in (23.6), and its cdf is given by (23.7) The linear transformation Y = u nornial random variable with pdf

and cdf
1
$O X
(cc
(x  u)2
< a < co, o > 0) genera.tes a
,
oo
< x < co,
(23.8)
ENTROPY
211
In the sequel, we will use the notation Y N ( a ,0 2 )to denote a random variable Y having the normal distribution with location parameter a and scale parameter cr > 0. Shortly, we will show that a and a2 are, in fact, the mean N ( 0 , l ) will denote that X has and variance of Y , respectively. Then, X the standard normal distribution with pdf and cdf as in (23.6) and (23.7), respectively. The normal density function first appeared in the papers of de Moivre a t the beginning of the eighteenth century as an auxiliary function that approximated binomial probabilities. Some decades later, the normal distribution was given by Gauss and Laplace in the theory of errors and the least squares method, respectively. For this reason, the normal distribution is also sometimes referred t o as Gaussian law, GaussLaplace distribution, Gaussian distribution, and the second law of Laplace. N
N
23.3
Mode
It is easy t o see that normal N ( a ,0') distribution is unimodal. From (23.8), we see that
which when equated to 0, yields the mode to be the location parameter a, and the maximal value of the pdf p ( a , 0 , x) is then readily obtained from (23.8) to be 1/(0&).
23.4
Entropy
The entropy of a normal distribution possesses an interesting property.
Exercise 23.1 Let Y
N
N ( a ,r 2 ) Show . that its entropy H ( Y ) is given by
1 H ( Y )= 2
+ log(aJ2.rr).
(23.10)
It is of interest to mention here that among all distributions with fixed mean a and vxiance 02, the maximal value of the entropy is attained for the normal N ( a ,0 ' ) distribution.
212
NOR,MAL DISTRIBUTION
23.5
Tail Behavior
The normal distribution function has light tails. Let X N ( 0 , l ) . From Table 23.1, which presents values of the standard normal distribution function a(.), we have N
a(l) + 1 2{1 @(a)}
@(1)= 2{1
P{l[l > 1) P{ > 2)
=
P{I 3)
=
., 2{1  @ ( 3 ) }= 0.0027.. . ,
P{I[I > 4)
=
2{1
=




Q(1)) = 0.3173.. . ,
= 0.0455..
@(4)} = 0.000063.. . .
It is easy t o obtain that for any x
> 0,
(23.11) Simila.rly, for any x
> 0,
I
x3
1
d%
p
/
2
(23.12)
Using (23.12), we get the following lower bound for the tail of the normal distribution function: 1 @(x)
=
1
.i,
”
c t 2 l 2d t
(23.13) Hence, the asympt,otic behavior of 1  @(x) is determined by the inequalities
TAIL BEHAVIOR
J:
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20 0.22 0.24 0.26 0.28 0.30 0.32 0.34 0.36 0.38 0.40 0.42 0.44 0.46 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 0.64 0.66 0.68 0.70 0.72 0.74 0.76 0.78 0.80 0.82 0.84 0.86
@.(XI
0.5000 0.5080 0.5160 0.5239 0.5319 0.5398 0.5478 0.5557 0.5636 0.5714 0.5793 0.5871 0.5948 0.6026 0.6103 0.6179 0.6255 0.6331 0.6406 0.6480 0.6554 0.6628 0.6700 0.6772 0.6844 0.6915 0.6985 0.7054 0.7123 0.7190 0.7257 0.7324 0.7389 0.7454 0.7517 0.7580 0.7642 0.7704 0.7764 0.7823 0.7881 0.7939 0.7995 0.8051
x
0.88 0.90 0.92 0.94 0.96 0.98 1.00 1.02 1.04 1.06 1.08 1.10 1.12 1.14 1.16 1.18 1.20 1.22 1.24 1.26 1.28 1.30 1.32 1.34 1.36 1.38 1.40 1.42 1.44 1.46 1.48 1.50 1.52 1.54 1.56 1.58 1.60 1.62 1.64 1.66 1.68 1.70 1.72 1.74
@(X)
0.8106 0.8159 0.8212 0.8264 0.8315 0.8365 0.8413 0.8461 0.8508 0.8554 0.8599 0.8643 0.8686 0.8729 0.8770 0.8810 0.8849 0.8888 0.8925 0.8962 0.8997 0.9032 0.9066 0.9099 0.9131 0.9162 0.9192 0.9222 0.9251 0.9278 0.9306 0.9332 0.9357 0.9382 0.9406 0.9429 0.9452 0.9474 0.9495 0.9515 0.9535 0.9554 0.9573 0.9591
X
2 13
@(XI
1.76 0.9608 1.78 0.9625 1.80 0.9641 1.82 0.9656 1.84 0.9671 1.86 0.9686 1.88 0.9699 1.90 0.9713 1.92 0.9726 1.94 0.9738 1.96 0.9750 1.98 0.9761 2.00 0.9772 2.02 0.9783 2.04 0.9793 2.06 0.9803 2.08 0.9812 2.10 0.9821 2.12 0.9830 2.14 0.9838 2.16 0.9846 2.18 0.9854 2.20 0.9861 2.22 0.9868 2.24 0.9875 2.26 0.9881 2.28 0.9887 2.30 0.9893 2.32 0.9898 2.34 0.9904 2.36 0.9909 2.38 0.9913 2.40 0.9918 2.42 0.9922 2.44 0.9927 2.46 0.9931 2.48 0.9934 2.50 0.9938 2.52 0.9941 2.54 0.9945 2.56 0.9948 2.58 0.9951 2.60 0.9953 2.62 0.9956
X
2.64 2.66 2.68 2.70 2.72 2.74 2.76 2.78 2.80 2.82 2.84 2.86 2.88 2.90 2.92 2.94 2.96 2.98 3.00 3.02 3.04 3.06 3.08 3.10 3.12 3.14 3.16 3.18 3.20 3.22 3.24 3.26 3.28 3.30 3.32 3.34 3.36 3.38 3.40 3.42 3.44 3.46 3.48 3.50
@(x)
0.9959 0.9961 0.9963 0.9965 0.9967 0.9969 0.9971 0.9973 0.9974 0.9976 0.9977 0.9979 0.9980 0.9981 0.9982 0.9984 0.9985 0.9986 0.9987 0.9987 0.9988 0.9989 0.9990 0.9990 0.9991 0.9992 0.9992 0.9993 0.9993 0.9994 0.9994 0.9994 0.9995 0.9995 0.9995 0.9996 0.9996 0.9996 0.9997 0.9997 0.9997 0.9997 0.9997 0.9998
Table 23.1: Standard Normal Cumulative Probabilities
214
NORMAL DISTRIBUTION
which are valid for any positive
2.
It readily follows from (23.14) that
1 @ ( T )
1
N
(23.15)
 cp(.) X
a.s x 4cu, where is the pdf of the standard normal distribution [see (23.6)].
23.6
Characteristic F’unct ion
Frorn (23.1), we have the characteristic function of the standard normal N(O.1) distribution to be
f ( t ) = et2/2.
If Y has the general N ( u ,a 2 )distribution, we can use the relation Y = a+aX, where X N ( 0 ,l),in order to obtain the characteristic function of Y as N
“1
(23.16)
2
We see that fy(t) in (23.16) has the form exp{Pz(t)}, where Pz(t) is a polynomial of degree two. Marcirikiewicz (1939) has shown that if a characteristic function g ( t ) is expressed as exp{P,(t)}, where P,(t) is a polynomial of degree n, then there are only the following two possibilities: (a) n = 1, in which case g ( t ) = eiat (degenerate distribution); (b) n = 2, in which case g ( t ) = exp{iat

a2t2/2}(normal distribution).
In Chapter 12 we presented the definition of stable characteristic functions and stable distributions [see Eq. (12.10)].
Exercise 23.2 Prove that the characteristic function in (23.16) is stable, and hence all normal distributions are stable distributions. Exercise 23.3 Find the characteristic function of independent standard normal random variables.
XY whcri X a.nd Y are
Exercise 23.4 Let X I ,X z , X3, and X 4 be independent standard normal ra.ndom variables. Then, show that the random variable Z = X I X 2 X s X , has the Laplace distribution.
+
MOMENTS
23.7
215
Moments
The exponentially decreasing nature of the pdf in (23.8) entails the existence of all the moments of the normal distribution.
Moments about zero: Let X N ( 0 , l ) . Then, it is easy to see that the pdf p(z) in (23.6) is symmetric about 0, and hence N
= EX2"+l = 0
CYZn+l
(23.17)
for n = 1 , 2 , . . . . In particular, we have from (23.17) that a1
=EX =0
(23.18)
and (23.19) Next, the moments of even order are obtained as follows:

(an)! 2" n!'
~
n = l , 2 , ....
(23.20)
In part,icular, we obtain from (23.20) that
EX2=1
(23.21)
= E X 4 = 3.
(23.22)
a2 =
and
In general, if Y the formula
EY"
= E(a
=
a
+ OX
+ax)"=
N
N ( a , a 2 ) ,we can obtain its moments using
n.
a'a"'EX"', r=O
n = 1,2,.. .,
(23.23)
NORMAL DISTRIBUTION
216 which immediately yields
(23.24)
EY=u and
E Y 2 = a2
+ u2.
(23.25)


Central moments: Let X N ( 0 , l), V N ( 0 , u 2 ) ,and Y N ( a ,a2). Then, we have the following relations between central moments of these randoni variables: N
E ( Y  EY)"
= =
E ( V  EV)" = u " E ( X  E X ) " u"EX" = u r b a n , n = 1 , 2, . . . ,
(23.26)
where an are as given in (23.17) and (23.20). In particular, we have Var Y
= u2
Var
x = u2 a2 = u 2 .
(23.27)
We have thus shown that the location paranieter a and the scale parameter a2 of riornial N ( a ,a 2 )distribution are simply the mean and variance of the distribution, respectively.
Cumulants: In situations when the logarithm of a characteristic function is simpler to deal with than the characteristic function itself, it is convenient to use the cuniulants. If f ( t ) is the characteristic function of the random variable X , then the cumulant 7 k of degree k is defined as follows: (23.28) In particular, we have 71
=
72
=
73
=
(23.29)
EX, Var X , E(XEX)3

(23.30) (23.31)
If the moment E X k exists, then all cumulants 71,72, . . . ,7 k also exist. Let us now consider Y N ( a ,a 2 ) .Since its characteristic function ha.s the form f y ( t ) = exp
{
iat
(t)
,

we have log f y ( t )= iat
F}
fy
a2tl
.
2 Hence, we find y1 = a , 7 2 = a ' , and yk = 0 for Ic = 3 , 4 , . . . . Moreover, recalling the Marcinkiewicz's result for characteristic functions mentioned earlier, we obtain the following result: If for some n all cumulants 7 k ( k = n,n+ 1.. . .) of a random variable X are equal to zero, then X has either a degenerate or a norriial distribution. ~
~
CONVOLUTIONS AND DECOMPOSITIONS
Shape Characteristics
23.8 Let
Y
N
217
N ( a , ( s 2 ) .Then, from (23.26), we readily have
E (Y 
= 0 ~ ~ x=3 0,
(23.32)
from which we immediately obtain the Pearson coefficient of skewness to be 0. Next, we have from (23.26) and (23.22) that
E ( Y  EY)4= 04a4 = 304 ,
(23.33)
from which we immediately obtain the Pearson coefficient of kurtosis to be 3 . Thus, we have the normal distributions to be symmetric, unimodal, bellshaped, and mesokurtic distributions. Plots of normal density function presented in Figure 23.1 (for a = 0 and different values of 0 ) reveal these properties.
Remark 23.1 Recalling now that (see Section 1.4) distributions with coefficient of kurtosis smaller than 3 are classified as platykurtic (lighttailed) and those with larger than 3 are classified as leptokurtic (heavytailed), we simply realize that a distribution is considered to be lighttailed or heavytailed relative t o the normal distribution.
23.9
Convolutions and Decompositions
+
In order to find the distribution of the sum Y = Yl Y2 of two independent random variables Yk N ( a k , o;),k = 1,2, we must recall from (23.16) that characteristic functions f k ( t ) of these random variables have the form N
and hence
where a = a1 normal N ( u l also valid.
+ a2 and o2 = a? + 0.22.This immediately implies that Y has + a2, 0::+ 0,”)distribution. Of course, a more general result is
+
Exercise 23.5 For any n = 1 , 2 , . . ., the sum Yl Y2 + . . . + Y, of independent random variables Yk N ( a k ,o;),k = 1 , 2 , . . . ,n, has normal N ( a ,c2) distribution with mean a = a1 + . . . + a, and variance o2 = of . . . 0:. N
+ +
NORMAL DISTRIBUTION
218
   . N(0.4)
I
I
I
I
I
4
2
0
2
4
X
Figure 23.1. Plots of normal density function when a
=0
CONDITIONAL DISTRIBUTIONS
219
We can now state that any normal N ( a ,u 2 )distribution is decomposable, because it can be presented as a convolution of two normal N ( a l , a ! ) and N ( u  a l , a 2  a!) distributions, where CT! < a2 and a1 may take on any value. In fact, Cram& (1936) has proved that only "normal" decompositions are possible for normal distributions, viz., if Y N ( u , a 2 ) ,cc < a < 00, a2 > 0, and Y = Y1 Y2, where Y1 and Y2 are independent nondegenerate random variables, then there exist cc < a1 < cc and 0 < af < u2 such that Yl N ( a l , a : ) and yZ N ( u  u1,a2  a:). Thus, the family of normal distributions is closed with respect to the operations of convolution and decomposition. Recall that families of binomial and Poisson distributions also possess the same property. Furthermore, the characteristic function of normal N ( a ,a 2 ) distribution which is


+

{
f ( t ) = exp iat

can be presented in the form
T}
where
is also the characteristic function of a normal distribution. Thus, we have established that any normal distribution is infinitely divisible.
23.10
Conditional Distributions

Consider independent random variables X N ( 0 , l ) and Y N ( 0 , l ) . Then, V = X Y has normal N(O,2) distribution. Probability density functions of X , Y , and V are given by
+
N
(23.34) and
(23.35) from which we can find the conditional distribution of X given V conditional pdf p X l v ( z ( w ) is given by
u. The
(23.36)
NORMAL DISTRIBUTION
220
Observing that the RHS of (23.36) is the pdf of normal N (z3/2, l / 2 ) distribution, we conclude that the conditional distribution of X , given V = w, is normal with mean w/2 and variance 1/2. A similar result is valid in the general ca.se too.
Exercise 23.6 Find the conditional pdf p x , v ( z l v )for the case when X N ( u l , o f ) ,Y N N(a2,a5),with X and Y being independent, arid V = X + Y .
N
23.11
Independence of Linear Combinations

As noted earlier (see Exercise 23.3), the sum mally distributed random variables X I , N ( u k ,.;), normal
Xk of independent nork = 1 , 2 , . . . , n,,also has
distribution. The following genera.1result can also be proved similarly.
Exercise 23.7 For any coefficients b l , . . . , bn, prove that, the linear combination n k=l
of independent normally distributed random variables XI, 1 , 2 , . . . , n, also has normal / n
n
\
\k=l
k=l
1
N
N ( a k ,o f ) , k =
distribution.
Let us now consider two different linear combinations n
n
k=l
k=l
arid find the restriction on the coefficients b l , . . . , b,, e l , . . . , c,,, and parameters a l , . . . , a, and a:, . . . , a : , which provides the independence of L1 and La. It, is clear that the location parameters a l , . . . , an cannot influence this independence, so we will suppose that uk = 0 for k = 1 , 2 , . . . , n without loss
BERNSTEIN’S THEOREM
221
of any generality. Then, the linear forms L1 and L2 are independent if and only if EeiuLl+iwLz
holds for any real u and v. Using the fact that X k that

EeiUL1EeiVL2
N ( 0 ,o:), k
N
fk(t) = E e i t X k
=
(23.37)
1 , 2 , . . . , are independent, and
= et2l2
we get the following expression for the joint characteristic function of L1 and
L2;
f
(%V)

~ ~ i u L l + i v Lz ~~i
x:=,(bkU+Clcv)Xk
n
k=l
(23.38) Also, the characteristic functions of L1 and La are given by (23.39) and
(23.40) Equations (23.38)(23.40) then imply that the condition for independence of L1 and L2 in (23.37) holds iff n
(23.41) k=l
23.12
Bernstein’s Theorem
If we take a pair of i.i.d. normal random variables X and Y and construct linear combinations L1 = X Y and L2 = X  Y , then the condition (23.41) is certainly valid and so L1 and L2 are independent. Bernstein (1941) proved the converse of this result.
+
222
NORMAL DISTRIBUTION
+
Theorem 23.1 Let X and Y be i.i.d. random variables, and let L1 = X Y and L2 = X  Y a l s o be independent. Then, X and Y have either degenerate or normal distribution. We will present here briefly the main arguments used t o prove this theorem. (1) Indeed, the statement of the theorem is valid if X and Y have degenerate distribution. Hence, we will focus only on the nontrivial situation wherein X and Y have a nondegenerate distribution.
(2) Without loss of generality, we can suppose that X and Y are symmetric random variables with a common nonnegative real characteristic function f ( t ) . This is so because if X and Y have any characteristic function y ( t ) , we can produce symmetric random variables V = X  XI and U = Y  Yl, where X I and Yl have the same distribution as X and Y , respectively, and the random variables X , X 1 , Y , and Y1 are all independent. This symmetrization procedure gives us new independent ra.ndom variables with the common characteristic function f ( t ) = g ( t ) y (  t ) = 1g(t)I2, which is real and nonnegative. Due to the conditions of the theorem, X Y and X  Y as well as X1 Yl and XI Yl are independent, and so X U and X  U are also independent. Thus, the random variables V and U with a common real nonnegative characteristic function satisfy the conditions of the theorem. Suppose now that we have proved that V and U are normally distributed random variables. Sirice V is the sum of the initial independent random variables X and Y , we can apply CramWs result on decompositions of normal distributions stated earlier and obtain immediately that X and Y are also normally distributed random variables.
+
+
+
(3) Since X and Y have a. common real characteristic function f ( t ) , we have the cha,ract,eristicfunctions of L1 and L2 as
f l ( u )= Ee
ZUL1
 Eei7LXEeiUY 

f2(4
(23.42)
arid fi(v) = Ee i u L 2  E ~ ~ . u X E ~  ~= U fY (
~ ) f (  v= ) fz(t/).
We also have the joint characteristic function of
L1
(23.43)
and Lz as
(23.44)
(23.45)
BERNSTEIN’S THEOREM
(4) Taking u = nt and
2)
=t
223
in (23.45), we get
+
f [ ( n l)t]f[(n l ) t ]= f 2 ( n t ) f 2 ( t ) ,
n
=
1 , 2 , .. .
.
(23.46)
In particular, we have f ( 2 t ) = f4(Q
(23.47)
It follows immediately from (23.47) that f ( t ) # 0 for any t. Also, since f(0) = 1 and f ( t ) is a continuous function, there is a value a > 0 such that f ( t ) # 0 if ltl < a. Then, f(2t) # 0 if It1 < a, which means that f ( t ) # 0 for all t in the interval (2a,2a). Proceeding this way, for any n = 1 , 2 , . . . , we obtain that f ( t ) # 0 if jtl < 2na, and hence the nonnegative function f ( t ) must be strictly positive for any t. Now, from the equality [see (23.46) and (23.47)], f ( 3 t ) f ( t ) = f 2 ( 2 t ) f 2 ( t )= {f4(t)Kf2(t) = flO(t), we get the relation
f(3t) = f9(t).
(23.48)
Following this procedure, we obtain
f(m t ) = f r n 2 ( t ) which is true for any m
=
(23.49)
1 , 2 , . . . and any t.
(5) The standard technique now allows us t o get from (23.49) that (23.50) and (23.51) for any integers m and n, where c = f ( l ) ,0 < c 5 1. Since f ( t )is a continuous function, (23.51) holds true for any positive t:
f ( t ) = ct 2 .
(23.52 )
Since any real characteristic function is even, (23.52) holds for any t.
If c = 1, we obtain f ( t ) = 1 (the characteristic function of the degenerate distribution). If 0 < c < 1, then
f ( t )= e 2 t 2 / 2 , where u2 = 2 log c > 0, and the theorem is thus proved.
224
NORMAL DISTRIBUTION
DarmoisSkitovitch’s Theorem
23.13
In Bernstein’s theorem, we considered the simplest linear combinations L1 = X f Y and L2 = X  Y . Darmois (1951) and Skitovitch (1954) independently proved the following more general result. Theorem 23.2 L e t X I , (k = 1 , 2 , . . . , n ) be i n d e p e n d e n t nondegenerate ran
d o m variables, a n d let
where bl; and c k (k = 1 , 2 , . . . , n) are n o n z e r o real coeficients. I . L1 and L2 are independent, t h e n t h e r a n d o m variables X I , . . . , X , all have n o r m a l distributions. A very interesting corollary of Theorem 23.2 is the following. Let 
L1=X= We see t1ia.t
XI+...+X, n
and
L2
= X1

X.
n
k=l
k=l
where
c1 Ck
= =
1 1,
n
1
 
n
,
k = 2 , 3 , . . . ,n.
(23.53)
It then follows from DarmoisSkitovitch’s theorem that independence of L1 and L2 implies that X’s are all normally distributed. If X I , N ( Q , a;), k = 1 , 2 , . . . , n,then (23.41) shows that for the coefficients in (23.53), linear combinations L1 and L2 are independent if and only if


(23.54)
For example, if X I , N ( Q , a 2 ) ,k = 1 , 2 , . . . , n, then 0%= cr2 ( k = 1 , 2 , .. . ,n ) , in which case (23.54) holds, and so linear combinations L1 and L2 are independent. More generally, in fact, for independent normal random variables Xi; N ( a k , a2),k = 1 , 2 , . . . , n, the random vector ( X I  X , X2  X , . . . , X ,  X ) and X = ( X I . . . X , ) / n are independent. It is clear that in order t o prove this result,, we can take (without loss of any generality) X’s t o be standard

+ +
DARMOISSKITOVITCH’S THEOREM
225
normal N ( 0 , l ) random variables. Let us now consider the joint characteristic function of the random variables X , X1  X , X2  X , . . . , X n  X :
n
=
t  (tl
E e x p { ik=l xXk (tk+
+ + . .+ tn) t2
’
n
(23.55) Since X’s in (23.55) are independent random variables having a common characteristic function f ( t )= et2/2, we obtain
t  (tl
t2
n
k=l n
t  (tl
+ t 2 + . . . + tn) n
k=l
n
=
+ + .. .+ tn)
e x p {  ; qk=l t,+
t  (tl + t 2 + . . . + tn) n
(23.56) We can rewrite (23.56) as
where (23.58)
Equation (23.57) then readily implies the independence of X and the random vector ( X I  X , X2  X,. . . , X n  X ) . Furthermore, X and any random variable T ( X 1  X , . . . , X , are also independent. For example, if X I , X2, . . . , X n is a random sample from normal N ( a ,0 2 )distribution, then we can conclude that the sample mean X and the sample variance
x)
(23.60) are independent. Note that the converse is also true: If X I ,X 2 , . . . , X n is a random sample from a distribution and that X and S 2 are independent, then X’s are all normally distributed.
NORMAL DISTRIBUTION
226 Now, let
XI,^ 5 X2,n 5 . . . 5 Xn,n
be the order statistics obtained from the random sample XI, X2,. . . , X,. Then, the random vector XI.^  X ,. . . , X,,,,  X ) and the sample mean X are also independent. This immediately implies, for example, that the sample range Xn,n

XI,,
== (Xn,n  X )
(X1,n  X )
and the sample mean X are independent for samples from normal distribution.
23.14
Helmert’s Transformation
Helmert (1876) used direct transformation of variables to prove, when XI, . . . , X, is a random sample from normal N ( a , a 2 )distribution, the results that X and S2 are independent and also that ( n  1)S2/a2has a chisquare distribution with n  1 degrees of freedom. Helmert first showed that if YI, = XI,  X ( k = 1,.. . ,n ) , then the joint density function of Yl, . . . , Y,l (with Y, = Yl

. . .  Y,1)
and
X
is proportional t o
thus establishing the independence of X and any function of X1  X ,. . . , Xn X,including S2. In order to derive the distribution of S2, Helmert (1876) introduced the transformation
...
Then, froni the joint density function of Y1, . . . , Y,l
fi
(&)
nl
exp
{

1 (YH
given by
}
+ .. . + Y 3 ,
we obtain the joint density function of wl,. . . , wnl as
IDENTITY OF DISTRIBUTIONS O F LINEAR COMBINATIONS
227
Since this is the joint density function of n  1 independent normal N ( 0 ,u 2 ) random variables, and that 1
n
1 ” C y; = y C(.k a
,.EWE=,. nl
k= 1
k=l
k=l

2)2 =
( n 1)s’ 02
’
Helmert concluded that the variable ( n 1)S2/a2 has a chisquare distribution with n  1 degrees of freedom. The elegant transformation above given is referred to in the literature as Helmert’s transformation.
23.15
Identity of Distributions of Linear Combinations
Once again, let X I , variables, and
N
N ( a k , a z ) , Ic = 1 ’ 2 , . .. ,n, be independent random n k=l
k=l
It then follows from (23.39) and (23.40) that these linear combinations have the same distribution if n
n
bkak = k=l
k=l
n,
n
(23.61)
Ckak
and (23.62) If X ’ s have a common standard normal distribution, then the condition in (23.62) is equivalent to n
n
k= 1
k= 1
(23.63) For example, in this situation, (23.64) for any integers m and n and, in particular, (23.65) P6lya (1932) showed that if X1 and X2 are independent and identically distributed nondegenerate random variables having finite variances, then the equality in distribution of the random variables X1 and ( X I X z ) / f i characterizes
+
NORMAL DISTRIBUTION
228
the normal distribution. Marcinkiewicz (1939) later proved, under some restrictions on the coefficients bk and ck ( k = 1,.. . , n ) , that if X I , .. . , x, are independent random variables having a common distribution and finite momerits of a.11order, then
k=l
k=l
implies that X ' s all have normal distribution.
23.16
Asymptotic Relations
As already seen in Chapters 5, 7, 9, and 20, the normal distribution arises naturally a.s a limiting distribution of some sequences of binomial, negative binomial, Poisson, and gamma distributed random variables. A careful look at these situations reveals that the normal distribution has appeared there as a limiting distribution for suitably normalized sums of independent random variables. There are several modifications to the central limit theorem, which provide (under different restrictions on the random variables XI, X 2 , . . .) the convergence of sums S,  ES,
~TGzz'
+
+
where S, = X I . . . X,, to the normal distribution. This is the reason why the normal distributions plays an important role in probability theory and mat hematical statistics. Cha.nging the sum S, by the maxima Ad, = max(X1, X 2 , . . . , X n } , n = 1 , 2 , . . ., we get a different limiting scheme with the extreme value distributions (see Chapter 21) determining the asymptotic behavior of the normalized random variable Adn. It should be mentioned here that if X I , N ( 0 , I ) , k = 1 , 2 , . . ., are independent random variables, then

(23.66) as n
4 00, where
a,
=
42

+
log log n log 47r [email protected]
(23.67)
and (23.68)
TRANSFORMATIONS
23.17
229
Transformations
Consider two independent random variables R and 4,where 4 has the uniform U(0127r)distribution and R has its pdf as pR(r)= r
r
er2/2,
2o .
(23.69)
Note that
P { R < r ) = 1 e  r 2 I 2 ,
r
2 0,
(23.70)
which means that R has the Rayleigh distribution [see (21.12)], which is a special case of the Weibull distribution. Moreover, R can be expressed as (23.71)
R=J2X,
where X has the standard exponential E(l) distribution. Then, the joint density function of R and q5 is given by 1
r 2 0, o 5 cp 5 27r.
pR,a(r,p)=  r e  ~ ’ / ~ , 27r
(23.72)
Let us now consider two new random variables V = R sin 4 and W = R cos 4. In fact, R and q5 are the polar coordinates of the random point (V,W ) .Since the Jacobian of the polar transformation (TJ = r sin cp, w = r cos cp) equals T , we readily obtain the joint pdf of V and W as (23.73) Equation (23.73) implies that the random variables V and W are independent a.nd both have standard normal distribution. This result, called BoxMuller ’s transformation [Box and Muller (1958)], shows that we have the following representation for a pair of independent random variables having a common standa.rd normal N ( 0 , l ) distribution:
(v,W ) 5
( J 2 X s i n 2 7 r ~ ,d E c o s 2 7 r ~ ) ,
(23.74)
where X and U are independent, X has the standard exponential E(1) distribution, and U has the standard uniform U ( 0 , l ) distribution. We can obtain some interesting corollaries from (23.74). For example,
V2fW2 2
=x d
(23.75)
and so
V
VVTTF’d m 2
=
(sin 27rU, cos 27rU).
(2 3.76)
We see from (23.75) and (23.76) that the vector on the lefthand side of (23.76) does not depend on V 2 W 2and it has the uniform distribution on the unit circle. From (23.76), we also have
+
Z
=
v d

W
=
tan27rU.
(23.77)
NORMAL DISTRIBUTION
230 It is easy to see that
d
tan2rrU = t a n r a.nd so
z =d
ta.nr ( U 
t)
.
(23.78)
Comparing (23.78) with (12.5), we readily see that Z has the standard Cauchy C ( 0 , l ) distribution. Next, let us consider the random variables Yl =
2vw
VVTw
and
Y2 =
w2 v2 
vPTiv‘
(23.79)
Shepp (1964) proved that Yl and Y2 are independent random variables having standard normal distribution. This result can easily be checked by using (23.74). We have
(Y~,
d
(2m sin 27ru cos 2 7 r ~ ,2J2X(cos2 27ru  sin2 27ru)
5
(msin47r~m , c o s 4 7 r ~ .j
(23.80)
Taking into account that (sin 4rU, cos 47rU) and hence
2
( ~ 1 ~, 2 )
d
=
(sin 27rU, cos 27rU)
(asin 2 7 r ~ ,
cos 27ru) ,
we immediately obt,ain from (23.74) that
which proves Shepp’s (1964) result. Bansal et al. (1999) proved the following converse of Shepp’s result. Let V and W be independent and identically distributed random variables, and let Y1 and Y2 be as defined in (23.79). If there exist real u and b with u2 b2 = 1 such that aY1 bY2 has the standard normal distribution, then V and W are standard normal N ( 0 , l ) random variables. Now, summarizing the results given above, we have the following characterization of the normal distribution: “Let V and W be independent and identically distributed random variables. Then, V N(O,l) and W N ( 0 , l ) iff Yl has the standard normal distribution. The same is valid for Y2 as well.” Conling back to (23.75), we see that X = ( V 2 W 2 ) / 2has the standard exponential E ( 1) distribution with characteristic function (1 it)’. Since X is a sum of two independent and identically distributed random variables V2/2
+
+


+
TRANSFORMATIONS
231
and W 2 / 2 ,we obtain the characteristic function of V2/2 t o be (1 which corresponds to r 0 , l ) with pdf
(a,
1
e”,
6
x >O.
Then, the squared standard normal variable V 2 has
&
e+,
r (i,0,2) with pdf
x > 0.
If we now consider the sum (23.81) k=l
where Vl, V2,. . . ,Vn are i.i.d. normal N ( 0 , l ) random variables, then the reproductive property of gamma distributions readily reveals that Sn has I‘ (n/2,0,2) distribution. In Chapter 20 we mentioned that this special case of r (n/2,0,2) distributions, where n is an integer, is called a chisquare ( x 2 ) distribution with n degrees of freedom. Based on (20.24), the following result is then valid: For any k = 1 , 2 , . . . , n, the random variables
and Sn are independent, and Tk,n has the standard beta ution. In particular, if n = 2, T1,2 =
v?
v,”+ vz”
and
T2,2 =
(i, 2 ’) distrib

vz” v,2 + vz”
have the standard arcsine distribution.
Exercise 23.8 Show that the quotient (V,” form U ( 0 , l ) distribution.
+ K2)/Sq has the standard uni
Exercise 23.9 Show that the pdf of (23.82) is given by (23.83)
NORMAL DISTRIBUTION
232
The distribution with pdf (23.83) is called as chi distribution (x distribution) with n degrees of freedom. Note that (23.83), when n = 2, corresponds to the Rayleigh density in (23.69). The case when n = 3 with pdf
is called the standard Maxwell distribution. Consider now the quotient
T=
d
V
m
=
V
m
l
(23.85)
where V, V l , . . . , V, are all independent random variables having standard normal distribution. The numerator and denominator of (23.85) are independent arid have normal distribution and chi distribution with n degrees of freedom, respect>ively.
Exercise 23.10 Show tha.t the pdf of T is given by
The distribution with pdf (23.86) is called Student’s t distribution with n degrees of freedom. Note that Student’s t distribution with one degree of freedom (i.e.] case n = 1) is just the sta.ndard Cauchy distribution.
Exercise 23.11 As n + 30, show that the t density in (23.86) converges to the standard normal density function.
Now, let Y = 1/V2 be the reciprocal of a squared standa.rd normal N ( 0 , l ) random variablc. Then, it can be shown that the pdf of Y is
( 23.87) It can also he shown that the characteristic function of Y is given by
(23.88)
TRANSFORMATIONS
233
It is not difficult t o check that f y ( t ) is a stable characteristic function, and the random variable with pdf (23.87) has a stable distribution. It is of interest to mention here that there are only three stable distributions that possess pdf's in a simple analytical form: normal, Cauchy, and (23.87). Of course, Y as well as any other stable random variable has an infinitely divisible distribution. We may observe that many distributions related to the normal distribution are infinitely divisible. However, in order to show that not all distributions related closely to normal are infinitely divisible or even decomposable, we will give the following example. It is easy to check that if a random variable X with a characteristic function f ( t ) and a pdf p(x) has a finite second moment then f*(t)= f ( 2 ) ( t ) is indeed a characteristic function which corresponds .f (2) (0) to the pdf a2,
Now, let X

p*(x)
X2P(X) =ff2
N ( 0 , l ) . Then,
consequently, (23.89)
f * ( t ) = (1 t 2 ) e  t 2 / 2 is the characteristic function of a random variable with pdf
(23.90) The characteristic function in (23.89) and hence the distribution in (23.90) are indecomposable. Let X N(0,l) and X = a+ blog Y . Then, Y is said t o have a lognormal distribution with parameters a and b. By a simple transformation of random variables, we find the pdf of Y as

b
(23.91)
We may take b to be positive without loss of any generality, since  X has the same distribution as X . An alternative reparametrization is obtained by replacing the parameters a and b by the mean m and standard deviation 0 of the random va.riable l o g y . Then, the two sets of parameters satisfy the relationships 7n =
U
 
b
and
1 b'
(23.92)
L T =
so that we have X = (log Y  m ) / a ,and the lognormal pdf under this reparametrization is given by 1
1 (logy  m)2 2 02
>
Y>O.
(23.93)
234
NORMAL DISTRIBUTION
The logiiormal distribution is also sometimes called the CobbDouglas distribution in the economics literature.
Exercise 23.12 Using the relationship X = ( l o g y  m ) / a , where X is a standard normal variable, show that, the kth moment of Y is given by
E ( Y k )= E (e k ( r n + o X ) )
(23.94)
= ekm+Bk2a2
Then, deduce that
EY where w
= em
&
and
Var Y = eZmu(u  l),
= euL.
Lognormal distributions possess many interesting properties and have also found important applications in diverse fields. A detailed account of these developments on lognormal distributions can be found in the books of Aitchison and Brown (1957) and Crow and Shimizu (1988). Along the same lines, Johnson (1949) considered the following transformations:
X
=a
+ blog (L) and 1Y
X
=a
+ bsinhI
Y,
(23.95)
where X is once again distributed as standard normal. The distributions of Y in these two ca.ses are called Johnson's Sg and Su distributions, respectively. These distributions have been studied rather extensively in both the statistical and applied literature .
Exercise 23.13 By transformation of variables, derive the densities of Johnson's Sn and S" distributions. Exercise 23.14 For the Su distribution, show that the mean and variance are
(23.96) respectively.
CHAPTER 24
MISCELLANEA 24.1
Introduction
In the preceding thirteen chapters, we have described some of the most important and fundamental continuous distributions. In this chapter we describe briefly some more continuous distributions which play an important rolc either in statistical inferential problems or in applications.
24.2
Linnik Distribution
In Chapter 19 we found that a random variable V with pdf 1 p v ( x ) =  e?, 2
m
< x < m,
has as its characteristic function [see Eq. (19.5)] (24.1) Such a random variable V can be represented as
vgaw,
(24.2)
where X and W are independent random variables with X having the standard exponential E ( 1) distribution and W having the normal N(O,2) distribution with characteristic function
gw(t) = EeitW
(24.3)
= et2.
This result can be established through the use of characteristic functions as follows:
fv(t)
=
Ee itV

EeitJSTW
eXEeitfiW
235
dz
236
MISCELLANEA =
1,
e"gw(t&) dx
(24.4)
a
Thus, from (24.4), we observe that V = W has the standard Laplace L( 1) distribution. Similarly, let us consider the random variable
Y
=xz>
where X and Z are independent random variables with X once again having the standard exponential E(1)distribution and 2 having the standard Cauchy C(0,l) distribution with characteristic function gz(t) =
(24.5)
= elti.
Exercise 24.1 Show that the characteristic function of Y 1 f Y ( t ) = EeitY = 1 It1 .
=X
Z is
+
(24.6)
Note that in the two examples above, W N(O,2) and Z C(0,l) have symmetric stable distributions with characteristic functions of the form exp(ltla), where (Y = 2 in the first case and (Y = 1 in the second case. Now, more generally, starting with a stable random variable W ( a )(for 0 < Q 5 2) with characteristic function N
N
let us consider the random variable
Y(a!)= X""W(a),
(24.8)
where X a.nd W ( a ) are independent random variables with X once again having the standard exponential E ( 1) distribution. Then, we readily obtain the chara.cteristic function of Y ( a )as
(24.9)
INVERSE GAUSSIAN DISTRIBUTION
237
Thus, we have established in (24.9) that (24.10) (for any 0 < Q 5 2) is indeed the characteristic function of a random variable Y ( a )which is as defined in (24.8). The distribution with characteristic function (24.10) is called the Linnik distribution, since Linnik (1953, 1963) was the first t o prove that the RHS of (24.10) is a characteristic function for any 0 < Q 5 2. But this simple proof of the result presented here is due t o Devroye (1990). In addition, from (24.8), we immediately have (24.11)
E {Y(a))' = EXe/" E { W ( Q ) } ' . Since
exists for any C > 0, the existence of the moment E {Y(a))' is determined by the existence of the moment E { W ( Q )e .} Hence, we have
{ ly(a)le}< 02 if and only if 0 < !< Q = 2.
24.3
Q
(24.12)
< 2 , a.nd that (24.12) is true for any !> 0 when
Inverse Gaussian Distribution
The twoparameter inverse Gaussian distribution, denoted by I G ( p ,A), has its pdf as
p x ( z ) = /=exp
271.x3
{

~
A
2P
2
( p) z
and the corresponding cdf as
Fx(z)
=
a)
(E(;

1))
+
,
z
> 0, A, p > 0, (24.13)
(8 (; +
1))
z
> 0.
, (24.14)
The characteristic function of IG (p ,A) can be shown t o be k.
(24.15)
238
MISCELLANEA
Exercise 24.2 From the characteristic function in (24.15), show that E X p and Var X = p3/X.
=
Exercise 24.3 Show that Pearson coefficients of skewness and kurtosis are given by
respectively, thus revealing that IG(p,A) distributions are positively skewed and leptokurtic. Note that these distributions are represented by the line 7 2 = 3 t 5$/3 in the ($, y2)plaiie.
By taking X = p2 in (24.13), we obtain the oneparameter inverse Gaussian distribution with pdf
,
x > 0,p > 0,
(24.16)
denoted by I G ( p ,p’). Another oneparameter inverse Gaussian distribution may be derived from (24.13) by let,ting p + 00. This results in the pdf
Exercise 24.4 Suppose X I ,X 2 , . . . ,X , are independent inverse Gaussian random variables with X , distributed as IG(p,, A,). Then, using the characteristic function in (24.15), show that X,X,/p: is distributed as I G ( p ,p 2 ) , where p = C:=l A z / p z . Show then, when p, = p and A, = X for all i = 1 . 2 . . . . , n, that the sample mean X is distributed as I G ( p ,nA).
x;&
Inverse Gaussian distributions have many properties analogous to those of normal distributions. Hence, considerable attention has been paid in the literature t o inferential procedures for inverse Gaussian distributions as well as their applications. For a detailed discussion on thew developments, one may refer to the books by C h h i h r a and Folks (1989) and Seshadri (1993, 1998).
CHISQUARE DISTRIBUTION
24.4
239
Chisquare Distribution
In Chapter 20 (see, for example, Section 20.2), we made a passing remark that the specia.1case of gamma r (n/2,0,2) distribution (where n is a positive integer) is called the chisquare (x2) distribution with n degrees of freedom. We shall denote the corresponding random variable by x;. Then, from (20.6), we have its density function as 1 (n/2)
Pxz,(.) =
e2/2
x(n/2)1
,
o<x<m.
From (20.9), we also have the characteristic function of
(24.17)
xi as
fxz, ( t )= (1 2it)
(24.18)
From (20.15) and (20.17), we have the mean and variance of
Ex: = n
and
Var
x:
xn2 = 2n.
as (24.19)
Furthermore, from (20.18) and (20.19), we have the coefficients of skewness and kurtosis of as
xi
(24.20) Also, as shown in Chapter 20, the limiting distribution of the sequence (x:  n ) / G is standard normal. Next, let and x i be two independent chisquare random variables, and x i . Then, from (24.18), we obtain the characteristic function let x2 = of x2 as
xi +
fX*(t) = Eeitx2 = EeitXz EeitXi
=
4(nfm)/2
(1  2.
(24.21)
which readily implies that x2 has a chisquare distribution with (n+m)degrees of freedom. On the other hand, if x i and X are independent random variables with X having an arbitrary distribution, and if x2 = X is distributed as chisquare with ( n m ) degrees of freedom (where m is a positive integer), then the characteristic function of X is
xi +
+
xk.
which implies that X is necessarily distributed as Let X I , . . . , X,n be independent standard normal N ( 0 , l ) random variables. Then, as noted in Chapter 23 [see, for example, Eq. (23.81)],
k= 1
follows a chisquare distribution with n degrees of freedom. More generally, the following result can be established.
240
MISCELLANEA
Exercise 24.5 Let Y l ,. . . , Y, be a random sample from the normal N ( a ,g2) Yk denote the sample mean. Then, show that distribution, and = CE==, [see Eq. (23.60)]
v
( 24.23)
It is because of this fact that chisquare distributions play a very important role in statistical inferential problems. A booklength account of chisquare distributions, discussing in great detail their various properties and applications, is available [Lancaster (1969)l.
24.5
t Distribution
Let X , X I , . . . , X , be i.i.d. random variables having standard normal N ( 0 , l ) distribut,ion. Then, consider the random variable [see also Eq. (23.85)] (24.24) where the numerator and denominator are independent with the numerator having a standard normal distribution and S, having a chisquare distribution with n degrees of freedom. Then, as given in Exercise 23.8, the pdf of this random variable is given by [see Eq. (23.86)]
which is called Student’s t distribution with n degrees of freedom. Let us denote this distribution by t,. This is a special form of Karl Pearson’s Type VII distribution. Since “Student” (1908) was the first to obtain this result, it is called Student’s distribution. But sometimes this distribution is called Fisher’s distribution. More generally, the following result can be established. Exercise 24.6 Let Yl,. . . , Y, be a random sample from the normal N ( a ,g 2 ) distribution, and = C;=,Y k / n denote the sample mean. Then, with S2 as defined in (24.23), show that
t DISTRIBUTION
24 1
It is for this reason that t distributions play a very important role in statistical inferential problems. d From the density function of X = t, in (24.25), it can be shown that the r t h moment of X is finite only for r < n. Since the density function is symmetric about IC = 0, all odd moments of X are zero. If r is even, it can be shown that the r t h moment of X (which are also central moments) is given by

~ _ _ _ _ _
n'/2
1 . 3 . . . ( r  1)
( n  r ) ( n r
+ 2 ) . . . (n

2)
( 24.2 7)
~
Exercise 24.7 Derive the formula in (24.27).
From the expressions above, we readily obtain the mean, variance, and d coefficients of skewness and kurtosis of X = t, as
EX n(X)
=
0,
=
0
n a), n2 3 ( n  2) and y2(X) = ( n > 4). n4 Var X
= (n, >
(24.28)
It is evident that the t distributions are symmetric, unimodal, bellshaped and leptokurtic distributions. In addition, as mentioned earlier in Exercise 23.9, t , distributions converge in limit (as n + m) to the standard normal distribution. Plots of the t density function presented in Figures 24.1 and 24.2 reveal these properties.
Exercise 24.8 Let X and Y be i.i.d. random variables with t , distribution. Then, show that
(24.29) a result established by Cacoullos (1965).
MISCELLANEA
242
In a recently published article, Jones (2002) observed that the t 2 distribution has simple forins for its distribution and quantile functions which lead to simple calculations for many properties and measures relating to this distributiori. The t density in (24.25) reduces, for the case n = 2, simply to P2(t) =
1
< t < 00,
m
(2 + t 2 ) 3 / 2 '
(24.30)
from which we readily obta.in the cdf as

L2

I,,,
( t / 4)
tan
1 h s e c 2 6 dQ 23/2(sec26 ) 3 / 2
(setting 7~ =
tan'(t/JZ)
Jztan 6 )
d6
C O S ~
2
; { l + + j (v5 ) )} 1
=
tan2 (tan'
m
< t < 00.
Exercise 24.9 From (24.31), show that the quantile function of the tribution is 2u  1 FT'(u) = O
(24.31)
t2
dis
&qKq'
Proceeding similarly, the following expressions can be obtained correspondirig to n = 3, n = 4, n = 5, and 'ri = 6: ~ 3 ( t )=
F,(t)
=
Fs(t)
=
F6(t)
=
(5)+
+
tan' 1 2 7 r 1 t ( 6 $ t 2 ) 2 2(4+t2)3I2 ' 1
d3t 7r(3
+
1 2 7 1 + 2 
1 + tan' r
+ + +
t(135 30t2 2t4) 4(6 t 2 ) 5 / 2
+ t2)
'
t DISTRIBUTION
243
LD
0
U 0
'
.,'
m
0
LL
n
a
N 0

0
0
.
0 I
6
I
4
.......
I
I
I
I
I
2
0
2
4
6
t
Figure 24.1. Plots of t density function when n = 1 , 2 , 4
MISCELLANEA
244
Lo
0
U
0
m
0
LL
n Q
rn
0
7
0
0 0
I
6
1
4
1
1
1
1
1
2
0
2
4
6
t
Figure 24.2. Plots of t density function when n = 10,20,30
F DISTRIBUTION
24.6 Let V

245
F Distribution xk and W x i be independent random variables. N
x=V / m W/n

Further, let
n.  V
W
m
(24.32) '
Then, it can be shown that the pdf of X is given by
This is called Snedecor's F distribution with ( m , n ) degrees of freedom, as its original derivation is due t o Snedecor (1934), and we shall denote it by Fm,,. This is related to Karl Pearson's Type VI distribution, which is a beta distribution of the second kind mentioned earlier (see Section 16.4).
Exercise 24.10 Derive the density function in (24.33).
From (24.32), we obtain the r t h moment of X
EX'
= =
for r
X
N
(t)' (t)'
N
Fm,7L as
E(V') E(W')
+
+
m ( m 2). . . (rn 2r  2) (n  2)(n  4) ... ( n 2r)
(24.34)
< n/2. From (24.34), we immediately obtain the mean and variance of Fm3nas EX=
n
n2
(n > 2) and Var X
Exercise 24.11 If X
d
=
=
+
2n2(m 12  2) (n> 4). m(n  2)2(n  4)
F,,,, then show that
fi

(24.35)
l/a) /2 d t,.
246
24.7
MISCELLANEA
Noncentral Distributions
In Section 24.4 we noted that when X I , X 2 , . . . , X, are i.i.d. N ( 0 , l ) random variables, the variable S, = CE=,X z has a chisquare distribution with n degrees of freedom. Now, consider the distribution of the variable n
sk c ( x k + a k ) 2 .
(24.36)
k=l
The distribution of Sk depends on a l , a2, . . . , a, only through X = C:=,a:, and is called thc noncentral chisquare distribution with n degrees of freedom and Iioncentality parameter X = la^. When X = 0 (i.e., when all a l , . . . , a, are zero), this noncentral chisquare distribution becomes the (central) chisquare distribution in (24.17). Exercise 24.12 Let Y1,Y2,. . . , Yn be independent random variables with Y k distributed as N ( Q , g 2 ) ,and Y = Cy=,Y k / n denote the sample mean. Then, show that (nl)S2 1 7L 02
=02
C(Yk 
Y ) 2
k=l
x:=l
is distributed as noncentral chisquare with n  1 degrees of freedom and noncentmlity parameter X = x:=,(ak  u ) 2 / a 2where , zi = akin. Exercise 24.13 From (24.36), derive the mean and variance of the noncentral chisquare distribution with n degrees of freedom and noriceritrality parameter x = u;.
x:=,
In a similar manner, we can define noncentral t and noncentral F distributions which are useful in studying the power properties o f t and F tests. For example, in Eq. (24.32), we defined the F distribution with ( m ,n) degrees of freedom as thn distribution of the variable
where V arid W are independent chisquare random variables with m and n degrees of freedom, respectively. Now, consider the distribution of the variable
(24.3 7 ) where V’ and W’ are independent noncentral chisquare random variables with m and n degrees of freedom and noncentrality parameters XI and X2, respectively. The distribution of X’ is called the doubly noncentral F distribution witjh (m, n ) degrees of freedom and noncentrality parameters ( X I , X2). In the special case when A 2 = 0 (i.e., when there is a central chisquare in the denominator), the distribution of X’ is called the (singly) noncentrul F distribution with ( m ,n) degrees of freedom and noncentrality parameter XI.
Part I11
MULTIVARIATE DISTRIBUTIONS
This Page Intentionally Left Blank
CHAPTER 25
MULTINOMIAL DISTRIBUTION 2 5.1
Introduction
The multinomial distribution, being the multivariate generalization of the binomial distribution discussed in Chapter 5, is one of the most important and interesting multivariate discrete distributions. Consider a sequence of independent and idential trials each of which can result in one of k possible mutually exclusive and collectively exhaustive events, say, A l , Az, . . . , A k , with respectively probabilities p1 , p 2 , . . . ,p k , where p l pa . .. p k = 1. Such trials are termed multinomial trzals. Let Ye = ( Y I ,Yz,!, ~ , . . . , Y k , e ) , 1 = 1 , 2 , . . . , be the indicator vector variables, that is, Y,,e takes on the value 1 if the event A, ( j = 1 , 2 , . . . , k ) is the outcome of the 4th trial and the value 0 if the event A, is not the outcome of the l t h trial. Note that the variables Yl,e,YZ,e,.. . , Yk,! (which are the components of the vector Ye) are dependent, and that
+
+
+
Y1,e
+ Y2,e + . . + Y k , ! = 1, ’
e = 1,2,....
For any n = 1 , 2 , . . . , let us now define the random vector X,
(25.1) =
(XI,,, X Z , ~ ,
,Xk,n) as
x,
(25.2) + YZ + . . . + Y,, n = 1 , 2 , . . . , = y 3 , ~+ y3,2 + . . . + %,, ( j = 1 . 2 , . . . , k ) is the number of occur= Y1
where X,,, rences of event A, in the n multinomial trials. In other words, the random vector X, is simply a counter which gives us the number of occurrences of the events A l , A z , . . . , Ak in the n multinoniial trials, and hence
X l , , + X z , , + ~ ~ ~ + X k ,=, n , n = 1 . 2 , . . . .
(25.3)
Then, simple probability arguments readily yield
P, ( m l , m z , . . , n
~ k ) =
Pr { X I , , = m l , X ~=, m2,. ~ .., X ~ C= ,, m k }
m,
= 0,. ..,n,
249
ml
+ . . . + mk = n.
(25.4)
250
25.2
MULTINOMIAL DISTRIBUTION
Notations
A random vector X, = ( X l , , , x,,,,. . . , X k , , ) having the joint probability mass function as in (25.4) is said t o have the multinomial M ( n ,p l , p2, . . . ,p k ) distribution. In the case when k = 3, the distribution is also referred to a.s the
trinomial distribution.

Remark 25.1 The random vector x, M ( n , p l , p z , . . . , p k ) is actually ( k 1)dimensional since its components XI,^, Xz,,, . . . , X k , , satisfy the relationship XI,, X 2 , n . . f x k , n = n
+
+
'
and, consequently, one of the components (say, x k , n ) can be expressed as xk,, =n

XI,,

X z , ,  . . .  xk 1 ,n.
Hence, the distribution of the random vector X, = (XI,,, X 2 , , , . . . , Xk,lL) is completely determined by the distribution of the ( k  1)dirnensional random vector XI,^, X2,,,. . . , Xkl,?L). For cxaniple, when k = 2, the probabilities P,(ml, m.2) in (25.4) simply become
which are the binomial probabilities.
25.3
Compositions
Due to the probability interpretation of multinomial distributions given above, it readily follows that if independent vectors Y1, Y2,. . . ,Y, all have multinomial h ' ( l , p 1 , p 2 , . . . , p k ) distribution, then t h e s u m x , = Y 1 + Y 2 + . . . + Y n has the iiiult,iriomial M ( n , p l , p 2 , . . . , p k ) distribution. In addition, if X M(n1,p1.p2.. . . , p k ) and Y M ( n 2 ,p l , p a , . . . , p k ) are independent multinomial random vectors, then X Y is distributed as the sum Y1 Y2 . . . Y,,+,, of i.i.d. multinomial M ( l , p l , p z , . . . ,p,+)random vectors, and hence, is distributed as multinomial M(nl n2,p1,pzr.. .,pk).

+
+
+ +
+
25.4
Marginal Distributions
The fact that the multinomial distribution is the joint distribution of the number of occurrences of the events Al, A2,. . . ,A,+ in n rnultinornial trials enables ub to derive easily any marginal distribution of interest. Suppose that we are interested in finding the marginal probabilities
P r ( X 1 , , = m ~ , X a=, m ~,...,X,,,=m,}
CONDITIONAL DISTRIBUTIONS for j
< k , when X,

25 1
M ( n , p l , p z , .. . , p k ) . We first note that
P r { X I , , = m l , . . . ,X j , ,
=mj}
= Pr {XI,, = m l , . . . , X j , , = mj, V = m } ,
(25.6)
where
evidently, V denotes the number of occurrences of the event A = Aj+l U Aj+2 U.. .U A k in the n multinomial trials with the corresponding probability of occurrence being
Pr{A} = pj+l
+ ... + p k
=
1 p l

. . .  p .3

P (say) .
Then, the random vector (XI,,,. . . , Xj,,, V) clearly has the multinomial M ( j l , p l , . . . , p j , p ) distribution; then, using (25.4) and (25.6), we have
+
Pr
= m l , . .., X j , , = m j }
=
P, ( m l ,. . . ,mj,m )
j
j
m + C m i = n and p + C p i = l . i=l i=l (25.7) In particular, for j of XI , , as Pr
=
1, we simply obtain from (25.7) the marginal distribution
= rnl} =
n!
p Y l ( 1  pl)nml , ml
ml! ( n ml)!
= 0 ,..., n,
(25.8)
which simply reveals that the marginal distribution of X I , , is binomial B ( n , p l ) Similarly, we have X , , B(n,p,) for any r = 1 , 2 , . . . , k . N
2 5.5

Conditional Distributions
M ( n , p l , p z , .. . , p k ) . Consider now the conditional distribution of (X,+i,,,. . . , X k , 7 L )given , ( X i , , = m i , . . . ,X,,, = m,) , defined by
Let X,,
Pr{X,+i,,
= m3+1,.
.. ,Xk,,
= mk
1 XI,,
= m l , .. .
,x,,,= m,} (25.9)
Substituting the expressions in (25.4) and (25.7) into (25.9), we obtain
MULTINOMIAL DISTRIBUTION
252
Pr {Xj+l,, = n ~ j +. .~. ,,X k , ,
=mk
1 X1,,
= m l , . . . ,Xj,,, = m j ]
(25.10)
+. +
+. +
for mj+l . . mk = n  (ml . . m j ) ,and 0 otherwise. From (25.10), we readily observe that the conditional distribution of ( X j + l , n ,. . . ,Xk,7L),given = m l , . . . ,Xj,, = m j , is multinomial M ( n  m , y j t l , . . . , y k ) , where m = 'tn1 . . . m j , yi = p i / p (for i = j 1 , .. . , k ) , and p = p j + l + . . . + p k . Since the dependence on ml, m2,. . . , mj in (25.10) is only through the sum 7n1 ni2 . . . mj, we readily note t1ia.t
+ + + + +
+
P r {Xj+l,, = mj+l,.. . , X k , , = m
k
I
= m l , .. . . X j , , = mj)
(25.11) hence, the conditional distribution of (x2+l,n, . . . , X k , l L ) , given Xl,+ . . . X 2 , n= r n , is also the same multinomial M ( n  7)2,y J + l , . . . , y k ) distribution.
+
25.6
+
Moments
Let x,, :( X I,,,, . . . , x k , n ) M ( n . p l . . . . , p k ) . Then, since the marginal distribution of X,,, ( r = 1 , 2 , . . . , k ) is binomial B(n,p,), we readily have N
EX,,,,
= 7lp, = e,
and Var Xr,n = np,(l

p,) = (T, 2 .
(25.12)
Next, in order t o derive the correlation between the variables X,,,, ( r 1.. . . , k ) , we shall first, find the covariance using the formula
where the last equality follows from the fact that
=
MOMENTS
j
i
i
j
=
253
Cj ~r { x ~ j ,> E~(Xr,nIXs,n =
j
=E
=j
) (25.14)
{Xs,n E (Xr,nlXs,n)}.
Now, we shall explain how we can find the regression E (Xr,nlXs,,) required in (25.13). For the sake of simplicity, let us consider the case when T = 2 and s = 1. Using the fact that the conditional distribution of the vector (X2,,,. . . , X k , n ) , given = m, is multinomial M ( n  m, q 2 , . . . , q k ) , where qi = pi/( 1  p l ) (i = 2,. . . , k ) , we readily have the conditional distribution of X 2 , n , given XI,^, = m, is binomial B ( n  m, q 2 ) ; hence, (25.15) from which we obtain the regression of
X2,,
to be
on
Using this fact in (25.13), we obtain 012
= 
E XI,^ XZ,,)e1e2 E 1x1,~ E (X2,nlt1,n)} e1e2 EP2 1P1 Pa lpl

( n  Xl,n)P2 1 P1
~
( n XI,,)} 2

e1e2
{ n Pl nP1(1P1)n2P?}
nPlP2.
7L2PlP2
(25.16)
Similarly, we have
From (25.12) and (25.17), we thus have the covariance matrix of X, (Xl,n,.. . , X k , n ) to be
=
(25.18) for 1 5 T , S 5 k . Furthermore, from (25.12) and (25.17), we also find the correlation coefficient between Xr,n and X s , n (1 5 T < s 5 k ) to be (25.19)
254
MULTINOMIAL DISTRIBUTION
It is important t o note that all the correlation coefficients are negative in a multinomial distribution.
Exercise 25.1 Derive the multiple regression function of . .. Xy,n.
XT+1,7Lon
XI,^,
1
25.7
Generating Function and Characteristic Function
Consider a random vector Y = (Y1,. . . , Yk)having M ( l , p l ,. . . ,pk) distribution. As pointed out earlier, the distribution of this vector is determined by the nonzero probabilities (for T = 1,. . . , k ) py = P r {Yl = 0 , . . . , Yrl= 0, Yr 1,Yy+1= 0 , . . . , Yk = o} .
Then, it is evident that the generating function
$ ( s 1 , s2,..
(25.20)
. , sk) of Y is (25.21)
+
+
Since X, N M ( n , p l , .. . ,pk) is distributed as the sum Y1 . . . Y, [see Eq. (25.2)], where Y1,.. . , Y n are i.i.d. multinomial M ( l , p l , .. . ,pk) variables with generating function as in (25.21), we readily obtain the generating function of X, as
n
=
(Q(S1,
. . . ,Sk)}n
=
From (25.22), we deduce the generating function of m = l , . . . , k  1 ) as Rn(s1,.
. . ,S r n )
=
Pn(S1,.
. . , sm, 1,.. . ,I)
In paxticular, when m = 1, we obtain from (25.23)
. . , X m , 7 1 (for )
GENERATING FUNCTION AND CHARACTERISTIC FUNCTION 255 which readily reveals that X I , , is distributed as binomial B ( n , p l ) (as noted earlier). Further, we obtain from (25.23) the generating function of the sum X I , , . . . Xm,n (for m = I , . . . , k  1) as
+ +
EsX13,+...+X,,
=Rn(S,.
. . ,s ) =
+
{+ 1
(8 
1)e p r } ; ' ,
r=l
+ +
(25.25)
which reveals that the sum XI,^ . . . Xm,n is distributed as binomial B (n, p r ) . Note that when m = k , the sum XI,^ . . . Xk,n has a degenerate distribution since X I , , + . . . Xk,, = n.
c7==l
+
+
Exercise 25.2 From the generating function of X, M ( n , p l , . . . , p k ) in (25.22), establish the expressions of means, variances, and covariances derived in (25.12) and (25.17). N
Exercise 25.3 From the generating function of (XI,,, . . . , X m , n ) in (25.23), prove that if m > n and m 5 k , then E (XI,,.. . Xm,+)= 0. Also, argue in this case that this expression must be true due to the fact that at least one of the X r , n ' ~ must be 0 since . . . Xk,, = n.
+
+
From (25.22), we immediately obtain the characteristic function of X n M ( n , ~ l ., .., ~ l i as ) N
E
f n ( t l , .. . , tk) =
{
ei(tlXl,~+'..+tkxk,n)
P, (eitl, . . . ,e i t k )
3
(25.26) In addition, from (25.23), we readily obtain the characteristic function of (XI,,, . . . , X m , n ) (for m = I , . . . ,k  1) as g,(tl,. . . , tm)
= =
R, ( e i t l , .. . ,eitm)
{+ 1
n
m
x p r
(,it,

1)) .
(25.27)
r=l
Exercise 25.4 From (25.27), deduce the characteristic function of the sum XI,^ . . . Xm,, (for m = 1 , 2 , . . . , k  1) and show that it corresponds to that of the binomial B (n, pr).
+ +
xr=l
256
25.8
MULTINOMIAL DISTRIBUTION
Limit Theorems
Let 11s now consider the sequence of random vectors
Xn
= ( X 1 , n r . .. , X k , n ) N
M(n,pl,.. . ,
(25.28)
~ k ) ,
where p k = I  C“’ p,. Let p , = A T / n for T = 1,.. . , k  1. Then, for nz = k1, the characteristic function of . . , X ~  I , ~in&(25.27) ) becomes n
(25.29) Letttirig n
4
00
in (25.29), we observe that
where h,(t) = exp {A, (ezt 1)) is the characteristic function of the Poisson .(A,.) distribution (for T = 1,.. . , k  1). Hence, we observe from (25.30) the components XI,,, . . . , X k  l , , of the multinomial random tha.t, a.s ‘n+ x, vector X, in (25.28) are asymptotically independent and that the marginal distribution of X , , converges to the Poisson .(A,) distribution for any r = 1 , 2 , . . . , k  1. ~
+
+
Exercise 25.5 Using a similar argument, show that XI,, . . . X m , , (for rn = 1,.. . , k  1) converges to the Poisson 7r ( C r = l A,) distribution.
Next, let tors
11s corisider
the sequence of the ( k  1)dimensional raridorri vec
Let h,,(tl.. . . , t k  1 ) be the characteristic function of W, in (25.31). Then, it follows froni (25.27) that
(25.32)
Exercise 25.6 As n to
LIMIT THEOREMS
257
show that h,(tl,.
. . , t k  l ) in (25.32) converges
4 m,
where
is the correlation coefficient between Xr , n and X s , n derived in (25.19).
From (25.33), we see that the limiting characteristic function of the random variable XT.n  npr Wr,n = nP ( 1  P 1
dy
(it:),
becomes exp which readily implies that the limiting distribution of the random variable Wr,, is indeed standard normal (for T = 1,.. . , k  1). Furthermore, in Chapter 26, we will see that the limiting characteristic function of W, in (25.33) corresponds to that of a multivariate normal distribution with mean vector (0,. . . , 0) and covariance matrix ifi=i
(25.34)
for 1 5 i , j 5 k  1. Hence, we have the asymptotic distribution of the random vector W, in (25.31) to be multivariate normal.
This Page Intentionally Left Blank
CHAPTER 26
MULTIVARIATE NORMAL DISTRIBUTION 26.1
Introduction
The multivariate normal distribution is the most important and interesting multivariate distribution and based on it, a huge body of multivariate analysis has been developed. In this chapter we present a brief description of the multivariate normal distribution and some of its basic properties. For a detailed discussion on multivariate normal distribution and its properties, one may refer t o the book by Tong (1990). At the end of Chapter 25 (see Exercise 25.6), we found that the limiting distribution of a sequence of multinomial random variables has its characteristic function as [see Eq. (25.33)]
h ( t l , .. . ,tkl)
= exp
{

1
Q ( t l , . . . , tk1)) ,
(26.1)
where Q (tl, . . . ,tk1) is the quadratic form
Q(t1,. . . d k  1 ) =
k1
Ct:+ 2 C
Prstrts.
(26.2)
l
r=l
The quadratic form Q (tl, . . . ,tk1) in (26.2) can be written in matrix notation as
Q(t) = tCt’,
(26.3)
where t is a row vector ( t l , . . . , tkl), and C is a (k1) x (k1) real symmetric positive definite matrix with (i, i)th element as 1 and (i, j ) t h element as p i J . In fact, C is the covariance matrix of a random vector (Y1,. . . , Yk1) with variances 1 and covariances p i j (which are also the correlation coefficients). It can be shown that (we state this result without proof)
259
260
MULTIVARIATE NORMAL DISTRIBUTION
where
with 1x1 denoting the determinant of the matrix C, Q'(t) = tC't/, and C' denoting the inverse of the matrix C. Eqmtion (26.4) implies that the nonnegative function p ( z 1 , . . . , Zk1) satisfies the condition
(26.6)
= 1
using (26.2). Hence, p ( z 1 , . . . , ~ random variable, and
h ( t l , . .. , t k  1 )
k  1 )
is the pdf of some ( k  1)dimensional
Q ( t l ,... . t k  I ) }
= exp
(26.7)
is indeed the characteristic function of this distribution
26.2
Notations
Let C = (crLj)tj=' be a (,a x n) real symmetric positive definite matrix. Note that such a matrix may be a matrix of secondorder moments of somc ndimensional distribution. Let us now define the quadratic form corresponding to C as
Q(t) = t C t '
c n
=
orrt;
+2
r=l
1
(26.8)
17rstrts.
ljr<s
As twforc:, let C' denote the inverse of the matrix C and Q' (t) = tClt'. Then, a random vector Y = (Yl,. . . , Yn)is said to have a multivariate normal MlV(0,C ) distribution, if it,s pdf is of the form g(z1,.. . , z),
=
r nexp { 1

. . , z,)
~1(zl,.
},
(26.9)
and its characteristic function is (26.10) where the quadratic form Q(t1,.. . , t n ) is as in (26.8). In this case, the first parameter 0 in the notation MlV(0, C) is a row vector of dimension ( n x 1)
NOTATIONS
261
with all of its elements being 0. From the characteristic function of Y in (26.10), we then readily find that EY,
for
=0
T
= 1 , .. . , n ,
(26.11)
and that C is indeed the matrix of the secondorder moments of Y , that is,
Va.r Y, Cov(Y,,Ys)
= =
oTr for 1 5 T 5 n, urS for 1s r < s 5 n.
(26.12)
Using the random vector Y, we can now introduce a new random vector
where m = ( m l ,. . . , m,) is the vector of means of the components of the random vector X.
Exercise 26.1 From (26.9), show that the pdf of the random vector X is given by p(z1,. . . , z,)
=
1
m
(26.14)
Exercise 26.2 From (26.10), show that the characteristic function of the random vector X is given by
(26.15) where the quadratic form Q(t1,.. . , t,) is as in (26.8).
We then say that such a random vector X = ( X l , . . . , X,) has a multivariate normal distribution (in n dimensions, of course) with mean vector m and covariance matrix C, and we denote it by X M N ( m ,C). N
262
26.3
MULTIVARIATE NORMAL DISTRIBUTION
Marginal Distributions
Let X = ( X I , .. . , X n ) MN(m,C), and V = ( X I , .. . , X e ) , !< n. Then, from the characteristic function of X in (26.15), we readily obtain the characteristic function of V as N
~ei("xl+.+tPxe)
= f ( t 1 , . . . ,t,,O, . . . , O )
(26.16) Equation (26.16) immediately implies that the random vector V .t < n, is distributed as MN(m('), C(')), where
=
( X I ,. . . , X e ) ,
In particular, we observe that marginally X I N(m1,nl1),where ml E X 1 and 011 = Var X I . Similarly, it can be shown that marginally, X,. N ( ~ , . . O ; for ~ ) any T = 1,.. . ,n. N
26.4
= N
Distributions of Sums
Let X and Y be two independent (ndimensional) multivariate normal random vectors with parameters (m('),C(')) and (m('),C(2)),respectively. Further, let V be a new random vector defined as V = X Y .
+
Exercise 26.3 Then, using Eq. (26.15), prove that V MN(m(1)+m(2), C(')t C(')). More generally, if X, M N ( m ( J )C")) , ( j = 1 , . . . , k ) are independent k random vectors, prove that C,=, X, MN(m, C), where N
26.5
N

k
k
j=1
j=1
Linear Combinations of Components

Lct X = ( X I , .. . , X n ) MN(m, C). Now, let us consider the linear combination L = C:==, c,X, = Xc', where c = ( e l , . . . , cn). Then, from the
INDEPENDENCE OF COMPONENTS
263
characteristic furictiori of X in (26.15), we readily obtain the characteristic function of L as
=
f ( C l t , .. . , cnt)
(26.17) Equation (26.17) readily reveals that the linear combination L is distributed as normal N(nz,0 2 ) ,where n
m=
C crmr = mc’ T=l
and
26.6
Independence of Components
It is easy t o find conditions under which the components of the multivariate normal vector X = ( X I , .. . , X,) M N ( m ,C) are all independent. Since in this case, when X I , .. . , X , are all independent, the characteristic function of X in (26.15) must satisfy the condition N
n n
f ( t l ,‘ . .
ltn)
=
f ( 0 ,. . . 10,tT,o,. . . > O),
(26.18)
r=l
we immediately observe that the components of the random vector X are independent if and only if uij = 0
for 1 5 i
< j 5 n.
(26.19)
In other words, the components X I , . . . , X , are all independent if and only if the covariance matrix C is a diagonal matrix.
MULTIVARIATE NORMAL DISTRIBUTION
264
26.7
Linear Transformations

Let X = (Xl,. . . , X n ) M N ( m ,C ) . Further, let Y = (Yl,. . . , Y,) = XC be a linear transformation of X, where C is a ( n x n ) nonsingular matrix.
Exercise 26.4 Then, prove that Y
= (Y1,.
. . , Y,)
N
MN(mC,C’CC).
It is well known from matrix theory that for any symmetric ( nx n) matrix C, there exists an ( n x n ) orthogonal matrix C such that C’CC is a diagonal matrix; be reminded that a matrix C is said to be an orthogonal matrix if CC’ = I,, where I, denotes an identity matrix of order ( n x n ) . From this property, it is clear that if C’CC is a diagonal matrix, then the components of the random vector Y = ( Y l , . . , Y,) MN(mC,C’CC) will all be iridependent. Hence, if X = ( X l , .. . , X,) M N ( m ,C), then there exists an orthogonal linear transformation Y = XC that generates independent normal random variables

N
y,. = q r X 1 + ’ .
‘
+ c,,x,,
T
=
1... . ,n.
(26.20)
Moreover, we see that X can be expressed as X = YB, where B = C’ is also an orthogonal matrix. This implies that the components X I , .. . , X, of any multivariate normal random vector X can be expressed as linear combinations
x,= b,lYI + . . . + b,,Y,,
T
= 1 , .. . , n
(26.21)
of independent normal random variables Yl, . . . , Y,. The orthogonality of the transformations in (26.20) and (26.21) can be exploited t o obtain the following relations:
c n
Y,” = YY’
1.=1
=
xc (XC)’ = X(CC’)X’ = XInX’ = xx’ =
c n
r=l
X; (26.22)
a.nd n
(Y  EY) (Y  EY)’ r=l
(XC  EXC) (XC  EXC)’ (X  EX) CC’ (X  EX)’ (X  EX) I, (X  EX)’ (X  EX) (X  EX)’
c n
(Xr 
r=l
’
(26.23)
BIVARIATE NORMAL DISTRIBUTION
265
It follows, for example, from (26.22) and (26.23) that n
n
r=l
r=l
(26.24) and (26.25) r=l
26.8
r=l
Bivariate Normal Distribution
In this section we discuss in more detail the special case when n = 2, that is, the twodimensional random vector ( X I ,X2) having a bivariate normal distribution. In this case, we denote the mean vector by (ml,m2)and the covariance matrix by (26.26) where 02 = Var X I , ( k = 1,2), p0102 = Cov(X1,X2), and p is the correlation coefficient between X1 and X z . Since this bivariate normal distribution depends on five parameters, we shall denote it by B N ( m 1 ,m2, a:, 02, p ) .
Characteristic function: The characteristic function for this special case is deduced from (26.15) t o be h(t1,t 2 )
= exp
{i (rn1tlt mZt2)
1 (oft: 2
 
+ 2palaztlt2 + a,2t;)
1
. (26.2 7)
Note that the expressions of the mean vector and the covariance matrix given above can also be obtained easily from the characteristic function in (26.27).
Density function: From (26.26), we note that the determinant of the matrix C is 1x1 = afa,2(1  p 2 ) , which is positive if as > 0, a$ > 0, and IpI < 1. If IpI = 1, then there exists a linear dependence between the variables X I and X2 and, therefore, the vector ( X I ,X 2 ) has a degenerate normal distribution. The inverse of the matrix C in (26.26) can easily be shown to be (26.28) Upon substituting for these expressions in (26.14) and simplifying, we obtain the pdf of ( X 1 , X z )to be
266
MULTIVARIATE NORMAL DISTRIBUTION
2p(y) (F(%2)2 ) +
(26.29)
Exercise 26.5 By integrating the pdf p ( z 1 , z2) in (26.29) with respect to 2 2 and z l , show tha.t the marginal distributions of X1 and X z are N ( n ~ 1a:,) and N(rn2,a;), respectively.
It should, however, be mentioned here that if the marginal distributions and X2 are both normal, it does not necessarily imply that the vector ( X I ,X z ) should be distributed as bivariate normal. In order to see this, let us consider the bivariate density function
of
X1
+exp{

1 2 ( 1  p2)
(4

2P2122
+ z;)
11
,
(26.30) which is a mixture of the densities of BN(O,O,1,1, p ) and BN(O,O,1,1,p).
Exercise 26.6 If the bivariate random vector (X1,Xa) has its pdf as in (26.30), then show that XI N ( 0 ,l),X2 N ( 0 ,l ) ,and Cov(X1, X 2 ) 0. N
N
But, as seen earlier, among all bivariate normal distributions, only BN(O,O,1,I, 0) with pdf (26.31) can provide such properties for the marginal distributions. Since the pdf h ( s l ,~ 2 in) (26.31) does not equal the pdf g(z1, z2) in (26.30), we can conclude that g ( z l , Q) in (26.30) is not a bivariate normal density function, but it does have both its marginal distributions to be normal.
Some relationships: Let V and W be independent standard normal variables. Further, let XI = V
and
XZ=pV+ J v W ,
(26.32)
BIVARIATE NORMAL DISTRIBUTION
267
where IpI < 1. Then, we readily have the characteristic function of the bivariate random vector ( X I ,X 2 ) to be
f(t1,ta)
=
~eitlXl+it2X2
{ +t2p)V +
=
E exp

Eei(tl+t2P)VEeit2J1p2W
=
exp
=
exp
{ {
i(t1


1 2 1  (t: 2
(tl+ t2p)2
i t 2 JW}

1 Z t i ( l p2)}
+ 2pt1t2 + t;)
I
,
(26.33)
which, when compared with (26.27), readily implies that ( X I ,X 2 ) is distributed as BN(O,O,1,1, p).
Exercise 26.7 Establish this result by using the Jacobian method on the density function. Exercise 26.8 More generally, prove that the bivariate random vector ( X I ,X 2 ) , where
X1 =ml + a l V
and
X2 = r n z + u 2
is distributed as B N ( m l , m 2 , a ~ , a & p ) .
Conditional distributions: Let the random vector ( X l r X 2 )be distributed as BN(rn1, m2, uY,u;,p). In this case, as seen earlier, X1 N(m1,n:) and X2 N(rn2,a;). From (26.29), we can then obtain the conditional density function of X I , given X z = 5 2 , as N

268
MULTIVARIATE NORMAL DISTRIBUTION
(26.35) where (26.36) Froin (26.35), it is clear t,ha.tthe conditional distribution of X I , given X , = 2 2 , is simply N ( X ( I C . L ) , (Tp~2() ~ ) , where X ( z 2 ) is as given in (26.36).
Exercise 26.9 Proceeding similarly, prove that the conditional distribution of X,, given X I = 2 1 , is N(X*(z1),0$(1 p 2 ) ) , where X*(ZC,) = m2
+ pa2
(26.37)
Regressions: From Eq. (26.35), we immediately have the conditiona.1 mean of
which is a linear function in
z2. Similarly,
X1
to be
we have from Exercise 26.9 that
which is a linear function in 2 1 . Thus, for a bivariate normal distribution, both regression functions (of X1 on X2 and of X 2 on X I ) are linear. Furthermore, we note that the conditional variances are at most as large as the corresponding uncondit,ional variances. In fact, the conditional variances a.re strictly snialler than the corresponding unconditional variances whenever thp correlation coefficient lpi z 0.
CHAPTER 27
DIRICHLET DISTRIBUTION 27.1
Introduction
In Chapter 18, when dealing with exponential order statistics, we established an important property of order statistics U1,n 5 UZ,, 5 . . . 5 as [see Eq. (18.25)] (27.1)
+ + +
+
where S k = Yl YZ . . . Y k ( k = 1 , 2 , .. . , n 1) is a sum of independent and identically distributed standard exponential random variables. Now, let T k , n = Uk>n Ukl,, ( k = 1 , 2 , . . . ,n), with the convention that Uo,, 3 0, denote the uniform spacings. Then, from (27.1) it is evident that (27.2) Clearly, therefore, the joint distribution of the uniform spacings TI,^, . . . , Tn,%is the same as the joint distribution of Y1/Sn+l,.. . , Yn/Sn+l.In the simplest case of k = n = 1, it is clear that we have only one variable Yl/(Yl Y z ) and that its distribution is the same as that of TI,^ = U1,1 = U1; in other words, Yl/(YI Y z )has the standard uniform U ( 0 , l ) distribution. In the general case, however, the distribution is somewhat involved and we will now proceed to derive it. With Y1,Y2.. . . , Y,+l being independent standard exponential random variables, we have their joint density function as
+
+
P Y ~ , , ~ , + l ( Y l , . . . , Y Y n + l ) =e("+
+Yn+l),
2
~ l , . . . , ~ n + l0.
(27.3)
Consider now the transformation V1 = Y1,V2 = Yz,
. . . , V,
= Yn and Vn+l= Yl
+ . . . + Yn+l.
(27.4)
Then, after noting that the Jacobian of this transformation is 1, we obtain from (27.3) the joint density function of V1,V2,.. . , Vn+l as P v ~ , ,v,,,
,
( c 1 , .. . tJn+1)=
e""+',
211,.
. . ,V n 2 0 ,
n
C Z=1
269
5 v,+I.
21%
(27.5)
270
DIRICHLET DISTRIBUTION
Now, consider the transformation
x1 =
Vl V,+l
~
,
... ,X,=
V, Vn+l
Xn+l = K+l
and
(27.6)
or equivalently,
Vl
= XIXn+l, . . .
, Vn = Xn Xn +lr and
Vn+l = Xn+l.
(27.7)
From (27.7), it is clear that the Jacobian of this transformation is Xz+l. Then, we obtain from (27.5) the joint density function of XI, X 2 , . . . ,Xn+l as P X ,..., ~ x,+, (21,. . . ,2n+1) = e""+'
05
21,.
Integrating out the variable function of X I , . . . , X , as
.., 2 ,
z,+1
< 1, 0 5
n
x:+11
5 1, ~ , + 12 0.
(27.8)
i=l
from (27.8), we then obtain the joint density n
~x~,...,x~(21,...,2,)=~!, 0 < ~ 1 , . . . , 2 , I1, 0 < c ~ I 1 .
(27.9)
i=l
In addition, from the density functions in (27.8) and (27.9), we readily observe that the random vector (XI,. . . ,X n ) and the random variable Xn+l are statistically independent. We thus have the joint density function of the uniform spacings TI,,, . . . , T,,n to be given by (27.9). If A, denotes the region
A,=
i
n
I
, . . . ,x,): O < Z ~, . . . ,~ ~ 50 1 < C , ~ i < l,
(21
i=l
(27.10)
then we imniediately have from the joint density function in (27.9) that (27.11)
It turns out that (27.11) is the simplest case of the wellknown Dirichlet integral formula, which, in its general form, is [Dirichlet (1839)]
for ak's positive, where I?(.) denotes the complete gamma function. For details on the history of the Dirichlet integral formula above and its rolo in probability and statistics, one may refer to Gupta and Richards (2001).
DERIVATION OF DIRICHLET FORMULA
27.2
271
Derivation of Dirichlet Formula
In deriving the simpler formula in (27.11), we started with the random variables Y1, . . . , Yn+l as independent standard exponential random variables. As a natural generalization, let us now assume that Y1,. . . , Yn+l are independent gamma random variables with Yk r ( a k ,0, l),where ak > 0. Then, the joint density function of Y1, . . . ,Yn+lis N
With the transformation in (27.4), we obtain the joint density function of Vl,. . , Vn+l as '
Now making the transformation in (27.7) (with the Jacobian as X:+l), obtain from (27.14) the joint density function of X I , . . . ,Xn+las P X 1 , ...,Xn+1 (51,.. '
x (1

we
,Zn+l)
p)an+l', n
Upon integrating out the variable xn+l in (27.15), we then obtain the joint density function of X I , . . . , Xn as PXl,...,x, (21, . . . , Z n )
c n
0 5 21, . . . , 2 n 5 1, 0 5
22
i=l
I 1.
(27.16)
272
DIRICHLET DISTRIBUTION
Indeed, from the fact that (27.16) represents a density function, we readily obtain
which is exactly the Dirichlet integral formula presented in (27.12). We thus have a multivariate density function in (27.16) which is very closely related to the multidimensional integral in (27.12) evaluated by Dirichlet (1839).
Notations
27.3
A random vector X = ( X I ,. . . , X,) is said to have an ndimensional standard Dirichlet distribution with positive parameters a l , . . . , a,+l if its density function is given by (27.16) and is denoted by X D,(al,. . . , a,+l). Note that when
71. =

1, (27.16) reduces to
which is nothing but the standard beta distribution discussed in Chapter 16. Indeed, the linear transformation of the random va.riables X I , . . . ,X , will yield the random vector (bl c1X1,. . . , b, cnXn) having an ndimensional general Dirichlet distribution with shape parameter ( a l ,. . . , a,+l), location parameters ( b l , . . . , b,), and scale parameters (c1,. . . , c,). However, for the rest, of this chapter we consider only the standard Dirichlet distribution in (27.16), due t o its simplicity.
+
+
Marginal Distributions
27.4
Let X D n ( a l , . . . ,a,+l). Then, as seen in Section 27.2, the components XI, . . . , X , admit the representations N
(27.17) and yk
d
XI, =

~
s,.+1
,
k = 1 , . . . ,n,
(27.18)
where Y1. . . . , Yn+l are independent standard gamma random variables with Yk r ( a k ,0,1) and = Y1 .. Y,+l. From the properties of gamma distributions (see Section 20.7), it is then known that
sn+l
S,,,

+. +
r ( a , 0 , l ) with a = a1
+ . . . + a,+l,
MARGINAL DISTRIBUTIONS
Sn+l
Yk
N
r ( a  a k , 0,1) (independent of
and d
y k
&
XI, = sn+l
273
y k
y k
+ (!%+I
Yk)

Yk)
(27.19)
B e ( a k , u  arc);
that is, the margina.1 distribution of X k is beta B e ( a k , a  u k ) , k = 1,.. . ,n (note that this is just a Dirichlet distribution with n = 1). Thus, the Dirichlet distribution forms a natural mult,ivariate generalization of the beta distribution. Exercise 27.1 From the density function of X in (27.16), show by means of direct integration that the marginal distribution of x k is B e ( a k , a  a k ) .
For twodimensional marginal distributions, it follows from (27.17) that for any 1 5 k < e 5 n,
where Yk a t , 0, l), and that N
(27.20)

0, I ) , Z =  Y k  5 r(a ak 0, I),5 and 2 are independent. Thus, (27.20) readily implies
Y k , Ye,
N
(27.21) In a similar manner, we find that
for any
T
= 1 , . . . ,n. and 1 5 k(1)
< k ( 2 ) < . . . < k ( r ) 5 n.
Exercise 27.2 If (XI,.. . ,XS)
N
(Xl,x2 + x3,x4
D ~ ( a 1 ,. . , a 7 ) , show that
+ x5 + X S )
Exercise 27.3 Let (XI,.. . , X n ) of Wk = XI + . . . X k .
+
N
DD3(Ul,a2
+ a3, + a5 + a(j,a7). a4
D n ( a l , . . . ,u,+l). Find the distribution
Exercise 27.4 Let (XI,X 2 ) Dz(a1, a2, a 3 ) . Obtain the conditional density function of XI, given X2 = 2 2 . Do you observe some connection to the beta distribution? Derive an expression for the conditional mean E(XIIX2 = 2 2 ) and comment. N
2 74
DIRICHLET DISTRIBUTION
Marginal Moments
27.5
Let X V , ( a l , . . . ,a,+l). In this case, as shown in Section 27.4, the marginal distribution of Xk is beta Be(ak, a  a k ) , where a = a1 . . . a,+l. Then, from the formu1a.s of moments of beta distribution presented in Section 16.5, we immediately have
+ +
N
27.6

Product Moments
Let X D n ( a l , .. . , a,+l). Then, from the density function in (27.16), we have the product moment of order ( a 1 , .. . , a,) as
E ( X p l .. . X:")
where the last equality follows by an application of the Dirichlet integral formula. in (27.12). In particula.r, we obtain from (27.23) that
and
Exercise 27.5 Let X N D T L ( a l., . , lation coefficient between X I , and X!.
Derive the covariance and corre
2 75
DIRICHLET DISTRIBUTION OF SECOND KIND
27.7
Dirichlet Distribution of Second Kind
In Chapter 16, when dealing with a beta random variable X probability density function

B e ( a , b) with
by considering the transformation Y = X/(1  X ) or equivalently X = Y / ( 1 Y ) ,we introduced the beta distribution of the second kind with probability density function
+

In a similar manner, we shall now introduce the Dirichlet distribution of the second kind. Specifically, let X D,(al,. . . , a,+l), where a k > 0 ( k = 1 , . . . , n + 1) are positive parameters. Now, consider the transformation
x1 . . . ,y, Y1 = 1  x1 . . . Xn ’
1
1
x n
x1

. . .  xrl
(27.24)
or equivalently, x1=
Yl ... l+Y1+...+Yn’
,x, 1 + Y1 +Yn. . ’ + Y,‘ 1
(27.25)
+ +
Then, it can be shown that the Jacobian of this transformation is (1 Y1 . . . + Y,)p(ntl).Then, we readily obtain from (27.16) the density function of Y = (YI,.. . ,Yn)as
The density function (27.26) is the Dirichlet density of the second kind.
Exercise 27.6 Show that the Jacobian of the transformation in (27.25) is (1+ Yl + . . . + Yrl)(n+l) (use elementary row and column operations). Exercise 27.7 Suppose that Y has a Dirichlet distribution of t,he second kind in (27.26). Derive explicit expressions for EYk, Var Yk, cov(Yk,f i ) , and correlation p(Yk,Ye).
276
27.8
DIRICHLET DISTRIBUTION
Liouville Distribution
Liouville (1839) generalized the Dirichlet integral formula in (27.12) by establishing that
where a l , . . . , u, are positive parameters, 2 1 , . . . , 2 , are positive, and f ( . ) is a suitably chosen function. It is clear that if we set h = 1 and choose f ( t ) = (1 t)an+ll, (27.27) readily reduces to the Dirichlet integral formula in (27.12). Also, by letting h + 00 in (27.27), we obtain the Liouuille integral formula
where ~ 1 ,. .. , ulL> 0 and tal+."+anl f ( t )is integrable on ( 0 ,co). The Liouville integral formula in (27.28) readily yields Liouwillr distribution with probability density function
p x l ,...)x n ( Z 1 , . . . , 2 , ) 21,
...,% > O ,
= a1
c f(21 + . ' . + 2 , )
,..., a,>O,
2y11
..' x a n T L  l
l
(27.29)
where C is a normalizing constant a.nd f ( . ) is a nonnegative function such that f ( t ) t " i + ' . ' f a . z  1 is integrable . on (0, co). For a historical view a.nd details on the Liouville distribution, one may refer to Gupta and Richards (2001). '
Exercise 27.8 Show that the Dirichlet distribution of the second kind in (27.26) is a Liouville distribution by choosing the function f ( t ) appropriately and then determining the constant C from the Lioiiville integral forniiila in (27.28).
APPENDIX PIONEERS IN DISTRIBUTION THEORY
As is evident from the preceding chapters, several prominent mathematicians and statisticians have made pioneering contributions to the area of statistical distribution theory. To give students a historical sense of developments in this important and fundamental area of statistics, we present here a brief biographical sketch of these major contributors. Bernoulli, Jakob Born  January 6,1655, in Basel, Switzerland Died  August 16, 1705, in Basel, Switzerland Jakob Bernoulli was the first of the Bernoulli family of Swiss mathematicians. His work Ars Conjectandi (The Art of Conjecturing),published posthumously in 1713 by his nephew N. Bernoulli, contained the Bernoulli law of large numbers for Bernoulli sequences of independent trials. Usually, a random variable, taking values 1 and 0 with probabilities p and 1  p , 0 p 1, is said to have the Bernoulli distribution. Sometimes, the binomial distributions, which are convolutions of Bernoulli distributions, are also called the Bernoulli distributions.
< <
Burr, Irving W. Born April 9, 1908, in Fallon, Nevada, United States Died  March 13, 1989, in Sequim, Washington, United States ~
Burr, in a famous paper in 1942, proposed a number of forms of explicit cumulative distribution functions that might be useful for purposes of graduation. There were 12 different forms presented in the paper, which have since come to be known as the B w r system of distributions, have been studied quite extensively in the literature. A number of wellknown distributions such as the uniform, Rayleigh, logistic, and loglogistic are present in Burr’s system as special cases. In the years following, Burr worked on inferential problems and fitting methods for some of these forms of distributions. In one of his last 277
278
APPENDIX
papers (coauthored with E. S. Pearson and N. L. Johnson), he also made an extensive comparison of different systems of frequency curves.
Cauchy, August inLouis Born Died
~~
August 21, 1789, in Paris, France May 23, 1857, in Sceaux (near Paris), France
AugustinLouis Cauchy was a renowned French mathematician. He investigated the socalled Cauchy functions p(x,h, a ) , which had Fourier transformations of the form
f ( t ) = exp (hiti") ,
h > 0, a
> 0.
It was proved later that p(x,h, a ) , 0 < Q 5 2, are indeed probability density functions. The special case of the distribution with pdf
and with characteristic function
f(t) = exP(ltl) is called the standard Cauchy distribution. The general Caiichy distribution, of course, has the pdf p((x  u ) , h, 1). Dirichlet, Johann Peter Gustav Lejeune
Born Died
~
~
February 13, 1805, in Duren, French Empire (now Germany) May 5, 1859, in Gottingen, Hanover, Germany
Lejcune Dirichlet's family came from the neighborhood of Ligge in Belgium and not, as many had claimed, from France. Dirichlet had some of the renowned mathematicians as teachers and profited greatly from his contacts with Biot, Fourier, Hachette, La.place, Legendre, and Poisson. Dirichlet made pioneering contributions to different areas of mathematics starting with his famous first paper on Ferrnat's last theorem. In 1839, Dirichlet established the general integral formula
The above ndimensional integral is now known as the Dirichlet integral and the probability distribution arising from the integral formula as the Dirichlet distribution.
APPENDIX
2 79
Fisher, Ronald Aylmer Born Died
~
February 17, 1890, in London, England July 29, 1962, in Adelaide, Australia
Fisher was a renowned British statistician and geneticist and is generally regarded as the founder of the field of statistics. The Fisher's Fdistribution, having the pdf
plays a very important role in many statistical inferential problems. One more distribution used in the analysis of variance, called the Fisher's zdistribution, has the pdf
where m, 72
= 1 , 2 , .. .
.
F'r&chet,Re&Maurice Born Died

September 2, 1878, in Maligny, France June 4, 1973, in Paris, France
Frkhet was a renowned French mathematician who successfully combined his work in the areas of topology and the theory of abstract spaces to make his essential contribution to statistics. In 1927, he derived one of the three possible limiting distributions for extremes. Hence, the family of distributions with cdf G(z,cr) = exp(2a), 2 > 0, a! > 0, is sometimes referred to as the Fre'chet type of distributions.
Gauss, Carl F'riedrich Born April 30, 1777, in Braunschweig, Duchy of Brunswick, Germany Died  February 23, 1855, in GGttingen, Hanover, Germany ~
Gauss is undisputably one of the greatest mathematicians of all time. In his theory of errors in 1809, Gauss suggested that the normal distributions for erros with density
For this rea.son, the normal distribution is often called the Gaussian law or as the Gaussian distribution.
280
APPENDIX
Gnedenko, Boris Vladimirovich
Born Died
January 1, 1912, in Simbirsk (Ulyanovsk), Russia December 27, 1995, in Moscow, Russia
~
~
In the first half of the twentieth century, foundations for the theory of extreme values were laid by R.M. Frkchet, R. von Mises, R. A. Fisher, and L. H. C. Tippett. Consolidating these works with his own, Gnedcnko produced his outstanding paper in 1943 t,hat discussed at great length the asymptotic behavior of extremes for independent and identically distributed random variables. For this reason, sometimes the limiting distributions of maximal values are called three types of Gnedenko ’s limiting distributions. Gosset, William Sealey
Born Died
June 13, 1876, in Canterbury, England October 16, 1937, in Beaconsfield, England
~
Being a chemist and later a successful statistical assistant in the Guiniiess brewery, Gosset did important ea.rly work on statistics, which he wrote under the pseudonym Student. In 1908, he proposed the use of the ttest for quality control purposes in brewing industry. The corresponding distribution has pdf
and it is ca.lled Student’s t distribution with n degrees offreedom. Helmert, F’riedrich Robert
Born Died
July 31, 1843, in Freiberg (Saxony), Germany June 15, 1917, in Potsdam (Prussia.), Germany
~
~
Helmert was a famous German mathematical physicist whose main research was in geodesy, which led him to investigate several statistical problems. In a famous paper in 1876, Helmert first proved, for a randoni sample X I ,. . . ,X , from a norrrial N ( a ,0 2 ) distribution, the independence of X arid any function of X I  X ,. . . , X ,  X ,including the variable S2 = C,”=,(X, X ) 2 / a 2 .Then, using a very interesting transformation of variables, which is now referred t o in the literature as Helmert’s transformataon, he proved that S2 is distributed as chisquare with n  1 degrees of freedom. ~
Johnson, Norman Lloyd
Born
~
January 9, 1917, in Ilford, Essex, England
Johnson, having started his statistical career in London in 1940s, came into close contact and collabora.tion with such eminent statisticians as B. L.
APPENDIX
281
Welch, Egon Pearson, and F. N. David. Motivated by the Pearson family of distributions and the idea that it would be very convenient to have a family of distributions, produced by a simple transformation of normal variables, such that for any pair of values (&, 7 2 ) there is just one member of this family of distributions, Johnson in 1949 proposed a collection of three transformations. These resulted in lognormal, S g , and Su distributions, which are now referred to as Johnson's system of distributions. In addition, his book System of Frequency Curves (coauthored with W. P. Elderton), published by Cambridge University Press, and the series of four volumes on Distributions in Statistics (coauthored with S . Kotz), published by John Wiley & Sons and revised, have become classic references to anyone working on statistical distributions and their applications.
Kolmogorov, Andrey Nikolayevich Born Died
April 25, 1903, in Tambov, Russia October 20, 1987, in Moscow, Russia
~
~
Kolmogorov, one of the most outstanding mathematicians of the twentieth century, produced many remarkable results in different fields of mathematics. His deep work initiated developments on many new directions in modern probability theory and its applications. In 1933, he proposed one of the most popular goodnessoffit tests called the KolmogorovSmirnov test. From this test procedure, a new distribution with cdf
originated, which is called the Kolmogorov distribution in the literature.
Kotz, Samuel Born

August 28, 1930, in Harbin, China
Kotz has made significant contributions to many areas of statistics, most notably in the area of statistical distribution theory. His four volumes on Distributions in Statistics (coauthored with N. L. Johnson), published by John Wiley & Sons in 1970 and revised, have become classic references to anyone working on statistical distributions and their applications. His 1978 monograph Characterszation of Probability Distributions (coauthored with J. Galambos), published by SpringerVerlag, and the 1989 book Multivariate Symmetric Distributions (coauthored with K. T. Fang and K. W. Ng), published by Chapman & Hall, have become important sources for researchers working on characterization problems and multivariate distribution theory. A family of elliptically symmetric distributions, which includes t,hP multivariate normal distribution as a special case, are known by the name Kotztype elliptical distributions in the literature.
APPENDIX
282 Laplace, PierreSimon
Born Died
March 23, 1749, in BeaumountenAuge, France Ma.rch 5, 1827, in Paris, France
~
~
Laplace was a renowned French astronomer, mathematician, and physicist. In 1812, he published his famous work The'orie analytique des probabilite's (A n alytic Theory of Probabilrty), wherein he rationalized the necessity to consider and investigate two statistical distributions, both of which carry his name. The first one is the distribution with pdf
where m < n < 00 and X > 0, which is called the Laplace distribution and sometimcs the first law of Laplace. The second one is the normal, which is sometimes ca.lled the second law of Laplace or GaussLaplace distribution. Linnik, Yuri Vladimirovich
Born Died
January 21, 1915, in Belaya Tserkov, Russia (Ukraine) June 30, 1972 , in Leningrad (St. Petersburg), Russia
~
~~
Linnik was the founder of the modern St. Petersburg (Leningrad) school of probability and mathematical statistics. He was the first to prove tha.t
f a ( t ) = (1 + Itla)' (for any 0 < Q 5 2 ) is indeed a characteristic function of some random variable. For this reason, distributions with characteristic functions f a ( t ) are known in the literature as the Linnik distributions. His famous book Charmterization Problems of Mathematical Statistics (coauthored with A. Ka.gan and C. R. Rao), published in 1973 by John Wiley & Sons, also became a basic source of reference, inspiration, and ideas for many involved with research on charact*erizations of probability distributions. Maxwell, James Clerk
Born Died
~
June 13, 1831, in Edinburgh, Scotland November 5, 1879, in Cambridge, England
Jaiiics Maxwell was a farnous Scottish physicist who is regarded as the founder of classical thermodynamics. In 1859, Maxwell was first to suggest that the velocities of molecules in a gas, previously assumed to be equal, must follow the chi distribution with three degrees of freedom having the density
This distribution, therefore, is aptly called the Maxwell distribution.
APPENDIX
283
Pareto, Vilfredo Born  July 15, 1848, in Paris, France Died  August 20, 1923, in Geneva, Switzerland Pareto was a renowned Italian economist and sociologist. He was one of the first who tried t o explain and solve economic problems with the help of statistics. In 1897, he formulated his law of income distributions, where cdf’s of the form
played a very important role. For this reason, these distributions are referred to in the literature as the Pareto distributions.
Pascal, Blaise Born Died

June 19, 1G23, in ClermontFerrand, France August 19, 1662, in Paris, France
Pascal was a famous French mathematician, physicist, religious philosopher, and one of the founders of probability theory. A discrete random variable taking on values 0 , 1 , . . . with probabilities
where 0 < p < 1 and r is a positive integer, is said to have the Pascal distribution with parameters p and r. This distribution is, of course, a special case of the negative binomial distributions.
Pearson, Egon Sharpe Born August 11, 1895, in Hampstead, London, England Died  June 12, 1980, in Midhurst, Sussex, England 
Egon Pearson, the only son of the eminent British statistician Karl Pearson, was influenced by his father in his early academic career and later by his correspondence and association with “Student” (W. S. Gosset) and Jerzy Neyman. His collaboration with Neyman resulted in the nowfamous NeymanPearson approach to hypothesis testing. He successfully revised Karl Pearson’s Tables for Statisticians and Biometricians jointly with L. J. Comrie, and later with H. 0. Hartley producing Biometrika Tables for Statisticians. Egon Pearson constructed many important statistical tables, including those of percentage points of Pearson curves and distribution of skewness and kurtosis coefficients. Right up to his death, he continued his work on statistical distributions, and, in fact, his last paper (coauthored with N. L. Johnson and I. W. Burr) dealt with a comparison of different systems of frequency curves.
284
APPENDIX
Pearson, Karl Born Died

March 27, 1857, in London, England April 27, 1936, in Coldharbour, Surrey, England
Karl Pearson, an eminent British statistician, is regarded as the founding father of sta.tistica1 distribution theory. Inspired by the famous book Natural Inheritance of Francis Galton published in 1889, he started his work on evolution by examining large data sets (collected by F. Galton) when he noted systema.tic departures from normality in most cases. This led him to the development of the P e a r s o n s y s t e m of frequency curves in 1895. In addition, he also prepared Tables for Statisticians and B i o m e t r i c i a n s in order to facilitate the use of statistical methods by practitioners. With the support of F. Galton and W. F. R. Weldon, he founded the nowprestigious journal B i o m e t r i k a in 1901, which he also edited from it,s inception until his death in 1936.
Poisson, SirneonDenis Born Died
June 21, 1781, in Pithiviers, France April 25, 1840, in Sceaux (near Paris), Paris
~
~
Poisson was a renowned French mathematician who made fundamental work in applications of mathematics to problems in electricity, ma.gnetism, and mechanics. In 1837, in his work Recherches s u r la probabilitk des j u g e m e n t s e n mati2re criminelle e t e n m a t i i r e civile, pre'ce'de'es des r2gles ye'ne'rales d u calcul des probnbilite's, the Poisson distribution first appeared as an approximation to the binomial distribution. A random variable has the P o i s s o n distribution if it takes on values 0,1,. . . with probabilities
P d y a , George Born Died
~
December 13, 1887, in Budapest, Hungary Scptember 7, 1985, in Palo Alto, California, United States
P6lya was a famous Hungarianborn U.S. mathematician who is known for his significant contributions to combinatorics, number theory, and probability theory. He was also an author of some popular books on the problemsolving process, such as How t o Solve It and Mathematical Discovery. In 1923, P6lya discussed a special urn model which generated a probability distribution with probabilities 7L
p(p
+ a )' ..{ p + ( k

l)a}q(q
+ a )' . .{ q + ( n

k

l)a}
, . . (1 + ( n l ) a } corresponding to the values 0,1,. . . ,n, where 0 < p < 1, q = 1  p , a > 0 , pk = ( k )
+ a ) ( l+ 2 0 )
(1
'
and n = 1 , 2 , . . . . For this reason, this distribution is called as the Pdlya distribution.
APPENDIX
285
Rao , Ca1yampudi Radhakrishna Born

September 10, 1920, in Huvvinna Hadagalli (Karnataka), India
Rao is considered to be on the most creative thinkers in the field of statistics and one of the few pioneers who brought statistical theory t o its maturity in the twentieth century. He has made numerous fundamental contributions to different areas of statistics. The RaoRubin characterization of the Poisson distribution and the LauRao characterization theorem based on integrated Cauchy functional equation, published in 1964 and 1982, generated great interest in characterizations of statistical distributions. Another significant contribution he made in the area of stat,istical distribution theory is on weighted distributions. In addition, his books Characterization Problems of Mathernatical Statistics (coauthored with A. Kagan and Yu. V. Linnik), published in 1973 by John Wiley & Sons, and ChoquetDeny Type Functional Equations with Applications to Stochastic Models (coauthored with D. N. Shanbhag), published in 1994 by John Wiley & Sons, have also become a basic source of reference, inspiration, and ideas for many involved with research on chara.cterizations of probability distributions.
Rayleigh, 3rd Baron (Strutt, John William) Born  November 12,1842, in Langford Grove (near Maldon), England Died  June 30, 1919, in Terling Place (near Witham), England Lord Rayleigh was a farnous English physicist who was awarded the Nobel Prize for Physics in 1904 for his discoveries in the fields of acoustics and optics. In 1880, in his work connected with the wave theory of light, Rayleigh considered a distribution with pdf of the form
This distribution is aptly called the Rayleigh distribution in the literature.
Snedecor, George Waddel Born Died


October 20, 1881, in Memphis, Tennessee, United States February 15, 1974, in Amherst, Massachusetts, United States
Snedecor was a famous U.S. statistician and was the first director of the Statistical Laboratory a t Iowa State University, the first of its kind in the United States. In 1948, he also served as the president of the American Statistical Association. The Fisher Fdistribution (mentioned earlier) having the pdf
x>0, a>0,
p>o,
286
APPENDIX
is sometimes called as the Snedecor distribution or the FisherSnedecor distribution. In 1937, this distribution was tabulated by Snedecor. Student (see Gosset, William Sealey) Tukey, John Wilder
Born  July 16, 1915, in New Bedford, Massachusetts, United States Died  July 26, 2000, in Princeton, New Jersey, United States Tukey was a famous American statistician who made pioneering contributions to many different areas of statistics, most notably on robust inference. To facilitate computationally easy Monte Carlo evaluation of the robustness properties of normalbased inferential procedures, Tukey in 1962 proposed the transformat ion  (1  X)X if X # 0 Y={ x if X t 0, 1%
xx
(A)
where the random variable X has a standard uniform U ( 0 , l ) distribution. For different choices of the shape parameter A, the transformation above produces a lightta.iled or heavytailed distributions for the random variable X in addition t o providing very good approximations for normal and t distributions. The distributions of X have come to be known as Tukey’s (sym,metric) lambdu distributions. Weibull, Waloddi
Born Died
~
June 18, 1887, in SchleswigHolstein, Sweden October 12, 1979, in Annecy, France
Wa.loddi Weibull was a famous Swedish engineer and physicist who, in 1939, used distributions of the form 2
< 0, oo
X>0,
to represent the distribution of the breaking strength of materials. In fact, these are limiting distributions of minimal values. In 1951, Weibull also demonstrated a close agreement between many different sets of data and those predicted with the fitted Weibull model, with the data. sets used in this study relating t o as diverse characteristics as the strength of Bofors’ steel, fiber strength of Indian cotton, length of syrtoideas, fatigue life of an ST37 steel, statures of adult males born in the British Isles, and breadth of the beans Phaseolus vulga>ris.For this reason, the distributions presented
APPENDIX
287
above are called Weibu,ll distributions, and they have become the most popular and commonly used statistical models for lifetime data. Sometimes, the distributions of maximal values with cdf’s
are also called Weibulltype distributions.
This Page Intentionally Left Blank
BIBLIOGRAPHY Aitchison, J. and Brown, J. A. C. (1957). The Lognormal Distribution, Cambridge University Press, Cambridge, England. Arnold, B. C. (1983). Pareto Distributions, International Cooperative Publishing House, Fairland, MD. Arnold, B. C., Castillo, E. and Sarabia, J.M. (1999). Conditionally Specified Distributions, SpringerVerlag, New York. Balakrishnan, N. (Ed.) (1992). Handbook of the Logistic Distribution, Marcel Dekker, New York. Balakrishnan, N. and Basu, A. P. (Eds.) (1995). The Exponential Distribution: Theory, Methods and Applications, Gordon & Breach Science Publishers, Newark, NJ. Balakrishnan, N. and Koutras, M. V. (2002). R u n s and Scans with Applications, John Wiley & Sons, New York. Bansal, N., Hamedani, G. G., Key, E. S., Volkmer, H., Zhang, H. and Behboodian, J. (1999). Some characterizations of the normal distribution, Statistics & Probability Letters, 42, 393400. Bernstein, S. N. (1941). Sur line propriktk charactkristique de la loi de Gauss, Transactions of the Leningrad Polytechnical Institute, 3,2122. Box, G. E. P. and Muller, M. E. (1958). A note on the generation of random normal deviates, Annals of Mathematical Statistics, 29, 610611. Burr, I. W. (1942). Cumulative frequency functions, Annals of Mathematical Statistics, 13, 215232. Cacoullos, T. (1965). A relation between t and F distributions, Journal of the A m e r i c a Statistical Association, 60, 528531. Correction, 60, 1249. Chhikara., R. S. and Folks, J. L. (1989). The Inverse Gai~ssianDistribution: Theory, Methodology and Applications, Marcel Dekker, New York. 289
290
BIBLIOGRAPHY
Consul, P. C. (1989). Generalized Poisson Distributions: Properties and Applications, Marcel Dekker, New York. Cram&, H. (1936). Uber eirie Eigenscha.ft der normalen Verteilungsfunktion, Mathematische Zeitschrift, 41, 405414. Crow, E. L. and Shimizu, K. (Eds.) (1988). Lognormal Distributions: Theory an,d Applications, Marcel Dekker, New York. Darmois, D. (1951). Sur diverses propriktks charactkristique de la loi de probabilitk de LaplaceGauss, Bulletin of the International Statistical Institute, 23(11), 7982. Devroye, L. (1990). A note on Linnik’s distribution, Statistics & Probability Letters, 9, 305 306. Diriclilet, J . P. G. L. (1839). Sur une nouvelle mhthode pour la dktermination des intkgrales multiples, Lioi~iiille,Journal de Mathe‘matiques, Series I, 4, 164168. Reprinted in Dirichlet’s Werke (Eds., L. Kronecker and L. Fuchs), 1969, Vol. 1, pp. 377380, Chelsea, New York. Douglas, J . B. (1980). Analysis with Standard Contagious Distributions, International Cooperative Publishing House, Burtonsville, MD. Eggenberger, F. and P d y a , G. (1923). Uber die Statistik verketetter Vorgange, Zeitschrift fur Angewandte Mathematik und Mechanik, 1,279289. Evans, M., Peacock, B. and Hastings, N. (2000). Statistical Distributions, 3rd rd., John Wiley & Sons, New York. Feller, W. (1943). On a general class of “contagious” distributions, Annals of Mathematical Statistics, 14, 389~400. Feller, W. (1968). An Introduction to Probability Theory and Its Applications, Vol. 1, 3rd ed., John Wiley & Sons, New York. Gilchrist, W. G. (2000). Statistical Modelling with Quantile Functions, Chapman & Hall/CRC, London, England. Godambe, A. V. a.nd Patil, G. P. (1975). Some characteriza,tions involving additivity and infinite divisibility and their applications to Poisson mixtures and Poisson sums, In Statistical Distributions in Scientific Work ~ 3: Characterizations and Applications (Eds., G. P. Patil, S. Kotz and J. K. Ord), pp. 339351, Reidel, Dordrecht. Govindarajulu, Z. (1963). Relationships a.mong moments of order statistics in samples from two related populations, Technometrics, 5 , 514518. Gradshteyn, I. S. and Ryzhik, I. M. (Eds.) (1994). Table of Integrals, Series, and Products, 5th ed., Academic Press, San Diego, CA. Gupta, R. D. and Richards, D. St. P. (2001). The history of the Dirichlet and Liouville distributions, International Statistical Review, 69, 433446.
BIBLIOGRAPHY
291
Haight, F. A. (1967). Handbook of the Poisson Distribution, John Wiley & Sons, New York. Helmert, F. R. (1876). Die Genauigkeit der Formel von Peters zue Berechnung des wahrscheinlichen Beobachtungsfehlers directer Beobachtungen gleicher Genauigkeit, Astronomische Nachrichten, 88, columns 113120. Johnson, N. L. (1949). Systems of frequency curves generated by methods of translation, Biometrika, 36, 149176. Johnson, N. L. and Kotz, S. (1990). Use of moments in deriving distributions and some characterizations, The Mathematical Scientist, 15, 4252. Johnson, N. L., Kotz, S. and Balakrishnan, N. (1994). Continuous Univariate Distributions, Vol. 1, 2nd ed., John Wiley & Sons, New York. Johnson, N. L., Kotz, S. and Balakrishnan, N. (1995). Continuous Univariate Distributions, Vol. 2, 2nd ed., John Wiley & Sons, New York. Johnson, N. L., Kotz, S. and Balakrishnan, N. (1997). Discrete Multivariate Distributions, John Wiley & Sons, New York. Johnson, N. L., Kotz, S. and Kemp, A. W. (1992). Univariate Discrete Distributions, 2nd ed., John Wiley & Sons, New York. Jones, M. C. (2002). Student’s simplest distribution, Journal of the Royal Statistical Society, Series D, 51, 4149. Kemp, C. D. (1967). ‘StutteringPoisson’ distributions, Journal of the Statistical and Social Enquiry Society of Ireland, 21, 151157. Kendall, D. G. (1953). Stochastic processes occuring in the theory of queues and their analysis by the method of the imbedded Markov chain, Annals of Mathematical Statistics, 24, 338354. Klugman, S. A., Panjer, H. H. and Willmot, G. E. (1998). Loss Models: From Data to Decisions, John Wiley & Sons, New York. Kolmogorov, A. N. (1933). Sulla determinazi6ne empirica di una lkgge di distribuzihne, Giornale d i Istituto Italaliano degli Attuari, 4, 8393. Kotz, S., Balakrishnan, N. and Johnson, N. L. (2000). Continuous Multivariate Distributions, Vol. 1, 2nd ed., John Wiley & Sons, New York. Kotz, S., Kozubowski, T. J . and Podgbrski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance, Birkhauser, Boston. Krasner, M. and Ranulac, B. (1937). Sur line propri6t6 des polynomes de la division du cercle, Comptes Rendus Hebdomadaizes des Se‘ances de 1’Acade‘mie des Sciences, Paris, 204, 397399. Krysicki, W. (1999). On some new properties of the beta distribution, Statistics & Probability Letters, 42, 131137.
292
BIBLIOGRAPHY
Lancaster, H. 0. (1969). The ChiSquared Distribution, John Wiley & Sons, New York. Laplace, P. S. (1774). Mkmoire sur las probabilitk des causes par les isvknemens, Mkmoires de Mathe'matique et de Physique, 6, 621656. Linnik, Yu. V. (1953). Linear forms and statistical criteria I, 11, Ukrainian Mathematical Journal, 5, 207243, 247290. Linnik, Yu. V. (1963). Linear forms and statistical criteria I, 11, in Selected Translations an Mathematical Statistics and Probability, Vol. 3, American Mathematical Society, Providence, RI, pp. 190. Liouville, J. (1839). Note sur quelques intkgrales dkfinies, Journal de Mathe'matiques Pures et Applique'es (Liouville 's Journal), 4, 225235. Lukacs, E. (1965). A characterization of the gamma distribution, Annals of Mathematical Statistics, 26, 319324. Marcinkiewicz, J. (1939). Sur une propriktk de la loi de Gauss, Mathematische Zeitschrift, 44, 612618. Marsaglia, G. (1974). Extension and applications of Lukacs' characterization of the gamma distribution, in Proceedings of the Symposium on Statistics and Related Topics, Carleton University, Ottawa, Ontario, Canada. Patel, J. K. and Read, C. B. (1997). Handbook of the Normal Distribution, 2nd ed., Marcel Dekker, New York. Pea.rson, K. (1895). Contributions t o the ma.thematica1 theory of evolution. 11. Skew variations in homogeneous ma.teria1, Philosophical Transactions of the Royal Society of London, Series A , 186, 343414. Perks, W . F. (1932). On some experiments in the graduation of mortality statistics, Journal of the Institute of Actuaries, 58, 1257. Pblya, G. (1930). Sur quelques points de la thkorie des probabilitks, Annales de l'lnstitut H. Poincare', 1, 117161. P d y a , G. (1932). Verleitung des Gauss'schen Fehlergesetzes ails eirier Funktionalgleichung, Mathematische Zeitschrifl, 18, 185188. Proctor, J . W. (1987). Estimation of two generalized curves covering the Pearson system, Proceedings of the A S A Section on Statistical Computing, 287292. Raikov, D. (1937a). On a property of the polynomials of circle division, Matematicheskii Sbornik, 44, 379381. Raikov, D. A. (1937b). On the decomposition of the Poisson law, Doklady Akadernii Nauk SSSR, 14, 811. Raikov, D. A. (1938). On the decomposition of Gauss and Poisson Laws, Izvestia Akademii Nauk SSSR, Serija MatematiCeskie, 2, 91124.
BIBLIOGRAPHY
293
Rao, C. R. (1965). On discrete distributions arising out of methods of ascertainment, in Classical and Contagious Discrete Distributions (Ed., G. P. Patil), Pergamon Press, Oxford, England, pp. 320332; see also Sankhya, Series A , 27, 311324. Rao, C. R. and Rubin, H. (1964). On a characterization of the Poisson distribution, Sankhya, Series A , 26, 2955298. Sen, A. and Balakrishnan, N. (1999). Convolution of geometrics and a reliability problem, Statistics & Probability Letters, 43,421426. Seshadri, V. (1993). The Inverse Gaussian Distribution: A Case Study in Exponential Families, Oxford University Press, Oxford, England. Seshadri, V. (1998). The Inverse Gaussian Distribution: Statistical Theory and Applications, Lecture Notes in Statistics No. 137, SpringerVerlag, New York. Shepp, L. (1964). Normal functions of normal random variables, SIAM Review, 6 , 459460. Skitovitch, V. P. (1954). Linear forms of independent random variables and the normal distribution law, Izvestia Akademii Nauk SSSR, Serija MatematiEeskie, 18, 185200. Snedecor, G. W. (1934). Calculation and Interpretation of the Analysis of Variance, Collegiate Press, Ames, Iowa. Srivastava, R. C. and Srivastava, A. B. L. (1970). On a characterization of the Poisson distribution, Journal of Applied Probability, 7,497501. Stuart, A. and Ord, J. K. (1994). Kendall’s Advanced Theory of Statistics, Vol. I, Distribution Theory, 6th ed., Edward Arnold, London. “Student” (1908). On the probable error of the mean, Biometrika, 6, 125. Tong, Y. L. (1990). The Multivariate Normal Distribution, SpringerVerlag, New York. Tukey, J. W. (1962). The future of data analysis, Annals of Mathemutical Statistics, 33, 167. Verhulst, P. J. (1838). Notice sur la loi que la population suit dans sons accroissement, Cor. Mathe‘matiques et Physique, 10, 113121. Verhulst, P. J. (1845). Recherches mathkmatiques sur la loi d’accroissement de la population, Acade‘mie de Bruxelles, 18, 138. Wimmer, G. and Altmann, G. (1999). Thesaurus of Uniuaraite Discrete Probability Distributions, STAMM Verlag, Essen, Germany.
This Page Intentionally Left Blank
AUTHOR INDEX Aitchison, J., 234, 289 Altmann, G., xv, 293 Arnold, B.C., 18, 133, 289
Folks, J. L., 238, 289 Frkchet, R.M., 279, 280 Fuchs, L., 290
Ba.lakrishnan, N., xv, xvi, 68, 72, 157, 197, 203, 289, 291, 293 Bansal, N., 230, 289 Basu, A. P., 157, 289 Behboodian, J., 230, 289 Bernoulli, J., 277 Bernoulli, N., 277 Bernstein, S. N., 221, 289 Box,G. E. P., 229, 289 Brown, J . A. C., 234, 289 Burr, I. W., 277, 283, 289
Galambos, J., 281 Galton, F., 284 Gauss, C. F., 279 Gilchrist, W. G., 3, 290 Gnedenko, B. V., 280 Godambe, A. V., 98, 290 Gosset, W. S., 280, 283, 290 Govindarajulu, Z., 176, 290 Gradshteyn, I. S., 31, 290 Gupta, R. D., 270, 276, 290 Haight, F. A., 89, 291 Hamedani, G. G., 230, 289 Hartley, H. O., 283 Hastings, N., xv, 290 Helmert, F. R., 226, 280, 291
Cacoullos, T., 241, 289 Castillo, E., 18, 289 Cauchy, A.L., 278 Chhikara, R. S., 238, 289 Comrie, L. J., 283 Consul, P. C., 100, 290 Cram&, H., 219, 290 Crow, E. L., 234, 290
Johnson, N. L., xv, 98, 148, 234, 278, 280, 281, 283, 291 Jones, M. C., 242, 291 Kagan, A., 282, 285 Kemp, A. W., xv, 291 Kemp, C. D., 98, 291 Kendall, D. G., 39, 291 Key, E. S., 230, 289 Klugman, S. A., 74, 291 Kolmogorov, A. N., 281, 291 Kotz, S., xv, 98, 148, 169, 281, 290, 291 Koutras, M. V., 72, 289 Kozubowski, T. %J., 169, 291 Krasner, M., 36, 291 Kronecker, L., 290 Krysicki, W., 148, 156, 291
Darmois, D., 224, 290 David, F. N., 281 Devroye, L., 237, 290 Dirichlet, J . P. G. L., 270, 272, 278, 290 Douglas, J. B., 98, 290 Eggenberger, F., 101, 290 Elderton, W. P., 281 Evans, M., xv, 290 Fang, K. T., 281 Feller, W., 98, 290 Fisher, R. A., 279, 280 295
296
AUTHOR INDEX
Lancast,er, H. O., 240, 292 Laplace, P. S., 169, 282, 292 Linnik, Yu. V., 237, 282, 285, 292 Liouville, J., 276, 292 Lukacs, E., 186, 292
Rayleigh, 285 Read, C. B., 210, 292 Richards, D. St. P., 270, 276, 290 Rubin, H., 99, 293 Ryzhik, I. M., 31, 290
Marciiikiewicz, J., 214, 228, 292 Marsaglia, G., 186, 292 Maxwell, J. C., 282 Muller, M. E., 229, 289
Sarabia, J.M., 18, 289 Sen, A., 68, 293 Seshadri, V., 238, 293 Shanbhag, D. N., 285 Shepp, L., 230, 293 Shimizu, K., 234, 290 Skitovitch, V. F’., 224, 293 Snedecor, G. W., 245, 285, 286, 293 Srivasta.va, A. B. L., 99, 293 Srivastava, R. C., 99, 293 Stuart, A., xv, 293 ‘Student’, 240, 286, 293
Nevzorov, V. B., xvi Neyman, J., 283 Ng, K. W., 281
Ord, J . K., xv, 290, 293 Panjer, H. H., 74, 291 Pareto, V., 283 Pascal, B., 283 Patel, J. K., 210, 292 Patil, G. P., 98, 290, 293 Peacock, B., xv, 290 Pearson, E. S., 278, 281, 283 Pearson, K., 8, 283, 284, 292 Perks, W. F., 206, 292 Podgbrski, K., 169, 291 Poisson, S.D., 284 Pblya., G., 101, 227, 284, 290, 292 Proctor, J. W., 114, 292 Raikov, D., 36, 92, 292 Ranulac, B., 36, 291 Rao, C. It., 99, 282, 285, 293
Tippett, L. H. C., 280 Tong, Y. L., 259, 293 Tukey, J. W., 118, 286, 293 Verhulst, P. J., 197, 293 Volkmer, H., 230, 289 von Mises, R., 280 Weibull, W., 286 Welch, B. L., 281 Weldon, W. F. R., 284 Wilmot, G. E., 74, 291 Wimmer, G., xv, 293 Zhang, H., 230, 289
SUBJECT INDEX Absolutely continuous bivariate distribution, 16, 18, 19, 21 Absolutely continuous distribution, 2, 9 Actuarial analysis, 74 Arcsine distribution, 140, 149, 151156, 186, 231 Characteristic function, 154 Characterizations, 155 Decompositions, 156 Introduction, 151 Moments, 153, 154 Notations, 151, 152 Relationships with other distributions, 155 Shape characteristics, 154 Asymptotic relations, 228 Auxiliary function, 211
Moments, 141 Notations, 140 Relationships with other distributions, 149 Shape characteristics, 147 Some transformations, 141 Beta distribution of the second kind, 140, 275 Beta function, 101, 102, 118, 139 Bilateral exponential distribution, 170 Binomial distribution, 39, 46, 4961, 63, 69, 80, 83, 88, 89, 9496, 98, 99, 102, 211, 219, 228, 249, 252, 253, 255, 277 Conditional probabilities, 58, 59 Convolutions, 56 Decompositions, 56, 57 Generating function and characteristic function, 50 Introduction, 49 Maximum probabilities, 53 Mixtures, 57, 58 Moments, 5053 Notations, 49 Tail probabilities, 59 Useful representation, 50 Binomial expansion, 175 Borel set, 1, 15 Borel aalgebra, 1, 15 BoxMuller’s transformation, 229 Brownian motion, 152 Burr system of distributions, 277
Bellshaped curve, 140, 241 Bernoulli distribution, 37,4350,56, 63, 69, 72, 87, 98, 112, 2 77 Convolutions, 45, 46 Introduction, 43 Maximal values, 46, 47 Moments, 44, 45 Notations, 43, 44 Relationships with other distributions, 47, 48 Bernstein’s theorem, 221223 Beta distribution, 8, 59, 115, 117, 139149, 151, 152, 155, 156, 186, 207, 231, 245, 2722 75 Characteristic function, 147 Decompositions, 148, 149 Introduction, 139 Mode, 140
Cauchy distribution, 12, 117, 119122, 170, 230, 232, 233, 236, 278 297
298
SUBJECT INDEX
Characteristic function, 120 Convolutions, 120 Decompositions, 121 Moments, 120 Notations, 119 Stable distributions, 1 2 1 Transformations, 121 Cauchy function, 278 CauchySchwarz inequality, 8 Central limit theorem, 228 Characteristic function, 1014, 22, 23, 33, 34, 37, 40, 45, 46, 50, 56, 60, 61, 64, 74, 81, 90, 96, 97, 110, 111, 120, 121, 125, 126, 131, 132, 147, 148, 154, 158, 159, 170, 173, 181, 185, 188, 195, 201, 203206, 209, 210, 214, 216, 217, 219, 222, 223, 225, 230239, 254263, 265, 267, 278, 282 Chi distribution, 232, 282 Chsquare distribution, 180, 226, 227, 231, 239, 240, 245, 246, 280 Closed under minima, 134 CobbDouglas distribution, 234 Compositions, 250 Compound Poisson distribution, 98 Conditional distribution, 18, 20, 58, 71, 83, 94, 95, 99, 185187, 219, 220, 251253, 267, 268, 273 Coriditional expectation, 20, 21 Conditional mean, 20, 21, 273 Conditional probabilities, 58, 59, 71, 72, 94, 95 conditional variance, 268 Contour integration, 120 Convex function, 10 Convolution formula, 201 Convolutions, 34, 35, 41, 45, 46, 56, 68, 69, 76, 92, 111, 120, 164, 165, 173, 185, 217219 Correlation coefficient, 19, 20, 116, 252254, 257, 259, 265, 268, 274, 275
Correlation matrix, 19 Countable set, 16 Covariance, 19, 116, 164, 252, 255, 259, 274 Covariance matrix, 19, 253, 257, 259, 261, 263, 265 Cramkr’s result, 222 Curnulants, 216 DarmoisSkitovitch’s theorem, 224226 Decompositions, 14, 35, 36, 41, 56, 57, 69, 70, 76, 80, 92, 111, 121, 148, 149, 156, 165, 166, 174, 185, 204, 217219 Degenerate distribution, 14,36,3941, 49, 214, 216, 222, 223 Convolution, 41 Decomposition, 41 Independence, 40 Introduction, 39 Moments, 39, 40 Degenerate normal distribution, 265 Destructive process, 99 Determinant, 24, 260, 265 Diagonal matrix, 263, 264 Differential equation, 8, 210 Dirichlet distribution, 269276, 278 Derivation of Diriclllet formula, 271, 272 Dirichlet, distribution of the second kind, 275 Introduction, 269, 270 Liouville distribution, 276 Marginal distributions, 272 Marginal moments, 274 Notations, 272 Product moments, 274 Dirichlet distribution of the second kind, 275, 276 Dirichlet integral formula, 27272, 274, 276, 278 Discrete bivariate distribution, 16, 1820 Discrete random walk, 47 Discrete rectangular distribution, 29
299
SUBJECT INDEX Discrete uniform distribution, 2937, 43, 71, 107, 117 Convolutions, 34, 35 Decompositions, 35, 36 Entropy, 36 Generating function and characteristic function, 33, 34 Introduction, 29 Moments, 3033 Notations, 29, 30 Relationships with other distributions, 36, 37 Distributions Absolutely continuous biva.riate Arcsine Bernoulli Beta Bilateral exponential Binomial Bivariate normal Burr system of Cauchy Chi Chisquare CobbDouglas Compound Poisson Conditional Degenerate Degenerate normal Dirichlet Dirichlet of the second kind Discrete bivariate Discrete rectangular Discrete uniform Double exponential Doubly exponential Elliptically symmetric Erlang Exponential Extreme value F First law of Laplace FisherSnedecor Fisher’s Fisher’s z Folded Frkhettype
Gamma Gaussian GaussLaplace Generalized arcsine Generalized extreme value Generalized gamma Generalized logistic Type I Type I1 Type I11 Type IV Generalized Poisson Generalized uniform Geometric Geometric of order k Half logistic Hypergeometric Infinitely divisible Inverse Gaussian Johnson’s system of Kolmogorov LagrangianPoisson Laplace Lattice Limiting Linnik Liouville Logarithmic Loggamma Logistic Loglogistic Lognormal LogWeibull Maxwell Mixture Mult inomial Mu1t ivariat e normal Negative binomial Negative hypergeometric Noncentral Chisquare
F
t
Normal Pareto Pareto of the second kind Pascal Pearson’s family of
300
SUBJECT INDEX Poisson Poissonstoppedsum P6lya. Power Rayleigh Rectangular Restricted generalized Poisson Runrelated
SB SU Sechsquared Second law of Laplace Snedecor Stable Stuttering Poisson Symmct ric symmetric uniform t Triangular Trinomial Truncated Tukey’s lambda Tukey’s symmetric lambda Type VI Type VII Twotailed exponential Uniform Weibulltype Weighted Double exponential distribution, 170, 191 Doubly exponential distribution, 191 Elementary events, 15 Elliptjicallysymmetric distributions, 281 Entropy, 8, 9, 36, 45, 70, 110, 131, 137, 162, 172, 211 Erlang distribution, 39 Euclidean space, 15 Euler’s constant, 195 Euler’s formula, 204 Expectation, 5, 7, 50, 109 Exponential decay, 160 Exponential distribution, 39, 113, 114, 117, 157167, 169, 170, 174, 175, 177, 179, 180, 186, 187, 189, 192, 194,
229, 230, 235, 236, 269, 271 Convolutions, 164, 165 Decompositions, 165, 166 Distributions of minima, 162 Entropy, 162 Introduction, 157 Lack of memory property, 167 Laplace transform and characteristic function, 158, 159 Moments, 159, 160 Notations, 157, 158 Shape characteristics, 160 Uniform and exponential order statistics, 163, 164 Exponentially decreasing tails, 198, 199 Extremes, 280 Extreme value distributions, 170, 189197,201,204,206,228 Generalized extreme value distributions, 193, 194 Introduction, 189, 190 Limiting distributions of maximal values, 190, 191 Limiting distributions of minimal values, 191 Moments, 194196 Relationships between extreme value distributions, 191
F distribution, 245, 246, 279, 285 Fermat’s last theorem, 278 First law of Laplace, 170 FisherSnedecor distribution, 286 Fisher’s distribution, 240 Fisher’s zdistribution, 279 Folded distribution, 176 Fourier transform, 278 Fractional part, 166 FrGchettype distribution, 191, 279 F test, 246 Functional equation, 18, 190 Gamma distribution, 92, 165, 179188, 206, 207, 208, 228, 231, 239, 271, 272
SUBJECT INDEX Conditional distributions and independence, 185, 187 Convolutions and decompositions, 185 Introduction, 179 Laplace transform and characteristic function, 181 Limiting distributions, 187, 188 Mode, 180 Moments, 181, 182 Notations, 180 Shape characteristics, 182 Gamma function, 75,179, 195,204, 2 70 Gaussian distribution, 211 Gaussian law, 211 GaussLaplace distribution, 2 11 Generalized arcsine distribution, 140, 155 Generalized extreme value distributions, 193, 194 Generalized gamma distribution, 184 Generalized logistic distributions, 205207 Type I, 205, 206 Type 11, 205, 206 Type 111, 206, 207 Type IV, 206, 207 Generalized Poisson distribution, 98, 100 Generalized uniform distribution, 114 Generating function, 10, 11, 13, 22, 24, 33, 34, 36, 50, 64, 68, 69, 7274, 81, 84, 90, 92, 96, 98, 99, 254, 255 Geodesy, 280 Geometric distribution, 39, 48, 6375, 80, 102, 166 Conditional probabilities, 71, 72 Convolutions, 68, 69 Decompositions, 69, 70 Entropy, 70 Generating function and characteristic function, 64 Geometric distribution of order k , 72 Introduction, 63
30 I
Moments, 6467 Notations, 63 Tail probabilities, 64 Goodnessoffit, 281 Half logistic distribution, 203, 204 Heavytailed distribution, 286 Helmert’s transformation, 226, 227, 280 Hypergeometric distribution, 59, 71, 8389, 102 Characteristic function, 84 Generating function, 84 Introduction, 83 Limiting distributions, 88 Moments, 8487 Notations, 83, 84 Hypergeometric function, 33, 84, 148. 154 Identity matrix, 264 Incomplete beta function, 139 Indecomposable, 14, 35, 36, 156 Independent random variables, 16, 17, 2123, 40, 128, 135, 148, 152, 156, 162, 166, 185187, 220229, 263,264, 270 Inferential problems, 180, 240, 241 Infinitely divisible characteristic function, 14 Infinitely divisible distribution, 14, 57, 70, 80, 92, 166, 174, 185, 219, 233 Integer part, 166 Integrated Cauchy functional equation, 285 Interarrival times, 39 Inverse distribution function, 112 Inverse Fourier transform, 170, 209 Inverse Gaussian distribution, 237, 238 Inverse transformation, 24 Jacobian, 24,25, 186,229, 267,269271, 275 Johnson’s system of, 234, 281 Joint characteristic function, 221, 222, 225
302
SUBJECT INDEX
Joint probability density function, 17, 18, 115, 116, 185187, 226, 227, 229, 270, 271 Kolmogorov distribution, 281 KolmogorovSmirnov test, 281 Kurtosis, 7, 8, 44, 45, 52, 53, 66, 76, 91, 110, 125, 129, 147, 154, 160, 172, 182, 201, 238, 241, 283 Lack of memory property, 71, 167 LagrangianPoisson distribution, 100 Laplace distribution, 165, 169177, 191, 204, 214, 236, 282 Characteristic function, 170 Convolutions, 173 Decompositions, 174 Entropy, 172 Introduction, 169 Moments, 171 Notations, 169, 170 Shape characteristics, 172 Laplace transform, 158, 159, 164, 179, 181 Lattice distribution, 29 LauRao characterization theorem, 285 Law of large numbers, 277 Least squares method, 211 Leptokurtic distribution, 8, 53, 66, 76, 91, 147, 160, 172, 182, 201, 238, 241 Lifetime data analysis, 184, 196 Lighttailed distribution, 286 Limiting distribution, 37, 39, 5961, 81, 88, 96, 182, 187193,209,228, 256259, 280, 286 Limit, process, 152 Limit theorems, 256, 257 Linear combinations, 220228, 262, 263 Linear dependence, 265 Linear regression, 268 Linnik distribution, 235237, 282 Liouville integral formula, 276
Location parameter, 4, 6, 30, 127, 134, 158, 169, 184, 190, 198, 211, 216, 220, 272 Logarithmic distribution, 99 Loggamma distribution, 184, 207 Logistic distribution, 197207, 277 Characteristic function, 201 Decompositions, 204 Generalized logistic distributions, 205207 Introduction, 197 Moments, 199201 Notations, 197, 198 Order statistics, 205 Relationships with other distributions, 203 Shape characteristics, 201 Loglogistic distribution, 277 Lognormal distribution, 233, 234, 281 LogWeibull distribution, 191 Marginal distribution function, 1618, 21, 250252, 266, 273 Maxima, 46,47, 113, 114, 127, 128, 189191, 193, 228, 280 Maximal sum, 152 Maxwell distribution, 232, 282 Mean, 5, 8, 30, 72, 100, 103, 115, 118, 153, 154, 192, 203, 205, 211, 216, 217, 220, 233, 234, 239, 241, 245, 246, 255 Mean vector, 19, 257, 261, 265 Measurable function, 20, 21 Mesokurtic distribution, 8, 53, 147 Minima, 113, 114, 134, 162, 189191, 286 Mixture distribution, 57, 58, 9699 Moment generating function, 12, 184 Moments, 4, 7, 8, 12, 3033, 39, 40, 44, 45, 5053, 6467, 7476, 8487,90,91, 108110, 118, 120, 124, 125, 129, 136, 137, 141146, 153155, 159, 160, 171, 174, 175, 181, 182, 184, 186, 194196, 199201, 206, 215,216,
SUBJECT INDEX
228, 234, 237, 241, 245, 252254 about p , 125 about zero, 46, 30, 44, 50, 52, 65, 75, 87, 91, 108, 124, 129, 136, 145, 153, 155, 159, 171, 181, 199, 200, 215, 216 Central, 5, 6, 8, 30, 44, 50, 52, 66, 76, 91, 109, 124, 125, 136, 137, 145, 153, 155, 160, 171, 182, 200, 216 Factorial, 6, 7, 10, 30, 31, 51, 64, 65, 74, 75, 86, 90 Marginal, 274 of negative order, 146, 181 Product, 19, 176, 274 Secondorder, 260, 261 Monte Carlo evaluation, 286 Multinomial distribution, 249257, 259 Compositions, 250 Conditional distributions, 251, 252 Generating function and characteristic function, 254, 255 Introduction, 249 Limit theorems, 256, 257 Marginal distributions, 250, 251 Moments, 252254 Notations, 250 Multinomial trials, 249251 Multiple regression, 254 Multivariate normal distribution, 257, 259268 Bivariate normal distribution, 265268 Distributions of sums, 262 Independence of components, 263 Introduction, 259, 260 Linear combinations of components, 262, 263 Linear transformations, 264, 265 Marginal distributions, 262 Notations, 260, 261 Negative binomial distribution, 69,
303
70, 7381, 89, 102, 209, 228 Convolutions and decompositions, 7680 Generating function and characteristic function, 74 Introduction, 73, 74 Limiting distributions, 81 Moments, 7476 Notations, 74 Tail probabilities, 80 Negative hypergeometric distribution, 102, 103 NeymanPearson approach, 283 Noncentral distributions, 246 Chisquare, 246 F , 246 t , 246 Noncentrality parameter, 246 Normal distribution, 8, 39, 53, 61, 76, 81, 91, 96, 170, 182, 188, 207, 209235, 239241, 246, 257, 266, 279, 282, 286 Asymptotic relations, 228 Bernstein’s theorem, 221223 Characteristic function, 214, 215 Conditional distributions, 219, 220 Convolutions and decompositions, 217219 DarmoisSkitovitch’s theorem, 224, 226 Entropy, 211 Helmert’s transformation, 226, 227 Identity of distributions of linear combinations, 227, 228 Independence of linear combinations, 220, 221 Introduction, 209, 210 Mode, 211 Moments, 215, 216 Notations, 210, 211 Shape characteristics, 217 Tail behavior, 212214 Transformations, 229234 Normalized extremes, 189191
304
SUBJECT INDEX
Normalized sums, 228 Normalizing constants, 189191 Orderpreserving transformation, 117 Order statistics, 114117, 128, 135, 139, 140, 157, 163, 164, 174176, 187,205,226, 269 Orthogonal linear transformation, 264 Orthogonal matrix, 264 Pareto distribution, 133138, 141, 283 Distributions of minimal Values, 134, 135 Entropy, 137 Introduction, 133 Moments, 136, 137 Notations, 133 Pascal distribution, 74, 102, 283 Pearson family of distributions, 281, 283, 284 Pearson plane, 184 Platykurtic distribution, 8, 53, 110, 125, 147, 154 Poisson distribution, 39, GO, 61, 81, 88100, 179, 180, 209, 219, 228, 256, 284, 285 Conditional probabilities, 94, 95 Convolutions, 92 Decompositions, 92 Generating function and characteristic function, 90 Introduction, 89 Limiting distribution, 96 Maximal probability, 95 Mixtures, 9699 Moments, 90, 91 Notations, 89 RaoRubin characterization, 99 Tail probabilities, 91, 92 Poissonstoppedsum distribution, 98100 Polar coordinates, 229 Polar transformation, 229 P6lya criterion, 126 P6lya distribution, 101, 102, 284
P6lya urn scheme, 102, 103 Power, 246 Power distribution, 127133, 135, 136, 140, 147, 149 Characteristic function, 131 Distributions of maximal Values, 128 Entropy, 131 Introduction, 127 Moments, 129 Notations, 127 Probability integral tmnsform, 112, 155, 203 Probability measure, 1, 15 Probability space, 1, 3, 15 Quadratic form, 259261 Quantile density function, 3 Quantile function, 3, 242 Queueing theory, 39 Random number generation, 30, 112 Random processes, 152 RaoRubin characterization, 99, 285 Rayleigh distribution, 192, 229, 232, 277, 285 Rectangular distribution, 108 Recurrence relation, 131, 160, 175177, 180 Regression, 21, 253, 268 Reliability problems, 196 Reproductive property, 231 Restricted generalized Poisson distribution, 100 Reverse Jshaped, 160 Riemann zeta function, 200 Robustness, 286 Runrelated distributions, 72
Sg distribution, 281 Su distribution, 281
Same type of distribution, 4, 6, 30 Sample mean, 225, 238, 240, 246 Sample variance, 225 Scale parameter, 4, 30, 127, 134, 158, 169, 173, 184, 185, 190, 198, 211, 216, 272 Sechsquared distribution, 198
SUBJECT INDEX Second law of Laplace, 211 Service times, 39 Shape parameter, 127,134,182,184, 286 Shepp’s result, 230 aalgebra, 1, 15 Skewed distribution, 7, 8, 66, 76, 91, 125, 147, 160, 182,206, 207, 238 Negatively, 8, 125, 147, 206, 207 Positively, 8, 66, 76, 91, 125, 147, 160, 182, 206, 238 Skewness, 7, 8, 44, 52, 53, 66, 76, 91, 110, 125, 129, 147, 154, 160, 172, 182, 201, 238, 241, 283 Snedecor distribution, 286 Spacings, 146, 163 Stable distribution, 14, 121, 214, 233, 236 Standard deviation, 233 Statistical modelling, 3 Stuttering Poisson distribution, 98 Symmetric distribution, 8, 44, 140, 154, 169, 171173, 176, 201, 205, 207, 241 Symmetric positive definite matrix, 259, 260 Symmetric uniform distribution, 110, 126 Tail probabilities, 59, 64, 80, 91, 92
t distribution, 8, 232, 240245, 280,
286 Theorem of total probability, 57 Theory of errors, 211 Transformation, 24, 25, 121, 140, 141, 151, 157, 170, 210, 229234,264,265, 269273, 275, 286 Triangular distribution, 111, 117, 123126 Characteristic function, 125, 126 Introduction, 123 Moments, 124, 125 Notations, 123 Trinomial distribution, 250
305
Truncated distribution, 204 t test, 246, 280 Tukey’s lambda distribution, 118 Tukey’s symmetric lambda distribution, 118, 286 Twotailed exponential distribution, 170 Type I11 line, 184 Type VI distribution, 245 Type VII distribution, 240 Uniform distribution, 37, 107119, 123, 127, 128, 131, 133, 136, 139, 140, 147, 149, 163, 164, 186, 187, 189, 203, 229, 231, 269, 270, 277, 286 Characteristic function, 110 Convolutions, 111 Decompositions, 111 Distributions of niinima and rnaxima, 113 Entropy, 110 Introduction, 107 Moments, 108 Notations, 107 Order statistics, 114 117 Probability integral transform, 112 Relationships with other distributions, 117, 118 Uniform spacings, 146, 269, 270 Unimodal distribution, 7 Urn interpretation, 84 Ushaped curve, 141, 151, 154 Variance, 5, 7, 20, 30, 40, 44, 50, 52, 72, 100, 103, 109, 115, 118, 124, 129, 137, 145, 154, 160, 164, 171, 182, 192, 203, 205, 211, 216, 217, 220, 227, 234, 239, 241, 245, 246, 255, 259, 268 Weibulltype distribution, 191, 229, 287 Weighted distribution, 285