Linear models: A mean model approach

Preface

Linear models is a broad and diversified subject area. Because the subject area is so vast, no attempt was made in this text to cover all possible linear models topics. Rather, the objective of this book is to cover a series of introductory topics that give a student a solid foundation in the study of linear models. The text is intended for graduate students who are interested in linear statistical modeling. It has been my experience that students in this group enter a linear models course with some exposure to mathematical statistics, linear algebra, normal distribution theory, linear regression, and design of experiments. The attempt here is to build on that experience and to develop these subject areas within the linear models framework. The early chapters of the text concentrate on the linear algebra and normal distribution theory needed for a linear models study. Examples of experiments with complete, balanced designs are introduced early in the text to give the student a familiar foundation on which to build. Chapter 4 of the text concentrates entirely on complete, balanced models. This early dedication to complete, balanced models is intentional. It has been my experience that students are generally more comfortable learning structured material. Therefore, the structured rules that apply to complete, balanced designs give the student a set of learnable tools on which to

xi

xii

Preface

build confidence. Later chapters of the text then expand the discussion to more complicated incomplete, unbalanced and mixed models. The same tools learned for the balanced, complete models are simply expanded to apply to these more complicated cases. The hope is that the text progresses in an orderly manner with one topic building on the next. I thank all the people who contributed to this text. First, I thank Virgil Anderson for introducing me to the wonders of statistics. Special thanks go to Julie Sawyer, Laura Coombs, and David Weeks. Julie and Laura helped edit the text. Julie also contributed heavily to the development of Chapters 4 and 8. David has generally served as a sounding board during the writing process. He has listened to my ideas and contributed many of his own. Finally, and most importantly, I thank my wife, Diane, for her generosity and support.

1

Linear Algebra and Related Introductory Topics

A summary of relevant linear algebra concepts is presented in this chapter. Throughout the text boldfaced letters such as A, U, T, X, Y, t, g, u are used to represent matrices and vectors, italicized capital letters such as Y, U, T, E, F are used to represent random variables, and lowercase italicized letters such as r, s, t, n, c are used as constants.

1.1

ELEMENTARY

MATRIX CONCEPTS

The following list of definitions provides a brief summary of some useful matrix operations.

Matrix: An r • s matrix A is a rectangular array of elements with r rows and s columns. An r x 1 vector Y is a matrix with r rows and 1 column. Matrix elements are restricted to real numbers throughout the text.

Definition 1.1.1

Transpose: If A is an n x s matrix, then the transpose of A, denoted by A', is an s x n matrix formed by interchanging the rows and columns of A. Definition 1.1.2

2

Linear Models

Definition

1.1.3 Identity Matrix, Matrix of Ones and Zeros: In represents an n x n identity matrix, Jn is an n x n matrix of ones, In is an n x 1 vector of ones, and 0m • is an m x n matrix of zeros. Definition 1.1.4 Multiplication of Matrices: Let aij represent the ij th e l e m e n t of an r x s matrix A with i = 1 . . . . . r rows and j = 1 . . . . . s columns. Likewise, let bjk represent the j k th element of an s x t matrix B with j = 1 . . . . . s rows and k - 1 . . . . . t columns. The matrix multiplication of A and B is represented by A B = C where C is an r x t matrix whose ik th element C i k - aijbjk. If the r x s matrix A is multiplied by a scalar d, then the resulting r x s matrix d A has ij th element daij.

~--~j=l

Example 1.1.1

The following matrix multiplications c o m m o n l y occur. l'nln = n lnl'n

= Jn

JnJn = nJn

lln (In-ljn):Olxn

J: (In- 1J,,)=On• (in-1Jn) (I,,-1Jn) = (in-1jn). Definition 1.1.5 Addition of Matrices" The sum of two r x s matrices A and B is represented by A + B = C where C is the r x s matrix whose ij th element

Cij : aij 9 bij. Definition 1.1.6 Inverse of a Matrix: An n x n matrix A has an inverse if A A -1 -- A - 1 A = In where the n x n inverse matrix is denoted by A -1 . Definition 1.1.7 Singularity: If an n • n matrix A has an inverse then A is a nonsingular matrix. If A does not have an inverse then A is a singular matrix.

i th

Definition 1.1.8 Diagonal Matrix: Let aii be the diagonal element of an n • n matrix A. Let aij be the ij th off-diagonal element of A for i :/: j . Then A is a diagonal matrix if all the off-diagonal elements aij equal zero.

Definition1.1.9

Trace of a Square Matrix: The trace of an n x n matrix A, denoted by tr(A), is the sum of the diagonal elements of A. That is, tr(A) = a//

1

Linear Algebra

3

It is assumed that the reader is familiar with the definition of the determinant of a square matrix. Therefore, a rigorous definition is omitted. The next definition actually provides the notation used for a determinant.

Definition 1.1.10

Determinant of a Square Matrix: Let det(A) = IAI denote the determinant of an n x n matrix A. Note det(A) = 0 if A is singular.

Definition 1.1.11

Symmetric Matrix: An n x n matrix A is symmetric if A = A'.

Definition 1.1.12

Linear Dependence and the Rank of a Matrix: Let A be an n x s matrix (s _< n) where al . . . . . as represent the s n x 1 column vectors of A. The s vectors al . . . . . as are linearly dependent provided there exists s elements kl . . . . . ks, not all zero, such that klal + . - . + ksas - 0. Otherwise, the s vectors are linearly independent. Furthermore, if there are exactly r _< s vectors of the set al . . . . . as which are linearly independent, while the remaining s - r can be expressed as a linear combination of these r vectors, then the rank of A, denoted by rank (A), is r. The following list shows the results of the preceding definitions and are stated without proof:

Result 1.1:

Let A and B each be n x n nonsingular matrices. Then (AB)

Result 1.2:

-1

-- B-1A-1.

Let A and B be any two matrices such that AB is defined. Then (AB)' = B'A'.

Result 1.3:

Let A be any matrix. The A'A and AA' are symmetric.

Result 1.4:

Let A and B each be n x n matrices. Then det(AB) = [det(A)] [det(B)].

Result 1.5:

Let A and B be m x n and n x m matrices, respectively. Then tr(AB) = tr(BA).

Quadratic forms play a key role in linear model theory. The following definitions introduce quadratic forms.

Definition 1.1.13 Quadratic Forms: A function f(Xl . . . . . Xn) is a quadratic form if f (xl ' .. " , xn) = ~~i~=l ~ ~j = l aijxixj - - X ' A X where X = ( x 1 . . . . . Xn)' is an n x 1 vector and A is an n x n symmetric matrix whose ij th element is aij.

4

Linear Models

Example 1.1.2

Let f (xl, x2, X3) : X 2 -Jr-3X~ + 4X~ + XlX2 + 2X2X3. Then f ( x l , x2, x3) = X'AX is a quadratic form with X = (Xl, x2, x3)' and

A

1 0.5 0

_._

0.5 3 1

0] 1 . 4

The symmetric matrix A is constructed by setting coefficient on the xixj term for i # j.

Example 1.1.3

aij

and a j i equal to one-half the

Quadratic forms are very useful for defining sums of squares.

For example, let

f ( x l . . . . . Xn) -- ~

x 2 -- X'X : X'InX

i=1

where the n • 1 vector X = (xl . . . . . X n ) mean is another common example. Let

t.

The sum of squares around the sample

n

f (xl . . . . .

Xn)

:

Z(xi

-

~-)2

i=1

= s

x 2 _ n2 -2

i=1

- - __ n

Xi i=l

Xi i=l

= X ' X _ _1 (X, ln)(1,nX) n

= X' [In -- -nlIn l'n] x

Definition 1.1.14 Orthogonal Matrix: An n x n matrix P is orthogonal if and only if p-1 _ p,. Therefore, PP' = P'P = In. If P is written as (Pl, P2 . . . . . Pn) where Pi is an n x 1 column vector of P for i - 1 . . . . . n, then necessary and sufficient conditions for P to be orthogonal are (i)

P'iPi = 1 for each i = 1. . . . . n and

(ii)

p'~pj = 0 for any i # j.

1

Linear Algebra

E x a m p l e 1.1.4

V

5

Let the n x n matrix - l l v/-d

11ff2

11,vi-6

...

l l,ff n (n - 1)

l l,viJ

-11ff2

11,ff-6

...

l l ~/n (n - 1)

llU'J

0

-21J-6

...

lldn(n-

1)

l l,vi-d

0

0

...

l l d n (n -

l)

. l l ff-ff

0

0

....

'

~/ (n - 1 ) I n

where PP' = P'P = In. The columns of P are created as follows: Pl = ( 1 / r P2 -- ( 1 / r

1 2 + " ' - q - 12) (1, 1, 1,1 . . . . . 1 > ' = (1/V/'n)In -q-(-1) 2) ( 1 , - 1 , 0, 0 . . . . . 0)'

P3 = ( l / v / 1 2 -4- l a + (-2> a) (1, 1 , - 2 , 0 . . . . . 0>'

Pn = ( l / v / 1 2 -t- 12 + " "

+ 12 + ( - ( n - 1)) 2) (1, 1 . . . . .

1,-(n - 1))'.

The matrix P' in Example 1.1.4 is generally referred to as an n-dimensional Helmert matrix. The Helmert matrix has some interesting properties. Write P as P -- (pl[Pn) where the n x 1 vector Pl = (1/~/~)ln and the n x (n - 1) matrix Pn -" (P2, P3 . . . . . Pn) then PIP; = 1 j n n

P'lPi = 1 P'IPn = O l x ( n - 1 )

n

P'n Pn = In- 1. The (n - 1) x n matrix P'n will be referred to as the lower portion of an n-dimensional Helmert matrix. If X is an n x 1 vector and A is an n x n matrix, then AX defines n linear combinations of the elements of X. Such transformations from X to AX are very useful in linear models. Of particular interest are transformations of the vector X that produce multiples of X. That is, we are interested in transformations that

6

Linear

Models

satisfy the relationship AX = ZX where ~. is a scalar multiple. The above relationship holds if and only if IzI~ - AI - 0. But the determinant of ZIn - A is an n th degree polynomial in ~.. Thus, there are exactly n values of ~. that satisfy I~.In - AI -- 0. These n values of ~. are called the n eigenvalues of the matrix A. They are denoted by Zl, ~.2. . . . . ~.~. Corresponding to each eigenvalue ~.i there is an n x 1 vector X i that satisfies AXi

--

)~iXi

where Xi is called the i th eigenvector of the matrix A corresponding to the eigenvalue Zi.

Example 1.1.5 Find the eigenvalues and vectors of the 3 • 3 matrix A = 0.613 + 0.4J3. First, set IZI3 - AI - 0. This relationship produces the cubic equation ,k3 - 3~ 2 + 2 . 5 2 ~ . - 0.648 = 0 or

(~. -

1.8)()~ -

0.6)(,k -

0 . 6 ) = 0.

Therefore, the eigenvalues of A are ~.1 = 1.8, Z2 = )~3 = 0.6. Next, find vectors Xi that satisfy (A - ~ . i I 3 ) X i - - 0 3 x l for each i = 1, 2, 3. For Zl = 1.8, (A - 1.813)X1 = 03xl or (-1.213 + 0.4J3)X1 = 03xl. The vector X1 = (1/~/3)13 satisfies this relationship. For ~.2 = ~.3 = 0.6, (A - 0 . 6 1 3 ) X i = 03,,1 or l~Xi - 03xl f o r / = 2,3. The vectors X2 = ( 1 / ~ / 2 , - 1 / ~ / 2 , 0)' and X3 = (1/~/-6, 1/~/6, -2/~/-6)' satisfy this condition. Note that vectors X1, X2, X3 are normalized and orthogonal since X~X1 = X~X2 = X~X3 = 1 and X~X2 = x

x3 = x

x3 = 0.

The following theorems address the uniqueness or nonuniqueness of the eigenvector associated with each eigenvalue. 1.1.1 eigenvalue.

Theorem

There exists at least one eigenvector corresponding to each

1.1.2 If an n • n matrix A has n distinct eigenvalues, then there exist exactly n linearly independent eigenvectors, one associated with each eigenvalue. Theorem

In the next theorem and corollary a symmetric matrix is defined in terms of its eigenvalues and eigenvectors.

1

Linear Algebra

7

Theorem 1.1.3

Let A be an n x n symmetric matrix. There exists an n x n orthogonal matrix P such that P'AP = D where D is a diagonal matrix whose diagonal elements are the eigenvalues o f A and where the columns o f P are the orthogonal, normalized eigenvectors o f A. The i th column o f P (i.e., the i th eigenvectors o f A) corresponds to the i th diagonal element o f D f o r i = 1 . . . . . n. E x a m p l e 1.1.6

Let A be the 3 x 3 matrix from Example 1.1.5. Then P'AP = D or

1/~/~

-1/~/~

l/v/6

1/~/~

1,0111 04 04111, -2/vr6J

[108o o0] 0

0.4

1

0.4

l/~d/3

-lIVe2

0.4

0.4

1

1/~/r3

0

1, l 1/~/6 / -2/~/~J

0

Theorem 1.1.3 can be used to relate the trace and determinant of a symmetric matrix to its eigenvalues. Theorem 1.1.4 If A is an n x n symmetric matrix then tr(A) = E i L 1 )~i and det(A) = 1-Iinl ~.i. Proof:

Let P'AP - D then tr(A) = tr(APP') - tr(P'AP) = tr(D) = ~

)~i.

i=1

det(A) = det(APP') = det(P'AP) = det(D)

f i ~,i. i=l

The number of times an eigenvalue occurs is the multiplicity of the value. This idea is formalized in the next definition.

Definition 1.1.15

Multiplicity: The n x n matrix A has eigenvalue ~.* with multiplicity m _< n if m of the eigenvalues of A equal )c*.

E x a m p l e 1.1.7 All the n eigenvalues of the identity matrix In equal 1. Therefore, In has eigenvalue 1 with multiplicity n. E x a m p l e 1.1.8 Find the eigenvalues and eigenvectors of the n x n matrix G = (a - b)In + bJn. First, note that

G[(1/C~)I,,] = [a + (n - 1)b](1/C"n)ln.

8

Linear Models

Therefore, a + (n - 1)b is an eigenvalue of matrix G with corresponding n o r m a l i z e d e i g e n v e c t o r (l~/-ff)ln. Next, take any n x 1 vector X such that l'nX = 0. (One set of n - 1 vectors that satisfies l'nX = 0 are the c o l u m n vectors P2, P3 . . . . . Pn f r o m E x a m p l e 1.1.4.) Rewrite G = (a - b ) l n + b l n l t n . Therefore, G X = (a - b ) X + b l n l ' n X -

(a - b ) X

and matrix G has eigenvalue a - b. Furthermore, I~.In - GI -

I(Z - a + b)In - bJnl

= [a + (n - 1)b](a

- b) n-1.

Therefore, eigenvalue a + (n - 1)b has multiplicity 1 and eigenvalue a - b has multiplicity n - 1. Note that the 3 x 3 matrix A in E x a m p l e 1.1.5 is a special case of matrix G with a = 1, b = 0.4, and n = 3. It will be convenient at times to separate a matrix into its submatrix c o m p o n e n t s . Such a separation is called partitioning.

Definition 1.1.16

P a r t i t i o n i n g a M a t r i x : If A is an m x n matrix then A can be

separated or partitioned as

A=

JAil A12 1 A21 A22

w h e r e Aij is an m i x n j matrix for i, j = 1, 2, m = rnl +

m2 and

n

=nm + n2.

M o s t of the square matrices used in this text are either positive definite or positive semidefinite. These two general matrix types are described in the following definitions.

Definition 1.1.17

P o s i t i v e S e m i d e f i n i t e M a t r i x : An n • n matrix A is positive

semidefinite if

(i) (ii) (iii)

A = A', Y ' A Y >_ 0 for all n • 1 real vectors Y, and Y ' A Y = 0 for at least one n • 1 n o n z e r o real vector Y. The matrix J,, is positive semidefinite because Jn = J',,, Y ' J n Y =

Example 1.1.9

(ltnY)t(ltn Y) = (y~in=l yi) 2 >__0 for Y = (Yl . . . . . Yn)t and Y'JnY -: 0 for Y = (1,-1,o

.....

o)'.

Definition 1.1.18 nite if

P o s i t i v e D e f i n i t e M a t r i x : An n x n matrix A is positive deft-

1

Linear Algebra

(i) (ii)

9

A = A' and Y'AY > 0 for all nonzero n x 1 real vectors Y.

E x a m p l e 1.1.10 The n x n identity matrix In is positive definite because In is symmetric and Y'InY > 0 for all nonzero n x 1 real vectors Y. Theorem

(i) (ii)

1.1.5

Let A be an n x n positive definite matrix. Then

there exists an n x n matrix B o f rank n such that A = BB' and the eigenvalues o f A are all positive.

The following example demonstrates how the matrix B in Theorem 1.1.5 can be constructed. E x a m p l e 1.1.11 Let A be an n x n positive definite matrix. Thus, A = A' and by Theorem 1.1.3 there exists n x n matrices P and D such that f l A P = D where P is the orthogonal matrix whose columns are the eigenvectors of A, and D is the corresponding diagonal matrix of eigenvalues. Therefore, A = PDP' = pD1/ED1/Ep ' = BB' where D 1/2 is an n x n diagonal matrix whose i th diagonal element is ~,]/2 and B = PD 1/2.

Certain square matrices have the characteristic that A 2 = A. For example, let 1 A = In - g Jn. Then

A2= (In-lJn) 2-"

(In-lJn)n (In-lJn)n

= I n -- 1 j n -- _Jn 1 + 1jn n n n

"-'A. Matrices of this type are introduced in the next definition. Definition 1.1.19 (i) (ii)

Idempotent Matrices: Let A be an n x n matrix. Then

A is idempotent if A 2 = A and A is symmetric, idempotent if A = A 2 and A = A'.

Note that if A is idempotent of rank n then A - In. In linear model applications, idempotent matrices generally occur in the context of quadratic forms. Since the matrix in a quadratic form is symmetric, we generally restrict our attention to symmetric, idempotent matrices.

10

Linear Models

Theorem 1.1.6

Let B be an n x n symmetric, idempotent matrix o f rank r < n. Then B is positive semidefinite.

The next t h e o r e m will prove useful w h e n e x a m i n i n g sums of squares in A N O V A problems.

Theorem 1.1.7 ns f o r s (i)

1,..

~-

ArAs

=

"'

Let A1. . . . . A m be n x n symmetric matrices where rank (As) = m and ~ s =m l A s • In. I7 ~ s = m l n s __ n, then f o r r 5~ s, r, s = 1 . . . . . m a n d

On x n

As = A2 f o r s = 1 . . . . .

(ii)

m.

The eigenvalues of the matrix In - 1j~ are derived in the next example. n E x a m p l e 1.1.12 (a-

The symmetric, i d e m p o t e n t matrix In -

b)In + bJn with a = 1 - n1 and b -

n1 J~ takes the form

- f i1" Therefore, by E x a m p l e 1 1.8, the

1

1

1

eigenvalues of In - n Jn are a + (n - 1)b = (1 - fi) + (n - 1 ) ( - ~ )

= 0 with

multiplicity 1 and a - b = (1 - 1) _ ( _ 1) = 1 with multiplicity n - 1. The result that the eigenvalues of an i d e m p o t e n t matrix are all zeros and ones is g e n e r a l i z e d in the next theorem.

Theorem 1.1.8 The eigenvalues o f an n x n symmetric matrix A o f rank r < n are 1, multiplicity r and O, multiplicity n - r if a n d only if A is idempotent. Proof:

The p r o o f is given by Graybill (1976, p. 39).

II

The following t h e o r e m relates the trace and the rank of a symmetric, i d e m p o t e n t matrix.

Theorem 1.1.9

I f A is an n x n symmetric, idempotent matrix then tr(A) =

rank(A). Proof:

The p r o o f is left to the reader.

II 1

In the next e x a m p l e the matrix I,, - n J,, is written as a function of n - 1 of its eigenvalues. E x a m p l e 1.1.13

1

The n • n matrix In - nJn takes the form (a - b)I,, + bJn with

a - 1 - 1n and b = - !n . By E x a m p l e s 1.1.8 and 1.1.12, the n - 1 eigenvalues equal to 1 have c o r r e s p o n d i n g eigenvectors equal to the n - 1 c o l u m n s of Pn w h e r e P'n is the (n - 1) x n lower portion of an n - d i m e n s i o n a l H e l m e r t matrix. Further, 1

In -- nJn = PnPtn 9

1

Linear Algebra

11

This representation of a symmetric idempotent matrix is generalized in the next theorem.

Theorem 1.1.10

/ f A is an n x n symmetric, idempotent matrix o f rank r then A = PP' where P is an n x r matrix whose columns are the eigenvectors o f A associated with the r eigenvalues equal to 1. Proof:

Let A be an n x n symmetric, idempotent matrix of rank r. By Theorem

1.1.3, R'AR=

[ Ir0 00]

where R = [PIQ] is the n x n matrix of eigenvectors of A, P is the n x r matrix whose r columns are the eigenvectors associated with the r eigenvalues 1, and Q is the n x (n - r) matrix of eigenvectors associated with the (n - r) eigenvalues 0. Therefore, A=[PQ]

[ lr

0

0O] [ PQ'' 1 = P P "

Furthermore, P'P = lr because P is an n x r matrix oforthogonal eigenvectors.

I

If X'AX is a quadratic form with n x 1 vector X and n x n symmetric matrix A, then X'AX is a quadratic form constructed from an n-dimensional vector. The following example uses Theorem 1.1. l0 to show that if A is an n x n symmetric, idempotent matrix of rank r < n then the quadratic form X'AX can be rewritten as a quadratic form constructed from an r-dimensional vector. Let X'AX be a quadratic form with n x 1 vector X and n x n symmetric, idempotent matrix A of rank r < n. By Theorem 1.1.10, X'AX = X'PP'X = Z ' Z where P is an n x r matrix of eigenvectors of A associated with the eigenvalues 1 and Z = P'X is an r x 1 vector. For a more specific example, note that X ' ( I n - n1 J n ) X = X ' P n P ' n X = Z'Z where P',, is the (n - 1) x n lower portion of an n-dimensional Helmert matrix and Z = P'n X is an (n - 1) • 1 vector.

E x a m p l e 1.1.14

Later sections of the text cover covariance matrices and quadratic forms within the context of complete, balanced data structures. A data set is complete if all combinations of the levels of the factors contain data. A data set is balanced if the number of observations in each level of any factor is constant. Kronecker product notation will prove very useful when discussing covariance matrices and quadratic forms for balanced data structures. The next section of the text therefore provides some useful Kronecker product results.

12 1.2

Linear M o d e l s KRONECKER

PRODUCTS

Kronecker products will be used extensively in this text. In this section the Kronecker product operation is defined and a number of related theorems are listed without proof 9

Definition 1.2.1 Kronecker Product: If A is an r x s matrix with i j th element aij for i = 1 . . . . . r and j = 1 . . . . . s, and B is any t x v matrix, then the Kronecker product of A and B, denoted by A | B, is the rt x s v matrix formed by multiplying each aij element by the entire matrix B. That is, allB

al2B

...

alsB

a21B

a22B

...

a2sB

9

.

9149

9

arzB

""

araB

A| arlB

T h e o r e m 1.2.1

Let A and B be any matrices. Then (A | B)' = A' | B'.

Example 1.2.1

[1 a | (2, 1, 4)1' = 1'a |

(2, 1, 4)'.

T h e o r e m 1.2.2 Let A, B, and C be any matrices and let a be a scalar. Then aA | B | C = a(A | | C = A | (aB | C).

Example 1.2.2 T h e o r e m 1.2.3

1

1

aJa @ Jb @ Jc -- a[(Ja @ Jb) ~) Jc] = Ja | Let A and B be any square matrices 9

( abJl | Jc). Then t r ( A |

B) --

[tr(A)][tr(B)].

Example 1.2.3 (a-

1)(n-

t r [ ( I a - aJa) 1 @

(I n --

1Jn)] -- tr[(Ia -- 1Ja)]tr[(In - 1Jn)] --

1).

T h e o r e m 1.2.4

Let A be an r x s matrix, B be a t x u matrix, C be an s x v matrix, and D be a u • w matrix 9 Then (A | B)(C | D) = AC | BD.

Example 1.2.4

[la @ Jn ] [ (la -- aJa) 1 1 1 1 @ ~Jn] = Ia(Ia - aJa) @ nJnJn -- (Ia --

1Ja)|176 T h e o r e m 1.2.5

Let A and B be m x m and n x n nonsingular matrices, respectively. Then the inverse o f A | B is (A | B) -1 = A -1 | B -1.

Example 1.2.5

[(Ia + ~Ja) ~ (In + flJn)] -1 = [Ia -- (~/(1 + a u ) ) J a ] | (fl/(1 4- nfl))Jn] for 0 < a, ft.

[In --

1

Linear Algebra

T h e o r e m 1.2.6

13

Let A and B be m x n matrices and let C be a p x q matrix.

Then (A + B) |

C = (A | C) + (B | C). (Ia - - a l Ja ) @ (In --~1Jn ) = [la ~) (In _ n[ J n1) ] _ aajl | (In - -Jn)]nl __

E x a m p l e 1.2.6 1

1

la @ In -- la @ nJn - a Ja @ In

+

1 1 a Ja n Jn 9

T h e o r e m 1.2.7 Let A be an m x m matrix with eigenvalues O~1 . . . . . Olm and let B be an n x n matrix with eigenvalues ~1 . . . . . fin. Then the eigenvalues o f A | B (or B | A ) are the m n values Oti ~ j f o r i = 1 . . . . . m and j = 1 . . . . . n.

E x a m p l e 1.2.7

From Example 1.1.8, the eigenvalues of In

-

1

nJn are 1 with

1

multiplicity n - 1 and 0 with multiplicity 1. Likewise, Ia - aJa has eigenvalues 1 with multiplicity a - 1 and 0 with multiplicity 1. Therefore, the eigenvalues 1 1 of (la - aJa) @ (In -- nJn) are 1 with multiplicity (a - 1)(n - 1) and 0 with multiplicity a + n - 1. T h e o r e m 1.2.8 Let A be an m • n matrix o f rank r and let B be a p • q matrix o f rank s. Then A | B has rank rs.

E x a m p l e 1.2.8

rank[(Ia - a1Ja) |

I. ] =[rank (Ia -- a1Ja) ] [rank (In) ] : (a - 1)n.

T h e o r e m 1.2.9 Let A be an m x m symmetric, idempotent matrix o f rank r and let B be an n x n symmetric, idempotent matrix o f rank s. Then A | B is an m n x m n symmetric, idempotent matrix where t r ( A | B) = rank (A | B) = rs.

E x a m p l e 1.2.9

tr[ (I a - aJa) 1 | In] = rank[(Ia - a1J a ) |

In] = (a -- 1)n.

The following example demonstrates that Kronecker products are useful for describing sums of squares in complete, balanced ANOVA problems. Consider a one-way classification where there are r replicate observations nested in each of the t levels of a fixed factor. Let Yij represent the jth replicate observation in the i th level of the fixed factor for i = 1 . . . . . t and j -- 1 . . . . . r. Define the tr x 1 vector of observations Y (yll . . . . . Ylr . . . . . Ytl . . . . . Ytr)'. The layout for this experiment is given in Figure 1.2.1. The ANOVA table is presented in Table 1.2.1. Note that the sums of squares are written in summation notation and as quadratic forms, Y'AmY, for m = 1 . . . . . 4. The objective is to demonstrate that the tr x tr matrices Am can be expressed as Kronecker products. Each matrix Am is derived later. Note E x a m p l e 1.2.10

y.. = [1/(rt)llttr Y = [ 1 / ( r t ) ] ( l t | l r ) t y

14

Linear Models Fixed Factor (i) 2

1 Replicates (j)

Figure 1.2.1

t

Yll Y12

Y21 Y22

Ytl

Ylr

Y2r

Ytr

Yt2

One-Way Layout.

Table 1.2.1 One-Way ANOVA Table Source

df

Mean

1

r

Fixed factor

t - 1

r

~ i - i 2j=_l (yi.- y..)2 = Y'A2Y

Nested replicates

t (r - l)

~-~i=l ~-'~j=l (Yij - Yi.) 2 = Y'A3Y

Total

tr

~ i = 1 E j=1 22

SS 92

-- Y'A1 Y

r

r

-- Y'A4Y

and

(Yl . . . . . . Yr.)' =

( 1,) It |

-1 r

r

Y.

Therefore, the sum of squares due to the mean is given by

~-~~'~

y2.. = rty2..

i=1 j = l

= rt{[1/(rt)](lt

| lr)'Y}'{[1/(rt)](lt

| lr)'Y}

= Y'A1Y where the tr x tr matrix A1 -- 1Jt @ 1Jr. The sum of squares due to the fixed factor is

1

Linear Algebra

15 t

( y i . _ y..)2 = r Z

i=1 j=l

try2..

y/2. _

i=1

=Y'[It|

Jt|

=Yt[(It-~Jt)| = Y'AEY where the tr x tr matrix A2 = (it - 7Jt) 1 @ r1Jr" The sum of squares due to the nested replicates is (Yiji=l

yi.)2 :

j=l

y/~ _ r ~--~ y2. i=l

j=l

i=1

= Y'[It | I r ] Y - Y' [It | !Jrl Y

='[It (Ir l r)l = Y'A3Y where the

t r x t r matrix

A3 = It ~ (Ir -- r1Jr). Finally, the sum of squares total is

~--~ ~

y2 .__ Y'[It ~ Ir]Y : Y'A4Y

i--1 j=l

where the t r x t r matrix A4 -- It t~ Ir. The derivations of the sums of squares matrices Am can be tedious. In Chapter 4 an algorithm is provided for determining the sums of squares matrices for complete, balanced designs with any number of main effects, interactions, or nested factors. This algorithm makes the calculation of sums of squares matrices Am very simple. This section concludes with a matrix operator that will prove useful in Chapter 8. Definition 1.2.2 BIB Product: If B is a c x d matrix and A is an a x b matrix where each column of A has c _< a nonzero elements and a - c zero elements, then the BIB product of matrices A and B, denoted by A n B, is the a x bd matrix formed by multiplying each zero element in the i th column of A by a 1 x d row

16

Linear Models

vector of zeros and multiplying the jth nonzero element in the ith column of A by the jth row of B for i = 1. . . . . b and j = 1. . . . . c. Let the 3 x 3 matrix A = J3 - I3 and the 2 x 2 matrix B = 1 I2 - 2J2" Then the 3 x 6 BIB product matrix

E x a m p l e 1.2.11

ADB=

I =

0 1 1 0 1 1

1] 1 [] 0

0 1/2 -1/2

0 -1/2 1/2

1/2 -1/2 1/2 0 -1/2

-1/2] 1/2 -1/2 0 1/2

1/2 -1/2 0

-1/21 1/2 . 0

Theorem 1.2.10

I f A is an a x b matrix with c < a nonzero elements p e r c o l u m n a n d a - c zeros p e r column; B1, B2, a n d B are each c • d matrices; D is a b x b diagonal matrix o f rank b; Z is a d x 1 vector; a n d Y is a c x 1 vector, then (i) (ii)

[A D B][D | Z] = AD D BZ [A [] Y][D | Z'] = AD [] YZ'

(iii)

D9

= D|

Y'

(iv)

[A I-I(B1 + B2)] - [A El B1] + [A D B2].

Let A be any 3 • 3 matrix with two nonzero elements per 1 column and let B = I2 - 2J2. Then

E x a m p l e 1.2.12

E (')

[At2B][13|

A{2

I2-~J2

[I3 | 12][I3 | 1~]

= IA rq ( ( 1 2 - ~ J 2 ) 1 2 ) ] = [A 7] 02• :

1.3

[I3 | 1~]

@ 1~]

03x6.

RANDOM VECTORS

Let the n x 1 random vector Y = (Y1, Y2. . . . . Yn)' where Yi is a random variable for i = 1. . . . . n. The vector Y is a random entity. Therefore, Y has an expectation; each element of Y has a variance; and any two elements of Y have a covariance

1

Linear Algebra

17

(assuming the expectations, variances, and covariances exist). The following definitions and theorems describe the structure of random vectors.

Definition 1.3.1

Joint Probability Distribution: The probability distribution of the n x 1 random vector Y = ( Y 1 . . . . . Yn)' equals the joint probability distribution of Y1. . . . . Yn. Denote the distribution of Y by f v ( y ) - fv(Yl . . . . . Yn).

Definition 1.3.2

Expectation o f a R a n d o m Vector: The expected value of the n x 1 random vector Y = (Y1 . . . . . Yn)' is given by E(Y) = [E(Y1) . . . . . E(Yn)]'.

Definition 1.3.3

Covariance Matrix o f a R a n d o m Vector Y" The n x 1 random vector Y - (Y1 . . . . . Yn)' has n x n covariance matrix given by

cov(Y) - E { [ Y - E ( Y ) ] [ Y - E(Y)]'}. The i j th element of cov(Y) equals E{[Yi - E(Yi)][Yj - E(Yj)]} for i, j - 1 . . . . . n.

Definition 1.3.4

Linear Transformations o f a R a n d o m Vector Y: I f B is an m • n matrix of constants and Y is an n x 1 random vector, then the m x 1 random vector BY represents m linear transformations of Y.

The following theorem provides the covariance matrix of linear transformations of a random vector.

Theorem 1.3.1

I f B is an m x n matrix o f constants, Y is an n x 1 random vector, and cov(Y) is the n x n covariance matrix o f Y, then the m • 1 random vector BY has an m x m covariance matrix given by B[cov(Y)]B'. Proof."

cov(BY) = E { [ B Y - E(BY)][BY - E(BY)]'} = E{B[Y - E(Y)][Y - E(Y)]'B'} = B{E{[Y - E(Y)][Y - E(Y)]'}}B' = B[cov(Y)]B'.

II

The next theorem provides the expected value of a quadratic form.

Theorem 1.3.2 Let Y be an n x 1 random vector with mean v e c t o r / z = E(Y) and n x n covariance matrix ~ - cov(Y) then E(Y'AY) = t r ( A ~ ) + / z ' A # where A is any n x n symmetric matrix o f constants.

18

Linear Models

Proof:

Since (Y - / z ) ' A ( Y - / z ) is a scalar and using Result 1.5,

(Y- #)'A(Y-

tz) = t r [ ( Y - / z ) ' A ( Y - / z ) ]

= tr[A(Y-/z)(Y-/z)'].

Therefore, E[Y'AY] = E [ ( Y - / x ) ' A ( Y - / z ) = E{tr[A(Y-/z)(Y-

+ 2Y'A/z - / z ' A # ] #)']} + 2E(Y'A/x) - # ' A #

= tr[AE{ (Y - /z) (Y - /x)'}] + / x ' A # = tr[A~] + / z ' A # .

l

The moment generating function of a random vector is used extensively in the next chapter. The following definitions and theorems provide some general moment generating function results.

Definition 1.3.5 M o m e n t Generating Function ( M G F ) o f a R a n d o m Vector Y: The MGF of an n x 1 random vector Y is given by my(t) = E(e t'v) where the n • 1 vector of constants t = (tl . . . . . tn)' if the expectation exists for - h < ti < h where h > 0 and i -- 1. . . . . n. There is a one-to-one correspondence between the probability distribution of Y and the MGF of Y, if the MGF exists. Therefore, the probability distribution of Y can be identified if the MGF of Y can be found. The following two theorems and corollary are used to derive the MGF of a random vector Y.

Theorem 1.3.3

Let the n x 1 random vector Y = (Y'I, Y2 . . . . . Y~)' where m Yi is an ni • 1 random vector f o r i = 1 . . . . . m and n = ~-~i=l ni. Let m e ( . ) , mv~ (.) . . . . . m y m(.) represent the M G F s o f Y, Y1 . . . . . Ym respectively. The vectors Y1 . . . . . Ym are mutually independent if and only if

my(t) = my~ (tl)mv2 ( t 2 ) . . .

m y m (tm)

f o r all t = (t~ . . . . . t'm)' on the open rectangle around O.

Theorem 1.3.4 I f Y is an n x 1 random vector, g is an n • 1 vector o f constants, and c is a scalar constant, then mg, y(c) = my(cg). Proof:

mg,u

= E[e cg'u = E[e ~cg)'u = mu

II

1

Linear Algebra

19

Let mr, (.) . . . . . myra (.) represent the MGFs of the independent random variables Y1 . . . . . Ym, respectively. If Z = ~-~i%1 gi then the M G F of Z is given by C o r o l l a r y 1.3.4

m mz(s) = I-[ mr,, (s). i=1

Proof." mz(s)

= ml~,y(s)

--

my(slm)

by Theorem 1.3.4

m

= I I mri (s)

by Theorem 1.3.3.

II

i=1

Moment generating functions are used in the next example to derive the distribution of the sum of independent chi-square random variables.

Example 1.3.1 Let Y1 . . . . . Ym be m independent central chi-square random variables where Yi and n i degrees of freedom for i = 1. . . . . m. For any i mri (t) = E(e tri)

etyi [I"(ni/2)2ni/2] -1 Yini/2-1 e-Yi/2dyi : fO ~176 ---- [I'(ni/2)2ni/2] -1 fO0~176 yini/2- le-y i/(l/2-t) -I dyi = [I'(ni/Z)Zn~/2] -1 [ F ( n i / 2 ) ( 1 / 2 - t) -n'/2] = (1 - 2t)-n'/2. Let Z = ~-~im__lYi. By Corollary 1.3.4, m

mz(t) = 1-I mr~ (t) i=1

= ( 1 - 2t) - Y']~2, n~/2.

Therefore, y]im=l Yi is distributed as a central chi-square random variable with m )-~i=1 ni degrees of freedom. The next theorem is useful when dealing with functions of independent random vectors.

20

Linear Models

Theorem 1.3.5 Let gl(Y1) . . . . . gm(Ym) be m functions o f the random vectors Y1 . . . . . Y m , respectively. I f Y1 . . . . . Y m are m u t u a l l y i n d e p e n d e n t , t h e n g l . . . . . gm are mutually independent. The next example demonstrates that the sum of squares of n independent N1 (0, 1) random variables has a central chi-square distribution with n degrees of freedom. E x a m p l e 1.3.2 Let Z 1 . . . . . Z n be a random sample of normally distributed random variables with mean 0 and variance 1. Let u = Z 2 for i = 1 . . . . . n. The m o m e n t generating function of Yi is mri (t) = mz2i (t) = E(e tz2i ) =

F f

(2yr)-l/2etZ2-Z2/2dz i

oo

__

z~ (27r)-l/2e-(1-Et)z2 /2dzi oo

-- (1 - 2t) -1/2. That is, each Yi has a central chi-square distribution with one degree of freedom. Furthermore, by Theorem 1.3.5, the Yi's are independent random variables. n n Therefore, by Example 1.3.1, ~~i=1 Yi = ~--~i=1 Z/2 is a central chi-square random variable with n degrees of freedom.

EXERCISES 1

1

1. Find an mn x (n - 1) matrix P such that PP' = m Jm @ (In -- nJn)" n

~

/'/

n

2. Let S1 = ~~/=1 Ui (Vi - V), $2 --" ~~i=1 (Ui - ~ ) 2 , and $3 = E i = I (Vi - V ) 2. If A = (I,, - 1 J n ) ( I n - V V ' ) ( I n -- ~Jn), U = ( g l . . . . . gn) t, and V = (V1,. , Vn)', is the statement $2 - ($21/$3) = U'AU true or false? If the statement is true, verify that A is correct. If the statement A is false, find the correct form of A. 1

1

3. Let V = lm @ [a121, + a~Jn], A1 : (Im -- mJm) @ nJn, and A2 : lm N (In -1jn). n

(a) Show A1A2 = 0mn • (b) Define an mn x mn matrix C such that Ira,, = A1 + A2 + 12.

1

Linear Algebra

21

(c) Let the m n x 1 vector Y = (Yll . . . . . Ymn)' and let C be defined as in part b. Is the following statement true or false?: Y ' C Y = [~--~im1 ~'~=1 Yij]2/ (ran). If the statement is true, verify it. If the statement is false, redefine Y ' C Y in terms of the Yij's. (d) Define constants k l, k2, and m n x m n idempotent matrices C1 and C2 such that A1V = klC1 and A2V -- k2C2. Verify that C1 and C2 are idempotent. 4. Find the inverse of the matrix 0.414 + 0.6J4.

(I.-W)

5. Show that the inverse of the matrix (In + VV') is \ l+V'V

where V is an n x 1

vector. 6. Use the result in Exercise 5 to find the inverse of matrix (a - b)In + bJ,~ where a and b are positive constants. 7. Let V = In | [(1 - p)12 + pJ2] where - 1 < p < 1. (a) Find the 2n eigenvalues of V. (b) Find a nonsingular 2n x 2n matrix Q such that V = QQ'. 1

1

1

1

1

8. Let A1 -- mJm | nJn , A2 - (Im - mJm ) | nJn, and A3 = In | (In _ nJn). Find ~-]~=1Ai, A i A j for all i, j = 1, 2, 3 and ~~=1 ciAi where Cl - c2 = l+(m-1)bandca=l-bfor1 < b < 1. (m-l) 9. Define n x n matrices P and D such that (a - b)In -+- bJn = PDP' where D is a diagonal matrix and a and b are constants. 10. Let B

~.

In, 0

--Y] 12Y]21 In2

[

Y]ll

and

~-

~"]12 ]

~-']21 ~-']22

where E is a symmetric matrix and the 'Y']ij are ni • nj matrices. Find B E B ' . 1

1

-

1

11. Let A1 -- mJm | nJn, A2 = X(X'X) 1X/, A3 - Im | (In -- nJn) and X X + @ In where X + is an m • p matrix such that l'm X+ = 01xp. Find the m n x m n matrix A4 such that I m | form.

In -- ~-~4=1 Ai. Express A4 in its simplest

12. Is the matrix A4 in Exercise 11 idempotent? 13. Let Y be an n • 1 random vector with n • n covariance matrix cov(Y) = al2In -4- a2Jn. Define Z = P'Y where the n x n matrix P = (lnlP,,) and the n x (n - 1) matrix Pn are defined in Example 1.1.4. Find cov(P'Y).

22

Linear Model:,,;

14. L e t Y be a bt x 1 r a n d o m v e c t o r w i t h E ( Y ) = lb @ (/z 1. . . . . /s and c o v ( Y ) cr2[Ib @ Jt] + a2T[Ib ~ (It -- 7Jt)]. 1 1 @ 7Jt, 1 D e f i n e A1 = ~Jb A2 - ( I b 1Jb) | 1Jt , A3 = (lb -- gJb) 1 l | (It -- 7 J t ) . F i n d E ( Y ' A i Y ) for i = 1, 2, 3. 15. L e t the btr x 1 v e c t o r Y -- (Ylll . . . . . Ybtr)' and let b Sl -- E ~ ~ ' Y

Yllr, Y121 . . . . .

Y12r . . . . .

Ybtl . . . .

"2 . . . .

i=1 j = l k=l

b~r i=1 j = l k=l

S3 -- E E (y'j" -- ~ ' " i=1 j=l k=l

54 -- E

.)2,

(YiJ" -- Fi'" -- Y.j. + Y . . .)2

i=1 j = l k=l

s5 = ~

(Yij, - Yij.) 2.

i=1 j = l k=l

Derive

btr x btr m a t r i c e s

Am for m = 1 . . . . .

5 where

Sm =

Y'AmY.

2

Multivariate Normal Distribution

In later chapters we will investigate linear models with normally distributed error structures. Therefore, this chapter concentrates on some important concepts related to the multivariate normal distribution.

2.1

MULTIVARIATE NORMAL DISTRIBUTION FUNCTION

Let Z 1 . . . . . Z n be independent, identically distributed normal random variables with mean 0 and variance 1. The marginal distribution of Zi is

fzi (zi) = (2zr)-l/2e-Z{/2

-~

< zi < oo

for i = 1 . . . . . n. Since the Zi's are independent random variables, the joint probability distribution of the n x 1 random vector Z = (Z1 . . . . . Zn)' is fz(z) = (27r)-n/2e-Ei"--lZ2/2 = (2~)-n/2e-Z'Z/2

23

--O0

< Zi < O0

24

L i n e a r Model~

for i = 1 . . . . . n. Let the n x 1 vector Y = G Z +/~ where G is an n x n nonsingula~t matrix a n d / z is an n x 1 vector. The joint distribution of the n x 1 random vector Y is f y ( y ) = I~1-1/2 (2n')-n/2e-{(Y-U)'Y:'-I (y-/.~)}/2. where ~ = G G ' is an n x n positive definite matrix and the Jacobian for the transformation Z = G -1 ( Y - / ~ ) is IGG'1-1/2 - I~1-1/2. The function f y (y) is the multivariate normal distribution of an n x 1 random vector Y with n x 1 mean vector/_t and n • n positive definite covariance matrix ~ . The following notation will be used to represent this distribution: the n x 1 random vector Y ~ Nn (/~, ~ ) . The m o m e n t generating function of an n-dimensional multivariate normal random vector is provided in the next theorem.

T h e o r e m 2.1.1

Let the n x 1 random vector Y ~ Nn (lz, ~,). The M G F o f Y i,', m y ( t ) = e t't~+t'~t/2

where the n x 1 v e c t o r t = (tl . . . . . tn)' f o r - h

< ti < h, h > O, and i -

1. . . . . n

Proof: Let the n x 1 random vector Z = ( Z 1 . . . . . Z n)' where Z i are independent identically distributed N1 (0, 1) random variables. The n x 1 random vector Y G Z + / ~ --~ Nn (/~, N) where N = G G ' . Therefore, m y ( t ) = Ev[e t'v]

= Ez

=

[e t'(GZ+#)]

=/.../(2rr)-n/2et'(Gz+U)e-Z'Z/2dz

=et'U+t'y,t/2f...f(2rr)-n/2et-(z-C,'t)'(z-G't)}/2dz

: e t'#+t'~2t/2.

II

We will now consider distributions of linear transformations of the random vector Y when Y ~ Nn (/z, ~ ) . The following theorem provides the joint probabilit~ distribution of the m • 1 random vector BY + b where B is an m x n matrix of constants and b is an m x 1 vector of constants.

T h e o r e m 2.1.2 / f Y is an n x 1 random vector distributed Nn (/~, ~ ) , B is at~ m x n matrix o f constants with m Fq,n-p • . Note that Y' (I n -- n1J n ) Y = SSRegR + S S E R = S S R e g F + S S E F , which implies SSER - SSEF = SSRegF - SSRegR where SSRegF and SSRegR are the sum of squares regression for the full and reduced models, respectively. Therefore, the likelihood ratio statistic for this problem also takes the form V = (SSRegF -- SSRegR)/q SSEF/(n - p)

Example 6.4.2

'~ Fq,n-p()~).

Consider a special case of the problem posed in Example 6.4.1. From Example 6.4.1 let the n x p matrix X = ( X I l X 2 ) w h e r e X1 equals the n x 1 vector 1,, and X2 equals the n x (p - 1) matrix Xc such that l'nXc = 01• and partition the p x 1 vector/3 as (/501/51. . . . . tip-l)'. The objective is to test

6

M a x i m u m Likelihood E s t i m a t i o n

125

the hypothesis H0 " /~1 = " ' " - - / ~ p - 1 = 0 versus H1 " not all /~i : 0 for i = 1 . . . . . p -- 1. F r o m E x a m p l e 6.4.1, the likelihood ratio statistic equals V = {Y'(In - ln(l',,1,,)-ll'n)Y - Y'(I,, - X ( X ' X ) - l x ' ) Y } / ( p

- 1)

Y'(I,, - X ( X ' X ) - ~ X ' ) Y / ( n - p) Y'[X(X'X)-~X '-

88

1)

Y'(I,, - X ( X ' X ) - I X ' ) Y / ( n

- p)

Y'[Xc (XcXc) ' - 1 ,Xc]Y/( p = Y'(I,,- X(X'X)-~X')Y/(n-

1) p) "" Fp_l,,,_p(~.).

Therefore, V equals the m e a n square regression due to/~1 . . . . . /3p_1 divided by the m ean square residual, as depicted in Table 5.3.1.

Example 6.4.3

Consider the one-way classification from E x a m p l e 2.1.4. The experiment can be m o d e l e d as Y = X/3 + E where the tr x 1 vector Y = (Yll . . . . . Ylr . . . . . Ytl . . . . . Ytr)', the tr x t matrix X = [It | lr], the t x 1 vector /~ -" (/~1 . . . . . /~t)t .~ (~L/,1. . . . . ]-s w h e r e / x l . . . . . ~L/~t are defined in E x a m p l e 2.1.4, and the n x 1 r a n d o m vector E " ~ N t r ( 0 , O.R(T)2It | Ir). The objective is to construct the likelihood ratio test for the hypothesis H0 : /~1 = . . . . /~t versus H1 : not all/~i equal. The hypothesis H0 is equivalent to the hypothesis H/3 = h with the (t - 1) x t matrix H = P't and the (p - 1) x 1 vector h = 0 where P't is the (t - 1) x t lower portion of a t-dimensional He l me r t matrix with

e'tet

:

It-l,

Pte' t : It - 7Jt, 1

and

ltPt

-- 0 i x ( t - I ) .

Note

= (X'X)-1 X ' Y = [ (It ~ lr)t (i t ~ l r ) 1-1 (It |

--

H/3-h=P; [H(X'X)-IH']

(1) It|

r

lr)'Y

Y

It N r r j Y =

-1 = {P't[(It ~ l r ) ' ( I t

(1) P;|

@ lr)]-lpt}

Y

-1

= (l~tPt) -1 ~ r = I t - 1 ~ r

It | L - X ( X ' X ) - I X' - It | L

-- {(It ~ lr)[(It ~ lr)'(It ~ l r ) ] - l ( I t ~ lr)'} :-It~)Ir--

=It~(Ir-ljr)

It@-Jr r

126

Linear Models

Therefore, the third form of the likelihood ratio statistic is given by

Y'

[(

P't @ ll'r

(It-1 ~

r)

(

P't ~

ll'r

)]

Y / ( t - 1)

V= Y'It |

(Ir-

s

--t)

Yt[(It - lJt) | lJr]Y/(t -1) =

~

Ft_l,t(r_l)(~.).

- rJr) 1 ] V/[t(r - 1)]

Y'[It|

Therefore, V equals the mean square for the treatments divided by the mean square for the nested replicates, which is the usual ANOVA test for equality of the treatment means.

CONFIDENCE BANDS ON LINEAR COMBINATIONS OF/3

6.5

The likelihood ratio statistic is now used to construct confidence bands on individual linear combinations of/3. Assume the model Y = X/3 + E where E ~ Nn (0, O'2In). Let the q x p matrix H and the q x 1 vector h from Section 6.4 equal a 1 x p row vector g' and a scalar go, respectively. The hypothesis H0 : I-I/3 = h versus H1 : I-I/3 ~ h becomes Ho : g'/3 = go versus HI : g'/3 ~: go. The third form of the likelihood ratio statistic equals

go) v = [Y'(I.- X(X'X)-lX')Y/(n-

p)][g'(X'X)-lg] ~

( g ' / 3 - go) = {[Y'(In - X ( X ' X ) - l X ' ) Y / ( n -

p)][g'(X'X)-lg]} 1/2

Fl,n-p(Z)

or

~

tn_p(X)

Therefore,

1m y

m

P(

,~y / 2

~ t n _ p _
F,2,1 • where F,2,1 • is the 100(1 y) percentile point of a central F distribution with 2 and 1 degrees of freedom. Note that Ho is equivalent to hypothesis that there is no treatment effect. The Type I sums of squares can also be used to provide unbiased estimators of the variance components cr2 and tr2T . The mean square for B T l l z , B , T provides an unbiased estimator of cr2T since ~.4 = 0 and E(Y'A4Y/1) = E(cr2 TX2 (0)) - trOT. Constructing an unbiased estimator for cr2 involves a little more work. In complete balanced designs, the sum of squares for blocks can be used to find an unbiased estimator for cr2. However, in this balanced, incomplete design problem, the block effect is confounded with the treatment effect. Therefore, the Type I sum of squares for Block(B)l/z has a noncentrality parameter ~.2 > 0 and cannot be used directly with Y'A4Y to form an unbiased estimator of cr2. One solution to this problem is to calculate the sum of squares due to blocks after the overall mean and the treatment effect have been removed. After doing so, the block effect does not contain any treatment effects. As a result, the Type I sum of squares due to Block (B)I#, T has a zero noncentrality parameter and can be used with Y'AaY to construct an unbiased estimator of tr2. The Type I sum of squares due to Block (B)I#, T is given by SS Block(B)l#, T = Y A 3 Y *' where A~ = T~ ~,~1 t w , , ' ~v ,1 ) - 1TT, -- S~(S~tS~) -1 Syz, with S~ = M [ X I IX2] and T 91 -_ M[XIIX2 IZ1]. Note that the matrices S~ and T7 now order the overall mean matrix X1 first, the treatment matrix X2 second, and the block matrix Z~ third. From the PROC IML output, the 6 • 6 matrix A~ for the example data set equals

,

A3=6

2

1

1

2

1

1

-1

-1

-1

-1

-2

2

1

-1

2

1

-2

-2

1

2

-1

1 2

1

1 -1

-1 -1

1

-2

1

2

1

-1

1

1

2

"

Furthermore, tr(A~E) - (3tr 2 + tr2T) and (X/~)tA~ (X/~) = (~2, ~3, ~1, ~3, ~1, ~ 2 ) A ; (~2, ~3,/~1, ~3, ~1, ~2) t -" 0.

7

Unbalanced Designs and Missing Data

143

1 '[A; Therefore, an unbiased estimator of tr~ is provided by the quadratic form ~Y A4]Y since

E

Y'[A; - A4IY

= g{(3cr 2 +

~T) -- ~r} =

~

The procedure just described is now generalized. Let the n* x 1 vector Y* represent the observations from a complete, balanced factorial experiment with model Y* - X*/3 + E* where X* is an n* x p matrix of constants,/3 is a p x 1 vector of unknown parameters, and the n* x 1 random vector E* -~ Nn (0, E*) where E* can be expressed as a function of one or more unknown parameters. Suppose the n x 1 vector Y represents the actual observed data with n < n* where n* is the number of observations in the complete data set, n is the number of actual observations, and n* - n is the number of missing observations. The n x 1 random vector Y = MY* ~ Nn (Xfl, E) where M is an n x n* pattern matrix of zeros and ones, X = MX*, and E = M E * M ' . Each of the n rows of M has a single value of one and (n* - 1) zeros. The ij th element of M is a 1 when the i th element in the actual data vector Y matches the jth element in the complete data vector Y* for i = 1 . . . . . n and j = 1 . . . . . n*. Furthermore, the n x n matrix M M ' = In and the n* x n* matrix M ' M is an idempotent, diagonal matrix of rank n with n ones and n* - n zeros on the diagonal. The ones on the diagonal of M ' M correspond to the ordered location of the actual data points in the complete data vector Y* and the zeros on the diagonal of M ' M correspond to the ordered location of the missing data in the complete data vector Y*. Finally, l e t Xs(XtsXs)-lXts a n d Zs(Z'sZs)-lZts be the sum of squares matrices for the fixed and random effects in the complete data set for s - 1 . . . . . m where rank(X,) = ps >_ 0, rank(Zs) = q~ > 0, a n d X 1 = In*. Let Ss = M [ X l l Z l l X 2 l - ' - IZ~-llXs] and Ts = M [ X l l Z l l X 2 l . . . IXslZ~] for s - 1 . . . . . m. The Type I sum of squares for the mean is Y ' S l ( S ' l S l ) - l s ~ y = Y' nJn 1 Y. The Type I sums of squares for the intermediate fixed effects take the form t Y'Ss(S'sS,) --1 SsY - Y ' T s - I ( T 's _

1

/ Ts-1)- 1 Ts_IY.

The Type I sum of squares for the intermediate random effects take the form ! Y'T~(T'~Ts)- 1 T~Y - Y'S~(S'~Ss)-IS's Y

for s = 2 . . . . . < m. However, the missing data may cause some of these Type I sum of squares matrices to have zero rank. Furthermore, it may be necessary to calculate Type I sums of squares in various orders to obtain unbiased estimators of the variance components. The estimation of variance components with Type I sums of squares is discussed in detail in Chapter 10.

144

7.3

Linear M o d e l s

USING REPLICATION AND PATTERN MATRICES TOGETHER

Replication and pattern matrices can be used together in factorial experiments where certain combinations of the factors are missing and other combinations of the factors contain an unequal number of replicate observations. For example, consider the two-way cross classification described in Figure 7.3.1. The experiment contains three random blocks and three fixed treatment levels. The data set contains no observations in the (1, 1), (2, 2), and (3, 3) block treatment combinations and either one or two observations in the other six combinations. As in Section 7.2, begin by examining an experiment with exactly one observation in each of the nine block treatment combinations. In this complete, balanced design the 9 x 1 random vector Y* = (Ylll, Y121, Y131, Y211, Y221, Y231, Y311, Y321, Y331)'. Use the model Y* = X*/3 + E where E --~ N9(0, ]E*). The matrices X*,/3, E, E*, X1, Z1, X2, Z2, and M are defined as in Section 7.2. For the data set in Figure 7.3.1, let the 9 x 6 replication matrix R identify the nine replicate observations in the six block treatment combinations that contain data. The replication matrix R is given by 1

12 R m

0 12

0

1

12 Finally, let the 9 • 1 random vector of actual observations Y -- (YI21, Y131, Y132, Y211, Y212, Y231, Y311, Y321, Y322) t. Therefore, Y ~ N9(X/~, ]~) where X/3 = RMX*/3 - R(fl2, f13, ill, f13, ill, f12) t (f12, f131~ flll 2, f13, ill, fl212

Random blocks (i) Y211 Y212 Fixed treatments

(j)

Figure 7.3.1

Y321 Y322

Y121 YI31 Y132

r311

Y231

Missing Data Example with Unequal Replication.

7

Unbalanced Designs and Missing Data

145

Table 7.3.1 Type I Sums of Squares for the Missing Data Example with Unequal Replication df

Type I SS

Overall mean/z

1

Y'S1 (S'l$1)-1S'I Y

Block (B)I#

2

Treatment (T)I#, B

2

Y'[T1 (T'IT1)-IT'I - $1 (S'IS1)-Is'I]Y = Y'AzY Y'[S2(S~S2)-Is~ - Tl (T'lT1)-lT'l]V = Y'AaY

BTII,z, B, T

1

Y'[RD-1R ' - S2(S~S2)-Is~]Y

= Y'A4Y

Pure error

3

Y'[I9 - RD-IR']Y

= Y'AsY

9

y'y

Source

Total

= Y'A1Y

and E - RME*M'R' + cr2(BT)I9

=R(a2[13|174 The Type I sums of squares for this problem are presented in Table 7.3.1 using matrices R, D, Sl = RMX1, T1 = RM[XIlZ1], and S2 = RM[XIlZIlX2]. The sums of squares matrices A1 . . . . . A5 in Table 7.3.1, the matrices A1 EA1 . . . . . AsEAs, A~, A~EA~, and the noncentrality parameters ,kl . . . . . )~5, ~.~ were calculated numerically using PROC IML in SAS. The PROC IML output for this section is presented in Section A2.2 of Appendix 2. From the PROC IML output note that A I ~ A 1 --

(3~2B

AEEA2 =

3or2

A3EA3 -

2)

+ 2t72BT + CrR(BT) A1 -Jr-~0"2BT -+-tTR(BT)

( -~O'BT 42 2 )) A 3 + O'R(BT

42 2 )) A 4 A4]~A4 -- ( -~O'BT + O'R(BT

AsEA5 = (O'R2(BT))A5.

146

Linear Models

Therefore, by Corollary 3.1.2(a),

Y'AzY ~ 3a2 + gcrnr + ~(Br) X22(X2)

2 X2()v3) Y'A3Y "~ gerBr+ ~R(Br) Y'A4Y ~"

if2 T + O'R(BT)

2 x~(Xs) Y'AsY ~ ~R(Br)

where )~l- (X~)'A1(X/3)/[2 (3cr~ + ~cr2r + tr~(Br)) ] = (~1 + ~2 + ~3)~/2 3o~ + g o 2 + o 2 X2 = (X/3)'A2(X~)/ E( 2 3aB2 + 5crB2r 2 + Crg2(Br)

-(flZl+f12+f123 -- ~1~2 -- ~1~3 -- ~2~3)/ [3 ( 3or2 + --

52~2Br+ o'2(Br))]

4

= 2(/~12+ r + f132-/~1/~2-/~lf13~,4- (X~)/A4(X/~)/[2 (~O'2T

f12f13)/[(4O'2T +

2 30"R(BT))]

-]-0"2(BT))1

--0.

•5 -- (X/3)'As(X/~)/(2cr~(,r)) =0.

The quadraticforms 3[Y'A4Y-(Y'AsY/3)] and Y'AsY/3 are unbiasedestimators of er2r and a2(nr), respectively,since 3 3 4 E{~[Y'A4Y- (Y'AsY/3)]}- E{~ [(~tr~r + ~r~(sr))X~(0) -er2(Br) (X2(0)/3)] } - ~ 2 r

7

Unbalanced Designs and Missing Data

147

and

E[Y'AsY/3] = E[o2(B:r)X2(0)/3]-- o2(B:r). Furthermore, As E A t = 06x6 for all s # t, s, t = 1 . . . . . 5. By Theorem 3.2.1, the five sums of squares Y'A1Y . . . . . Y'AsY, are mutually independent. Therefore, F* =

Y'A3Y/2 Y'AaY/1

~ F2,1 (X3)

where X3 = 0 under the hypothesis H0 :/31 = ~2 = ~3. A Y level rejection region for the hypothesis Ho :/~l =/32 =/33 versus H1: not all/3's equal is to reject Ho if F* > F,2,1 • where F,2,1 • is the 100(1 - ~,) percentile point of a central F distribution with 2 and 1 degrees of freedom. The Type I sum of squares due to Block (B)l/z, T is given by SS Block ( B ) I # , T - Y'A~Y where A~ . _ T ~ ( T ~ " T ~ ) - I T 91 , - S 2 ,~t ' g' 2l , t ~ ,'-'2) - 1 S~, with S~ - RM[XIIX2] and T ,1 = RM[XIIXzIZ1]. Furthermore, fl'X'A~Xfl = 0, t r ( A ~ E ) - 4or2 + 40.2T _+. 2 4 2 2 2trR(Br ), and E[Y'A~Y] - 4or2 + ~crsr + 2erR(st ). Therefore, an unbiased esti-

mator of tr 2 is provided by 1 {[Y'A~Y] - [Y'A4Y + (Y'AsY/3)]} since 1 ~E{[Y'A~Y] -[Y'A4Y + (Y'AsY/3)}

1 4 = ~{ (40"2+ ~o' 2:r+ 2o~(8/,))- E[(~-4

-Jr- tT R2 ( B T )

(X2(0)/3)]}

= o r b2.

EXERCISES 1. If B1, B2, B3, and B4 are the n x n sum of squares matrices in Table 7.1.1, prove B ] = Br for r -- 1 . . . . . 4 and BrBs - - O n x n for r 5~ s. 2. In Example 7.1.1 prove that the sums of squares due to the mean, regression, and pure error are distributed as multiples of chi-square random variables. Find the three noncentrality parameters in terms of/51,/32, and/53. 3. In Example 7.1.2 let b - t - 3, rll = r22 - r33 -- 2, r12, r23 - r31 -- 1, rl3 - r21 = r32 - 3, and thus n - 18. Let B1, B2, B3, B4, B5 be the Type I

148

Linear

Models

sum of squares matrices for the overall mean #, Bl/z, TIlz,B, BTIIz,B,T, and R(BT)IIz, B,T, BT, respectively. Construct and B1, B2, B3, B4, B5. 4. From Exercise 3, construct the n x n covariance matrix E. 5. From Exercises 3 and 4, calculate t r ( B r E ) and/3tX~RtBrRXd~ for r 1 .....

-

5.

6. From Exercise 3, find the distributions of YtBrY for r = 1. . . . . 5. Are these five variables mutually independent?

8

Balanced Incomplete Block Designs

The analysis of any balanced incomplete block design (BIBD) is developed in this chapter.

8.1

GENERAL BALANCED INCOMPLETE BLOCK DESIGN

In Section 7.2 a special case of a balanced incomplete block design was discussed. As shown in Figure 7.2.1, the special case has three random blocks, three fixed treatments, two observations in every block, and two replicate observations per treatment, with six of the nine block treatment combinations containing data. We adopt Yates's/Kempthorne's notation to characterize the general class of BIBDs. Let b = the number of blocks t = the number of treatments

149

150

Linear Models Random blocks (i) YI 1 Fixed treatments

(j)

Y31

Y51

Y42

Y12 Y23

Y62

II33

I'24

I'63 r44

r54

Figure 8.1.1 Balanced Incomplete Block Example.

k = the n u m b e r of observations per block r = the n u m b e r of replicate observations per treatment. The general BIBD has b random blocks, t fixed treatments, k observations per block, and r replicate observations per treatment, with bk (or tr) of the bt block treatment combinations containing data. Furthermore, the number of times any two treatments occur together in a block is ~. = r(k - 1)/(t - 1). In the Figure 7.2.1 example, b = 3, t = 3, k = 2, and r = 2. The total number of block treatment combinations containing data is bk or tr, establishing the relationship bk = tr. To obtain a design where each block contains k treatments, the number of blocks equals all combinations of treatments taken k at a time or b = t!/[k!(t - k)!]. A second example of a BIBD is depicted in Figure 8.1.1. This design has b = 6 random blocks, t = 4 fixed treatments, k = 2 observations in every block, and r -- 3 replicate observations per treatment, with bk - 12 of the bt = 24 block treatment combinations containing data. Next we seek a model for the general balanced incomplete block design. To this end, begin with a model for the complete balanced design with b blocks, t treatments, and one observation per block/treatment combination. The model is Y* = X*'r + E* where the bt • 1 vector Y* = ( Y l l . . . . . Y l t . . . . . Y b l . . . . . Y b t ) ' , the bt x t matrix X* = l b @ It, the t x 1 vector of unknown treatments means r = (rl . . . . . rt)' and the bt x 1 random vector E* = ( E l l . . . . . Elt . . . . . Ebl . . . . . Ebt)' ~ N b t ( O , ~ * ) where

~* -- r [Ib | Jt ] -q- cr2T [Ib | (It - ~ Jt ) ] . Let the bk x 1 vector Y represent the actual observed data for the balanced incomplete block design. Note that Y can be represented by Y = MY* where M

8

Balanced Incomplete Block Designs

is a bk

x bt pattern matrix M.

151

The matrix M takes the form M1

M

M2

0

0

".

_~

Mb where Mi is a k x t matrix for i = 1. . . . . b. Furthermore, MM' = Ib | Ik and M(Ib | Jt)M' - [[M1JtM'I

L

"..

0

0MbJtM'b 1

-- Ib |

Each of the k rows of Mi has t - 1 zeros and a single value of one where the one indicates the treatment level of the jth observation in the i th block for j -- 1. . . . . k. Therefore, the model used for the balanced incomplete block design is Y - X-r + E where the bk x t matrix X = MX*, E .-~ Nbk (0, ]E) and E = ME*M'

--M{cr2[Ib|174

(It-~Jt)])M'

= tr2M[lb | Jt]Mt--[ - O'2T [M(1o | It)M t - 1M(Ibt | J,)M'] = cr2[lb | Jk] + cr2r [Ib | ( I k - t Rearranging terms, E can also be written as E=

(or 2 -

~cr2r)[Ib|

+ cr2r[Ib |

or

E=

(kcr2+

t -t k c r Z r ) [ l b |

1jk ]

+ trZr [ib | (ik _ ~jk) ]

1 This last form of E can be used to find E -1. Note [lb | ~Jk] and [Ib | (lk l ~ j k ) ] are idempotent matrices where [Ib | (l~jk)] [Ib | (Ik -- ~J~l)] = 0. Therefore,

t -k t

)-1

152

Linear Models

Table 8.2.1 First ANOVA Table with Type I Sum of Squares for BIBD

8.2

Source

df

Type I SS

Overall mean #

1

Y'A,Y=Y'~Jb|

Block (B)l/z

b-

Treatment (T)I/z, B

t - 1

Y'AaY

B T I # , B, T

bk - b - t + 1

Y'A4Y

Total

bk

y'y

ANALYSIS

1

Y'AzY = Y'(Ib - ~ J b ) |

OF THE GENERAL

88

CASE

The treatment differences can be examined and the variance parameters cr~ and cr~r can be estimated by calculating the Type I sums of squares from two ANOVA tables. Some notation is necessary to describe these calculations. Let the bt x 1 matrix X1 = lb | l t , the bt x (b - 1) matrix Z1 = Qb @ 1/, the bt x (t - 1) matrix X2 = Ib @ Q t and the bt x ( b - 1)(t - 1) matrix 2 2 = Q b | Q t where Qb and Q t are b x (b - 1) and t x (t - 1) matrices defined as in Section 5.7. Furthermore, letS1 = MX1, T1 = M[X11Z1], S2 = M[X11Z1 IX2], and T2 = M[X11ZllX21Z2]. The first ANOVA table is presented in Table 8.2.1. The Type I sums of squares due to the overall mean ~, due to Blocks (B)I/z, Treatments ( T ) I # , B, and BTIIz, B, T are represented by Y'A1Y, Y'A2Y, Y'A3Y, and Y'A4Y, respectively, where A1 = S1 (S'lS1) -1 S '1 -- ~Jb 1 @ ~Jk 1 A2 = T I ( T ' I T 1 )

_~T ,1 -

S1

(S,lSl) - 1,S 1 - - ( Ib

1 )

-- ~ J b

|

1 ~Jk

A3 = $2(S~$2) - 1S 2, - T1 (T'1T1) 1T'l A4 - (Ib @ Ik) -- A1 - A2 - A3. Note A1 + A2 - lb | By Theorem 1.1.7, A2,, Au(A1 + A2) - Au (Ib @

1 4 Au -- Ib | Ik, and ~-~.4u=1 rank(Au) - bk. ~J~, ~--~.u:l Au for u - 1 . . . . . 4 and AuAv - 0 for u -76- v. Also 1 ~Jk) -- 0 or Au(Ib | Jk) = 0 for u -- 3, 4. Therefore,

AuX -- cr~Au[Ib Q Jk] + cr~rAu [Ib Q (Ik -- ~Jk) l -- cr~Au for u -- 3, 4. Furthermore, Au E - (kcr~ + ~-~ crBer)Au for u -- 1, 2. Therefore, by Corollary 3.1.2(a), Y'A1Y "" alX~()~l), Y'AeY ~ aex~_l(~.2), Y'A3Y "~

8

Balanced Incomplete Block Designs

153

Table 8.2.2 Second ANOVA Table with Type I Sum of Squares for BIBD Source

df

Type I SS

Overall mean/z

1

Y'A 1Y = Y' ~ Jb |

Treatments (T)I#

t - 1

Y'A~Y

Block (B)I#, T

b-

Y'A~Y

BTI#, B, T

bk - b - t + 1

Y'A4Y

Total

bk

Y'Y

1

1

~ Jk Y

t - k G2

a3xt21 (~.3), and Y'AaY ~ a4x b2k - t - b + l (~,4) where al - ae - (ka 2 + -7-- Br) a3 -- a4 -- a2T, ~u -- ( X ' r ) ' A u ( X ' r ) / ( 2 a u ) for u - 1, 2, 3 and ~ 4 - - 0 . Finally,

by Theorem 3.2.1, Y'A1 Y, Y'AeY, Y'A3Y, and Y'AnY are mutually independent. A test on the treatment means can be constructed by noting that ri . . . . . "rt implies ~3 = 0. Therefore, a y level rejection region for the hypothesis H0 9 ri rt versus Hi " not all "rj's equal is to reject H0 if F* > F t -• i , b k - t - b + i where " " " --F

*

_._

Y'A3Y/(t - 1) Y'A4Y/(bk

Unbiased estimators of cr~ and r estimator of O-~r is provided by ..2 = Y ' A 4 Y / ( b k O'BT

- t - b + 1)

can also be constructed. An unbiased

- t - b + 1)

since Y'AnY ~ O ' 2 T X b2 k - t - b + i (/~4 -- 0) An unbiased estimator of a~ can be developed using a second ANOVA table. Let the Type I sums of squares due to the overall mean/z and due to B T I # , B, T be represented by Y'A1Y and Y'A4Y, as before. Let the Type I sums of squares due to Treatments (T)I# and due to Blocks (B)I#, T be represented by Y'A~Y and Y'A~Y, respectively. The matrices A1 and A4 are defined in Table 8.2.1. Matrices A~ and A~ can be constructed by setting S~ - - M [ X i I X 2 ] and T~ - - M [ X i I X 2 I Z i ] . Then A~

--

S~(S~tS~)-is~ t - Ai

A~

t , T , t T , -1 ,t , ,t , - i ,t -- T~l 1 -i) T1 -- 8 2 ( 8 2 8 2 ) 82.

The second ANOVA table is presented in Table 8.2.2. The expected mean square for Blocks (B)I#, T is now derived and then used to generate an unbiased estimator of aBe. First, let G be the t x bk matrix such that GY = (~'.i . . . . . ~'.t)'. The t x t covariance matrix G E G ' has diagonal elements equal to ( a 2 + L ~ a 2 B T ) / r and off-diagonals equal to ( k - 1 ) ( a 2 - - T a 2i B T ) / [ r ( t - - 1 ) ] .

154

Linear Models

The sum of squares due to Treatments (T)Ib, can be reexpressed as rY'G' (I/ Jt)GY. Therefore, A~ - rG' (I/ -- ~Jtl )G and tr[A~E]=rtr[G'(It-~Jt)GE]

= r t r (It -- ~ J t ) G~G']

t(~

=r

t-1

- tr + r

t-1

1

t

+ t(k-1)(cr~-~

1 = (t -- k)cr2 + t[(t - 1) 2 + (k - 1)]a2T .

But A2 + A3 -- A~ + A~, therefore tr[A~E] = tr[(A2 + A3)]~] - tr[A~E] =(b-l) -

{

(bk -

[

kcr2 +

t

1

trZr +(t--1)crZr

,

(t-k)o'~+t[(t-1 t)~ 2 +

b(t - k ) t

+(k-1)lo~r

}

o2 r.

Furthermore, E[Y'A~Y/(b- 1)] : tr[A~E]/(b- 1) (bk-t) = ~tr~"(b1-

+

b(t-k) Cr2r t ( b - 1)

since -r'X'A~X-r - O. Therefore, an unbiased estimator of t~B 2 is provided by l

6 . 2 _ (bk- t)

[y,A~y -

b(t-k) Y'A4Y] t(bk - b - t + l)

because 1_ E { (bk 1

(bk - t)

[[Y'A~Y_ t)

b(t - k) Y'A4Y] } = t(bk - b - t + 1)

(bk - t)a 2 +

b(t - k ) .2 OBT t

b~t;

~)~] _ o~

8

Balanced Incomplete Block Designs

155

Treatment comparisons are of particular interest in balanced incomplete block designs. Treatment comparisons can be described as h'-r where h is a t x 1 vector of constants. By the Gauss-Markov theorem, the best linear unbiased estimator of h ' r is given by h ' [ X ' Z - 1 X ] - I x ' z - 1 Y where Z -1 is given in Section 8.1. Therefore, the BLUE of h'-r is a function of the unknown parameters tr 2 and a2 r. However, an estimated covariance matrix, Z, can be constructed by using Z with crB 2 and cr2r replaced by t~2 and 82 r, respectively. An estimator of h'-r is provided by h ' [ X ' Z - 1 X ] - I x ' z - 1 Y .

8.3

MATRIX DERIVATIONS OF KEMPTHORNE'S INTERBLOCK AND INTRABLOCK TREATMENT DIFFERENCE ESTIMATORS

In Section 26.4 of Kempthorne's (1952) Design and Analysis of Experiments text, he develops two types of treatment comparison estimators for balanced incomplete block designs. The first estimator is derived from intrablock information within the blocks. The second estimator is derived from interblock information between the blocks. The purpose of this section is to develop Kempthorne's estimators in matrix form and to relate these estimators to the discussion presented in Sections 8.1 and 8.2 of this text. Within this section we adopt Kempthorne's notation, namely: Vj = the total of all observations in treatment j Tj = the total of all observations in blocks containing treatment j Qj = Vj - T j / k . Kempthorne indicates that differences between treatments j and j ' (i.e., rj - rj,) can be estimated by 01 = k(t - 1)[Qj - Q j , ] / [ t r ( k 02 - ( t -

1)[Tj - T j , ] / [ r ( t - k)].

1)]

(1) (2)

The statistics 01 and ~)2 are unbiased estimators of rj - rj, where 01 is derived from intrablock information and ~)2 is derived from interblock information. The following summary provides matrix procedures for calculating estimators (1) and (2).

Estimator I Let A1 and A3 be the bk x bk idempotent matrices constructed in Section 8.2 where rank(A1) = 1 and rank(A3) - t - 1. Let A1 = R1R'1 and A3 = R3R~

156

Linear Models

where R1 -- ( 1 / ~ / ~ ) l b @ lk and R3 is the bk x (t - 1) matrix whose columns are the eigenvectors of A3 that correspond to the t - 1 eigenvalues equal to 1. The estimator ~1 is given by

~1 = g ' ( R ' X ) - 1R'Y where the bk x t matrix X and the bk x 1 vector Y are defined in Section 8.1, the bk x t matrix R - [R11R3], and g is a t x 1 vector with a one in row j, a minus one in row j ' and zeros elsewhere. A second form of the estimator ~)l can be developed by a different procedure. First, construct a t • b matrix N. The i j th element of N equals 1 if the i j th block treatment combination in the BIBD factorial contains an observation and the i j th element of N equals 0 if the i j th block treatment combination is empty for i - 1 . . . . . b and j = 1 . . . . . t. For example, the 3 x 3 matrix N corresponding to the data in Figure 7.2.1 is given by

0, '1 1 1

N

0 1

1 0

and the 4 x 6 matrix N corresponding to the data in Figure 8.1.1 is given by

N __

1

0

1

0

1

0

i

0 1 1

0

1

0

1

1 0

0 1

0 1

1 0

"

The matrix N is sometimes called the incidence matrix. T h e estimator t)l is given by ~1 -- { k ( t -

1)/[tr(k-

[ ( ')]

1)]}g' N []

Ik -- ~Jk

Y

where N is the t x b matrix constructed above and N U(I~ - ~Jk) is a t • bk BIB product matrix. For example, the 3 x 6 matrix [N 0 (Ik -- ~1Jk)] corresponding to the data in Figure 7.2.1 is given by

E / N[]

I2-

=~

,[0 0,,, 1

-1

-1

1

0

-1

0

-1

1

0

,] 1

0

corresponding to the data in Figure 8.1.1 and the 4 x 12 matrix [N n(l~ - ~J~)] 1

8

Balanced Incomplete Block Designs

157

is given by 1-1 0 0 1-1 0 0 1-1 0 0 [F ~/' ~1j2"~] )] -1 1 0 0 0 0 1 - 1 0 0 1 - 1 N[] I2 -- 1/2 0 0 1 -1 -1 1 0 0 0 0 -1 1 " 0 0-1 1 0 0-1 1-1 1 0 0 The following relationships are used to derive the variance of 01. g'g

= 2

g'Jtg

= 0

IN ff] (Ik -- ~Jk) ] [Ib~ lk] --"Ot• [N [Z](Ik -- ~Jk) l [Ib@Jk] -- Otxbk =tr(k-1)[It 1)

1 ~J/~)l [N ff] ( I k - ~Jk)l

IN D (Ik -

k(t-

1jt] -t

"

Therefore, var(01) =

{k(t-1)/[tr(k-1)]}2g'[ND(Ik-~Jk) •174174

_

1

tJk) } [ N D ( I k - - ~ J k ) l g

=o'~r{k(t-1)/[tr(k-1)]}2g'[ND(Ik-~Jk)J x [ND(Ik-~Jk)]'g

=aZr,k(t-1)/[tr(k-1)]}g'(lt-~Jt)g -- 2k(t- 1)azr/[tr(k- 1)]. The matrix IN D (Ik- 1Jk)] is used to create the estimator 01. This estimator is constructed from the treatment effect after the effects due to the overall mean and blocks have been removed. Similarly, Y'A3Y is the sum of squares due to treatments after the overall mean and blocks have been removed. Therefore, [N ffl (Ik -- ~Jk)] 1 and A3 are related and can be shown to satisfy the relationship A3 =

tr(kk(t-1)_1) [ND (Ik - ~ l J k ) ] '

IN [2 ( I k - ~ J k ) ]

lab

Linear Models

There is a one-to-one relationship between the t • b incidence matrix N and the bk x bt pattern matrix M. In Appendix 3 a SAS computer program generates the bk x bt pattern matrix M for a balanced incomplete design, when the dimensions

b, t, k, and the t x b incidence matrix N are supplied as inputs. A second SAS program generates the incidence matrix N, when the dimensions b, t, k, and the pattern matrix M are supplied as inputs.

Estimator

2

The estimator 02 is given by 0 2 ~-

(t - 1)g'[N | l ' k l Y / [ r ( t - k)]

where g, N, and Y are defined as in estimator 1. The following relationships are used to derive the variance of 02" NN'

k) It + ~ J t (t - 1) (t - k)

and

r (t

g'NN'g = 2r(t - k ) / ( t - 1).

Therefore,

var(02) = {(t •

1)2/[r2(t

k)2]}gt[N | 1~]

-

{

[ ( ')} ( ) 1

cr2[Ib | Jk] + cr2r Ib |

Ik -- t J k

= { ( t - 1 ) 2 k / [ r 2 ( t - k)2]} kcr2 + _

2k(t-

r(t -k)

1)

[

+

t

cr2r

[N' | lk]g g'NN'g

.

t

Finally, Kempthorne suggests constructing the best linear unbiased estimator of the treatment differences by combining 01 and 02, weighting inversely as their variances. That is, the BLUE of rj - r j, is given by

03 -- 01/var(01) -~- 02/var(02) . [ 1/ var(01) + 1/ var(02)] Note that 03 is a function of cr2 and a z r since var(01) and var(02) are functions of cr2 and a z r . However, as discussed in Section 8.2, a 2 and crZr can be estimated by 8 2 and 6 2 r , respectively. Therefore, 03 can be estimated by

0~ --

01/var(01) -+- 02/~(02) [1/~'r(01) @ 1/ffa'r(02)]

8

Balanced Incomplete Block Designs

159

where f"~(01) and ~ ( 0 2 ) equal var(01) and var(02), respectively, with o'2 and o'2 r replaced by ~2 and ~2 r. It should be noted that the estimator 0~ given above equals the estimator h'[X'E-1X]-IX'~,-1Y given in Section 8.2 when h = g. ^

EXERCISES Use the example design given in Figure 8.1.1 to answer Exercises 1-11. 1. Define the bk x bt pattern matrix M, identifying the t x t matrices M1 . . . . . Mb explicitly. 2. Construct the matrices A1, A2, A3, A4, A~, and A~. 3. Verify that ArE = (ko'~ + tt-Zko'~r)Ar for r = 1, 2 and ArE = o'~rAr for r = 3,4. 4. Verify that

~,4 =

0.

5. Verify that

~.3 =

0

under H0

" Z'l - - - - .

-- zt.

6. Construct the t x bk matrix G such that GY = (~'.1. . . . . ~'.t)'. 7. Verify that G E G ' is a t x t matrix with (o'2 + t~tlo'Er)/r on the diagonal and 7 o ' B r ) / [ r ( t -- 1)] on the off-diagonal.

8. Verify that A~ = rG' (It - 71jt)G.

1 - 1 )2 + (k - 1)]o'2r. 9. Verify that tr(A~E) = (t - k)o" 2 + -i[(t 10. Verify that tr(A~E) = (bk - t)o" 2 + br

o'2 r.

11. Verify that 7-'X'A~XT" = 0. Use the following data set of answer Exercises 12-16. Random blocks

1 Fixed treatments

2

3

7

12

8 15

16 20

12. Calculate the Type I sum of squares for Tables 8.2.1 and 8.2.2. 13. Compute unbiased estimates of o-2 and o'27..

160 14. Test the hypothesis H0 : y = 0.05.

Linear Models Z'I

-"

Z'2

=

Z'3 versus H1 : n o t all r j ' s are equal. Llse

15. C o m p u t e Kempthorne's estimates of 01 and 02 for the differences rl - r2, r~t r3, and r2 - r3. 16. C o m p u t e Kempthorne's estimates of 03 for the differences rl - r2, rl - r3, and r2 - r3 and then verify that these three estimates equal h ' ( X ' E - 1X)- 1X, ~:-~Y when h = (1, - 1, 0)', (1, 0, - 1)' and (0, 1, - 1)', respectively.

9

Less Than Full Rank Models

In Chapter 7 models of the form Y = RXdfl + E were discussed when the k • p matrix Xd had full column rank p. In this chapter, models of the same form are developed when the matrix Xd does not have full column rank.

9.1

MODEL

ASSUMPTIONS

AND EXAMPLES

Consider the model Y = RXd/~ + E where Y is an n • 1 random vector of observations, R is an n • k replication matrix, Xd is a k • p known matrix of rank k < p, and E is an n • 1 vector of random errors. For the present assume E(E) = 0 and coy(E) = a2In . The assumption E ,~ Nn (0, a2In) will be added in Section 9.3 when hypothesis testing and confidence intervals are discussed. In Section 9.5 the analysis is described when E ~ Nn (0, cr2V) and V is an n • n positive definite matrix. The least squares estimators o f / 3 are obtained by minimizing the quadratic form (Y - RXd/3)'(Y - RXd/3) with respect to/3. The solution/9, satisfies the

161

162

Linear Models

normal equations X~OXa/3 = X ~ R ' Y where the nonsingular k x k matrix D = R'R. Since the p x p matrix X~DXa has rank k < p, X~DXa is singular and the usual least-squares solution/3 = ( X ~ D X a ) - l X ~ R ' Y does not exist. Therefore, the analysis approach described in Section 7.1 is not appropriate and an alternative solution must be found. In Section 9.2 the mean model solution is developed. Before we proceed with the mean model, a few examples of less than full rank models are presented.

Example 9.1.1

Searle (1971, p.165) discussed an experiment introduced by Federer (1955). In the experiment, a fixed treatment A with three levels has rl = 3, r2 = 2, and r3 = 1 replicate observations per level. Let Yij represent the jth observation in the ith treatment level for i = 1, 2, 3 and j - 1. . . . ri with the 6 x 1 random vector Y - (Yll, Y12, Y13, Y21, Y22, Y31)'. The experimental layout is presented in Figure 9.1.1. Searle (1971) employs the less than full rank model Yij = ot + oli + Eij

where ct is the overall mean, O~i is the effect of the ith treatment level, and Eij is a random error term particular to observation Yij. The model can be rewritten in matrix form as Y = RXa/3 = E where the 6 • 3 replication matrix R, the 3 x 4 matrix Xd, the 4 • 1 vector/3 and the 6 • 1 error vector E are given by

R =

113 01 I11001 12

0

'

Xd - -

1

1

0

1

0

1

0

0

1

'

/3 =

c~1 t3/2

ct3

and E = (Ell, El2, El3, E21, E22, E31 /. The 3 x 4 matrix Xd has rank 3 and therefore X~DXd is singular. In this case n - 6, p - 4, and k = 3.

Fixed factor A(i) 1 2 3 Yll

Y21

Y12

Y22

Y31

Replicates (j) Y13

Figure 9.1.1

Searle's (1971) Less than Full Rank Example.

9

Less Than Full Rank Models

163

The main difficulty with the model in Example 9.1.1 is that p -- 4 fixed parameters (ct, C~l, c~2, and ct3) are being used to depict k -- 3 distinct fixed treatment levels. Therefore k < p and the less than full rank model overparameterizes the problem. Less than full rank models can also originate in experiments with missing data. In Chapter 7 pattern matrices proved very useful when analyzing missing data experiments. However, as shown in the next example, pattern matrices do not in general solve the difficulties associated with the less than full rank model.

Example 9.1.2 Consider the two-way layout described in Figure 9.1.2. Fixed factor A has three levels, fixed factor B has two levels, and there are r ll = r32 = 2 and rl2 -- r21 - r31 = 1 replicate observations per A, B combination. Note there are no observations in the (i, j ) = (2, 2) A, B combination. As in Section 7.3, we develop the model for this experiment using a pattem matrix. Let Y* = X*3 + E* describe an experiment with one observation in each of the six distinct combinations of factors A and B where the 6 x 1 random vector Y* = (Ylll, YI21, Y211, Y221, Y311, Y321)', the 6 x 6 matrix X* - [X11Xx[X3IX4], the 6 x 1 vector/3 = (i31. . . . . /36)' and the 6 x 1 error vector E* = ( E l l l , El21, E211, E221, E311, E321) t with X 1 = 13 | 12, X2 -- Q3 @ 12, X3 = 13 | Q2, X4 = Q3 | Q2, Q2 = (1, - 1 ) ' , and

Q3 -

E 1 1] -1 0

1 -2

Let the 7 x 1 random vector of actual observations Y = (Ylll, Y112, Y121, Y211, Y311, Y321, Y322)'. Therefore, the model for the actual data set is

Y = RMX*~ + E where the 7 x 5 replication matrix R, the 5 x 6 pattern matrix M, and the 7 x 1

Fixed factor A (i) 1

Ylll Yll2

2

Y121

Fixed factor B (j)

Figure 9.1.2

V211

V311 Y321 Y322

Less than Full Rank Example Using a Pattem Matrix.

164

Linear Models

vector E are given by 12

1 R=

0 1

0

,M1

12

1 0 0 0 0

0 1 0 0 0

0 0 1 0 0

0 0 0 0 0

0 0 0 1 0

0 0 0 0 1

and E = (Elll, El12, El21, E211, E311, E321, E322)'. The preceding model can be rewritten as Y = RXd/~ + E where the 5 • 6 matrix Xd = MX*. In this problem, n = 7, p = 6, k = 5, with X~DXd is a 6 • 6 singular matrix of rank 5. The main difficulty with Example 9.1.2 is that p - 6 fixed parameters are used to depict the k - 5 distinct fixed treatment combinations that contain data. Note that the use of a pattern matrix did not solve the overparameterization problem. In the next section the mean model is introduced to solve this overparameterization problem.

9.2

THE MEAN MODEL SOLUTION

In less than full rank models, the number of fixed parameters is greater than the number of distinct fixed treatment combinations that contain data. As a consequence, the least-squares estimator of/3 does not exist and the analysis cannot be carried out as before. One solution to the problem is to use a mean model where the number of fixed parameters equals the number of distinct fixed treatment cornbinations that contain data. Examples 9.1.1 and 9.1.2 are now reintroduced to illustrate how the mean model is formulated.

Example 9.2.1 Reconsider the experiment described in Example 9.1.1. Let E(Yij) = [,,I,i represent the expected value of the jth observation in the ith fixed treatment level. Use the mean model

Yij = #i -'1"-Eij. In matrix form the mean model is given by Y=R/~+E where the 3 • 1 vector/~ = (/zl, #2,/z3)' and where Y, R, and E are defined as in Example 9.1.1. Note the 6 • 3 replication matrix R has full column rank k = 3.

9

Less T h a n Full R a n k M o d e l s

165

Example 9.2.2

Reconsider the experiment described in Example 9.1.2. Let represent the expected value of the k th observation in the ij th combination of fixed factors A and B. Use the mean model E(Yijk)

--

]s

Y=R#+E where the 5 x 1 vector # = ( / s 1 6 3 1 6 3 1 6 3 and where Y, R, and E are defined as in Example 9.1.2. Note the 7 x 5 replication matrix R has full column rank k - 5. In general, the less than full rank model is given by Y - RXd/~ + E

where the k x p matrix Xd had rank k < p. The equivalent mean model is Y=R#+E where the n x k replication matrix R has full column rank k and the elements of the k x 1 mean vector # are the expected values of the observations in the k fixed factor combinations that contain data. Since the two models are equivalent R # -- RXd/~. Premultiplying each side of this relationship by ( R ' R ) - I R ' produces # = Xd/3.

This equation defines the relationship between the vector/~ from the mean model and the vector/5 from the overparameterized model.

9.3

M E A N M O D E L A N A L Y S I S W H E N C O V ( E ) - o'2In

The analysis of the mean model follows the same analysis sequence provided in Chapter 5. Since the n x k replication matrix R has full column rank, the ordinary least-squares estimator of the k x 1 vector/~ is given by /2 = ( R ' R ) - I R ' y = D - 1 R ' y = ~,

where Y = D - t R ' y is the k x 1 vector whose elements are the averages of the observations in the k distinct fixed factor combinations that contain data. The least-squares estimator/2 is an unbiased estimator of/z since E(/~) = E [ ( R ' R ) - I R ' y ] = (R'R)-IR'R/~ =/~.

166

Linear Models Table 9.3.1

Mean Model ANOVA Table Source

df

SS

Overall mean

1

Treatment combinations

k-

y , n1J n Y

Residual

n - k

Y'[In - R D - 1 R ' ] Y

Total

n

Y'Y

1

= Y'A1Y

Y'[RD-1R '-

1Jn]Y = Y'A2Y = Y'ApeY

The k x k covariance matrix of/2 is given by COV(/2) - - ( R ' R ) - I R

'(o'2In)R(R'R)

-1 -

c r 2 D -1

and the least-squares estimator of tr 2 is $ 2 _ V'[I~ - R ( R ' R ) - I R ' ] Y / ( n = V'[In - R D - 1 R ' ] Y / ( n

- k)

- k)

= Y'ApeY/(n - k) where Ape is the n x n pure error sum of squares matrix originally defined in Section 5.5. The quadratic form $2 provides an unbiased estimator of O"2 since E($ 2) - E[Y'ApeY/(n - k)]

= {tr[Ape(crZIn)] +/_t'R'ApeR/.t}/(n - k) =O"

2

where tr(Ape) = n - k and ApeR - 0nxk. Furthermore, by Theorem 5.2.1, the least-squares estimator t ' # -- t'Y is the BLUE of t'/.t for any k x 1 nonzero vector t. An ANOVA table that partitions the total sum of squares for the mean model is presented in Table 9.3.1. The expected mean squares are calculated below using Theorem 1.3.2 with the k x 1 mean vector/, - ( # 1 , # 2 . . . . . #k) t.

EMS (overall mean) - E [ Y '1nJn Y ]

= tr[(1/n)cr2jn] + # ' R ' - J n R # I n

1

k

= cr2 + - Z ?1

k

Zrurv#u#v

u=l v=l

9

Less T h a n Full R a n k Models

167

EMS (treatment c~176

= E IY' (RD-1R' - ljn) Y] /(k -- { tr[0.ZRD-

1R' ] =

,-n =

(n -

~'R'RD-1R'R~

_1, ru (ru - 1)lZ2u

1)0.2 + u--1

- (2/n)

k

1

rur~lZulZv /(k-

Z

l)

u=l,u F~,f.

186

Linear Models

All of these fixed portion calculations are performed for an example problem in the next section.

10.6

FIXED PORTION ANALYSIS: A NUMERICAL E X A M P L E

An estimate of #, a confidence band on t'#, and a hypothesis test on the fixed treatment effect are now calculated using the data in Section 10.4. The numerical calculations are performed using the SAS PROC IML procedure detailed in Section A5.3 of Appendix 5. All calculations are made assuming a finite model with Type I sums of squares estimates for ~ , cr~r, and aR(BT)" 2 From Section A5.3 the estimate of ~ = (#1,/22)' is given by ~w - ( ~ , / ~ 2 ) ' -

(C'R'E-1RC) - 1 C ' R ' ~ - ~ Y -

[227.94] 182.62 "

The estimated covariance matrix of the weighted least squares estimator of ~ is co~-~(~w) = (C,R,~_IRC)_I = [ 295.78 88.33 An estimate of the BLUE of/x~

-

#2

"-- t t j L t

,~ t~w--(1,-1)

88.33] 418.14 "

fort' = (1, - 1 ) is given by

[227.94] 182.62 =45.32.

The variance of the BLUE of #~ - #2 is estimated by var(t ~w) = t ' ( C ' R ' s

~t 88.33

Therefore, 95% confidence bands on

~1

418.14 --

~2

-1

= 537.26.

are

t'~w + Zo.o25v / ~ ( t ' ~ w ) - 45.32 -4- 1.96~/537.26 = (-0.11, 90.75). A Satterthwaite test on the treatment means can be constructed. Note that E [ Y ' A 2 Y / 1 ] - a 2 + 1.5r

+

r

*(T)

where Y'AzY is the Type I sum of squares due to treatments given the overall mean and 9 (T) is the fixed portion of the treatment EMS. Furthermore, 0.2 + 1.5cr2r + tx~(sr ) -- c,E[Y'A3Y/2] + ceE[Y'A4Y] + c3E[Y'AsY/3] where Cl = 1/2, c2 = 1.3/1.2, and c3 = -0.7/1.2. Therefore, the Satterthwaite

10

General Mixed Model

187

statistic to test a significant treatment effect is MSF/W =

=

Y'A2Y/1 Cl (Y'A3Y/2) + c2(Y'A4Y/1) + c3(Y'AsY/3) 6728 1419.51

= 4.74.

The degree of freedom for the denominator of the Satterthwaite statistic is W2

~= / (clY'A3Y/2)22"~ (c2Y'A4Y/1)21"q- (c3Y'A}sY/3)2 3 =

(1419.51) 2

{ (2770"8/4)22-~ [(1"3/1"2)(740"03)]21 ~- [(-~

= 2.28.

Since M S F / W -- 4.74 < 18.5 - F 1,2 ~176, do not reject the hypothesis that there is no treatment effect. EXERCISES For Exercises 1-6, assume that E has a multivariate normal distribution with mean vector 0 and covariance matrix E = ~-'~7=1o'}UfU~. 1. Write the general mixed model Y = RC/~ + E for the two-way cross classification described in Example 4.5.2. Define all terms and distributions explicitly. 2. Write the general mixed model Y = RC/~ + E for the split plot design described in Example 4.5.3. Define all terms and distributions explicitly. 3. Write the general mixed model Y = RC/~ + E for the experiment described in Example 4.5.4. Define all terms and distributions explicitly. 4. Write the general mixed model Y = RC/~ + E for the experiment described in Figure 7.3.1. Define all terms and distributions explicitly. 5. Write the general mixed model Y = RC/~ + E for the experiment described in Exercise 12 in Chapter 5. Define all terms and distributions explicitly. 6. Write the general mixed model Y = R C ~ + E for the experiment described in Exercise 7 in Chapter 4. Define all terms and distributions explicitly. 7. Letthe 8 x 1 vector of observations Y = (Ylll, Yl12, Y121, Y122, Y211, Y:I:, Y2:l, Y222)' = (2, 5, 10, 15, 17, 14, 39, 41)' represent the data for the experiment described in Example 10.2.1 with b = t = r = 2.

188

Linear Models

1

W. Plot (j) Split plot (k)

1

23.7

2

28.9

3

17.8

Figure 10.6.1

37.2

Replicates (i) 2 W. Plot (j)

3 W. Plot (j)

21.4 27.9

60.6

21.9 40.5

32.6

21.2

45.1

41.6

Split Plot Data Set for Exercise 9.

(a) Find the Type I sum of squares estimates of cr2, a finite model.

crZr, and tYR(BT ) 2 assuming

crZ(Br)

(b) Find the REML estimates of a 2, crZr, and assuming a finite model. Are the Type I sum of squares and REML estimates of the three variance parameters equal? 8. The observations in Table E 10.1 represent the data from the split plot experiment described in Example 4.5.3 with r = s -- 3, t = 2, and with some of the observations missing. (a) Write the general mixed model Y -- RC/~ + E for this experiment where E ~ N13(0, ~]) and Z - E ) = I o'}UTU~. Define all terms and distributions explicitly. (b) Find the Type I sum of squares and REML estimates of the variance parameters defined in part a assuming a finite model. (c) Find the Type I sum of squares and REML estimates of the variance parameters defined in part a assuming an infinite model. (d) Calculate the estimates/~w and cb'~(/2w). (e) Construct a 95% confidence band on the difference between the two whole plot treatment means. (f) Construct a Satterthwaite test for the hypothesis that there is no significant whole plot treatment effect. (g) Construct a Satterthwaite test for the hypothesis that there is no significant split plot treatment effect.

A P P E N D I X 1: C O M P U T E R O U T P U T F O R C H A P T E R 5

Table A1.1 provides a summary of the notation used in the Chapter 5 text and the coresponding names used in the SAS PROC IML computer program. Table AI.1 Chapter 5 Notation and Corresponding PROC IML Program Names Chapter 5 Notation

PROC IML Program Name

Y, X1, Xc, X /~, 6.2 yt 1 nJ.Y

Y, X1, XC, X BHAT, SIG2HAT SSMEAN SSREG SSRES SSTOT WBETAHAT, WSIG2HAT SSWMEAN

ytXc(X'cXc)-IX'cY 1 Y'[In - ~J. - Xc(X'cXc)-~X'c]Y Y'Y /~w, (7^w2 y , v - i [ln (1" V -1 ln) -1 l'n ]V- ] Y Y ' V - 1 [ X ( X ' V - 1 X ) - I X' -

ln(l'nV-11n)-lltn]V-1y

Y'[V -1 - V - 1 X ( X ' V - 1 X ) - I x ' v - 1 ] Y Y'V- 1y Y'ApeY Y'[In -- X(X'X)-]X ' - Ape]Y SS(x] I overall mean) SS(x21 overall mean, xl )

SSWREG

SSWRES S SWTOT SSPE SSLOF SSX1 SSX2

The SAS PROC IML program and output follow:

PROC IML; Y={1.7, 2.0, 1.9, 1.6, 3.2, 2.0, 2.5, 5.4, 5.7, 5.1}; Xl = J(lO, 1, 1); XC = { - 1 5 - 1 0 2 , - 1 5 -102, - 1 5 -102, - 1 5 -102, -15 18, 15 -102, 15 -102, 15 198, 15 198, 15 198 }; X = Xll IXC; BHAT = INV(T(X)*X)*T(X)*Y;

189

190

Linear Models

SIG2HAT = (1/7)#T(Y)*(I(10)- X*INV(T(X)*X)*T(X))*Y; A 1 = X 1 *INV(T(Xl )*Xl )*T(Xl ); A2 = XC*INV(T(XC)*XC)*T(XC); A3 = 1(10) - A1 - A 2 ; S S M E A N = T(Y)*AI*Y; SSREG = T(Y)*A2*Y; SSRES = T(Y)*A3*Y; SSTOT = T(Y)*Y; V = B L O C K ( l , 2) @ 1(5); WBETAHAT = INV(T(X)*INV(V)*X)*T(X)*INV(V)*Y; WSIG2HAT=( 1/7)#T(Y)*(INV(V)- INV(V)*X*I NV(T(X)*INV(V)*X) *T(X)*INV(V))*Y; AV1 = INV(V)*Xl *INV(T(Xl )*INV(V)*Xl )*T(Xl )*INV(V); AV2 = INV(V)*X*INV(T(X)*INV(V)*X)*T(X)*INV(V)- AV1; AV3 = INV(V) - AVl - A V 2 ; S S W M E A N = T(Y)*AVI*Y; S S W R E G = T(Y)*AV2*Y; S S W R E S = T(Y)*AV3*Y; SSWTOT = T(Y)*INV(V)*Y; B1 = 1(4) - J(4,4,1/4); B2 = 0; B3 = 1(2) - J(2,2,1/2); B4 = 1(3)- J(3,3,1/3); APE = BLOCK(B1 ,B2,B3,B4); ALOF = A 3 - A P E ; SSLOF = T(Y)*ALOF*Y; SSPE = T(Y)*APE*Y; R1 = X(11:10,11); R2 = X(I1:10,1:21); T1 = RI*INV(T(R1)*R1)*T(R1); T2 = R2*INV(T(R2)*R2)*T(R2)-T1; T3 = X*INV(T(X)*X)*T(X)-R2*INV(T(R2)*R2)*T(R2); S S X l = T(Y)*T2*Y; SSX2 = T(Y)*T3*Y; PRINT BHAT SIG2HAT SSMEAN SSREG SSRES SSTOT WBETAHAT WSIG2HAT SSWMEAN SSWREG SSWRES SSWTOT SSLOF SSPE S S X l SSX2;

Appendix 1

191

BHAT SIG2HAT SSMEAN SSREG 3.11 0.0598812 96.72124.069831 0.0134819 0.0106124 WBETAHAT WSIG2HAT SSWMEAN SSWREG 3.11 0.0379176 57.408333 14.581244 0.013 0.0107051 SSLOF 0.0141687 QUIT; RUN;

SSPE 0.405

SSRES 0.4191687

SSTOT 121.21

SSWRES SSWTOT 0.2654231 72.255

SSXl SSX2 1 0 . 6 0 9 13.460831

A P P E N D I X 2: C O M P U T E R O U T P U T F O R C H A P T E R 7

A2.1

C O M P U T E R O U T P U T F O R S E C T I O N 7.2

Table A2.1 provides a summary of the notation used in Section 7.2 of the text and the corresponding names used in the SAS PROC IML computer program. Table A2.1 Section 7.2 Notation and Corresponding PROC IML Program Names Section 7.2 Notation

PROC IML Program Name

X* I3 | J3 I3 | (I3 -- 89 X1, Q3, z1, x2, z2 M, X = MX* M[I3 @ J3]M' M[I3 | (I3 -- ~1J3)]M t Sl,T1 A1 . . . . . A4 A1M[I3 | J3]M' . . . . . A4M[I3 | J3]M t A1M[I3 | (I3 - 89 ~. . . . . A4M[I3 | (I3 - 89 XtA1X . . . . . XIA4 X S~, T~ A~, A; X fA~X tr[A~M(I3 | J3)M'] tr{A~M[I3 | (I3 - ~1J3)]M t }

XSTAR SIG 1 SIG2 x1, Q3, z1, x2, z 2 M, X SIGM1 SIGM2 S1, T1 A1 . . . . . A4 A1SIGM1 . . . . . A4SIGM1 A1SIGM2 . . . . . A4SIGM2 XA1X . . . . . XA4X S2STAR, T1STAR A2STAR, A3STAR XA3STARX TRA3ST1 TRA3ST2

~

The SAS PROC IML program for Section 7.2 and the output follow:

PROC IML; XSTAR=J(3, 1, 1)@1(3) SIG1=I(3)@J(3, 3, 1); SIG2=I(3)@(I(3)-(1/3)#J(3, 3, 1)); Xl=J(9, 1, 1); Q 3 = { 1 1, -1 1, 0 - 2 }; ZI=Q3@J(3, 1, 1); X2=J(3, 1, 1)@Q3; Z2=Q3@Q3; 193

194

M={0 1 0 0 0 0 0 0 0 0 0 0 0

Linear Models

0 1 0 0 0 0

0 0 1 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0 1 0 0 1 1 0 }

0 0 0 0 0 ;

0, 0, 0, 0, 0,

X=M*XSTAR; SlGM1 =M*SlG 1*T(M); SlGM2=M*SlG2*T(M); SI=M*Xl; TI=M*(XllIZ1); S2=M*(Xl IlZl IIX2); A1 =$1 *1NV(T(S1)*$1 )*T(S 1); A2=T1 *1NV(T(T1)*T1 )*T(T1 )-A1; A3=S2*INV(T(S2)*S2)*T(S2)-A1 -A2; A4= I(3)@ I(2)-A 1-A2-A3; A1SIGM1 =A1 *SIGM 1; A1SIGM2=AI*SIGM2; A2SIGM1 =A2*SIGM1; A2SIGM2=A2*SIGM2; A3SIGM1 =A3*SIGM1; A3SIGM2=A3*SIGM2; A4SIGMI=A4*SIGM1; A4SIGM2=A4*SIGM2; XAlX=T(X)*AI*X; XA2X=T(X)*A2*X; XA3X=T(X)*A3*X; XA4X=T(X)*A4*X; S2STAR=M*(Xl IIX2); T1STAR=M*(Xl IIX211Zl); A2STAR=S2STAR* INV(T(S2STAR)*S2STAR)*T(S2STAR)-A1 ; A3STAR=T1STAR*I NV(T(T1STAR)*T 1STAR)*T(T1STAR)-A2STA R-A 1; XA3STARX=T(X)*A3STAR*X; TRA3ST1 =TRAC E(A3STAR*SIG M 1); TRA3ST2=TRAC E(A3STAR*SIG M2); PRINT SIG1 SIG2 SIGM1 SIGM2 A1 A2 A3 A4 A1SIGM1 A1SIGM2 A2SIGM1 A2SIGM2 A3SIGM1 A3SIGM2 A4SIGM1 A4SIGM2 XA1X XA2X XA3X XA4X A3STAR XA3STARX TRA3ST1 TRA3ST2

Appendix 2

195

SIG1 1 1 1 0 0 0 0 0 0

1 1 1 0 0 0 0 0 0

1 1 1 0 0 0 0 0 0

0 0 0 1 1 1 0 0 0

0 0 0 1 1 1 0 0 0

0 0 0 1 1 1 0 0 0

0 0 0 0 0 0 1 1 1

0 0 0 0 0 0 1 1 1

0 0 0 0 0 0 1 1 1

SIG2 0.6667 -0.333 -0.333 0 0 0 0 0 0 -0.333 0.6667 -0.333 0 0 0 0 0 0 -0.333 -0.333 0.6667 0 0 0 0 0 0 0 0 0 0.6667 -0.333 -0.333 0 0 0 0 0 0 -0.333 0.6667 -0.333 0 0 0 0 0 0 -0.333 -0.333 0.6667 0 0 0 0 0 0 0 0 0 0.6667 -0.333 -0.333 0 0 0 0 0 0 -0.333 0.6667 -0.333 0 0 0 0 0 0 -0.333 -0.333 0.6667 SIGM1 1 1 0 0 0 0

1 1 0 0 0 0

0 0 1 1 0 0

0 0 1 1 0 0

0 0 0 0 1 1

0 0 0 0 1 1

SIGM2 0.6667 -0.333 0 0 0 0

-0.333 0.6667 0 0 0 0

0 0 0.6667 -0.333 0 0

0 0 -0.333 0.6667 0 0

0 0 0 0 0.6667 -0.333

0 0 0 0 -0.333 0.6667

196

Linear Models

A1 0.166667 0.166667 0.166667 0.166667 0.166667 0.166667

0.166667 0.166667 0.166667 0.166667 0.166667 0.166667

0.166667 0.166667 0.166667 0.166667 0.166667 0.166667

0.166667 0.166667 0.166667 0.166667 0.166667 0.166667

0.166667 0.166667 0.166667 0.166667 0.166667 0.166667

0.166667 0.166667 0.166667 0.166667 0.166667 0.166667

0.333333 0.333333 -0.16667 -0.16667 -0.16667 -0.16667

-0.16667 -0.16667 0.333333 0.333333 -0.16667 -0.16667

-0.16667 -0.16667 0.333333 0.333333 -0.16667 -0.16667

-0.16667 -0.16667 -0.16667 -0.16667 0.333333 0.333333

-0.16667 -0.16667 -0.16667 -0.16667 0.333333 0.333333

-0.33333 0.333333 -0.16667 0.166667 0.166667 -0.16667

0.166667 -0.16667 0.333333 -0.33333 0.166667 -0.16667

-0.16667 0.166667 -0.33333 0.333333 -0.16667 0.166667

-0.16667 0.166667 0.166667 -0.16667 0.333333 -0.33333

0.166667 -0.16667 -0.16667 0.166667 -0.33333 0.333333

-0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667

-0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667

0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667

0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667

-0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667

A2 0.333333 0.333333 -0.16667 -0.16667 -0.16667 -0.16667 A3 0.333333 -0.33333 0.166667 -0.16667 -0.16667 0.166667 A4 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 A1SIGM1 0.333333 0.333333 0.333333 0.333333 0.333333 0.333333

0.333333 0.333333 0.333333 0.333333 0.333333 0.333333

0.333333 0.333333 0.333333 0.333333 0.333333 0.333333

0.333333 0.333333 0.333333 0.333333 0.333333 0.333333

0.333333 0.333333 0.333333 0.333333 0.333333 0.333333

0.333333 0.333333 0.333333 0.333333 0.333333 0.333333

Appendix 2

197

A1SIGM2 0.055556 0.055556 0.055556 0.055556 0.055556 0.055556

0.055556 0.055556 0.055556 0.055556 0.055556 0.055556

0.055556 0.055556 0.055556 0.055556 0.055556 0.055556

0.055556 0.055556 0.055556 0.055556 0.055556 0.055556

0.055556 0.055556 0.055556 0.055556 0.055556 0.055556

0.055556 0.055556 0.055556 0.055556 0.055556 0.055556

A2SIGM1 0.666667 0.666667 -0.33333 -0.33333 -0.33333 -0.33333

0.666667 0.666667 -0.33333 -0.33333 -0.33333 -0.33333

-0.33333 -0.33333 0.666667 0.666667 -0.33333 -0.33333

-0.33333 -0.33333 0.666667 0.666667 -0.33333 -0.33333

-0.33333 -0.33333 -0.33333 -0.33333 0.666667 0.666667

-0.33333 -0.33333 -0.33333 -0.33333 0.666667 0.666667

0.111111 0.111111 -0.05556 -0.05556 -0.05556 -0.05556

-0.05556 -0.05556 0.111111 0.111111 -0.05556 -0.05556

-0.05556 -0.05556 0.111111 0.111111 -0.05556 -0.05556

-0.05556 -0.05556 -0.05556 -0.05556 0.111111 0.111111

-0.05556 -0.05556 -0.05556 -0.05556 0.111111 0.111111

A2SIGM2 0.111111 0.111111 -0.05556 -0.05556 -0.05556 -0.05556 A3SIGM1 1.1E-16 1.1E-165.51E-175.51E-17 0 0 1.65E - 16 1.65E - 16 5.51E - 17 5.51E - 17 0 0 5.51E-175.51E-17 1.1E-16 1.1E-16 0 0 5.51E - 17 5.51E - 17 1.65E - 16 1.65E - 16 0 0 0 0 0 0 - 1 . 1 E - 16 - 1 . 1 E - 16 0 0 0 0 - 1 . 1 E - 16 - 1 . 1 E - 16 A3SIGM2 0.333333 -0.33333 0.166667 -0.16667 -0.16667 0.166667 -0.33333 0.333333 -0.16667 0.166667 0.166667 -0.16667 0.166667 -0.16667 0.333333 -0.33333 0.166667 -0.16667 -0.16667 0.166667 -0.33333 0.333333 -0.16667 0.166667 -0.16667 0.166667 0.166667 -0.16667 0.333333 -0.33333 0.166667 -0.16667 -0.16667 0.166667 -0.33333 0.333333

198

Linear Models

A4SIGM1 1 . 1 E - 16 1 . 1 E - 16 0 0 0 0 5 . 5 1 E - 17 5 . 5 1 E - 17 0 0 0 0 0 0 1.1E-16 1.1E-16 0 0 0 0 5 . 5 1 E - 17 5 . 5 1 E - 17 0 0 0 0 0 0 1 . 1 E - 16 1 . 1 E - 16 0 0 0 0 1 . 1 E - 16 1 . 1 E - 16 A4SIGM2 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667 -0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667 0.166667 -0.16667 -0.16667 0.166667 XA 1X

XA2X

0.666667 0.666667 0.666667 0.666667 0.666667 0.666667 0.666667 0.666667 0.666667

0.333333 -0.16667 -0.16667 -0.16667 0.333333 -0.16667 -0.16667 -0.16667 0.333333

XA3X

XA4X

1 -0.5 -0.5 -0.5 1 -0.5 -0.5 -0.5 1

1.65E-16 2.76E-17 2.76E-17 2.76E-17 1.65E-16 2.76E-17 -2.8E-17 -2.7E-17 1.65E-16

A3STAR 0.333333 0.166667 0.166667 -0.16667 -0.16667 -0.33333 0.166667 0.333333 -0.16667 -0.33333 0.166667 -0.16667 0.166667 -0.16667 0.333333 0.166667 -0.33333 -0.16667 -0.16667 -0.33333 0.166667 0.333333 -0.16667 0.166667 -0.16667 0.166667 -0.33333 -0.16667 0.333333 0.166667 -0.33333 -0.16667 -0.16667 0.166667 0.166667 0.333333 XA3STARX 2.76E-16 1.65E-16 0 QUIT; RUN;

TRA3ST1 TRA3ST2 1.65E-16 2.76E-16 0

0 0 -2.2E - 16

3

1

Appendix 2

199

In the following discussion, the computer output is translated into the results that appear in Section 7.2. 13 | J3 - SIG1 13|

113-=J3

=SIG2

SO

tr2[I3 |

+ cr2T [I3| ( I 3 - - ~ J 3 ) ] - tr2 SIG1 + tr2T SIG2 M[I3 | J3]M' = 13 | J2 = SIGM1

M[13|

1

_1~J3) M'= [13| (12 _ ~ j2)] = SIGM2

so

M{cr2[13 |

+t72T [13| (13--~J3)] }M '-Cr2 SIGMI +O'2T SIGM2 1

1

~J3 | ~J2 = A1

( 13- 3J3 ) | ~J2 1 = A2

1121 _l I 1

-

1

3 -1

2

= A3

2

1

1] 1 1 | ~J2 = A4

1 1

-1

3 1

-1

1

1 |

1

1

A1M[I3 | J3 ]M' = A 1SIGM 1 = 2A 1 1

A1M [13 | (13 - ~J3)l M' = A1SIGM2 = -A1 3 SO

AIM{or2[13 | J3] + tr2r [13 | (13 - ~J3)] } M' = o.2 A1SIGM1 + cr2r A 1SIGM2

200

Linear Models

A2M[13 | J3]M' = A2SIGM1 = 2A2 A2M[13 | (I3

_

1j

1

SO

A2M{cr2[13 |

+trOT [13| (13--~J3)] }M' = o.2A2SIGM 1 + cr~rA2SIGM2 1

A3M[I3 | J3]M' -- A3SIGM1 - -

06x 6

[ (13 - ~J3')M'- A3SIGM2 = A3

A3M 13 | SO

A3M{tr2[13 |

+cr2r [13@ (13 - ~ 1j 3)]} M' = tr2A3SIGM1 + o2r A3SIGM2 = cr~r A3 A4M[13 | J3]M' = A4SIGM1 = 06x6 1

A4M [13 | (13 - ~J3) ] M' - A4SIGM2 -- A4 SO

A4M{cr~[13 | J3] + cr~r [13 | (I3 - ~J3) ] }M' - o-2A4SIGM1 + cr~rA4SIGM2 - cr~rA4 2 X'A1X - XA1X- :J3 3 SO

2 /3'X'A1X/3 =/3'~J3/3 - ~(/31 + f12 -4- f13) 2

Appendix 2

201

X'A2X : XA2X = ~1 13- ~

3

SO

~'X'AzX/~ - / ~ ' ~

1j) 1 13 - ~ 3 /~ = ~(/~12+ / ~ + r - ~1/~2 -/~/~3 - r

2

X'A3X=XA3X= ( I 3 - ~ J 3 ) SO

/~'X'A3XC] : / ~ ' (I3

--

~J3),8 = (,62 +/~2 + ,823 --

X ' A 4 X ---

XA4X-

/~1] ~2 -- /~1r

-- /~21~3)2

03x3

so

fl'X'A4Xfl = 0 !

,

X A3X :

XA3STARX - -

03x3

SO

/~'X'A~Xfl -- 0 tr[A~o'seM(I3 | J3)M' + tr{A~o~rM [13 | (13 - ~J3)]M'} = cr~TRA3ST1 + cr2TTRA3ST2 : 34 +

A2.2

COMPUTER OUTPUT FOR SECTION 7.3

Table A2.2 provides a summary of the notation used in section 7.3 of the text and the corresponding names used in the SAS PROC IML computer program. The program names XSTAR, SIG1, SIG2, X1, Q3, Z1, X2, Z2, and M are defined as in Section A. 1 and therefore are not redefined in Table A2.2.

202

Linear Models Table A2.2 Section 7.3 Notation and Corresponding PROC IML Program Names Section 7.3 Notation

PROC IML Program Name

R,D

R,D SIGRM1 SIGRM2 SIGRM3 X S1, T1, $2 A1 . . . . . A5 A1SIGRM1 . . . . . A5SIGRM1 A1SIGRM2 . . . . . A5SIGRM2

RM[13 | J3]M'R' RM[13 | (13 - ~J3)]M 1 ,R , RMI9M~R ' X = RMX*

S1, T1,S2 A1 . . . . . A5 A1RM[I3 | J3]M'R'A1 . . . . . AsRM[I3 | Ja]M'R'A5 A1RM[I3 | (I3 1Ja)]M'R'A1 . . . . . A5RM[I3 | (I3 - 89 AII9A1 . . . . . A519A5 X/A1X . . . . . X'AsX -

-

s~, TT A~,A; A~RM[13 | Ja]M'R'A2, A;RM[I3 | Ja]M'R'A; 1 )]M'R'A~ A~RM[I3 | (I3 - ~J3 A~RM[I3 | (I3 - ~ ~J3)]M'R'A~ AEI9A2, AaI9A 3 * X'A~X X ' A2X, tr{A~RM[I3 | J3]M'R'} tr{A~RM[I3 | (13 - ~J3)]M 1 ,R , } tr{A~I9}

A 1SIGRM3 . . . . . A5 SIGRM3 XA1X . . . . . XA5X S2STAR, T1 STAR A2STAR, A3STAR A2STARM 1, A3STARM 1 A2STARM2 A3STARM2 A2STARM3, A3STARM3 XA2STARX, XA3STARX TRA3ST1 TRA3ST2 TRA3ST3

The SAS PROC IML program for Section 7.3 follows:

PROC IML;

XSTAR=J(3, 1, 1)@1(3); SIG1=I(3)@J(3, 3, 1); SIG2=I(3)@(I(3)-(1/3)#J(3, 3, 1)); Xl=J(9, 1, 1); Q3={ 1 1, -1 1, 0 -2}; ZI=Q3@J(3, 1, 1); X2=J(3, 1, 1)@Q3; Z2=Q3@Q3; M={O 1 0 001 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 00 0 0 1 0 0 0 0 1 0 } ;

O, O, O, O, O,

Appendix 2

203

R=BLOCK(1,J(2, 1, 1), J(2, 1, 1), 1, 1, J(2, 1, 1)); D=T(R)*R; X=R*M*XSTAR; SIGRM 1=R*M*SIG 1*T(M)*T(R); SIGRM2=R*M*SIG2*T(M)*T(R); SIGRM3=I(9); $1 =R*M*X1; TI=R*M*(XlIIZl); S2=R*M*(Xl IlZl I IX2); A1 =$1 *INV(T(S 1)*$1 )*T(S 1); A2=TI*INV(T(T1)*T1)*T(T1)-A1; A3=S2*INV(T(S2)*S2)*T(S2)-A1 - A2; A4=R*INV(D)*T(R) - A1 - A 2 -A3; A5=1(9)- R*INV(D)*T(R); A1SIGRM 1=A1 *SIGRM 1*A1; A 1SIG RM2=A 1*SIG RM2*A 1 ; A 1SIG RM3=A 1*SIG RM3*A 1 ; A2SIG RM 1=A2*SIG RM 1*A2; A2SIGRM2=A2*SIGRM2*A2; A2SIGRM3=A2*SIGRM3*A2; A3SIG RM 1=A3*SIG RM 1*A3; A3SIGRM2=A3*SIGRM2*A3; A3SIG RM3=A3*SIG RM3*A3; A4SIG RM 1=A4*SIG RM 1*A4; A4SIG RM2=A4*SIGRM2*A4; A4SIG RM3=A4*SIGRM3*A4; A5SIG aM 1=A5*SIG aM 1*A5; A5SIGRM2=A5*SIGRM2*A5; A5SIGRM3=A5*SIGRM3*A5; XAlX = T(X)*AI*X; XA2X = T(X)*A2*X; XA3X = T(X)*A3*X; XA4X = T(X)*A4*X; XA5X = T(X)*A5*X; S2STAR=R*M*(Xl IIX2); T1STAR=R*M*(Xl IIX211Z1); A2STAR=S2STAR*INV(T(S2STAR)*S2STAR)*T(S2STAR)-A1 ; A3STAR=T1 STAR* IN V(T(T1STAR)*T 1STAR)*T(T1STAR)-A2STAR-A1 ; A2STARM 1=A2STA R* SIGRM 1*A2STAR; A2STARM2=A2STAR*SIGRM2*A2STAR; A2STARM3=A2STAR*SIGRM3*A2STAR; A3STA RM 1=A3STAR*SIG RM 1*A3STAR;

204

Linear Models

A3STARM2=A3STAR*SIGRM2*A3STAR; A3STARM3=A3STAR*SIG RM3*A3STAR; XA2STARX = T(X)*A2STAR*X; XA3STARX = T(X)*A3STAR*X; TRA3ST1 =TRACE(A3STAR*SIGRM 1); TRA3ST2=TRACE(A3STAR*SIG RM2); TRA3ST3=TRACE(A3STAR*SIG RM3); PRINT A1 A2 A3 A4 A5 A1SIGRM1 A1SIGRM2 A1SIGRM3 A2SIGRM1 A2SIGRM2 A2SIGRM3 A3SIGRM1 A3SIGRM2 A3SIGRM3 A4SIGRM1 A4SIGRM2 A4SIGRM3 A5SIGRM1 A5SIGRM2 A5SIGRM3 XAlX XA2X XA3X XA4X XA5X XA3STARX TRA3ST1 TRA3ST2 TRA3ST3; QUIT; RUN; Since the program output is lengthy, it is omitted. In the following discussion, the computer output is translated into the results that appear in Section 7.3.

A1RM[I3 | J3]M'R'A1 = AISIGRMI = 3A1 AI RM

13 i~)

13 -- ~ J3

A1 - - m I SIGRM2 = 3 A I I 9 A 1 ---= A I S I G R M 3

so

AIEA1--

1

= A1

( 3cr~ + ~CrST 22 +C~2(8T)) A1

A2RM[I3 | J3]M'R'A2 = A2SIGRMI = 3A2

[ (

AERM 13|

13-g

3 M'R'A2=A2SIGRM2=~ 1,)1 A219A2 = A2SIGRM3 = A2

so

A2EA2-

2 2 )A2 3cr~ + ~crEr + crg(sr)

A3RM[I3 | J3]M'R'A3 = A3SIGRM1

[ (

A3RM 13 |

= 09•

1)1 M'R'A3 = A3SIGRM2 = ~A3 4

13 - ~J3

A319A3 = A3SIGRM3 = A3

Appendix 2

205

SO

A4RM[I3 | J3]M'R'A4 = A4SIGRM1

= 09•

4 A4RM [13 | (13 - ~J3) ] M'R'A4 - A4SIGRM2 - ~A4 A419A4 - A4SIGRM3 - A4 SO A4EA4 -

4 2 -+-CrR(BT 2 ))A4 -~rBT

AsRM[13 | J3]M'R'A5 - A5SIGRM1 - 09x9 AsRM [I3 | (13 - ~J3) ] M'R'A5 - A5SIGRM2 - 09x9 AsI9A5 - A5SIGRM3 = As so 2 ) As AsEA5 - oR(sr

X'A1X = XA1X - J3 so /3'X'A1X,O : ,0'J3/3- (131 -+- f12 -4- f13)2

(l)

X ' A 2 X - XA2X = 2 I3 -

5

J3

so

,,X,A2X,-,,2(

I3 - -

X'A3X = X A 3 X -

2

~J3 /3 - 5(/~12 + / ~ + / ~

- ~1/~2 - / ~ 1 / ~ 3 - / ~ 2 / ~ 3 ) 2

4( l)

~ 13 - ~J3

so

/3,XtA3X~_~,4~ ( 13 -

1 ) /3 - ~(/~12 4 ~J3 + f122 + / ~ -/~1r

X'A4X - XA4X - 03x 3

-/~1/~3 - r

2

206

Linear Models

so /3'X'A4X/~

-

0

X'AsX = XA5X

-- 03•

so

/TX'AsX/3 - 0

X'A~X - XA3STARX

-- 03•

SO !

!

,

/3 X AgX/3 = 0

tr[A~cr2RM(13 | J3)M'R'] + tr A3CrBTRM 13 | 9

13 - ~J3

2

+ tr{AaCrR(BT)I9} = ~2 TRA3ST1 + cr2r TRA3ST2 2 TRA3ST3 + aR(BT) =

4ff 2

2 ). + ~02T + 2tTR(BT

) M'R'}

A P P E N D I X 3: C O M P U T E R O U T P U T F O R C H A P T E R 8

Table A3. l provides a summary of the notation used in Section 8.3 of the text and the corresponding names used in the SAS PROC IML computer programs and outputs. Table A3.1 Section 8.3 Notation and Corresponding PROC IML Program Names Section 8.3 Notation

PROC IML Program Names

b,t,k

B,T,K N,M

N,M,

The SAS PROC IML programs and the outputs for Section 8.3 follow:

PROC IML; THIS PROGRAM USES THE DIMENSIONS B, T, AND K AND THE T x B INCIDENCE MATRIX N TO CREATE THE BK 9 BT PATTERN MATRIX M FOR A BALANCED INCOMPLETE BLOCK DESIGN. THE PROGRAM IS RUN FOR THE EXAMPLE IN FIGURE 7.2.1 WITH B=T=3, K=R=2, LAMBDA=I. THE PROGRAM CAN BE GENERALIZED TO PRODUCE THE BK x BT PATTERN MATRIX M FOR ANY BALANCED INCOMPLETE DESIGN BY SIMPLY INPUTING THE APPROPRIATE DIMENSIONS B, T, K AND THE APPROPRIATE INCIDENCE MATRIX N. B=3; T=3; K=2; N={ 0 1 1, 1 0 1, 1 1 0 }; BK=B#K; BT=B#T; M=J(BK, BT, 0); ROW=I ; COL=0; DO I=1 TO B; DO J=l TO T; COL=COL+ 1 ; IF N(IJ, II)=1 THEN M(IROW, COLI)=I; IF N(IJ, II)=1 THEN ROW=ROW+l; END; END; PRINT M;

207

208

M 0 0 0 0 0 0

Linear Models

1 0 0 0 0 0

0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0

0 0 0 1 0 0

0 0 0 0 1 0

0 0 0 0 0 1

0 0 0 0 0 0

QUIT; RUN; PROC IML; THIS PROGRAM USES THE DIMENSIONS B, T, AND K AND THE BK x BT PATTERN MATRIX M TO CREATE THE T 9 B INCIDENCE MATRIX N FOR A BALANCED INCOMPLETE BLOCK DESIGN. THE PROGRAM IS RUN FOR THE EXAMPLE IN FIGURE 7.2.1 WITH B=T=3, K=R=2, LAMBDA=I . THE PROGRAM CAN BE GENERALIZED TO PRODUCE THE T x B INCIDENCE MATRIX N FOR ANY BALANCED INCOMPLETE BLOCK DESIGN BY SIMPLY INPUTING THE APPROPRIATE DIMENSIONS B, T, K AND THE APPROPRIATE PATTERN MATRIX M. B=3; T=3; K=2; M= { 0 1 0 0 0 0 0 0 0, 0 0 1 0 0 0 0 0 0, 0 0 0 1 0 0 0 0 0, 0 0 0 0 0 1 0 0 0, 0 0 0 0 0 0 1 0 0, 0 0 0 0 0 0 0 1 0 BK=B# K; N=J(T, B, 0); ROW=I ; COL=0; DO I=1 TO B; DO J=l TO T; COL=COL+ 1 ; IF M(IROW, COLI)=I THEN N(IJ, II)=1; IF M(IROW, COLI)=I & ROW < BK THEN ROW=ROW+l; END; END; PRINT N; N 0 1 1 1 0 1 1 1 0 QUIT; RUN;

A P P E N D I X 4: C O M P U T E R O U T P U T F O R C H A P T E R 9

Table A4.1 provides a summary of the notation used in Section 9.4 of the text and the corresponding names used in the SAS PROC GLM computer program and output.

Table A4.1 Section 9.4 Notation and Corresponding PROC GLM Program Names Section 9.4 Notation

Proc GML Program Names

Levels of factor A Replication number Observed Y

A R Y

The SAS PROC GLM program and the output for Section 9.4 follow:

DATA A; INPUT A R Y; CARDS; 1 1 101 1 2 105 1 3 94 2 1 84 2 2 88 3 1 32 PROC GLM DATA=A; CLASSES A; MODEL Y=A/P SOLUTION; PROC GLM DATA=A; CLASSES A; MODEL Y=A/P NOINT SOLUTION; QUIT; RUN;

209

210

Linear Models

General Linear Models Sums of Squares and Estimates for the Less Than Full Rank Model Dependent Variable: Y Source Model Error Cor. Total Parameter

DF

Sum of Squares

Mean Square

2 3 5

3480.00 70.00 3550.00

1740.00 23.33

Estimate

INTERCEPT A

32.00 68.00 54.00 0.00

B B B B

F Value

Pr>F

74.57

0.0028

TforH0: Parameter=0

Pr>ITI

StdErrorof Estimate

6.62 12.19 9.13 .

0.0070 0.0012 0.0028 .

4.83045892 5.57773351 5.91607978 .

NOTE" The X'X matrix has been found to be singular and a generalized inverse was used to solve the normal equations. Estimates followed by the letter 'B' are biased, and are not unique estimators of the parameters. General Linear Models Estimates for the Full Rank Mean Model Parameter A

1 2 3

TforH0: Parameter=0

Pr>ITI

Estimate

StdErrorof Estimate

100.00 86.00 32.00

35.86 25.18 6.62

0.0001 0.0001 0.0070

2.78886676 3.41565026 4.83045892

In the following discussion, the preceding computer output is translated into the results that appear in Section 9.4. The less than full rank parameter estimates in the output are titled INTERCEPT, A 1, 2, 3. These four estimates are SAS's solution for/~0 = (32, 68, 54, 0)'. The mean model estimates in the output are titled A 1, 2, 3. These three estimated equal/2 - (100, 86, 32)'. Note that the ANOVA table given in the SAS output is generated by the PROC GLM statement for the less than full rank model. However, the sums of squares presented are equivalent for the mean model and the less than full rank model. Therefore, the sum of squares labeled MODEL in the output is the sum of squares due to the treatment combinations for the data in Figure 9.4.1. Likewise, the sums of squares labeled error and cor. total in the output are the sums of squares due

Appendix 4

211

to the residual and the sum of squares total minus the sum of squares due to the overall mean for the data in Figure 9.4.1, respectively. As a final comment, for a problem with more than one fixed factor, the classes statement should list the names of all fixed factors and the model statement should list the highest order fixed factor interaction between the ' = ' sign and the ' / ' . For example, if there are three fixed factors A, B, and C, then the CLASSES statement is

CLASSES A B C; and the model statement is

MODEL Y=A*B*C/P SOLUTION; (for the less than full rank model) MODEL Y=A*B*C/P NOINT SOLUTION; (for the mean model).

APPENDIX 5: COMPUTER OUTPUT FOR CHAPTER 10

A5.1

C O M P U T E R O U T P U T F O R S E C T I O N 10.2

Table A5.1 provides a summary of the notation used in Section 10.2 of the text and the corresponding names used in the SAS PROC IML computer program and output. Table A5.1 Section 10.2 Notation and Corresponding PROC IML Program Names Section 10.2 Notation

PROC IML Program Names

I3 | J2,13 ~ (I2 - 1j2), Xl, Q2, Q3 x2, z1, z2, M, R, D, C RM(13 | J2)M'R', RM[I3 | (I2 - Ij2)]M'R', I8 TI, A1, A2, A3, A4, A5 tr[A1RM(I3 | J2)M'R'] . . . . . tr[A5RM(I3 | JE)M'R'] tr{A1RM[I3 | (I2 - 89 . . . . . tr[As[I3 | (I2 - 1J2)]} tr(AlI8) . . . . . tr(AsI8) C'R~AIRC . . . . . C'R'AsRC

SIG 1, SIG2, X 1, Q2, Q3 X2, z1, z2, M, R, D, C SIGRM1, SIGRM2, SIGRM3 T1, A1, A2, A3, A4, A5 TRA 1SIG 1. . . . . TRA5 SIG 1 TRA 1SIG2 . . . . . TRA5 SIG2 TRA1SIG3 . . . . . TRA5SIG3 RCA1RC . . . . . RCA5RC

The SAS PROC IML program and output for Section 10.2 follow:

PROC IML; SIG1=I(3)@J(2, 2, 1); SIG2=I(3)@(I(2)-(1/2)#J(2, 2, 1)); Xl=J(6, 1, 1); Q2={1, -1}; Q3={ 1 1, -1 1, o

-2};

X2=J(3, 1, 1) @Q2; ZI=Q3@J(2, 1, 1); Z2=Q3@Q2; M={1 0 0 0 0 0 , 010000, 001000, 000010, 000001}; R=BLOCK (1, 1, J(2, 1, 1), 1, J(3, 1, 1)); D=T(R)*R; 213

214

Linear Models

C={1 0, 0 1, 1 0, 1 0, o 1}; SIGRM 1=R*M*SIG 1*T(M)*T(R); SIGRM2=R*M*SIG2*T(M)*T(R); SIGRM3=I(8); SI=R*M*Xl; S2=R*M*(Xl IIX2); TI=R*M*(Xl IIX211Zl); A1 =$1 * INV(T(S 1)*$1 )*T(S 1); A2=S2*INV(T(S2)*S2)*T(S2) - A1; A3=TI*INV(T(T1)*T1)*T(T1) - A1 - A2; A4=R*INV(D)*T(R) - A1 - A 2 -A3;

A5=1(8)- R*INV(D)*T(R); TRA 1SIG 1=TRAC E(A 1*SIG RM1); TRA 1S IG2=TRAC E(A 1*SIG RM2); TRA 1S IG3=TRAC E(A 1*SIG RM3); TRA2SIG 1=TRACE(A2*SIG RM1); TRA2SIG2=TRACE(A2*SIG RM2); TRA2SIG3=TRACE(A2*SIG RM3); TRA3SIG 1=TRAC E(A3*SIG RM1); TRA3SIG2=TRACE(A3*SIG RM2); TRA3SIG3=TRACE(A3*SIG RM3); TRA4S IG 1=TRACE(A4*SIG RM1); TRA4SIG2=TRAC E(A4* S IG RM2); TRA4SIG3=TRACE(A4*SIG RM3); TRA5SIG 1=TRACE(A5*SIG RM 1); TRA5SIG2=TRAC E(A5* S IG RM2); TRA5SIG3=TRACE(A5*SIG RM3); RCA1RC = T(C)*T(R)*A1 *R'C; RCA2RC = T(C)*T(R)*A2*R*C; RCA3RC = T(C)*T(R)*A3*R*C; RCA4RC = T(C)*T(R)*A4*R*C; RCA5RC = T(C)*T(R)*A5*R*C; PRINT TRA3SIG1 TRA3SIG2 TRA3SIG3 TRA4SIG1 TRA4SIG2 TRA4SIG3 TRA5SIG1 TRA5SIG2 TRA5SIG3 RCA1RC RCA2RC RCA3RC RCA4RC RCA5RC;

Appendix 5

215

TRA3SIG1 TRA3SIG2 TRA3SIG3 TRA4SIG1 TRA4SIG2 TRA4SIG3 4 0.8 2 0 1.2 1 TRA5SIG1 TRA5SIG2 TRA5SIG3 0 0 3 RCA1RC 2 2 2 2

RCA2RC 2 -2 -2 2

RCA3RC 0 0 0 0

RCA4RC 0 0 0 0

RCA5RC 0 0 0 0

QUIT; RUN;

In the following discussion, the computer output is translated into the results that appear in Section 10.2. E[Y'A3Y] = ~r~tr[A3RM(I3 | J2)M'R'] 1

+ tr2rtr{A3RM [13 | (12 - ~J2) ] R'M' } + cr2(Br)tr(A318) -- crETRAaSIG1 + trErTRAaSIG2 + cr2(sr)TRAaSIG3 = 4o"2 + 0.8trZr + 2crZ(Br) E[Y'A4Y] = trZtr[A4RM(I3 | Jz)M'R'] + trzTtr{A4RM [13 | (12 - ~J2) ] R'M'} + crZ(Br)tr(A418) = crZTRA4SIG1 + aZTTRA4SIG2 + crZ(BT)TRA4SIG3 2 = 1.2trzr + aR(Br)

E[Y'AsY] -- crZtr[AsRM(13 | Jz)M'R'] + crZrtr{AsRM[13 | ( 1 2 - ~ J 2 ) ] R ' M ' } + ~rz(sr)tr(Asl8) = cr2TRA5SIG 1 + a2rTRA5SIG2 + tr2(Br)TRA5SIG3 = 3a2(Rr) Note that C'R'A3RC = RCA3RC = 02x2, C'R'A4RC = RCA4RC = 02• and C'R'AsRC = RCA5RC = 02• which implies #'C'R'A3RC# = #'C'R'AaRC# = #'C'R'AsRC# = 0. Therefore, the E[Y'A3Y], E[Y'AaY], and E[Y'AsY] do not involve #.

216

A5.2

Linear Models

C O M P U T E R O U T P U T F O R S E C T I O N 10.4

Table A5.2 provides a summary of the notation used in Section 10.4 of the text and the corresponding names used in the SAS PROC VARCOMP computer program and output. Table A5.2 Section 10.4 Notation and Corresponding PROC VARCOMP Program Names Section 10.4 Notation

PROC VARCOMP Program/Output Names

B,T,R,Y,R(BT) , ~'~r, , t T R2( B T ) * , , ( r )

B, T, R, Y, Error var(B ), var(T 9 B ), var(Error), Q (T)

~

The SAS program and output for Section 10.4 follow:

DATA A; INPUT B T R Y; CARDS; 1 1 1 237 1 2 1 178 2 1 1 249 2 1 2 268 31 1 186 3 2 1 183 3 2 2 182 323165 PROC VARCOMP METHOD=TYPE1; CLASS T B; MODEL Y=TIB / FIXED=l; RUN; PROC VARCOMP METHOD=REML; CLASS T B; MODEL Y=TIB / FIXED=l; QUIT; RUN; Variance Components Estimation Procedure Class Level Information Class Levels Values T 2 12 B 3 123 Number of observations in data set = 8 Dependent Variable : Y

Appendix 5

217

TYPE 1 SS Variance Component Estimation Procedure Source T B T*B Error Corrected Total Source T B T*B Error

DF 1 2 1 3 7

Type I SS 6728.0000 2770.8000 740.0333 385.1667 10624.0000

Type I MS 6728.0000 1385.4000 740.0333 128.3889

Expected Mean Square Var(Error) + 2 Var(T*B) + Var(B) + Q(T) Var(Error) + 1.4 Var(T*B) + 2 Var (B) Var(Error) + 1.2 Var(T*B) Var(Error)

Variance Component Var(B) Var(T*B) Var(Error)

Estimate 271.7129 509.7037 128.3889

RESTRICTED MAXIMUM LIKELIHOOD Variance Components Estimation Procedure Iteration Objective Var(B) Var(T*B) Var(Error) 0 35.946297 30.899 616.604 158.454 1 35.875323 51.780 663.951 144.162 2 35.852268 66.005 693.601 136.830 3 35.845827 81.878 701.484 133.406 4 35.843705 86.686 711.185 131.422 5 35.843069 87.877 718.188 130.334 6 35.842890 88.250 722.237 129.757 7 35.842842 88.421 724.385 129.458 8 35.842830 88.509 725.494 129.305 9 35.842827 88.553 726.060 129.227 10 35.842826 88.576 726.348 129.187 11 35.842825 88.587 726.493 129.167 12 35.842825 88.593 726.567 129.157 13 35.842825 88.596 726.605 129.152 14 35.842825 88.598 726.624 129.149 Convergence criteria met In the following discussion, the computer output is translated into the results that appear in Section 10.4.

218

Linear Models

Y'A2Y = Type I SS due to T = 6728.0 Y'A3Y = Type I SS due to B = 2770.7 Y'AnY = Type I SS due to T*B = 740.03 Y'AsY = Type I SS due to Error = 385.17

E[Y'A3Y/2] = 2Var(B) + 1.4 Var(T*B) + Var(Error) 2 = 2a2, + 1.4azr, + aR(BT),

E[Y'AaY/1] - 1.2 Var(T*B) + Var(Error) 2 = 1.2o-2~. + crR(BT).

E[Y'AsY/3] -

A5.3

Var(Error) -

aR(BT) ,2

C O M P U T E R O U T P U T F O R S E C T I O N 10.6

Table A5.3.1 provides a summary of the notation used in Section 10.6 of the text and the corresponding names used in the SAS PROC IML computer program and output.

Table A5.3 Section 10.6 Notation and Corresponding PROC IML Program Names Section 10.6 Notation

PROC IML Program Names

13 | J2,13 | (I2 - 132), XI, Q2, Q3 X2, Z1, Z2, M, R, D, C RM(I3 | J2)M'R', RM[I3 | (I2 - 89

SIG 1, SIG2, X 1, Q2, Q3 x2, z1, z2, M, R, D, C SIGRM1, SIGRM2, SIGRM3 COVHAT, MUHAT, COVMUHAT T, TMU, VARTMU

18

~:, ~w, co~'~(#w) t, t'~, v~ar(t', ~w)

t'/~w + Z0.0e5V/~ar(t'/2w) Sl, S2, TI, AI, A2, A3, A4, A5 W, f, Y'AzY/W

LOWERLIM, UPPERLIM S1, $2, T1, A1, A2, A3, A4, A5 W, F, TEST

The SAS PROC IML program and output for Section 10.6 follow:

PROC IML; Y={237, 178, 249, 268, 186, 183, 182, 165}; SIG1=I(3)@J(2, 2, 1); SIG2=I(3)@(I(2)-(1/2)#J(2, 2, 1)); Xl=J(6, 1, 1);

Appendix 5

Q2={1, -1}; Q3={ 1 1, -1 1, o -2}; X2=J(3, 1, 1) @Q2; ZI=Q3@J(2, 1, 1); Z2=Q3@Q2; M={1 0 0 0 0 0, 0 1 0 0 0 0, 0 0 1 0 0 0, 0 0 0 0 1 0, 0 0 0 0 0 1}; R=BLOCK(1, 1, J(2, 1, 1), 1, J(3, 1, 1)); D=T(R)*R; C={1 0, 0 1, 1 0, 1 0, 0 1}; SIG RM 1=R*M*SIG 1*T(M)*T(R); SIG RM2=R*M*SIG2*T(M)*T(R); SIGRM3=I(8); COVHAT=526.56#SIGRM1 + 509.70#SIGRM2 + 128.39#SIGRM3; M U HAT=I NV(T(C)*T(R)* INV(COVHAT)* R*C)*T(C)*T(R) *INV(COVHAT)*Y; COVM U HAT=I NV(T(C)*T(R)*I NV(COVHAT)* R'C); T={1, -1)}; TMU=T(T)*MUHAT; VARTMU=T(T)*COVMUHAT*T; LOWERLI M=TMU- 1.96#(VARTM U**.5); UPPERLIM=TMU+ 1.96#(VARTM U**.5); SI=R*M*Xl; S2=R*M*(X111X2); T1 =R*M*(XI IIX211Zl ); A1 =$1 *1NV(T(S 1)*$1 )*T(S 1); A2=S2*INV(T(S2)*S2)*T(S2) - A1; A3=TI*INV(T(T1)*T1)*T(T1)- A1 - A2; A4=R*INV(D)*T(R)- A1 -A2 -A3; A5=1(8)- R*INV(D)*T(R); W=.25#T(Y)*A3*Y + (1.3/1.2)#T(Y)*A4*Y - (0.7/3.6)#T(Y)*A5*Y; F=W**2/((((.25#T(Y)*A3*Y)**2)/2) + (((1.3/1.2)#T(Y)*A4*Y)**2) + (((0.7#T(Y)*A5*Y/3.6)*'2)/3));

219

220

Linear Models

TEST=T(Y)*A2*Y/W; PRINT MUHATCOVMUHATTMU VARTMU LOWERLIM UPPERLIM W F TEST; MUHAT 227.94 182.62153 LOWERLIM -0.111990 QUIT; RUN;

COVMUHAT 295.7818 88.33466 88.33466 418.14449 UPPERLIM 90.748923

W 1419.5086

TMU 45.318467

F 2.2780974

VARTMU 537.25697

TEST 4.739663

References and Related Literature

Anderson, R. L., and Bancroft, T. A. (1952), Statistical Theory in Research, McGraw-Hill Book Co., New York. Anderson, T. W. (1958), An Introduction to Multivariate Statistical Analysis, John Wiley & Sons, New York. Anderson, V. L., and McLean, R. A. (1974), Design of Experiments, A Realistic Approach, Marcel Dekker, New York. Arnold, S. F. (1981), The Theory of Linear Models and Multivariate Analysis, John Wiley & Sons, New York. Bhat, B. R. (1962), "On the distribution of certain quadratic forms in normal variates," Journal of the Royal Statistical Society, Ser. B 24, 148-151. Box, G. E. P., Hunter, W. G., and Hunter, J. S. (1978), Statistics for Experimenters, John Wiley & Sons, New York. Chew, V. (1970), "Covariance matrix estimation in linear models," Journal of the American Statistical Society 65, 173-181. Cochran, W. D., and Cox, G. M. (1957), Experimental Designs. John Wiley & Sons, New York. Davenport, J. M. (1971), "A comparison of some approximate F-tests," Technometrics 15, 779-790. Draper, N. R., and Smith, H. (1966), Applied Regression Analysis, John Wiley & Sons, New York. Federer, W. T. (1955), Experimental Design, Macmillan, New York. Graybill, E A. (1954), "On quadratic estimates of variance components," Annals of Mathematical Statistics, 25, No. 2, 367-372.

221

222

References and Related Literature

Graybill, E A. (1961), An Introduction to Linear Statistical Models, Volume 1, McGraw-Hill Book Co., New York. Graybill, E A. (1969), An Introduction to Matrices with Applications in Statistics, Wadsworth Publishing Co., Belmont, CA. Graybill, E A. (1976), Theory and Application of the Linear Model, Wadsworth & Brooks/Cole, Pacific Grove, CA. Guttman, I. (1982), The Analysis of Linear Models, John Wiley & Sons, New York. Harville, D. A. (1977), "Maximum likelihood approaches to variance component estimation and to related problems," Journal of the American Statistical Association 72, No. 358, 320-338. Hocking, R. R. (1985), The Analysis of Linear Models, Brooks/Cole, Publishing Co., Belmont, CA. Hogg, R. V., and Craig, A. T. (1958), "On the decomposition of certain )~2 variables," Annals (?f Mathematical Statistics 29, 608-610. John, P. W. M. (1971), Statistical Design and Analysis of Experiments, Macmillan, New York. Kempthome, O. (1952, 1967), The Design and Analysis of Experiments, John Wiley & Sons, New York. Kshirsagar, A. M. (1983), A Course in Linear Models, Marcel Dekker, New York. Milliken, G. A., and Johnson, D. E. (1992), Analysis of Messy Data, Volme 1: Designed Experiments, Von Nostrand Reinhold, New York. Morrison, D. (1967), Multivariate Statistical Methods, McGraw-Hill Book Co., New York. Moser, B. K. (1987), "Generalized F variates in the general linear model," Communications in Statistics: Theory and methods 16, 1867-1884. Moser, B. K., and McCann, M. H. (1996), "Maximum likelihood and restricted maximum likelihood estimators as functions of ordinary least squares and analysis of variance estimators," Communications in Statistics: Theory and Methods, 25, No. 3. Moser, B. K., and Lin, Y. (1992), "Equivalence of the corrected F test and the weighted least squares procedure," The American Statistician 46, No. 2, 122-124. Moser, B. K., and Marco, V. R. (1988), "Bayesian outlier testing using the predictive distribution for a linear model of constant intraclass form," Communications in Statistics: Theory and Methods 17, 849-860. Moser, B. K., Stevens, G. R., and Watts, C. L. (1989), "The two sample T test versus Satterthwaite's approximate F test," Communications in Statistics: Theory and Methods, 18, 3963-3976. Myers, R. H., and Milton, J. S. (1991), A First Course in the Theory of Linear Statistical Models, PWS-Kent Publishing Co., Boston. Nel, D. G., van der Merwe, C. A., and Moser, B. K. (1990), "The exact distributions of the univariate and multivariate Behrens-Fisher statistic with a comparison of several solutions in the univariate case," Communications in Statistics: Theory and Methods 19, 279-298. Neter, J. Wasserman, W., and Kutner, M. H. (1983), Applied Linear Regression Models, Richard D. Irwin, Homewood, IL. Patterson, H. D., and Thompson, R. (1971), "Recovery of interblock information when block sizes are unequal," Biometrika 58, 545-554. Patterson, H. D., and Thompson, R. (1974), "Maximum likelihood estimation of components of variance," Proceedings of the 8 th International Biometric Conference, pp. 197-207. Pavur, R. J., and Lewis, T. O. (1983), "Unbiased F tests for factorial experiments for correlated data," Communications in Statistics: Theory and Methods 13, 3155-3172. Puntanen, S., and Styan, G. P. (1989), "The equality of the ordinary least squares estimator and the best linear unbiased estimator," The American Statistician 43, No. 3, 153-161. Rao, C. R. (1965), Linear Statistical Inference and Its Applications, John Wiley & Sons, New York. Russell, T. S., and Bradley, R. A. (1958), "One-way variances in a two-way classification," Biometrika, 45, 111-129. Satterthwaite, F. E. (1946), "An approximate distribution of estimates of variance components," Biometrics Bulletin 2, 110-114.

References and ~telated Literature

223

Scariano, S. M., and Davenport, J. M. (1984), "Corrected F-tests in the general linear model," Communications in Statistics: Theory and Methods 13, 3155-3172. Scariano, S. M., Neill, J. W., and Davenport, J. M. (1984), "Testing regression function adequacy with correlation and without replication," Communications in Statistics." Theory and Methods, 13, 1227-1237. Scheff6, H. (1959), The Analysis of Variance, John Wiley & Sons, New York. Searle, S. R. (1971), Linear Models, John Wiley & Sons, New York. Searle, S. R. (1982), Matrix Algebra Useful for Statistics, John Wiley & Sons, New York. Searle, S. R. (1987), Linear Models for Unbalanced Data, John Wiley & Sons, New York. Seber, G. A. F. (1977), Linear Regression Analysis, John Wiley & Sons, New York. Smith, J. H., and Lewis, T. O. (1980), "Determining the effects of intraclass correlation on factorial experiments," Communications in Statistics: Theory and Methods: 9, 1353-1364. Smith, J. H., and Lewis, T. O. (1982), "Effects of intraclass correlation on covariance analysis," Communications in Statistcs: Theory and Methods, 11, 71-80. Snedecor, W. G., and Cochran, W. G. (1978), Statistical Methods, The Iowa State University Press, Ames, IA. Steel, R. G., and Torrie, J. H. (1980), Principles and Procedures of Statistics, McGraw-Hill Book Co., New York. Thompson, W. A. (1962), "The problem of negative estimates of variance components," Annals of Mathematical Statistics 33, 273-289. Weeks, D. L., and Trail, S. M. (1973), "Extended complete block designs generated by BIBD," Biometrics 29, No. 3,565-578. Weeks, D. L., and Urquhart, N. S. (1978), Linear models in messy data: Some problems and alternatives," Biometrics, 34, No. 4, 696-705. Weeks, D. L., and Williams, D. R. (1964), "A note on the determination of correctness in an N-way cross classification," Technometrics, 6, 319-324. Zyskind, G. (1967), "On canonical forms, non-negative covariance matrices and best and simple least squares linear estimators in linear models," Annals of Mathematical Statistics 38, 1092-1109. Zyskind, G., and Martin, E B. (1969), "On best linear estimation and a general Gauss-Markov theorem in linear models with arbitrary non-negative covariance structure," SLAM, Journal of Applied Mathematics 17, 1190-1202.

Subject Index

ANOVA, 10, 13-15, 34-36, 49-50, 57, 77-80, 87-88, 90, 103, 114, 127, 134, 137, 152-153, 166, 172-174, 210 Assumptions, 54-58, 91, 161-162

Confidence bands, 126-127, 168, 173-175, 185-186, 188 Chi-square distribution, 19-20, 33, 35, 38, 41-50 Completeness, 108, 110

Determinent of matrix, s e e Matrix Balanced incomplete block design, 138, 149-159 BIB product, s e e Matrix BLUE, 86-87, 89, 100-101,118, 155, 158, 166, 170-171,185-186

Complete, balanced factorial, 11, 13, 15, 29, 53-77, 91, 97-100

Eigenvalues and vectors, 6-11, 13, 21, 36, 41-42, 44, 115, 156 Estimable function, 168-171 Expectation conditional, 29-32, 39 of a quadratic form, 17 of a vector, 17

225

226

Expected mean square, 57, 64-66, 68-70, 72-73, 76-77, 88, 103, 153, 166, 184-185

Linear M o d e l s

Kronecker product, 12-16, 52, 59, 61-63, 98, 131

!

F distribution, 47-49, 77-80, 142, 147 Finite model, s e e Model

Gauss-Markov, 86-87, 89, 118, 155, 170 General linear model, 96-97, s e e a l s o Model ANOVA for, 87 interval estimation, 126-127 point estimation, 86-87, 105-108 replication, 131-137, 144-147 tests for, 119-126

Lack of fit, s e e Least squares regression Lagrange multipliers, 121 Least squares regression lack of fit, 91-94, 102, 133-134 partitioned SS, 87, 91, 94-97, 135 pure error, 91-94, 102, 133-134, 147, 166 ordinary, 85-86, 94, 107, 111, 114, 132-133, 161 weighted, 89-91, 173 Likelihood function, 105, 115, 119-120, 182 Likelihood ratio test, 119-126

m

Helmert, s e e Matrix Hypothesis tests for regression model, 91, 93-94, 101, 125 for factorial experiments, 48, 78, 80, 102, 125-126, 142, 147, 153, 160, 174

Idempotent matrix, s e e Matrix Identity, s e e Matrix Incidence, s e e Matrix Incomplete block design, s e e Balanced incomplete block design Independence of random vectors, 26 of quadratic forms, 45 of random vector and quadratic form, 46 Infinite model, s e e Model Interval estimation, s e e Confidence bands Invariance property, 108, 113-114 Inverse of a matrix, s e e Matrix

Marginal distribution, 25, 30, 39 Matrix BIB product, 15-16, 156-157 covariance, s e e a l s o Random vector, 1l, 17, 24, 53, 58-77, 111, 118, 123, 135-136, 138, 148, 153-155 determinant, of 3, 6-7 diagonal, 2, 7, 9, 16, 21, 132 Helmert, 5, 10-11, 25, 33, 35, 98, 111, 125, 136-137, 178, 182 idempotent, 9-11, 13, 21, 35-36, 41-50, 67, 92, 96, 100, 106, 114, 140, 143, 151,155 identity, 2, 9 incidence, 156 inverse, 2, 12, 21 multiplicity, 7-8, 10, 13 of ones, 2 orthogonal, 4, 6-7, 9, 11, 44, 51 partitioning, 8, 25, 94, 124, 135 pattern, 138-147, 150-152, 158, 163-164, 181

Subject Index positive definite, 8-9 possible semidefinite, 8, 10 rank of, 3, 9-11, 13, 16 replication, 131-137, 144-147, 161-165, 178, 180-181 singularity, 2, 12, 21 symmetric, 3-4, 7, 9-11, 13, 17 trace of, 2, 7, 10 transpose of, 1 Maximum likelihood estimator, 105-109, 111-119, 182 Minimum variance unbiased estimate, s e e UMVUE Mean model, s e e Model Mixed model, s e e Model Model finite, 53-56, 58-60, 64, 178, 183-184, 186 general linear, 97-100, 105 infinite, 56-58, 60, 64, 178, 183-184 less than full rank, 161-164 mean, 28, 164-174, 180 mixed, 177-187 Moment generating function, 18-20, 24, 26, 42 Multivariate normal distribution conditional, 29-32 independence, 26-28 of Y, 24 of transformation BY + b, 24 marginal distribution, 25 MGF of, 24

One-way classification, 13, 27, 47-48, 107-108, 125, 134 Overparameterization, 163-165, 168

Pattern, s e e Matrix Point estimation, s e e a l s o BLUE and UMVUE for balanced incomplete block design, 138-143, 152-159

227 for general linear model, 105-107 least squares, 84-85 variance parameters, 85, 106, 142-143, 152-153, 179-182 Probability distribution, s e e a l s o Random vector chi-square, 19-20, 33-35, 41-50 conditional, 29-32 multivariate normal, 23-36 noncentral corrected F, 48 noncentral F, 47 noncentral t, 47 PROC GLM, 171,209 PROC IML, 85, 140-141,145, 180, 186, 189, 193, 201-202, 207, 213,218 PROC VARCOMP, 183-184, 216 Pure error, s e e Least squares regression

Quadratic forms, 3, 9, 11, 13, 17, 32-36, 41-50, 63, 85, 143, 146, 161,166 independence of, 45-46

Random vector covariance matrix of, 17, 21, 28-29, 58-74, 96, 111,135, 138, 140, 145, 150-151,155, 181,185-186 expectation of, 17 linear transformation, of 17, 24 MGF of, 18-20, 24-26, 42 probability distribution, 17-18 Rank of matrix, s e e Matrix Regression, s e e Least squares regression Rejection region, 49, 94, 124, 128, 142, 147, 153, 168, 174, 185, 187 Reparameterization, 60, 66, 183 Replication, s e e also Matrix, 47-48, 53-58, 60-61, 63, 66-67, 70, 77, 91, 97-98, 107, 126, 150 REML, 182-184, 188

228

Satterthwaite test, 185-188 Sufficiency, 108-111 Sum of squares, 4, 14, 20, 33-35, 47, 49, 60-69, 71, 74, 77-80, 87, 91-96, 98, 100-103, 107, 112, 124, 131, 133-135, 140-143, 145, 147-148, 166, 174 type I, 94-96, 135, 137, 140-143, 145, 147, 152-153, 159, 179-182, 186

Linear Models for variance parameter, 49 likelihood ratio, 124 Satterthwaite, 185 Trace of matrix, s e e Matrix Transformation, 17, 24 Transpose, s e e Matrix Two-way cross classification, 28, 34, 45, 49-50, 53, 66, 69, 135 Type I SS, s e e Sum of squares u

UMVUE, 110-111, 128 t distribution, 47-48, 126, 168 Tests F test, 49, 57, 94, 124, 142, 153, 168, 174, 185 for lack of fit, 91-94 for noncentrality parameters, 48 for regression, 124

Variance components, 135-136, 142-143, 179 Vector, s e e Random vector