Studnes ffiil by SydneyAfriat M.V.RamaSastry GerhardTintner
Vandenhoeck & Ruprecht
Studiesin Correlation Multivariate...
52 downloads
910 Views
4MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Studnes ffiil by SydneyAfriat M.V.RamaSastry GerhardTintner
Vandenhoeck & Ruprecht
Studiesin Correlation Multivariate Analysisand Econometrics
by SydneyAfriat, M. V. Rama Sastry and GerhardTintner
Vandenhoeck& Ruprechtin Gcittingen
Contents
lntroduction. Sydney Afriat Regression and Projection.
Studies in the Algebra of 10
Statistical Correlation. M. V, Rama Sastry and Gerhard Tinher Multivariate
Bibtiography
Analysis.
Econometrics
and Infonnation
Theory
' ' ' ' ' 108
"
' 142
Abstract
The first part gives a geometric and algebraic background to general methods of regression and correlation,
taking into account particulary methods of
multivariate analysis. In the second part Canonical correlation, root criterion, of information
largest
principal components, weighted regression, the relation theory to econometric
estimation and to canonical correlation
and trace correlation is considered. Bayes estimation or regression models is also briefly discussed.
Introduc t ion
s c h e m et o s h o w l e a s t - s q u a r e s l i n e a r regression as resolving sanple vectols of dependent variables i-nto two orthogonal components, a regressional component It
is a familiar
in a space spanned by sample vectors of the i . n d e p e n d e n tv a r i a b l e s , a n d a r e s i d u a l c o n p o n e n t l y i n g lying
in
t h e o r t h o g o n a l c o m p l e m e n t o f t h a t s p a c e . T h i - s s c h e m ew a s i n t r o d u c e d b y K o l m o g o r o v ( 1 9 4 6 )' B u t t h e probably first step from this
to considering
the linear
transformations
these athogonal conponents of sample vectors, these being the complementary pair of symmetri-cidempotents which are the orthogonal projectors on the space and its orthogonal conplement, and then to analysis of relations
which obtain
between variables directly in terms of these projectols' is one that does not appear to have been explored' The advantage of this step apart fron its showing the way to new concepts is that it makes an enlargement methods j.n a natural formalism for handling sone
some possibly of didactic farniliar
theory vrith a direct
geonetrical
neaning, and it
di-sp1ay of
its
algebraical-
gives the framework for
simple
proofs and formulae. Proofs of some relevant algebraical propositions have already been gi-ven elsewhere (Afriat' 1957 and 1956). The view taken of regressl-onanalysis is concepts. orthogonal one that does not depend on distribution projection is presented as the fundamental princi-p1e and the least-squares principle is derived as a property of it. The association of a di-stribution with a sanple is a refinements of this view, since and it can also be considered to justify it'But this vlew can just as well be taken as primitive, in its compatibility with it the distribution nethod can be taken
method that yields
certain
The only to have a crj-tical part of its own justification. that can be given to the probability claim to priority approach ls that it is part of a more general distribution artifice
for
the interpretation
o f d a t a . W h e nr e f e r e n c e
-
is made to a variate,variance, which terns properly distribution,
all
d
-
covariance or correlation,
belong to concepts relating
that is neant is
is determined on objects
to a
a variable whose value
and certain
functj-ons of value
which, when a variable
l-s interpreted
sanple as a population
sample, can also be lnterpreted
as a variate
and a
as sanple neasures which, in the case of normality, respond in the usual way to those things Hotelling
(1935) has given an analysis of the relation
between two sets of variates theory.
cor-
in the population.
in his
c a n o n i - c a 1c o r r e l a t i o n
It
can be fornulated as an analysis of the relative of two sub-spaces of a Euclidean space, spanned b y v e c t o r s o f r n e a s u r e m e n t s ,a s h e h a s p o i n t e d o u t . T h i s position
analysis
provides a cornplete set of orthogonal invariants
which characterize the figure
formed by the spaces, in terms of a set of angles determined between them. Such a characteri-zation of a pair of spaces by angles was demonstrated by Jordan (1875), using synthetic methods; and a further
such account has been given by Somerville (j929). Algebrai-ca1 method for determination of the angles has been investigated by Schoute (1905), and more recently by Flanders (1948). Hotelling's
theory shows another
approach, equivalent to consideration of directions whose variation in the spaces leaves the angle between thern stationary,
but this
a p p r o a , c hd o e s n o t a c c o u n t f o r
the sub-space in each space which are orthogonal to the other space. A further rnethod, shownby Afriat (1956 and ,l957), j.s in accordance with the nethod of the present investigation, and proceeds entirely values and latent
by a consideration of the characteristic
vectors of the pair
orthogonal projectors
of products of the
on the spaces.
l v h i1. e a c o m p l e t e s e t o f c a n o n i c a l a n g l e s , o r t h e i r
- 9 -
cosines which are the canonical correlations, spaces, or the sets of variates,
there are coefficients
that express sumnaryaspects of this
relation.
which have such a nature,
and alienation
identical
and which can be seen to be algebraically the here-defined coefficients one space with projectors
another. Presented in terms of the orthogonal
algebraical properties properties
are immediately clear.
for
any numberof variates
coefficient
a set of several, this the correlation
itself
coeffi-cient
(1957). It it
and
being a genera)izationof
of Pearson defined between a have been derived
coefficient
h a s h a d s u b s e q u e n tc o n s i d e r a t i o n ,
though with a different has applied
of the multiple
defined between one variate
T" ^h" c n r ronerties of this
by Afriat
S o m en e w
represents a
shown. One of these coefficients
correlation
n:ir^
geornetrical rneaning and
kind are here introduced, and their
of this
generalizatlon
with
and separation of
of inclusion
on the spaces, their
coefficients
l{i1ks
(1935) have defined coefficients
(1932) and Hotelling of correlation
between the
of the relation
complete specification
glves a
derivation,
by Hooper (1959), who
to the sirnultaneous equation method in
econonetrics. A multivariate generaTization of the concept oI variance which has been studied by Wilks (1932) and Anderson (1958) enables an alternative the method of least
characterization
to be gi-ven for
squares. A formula is
shown vihich
expresses the variance of a set of variates the variances for
conplementary subsets, together with
here-defined coefficient Geometrically, this Formula for
in terms of the
of separation between them.
is a generalization
of the familiar
the area of a parallelogram, in terms of the
lengths of a pair
of edges and the sine of the angle between
them. A multidirnensional generalization
is shownalso for
the formula which gives the area as the product of base with height.
RECRESSIA ON N DP R O J E C T I O N Studies
tlie Algebra
in
and Ceometry
Correlati.on
of Statistical
by Sydney AFRIAT Part I Prediction Let U be a unlverse of objects, and 1et Y be a set of i n t h e r e a l n u n b e r f i e l d K r r ' h o s ev a l u e s c a n
variables
Then ll] K can denote
be determincd on anv ohject in U.
V determined on anv object
the value of an1'variable X a
LI.
,\ sanple of size N consists in a set of N objects taken .l , . . . , N. .,\nexperi-ment deterrnines the values of variables in V on every object in f r o r n I J , r n , hci h c a n b e d e n o t e d
the sample. is
In particular if
characteristic
there is one variable u that identified
the sample is
with the
u n i f o r m c o n di t i o n o f t h e e x p e r i m e n t , t h a t t a k e s t h e v a l u e I on everv ohject, is
The samolesnacc for the exDerintent the Iruclidean space E = K" , of dimension equal to the
samolesize. for
Tlie sample vector r"hose elenents tlre
of
\
n v uhJ i c we !t s
11] (i
).
in
[]19
a variahle
=1,
.. ., ^) d ^l l *t P- rl U .^
t h e u n i f o r n r c o . n d it i o n
A multivarialrlc its
basic
is
components.
are
defined
the
n-t'ector
l\lael
N) are the rralues of x on rr r"r . - ^ - +L ii L cu rra ra l a r tL hr r cv rsa nr . nr Pna l\ e V e c t o I I ' d t
variable
the n-vector whose elenents
x is
u is all
ll
u
equal
by an1' set
Thr.rsa p-l'ariable
= I lvhere I to
of
is
1.
variables,
can be denoted
ca11ed
-1 1 *p), so a l-variable is then just one
(x1, ..., in V.
variable
sanple natrix
A 1r-variable has associated with it I . 1 -= ( I 1 - , . . . , I l - ) o f o r d e r N * 1 , ^r
1
p conponents.
w h o s ep c o l u m n sa r e t h e l s a m p l " u " 8 t o r u f o r i t s comnonents -
n
Thesp
a
\rcctnrq
qnzn a H " .
s. r : h " - q- n a' c e"
F X
.i Il
the
be called the sample span of x.
sample space E whlch will Thus
n * , =h r r ) = l r r * r ,. . . , r ' * . , , w h er e = {Nl"c: ctKP J,
trl*
again,
NIx., ..., "1
NI*"t
= {ltl*,c, + "1
C i v e n a r n u lt i v a r i a b l e
x say with
p t q-matrix
c c Knl ,
+ Nl"-co:c1r .'f r
,..
p conponents,
.p.K}
and a
ivhich can be tlenotecl
a nul tivariable
Y
y = x c , i s d e f i n e d r v i t h q c o m p o n e n t sd e n o t e d
'Y J; eK are
where c;;
I.J
of
' lc p' Lt Co' lr l nr r i l nl r oq u, u
X C ; =
) X i C ,
J
r
I
r J
and ci
the elements,
eKP are the
colunns,
J
Its
c.
=
value
l
n r nr u
da r nr j w l
is
determined rvhen tlie value
nhi ec f
: n d r rd \ r
( ,
ir n r r
in the s,rmnle. its
the ohiects
from the sannle rnatrix of
rI ' ^d -r + L; r^\ "r r1r (^r
x
of
is
l ;cd
r
to
t
s:mnle metrix
is
determined
x hv the relation
trf*. = M*., that is, trl*.. = Il*ci' ) "
and again
l" \xf l c l i * . . . * * p . p j = '\ *. l -eI -) 'I ,"t , * a ,
'
5lnce
fl)en
g-
.I .l X
autornaticalll Lr 9t*; contained ,y
-
IL
Special
Ldll
in L^ L,g
thc
,
r- .i
that
t follows
exanlples are
e a
of
lr .r i r nr !e cer T
tl-rebasic
that
x.
1,...,Q),
y = xc then
it
tlre sanrplcslan of
is,
sample slan
^^l l d l r L lup r ]
=
+...+ Mx cni(j p,,
y
is
Any such variablc
a npnf Ln Um l , nl n ,urtgrtL
n u fI
v
t
n u rr
x-ccnrporrentsxj.
2 lhr o
i\ ' - . r^vml ln' ^l /nvar nl Lf r l L .
BLlt ncw 1et mntri ces U
^
x,
be any rnultivariables
I
l" r\r 1 -i n
t
the exnerinent.
t
rolrfinn
y.
=
v
hvnothcsis
of
a
avnarimenr
i c
r ' l r 1 t I i - . = l' rxl -C - .""--
to
with
sample
The condition lro rdmittorl
for
hv
the
t[e
But thC cOndition that this y that E--:E... Let e-, e,, denote the y r(' y x orthogonal projectors on Ex,Ey. Then, since exX = X = if and only if X Ex, this condition is or "*"y "y, couivalentlv e ll = Il y x y
holds for somec is
T H I O R I I i I] :
I I x , y b e r n u tl i v a r i a b l e s
N I - , 1 . 1 .j.n
an expcrincnt,
nrniectors
on
conclition
that
cxperimcnt
If
is
their
and if
= ev,
"*"y
e., are thc
I -
orthogonal
E-, E.,, then the
y = xc be adnissible
a relation that
ev,
qnrnq
c eu r m n lr p r r l '
r
w i t h s a n p le m a t r i c c s
or
i.n the c * 1 r 1 .=.
equiValently
s u c l r a r c l a t j o n l r o l d s , y c a n l . e s a - i d t o l - ' el r c d j c t a h l c
fronrx, and any nultivariblc
x c r t ' h o s ci d e n t j t y w i t h y i s e x p e r i n ) e n t al yl a d n i s s i b l e d c f i n e s a n x - l r c d i c t o r o f y . A multivarialrle mcnt, i f
no one of
x Iii ll its
be called
regular,
conponcnts is
in
the e.rperi-
prcJictrble
fronr
the otlrers- otlrerr.'ico flrn .^mn^nantS aIe nrrltiCOllinear. An identical
condition
of
regularity
is
that
no linear
rclation
b e t w e c l ) t l r e c o n r p o r r c n t sl , . r ee x p e r i m c n t a l l y
wlrich is
to
sa y no linear
conponent sanple sanple
matrjx is
p.
In
L . r ea t
least
the
T l l l 0 l l l ; l r 12 :
vectors,
llr,
span,
relation
equal
exists
equivalentlyr, to
the
nunber
p o{
The condition
tlre rank
dirnension of
thj.s case certainly
the
the
of
the
sanple
s a n r p J , es i z e
conutonents in
that
admissible,
betrr'een the
N must
x.
a multivariable
x witlr
s a n p l c m a t r i x l t l . ,l r-o r e o t t l a r i s t l r r t t h n m n t r i v \ r t \ l - . b c " ' X ' r x regular, in which case the orthogonal projector on the s a n r p le s p a n I l - . i s
e 'x
T'hus, for
all
t
e KP,
= \1 "x
t1"tx1" 'x1/ 1 - 1"1x "
_ 13 _ I r 1t = 0 = ) l \ 'l l ; = 0 x x x and .I ,\rx1l. \. x' -t = o J t r .l , lx' , \,txi =. o vr \ \ :, , xr .)/ r r, .l .i _t \) . =, 0 + 1, ,r . 1xt -= 0 . T h a t i s .' l t l t = 0x ( + I I l N I t x= O . w h i cr h s h o w s xt l r a t I l . . a n J I r 'l l t l * h a v e t h e x
1s regtrlar,
r-2 q e
i t
i c
sane rank, i-f and only
f o r r n uI a f o r
thc
llillx
r:.atriX cx,
tlrc orthogonaJ
is
of
i , l c n r n n t n n t
t l r r l
'x is
range,
whi ch
pro j ector
on Lx ,
It
is
consitlcrc.l is
rieJI -
s 1 ' r n n ; e t r i ca n d
=
identical
say n.
It
- x ' o' x2 a
=
rr'ith the remains
to
'x' o
ortlrogonal shoiv that
projector
olr
Il.= L*.
Tiius, if Xe R, then X = Y for since Ye " *" ) = = e*Y = X, and hence X = il* (llillx) "*\ ";Y lr'hich shornsX e Iir. Thus Rq lrx. Conversely,
l, so th:rt 1' I l x \ ,
X e L*, = extrlxt = Irl*t = X,
X = Irl*t for s ince
in
case
l S t
A l
if
some t. KP, so that "*X = c*NI* N I * , r v i ' r ci h s h o w s X e R , l h u s I ; * ! . I i ,
tr* = R,
which
in
rank p,
p.ivcn I'y thc
j-nverse exists.
definett sir)ce the
its
if
p,
rank
i.s of
re c r r l ql .
Tn thi s casc
IIence, it
so llx
then
and hence
as required.
Another
artunrent
idcmpotents,
trace
is
as fol lorts.
e* = rank e*.
lly tcncral
propclty
of
But
t r a c e e x = t r a c e N l x( I l i l l x )
- llii
= t r a c e } " i ft ' i * ( 1 1 i l l x ) 1 = trace I
p
Ilence rank e* = p. But c1earl-y R - E*, anclsince now also d j - mR = d i m l r , i t f o l l o w s t l . r a t R = l * .
y is predictable
T H E O R E 3M: I f
from x and x is regular then
of y is uniquely xc*r,where
the x-predictor
1
(MiMx)'MiMy.
.*y
' ' .y- ', w" "h 'e' r*o a- x i c B y T h e o r e r n 1 ,' e ' x. ."My. . = M
tha
nrrh4g6n3l
p r o j e c t o r o n t h e s a r n p l es p a n o f x , a n d b y T h e o r e m2 , e
x
'Ilr . x
= I'1fM'M I x' x x'
^h u- L- v^ r- sA r; r- r '^ r 1/ . , ' ,- [(\4:\r-) x L ' x x ' that
is
anv n !r
"
-xy
r h a n r { - ' . M . . c= I l . ' . M . . ,h e n c e , "y',y, """
rocrrlapair
the
by
if
this
regularity,
holds again
Regressi-on Now consider
x,
= M.,, y '
tJt.,] x y ,
\ 1 _ - c= M . . w h e r e c = c - , , . C o n v e r s e l y , x Y X Y
fny
2.
- t
of
t-t
I
x
any two nultivariables 1' -
=
residual
idenpotents
-s "o ' xc '
projectors,
orthogonal
and its
'e x t
orthogonal span of
e e X
complementarity
= O' . e
X
* 6
X
X
and assume x
sample span E* of
rnich
a pair
They are
y
are a cOmolementafy
onto the
complement i* x.
wj.th the
"
x,
of
can he called synmetric
properties
= 1.
Consider the resolution M
" Y= of the sample natrix
a
t r , t "x 'Y
of y,
the orthogonal projection samplespan of x and its
+
l
V
"x Y'
into
componentsobtained by
of sample vectors onto the orthogonal complement.
is
- 1 5 -
If
vI
"
i sY n r e d i c t a h l e e
In
fhis
case.
as
X
then
from x
My = My ' ex \ {y = 0 . been remarked there
has
c e KP
exists
Y
Such that
M.. = M-
an X-predictor
y.
of
r c '
Y
Tt
en.l then ve is
u r r u
i s a nred ictot
ll il n
tL hl l eE
^i ',i.. Sense O urf B lvrllts )EllJE
M r ' ta x LcJ' ) r ' r? y -= ,M ' ,d x c t f- = M
as the value of y in any ohject a, determinedfrom the value U3 of x on that object. The predictions are exactly x confirmed in the experiment for every object in the sample. to assume
That heing the case, there is no prohibi tion it
as universal RU r r Lf D
nr
for obiects in U.
nanarrllv SsrlqLat ) /
n ur ir vv rq l ^evn. f. lLvr l vF Y
it
Will =
-e x "ey
nOt
fg
thc
I t
i s
nossi hl e.
Q \ y-. - .
r-ese
-e x , \, {y
t. h, ,a- t. for
=
vt t y ,
i nstance,
that e*Nl,= 0, or equivalentlY e*e, = O. Since then also e.,e- = 0, this f
d e f i n e s a s y m m e t r i c a l -r e l a t i o n
between x,
^
y by which
they
are
said
considered resolution
to
For the
be uncorrelated.
in this
case
e . . N ' 1=. . 0 , X Y
e . . V , , = l t , ,. X Y Y
These are two extreme cases that have been considered. ts a set of vectors in "*ty E - x- . = .[ M - . ], t h e r e a l w a y s e x i s t s a m a t r i x c . Kql s u c h t h a t x., But generally, since always
e*\1, = M*c . If
x i.s regular then c = .*y uniquely. 0therwise, there is
an infinity
of such matrices, of the forn c + t,
^-J + ;r )a n n r t i c r r l : rr o v rn r ! er dllu L variety
dlrl
where c is
^lution of V.-t = O, the
Jw
of thern corresponding to the variety
X
of
adnissible
_ 10 _ linear relations between the conponentsof x. Noweven when y is not exactly predictable
fron x,
a
= xc can be generally introduced with the
variable f(x)
role of an x-predictor of y.
It
defines the regression
of y upon x, with c as regressionmatrix, and it
coincides
with the x-predictor of y in case y is predictable from x. rf
ic
rrninilo
*'ith
=
c
-c x- .y, -' ,
in
case
x
is
It
regular.
is
are uncorrelated. For any object a,
nu11 in case x,y
= N{3c NIl ^ r(x) c a n d e f i n e a n e x p e c t e dv a l u e l , f l o f y , g i v e n t h e v a l u e Ml of x. ,n" X
,rr""
""tr**
oy Y, = y - i t * l d e f i n e s t h e
r e g r c s s i o n r e s i d u a l , o r u n e x p e c t e dp a r t o f y r e l a t i v e t o x.
I n h a r m o n yw l t h t h i s
vatue i(*) with x.
it
is
seen that the expected
of z ts identically
zero, or z is uncorrelated
For = e NI^, . = e M = e - x "My - y " x "ex l r' ly = O . ztxJ x z (x)
Thus the consi-dered resolution N{
--
v
can be put in the form
My - y^ , txJ
tr^ lvl
ytxJ
characteri,zed by the conditions
ti n?
anrli
rre l onf
r*; l rz
i f *l Thrrs v
is
= u * M y '^ ' r - i (*) = 6*My = xc where e*N{, = lvt*c.
resolved
into
a sum of
variables
Y = i i * l+ z an expectea part i1*; unexpected residual
= XC, predictable z = y - y(x),
from x,
and an
which is uncorrelated
with x and has expected value zero relative
to x.
_ t 7 -
The rnatrix c has been characterized
in two equivalent
exMy = M*s, and also by the relation
ways, by the relation
= 0 which says that y - xc be uncorrelated "*My-*. These relations are equivalent, since
x'
= e*M*c = M*a = M*c,
"*M*a
= \{*. follows e*Mr_*. = €* (vy - tl*.) "*My = [1*.' and f rom e-M.,-.- =O follows e-M.. = X Y x c ^ Y "*M*.
= M*.-M*.=0,
so fron
4: For any variable THEOREM exists
with
y in an experiment there
x,
= M*., or equivalently "*My = cxy uniquely. x is regular then.
a matrix c such that
= o, and if "*My_*. Now two further that
c. One is
ways wi-11 be shown for from the method of
farniliar
characterizing least
squares
I i n e a r r e g r e s s i o n , a n d t h e o t h e r i s k n o w nb u t l e s s Instead of the usual methods of the calculus
fanlliar.
algebraic nethods will
be used.
x is regular,
T H E O R E 5M: I f
=
e -xy
then c = c
is defined
so ,that
ft 'M ' x ') - xr M
xy
- 1'
MrM 'x 'y
eives the absolute minimun for
the trace and determinant of Nl|_*.My_*.. Since e it
follows
+ e
X
1 , e x M x = M * , 6 * M * = 0 a n d e * l r ' l r =M * . * y ,
X
+L ^ + L I I d
L
l, I v
y
l
-
f
ii
l
l
x
^
-
-
o .
l'M x \,-,Y
V ' ' x (t "cx y
and then,
Afrtat!
s i n c e a l s o e x- - i s
Studles
M c) X-, -
cl
synnetric
+ tr - X f\'t sY *
-
M c)
-6x "My and idemPotent,
that
- t 8 -
( M . ,-
-
M-c)'(M. /
(c \"xY
=
M " X "c )/
' Xr"M x (\ "ex Y !r -/ ) r' M
-
c)+ M'e M "Y"x"Y
= (' Mx- . cx- . .y. -1 xt { - - c )' ' ( xM . x_ c -y. .M-x- _ c ) + (", which, witfr i(x)
= *.*y,
M*.*y)'(My
M*.*y)
can be stated
+ M' ^ M^ My- '--x- -c_ My--, x. .c^ = I l l 1rl ' y (x) -xc y (x) -xc y-y (x) y-y (x)
It
follows
immediately
theorems
the
matrices
(Mirsky
trace
this
identity,
and determinant
of
by the well-known
non-negati,ve
defj-nite
1955), (a'a
trace det
from
+ b'b)
(ara + b'b)
= tlace
a'a
+ b'b
= det ara + det b'b
that t r a c -e-
-
.l t' 'y[ - x' _c_ "I {y - x c
trace
^, M' yt - y
.M (x) "y-y (x) ,
and
t ti-*.ty-xc where the equalities
det Mr-i(*)My-i(*)'
hold if
and only if
= 0 M . : . ,. M^, , -' y ( x ) - x c ' Y . t x . )- x c but this is if
and only it
nti(")_*. = O.But Mi(x)_xc=
= M*.-..-*. = M*(c*r-c), and since x is regular, M*(c*y- .) xy if and only if c_,, - c = 0. Hence, the equalities hold .if
rr
--J ; t - ^ ' -= a n o ^o- n1 .r,y 1 r a cxl,
=O
a -s r- ^e^ q. .u: -i r e d .
The least squares principle
is shown in the part of the
t h e o r e m c o n c e r n i n g t h e m i n i m u mt r a c e . A p r o o f o f t h e p a r t of the theorem concerning the minimumdeterminant has been given by F. Sand (in a private
communication), by
nethods of the calculus, using the formul"
l"l-1a]"1=tracela-1aa1.
- 1 9 -
3. Experinantal
configuration
Certain relations
between variables x, y in an of the
experinent can be presented as an alalysis
nrniarrnr
+ L ^ + L l r d L
= u*t*y
'y-y which shows the relation
of the residual variance \'r_i in
the regression of y upon x,
to the variance V*. Also, as
follows from the sameidentity, v^ = v l r y x I -
ti
^
x x y
since Vi = e*r1r, - e e t\/2 x y t
- 3 0 -
remark, it
As a flnal identity
to be noted that the determinental
is
which gives the generalized parallogran
1aw, which
can be st&d T,II M N'I'N' x x x y '\ ,'1ryv
x
= l r , l r * l l u ; r . r r l-l t "*"rl
l \ 1r l \ ' l
)'y
where l ' Ix),- 1 l r^{ ,x. , e- x = l' .' {x (t N' 'x,N f n o o f h o r
w i t h
t h n
-1N,1 e ' y. = \ {' y (. r ', 'yl- ;' ,y4' ' ; y- ,.
n r n n n q i t i n n
O < l l - € * € , 1 < 1 , wherc
there
ii -S'
!e\ O , t U. d. I^r 1 L /:
+,,
^-r
d l l u
I
I
:c r l
dar n r u.
l
vn i n l r lj rr
l t ' l - ' - r 1=- -O , s h o w s t h a t x y ,\ lI ,' Vx ' "t rx, |t || |' vV' ' u l I M ' T I I {' M v l x x x y
'.if
p
p
=
O,
or equivalently
'\ 4' ry\ '{ x
'
\4r[4
y y
which is an equality of Fischer ('l908), from which follows t h e w e l l - k n o w n d e t e r m i n e n t a l i n e q t r a l i t y o f t i a d a m a r d( 1 8 9 3 ) . Another proof is thus obtained of these inequalities, the added explicit
qualification
with
shownby the factor
I' f - e x- ev. .,1' , a n d t h e p r o v i s i o n t h a t t h e e q u a l i t y i s a t t a i n e d = 0 . B e l l r n a n( 1 9 6 0 , p . 1 3 7 ) r e m a r k s t h a t o n l 'y .i f l Mt J M v .. " l l a d a m a r d ' si n e q u a l i t y i s o n e o f - t h e m o s t - p r o v e d r e s u l t s i n analysis, with well over one hundred proofs in the literature."
-
-
JI
PAP.T I I
'I
Ir'inein:l nAl ln, vw
^
I t O t
n l l r L
) 1 , d 1
nf -L
neirs
^*^
l l
d r
I
U l
d l g
nf
snrccs
^L1:^rrc
s
u u l
cornponents lr'hich are
- . ^ i - , . Lw :l ^l l 1l l .l
O
' L;
q n a c pq cJ
tL uw -v n
of
lair
redrretinn
- . (+l l1u .F^u ^r ^l ( -l ^! r
n l la l dw j /
r Y u q
inclined +L U^
hl , es
rI eq ud ur Lr e n c d u
f g
each other,
to
+( i Ll '. e S e a n d
tO
aiSO
3
and
eacll
which are mutualy inclined, are
other. The prjncipal lair,
o b t a i n c d a s t h e o r t h o g o n aI p r o j e c t i o n s o f t l l c s p a c e s i n l ai r e a c h o t h e r : a n d t h e o t h e r - t h e r o si d r r . an
are the
I
con,plernents in
orthogonal in
fhc
nrinein:l
n T n n . r t \ /t ) l , r u P l r
n u fl
projections
nair-
hu eq ir nl lob
each
lhe
ri nr r
in
the
prolositi-on
The fundamental
thc
sfaces
nrincinel
rr A a ri nl ,r rnve! eu lr sJ r L L
each otiler
of
of
nair
+ l ' ^ + L l l d (
of
either
also
+ l - ^ ' ' L l l q j /
have
- * a d l q
+ L F L l l s
space the
n v rr l hL nr cr un bn ur lr r q r
spaces.
fron
which
thj-s reductj-on
i s o b t a i n e d i s c x p r c s s e t Ji - n t h e s c h e n e : E @ k x y x y
t
x
t / ty
where the that
is
aplears in
indicated
relations
e..[,. is ^ y
i n c l inecl to
are 6 . ,, I
as orthogonaI
to both
C
x
and ty are oblique
and the R*r,
whic)r
extr and \ / , c a n b e d e f i n e d
t h T c n c n u i r ' : l p n t w a y s:
fq* o"*
xy
t h r t
the
.i q" ,
2 s
nrniectinn n rn i e e ti n n nrthnonnel
the in of
=
I
,
( e;.Ory I \E,. o"y E*
o r t h o g o n a l c o n r p l e n r e n ti n
t-_ o t r_. j n
n rn i a a t
,),, or of sinply
t-. of
E,,, or r
of
t 1 , o
^ r f l r n c n n r l
+ l - ^ Lltg
^ , + l ^ ^ ^ - ^ 1 vr Lrrvl:;urror
, y , o r o f s i n r p l y {.,, or o f the
ion of
qx in
6 y . A pnr ' I / v'
i n o " o
t lr
is
schene to
-
-
Ja
and then again with the spaces
one space with the other,
i n t e r c! rhr u:r nr a o- \ e rs ,1 . t h e r e i s o b t a i n e d t h e f u r t h e r
-
!x | o l
'l'he is
by the
l
t
)
with
for
1 -
c,c..,
)
=
con'ponents
A
ly
"* the
spaces interchanged. tlre rcLluction is
anC fi*..ts ^)
)
ortitogonaJ. projectiors
the
e*er,
the pair.
principal
the
obtair.ing
e.. [,, as tlre rangc of ,\
of
rclation
same relation
An algori thn
reduction of
ex{e)- 1r) and the
t n
.
t.="y e^-!y*
property
reciprocal
stated
'/'---ix [1' Txy | ..-l ? O ; and let
\/Fctor of e e ^
, s o/ t h a t
U be any corresponding unit
is
latent
)
e - - e - - U = U U ' ,U r l J = ' l; x y and define V by e U = V',.
v
-
Then e*V = Uu
, and V'V = 1;
when U,V are a normal reciprocal
pair
of vectors in the
s -p a c e s^ € . - - ,6 y_ . . ^ ^ ^ ^ r , U. l ri r hr B^ r ' / l, '
n L L U I
tL nU
of the biprojectors reciprocal spaces with
pair
2nv
d r l /
n i l n u ln l - ' 4Fq rl u^
ChafaCtefiStiC \
ValUe
UZ
on the spaces, there corresponds a normal
of vectors making a proper angle between the
cosine u.
It
has already appeared that
the square
of the cosine of every proper angle between the spaces is characteristic value of the hiprojectors.
Thus it
that the characteri.stic values of the bi-projectors
is seen on the
s n a c e s a r e D r e c i s c l v t h c s n r r n r c so f t h e c o s i n e s o f t h e proper angles between them.
a
- . 1 0 -
5. Rank and mult j-plicity r T " 'a" r ' - x, 0 r o \ < o , < " / 2 ) d e n o t e t h e n u l l - s p a c e o f t h e m a t r i x ) ,1 - e X. - eV .' - , w h e r e l = c o s 2 o . T h e n e v i d e n t l y t * r o
t
t y
;
" *
and €.- I 0- just x 0'
when s is a proper angle between the
spaces, which is just rl r* a: ul rs r o
'
i n
u , hi . h
whenl is a biprojector
.aSe
i -_
fnr
the
latent
Vectors
r d
and the corresponding space E.,
have
Y t a
the samedimension ra of
reflexive
associated wi,th a proper angle o.
T h e s ^p a c e / X
r which will
be taken to define the
o as a proper angle between the given spaces.
Si-nce latent different d
Of
tha lharacterist i c value tr , which span
directions
Y
composed
X , 0
^r a 6
rank
is
characteristic
vectors
of a matrix
corresponding to
characterist ic values are independent,the spaces
a r e i_ n" -d- er -p e p r l e n t : n d t h e r e f o r e t h e d i m e n s i o n O f t h e i r
union is the sum of their
dirnensions. Thus
O r- x , c c ; and correspondingly t
e ' x i- I
r' o .
u,)-
pair of hasesfor the spaces.In this case (Ui, Vi)(i will
be normal reciprocal "*
...,r)
pairs of vectors on the spaces, with
Ia y -= fLr, r, . . . r r, fr Jl t 1r
o
= l,
^
F
"y.-*
-= [Ur "' l , . . . ,
v J
"fj
and
A
xy
The problem for spaces is,
constituting a total
reduction
vu ql J
of a pair
a pure matrix formulation,
of
that of
the bases, with which the spaces happen to be an equivalent canonical pair;
sinultaneously pair.
r . '
= f u L t r r * 1 , . . .' ,. -t 'rp rJ ', o^ t y x L V r * l ' " "
to give it
transforming given, into
= =
rotating
alternatively,
orthonormal bases into
to
a canonical
The analysis which has been given shows the possibility
of this,
together with an algebraic algorithm for
Alternative
the realization
computational procedures are as fo1lows.
f i I
Q i n n o
\-,
' x M" x '
l\4t
' x - y l"\ ,x1
Nlrc
e r e A n a ' i r o f s v m m e t r i cm a t r i c e s . o f w h i c h t h e f i r s t poritive
is they
a n d t h e s e c o n dn o n - n e g a t i v e d e f i n i t e ,
definite,
can be simultaneously transformed into
matrl-x, and a
the unit
n o n - n e g a t i v e d i a g o n a l m a t r i x ; t h u s , r , ' i t h s o m er e g u l a r s q u a r e natrix o, ^t\l t
\ t'
=
xt"
1'
, s. , t' \ 1xt ' ey. , 'V x. o-
I Z , u
=
n\
t
l,
o /
\ 0
w h e r ep i s a r e a l d i a g o n a l m a t r i x w i t h n o n - z e r o e l e m e n t s , of order the rank r of exey. Take LI = Mx o = (tJoul), V* = erLJ = (\': VT) , f h a
n r r f
i f i n n
€L
^ f , UI
1 1 . ' IruLudrr/
these
^ - - ; L ; 1 ^ + : " ' dlrlllrrlr(rLf
are the projectors
'rb,
on tlle
s u p p l e r n e n t a r ys e t o f s u l r s p a c e s f o r n e d b y t l r e i r r a n g , e s . l i i t h a n y i r l e m p o t e n t ,i t s c o m p l e m e n t a r y ,a n d i t its
ranse narallcl
projector if
it
is
range and null-space are
iCentifieC with
tn ifs
n r r l l - ' sr n a c e . I t
thc projector
on
iS an oltltogonal
is synmetric, in which case its
range and
null-space are ortlrogonal conplcnents; and otllerrvise it
js
an oblique proj ector. Ilore generally, rclative are just
t o a n y s e t o f s u b s p a c c sw h i c l r
independent,and not necessarily supplementray'the
s u m o f w h o s e d i - m e n s i o n si s
the dimension of their
sunr,not
I t e c e s s a r i l y t h e w h o l e s f a c e , t h e r e r n a yb e t a k e n t l r e o r t h o g o t r a l projection
of any vector onto their
union, and then, with
vector obtaincd, tlte o['Iique projcctions
Afrtet:
StualleB
the
on the supp-lenicntary
- 5 0 -
set which the spaces forrn relative r* ur t2 v 2 4 !n L lv
\rtr.tnr
is
one of which is
intO
feSOlVed
In this
to thei-r union. nair
a
into
resolved further
oI
coTrnnnnntq
orthoron:]
a set of oblique corn-
ponents in the given spaces, while the other belongs to the orthogonal conplenent of their If thon
union.
the supplenentary spaces are nutually nrn'iectinn
nn
eech
snAce
relative
to
orthogonal, set
the
is
the
s a m ea s o r t h o g o n a l p r o j e c t i o n o n t h a t s p a c e , t l r a t i s t h e nrnicctinn
nr
Spl i t Thn min:tion
n f' nn
now ariSes
nv r' "nJi e C t O r s
tlroir
rrnion
terms of I f
to
in
two spaces
Ex =
-
its
nrthnonn:l
exnlicit
to
^
rvjth
-\
rosnoct f
otheli
to
comnlonrpnt!
v
| ] l l , r
t"h"t r v/
relative
x
nar:l
t].a
-*a.i6-r^t'
lel
t^ ^f
,
fof
whjch
by
a n d , e Q U i v a l e n t l y ,t
/
r tx
relaf narcl P'
irre lol
fn rn
in
spaces.
'a l l ' l " t " 1l tr I II '"
there is defined the projector, which maybe denotedbI n v n, r te
to
A fnr1pgla can be
Are senerate.
given
qn.ces.
of
al ternatively,
on the
deter-
enmnl pnrFnt2TV
2Tp
the other
the
mejr
the
e
o,
a
foI
r ' l Ey = harJ on which
are c are
to
ir:st qcnerAfo
projectors
e - e . ,1 *
formrrlee
the bases, or
r l ll.lxJ ,
criteria
It
thc
terns of
nrnicetors the
condition
to
on one parallel
the orthogonal
nrtlrnonnrl
aS
r-nsc thnw ern
in
givcn directly
lel
defined
olro n:ral1c1
more generally,
or
n:rel
pro jecl-ors
orthof onal
nupstinn
ci ther
qnAcF
thrt
their tllg
rrninn
.. rx
COmplenentary
that
,y,
SpaCe
n v'
"*lu, is giVen
b y t h e u n i o n E - - O i ^ . . . o f 8' Y .. with the orthogonal complernent Y-'x,Y r' x r y
r5 x
of "
tlreir
lniOn: '
JS vO
e L x l, y
and nrrlI-qnrce Ee;..... a e r ' \ '
t l
-i S
tho
,'nin,'a
i,la--nrcnt
uifh
renoe
eithe
-51 on the spacest
Then, in terms of the orthogonal projectors there is
the formula -
"*[y for
( ' l
-
\'
o
o
"x\,
-
c
" x e- y l,
more immediately in
or'
the determination of u*lyi
( 1
l - 1o
"x.y,
terns of bases, and in a form which directly
generalizes the
formul a 1
= V* (l\1i]4x) 'Ml
"* for
on a space in terns of a base,
the orthogonal projector
there is
the formula e-
From here it
= M' x tr N' x, 'tY' e ' - 1 u' x' 'EY ' ' ' x- 1 M
x IY
is noted, incidentally,
'Z y o' x l y
=
that
a-
".y t*
The sum of the complenentary oblique projectors union is
nei r of snaces relati-ve to their
on a
the orthogonal
nrniectnr on their union: =
* Now any vector
z =
Z in
"*,y
"yl*
"*ly
t has a unique resolution
* 6*,rt * "*r>'z "yl*Z
' i n t c r c o m n o n e n t s -i n- - t- x ' C - -r y -X - . - y. - a n d C If
C-., t.. are, ^ y
complementary,
noreo\rer,
€ -A- r. _. .I = t
e-
and
^ t f
.,
so that
= 1,
sirnply that
for which the condition now is
P + Q = N, rhen c
e " Y l x a r e c o m p l e m e n t a r yp r o j e c t o r s ' of spaces; and in this
to a complenentarypair
'a * +y |
p
' y.
=
lx
1
on and parallel case
- 5 2 -
projectors.
3 . I t ' l u 1 t i p 1 vs p l i t
p, e, r,
e -L, ' . . . b e a n y s p a c e s o f d i m e n s i o n
€.., y'
Now 1et [-., x'
which are independent, and so forrn a supple-
...
to their
nentary set relative
union ,*,r,r,,.
of dimension
Then there is determined the projective
p + q + r + resolution 7
"
- =- x | yc , r ,, . , . 7'
cf any vector Z in t andE-... L r Y r L r
+=
y l cx r z , r . , a. "
!
+
' ;x , r -t z , .2 ' ,
i n t o c o m p o n e n t si n € * , e r ,
,
,
The orthogonal projector on the union has
" '
the decomposition = * "' "rl*rzr,.,+ "*lyrzr,,, "*ryrzr.,. t h e s u r no f m u t u a l l y a n n i h i l a t i n g p r o j e c t o r s , o n e a c h
into
para11e1 to the union of the other with nl omonf
^f
fho i r
the projector
gnl6n
all
ThUS,
tOgether.
on C.. parallel
the orthogonal corn€_t ., . xlY'2
it
""
t o t h e c o m p l e m e n t a r ys p a c e
given by
t ), , @ -c , e . . . C -I XEr I -r .7 r, .-. , '; rn.l
m n r ar nv rvr a r!
L
r
t
= o' "*lyrzr,,, a n' y b a s e o b t a i n e d f o r C - . , A t J t
" '
)
Sqga1.e
rank e*rYrzr,,, = trace If \l ' "\ {x
I \' ly
M--..
x t I : "
ls
M
""'xryr.,.
t h e c o r n p o n e n t so f
ic
nhtainp.l
aS
a
fegUlaf
then matfiX,
the inverse of which, when transposed and
c o n f o r m a b l yp a r t i t i o n e d ,
define matrices N-, N.,... ;
" ' X r Yr . , . '
thus ; f' M x My . . . l -xt r Y r . . ). -' 1 = ( Nx Ny . . . Nx , Y t , , ), '' . The projectors
which have been considered can be computed thus: r Y , 2 , , . . "€ x 1
=
M
" x N" lx '
"'
5 3 4. Partial
regression
Consider three factors Xt Y, z and the regression of z on the factor
c o m p o s e do u t o f x ,
together.
It
obtains the
resolution M " z = e" x r I r ' zl + 6" x r I M Fnr
thc
rForcqsion:l
nert
here
n ' , -=
^
f,,
there
ll
''"ry
the separation
is
t*ry;t
"*ry = L" .x, - l xr l ,
into
further
parts
Y'z
+ l\{ " Yr -
Y lx , z
corresponding to the
nc
rf
i t i nn
\' 1 ' X r Y = r\ \' 4' x M . , ) ; .r an4 conformably, r
x,Yiz
=/'*lt,t\
\,r1*,,/ Now
* Yryl *t*l y x,z ,z is
the regression of z on x, y together, given as a sun
o f c o m p l e m e n t a r yp a r t i a l of z on x partially
regressions, defining the regression
with respect to y'
and on y partially
with respect to x. It
is required to have a formula for
the partial
regression
m a t r i x r ^- l l , , ,L b e J o n g i n gt o t h e r e g r e s s i o n o f z o n x p a r t i a l l y I t with respect to )'. Splitting
the orthogonal projsqtor ex,y
into oblique componcnts,corresponding to the resolution its
range t*,,
t n t o c o m P l e m e n t sC * , t
= *V, y '= , - c " *\ l yl ' t + '
e
:
' y \l x l ' z '
of
- 5 4 Substitrrting fron the formuja rvhich has been given for snlit
follows inmcdiateiy that
r r r o i e c t o r " s .i t
-
= (l!|erl'!*)
t*1y,, ,-lirneilw
- l
' l ' t i -e * l t i ,
tlrc cnrreqnnnrlino
o e n n r qr lr ir -r ia nl o E ;
u r r L L L r /
the
, formtrla
[or
a
total
regress ion. forn'tr1ait
Irrom tliis natrix
Ir L^ Lo .r la! qJ cJ ri n u rn ,
n r l t r i r
n f
-z
repress ion
r e [ r , r e s s i o t tn a t r i x .
can be cxflressed as a total
n e r f ( I i da r l l r d l
to y,
seen tl)at a part ial
is
n - l- l
A
a p p c a r s t l ) e s a m ea s t l r e t o t a l
P d l
: ^ r 1 " I ) L i d l
. ' : * h h l L r r
The rI o s n e c t
relressiotr ntatrix of
tlrc residuals in tlre reEression oI z on y on t])c rcsidlrals in tlre regressj.on of x on y.
for
these reslduals fornr the
matri-ccs * x
-
*
-
. ' ! l' i " = c "111' " - x ' 1 " z, 1 " =- eY |" 7
'
u l r i e l r r - n n s i d o r e , aj s t l t e n e a s u r e n c n t n a t r i c c s o f ,^ = 2 -
z0),
i*
= s - s(yJ, cive r*x."x ^
5 . Inversion
, L
= I_1., -, ^ t )
r L
anclDartition
z = (x,y)
Consirler a factor Its
Iactors
neasrirencntrnatrix has the
corfoscrl of
sulrfactors x,
narf
fnrn
i+inrp.l
=
"" !
.
y.
(I1.1.,,) \
J
a l r t l c o r l ' e s p c n t l i n g 1) ' , riz))z
-
/r.'x"x
I \rr;lrx 'llro
nrrncf
inn
n o n 1 s t o o L ' t . a i . nt h e r' l , 1| \ ! ) 7"7'
-1 z z
where i i = f ' \ x \ yl ' z
li'
xl\ly \ ,
,'j-"I
inverse in a sinrilar forn
- 5 5 -
confornably with Mr, and moreover, is
is partitioned
such
th at
N i M ,= 1 . It
that this
i s n o t i - m m e c l i a t e l yo b v i o u s ,
though it
can be verif ied,
s c h e m ei s o h t a i n e d b y - l . l, y r, "vy' E N . . x- -= -ey ,t r, xt [t ,r ., '1* ].ye -, 1- l/t - ) - 1, N - xr ' 1 . . ) - 1 . "y = 6 x t
It
is noted that,
f o r a n y m a t r i x t l w i t h i n d e p e n d e n tc o l u m n s , of matrices of the sameorder such that
there is a variety
- 1' = N ' N , N ' M = . 1; ( l ' '1M )
for
a special example, -,I ', N = U(Nl'\1)
exanple, as just
and there is a further ,w.a, . +L: trr . d^ r-r .j ,l
n a + +L .r i *L ;r l O n i n P d r
COlUmnS Of
the
indicated,
associated
14.
Now Nl- \': = I'l-.N-'- + I|..N.'., y y' 7 z x x which obviously
n rn jp6tnrs y r w J l u L v l J .
4--ihi
ll daL tI ir nr tcs
snl i t
nroiectors.
d t l l l l l r r
one projector
gives
e '
Tr ihl cL rnr
there
x ,l_ y
=
i s
V \ 1|
,
ri rnr
as a sum of
Vi-ew
made the
Of
tlre
two mutually
fOrmUla
fOr
identification
= M,,N,', ,
c
x ' x ' * Y l *
, )
and hence, though otherwise not obviously, e- = i'lrNi C i v e n a r e c t a n g u l a r m a t r i x \ 1 w i t h l l u { ' MI I 0 , i t according to the definition - 1'
inverse given by \1
is,
whi,Ir is
thc
The natrix
transpose oI
thc generalized
h e s t" h" "e nF r' "orn- e' -r- t- i e s - l
(fi{'M)
N defined by
= N',
NI that
of Penrose (1955), a generalized
- 1 = (M'M) 'NI' - l
has,
= N'N
, N'I'l
= 1.
inverse,
- 5 6 -
to make further
is now interesti4g
It
observation on the
fo rn o f
( l u ; l \ 1 21) = n - l N , , N l l l , = 1 , w h er e N, = (NxNy)
( ) . l x l , l y ),
Il
Thus, R l "' X1 ' l 'l \ Y = " "1 x "' Yt "x = o 'Y
are the rcsiduals in tlre regrcssions of x, 1'on eaclrotlier; and 1 |
1 |
,n . x -- , I r l x r , I ) . y - - . rf , ly
h. v .
Ivnerimnntnl
L / \ l \ r r r , ! r r L u r
TLtL
^-., dtLf
^-.-^-; sAI,sr
a fictitiorrs
r
fr h" 11 v' ^v 6r r d
1-ho
n,pn t
evncri
u r r \ r
rnen t
^ 1 , . ^ . . d r r \ d /
facto
rrrrrlrenop,lv
nnd m , , Lc n n \alucs
ru rr r nr ri vf a nl rrr nt ' i l v ; + I L
i ^ l >
s nossil'leto cntertai
t o b e C e n o tc d h 1 ' I ,
r
it
ho
rrhiclr remains
define s the unifornr factor
k i n g the valLle 1 o n a n y o b j e c t . I t s
rJeasurelncltt natr
thus
i ^ A J
i r -
t 1, = / 1 \ t l l
I ' l
tt '' ll \ r /
the vector with N o w c o n si d e r uniform factor
^ l
t
+1.e
i
+ ^ L >
g^ 1 a^ -q ^t !" t .g+l l^
r e g r e s si o n o f
I , thu
I,I
X
c -tri I
x
*
t
i
lr x
Since
,\ 1. rI l't' I it
L5
follorvs that I
N;TI\1i
equal to 1. anv
factor
x
nn
thg
o
-
D'J
that
and therefore
e r l r l *= I r l r i = n ' i ( r l w h er e Y "
1 N NIl 1 X1
' rx ' T-
=
(I')
of x in tlre cxperin'cnt, it
defines the mean value jts
^
being
c o e ff i c i e n t s o n t h e u n i f o r n f a c t o r ;
vec,o. o*n
e x p e c t e d r r a l u e o n a n y f r - r r t h e r o bj e c t ,
and i-t gives its
unif orrnlty. the residuals 6rlrl* r e g r e s s i o n m e a s u r et h e d e v i a t i o n i = x o f x f r o m i t s
s u bj e c t t o t h e f i c t i t i o u s in this
mean, tlius : -
l t
^
x-x
7/
r
l ' r r l fL r iPnr \l o
'[he of
r^ n! oA rr P! S S
r : u r
rr-'gression
x
iOnS
a factor (x,y,2,.
w of
a factor
of
composedof
dimension p+q+1+
X r yr z t . . .
l l
r
.l
subfactors
d i n r e n s i o n p r g r r r . . . h a s the form
of
( - , l r \ ^ , J r . . ' ' r v \r ^ r ) , '
^
, w
/ ; L
s
. . ; r * Y t y X r z . . .; w
r v h er e
xlY,z,
';l!'
xrY' "
Ylx,z,
Just as the total
l'rasthe determi-natlon
t {
-
"'(*ryr...)r hu rt r
n , uarn, vn rc r q v rv t l( r, n
l
nrniee
t
so the
inn
--
rt r \I X I
r
xl Y, z, . . . ll{
E
XrIr
XrIr.'
"xl y ,z
parts N1
h ave
tr1 the deternrinations
- 5 8 h u rl r
n w hu l r r iYn ur rs n
nrnioe
tinnc
lrv
, l \e tLo\ r m i n e d u
,
tlre
o b I I o,u c' J)1'o J e c t o r s
into which thc orthogonal proj ector is spli l - r a v eb c e n o b t a i n e d i n
t
But
these
the f orm -
" * fy , r , .
n t
' ' ' xl i' l t x t
.
..'
so that I lr , x \1 y,2 ,.. whence
flro snt
nf
=
t\i 'l \ x1 " \ xi ' l '
rcoressinn
nrrtiel
' r '
t
nntrices
are obtainecl
i n t h e fo rnr I
R u .
\ l r ra ltL ri n l nL , ' !n vnr fr ai cf r r r r t i n n s t , r
as varioLrs binalf
appJyinp. to factors
any Iair
of
througlr their
a pair" of
harre been considcreC,
rclations
spaces, anJ tlrereforc sf ans;
spaces attajns
to
or
or
tieparts f1.on then,
dcpartrrr(' J:ron thcr: fi\cn
coefficicnts
arc
of
l'y ccrtain
tlre spaces.
flrtl renain
for
rOIC
thc
jnvari ant
attainn'ent
cocft'icicnts,
l'lrese rclations
consic'lercclfr,rnciions of
forn'ctl b;' tlrc slaccs,
so the
l : o l . li r r p b c t r i e c n r .
ancl tt'],ich har,e criteria
togetller,
c -elf i n c ' 1 1a s f l r n c t i o n s
har,e been
i n r t l . i c l ra n y c o r I i I u r a t i c n
spflccs, b1'definiril: n-ilry rclations,
to
any fair
t l ' n e n r . fi o U f a t l O n s O f t l r l e i ' O f
sArc pi olrt lrc rlonn fol
spaces taken
to
and coeffici-ents
t le f i n e , l u i . i c i , m c r s u r c t l r t ' e \ t e r r t of
'
r , \ .
Just
of
r \: ^ ' X ' t.l' 1 r '
I
X I Y' 2 ,
attcl
configrrrations
un\lcr ortltoqona l
transfornations. The nrost obvior-rsnLrltiple for
a r r y r r u r n l r c ro f
qnpci:l r{^*^ lluIL
thrn
this
1 + l ^ l l L i l L ' , I ( r t( l l J l l
sp6qc's is iS
relation
to
be conslderecl
nutrral ortlrr:lionalit)'.
tlte lelation
+( |1L. ; ^ ^. rlr'llu(rurl ur
of
):ore
ntutual distinctneSs.
.r .j ^ Lllr)
- - 1- r :^il ld( lulr,
. n l ,i cl ,
i s
- 5 9 -
of linear
the occurrence of a coincidence, is a relation d e p e n d e n c eb e t w e e n t h e o r t h o g o n a l p r o j e c t o r s
on the spaces,
which may be terined cornposabilitv. An equivalence of the the condition that there exist
condition of composability is
unions identical
two subsets of the spaces, which have their and within
each of which the spaces are mutually orthogonal. o r t h o s" o n a l
+
t, distinct A coeffi.cient will
extremes
composable
o p p o s1 t e s ^-_-/
coincident
t
be defined which sets the configuration
formed by any three spaces in a scale between the extremes oI orthogonality and conposability. For three spaces, composability is that either a pair of them are itlentical, or a pair
are orthogonal and the third
their
uni-on. A natural
the coeffici.ent to any numberof spaces is
extension for readily
is
suggested, and partly
established.
Let the number xtY rz
determine(l for
,
a*, C*,
tr*
'
'r,
ct*
'r,
1
a n y t h r e e s p a c e s s y m m e t r i c a l l y i n t e - r m so f t h e
coefficients
of association between thej.r pai.rs, be taken
define their
coefficient
of dissociation.
lt
can be sholn
to have the property of being non-negative, and equal to if
and only if
to
the three spaces are composable. It
be seen to be at most one, and equal to one if
will
and only
t h e t h r e e s p a c e s a r e m u t u a iI v o r t h o g o n a l . F o r i t
is
zero now if
seen that
- 6 0 -
o < K " = ""
t = t(" YCz - c' ' x Yc" x 7)t z = ( 1 - " cx yz' t n- c" x 2 z) "' .j 2 = -z -2 -
ryr t
J*y)*,
L*yL*,
'ty.-
. 1, < s1,.s?.^ f
^ L
the bounds being obtained since the S and C coefficients
It
follows
equals 1 if all
between the three spaces
the K-coefficient
that
and only if
the spaces permuted.
wjth
bounded between O and 1; and sirnilarly
are
the S-coefficients betweenthe pairs
equai 1, which is if
the spacesare mutually
and only if
^-+L^^^-^1 v I LrruEvrrdr. ^ - - ^ - A ; h ^ r ' nLLVr srrrBa
jr t
o t n * , r , ,4 1 ' K-_,. _ = 0€:Xomposable X,Y,Z = 1ts;'orthogonal. In terms of orthogonal projectors,
composability is one of
the conditi-ons e _ _= e , , , o r e _ e . , = O a n d e . , + e y. . = c _ , x ^ y x z ' ) or one of the conditions obtained by permuting x, y, Now for
any numberof spaces, corresponding to x, y,
d ef i n e v
-
1
C
X t Y r Z r ' . ' c
C
xy
l
xz:
f f
C zx
zy
L
1
;
can be shown that K
XtYrZr"'
> 0 = 0
It
C
'
'
) ^
Then it
z in these.
€J
conposable
is obvious that
Kx r Y r Z , , .
= 1 Q)
elllegonal
z,
-61 -
and it
that
to conjecture
is natural
s 1
xrYrzr"'
= 1-
orthogona
a s w i 11 b e p r o v e d . Fron the identity Mt
N'tr!
N l l\
and the properti,es 1 - efi
, it
with equality induction
that
= l v ' M lNl ' N i1l "rI
li{rM
w hi c h h a v e b e e n e s t a b l i s h e d f o r t h e q u a n t i t Y
f o l l o w s that MIM
! l ' NI L
NIM
N'N
i r r c f
i f
MtN = 0.
M ' M l lN ' N l
(Fidrer, 1908),
It follows immediatelyby
if
A =l A'.'A.,, l
"
I ^ztAzz | t " " " ' is any synmetrically partitioned positive
def j.nite
matrix,
then
l A l - ([ A 1 1|] o r r l with equality
just
if
the non-diagonal conponentmatrices
all
A.. are nu11. 1J
In particular,
w i t h c o m p o n e n t sa l l
i.nequality is obtained.
of order 1, fladamard's
An advantage of this
the most general inequality
approach is
is obtained directly,
with necessaryand sufficient
the most proved results proofs in the literature.
together
c o n d i t i o n s f o r e q u a li t y .
( 1 9 6 0 , p . 1 3 7) r e m a r k s t h a t " t l a d a m a r d ' s i n e q u a l i t y
that
Bellman
is one of
in analysis, with well over one hundred "
- 6 2 An appli.cation of Iladamard'stheory (1893) on the i n m e d i ,a t e l y e s t a b l i s h e s
n'atrix
deterrninant of a positit'e
t h e c o nj e c t u r e t h a t r . v a sn a d e a b o u t t h e r n a x i m u mo f t h e o f J i s s o c i a t i o t t b e t w e c n s p a c e" c-2 x ' -eY ' coefficient X-... r ) r . . . which is that its value is 1, and that it is attained just wlicn tlre spaccs arc mutual1y orthogonal. lrlorcover, tlre t l ' L e o r e m ,. 1i s c o v e r e d b y F i s c h e r ( 1 9 0 6 ) '
nore general forrn of thjs
antl Pu, ( ,,
€- ^* , € - --). , .
K with
and
be
v "*ry,
.,,;arbr...)rrr
tlren this
cocf f icient
jrrst
if
all
i,,cr
il fI
U I L l t U E U l l d f l L j /
J U > t
the
sets . StiIt
L | 1 9 |
)
]
I ies
a set
if
it
Cefined
holds
for
an1'
Ii lK r a r b r . . .r . . / " ^ r y r . . . " I r b , bethreen O and 1; beitlg
:ero
conrllosablc,antl 1
hI Le t w e o n
nrore general
coefficients
SnACOs
talien
ffOnl
can be definecl,
s n a c c s a r C C o n r ihn e d i n t O
a n d s o f o r t h u l ) r c s t t ' i c t c tl y ,
are
def inition
^r,,-..-l^1.^1.rs
,
sets
preliottsly
d r t \ d / r l r r r u l u J
r .h n n q n l q c rf
i n r n r n i n n l r ' rt .
for
=K "Xryr,.. also
the
that
different
slraccs tal'cn togetlter arc
^-+l,a^^h^1i+,.
different
noted
t h c r - e i - s n r a c l et h e
Norv if
in
spaces
holds
conrfosaL;i1ity condition subset.
if
only
shouid
It
orthogonal.
aticlso forth,
4i\.. .. K., l ., ^ r. X r ) ' r . . . Y r D r . . . t r r I r . . , r a r l l r . r . r , . . if
equalitf
sel'era1 sets of spaces
Thus, for
leads to nlore general results.
f urther
S e t S,
l r e s c r v i n g a n a n a l o g o u ss c l l e n e
a t e v e l y s t a g , e , s h o r v i n ga n i n c x h a u s t i b l e c o n b i n a t o r i a l , qelrnnrtie
ellv
I
r! o - ll'\ r o , l r r e t i v e
nrnnFrtv
of
tl'e
rlissoci.atiOn
coefficient.
,.
l.irnits of
a s s o ci a t i o n
T h e r e i s n o t " t o b e s ] ' r o w nh o l v t l t e r e l a t i o n b e t w e e n a p a i r of sfaces, or a fair
of factors through their
by internreCiate relation
to a thircl .
spans, is limited
- 6 3 -
A l r e a d y t h e r e l . r a sb e e n s h o w n t h e i n e q u a l i t y ,2 LxyLx2J I
l.yr
irith equality i I and only i f oenor'2!
It
,liscrcn2nev
in
the
fo1lolr'sthat C.._ is
the slaces are comlosahle, tlle
cnrrr'litw
limited
y L
- S S C C "xy'xz xy'xz r^rharn anrroI i rv
anclonly if
tc
one
-Z _2 -*y)xr,
or
is
with with
r l if f e r e n r - c o f
is
f actor
associated
a third
of
is
limj.ts
is
'
attained
if
gir.,en.
are
for
the
of association and There is
cosine
of
to
be noted
a sum ancl
t h r o u g h j - t s m e a s r r r c n e n tn a t r i x
z,
a nultj-nornral di.stribution
clesirable
futtctions of
of association between a
characteristics
have been defined
to
1
( t . 1 1f r z )
itlentify
between x,
;
the
of
which
nteasurements I'ith
tIe
have been forned
)'i
ther
wit)r paranreter natrix
parameters,
characteriStics
These coefficj-ents
Ilr,
certerin coef f icients
as fr-rnctions of
the distribution
annerr 2s ,lirect
relaticn
the
coefficients
tl.ie formulae
A = and i t
"'*'Y'z'
rnclcs.
10. llistribution a
L
tlre spaces are conposable.
dissociation
Itith
hrr
4 '(-]x y C 'xz 3 ' - x z -+ 'Sr y ' S
z C = "yz
otlter
l i n r i t e c l w l ' i e nt l i e i r
an anology
oirrpn
tl-rus:
In this way the coefficient o r *a^i 'r
hpino
so that
they can
distribgtion. relative
to
a partition
and tlle measurenents have tire corre-
s n o n , l in o n : r t i t i n n lti. = L
^
[ ] ' _ l 'f 1 , , ) ,
f r o n w hj - c l . rt h e c o e f f i c i e n t s
are calcr.rlated. Correspondingly,
the parametcr matrix A, and its
inverse
- M -
r = A -1 = u ;'r, , herro nrrtitions
r = f L ^ *t - ,
o = / o * *^ *'r \
l
tl n
\
/
/
l r
)
A
\"yx"yyl
t
\
r
r
i
I
YYI
\Y*
where the diagottal subrnatrices are square, or of x, y.
t h e d j . r n e n s i o n sp , q t trly'NIx
t
r
,
"*
xy
= lt1*'l'1,
-1
. A * * = ( I : x ' 6 y l l x . ) - r , , \1x r
giving
The formulae the
coefficient
the
c li s t r i b u t i o n
-Rx2) ' or
the
--1 -
yx'**'*1'
1
)
''xY
"xI
t
'
of
between x'
s*,
R'
correlation y
and
as functions
of
are 2
.' - 1
1
= yy ' sxY lA lb I . \X' XX
x
rr /
ir nr fi n( L r er lLr re rnco l ci rr ' l l . L u .
. ^ r r ^ l a f
i n n
e n d
S i m i l : r l qwL t l ,
n f
i n e
fnr
l r r s i n n
I '- * ? y -_ lI I - r x x A x x L' a s s o c i a t i o n a n d < d i s s o cai t i o n a r e d e t e r -
of
The cocfficients
,
coefficient
r e s i c l u a1
of
the coefficients
fn
-l 't-1 = (r rt !r l ^'r 6l y l : x ) - ' ) 1 " ' e r e * l , i r ( ], \; 1 ,1 . , ""^.r.,
the
r,'i th
1
-
paralneters
sanre fornulae
enrre
,
separation
of
= trace ,
lxplicitly
nr,--lar
nj-ned by )
a*,
^
4
1
^
The canonlcal correlation
2
roots,
and their
multiplicities
together rvith their
multi-
of the equati-on , 2 lu t**A**-
. tillcrc \
^
coefficients
are determined fron the posltive p1icities,
?
Rxy/pq,tir=t-c"y
2= r -
| 1l=
0
7 L- , or tlre sane equation tritlr x, y interclranged.
- 6 5 -
Q u a d r a t i c d e c o n p o si t i o n
1 1 A I
The follorving
of the algebraical
is a reformulation
n r n n n q it i n n n n t h i c h t h e t l t e o r e n o f C o c h r a n ( 1 9 3 4 ) o n t l - r e of quadratic forn's depends, in which ortlrogonal
distribution projectors
and rvhich shows a significance
again have a roIe,
the forrn in whlch a surn-of-squares decornposition has been
for
derived fron a regression. If
are positive definite
a,b,
of rank p, q,
s y r n m e t r i cr a t r i c e s
and of order l'J, such that
a + b +
1,
then p + q + . . . = N if
and only if
there exists a deconlosition of the rrnit
matrix into mutually orthogonal projectors
'.e'
f,
that is a
and an orthogonal
+
f
+
=
1I
n F t
n v
,
,
t r a n s f o r n n a t i o n L I, s u c l - r t h a t a = l l' e U ,
L2
=
G e n e r a! i z a t i o n i n
l l ' il h e r t
b = U'fU,
snace
The analysis i+hich has been given for pairs of subspaces of a fi-nite
dimensional Euclidean space generalizes wi-th only
the slightest
m o d i f i c a t i o n s o f r n e t h o dC i r e c t l y
dimensional unitary
space; and it
can be put in a spectral
forn which can be i.nterpreted in llilbert that generalization.
A different
question of the unitary
invariants
h a s b e e n m a d e b y I l i - x m i , e r( 1 9 4 8 ) .
to a finj"te
space' and suggests
approach to the related of a paj,r of subspaces
- 6 6 -
b e a p a i - r o f s u b s p a c e so f a H i l b e r t
Let €,T and let
e, f be the orthogonal projectors
:re
nrir
the
of
haveC, fi fot
i r 1c m n o t e n t
ltcrmitian
tp^r"{
,
on them; so er f
oncrators
in
whiCh
ranges.
their
Nou efe, fef are a pair of non-negativeIlermitian o p e r a t o r s w h i c h h a v e t h e s a m es p e c t r u m I , e x c l r r d i n g0 , w h i c h i s a c o r n n a c ts e t o n t h e r e a l a x i s b e t w e e n 0 a n d 1 . Theq following the formulatjon oI thc spectral theoremfor l l e r m i t i a n o p e r a t o r s g i v e n i n I I a l m o sf 1 9 5 1 ) , t h e r c e x i s t u n i q u e s p e c t r a l n r e a s u r e sE ( o ) , F ( o ) ( " € I
),
defined on
,4
s u b s p a c e< s . Yo f I , s u c h t h a t efe =
, fef = ('^ar(^)
[rdr(.r)
J
where,
in
addition
to
the
E ( o ) E ( p )= O , rnrried
hv
the
J
autonatic
F ( o J F ( p )= 0 cnnefrel
oennrrl
there are the further
orthogonali.ties (o ao
theOfem
fOf
= 0)
,
I l e f m i t .i .a. n. .
. - lO - .n
eTatOrS.
orthogonalities
E ( o ) F ( p )= o
(oAo
which arise frorn the peculiar
= 0) ,
form of construction of this
p a i r o f I I e r m ti i a n o p e r a t o r s o u t o f a p a i r o f l l e r m i t i a n i d e m p o t e n t s .\ l o r e o v e r , t h e s p e c t r a l m e a s u r c sE ( l ) , the
DOS it
ive
i r-a ts,rvr
-S/Dt /e! c! r r u ' r '
orthogonal projections
tl-,^
ef,
F(l) of
^ vr +LL r i u l { u t r d1r -p*l ^r r Ji s^L^ L+ L1 r S
On
the
f€of F and €.on € and T ,
The matter is easily
settled
is concerned;but it
calls
so far as the point
spectrurn
for further analysis in respect to
t h e a p p r o x i m a t ep a r t o f t h e s p e c t r u m , u s i n g t h e I a c t t h a t t h c o p e r a t o r s c o n s i d e r e d c a n b e a p p r o x i r n a t e db y o p e r a t o r s w i t h nnint
qnaafra
-
oa
-
A P P E N I ] ] XI O N T H E L A T E N TV E C T O R SA N D C H A R A C T E R I S T IV CA L U E SO F P R O D U C T S O F P A i R S O F S Y N { M E T R IIC DEMPOTENTS
Let e,f ". be a pair of symmetric idempotents with real elements. Then F o r a s y m n e t r i c i d e m p o t e n tm a t r i x e , s , = e , e 2 =
the protJucts ef, = ftet
= fe;
fe are transposes of each othcr: (ef)'
s o t h e y h a v e t h e s a r n ec h a r a c t e r i s t i c
and these values and nulti-
w i t h t h e s a m em u l t i p l i c i t i e s ; plicities
characterize a symmetrical relation
THE0REM I:
values,
The characteristic
between e and f.
v a l u e s o f e f a r e a 1 . l -r e a l .
at
least zero and at most unlty. values of ef = eeff are the sane as
The characteristlc those of feef = (ef)ref, definite; all
symmetric and non-negative
hence they are real and non-negative; and they are
zero if If
whi,chis
and only if
ef = 0.
e f I U , l e t t rh e a n y n o n - z e r o c h a r a c t e r i s t i c
necessarily rea1, and 1et tJ be a real belongs to it,
latent
value,
vector which
so that efU = Ur eU = eefUl-l
Hence
Sincer I O, -1 = eftjr = U.
tJrfU = U'efU = U'U). ,
which gives I = U'fll/U'U. But the stationary values of X'fX/X'X values of f,
are the characteristic the calculus;
as appears by methods of
and these values are all
X'fX is a continuous function
zero or unity.
j-n the closed region X'X = 1, the
maxinun and minimumvalues are anong these stationary and so it
follows
values,
that 0 ' d c r . r t - r t c lt 'l 1 ' e @ J i r r Cj " . cl n t . ' l ' h e y
be
ancl tlrcir
sqLrarenatrix.
to
ortltogonal
1 - o I r ,l i i c h
exists
a n,atri-x a
aitclthe
is
e\:c'ry
I, I,
t l - , c ) 't r r c s a j t l
of
orclel qx I
sufficicnt
trnion
l'lrespaces are
r) r ' ) r i c l rc : r s c c i s s a i . l t o l c
anrl conve'rse1,v; ancl a necessarl'and ts
tite nu11 space,
case p + Q = n,
onc is
[rre
(frf) of
to
saitl
= Cf ; othcrt',isc
*| t r - 7J - , i l
is
tirc sprccs,
V e c t c ) r - i r r t l r c ' o t l t c - r ' , t l t c c o n r lj t i o r or
tlte bascs,
cP
tlrcl'are'scparatc
a r c g r rl a r
e \ ' 6 r r ' ) '\ ' e c t o r
t nt
the jojrr
a l r a s c ; o t h e r ' \ i i s c ' t l r c ' 1 'a r c
;lrc callccl conIlerrentnrl
rt
intersection
anclonly'i.f
anclrvith bascs
c li n e n s i o n p , q
corrcs]-ronCingll',also
if
the case,if
sy.;t1o r f also
tr" an,v spaces in"C of
= 6, to
he
ilrclrr.lc,l such that
conclition
for
I
this
IpI1, = I1., r''lrich in|lics l. = |o
rr,itird =
- 1 1 ,' 1 . . (l 'l ; '],
. \ ec o r c l i . n g J y , t h c
s1."1qg5 ale
i