This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
0 , (ii) y* = oo but lim b (e )= 0 for e -+ 0 .
This page intentionally left blank
CHAPTER III
M-, L-, and R-estimates
5. M-estimates. Any estimate T,. defined by a minimum problem of the form n
(5.1 )
LP(xj; T,.)=min! 1
or by an implicit equation " L t/f(xi; T,.)=0,
(5.2)
1
where p is an arbitrary function, t/f(x, 8)=(iJ/iJ8)p(x; 8), shall be called an M-estimate (or maximum likelihood type estimate; note that p(x; 8)= -log f(x; 8) gives the ordinary M.L.-estimate). We are particularly interested in location estimates or
I p(xi- T,.)=min!
If we write the last equation as (5.3 )
I
wi
·(xi- T,.)=O
with Wj=
t/f(xj- T,.) xi- Tn
we obtain a representation of T,. as a weighted mean T.
"
L W jX j = L Wj
with weights depending on the sample. Our favorite choices will be of the form for lxl�c.
p(x)=x2/2 =clxl- c2 /2 t/f(x) = [x ]�c=-c
for lxl > c, for x < -c,
=x
for -c�x�c.
=c
for x >c,
13
14
CHAPTER III
leading to weights c
lx;
_
Tnl
for lx;- Tn l >c.
All three versions (5. 1 ), (5.2), (5.3) are essentially equivalent. Note that the functional version of the first form
J p(x- T(F))F(dx)= min! may cause trouble. For instance the median corresponds to p(x)= lxl, and
J lx- TIF(dx)=oo unless F has a finite first absolute moment. This is o.k. if we examine instead
J (lx- Tl-lxi)F(dx) = min!. Influence curve of an M-estimate. Put F, = (1 - t)Fo+ tF1;
then the influence curve IC(x; F0, T) is the ordinary derivative
T=
[ dd T(F;)] (
t=O
with F1 = l>x.
In particular, for an M-estimate, i.e. for the functional T(F) defined by
J l{!(x; T(F))F(dx)=O we obtain by inserting F, for F and taking the derivative (with l{!'(x, 8)= (a/iJ8)l{!(x, 8)):
J l{!(x; T(Fo))d(Fl- Fo)+ t J l{!'(x; T(Fo))Fo(dx) or
T=
=
0
J l{!(x; T(Fo))Fl(dx) -J l{!'(x; T(Fo))Fo(dx)"
After putting F1 = l>x, we obtain
IC(x; Fo, T)=
l{!(x; T(Fo))
- J l{!'(x; T(Fo))Fo(dx)"
So let us remember that the influence curve of an M-estimate is simply propor tional to 1{1.
M-, L-, AND R-ESTIMATES
15
Breakdown and continuity properties of M-estimates. Take the location case, with T(F) defined by
J l{!(x- T(F))F(dx) = 0. Assume that 1{1 is nondecreasing, but not necessarily continuous. Then A(t , F) =
J l{!(x-t )F(dx)
is decreasing in t , increasing in F. T(F) is not necessarily unique; we have T*� T(F)� T** with T* =sup {tiA(t , F ) > 0} , T** = inf {tiA(t , F) xo +e, forx�xo +e,
=0 where x0 is defined by
Fo(xo) = e. Thus
A (t, F)�A(t, Ft)= Define
J"" l{!(x- t + e )F0(dx ) +el{!( oo). xo
b+(e )=sup { T(F)iddFo, F )�e} = inf {tiA(t , Ft) 0}; then the maximum asymptotic bias is bt(E ) = max {b+(e )- T{ Fo), T{ Fo)-b-(e )}.
CHAPTER III
16
Breakdown. b+(e) < b+(1)=co holds iff 1/1(00) < CX)
and lim,\ (t, F1)= (1-e)1/1(-co)+ ei/J(+co) < 0.
, .... oo
For h_(e) a similar relation holds, with the roles of +co and -co interchanged. It follows that the breakdown point is e*=_!!_ 1+TJ
with
.{
11 =mm
-
}
1/J(-co) 1/J(+co) 1/J(+co)' 1/1(-co) .
If 1/J(+co)=-1/J(-co), we have e*=!. Continuity properties. Putk =1/J(co)- 1/1(-co). Then A(t+ e, Fo)-ke�A(t, F)�A(t- e, Fo)+ke.
Hence if (i) 1/J is bounded, and (ii) A(t, Fo) has a unique zero at t= T(Fo), then T(F)-+ T(F0) as e-+ 0. It follows that Tis weakly continuous at Fo. The conditions are also necessary. Examples. (i) The median, corresponding to 1/J(x)=sign (x), is a continuous functional at every Fo, whose median is uniquely defined. (ii) If 1/J is bounded and strictly monotone, then the corresponding M-estimate is everywhere continuous. 6. L·estimates. Let M be a signed measure on (0, 1). Then the functional
(6. 1) induces the estimates
J
T(F)= F-1(t)M(dt) "
T,. = T(F,.) = L a,.;X i-1
with
In order that the T,. are translation invariant, the total algebraic mass of M should be 1 . As a special case, we first derive the influence function of T, =F-1(t) , the t-quantile. We take the derivative of Fs(F;1(t))=t
M-, L-, AND R-ESTIMATES
17
at s =0 , with F. = (1- s )Fo + sF1. and obtain or
Ft (Fo'(t))- Fo (Fo'(t))+ /o (Fo1(t))Tr =o,
(6 2) .
Example 1. For the median (t =!) we have 1 IC(x; F, Tt12 )=2/ (F '(!
))'
x < p-'( f),
1 The general case is now obtained by linear superposition: (6.3)
J =J M(dt)- fcx> M(dt). f ( F \t)) f (F!'(t))
IC(x;F, T)= IC(x;F, Tt)M(dt)
Example 2. If T(F)=L (3;F-1(t;), then IC has jumps of size (3;//(F-1(t;)) at the points x=F -1(t;). If M has a density m, then we may differentiate the expression (6.3) and obtain the more easily remembered formula (6 .4)
d iC(x;F, T)= m(F(x)). dx
Example 3. The a-trimmed mean 1 T(F)= - a 1 2
r-"' p-'(t) dt
has an influence curve of the form shown in Fig. 2.
I slope= -
l-2a
� I I I I
1 F-'(a)
r'(l-a) F10.2
CHAPTER III
18
Example 4. The a -Winsorized mean. For g/n =a, 1 X.. w =-(gX(g+t>+ X(g+tl+ · n
· ·
+ X(n-g>+ gX(n-g>)·
The corresponding functional
r-a p-1(t)dt+a(P-1(a)+P-1(1-a))
has the influence curve shown in Fig. 3 .
. , ----.-a--- -
f(F 1(1 -a))
a f(F 1(a))
� I
FIG.3 Breakdown and continuity properties of L-estimates. Assume that M is a positive measure, with support contained in [a, 1 -a] where 0 < a a!; leta be the largest such number. As in the preceding section, we calculate
b+(e ) = sup { T(F)I dL(Po, F) ae}
{ J F-1(t)M(dt) }
= sup
J
= F11(t}M(dt)
J
=e+ P01(t+e)M(dt), and symmetrically,
J
b_(e)=-e+ P01(t-e)M(dt). Clearly, these formulas make sense only if the support of M is contained in [e, 1-e]; otherwise b+ or -b _ or both will be oo. It follows that e*=a. Moreover, as P01(t+e)-P01(t) � 0 for e � 0, except at the discontinuity points of Po\ we conclude that
b+(e)- T(Po)=e+
J [PiJ1(t+e)-P01(t)]M(dt)
converges to 0 as e �o, unless P01 and the distribution function of M have common discontinuity points, and similarly for T(Fo)- b-(e).
19
M-, L-, AND R-ESTIMATES
It follows that Tis continuous at all Fo where it is well defined, i.e. at those Fo for which F01 and M do not have common discontinuity points. 7. R-estimates. Consider a two-sample rank test for shift: let Xt. · · · , Xm and Yt. · · · , Yn be two independent samples with distributions F(x) and G(x) = F(x - a) respectively. Merge the two samples into one of size m + n, and let R; be the rank of X; in the combined sample. Let a;== a(i) be some given scores; then
base a test of a= 0 against a> 0 on the test statistic (7. 1 )
Sm,n
1 m L a (R; ).
m i=I
==-
Usually, one assumes that the scores a; are generated by some function J as follows: (7.2)
a; ==J
(m + n + 1). i
But there are also other possibilities for deriving the a; from J, and we shall prefer to work with
i i/(m+n) a; = (m + n )
(7.3)
(i-1)/(m+n)
J(t) dt.
Assume for simplicity m = n and put (7.4)
S(F, G) =
r J(!s +!G{F-1(s ))) ds.
Then Sn,n = S(Fn, Gn). where Fno Gn are the sample distribution functions of (xt. · · · , Xn) and (Yt . · · · , Yn) respectively, provided we define the a; by (7.3). One can derive estimates of shift dn and location Tn from such tests: (i) adjust An such that Sn,n = 0 when computed from (Xt. · · · , Xn) and (Yt
dn, ·
· ·
, Yn - dn)•
(ii) adjust Tn such that Sn,n =0 when computed from (x t. · · · , X n) and (2Tn Xt. · · 2Tn - Xn). In this case, a mirror image of the x-sample serves as a stand-in for the missing second sample. (Note that it may not be possible to achieve an exact zero, Sn,n being a discontinuous function.) Example. The Wilcoxon test corresponds to J(t) = t - ! and leads to the Hodges-Lehmann estimate Tn = med {(x; + xi)/2}. In terms of functionals, this means that our estimate of location derives from T(F), defined by the implicit equation ·
(7.5)
,
f J{!(s + 1- F(2T(F)- F-1(s))]} ds
=
0.
From now on we shall assume that 1(1- t) = - l(t), 0 < t < 1 .
20
CHAPTER III
Influence curve. As in the preceding sections, we find it by inserting F, for F into
(7 .5) and taking the derivative with respect to t at t=0. After some calculations
this gives
U(x)- J U(x)f(x) dx IC( x'. F.' n= ' J U'(x)f(x) dx where f is the density of F, and U is defined by its derivative
U'(x)=J'{![F(x )+ 1 - F(2T(F)- x) ]}{(2 T(F)- x). If the true underlying distribution F is symmetric, there is a considerable simplification: U(x)= J(F(x )), and thus
J(F(x)) IC(x;F, n= JJ'(F(x))f(xi dx
for symmetric F.
Example 1 . Hodges-Lehmann estimate:
!- F(2T(F)- x) (general F). IC(x; F , n= Jf (2T(F) - x){(x) dx =
F(x)- ! Jf(x)2 dx
(symmetric F).
Example 2. Normal scores estimate J(t)=�-1(t): �-1 (F(x)) IC(x ;F, n= Jf(x)2/cf>[� 1(F(x))] dx
. (symmetnc F)
In particular:
IC(x;�. n=x. Breakdown. The maximum bias b1(e) and the breakdown point e* can be worked out as in the preceding sections if J is monotone; e* is that value of e for which 1 l-e/2 J(s) ds. J(s) ds= 1/2 1-e/2
i
Hodges-Lehmann:
1
e* =1-1/./2::::.0.293
Normal Scores: Note that the normal scores estimate is robust at�. even though the influence curve is unbounded there. 8. Asymptotic properties of M-estimates. The influence function gives a nice, unified description of the asymptotic properties of arbitrary statistical functionals
M-, L-, AND R-ESTIMATES
21
T(F,.): it is usually true that J'n(T(F,.)- T(F )) is asymptotically normal with mean 0 and variance
J IC(x; F, T)2F(dx).
However, proofs via the influence function are rarely viable (or only under too restrictive regularity assumptions.) We shall now sketch a rigorous treatment of the asymptotic properties of M-estimates. Assume that 1/J(x, t) is measurable in x and decreasing ( =nonincreasing) in t, from strictly positive to strictly negative values. Put
{ l � 1/J(x;, t)>O} , 1!*= inf { l � 1/J(x;, t)/2. M;
l.{! (x) = x
sample mean, nonrobust,
L:
m(t) = 1
R:
J(t) = -1(t) normal scores estimate, robust.
sample mean, nonrobust,
(The normal scores estimate loses its high efficiency very quickly when only a small amount of far-out contamination is added and is soon surpassed by the Hodges Lehmann estimate. Thus, although it is both robust and fully efficient at the normal model, I would hesitate to recommend the normal scores estimate for practical use.) x Example 2. Logistic distribution F0(x) = 1/(1 +e- ). M:
l.{!(x) = tanh (x/2) robust,
L:
m(t)=6t(1 - t)
nonrobust,
R:
J(t) = t - !
robust
(Hodges-Lehmann estimate). Remember that an L-estimate is robust iff support (M) = [a, 1- a] for some 0 X o, where fP is the standard normal density. Note that F+ is symmetric around x0, and that also belongs to Pl'e. Thus, we must have for any translation invariant functional T. It is obvious from this that none can have a smaller absolute bias than x0 at F+ and F_ simultaneously. For the median, we have (rather trivially) b1(e ) = b (s ), and thus we have shown that the sample median minimizes the maximal asymptotic bias. We did not use any particular property of Pl'e, and the same argument carries through with little changes for other distributions than the normal and other types of neighborhoods. It appears, not surprisingly, that the sample median is the estimate of choice for extremely large sample sizes, where the possible bias becomes more important than the standard deviation of the estimate (which is of the order 1 /../n).
29
CHAPTER IV
30
12. Minimax asymptotic variance. In the following, rJ' will be some neighbor hood of the normal distribution , consisting of symmetric distributions only, e.g.
r!P = {FIF
or
rJ'
=
(1 - e ) + eH, H symmetric},
= {Fi d*(, F) � F symmetric}. e,
It is convenient to assume that rJ' be convex and compact in a suitable topology (the vague topology: the weakest that F � J r/1 dF is continuous for all continuous r/1 with compact support). We allow rJ' to contain substochastic measures (i.e. probability measures putting mass at ±oo); these may be thought to formalize the possibility of infinitely bad outliers. The problem is to estimate location fJ in the family fJ), F E The theory is described in some detail in Huber (1964 ); I only sketch the salient points here. First, we have to minimize Fisher information over rJ'. 1 . Define Fisher information as
F(x -
r!P.
I(F)
= !�C� (JJ r/1'r/12dFf dF
'
(where �i is the set of continuously differentiable functions with compact support). 2. THEOREM. (i) J(F) < oo. (ii) J < oo. I(F) J (Liggett). Define a linear functional A by
The following two assertions are equivalent: F has an absolutely continuous density f, and (f'Ifitdx In either case, = (f'I/)2/ dx. Proof Ar/1 - f r{l' dF =
Note that J(F) is the square of the L2 -norm of A, hence A is bounded if I(F) is finite. By Riesz' theorem then there is a E L2 (F) such that
Ar{l = f r{lg dF
g
for all r/1 E L2 (F).
We do not know yet whether F has an absolutely continuous density, but if it has, then Ar/1 =
hence
-
J r/l'f dx = J r/ljtdx,
g = f'If. We thus define f(x) = {<x g(y )F(dy ).
The rest of the proof is a matter of straightforward verification.
ASYMPTOTIC MINIMAX THEORY
31
3 . I( · ) i s lower semi-continuous (being the supremum of a set of continuous functions); hence it attains its minimum on the compact set P/J, say at F0• 4. I( ) is convex. 5 . If /0 > 0, then F0 is unique. 6. The formal expression for the inverse of the asymptotic variance of an M-estimate of location, ·
1 A (F, 1/!o) is convex in F. 7. Take 1/Jo -!�/fo . Then =
1 A (Fo, 1/Jo)
8. Let F, = (1 - t)Fo + tFt, I(Ft ) < oo. Then, explicit calculation gives
[d 1 dt A (F,, 1/10)J -
r=o
=
=
l(Fo).
[ d I(F,) dt J -
t=O
2:: 0.
-
It follows that the asymptotic variance A (F, 1/10) attains its maximum over PJ at F0, but there the M-estimate is asymptotically efficient. Hence the M.L.-estimate for location based on F0 is minimax over P/J. Example. For e-contamination, we obtain 1 - E -p(x) 1Io(x ) = -- e
../2;
with
l x l � c.
where c = c (e ). The L- and R-estimates which are efficient at F0 do not necessarily yield minimax solutions, since convexity fails (point 6 in the above sketch of the proof). There are in fact counter-examples (Sacks and Ylvisaker (1972)). However, in the important case of symmetric e -contamination, the conclusion remains true for both L- and R-estimates (Jaeckel (197 1a)). Variants. Note that the least informative distribution Fo has exponential tails, i.e. they might be slimmer(!) than what one would expect in practice. So it might be worthwhile to increase the maximum risk a little bit beyond the minimax value in order to gain a better performance at long tailed distributions.
32
CHAPTER IV
This can be done as follows. Consider M-estimates, and minimize the maximal asymptotic variance subject to the side condition 1/!(x) = 0
for /x / > q.
The solution for contaminated normal distributions is of the form (Collins
(1976)) (see Fig. 4):
1/!(x ) = x,
O � x � c.
1/!(x ) = b tanh [tb (q - x )],
c �x �q
=0
X � q,
1/1(-x ) = -1/!(x ). l/l(x )
X
FIG . 4
The values o f c and b o f course depend on e . The actual performance does not depend very much o n exactly how 1/1 rede scends to 0, only one should make sure that it does not do it too steeply; in particular 1 1/1'1 should be small when / 1/1 / is large. Hampel's extremal problem (Hampel (1968) ) Assume that the model is a general one-parameter family of densities f(x, 6) and estimate 6 by an M-estimate based on some function 1/!(x, 6), i.e. .
n
L 1/!(X;, Tn) = 0. 1
Assume that T is "Fisher consistent" at the model, i.e.
J 1/!(x, 6)/(x, 6) dx = 0.
ASYMPTOTIC MINIMAX THEORY
Then
1/J(x, 8) J (aja8)1/J (x, 8)/ (x, 8) dx
IC(x; F6, T)= -
_
33
1/J(x, 8 ) J 1/J(x, 8)(aja O)f(x, 8 ) dx
and the asymptotic variance at the model is
f 1/!(x, 8ff(x, 8 ) dx
A (F6, "' )= (J 1/! (x, 8 )(aj a8)f(x, O) dxi ·
Hampel's extremal problem now is to put a bound on the gross error sensitivity: sup IIC(x; FIJ, nl � kll X
with some appropriately chosen function kll and, subject to this side condition, to minimize the asymptotic variance A (F6, 1/1 ) at the model. The solution is of the form ·'·
(
"' x, O)=
[ (aja8)f(x, 8 ) f(x, 0)
where we have used the notation [ x]�
=
a
for x � a,
= x for a < x < b, =
b
for x ?;. b.
The functions alh b6 are somewhat difficult to determine, and if ke is too small, there is no solution at all. It might therefore be preferable to start with choosing b9• A reasonable choice might be b6 = c.Jl(F6)
with c between 1 and 2, and where
is the Fisher information. Then one determines a6 (so that the estimate is Fisher consistent), and finally, one finds k6•
This page intentionally left blank
CHAPTER V
Multiparameter Problems
13. Generalities. As far as M-estimates are concerned, most concepts of the preceding chapters generalize to vector valued parameters. Asymptotic normality was treated by Huber (1967). The Fisher information matrix and the inverse of the asymptotic covariance matrix of the estimate are convex functions of the true underlying distribution, matrices being ordered by positive definiteness. Since this is not a lattice ordering, it is not in general possible to find a distribution minimizing Fisher information. But if there is one, the corresponding maximum likelihood estimate possesses an asymptotic minimax property: it minimizes the maximal asymptotic variance among all distributions for which it is Fisher consistent. 14. Regression. Assume that p unknown parameters ( 61 . · · · , 6p) = 9T are to be estimated from n observations (y1, · · · , Yn) = yT, to which they are related by
(14.1)
The f; are known functions, often assumed to be linear in 8, and the u ; are independent random errors with approximately identical distributions . One wants to estimate the unknown true 8 by a value 9 such that the residuals (14 . 2)
6.; = 6.;(8) = y; - /;(8)
are made "as small as possible" . Classically, this is interpreted (Gauss, Legendre) as (14.3)
L 6-� = min!, i
or, almost equivalently, by taking derivatives: j = 1, . . . , p.
(14.4)
Unfortunately, this classical approach is highly sensitive to occasional gross errors. As a remedy, one may replace the square in (14 . 3) by a less rapidly increasing function p : (14.5)
L p (6.; ) = min!, 35
CHAPTER V
36
or, instead of (14.4), to solve j = 1, . . . , p,
(14.6)
for 9 with I{! = p '. There are also other possibilities to robustify (14.3). For instance, Jureckova (1971) and Jaeckel (1972) have proposed to replace the residuals A; in (14 . 4) by their ranks (or more generally, by a function of their ranks). Possibly, it might be good to safeguard against errors in the {; by modifying also the second factor in (14.4), e.g. by replacing also a{;/a8i by its rank in (afda8i> · · · , afn/a8i), but the consequences of these ideas have only begun to be investigated (Hill (1977)). In any case, the empirical evidence available to date suggests that the M estimate approach (14.5), (14.6) is easier to handle and more flexible, and even has slightly better statistical properties than the approaches based on R- and L-estimates. There is only one minor disadvantage: one must simultaneously estimate a scale parameter S in order to make it scale invariant, e.g.
LP
(i) = min!
where S is determined simultaneously from n
� L 1/1( �r = f3 = E.p(l{!(x )2). P
In the regression case I would prefer this S to, say, the median absolute value of the residuals, since it is more easily tractable in theory (convergence proofs) and since it fits better into the established flow of calculation of 8 in large least squares problems. In order that robust regression works, the observation y; should not have an overriding influence on the fitted value
9j = {; (8).
To clarify the issues, take the classical least squares case and assume the f; to be linear: Then
y = fy
with If var (yi) and
=
u2, we obtain var (A;) = var (y; - M=(1 - Ya)u2 ,
where Yu is the ith diagonal element of f.
37
MULTIPARAMETER PROBLEMS
Note that tr (f) = p, so max 'Y;; � ave y;; = p/ n ; in some sense, 1 / y;; is the effective number of observations entering into the determination of y;. If 'Yu is close to 1 , y; is essentially determined by alone, may have an undue leverage on the determination of certain parameters, and it may well be impossible to decide whether contains a gross error or not. The asymptotic theory of robust regression works if e = max y;; goes to 0 sufficiently fast when p and n tend to infinity; "sufficiently fast" may be taken to mean ep 2 � 0 or (with slightly weaker results) ep � 0. If only e � 0, there may be trouble if the observational errors have an asymmet ric distribution and p is extremely large (above 1 00). This effect has been experimentally verified in a specifically designed Monte Carlo study (Huber (1973a)), but for all practical proposes, e � 0 seems to be o.k. Note that e � 0 implies p/ n � 0. As already mentioned, we propose to enforce scale invariance by estimating a scale parameter u simultaneously with 6. This can be done elegantly by minimiz ing an expression of the form
y;
y;
Q (6, u) = L P
y;
(y' -/.·' (6)) u + au,
u > O.
u
(The more natural looking expression derived from the simultaneous M.L. problem, which contains log u, would not allow us to push through a simple convergence proof for the numerical calculations.) In the above, p � 0 is a convex function, p(O) = 0, which should satisfy . p (x ) = c � co . 0 < hm X1 lxl-+oo -1-
If c ) . 3. Compute a new value for u by (u<m + l })2
4. "Winsorize" the residuals: Z;
=�I x(!�::) (u<m y ( A (m ) ) L.lj
(m + l}
= 1/J O"(m + l)
5 . Compute the partial derivatives
Cik =
0"
•
i = 1,
·
·
· , n.
a{;(o<m l) . iJ(h
6. Solve for 7.
7 . Put
o<m + l)
=
o< m ) + q7
where 0 < q < 2 is an arbitrary (fixed) relaxation factor. 8. Stop iterating and go to step 9 if the parameters change by less than e times their standard deviation, i.e. if for all j '-
I Tj " I < t: Y CjjO"
(m+l)
where cii is the jth diagonal element of the matrix C = (Crcr1 ; otherwise put m := m + 1 and go to step 2.
40
CHAPTER V
9. Estimate 9 by e<m + l l,
var (y;) by (O"(m+l lf, 2 1 var (0} by K L zr( C TC)- 1 . n -p
--
Here K is a correction factor (Huber (1973, p. 8 12ff)):
1 +....:.!. (p/n ) var-'-1{1'/(ave 1/1') --'..: :.._:_-;_c_ K =-
2 ave 1{1' ( Ll; ) , 1 ave 1{1 = ;; £.. 1/1 O" m +l) = ( , 1 [ ·( Ll; ) ]2 var 1{1 = ;; £.. 1{1 +l -
where
" "
P, ,
·
u <m
)
P,
·
In the case of "proposal 2", l{l(x ) = [x ]��. these expressions simplify: p, then is the fraction of non-Winsorized residuals, and K=
1 + (p/ n }(1 - p, )/p, . p,
Under mild assumptions, this algorithm converges (see Huber and Dutter (1 974)): assume that p (x )/x is convex for x < 0 and concave for x > 0, that O < p" ;§ 1 , and that the [; are linear. Then one can show that Q (O(m ), O"(m )) � Q (e<m l, u <m + l )) � Q (e<m + l ), O" (m +l l),
and that these inequalities are strict, unless O"(m) = O'(m + l ) or 9(m ) 9(m + l) respec tively. The sequence (o<m >, O"<m l) has at least one accumulation point (this follows from a standard compactness argument), and every accumulation point minimizes Q (O, u ) If the minimum is unique, this of course implies convergence. Remark. The selection of starting values (9b and we are seeking a pair Qi e @>i of closest neighbors, making the testing problem hardest (see Fig. 7).
FIG. 7 49
50
CHAPTER VI
If the likelihood ratio 1r(x) = q1 (x )/ q0(x) between a certain pair Qi E r!Pi satisfies for all t,
(17.3) (17.4)
sup P{1r(x ) > t} = Oo{1T(x) > t},
Pe f11'o
inf P{1r(x ) > t} = Q1{1r(x) > t},
Pe f11' 1
then, clearly, the one-sample Neyman-Pearson tests between 00 and Q, are minimax tests between f!Po and r!P1. One easily proves that this property carries over to any sample size. Note that it is equivalent to: for P E f!P0, log 1T is stochastically largest when P = Oo; hence L log 1r(x;), with ..2?(x;) e r!P0, becomes stochastically largest when ..2?(x;) = Oo (see, e.g. Lehmann (1959, Lem. 1, p. 7 3)). The existence of such a least favorable pair is not self-evident, and it was in fact a great surprise that the "usual" sets r!Pi all possessed it, and that the likelihood ratio 1r(x ) even had a simple structure (17.1) (Huber (1 965)). This has to do with the following: the "usual" neighborhoods r!P can be described in terms of a two-alternating capacity v, that is
( 1 7.5)
{fJ> = {P e At IVA, P(A) � v (A)}
where v is a set function satisfying (!1 being a complete, separable metrizable space, A, B, being Borel subsets of !1):
(i ) v ( such that for all i, P(A;) = v (Ai). This simultaneous maximizing over a monotone family of sets occurs in ( 1 7.3) and is needed for the minimax property to hold. Examples. Assume that n is a finite set, for simplicity, and let P0 be a fixed probability. (i ) v (A ) = (1 - e )Po(A ) + e for A cF
