A Berry-Esseen Bound For Least Squares Error Variance Estimators Of Regression Parameters

Lithuanian Mathematical Journal, Vol. 39, No. 1, 1999

A BERRY-ESSEEN BOUND FOR LEAST SQUARES ERROR ESTIMATORS OF REGRESSION PARAMETERS

VARIANCE

M. Bloznelis and A. Ra~kauskas Abstract. We construct a precise Berry-Esseen bound for the least squares error variance estimators of regression parameters. Our bound depends explicitly on the sequence of design variables and is of the order O(N -t/2) if this sequence is "regular" enough. Key words: Berry-Esseen bound, least squares estimators, error variance estimators, linear regression.

1. I N T R O D U C T I O N AND RESULTS Consider the linear model

Yi=clq-flXiq-gi,

i = 1 . . . . . N,

(I.1)

where Yl, - . . , Y~' are the observed responses to given variables (design points) X = (Xl, . . . , Xlv), and where ot and fl are unknown regression parameters to be estimated. Here el . . . . . e~v are unobservable errors. We assume that el . . . . . ere are independent identically distributed (i.i.d.) mean zero random variables. The ordinary least squares (OLS) estimators of the parameters ce and fl are defined by f = ' Y - / 3 X" and

/~ ----))-2(xlyl -I---" -4- XNYN)

(see [8]). Here xk = Xk -- X , Yk -~- Yk -- "Y, and "l) = (x 2 -I- . . . q- x 2 ) 1/2. We are interested in the rate of the normal approximation of f - tr and/3 - fl in the case where the variance Ee 2 is unknown. In this c a s e 0-2 is estimated by

0 -2 =

s2 =

(~ +...

+ ~)/(u

-

2),

~k = rk - a - ~ xk

(see [8]) and is used to normalize f - ct and/3 - ft. By the central limit theorem, for large N, distributions of the statistics

a-~ Ol = O I ( X ) =

~ ,

Qs

/~-/~ 02 = 0 2 ( X )

-- - - ,

);s

where

Q=

-~+~]

,

can be approximated by the standard normal distribution. We construct a bound for the accuracy of the normal approximation o f these statistics.

Supported by the Lithuanian State Science and Studies Foundation. Vilnius University, Naugarduko 24; Institute of Mathematics and Informatics, Akademijos 4, 2600 Vilnius, Lithuania. Published in Lietuvos Matematikos Rinkinys, Vol. 39, No. 1, pp. 1-8, January-March, 1999. Original article submitted May 27, 1998. 0363-1672/99/3901-0(~1522.00 9 1999 Kluwer Academic/Plenum Publishers

2

M. Bloznelis and A. Ra&auskas

The Berry-Esseen bound given in Theorem 1.1 below is precise and depends only on the ratio fl3/cr 3, where fl~3 = E[ei 13, and the fraction Ixkl3 1;3

g = gX -- EN--1

If V = 0 , p u t 8 = 1 andOi = 0 . THEOREM 1.1.

Assume that r 2 > O. Then there exists an absolute constant c > 0 such that

suplP{0,. ~<x} x

-

O(x) I ~< cE 3---L3 i = 1, 2, 0"3'

where dO(x ) denotes the standard normal distribution function.

Here and below, c, co, cl . . . . denote generic absolute constants. By c(T1, T2 . . . . ) w e denote a constant which depends only on the quantities indicated in the brackets. If the design points X are regular enough, e.g., they are regularly spaced, Xk = a + k t, k = 1, . . . , N, for some a, t, then Theorem I. 1 yields the Berry-Esseen bound /~3 ' suplP{Oix ~< x} - O(x) l ~< c 4~'cr3

for

i = 1,2.

The rate O ( N -tIE) is achieved also when the design points are taken at random and independently of the errors. Consider the linear model (1.1) in the case where the design points 2' = (2"1,. -., 2"~') are i.i.d, random variables independent of el . . . . . etc. Write F3 = E I2"l --E2"t[ 3 and r 2 = Var2"l. Theorem 1.1 implies the following bound. COROLLARY 1.1.

Assume that cr2 > 0 and r z > O. Then

suplP{~

~ x] - ,t,(x) I ~< c J ~/33Y3 3 r3'

for

i=1,2.

It is easy to recognize the structural similarity between the statistics 0t and 02 and the Student statistics, based on non-identically distributed observations. The Berry-Esseen bound for the Student test was proved by Bentkus and Gttze [1] and extended to the non-i.i.d, case by Bentkus, Bloznelis, and Gttze [2]. In order to obtain the optimal results for error variance estimators of regression parameters we extend the methods developed in these papers and then apply them to the statistics 01 and 02. It seems that the traditional technique, see, e.g., [7], does not allow one to obtain the optimal results: it involves moments of order which is higher than the optimal one. The rest of the paper is divided into two sections. In Section 2, a general Berry-Esseen type bound for statistics such as 01 and 02 without the assumption that the second moment of errors 8i is finite is formulated. Theorem 1. I is a simple consequence of this result. Proofs are given in Section 3. 2. A G E N E R A L R E S U L T Given two sequences of real numbers w = {wl, . . . , WN} and z = {zl . . . . , ZN} such that N

E

N

w/2= 1

i=1

and

E

z 2 = 1,

i=1

we denote T = T ( w , z) = S ( w ) / s ( z ) ,

(2.1)

Berry-Esseen bound

3

w~aere N

S(1o) = E

s2(Z) = ~2 _ N-IS2(z),

wigi'

N

and

= (el + . . . + e r e ) I N

~2 = N - I E ( e i

_ ~2.

i=1

Let E(w) = )--~=1 Iwil 3. Note that by (2.1), N

E(w) >1 N -1/2

and

Elwil~< N 1/2.

(2.2)

i=1

Clearly, the same inequalities hold for ~(z) and ~-~/N=t[Zil as well. Define the number a 2 = a 2 by the truncated second moment equation: a 2 = s u p { b : Eet21{el2 < ~ b N } />b},

a/>0.

It is known (and easy to show) that for any random variable (r.v.) el and any N such a number a exists, and a is the largest solution of the equation a 2 = Ee21{el2 ~< a2N}. Moreover, if r < c~, then a ~< 0". If o" = 00, then aN --+ (30 as N --* 00. In order to formulate a general result we introduce the truncated random variables

ri = a - l N - l / 2 e i l { e 2 i ~a2N} +P{IAI >i 3/4}. By Chebyshev's inequality, Lemma 3.1 (see below), and the inequality N -1/2 ~< N E Irt 13,we get P{Irtl/> 1/4} ~< cE [O[3/z ~ 1/4} ~< c N -I O, andk=l,2

i

E ~r

N