Theory of Preliminary Test and Stein-Type Estimation with Applications

Theory of Preliminary Test and Stein-Type Estimation with Applications A. K. Md. Ehsanes Saleh Carleton University Ott...

Author: A. K. Md. Ehsanes Saleh

33 downloads 1265 Views 22MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Theory of Preliminary Test and Stein-Type Estimation with Applications

A. K. Md. Ehsanes Saleh Carleton University Ottawa, Canada

@ZELENCE A JOHN WILEY & SONS, INC., PUBLICATION

This Page Intentionally Left Blank


WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A. SHEWHART and SAMUEL S. WILKS Editors: David 1Balding, Noel A. C. Cressie, Nicholas I. Fisher: Iain M. Johnstone, 1B. Kadune, Geert Molenberghs, Louise M. Ryan, David W Scott, Adrian F M. Smith, Jozef L. Teugels Editors Emeriti: Ec Barnett, 1 Stuart Huntec David G. Kendall

A complete list of the titles in this series appears at the end of this volume.


A. K. Md. Ehsanes Saleh Carleton University Ottawa, Canada

@ZELENCE A JOHN WILEY & SONS, INC., PUBLICATION

Copyright 02006 by John Wiley & Sons, Inc. A11 rights reserved. Published by John Wiley & Sons, lnc., Hoboken, New Jersey. Published simultaneouslyin Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the I976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 11 1 River Street, Hoboken, NJ 07030, (201) 748-601 1, fax (201) 748-6008, or online at http://w.wiley.codgo/permission. Limit of Liability/Disclaimerof Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completenessof the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-PublicationData:

Saleh, A. K. Md. Ehsanes. Theory of preliminary test and Stein-type estimation with applications. / A.K. Md. Ehsanes Saleh. p. cm. Includes bibliographical references and index. ISBN-I3 978-0-471-563754 (acid-free paper) ISBN-I0 0-471-56375-7 (acid-free paper) 1. Parameter estimation. 2. Regression analysis. 3. Bayesian statistical decision theory. I. Title. QA276.8.8257 2006 2005050196 Printed in the United States of America 109 8 7 6 5 4 3 2 1

To SHAHIDARA, my wife


Contents List of Figures

xvii

List of Tables

xix

Preface

xxi

1 Introduction 1.1 Objective of This Book . . . . . . . . . . . . . . . . . . . . . . 1.2 Statistical Decision Principle . . . . . . . . . . . . . . . . . . . 1.3 Quadratic Loss Function . . . . . . . . . . . . . . . . . . . . . . 1.4 Some Statistical Models with Preliminaries . . . . . . . . . . . 1.4.1 Mean and Simple Linear Models . . . . . . . . . . . . . 1.4.2 One-Sample Multivariate Model . . . . . . . . . . . . . 1.4.3 ANOVAModels . . . . . . . . . . . . . . . . . . . . . . 1.4.4 Parallelism Models . . . . . . . . . . . . . . . . . . . . . 1.4.5 Multiple Regression Model and General Linear Hypothesis 1.4.6 Simple Multivariate Linear Model . . . . . . . . . . . . 1.4.7 Discrete Data Models . . . . . . . . . . . . . . . . . . . 1.5 Organization of the Book . . . . . . . . . . . . . . . . . . . . . 1.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 6 6 8 9 9 12 13 14 15 19 20 23 23 23

2 Preliminaries 2.1 Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Chi-square Distribution and Properties . . . . . . . . . . . . . . 2.3 Some Results from Multivariate Normal Theory . . . . . . . . . 2.4 Beta Distribution and Applications . . . . . . . . . . . . . . . . 2.5 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . 2.5.1 Binomial Distribution . . . . . . . . . . . . . . . . . . . 2.5.2 Multinomial Distribution . . . . . . . . . . . . . . . . . 2.6 Matrix Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Large Sample Theory . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Four Types of Convergence . . . . . . . . . . . . . . . . 2.7.2 Law of Large Numbers . . . . . . . . . . . . . . . . . . .

29 29 30 33 34 36 36 37 38 40 41 42

vii

...

CONTENTS

Vlll

2.7.3 Central Limit Theorems . . . . . . . . . . . . . . . . . . 2.8 Nonparametric Theory: Preliminaries . . . . . . . . . . . . . . . 2.8.1 Order-Statistics, Ranks, and Sign Statistics . . . . . . . 2.8.2 Linear rank-statistics (LRS) . . . . . . . . . . . . . . . . 2.8.3 R.ank Estimators of the Parameters of Various Models . 2.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

43 45 45 46 50 53

55 3 Preliminary Test Estimation 3.1 Simple Linear Model. Estimators. and Tests . . . . . . . . . . . 56 3.1.1 Simple Linear Model . . . . . . . . . . . . . . . . . . . . 56 3.1.2 Estimation of the Intercept and Slope Parameter . . . . 56 3.1.3 Test for the Slope Parameter . . . . . . . . . . . . . . . 57 3.2 P T E of the Intercept Parameter . . . . . . . . . . . . . . . . . 57 3.2.1 UE. R E and P T E of the Intercept Parameter . . . . . . 58 3.2.2 Bias and MSE Expressions . . . . . . . . . . . . . . . . 58 3.2.3 Comparison of bias and mse functions . . . . . . . . . . 61 3.2.4 Optimum Level of Significance of Preliminary Test . . . 63 3.3 Two-Sample Problem and Pooling of Means . . . . . . . . . . . 67 3.3.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.3.2 Estimation and Test of the Difference between Two Means 67 3.3.3 Bias and mse Expression of the Three Estimators of a Mean . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.4 One-Sample Problem: Estimation of Mean . . . . . . . . . . . . 73 3.4.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 3.4.2 Unrestricted, Restricted, and Preliminary Test Estimators 73 3.4.3 Bias, mse, and Analysis of Efficiency . . . . . . . . . . . 74 3.5 An Alternative Approach . . . . . . . . . . . . . . . . . . . . . 76 3.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 76 3.5.2 One-Sample Problem . . . . . . . . . . . . . . . . . . . . 77 3.5.3 Comparison of PTE, and SE 62 . . . . . . . . . . 80 3.5.4 Simple Linear Model and Shrinkage Estimation . . . . . 82 3.5.5 The Two-Sample Problem and Shrinkage Estimation . . 86 3.6 Estimation with Nonnormal Errors . . . . . . . . . . . . . . . . 88 3.6.1 Unrestricted, Restricted, Preliminary Test and Shrinkage Estimators, and the Test of Slope . . . . . . . . . . 89 3.6.2 Conditions for Asymptotic Normality of the Unrestricted Estimators of Intercept and Slope Parameters . 89 3.6.3 Asymptotic Distributional Bias and Mean Square Error Expressions, and Efficiency Analysis . . . . . . . . . . . 92 3.7 Two-Sample Problem and Estimation of Mean . . . . . . . . . 99 3.8 One-Sample Problem and Estimation of the Mean . . . . . . . 101 3.9 Stein Estimation of Variance: One-Sample Problem . . . . . . . 103 3.10 Nonparametric Methods: R-Estimation . . . . . . . . . . . . . . 109 3.10.1 Model and Assumptions . . . . . . . . . . . . . . . . . . 109 3.10.2 Test of Hypothesis . . . . . . . . . . . . . . . . . . . . . 110 3.10.3 Estimation of Intercept and Slope Parameters . . . . . . 111

f?LT

CONTENTS

ix

3.10.4 Asymptotic Distribution of Various Estimators and Their ADB and ADhlSE Expressions . . . . . . . . . . 112 3.1 1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 3.12 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

125 4.1 Statistical Model. Estimation. and Tests . . . . . . . . . . . . . 126 4.2 Preliminary Test Estimation . . . . . . . . . . . . . . . . . . . . 129 4.3 Stein-Type Estimators . . . . . . . . . . . . . . . . . . . . . . . 136 4.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 136 4.3.2 James-Stein Estimator (JSE) . . . . . . . . . . . . . . . 139 4.3.3 Positive-Rule Stein Estimator (PRSE) . . . . . . . . . . 143 4.3.4 Sclove-Morris-Radhakrishnan Modifications . . . . . . . 148 4.4 Derivation of the Stein-Type Estimators . . . . . . . . . . . . . 151 4.4.1 Risk Difference Representation Approach . . . . . . . . 151 4.4.2 Empirical Bayes Estimation (EBE) Approach . . . . . . 154 4.4.3 Quasi-empirical Bayes or Preliminary Test Estimation Approach . . . . . . . . . . . . . . . . . . . . . . . . . . 157 4.4.4 How Close is the JS Estimator t o the Bayes Estimator? 159 4.5 Stein-Type Estimation When the Variance is Unknown . . . . . 161 4.5.1 Introduction: Model, Estimators, and Tests . . . . . . . 161 4.5.2 Preliminary Test and Stein-Type Estimators . . . . . . 161 4.5.3 Empirical Bayes Estimation When the Variance Is Unknown . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 4.5.4 Bias, MSE Matrices, and Risk Expressions . . . . . . . 163 4.5.5 Risk Analysis of the Estimators . . . . . . . . . . . . . . 166 4.5.6 An Alternative Improved Estimator of 6 . . . . . . . . . 171 4.6 Stein-Type Estimation: Nonnormal Distributions . . . . . . . . 174 4.6.1 Model, Estimation, and Test . . . . . . . . . . . . . . . 174 4.6.2 Preliminary Test (or Quasi-empirical Bayes) Approach to Stein-Type Estimation of the Mean Vector . . . . . . 175 4.6.3 Asymptotic Distributional Bias Vector, Quadratic Bias, MSE Matrix, and Risk Expressions of the Estimators . 175 4.7 Improving James-Stein Estimator Toward Admissible Estimator 180 4.7.1 Introductions . . . . . . . . . . . . . . . . . . . . . . . . 180 4.7.2 Improving 6 , via P T E . . . . . . . . . . . . . . . . . . . 180 4.7.3 Iterative P T E t o Obtain an Admissible Estimator . . . 182 4.7.4 Extension t o the Case Where the Variance Is Unknown 183 4.8 Confidence Set Estimation Based on Stein-Type Estimators . . 185 4.8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 185 4.8.2 Properties of the Recentered Confidence Set Based on PRSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 4.8.3 Confidence Set Based on Preliminary Test Estimator . . 192 4.8.4 Asymptotic Theory of Recentered Confidence Sets and Domination of Positive-Rule Coverage Probability . . . 195 4.9 Nonparametric Methods: R-Estimation . . . . . . . . . . . . . . 197

4 Stein-Type Estimation

CONTENTS

X

Model and Assumptions . . . . . . . . . . . . . . . . . . Test of Hypothesis . . . . . . . . . . . . . . . . . . . . . Estimation of the Location Parameter . . . . . . . . . . ADB, ADQB. ADMSE, and ADQR of the Estimators of Location Parameters . . . . . . . . . . . . . . . . . . 4.9.5 Asymptotic Properties of Confidence Sets . . . . . . . . 4.10 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.11 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9.1 4.9.2 4.9.3 4.9.4

5

198 198 199 200 204 205 206

ANOVA Model 213 5.1 Model. Estimation. and Tests . . . . . . . . . . . . . . . . . . . 214 5.1.1 ANOVA model . . . . . . . . . . . . . . . . . . . . . . . 214 5.1.2 Estimation of the Parameters of the One-way ANOVA Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 5.1.3 Test of Equality of t h e Treatment Means . . . . . . . . 215 5.2 Preliminary Test Approach and Stein-Type Estimators . . . . . 218 5.2.1 Preliminary Test Approach (or Quasi-empirical Bayes Approach) . . . . . . . . . . . . . . . . . . . . . . . . . . 218 5.2.2 Bayes and Empirical Bayes Estimators of Treatment Means . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 5.3 Bias, Quadratic Bias, MSE. and Risk Expressions . . . . . . . . 221 5.3.1 Bias Expressions . . . . . . . . . . . . . . . . . . . . . . 221 5.3.2 MSE Matrix and Risk Expressions . . . . . . . . . . . . 224 5.4 Risk Analysis and Risk Efficiency - . . . . . . . . . . . . . . . . . 229 5.4.1 Comparison of 8, and 8, . . . . . . . . . . . . . . . . . . 229 +.PT 5.4.2 Comparison of 8, and 6,(6,) . . . . . . . . . . . . . . 230 -s -s+ 5.4.3 Comparison of 8,. 8, and 8, . . . . . . . . . . . . . . 232 5.5 MSE Matrix Analysis and Efficiency . . . . . . . . . . . . . . . 234 5.5.1 Comparison of 6, and 6, . . . . . . . . . . . . . . . . . 234 ..PT 5.5.2 Comparison of On Relative t o 6, and 6, . . . . . . . . 235 -s -s . S+ 5.5.3 Comparison of 6, and 6 , (8, and 8, ) . . . . . . . . . 237 240 5.6 Improving the P T E . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 ANOVA Model: Nonnormal Errors . . . . . . . . . . . . . . . . 242 5.7.1 Estimation and Test of Hypothesis . . . . . . . . . . . . 243 5.7.2 Preliminary Test and Stein-Type Estimators . . . . . . 244 5.8 ADB. ADQB. ADMSE, and ADQR . of the Estimators . . . . . 244 5.8.1 Asymptotic Distribution of the Estimators under Fixed Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . 245 5.8.2 Asymptotic Distribution of the Estimators under Local Alternatives . . . . . . . . . . . . . . . . . . . . pT . ,g . 246 5.8.3 ADB. ADQB, MSE-Matrices, and ADQR . of 8, 8, . S+ and 8, . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 250 5.9 Confidence Set Estimation . . . . . . . . . . . . . . . . . . . . . 5.9.1 Confidence Sets and Coverage Probabilities . . . . . . . 251

.

^

^

CONTENTS

xi

5.9.2 Analysis of the Confidence Sets . . . . . . . . . . . . . . 253 5.10 Asymptotic Theory of Confidence Set Estimation . . . . . . . . 258 5.10.1 Asymptotic Representations of Normalized Estimators under Fixed Alternatives . . . . . . . . . . . . . . . . . 258 5.10.2 Asymptotic Coverage Probability of the Confidence Sets under Local Alternatives . . . . . . . . . . . . . . . . . . 259 5.11 Nonparametric Methods: R-Estimation . . . . . . . . . . . . . . 260 5.11.1 Model, Assumptions. and Linear Rank Statistics (LRS) 260 5.11.2 Preliminary Test and Stein-Type Estimators . . . . . . 263 5.11.3 Asymptotic Distributional Properties of R-Estimators . 263 5.11.4 ADB, ADQB. ADMSE. and ADQR . . . . . . . . . . . 265 5.12 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 5.13 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 6 Parallelism Model 271 6.1 Model. Estimation. and Test of Hypothesis . . . . . . . . . . . 272 6.1.1 Parallelism Model . . . . . . . . . . . . . . . . . . . . . 272 6.1.2 Estimation of the Intercept and Slope Parameters . . . 272 6.1.3 Test of Parallelism . . . . . . . . . . . . . . . . . . . . . 274 6.2 Preliminary Test and Stein-Type Estimators . . . . . . . . . . . 275 6.2.1 The Estimators of Intercepts and Slopes . . . . . . . . . 276 6.2.2 Bayes and Empirical Bayes Estimators of Intercepts and Slopes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 6.3 Bias, Quadratic Bias, MSE Matrices. and Risk Expressions . . 280 6.3.1 Unrestricted Estimators of p and 8 . . . . . . . . . . . . 280 6.3.2 Restricted Estimators of p and 8 . . . . . . . . . . . . . 280 6.3.3 Preliminary Test Estimators of ,B and 8 . . . . . . . . . 281 6.3.4 James-Stein-type Estimators of p and 6 . . . . . . . . . 281 6.3.5 Positive-Rule Stein Estimators of p and 8 . . . . . . . . 282 6.4 Comparison of the Estimators of the Intercept Parameter . . . 283 6.4.1 Bias Comparison of the Estimators of the Intercept Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 6.4.2 MSE-matrix Comparisons . . . . . . . . . . . . . . . . . 284 6.4.3 Weighted Risk Comparisons of the Estimators . . . . . 289 6.5 Estimation of the Regression Parameters: Nonnormal Errors . 292 6.5.1 Unrestricted. Restricted. Preliminary Test. James-Stein and Positive-Rule Stein Estimators and Test of Hypothesis293 6.5.2 Conditions for Asymptotic Properties of the Estimators and Their Distributions . . . . . . . . . . . . . . . . . . 294 6.5.3 Asymptotic Distributions of the Estimators . . . . . . . 295 6.5.4 Expressions for ADB. ADQB. ADMSE. and ADQR of the Estimators . . . . . . . . . . . . . . . . . . . . . . . 298 6.6 Asymptotic Distributional Risk Properties . . . . . . . . . . . . 302 6.6.1 Comparison of 6, and 6 , . . . . . . . . . . . . . . . . . 302

6.6.2

..PT

Comparison of 6 ,

and On(0, ) -

A

..............

302

CONTENTS

xii 6.6.3

6.9 6.10

6.11 6.12

...

-S

..PT

. . . . . . . . . . . . . . . . 305 . -PT 6.6.5 Comparison of 8, and 8,, 8,, 8, . . . . . . . . . . 305 Asymptotic Distributional MSE-matrix Properties . . . . . . . 307 Confidence Set Estimation: Normal Case . . . . . . . . . . . . . 311 6.8.1 Confidence Sets for the Slope Parameters . . . . . . . . 311 6.8.2 Analysis of Coverage Probabilities . . . . . . . . . . . . 314 6.8.3 Confidence Sets for the Intercept Parameters when o2 isKnown . . . . . . . . . . . . . . . . . . . . . . . . . . 318 Confidence Set Estimation: Nonnormal Case . . . . . . . . . . . 319 Nonparametric Methods: R-Estimation . . . . . . . . . . . . . . 321 6.10.1 Model, Assumptions, and Linear Rank Statistics . . . . 321 6.10.2 R-Estimation and Test of Hypothesis . . . . . . . . . . . 323 6.10.3 Estimation of the Intercepts Oa and the Slope pa . . . . 323 6.10.4 Asymptotic Distribution of the R-Estimators of the 325 Slope Vector . . . . . . . . . . . . . . . . . . . . . . . . 6.10.5 Asymptotic Distributional Properties of the R-Estimators of Intercepts . . . . . . . . . . . . . . . . . . . . . . . . 330 6.10.6 Confidence Sets for Intercept and Slope Parameters . . 336 337 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337 6.6.4

6.7 6.8

-S

Comparison of 8, and 8,(8, ) . . . . . . . . . . . . . . . 304 Comparison of 8, and 8, S+

-5

7 Multiple Regression Model 339 7.1 Model. Estimation. and Tests . . . . . . . . . . . . . . . . . . . 340 7.1.1 Estimation of Regression Parameters of the Model . . . 340 7.1.2 Test of the Null Hypothesis. HP = h . . . . . . . . . . . 341 7.2 Preliminary Test and Stein-Type Estimation . . . . . . . . . . 343 7.2.1 Preliminary Test (or Quasi-empirical Bayes) Approach . 343 7.2.2 Bayes and Empirical Bayes Estimators of the Regression Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 344 7.3 Bias. Quadratic Bias. MSE. and Quadratic Risks . . . . . . . . 348 7.3.1 Bias Expressions . . . . . . . . . . . . . . . . . . . . . . 348 7.3.2 MSE Matrices and Weighted Risks of the Estimators . . 350 7.4 Risk Analysis of the Estimators . . . . . . . . . . . . . . . . . . 355 7.5 MSE-Matrix Analysis of the Estimators . . . . . . . . . . . . . 362 369 7.6 Improving the P T E . . . . . . . . . . . . . . . . . . . . . . . . . 7.7 Multiple Regression Model: Nonnormal Errors . . . . . . . . . . 370 7.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 370 7.7.2 Estimation of Regression Parameters and Test of the Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . 370 7.7.3 Preliminary Test and Stein-Type Estimation . . . . . . 371 7.8 Asymptotic Distribution of the Estimators . . . . . . . . . . . . 371 7.8.1 Asymptotic Distribution of the Estimators under Fixed Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . 372

CONTENTS

...

Xlll

7.8.2

7.9

7.10 7.11

7.12 7.13

Asymptotic Distribution of the Estimators under Local Alternatives. and ADB. ADQB. ADMSE. and ADQR . 374 7.8.3 ADQR Analysis . . . . . . . . . . . . . . . . . . . . . . 380 Confidence Set Estimation . . . . . . . . . . . . . . . . . . . . . 383 7.9.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 383 7.9.2 Confidence Sets and the Coverage Probabilities . . . . . 385 7.9.3 Analysis of the Coverage Probabilities . . . . . . . . . . 387 Asymptotic Theory of Confidence Sets . . . . . . . . . . . . . . 391 7.10.1 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . 391 7.10.2 Asymptotic Properties of Confidence Sets . . . . . . . . 392 Nonparametric Methods: R-Estimation . . . . . . . . . . . . . . 394 7.11.1 Linear Rank Statistics. R-Estimators and Confidence Sets394 7.11.2 Asymptotic Distributional Properties of the R-estimators396 7.11.3 Asymptotic Properties of the Recentered Confidence Sets Based on R-Estimators . . . . . . . . . . . . . . . . 399 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

8 Regression Model: Stochastic Subspace 403 8.1 The Model. Estimation. and Test of Hypothesis . . . . . . . . . 404 8.1.1 The Model Formulation . . . . . . . . . . . . . . . . . . 404 8.1.2 Mixed Model Estimation . . . . . . . . . . . . . . . . . 405 8.1.3 Test of Hypothesis . . . . . . . . . . . . . . . . . . . . . 406 8.1.4 Preliminary Test and Stein-type Mixed Estimators . . . 407 8.2 Bias. MSE. and Risks . . . . . . . . . . . . . . . . . . . . . . . 408 8.2.1 Bias and Quadratic Bias Expressions . . . . . . . . . . . 408 8.2.2 MSE Matrix and Risk Expressions . . . . . . . . . . . . 409 8.2.3 hlSE Matrix Comparisons of the Estimators . . . . . . . 411 8.2.4 Risk Comparisons of the Estimations . . . . . . . . . . . 415 8.3 Estimation with Prior Information . . . . . . . . . . . . . . . . 418 8.3.1 Estimation of PI and Test of H& = HIP, . . . . . . . 419 8.3.2 The Mixed Estimators . . . . . . . . . . . . . . . . . . . 420 8.3.3 Bias Expressions . . . . . . . . . . . . . . . . . . . . . . 421 8.3.4 MSE Matrix and Risk Expressions . . . . . . . . . . . . 421 8.4 Stochastic Subspace Hypothesis: Nonnormal Errors . . . . . . . 422 8.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 422 8.4.2 Estimation of the Parameters and Test of Hypothesis . 423 8.4.3 Preliminary Test and Stein-type Estimators . . . . . . . 424 8.5 Asymptotic Distribution of the Estimators . . . . . . . . . . . . 424 8.5.1 Asymptotic Distribution of the Estimators under Fixed Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . 425 8.5.2 Asymptotic Distribution of the Estimators under Local Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . 427 8.6 Confidence Set Estimation: Stochastic Hypothesis . . . . . . . . 429 8.7 R-Estimation: Stochastic Hypothesis . . . . . . . . . . . . . . . 430 8.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

CONTENTS

xiv 8.9

Problems

..............................

436

9 Ridge Regression 439 9.1 Ridge Regression Estimators . . . . . . . . . . . . . . . . . . . 441 9.1.1 Ridge Regression with Normal Errors . . . . . . . . . . 441 9.1.2 Nonparametric Ridge Regression Estimators . . . . . . . 442 9.2 Ridge R.egression as Bayesian Regression Estimators . . . . . . 443 9.3 Bias Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . 444 . PT 9.3.1 Bias Vector of p,, (k) . . . . . . . . . . . . . . . . . . . 444 9.4 Covariance, MSE Matrix. and Risk Functions . . . . . . . . . . 446 9.5 Performance of Estimators . . . . . . . . . . . . . . . . . . . . . 450 9.6 Estimation of the Ridge Parameter . . . . . . . . . . . . . . . . 461 9.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464 9.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464 10 Regression Models with Autocorrelated Errors 469 10.1 Simple Linear Model with Autocorrelated Errors . . . . . . . . 470 10.1.1 Estimation of the Intercept and Slope Parameters when p is Known . . . . . . . . . . . . . . . . . . . . . . . . . 470 10.1.2 Preliminary Test and S-Estimation of ,5 and 6 . . . . . 472 10.1.3 Estimation of the Intercept and Slope Parameters When Autocorrelation Is Unknown . . . . . . . . . . . . . . . 474 10.2 Multiple Regression Model with Autocorrelation . . . . . . . . 478 10.2.1 Estimation of p and Test of Hypothesis of HP = h . . . 479 10.2.2 Preliminary Test, James-Stein and Positive-Rule SteinType Estimators of p . . . . . . . . . . . . . . . . . . . 479 10.3 Bias, MSE Matrices, and the Risk of Estimators When p Is Known . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480 10.4 ADB, ADMSE, and ADQR of the Estimators ( p Unknown) . . 483 10.5 Estimation of Regression Parameters When p Is Near Zero . . . 485 10.5.1 Preliminary Test and Stein-Type Estimators (Chen and Saleh, 1993) . . . . . . . . . . . . . . . . . . . . . . . . . 485 10.5.2 Design of Monte Carlo Experiment . . . . . . . . . . . . 487 10.5.3 Empirical Results and Conclusions . . . . . . . . . . . . 487 10.6 Estimation of Parameters of an Autoregressive Gaussian Process493 10.6.1 Estimation and Test of Hypothesis . . . . . . . . . . . . 494 10.6.2 Asymptotic Theory of the Estimators and the TestStatistics . . . . . . . . . . . . . . . . . . . . . . . . . . 495 10.6.3 ADB, ADMSE Matrices, and ADQR of the Estimators 497 10.7 R-Estimation of the Parameters of the AR[p]-Models . . . . . . 498 10.7.1 R-Estimation of the Parameters of the A R b ] Model . . 499 10.7.2 Tests of Hypothesis and Improved R-Estimators of 6 . . 500 10.7.3 Asymptotic Bias, MSE Matrix, and Risks of the REstimators . . . . . . . . . . . . . . . . . . . . . . . . . 501 10.8 R-Estimation of the Parameters with AR[1] Errors . . . . . . . 503 10.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505

CONTENTS 10.10Problems

xv

..............................

505

11 Multivariate Models 509 11.1 Point and Set Estimation of the Mean Vector of an MND . . . 510 11.1.1 Model, Estimation. and Test of Hypothesis . . . . . . . 510 11.1.2 Bias. QB. MSE Matrix. and Weighted Risk Expressions of the Estimators . . . . . . . . . . . . . . . . . . . . . . 512 11.1.3 Risk and MSE Analysis of the Estimators . . . . . . . . 513 11.2 U-statistics Approach to Estimation . . . . . . . . . . . . . . . 516 11.2.1 Asymptotic Properties of Point and Set Estimation under Fixed Alternatives . . . . . . . . . . . . . . . . . . . 518 11.2.2 Asymptotic Properties of the Point and Set Estimation under Local Alternatives . . . . . . . . . . . . . . . . . . 519 11.3 Nonparametric Methods: R-estimation . . . . . . . . . . . . . . 522 11.3.1 Asymptotic Properties of the Point Estimators . . . . . 524 11.3.2 Asymptotic Properties Confidence Sets . . . . . . . . . 528 11.4 Simple Multivariate Linear Regression Model . . . . . . . . . . 530 11.4.1 Model, Estimation and Tests . . . . . . . . . . . . . . . 530 11.4.2 Preliminary Test and Stein-Type Estimators . . . . . . 531 11.4.3 Bias. Quadratic Bias, MSE Matrices, and Risk Expressions of the Estimators . . . . . . . . . . . . . . . . . . . 532 11.4.4 Two-Sample Problem and Estimation of the Means . . . 535 11.4.5 Confidence Sets for the Slope and Intercept Parameters 538 11.5 R-estimation and Confidence Sets for Simple Multivariate Model539 11.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 539 11.5.2 Asymptotic Properties of the R-estimators . . . . . . . 541 11.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 11.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 12 Discrete Data Models 549 12.1 Product of Bernoulli Models . . . . . . . . . . . . . . . . . . . . 550 12.1.1 Model. Estimation. and Test . . . . . . . . . . . . . . . 550 12.1.2 Bayes and Empirical Bayes Estimation . . . . . . . . . . 552 12.1.3 Asymptotic Theory of the Estimators and the Test of Departure . . . . . . . . . . . . . . . . . . . . . . . . . . 554 12.1.4 ADB. ADQB. ADMSE. and ADQR . of Estimators . . . 558 12.1.5 Analysis of the Properties of Estimators . . . . . . . . . 559 12.1.6 Baseball Data Analysis . . . . . . . . . . . . . . . . . . 564 12.1.7 Asymptotic Properties of Confidence Sets . . . . . . . . 567 12.2 Product Binomial Distributions . . . . . . . . . . . . . . . . . . 569 12.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 569 12.2.2 Model. Estimation. and Test of Hypothesis . . . . . . . 569 12.2.3 Asymptotic Theory of the Estimators and the TestStatistics . . . . . . . . . . . . . . . . . . . . . . . . . . 572 12.2.4 ADB. ADQB. ADMSE, and ADQR of the Estimators . 574

CONTENTS

xvi

12.2.5 Estimation of Odds Ratio under Uncertain Zero Partial Association . . . . . . . . . . . . . . . . . . . . . . . . . 579 12.2.6 Odds Ratios: Application to Meta-analysis of Clinical Trials . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580 12.3 Product of Multinomial Models . . . . . . . . . . . . . . . . . . 584 12.3.1 The Product of Multinomial Models . . . . . . . . . . . 584 12.3.2 Estimation of the Parameters . . . . . . . . . . . . . . . 585 12.3.3 Test of Independence in an T x c Contingency Table . . 585 12.3.4 Preliminary Test and Stein-Type Estimators of the Cell Probabilities . . . . . . . . . . . . . . . . . . . . . . . . 586 12.3.5 Bayes and Empirical Bayes Method . . . . . . . . . . . 586 12.3.6 Asymptotic Properties . . . . . . . . . . . . . . . . . . . 589 12.3.7 Asymptotic Properties of the Estimators under Local Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . 594 12.3.8 Analysis of the Asymptotic Properties of the Estimators 597 12.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599 12.5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599

References

60 1

Glossary

613

Authors Index

615

Subject Index

621

List of Figures 3.1 Display of predicted batting averages based on Stein’s formula . 3.2.1 Graph of quadratic bias functions of the estimators . . . . . 3.2.2 Graph of MRE(8,; 8,) and MRE(eET;&) .......... 3.3.1 Graph of MRE ( f i ~bl) ; and MRE(firT;i i 1 ) . . . . . . . . . . 3.5.1 Graph of the relative efficiency of SE and PTE for different valuesofa . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Graph of AMRE of 6zT and 8: relative to 6, . . . . . . . . . 3.9.1 Graph of [ ~ ~ S ( L , ) ] ~ { E ( X ~ 2q5s(L,)E(x2ILn) IL,)} 1 . .

+

I)

3

. 61 . 63 . 73 . 81 . 95

. 108

..PT

APT l / p

4.2.1 Graphs of plMz (6, and R2 ( 6 , ; Ip) . . . . . . . . . . . 130 4.3.1 Geometrical representation of Stein’s idea . . . . . . . . . . . . 137 -S 4.3.2 Graphs of & ( @ ; I p ) and p[M3(6,)I1/p . . , . . . . . . . . . . 141

-s -

-s -

4.3.3 Graphs of MRE = MRE(6,; 6,) and RRE =RRE(6,; 6,)

..PT

-S

. . 143

4.3.4 Graph of QB of estimators: P T E = 6 , , AS+ PRSE = 6 , .......................... 4.3.5 Graph of the Risks of the Estimators . . . . . . . . . . . . . JSE

=

6 , and

. . 145 . . 147 PT+ ..PT 4.3.6 Graph of Rg (6, ; Ip) and Rz ( 6 , ;Ip) . . . . . . . . . . . . . 149 AS+. APT+4.3.7 Graphs of R4(6, ,Ip)and Rg(6, ,Ip) . . . . . . . . . . . . . 151 4.4.1 Empirical Bayes Tree . . . . . . . . . . . . . . . . . . . . . . 154 4.5.1 Graph of QB of Estimators: PTE, JSE and PRSE . . . . . . 164 PT S+PT+. 4.5.2 Graph of ; o-’I,), , a-’I,), and Rg(8, , G - ~ I , )170 -IS

S+

-S

4.5.3 Graph of Rs(6, ; O - ~ I , ) R4(6, , ; O-~I,), and R@,; u - ~ I , ) 173 ~

12.2.la Predicted odds ratios . . . . . . . . . . . . . . . . . 12.2.lb Confidence Intervals of odds ratios . . . . . . . . . . 12.2.2a Predicted odds ratios (Deleting Fallis) . . . . . . . . 12.2.2b Confidence Intervals of odds ratios (Deleting Fallis)

xvii

. . . . . 581 . . . . . 581 . . . . . 581 . . . . . 581


List of Tables 1.1.1 Batting averages of 18 players

...................

3.2.1 Maximum and Minimum Guaranteed Efficiencies for n = 8 . . 3.2.2 Maximum and Minimum Guaranteed Efficiencies for n = 12 and Z2/Q = 0.1(0.2)0.9. . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Maximum and Minimum Guaranteed Efficiencies . . . . . . . . 3.3.2 Maximum and Minimum Guaranteed Efficiencies . . . . . . . . 3.3.3 Maximum and Minimum Guaranteed Efficiencies . . . . . . . . 3.4.1 Minimum and Maximum Efficiency of P T E . . . . . . . . . . . 3.5.1 Maximum and Minimum Efficiencies of SE and Efficiency of P T E at A, for Selected a . . . . . . . . . . . . . . . . . . . . . 3.5.2 Minimum and Maximum Relative Efficiency of SE and P T E for n = 8, a = .05(.10).45 and = 1(0.5)5 . . . . . . . . . . . 3.5.3 Minimum and Maximum Relative Efficiency of SE and PTE for a = 0.05(0.10)0.45 and for Selected Samples . . . . . . . . 3.6.1 Maximum and Minimum Guaranteed Asymptotic Efficiencies ofPTE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6.2 Maximum and Minimum Guaranteed Asymptotic Efficiencies ofPTE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

$

3 65 66 70 71 72 76 81

85 88 97 98

4.2.1 Maximum and Minimum Guaranteed MSE Based Efficiencies . 134 4.2.2 Maximum and Minimum Guaranteed risk Based Efficiencies . . 136 4.3.1 Risk Gain of PRSE over JSE . . . . . . . . . . . . . . . . . . . 148 4.8.1 Decomposition of the Coverage Probability . . . . . . . . . . . 191 4.8.2 Some Upper Bounds of c for y = .10 and .05 . . . . . . 192 4.8.3 Coverage Probabilities for the Set CPT(6ET(a))with y = .10 and 0 = .05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194 ..PT 4.8.4 Coverage Probabilities for the Set CPT(6, ( a ) )with y = 0.10 and a = 0.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 4.10.1 Properties of Estimators . . . . . . . . . . . . . . . . . . . . . 205 5.5.1 Maximum and Minimum Guaranteed Efficiencies

. . . . . . . . 236

10.4.1 Empirical Risks for Different Estimators Prior t o Testing . . . 488 xix

xx

LIST OF TABLES 10.4.2 Empirical Risks for Different Estimators Prior t o Testing-Shrinkage Estimates . . . . . . . . . . . . . . . . . . 488 10.4.3 Empirical Risk Values for P T E Based on D-W and statistic, a = 0.01. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489 10.4.4 Empirical Risk Values for Shrinkage P T E Based on D-W and Statistic, a = 0.01 . . . . . . . . . . . . . . . . . . . . . . . . 489 10.4.5 Empirical Risk Values for P T E Based on D-W and G1 Statistic, Q = 0.05. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490 10.4.6 Empirical Risk Values for Shrinkage P T E Based on D-W and Statistic, a = 0.05 . . . . . . . . . . . . . . . . . . . . . . . . 490 10.4.7 Empirical Risks for Different Estimators prior t o Testing . . . 491 10.4.8 Empirical Risks for Different Estimators prior to Testing491 Shrinkage Estimates . . . . . . . . . . . . . . . . . . . . . . . . 10.4.9 Empirical Risk Values for P T E Based on D-W and Statistic, cy = 0.01. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491 10.4.10 Empirical Risk Values for Shrinkage P T E Based on D-W and Statistic, Q = 0.01 . . . . . . . . . . . . . . . . . . . . . 492 10.4.11 Empirical Risk Values for PTE Based on D-W and Statistic, a = 0.05. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 10.4.12 Empirical Risk Values for Shrinkage P T E Based on D-W and 41 Statistic, Q = 0.05 . . . . . . . . . . . . . . . . . . . . . 493 12.1.1 Maximum Relative Efficiencies of the RMLE, PTE, and SE and the Intersection Efficiencies for the P T E and SE for each a with Corresponding A,-Values for pValues for a = 0.05(0.05)0.25 and p = 4(2)16 . . . . . . . . . . . . . . . . 564 12.1.2 True Value (13:) and Estimated Values of Bi Based on Efron-Morris (EM), Empirical Bayes (EB) and Ali and Saleh Estimators, 6rT,d:, and 6;’ . . . . . . . . . . . . . . . . . . . 566 12.1.3 Estimated Average Loss for the Estimators. . . . . . . . . . . 566 12.2.1 Incidence of pre-eclampsia in nine randomized trials . . . . . . 581 12.2.2 Various Estimators of Odd-Ratios . . . . . . . . . . . . . . . . 581 12.2.3 Revised Estimators of ORs after Deleting O R “Fallis” . . . . 584

Preface The estimation of parameters of a model with “uncertain prior information” on the parameters of interest began with Bancroft (1944, Annals of Mathematical Statistics 15: 19g204). Bancroft introduced the “preliminary test estimation” on the classical front. although Bayesian methods already existed. But the real breakthrough came when Stein (1956, Proceedings of the Third Berkeley Symposium 1, pp. 197-206) and James and Stein (1961, Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability 2, pp. 361-379) proved that the sample mean in a multivariate normal model is not admissible, under a quadratic loss, for dimensions more than two. This discovery, known as the Stein paradox, basically undermined the criteria, like unbiasedness, equivariance, and the MLE properties, that were the backbone of statistical estimation theory. Stein’s theory does not only improve the point estimators of location parameters (mean, regression parameters, etc.) but also improves the traditional confidence sets by sets recentered a t the Stein estimators. Preliminary test estimation of locations generally performs poorly and is not uniformly better than the MLEs/LSEs, but Stein (1964, Annals of the Institute of Statistical Mathematics 16, pp. 155-160) showed that, in the case of variance estimation, preliminary test estimators are uniformly better than the usual variance estimators. In addition, the preliminary test approach improves upon the standard James-Stein estimators and leads to admissible (generalized Bayes) estimators. Due to the immense impact of Stein’s approach on estimation, scores of technical papers appeared in the literature in various areas of applications. The book The Statistical Implications of Pre-test and Stein-Rule Estimations in Econometrics by Judge and Bock (1978), was the first attempt directed toward the econometrics area of applications. Recently, another book, Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estimators by Marvin H. J . Gruber (1998) has directed the estimation toward student/teacher as descriptive applied research material. So far, there has been no book of statistics on the topic of preliminary test and Stein-type estimation. This led me to develop such a book that would be beneficial to graduate students, researchers, and teachers alike. The mathematics and statistics prerequisites for this book are modest. It is assumed that the reader has had a basic course in mathematical statistics, xxi

xxii

PREFACE

preferably having used textbooks such as An Introduction to Probability Theory and Mathematical Statistics by Rohatgi and Saleh (2001, Wiley), Statistical Inference by Casella and Berger (1990, Brooks/Cole), Mathematical Statistics: basic ideas and selected topics by Bickel and Doksum (2001, Prentice Hall) or Introduction to Mathematical Statistics by Hogg, McKean and Craig (2005, Prentice Hall). The aim of the book is to provide a clear and balanced introduction to preliminary test and Stein-type estimation theory for students/teachers of statistics programs. We start with the two-sample problem of pooling means in a general setup, in order to help the readers understand the results and calculations that are being used in every step of the development for the solution to the problem. Then, from chapter to chapter, we raise the level of discussion. The book consists of twelve chapters. Chapter 1 gives the introduction to the preliminary test and Stein-type estimation, with details on each model, including simple linear regression, the ANOVA model, the parallelism model, the multiple regression model, and the multivariate simple linear model. Chapter 2 contains some basic results used in the book. Chapter 3 covers the introduction to the preliminary test estimation, with applications to the one-sample problems, two-sample problems, and simple linear models. Also included is Stein’s variance estimation. Chapter 4 gives an introduction to Stein-type estimation. In addition, the chapter shows that the Stein-type estimation of the mean vector is a n empirical Rayes estimator, and the Sclove, Morris, and R.adhkrishnan (1972, Annals of Mathematical Statistics 43: 1481-1490) modification is carried out to obtain the positive-rule Stein estimator. Some asymptotic results are given for nonnormal situations. To complete the chapter, recentered confidence sets are studied for the mean vector. Chapter 5 contains similar details of the ANOVA model, and Chapter 6, deals with the parallelism models. Chapter 7 contains the discussion of the multiple regression model with subset restrictions, while Chapter 8 is concerned with the multiple regression model with stochastic subset restrictions. The topic of ridge regression introduced by Horel and Kennard (1970, Technometrics 12: 55-67) is presented in Chapter 9. Chapter 10 contains the multiple regression models with autocorrelated errors. Finally, in Chapters 11 and 12 we discuss the one-sample to two-sample problems and simple linear models in a multivariate setup and some discrete data models. The contents of the book may be covered in one semester. Various problems are included to enhance the knowledge of application of the theory. I am grateful to my wife, Shahidara Saleh, for her support over our 54 years of marriage. While I remained a student all my life, she bore more than her share of chores and provided me with words of encouragement. My granddaughters, Jasmine and Sarah, watched me work on the manuscript, prepared tea a t frequent intervals, and as a result, Sarah became interested in mathematics and statistics. Thanks are due to Professor E.O. Kreyszig, who read the manuscript thoroughly, with great interest, and edited the early versions page by page to

PREFACE

xxiii

bring it t o this final form. I am also grateful to Professor Vijay K. Rohatgi for checking some portions of the book for clarity. I wish to thank Gillian Murray for diligently w i n g the manuscript; without her help, the book could not have been completed. Also, special thanks are due to H. M. Kim, who read the typed version and produced all the graphs and tables in the book, as well as to Ann Woodside, Drs. Bashir Ullah, M. Oulde Haye, Patrick Farrell, Shahjahan Khan, and B.M. Golam Kibria, who read various chapters and made many good suggestions to improve readability. My thanks are due to Wiley & Sons, Inc. and to authors for the copyright permissions on several books which helped me to finish the project. Finally, I am grateful to NSERC for supporting my research for the last three decades, the outcome of which is, besides extensive publications, twelve Ph.D. theses, several postdoctorals and MSc. students, and this book. A. K. Md. Ehsanes Saleh


Chapter 1

Introduction Outline 1.1 Objective of This Book 1.2 Statistical Decision Principle 1.3 Quadratic Loss Function 1.4 Some Statistical Models with Preliminaries 1.5 Organization of the Book 1.6 Conclusions 1.7 Problems

In problems of statistical inference, the use of prior information on some or all of the parameters in a statistical model usually leads to improved inference procedures. For some or all parameters of the model, the prior information may be known or may be uncertain. The known prior information is generally incorporated in the model in the form of a constraint, giving rise to a restricted model. The analysis of such restricted models leads to an improved statistical procedure when such constraints hold compared to the Unrestricted model. The estimators (or tests) resulting from a restricted (unrestricted) model are the restricted (unrestricted) estimators (or tests) of the parameters of the model. The validity and efficiency of restricted model analysis retains its properties over the restricted parameter space induced by the constraints, while the same holds for the unrestricted model analysis over the entire parameter space. Therefore, the results of an analysis of the restricted/unrestricted model need to weigh the loss of efficiency against the validity of the constraints in order to choose between two extreme inference techniques. Choosing restricted estimation/tests may be justified when full confidence may be placed in the prior information. When we encounter statistical models with “uncertain prior information” in the form of a constraint on the parameters, we may consider the “uncertain constraints” as a “nuisance parameter” in the statistical analysis of the full model. The preferred procedure to eliminate this uncertainty of the prior 1

2

Chapter 1. Introduction

information in a model is to use the Fisher’s recipe. This procedure calls for elimination of the nuisance parameter by a so-called preliminary test. The restricted or unrestricted model is chosen based on the validity of the uncertain prior information. The result is a compromise inference procedure between the two extremes. Bancroft (1944, 1964, 1965) was first to implement the idea of preliminary test estimation (PTE) in an ANOVA setup to analyze the effect of preliminary test (PT) on the estimation of variance. The idea was borrowed from a suggestion in Snedecor’s (1938) book on testing differences between two means after testing the equality of variances that are unknown. If the test accepts the equality of two variances, then the usual t-test is to be used with pooled estimate of variance; otherwise, it falls in the category of Behrens’-Fisher problem. As it became clear, the preliminary test estimation procedure depends heavily on the level of significance and yields only two extreme choices as an estimator. The natural question is what would be the optimum size of the level of significance in a preliminary test estimation procedure. Later Nan and Bancroft (1968) provided a maximin procedure t o determine the size of a preliminary test for the estimation of a mean in a two-sample problem to obtain an optimum “preliminary test estimator” of the mean. Mosteller (1948) looked at the problem from a Bayesian point of view and almost gave the idea of empirical Bayes estimation. Kitagawa (1963) followed the idea of preliminary test on successive occasions and obtained the distribution and moments of the preliminary test estimators in the two-sample problem. Bozivich, Bancroft, and Hartley (1956) studied the properties of the size of the preliminary test as well as the power of the test followed by a preliminary test. Bennett (1952, 1956), Huntsberger (1955), and Asano and Sato (1962) followed the application of preliminary test approach in various directions. Cohen (1965) proved that preliminary test estimators are inadmissible with no alternative suggestion for a superior estimator. Charles Stein (1955, 1956; see also 1981) followed by James and Stein (1961) discovered a paradoxical statistic which undermined a century and a half of work on estimation theory, going back t o Gauss, Legendre, Fisher, and Rao. The result was that the sample mean vector is inadmissible under a quadratic loss for the mean vector of a pdimensional multivariate normal distribution for p 2 3. After 25 years of resistance to Stein’s ideas, the paradox has diminished and Stein’s ideas are being incorporated into many applied and theoretical statistics. Not only that, Efron (1995) in his article “The Statistical Century” lists “empirical Bayes and James-Stein estimations” as important future topics among a dozen of his favorite topics in applied statistics. In traditional statistical methodology, usually the sample mean is appropriate t o estimate the population mean, as it can be proved that no other estimation rule is uniformly better than the sample mean. The paradoxical element of Stein’s work is that it contradicts this basic law of statistical theory. This assertion is shown by the baseball data analysis of Efron and Morris (1973, 1975). For example, if we have three or more baseball players and we are interested in predicting future batting averages for each of the


3

players, then the statistician who uses Stein’s method can expect to predict the future batting more accurately than the predicted values based on the separate batting average of the players. Stein’s method can be illustrated based on the famous baseball data of Efron and Morris (1975). Table 1.1 shows the first 45 batting averages for 18 players. Table 1.1.1Batting Averages of 18 Players

after 45

average Source: Adapted from table 1 of Efron and Morris ( 975) The first step in applying Stein’s method is to compute the grand mean of all these averages, which is 6 = 0.2654 in this case. The idea is to shrink the 18 averages toward the grand mean, which is a reasonable quantity under consideration. The next step is to determine the weighted squared distance between the 18 averages in the table and 18 grand means values, which becomes 19.045, and calculate the adjusting factor c = [l - 15/19.045] = 0.212, which is called the shrinkage constant. Here 15 equals the degrees of freedom (d.f.) of the weighted distance minus two. Then the formula for the ith player’s predicted average is obtained by the James-Stein’s formula for the ith predicted average 6;’ of the ith player, namely, 6,’ = 6 c(& - 6), i = 1,2,. . . ,18. This means that the first player has the predicted average

+

87 = 0.265 + 0.212(0.400 - 0.265) = 0.294, and the 18th player has the average

if8= 0.265 + 0.212(0.156 - 0.265) = 0.243. Actual batting averages were 0.346 and 0.200, respectively. The preliminary test estimator in this case is 0.265 a t the 0.15 level of significance. The usual sample mean vector has the following properties: (I) best unbiased estimator, (2) best equivariant estimator, (3) maximum likelihood estimator, and (4) minimax estimator. Yet the performance is inferior in terms of the expected squared loss relative to the James-Stein estimator. Observe from Figure 1.1 that the predicted batting averages are more concentrated around the grand mean compared to the original batting averages. Stein’s method pulls the original estimates toward the grand mean. Berger (1980a, 1985) has a detailed account of some of these developments, mostly related to the classical multinormal and specific types of exponential


4

0 5

--0.356 -3 0.333 -- 4 0.311

0 265.

__

5 8 6

-- 8 -- 9 8 10 0.222 -- 11 l o 15 0 2 -- 16

1

1

0 257

0 244

0.178

--

I

_.

0 294 1 0.289 ~- 2 0 280 _- 4 0.275 -- 5 8 6 0 270 -- 7 0.266 -- 8 0 261 -- 9 8 10 0 256 -- 11 lo 15 0 252 -- 16 0.245 -- 3 0 . 2 4 3 -_ 18 0.241 17

__

17

0.156 -- 18

0 Batting Average

Predicted Batting Average

Figure 1.1 Display of predicted batting averages based on Stein’s formula


5

families of distributions. For discrete multivariate Stein-type estimators we refer the reader to Ghosh, Hwang, and Tsui (1983). Estimators that uniformly improve over standard (least squares or maximum likelihood estimators among others) estimators are usually termed Steintype estimators in honour of Charles Stein. Basically, Stein-type estimators may be obtained via a decision-theoretic approach (risk-difference approach) due to Stein or the empirical Bayes approach (EBE) due t o Robbins (1955) and Efron and Morris (1972, 1973, 1977) (see also Casella, 1985). “Empirical Bayes” is a term that has many meanings, reflecting different approaches to solving problems. Efron and Morris (1973) applied this method to justify the Stein-type estimation. The general approach of empirical Bayes estimation may be depicted as

I Statistical Estimation 1

[Classical]1

-

1

1

p&zq

The empirical Bayes method sits in between the classical (Neyman-Pearson) and the Bayesian approach borrowing pieces from each. However, Saleh and Sen (1978-1986), Sen and Saleh (1979-1987), and Casella (1985) pointed out that the Stein-type estimators involve an appropriate test-statistics for testing the adequacy of an uncertain prior information on the parameter space, which is incorporated in the actual formulation of the estimator. Stein-type estimators adjust the unrestricted estimator (for the full model) by an amount of the difference between unrestricted and restricted estimators scaled by the adjusted test-statistics for the uncertain prior information. Generally, these test-statistics are the normalized distance between the unrestricted and restricted estimators and follow a noncentral chi-square or F-distribution with appropriate degrees of freedom. The risk or MSE of Stein-type estimators depends on the noncentrality parameter, which reflects the distance between the full model and the restricted model. The preliminary test estimators (PTE) are the precursors of the Stein-type estimators, and a careful look at the P T E reveals that a simple replacement of the indicator function by a multiple of the reciprocal of the test-statistics defines the Stein-type estimators. This procedure will be known in this book as the preliminary test approach (or alternatively, we may call it a quasiempirical Bayes approach) to shrinkage estimators. This procedure has the far-fetched implication that it combines the two diverse areas of robust estimation and shrinkage estimation. On this ground Saleh and Sen (19781986) took this path to broaden the improvement of standard rank estimators to shrinkage estimators. Among many benefits of Stein-type estimators is that they can be used as a predictive tool for events of interest. For example, they could be used


6

to predict batting averages of individual baseball players from a collection of many batting averages of several baseball players in a season (see Efron, 1975), or meta-analysis of several case control studies, or forecasting, based on timeseries data-fitting autoregressive models among others. A broad area known as “small area estimation” has evolved based on the Stein-type estimation for predicting in a small area where the sample size is very small. The methods described in this book provide the most useful techniques for combining data from various sources, such as in meta-analysis and data mining and in many other modern topics.

1.1

Objective of This Book

The main object of the book is to provide the readers with the mathematical and statistical foundations of the preliminary test and shrinkage estimation. This will enlarge the scope for use in applied areas. Basically we will present analytical properties of the five estimators: unrestricted estimator (UE), restricted estimator (RE), preliminary test estimator (PTE), Stein-type shrinkage estimator (SE), and positive-rule shrinkage estimator (PRSE) to show how they can be applied to the standard models, such as linear models, parallelism models, ANOVA models, multiple regression model, and their variations, multivariate models, and discrete data models. We usually assume the error distribution in these models to be normal. We also include nonparametric methods and models with nonnormal errors and provide asymptotic results.

1.2

Statistical Decision Principle

The basic elements of a statistical decision theory are the following: (1) A random experiment represented by the triplet ( 3 ,B,Po)where X is the sample space of the experiment, B is the o-algebra generated by X, and Po is the class of probability distributions over X indexed by the parameter 8 E R, where R is the parameter space. (2) A nonempty set A of actions at the disposal of a statistician. ( 3 ) A loss function, L(8,a ) , that is a real-valued function defined on R x A. These elements are related as shown:

I

I

1.2. Statistical Decision Principle

7

If X, = x, = ( 2 1 , . . . ,2,)’ is observed, the statistician takes the action a = b(x,) E A, where 6 is a decision rule belonging t o a nonempty set V. If A = R, then the problem is one of estimation. If A = { a l ,a z } , then any decision rule 6 partitions the sample space X into two disjoint sets C and ??. If 6(x) = a1 E C or 6(x) = a2 E then action a1 or a2 is taken, respectively. This problem relates to testing of hypotheses, where a1 relates t o rejecting a null-hypothesis and a2 relates to accepting the null hypothesis. The loss incurred during the process of choosing elements of a E A is defined by L(6,a). The action a is the result based on a function 6 E V such that 6(x) = a. Then the loss L(B,b(x))is indicative of the action a E A. It is then a random variable. The average of the loss L(6,6(x)) is called the risk R ( 0 , b )= E~{L(6,6(x))}= L(6,6(x))dPe(x) associated with the use of the decision rule 6 from the class V. The fundamental problem of statistical decision theory is the selection of 6 E ’D such that the risk R(0,6)is minimum for all 6 E D.We thus need to specify some criterion to compare various decision rules.

c,

Definition 1.1. A decision rule 6* is said to dominate another decision rule 6’ if for all 8 E 52, R(8,6*) 5 R(6,6’). If, in addition, for some 6 E R, R(6,6*)< R(6,do), then 6’ strictly dominates 6’. Definition 1.2. A decision rule 6* is said to be admissible if for any other decision rule, 6’ E ’D, R(6,6*)5 R(6,6’) for all 6 E R. The criteria used to limit the class of V of decision rules are “unbiasedness” “invariance” and “sufficiency”. Another possible criterion is the LLminimaxf’ principle. Definition 1.3. A decision rule 6*(x)is minimax within the class V ,if 6‘ E V and sup R(6,6*)5 sup R(6,6) for all 6 E V . BEn BE n The Bayes principle with respect to a prior distribution r ( 6 ) is frequently used to choose a minimax decision rule. Then the Bayes risk of the decision rule 6(x) is given by

611.

P ( T , 6 ) = &“6,

Definition 1.4. A decision rule bB is called a Bayes decision rule if

For the estimation of 6 E R c R’ based on the loss function (6 - 6)2, a Bayes decision rule hB is given by

bB(.) = qelx = 4 =

I

eh(qz)d6;

8


this is called the Bayes estimator of 0, where h(6iz)is the posterior distribution of 6 given X = z. For details the readers are referred t o Rohatgi and Saleh (2001). One of many methods of determining minimax decision rules is to find a Bayes rule based on the prior ~ ( 0 )say, , b B ( z )such that the risk, R ( 6 , S B ) = constant for all 6 E a; then S B is a minimax decision rule. See Rohatgi and Saleh (2001, thm 8.8.2) in connection with the estimation problem.

1.3

Quadratic Loss Function

In this book we evaluate the estimators in a decision-theoretic setup based on quadratic loss functions among other loss functions such as (1) the absolute deviation function l6-6(z)j and (2) the Linex function [exp(ab(z))-bb(s) - 11 for analytical simplicity. Let 6 = (61,. . . , 6,)’ be a vector of decision rules based on a sample of size n and let 6 = (01,. . . ,6,)’ E RP. Then we have a weighted square loss function defined by

q e ,6) = n(iqx) - e)’w(s(x)- e) = nIlW

-

~ I I L

where W is a positive semi-definite (p.s.d.) matrix of weights. As a special case we also discuss unweighted quadratic loss

L ( 6 , 6 )= 4 1w - 42, where W = I, is the identity matrix. Then the corresponding risk function is given by

~ ( 6 ~=6nEOIP(x) ) - 6IIL = nEellG(x) - 6jj2, if W = I,.

Under the weighted quadratic loss function, the Bayes estimator is the mean of the posteriori distribution given X = z corresponding to the prior distribution r ( 0 ) .If W = I,, then we obtain an unbiased estimator with minimum variance. If we formulate the estimator as unbiased and linear, then the results coincide with the Gauss-Markoff theory having minimum risk or best linear unbiased estimator (BLUE). Another quantity we may consider is the mean square error (MSE) matrix defined by

M = n ~ 6 [ ( q x-) e)(s(x)- e)q. Note that the weighted quadratic risk function is R(8,6)= t r ( M W ) . The idea of using the criterion based on M is to find estimators whose mean square

1.4. Some Statistical Models with Preliminaries

9

error is small, while the R(B,6) minimizes the weighted sum of the mean squares. An estimator 6 is as good as 6’ if

R(t’8,t’s) I R(t’6,i?’6’) for all 6 E RP for a given nonzero vector i? = ([I,..

. ,tP)’. In other words, we must have

t’ { ~ E O [ ( ~- ~~)(S*(X) ( Z ) - O)’] - ~ E O [ ( ~-( 6)(8(~) X) - B ) ’ ] } f? 2 0 for all t # 0 and for all 6 E RP for the mean square error.

1.4 Some Statistical Models with Preliminaries In this section we consider some basic statistical models that are frequently used in applied statistical methodology, with preliminary information with regard to estimation and testing of hypotheses. These will be used t o discuss the preliminary test and shrinkage estimation in later chapters. Some nonparametric methods are also given with preliminaries in Chapter 2.

1.4.1

Mean and Simple Linear Models

Consider the simple model

Y

= 61,

+ El

(1.4.1)

where Y = (Yl,.. . ,Y,)’ is the observation vector of n components, 1, = (1, ... , 1)’ is a vector of an n-tuple of one’s, 6 is a constant, and E = (€1, . . . ,E ~ ) ’is a vector of n independent components distributed as N,(O, u21n), I, is the identity matrix of order n. Using the sample information and the specification (1.4.1), we obtain the unrestricted estimator (UE) of 6 by minimizing the quadratic form

(Y - Ol,)’(Y

- 61,)

(1.4.2)

as

6, = (l;ln)-ll;Y

=

Y,

(1.4.3)

where ‘L is the sample mean, distributed as N(6,u 2 / n ) .Further, the unbiased estimator of o2 is given by s2 =

-(Y - f?,l,)’(Y n-1

- 6,1,).

(1.4.4)


10 In order to test the null hypothesis the test-statistic

L, =

7218,

No : 8 = 80 against H A : 8 #

- 8012 u2

80, we use

if o2 is known

(1.4.5) The exact distribution of C, under H A (r2known) follows a noncentral chisquare distribution with one degree of freedom (d.f.) and noncentrality parameter A2/2, where (1.4.6) while L , (a2unknown) follows a noncentral F-distribution with ( 1 , ~1) - d.f. having the same noncentrality parameter (1.4.6). If 6 = 80, the distributions above reduce t o the central chi-square and the F-distribution, respectively. Next, consider the simple linear model

Y=Bl,fpx+€,

(1.4.7)

. . . ,Y,)’and x = (q, . . . ,2,)’ are the observation vector and where Y = (Yl, fixed vector of known constants, while E = ( € 1 , . . . ,E,)’ is the error vector of the model distributed as Nn(0,021,). Based on the LS/ML principle, the estimators of (8,p)’ is given by ( 1.4.8) where

1 Q = X’X - - ( 1 ’ , ~ ) ~ . n

( 1.4 9)

The exact distribution of (en,,&)’ is a bivariate normal distribution with mean vector (8,p)’ and covariance matrix

(1.4.10) In order to test the null hypothesis Ho : p = ,& against H A : p use the test-statistic

L, =

IPn - Pol2&

s:

1

#

Po, we

(1.4.11)

where (1.4.12)


11

The exact distribution of L, under H A follows a noncentral F-distribution with (1,n - 2) d.f. and noncentrality parameter A2/2,

( 1.4.13) Under H o , it follows a central F-distribution. Similarly, in order to test the null hypothesis HO : 6 = 60 against H A : 8 # 60, we use the test-statistic

(1.4.14) which follows a noncentral F-distribution with (1,n - 2) d.f. and noncentrality parameter A2/2, where

(1.4.15) Under Ho, it follows a central F-distribution. Further, in order to test the null hypothesis HO : 8 = 60, 0 = PO against HA : 8 # 80, ,d # 00,we use the test-statistic

C,

= ~,’(6, - 6 0 ,

,Bn - 00)’

-

nx n?f

Q+n?f2

)(

1:

)

(1.4.16)

whose exact distribution is a noncentral F-distribution with (2, n - 2) d.f. and noncentrality parameter A2/2, where

Under Ho, C, follows a central F-distribution. Finally, let x = (0,. . . , 0 , 1,. . . , 1)’ in the linear model where there are n1 zero’s and 722 1’s in the vector. Then we obtain the two-sample problem of estimating means and testing difference between two means. Here p1 = 8,

p2 = 8 + P ,

and p2

--PI =

0.

(1.4.18)

n n Also Z = Thus, the mle of p1 is y1 Q = *,and - n i ( nn2 Q i+nz)’ and that of p2 is j j 2 . For testing Ho : p2 = p1 against H A : 1.12 # p ~ we , use the likelihood test -2

(1.4.19)

12


where 13, follows a noncentral F-distribution with (1,nl noncentrality parameter A2/2

+ n2 - 2) d.f. and (1.4.21)

Under Ho, it follows a central F-distribution. These models will be considered in Chapter 3 for further estimators of 6 or (0, p)’.

1.4.2 One-Sample Multivariate Model Let

Y1,Y2,.. .

, Y N be N observation vectors satisfying the model

Y , = ~ + E , a = 1 , 2 ,..., N .

(1.4.22)

Here Y, = (Yal,.. . ,Yap)‘,8 = (61,.. . ,eP)’,and E, = (€,I,. . . ,E,~)’ for E, Np(O,E);X is the covariance matrix of the error vectors E,, and the error vectors €1, € 2 , ... ,E N are mutually independent. The LSE/MLE of 8 is given by the sample mean vector Y = (71,. . . ,Yp)’ such that

-

( 1.4.23) In order to test the null hypothesis HO : 6 = 80 against H A : 8 # the likelihood ratio test-statistic

80,we

use

1 3 =~ N ( Y - 6o)’X-’(Y- 80) if X is known = NllY - &11&-1,

LN = N ( Y - & , ) ’ S - ’ ( y

- 80) if

= NllY - 6ol\&l= T2

X is unknown

(Hotelling’s T 2 ) ,

(1.4.24)

where N

s = C(Ya- Y)(Y, - Y)’. a=l

We then write

For detailed information, see Anderson (1984)’ Srivastava and Khatri (1979)’ and Rao (1973),


13

The exact distribution of C, (when C is known) is a noncentral chi-square distribution with p d.f. and (when C is unknown) it follows a noncentral Fdistribution with ( p , N - p ) d.f., having the noncentrality parameter A2/2 in both cases, where

A2 = N ( 6 - 60)’E-~(6 -60) = N1/6 - 601l5-1.

(1.4.25)

If 6 = 80, then it follows a central chi-squared and F-distribution, respectively. Now, if we assume X = u21plwe have to estimate only one parameter in the covariance matrix. Estimation of the mean vector 6 = (01,.. . ,6,)’ is considered in Chapter 4 when C = I,, and the general estimation problem when X is unknown is considered in Chapter 11.

1.4.3 ANOVA Models Let the observation vector Y be modeled as

Y=B6+&,

(1.4.26)

where

y 6 E

= (Yll,... ,Ylnl;Y21;...Yznz;... ;Ypl, ... 7 Ypn, )‘ = ( 0 1 , . . . ,6,)‘

= (€11,. * * ,Elnl;.

. . ;&PI,.* . ,E p n , ) /

B = Block diagonal matrix = Diag(l,, ,. . . , In,) Int = ( I , . . . ,1)’ a ni-tuple of ones 72

= n l + . . . + n.,

(1.4.27a) (1.4.27b) ( 1.4.27~) (1.4.27d) (1.4.27e) (1.4.27f)

The joint distribution of the components of E is &(O, a2B),and u2 is the variance of errors. Using the LS principle/MLE method, we obtain the estimator of 6 as

e,

=

(B’B)-’B’Y = N-~B’Y,

where N = B’B = Diag(n1,. . . , 72,). of u2 is then given by

en -

s; =

The corresponding unbiased estimator

-(Y - Be,)’(Y - Be,). n-p

(1.4.28)

(1.4.29)

N,(6, a2N-l) and (n - p)s2/a2 follows a central chi-square Moreover, distribution with ( n - p ) d.f. independent of 6,.


14

In order to test the null hypothesis HO : 6 = 601pagainst H A = 6 # &lP, where 90 is a constant, we use the likelihood ratio test-statistic (1.4.30) where H = ( I p - $1,1LN) and 601, = :lPlbN6, is the pooled estimator of 6 . The exact distribution of C, is a noncentral F-distribution with (p-1, n-p) d.f. and noncentrality parameter A2/2 where

A2 = (e’H’NH6)

(1.4.31)

62

If 6 = 601p,it follows the central F-distribution. This ANOVA model will be the subject of Chapter 5.

1.4.4 Parallelism Models Let Yna=6z1n*+flzxz+E,,

i=l,2,

-

... ,p,

(1.4.32)

where Y n z = ( y i i , . . . in,)', X, = (GI,. . . ,zznZ)’ and E , = (&,I,. .. , E , ~ , ) ’ . In addition, E , N ( 0 , a 2 1 , , ) , where In% is the identity matrix of order n,. Thus, we have p linear models with different intercepts and slope parameters. If = ... = p p = p (same), then we have p parallel simple linear models used in many problems of bioassay and shelf-life determinations in pharmaceutical problems. Now, consider the LSE/MLE - of the parameters {(6,,,5,)’;z= l , . .. ,p}. Let us define P = (Ti,... , Y p ) ’as the vector of means of the observations in p models, k = (Z1, . . . ,Zp)’ being the mean vector of the known z-vectors, and let = . . . ,&)’ be the LSE/MLE of the slopes in p-models. Then the intercept vector 6 = (61,. . . ,OP)’ may be estimated as 6, = ( e l , . .. ,I$)’, where

Pn (p,

8,=T,-p,Z,,

i = l , . . . ,p.

(1.4.33)

Let

T, = Diag(Z1,. . . ,Z,).

-

-

-

(1.4.34)

Then we can write the vector 6 , = (61,. . . ,6,)’ compactly as

0, = -iT - T,P,.

( 1.4.35)

If we want to test the null hypothesis HO : p = flol,, where POis a specified constant, against H A : /3 # polp (parallel vs. not parallel), we can use the LR test-statistic (1.4.36)

15

1.4. Some Statistical hdodels with Preliminaries where 1 nQ

H = I p - -1

P

l‘D;;,

nQ = X n i Q i ,

(1.4.3713)

D i . = Diag(niQ1,. . . ,npQp), 1

(1.4.37~)

niQi = X : X ~- -(Ikixi), ni

c(Yz P

s: = (n - 2p)-l

z=1

( 1.4.37a)

i=l

- &lnz - &xz)’(Yz- &lnZ - ,&xz).

(1.4.37d)

The exact distribution of L, follows a noncentral F-distribution with ( p , n - 2p) d.f. and noncentrality parameter A2/2, where

1 1 A2 = ,zP’H’D~~HP= - p ( P - Polp)’D22(P- P o l p ) .

(1.4.38)

Under Ho, L,, follows a central F-distribution with (p - 1, n - 2 p ) d.f. We will discuss this model in Chapter 6.

1.4.5 Multiple Regression Model and General Linear Hypothesis Consider the multiple regression model

Y=xp+€,

(1.4.39)

where Y is n x 1 vector, X is a n x p matrix of known constants, P = (PI,.. . ,,LIP)’ is a p x 1 vector and E is a n x 1 vector of errors distributed as N,(O, &,). In many applied problems one formulates the response variable Y as above and tries to assess the linear hypotheses of the form

(1.4.40) In this case, we are interested in estimating /3 under the constraint HP = h and to test a hypothesis concerning the constraint. Thus, the unrestricted LSE of ,f3 is given by

p, = (x’x)-’x’Y

=c-lx’y,

c = X’X.

When the restriction is applied, we obtain the estimator

bn = p , - C-lH-’(HC-’H’)-l(Hp,

(1.4.41)

p,,

- h).

(1.4.42)


16

In order t o test the null hypothesis Ho : HP = h against H A : HP # h, we use the test-statistic

c,

= q-1s;2

{ (HP, - h)’(HC-’H’)-l(HP,

- h)} ,

(1.4.43)

where s: = ( n - p)-’(Y - XP,)’(Y

-

XP,).

(1.4.44)

The exact distribution of C, is a noncentral F-distribution with (q, n - p ) d.f. and noncentrality parameter A2/2, where

A’ = (HP - h)’(HC-lH)-l(HP - h)/c2.

(1.4.45)

Under Ho, 13, follows a central F-distribution. Further, the estimation of p will be discussed in Chapter 7. There are three important variations of the basic model discussed above: (1) (X’X) is a singular/ill-conditioned matrix that prevents a reasonable inverse of (X’X). This means that there is multicollinearity among the elements of X, and X has rank ( 5 p ) or one of the characteristic roots of X’X is very small. (2) The restriction h = HP is not exact; rather, it is of the form

h = HP + V ,

v

N

Nq(6,a2S2),

(1.4.46)

where 6 # 0 and fl is a q x q matrix of known constants that reflects the precision of an earlier sample study. (3) The error vector in the model (1.4.39) has components related by a first-order stationery autoregressive process.

In case (l),the LSE of

p is obtained

by solving the normal equations

(X’X)P = X’Y.

(1.4.47)

The general solution for p is /3 = GX‘Y where G is the generalized inverse of (X’X) and the solution for p is non unique, and for the non-full rank, there exists an infinite number of solutions. Hoerl and Kennard (1970) devised a method to overcome the difficulties above. Thus, they proposed the solution called the “ridge estimator” of p, which is defined by

BHK = (X‘X + kIJ1X’Y

= (C

+ kIJlX’Y,

(1.4.48)

where k > 0 is a positive real number known as “ridge constant” for the estimator of p. The computation of the trace of

(C)-’

and

(C+kI,)-l

( 1.4.49)

shows that P

.

(1.4.50)

17


. , X p are the characteristic roots of X'X

where XI,..

tr[C+

kip]-' =

= C and

Xi

P

i=l ( X i

+ k)'

(1.4.51) '

Hence, tr[C

+ kip]-' 5 tr(C-'),

(1.4.52)

aHK

and the ridge estimator has smaller variability than /jn. There are various ways of deriving the ridge estimator of p and the basic problem becomes that of the estimation of k . The ridge estimator will be discussed in Chapter 9. As for the case (a), consider the model

Y=xp+E

(1.4.53)

-

subject to h = HP + v, where v N b ( O , a2C?),instead of assuming v to be non stochastic equaling 0. Here, C? is a known q x q invertible matrix. The covariance matrix may reflect the information from previous samples or some prior information regarding the sizes of the elements of 0. Thus, we can write (1.4.54) where (1.4.55) After suitable transformations we can write the model above as

[

Y f2t-'/2h

] [? =

C?-01/2

] [ c p+6 ] [ +

&

fi-'/'(v - 6)

]

(1.4.56)

subject t o =O

or

6=0.

(1.4.57)

It is clearly

Now, the unrestricted estimator (i.e. without model restriction) of

-

is

[ pn 3 h

[

by the LSE principle, with the unbiased estimator of o2 as s; =

-(Y'Y - y'xc-'X'Y). n-p

(1.4.59)

18


The restricted estimator of the parameters are

[

R A 6

] [ p,

- C-lH’[(HC-lH’) + O]-l(Hp, - h) h - a[(HC-’H’) + a]-l(HB, - h)

=

.

(1.4.60)

The test-statistic for testing the null hypothesis Ho : h = H,f3 is given by

cn =

(HB,

- h)’[HC-lH’

+ s2]-1Hp,

- h) 7

QS2

+

(1.4.61)

which under H A : h = HP 6, S # 0 follows a noncentral F-distribution with (9, n - p ) d.f. and noncentrality parameter A2/2, where a2

=

G’[HC-IH‘ + 0]-16

(1.4.62)

02

This model will be discussed in Chapter 10. Some background material may be obtained from Graybill (1976), Gruber (1998), Judge and Bock (1978) among others. Finally, the regression model that arises particularly in econometrics may be stated as

Y=Xp+E, where E = ( € 1 , . . . ,E,)’ and the components of stationary autoregressive process et = pet-1

+ at,

E

are related by a first-order

~ ( v t= ) 0, Var(v,2) = a : ,

(

for all t ,

7

(1.4.63)

and lpI < 1. This autoregressive specification implies that

[i

E(ee’)= D 2 R = o2

where

R-1 =

1

fn-‘

lt’

P 1

... ...

Pn-2

...

0

...

pn-1

... lfp2 -P

-p

j ,

(1.4.64)

(1.4.65)

and has n - 1 characteristic roots equaling (1 - p 2 ) - I with one root equal to one. Using the generalized LSE principle, we obtain the LSE defined by

b(R) = (XR2S1X)-1X’R-1Y,

(1.4.66)

where E[b(R)]= P and Cov[b(S2)]= o ~ ( X ’ R - ~ X ) -whenever ’ R is known. If R is unknown, then we estimate p, say, by b and plug in &3) in the expression above to obtain b(h2).Various other estimates are desired and will be obtained in Chapter 10.

19


1.4.6 Simple Multivariate Linear Model Consider the simple multivariate linear model

Y , = e + px,

+ E,,

E,

- N,(o,XI,

. . ,N ,

= 1,.

(1.4.67)

-

where Y , = (Y,l,. . . ,Y,,)’ is the observation vector, x, is a fixed known constant and E, = ( & , I , . . . ,E,,)’ N,(O, X).The LSE/MLE of 8 and p are given by SN = N PN =

T-PNT,

(1.4.68a)

(Y, - Y ) ( x , - z)

Q

,=l

( 1.4.68b)

with N

Q=

C(xa- S ) 2 .

(1.4.68~)

,=I

In order t o test the null hypothesis Ho : p = 0 against H A : p the test-statistic

CN

= Q ( & S - l f i N ) = T 2 (Hotelling’s T 2 ) ,

# 0, we use (1.4.69)

where

is the unbiased estimator of C. The exact distribution of C N is the Hotelling’s T 2 distribution. Equivalently, C, has a noncentral F-distribution with ( p , N - p ) d.f. and noncentrality parameter A2/2, where

A2 = Q(j3’X-1p).

(1.4.71)

As a special case, let us assume 2, = 0 if Q = 1,... ,N1, and 1 if Q = NI + 1,. .. ,N . Then we have the two-sample multivariate mean problem where

Also,

Nl

-

X=

Nl

+ N2

and

Q=

Nl N2 N1+ N2’

( 1.4.73)

Chapter 1 . Introduction

20

The mle of p 1 is 71 and that of pz is Yz based on the samples of sizes N1 and N2, respectively. Further, the mle of X is the pooled sample covariance

+

c N

( Y , -Yz)(Y, -Yz)’

a=Ni+l

In order to test the null hypothesis Ho : p 2 = p l against H A : pz # p l , we use the LR test-statistic as Y z - Y1) L N = N 1 N 2 (Pz- Y1)I s,-1 (Ni + Nz = T2

( 1.4.75)

(Hotelling’s 7’’)

The exact distribution is a noncentral F distribution with ( p , N1 d.f. and noncentrality parameter A 2 / 2 ,where

+ Nz - p ) (1.4.76)

Under Ho, it follows a central F-distribution. More estimation will be discussed in Chapter 11 for the regression and the two-sample problems.

1.4.7 Discrete Data Models Here we consider three selected discrete data models.

Product Bernoulli Distributions. Let { ( x i l , .. . ,xin, li = 1,.. . ,p } be a set of mutually independent Bernoulli random variables (r.v.’s) with joint distribution b

n,

(1.4.77) Let Y = ( y l , . . . , y p ) , y z = C;:, z t 3 .Then the MLE of 6 = (el,. . . ,e,)’ is given by en = (81,.. . ,jp)’, where 6z = n, ‘yi, i = 1,. . , p . If 81 = . . . = OP = 80 (unknown), then the MLE of 00 is 6on = n-l(y1 . . . y p ) . In vector notation, we then write

+ +

6, = (61, ... ,8,)’,

6, 6,

= 60,lp, = l,lbNB,,

1, = (1,. . . , l)’,

N = Diag(nl, . . . , n p ) .

(1.4.78)


21

In order to test the null hypothesis Ho : 8 = 6 0 1 p , we use the test-statistic

D, = n(6,

-

h,)'pg1(8, -

(1.4.79)

where

9;'

= [&(l - &)]-'Diag

( s l . . .3). n n

As n + m, D, approximately follows a chi-square distribution with ( p - 1) d.f. under Ho. Product Binomial Distributions. Consider k mutually independent 2 x 2

f""l

tables where each row is the outcome

...

521

n21 - 2 2 1

... n2J

- 523

52k

n2k

-52k

(1.4.80) of a binomial experiment. Then, the joint distribution of 1,.. . , k} is the product binomial given by

{(z131z23)jj =

(1.4.81) We are interested mostly in the estimation of the ((odds-ratios", 7c) = (41,.. . ,$k)' and homogeneity of the odds-ratios, meaning 7c) = $elk ($0 is a scalar), where

( 1.4.82) By invariance properties of MLE, we have the mle of II, as (1.4.83) and el, and 8z3 are the MLE of and 0z3, respectively. If 7c) = $ O l k holds, then the common estimator of $0 due to Gart (1992) is given by k

k

(1.4.84) j=1

where I;;

=

+ n I J & j ( l - &J 1

1 n2,&3(1

- &,)'

j = 1, ... ,k.

(1.4.85)

In order to test the homogeneity of the odds-ratiosl we consider the Waldtype statistic


22

where W = Diag(G1,. . . ,Gk). The asymptotic distribution of D , follows a central chi-square distribution with (k - 1) d.f. under Ho : = $olk. Product Multinomial Distributions. Consider a ( r x c)-contingency tables for two traits. Let xV stand for the cell frequency of the (z,j)-cell for z = l , . . . , T ( 2 2) and j = 1,. . . , c ( 2 a), and let xz+ = C,”=, xZJ,x + ~-

+

El=,x z J ,and n = Crzl xzJ. The probability distribution of the vector frequency n = (211 . ..xlc,x21, . . . ,xzC,.. . , z r l , . . . ,xrc)’ is given by the product multinomial distribution n!

{

-1 fi(X..d}

ITJ=1 r

c

(1.4.87)

JJ~zJ)21~,

z=1

where 6 = {0,,12 = 1,.. . ,r,

J

. . ,c} with 6 ’ l k

= 1,.

C

r

C

Oi+ =

= 1. Define

Oij

and

0+j =

C

oi,.

(1.4.88)

i=l

j=1

If there is independence structure in the table, then we have

Note that in (1.4.87) we have rc - 1 free parameters, while in (1.4.89) we deal with T + c - 2 free parameters. In order to test for the null hypothesis

Ho : 0ij = 0i+ . 0+3

for all ( i , j ) ,

we consider the following chi-square test:

(1.4.90) where 6, = (&+, . .. ,elc,.. . , & I , . . . ,BTC)’,

x-. 023. -- -2

and an=

with

I

8i,

.

.

( & + , . - . , & c , * - ., e r l , - . - ,6rc)’,

= &+ . 0 + j , 0i+ = x i + / n and

a+, = x + j / n . Also,

9 = Diag(811,. . . ,e l c , . . . ,&.I,. . . ,&).

(1.4.91)

It may be shown, following Bishop et al. (1975) and Agresti (1990), that D, closely follows the chi-square distribution with ( r - l)(c - 1) d.f. as n -+ m. The improved estimators of the parameters of the three models will be the subject of Chapter 12.

1.5. Organization of the Book

1.5

23

Organization of the Book

The book is divided into 12 chapters that cover most of the useful models in applied statistics. In Chapter 2 we present properties of the normal and chisquare distributions together with results involving the multivariate normal and discrete distributions. Matrix results and formulas for the calculations of risks of the estimators, together with the Stein formulas, are provided for ready references. Some preliminaries of nonparametric methods are also included in this chapter. Preliminary test estimation is introduced in Chapter 3 with the simple linear model. Chapter 4 involves the introduction of Stein-type shrinkage estimation in a simple multivariate model. A general development of the estimation of the mean vector when the covariance matrix is unknown is deferred until Chapter 11. The ANOVA model is discussed in Chapter 5, and Chapter 6 extends the results of the simple linear model to several linear models that may be parallel. Chapter 7 deals with the general linear model with linear hypothesis and estimation regression parameters. Chapters 8 and 9 contain extensions of Chapter 7 in two ways: (1) parameters are restricted by stochastic constraints, and (2) the design matrix may be ill-conditioned, leading to “ridge regression.” In Chapter 10 we consider the linear regression model where the errors are generated by a first-order stationary autoregressive process. Chapter 11 deals with the general problem of estimation of parameters with one-sample, twosample, and simple multivariate linear models when the error distribution is a pdimensional normal. Chapter 12 deals with three basic discrete data models, namely, product Bernoulli distributions, product binomial distributions, and the product multinomial distributions and the related estimation of the parameters and application to meta-analysis.

1.6

Conclusions

In this chapter we presented a historical perspective on the preliminary test and Stein-type estimators, illustrated by an example of the “baseball data” and the objective of the book. We also discussed the decision-theoretic approach to estimation and testing. Finally, we have presented the models that will be covered in Chapters 2 through 12 of this book.

1.7 Problems 1. (Refer to Section 1.4.1) Consider the model Y = 01, Show that

-

-

(a) en = y ,

(b) si = &(Y - &l,)’(Y - &l,)

+ E.


24

( c ) Show that the likelihood ratio test for HO : 8 = 60 versus H A : 6 # 80 is given by

L,

nl&

=

-

-

&,I2

0 2

- nI6n - 6oI2 -

if

0’

is known

if

0’

is unknown.

sE

(d) What is the distribution of Ln under Ho and under HA?

2. Consider the model (Equation 1.4.7)

Y

= 81,

+ Px +

E

E,

-

N,(O, 0’1,)

(a) Show that the LSE/MLE of (6,p) is given by i$[X’Y

Y

-pnz

-

;(lwlLY)l

with covariance matrix

(b) Show that the likelihood ratio test for HO : ,O = 00 against H A : /3 # ,f30is given by

L , = nIpn

-

02

,001~Q

if o2 is known

(c) Determine the distribution of L,.

3. Consider the two-sample problem by taking x = (0,. . . , O , 1,.. . , 1)’ in the linear model, then verify (1.4.19)-(1.4.21) and distribution of L,.

4. Let Y1,Y 2 , .. . , Y Nbe N pvector observations

Y , = e+E,, Let

y

= N-I

C,”=,Y,

a = 1,... , N ,

E,

N,(o,c).

and S = C,=I(Y,- y ) ( Y , N

-y)’.

(a) Show that

L,

=N

( y - Oo)’E-’(Y- 8,) if C is known

= N ( T - OO)’S-’(F- 00) if C is unknown

is the likelihood ratio statistic for testing Ho : 8 =

H~ : e #

eo.

60

against

25

1,7. Problems

(b) Determine the distribution of Ln under H0 and H A .

5. Consider the linear model (ANOVA)

Y = B6 + E ,

- Nn(O,a2B)

E

as defined by (1.4.26)-(1.4.27), where 6 = ($1,. . . , $ , ) l . (a) Show that the likelihood ratio statistic for testing HO : 8 = against H A : 6 # $01, is given by

1 -

13, = -$;H'NH6, Is2

-

( P - 1)s:

$01,

if rs2 is known

(8nHfNH8n)if

a2 is unknown.

(b) What is the distribution of L, under Ho as well as under HA? 6. Consider p simple linear models

-

Y n a = $,In,

+ ,&x, +

€2,

i = 1,. . . , p ,

where E , Nnt(O,0~1,~). Show that LSE/MLE of 6 = ($1,. . . ,19,)l and that /3 = (01,. .. ,&,)I are

en

= ($1, f . .

, e,),

e, = L, - b,T,,

i = 1,. . . , p ,

and

In order to test the null hypothesis Ho : ,f3 = the likelihood ratio statistic is given by

pol,,

polp against H A

: ,f3

#

Ln = (PlnH'D;iHBn) ( P - 1)s: (by (1.4.36)), where 0;; = Diag(nlQ1,. . . , n p Q p ) .

7. Consider the multiple regression model

Y = xp + E ,

E

-

Nn(0,o21,).

Find the likelihood ratio test for the null hypothesis H0 : HP = h against H A : H P # h given by (1.4.40), and determine its distribution.

26

Chapter 1. Introduction 8. Consider the multivariate simple linear model

Y, =8

+

PX,

+

E,

E,,

-

N,(O, C).

Show that the likelihood ratio statistic for testing HO : p = 0 against H A : p # 0 is given by CN =

Q(&s-~P~)

where S is defined by (1.4.70). What is the distribution of L N ? 9. (Refer to Section 1.4.7a)

(a) For- the product -Bernoulli distributions verify that the MLE of 6 i s 6 , = ( 8 1 , ... ,8,)'. (b) Show that Cov(8,) = Diag . . , e,( 1-op) I

(y,. n P

(c) If 6 = O01,, show that the MLE of O01, is

1, IbN -

eon= On, n

N = Diag(n1,. . . ,n,)

1P1; with Cov(60,) = -. n

-

) 0.,

(d) Consider the r.v. ( 6 , - 60,) = Ho : 6 = 801,

Show that under

(e) Show that

D, = n(8, - 60,) I C,-1 ( -6 , - eon), where C,' = [&( 1 - 60)]-'Diag n I / n , . .. ,n,/n), approximately follows the chi-square distribution with ( p - 1) d.f. under Ho.

(

10. (Refer to Section 1.4.7 and k (2 x 2) contingency tables) Define

(a) Show that the approximate variance of

j = 1,2,. .. ,k, as n + j = n l j nij/n+j -+ X i j (< a).

+ n2j

Gj

is

-+

co in such a way that

1.7. Problems (b) Let 40,

27

40, = w-’cf_,wjq,.

Show that the asymptotic variance of

is +:w-l.

(c) Show that

D, = G;& (:

- &,I~)’w(& - ?jIJnlk),

w = Diag(h1,. . . , ~ k )

follow approximately the chi-square distribution with ( k - 1) d.f. as n -+ 00.

11. Let (XI,. . . ,z,)’

follow the multinomial distribution

(a) Show that the MLE of 8 = (0, ,... ,&)’ is Oj = j = 1,... , k . I

?,

6,

=

(81,... , 6 k ) ’ ,

(b) n ~ o v ( 6 , )= Diag(B1,. .. , O k ) - 88’ = ~ ( 6 ) .

12. (Continuation of Problem 11) (a) Show that V &(en - 8)21Nk{O,x(O)}.

(b) D, = n(8, - ne)’Z;’(6, - no) approximately follows the chisquare distribution with k d.f. as n -+ c ~ . (c) (Refer to Section 1.4.7~)For the T x c contingency table, show that ..-I ,. D, = n(6, - 6 , ) ’ ~(8,~ - en)

where 6,, 6, and $, are given by (1.4.90)-(1.4.91). Show that D, approximately follows the chi-square distribution with (T - l)(c- 1) d.f. as n -+00 under HO : 8ij = Oi+ . O+j for all ( i , j ) .


Chapter 2

Preliminaries Outline 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10

Normal Distribution

Chi-square Distribution and Properties Some Multivariate Normal Theory Results Beta Distribution and Applications Discrete Distributions Matrix Results Large Sample Theory Nonparametric Theory: Preliminaries Conclusions Problems

In this chapter we provide summary of the properties of statistics related t o normal and chi-square distributions. Particularly, the Stein Lemma and its extension is provided for application in subsequent chapters. In addition, we provide some core results on matrix theory, multivariate discrete and continuous distributions, beta distributions, large sample results, and nonparametric theory for application in subsequent chapters.

2.1

Normal Distribution

The most important distribution in statistical theory is the normal distribution. The probability density function (pdf) of the normal distribution may be denoted as N ( p ,02), which is written as follows:

(2.1.1) where p is the mean and o2 is the variance of the random variable X . Some of the properties of a normal variable may be listed as follows:

29

Chapter 2. Preliminaries

30

If X is N ( p ,a’),then 2 = X - p / u is N(0,l ) , the standard normal variable. 0

-

If X1 and X2 are independent, such that X , N ( p i ,u;), ( i = 1,2), then X1 fX2 is distributed as a normal variable with mean p1 fpz and variance 0: u$.

+

0

If X I , . .. , X , are n independent N ( p ,u 2 ) variables, then the sample mean = :(XI . - . x,)is ~ ( p$). ,

x

+ +

If X I , .. . ,X, are independent random variables having any distribution with common mean p and common variance u z , then f i ( X - p ) / u is distributed as N ( 0 , l ) as n -+ 00. This is known as the central limit theorem (CLT). These properties are amply listed in Cramer (1946), Hogg, McKean and Craig (2005), Rohatgi and Saleh (2001), Judge and Bock (1978), and Bickel and Doksum (2001) among many others. 0

(Stein’s Indentity) If X is N(6,u’) and if p(X) is a differentiable function satisfying EIp’(X)I < 00, then

E [ Y ( X ) ( X- 611 = 02E[cp’(X)1.

(2.1.2)

(Extended Stein’s Indentity) If X = ( X I , .. . , X,)’ is Np(O,~ ’ 1 , ) and if a function p(X) = (pl(X), . . ,p,(X))’ is partially differentiable, that is, (2.1.3a) and pi(X) is a continuous function of X i for all vectors, Xlil = ( X I , . . . , X i - l , X i + l , , . . ,Xp)’ then (2.1.3b) provided that

EllP(X)1I2< Note that for p = 1, (2.1.3b) reduces to (2.1.2). For the Stein identities, see Casella and Berger (1990) and Hoffmann (1992) among others.

2.2

Chi-square Distribution and Properties

If X is N(0,l), then X 2 is distributed as a central chi-square variable with one degree of freedom denoted as x:. However, if X is N ( p , u 2 ) ,then X2 is distributed as a noncentral chi-square variable x:(A2) with one degrees of

2.2. Chi-square Distribution and Properties

31

&

freedom and noncentrality parameter = fa2.The pdf of a noncentral chi-square variable with v degrees of freedom and noncentrality parameter ;A2 may be written as

+

where hv+2,(x2;0) is the pdf of the central chi-square distribution with v 2r degrees of freedom. The pdf of a central chi-square distribution with v degrees of freedom may be written as

(2.2.1b) The mean and variance of this distribution are Y and 2v, respectively. The first four cumulants of the noncentral chi-square distribution at (2.2.la) are given by ~1 = v

+ A2,

and ~4

= 48(v

~2 =

2(v

+ 2A2),

63

+

= 8 ( ~3A2),

(2.2.2)

+ 4A').

We denote by Hy(x; A') the cdf of a noncentral chi-square distribution with v degrees of freedom, and noncentrality parameter A2/2. Some of the results involving noncentral chi-square distributions (see also Judge and Bock, 1978, Appendix B) are given below in theorem/corollary form. Note that Hv+2(x;0) = - h v + 2 ( z ; 0)

+ Hv(z;0).

(2.2.3)

Theorem 1. Let xz be a central chi-square variable with v degrees of freedom and let p(xz) be a measurable function of xz. Then

Theorem 2. Let x?(A') be a noncentral chi-square variable with v degrees of freedom and noncentrality parameter A2/2, and let (p(x;(A2)) be a measurable function of xE(A2). Then

Theorem 3. Let xz(A') be a noncentral chi-square variable with v degrees of freedom and noncentrality parameter A2/2, and let (p(x;(A2)) be a measurable function of xZ(A2). Then

32


Further to the above, we have the following results involving the expectation of a product of a normal variable and functions of a noncentral chi-square variable:

Theorem 4. If 2

-N(A7

1) and ~ ( 2 is a~Borel ) measurable function, then

E [ 2 d 2 2 ) ]= A + ( X W ) ) ] .

Theorem 5. If 2

- N ( A , 1)

(2.2.7)

and ~ ( 2 is a~Borel ) measurable function, then

r;.[2”(2”,]= + 4 x ; ( A 2 ) ) ]

+ A2E[C“(x;(A2))].

(2.2.8)

Theorem 6. Let Z = (21, . ..,Z,)’ be pdimensional normal vector distributed as N p ( p Ip). , Then for a measurable function p we have JqZdZ’Z)] = PE[io(x;+2(w)]*

(2.2.9)

Theorem 7. Let Z = (21, .. . ,2,)’ be a pdimensional normal vector distributed as N,(p, I,). Then for a measurable function p we have

Theorem 8. Let Z = (21, . . . 2,)’be a pdimensional normal vector distributed as N p ( p I,,). Then for a measurable function p we have for a positive definite matrix A E[Z’AzdZ’Z)] = t r ( A ) E [p(x;+2(A2))] + P’APE[V(X;+~(A~))]. (2.2.11) For proof of Theorems 1 through 8 see Judge and Bock (1978). Finally, we may obtain the following useful identities:

where Er stands for the expectation with respect to a Poisson variable r with mean A2/2.

A2E[Xc:~(A2)] = 1 - ( p - ~ ) E [ x ; ~ ( A ~ ) ] .

(2.2.13~)

2.3. Some Results from Multivariate Normal Theory

33 (2.2.13e)

(2.2.13f)

Some Results from Multivariate Normal Theory

2.3

Let x be a pvariate normal vector with mean vector p and covariate C be denoted by N P ( pX). , If XI,... ,X N be a random sample of size N from NPCP, The joint pdf of XI,. . . ,x~ is given by

Let f =

N & Ca=l x,,

and let N

v = C(X,- f)(x, - rr)'

(2.3.2)

a=l

Then we have the decomposition given by N

C(X,a=l

-

p)' =

v + N(%

-

-

4'.

(2.3.3)

Note that 2 and V / N - 1 are unbiased estimators of p and C, respectively, while V / N is the mle of C and is a biased estimator of C. With respect t o N.(p,C) the following properties may be noted:

I. X

N

NP(p,&X)

2. V W P ( Cn, ) ,where W P ( Cn, ) stands for the Wishart distribution with n = N - 1 degrees of freedom. Then the following results may be verified N

a. E [ V ]= nC, b. E [ V A V ]= n(n+l ) X A C + n t r ( A C ) C , where A is symmetric, and


34 c. E [ v - l ] =

Ax-!

3. Let T 2 = NnS’V-’% Then

n-p+lT2 N-pT2 v -=---F~ N, P (A ) P n P n

(2.3.4)

where V stands for “equal in distribution” and F p , ~ - p ( A 2denotes ) a noncentral F-distribution with ( p , N - p ) degrees of freedom and noncentrality parameter A2/2, where A2 = Np’E-lp. Further, we note that

which proves the assertion (2.3.4). Here Fm,,(A2) stands for the noncentral F-variable with (m,n)degrees of freedom and noncentral parameter

A2/2.

Theorem 1. If X

- Np($,X),

then

E[X’AX] = t r [ A E ]

Theorem 2. Let C function g we have

- Wp(n,E).

+ $’A$.

Then for any vector

P and measurable

Theorem 3. If X1,Xz ,... , X N are i.i.d.r.v. with distribution Afp($,E) and if 6 has the prior distribution AfP(&,A),then the joint distribution of (o’, XL)’is

For more properties of the multivariate normal distribution, see Anderson (1984) and Srivastava and Khatri (1979) among others.

2.4 Beta Distribution and Applications Let xz, and x:, be two independent central chi-square variables with u1 and u2

degrees of freedom , respectively. Define

X=

2 XU,

x:, + x:*

.

(2.4.1)

Then X is distributed as a beta-variable with the pdf given by (2.4.2)

2.4. Beta Distribution and Applications Themean and variance of X are v l / q respectively.

35

+ v2 and vlv2(vl+v2)-1(v1+v2+1)-1,

Application I. Compute E [ X I ( X > c ) ] ,where I ( A ) is the indicator function of the set A:

-

since

Al

v1 + vz

B ( % + I,?) B(%,%)

1 xu'+2/2-1

(1 - x ) V 2 / 2 - 1

B(Y,Y)

dx,

(2.4.3)

r(+)r(v) vl -r($$('1+,.z+2) v1 + VZ' Application 11. If F;,m = x:/xL with x: and x; independent chi-square variables, then the statistic G = x: + x; is independent of F;,m. Thus, we -

write

(2.4.4) Now, find the E[x;I(F;,, > c ) ] .In this case, we write

(2.4.5) where x is a B

(

p+2 fIf

2 7 2 )

-variable.

+

2 and F;+zr = x ; + ~ ~ / x ; Application 111. Let T = x;+~,. xm Consider . E[cp(F;+z,,,)x&2,], where cp is a B-measurable function. Then


36

Also,

and m

E[v(F;+2T,m)F;&m]

=

-2

+

2r

For more information on these properties, see Sclove et al. (1972).

2.5

Discrete Distributions

In this section we only discuss the binomial and the multinomial discrete distributions with their properties useful for our purposes.

Binomial Distribution Let z l , . . . ,x, be n independent Bernoulli variables with 2.5.1 y=

n

xi follows the binomial distribution (;)oy(i

- e)n--y,

parameter 6. Then

= 0, I , . . . , n.

(2.5.1)

The first moment and the next three central moments of y are given by n6, n6(1 - O ) , n6(1 - 6 ) ( l - 26), and n6(1 - 6)[l + 3(n - 2)6(l - B ) ] , respectively. One may see that the skewness and kurtosis of the distribution tend to 0 and 3, as n -+ co,supporting the fact that the binomial distribution may be approximated by the normal distribution and we have the De Moivre-Laplace theorem (see Renyi, 1970, page 204).

Theorem 1. The binomial distribution

may be approximated as follows:

and if

--OO

lim

n-05

< a < b < +co, then we have

C a < d Zn.e2(!i.--e 0, i = 1,. .. ,n. Theorem 2. Consider the partition of the nonsingular matrix A given by (2.6.2) where A22 is nonsingular. Let A11.2 = A11 - A12Ai.Azl. Then

Further, (2.6.4)

Theorem 3. If A is an n x n positive definite matrix and B is an n x m ( m < n ) matrix with rank m, then B'AB is positive definite.

39

2.6. Matrix Results

If A is an n x n idempotent matrix, meaning A = A’ and AA = A, then as long as the rank of A is r , A has r positive roots and n - r roots equal t o 0, the rank of A is equal to tr(A), and there exists an orthogonal matrix r such that (2.6.5)

Theorem 4. If A and B are two symmetric matrices, a necessary and sufficient condition for an orthogonal matrix I’ t o exist, such that I”Ar = D1 and r’Br = D2 where D1 and D2 are both diagonal matrices, is that A and B satisfy AB = BA, meaning A and B commute. Theorem 5 (Courant Theorem). Let XI, ’ . ,A, be the characteristic roots of an n x n matrix A such that min X i = A’, max X i = A,, and let q ,. . . ,21, be the characteristic vectors. Then A = A l q v i ... A n V n v L , I = V ~ V ; .. . u,vU:, : sup x’Ax = A, and inf

+ +

(;.;-)

+

+

(2.6.6) where min X i = Ch,i,(A)

max X i = Ch,,(A).

and

2

(2.6.7)

2

Theorem 6. If A is an n x n matrix and b is an n x 1 vector. Then max x

x’bb’x = b’A-’b. x‘Ax

(2 A.8)

~

Theorem 7. If A is an n x n positive definite matrix and b is an n x 1 vector, a necessary and sufficient condition for A-’ - bb’ to be positive definite is that b’Ab is less than 1. Further, bb’ - A-’ cannot be negative definite for n exceeding 1.

(

)

Theorem 8. Let A and D be nonsingular matrices of order B be a pl x p2 matrix. Then

(A

+ BDB’)-’

= A-1

- A-’B

pl

and pz and

(B ’ A - ~ B+ D - ~ ) - ’ B / A - ~ .

(2.6.9)

Theorem 9. If A and D are square matrices such that A is nonsingular, then (2.6.10)

Theorem 10. If A be a p x p matrix and U a column p x 1 vector, then

/A + UU’I= I A(I~ + U’A-’U)

if

+

I A ~o

(2.6.11)


40

Application I. (2.6.12)

+ 7’) and PX= X(X’X)-lX’ and X is a n x p matrix. where B = 02/(02 Here, B = X and D = (X’X)-’ of Theorem 8. Application 11. (a1

+ bwwy = -a1( I + a + bW’W

(2.6.13)

+ bWW‘( = a p(1 + -ab W’W).

(2.6.14)

and

laI

Theorem 11. Let A be a p x p positive definite matrix, B a p x m matrix, and B1 a p x n matrix such that the rank of B1 equals p minus the rank of B and B’B = 0. Then

is a positive semidefinite matrix of rank = p - rank B.

Theorem 12. If A and B are two p x p positive matrices, then

+

A - A(A + B)-’A = (A-’ B-’)-’, (2.6.15) (i) meaning the 1.h.s. equals the inverse of the sum of the inverse of A and B. (2.6.16) I - A(A + B)-l = B(A B)-’, (ii) meaning the 1.h.s. equals the product of B and the inverse of the sum of A and B.

+

For more results on matrix theory see Anderson (1984), Rao (1973), and Srivastava and Khatri (1979) among others.

2.7

Large Sample Theory

Finite sample theory is generally based on the assumption of normality. In practice, this assumption may not hold and asymptotic theory is invoked. The results are parallel to normal theory. In the understanding of asymptotic theory, we look forward to four distinct types of convergence of sequence of random variables to a limit. We provide only a limited amount of results which achieve our goal in the book. For details, we refer the readers t o Feller (1957), Renyi (1970), Rohatgi and Saleh(2001), Sen and Singer (1993), and Ferguson(1996). In the next section we provide the definition of each type of convergence.

41

2.7. Large Sample Theory

2.7.1

Four Types of Convergence

There are two basic forms of convergence (1) sequences of distribution functions F,(z) converging to a distribution function F ( z ) for all continuity points, z as n -+ 00, and (2) sequences of random variables { X , In = 1,2, . . . } converging t o a random variable X as n -+ cw. First we consider the convergence of sequences of distribution functions.

Definition 2.1. A sequence of random variables (r.v.'s) {T,} cohverges in V F T ( z ) for all continuity points

distribution to T, meaning T,+T, if FT, (z) z of F T ( 5 ) .

-+

Convergence in distribution does not imply convergence of moments, nor does it imply convergence of probability density functions.

Definition 2.2. A sequence of r.v. {T,} converges in probability to a r.v. T if for every E > 0,

P{IT,-TI>e}+O

as n-+oo.

This is written as

T,~T

or

T, = T + o , ( ~ ) .

The convergence says that the sequence of probabilities P{IT, - TI} > E} converges to 0 as n + 03. It does not say that IT, - TI < E for n 2 N for a suitable sample size, N. As a consequence of the definition we can easily see that g(Tn)'g(T)

as n + w , P

P

when g is a continuous function. Hence, g(T,)+g(c) when T,-+c, a constant. The following theorem is important: P

V

Theorem 1. T,+T implies T,+T as n

-+

where c is

03.

Definition 2.3. A sequence of r.v.'s {T,} such that EIT,I' < 03 for some > 0 converges in the rthmean t o the r.v. T if EITI' < 00 and EIT,-TI' -+0 as n -+ co.

T

This is written as

T,~T

as n+cw. tth

Convergence in rth means implies convergence in probability, meaning T, +T P for some T > 0 implies T,-+T. If T = 2, then Var(T,) -+ Var(T). In statistical methodology, convergence in the quadratic mean is very useful and will be used frequently in this book. Finally, we consider the concept of almost sure convergence, sometimes called convergence of a sequence of r.v.'s {T,} to T with probability one or strong convergence of {Tn}to T.

42


Definition 2.4. A sequence of r.v.'s {T,} converges almost sure to the r.v. T , meaning Tna3T,if P{limn-m T, = T } = 1. The relationship of the four types of convergence concepts is given by the following diagram:

Thus, we see that convergence in distribution is the weakest of all. We observe that this convergence in distribution is implied by the rest of the three and is most important for statistical inference since the limiting distribution is used to determine the asymptotic distributional bias, risks and confidence sets or the significance of tests of parameters of interest.

2.7.2

Law of Large Numbers

When {T,} convergences to a constant, say, c, then it falls in the domain of the P

V

law of large numbers. Accordingly, when T,+c or Tn-+c,then we have the weak law of large numbers. If TnaZclthen we have the strong law of large numbers. If a distribution has a finite second moment, the law of large numbers is simplest and useful for statistical inference and the convergence is in quadratic mean. Three laws of large numbers are related t o the idea of consistency of estimators/tests. Weak laws are proved based on characteristic functions and the continuity theorem. The following theorem states the weak/strong law of large numbers with regard to the sample mean: Theorem 2. Let X I , mean. Then:

. . ,X , be i.i.d., and let

xn= cy=l X , be the sample

(i) (Weak law) If E I X J < 00, then X , z E ( Y ) = p. (ii) (Strong law) X , " Z p if and only if E ( X ) < 00 and p = E ( X ) . (iii) If EIX1' < 00, then

-

xn2nd mean

p =E(X).

The proof may be obtained by (1) the characteristic function, (2) Chebycheff's inequality, or (3) the Borel-Cantelli Lemma (a.s.) , which are stated below.

2.7. Large Sample Theory

43

Chebycheff's Inequality If T is a nonnegative r.v. such that ElTl < co,

then for every c > 0,

1

P{T > c E ( T ) }< -. C

Borel-Cantelli Lemma. Let {A,} C:='=, P(A,) < 00. Then (1) P(limA,)

be a sequence of events such that

= 0,

(2) If {A,} are independent events, then P(limA,) = 1.

Markov Inequality Let T be a nonnegative r.v. with finite rth mean, meaning pi = E(T)' < 00, for some T > 0, Then for every € > 0,

P(T > c ) 5 c-'p;.

2.7.3

Central Limit Theorems

In determining confidence intervals or the level of significance a in test of a hypothesis, we need the exact distribution of the related statistics. In the nonnormal case, under certain regularity conditions, we may determine the asymptotic distribution that may be used to set up confidence intervals, and determine type I and power of tests based on the statistic in question. It is well known that if X I , . . . ,X , are i.i.d. r.v. from a cdf F ( z ) with finite mean p and variance 0 2 ,then for the sample mean X, we have

f i - - p ) N(O,1) ff

-(X,

as n --f

N

CQ.

This is the basic central limit theorem (CLT). However, there are situations where X I , . . . , X , are not i.i.d. r.v. but satisfy

E(X2) = 0 , V ( X 2 )= ffz2 < 00, Now, let T, = X1 question:

+ ... + X ,

When is T,/o,

N(0,l) as n

-

and --+

0 ;

= of

(2

= 1,. . . ,n).

+ - . -+ o;. Then we may ask the

CQ?

The answer lies in the Lindeberg-Feller theorem, which gives the uniform integrability condition: If for every 6 > 0,

then m

1

ffn

-

N(0,l).

44


The next theorem called the Hajek-Sidak CLT deals with the asymptotic normality of a linear combination of i.i.d. r.v.’s X I , . . . ,X , meaning of T, = a l X l + ... + a n X n , where E ( X i ) = p and E ( X i - p)’ are both finite.

Hajek-Sidak CLT. Let X I , . . . ,X, be i.i.d. r.v. such that E ( X i ) = p and VarXi = u2 are both finite. Let

T, = alX1 + . . . + anXn. Then

whenever the Noether condition

is satisfied. Sometimes we have to deal with functions of two or more statistics, say, P

g(T,,S,)where FT,,(z) -+ F ( z ) and Sn+c (constant). Then we need the limiting distribution of g(T,,Sn). In this case, we can use the Slusky theorem. Stated in a simplest form the Slutsky theorem is as follows: Let X I , . . . ,X, be a sequence of r.v. with distributions F l , . . . ,F,, and suppose F,(z) -+ F ( z ) as n + 03. Further, let Y1,.. . ,Y, be another set of r.v.’s P

such that Y, + c. Set V, = X, as n + 03, the distribution of

(1) V, tends to F ( z

+ Y,,

W, = X,Y,

and 2, = X,/Y,.

Then,

+ c)

(2) W, tends to F ( c z ) (3) 2, tends to F ( z / c ) for c > 0 as n

-+

co.

An important concept that dominates the asymptotic theory of rankstatistics is that of “contiguity of probability measures.” We define the concept in the following:

Definition 2.5. Let {p,} and {q,} be the sequence of simple hypothesis densities defined in a measure space ( y ,I?,, p n ) . If

Pn(A,)

-+

0 implies &,(A,)

+0

as n

+ 03,

then {Q,}are contiguous to {Pn}. Here 8 P ( A n ) / a p , = p , and aQ(An)/dpn= 4,. Generally, we are interested in the asymptotic distribution of statistics {T,(Y)}. Then convergence of {T,(Y)} -+ 0 under {P,} implies {T,(Y)} -+ 0 under {Q,} if {Q,} is contiguous to {P,}.

2.8. Nonparametric Theory: Preliminaries

45

In this context the likelihood ratio statistics plays an important part. Consider the likelihood ratio statistic

L,(Y)

=fi

if p,>O

=1 = co

ifp,>O if 0 = p ,

P77

< qn.

If log L,(Y) is asymptotically N ( - a 2 / 2 , a’) under {P,}, then {Q,} is contiguous to {P,} (LeCam’s lemma 1). Let T,(Y) be a statistic. Assume that under {P,}, (T,(Y),logL,(Y)) has asymptotically bivariate normal distribution with mean ( e l , & ) and covariance matrix

( :;:

+

012 022

) . Then, under

{ Q,}, the asymptotic distribution

of T,(Y) is N(p1 0 1 2 , all). (LeCam’s lemma 3). LeCam’s lemma 2 gives conditions when log L,(Y) is asymptotically ~ ( - ~ a ~ , a ~ ) . For further information on contiguity see Hajek and Sidak (1967), Hajek, Sidak, and Sen (1999), Puri and Sen (1986), and JureEkova and Sen (1996) among others. ~

2.8

Nonparametric Theory: Preliminaries

In this section we present some of the basic preliminaries of nonparametric methods which are relevant in the discussion of the R-estimators of the parameters of the models in various chapters. The proofs and details of technicalities are available in Hajek (1969), Hajek and Sidak (1967), Hajek, Sidak, and Sen (1999), Puri and Sen (1986), Randles and Wolfe (1979), and Sen (1981) among others.

2.8.1

Order-Statistics, Ranks, and Sign Statistics

Let Yl,Yz,. . . ,Y, be n i.i.d. r.v. with absolutely continuous cdf F ( y ) with absolutely continuous pdf f(y). Let Y ( . )= (ql), .. . , be the orderstatistic vector corresponding to the i.i.d. r.v. Y = (Y1,. .. ,Y,)’, which we write as

q,))’

y = (yl,. . * ,yn)’ = ( where Rj is the rank of Y ( . )is

n

~ R I ).,. 7 qR,))’

6 among Y1,.. . ,Y,.

(2.8.1)

Thus, the joint distribution of

n

n!

i=l

f ( Y ( , ) ) on - co < Y(1) < . . . < Y(n)

< 00,

(2.8.2)

and the joint distribution of R = (R1,.. . , R,)’ of the rank-vector is

1 n!

for R E R,,

(2.8.3)


46

where R, is the set of n! permutations of the integer-vector (1,2,. . . ,n). If the pdf of y3 ( j = 1,. . . ,n ) is symmetric about 0, then we may define four statistics relevant to the distribution: (i) The absolute-value statistics

(ii) The absolute-value order-statistics

(iii) The rank-statistic, Rt = (RF,. . . ,I?:)’,

where

Rl is the rank of lY,l among lY11,. . . , /Y,l.

(2.8.4~)

(iv) The sign-statistic S = (sign Yl ,. . . , sign Y,). Clearly,

Y , = ly31 signYj,

j = 1,. . . ,n.

(2.8.4d)

The distributions of lYl(.), R+, and S are independent with distributions given by (i) 2, n! (ii) (iii)

n:=,f ( z i ) , O < zi < ...

(ii) v a r [ ~ n ( o )= l a=]

and (i) EfLn(O)I = 0,

Standard Scores. For the symmetric location distributions F ( z - 0) and for the test of 0 = 0, the scores are

where F-’(U), (0 < u < 1) is the quantile of order u of the cdf F ( y ) and the score generating function is defined by


48

The Wilcoxon test is associated with the score function = u,

+(ZL)

0

< ZL < 1,

(2.8.14a)

and the normal score test is associated with (2.8.14b) For the regression model, the scores are defined by (2.8.15a) with the score generating function (2.8.1513)

Theorem 2. Let n

T,(o) = n-1/2

CU:(R+) signx,

(2.8.16)

i=l

where the scores {uk ( 2 ) li = 1, . . . ,n} converge to some square integrable function & ( u ) such that J ; [ ~ + ( Z L ) ] ~ ~ Z C > 0. Then Tn(0)follows approximately the normal distribution with mean 0 and variance J : [ ~ + ( z L ) ] ’ ~ z L . If

‘ p + ( z ~ ) = ++(zL,

f),then the variance is I ( f ) .

Theorem 3. Let n

L,(o) = n-1/2

C(xi- ~,)a,(~i),

(2.8.17)

i=l

where the scores { u n ( i ) ( i= 1 , . . . ,n } converge to some square integrable function p(zi) such that A; = J;[q(u)- (pI2dzi > 0, (p = s,’q(u)dzi. Then L n ( 0 )follows approximately the normal distribution with mean 0 and variance AQ : and

=

( lim n-’Qn) n-cc

1

1

0

[p(u)- (pl2dzi, Qn =

c(zz n

- %n)2

i=l

Q = limn-’&,.

If ~ ( z L )= +(zL, f),then (p = 0 and (2.8.18)

49

2.8. Nonparametric Theory: Preliminaries Theorem 4. (Saleh and Sen, 1978). Let

T,(o)=

n

n-1/2Ca,+(~F) sign u,, z=1

n

Ln(0) = n-1/2

X(q 2=1

(2.8.19)

- Zn)an(Rz),

so

where { a R ( i ) } and { a n ( i ) } converge to some square integrable functions @(u) and ~ ( 2 1 )such that A; = J;[y(a) - 3]2d26 > 0, 3 = 1 cp(u)da and

$ [ y + ( ~ ) ] 0) and 8i2) = inf{a : T,(a) on Puri and Sen (1986) we state the following theorem:

< O}. Then, based

Theorem 5. Under the assumed regularity condition as n 4 co,

where

(2.8.29)

Location and Regression Model. Consider the location and regression model

Yn = 91,

+ ,Bx+ el

(2.8.30)

51

2.8. Nonparametric Theory: Preliminaries

where Y , = (Yl,. . . ,Yn)',x = ( ~ 1 , ... ,xn)', and e = ( e l , . . . , e n ) and the error distribution satisfy the same condition as in Section 2.8.4. Then, consider the linear rank-statistics n

sign (Y,- a - bxi),

Tn(a,b) = n-l12 Ca:(R!) i=l where R: i s t h e r a n k o f IY,-a-bziI and

among Ifi-u-bx11,

... ,IYn-a-bxnj,

n

Ln(b)= nd112 c ( x i - I n ) a n ( R i ) ,

(2.8.31)

i=l

where Ri is the rank of Y, - bxi and Y1 - bxl, ... ,Yn - b,xn. Then the Restimators of (6, P) are given by

1 p" -- 2-(&') + pi2)), where pi2)= inf{b : ~ , ( b )< 01,

,&')

= sup{b : Ln(b) > 0},

and

1 -

+ 6?)), where 6:) = inf{a : ~,(a,b,) < 01. 6:)

= 2(6i1)

6;) = S U ~ { U: Tn(a,&) > 0) (2.8.32)

By the fact that T,(O,O) and Ln(0) are symmetrically distributed around the origin, 0, the asymptotic distribution of the estimators is as given in the following theorem:

Theorem 6. (Adichie, 1967). Under the assumed regularity condition and maxl 0 and 6 E RP and 11 . 11 denotes the pdimensional Euclidean norm (2.8.42)

(ii)

lim Po

n-00

{

sup 1601 0 and 2 # 0. The quadratic bias of 6, is linear in A’, while the quadratic bias of 6FT increases to a maximum as A2 moves away from the origin, and then decreases toward zero as A2 co. Now we compare the MSE functions of the restricted and preliminary test estimators with respect to the traditional estimator, 6,. The mean-square relative efficiency (MRE) of 6, compared t o 6, may be written as ---f

(

( + -A2

MRE(6,;6,) = 1 + ni2) 1

nz2

)-’

.

(3.2.12)

The efficiency is a decreasing function of A’. Under Ho it has the maximum value (l+$)

21

(3.2.13)

Chapter 3. Preliminary Test Estimation

62

0

2-

.-_ 4 v)

0

QB

~.

0-

a = 0.15

4

O

2

0

4

A2

6

8

10

Figure 3.2.1 Graph of quadratic bias functions of the estimators and

>

< >

MRE(6,; 6,) (1 according as A2-1.

(3.2.14)

Thus, 6, performs better than 6, whenever A2 < 1; otherwise, 6, performs better. The relative efficiency of 8zT compared to 6, is given by

MRE(6,PT;6,) = [l + g(A')]-',

(3.2.15)

where

(3.2.16) Under Ho, it has the maximum value

MRE@,PT;e,) = (3.2.17)

3.2. PTE of the Intercept Parameter

63

a

a

= 0.15

Eflciency

= 0.20

a

= 0.25

of PTE re1 UE

l7

MRE N

a = 0.25

0

a = 0.20

5

20

15

10

A2

Figure 3.2.2 Graph of MRE(6,;

8,)

and MRE(6zT;6,)

and MRE(6zT;8,) 2 11 according as

Hence, 6zT performs better than 6, if A' better than 6,". Since

1 2G3,m (?F~,rn(a); A')

(3.2.18) is

I K ( F I , ~ (A'); ~ ) ;otherwise, 6,

- G5,m ( i F l , m ( a ) ; A')

> 0,

(3.2.19)

(3.2.20) The graphs of the MRE(6, : 8,) and MRE(6LT : 6,) are shown in Figure 3.2.2.

3.2.4 Optimum Level of Significance of Preliminary Test Consider the relative efficiency of

MRE(a, A'), we have

6,"

compared to

MRE(cu,A2) = [1+g(A2)]-'.

6,.

Denoting it by

(3.2.21)


64

The graph of MRE(a,A’), as a function of A’ for fixed a , is decreasing, crossing the 1-line t o a minimum at A’ = A i ( a ) (say); then it increases toward the 1-line as A’ -+ m. The maximum value of MRE(a, A’) occurs a t A’ = 0 with the value max MRE(a, A’) = A2

for all a E A, the set of possible values of a. The value of MRE(a,O) decreases as a increases. On the other hand, if a = 0 and A2 varies, the graphs of MRE(0,A’) = 1 and MRE(1,A’) intersect a t A’ = 1. In general, MR.E(al,A’) and MRE(a2,A’) intersect within the interval 0 5 A’ 5 1; the value of A’ a t the intersection increases as a increases. Therefore, for two different a-values, MRE(a1, A2) and MRE(a2, A’) will always intersect below the 1-line. In order to obtain an estimator (PTE) with a minimum guaranteed efficiency Eo,we adopt the following procedure: If 0 5 A’ 5 1,we always choose 6, because MR.E(a,A’) 2 1 in this interval. However, in general, A’ is unknown, so there is no way to choose an estimator that is uniformly best. For this reason, we select a n estimator with minimum guaranteed efficiency, such as Eo, and look for a suitable a from the set A0 = {alMRE(a,A’) 2 Eo}. The estimator chosen maximizes MRE(a, A’) over all a E A0 and A’. Thus, we solve the following equation for the optimum a*: min MRE(a, A’) = E ( a ,A:(*)) A2

= Eo.

(3.2.23)

The solution a* obtained this way gives the P T E with minimum guaranteed efficiency Eo, which may increase toward MRE(a*, 0) given by (3.2.22). Tables 3.2.1 and 3.2.2 give selected values of T’/Q and a = 0.05(0.05)0.50 for the procedure of choosing the level ( a * )of significance for n = 8 and Z2/Q = 1(.5)5, n = 12, and Z’/Q = 0.1(0.1)0.9.

3.2. PTE of the Intercept Parameter

ff -

LO!

Table 3.2.1 Maximum and Minimum Guaranteed Efficiencies for n = 8 and T 2 / Q = 1.0(0.5)5.0

/&

-2

Emax Emin A2in

1.11

Emax Emin A2in

1.11

Emax Emin A L n

1.21

Emax Emin Akin

l.2!

Emax Emin A2in

1.31

Emax Emin Akin

j.31

Emax Emin A2in

1.41

Emax Emin Azin

1.4!

Emax Emin Akin

1.51

Emax Emin

-

65

+ n & I

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

5.0

3.29753 3.16716 3.81283 3.94493 4.04011 4.11195 4.16810 4.21320 4.25021 0.32597 0.31773 0.31353 0.31099 0.30929 0.30806 0.30714 0.30643 0.30585 7.05937 7.05937 5.53023 7.05937 7.05937 7.05937 7.05937 7.05937 7.05937 2.27544 2.39283 2.46001 2.50352 2.53400 2.55654 2.57389 2.58764 2.59883 0.44789 0.43855 0.43378 0.43087 0.42892 0.42752 0.42646 0.42564 0.42497 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 1.82656 1.88654 1.91991 1.94117 1.95589 1.96676 1.97496 1.98149 1.98678 0.54227 0.53289 0.52805 0.52510 0.52311 0.52161 0.52061 0.51976 0.51909 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 1.40934 1.56154 1.64097 1.68976 1.72277 1.74659 1.76458 1.77866 1.78997 0.67415 0.62561 0.60605 0.59549 0.58887 0.58434 0.58104 0.57853 0.57656 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 1.38335 1.40405 1.41526 1.42230 1.42712 1.43063 1.43330 1.43540 1.43710 0.70912 0.70127 0.69719 0.69469 0.69299 0.69177 0.69085 0.69013 0.68955 0.60838 0.60838 0.60838 0.60838 0.60838 0.60838 0.60838 0.60838 0.60838 1.30073 1.31595 1.32416 1.32929 1.33280 1.33535 1.33729 1.3882 1.34005 0.74827 0.74109 0.73735 0.73505 0.73349 0.73237 0.73152 0.73086 0.73033 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 1.22121 1.23169 1.23731 1.24082 1.24321 1.24495 1.24627 1.24731 1.24815 0.79911 0.79298 0.78978 0.78781 0.78647 0.78551 0.78478 0.78421 0.78375 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 1.09488 1.12019 1.13192 1.13868 1.14309 1.14618 1.14847 1.15014 1.15165 0.89949 0.87846 0.86936 0.86427 0.86102 0.85877 0.85712 0.85585 0.85485 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 1.11847 1.12359 1.12632 1.12802 1.12917 1.13001 1.13065 1.13115 1.13155 0.87938 0.87532 0.87318 0.87187 0.87097 0.87033 0.86984 0.86946 0.86915 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 1.08512 1.08868 1.09058 1.09176 1.09256 1.09314 1.09358 1.09393 1.09420 0.90992 0.90677 0.90512 0.90410 0.90340 0.90290 0.90252 0.90223 0.90199 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621


66

Table 3.2.2 bfaximum and Minimum Guaranteed Efficiencies for n = 12 and -2 L =O.l(O.2)O.g Q cy -

Z2/Q

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.01 Em,, 1.85798 2.33498 2.63864 2.84893 3.00317 3.12114 3.21428 3.28968 3.35198

Em,, 0.47754 0.42471 0.40464 0.39408 0.38755 0.38313 0.37992 0.37750 0.37560 A”,,, 5.53022 5.53022 5.53023 5.53022 5.53022 5.53022 5.53022 5.53022 5.53022 0.11 Em,, 1.57358 1.82247 1.96155 2.05035 2.11197 2.15723 2.19189 2.21927 2.24145 Em,, 1.40934 1.56154 1.64097 1.68976 1.72277 1.74659 1.76458 1.77866 1.78997 A;,, 0.67415 0.62561 0.60605 0.59549 0.58887 0.58434 0.58104 0.57853 0.57656

0.11 Em,, 1.40934 1.56154 1.64097 1.68976 1.72277 1.74659 1.76458 1.77866 1.78997 Em,, 0.67415 0.62561 0.60605 0.59549 0.58887 0.58434 0.58104 0.57853 0.57656 A:, 3.94090 3.94090 3.94090 3.94090 3.94090 3.94090 3.94090 3.94090 3.94090 0.21 Em,, 1.30133 1.40191 1.45223 1.48234 1.50256 1.51695 1.52774 1.53613 1.54284 Em,, 0.73816 0.69495 0.67714 0.66744 0.66133 0.65713 0.65406 0.65173 0.64989 A:, 3.60055 3.60055 3.60055 3.60055 3.60055 3.60055 3.60055 3.60055 3.60055 0.2, Em,, 1.22209 1.29033 1.32345 1.34302 1.35594 1.36510 1.37195 1.37725 1.38148 Em,, 0.79359 0.75642 0.74086 0.73231 0.72691 0.72318 0.72046 0.71838 0.71674 A’,,n 3.35268 3.35268 3.35268 3.35268 3.35268 3.35268 3.35268 3.35268 3.35268 0.31 Em,, 1.16931 1.21843 1.24181 1.25548 1.26446 1.27080 1.27552 1.27917 1.28207 Em,, 0.83380 0.80206 0.78860 0.781 16 0.77644 0.77318 0.77079 0.76897 0.76753 A;,, 3.18236 3.18236 3.18236 3.18236 3.18236 3.18236 3.18236 3.18236 3.18236 0.31 Emax1.12716 1.16235 1.17885 1.18842 1.19468 1.19908 1.20235 1.20487 1.20688 Em,, 0.86973 0.84357 0.83234 0.82610 0.82213 0.81938 0.81737 0.81582 0.81461 A’,,, 3.04378 3.04378 3.04378 3.04378 3.04378 3.04378 3.04378 3.04378 3.04378 0.41 Em,, 1.09488 1.12019 1.13192 1.13868 1.14309 1.14618 1.14847 1.15014 1.15165 Em,, 0.89949 0.87846 0.86936 0.86427 0.86102 0.85877 0.85712 0.85585 0.85485 A’,,, 2.93352 2.93352 2.93352 2.93352 2.93352 2.93352 2.93352 2.93352 2.93352 0.4% Em,, 1.06997 1.08809 1.09642 1.10120 1.10431 1.10648 1.10810 1.10934 1.11032 Em,, 0.92393 0.90750 0.90031 0.89629 0.89372 0.89193 0.88961 0.88961 0.88881 A;,, 2.84422 2.84422 2.84422 2.84422 2.84422 2.84422 2.84422 2.84422 2.84422 0.5r Em,, 1.05070 1.06354 1.06940 1.07276 1.07493 1.07645 1.07758 1.07844 1.07913 Em,, 0.94378 0.93131 0.92584 0.92275 0.92077 0.91940 0.91838 0.91761 0.91700 2.77103 2.77103 2.77103 2.77103 2.77103 2.77103 2.77103 2.77103 2.77103 IA,:

3.3. Two-Sample Problem and Pooling of Means

67

To illustrate the use of the tables, we consider n = 12 and Z2/Q = 0.40. We want a P T E with minimum guaranteed efficiency EO = 0.7811 (say). Then, using the table for n = 12 and EO = 0.7811 we find that a* = 0.30 with maximum efficiency E* = 1.2548.

3.3 3.3.1

Two-Sample Problem and Pooling of Means Model

Consider again the simple linear model (3.2.1)

Y=$l,+px+e,

(3.3.1)

-

where x = (0,. .. '0, 1 , . . . ,1)' is an n x 1 vector of n1 zeros and n2 one's such that n = n1 n2. Also, e N,(O, u21,), and in this case

+

p1=8, p 2 = 8 + P

and p = p 2 - p 1 .

(3.3.2)

Further,

3.3.2

Estimation and Test of the Difference between Two Means

The maximum likelihood estimators of p1, p2, and u2 are

where

Here y1 is the sample mean of the first n1 y-observations while j j 2 is the sample mean of the last n2 y-observations, and s; is the pooled sample variance of the y-observations. For testing the null hypothesis HO : 112 = p1 against H A : p2 # p1, the likelihood ratio test-statistic is given by (3.3.5)

L, follows a noncentral F-distribution with (1, n1 trality parameter A2/2, where

+ 722.2- 2) d.f. and noncen(3.3.6)

68


Under Ho, 13, follows a central F-distribution. Finally, the unrestricted, restricted, and preliminary test estimators of p1 are given by

and (3.3.7) where I ( A ) is the indicator function of the set A and F I , ~ (is ~the ) upper a-level critical value of the central F-distribution with (1, m = n1 722 - 2) d.f.

+

3.3.3 Bias and mse Expression of the Three Estimators of a Mean To obtain the bias, quadratic bias, and mse expressions of the estimators of p1, we simply substitute in the expressions of Theorems 3.2.1 and 3.2.2 corresponding values of Z and Q given by (3.3.3). Thus, for the bias and quadratic bias we have the following theorem:

(3.3.9)


69

It is clear from the expressions of the mse's of fi1 and fifT that the relative efficiencies of j i 1 and jiFT compared to @I are given by

MRE(,hl;bl) =

{2 +nz n1

-1

n1

(3.3.1Oa)

+nz

and

respectively, where

Note that MRE(ji1;bl) is a decreasing function of A2. The maximum value of M R E ( f i 1 ; bl) occurs at A' = 0 with the value 1 + n2/n1; and as A2 + 03, MRE(ji1;f i 1 ) tends to zero. Further, j i 1 performs better than b1 for A' in the interval [ O , l ] , while jll performs better than outside this interval. Next, we compare firTrelative to f i l . In this case, the efficiency has the maximum value

MRE(bFT;bl)=

(31F ~ , m ( ~ ) ;

-1

0)]

( 2 1),

(3.3.11)

so it decreases monotonically, crossing the 1-line a t A2 = A i i n ,and then increases toward the 1-line as A' tends to infinity. The graphs of the M R E ( j i 1 : b1) and the MRE(ji;* : fi1) are shown in Figure 3.3.1. Based on the analysis of Section 3.2.4, the optimum level of significance a* with minimum guaranteed efficiency EO is obtained by solving the following equation for (Y: min MRE(a, A2) = E ( a ,A $ ( a ) )= Eo A2

(3.3.12)

as in the case of (3.2.23). Tables 3.3.1 through 3.3.3 of maximum and minimum guaranteed values are provided for (Y = 0.05(0.05)0.50 selected pair of integers (711,722). To illustrate the use of the table, let us choose nl = 8, n 2 = 16, and EO= 0.66. Then using the corresponding tabular values, we obtain a* = 0.15. The efficiency can go up to 1.442 if A' -+ 0.


70

Table 3.3.1 Maximum and Minimum Guaranteed Efficiencies a\(n1,722)

0.05 E'

1.6445 1.4451 1.3400 1.2750 2.0301 1.7587 1.6005 1.4969 0.4623 0.5477 0.6092 0.6558 0.4380 0.4878 0.5294 0.5647 Ao 7.0594 6.3232 5.9338 5.6934 5.9338 5.6934 5.5302 5.4123

Eo 0.10 E'

1.4605 1.3258 1.2522 1.2058 1.6746 1.5182 1.4208 1.3542 Eo 0.5905 0.6647 0.7157 0.7530 0.5572 0.6039 0.6415 0.6726 Ao 5.3280 4.9274 4.7087 4.5712 4.7087 4.5712 4.4767 4.4078

0.15 E'

1.3415 1.2458 1.1921 1.1577 1.4755 1.3745 1.3089 1.2629

Eo 0.6781 0.7410 0.7830 0.8131 0.6434 0.6851 0.7180 0.7446 Ao 4.5088 4.2448 4.0983 4.0053 4.0983 4.0053 3.9409 3.8938 0.20 E'

1.2578 1.1879 1.1479 1.1220 1.3471 1.2779 1.2317 1.1988

Eo 0.7447 0.7973 0.8317 0.8560 0.7119 0.7482 0.7764 0.7988 Ao 4.0090 3.8206 3.7150 3.5475 3.7150 3.5475 3.5005 3.5661 0.25 E'

1.1962 1.1443 1.1142 1.0946 1.2579 1.2089 1.1756 1.1514

Eo 0.7978 0.8411 0.8690 0.8884 0.7683 0.7993 0.8229 0.8415 Ao 3.5667 3.5266 3.4474 3.3965 3.4474 3.3965 3.3611 3.3350 0.30 E'

1.1495 1.1107 1.0880 1.0731 1.1931 1.1577 1.1333 1.1155

Eo 0.8409 0.8760 0.8983 0.9137 0.8154 0.8412 0.8606 0.8758 Ao 3.4162 3.3095 3.2488 3.2097 3.2488 3.2097 3.1824 3.1622 0.35 E'

1.1135 1.0845 1.0674 1.0561 1.1445 1.1188 1.1009 1.0877

Eo 0.8761 0.9042 0.9217 0.9338 0.8548 0.8758 0.8915 0.9037 Ao 3.2250 3.1426 3.0955 3.0651 3.0955 3.0651 3.0438 3.0280 0.40 E'

1.0854 1.0638 1.0510 1.0425 1.1076 1.0889 1.0757 1.0660

Eo 0.9049 0.9269 0.9405 0.9498 0.8877 0.9044 0.9168 0.9263

00 3.0747 3.0107 2.9739 2.9501 2.9739 2.9501 2.9335 2.9211 0.45 E'

1.0634 1.0475 1.0381 1.0318 1.0792 1.0657 1.0561 1.0490

Eo 0.9284 0.9451 0.9555 0.9625 0.9148 0.9278 0.9373 0.9446 Ao 2.9542 2.9045 2.8758 2.8573 2.8758 2.8573 2.8442 2.8345

0.50 E* 1.0462 1.0347 1.0279 1.0233 1.0573 1.0477 1.0408 1.0357 Eo 0.9472 0.9598 0.9674 0.9726 0.9369 0.9467 0.9538 0.9593 Ao 2.8562 2.8178 2.7956 2.7812 2.7956 2.7812 2.7710 2.7635


71

Table 3.3.2 Maximum and Minimum Guaranteed Efficiencies

a\(% ,nz) (4, 12) (6, 12) (8, 12) (10, 12) (4, 16) (6, 16) (8, 16) (10, 16) 0.05 E*

2.2873 1.9918 1.8067 0.4286 0.4637 0.4948 5.5302 5.4123 5.3232

1.6798 2.4712 2.1720 1.9740 0.5224 0.4234 0.4506 0.4753 5.2534 5.3232 5.2534 5.1973

1.8332 0.4979 5.1512

1.7993 1.6456 1.5416 0.5440 0.5780 0.6072 4.4767 4.4078 4.3554

1.4664 1.8811 1.7363 1.6324 0.6325 0.5369 0.5635 0.5873 4.3142 4.3554 4.3142 4.2809

1.5542 0.6085 4.2534

1.5480 1.4540 1.3876 0.6293 0.6603 0.6866 3.9409 3.8938 3.8578

1.3381 1.5934 1.5081 1.4442 0.7090 0.6216 0.6463 0.6679 3.8294 3.8578 3.8294 3.8064

1.3947 0.6871 3.7874

1.3932 1.3310 1.2859 0.6983 0.7258 0.7487 3.5005 3.5661 3.5397

1.2516 1.4213 1.3661 1.3238 0.7680 0.6908 0.7129 0.7320 3.5188 3.5397 3.5188 3.5019

1.2902 0.7487 3.4880

0.25 E* 1.28887 1.2458 1.2140 Eo 0.7559 0.7797 0.7992 Ao 3.3611 3.3350 3.3149

1.1895 1.3072 1.2696 1.2403 0.8155 0.7490 0.7683 0.7847 3.2991 3.3149 2.2991 3.2862

1.2167 0.7990 3.2756

0.30 E'

1.2143 1.1839 1.1610 0.8045 0.8246 0.8409 3.1824 3.1622 3.1467

1.1432 1.2269 1.2006 1.1797 0.8544 0.7985 0.8148 0.8287 3.1344 3.1467 3.1344 3.1245

1.1628 0.8406 3.1163

1.1594 1.1376 1.1210 0.8456 0.8621 0.8754 3.0438 3.0280 3.0159

1.1080 1.1681 1.1494 1.1345 0.8863 0.8405 0.8540 0.8654 3.0063 3.0159 3.0063 2.9986

1.1222 0.8751 2.9921

Eo Ao

1.1181 1.1024 1,0903 0.8802 0.8934 0.9040 2.9335 2.9211 2.9117

1.0809 1.1242 1.1108 1.1000 0.9126 0.8759 0.8868 0.8959 2.9041 2.9117 2.9041 2.8980

1.0911 0.9037 2.8929

0.45 E* Eo Ao

1.0866 1.0753 1.0666 0.9088 0.9192 0.9274 2.8442 2.8345 2.8270

1.0598 1.0909 1.0813 1.0736 0.9341 0.9055 0.9140 0.9211 2.8212 2.8270 2.8212 2.8164

1.0672 0.9271 2.8124

0.50 E*

1.0625 1.0545 1.0483 0.9323 0.9401 0.9463 2.7710 2.7635 2.7577

1.0434 1.0655 1.0587 1.9532 0.9514 0.9297 0.9362 0.9416 2.7530 2.7577 2.7530 2.7493

1.9487 0.9462 2.7462

Eo Ao 0.10 E'

Eo Ao 0.15 E"

Eo Ao 0.20 E'

Eo Ao

Eo Ao 0.35 E"

Eo Ao 0.40 E*

Eo Ao

72

Chapter 3. Preliminary Test Estimation Table 3.3.3 Maximum and Minimum Guaranteed Efficiencies a\(n1, n2) 0.05 E'

(117 12)

1.8113 1.6673

1.6291 0.5365 5.1973

1.6189 0.5428 5.1127

1.6114 0.5474 5.0521

1.4351 0.6451 4.2809

1.4280 0.6501 4.2304

1.422$ 0.6538 4.1941

1.3171 0.7199 3.8064

1.3119 0.7240 3.7715

1.3081 0.7265 3.7464

1.2367 0.7773 3.5019

1.2328 0.7806 3.4760

1.2301 0.783C 3.4574

1.1787 0.8233 3.2862

1.1758 0.8259 3.2667

1.1736 0.827s 3.2525

1.1354 0.8607 3.1245

1.1332 0.8629 3.1093

1.1316 0.8644 3.0983

1.1022 0.8915 2.9986

1.1006 0.8932 2.9867

1.0994 0.8944 2.9780

1.0766 0.9167 2.8980

1.0754 0.9180 2.8887

1.0745 0.9189 2.8819

1.0567 0.9372 2.8164

1.0558 0.9382 2.8090

1.0551 0.9389 2.8037

1.4120 0.9537 2.7493

1.0405 0.9545 2.7436

1.0400 0.9550 2.7394

Eo 0.4293 0.5133

Ao 7.0594 5.5302 0.10 E'

1.5633 1.4618

Eo 0.5579 0.6266

Ao 5.3280 4.4767 0.15 E'

1.4102 1.3364

Eo 0.6482 0.7048 Ao 4.5088 3.9409 0.20 E'

1.3059 1.2511

Eo 0.7158 0.7650

Ao 4.0090 3.5005 0.25 E"

1.2307 1.1895

Eo 0.7754 0.8133 Ao 3.5667 3.3611 0.30 E'

1.1746

1.1435

Eo 0.8222 0.8527 Ao 3.4162 3.1824 0.35 E'

1.1318 1.1084

Eo 0.8609 0.8851 Ao 3.2250 3.0438 0.40 E'

1.0987 1.0812

Ea 0.8928 0.9117 Ao 3.0747 2.9335 0.45 E'

1.0731 1.0601

Eo 0.9190 D.9334 Ao 2.9542 2.8442 0.50 E'

1.0531 1.0436

Eo 0.9402 9.9509

An

(13, 14) (15, 16:

(7, 8)

(3, 4)

2.8562

2.7710

73

3.4. One-Sample Problem: Estimation of Mean

a = 0 IS a = 0.20 a = 0.25 Efficcrency of RE re/ UE

Efficiency of PTE re1 UE forn,=l ondn,=4

a = 0.25

x!

a = 0.20

,

0

5

I

I

10

15

A2

I

20

Figure 3.3.1 Graph of MRE(,Gl; ,GI) and MRE($rT; ,GI)

3.4 3.4.1

One-Sample Problem: Estimation of Mean Model

Consider the simple linear model

Y =81,+0x+e

-

where e Nn(0,a21,)and x = 0. Then we have the reduced model for the mean given by

Y

= 81,

+ e,

(3.4.1)

and the variance of the errors is a’. Our problem is to estimate 8 when it is suspected that 6 may be equal to 80. As before, we define the unrestricted, restricted, and preliminary test estimators of 8 in the next section.

3.4.2 Unrestricted, Restricted, and Preliminary Test Estimators Using the sample information and (3.4.1), we minimize the quadratic form

(Y - 6l,)’(Y - 61,),

(3.4.2)

to obtain the unrestricted estimator of 8 as

8,

=(lkl,)-l~k= ~

7

(sample mean).

(3.4.3)

74


Clearly, E(6,) = 6 and Var(8,) = u’/n. Further, the unbiased estimator of u2 is given by

1 m

sp = -(Y

-

6 , 1 , ) ’ ( ~ - 6,1,),

In order t o test the null hypothesis HO : 6 = use the likelihood ratio test-statistic

m = n - 1. 60

against H A : 6

(3.4.4)

#

60, we

(3.4.5) The exact distribution of C, is the noncentral F-distribution with (1,m ) d.f. and noncentrality parameter A2/2,where A2 = n(6 - 6o)’/u2. Under Ho, it follows a central F-distribution. Thus, a t the a-level of significance we reject HO if C, 2 Fl,,(a), where Fl,,(cr) is the upper a-level critical value of the central F-distribution with (1,m ) d.f. The restricted estimator of 6 is simply 80, though we could combine 6, and 60 as d6, (1 - d)&, where 0 < d < 1 is the coefficient of distrust on the value 6 = 60 (type I1 error in testing Ho). However, in this case we will assume d = 0, meaning no distrust on 6 = 60. So we define the preliminary test estimator of 6 as

+

GET

+

= @01(Cn< ~l,rn(a))6 n I ( C n 2 ~l,rn(a))

= 6, - (6, - Qo)1(Cn < ~l,rn(a))?

(3.4.6)

where Fl,,(a) is the a-level critical value.

3.4.3 Bias, mse, and Analysis of Efficiency The bias expressions of

6,

and OT:

are

b l(6 , ) = 0,

&(en) = 0,

(3.4.7a)

Similarly, U’

Ml(6J = n

(3.4.8a)

75

3.4. One-Sample Problem: Estimation of Mean and

A&(ET)= n [1 0’

-

(

G3,m ;FLm(a); A2)

{

+A2 2G3,m (;Fl,m(a); A’) Thus, the efficiency expression for

8,”

to

1

- G5,m ( s F l , m ( a ) ; A2)}]

&, is given by,

en) = [ 1 + g(A’)]-’,

MRE(6,PT;

(3.4.8b) (3.4.9)

where

Hence,

8KT performs better than gn 0

whenever

I A’ 5 K(Fi,,(a); A’),

(3.4.11)

where

To obtain a P T E with minimum guaranteed efficiency Eo, we solve the equation min M R E ( a ,A2) = E ( a ,Ai(a)) = Eo A2

(3.4.13)

to find the optimum a-value a* as before. Table 3.4.1 gives the maximum and

minimum efficiency of the PTE of 19for various a and sample sizes.


76

Table 3.4.1 Minimum and Maximum Efficiency of PTE for n = 8 and T 2 / Q = 0.1(0.1)0.9 a\n 0.05 E'

Eo A0 0.10 E' Eo A0 0.15 E' Eo A0 0.20 E' Eo A0 0.25 E' Eo A0 0.30 E' Eo A0 0.35 E' Eo A, 0.40 E' Eo A0 0.45 E'

Eo A0 0.50 E'

Eo A0

3.5

10 4.2577 0.3350 6.1009 2.5564 0.4500 4.8042 0.6827 0.5405 4.1626 1.6406 0.6174 3.7616 1.4514 0.6844 3.4811 1.3256 0.7430 3.2756 1.2374 0.7941 3.1163 1.1732 0.8381 2.9903 1.1256 0.8756 2.8885 1.0899 0.9068 2.8054

15 4.0063 0.3600 5.5303 2.4529 0.4722 4.4762 0.6892 0.5601 2.9403 1.6033 0.6345 3.6006 1.4259 0.6990 3.3612 1.3077 0.7553 3.1824 1.2245 0.8043 3.0429 1.1639 0.8463 2.9335 1.1189 0.8820 2.8442 1.0851 0.9118 2.7695

20 3.8912 0.3720 5.2857 2.4052 0.4828 4.3337 0.6924 0.5693 3.8429 1.5860 0.6425 3.5278 1.4141 0.7059 3.3066 1.2994 0.7611 3.1402 1.2185 0.8090 3.0110 1.1596 0.8501 2.9077 1.1158 0.8850 2.8240 1.0829 0.9140 2.7541

25 3.8252 0.3790 5.1514 2.3778 0.4889 4.2535 0.6943 0.5747 3.7875 1.5760 0.6471 3.4873 1.4073 0.7098 3.2757 1.2946 0.7645 3.1150 1.2150 0.8117 2.9922 1.1571 0.8523 2.8930 1.1140 0.8867 2.8125 1.0816 0.9153 2.7461

30 3.7825 0.3836 5.0656 2.3600 0.4929 4.2022 0.6955 0.5782 3.7520 1.5695 0.6502 3.4612 1.4029 0.7124 3.2557 1.2914 0.7666 3.0997 1.2128 0.8135 2.9801 1.1555 0.8538 2.8835 1.1128 0.8878 2.8050 1.0808 0.9162 2.7403

35 3.7525 0.3868 5.0063 2.3475 0.4958 4.2657 0.6964 0.5807 3.7272 1.5649 0.6523 3.4430 1.3998 0.7134 3.2417 1.2892 0.7682 3.0890 1.2112 0.8148 2.9716 1.1543 0.8548 2.8768 1.1120 0.8887 2.7979 1.0802 0.9168 2.7362

40 3.7304 0.3893 4.9629 2.3383 0.4979 4.1397 0.6971 0.5825 2.7090 1.5616 0.6539 3.4295 1.3975 0.7156 3.2314 1.2876 0.7693 3.0811 1.2100 0.8157 2.9653 1.1535 0.8555 2.8719 1.1114 0.8892 2.7941 1.0797 0.9172 2,7332

An Alternative Approach to Preliminary Test Estimation

3.5.1 Introduction In the study of the properties of the preliminary test estimators, we note that the optimum preliminary test estimators have basically two characteristics:

(1) They depend heavily on the level of significance (generally a 2 0.20), which is a nonstandard choice of level of significance. (2) They provide only two choices for the estimator, namely the restricted estimator or the unrestricted estimator on the result of the preliminary test.

3.5. An Alternative Approach

77

To eliminate the dependency on the level of significance and the extreme choices for the estimator, we devise an alternative estimator that will shrink toward a targeted prior value of the parameter under consideration and provide interpolated values of the estimator between the extreme choices. The interpolated values depend on the value (not the test result) of the test-statistic. The targeted value of the parameter is called the natural origin. This approach was introduced by Rodrigues (1987) and elaborated on by Bolfarine and Zacks (1991) in connection with the prediction problem in finite populations. We elaborate the methodology in detail in our context. We first consider the one-sample problem of Section 3.4 to introduce the idea of such an alternative estimator.

3.5.2 One-Sample Problem Consider the one-sample model of Section 3.4, which is

Y = 81,

-

+ e,

(3.5.1)

where e Nn(O,0~1,). It is suspected that 8 may equal 80 (natural origin). So we wish to estimate 6 based on a random sample (yl,... , y n ) and the uncertain prior value 80. Then we have the preliminary test estimator of 8 given by (3.4.6) based on 80, the unrestricted estimator, 6, and the test-statistic L,, as

p T = g-n where

-

(6 - 8o)l(Ln < ~ l , m ( a ) ) ;( m = R. - 11,

(3.5.2)

C, is the test-statistic given by (3.4.5) (3.5.3)

F1,n(a) is the upper a-level critical value from a central F-distribution with (1,m) d.f. The mean-square error of 6cT is

while the mean-square error of is M l ( 8 , ) = u 2 / n . The expression (3.5.4) depends heavily on the level of significance. Thus applying the maxmin rule given a t (3.4.13), we resolve the problem of choosing optimum a. To avoid the dependency of the estimator on a , we devise the following shrinkage estimator(SE) based on 80 and the sample (y1,. . . ,y,)

78


via On and C,: (3.5.5a) (3.5.5b) (3.5.5c) where c is an appropriate nonnegative bounded constant. This estimator was introduced by Khan and Saleh (2001). Notice that 0: is similar to 6," (see 3.5.2) where we have replaced the indicator function I ( & < F~,,(cY))by a continuous decreasing function c1Ci/2j-1 of L,. Thus, instead of two extreme choices, namely 60 or 6,, 0: provides the choice for all values between 80 and depending on the value of C, for a given sample. 6: is a smooth version of OrT,where 62 4 On and 6rT -+ 8, as C, + 0;) and 62 + $0 when 1LA'2j-1 -+ c-l, while 6rT -+ 80 for small values of C,. The bias of 6: is seen to be -

A

en

where 2

-

(3.5.6)

N ( A , 1) with A = fi(e - do)/.

Theorem 1. If 2

-

N ( A , l), then

[ti,1

E - =1-2@(-A),

(3.5.7)

where @ is the cdf of the standard normal distribution. The proof is straightforward (see the problems a t the end of the chapter; see also Rodrigues, 1987, and Bolfarine and Zacks, 1991). Hence the bias expression (3.5.6) becomes C(3

--Kn(2@(A) n

where K, =

- 1)'

The quadratic bias function of

B'&)

6: = {:c "

is further given by

2@(A)- 1)

2

.

(3.5.8)

The quadratic bias expression for the P T E is

(3.59)


79

which is dependent on CY. B4(6:) is a nondecreasing function of A2 with initial value zero and increases monotonically to c2K2 as A2 m. On the other hand, B3(irT)is a function of ((.,A) and, for fixed C Y , begins with the initial value zero and increases to a maximum a t some point A = Ao; then it decreases gradually to zero again. Thus, except for small and large values of A, the bias of 6; is smaller than the bias of 6,"'. Next, we consider the mean-square error of 6:, which is .--)

(3.5.10)

To evaluate (3.5.11), we use the following theorem:

Theorem 2. If 2

-

E ( [ Z [=) Proof.

c

n/(A, l ) , then e-A2/2

+ A{2@(A)- l}.

(3.5.12)

The pdf of 1 2 1 is (3.5.13)

=

zQ(z)dz

=

e-A2/2

+

1;

z+(z)dz

+ A{2@(A)- I}

+ A{2@(A)- l}.

(3.5.14)

Hence, the mse of 6: is given by (35.15)


80 The value of c that minimizes

M4(@)

is

(3.5.16)

7

which depends on A2. To make c* independent of A2, we choose c as

co = K , E .

(3.5.17)

Thus, the optimum value of M 4 ( i z ) reduces t o

(3.5.18)

3.5.3 Comparison of PTE, .:8

and SE

6,

The relative efficiency of 0: compared t o

2 { -~::[2

RE@: : 6,) = 1 while that of

8:

is given by

e-A2/2 -

7f

(3.5.19)

6,PT is given by

R E ( 8 c T :e n ) = [1 -G3,m(3F1,m(a);A2) 1 -1

+A2{ 2G3,m(3F~,m(a);A2) 1 - G S , ~ ( $ F I , ~ ( ~ ) ; A ~ .) } ](3.5.20) Under the null hypothesis Ho : 6 = 60, A2 = 0; hence,

[

1-1

R E ( @ : 6,) = 1 - ZK: ~

21,

(3.5.21)

while 1

RE(dcT : 6,) = [1 -G3,m(5F1,m(a);0)]-1 2 1

(3.5.22)

depending on the size of a. As A2 -+ co,

RE@

: 6,)

-+

[1+ ;K 2 :]

-1

(3.5.23)

and R E ( d r T : 6,)

-+

1.

5

(3.5.24) 1

In general, RE(@ : 6,) decreases from [I - K:]at A2 = 0 and crosses the 1-line at A2 = In 4 (= 1.38) then drops to the minimum value [l ZK:]-' ( 5

+


81

+

1) as A2 + m. The loss of efficiency is 1 - [l :K:]-', while the gain in efficiency is [I - ZKi1-l. Thus, for 0 5 A2 5 ln4, 62 performs better than otherwise, 8, performs better outside this interval. In the case of 6fT, the graph of RE(6rT; decreases from (3.5.22) t o a minimum at A = A*, and then increases to 1 as A --t m. It performs better than in the range 0 5 A2 5 1. Thus, the range of better performance of 6: compared to OFT is increased by an amount 0.38. The following Table 3.5.1 gives the performance of 0; and if* (for each selected level of significance and sample sizes, n).

en;

an

1

en)

Table 3.5.1 Maximum and Minimum Efficiencies of SE and Efficiency of PTE at A0 for Selected a a\n 10 15 20 25 30 35 40 Em"" 2.6261 2.6692 2.6903 2.7029 2.7112 1.7171 1.7215 Emin 0.6176 0.6152 0.6141 0.6135 0.6131 0.6128 0.6125 0.05 En, 0.6408 0.6466 0.6498 0.6518 0.6532 0.6542 0.6550 Eo 0.3350 0.3600 0.3720 0.3790 0.3836 0.3868 0.3893 A0 6.1009 5.5303 5.2857 5.1514 5.0656 5.0063 4.9629 0.15 EA, 0.6827 0.6892 0.6924 0.6943 0.6955 0.6964 0.6971 Eo 0.5405 0.5601 0.5693 0.5747 0.5782 0.5807 0.5825 4,1626 2.9403 3,8429 3,7875 3.7520 3.7272 2.7090 A0 0.25 E A ~0.7133 0.7182 9.7206 0.7220 0.7229 0.7236 0.7241 Eo 0.6844 0.6990 0.7059 0.7098 0.7124 0.7134 0.7156 A0 3.4811 3.3612 3.3066 3.2757 3.2557 3.2417 3.2314 0.35 EA, 0.7361 0.7395 0.7410 0.7420 0.7426 0.7420 9.7433 Eo 0.7941 0.8043 0.8090 0.8117 0.8135 0.8148 0.8157 A0 3.1163 3.0429 3.0110 2.9922 2.9801 2.9716 2.9653 0.45 EA, 0.7536 0.7555 0.7564 0.7569 0.7572 0.7576 0.7578 Eo 0.8756 0.8820 0.8850 0.8867 0.8878 0.8887 0.8892 A0 2.8885 2.8442 2.8240 2.8125 2.8050 2.7979 2.7941

The first two rows of the table give the maximum and minimum relative efficiency of SE for selected sample sizes. The maximum relative efficiency of SE increases as the sample size n increases, and as n m, it tends to r/r - 2 while the minimum efficiency decreases as n increases. Finally, as n -+ ca, it 1 tends to .rr/r+ 2. The maximum relative efficiency at A' = 0 is [l - $K:]- , which varies depending on the sample sizes. The remaining rows of the Table 3.5.1 contain the minimum relative efficiency, Eo of the P T E at A = A,, which is recorded for each (Y = 0.05(0.10)0.45 with the corresponding efficiency ( E n o )of SE for A = A,. The graph of the relative efficiency of SE against A, is plotted in Figure 3.5.1 along with the graph of relative efficiency of P T E showing the relative positions of the efficiency over the Ao-values. From Figure 3.5.1 it is clear that the relative efficiency of PTE compared t o UE is smaller for A > 1 and that of SE relative to UE is smaller for A > 1.38. Thus, SE dominates UE over the wider interval [0, 1.381 relative to the PTE -+


82

:1

RE of SE and PTE a:

\

a = 0.15

D

0

I

I

10

30

20

A?

40

Figure 3.5.1 Graph of the relative efficiency of SE and PTE for different values of a in the interval [0, 11. Also from Figure 3.5.1 we see that SE has higher relative efficiencythan PTE over all A at which efficiency of PTE is smallest.

In general, there is no uniform domination of SE over the PTE for all A and every a. When A is near the origin, SE over performs PTE for some a. Also, at A = 0, the relative efficiency of SE is much higher than that of the PTE except for small a. Further, SE is independent of a. Thus, considering the overall performance of SE relative to PTE,SE is preferable to PTE because it produces interpolated estimators that are free of the level of significance, a.

3.5.4 Simple Linear Model and Shrinkage Estimation Consider the simple model (3.1.1) again,

Y = 81,

+ px + e,

e

- N,(o,2 1 ~ ) ~

(3.5.25)

and the estimator of (0, p) as given by (3.1.2) and that for testing given by (3.1.6). These estimators are repeated here as

(k)

=

(

P - pnx

&[X’Y - i(l;x)(l;x)]

)

p = ,& as (3.5.26)

and

(3.5.27)

83

3.5. An Alternative Approach where Q = x’x - i(1;x)’ and s: = ( n - 2 ) - l ( Y

-

e,ln- PnX)’(Y- e,l,

- JnX).

(3.5.28)

Consider the P T E and SE of ,B following Section 3.5.1 as

finPT

- Pn - ( P n - P ~ ) I ( L

< ~ l , m ( a ) ) , rn = R. - 2,

(3.5.29)

and

(3.5.30) respectively, where c is the nonnegative shrinking constant. Similarly, the PTE and SE of 8 are given by =

e;T

e, + (Pn- po)zr(cn< F1,m(cy)),

rn = n - 2,

(3.5.31)

and

(3.5.32)

(e;,

respectively. The shrinkage estimators fi:) have been studied by Khan, Hoque, and Saleh (2002). The bias expressions for the SE of /3 and 8 may be obtained as C

a

b 4 ( f i 3 = --E(se)E

where Z

-

(3.5.33)

N ( A , 1) with A = (P - , B O ) Q ~ / ~and / Owhere (3.5.34)

r(d) G&. 2

with K , = Similarly the mse expression for the SE of are, respectively, given by

p and 8


84 and

M4(6:) = E [ ( 6 , - 8)2]

C2P2 + -E(se) + 2-E(se)E C

2

a

Q

(3.5.36) using the optimum value of c as before. The relative efficiency of $2 with respect to are

p and of 02 with respect to 6,

RE(^: : pn) = [I - 7r

- l)] - l

(3.5.37)

and

en)= [I

R E ( ~ , :S

2

-

;K:-

9 1

-1

(; + -ii-) Q z2

(2

e-a2/2 -

i)]

-1

,

(3.5.38)

respectively. If A2 = 0,

2

RE@: : Pn) = [l - ;K;]

-1

(21)

(3.5.39)

and

RE(8f : 6,)

=

2 22 1 f 2 [ l - --K;--(7r Q n+Q

depending on the value of

(3.5.40)

z2/Q.On the other hand, as A2 + o,

RE@:

:

6,)

= [l

+ ;Kz] 2

-1

( 5 1)

(3.5.41)

and

RE(6f:in)=

(3.5.42)

Note that both relative efficiencies are decreasing functions of A2 with the highest efficiency at A2 = 0. Some values of the efficiency RE(@ : 6,) are given in Table 3.5.2 for chosen values of the coefficient of variation of the z-values. The efficiency RE(,@ : is available in Table 3.5.1, since it is similar to the one-sample problem.

P,)

85


Table 3.5.2 Minimum and Maximum Relative Efficiencies of SE and PTE for n = 8, a = 0.05(0.10)0.45 and = 1(0.5)5

a\?'lQ E"""

Emin

0.05 En,

Eo A0 0.10 Ea, Eo A0 0.15 En, Eo A0 0.20 Ea, Eo A0 0.25 EA, Eo A0 0.30 En, Eo A0 0.35 EA, Eo A0 0.40 Eno Eo A0 0.45 Ea, Eo A0 0.50 Ea, Eo

A0

5

1 1.5 2 2.5 3 3.5 4 4.5 5 2.08696 2.17801 2.22951 2.26263 2.28571 2.30273 2.31579 2.32613 2.33452 0.65756 0.64901 0.64457 0.64186 0.64002 0.63870 0.63770 0.63692 0.63630 0.67101 0.66262 0.65827 0.65560 0.65380 0.65250 0.65152 0.65076 0.65014 0.32597 0.31773 0.31353 0.31099 0.30929 0.30806 0.30714 0.30643 0.30585 7.05937 7.05937 7.05937 7.05937 7.05937 7.05937 7.05937 7.05937 7.05937 0.69048 0.68236 0.67814 0.67555 0.67380 0.67254 0.67159 0.67084 0.67024 0.44787 0.43855 0.43378 0.43087 0.42892 0.42752 0.42646 0.42564 0.42497 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 5.32801 0.70845 0.70060 0.69651 0.69400 0.69231 0.69108 0.69016 0.68944 0.68886 0.54227 0.53289 0.52805 0.52510 0.52311 0.52168 0.52061 0.51976 0.51909 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 4.50883 0.72438 0.71678 0.71282 0.71040 0.70875 0.70757 0.70667 0.70597 0.70541 0.62136 0.61244 0.60782 0.60499 0.60309 0.60172 0.60068 0.59987 0.59922 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 4.00897 0.74109 0.73378 0.72997 0.72763 0.72605 0.72491 0.72405 0.72337 0.72283 0.70912 0.70127 0.69719 0.69469 0.69299 0.69177 0.69085 0.69013 0.68955 3.60838 3.60838 3.60838 3.60838 3.60838 3.60838 3.60838 3.60838 3.60838 0.75070 0.74358 0.73986 0.73758 0.73603 0.73492 0.73407 0.73341 0.73288 0.74827 0.74109 0.73735 0.73505 0.73349 0.73237 0.73152 0.73086 0.73033 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 3.41621 0.76154 0.75461 0.75100 0.74878 0.74728 0.74620 0.74538 0.74474 0.74422 0.7991 I 0.79298 0.78978 0.78781 0.78647 0.78551 0.78478 0.78421 0.78375 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 3.22498 0.77106 0.76432 0.76081 0.75865 0.75719 0.75613 0.75533 0.75471 0.75421 0.84262 0.83755 0.83489 0.83326 0.83215 0.83134 0.83074 0.83026 0.82988 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 3.07468 0.77941 0.77286 0.76943 0.76732 0.76590 0.76487 0.76409 0.76348 0.76299 0.87938 0.87532 0.87318 0.87187 0.87097 0.87033 0.86984 0.86946 0.86915 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 2.95417 0.78673 0.78033 0.77699 0.77493 0.77354 0.77253 0.77177 0.77118 0.77070 0.90992 0.90677 0.90512 0.90410 0.90340 0.90290 0.90252 0.90223 0.90199 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621 2.85621

Note: 01 is the level of significance, Em"" is the maximum efficiency of SE, Emin is the minimum efficiency of SE, EOis the minimum efficiency of PTE, Eno is the minimum efficiency of SE a t A,, and A0 is the value of A a t which the minimum efficiency of PTE occurs.


86

The Two-Sample Problem and Shrinkage Estimation Consider the linear method (3.5.25), where x = (0,. . ,0; 1 , . . . , 1)' is an n x 1 3.5.5

vector with n1 zero's and 122 1's such that n = n1 +n2. Also, e and in this case, n2

-

x=-

nl

71.1732

+ n 2 Q = -n1+ n2

For this model, Y = 61,

and

+ ,Ox+ e ,

,LL~=

6

and

'5

--

Q

- N,(O,

n2

ni (ni + n2) '

u21n),

(3.5.43)

p2 = 6 + P

(3.5.44)

fi2 = g2.

(3.5.45)

and p2 as

with the estimators of

= $3

fi1

and

The unbiased estimator of n2 is s;, where

+

s; = (ni n2 - 2)-l{

ni

n

j=1

j=nl+l

C(yj- 3il2 + C

(yj - 3212

1.

(3.5.46)

For the test of the hypothesis Ho,

HO : PI = PZ against PI # PZ we consider the statistic

L,

=

n1n2 n1 +n2

(fi2 -

~

s;

fid2

(3.5.47)

(3.5.48)

Now the shrinkage estimator of p1 may be written as

(3.5.49)

+

+

where f i n = nlfil n2fi2/n1 n2. The bias of jif is given by

-

=

-cod-

1 + -K, n1 1

722

(2@(A)- l),

where 2 N(A, 1) with A2 = nlnz(p2 - pl)'/(nl Similarly, the mse of f i f is then calculated as

M(b3 = E"+s

-Pd2]

+ n2)02.

(3.5.50)


with the optimum c-value as m

87

K

, given earlier and K , is defined by (3.5.52)

Hence, the R E ( b f ; f i l )is

(3.5.53)

RE($;

$1)

is a decreasing function of A2. At A2 = 0 ,

while for A2 -+ 00,

(3.5.55) It is known from Section 3.3 that the relative efficiency of jirT is

see (3.3.10b). The maximum value of RE(jifT;b1) is

and the minimum value for each (Y occurs a t A2 = A: with the value [l + g(A;)]-' and A; depending on the size of a. Table 3.5.3 provides the relative efficiencies of SE and PTE for each (Y at A2 = A: to show that SE out forms the PTE for many a-values. Thus, SE is a preferred estimator of p1.


88

for a = .05(.10).45 and for selected samples ~ \ ( n ina) ,

Em,, Emin .05 EA, Eo

.I0 .I5

.20

A0 EA, Eo A0

EA, Eo

A0 EA,

Eo A0 .25

.30

.35

.40

.45

I

.50

Ea, Eo A0 Ea, Eo A0 EA, Eo A0 EA, Eo A0 EA,

Eo A, Ea, Eo

Ao-

(4,4) 1.4144 0.7734 0.7838 0.4623 7.0594 0.7986 0.5905 5.3280 0.8120 0.6781 4.5088 0.8237 0.7447 4.0090 0.8338 0.7978 3.5667 0.8426 0.8409 3.4162 0.8502 0.8761 3.2250 0.8569 0.9049 3.0747 0.8627 0.9284 2.9542 0.8677 0.9472 2.8562

(43) 1.6771 0.7124 0.7341 0.4380 5.9338 0.7535 0.5572 4.7087 0.7694 0.6434 4.0983 0.7826 0.7119 3.7150 0.7939 0.7883 3.4474 0.8034 0.8154 3.2488 0.8117 0.8548 3.0955 0.8189 0.8877 2.9739 0.8251 0.9148 2.8758 0.8304 0.9369 2.7956

(4,121 1.8544 0.6846 0.7129 0.4286 5.5302 0.7340 0.5440 4.4767 0.7506 0.6293 3.9409 0.7643 0.6983 3.6005 0.7757 0.7559 3.3611 0.7855 0.8045 3.1824 0.7939 0.8456 3.0438 0.8012 0.8802 2.9335 0.8075 0.9088 2.8442 0.8129 0.9323 2.7710

(416) ( w 6 ) 1.9816 1.4770 0.6688 0.7559 0.7012 0.7870 0.4234 0.5474 5.3232 5.0520 0.7230 0.8041 0.5309 0.6538 4.3554 4.1941 0.7400 0.8172 0.6216 0.7269 3.8578 3.7462 0.7538 0.8276 0.6908 0.7830 3.5397 3.4574 0.7654 0.8362 0.7490 0.8279 3.3149 3.2525 0.7752 0.8433 0.7985 0.8644 3.1467 3.0983 0.7837 0.8494 0.8405 0.8944 3.0159 2.9780 0.7909 0.8546 0.8759 0.9189 2.9117 2.8819 0.7972 0.8591 0.9055 0.9389 2.8270 2.8037 0.8027 0.8630 0.9297 0.9550 2.7577 2.7394

~

1

Note: a is the level of significance, n1 and n2 are the sample sizes, Em,, is the maximum efficiency of SE, Emin is the minimum efficiency of SE, Eo is the minimum efficiency of PTE, EA, is the minimum efficiency of SE at Ao, A0 is the value of A a t which the minimum efficiency of P T E occurs.

3.6

Estimation with Non-Normal Errors

In this section we consider the properties of the unrestricted, restricted, and preliminary test and shrinkage estimators in a simple linear model when the components of the error vector e = ( e l , . . . ,en)’ in (3.2.1) are

3.6. Estimation with Nonnormal Errors

89

independent and

E ( e ) = 0 and E(ee’) = 021n and the distribution of e is nonnormal, say, F ( e ) =

3.6.1

n,”=,Fo(ej).

(3.6.1)

Unrestricted, Restricted, Preliminary Test and Shrinkage Estimators, and the Test of Slope

Following the notations of Section 3.1 and 3.2, the unrestricted and the restricted estimators of 6 and p are

6 and

-y - ,&I,

pn = $

6, . I . [

P = PO)

= ?j - POT (when

- ;(l:x)(l;x) 1

1

.

(3.6.2)

As for the test of HO : P = PO,the test-statistic L, as in (3.1.6) may be used (3.6.3) Therefore, the preliminary test estimator 6zT is simply,

iy

=

e, - (6, - ijn)l(Ln
5 m+2 That is, @O(C,) is closer to the minimizing value than l / ( m shown in Figure 3.9.1, it is clear that for each A’ and L,,

(3.9.37)

+ 2). So, as

(3.9.38) It follows that 4s(L,)msi dominates m.$/(m + 2) uniformly in A2. Further, it is clear that the PTE defined in (3.9.27) equals 6:”l

whenever F I , ~ ( C=Y )

3.10. Nonparametric Methods: R-Estimation

+

109

2). This may be obtained by minimizing the mse of 6:[pT1for varying Fl+(a). Thus, a* = 1 - Gl,,(m/(m 2);O) is the optimum level of significance for the PTE, 8,ZIPTI .

m(m

3.10

+

Nonparametric Methods: R-Estimation

In this section we consider nonparametric methods toward R-estimation of the intercept and the slope parameters of the simple linear model when it is suspected that the slope parameter may assume a pre-specified value. This procedure broadens the class of distributions and a t the same time achieve robustness of the estimators against contamination in the data set. R-estimation of a location parameter after a preliminary test on regression is due to Saleh and Sen (1978), and estimation of a location parameter after a preliminary test is due to Tamura (1967) based on U-statistics. We invoke asymptotic theory to develop the properties of the estimators we discuss. In the following sections we state the model, assumptions, and test of hypothesis, and define the estimators we consider.

3.10.1 Model and Assumptions Let y1, . . . ,y n be the response variables corresponding t o preassigned values x1, . . . ,2, satisfying the linear model y,=B+px,+e,,

j=l,.-.,n.

(3.10.1)

The distribution function (d.f.) F(e) of the errors belong t o a class, .T, of symmetric absolutely continuous distribution function (cdf) with absolutely continuous density function (pdf), f (e) such that the Fisher's information is finite. (3.10.2) In addition, the covariates (i) l i m , - , m ~ , (ii)

= 20,

+

= QO

X I ,. . .

,x,

satisfy the following conditions:

IzOl < co. < m, Q, = c:="=,xi

(iii) The 2's are bounded and maxi

(3.10.3a) - z,)~.

4

0 as n

(3.10.3b) -+

00.

(3.10.3~)

The linear model (10.3.1) together with the assumptions (3.10.2) through (3.10.3a,b,c) is the basis of further investigations on test of the hypothesis B = 0,p = 0 or ,d = 0, and the estimate of ( 6 , p ) using the sample { ( x l ? Y l ) ? - .,(zn,Yn)). -


110

3.10.2

Test of Hypothesis

In nonparametric methods of statistics, one first develops test-statistic for testing the parameters of interest and in the second stage uses the test-statistic to define the estimators of the parameter by inversion technique. In the model (3.10.1) under consideration we consider the test of hypothesis, HO : 6' = 0,p = 0 against the alternatives that 8 # 0,p # 0 simultaneously. As such we consider the test-statistics well known in the literature (e.g., see Adichie, 1967; Hodges and Lehman, 1963; Puri and Sen, 1986) as follows: Let 4 = { & ( u )0, < u < 1) be a nondecreasing, skew-symmetric (i.e., 4(u) 4(1 - 21) = 0, 0 < u < 1) and square integrable score function f = { ~ * ( P L= ) 4((1 u ) / 2 ) ,0 < u < l}, and for every n(> l), let

+

+

or

where Unl < . - . < U,, are the ordered r.v's of a sample of size n from the rectangular cdf on (0.1). Finally, let yn = (yl,... ,y,)' and for every real ( a , b ) , and define y n ( a , b ) = yn - a l , - bx,, where 1, = (1,e.e ,1) and x, = (21, . . . ,z,)'. Consider then the statistics

istherankofyi-a-bzi (or Iyi-a-bsiI) among whereRni(a,b) (orR,+i(a,b)) y i - u - b z l , - . - ,yn-a-bz, (or ( y ~ - a - b s l j , . . . , /y,-a-bz,I),for i = I , . . . ,n. Note that R,i(a,b) = R,i(O,b) for every real a, and hence, R,i(a,b) does not depend on a. We write it as L,(b); also, we write R,i(O,O) = R,i for i = 1,... ,n* Note that for every given y n and b, T,(a,b), is \ in a : -00 < a < 00, and for every given y,, L,(b) is \ in b : -00 < b < 00. Also, if in the model (3.10.1) we let 8 = p = 0, then T,(O,O) and L,(O) both (marginally) have distributions symmetric about 0. Thus for the one-sided test of HO : p = 0 against H A : p > 0, our test consists in accepting or rejecting HO according as L,(O) is < or 2 L,,,, where P{L,(O) 2 L,,,IHo) 5 a , 0 < Q < 1, and Q is the level of significance of the test. Similarly, for the test of HO : 8 = 0 against H A : 8 > 0, the test consists of accepting and rejecting HO according as T,(O, 0) < or T,,,, where PO{Tn(O,O) 2 T,,IHo} F: Q. Since we will be estimating 8 and p when it is suspected that is zero, we concentrate on the test of p that is, Ho : p = 0 against H A : p > 0, using the statistics ILn(0)I or L ~ , ( o Let ).

>

111

3.10. Nonparametric Methods: R-Estimation A$ =

s:

(s,' 4 ( ~ ) d u ) ~ ,

(3.10.5a)

, 0 < 21 < 1,

(3.10.5b)

42(7~)d7 -~

f' F-' u))

+ ( u ) = - f;F-';u))

A$ = I ( f ) =

r(+,4) = s,:

Jt

(3.10.5~)

@(U)~W,

(3.10.5d)

+(71)4(7&)d74

and consider the estimate of A$, by (3.10.6)

Let z, be the upper 0-level critical value from h i ( 0 , l ) . Then

& ~ Q ; ~ / ~ A ; ~+ L ~z ,, , a s n + w .

(3.10.7)

Alternatively, we can write

nQ,1A,2LE,,

--f

x:(cu)

as n

-+ 03,

(3.10.8)

where x:(cr) is the a-level critical value from a central chi-square distribution with 1 d.f. For small samples, L,,,, can be computed by direct enumeration of the null distribution of L,(O) generated by n! equally likely permutation of the ranks R1, . . . ,R,. These results follow from the fact that under the condition (3.10.3a, b, c) and under Ho we have (see Hajek and Sidak, 1967; Hajek, Sidak, and Sen, 1999) (3.10.9) Consequently, lim P{nQ,1A,2L,(0)

n-cc

5 ZIH~} = H ~ ( 0). z;

(3.10.10)

Estimation of Intercept and Slope Parameters We focus now on the estimation of 6 and p based on the test-statistics T,(a, 3.10.3

b)

and L,(b) as follows (see Adichie, 1967; and Puri and Sen, 1986): Let

6i1) = sup{a : T,(a,O) > 0} and 6i2) = inf{a : T,(a,O) < 0 } ,

6n -- q2 & 1 )

(3.10.1l a )

+ &2)),

= sup{a : L,(b)

> 0 } and Bi2)= inf{a : L,(b) < 0 } ,


112

pn -- 21 6;’)

( p+

p),

= sup{a : T,(a,pn)> 0}

en = +(&I)

(3.10.1 1b)

and

6i2’ = inf{a : Tn(a,,8,) < 0},

+ &Z’).

(3.10.1 lc)

Then 6, is a translation-invariant, robust, and consistent estimator of 8 when ,B = 0, while Jn is a similar estimator when ,B is unspecified. For the preliminary test estimator of p and 8, we consider the statistic

C,=~Q,’A,~L;(O)

V

=

x;

(3.10.12)

asn--,m

from (3.10.10) under Ho. Let x?(a)be the a-level upper critical value from a central chi-square distribution with 1 d.f., then the PTE of /? is defined by

BrT =

Pn

- PnI(Cn

<x~(Q.)).

(3.10.13)

The S-estimator of ,B is defined by (3.10.13)

Similarly, the PTE and S-estimator of 8 are given by

enp,

=

Jn - (6, - in)1(.cn< &a))

(3.10.15)

and (3.10.16)

respectively.

3.10.4 Asymptotic Distribution of Various Estimators

and Their ADB and ADMSE Expressions

In order to obtain the asymptotic distributions of various statistics, we will primarily use (3.10.9) and the JureEkova’s (1969) linearity of rank-statistics under Ho : p = 0 as n -i00. The following holds: (i) sup{fipn(n-1/2b) - ~ ~ ( 0n -)1 / 2 b ~ 0 ? ( 4 , 4 )jbl1 ;I IC) 5

+

o

(ii) s u p { h ~ ~ , ( n - ; a n-’/2b) , - T,(o, 0) +n-112(a

+ bZ,)?(.JJ, 4)l; la1 Ik , Ibl I k }

P

+

0

(3.10.17)

where k is a positive constant. Furthermore, under the stated conditions we have


&(en

113

- N(0,

4)) when p # 0. Note that L, is a consistent test against /3 # 0. So, for fixed alternatives, the asymptotic distributions of fi@, - p),&(b:T - 0) and ,/E(,L?:Ip) are all equivalent to N(0,A:/?($, 4)Qo). Similar results hold for &(On (ii)

6 ) , fi(6,“ natives

- 6)

- 6 ) , and

&(I?,” - 6 ) . Thus we consider the class of local alter-

qn) : p(,)

(3.10.19)

= n-”2b

to obtain asymptotic distributions that provide fair comparisons of the properties of the estimators. Thus, Theorem 1. (Saleh and Sen, 1978). Under K(,) and the assumed regularity conditions, as n GO, ----f

(3.10.20)

Qo

)’

Proof. Note that by (3.10.11b), (3.10.9), and (3.10.17), we have fiIp-pI = Op(l), where under {K(,)},n1/2/3 = 6 = O(1). Thus, under {K(,)},&\jI = Op(l). Further under ,B = 0 by (3.10.17) we obtain n’/’L,(O) = n1/’pQ0y($, 4 ) + op(l). Hence, using contiguity of probability measures under {K(,)} and to that of P = 0, we obtain under {K(,)} as n + GO,

[ &@ ] { ( W+,4)Qo) ( - P)

&L,(o)

0

’N2

;A:

1/r2(4,4>Qo 1 / ~ ( $ 4) , 1/~(+,+)

Q0

)I.

(3.10.21)

114


To prove (ii), we consider the hypothesis 0 = ,L? = 0 together with the linearity results based on (3.10.17)

J;IL(o)

=

hBn@r($, 4 ) + o p ( 1 )

(3.10.22)

and

Then we have

Hence, from the contiguity of measures under Kin, : 0 = 0, ,B(n) = n-1/2b, t o those under H,* : 0 = ,L? = 0, we obtain from above that as n -+m,

Finally, (fiT,(O, 0), fiLn(0))' under {KFn,} has the same distribution as (&Tn(O, n-1/26), &Ln(n-1/26)) under H,*. Hence, the asymptotic distribution of (&T,(O, n-1/2S), &Ln(n-1/2b)) is bivariate normal with mean vector by($, 4)(?&,So)'and dispersion matrix A; Diag(1, Q o ) . Thus under {K(,)l as n 00, ( f i m ( 0 , O), h L ( 0 ) ) '

- N2{b('IL,#)(zo,

QO)';

A'$ Diag(1, Qo)l (3.10.25)

The proof of (3.10.20(ii)) follows from (3.10.23) and (3.10.25). Again, by noting (3.10.11a) and (3.10.17), under {K(,l}, we have & ~ ~ T ( ' I L ,$1 = fiTn(0,O) + o p ( l ) ,

and the proof of (3.10.20(iii)) follows. Clearly, L,

3

(3.10.26)

xf(A2)as n -+ M.

Based on Theorem 3.10.1, we obtain the expressions for the asymptotic distributional bias (ADB) and the asymptotic distributional MSE (ADMSE) of various estimators of ,L? and 0 as given by the theorem below:

Theorem 2. Under {K(,)} and assumed regularity conditions, we have the ADB and ADMSE of the estimators of ,L? as n -+ M given in (A), (B), and (C) below. (A) UE,

A

(i) b l ( b , ) = 0 and (ii) M I ( & ) = 0 2 / Q 0with o2 = A$/-y2(+,4), (B) PTE, ,bzT

115


Proof. A(i) and A(ii) are obvious. We prove (B) and (C). The PTE of P may be written as = ,6 - PI(& < x?(a)). Thus, the ADB may be written as

BLT

lim E[&Pnl(Ln < x ? ( ~ ) ) I

n- w

= - n-cc lim E[~P(n)l(nB;Q,1A,2v2(@, 4)

< x?(Q))]

= -SH3(x?((Y); A2)

- --

CCT

m[2+(A)

- 11,

since Z

-

N ( A , 1)

116


The MSE of

8,:

is given by

-

=

-+

n-0;)

E

+ 2-AE[-] z Q0 Qo Izl z + c’ - 2cu2{E[lZl]- A’E[-I}]

0’

Q0

0’

= -[1

C(T2

U’

- 2c-E[IZl]

I4

Q0

=

+ n-oo lim

lim E[n(& -p)’]

=

2

(T’

- -(2ePA2/’

-{1 Qo

-

l)}, by taking c =

The next theorem gives the ADB and ABMSE of the estimators of 6.

Theorem 3. Under { K ( , ) } and the assumed regularity conditions, we have as n + 03, the ADB and ADMSE of the estimators of 6 as follows:

(A) UE,

en

(i) b l ( 8 , )

(B) RE,

=0

and

5).

(ii) M l ( 6 , ) = ~ ‘ ( 1 +

On

(i) b z ( e n ) = 650 and

+

(ii) M 2 ( i n )= (~’(1 $ A 2 )

(C) PTE, 6rT and

(i) b3(6cT) = bZoH3(X:(a);A’)

$)

(ii) M3(6LT) = ~ ’ ( 1 +

-

+%A2{2H3(x:(a); A’) (D) S-estimator, (9

^s

b4(On) -

9H3(x:(a);A2) -

H ~ ( x : ( o IA’)}. );

6: JQO P q A ) - 11

COZ06

(ii) M ~ ( J , s=) 0’ [(I

and

+ $){ 1 - 3 ( 2 e - ~ ’ / 2 - I


117

Proof. A(i) and (ii) are obvious. We consider B(i) and (ii). By the linearity results (3.10.26) and (3.10.22), we have = &(&+p50)+0,(1). Hence, the ADB and ADMSE are given as follows:.

&en

(i) (ii)

lim

n-cc

E[J;;(G,-

= lim E [ & ~ ~ Z O ] = 6 ~ 0 n+cc

lim ~ [ n ( 8-, e)’]

n-ce

= lirn E [ n ( 8 ,- a)’] n-cc

+ n-m lirn

+

E[nB;Z2] 2 lirn E [ n ( 8 ,- 6)pnZo]

+ 8 ) 5;n-cc lim ~ [ n j :+]265: = $ (1 + 8 )- 5;($ + 6’) + 265; 2

(1 - Qo

n-cc

-

=$(1+$)-&$+&;

( + $A2)

= a’ 1

from (3.2.9a).

To prove C(i) and (ii), consider (i)

lirn E[&(6,PT - 6 ) ] = - n-m lim E[&PnZol(Ln

n-cc

< x:(a))]

= 650H3(XT(a);A2) by Theorem 3.10.1;

(ii)

JLir E[n(Bn- 6 ) 2 ] +%; lirn E[np;l(Ln < x:(a))] + 220 lim E[n(fin- 6)p1(Ln< xr(cr))] = o 2 ( 1 + $) Z; lim E [ ~ ~ ; I; nlim E[n(eLT- f?)’] =

n-cc

n-m

n-

03

-

03

= CT’ (1

+ $) - $02H3(x:(a);

+ 26’%;H5(x:(a); = a’(l+

3)

A’)

- Z~’H~(XT((Y A2) );

A’)

- $02H3(x:(a);

A’)

+ @’{2H,(X:(Q.); A’) - Hdx?(a);A’)} = 0 2 { (1 + $) $H~(x:(Q);A’) -

+ $A’

[~H~(x:((Y); A’) - H~(x:(Q);A’)]}.

To prove D(i) and (ii), we consider

118


+(en

First note that - 6,) = fi,8,Z0+op(l), f i L n ( 0 )= fi&QOr($J,$) + o p ( l ) and limn+00 A: = A24,1imn+mi Q , = Qo. Hence, the above reduces to

+ c2c2$

52

-

2c~0

[ @filbnI7(W) n ( &-@)&A+

3

From the result of Theorem 3.10.2, we may draw the same conclusion on the properties of the estimators of ,B and 6 as in Section 3.6.

3.11

Conclusions

In this chapter we introduced the preliminary test estimation of the intercept parameter when the slope parameter has an a priori fixed value, and we compared its performance with the usual estimator under normal theory. In Section 3.5, we introduced an alternative estimator to PTE. Also, we included the Stein estimation of the variance as a PTE of . ' a In Section 3.6, we presented the asymptotic theory when the underlying error distribution is nonnormal. Similarly, two-sample and one-sample problems of estimating the mean were considered in Section 3.7 and 3.8. In this context the question of whether to pool or not t o pool the sample is answered via preliminary test and shrinkage estimators. In Section 3.10, we included the theory of R-estimation of regression parameters and the PTE, together with the S-estimators. The corresponding results when the errors follow a multivariate Student's t-distribution are presented in the problem set together with other results. In this way, a whole array of model assumptions are covered for the estimation of parameters for simple linear models.

3.12

Problems

1. Let Y = 61, I

+ e, e

-

N,(0,a21n). Show that

(a) Bn = (1Lln)-'lnY,

119

3.12. Problems (b) E(8,) = 6 and E(6, - 6)2 = 2). vo - 2

3.12. Problems

121

7. Consider the simple linear model where e

-

Y

= 61,

+ ,Ox + e ,

Mt(O,a,"I,).

(a) Show that the MLE of 6 and ,L? are the same as the least squares estimators, and the unbiased estimator of u," is s: = &(Y 6,l, - & X ) ' ( Y - e,1, - pnx,.

(b) The restricted estimator of 6 is 8, = P - ,&E when it is suspected that ,O may be equal to PO.Find its bias and mse. 8. Suppose vo > 2 and e

-

M t ( 0 , a,"In)for the linear model in Problem 7.

(a) Show that the LR test of the null hypothesis HO : p = 00 versus is given by test-statistic HA :p #

(b) Show that under H o , L, follows the central F-distribution with (1,m) d.f. (rn = n - a), while under H A , it follows the distribution defined by the pdf

g(L,;

vO) =

x(n

- 2)T+(1/2)

r>O

(which is not a noncentral F-distribution) where A*2=

3.

9. Define the preliminary test estimator of the intercept parameter as in (3.2.4) e,PT

= 6, - (6, - 6 , ) I ( L ,

< F1,m(a)).

(a) Show that the bias and MSE expressions are

02 (i + 5). Mz(6,) = c,"(i + $A*2),

(i) b l ( 8 , ) = 0, and M I ( & ) =

(ii) bz(8,) = -6T, and

6 = ,O - PO.

(iii) b3(eET) = -6GFA ( i F l , , ( ~ r ) ; A * ~ )m , = n - 2 , and


122 where

where zo =

F1*m(a)

. and j

m+F1,,(a)'

= 1,2.

(b) Compare the three estimators, namely, 8,, conclude that the performance of 6, and similar to that under the normal theory.

in,and 6zT of 0 and

GET

compared to

8,

is

10. (Two-Sample Problem). Consider the simple linear model in Problem 2, and assume the vector x t o be (0,. . . ,O; 1,. . , 1)' with n1 zero entries and nz entries of 1's. Assume e Mt(O,OzIn), n = n1 nz.

-

(a) Show that the MLE of p1 and tively. (b) Show that the LR test for

+

p2

are f i 1 = 1 1 and

p = p 2 - p1

ji2 = 1 2 ,

respec-

= 0 is given by statistic

L, = n l n Z ( g 2 - & ) 2 (n1 + 4 s ;

.

(c) Show that C, follows the central F-distribution with (1,nz+nz-2) d.f. under HO and under H A : p1 # p1, it follows the distribution whose pdf is defined by g(C,; A*', YO) given in Problem 8(b) with 7LlR.2 (32 - Y d 2 n1n2 , A*z = ___ L, = ___ n1 + n 2 s; n~+n2

(d) Consider the preliminary test estimator of

byT = f i 1 - ( f i 1

p1

b 2

-Pd2

a:

as

- bl)l(Ln < Fl,m(a))

where FI,,(a) is the upper a-level critical value of the central Fdistribution with (1,nl n 2 - 2) d.f. Show that the bias and mse expressions of f i 1 , b1, and brT are given by

+

(i)

bl(ji1)

= 0, B1(fi1) = 0 and Ml(fi1) =

2, a: = $,

YO

> 2.

3.12. Problems

123

where Gp:2i,m (&Fl,,(a); in Problem Sa(iii).

A*2) has the same definition as

11. (Application to Survey Sampling). Let P = (1,. . . ,N } denote a finite population of N units, where N is known. Associated with the kth unit of P , there is a vector (yk,xk)’, k = 1,... , N , assumed to be a random sample from the bivariate normal distribution. Assume that the population P was generated according to the model

Y = 191~+ ,Ox+ e, E ( e z ) = 0, Cov(e) = 0 2 1 ~ ,and e

-

NN(O,O~IN).

N To gain information about the population total = Ckzl Y k , a sample s of n units is selected from P according to some specified plan. Thus, we are using a superpopulation model in survey sampling. Let T = N - s , the

unobserved part of the population. The following predictor is considered:

TG = N T + ~ ( N - n ) ( Z , - IC,)&, -

-

1 where Y , = ; CiEs yi, Zr = ~1 - Cj=, n xj, 5, = stands for the generic estimator. Let

(i)

-

Pn

=

x’y - ;(l;x)(l;y) x’x - i(lnx)2

en = ( A-s2 s,2 =

’

1

, Q = X’X - -(1n~)2, n

1 (Y- e,ln n-1

1

- &X)’(Y -

e,l, - &x).

xi and

,&


124

T, and TzTare

(a) Show that the bias and mse expressions of

(i)

b(?,)

= 0 and b ( f z T = - ( N - n ) p l G ~ ,(~i . F ~ , ~ (A') a ) ;;

and

(b) If, in the model

Y

= 6'1~

+px+e,

e is distributed as M t ( 0 ,a z I ~as ) in Problem 6(a) show that = 0 and b(fLT) = - ( N - n)p1G3,rn(3Fl,m(a);A*2); (2) 1

6) b@,) (ii)

M1(Tn) = N f ( 1 - f)o: [l + Nf(1 -

f)w] and

where the G-functions are defined in Problem Sa(iii). 12. Prove Theorem 4 with respect to the estimate of

p.

13. Refer to Section 3.10 (a) Show that the exact distribution of T,(O,O) and L,(O) is symmetric of 0. (b) Show that the R-estimators 6, and ,& are invariant, robust, and consistent estimators of 6' and p, respectively. 14. Refer to Section 3.10.3. (a) Find the asymptotic efficiency of

fin

and relative to the LSE of

p.

(b) Find the asymptotic efficiency of 6 , and relative to the LSE of 6'.

6,)relative to the LSE

(c) Find the joint asymptotic efficiency of (6n, of (6'7 P).

Chapter 4

Stein-Type Estimation Outline 4.1 Statistical Model, Estimation, and Tests 4.2 Preliminary Test Estimation 4.3 Stein-Type estimators 4.4 Derivation of the Stein-Type Estimators 4.5 Stein-Type Estimation When the Variance Is Unknown 4.6 Stein-Type Estimation: Nonnormal Distributions 4.7 Improving James-Stein Estimator toward Admissible Estimator 4.8 Confidence Set Estimation Based on Stein-Type Estimators 4.9 Nonparametric Methods: R-Estimation 4.10 Conclusions 4.11 Problems

Stein-type estimation (also called shrinkage estimation ) was introduced by Stein(1956) and James and Stein (1961) in the statistical literature. The approach by Stein (1955, 1956) combines “uncertain prior information” on the parameters of interest and the sample observations from a Multiparameter statistical model. It is more sophisticated and improved than its pre-cursor, the preliminary test estimation. The application needs a parameter space with more than two dimensions. As pointed out in Section 3.5, the preliminary test estimation produces (1) only two values of the estimator, namely, the unrestricted estimator and the restricted estimator, and (2) depends heavily on the level of significance of the preliminary test. Stein-type estimators prcduce all possible values in between the unrestricted and restricted estimators depending on the sample values of the test-statistic used for the preliminary test, which shrinks toward the target vector parameter or its estimator. There is a substantial gain in the use of Stein-type estimators as the improvement carries over to the corresponding set-estimation problems. In this chapter we introduce the Stein-type estimation of the mean vector of a standard pvariate normal distribution. The usual maximum likelihood 125

Chapter 4. Stein-Type Estimation

126

estimation and test of hypothesis for the mean vector are presented. These results are then utilized to define (1) preliminary test estimator and (2) Steintype estimators of the mean vector, which includes the James-Stein and the positive rule estimators. Risk difference and empirical Bayes approaches are presented to derive the Stein-type estimators, beginning with the preliminary test approach. Comparisons of the preliminary test and Stein-type estimators are made relative to the maximum likelihood estimator based on the MSE matrices and the weighted risks under quadratic loss functions. Some asymptotic theory is presented for nonnormal distributions together with the R-estimation in a nonparametric setup. For the preliminary test estimators, tables are presented t o determine optimal size of the preliminary test. We also discuss recentered confidence sets with positive-rule Stein estimators for the normal model as well as for the nonnormal model and provide asymptotic domination of the recentered confidence set. Finite sample analysis of the efficiencies of the Stein-type estimators relative to the standard estimators are given along with the graphs of bias and efficiencies. Finally, problems are added to expand the results to multivariate t-distributions in addition to the related results.

4.1 Statistical Model, Estimation, and Tests Consider the simple location model

Y i= 8 + e i ,

i = 1, . . . , n ,

(4.1 .l)

where Yi = (Yli,.. . ,Y,i)’is the ith observation vector, ei = (eli,. . . ,epi)’ is the ith error vector and 6 = (61,.. . ,6p)’ is the vector parameter of the model. The primes mean transposition, as usual. I t is assumed that ei

- N,(O,I,),

i = 1,.. . ,n.

(4.1.2)

Using the information (4.1.1) and (4.1.2), we obtain the maximum likelihood estimator (MLE) of 6 as -

en = P = (F, . . ,Y,)’, *

where

is the sample mean vector and

(3

8 , = Y - N p @,-Ip .

(4.1.3)

(4.1.4)

Clearly, the bias vector, bl(6,) = 0 the quadratic bias, Bl(8,) = 0, and the normalized MSE matrix is Ml(8,) = E ( n ( 8 , - 6 ) ( 8 ,- 6 ) ’ } = I,. Finally, under the quadratic loss function

qe*,e)= nip* - 6&

=46*-

e)’w(e*- 6 )

(4.1.5)

4.1. Statistical hdodel, Estimation, and Tests the weighted risk of

6,

127

is and

Rl(6,; W) = tr[Ml(G,)W],

tr[M1(6,)W] = p if W = I,. (4.1.6) -

Notice that 6, consists of the marginal sample means &, = Y i , i = 1,2, . . . ,p , as its components (which are independent). For the test of the null hypothesis HO : 8 = 80 against H A : 8 # 8 0 , the likelihood ratio test yields the statistic

c,

so112.

(4.1.7)

eO1l2.

(4.1.8)

= rille, -

Under Ho, C, has a central chi-square distribution with p d.f. If 8 # 80, C, has the noncentral chi-square distribution with p d.f. and noncentrality

parameter A2/2, where

a2=

-

The estimator 6 , stands out because of the following optimality properties: it is a (i) (ii) (iii) (iv)

best unbiased estimator, best equivariant estimator, maximum likelihood estimator, minimax estimator.

In this chapter we utilize the expressions of the MSE matrices and risks to evaluate the performance of various estimators. We also consider the MSE and risk based efficiencies. At this stage it is useful t o remind the readers of the definition of inadmissibility of an estimator 8:: An estimator 8; is inadmissible for 8 if there exists an estimator 6 , such that

~ ~ w) ( 65 R2(8:; ~ ; w) for every value of 8 and strict inequality Ri(6,; W) < RZ(8:; W)

(4.1.9)

holds for some 8 E 0. In terms of MSE matrices we then have

for every value of 8 , and for some 8 E 0, strict inequality holds, meaning, (4.1.10) In order to compute the bias vector, quadratic bias, MSE matrix, and the weighted risk of an estimator of the type

8: = 6 n # ( n ~ ~ e n ~ ~ 2 ) ,

-

(4.1.11)

where $Jis a real-valued function and 6, Np(8,n-11p), we use the Stein’s (1956) formula given in the theorem below assuming 80 = 0 w.1.g.

128


Theorem 1. Under the assumed regularity conditions, we have the bias vector and the quadratic bias of 0; given by (i) b ( Q 3= - e E [ l - 4(x;+2(A2)] and (ii) B(@:) = -A2 { E [ 1- 4 ( ~ ; + 2 ( A ~ ) ] } ~A2 , = n//0jj2.

(4.1.12)

The MSE matrix and the weighted risk expressions under the loss function (4.1.5) are given by (iii)

M(@;) = I, E [ ~ ~ ( x ; + ~ ( A+~n) e) ]e ’ ~ [ 4 ~ ( ~ ; + ~ ( ~ 2 ) ) ] -

+

2nee’~[+(~;+~(a~))]

(4.1.13)

and (iv)

R(%;W) = tr(W)E[42(~g+2(A2))] fn@’We{1- 2E[4’(x;+2(A2))]

+E[~’(X;+~(A~))]}

+

= PE[42(~;+2(A2))] A2{ 1 - 2E[42(x;+2(A2))]

+ E[4(x;+4(A2))]}

and

if W = I.

(4.1.14)

4.2. Preliminary Test Estimation

129

This theorem will be used in the sequel to obtain relevant results. Our will be the following: choices of 4(n116n1/2)

and

4.2

Preliminary Test Estimation

One aspect of 6 , is that it ignores any relationship among the components of 6 = (61,. . . ,6,)’. If this question is raised, then the entire estimation procedure may have to be modified. As a start, we consider that the relationship among the components of 6 is defined by the null hypothesis HO : 6 = 0, meaning, 6 lies in the hyperplane defined by 8 = 0. So the restricted estimator of 6 is 6 , = 0. In this case, we consider the preliminary test estimator (PTE) of 6 as

where 0 is the restricted value defined by HOand I ( A ) is the indicator function of the set A and x;(a) is the upper a-level critical value of a central chi-square distribution with p d.f. As stated in Chapter 3, PTE was initiated by Bancroft (1944, 1964, 1965) and extended by Han and Bancroft (1968), among others, in a parametric setup. Saleh and Sen (1978) considered PTE within a nonparametric setup. The PTE of 6 heavily depends on the level of significance, a , of the test PT and the value of en fluctuates between two values, namely 6 , and 0 . N o A

Chapter 4 . Stein-Type Estimation

130

intermediate values between 0 and 6, are available. It is clear that the bias vector of 6, is bl(6,) = 0 with the quadratic bias (QB), = 0. The MSE matrix of 6, is Ml(8,) = I, with the risk Rl(6,; W) = t r ( W ) = p if w = I,.

&(a,)

..PT

..PT

), the quadratic bias (QB) &(On ), the PT MSE matrix Mz(8, ), and the weighted risk RZ(8, ;W) are given by the following expressions, by the results of Theorem 1 of Section 4.1: Similarly, the bias vector b’(8,

-

. PT

1

PT

(i) b ( 8 ,

Bz

) = - 8 H , + z ( x : ( a ) ; A’) and = A’ { H,+Z A’) }’,

(ET)

(X:w;

(4.2- 2 )

where H,(x:(a); A’) is the cdf of a noncentral chi-square distribution with I, d.f. and noncentrality parameter A 2 / 2 where A’ = nl18/j2. (ii)

MZ(~$)

= 1 , ~ -~ ~ + z ( x ; ( aA”>> >;

+ n68’{2H,+z(X;(Q); A2) - H p + 4 ( x 3 a ) ;A’)}.

(4.2.3)

and

&(DflT; =

W) = tr[M2(6ET)W] (1 - Hp+Z(X;(4; A”}

f(nB’W8) {2HP+’(X;(a); A’) - H p + 4 ( x 3 4 ;A”}

,

(4.2.4)

= P { l - Hp+Z(X;(Q);A”}

+A2 ( 2 H p + 2 ( x ; ( a ) A2) ; - H P + 4 ( x ; ( a ) A’)}, ;

if W

= I,.

(4.2.5)

- PT

Note that if A’ = 0, then &(On ) = 0; otherwise, it increases to a maximum and then drops down toward zero as A2 + 00. As regards the MSE matrix, we consider the pth root of the determinant of the matrix, that is,

+ A’ [ ~ H , + z ( x : ( ~A)2;) - H p + 4 ( x ; ( a ) ;A’)]}l/p. (4.2.6) APT l / p - PT Thus, p l M z ( 8 , I) = Rz(6, ;I,) = p { l - H p + ~ ( x ~ ( c y ) ; aOt )A’ } = 0 , and PT

..PT

as A’ diverts from the origin, both pIM2(6, ) / ‘ I p and RZ(8, ;I,) increase to a maximum crossing the pline, and then decrease monotonically toward the ,.PT value p as A2 + 03. See Figure 4.2.1. In general, the graph of pjMz(8, )I1/, PT remains below the graph of R2(8, ;I,) in the middle (0 < A’ < co),since these functions are drawn in the same scale. 6

A


* 0

131

,.* i’

-[

I

Figure 4.2.1 Graphs of Ma = p

1

I

I

I

M2 (*PT)/l’p 8,

I

1

. PT and R2 = R2(B, ; I p )

The following theorem states that there are points in the parameter space where the PTE has risk larger than the MLE 6,. It performs better in a small neighborhood of the origin. The theorem is due t o Scolve, Morris, and Radhakrishnan (1972).

Theorem 1.

(ne’we)> t r ( w ) if (netwe) < tr(W).

R ~ ( L ;W) ~ ;> t r ( W ) if < tr(W) Proof.

(4.2.7)


132

since x;+~is stochastically larger than x;+’. tr(W)

The r.h.s. of (4.2.8) equals

+ [(ne’WO) - tr(W)]Hp+2(xi(a);A’)

> tr(W)

if ( n o r w e ) > t r ( W ) . If W = I,, then

- PT

RZ(6, ;Ip)> p

(4.2.9)

if A’ > p .

(4.2.10)

Now, choose (n6’WB) such that t r ( W ) - 2(n6’W6) > 0. Then

R2(6ET;W) 5 [tr(W)- 2(n6’W6)]{l - Hp+4(x;(a);A2)] +(n6’W6)[1 - HP+4(xi(cy);A’)]

+ (n6’W6),

(4.2.11)

replacing Hp+2(x:(a);A’) by Hp+4(x:(a); A’). The r.h.s. of (4.2.11) after simplification equals tr(W)

+ [tr(W)

-

(n6’W6)]Hp+4(x~(a); A’) < t r ( W )

1 if ( n e ’ w 6 ) < - t r ( W ) 2

< tr(W).

(4.2.12)

. PT

If W = I,, then Rz(6, ; I p ) < p if A’ < i p < p. Since the risk Rl(8,;I,) = p and 8, is minimax, Theorem 4.2.1 states that no such P T E can be minimax. Further, as $ ( a ) --+ 00 (i.e., (Y -+ 0), the risk Rz(e,;I,)at the origin (i.e., A’ = 0) corresponding t o the PTE tends t o be p . That is to say, we can find a P T E that does arbitrarily well a t the origin at the cost of being poor elsewhere. Consider the properties of 6: with respect to 8, based on the MSE criterion. For this consider the difference C’MZ (6;’) C‘ - t ‘ C

(4.2.13)

for a given nonzero pvector C. Thus,

ere{1 - Hp+z(Xi(a); A’)}+ t’n66’t{ 2Hp+2(xi(a);A’) >
A;, then 8, is better. The cut-off point A; may be determined based on the inequality

and if

The efficiency expression allows us to determine the optimum level of significance for minimum guaranteed efficiency, Eo, as the solution a* of the equation min MRE(a, A’) = E ( a ,A,,,in(a)) = EO, A=

(4.2.19)

where Eo is a preassigned efficiency number for the PTE, and MRE(a, A2) -PT stands for MRE(8, ;8,) as a function of a and A2. The efficiency based on the optimum level a* may go up to Em, = (1 - Hp+2(xg(a);O)}-’.


134

Table 4.2.1.Maximum and Minimum Guaranteed MSE Based Efficiencies 8 10 12 14 4 6 P Emax 6.75908 7.89292 8.72360 9.37375 9.90415 10.34988 Emin 0.83930 0.90622 0.93840 0.95653 0.96781 0.97533 12.55403 16.40307 19.66492 22.53881 25.14251 27.51153 An i: Emax 3.92601 4.491 18 4.89829 5.21311 5.46767 5.67991 Emin 0.86685 0.92253 0.94938 0.96451 0.97391 0.98015 Akin 10.62223 14.08485 17.06601 19.72510 22.15249 24.40138

Emax Emin A;in

r

Emin Akin Emax Emin A?nin Emax Emin Aiin

i"

Emin Aiin Emin

Emin Aiin Emin

2.89783 3.26722 3.53047 0.73248 3.89487 4.02965 2.89783 3.26722 3.53047 3.73248 3.89487 4.01965 9.47256 12.67623 15.46918 17.97908 20.28298 22.42997 2.35590 0.90103 8.64957

3.17317 0.98570 20.96061

2.01845 0.91385 8.00872

2.64502 0.98767 19.77572

1.78732 0.92509 7.48503

2.28535 0.93935 18.77429

1.61900 0.93513 7.04355

1.75471 1.84936 1.92081 1.97751 2.02408 0.96168 0.97513 0.98284 0.98764 0.99082 9.58252 11.89341 14.02413 16.01690 17.90156

1.49112 0.94421 6.66328

1.60350 1.68169 1.74059 1.78724 1.82549 0.96682 0.97845 0.98516 0.98935 0.99212 9.07737 11.29477 13.35346 15.28787 17.12444

1.39096 0.95247 6.33049

1.48475 1.54992 1.59894 1.63770 1.66944 0.97151 0.98148 0.98727 0.99089 0.99329 8.62734 10.75802 12.74924 14.62948 16.42136

1.31074 0.96000 6.03573

1.38937 1.44382 1.48480 1.51718 1.54367 0.97581 0.98425 0.98919 0.99229 0.99434 8.22239 10.27088 12.19820 14.02726 15.77741

Table 4.2.1 may be used t o determine the optimum level of significance for the preliminary test (PT) when the MSE based criterion is used. For example, let p = 6, and suppose that we wish t o obtain a PTE with minimum guaranteed efficiency Eo = 0.95. Then Table 4.2.1 gives the value a* = 0.25 at the intersection of (Y = 0.25 and p = 6. Hence, the optimum level for the PT in this case is 0.25 with a maximum possible efficiency 2.22631. . PT Now, consider the relative risk efficiency (RRE) based on the risk of en


135

given by

(4.2.20) APT.-

-PT .

The graph of RRE(0, ,On)is similar to the graph of MRE(0, ;On).The maximum occurs at A’ = 0 with the same value (1 - Hp+2($(a);O)}-’. The to the value efficiency decreases, crossing the 1-line a t min RRE(a, ALin(a))= RR.E(a,ALin(a)),

(4.2.21)

A2

and then increases toward 1 as A2 increases toward -PT

-

03.

Here RRE(a,A2) APT

-

stands for RRE(6 , ; 0,) as a function of a and A’. Clearly, RRE(6 , ; 6,) $1 according as

(4.2.22) The optimum value of a may be obtained by solving the equation

minRRE(a, A’) = E ( a ,Amin(a))= EO A2

(4.2.23)

with a pre-assigned value Eo. Table 4.2.2 facilitates the determination of the optimum level of significance a* based on MSE matrix and risk efficiencies. As an application of Table 4.2.2, let p = 6. We wish to obtain a P T E with a t least 95% efficiency. We find in the crossing of 0.40 and 0.95099 (E0.95) the desired a-value as a* = 0.40 with a maximum efficiency 1.60350 as in the


136 case of MRE.

Table 4.2.2. Maximum and Minimum Guaranteed risk Based Efficiencies

Emin Alin

8 8.72360 0.81707 12.79251

Emin Alin

4.89829 0.87014 11.68294

5.67991 0.93879 17.01821

Emin Akin

4.02965 4.95530 16.26681

Emin Alin

3.17317 0.96591 15.71231 2.64502 0.97339 15.26500 2.28535 0.97897 14.88455

0.35 Emax 1.61900 Emin 0.91005 Akin 5.96289 0.4 Emax 1.49112 Emin 0.92589 Akin 5.76832

1.75471 0.94021 7.79095

1.84936 1.92081 1.97751 2.02408 0.95827 0.96989 0.97776 0.98326 9.54048 11.24068 12.90716 14.54916

1.60350 0.95099 7.56281

1.68169 1.74059 1.78724 1.82549 0.96604 0.97567 0.98215 0.98666 9.28777 10.96826 12.61807 14.24553

0.45 Emax 1.39096 E,in 0.93933 Akin 5.59431

1.48475 0.96002 7.35598

1.54992 1.59894 1.63770 1.66944 0.97245 0.98039 0.98571 0.98938 9.05691 10.71809 12.35159 13.96478

0.5 Emax 1.31074 Emin 0.95079 Akin 5.43611

1.38937 0.96763 7.16522

1.44382 1.48480 1.51718 1.54367 0.97781 0.98430 0.98862 0.99160 8.84227 10.48427 12.10153 13.70053

-

-

4.3 Stein-Type Estimators 4.3.1

Introduction

Preliminary test estimation introduced by Bancroft (1944, 1964, 1965) was an early approach t o the problem of estimation and test of hypothesis in cases where some uncertain prior information exists. We saw in Section 4.2 that the PTE of 8 fails to be minimax, although there is a small neighborhood near the origin where it performs better than the MLE, 6,.

4.3. Stein-Type Estimators

137

However, real breakthroughs came when Stein (1956) proved that there exist estimators that can improve on 8, under the loss function nll6, - ell2. He demonstrated that the estimator (4.3.1)

-

improves on 8,. The genesis of this estimator stems out of a simple consideration: If 8, estimates 8 closely, does it mean 116,112 will estimate l18]j2closely? Clearly,

+

~[nlt~n1121 = nllett2 p .

(4.3.2)

Thus, n\16,112 has missed n118112 by an amount p. To obtain the estimator O:, we will pursue the following geometric approach together with the Pythagorean theorem: Since 6, is an estimator of 8 based on a sample of size n from NP(8,1,), we set

4len - q2= P

(4.3.3a)

nl16nl12= nllelt2 + P.

(4.3.3b)

and Stein showed that instead of 6, the projection of 6 on 6, is a better estimate of 8. Denote this projection by (1 - a)8,. Then, from the geometrical representation given in Figure 4.3.1, we can conclude the following: (i) 1 and

1 ~ 1 =1 ~ nll8, - ell2-

(4.3.4)

= p - na2116,112

= n1/8,/12- p (ii) 11~112= n11e112 - n ( 1 Equating (4.3.4) with (4.3.5), we obtain

-

n(1-

(~)~1j8,11~. (4.3.5) (4.3.6)

Hence, the projection of 8 on

8,

is given by (4.3.1) and repeated here:

(4.3.7) The bias vector and the quadratic bias (QB) of 0: are given by (using Theorem 1)

b(8:) = -pOE (x;:~(A~)) and B(8:) = p2A2{E (x;:,(A2))}', where 1 1

; ! ( 2- ~ l ~ ) ~ 2( + p 2r)-l

E[x,'(A2)] = e-A2/2 r>O

(4.3.8)

138


Figure 4.3.1 Geometrical representation of Stein’s idea In order to compare the performance of 0: relative to 6,’ we consider the

MSE matrix and the weighted risk of 0; given by

M(6:) = I , - & [ 2 E ( ~ , ; 2 2 ( A ~ )-)P E ( x ; ; ~ ( A ~ ) ) ]

+ p(n0e’)[2E(X,;22(A2)) - 2E(x;i4(A2)) - PE(X,;41(A2))]

(4.3.9)

and

R(%;W) = tdM(0:)WI

= tr(w)- p t r ( W ) [2E(xL22(A2)) -~ E ( x L ~ ~ ( A ~ ) ) ]

+P(n@’W0)[2E(X;22 ( A 2 ) -2E(x;i2( ) A’)) - ~ E ( x , (A”,)] S~~ = P - P2 [2E(X,;22(A2)) - PE(X,;2(A2))]

+

+PA2 [2E(X;i2(AZ))- ~ E ( x ; ~ ~ ( A pE(x;;,(A2))] ~))

(4.3.10)

if W = I,. By taking the risk difference R ( 6 n ; I p )- R(O:;Ip),we find that the risk of 0: is smaller than the MLE 6,. Similarly, we consider the pth root of the


139

determinant of M ( 6 : ) :

IM(O:)l’/” =

{ 1 - P[2E(X,;Z2(A2)) - PE(X&(A2))]} { 1 - P[2E(X&(A2)) - p E ( x ; W ) ) ]

(p--l)/p

+ PA’ [~E(x;?~(A’)) - ~E(x;:~(A’)) + ~E(X;~~(A’))])’/~. (4.3.11) Thus, plM(O:)I’/P is smaller than p for all A’. Hence, the MLE 8, is inadmissible according t o the MSE criterion. As A’ + co,p/M(6:)11/p+ p , and for A’ = 0, we obtain plM(O:)l’/p = 2(1 - z)-’. On the other hand, P

co and a t the origin R ( 6 L ; I p )= 2(1 - $)-’ is the same as the MSE case. Except a t the endpoints the graph of plM(6L)lllp remains below the graph of R(6:; I p ) in the interval 0 < A’ < 00.

R ( O l ; I p ) p as A‘ --$

4.3.2

-+

James-Stein Estimator (JSE)

Although 6: dominates the MLE i j R , is it an optimum estimator of 6 ? To answer this question, consider the shrinkage estimator, (4.3.12) -S and determine the optimum constant c from the unweighted risk of O,(c). Now,

= P - 2c

+ ~~A’E(X;:~(A’)) ic’E(x~’(A’)).(4.3.13)

-S

Minimizing R ( 6 , (c);I p ) with respect to -2

c,

we obtain

+ 2 A 2 E ( ~ i $ 2 ( A 2 )+) 2d3(xp2(A2)) = 0

(4.3.14a)

Now, writing A ’ E [ x ~ ~ ~ ( A= ’ ) 1] - (p - ~)E[x,’(A’)], we obtain

-2

+ 2 { 1 - ( p - ~ ) E ( x , ~ ( A ~ )+) }2cE(xF2(A’))

= 0.

(4.3.1413)

This implies that the optimum value of c is Cf

=p-2.

(4.3.14~)


140

Thus, the optimum Stein-type estimator of 8 is given by James and Stein (1961) as (4.3.15) -S

Hence, the estimator 8, is known as the James-Stein estimator (JSE) in the literature. -s Note that if n [ ~ 6 , -+ ~ ~00, 2 then 8, = 6,, which matches with the PTE,

,.PT

8,

. However, if nl/6,/[2-+

-S

0, then 8, becomes a negative estimator. In order to obtain the right value, one has t o restrict n116,112 > p - 2. Stein (1966) defined the following estimator of 8:

(4.3.17) which is known as the positive-rule Stein estimator (PRSE) of 8. The -S

PRSE of 8 is a PTE. It combines 0 and 8, via testing Ho : 8 = 0 based on nl[6,1\2with the critical value p - 2. Since the arrival of JS estimator, there was a tremendous serge of research in Stein-type estimation to fill the void in different directions. See, for example, Baranchik (1970), Berger (1980, 1985), Bock (1988), Brown (1966, 1988), Casella (1989), Cellier, Fourdrinier and Robert (1989), Cohen and Strawderman (1973), Efron and Morris (1973), Saleh and Sen (1978-1986), Hoffmann (1992), Brewster and Zidek (1974), Casella (1985), and Kubokawa (1991, 1994) among others. In the sections that follow we present various techniques to improve the estimator 6,. -S In the sequel we will discuss the MLE 6,, the James-Stein estimator O,, . Si-

and the PRSE, 8, and variations thereof. We have seen that the Stein estimator 8: given in (4.3.1) is a biased estimator. The bias vector and the QB of the James-Stein estimator are b3(6;T) =

and

-(P - 2)8E(X&(A2))

-S

B3(On) = (P - 2)2A2{E.(~F:2(A2))}2,

(4.3.18) (4.3.19)

respectively. -S The MSE matrix and the weighted risk of 8, may be obtained as -S

M 3 ( 8 n ) = I,

-

(P - 2)I,{2JqX;:2(A2))

- (P - 2)E(XpS42(A2)))

+ (p2 - 4 ) n 8 8 ’ ~ ( ~ , ; 4 ~ ( ~ ~ ) )

(4.3.20)

4.3. Stein-TypeEstimators

141

and

R3(6:; W) = tr[M3(6z)W] = tr(W)

+ (P’

-

-

(P - 2 ) t r ( w ) { 2E(x,S22(A2) - ( p - ~)E(x;:~(A~))}

4) (nO’we)E(X& (A’))

= p - (p - 2)2E(~,2(A2))if

W = I,.

(4.3.21)

respectively. -S

The determinant of M3(6,) is given by

{ { 1- ( P

S

IM3(6,>/= 1 - ( P - ~)[~E(X;:~(A’))- ( p - ~)E(X;$,(A’,)]}~-~ x -

W E ( x ; 3 A 2 ) ) - ( P - 2)E(X,-,”2(A2))]

+(P2 - 4)A”(X&(A’))}.

(4.3.22)

-s

-s

At A’ = 0, R3(6,;Ip) = plM3(6n)11/P= 2 and as A’

4 m,

-s R3(6,;Ip)

-s and p[M3(6z)[’/pconverge to p . The graph of plM3(8,)l1/P remains below -S

the graph of R3(6,;IP) when A’ E (0, co).See Figure 4.3.2. -s Consider the comparison of 6 , relative to 6,. For a given nonzero vector t of p components we may evaluate the difference I

t’M3(6z)t- t’t.

(4.3.23)

Note that

s frM3(6,)t = t’t- ( p - 2)t’t{2E(x,;2,(A2))

- (p - ~ ) E ( x ; : ~ ( A ~ ) ) }

+ ( p 2 - 4)t’n66’tE(~;:~(A~)) -S

and t’M3 ( 6 , ) t (p

t’t < 0 according as

+ 2)t’n66’tE (

(A”,)

< t’t{~E(X;?~(A’))} - ( p - 2)E(x&(A2))). This implies

(4.3.24)

(4.3.25)


142

7

Risk of

S

M ; of S

-S

Figure 4.3.2 Graphs of R3 = R3(6:; Ip) and M3f = p[M3(8,)1'/P holds for all A2, which is equivalent t o pE[xP?2(A2))

< ( p + l ) ( p - ~ ) E [ x ; ~ ~ ( A ~for ) ] all A2

(4.3.28)

-S

and may not always hold. Hence, M3(8,) - Ml(6,) is not negative semi-S

definite (n.s.d.). Thus, 8, does not dominate 6, uniformly with respect to the MSE criterion. - s The efficiency of 8, based on the MSE matrix is given by

-s

MRE(8, : 6,) is a decreasing function of A2. At A2 = 0, it assumes the maximum value p/2 and as A2 -+ 00, the efficiency tapers down to unity from above. -S A consideration of the efficiency of 8, relative to 6, based on the risk


143

(W = Ip)expression leads t o

-s -

-) 2

RRE(6,;6,) = [1 - p ( l - P

2

(4.3.30)

E(x;~(A~))]-’.

-s RRE(6,; 6,) is also a decreasing function of A’ with a maximum value p/2 and tends to 1 as A2 + 00 from below. See Figure 4.3.3 for both kinds of

efficiencies. -S Now we consider the risk analysis of 6 , relative to weighted risk difference

6,

for general W. The

R@,; W) - R2(6:;W) = (P - 2)tr(W{2E(x,;2(A2))

- (P - 2)E(X;:2(A2))}

- (p2 - 4 ) n 6 ’ W 6 E ( ~ & ( A ~ ) )

(4.3.31)

2 0 if the matrix W satisfies the condition (4.3.32) -S

Thus, 6 , dominates 6, uniformly for all W , satisfying (4.3.31). Further, as A2 + 00, the risk difference tends to zero from below.

4.3.3 Positive-Rule Stein Estimator (PRSE) ., S+

We have motivated the PRSE, 6 ,

-S

from the James-Stein estimator 6 , so that

..S+

it may have negative coordinates. Clearly, 6 , defined a t (4.3.17) is a PTE. Also, this estimator can never change the sign of its coordinates, and hence the name. First, note that we may write

e:+ = 6:I

(I(nll6,$ > p - 2))

(4.3.33) ., S+

-S

in terms of the James-Stein estimator. Note also that njl6, lj2 5 116n112. The bias vector, QB, MSE matrix, and the risk expressions may be obtained as given below. . S+

Chapter 4 . Stein-Type Estimation

144

x-

MRE of S re1 UE RRE of S re1 UE

7

2-

z

RRE

I

I

Figure 4.3.3 Graphs of MRE

and (iii)

M4 (6;’)

where

I

I

-s

= MR.E(6,;

-

I

I

-s

-

6,) and RRE = RRE(6,; 6,)


145

where + stands for domination under the risk criterion. In passing we mention the properties of the QB of the estimators shown in Figure 4.3.4.


146

Ln

7

-

0

PTE

1

0

5

10

20

15

A2

25

30

. PT Figure 4.3.4 Graph of QB of estimators: PTE = 8, , JSE

,.S+

and PRSE = 8,

For a nonzero vector, l we have

-S

= 8,,

4.3. Stein-TypeEstimators

147

and the quantity in the second big bracket is negative. Thus, the whole expression is positive. Taking max (nl’$B’t/l’l)= A2, we have

E [ ( 1- ( P - 2)x,;22(A2))21(x~+2(A2) < P - 2)] +A2{2E[((p- 2)x&A2)

- 1)I(x&2(A2)

< p - 2)

+ E [ ( 1 - ( P - ~ ) x ~ ~ ~ ( A ’ ) ) ~ I ( x ~= (1 - ( P - 2>[2E(X&(A2)) - ( P - 2)E(X;:2(A2))] E[{1 - ( P - 2 ) x ~ ~ 2 ( A 2 ) } 2 1 ( x ~ + z (< AP2 )- a)]) ( P - - l ) l P x [m4(A2)]-’lP.

(4.3.45)

where

m4(A2) = (1 - ( P - 2) [2E(x;:2(A2))

- (P - ~

)E(x~$~(A~))]

+ (P2 - 4 ) A 2 E ( x 3 A 2 ) ) + A2{ 2 -

w - ( P - 2)x~~2(A2))I(x2,+2(A2) < P - a)])

E [ ( 1- ( P - 2)X,;24(A2))21(X2,+4(A2)< P - 2)]}. (4.3.46) ., S+

Similarly, the risk efficiency of 0, obtained as

relative to

a,,

with W = I, may be

-St*-

RRE(% =

{1

-

ten)

p - ’ ( p - 2)2E[x,2(A2)]

-

E[{1 - ( P - ~ ) X ; ~ ~ ( A ~ ) } ~ ~ ( < X P~-+2)] Z(A~)

+

P

-

E [ ( 1- ( P - ~ ) X ; ? ~ ( A ~ ) ) ~ I ( X < ~ +p ~- (2)]}-l. A~)

-1

2

A {2E[{1- ( P - 2 ) x ~ ~ 2 ( A 2 ) } 1 ( x ~ + z ( < A 2P )- 2)]

,.S+

Table 4.3.1 presents some risk gains of 8,

-S

(4.3.47)

over 8,. Also, see Figure 4.3.5 - -PT - S ..S+ for the risk of the four estimators, namely On,$, ,On, and 8, .


148

7

1 ~ =0.15 1

p=4

of UE, PTE, S andS+

m

PTE

0

I

I

I

5

10

15

20

I

I

25

30

A2

Figure 4.3.5 Graph of the risks of the four estimators

(UE =

en,PTE = 6,.

PT

- S and S + , S= On, -

a;+)

F

0 1 2 3 4 5 6 7 8 9 10 1.35914 1.20103 1.52558 1.08236 1.05536 1.03769 1.02583 1.01776 1.01222 1.00841 1.00578 1.47781 1.26169 1.16530 1.11089 1.07668 1.05389 1.03822 1.02724 1.01945 1.01390 1.00993 1.54504 1.29510 1.18779 1.12778 1.08993 1.06447 1.04672 1.03405 1.02490 1.01823 1.01335 10 1.59024 1.31722 1.20302 1.13959 1.09951 1.07239 1.05329 1.03949 1.02938 1.02190 1.01634 12 1.62348 1.33333 1.21430 1.14855 1.10698 1.07872 1.05866 1.04405 1.03322 1.02512 1.01901

4.3.4 Sclove-Morris-RadhakrishnanModifications ..S+ It is interesting to note that the PRSE, 6,

-s+

0,

has two representations, namely

-s

= ~,I(n11~,1I2> P - 21,

(4.3.48a)

which is a PTE based on 62 and 0 with the critical value ( p - 2), and As+

6,

*PT

= 8,

I1- ( P - 2 > ( ~ l l ~ n I I ) - 1 1 ,

(4.3.48b)

which is a Stein-type estimator where 6, is replaced by

eLPT= e,I(nIien112 >

-

21,

(4.3.48~)


0

0

149

I

I

I

5

10

15

1

20

A2

Figure 4.3.6 Graph of

R5

,.PT+

= R5 ( 6 ,

*

PT

;Ip) and R2 = R2(Bn ;Ip)

which is an ordinary PTE with critical value ( p - 2). We can represent the risk expression of 6,”’ in terms of the risk expression of 6:pT as follows: AS+.

Rs(6,

,W) =

R2(0:pT;W) - ( P - 2)tr(W)[2E(x,;22(A2)) - ( p - ~)E(X;:~(A’))] +(p2 - 4 ) ( n e ’ W 6 ) E ( ~ i($A2)) ~

{

[,+

1 + ( ~ - 2 ) t r ( W ) 2Ep 2THp+2r(P 1

w]

11

- 2; 0 ) [ + 2r)(p - 2 + 2T) 1 - ( p - 2)(n6’W0){ 2Ep [2rHP+2r(P - 2; 011

- ( p - 2)Ep ( p

4-2+2r(?J

+

-2Ep

[ ( p + 2r)(p1- 2 + 2T)

Hp-2+2r

1 + ( p - 2 ) ~ p( p + 2 + aT)( p + aT)~

[

( p - 2; O)] p + 2 (r p - 2;

o)]

}

(4.3.49)

where Ep stands for the expectation with respect to the Poisson r.v. r with mean A2/2 and H,(-; 0) is the cdf of a central chi-square variable with v d.f.

150


Also, we have the Sclove-Morris-Radhakrishnan (1972) modification of the

..PT

PTE, 8, as given below. First we have the usual one

..PT en = & ~ ( n i i k i i 2> x;(Q)),

(4.3.50)

. PT

and then we have the Stein-type modification of 8, given by (4.3.51) . PT+

The following theorem shows the uniform dominance of 8, PT+

Let R5(8,

;Ip)be the risk

PT+

of 8,

-PT APT+

. See Figure 4.3.6.

. PT

over 8,

.

- S+

Theorem 1. Let 8, , 8, and 8, be the estimators defined by (4.3.50), (4.3.51), and (4.3.17). Then, for p 2 3, . PT+

R2 (eET;Ip) 2 R5 (8,

;Ip) for all A’,

(4.3.52)

with strict inequality for at least one A’.

Proof.

. PT

Consider the risk difference R’(8,

RS(~;~+;I,) is the

replaced by

..PT+ ;Ip) - Rs(8, ;Ip), where

risk expression (4.3.36) with the critical value ( p - 2) difference is equal t o

x;(cr). Then the risk

P b - 2) [2E (X;;z(A’))

-

(P - 2 ) E (x;:,(A”,)]

1 (p+2r)(p- 2 + 2 r )

+p(p-2)[(p-2)Ep{

1 +A% - 2){ 2EP [%Hp+%

[

- (p2 - 4)A’E (x$(A2))

Hp-2fZr

(x;3(4;0)]

(x;(4;o)]

1

-2Ep ( p + 2r)(p - 2 + 2r) H p - 2 + 2 r 1 + ( P - 2 P P [ ( p + 2 r ) ( p- 2

+ 2T)

(x;(~);o)]

Hp+’r

( x ; ( 4 ;o)]

}.

All the terms in the brackets are positive, and the theorem follows.

(4.3.53)

4.4. Derivation of the Stein-Type Estimators

151

Further, consider the risk difference

The r.h.s. of (4.3.54) is positive if $ ( a ) S+

< ( p - 2). Thus, 6:'

APT+

dominates 6 ,

< ( p - 2 ) . 6 , is always minimax, and it is a minimax ,.PT ..PT+ substitute of 6 , whenever x;(a) I 2 ( p - 2 ) . If xE(a)> 2 ( p - 2 ) , then 6 , uniformly in A2 if $ ( a )

~

. PT ..S+ . PT ., PT+ dominates 8, but 6 , may not dominate 8, . In this case, 6 , may not be minimax. A P T APT+ S+ Thus, if x ; ( a ) < ( p - 2 ) , we can order the estimators 6 , , 6 , , and 8,

..

APT+

-PT

-S+

APT+

. See Figure 4.3.7.

as 8,

> 6 , for all $ ( a ) and A2 and 6 , > 6 ,

4.4

Derivation of the Stein-Type Estimators

In this section, we present some general methods of obtaining Stein-type estimators. They are

(1) Risk difference representation approach due to Stein (1956) and James and Stein (1961). (2) Empirical Bayes approach due to Efron and Morris (1973), and

(3) Preliminary test approach (or quasi-empirical Bayes approach) due t o Saleh and Sen (1978-1986).

4.4.1

Risk Difference Representation Approach

Let Ro(6,; 6 ) be the risk of 3, under the loss rille, - 6/12,and let R1(6,, 0) be the risk of another estimator 6,. In order to determine an efficient estimator, consider the risk difference

VR= ~~

~ ( 6e), -; ~ ~ ( 66,);.

(4.4.1)


152

N r

0 -

7

Risk of S + m-

01

0

I

I

I

I

5

10

15

20

Figure 4.3.7 Graphs of

A2 . S+

R4 = R4(8,

;Ip)and

R5

- PT+

= RS(8,

;Ip)

The aim is t o derive the estimator, S, such that VRo as a function of 6 is nonnegative and not identically zero. The Stein's identity (2.1.3) of Chapter 2 is an effective tool for transforming risk differences into an unbiased estimator of the "risk difference" considered as a parameter. This is given in the following theorem due to Hoffmann (1992):

Theorem 1. Let 6, = 8,-g($,), where g ( 8 , ) is partially differentiable and Ellg(6,112 < 03. Then


153

= ~0 [h(en)]9

(4.4.3)

where

(4.4.4) Then h(6,) is an unbiased estimator of the risk difference. If we can show that h(6,) is a positive-valued function for certain g ( 6 , ) satisfying the regularity conditions of Stein’s lemma 2 (2.1.3)’ then Theorem 4.4.1 states that 6; = 6 , - g(8,) has smaller quadratic risk than 6,. Now consider the class of estimators of 8 defined by

(4.4.5) based on a sample of size n from Np(e,Ip),and let

Ro(&

Q ) - R1(6,;

6,

be the MLE of 8. Then

0)

+ 2c(p - 2)A2E(~G:2(A2))] c2(p - 2 ) 2 E ( ~ p 2 ( A 2 + ) )2 ~ ( p 2)

= p - [ p + c2(p - 2 ) 2 E ( ~ p 2 ( A 2) )2c(p - 2)

= -2c(p - 2){l - ( p - 2 ) E ( x i 2 ( A 2 ) ) }= [2c(p -

-

c2(p- 2)2]E(~,2(Az))

= ( p - 2)2c(2 - c)E(xi2(A2)) > 0 for 0

< c < 2,

(4.4 .S)

where E ( x i 2 ( A 2 )is) defined by (2.2.13) in Chapter 2. Here (4.4.7) Then


154

(ii) h(8,) =

c(2-c)(p-Z)

difference,

118- 112

, (0 < c < 2) is the unbiased estimator of the risk

where

All conditions of Stein’s lemma 2 (Section 2.1.3) are satisfied to justify the result (4.4.6). For c = 1, we obtain the JS estimator. Thus, the JS estimator has smaller risk than 6,. As a matter of fact, the class of estimators defined by (4.4.5) has lesser risk than 6,.

4.4.2

Empirical Bayes Estimation (EBE) Approach

Similar t o R.obbins (1955), Efron and Morris (1973) developed the empirical Bayes estimation approach to obtain the JS estimator of 8 as follows: Since 6, Np(8,iIn), they incorporated the null hypothesis HO : 8 = 0 into the description of the prior distribution T of 8. Thus, assuming 6 Np(O,$Ip), they could obtain the joint distribution of (8’,6:) as

-

NZp{ (

-

), n-’(

is given by

:zi

816,

)}. Here the posterior distribution of 6

(1 ::2)1

- Np{

1

1 (1 - B)6,, -(I - B ) I p n

6 6 ,

while the marginal distribution of

is given by

,

(4.4.8)

(4.4.9)

N P ( 0 ,B-’Ip),

where

B = (1 + T~)-’,0 < B < 1.

(4.4.10)

The results (4.4.8) to (4.4.10) may be represented as an empirical Bayes tree given in Figure 4.4.1. The p parameters are tied down together by the underlying common distribution of the 6’s, but there are some differences in the 6’s in general. Now, each &Il&+ N ((1 - B)&,,, (1 - B ) ) so that

-

0

E ~illiji,, = (1 - B)B,,,, and the marginal distribution of mator of 8 may be calculated as

6,

=

fig;,, -

s,(e,)

=

i = I , . .. ,p,

is N ( 0 , B - l ) . Then the Bayes esti(1 - ~

p,.

(4.4.11)


155

Figure 4.4.1 Empirical Bayes tree The Bayes risk of 6, may be computed as P ( ~ , ; O ) = Ee{nII(Sr - e)I12}

+ B2Ee{nl18n112}- 2 B Ee{neb(6n- e)}

= Ee{nllen - 6112}

=~-2Bp+Bp=p(l-B).

(4.4.12)

However, the hyperparameter r2 or B in general is unknown. Therefore, we replace B by B, given by

(4.4.13) to obtain the JS estimator -s en=

(I - $-;!)". - - -

To justify the use of Bn, we note that

Bnljanl12

2

N

xp.

(4.4.14)

Thus,

(4.4.15)

156


and

E ( B , ) = ( p - 2)E,(nl/6,112)-’

(4.4.16)

= B.

This means that B, is an unbiased estimator of B and the J S estimator arises from the Bayes estimator when we replace B by B,. Notice that 6, can assume any value between 0 to 00 but B E (0,l). Therefore, we may restrict the value of n116,1/2 to more than ( p - 2 ) . This

,.S+

leads to the positiue-rule shrinkage estimator, 6,

We may summarize the empirical Bayes estimation approach as follows:

(1) Obtain a Bayes estimator based on a class of prior distributions (here it is N ( 0 , G I , ) ) that includes the “uncertain prior information” on the parameter of interest. (Here it is 6 = 0 . ) ( 2 ) Estimate the hyperparameter (here it is B ) based on the marginal distribution of the MLE/BLUE of 6 . (3) Plug in the estimator of B into the Bayes estimator of 6 , and obtain the EBE or Stein-type estimator. (Here it is (1 - B,)6,.) (4) If there is any restriction on B , adjust the estimator to set the estimator of B t o fulfill this requirement. (See positive-rule shrinkage estimator in Section 4.3.3.) For more information on EBE, see Deely and Lindley (1981). For the EBE approach a major difficulty is faced when formulating a suitable class of priors for a given problem and the estimation of the hyperparameter. In Chapter 12, we will be presenting the estimation of binomial and multinomial probabilities to illustrate these difficulties with approximate solutions. Finally, we consider the measure of improvement of an estimator 6 , over 60 as the ratio (4.4.18) where the numerator is the Bayes risk difference between 60 and 6 , and the denominator is the Bayes risk difference between 60 and 6,. Clearly,

K ( 6 , ; r ) 5 1.

(4.4.19)

If 6 , is uniformly better than 60, then

0 5 K(6,;n) 5 1. An almost Bayes estimator is characterized by a value of K(6,;

(4.4.20) T)

near 1.


157

4.4.3 Quasi-empirical Bayes or Preliminary Test Estimation Approach To obtain the Stein-type estimators using the P T E approach, the following steps may be followed for a given model with uncertain prior information: (I) Obtain an optimum unrestricted estimator 8, of 8 E 52 and an optimum restricted estimator 6, of 8 E w (e.g., by likelihood/least squares method). (2) Obtain an optimum (e.g., likelihood ratio type) test-statistic, L, for testing the ('uncertain prior information", say, 8 E w. (3) Define the preliminary test estimator of 8 as

- PT = 8, - (b, - ~ , ) I ( L ,< L,,~),

8,

where

e E w.

(4.4.21)

is the a-level critical value of L, from its distribution under Ho :

(4) Replace the indicator function I ( & < by a smooth decreasing function c L i l , where c is a suitable constant. Then the Stein-type estimator is defined by (4.4.22a) and the positive-rule Stein-type estimator by AS+

6,

= 6n

+ (1

-

cL;'}I(L, > c ) ( b , - 6,) for the PRSE of 8. (4.4.22b)

For the James-Stein case, 6, = 0 and L, = ~~118,11~, and both the PTE approach and the EBE approach give identical results. Here, (4.4.22a) and (4.4.22b) are based on a simple decreasing function cL;', but we will choose a more general function such as c+(L,), where $(L,) is a monotone decreasing function of the test-statistic L,. This allows us to cover various types of estimators. As a matter of fact we can write the four estimators compactly as (see Stein, 1966) 0: = $(.c,)&,

(4.4.23)

where

+Vn)

= 1, = I(Ln > x;(c.)), = 1- C L ; l ,

or

=

> c),

(1 - CL;')I(L,

APT - s

,.S+

c =p -2

yields the estimators b,, 8, ,On, and 8, , respectively. The quasi-empirical Bayes (or PTE) approach results in an identical, similar, or an approximation

158


of the EBE approach, hence, the name “quasi-empirical Bayes estimation (QEBE)”. We will consider this approach as a useful tool in many complicated situations in the chapters that follow. (5) The constant c in the formulation of the Stein-type estimators is chosen to be (i) c = (v - 2) if C, has a central chi-square distribution with v d.f. (ii) - (.1-2).2 u l ( v 2 + 2 ) if C , has a central F-distribution with (v1,v2) d.f.

In the James-Stein case here, c = ( p - 2 ) . As an example consider the estimation of the mean vector 6 for the p variate normal distribution N p (6 , C ) ,where E is the covariance matrix based on the sample Y 1 , .. . ,Y N Let . 7 and S be the sample mean vector, and let the sample covariance matrix have the Wishart distribution with ( N - 1) d.f. and expectation ( N - 1)X. Suppose that we need to estimate 6 with the loss function - 6)’C-’(8;

L(6:6) = N ( 8 ;

-

6),

(4.4.24)

which is the Mahalonobis distance between 6; and 6 . Suppose that 6 0 is a prior uncertain value of 6 . Then, using the P T E approach, we obtain the Stein-type estimator as

ht

= 60

+ (1 - cTG2) (F- 6,) ,

(4.4.25)

where TN -

N ( Y - 60)’S-’(Y - 6 0 )

(4.4.26)

is the Hotellings T2-statistic. Now, the distribution of [ T 2 / ( N- l)][(NV

p ) / p ]= F p , ~ -follows p, the F-distribution with ( p , N - p ) d.f. Hence,

c=

( P - 2)m

p(m

+ 2 ) ( N - 1)’

m=N-p,

(4.4.27)

is the optimum shrinkage constant. The EBE needs more analysis (see Efron and Morris, 1972) than this simple PTE approach as given below. Assume that the prior distribution of 6 is Np(60,N - I A ) and we have 6, n/,(6,N - I X ) . It may be shown that the joint-distribution of (6’,6;)’

-

>>

N-lA N-’A N-lA N-’(A C) * Consequently, the conditional distribution of 6 given 6~ is

is given by

NzP

Np

+

{ + A ( A+ X)-’(e, 60

- 60),N - ’ (A-’ + X-’)-’}

Hence, the Bayes estimator of 6 is given by

e;

=

eo+ A(A + x ) - l ( e N- eo)

= 60

+ ( I - B ) ( ~ N6 0 ) , B = X ( A + C)-’, -

(4.4.28)

159

4.4. Derivation of the Stein-Type Estimators with

+

Since (A X) is unknown, so is B, and we estimate B using the marginal distribution of Y , ( a = 1,.. . ,N ) , which is Np(80,(A+E)).Now, C,=, N (Ya&)(Y,- So)' follows the Wishart distribution, W P ( A + X), N ) , and we estimate (A+E) by

[+ c , N _ ~ ( Y ,

-eo)(y,

-

coy] = + { S N + N ( i j N - e 0 ) ( 6 N -

eo)'}, where S N = C,=l(Y,- e,)(Y, - 6 given by B N , N

B N =SN[SN

= [Ip

~ ) 'Hence, . an estimate of B is

+N ( 6 N -80)(6N

+ NsG1(6N - 8 , ) ( 6 N

-

-

&)']-'

?I')&

Then the empirical Bayes estimator of 8 is defined by -EB

8,

=

eo+ (I - B N ) ( 6 N eo). -

(4.4.29)

-s

Compare this expression with that of 8 , a t (4.4.25).

4.4.4 How Close is the JS Estimator to the Bayes Estimator? -S

-

We consider the shrinkage estimator 8,. Based on the assumption that 8 Np(O,$ I p ) = 7r as in Section 4.3.2, we know that the a posteriori distribution of 8 is N p { ( l- B)6,; (1 - B)Ip},where B = (1 ~ ~ ) - 0l < , B < 1. Hence,

+

P ( L 7 r )

p(&,

7r)

=p

(4.4.30)

= (1 - B)P,

and ~ ( 6 ,7 r;) - p(6,;

7r)

= Bp.

(4.4.31)

On the other hand, p(6,;

7r)

- p ( i f ; 7r) = E,

[I?,@,)

- R 3 ( 6 3 ] = ( p - q2E1,[ E ( X , 2 ( A 2 ) ) ] .

(4.4.32)

Now, as in Hoffmann (1992), we have

The r.h.s. may be written following Hoffmann (1992) as

(4.4.34)


160

Using the Fubini theorem, we obtain the r.h.s. as

/ fl [ / { -;(iJ, W

=

P

W

exp

0

-

8,)2 - n7~6:,}(27r)-'/~

-w

j=1

P

(- -8:)mi

(4.4.36)

+22~)-~/~exp

(4.4.37)

x exp

1

05

=

(1

nu 1 27L

+

+

Setting v = 27~/(1 2 u ) in (4.4.33), we get -

1

1

(1 - v)?-'exp

{ -zA2}dv 21

for p 2 3.

(4.4.38)

Thus, E(xF2(A2)) has the integral representation (4.4.34) that restricts the -S

) a decreasing function of shrinkage estimator 8, for p 2 4,and E ( x i 2 ( A 2 ) is A2. The maximum value of E ( x i 2 ( A 2 )is) l / ( p - 2 ) which drops down to 0 as A2 + 00. -S Thus, the risk difference Rl(8,) - RS(8,) equals ~

(P- 212 2

Jd

1

(I - 8

) F - I

exp

{

-

52) A 2 } d v ,

(4.4.39)

which varies from ( p - 2 ) (maximum) to 0 (minimum). Next, we compute the -S

Bayes risk of 8, given by

[qx,"A",>I

=Lp[ T I

E 7 T

1

(P- 212

p-2-1 2

(z) { 1

x

Substituting z =

[-,

u + B (I-B) [

-

exp

= B ( p - 2).

n

{ - ;nl18112}dv]

ll8Il2}d81 . . .do,

(I - u ) - 2 d v (

B n) . p/2

(4.4.40)

we obtain

(P- 2)2 (1 - B ) 2

-

exp

/

00

B

i=E

(-1 B

~ - ~ / ~ d z

1-B

(4.4.41)

4.5. Stein-Type Estimation When the Variance is Unknown

-s

161

Thus, the value of K(8,; n) equals ( p - 2)/p = 1- (2/p). For reasonably large

-s

values of p , the estimator 6, closely resembles the Bayes estimator.

4.5

Stein-Type Estimation When the Variance is Unknown

4.5.1 Introduction: Model, Estimators, and Tests Consider the model (4.1.1) Yi=8+ei,

where Y i= (Yli,.. . ,Ypi)’ and ei = Further, ei

i = 1 , . . . , n, ( e l i , . . . ,epi)’

- N,(o,

and 8 = (61,... ,6=)’. (4.5.1)

a2rP).

The MLE/LSE of 8 is the same 6 , as before, meaning but the unbiased estimator of u2 is given by

6,

= @I,.

-

.. ,Y p ) ’ ,

(4.5.2) Further, p ( n - l)s;/u2 follows a central chi-square distribution with rn = p ( n - 1) d.f. In order t o test the null hypothesis HO : 8 = 80 against H A : 8 # 80,we use the LR-test (4.5.3) that follows a central F-distribution with ( p ,m ) d.f. under Ho and a noncentral F-distribution with ( p , m ) d.f. and noncentrality parameter A2/2 (4.5.4) under the alternative.

4.5.2

Preliminary Test and Stein-Type Estimators

If a priori it is suspected that 8 may be 0 , then the preliminary test estimator of 8 is given by

162


where F p , m (is~the ) a-level critical value of a central F-distribution with ( P t m ) d-f. Similarly, the James-Stein estimator of 8 according to Section 4.4.3 is defined by (4.5.6) and the positive-rule shrinkage estimator is defined by (4.5.7) -S

It may be noted that if we write 8, as (1 - cL;')en, then it is easy to show that the optimum c equals ( p - 2 ) m / p ( m 2 ) .

+

4.5.3 Empirical Bayes Estimation When the Variance Is Unknown To obtain the empirical estimator of 8,we consider the class of priors

(3

8 - Np 0,-Ip

Hence,

elen

+

where B = a2(cr2 defined by

-

.

(4.5.8)

(4.5.9)

Np((1 - B)Gn,(1 - B)Ip},

0 < B < 1. Thus, the Bayes estimator of 6 is

(4.5.10)

6 , = (1 - B)&,

with Bayes risk p ( 1 - B). In the expression for 6,, B is unknown since 'rc and 'T are both unknown. To estimate B , we follow the same procedure as (4.4.14) to (4.4.17) where we consider cr2(02 ? - 'instead of 1 r 2 . We note that the marginal distribution of is Np(0,B-'Ip). Thus, Bn116((2has the central chi-square distribution with p d.f. Accordingly, the uniformly minimum variance unbiased estimator of (0' T ' ) - ~ is ( p - 2)/nli6n112. Also, the best scale invariant estimator of ' o is m s i / ( m + 2 ) . Thus, inserting the estimators of (a2 r2>-land o', we obtain the empirical Bayes estimator of 8 as

+

+

fie,,

+

+

(4.5.11a)


163 (4.5.1l b )

Thus, writing (4.5.12) where

din) = 1, = I(Cn

> Fp,m(a)),

= 1 - CLL1,

or = (1 - cL;l)I(L,

> c),

c=

(P - 2)"

+ 2) '

P("

APT - S

yields the estimators 8,,8, ,On,and b:+, respectively. Here the PT approach coincides with the EB approach. We may also consider the Stein estimator of u2 (see Section 3.8) in place of msp/m 2 in (4.5.11a, b) t o obtain another estimator of 6 (see Berry, 1994), namely

+

-IS

8, = (1 - c(m

+ 2)4s(~,)~;'}8,,

where

(

(4.5.13)

">

I Ln<m+2

+

'

and c = ( p - 2)m/p(m 2 ) . We study this estimator in Section 4.5.6. in terms of the test-statistic C, as in (4.5.3). . IS

-S

We will see later that 8, improves on On (4.5.6) uniformly in A2.

4.5.4

Bias, MSE Matrices, and Risk Expressions

The bias vector and quadratic bias expressions are (i) b1(8,)=O

and

B1(8,)=0.

PT

(4 h ( 8 , ) = 4Gp+2,,(&;A2)

(4.5.14a) . PT

and B 2 ( 8 ,

) = A2{G,+2,,(ta;A2)}

2

,

(4.5.14b) where Gul,u2 (.; A2) is the cdf of a noncentral F-distribution with (v1,uz) d.f. and noncentrality parameter A2/2 and

e.,

=

P + 2%&4-

(4.5.14~)


164

_I

n = 16

0 (9

S+

\

8QB

70

29 0

I

5

0

10

15

1

I

I

20

25

30

A2

Figure 4.5.1 Graph of QB of estimators: PTE, JSE, and PRSE

- PT , S = 8,,

(PT = 8,

and

B3

(6;)

= c’p2A2

-S

and S+ = 6;’)

{E ( x ~ ~ ~ ( A ’ ) ) } ~ ,

(4.5.14d)

The expressions are obtained following the procedure of Section 4.3 and applying the theorems in Chapter 2. Some graphs of QB of estimators are given in Figure 4.5.1. Similarly, the MSE matrices and the quadratic risk expressions may be obtained as (i) Ml(8,) = n’Ip and Rl(6,; W) = a 2 t r ( W ) = p if W = o - ~ I , . (4.5.15a)


165

166


if W = u-’IP.

(4.5.15g)

Risk Analysis of the Estimators

4.5.5

First compare the unrestricted estimator difference in this case is given by R1

6,

. PT

and the PTE, 8, . The risk

(6,; W) - R2(6ET;W )

= u2tr(W)Gp+z,m(&;A’)

-(ne’W8) {2Gp+z,m(L;A’) - Gp+4,m(&; A’)}.

(4.5.16)

Thus, the r.h.s. is nonnegative whenever

PT

In this interval, 8, APT 6, whenever

performs better than

6,,

and

6,


tr(W)Gp+z,m(a*; A2) > tl-w Chmin(W){2Gp+2,m(&; A’) - Gp+4,m(ez;A’)} - Chmin(W) ‘ (4.5.17b) -S Next, we compare 8, and 6,. Here, the risk difference A’ 2

R1(6,; W) - R&;

W)

+ [I - (’+2u2A2tr(W) 2 ) n 6 ’ W e ] 2 A 2 E ( x ~ ~ , ( A 2 ) ) } (4.5.18) . This risk difference is positive for all W satisfying the condition t r ( W ) > p-t2 C ~ r n a d W )- 2 ’

(4.5.19)

167


and b: dominates 6, for all A’ whenever W satisfies (4.5.19).The risk difference converges to zero as A’ -+ 03, and a t A’ = 0, it is 2(p+ m ) / ( m 2), which tends to 2 as m 4 00.

+

..PT

If we compare 8,

-S

and O n , we notice that under HO : 8 = 0 ,

R3(b:; W) = R2(6ET;W) + 02p{ Gp+2,m(&;0) - c}

-S

PT

Thus, the risk of 8, is bigger than the risk of 8, for all levels of significance a, satisfying A

PT

otherwise, the risk of 8, is bigger. The picture changes as A2 moves away from the origin. As A’ + co,the risk difference approaches zero. For intermediate value of A’, the two risks intersect. ..S+ -S Comparison of 8, and 8, shows that the risk difference is

The r.h.s. is negative, since

Thus, for all A2 E (0, co),

R4 S+

-S

(6;’

W) 5 R3

(6:;

(4.5.24)

W) , -S

AS+. 1s

a simple modification of 8,. PT Now, consider the modification of the PTE, 8, as defined at (4.5.5):

8, dominates On, and 8,

APT+

-PT

en = e n

(1 -&).

(4.5.25)

168



1 0 for all

(0, A2)

R5

169

and p 2 3. Hence,

(6ET+;W) 5 R2 (6ET;W)

for all ( a ,A2),

(4.5.31)

..PT+

PT

where strict inequality holds for some ( a ,A2). The risk of both 6 , and 6 , approaches a 2 t r ( W ) as A2 4 co. Both estimators share the property that as A2 + oc), their risks converge to a common limit a 2 t r ( W ) but the risk of PT+ PT . PT+ is remains below that of the risk of 6 , . This shows clearly that 6 , 6, ~

A

..PT

. Furthermore, neither

PT+

..PT

-

6, nor 6 , is superior t o 6,, their ..PT+ risk functions intersect as A2 increases leaving the origin. However, 6 , is ..PT a simple modification of 6 , similar to the positive-rule shrinkage estimator and valid for p 2 3. ..S+ APT+ Finally, we compare the PRSE, 6 , and the improved PTE, 6 , . Here the risk difference is superior t o 6 ,

A


170

“1 1

la = 015

R

0

0

1

I

I

1

I

1

5

10

15

20

25

30

A2

Figure 4.5.2 Graph of and

PT R2 = R2(8, ;oP2Ip), R4 = APT+. A

R5 = Rs(8,

R4(6:+;o-21,),

,O - ~ I , )

(4.5.32)

Note that la < c p / ( p implies F,,,(cY) < c.

+ 2 ) implies Fp,,(a)

< c and similarly l: < c p / ( p + 4) ..S+

The risk difference is positive for all Fp,m(cr)< c. Therefore, On

- PT+ formly better than 8, . The estimator 8,

AS+.

is uni-

,.S+

is minimax, and therefore, On is a minimax substitute for the PTE whenever F p , m ( ~) 2c.

..PT+

Then 8,

..PT

dominates 8,

..S+

as we saw before at (4.5.31), but On . PT+

PT

may not

. In this case, 8, is not minimax. See Figure 4.5.2 for com-s+ PT+ parison of 8, ,On , and On . dominate 8,

APT

e

171


4.5.6

An Alternative Improved Estimator of 8

In Section 4.5.2 we considered various estimators of 8, and we defined another IS

improved shrinkage estimator 8,

in (4.5.13) as

-IS

0, = (1 - c ( m

+2)4s(~n)~,~>8n,

where

1

4S(Ln) =

I

+

(e n > -

1 $Ln I mi:2)+m+p+2

(4.5.33)

(L, C ) - 6*,

(i) fisel(k2 - 6 ( , ) ) 6

177

-

S+

-

(iv) f i s ~ ' ( 6 , where c = ( p - 2), Then

(i) bl(6,) (ii)

=0

and Bl(6,)

= 0,

..PT bz(6, ) = aE(U1), where u1 = z - (Z + S*)I(l/Z+ 6*lj2 < $ ( a ) ) = 6 H p + ~ ( x ; ( a ) ; A 2 ) , ..PT

A2 = 116*jj2 and Bz(6, ) = A2{Hp+2(~~(a);A2)}2, -s

(iii) B3(6,) = aE(U2), where

Uz = Z - c(Z + S*)llZ + 6*11-2 = -cdE[~;:~(A~))l

&(at)

and

2

= c'A~{E[x;$~(A~)]} .

. s+

(iv) b4(O,

) = aE(U3), where

u3 = (z + 6*)(1- CllZ + 6*lI-2)I(jlz+ c5*1/-2 > c) - 6* = ~ E [ ( ~ - C X ~ ( A ' ) ) I ( & ~> ( Ac)]~ )- 6 =

-c6E[x;:2(A2)]

- ~ E [ ( ~ - c x , S ~ ~ ( A ~ )< ) Ic () ]X, ~~=(( Ap -~2 ))

= -6{CE[X;:2(A2)] -

+ Hp+2(c;A2)

.)I}

cE[x,S22(A2)I(x~$2(A2)
0).

(4.7.3)

Now, we combine the two estimators via a preliminary test procedure with the critical value k , as follows:

6,“(c1k ) = 6;Tl(nl[6,112> k )

+ 6;T(c)l(n1lenll2L: k )

= 6: - (6: - b , ” ( C ) ) I(nllBnl12 5 k ) = 6, - (c - p + 2)(nllenll2)-l1(nlle,1125 k)&.

(4.7.4)

-S

Minimizing the quadratic risk of 6n(c, k ) given by P + (c - P

-2(c

-

+ 2)2E[llZll-21(11Zl12< k ) ]

p + 2)E[llzll-2z’(z - 6)1(ljz112 < k ) ]

(4.7.5)

with respect to c, we obtain the optimum value of c as

-

where Z Np(S,Ip), 6=6 8 . Now, c ; ( k ,A 2 )can be rewritten as

c;(k;A2)= ( p - 2)

I(IIZ2ll < k ) } + E { (1 - Jq$)

E

{~

~ ( 1 1 ~ 1< 1k )2 }

1

A2 = 116/121 (4.7.7)

where U = Z’S/llSll. The explicit form of (4.7.7) is given by

c;(k; A2) = ( p - 2)

; - EP { * ~ p - 2 + 2 r ( k ; o ) ) + ~ p ( ka2)

{

EP &Hp-z+24k;

,

(4.7.8)

O)}

and H,(x;A2)is the cdf. of a noncentral chi squared distribution with Y d.f. and noncentrality parameter A2/2, and Ep stands for the expectation operator with respect to Poisson distribution with mean A2/2. Using (2.2.3), we can write


182 Now, rewriting (4.7.9) as

c;(k;A2) = ( p - 2) -

2hp(k;A2)

[J:

(4.7.10)

t-lh,(t; A2)dt] ’

we see that cr(k;A’) is less than or equal to

c;(k; 0 ) = ( p - 2) -

2h,(k; 0)

(4.7.11)

[sd” t-lh,(t; O)dt] ’

which is due to (2.2.7). Also, c;(k;O)is an increasing function of k and 0 @ ( k ;0) < p - 2. Further, we can write C ; ( k ; 0) as

k,, the ball of radius fi is excluded from Cs+(6)and the sets Cs+(6)and C’(6) intersect. Since Cs+(c) is the PRSE of 6 , Cs+(6) is farther away from the origin 6 than is C’(6). The r.h.s. of (4.8.10) may be written as 41 4 2 , where

+

= H&;

and

a2)1(n2 5 k,)

42 = pe {nip - ~

5 k,;

~ ( C ) I I ~

~LII~,II~

2 c} .

(4.8.12)

Clearly, the probability is discontinuous a t A’ = k, since the first term vanishes for A’ > k,. Before proving the next theorem, we consider the polar transformation t o describe the set Cs+(6).Let (4.8.13a) Then, for A’ = nII6/I2< k,, we have CS+(6)= { ( T I P ) : 7- 5 .!(P),

PE

[-T,T]},

(4.8.13b)

where (4.8.13~) and (4.8.13d)

(4.8.14a) k-r sin,& = -

A, (4.8.14b)

and

r!!(P) = A c o s P - d k , - A2sinp.

(4.8.14~)

Further, note that T- (P) > c. Also, T+ (P), is a decreasing function of 0,but the distance between 6 and the boundary of Cs+(6)increases with 0.The theorem that follows shows the discontinuity of the coverage probability.

190


Theorem 3. A2 = k y .

42

-S

= Pe(nll0 - 0,(c)/I2 5 k,; nlIan112 > c } is discontinuous a t

Proof. First, if A’ < k, we have 42

-s 2 = Pe(n.116 - 0,(c)II

5 k,; nllan[12> c } = 2K (4.8.15)

where

1 P-’sinp-2p exp { - - [ r 2 -2rAcosp+A2]} h(r,P)= 7and K = ( 2 7 ~ ) - ( P - ~ ) nfZ:(s,” /~ sin2 pdp), is the normalizing constant. As A increases to

A,r+(P)goes to

= k;. Therefore,

for A2 < k,, the r.h.s. of (4.8.15) reduces to

2K

ln1;‘’)

h(r,p)dr dp.

(4.8.16)

Second, if A2 > k,, the r.h.s. of (4.8.15) becomes (4.8.17)

I

since, as A decreases to

A,r+@)-+ Ic;

and r-(/3) -+ , h and ,& -+ n/2.

Combining (4.8.16) and (4.8.17), we can write

P ( C + ( 6 E ) )=

Hp(c;A2)1(A2< k,) + 2 :J~ J;(’) h(r,p)drdp

if

< k,,

H p ( c ;A2)1(A2< k,) + 2 J’/:~ J>(’) h(r,p)drdp if

> k-,.

Using the results given above, we present in Table 4.8.1 the components of the probability of coverage of Cs+(O,) for p = 3 , 6 , and 9.

4.8. Confidence Set Estimation Based on Stein-Type Estimators

191

Table 4.8.1 Decomposition of the Coverage Probability.

A $1 $2

+

$1

$2

A $1

=

$2

41 + $2 A

41 $2

41 + $2 A =

$1

$2 $1

+

$2

A

91

$2

$1

+ 42

0 0.25 0.50 0.75 1.0 1.25 1.5 0.1727 0.1682 0.1556 0.1366 0.1138 0.0900 0.0675 0.7799 0.7842 0.7964 0.8146 0.8364 0.8588 0.8795 0.9526 0.9524 0.9520 0.9512 0.9502 0.9488 0.9470 1.75 2.0 2.25 4 2.5 6 8 0.0479 0.0323 0.0206 0.0125 0.8969 0.9099 0.9182 ,9212 0.9060 0.9022 0.9012 0.9448 0.9422 0.9388 0.9337 0.9060 0.9022 0.9012

0 0.35 0.65 0.98 1.3 1.6 2 0.2383 0.2306 0.2087 0.1766 0.1394 0.1025 0.0698 0.7512 0.7588 0.7802 0.8116 0.8477 0.8833 0.9140 0.9895 0.9894 0.9889 0.9882 0.9871 0.9858 0.9838 2.3 2.6 2.9 3.3 5 7 9 .0441 .0257 .0138 .0068 0.9375 0.9530 0.9614 0.9637 0.9338 0.9169 0.9107 0.9816 0.9787 0.9752 0.9705 0.9338 0.9169 0.9107

0 0.38 1.1 0.77 1.5 1.9 2.3 0.2494 0.2380 0.2119 0.1741 0.1315 0.0909 0.0571 0.7497 0.7590 0.7849 0.8223 0.8644 0.9042 0.9370 .9991 0.9970 0.9968 0.9964 0.9959 0.9951 0.9941

A

2.7 3.1 3.4 3.8 5 7 9 11 0.0325 0.0166 0.0076 0.0031 0.9602 0.9742 0.9808 0.9821 0.9639 0.9388 0.9255 0.9176 $2 61 h .9927 0.9909 0.9884 0.9852 0.9639 0.9388 0.9255 0.9176 $1

=

+

The table shows the decomposition of the coverage probability (4.8.10) where

d1 = H&;

a2)1(a2 < k,)

(4.8.18a)

and (4.8.18b)


192

The different scales of A are intended t o provide homogeneous coverage of [0,&]. Some combined values of c were obtained in Hwang and Casella (1984)and given in the Table 4.8.2.

cTlo c'OS

3 0.892 0.810

4 1.710 1.602

5 2.536 2.408

6 3.666 3.223

7 4.199 4.042

8 5.035 4.866

9 5.872 5.692

10 6.711 6.521

In general, we note that 41 is decreasing as A moves toward and 4 2 is increasing in A. From Table 4.8.1 (111) we find that the maximum of 41 4 2 is almost unity (0.9991)and the minimum value is 0.9176,which is more than 0.90.The confidence coefficient of the set C("c;is thus always more than that

&

+

of CO(6). In concluding this section, we present without proof the asymptotic approximation of 4 2 as A' + 00 motivated by the fact that A2 > k,.

Theorem 4. (Hwang and Casella, 1984). As A' proximation of 4 2 is given by 42

C

(1 - y) - -[I 2A

- y - h ( y ) ] [c 2(p -

4 00,

the asymptotic ap-

a)] + O ( L ~ - ~ / ' ) ,(4.8.19a)

where (4.8.19b)

4.8.3 Confidence Set Based on Preliminary Test Estimator Now, consider the PTE of 6 :

where d is the critical value of the test-statistic nj6,11' a t some level of significance, say, a. The related confidence set is defined by

CPT(6y(a)) = {6 : nll6 - 6:T(a)j)25 k,}. From Section 4.2,we know that the quadratic risk of A

PT

Rz(6, ( a ) )= p

-

p H p ( d ;A')

+ A2{2Hp+2(d; A')

- PT

(4.8.20)

. PT en ( a )is given by

- Hp+4(d;A ')}.

(4.8.21)

The graph of the efficiency of 6, ( a )relative to 6, as a function of A2 may be . PT

described as follows: The maximum efficiency of 6, ( a )is [l-Hp+2(d; 0)I-l 2


193

1, which decreases, crossing the one-line a t some point near A2 = 1, and goes to a minimum value at, say, A2 = A:,,. It then increases monotonically toward the 1-line as A2 4 co. - PT The properties of the confidence set CPT(B, (a))are similar t o the properties of the point estimator given by Theorem 3. First we note the decomposition of the coverage probability

Pe {e E ~ p ~ ( i j z ~ ( a ) ) } = H,(d;

A2)1(A2< k,)

+ Pe {nlI8,/l2 > d; n118 ,.PT

I k,} . (4.8.22)

Then we conclude the inadmissibility of CPT(B, ( a ) )based on Theorem 5 given below. Theorem 5. (Robert and Saleh, 2001; Chiou and Saleh, 2002). (i) If A2 < k,, then P e { 0 E CPT(6:T(a))}2 Pe{e E Co((e,)}.

(ii) If k, 5 A2 I (&+

cE7(en)}.

(iii) If A' > (&+

&)2,

then Pe{O E CPT(6:T(a))} < Pe{e E

d)' then , Pe{8 E CPT(6ET(a))}= 1 - y.

(4.8.23)

..PT

If A2 < Ic, I: d, then Pe{O E CPT(Bn( a ) ) }is decreasing as a function of A2. At A2 = 0, it equals 1 - a , corresponding to the critical value d. As A2 increases to k,, the coverage probability of CPT(6zT(a)) drops to a minimum below 1 - y depending on the value of d, meaning the level of significance of the test for 0 = 0, and then monotonically increases toward 1 - y as A2 4 + Finally, the coverage probability becomes equal t o 1 - y as a2>

(6 a)'. (A+

There is a discontinuity of the coverage probability a t A2 = k,. Tabular values of the coverage probability are presented in Tables 4.8.3 and 4.8.4 for 90% confidence sets for p = 5(2)15 and the test levels a = 0.05 and 0.10. Proof.

(i) A2 < k,. From (4.8.22) we have

194


(ii) K7

< A’ 5 (A+ A)’.From (4.8.22) we have PA

(

CPT

(enPT ( a ) ) )= P

A ~ { ~>I d;nll6 I ~ ~- SnlI2 ~ I ~ 5 ICY}

5 P~2{nll6- en[[’ < Icy}

=

1 - y.

(6 + &)’. Note that (6 + 4)’< A. Now, njl6 k, ==+ fill6 en/[5 A,which in turn implies (iii) A’

>

-

en1I25

-

A - fillenll I fill6 - en11 5

and we have A 5

It follows that (4.8.22),

A,

fillenll + 6. Thus, we conclude that

& < fiI/enllwhich is equivalent to nllenll’ > d. Thus, from PA~{nj/8,11~ > d; ~~116 - 8,11’ < k-,} = P0{n//6- 6n1125 Icy} = 1 - y.

This completes the proof. Table 4.8.3 Coverage Probabilities for the Set and a = 0.05

A

p=5 0.9500 1 0.9356 2 0.9318 3 0.9289 4 0.9264 5 0.9243 6 0.9224 7 0.9206 8 0.9191 9 0.9176 10 0.5937 15 0.7773 20 0.8593 25 0.8890 50 0.9000 100 0.9000 0

p=7 0.9500 0.9350 0.9304 0.9271 0.9245 0.9222 0.9202 0.9184 0.9169 0.9154 0.9141 0.7296 0.8336 0.8777 0.9000 0.9000

p=9 0.9500 0.9347 0.9297 0.9262 0.9233 0.9210 0.9190 0.9172 0.9156 0.9142 0.9129 0.6871 0.8079 0.8650 0.9000 0.9000

p=ll 0.9500 0.9347 0.9293 0.9256 0.9226 0.9202 0.9181 0.9163 0.9147 0.9133 0.9121 0.9074 0.7826 0.8514 0.8999 0.9000

CPT(6F(a)) with y = 0.10 p=13 0.9500 0.9348 0.9291 0.9252 0.9221 0.9197 0.9176 0.9157 0.9141 0.9127 0.9115 0.9069 0.7580 0.8371 0.8997 0.9000

p=15 0.9500 0.9350 0.9289 0.9249 0.9218 0.9193 0.9171 0.9153 0.9137 0.9123 0.9110 0.9066 0.9039 0.8223 0.8995 0.9000


195

Table 4.8.4 Coverage Probabilities for the Set CPT(8ZT(a)) with y = 0.10 and Q! = 0.10

A

p=5 0.9000 0.9190 2 0.9196 3 0.9190 4 0.9180 5 0.9169 6 0.9157 7 0.9147 8 0.9136 9 0.9126 10 0.6937 15 0.8305 20 0.8809 25 0.8960 50 0.9000 100 0.9000 0 1

4.8.4

p=7 0.9000 0.9162 0.9167 0.9160 0.9151 0.9141 0.9130 0.9120 0.9111 0.9102 0.9094 0.7975 0.8657 0.8901 0.9000 0.9000

p=9 0.9000 0.9144 0.9148 0.9142 0.9133 0.9123 0.9114 0.9105 0.9096 0.9088 0.9080 0.7668 0.8498 0.8834 0.9000 0.9000

p=ll 0.9000 0.9131 0.9135 0.9129 0.9120 0.9111 0.9102 0.9094 0.9085 0.9078 0.9071 0.9044 0.8336 0.8758 0.9000 0.9000

p=13 0.9000 0.9122 0.9125 0.9119 0.9111 0.9102 0.9093 0.9085 0.9078 0.9071 0.9064 0.9039 0.8173 0.8675 0.8999 0.9000

p=15 0.9000 0.9114 0.9117 0.9111 0.9103 0.9095 0.9087 0.9079 0.9072 0.9065 0.9059 0.9036 0.9021 0.8588 0.8999 0.9000

Asymptotic Theory of Recentered Confidence Sets and Domination of Positive-Rule Coverage Probability

Consider the model (4.1.1) again, namely

Yi = 6 + e i , i = 1,... , n , where ei = (eli,. . . ,epi)’ vector. Further, assume

-

(4.8.24)

n,”=, Fo(eji) and 6 = (61,. . . ,6’)’

is the mean

E ( e i ) = 0 , and E(eie:)= 0~1,.

(4.8.25)

-

The unbiased estimator of 6 is 8, = (TI,.. . Yp)’and by the pvariate central limit theorem

&en

-

- 6)

The unbiased estimator of n2 is S$

= { p ( n - l)}-’tr

n/,(070~1,) as n -+ DC).

I’c(Yi

1

(4.8.26)

- 6)(Yi- 0)’

i=l

and s$ -+ g2 in probability as n + co. For the test of HO : 6 H A : 6 # 6 0 , we use the test-statistic

(4.8.27) = 60

against

(4.8.28)

196


Under Ho, as n -+ 00, the distribution of Ln is a central chi-square distribution with p d.f. For the asymptotic theory of confidence sets using PRSE, we assume that the prior uncertain information of 8 is equal to 0 (null vector). Then we define the Neyman-Pearson confidence set as follows: (4.8.29) is the y-level critical value of the chi-square distribution where k, = x:(y) with p d.f. Correspondingly, the confidence set based on PRSE is defined by

,.S+

(4.8.30)

where the positive-rule Stein estimator is given by (4.8.31a) and (4.8.31b) Thus, we have the following theorem.

Theorem 6. Under the fixed alternative 8 = 6 ,

Proof.

First consider the coverage probability

V

-

since f i ( 6 , - O ) / s y - + Z Np(O,Ip) as n 00, and k, = x;(y), a y-level critical value from the central chi-square distribution with p d.f. Note that under fixed alternative L, + 00 as n 00. Hence, L;' converges to 0 in first mean and -+

--f

~ ; r ( e f +-( ~e)) -+ SY

which implies that

0

q

as

n-+00,


197

Because of this equivalence of probability content of the two sets, we may consider a sequence of Pitman-type local alternatives and obtain the following theorem, which shows that the confidence set with PRSE as center of the sphere is locally better than the usual confidence set, Co(en):

Theorem 7. Under the local alternatives K(n) : 8(nl = n-l/'6, vector, Iim

n-03

PK(,,

6 a fixed

{c~+(o(,))} = H&;A~)I(A< ~ IC,)

Proof. First note that under K(n) - ~ ( n )-) C f i ' n s y

where Z

-

SY

IIe n (I2

3

-

c(Z + 0-16) /IZ + c-1611''

A&(O, Ip).Therefore,

2 p6{11z/12< k,} = 1 - y by Theorem 4.8.1.

(4.8.33)

Thus, we can use the results of (4.8.10) on the decomposition of the coverage probability of positive-rule estimators in the normal case. Theorems 4.8.2 and 4.8.3 pertain to the nonnormal and asymptotic case. Thus, the domination of the probability of coverage in this case is given by Table 4.8.1, which holds for the asymptotic case.

4.9

Nonparamet ric Met hods: R-Est imat ion

In this section, we consider the nonparametric methods of estimation of several location parameters 8 = (01,. . . ,0,)' when it is suspected that 8 may be 0. This broadens the scope of the Stein-type estimators in a genera1 class of symmetric distributions, and the robust Steins procedure ensues.


198

4.9.1 Model and Assumptions Let Y1, Y z , . . . ,Y, be n independent pdimensional response vectors such that

Y 3 = 6 + e 3 , j = l ,. . . , n,

(4.9.1)

holds. The error distribution F ( e 3 )is given by P

F(e,)=nFo(e,,), j = I , . . . ,n,

(4.9.2)

2=1

F with

where Fo belongs to the class of absolutely continuous distributions, absolutely continuous pdf fo(e,,) such that

(4.9.3) That is, the Fisher’s information is bounded. Our objective is the estimation of 6 when it is suspected that 8 may be 0 .

4.9.2

Test of Hypothesis

Consider the data set Y with independent rows and the corresponding ranks R$(bi) of the absolute deviations [ y Z j - biI among 1x1 - bi 1,. .. , lyZn - bi 1, i = 1, ... , p : y11

1 . ’

Yln

Rfl(bl) RA(bi)

* * -

...

R,+l(bP) . * .

Rf,(bl) RL(bi) R,+,(bP)

(4.9.4) For every n ( 2 1) and j = 1,. . . , p , we define a set of scores by setting (4.9.5) where 0 < U1, 5 . . . < U,, < 1 are the order statistics of a sample of size n from U(0,l). For every 21 E (0, l ) , (4.9.6) the 4j are all assumed to be nondecreasing, nonconstant and square integrable score-generating functions. Recall that R$(bi) is the rank of lxj - bil among 1 x 1 - bil, 1 x 2 - bil,.. . lYin - bil for i = 1,.. . , p and j = 1,.. . ,n. Now, consider the vector of rank-statistics Tn(b) =

... ,Tpn(bp))’,

(~ln(bl),

(4.9.7a)


199

where n

Tzn(bZ)= n-l xui(R;(b,))sign(Y,, j=1

- b,)

(4.9.7b)

for i = 1,.. . ,p. Note that Ti(bi) is \ in bi (i = 1,.. . , p ) (see Puri and Sen, 1971, Ch. 6). Let

A consistent estimator of A: may be defined as n

A: = ( n - 1)-' x[u:(i) - &:I2.

(4.9.9)

i=l

Then, as n + 03 &Tn(O)

2)

&(O,

A&).

(4.9.10)

(See Hajek, Sidak, and Sen, 1999; Sen, 1986; Puri and Sen, 1971). Thus, for testing HO: 6 against H A : 6 # 0, we consider the rank-test C N ,defined by

ell = n A i 2 llTn(O)Il2,

(4.9.11)

which approximately follows a central chi-square distribution with p d.f. under HO as n -+ 03 (e.g., Puri and Sen, 1986).

4.9.3 Estimation of the Location Parameter Now. we focus on the estimation of 6 based on the rank-statistic T(b), following Adichie (1967) and Puri and Sen (1986). Let

- (1) Bn = sup{b : T,(b) > 0},

- (2)

6 , = inf{b : T,(b) < O}.

(4.9.12)

Then. we define the UE of 6 as

-

6, The UE,

8,

=

1 -(1) - ( 2 ) ~ ( 6 f, 6 ,

).

(4.9.13)

is a translation-invariant and consistent estimator of 8 . The

RE of 6 is of course, 0 , which is prefixed by the null hypothesis H0.It may


200

be shown, following Puri and Sen (1971), Sen (1986), and Hajek, Sidak, and Sen (1999), that as n 00, ---f

-

fi(8n - 8)

(4.9.14)

&(O, A ; I y 2 ( + , 4 ) I p ) .

(See also Section 3.10.) Let x;(a) be the a-level critical value for testing Ho : 6 = 0 . Then, the PTE of 8 is defined by

-

APT

8, = 8, - B,I(L,

L ,

(4.9.1613)

where E, 0 as n -+ co. A better estimator of 6 without the confusion above S+ is the positive-rule Stein-type estimator 8, , where

-

--f

6:+ = (1 - dLC,lI(L, > d ) 6 , .

(4.9.17)

All these estimators parallel the estimators defined in Section 4.6.2.

4.9.4 ADB, ADQB, ADMSE, and ADQR of the Estimators of Location Parameters

It may be observed that the test-statistic L, defined by (4.7.10) is a consistent test as such, when n

--f

03,

., S+

APT - S

all three estimators, namely 8,

, On,and

6, that are equivalent in distribution to that of 6,. Hence, the asymptotic distributional bias, MSE matrices, and the risk expressions are identical to that of fi(6, - 8).As we have seen before, t o overcome this problem, we consider the class of local alternatives

K(,,

:

e(,)=

6 = (&, . . . ,bp)’.

(4.9.18)

Then we have the following theorem under K(,), using the linearity result of Section 3.10: sup

{ f i ITj,(n-l12bj)

for j = 1 , . . . , p .

+ n-l/’b,y(+,

- Tj,(O)

4)l; lbjl < k} 5 0 (4.9.19)

201


Proof.

Based on the linearity results given at (4.9.19), we have as n -+ 00,

h T n ( 0 )= & ( e n - q n ) ) T ( ? h4) + O p ( 1 ) .

(4.9.20)

Hence, using contiguity of probability measures under {K,,)} and t o that of 8 = 0 , we obtain, as n -, 00,

using (4.9.10). Now,

4 ) A , ' ( C ( 4 - 8,n)) = fiy($, 0)Az1(en- e(,)) - h-~($, 4)AL1enI(L
E,),

(4.9.23)

r.h.s. converges in distribution to the r.v.

z - c(Z + a-1S)llZ + a-16I1-2,

c =p

-

2.

(4.9.24)

Finally, h T W 7 d)A,'(b,S+ =

- 0(n,)

,/~~(+,4)~;1(iz+ - q,,) - C ( ~ - Y ( $ , ~ ) A L ~ ~ , ~ L > I ~ ( (4.9.25) C,

which converges in distribution to the r.v.

(Z + a-16)(1

-

+

Cl/Z+ a - 1 f 5 [ [ - 2 ) I ( ~ ~ za-1112> c).

As a result of Theorem 1, we obtain Theorem 2.

(4.9.26)

202


Theorem 2. Under {K(,)} and the assumed regularity conditions, as n -+00, (i)

fi(8,

- Q(,))

~ ~ ( A;/lr2(+, 0 ,

(4.9.27)

(ii) limP~~,,)(~A,’[T,(0)]’[T,(0)] I z}= H p ( z ;A’),

(4.9.28a)

where LIP(.; A2) is the cdf of a noncentral chi-square distribution with p d.f. and noncentrality parameter A2/2, where A2 = ))6112/021 o’ = A$/r2(+,,4)

(4.9.28b) -PT

(iii) The ADB, ADQB, ADMSE, and the ADQR of 6,, 8, are given below: For UE, we have

(en), bl(8,) = 0 and &(en)= 0 and

M1(6,) = 0’1, and R l ( 6 , ; W) = a’tr(W). For PTE,

,.PT

AS

, 8,, and

S+

8,

(4.9.29)

(an”), we have

b2(8, )

=

-q+2(x34;

. PT

A2) and B2(8, ) = A 2 { H p + 2 ( x 3 4 ;A2)I2; (4.9.30)

Mz(6n) = a 2 1 p { 1

- H~+Z(X;(Q); A’)}

+66’{2Hp+2(x;(Q);A2) - Hp+4(x34;A”} and

Rz(6;; W) = a2tr(W){1 - Hp+2(x2,(a);A2)}

4.9. Nonparam etric Methods: R-Es timation

203

and

The expressions above are similar to the expressions in Theorems 1 and 2 of Section 4.6.3. Hence, the asymptotic properties of the four estimators are the same as given there.

Proof. let

(i) See Theorem 1. (ii) The proof follows from (4.9.21). For (iii), we

u1 = z

-

(Z + a-'6)l(ljZ

+ 0-16)[/2 < Ice).

(4.9.34)

Then the bias vector is given by b2(eET)= E(aU1) =

-6Hp+2(xg(cu),A2).

Similarly, PT

M2(6, ) = D ~ E [ U , U=~c2Ip ] - 02 I p H p + 2 ( x ; ( a ) A2) ; +66'{2Hp+2(x3a); A2) - Hp+4(x2,(a);A2H. (4.9.35) -S

For the bias vector and MSE matrix of 6,, let u2 =

z

-

c(Z

+ a-16)"Z + a-161j-2.

Then compute -S

b3(6:) = aE(U2) and M3(6,) = o2E[U2U;].

(4.9.36)


204

-St

Similarly, the bias vector and the MSE matrix of 8, using the r.v.: u3 =

can be computed by

(Z + a-16)(1 - C l j Z f a - 1 6 ~ ~ - 2 ) I (+ ~ la-16j/2 z > c).

Also, compute

- S+ )

b4(@, M4

aE[U3],

(en -s+= a2E[U3Uj].

(4.9.37)

Asymptotic Properties of Confidence Sets

4.9.5

In this section, we consider the asymptotic properties of the following (1--y)'% confidence sets: CO(6,)

=

{4I@ - enIl2I Icy},

cPT (GPT n ( a ) )= in110 - 6t/T(a)/~2 I ky>, and

. S+

CS+(8,

(.I)

. S+

= {rill@ - 8,

(.)I2

5 k,>,

(4.9.38) (4.9.39) (4.9.40)

where Ic, = $(y). Similar to Theorem 4.8.6, we show that under fixed alternative, the asympPT st totic distributions of fi(8, ( a )- 8) and &(On (c) - 8 ) are the same as that of - 8).Hence, we consider the asymptotic properties of the confidence sets (4.9.26) through (4.9.28) under the local alternatives, A

&(an

qn,qn)= n-1/26.

To begin, it is easy to see that lim P~,,){nljO(,)- &/I2

n-m

5 k,}

= P(llZ112 < k7) = 1 - y.

(4.9.41)

Next, we consider Theorem 4.9.1 and use the results (i) through (iii) to compute the probability of the different sets. Thus, the asymptotic coverage prob..S+ abilities of CpT(6ET(a))and CS+(8, (c)) reduce to

and

(4.9.421;) The two expressions are similar to the expressions given in Theorem 4.8.7. Hence, the asymptotic coverage probabilities are similar to the finite sample coverage probabilities as given in Theorems 4.8.1 through 4.8.7.

205

4.10. Conclusions

4.10

Conclusions

In this chapter, we introduced the basic Stein-type estimators, in particular the James-Stein estimator together with the preliminary test estimator of the mean vector of a pvariate normal distribution with known and unknown covariance matrix. We have also presented three basic methods of obtaining Stein-type estimators. They are (1) the risk difference approach, (2) the empirical Bayes approach and the (3) the preliminary test (or quasi-empirical Bayes) approach. Of all these models, we particularly discussed the empirical Bayes and the preliminary test (or quasi-empirical Bayes) approach to obtain the Stein-type estimators. In general, we considered the (1) unrestricted (2) PTE, (3) Stein-type, and (4) positive-rule shrinkage estimators and evaluated their properties according to Table 4.10.1. Table 4.10.1 Properties of Estimators

Estimators

4

We showed the following among other results: (i) 0 = B1 (8,) 5 B4 (6;') 5 B3 (6:) 5 B2 (6:T) conditions on the level of significance, a

(-

(ii) R1 8,;W (iii) R3 (6;;

) 2 R3 (6,;W ) 2 R4 (*'+;W) 8, 2 R5 (6;;

0, MRE(6, ;tin) is decreasing function of A' for fixed cy until A' = A;,,. Then it increases toward 1 as

A'

+ 00.

5.5. MSE Matrix Analysis and Efficiency

237

To determine optimum level of significance a to obtain a minimum guaranteed efficiency Eo, we solve the equation for a: APT

minnz MRE(8,

-

;6,) = E ( a ,A$,(.))

= Eo.

(5.5.11)

Some tabular values are given in Table 5.5.1 for various p and n values.

5.5.3

Comparison of

6,

First, consider the comparison of difference is given by

-s

AS

S+

and 8, (8, and 6, )

6,

-s

and 8,. In this case, the MSE-matrix

(5.5.12) -S

For 6 , to dominate 6, based on MSE matrices, we must show that (5.5.13) is nonnegative definite. Thus, consider the quadratic form

02C’(HN-’C) {2E[xi;l(A2)] -(P

- (p-3)E[x;;i(A2)])

+ l)e’(ss’)eEIxi~3(A2)1

for a given nonzero vector C = maximizing over all t , we have

(tl,

(5.5.13)

. . . ,lP)’. Dividing by C’N-’l(> 0), and

(5.5.14)

-S

which does not always hold. Hence, 8, does not dominate 6, uniformly.

Chapter 5. ANOVA Model

238 -S

The MSE-efficiency of 6 , compared to 6 , is given by

-s Under Ho, the MRE (6,;6,) becomes

-s -

and as A2 -+ 00, MRE(6,; 6,) -+1. S+ -S In order to compare 6 , and 6,, we note from (5.3.6e) that

-

M4 (6;)

- M5

(6,”’)

The r.h.s. of (5.5.18) is positive-semidefinite, since

- S+

Thus, the MSE-based efficiency of 6 ,

-s

relative t o 6 , is given by

5.5. MSE Matrix Analysis and Efficiency

J51

--f

1 from above. -S . S+ The MSE-efficiency of 8, compared to 8, may be written as

239


240

-s+ -s

It may be shown that under Ho, the MRE (8, ; O n ) 2 1, and as A2 -s+ -s MRE (8, ;en)tends to 1 from above.

5.6

-+ co,

Improving the PTE . PT

As in Chapter 4, we consider the improvement of the PTE 8, of 8 for p We recall that the P T E of 8 is given by PT

based on 6, and given by

n ) G I Fp-l,m(cy))

0,

= 3, - (8, - 6

6,.

Now, for p 2 4, we have the Stein-type estimator of 6

APT+

Thus, for p 2 4, we may consider the estimator 8, 6, via the test on 6 = 001,. We may write APT+

8,

2 4.

.

= & l ( L< F p - l , m ( Q ) )

(5.6.1)

-S

by combining 8, and

-S

+ 6,I(L,

2 -s - s

Fp-l,n(Q))

< F p - l , m ( Q ) ) + 0, - 6 , I ( L < Fp-l,m(cx)) -s = 8, - (6, - G n ) l ( L < F p - l , m ( @ ) ) =6,I(L

-s A

PT

= 8,

- c(8, - ~,)c;'I(L2

F ~ - ~ , ~ ( ~ ) ) .

,.PT+ are then given

The bias vector, b6(6cT+)and the quadratic bias of 8, by

(5.6.3)

5.6. Improving the PTE

241

respectively. Note that (5.6.5) and (5.3.lf) are similar except for t,. The corresponding MSE-matrix and the risk expressions are given by

M6 (6ET+)= h’f4 (6:)

,.PT+

PT

respectively. The percentage improvements of 6 , over 6 , under the null hypothesis Ho for cr = 0.05 and 0.10 are given in Table 5.6.1.

-s

Similarly, we can make an improvement on 6 , by using the Stein-typeestimator of the variance 0 2 ,as in Section 4.5.6 of Chapter 4. We write estimator of 6 as -IS

.,

6, = 6,

+ { 1 - c ( m + 2)LG145s(L,)}(6, a,),

(5.6.8)

-

where

-S

This estimator improves over 6 , uniformly. The risk function of by

R7 (6; : o-~I,)= R4 (6; : 0-~1,) -

+

-IS en

( P - 3)2 [ m ( p 2 m (p+m+1)2 m+2

is given

+ 3)


242

Table 5.6.1 Tables for Percentage Improvement of

enPT+ over O PT n ~

a

2

18

20

22

24

26

28

30

32

0.05

4 5 6 7 8 9 10 11 12 13

19.62 36.20 46.82 52.87 56.14 57.82 58.52 58.49 57.74 56.08

20.16 37.29 48.27 54.54 57.97 59.86 60.90 61.39 61.41 60.92

20.59 38.15 49.39 55.78 59.28 61.29 62.52 63.30 63.74 63.83

20.95 38.85 50.28 56.74 60.28 62.34 63.39 64.66 65.37 65.82

21.25 39.43 51.01 57.51 61.06 63.15 64.59 65.70 66.59 67.82

21.50 39.91 51.61 58.13 61.68 63.80 65.29 66.51 67.54 68.40

21.72 40.32 52.11 58.65 62.20 64.32 65.87 67.16 68.30 69.29

21.91 40.68 52.54 59.09 62.63 64.76 66.34 67.70 68.90 70.01

a

2

18

20

22

24

26

28

30

32

0.10

4 5 6 7 8 9 10 11 12 13

22.59 38.94 48.39 53.40 56.02 57.40 58.01 58.00 57.32 55.79

23.12 39.94 49.67 54.85 57.64 59.27 60.26 60.80 60.89 60.48

23.54 40.73 50.66 55.94 58.83 60.60 61.83 62.70 63.25 63.43

23.88 41.37 51.45 56.79 59.73 61.61 63.00 64.09 64.93 65.48

24.17 41.89 52.09 57.47 60.45 62.40 63.90 65.16 66.19 66.99

24.42 42.33 52.62 58.03 61.03 63.03 64.62 65.99 67.18 68.15

24.63 42.71 53.07 58.49 61.51 63.55 65.21 66.67 67.97 69.07

24.81 43.03 53.45 58.89 61.92 63.98 65.69 67.24 68.62 69.82

P

P

Source: Bashir Khan Ph.D. Thesis, 1997

ANOVA Model: Nonnormal Errors

5.7

In this section, once again, we consider the ANOVA model

- nr='=, n;:,

Y=BB+E as in (5.1.1) and (5.1.2) except that E unknown but satisfies the conditions (i)

(5.7.1)

F ( E ~where ~ ) the cdf F is

E ( E )= 0 and (ii) E ( E E '= ) 021,, (0 < u2 < 00);

lim (A3,)

n-cc

= lim n-cc

lim (A,) = A

n+cc

(2) x j

and

=

J,

(0 < Aj

= I, - l,l&.

< l),

(5.7.2)

(5.7.3)

(5.7.4)

Further, lim J, = J = I, - l,l$

n-co

(5.7.5)

5.7. ANOVA Model: Nonnormal Errors

243

5.7.1 Estimation and Test of Hypothesis

We define unrestricted and restricted estimator of 6 as before, as - the 6, = (yl,.. . ,Yp)'and 6 , = tlPlLN6,, respectively. Also, the unbiased estimator of u2 is defined as P

ni

(5.7.6)

and s: is a consistent estimator of 0'. In order to test the null hypothesis HO : 6 = Ool,, we consider the statistic -I

L,

=

n6, Jkh,3,8,

(5.7.7)

s:

Under Ho, L, is distributed approximately as a central chi-square variable with ( p - 1) d.f. On the other hand, under fixed alternatives

Hd : 6 = 601,

+ 6,

(5.7.8)

we can write

and

Now, by the standard central limit theorem together with the Slusky's theorem, we have

(5.7.10) It follows that L, = ns;'l18, - 6n112+ 00 as n fixed alternatives H 6 ,

Pa{& > K }

-+ 1

for all

--f

co. Consequently, for all

K.

(5.7.11)

Therefore, to obtain a meaningful asymptotic distribution of L, as n we consider a class of local alternatives {IS(,)} defined by

q,): e(,) = eoip+ n-1/26.

+ 00,

(5.7.12)

Then we have the following theorem: Theorem 1. Under {K(,)} and the assumed regularity conditions, we obtain the following as n 4 co:


244

(5.7.13) (v) C, is asymptotically distributed as a noncentral chi-square with p - 1 d.f. and noncentral parameter A2/2, A2 = C T - ~ ( ~ J ' A J= S )S*J'AJ6*. The proof follows by defining r2 = A:l2Ip, then r1J?i= I, - Ak121pI;A:12.


5.7.2

We combine the unrestricted and the restricted estimators of 8 t o obtain the PTE and Stein-type estimators as follows: (i)

,.PT

e,

where (ii)

-s

= 6, -

(6, - &,)I(C, < c,,~),

(5.7.15a)

is the upper a-level critical value of C, for finite samples,

-

en = e,

(5.7.1513)

- c*(Bn - 6 , ) ~ ; 1 , c* = (p--J)m m+2 1

and

are the usual Stein-rule estimator and positive-rule Stein-estimator, respectively, since C, # 0 with probability one.

5.8

ADB, ADQB, ADMSE, and ADQR of the Estimators

In this section, we present the asymptotic distributional bias (ADB), quadratic bias (ADQB), MSE-matrices (ADMSE), and the quadratic risks (ADQR) of the estimators of 8. Accordingly, consider the estimator O;, and let W be a p.s.d. matrix and the quadratic loss function

qe;, e) = n(e:,- e)'w(e; - e) = tr[W{n(e; - e)(e; - B)'}]. (5.8.1) Let the ADM be M(8;) = E{71(0; - e)(e: e)'}. Then the ADQR -

of 0; is given by

R(0;;W) = tr[WM(e;)].

(5.8.2)

5.8. A D B , ADQB, ADMSE, and A D Q R of the Estimators

245

(0; -0) is equivalent In many situations, the asymptotic distribution of fis;' to f i u - ' ( 6 , - 0) under fixed alternatives 8 = 1901, 6, and thus t o obtain appropriate asymptotic distribution of 6s;' (0; - e),we consider the class of local alternatives

+

q,): qn)= oOi, + n-1/26

(5.8.3)

- @(,I) as

and define the asymptotic cdf of fis;l(e;

G;(x) = n-m lim P { &is;'(e;

-

t9(,)) 5 x/K(,)}

(5.8.4)

if it exists. Then the ABD and ADQB are defined by

b(8;)= J x d ~ ; ( x ) ,

(5.8.5)

5.8.1 Asymptotic Distribution of the Estimators under Fixed Alternatives First, note that under 0 = 001,

&is;l(en

+ 6, -

e) N,(o,A-~).

(5.8.9)

Now, consider the quadratic difference -2

ns,

APT

jp,

-

en112=

~S;~II(& -

= L,I(LTL

~ , ) I I ~ I ( L L E I , o
x:(a)), then 6: = 6 , A

( q = p - 1).

-*

-s

(iv) If g(C,) = (1 - c L i l ) ; 0 < c < 2(p - 3), then 8, = 6,. . S+

(v) If g(L,) = (l-cLil)I(L, > c ) , then 6; = On ( c ) ,which is the Lindley (1962) estimator of 6 . The estimator 0; shrinks toward the linear subspace defined by 6 = &l,.

5.9.1

Confidence Sets and Coverage Probabilities

We consider the class of confidence sets given by

c*(e;) = ( 6 ; a - 2 ~~6 6;iiL I x;(Y)}.

(5.9.6)

More specifically, we consider the following confidence sets: (i) CO(8,) = ( 6 : 0-~116- 6,I&

< $(y)}.

PT

(iii) C P T ( 6 , ( a ) )= ( 6 : 0-~116- 6,"(a)IIk

< xX(y)}.

-S

(iv) Cs(6:) = ( 6 : o-2)16 - 6,llL < $(r)}.

().

-

S+

CS+(~:+(~ =)()6 : a - 2 1 p - 6 ,

( c ) ~ ~ ~ - (n-'/2~,,(0)

5o

- x , ~ - ' / ~ T , * ( o )---f )P 0.

(5.1 1.16)

In vector forms we may write

n - 1 / 2 ~ n ( B-n (I, ) - ~ , ~ ; A ) ~ - ~ / ~ T ,Z( o0.)

(5.11.17)

Then, we may write

C,

= Ai2[T,(0)]'JA-1J'[T,(O)]

+ op(l),

(5.11.18)

where J = I, - l&A. Hence, under 8 = 0, as n -+ 00, C, approximately follows the central chi-square distribution with ( p - 1) d.f., since JA-'J' is an idempotent matrix with rank ( p - 1).


263

5.11.2 Preliminary Test and Stein-Type Estimators We combine the u n r e s t r i c t e d and the r e s t r i c t e d estimators of 0 to obtain the preliminary test estimator and the Stein-type estimators as in Section 5.7.2, which are given by

- PT = a, - (8, - hn)1(cn< c,,~),

(i) 8,

(5.11.19)

where is the a-level critical value from the distribution of C, under No which converges to as n -+ 03. The Stein-type estimators are given bY

~i-~(cr)

-s

-

(ii) 8, = 8, - c(an - B,)c;'I(c, where

E,

-s+

(ii) 8,

(5.11.20)

and c = ( p - 3) and

-+0

as n

= 8,

+ (1 - CC;~)I(C,

-+ CQ

> E,)

> c)(&

-

6,).

(5.11.21)

as in Section 5.7.2. Note that (5.11.20) involves I ( & > en), since C, assumes the value 0 with positive probability.

5.11.3

Asymptotic Distributional Properties of R-Es t imat ors

First of all we know from the basic theorems of Chapter 4 of Hajek and Sidak (1967), that as R CQ, -+

n-1/2T,(0) and under fixed alternatives C,

..PT

--+

00

-

Np(O,A$A)

as n

-s

-+

(5.1 1.22)

m. Also, under fixed alternatives ., S f

- 0) are equivalent all the estimators f i ( 0 , - 0), fi(0, - 0) and fi(0, in distribution t o that of fi(0, - 0). Hence, we consider the class of local alternatives

K(,) : qn) = 0~1,+ n-1/2S.

(5.11.23)

It may be mentioned that this class of alternatives is contiguous to that of 0 = 601,. Thus, we have the following theorem: Theorem 1. Under {K(,)} and the assumed regularity conditions w.r.t. the model (5.11.1), as n + 00, we have

&(a, (ii) &(a, (i)

- 601,)

- N,(6,02A-');

- 6,)

N,(J6,02JA-l), J = (I, - I,l;A),

-

o2 = A $ / T ~ ( $q5), ,


264

and = H p - l ( z ;A’), A’ = a-’ (6’J’AJ6). (vi) lim P { L , 5 x:JK(,)} a-Di)

(5.11.24)

Proof. Note that both 8, and 6, (or 8,) are translation-invariant estimators. Thus, without loss of generality, we may assume 8 = 0. Also, by the definition of the estimators (5.11.11a) and (5.11.10) and (5.11.15a, b) we have n’/’16, 61 = O p ( l ) ,while under {K(,)} n’/’(6’ - &lP) = 6 = O(1). Thus, n1/216nl (and n1/’/6,1) = Op(l).Observe also that under 8 = 0, n-l/’T,(O)

= An1/’8,y($,

4 ) + op(l).

(5.11.25)

Hence, utilizing contiguity under { K(,)} t o those under 8 = 0, we obtain the distribution of n’/’(6, - &lP) as N,(6, a2A-‘), 0’ = A”,/r’($, 4). Similarly, the asymptotic distribution of f i ( 6 , -8,) is N,(J6, a’JA-l) under {K(,)}, while under 8 = 0,

+

(Ae1 - l p l ~ ) n - l ~ ’ T , ( 0 =)&(en - en) oP(l).

(5.11.26)

Also, -

( l p l ~ ) ~ - l ’ ’ T n (=0n) ‘ / ’ ( 6 , - 601,) and under { K(,)},

nq8,

- 601,)

-

+ oP(l),

(5.11.27)

Np(0,a2B).

As for the asymptotic distribution of L,, as n 8=0as

-+ 00,

we consider 13, under

L, = Ai2[T,(0)]’J’A-1J[T,(0)] + op(l),

(5.11.28)

which follows a central chi-square with ( p - 1) d.f. Under {K(,)}, Ln is approximately distributed as a noncentral chi-square with ( p - 1) d.f. and noncentrality parameter $A2, A’ = Ai2--y2(+,4)(6’J’hJ6). As a consequence of Theorem 1, we obtain ADB, ADQB, ADMSE, and ADQR of the five estimators, as given below following Section 5.8.3.


265

5.11.4 ADB, ADQB, ADMSE, and ADQR For UE, ( 6 ) , (i) bl(6,) = 0 and Bl(6,) = 0; (ii) Ml(6,)

= a2A-’

(5.1I .28a)

(en),

For RE,

(i) bz(6,) = -6 and (ii)

and Rl(6,; W) = a2tr(WA-’).

M 2 ( e n ) = u2B

&(en) = A2,

A2 = 0-~(6’5’AJ6);

+ 66’ and Rz(6,; W) = a 2 t r ( W B )+ 6’Wd.

(5.11.28b)

.. PT

For PTE, (6, ),

- PT

(i) b3(6, ) = -JBHp+a(~;-l(~);A2) and

,.PT

2

B3(6, ) = A2{~p+1(X;-1(.);A’)} ;

..PT

(ii) M3(6, ) = u2A-’ - C ’ J A - ~ H , + ~ ( X E - ~A2) (~);

+ (Jdd’J’){

2Hp+1(x&1(a);A2) - Hp+3(x;-1(~);A2)}

and . PT &(en ;W) = cT2tr(WA-’)

- u2tr(WJA-1)Hp+1(x;-l(a); A’)

+ (”JWJ”) { 2 H p + 1 ( X;-I For SE, (i)

( a ) A’) ; - H p + 3 (x;- 1( a ) A’)} ;

*

(a:), -s

b4(6,) =

-(P - 3)JaE[x;-‘(a);

A’)]

and

B4(3:) = ( P -3)A2{E[~G:l(A2)]}2; (ii) M4(6:) = u’tr(WA-’) - (P-3)E[x;;I

R4(6:;W)

-

O~~~(WJA-’{~E[X~:~(A~)],

(A2)]} + (P-3) (P+ 1)(6’J’WJW [xi$ (A’)]

= a2tr(WA-’)

-

a2tr(WJA-’)

x {2E[Xp-:1(A2)]

- ( P - 3)E[X;:1(A2)]}

+ ( P - 3)(p + ~)(G’J’WJG)E[X~;~(A’)]. ..S+

For PRSE, (6, ),

(5.11.28d)

266

Chapter 5. ANOVA &lode1

where (5.11.30)

By computing (i) E[U1], a2EE[U1U’,]and a2E[U{WU1], (ii) E[Uz], a2E[U~Ua] and o’E[U;WUz],

and

5.11. Nonparametric hlethods: R-Estimation

267

(iii) E[U3],02E[U3Ui]and 02E[UiWU3],

(5.11.31)

we obtain the bias, MSE matrix, and the quadratic risks of the PTE, SE, and PRSE. Next, we discuss the recentered confidence sets based on the five R-estimators. First, we consider the usual (1-Y)% confidence set, namely Co(6,) defined by

~ ‘ ( 6 , )= { O

: nai2/16- l~,ll&5 k , } ,

(5.11.32)

a = A:/?:. For some positive a (< co),we define a where k, = $(y), and : consistent estimator of Y($, 4 ) by

1

2

Yn = P j=1 ~

T ~(6, ,

+ n-1/2alp) - T ~(6,, - n-I/2alP) 2njan

(5.11.33)

Then, the asymptotic coverage probability of the set, Co(an)may be written as 2 -1

P K ~ , , ) ( C * ( ~=n )n-wx lim ) PK(,,{A, n

n-cc lim

= Po(&2

5 Icy)

IlTn(O)ll%_l i Icy}

= 1 - y.

(5.11.34)

We define the confidence sets by

c*(e:) = ( 6 : na;21p - e;&

5 k,},

(5.1 1.35)

where 6; = A6,

+ ( I - A)eng(Ln),

A = l,lbDn,

(5.11.36)

as before (see Section 5.9.4).

Now, using the transformations as in Section 5.9.1, we have w1

where

n1/2on-11”D1/2an 1 n and w2 = n1/20;1rLDi/26n,

1

I’2 =

Dk‘21,. Then, as n

(5.11.37)

+ 00,

(5.11.38) Hence, as n 4 co,

na;2j16 - 6:11D,-(732 2 2 -~

2

+ )I I ~~ -I wlg(/IwiI12)I12.

(5.11.39)

The asymptotic coverage probability of the set C*(6:)is then given by lim P(C*(6:))

n-w

= Pq

{ “732

- w2)2

+ I l V l - ~lg(l/w1112)l1215 4 )


268

which is the same as the expression at (5.9.10). The rest of the analysis follows based on the material in Section 5.9.1 and 5.9.2 and is not repeated.

5.12

Conclusions

In this chapter, we considered the simple ANOVA problem with regard t o the improved estimation of the parameters of the model (5.1.1). Accordingly, we discussed the unrestricted, restricted, PTE, SE, and the PRSE of the parameters based on normal theory as well as the nonnormal theory in an asymptotic setup. We added the nonparametric R-estimation methods t o enhance the scope of the methodology. In addition, we discussed the recentered confidence set estimation and their finite sample as well as asymptotic properties of the coverage probabilities under local alternatives.

5.13 Problems 1. Prove Theorem 2 of Section 5.1.3. APT -S ,.S+ 2. Determine the bias-vector of 6 , , 8, and 8, using the expressions at (5.2.4-iii).

..S+ , 8,, and 6,

-PT - S

3. Determine the MSE matrices of 8, at (5.2.4i-iii).

using the expressions

..S+

4. Show that 6;' is the convex combination of 6, and 6,, and hence On is a preliminary test estimator of 6 for p 2 4. -S

5. Determine lM1(an)1,M2(6,)/,M s ( 6 r ) I JM4(6:)1, and M~(6;+)l,and verify the expressions for the relative efficiencies based on MSE matrices. 6. Prove Theorem 1 of Section 5.7.1. s+ 7. Consider (5.8.16), and show that E{n~;~/j6, - 6 / ' } 0 as n 4 oc) under fixed alternatives 6 = BOl, 6. .--)

+

8. Show that

fis;'

(6;

- 601,) 2

[(X+ S*)- (Y + Jd*)I(llY + J6*j12 < ~ & ~ ( a under ) ) ] local al-

ternatives 8(nl = 901,

( fiii,

+ n-'f26

where

)} (Section 5.8.2). V

9. Verify that &s;' (6: - &l,)= X - ( p - 3)(Y given a t (5.8.24). 10. Prove Theorem 5.8.1, equation (5.8.22).

1,

I

11. Determine (Ms (6t['T) IM1 (6;) and IMs (6:') sions given at (5.8.29), (5.8.32), and (5.8.36).

+ JS*)llY -+ J6*/1-2

I

using the expres-

5.13. Problems 12. Let

Y i= 8 + e,, i

= 1,. . . , n , where e, is distributed as f(e,;v,a’) =

c:=, eze,I ( ~ + ~ );/v,a2 ’ > 0. Define a,,

-PT

-S

,.S+

6,, 8, , 6,, and 6 , 4 1 + vu2 as given in Section 5.2.1. Determine the bias vector, MSE matrices and weighted risk expression for the five estimators. Prove Theorem 1 of Section 5.9.2. Prove (5.20.2a-d) of Section 5.10.1. Prove the asymptotic linearity results given by (5.11.15a, b). Show that \

13. 14. 15. 16.

269

I

n’/%, = n 1 ’ 2 8 ,

+ op(l),

where 6, = A l e n l + . . . + Apen, using the asymptotic linearity results. 17. Verify the expressions for ADR and ADMSE matrices and for ADQR. given by (5.11.28a+) using (5.11.31) as the computational step. 18. Prove that 13, a t (5.11.12) approximately follows the central chi-square distribution with ( p - 1) d.f. under Ho : 6 = 6 0 1 ~ .

+

19. Show that C, = n(6, - 6,)’A(6, - 8,) op(l). 20. Verify the expressions for the asymptotic coverage probabilities of the confidence sets given by (5.11.35) explicitly with that of the Section 5.9.


Chapter 6

Parallelism Model Outline 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10 6.11 6.12

Model, Estimation, and Test of Hypothesis Preliminary Test and Stein-Type Estimators Bias, Quadratic Bias, MSE Matrices, and Risk Expressions Comparison of the Estimators of the Intercept Parameter Estimation of the R.egression Parameters: Nonnormal Errors Asymptotic Distributional Risk Properties Asymptotic Distributional MSE-matrix Properties Confidence Set Estimation: Normal Case Confidence Set Estimation: Nonnormal Case Nonparametric Methods: R-estimation Conclusions Problems

In this chapter, we consider another important model, namely the parallelism model, belonging to the class of linear hypotheses useful in the analysis of bioassay data, shelf-life determination of pharmaceutical products, and profitability analysis of factory products in terms of costs and outputs, among other applications. In this model, as in the ANOVA model, p independent bivariate samples { ( x a l yal , ), (xa2 ya2 ), . . . , (xan,,,yan, ) = 1 . .. ,PI are considered such that y a j n/(Sa Paxa5,o2) for each pair ( c x , ~ )with fixed x a j . The parameters 6 = (61, . . . OP)’ and ,6? = (PI . . . ,PP)’are the intercept and slope vectors of the p lines, respectively, and o2 is the common unknown variance. In this model, it is common to test the parallelism hypothesis NO : p = P o l p (where Po is an unknown scalar) against the alternative HA: a t least one pair of the components of p differ. Hence, we designate this model as the “parallelism model”. The main objective of this chapter is to study the properties of various estimators of 6 and ,f3 when it is suspected but not

-

+

271

Chapter 6. Parallelism Model

272

certain that the parallelism hypothesis HO : p = P o l p holds. Accordingly, we consider (1) the unrestricted estimator (UE), (2) the restricted estimator (R.E), (3) the preliminary test estimator (PTE), (4) the James-Stein-type estimator (JSE) and (5) the positive-rule Stein estimator (PRSE) of 8 and p. In addition, we consider the properties of the recentered confidence sets based on these estimators as the center of the sets with fixed volume. Both small and large sample cases are treated for the problems stated above. In addition, we include the R-estimation theory of point and confidence set estimation. Interested readers may refer to Lambert and Saleh (1985), Akritus, Saleh, and Sen (1985), Saleh and Sen (1985a), Saleh and Hassanein (1986), and Robert and Saleh (1991) for more information.

Model, Estimation, and Test of Hypothesis

6.1

6.1.1. Parallelism Model Consider the p simple linear models

+ Pax, +

ya =

€ ,,

a = 1, * .

. ,p,

(6.1.1)

where

Y , = ( ~ ~ 1..,y a. R n ) ’ , 1,xa =

=

(%I

( I , . . . ,I)’ - a n,-tuple of 1k,

7 .

. . ,%na

(6.1.2)

)I,

N,,( 0 ,021n,), In, being the identity matrix of order n,. and E, Generally, the main objective of this model is the test of the parallelism hypothesis HO : = P o l p (POis an unknown scalar) against the alternative H A : ,B # polp. In this chapter, we pursue the problem of improving the usual estimators of 6 and p when it is suspected but not evident that HO holds. N

6.1.2

Estimation of the Intercept and Slope Parameters

Consider (6.1.1) and (6.1.2) using the sample information and the model. The unrestricted estimator of 6 and p are given by the LSE/MLE as follows: (a)

where

-

3,

and

= Y - T,B,

(i) Y

=

(b)

p, =

PI^^, . .. ,&,,)’,

,,... ,Ypn,)’, (ii) T,

(v1,

(6.1.3) = Diag(zl,, ,... , znp),

)(I:, Y-) 1 (iii) Pan, = x L Y a - & (1k,xn , (iv) n,Q, = x&xa - n, (1;- x, l29 n,Q,, (6.1.4) (v) Zxna = ,(lb,Xol) 1 and yen, = L n, ( l n n Y , ) . Finally, the unbiased estimator of o2 is given by

s : = ( n - 2p1-l

c P

llYa

a=l

-&Jnn

- P*,n,Xall 2 *

(6.1.5)

6.1. hdodel, Estimation, and Test of Hypothesis In the case where the slopes are equal, meaning estimators of 8 and P are given by

where

(i)

an

(iii) H = I, -

273

P

=

pol,,

(ban,... ,An)' = h n l p t (ii) n Q ; ; , (iv) DT; = Diag(nlQ1,. . . ,n,Q,). nQ =

1,l ;D

the restricted

= CZ=ln,Qa,

(6.1.6)

The following theorem gives the exact distribution of the estimators, whose proof is straightforward.

Theorem 1. Under the assumed conditions

( il 1 ) - N2, { ( 8 ) ; a 2( -TnD22 D1l

-TnD22 D22

( Penn -' { ( TnHP ) ( gii -P ) ;u2

N2,

where (i)

pl,

=

,b

1 1 D;, nQ

P

D'2

D22

)} ,

(6.1.7a)

)},

(6.1.7b)

.

, (ii) D11

+

= N-l

T,i , i ; ~ ,

(6.1.7d)

+ TnD22Tn; and

N =

and DT2 = - 2 nQ1 P l'Tn, P Diag (nl,.. . , n,), (iii) DT1 = N-' nQ (6.1.8) (iv) H is defined by (6.1.5 (iii)) and D22 is defined by (6.1.5iv). We will need these distributions later to compute the bias, the MSE matrices, and the weighted quadratic risks of the estimators of the form

where A is an idempotent matrix of rank 1 and g ( L n ) is a nondecreasing function of the test-statistic L, for testing the null hypotheses HO : P = Pol, against the alternative H A : p # ,001, given in Section 6.1.3 below.

Chapter 6. Pardelism Model

274

Table 6.1.1 Parallelism Data

First Sample

I I

Second Sample

I

pth Sample

Data

Estimate

I

Variance Estimate I

In the sequel, we will need the following statistics: (a) Pooled estimate of slope:

boon= nlQ16lnI +...+n,Q,PPnp nQ

(b) Pooled estimate of variance:

sz =

( n - 2p)-'

c",=, llya

-

-

ba,naXaI12*

6.1.3

Test of Parallelism

In this section, we consider the testing of the parallelism hypothesis Ho : p = polp. The test-statistic is given by the following theorem:

Theorem 2. The likelihood ratio(LR.) statistic for the null hypothesis Ho is given by (6.1.10) Under Ho, L, follows the central F-distribution with (p - 1 , m ) d.f., where m = n - 2p and under H A , it follows the noncentral F-distribution with ( p - 1,m ) d.f. and noncentrality parameter A2/2, where

(6.1.11) Proof. The likelihood ratio statistic is obtained by using the maximum of the likelihood function under HO and H A , which are given by

6.2. Preliminary Test and Stein-Type Estimators

275

and LA

= Sin(&)-nexp

{ - ;},

c P

2

nsA =

llya - 8 a ~ n a paxa112 = ms:,

a=l

respectively. Hence, the likelihood ratio statistic is the result of

(6.I. 13)

since P

P

a=l

m=l

C I I Y-~ 6a1na - Baxalj' = C I I Y-~

e a l n a - paxa/l2

+ P;H~D;~@,. (6.1.15)

p' H D; ;HP- as the L R statistic for testing Ho against Thus, we take L, = (p-l)s,z

HA. To obtain the distribution of 13, under H A (or Ho), we consider the orthogonal matrix I' = (rl,r2). Here rl is a p x p - 1 matrix, and I'z is a pvector such that rLrl = 0 and rlr; + r2ri= I, SO that = I, - r2rL.Also, I' diagonalizes the symmetric idempotent matrix D22 1/2H'Di;HDip of rank ( p - I), that is,

112 1/2 Defining w = a-1rD22 0, and q = a-lrDz2 0,we have w = N p ( q ,Ip).Now partition w = (wi,wz)' and q = ( q i ,q 2 ) ' , where w1 is a ( p - 1)-vector and

( :;

("I

; I,( ti1 :I}.

is a scalar random variable. Then Hence, llw1112 follows the noncentral chi-square distribution with ( p - 1) d.f.

w2

and noncentrality parameter A2/2 with A2 = ljqlll2 - O'H'D-,'HP. 4; Then, L, under H A follows the noncentral F-distribution with ( p - 1,m) d.f. and noncentrality parameter A2/2 since ms2/a2 follows the central chi-square distribution with m d.f. independent of Clearly, under Ho, C n follows the central F-distribution with ( p - 1,m) d.f.

Pn.

6.2


Following Chapter 5, we now consider various estimators of the intercept 6 and slope vectors p.

Chapter 6. Paralielism Model

276

6.2.1 The Estimators of Intercepts and Slopes The unrestricted and restricted estimators of 8 and @ are given in Section 6.1.2. We consider the preliminary test estimators of 8 and @ as follows:

KT= P , and

- PT

en

=

-

HP,W, < Fp--l,rn(a))

e, + T,HP,I(L,

c)}

(6.2.3a)

< c) + P;tI(L, > c )

bn + (1 - cLG1)1(L, > c)HP,

(6.2.3b)

with p 2 4. We present the following data set related to shelf-life, to illustrate the summary statistics and the test-statistics. The tabular values contain results of six experiments conducted to determine the shelf-life of a drug by setting fixed concentrations of a component of the drug and observing the times (years) to destroy the concentration.

6.2. Preliminary Test and Stein-Type Estimators

277

Table 6.2.1 Shelf-life Data Batch

Time Concentration (years) 1 0.014 100.4 0.230 100.3 0.514 99.7 0.769 99.2 1.074 98.9 1.533 98.2 2.030 97.3 3.071 95.7 4.649 94.5

2

3

0.022 0.118 0.272 0.566 1.165 2.022 3.077

100.7 100.6 100.3 99.4 98.6 97.6 96.4

0.025 0.275 0.547 0.797 1.041 2.058 2.519

100.2 99.7 99.2 99.0 98.8 96.4 96.2

Batch 4

Time Concentration (years) (%I 0.066 100.4 0.343 100.0 0.533 99.5 0.802 99.3 1.033 99.3 1.538 98.2

5

0.011 0.310 0.624 1.063

100.5 99.8 99.1 98.4

6

0.011 0.310 0.624 1.063

100.1 99.5 98.5 98.4

Adapted from Roberg and Stegman (1991)

Table 6.2.2 contains the summary statistics related to Table 6.2.1. Table 6.2.2 Individual Batch Regression Estimates Intercept Slope Batch (%) (%/year) 1 100.49 -1.515 2 100.66 -1.449 3 100.25 -1.682 4 100.45 -1.393 -1.999 5 100.45 6 99.98 -1.701 Pooled batch regression with 99.90 -1.534

Weight MSE niQi 0.019 14.63 0.043 7.83 0.062 5.14 0.035 1.21 0.61 0.011 0.124 0.61 equivalent slopes 0.047

The value of 13, = 1.64 with p-value 0.186.


278

6.2.2 Bayes and Empirical Bayes Estimators of Intercepts and Slopes

(@,,on)’

be the intercept and slope parameter, and let 8, = Let 6 = (6’, ..I ..I be the unrestricted estimator and 2, = (6,, 0,)’ the restricted estimator of (6’,/3’)’.We then write -I

-I

(6.2.4)

- NZp{ ( E:i Eiz )}

-

By sufficiency considerations, we can reduce the model to the observation of the statistic, 8,

6, a2

m*=n-2p+2and

a2x

c P

s: = ( n - 2p+ 2)-l

and m*s:

lly* - &Aa - pax,p.

~ where - ~ (6.2.5)

Cr=l

The estimator sp of a2 is the best scale invariant under the loss

L(a2,d ) = (1 -

$)

2

(6.2.6)

p

= p o l p , we

)} ,

(6.2.7)

and dominates the mle of a2.Since our null hypothesis Ho is select the prior

6

- NzP{ ( ) ( ; 7’

D1l

-TnD22

-TnD22 D22

which gives the posterior distribution of 6 conditionally on a2 as

where 6’ =

( :; ).

Conditionally on cr2, the associated posterior mean of

618, may be written as

Now, using the noninformative prior on a2, i.e., ..(a2)= 1/a2?we find that the Bayes estimate of a2 is s.: Similarly, the empirical Bayes estimator of 6* and (cr2 T ~ ) - ’ based on the marginal distribution of 8,, namely

+

( -ZTn-%2y22)} ’

(6.2.SO)

~

6.2. Preliminary Test and Stein-Type Estimators is given by

279

a, and 6,, since Em($,) = Polp

E(6,) = 6,

and

where Em[.]stands for the marginal expectation of the r.v. in and 8, are empirical Bayes estimators of P o l p and 6 . Further, COV[(&

-fin)] =

(2+ T’)HD~’H’

(6.2.11)

(.I.

Thus,

b,

(6.2.12)

and

(P, - b,)’Di;(P,

-

p,)

2

+ T2)&1.

(0’

(6.2.13)

Therefore,

and the estimator of 1/(02+ T’) is given by (6.2.15) Hence, the empirical Bayes estimator of 6 may be written as

=

8,

+ ( 1 - cL,1) (6, - 8,),

(6.2.16)

where

with m = n - 2p being the LR test for Ho : P = polp. The corresponding positive-rule empirical Bayes estimator of S is given by

&+:

= L,+

( 1 - c.c,1) I ( L , > c ) (6, - h,),

(6.2.18)

which coincides with the PTE approach for the shrinkage estimators (see Section 6.1.4).

Example 1. Consider the data of Tables 6.1.1 through 6.1.3. We illustrate the computation of estimators of the intercept vector 6 . The computation of the estimators of p may be obtained t o verify these computations. Table 6.1.2 shows that 13, = 1.64 with pvalue 0.186. At the 0.25 level of significance, the hypothesis of equality of the slopes is reAPT -S - S+ jected. The formulas for 0, , O n ,and 6 , are given by (6.2.la) through


280

(6.2.3b). Accordingly, we have (i) The unrestricted estimator of 8 is 6 3 7 = (100.49,100.60,100.25,100.45,100.45,99.98)’. (ii) The restricted estimator of 6 is 6 3 7 = (99.50,99.90,99.90,99.90,99.90,99.90)’. (iii) The PTE of

-

-PT

6 is 8,, = 6 3 7 = (100.49,100.60,100.25,100.45,100.45,99.98)’. (iv) The -S JSE of 8 is 8,, = (100.50,100.62,100.23,100.47,100.37,99.48)’, here c = -s+ - - S 0.556 and L, = 1.64, and (v) The PRSE of 8 is then 8,, - 8,, = (100.50,100.62,100.23,100.47,100.37,99.48)’.

Bias, Quadratic Bias, MSE Matrices, and Risk Expressions

6.3

In this section, we present the expressions for bias vector, quadratic bias, MSE matrices, and weighted quadratic risks of various estimators. They are given in the sequel based on Theorem 6.1.1.

6.3.1

Unrestricted Estimators of p and 8

(i) bl(P,) = 0, and Bl(P,)

= 0;

(ii) MI@,) = u2D22,and Rl(P,;W)

= u2 tr(WD22).

(6.3.1a)

Similarly, (i) bl(6,) = 0, and

&(en)= 0

(ii) Ml(6,) = 02Dll, and Rl(6,; W) = o2 tr(WD11).

(6.3.1b)

Restricted Estimators of p and 8 bz(P,) = -HP, and Bz(bn)= A’ = o - ~ P ’ H ’ D ~ ~ H P

6.3.2 (i) (ii)

M2(Pn)

=

nQ

+ P’H‘WHP. + HPP’H‘ and R2(gn;W) = a21’iw1, Q (6.3.2a)

Similarly, (i) bZ(6,) = T,HP, and Bz(6,) = CT-~P’H’TLD,-,’T,H/?; (ii) M z ( ~=~02D;, )

+ T,HPP’H’T,

P‘H‘TkWT,HP, where Dil = N-’

and Rz(6,; W) = u2 tr(WD;,)

+

T : I ~ ~ ; T ~ nQ ’

+

(6.3.2b)

6.3. Bias, Quadratic Bias, MSE Matrices, and Risk Expressions

28 1

and

and

%(bf;W) = a’tr(WDz2) - c(p - l)a’tr(WHDzz){2E[~~~~(A’)] (P- 3)E[x;$1(A2)]} + C(P’ - 1)(P’H’WHP)E[x,54,(A2)]. (6.3.4a) Similarly, (i)

-S

b4(en)= -S

c ( p - ~)T,HPE[X;;~(A’)] and

B4(6,) = ~ ’ ( -p ~)‘CT-’(P’H’T,D~~T,HP){ E[x&(A’)]}’;

282


6.4. Comparison of the Estimators of the Intercept Parameter . S+

(ii) Mg(6, ) X

x

= M4(6:) - a2(T,HD22H’T;)

E [ ( 1- c1F~~i,,(A2))21(F,+l,,(a2)< CI)]

{2

-

and

283

w - C1F,-:,(A2))~(Fp+2,,(A2)

< Cl)] < c2)]}*

E[(l - c2F,;13,,(A2))21(Fp+3,,(A2)

..S+

+ (T,HDzzH’T&)

Rg(6, ; W) = &(en;W) - cr2 tr(WT,HD22H’T’,) X

+

-S

E[(1- clF,-:l,,(A2))21(Fp+l,,(A2)

(2w

< .I)]

+ (P’H’TLWT,HP)

- C 1 F ~ ~ ~ , , ( A 2 ) ) 1 ( F P + ~ , , (< A 2C l)) ]

- E[(l - C Z F ~ ~ ~ , m ( A 2 ) ) 2 1 ( F p + 3 ,< m CZ)]} (A2)

(6.3.5b)

where E[Fp>11,,(A2] is the expectation of the reciprocal of the noncentral F-variable with ( p 1,m ) d.f. and noncentrality parameter A2/2. Similarly,

+

E [ F ~ ~ l , , ( A 2 > ~ ( F P + l , , ( A. (6.4.15)

Thus, for a given nonzero vector 4 we obtain by setting (6.4.15) 2 0 the expression below:

6.4. Comparison of the Estimators of the Intercept Parameter

287

Dividing by t’(TnD22T,)k?, and maximizing over all k?, we obtain

APT

Thus, 8,

6,

performs better than ,

PT

performs better than 8,

if A’ satisfies (6.4.17); otherwise, 6,

. PT

. The MR.E(B,

:

6,)

is given by

= 11, - DT; (T,HD22H’T,)Gp+l,m(ea; A’)

+ a-’D,-,’

(T,HPP’H’T,)

{

x 2GP+1,,(&; A’) - Gp+3,m(!:;

(6.4.18)

,.PT

Clearly, as a function of A’, MRE(8, : 6,) has a maximum at A’ = 0, decreases crossing the 1-line to a minimum, and then increases toward 1-line as A -+ 03. The same conclusion will be reached if we consider the trace of the matrices instead of the determinant. -S

Comparison of 8, Relative to is given by

6,.

In this case, the MSE-matrix difference

Ml(6,) - M1($) = c ( p - 1)(TnHD22H’T,) { ( P - 3)E[X;:1(A2)]

+ 2A2E[X,343(A2)]}

-c(p’ - 1)(T,HPP’H’TnE[~~:,(A2)].(6.4.19)

In order that the MSE difference is p.s.d., we must have for every nonzero vector 1, ( P + 1)(t’T,HPP’H’T,t)E

[xi:,

(A’ )]

I ( ~ ’ T , H D z z H ’ T , ~() p{ - 3)E[x;;1(A2)]

+ ~A’E[X;$~(A’)]}.

Dividing by t’TnD22T,t and maximizing over

(6.4.20)

t , we have

+

( P + 1)A2E[x&(A2)] I (P-~)E[X;$~(A’)] ~A’E[x;:~(A’)],

or


288

-S

The inequality (6.4.21) does not hold for all A’. Hence, 8, does not dominate 0, uniformly. Here we have the efficiency expression as

..S+

which is positive. Hence, 6 ,

-s

dominates 6, uniformly under the MSE criteria.

6.4. Comparison of the Estimators of the Intercept Parameter

289

Here we have the efficiency expression as

< ci)] - (T,HPP’H’T,)

x E [ ( 1 - .1F,-:l,,(A2))21(F,+l,,(Az)

x E [ ( 1 - c 2 F 6 ; ’ 3 , , ( A ” ) ) 2 r ( F , + 3 , m ( ~ 2]} - l’p,

(6.4.26)

-S

P+l and c2 = c(p-l). P+3 MRE(6;’ : 6,) is a decreasing function of where c1 = cO A’ with maximum a t A’ = 0, and as A .+ 00, it converges to 1 from above. -S Hence, 6:’ dominates 6 , uniformly with respect to MSE matrices.

6.4.3 Weighted Risk Comparisons of the Estimators In this section, we compare the estimators of 6 based on the weighted quadratic risk functions.

Comparison of 6, Relative to 6,. The risk-difference of the two estimators In this case, is given by

Ri(8,

: W) - Rz(6, : W) = 02tr[W(D11 - Dil)] -

(P’H’TnWTnHP). (6.4.27)

Thus, the risk-difference is f 0 according as

u-~(P’H’T,WT,HP)

2 tr[W(D11 - DT1)].
according as
according as

>

= c2u2 lim ~ { s : n ( P , ~ , > ’ D & ~ ( ( P-~ n-co

= c2u2 lim

n-+co

E(CZ1) = 0, where

since C n 2 0 and Cn (iii)

--f

00

s+

> ’ ~ ; ; s+ ~ (-b ,s>>

E(c,~(I- C L , ’ ) ~ I ( C ,

< c)>

= 0.

,.

Bnl - P T on, -S

(6.5.14b)

s

lim ~ { n s : s ; ~ ( p ,- 6 n-w

= ( p - 3),

in probability;

n-w

= u2 lim

c

bn)~;2>

S+

(6.5.14~)

Hence, pn , and pn are risk equivalent under fixed alternatives. Further, computing the asymptotic dispersion matrix of we obtain

a,,

(6.5.15) 00. &(fin - p) has unbounded dispersion Note that nIj6112 4 00 as n matrix. Hence, the asymptotic distribution of is degenerate. Thus, for any --f

pn

APT -s + 6 asymptotic distribution of ,Bn , 0, and b,S+is the same as that of fi(Pn - p), and that of bn is degenerate.

fixed alternatives Hb : /3 = ,001,

To obtain a meaningful asymptotic distribution of the estimators, we confine ourselves to the local alternatives defined by ~

( : ~ ~ ( =1 ~POI, 1

+ n-1/26,

(6.5.16)

6.5. Estimation of the Regression Parameters: Nonnormai Errors

297

where 6 = (61,. . . ,bE)’ is fixed, ,& E R1,and 61,. .. ,bp are not equal such that Cg=,XaQaba/Q = 0. The following theorem: gives the asymptotic distribution of various estimators under { K(,)} :

Theorem 2. Under 6.5.3, as n -+ 00,

{A’(,)}and assumed regularity conditions of Section

(iv) L, has asymptotically a noncentral chi-square distribution with p - 1 d.f. and noncentrality parameter A2/2, where

A2 = K 2(6’J’A;: J6) = a-2 (6’A;i 6 ),

(6.5.17)

since J6 = 6 from (6.5.16). Now, by virtue of the definition of the PTE of

&(SET- P ) = &(Pn

- P) -

&(Dn

and 8, we have

- BJ(L7Z < &>

and

- e) = &(fin

-

el + T,&(P,

Therefore, using the fact that we obtain the following theorem:

-BJI(C,

-+ x 2p m l ( a T, ) , -+

< t , , ~ . (6.5.18)

To and Theorem 6.6.1,

Theorem 3. Under { Kc,)} and assumed regularity conditions, the cdf of the estimates of /3 and 6 are given by (i)

G , P T @ ) ( ~6, ) = 11-+W lim p{fi(bET - 0) F x l ~ ( , ) ) = lim n-oo

n-o3 lim

p{ h

( B , - P ) Ix ;Ln < L , ~ I K ( ~ ) }

p { f i ( D n- 0 ) L x

;LL L,~IK(,)}

= Q P ( x- 6 ;0 , CJ~B)H,-~ ($1

( a ) ;A’) (6.5.19a)


298 where B = &lplL;

aP(x + Jz; 0, n2Ao)dGp(Z;0, u’Azz),

(6.5.19b)

+

E ( 6 ) = { Z ; O - ~ ( Z Js)’Ag.(Z + J6)2 x&-I(~)}. (6.5.19~) ap(x;p, E) is the pvariate normal distribution with mean p and covariance matrix C, and H,(-; A’) is the cdf of a noncentral chi-square distribution

where

function with m d.f. and noncentrality parameter A2/2.

-s

fi(gnS+

The asymptotic distributions of fi(pn- p ) and - p) are difficult to obtain but the asymptotic bias, MSE matrix, and weighted quadratic risk function can be computed writing

h~iii~ - (~= zAD,, ~ o l p )

1/2

-

(pn - ~

o l p)

cADzii2(bn-

an)^,'

and 6

1/2 -s+

~

(0,2

~

,- ~ o l p = )

J;EDT;,~,/’( P n - ~

O l p)

-1/2

C J ; E D ~ ~ , ~ -, / B~ n( P) L, i l

-

-{ (1 - CLi1)I(Ln < c ) } J ; E D ~(0, ~ ~- Bn).

(6.5.20a)

Similarly, we can write expressions for

(6ET- 0), 6DL:’2 (6: - 0) and fi

fiD;:” 6.5.4

(6:’

- 0). (6.5.20b)

Expressions for ADB, ADQB, ADMSE, and ADQR of the Estimators

Note that under (6.5.1) and (6.5.2) and the regularity conditions (6.5.10iiii), we can obtain the asymptotic distributional bias, MSE matrices, and the weighted risk expressions as follows:

Unrestricted Estimators. (i) b l ( p n ) = lim E n-m

{ fi p

(-n

- p ) IK

(ii) MI@,) = n-m lim E { n ( p n - P ) and

R1

Similarly,

(a,;W )

= 0% [wA22]

(4

=OandBl(pn)=O

( a n - P)’IK(n)} = a 2 A ~ 2

(6.5.21a)

6.5. Estimation of the Regression Parameters: Nonnorrnal Errors

and Rz(p,; W) = o2

1 W1’ P Q

E

+ G’WG.

299

(6.5.22a)

Similarly, (i) b2(6,) = lim E { &(en - O)lK(,)} = T o J G n-w

and

&(en) = U-’(S’J’TOA;~TOJG) = A*’;

(ii) M2(en)= lim E ( n ( 6 , - 0) (6, - e)’lK,,)} n-M

= 02(A;,

+ o-~ToJSG’J’TO)

+

and Rz(6,; W) = c~’tr[WA;~] (G’J’ToWToJG).

(6.5.2213)


300

and R4(6:; W)

= a2tr[WA11] - a 2 ( p - 3)tr[W(ToJAzzJ’To)]

X{2E[X;:l(A2)1

-

(P - 3)E[X,541(A2)]}

+( P - 3)( p - 1)(G’J’ToWToJS)E [~;:3(

A’)].

(6.5.2413)

Positive-Rule Stein Estimators. (i) bs(b:+) = -JS{Hp+i(P-3;

A ’ ) + ( ~ - ~ ) E [ X ~ ~ ~ ( A ’ )>~ p(-X 3 )~] }+ ~ ( A ’ )

6.5. Estimation of the Regression Parameters: Nonnormal Errors

301


302

Asymptotic Distributional Risk Properties

6.6

In this section, we provide the risk analysis of the estimators of 8 with the general quadratic loss function using the matrix W so that James-Stein and positiverule Stein estimators of p dominate. The properties of the estimators of /3 may be obtained following Chapter 5 with the ANOVA model.

6.6.1 Comparison of 6, and

bn

The risk-difference In this case, is given by

Rl(6, : W ) - Rz(6, :W) = a2{tr[W(All -A;,)]

- U-’(G’J’TOWTOJG)).

By Courant’s theorem,

Hence,

R l ( 6 , : W) - o’tr[W(All - A;,)]

+ ~’A’Chmin [ ( T o W T O ) A ~ ~ ]

5 Rz(6, : W) 5 Rl(6, : W) - u’tr[W(All - A i l ) ] +u2A2Chm, [(ToWTo)A22].

(6.6.1)

When A2 = 0, the bounds are equal. Thus, (6.6.1) means 6 , performs better than 6, whenever (6.6.2a) whereas

6,


6, whenever (6.6.2b)

6, performs better than 6, in the interval For W = A;:, tr[Ag:(A1l and worse outside the interval. (O’ Chm, [ (TO&; To)A22)]

)

6.6.2

*

PT

Comparison of 6, * PT

Here we compare 8,

and

6,.

and en(8,)

The risk-difference In this case, is as follows:

PT

R l ( 6 , : W) - R3(On : W) = a’tr[W(All

- (~’J’ToWTO J6) { 2Hp+i(x-;

1( a ) ; A’)

- Ay,)]H,+i(x~-l(*);A’)

- Hp+3(x;-, ( a ) A’)}. ;

(6.6.3)

6.6. Asymptotic Distributional Risk Properties

303

Thus, the R.H.S of (6.6.3) is nonnegative whenever

. PT

In this range, 8, ever


6,,

whereas

6,

performs better when-

t r [ w ( A l ~- A ; i ) ] H p + 1(xf-1 ( a ) A’) ; C ~ ~ ~ ~ [ ( T O W T O{2Hp+1(~;-1(a); )AZZ] A’) - H~+~(x;-~(cY); A’)}. (6.6.4b) ,.PT Under Ho, 8, is superior to since the risk-difference is positive for all a.

A’ 2

a,

. PT

We can describe the graph of R3(8, 0’

: W) as

follows: It begins with a value

t r [ W A l l ] - o’tr[W(Aii - A;l)]Hp+i(xg-l(a); 0)

(6.6.5)

at A’ = 0, then increases, crossing the risk of 6, t o a maximum, and then drops gradually toward u’tr[WA,,] as A’ 4 co. By setting a = 0, it can be shown that PT

R2(an: W) = R3(6, A

:W).

(6.6.6a)

For cr = 1, however, we obtain

~ ~ ( : 6w), = ~ ~., PT ( 8 : w). , ..PT and 6, . Both are superior to 8,

NOW,we compare 6, In general, the risk-difference is given by

~ ’ ( 6: ,w) - ~ = -u’tr[W(All

(6.6.6b) under Ho : /3 = polp.

PT

~ ( 8 : w) , - A;,)]

+

[I - H , + 1 ( ~ i - ~ ( a ) ; A ’ ) ] (G’J’ToWToJG)

~ ( 1 2Hp+1(xf-1(a);A2) + Hp+3(x:-1(a);A2)}. PT

Thus, 8,

(6.6.7)

performs better than 6n whenever

t f [ W ( A l l - A;,)] El - HP+l(x;-l(~);A’)] C h m m [ ( T o W T O ) A Z{~1]- 2Hp+l(X;-i(a); A’) + Hp+3(x~-1(a);A2)}’ (6.6.8a) while for

A2 5

tr[W(h11 - A;,)] [I - Hptl(x;-l(~);A’)] Chmin [(ToWTO)AZZ] { 1 - 2Hp+1(~;-1(a); A’) + Hp+J(x;-1(a); A’)} ’ (6.6.8b) ,.PT 6 , performs better than 8, . Under Ho, the risk ordering is given by

A’ 2

- PT

~ ~ ( :6w) , >~ ~ ( 8 , >~ : w) depending on the size of a.

~ (: w6 ) ,,

(6.6.9)


304 -S

6.6.3 Comparison of 8, and

en(en)

-S

In order to compare 8, and 6,, we consider the risk-difference given by Rl(6, : W) - R4(6: : W)

(P- 3 ) E [xi21(A2)]

= a2( p - 3)tr [W(TO JAZZJ‘To)]

The risk-difference is positive for all W such that

+

p 1 tr[W(ToJA22J’To)] 2 2. C L , [(ToWTo)Azz] -S

Thus, 8, uniformly dominates

6,.

Further, as A2 -+ 03,

: W) -+

R&

(6.6.11)

R , ( e , : W).

-S

To compare 8, and 8,, we can write R4 (6: : W) = R2 (

6,

-(G’J’ToWToJG) +[l-

(p

-

: W)

+ a 2 t r[W(ToJAZZJ’To)]

a 2 ( p - 3)tr[W(ToJA22J’To)] ( p - ~ ) E [ x ; ~ ~ ( A ~ ) ]

+ ~)(G’J’T~WT~JS)

(6.6.12)

2a2A2tr[W(ToJA22J’To)]

Under Ho, this becomes

R4(6z :W)

=

&(en

:W )

+ a2tr[W(ToJA2zJ’To)]

x ( 1 - ( P - 3 ) 2E

x-4 p + l (A2)] 2

,

(6.6.13a)

: W)

while

R2(6, : W) = R,(e, : W) - o2tr[W(ToJAz2J’T0)] 5 Rl(6, : W). (6.6.13b) -S Thus, 6 , performs better than 8, under Ho. However, as 6 moves away from the origin, o-2(G’J’ToWJ6) increases, and the risk of 6 , becomes unbounded -S

while the risk of 8, remains below the risk of 6, and merges with it as A -S -s Thus, 8, dominates 8, outside an interval around the origin.

-+

00.

6.6. Asymptotic Distributional Risk Properties

305

,.PT

-S

Comparison of 8, and 8,

6.6.4

PT

-S

Now, we compare 6 , and 6 ,

. First note that under Ho,

HP+l(x;-l(a);o) > (P - 3) -S

(6.6.15)

- PT

for some a. The risk of 6 , is smaller than that of the risk of 6 , when the critical value ~ ; - ~ ( o l ) satisfies the opposite relation in (6.6.15). This means -PT

-S

that 6 , does not always dominate 6 , follows: . PT

R z ( 6 , : W) < R3(6,

under

Ho.We can order the risks as

< R4(6: : W) < Rl(8, : W)

: W)

(6.6.16)

xi

when satisfies (6.6.16). This picture changes as A’ moves away from 0. As A2 -+ 00, the risk of PT -S 6 , and 6 , converge to the risk of 8,. For a reasonable a-value, the risk of A

., PT

-S

6 , is smaller than the risk of 6 , for a , satisfying (6.6.15) and p 2 4. Thus, none of the estimators dominates the other uniformly.

-

Sf

Comparison of 8,

6.6.5

-S

-S

,.PT

and On,6,, 8,

,.S+ . The risk-difference is given by

Finally, we compare 6 , and 6 ,

R.F,(~:+ : W) - R4(6: : W) = -u2 tr[W(ToJA22J’To)] x E [ ( 1 - (P - 3)~,;2l(A~))~1(xZp+l(A’) < P - 3)]

-(G’J’TOWT~JG)(~E[(~ - ( P - ~ ) X , ~ ~ ~ ( A ’ ) ) I ( X+p~ (A 3)]~ ) +2E[(1 - ( P - ~ ) X , - , ~ ~ ( A ~ ) ) I ( X > ; +p~-( 3)]} A ~ ) 2 0. S+

(6.7.13)

-S

Hence, 8, dominates 8, uniformly. Furthermore, A

,.S+ ) 2 0

Mi(8,)

- M4(8,

M4(6:)

. S+ - M5(8, )

and

2 0.

(6.7.14)

MI(&),

(6.7.15)

So we can order the MSE matrices as

M5(6:+)

4

M4(0:)

4

where 4 means “domination”.

-s+ -s

: 8,) is then given by : 8,) -S = 1 1, - c~~{M~(~:)}-~(ToJA~~J’To)

The AMRE(8,

AMRE(h:+


310

Under Ho, (6.7.17) becomes

P-3 a 2 ( A i i -A;,) - -0 P- 1

2

2 (ToJAzzJ’To) = - - - - - ~ ~ ( A i i -A;,). P-1

(6.7.18)

-s

The expression (6.7.18) is p.s.d. Hence, b, performed better than 6, under Ho. In general, the R.H.S. of (6.7.17) is negative-semidefinite(n.s.d) if for every nonzero vector f?, we have

t‘(Aii -(P

- A;,)t

- 3)f?’(ToJA22J’To)f?{2E:[x~~~(A2))I - (P - ~)E[x;:~(A~)]}

-e’(ToJAzzJ’To)t{ (P - l ) ( p - 3)E[x&(A2)] - 1) 5 0.

(6.7.19)

Dividing by t’ToA22Tot and maximizing over all 1, we obtain

1 - (P- 3){ 2E:[X,S21(A2)I- ( P - 3)qX;:l(A2)]}

-A2{ (P - 1 ) b- 3)E[x;:1(A2)]

- 1)

50

or

“S

Thus, 0, dominates 6, whenever (6.7.20) is satisfied; otherwise, 6 , dominates -S

-S

8,. Thus, neither 6, nor 6, dominates the other. The asymptotic efficiency -S of 6, relative to 6, is then given by

where AMRE(6: : 6,) and AMRE(6, : 6,) are given by (6.7.16) and (6.7.4), respectively.

6.8. Confidence Set Estimation: Normal Case -S

, ,

311

PT

Comparison of 6, and 6, . We may write the MSE-matrix difference as

~(63 ~ ~ ( 6 5 1Q2(nll ~) =

-

-ny,)[~,+~(x~-~(~);~2)

-(P - 3){2E[X;:l(A2)]

- ( P - 3)E[X;:1(A2)]}]

+(ToJ~~’J’T [(P o-) I ) ( P

- {2E [x;:l

>I - E Ix;A

(A

- 3)E[x&(A2)]

>I}I.

(6.7.22)

(A2

The expression (6.7.22) is p.s.d. whenever

PT

Hence, 6 ,

(6.7.23) -S -S is superior to 6 , whenever (6.7.23) is satisfied; otherwise, 6, is

. PT

superior to 8, . -S PT The asymptotic efficiency of 6, relative t o 6, is given by (6.7.24) where AMR,E(6; : (6.7.8), respectively.

8,)

and AMRE(6ET : PT

Similarly, we can compare 6, and are given in the problems section 6.12. A

6.8

en) are given by

..s en, 6,

., S+

and 6,, 6,

(6.7.12) and

..PT and 6, , which

Confidence Set Estimation: Normal Case

So far, we have considered point estimators of the intercept and slope parameters in the parallelism model from preliminary test and Stein’s perspectives. However, along with the point estimators, it is important t o provide some information on confidence sets. This section is set out t o study this problem.

6.8.1

Confidence Sets for the S l o p e Parameters

In this section, we consider the confidence set estimation of the slope parameters, p = ,,Bp)’ when it is suspected that p = p o l p holds in the parallelism model. Consider, again, the model (,&,.a.

-

Y , = doIne

+ pax, +

E,,

(Y

.

= 1,.. , p ,

(6.8.1)

N,, ( 0 ,a21,_) as stated in Section 6.1.1 and o2 is known. Thus, where E, we consider several estimators of p of the type P;C, = APn + (1,

- A)Png(Ln),

(6.8.2)


312 where

L,

=

a - 2 B k ( ~ -p A)’D;;

(I, - A)&

(6.8.3)

and g(C,) is a nondecreasing function of the test-statistic for testing the null hypothesis HO : = p o l p against the alternative H A : P # polp. Here we consider A t o be

(6.8.4)

I, - A is an idempotent matrix of rank p - 1. We limit g(C,) to the following selected functions: (i) g(13,) = 1, then ,B: = (ii) g(L,)

= 0,

then ,f3: = P,;

(iii) g(&) = I ( L (iv) g(L,)

=

p,;

PT

> ~ ; - ~ ( a )then ) , PA = Pn

-s

(1 - cL;’), 0 < c < 2 ( p - 3) then 0: = P,;

and (v) g(L,)

;

=

(1 - cL;‘)I(L, > c ) , then ,Bz

=

- s+

P, .

(6.8.5)

Following Chapter 5 (Section 5.9.1), we define the five confidence sets as

We like to provide the properties of the coverage probabilities of these five sets. As before, let r = (r1,rz)be an orthogonal matrix, where is p x p - 1 and I’2 is a pvector such that = I, - r2I’;.Also, r diagonizes the symmetric idempotent matrix D:i2H’D:i2HD:i2 of rank ( p l), meaning rD2, 112H‘Dii2HD:i2r’ = Further, choose r2 = 0 0

(

).

6.8. Confidence Set Estimation: Normal Case

313

(6.8.7) where q1 = Hence,

$ r ; ~ i i /and ~ a772 = 'a r2 f2 ~2 - 1 / 2 ~ '

2

a-211P - P:I/*;:

2

= 11711 - w1g(11w11I2)II

+ (772 - w2)2 .

(6.8.8)

We can then write the coverage probability of the set C*(P7tl)=

{ P : a-211P -a$),

I XJY)) 2

(6.8.9)

as

PV{"772 - w2)2 + 11711 - ~ 1 ~ ( 1 1 ~ 1 1 1 2 ) 1 152x;(d} 1 =

I

x;(Y)

Pv{llrll

- wlg(l/w1112)112I (X;w

- t)+}dHl(t,O),

(6.8.10)

where H,(- : A') is the cdf of a noncentral chi-square distribution with v d.f. and noncentrality parameter A2/2. Specifically, we may write the coverage probability as follows: (i) If g(I[w11[2)= 1, then by definition

1 - y = Po{ (772 - w2I2 + 11711 - will2 I x;(Y))

= Hp(x;(m).

(6.8.11)

(ii) If g(j[wlj(2)= 0, then we have

(6.8.12)

314


6.8.2 Analysis of Coverage Probabilities

Clearly, P ( C o ( p n ) = ) 1-7, which is a constant for all A '. Next, we note that

W R ( B , ) ) = Hl(x;(r) - A2;0),

(6.8.16)

which is a decreasing function of A ', has a maximum (11-7) at A' = 0, and decreases to zero when A ' 4 x;(y). The coverage probabilities of P ( C o ( p n ) ) and P ( C R ( B n )are ) equal when A' = A; where

A;

= x;(y) - Hcl(l- 7 ) .

Now, consider the confidence set ability is given by (6.8.13),

Hdx;(r)

+b

(6.8.17)

PT

CpT(p, ). In this case, the coverage probA

- A2;o)HP-l(X;-l(a); A2)

X;h)

~77{llrll

-w11I2 5 (x;(Y) - t)+;IIw11I2> x;-l(..))dHl(t,O),

and we have the following theorem: similar t o Theorem 5.9.1: Theorem 1.

(6.8.18)


315


316 or

A I llw1 /I + I(x;(Y) - t )+ I 1/2 . We then have

I($(?)

(6.8.24)

+ Xp-i(a) 1. < k i I I + /(x;(r)- t)+I‘/’.

- t)+I1/’

(6.8.25)

Thus, from (6.8.23) we have the R.H.S. as Hence, j/wlj12 > X:-~(CY).

{ llrll = Pql{

-

w1 112

(712 -

= Po{ ( 7 2 -

< (x;(Y)

42 + 11%

- t)+;IIw1

-0

1

112

5

I? > x;-lW}

x;w; ll

Wl

112 > x;-1w}

+ 11171 - w1 Il2 I x;(?.)} = 1 - Y-

(6.8.26)

This completes the proof. The graph of the coverage probability of CPT(p, ( a ) )as a function of A2 may be described as follows: As function of A2, the coverage probability is decreasing in the range 0 5 A2 5 xi(y) with a maximum a t A2 = 0. At A’ = $(y) there is discontinuity, and it drops to *

1

X;(Y)

PT

2 ~~-~(a)}dHl(t,O), (6.8.27) Prll{X;-15 (x;(Y)- t ) +.,xP-l(A2) 2

then increases as

A’ increases in the range

x;w

< A2 < ( X P ( Y ) + xp-l(4)2,

(6.8.28)

and eventually goes to 1 - y as A exceeds the limit

(XP(Y)+ Xp-1 Next, we consider the coverage probability of the set case, we have

P{CSf(p:+(c))}

Cs+(p, (c)).In this A

S+

= Hl(x;(r) - A2;0)Hp-1(c;A2)

5 (x;(y) - t ) + ;llw11I2> C}dHl(t,O)2 (1 - 7). (6.8.29) The first term on the R.H.S. of (6.8.29) is decreasing as a function of A’, and the second term is increasing in A2 and for all values of A’ the coverage prob. s+ ability is a t least (1 - 7). Hence, Cs+(p, (c)) uniformly dominates Co(B,) in coverage probability.


317

Before we prove the assertion (6.8.29), we define the functions

(6.8.30a) and

(6.8.3013) . s+

Theorem 2. Cs+(pn(c)) has higher coverage probability than Co(p,) for all A2 such that 0 < c < CO, where co is the minimum of the two unique solution

Ml(G (x;(Y)

- t ) + )= 1

and

(6.8.31)

Proof. We have to show that for every 0 < b2 < x:(y),

HI(Xg(Y)

- A2;0)H,-1(c; A2)

It is sufficient to establish that M l ( c ; b) 2 1 and Mz(c; b) 2 1 for all c E (0, CO) and b2 E (0, $(y)). First, we prove that Ml(c;b) 2 1 for b E (O,.X;(Y)). Note that MI (c; b) for each fixed value of b is decreasing in c. Hence, it is sufficient to establish MI (c*;b) 2 1 where c* satisfies MI (c*;xp(y))= 1. Note that a2

-logMl(c*; b) < 0. db2


318

0

logM1 c; b is strictly decreasing in b. Hence, M l ( c * ;b) is This is to say, (1) either strictly decreases to zero in b or (2) strictly increases to a unique maximum and then strictly decreases to zero. The first case does not hold, since

(6.8.33)

=1-y.

6.8.3

Confidence Sets for the Intercept Parameters when o2 is Known

In this section, we propose some confidence sets for the intercept parameters for the model (6.8.1) when it is suspected but not certain that Ho : p = pol, holds. Following Section 6.8.1, we define the five confidence sets given by the expressions below, assuming o2 is known:

We can write the confidence sets compactly as

6.9. Confidence Set Estimation: Nonnormal Case

319

b, + ( 6 , - b,)g(L,),

g ( L , ) is given by (6.8.2) and L, = o-~PLH’D;~H@,.By a suitable transformation (see Section 6.1.3), we then find that

where 6; =

O-2t18

- 8;112 = ( r l 2 -

4 + 11771 - wlg(llw1t12)112~

( Ei ) - { ( ;l ) ( ‘xl )}.

where Np (6.8.35) is then given by

;

P(c*(8:))= P{ 1171~ - wlg(itwl

(6.8.36)

The coverage probability of

tI2)P+ (72 - 4 I: X;(Y)},

(6.8.37)

which is the same as (6.8.10). Hence the details of Sections 6.8.1 and 6.8.2 are applicable t o evaluate properties of the confidence sets with center 8,, -PT - S . S+ 8, , On,and 8, , respectively, which are described in Section 6.8.3 replacing PA by 8:.

a,,

6.9

Confidence Set Estimation: Nonnormal Case

Now, consider the five estimators in a compact form as in Section 6.8.1:


320

where A, = lplbDT;/nQ -+ lplbAi;/Q as n -+ m. Then, under {K(,)} as n -+ m, by Theorem 1 of Section 6.5.3, we can write

where (6.9.5) Thus, under {K(,)} as n -+ m, we obtain the following asymptotic coverage probabilities according t o the choice of g( Ilwl

1. If g(llw1112) = 1, then (6.9.4) becomes po(ll771 - 4 12 + (v2 - W 2 I 2 5 x;(Y)}.

(6.9.6)

32 1


These expressions are similar to that of the expressions in Section 6.8.1. Hence, the analysis of the coverage probabilities is similar to that given in Section 6.8.2.

6.10

Nonparametric Methods: R-Estimation

In this section, we consider the nonparametric methods of estimation of the intercept and slope parameters of several simple linear models. These applications enlarge the scope of the theory of shrinkage estimation bringing it to a broader class of estimators with robustness properties.

6.10.1 Model, Assumptions, and Linear Rank Statistics Consider, again, the set of several simple linear models given by

Y, = &ln, +paxa + E,,

CY =

1,. .. , p ,

(6.10.1)

where

with the cdf defined by

and P

n,

(6.10.3)


322

2)

+

Let n = n1+ . . . np and A,’ = Diag (%, . . . , . It is assumed that { E , ~} are mutually independent and identically distributed with cdf F(e1, . . . ,c P ) defined by (6.10.3), where PO(.)belongs to the class F of absolutely continuous cdf with absolutely continuous pdf, fo(-) having finite “Fisher information” as follows:

(6.10.4)

ni (iii) lim - = Xi (0 < X i < l), meaning, limn-wA;l n-iw n Diag(X1,. . . ,X p ) .

= A-’ 0

=

(iv) Score functions a,(.) (and a:(.)) are generated by a function ~ ( z L ) u , E (0, l ) , which is a nondecreasing, skew symmetric (i.e., 4(~)+4(1--2~) =0 for all 11 E (0, l)),and square integrable. Let ~ + ( z L )= 4

(F),E (0, l ) , and set ZL

and a:(k) = E[4+(Ukn)]or

4’

(&),

k = 1,... ,n,

(6.10.5)

where 0 < U1, 5 ... 5 U,, < 1 are the order-statistics of a sample of size n from U(0,l). Let R,j(a,, b,) be the rank of (Y,j - a, - b,zaj) among (Y,l - a, - b,z,i), . .. , - a, - bazan,) and similarly, let R,+J(a,,b,) be the rank of {/Yaj - a , - b,zajl among l(Yal - a, - b,z,l)l,. . . ,I(Y,,, a, - boz,,_)l for a = 1 , . .. , p . Let the vector of linear rank-statistics (LRS) be

where

and

where (6.10.6b)

323

6.10. Nonparametric Methods: R-Estimation Further, let

c P

=

j=1

Ln,(b,)

(6.10.7)

= lbLn(b).

We will use the LRS above for the estimation and test of hypothesis-related problems regarding the intercepts and the slopes.

6.10.2

R-Estimation and Test of Hypothesis

As in Chapters 4 and 5, we define

and

Y(+, 4) =

/

1

0

(6.10.8)

+r(2h)d(2h)dZh.

We let na

A:_ = (n, - 11-l C ( a n , ( k ) - an,>2 k=l with sin_ = n,lCLrl(an,(k) and A: = (n-p)-'

(6.10.9a)

xE=l(n,-l)A:,.

Further,

=C(~cr~--n~)~, ~ = 1 , . - ,P. n,

Qn,

(6.10.9b)

j=1

Note that given x, = (z,~,. . . ,zan_)',L,_ (b,) \ b, for b, E (--00, m) and T,_(a,,b,) \ a, for fixed b,. Under the model (6.10.1) with 8, = pa = 0, Trim (0,O) and Lnm(0), both have distributions symmetric about 0. First, we note that from the basic theorems of Hajek and Sidak (1967) in =0 Chapter 4, we have that under 6, =

where A,'

= Diag(X1,.

.. ,Ap)

and A;

= Diag(XlQ1,.

. . ,X,Qp).

(6.10.11)

6.10.3 Estimation of the Intercepts 6, and the Slope We consider two unrestricted estimators of 8, and and (6.10.6b), respectively, as follows:

,&= -21 [ sup{b,

: Ln_(b,)

0, based

> 0} + inf{b, : L,_(b,) < O } ] ,

on (6.10.6a) (6.10.12a)


324

1

[ sup{aa : Tn, (aa 2

gna =

7

> 0) + inf{aa : Tn, (aa

Pn,

7

Pn,

p under p = polp is defined by

The restricted estimator of

1

Boon= 5[ sup{b : ~ i ( b > ) 01+ infib : LA(^) < 013. Hence, we denote the unrestricted estimators of 8 and

. . . ,JnP)’, and

6, = (g,

p, = (p,,,...

< 011. (6.10.12b)

(6.10.12~)

p by

respectively. (6.10.13)

,,&)I,

Let us denote the restricted estimator of 6 by

6n,

-

1

= ( 6 n l , - - -i o n p ) 7

(6.10.14a)

where

in,

=

1 2

{aa : Tn,(aa;h:on) > O }

-[SUP

+ inf {aa : ~n,(aa;h n > < o } ] (6.10.14b)

p = polp by p n = & l p . Test of Hypothesis: ,B = polp.For the test of the null hypothesis Ho : p =

and that of

p o l p ,we use the nonparametric test due to Sen (1969), which is based on the statistic P- 1

Ln =

C~ ~ ~ A L Z Q ; :

[ ~ n , ( o n ) ] ~ ,

(6.10.15)

a=l

where A i m and Q,, are given by (6.10.9a) and (6.10.9b), respectively. Under the null hypothesis Ho : p = ,&lp, L, is approximately distributed as a chisquare with ( p - 1) d.f. This allows us to define the PTE, SE, and PRSE of p as follows: (i)

bZT = P n - ( P n - B n ) I ( L n

(ii)

b: = P, where

(iii)

E,,

- d ( p n - B,>L,’

-+

0 as n -+ 00, and

< X;-,(~Y,>, I(L > E n ) ,

d =p -3

bf+ = 6, - (1 - ~ L Z ~ ) I (>Ld, ) ( ~ -, f i n > ,

(6.10.16a) (6.10.16b)

(6.10.16~)

respectively. Thus, we have defined five possible estimators of to generate five possible estimators of the intercept parameter 8 of the model (i) UE : 6 , = (ii) RE : hn =

(&, . . .

,en,)’;

(6.10.17a)

(en,,.. . ,en,,)’,;

- PT (6:?, . . . ,enP -PT ) ,

(iii) PTE : 8, =

I.

(6.10.17b) (6.10.18a)


-s

(iv) SE : 8, =

325

.

(s,Sl,.. . ,6,Sp)',

(6.10.19aj

where

6.10.4 Asymptotic Distribution of the R-Estimators of the Slope Vector First, note that by JureEkova's (1969) asymptotic linearity result (see Section 2.8 of Chapter 2) we have under ,B = 0, as n --+ 00 and R > 0,

{

sup n-'/2/Lna(n-'/'b,)

- L,, (0)

+ X,n'/2b,Q,y(T,!I,

$)llb,l

50 (6.10.21)

A;;

and A2 = (~-~(6'Ai;d).

Q

pn

p,,

(6.10.34)

PTOO~. First note that and are both translation-invariant estimators, and hence, we can assume without loss of generality, P = pol,. Also, by (6.10.12a) and (6.10.12~)together with (6.10.10) and the linearity results (6.10.21) through (6.10.23), we have n'/21pn-01 = O p ( l ) and n'/21fin-01 = O,(l), while under {K(,,)},n'/'pn = ,801, b = O(1). Thus under {K(,,)}, n'/2Pn = ,801, + 0,(1) and n1/2fin= pol, O,(l). Observe that under H' : 6 = P = 0 by (6.10.12a-c), n-'12Ln(0) = AY;n'/2fin-y(+,(6) + op(l). Hence, utilizing the contiguity of probability measures under KTn, : 0 = 0, P - ,801, = n-'126 to those under H', we obtain under Kin, as n 00, that n-'12Ln(0) has the same distribution a5 n-'12Ln(6) under

+

---f

+


328

H*. Thus, by (6.10.10) n-'/2L,(n-1/2(6) is asymptotically pvariate normal with mean-vector A;i6y($, 4 ) and dispersion matrix A;&.. .-Thus, under

- Np{&16d$, $1, 4).

{K(,)}, n-lI2L,(O)

Hence, n1I2(P, - Polp)

A&2).

Np(6,g2A22), a2 = A$,/r2($,

-

Similarly, by the contiguity of probability measures under K:,, t o those un-

der H ' , we obtain under KT,, as n --+ co,n1/2,b, = hl' n-1/2L,(0)y-1($, Q

P

o p ( l ) .By a similar argument as before, we have fi(P, -Polp) B = lplL/Q. By similar steps, we obtain under {K(,)}, (iii)

TZ'/~(P,- P,)

+

= (A22 - ~ l p l ~ ) n ' / 2 L , ( 0 ) ~ - 1 (o~p ,( ~ l ))

-

NP(J6,a2JA22).

Further, under {K(,)} as n

(6.10.35)

00,

A22n'/2L,(0)7-1 ($, 4 )

n1/2(D, - P o l p n'/2(& -

-

+)+ Np(6;a2B),

P,)

N N2,

{(

6!

),

(

g2

A22 A22 J'

JA22

JA22

+OP(1)

)}

(6.10.36)

by the same argument as before. Similarly, under {K(,)} as n 3 co,

(vi) To prove limn-m P { L , 5 Z I K ( ~ ,= } H p - l ( z ;A2), we note that under (0 - Polp) = 0, L, = nA~[nL,(0)]'JA22J'[Ln(0)] o p ( l ) ,which approximately follows a central chi-square distribution with ( p - 1) d.f. Also, under {K(,)}, as n -+ 00, we can write

+

L,

=

n(p, - ~ , ) J ~ A ; ~ J-@&J, + op(i).

(6.10.38)

By (iii) the expression above follows approximately a noncentral chi-square distribution with ( p - 1) d.f. and a noncentrality parameter A2/2 with A2 = 0-2( dfA; 6).

As a consequence of Theorem 6.10.1, we obtain the ADB, ADQB, ADMSE, and ADQR of the five estimators its given below.

For UE,

B,,

(i) bl(P,) = 0 (ii)

and

MI@,) = a2A22

For RE,

a,,

&(&) and

= 0;

R l ( P , ; W ) = u2tr(WA22).

(6.10.39a)


329

330

Chapter 6. Parallelism Model x I(X;+~(A~)
to obtain the positive-rule Stein estimator (PRSE), --f

Notice that (7.2.27) and (7.2.29) are the same as (7.2.3) and (7.2.4), respectively. The reader may refer to Ali and Saleh(l99lb) and Ghosh, Saleh, and Sen (1989) for more information on EBE

7.3

Bias, Quadratic Bias, MSE and Quadratic Risk Expressions

In this section, we obtain the bias, quadratic bias, MSE matrices, and the weighted risk expressions of the five estimators of P.

7.3.1 Bias Expressions The bias and quadratic bias expressions of the five estimators of the regression coefficients are given in the following theorem:

Theorem 1. (i) bl(P,) = 0 and B1(B,) = 0.

(ii)

-C-'H'(HC-'H')-'(HP

b2(Bn) =

&(a,)

(7.3.1) -

h) = 6 (say) and

= &(HP - h)'(HC-'H')-'(HP - h) = 6'C6/a2 = A2.

(iii) b3(p:T)

=

-C-'H'(HC-'H')-'(HP

- h)G,+2,,(e,;

A2)

= -6Gq+z,,(L; A2),

where & = &Fq,,(a), (iv)

bl(b:)

and

PT

&(On ) = A2{Gq+z,,(!,; A2)}2.

-dqC-'H'(HC-'H')-'(HP = -WE[x;?2 (A2)I

=

-~)E[x~:~(A~)]

7.3. Bias, Quadratic Bias, MSE, and Quadratic Risks

349

Proof. (i) is obvious. For others using Theorem 7.2.2, Theorem 2.2.6, and equation (7.2.4), we obtain the following:

(ii) E ( 6 , - P ) = -C-'H'(HC-lH')-l(HP (iii)

&(fin) = A'. (7.3.2) E(P, -0) = -C- 'H' (HC-lH')-lE{(HB - h)I(L, I Fq,,(a))} - h) and

..PT

= -C-'H'(HC-'H')-'(HP

by Theorem 2.2.6. Thus,

..PT

- h)Gq+2,,($;

A2)

) = A2{Gq+2,,(!,; A2)}2.

S

(iv) E(b, - P ) = -dqC-'HC-lH')-l(HP

- h)E[&2(A2)],

-s and B4(/3,) = d2q2A2{ E [ x ~ : ~ ( A ~ ) ]by } ~Theorem 2.1.16. (v)

a:+ -a)=E[Bz-P]-

E

[-P, - P , ) W n

+ dE [ ( P , - B,)L;'I(L,

I 41

I 41.

The first term is given by (iv). The second term equals

-S(Gq+z,,(di; A'),

dl = -d. Y+2

Now, the third term can be written as

E

[(p,- b , ) L z l l ( L n 5 d ) ] = di6E [F>'2,,I(Fq+2,m(A2)

Collecting the expressions in (7.3.2) and S+

b4(&),

I di)]

I

we obtain the expression

b5(p;+). The expression for &(On ) is easily verified. In order to study the bias of the estimators, we consider the quadratic bias * PT AS . S+ expressions B ~ ( P , ) , ~ 3 ~ 31, ,B~(P,),and B~(P,1. clearly, is

B~(B,),

a,

Chapter 7. Multiple Regression Model

350 unbiased and the bias of

a, is-unbounded. The bias of P , s -

. PT

S+

size of a and A2. The bias of P, and P, establish the relation

o = ~l(p,)I ~

s ( ~ f5+~ ) 4

depends on the

depend on A2 alone. Thus, we can

..

PT ( ~ 5 z ~) 3 ( ~ n 5. ~ 2 ( B n )

(7.3.3)

PT

under certain conditions on a. Note that for a = 0, B3(Pn ) = B2(Pn)7while A

. PT

-s

for a = 1,B3(p:T) = Bl(p,). The position of B3(P, ) and B4(Pn)switches, depending on the value of GU.

7.3.2

MSE Matrices and Weighted Risks of the Estimators

The MSE matrices and the weighted risk based on the loss function

L(P*,P)= (P’

- P)’W(P*- P ) =

are given in the following theorem:

p*- PI[;

(7.3.4)

7.3. Bias, Quadratic Bias, MSE, and Quadratic Risks

351

Proof. (i) is obvious. The remainder of the results follow by Theorems 7.2.2 and 7.2.6 and formula (7.2.4).First, consider

(4W b , ) = E ( b , - P)@, - P)’ = E { @ , - p) - C-lH’(HC-’H’)-l(Hfi,

{(p, - 0) - C-’H’(HC-’H’)-l(Hb,

= 02C-’

+ C-lH’(HC-lH’)-lE{(Hfi,

’

’

’

’

- h)} - h)}’

- h)(Hfi, - h)’} x

-’ ’

(HC- H’)- HC- - 2 C - H’(HC H’)- x E{(HP, - h M , - PI’>

= 02c-l+ C-~H’(HC-~H’)-~{~~(HC-~H’)

+ (HP - h)(HP- h)’}(HC-’H’)-lHC-’ -

~D~C-~H’(HC-’H’)-’HC-’

= 2c-1 - a

+

2 ~ - l ~ ’ ( ~ ~ - 1 ~ ’ ) - l ~68’. ~ - 1

(7.3.6a)

The risk expression is given by

Rz(fJ,; W) = 02tr[WA]+S’W6,A = C-’

- C-lH’(HC-’H’)-’HC-l.

(7.3.6b)

352

Chapter 7. hdultiple Regression Model

M ~ ( B : ~=) E

(iii) =

( ~ , P-~p)(jfT

-

p)’

{ ( P , - p) - C-’H’(HC-’H’)-’(HP, x { ( P - p) - C-’H’(HC-’H’)-’(HP,

= E [ ( P ,-

x

-

h)l(L, < Fq,m(a)} - h)l(L,

< Fq,m(a))}’

p ) ( P , - p)’] - 2E[C-’H‘(HC-’H’)-’(HP,

- h)

(P, - P>’l(Ln< % m ( a ) I

+ EIC-lH’(HC-lH’)-l(HPn

- h)(HP, - h)’

x (HC-~H’)-~HC-~I(L, ~e166‘e. ~-1}t

(7.5.3)

Hence, we have

e W e < ~’{c-~H’(Hc-~H’)-~Hc-~}~ e. ~

02e’c-1e

-

t’C-1.t

Therefore,

A2 = max

c

e’ { c- H’(HC- H’)- HC1‘66‘4 < max aze’c-le f?C-lt

}e 7

or

A2 F Ch,,[H’(HC-lH’)-lHC-ll

= 1,

(7.5.4)

7.5. h/lSE-Dlatrix Analysis of the Estimators

363

since C-1/2H’(HC-1H’)-1HC-1/2 is an idempotent matrix with maximum characteristic value equal to 1. Thus, (7.5.1) is p.s.d. if and only if A2 5 1. Therefore, performs better than when A2 5 1; otherwise, performs better than The range of the domination of over is bigger in the case of risk analysis. For W = C, the range of risk domination is [0,q] and q 2 1. The MSEbased efficiency of relative to B, is meaningless, since

6, 6,.

M R E ( ~ , ;6,) =

6,

6,

a,

pP - H’(Hc-~H’)-~Hc-~

p,

p,

- a - 2 ~ 6 ~ / j= - ”0,~ (7.5.5)

while the risk eficiency, which is meaningful, is given by

(7.5.6) For W = C , tr(A11) = q and tr(WC-’) = p so that (7.5.6) reduces t o PKP - Q )

+ a21-l.

(7.5.7)

6,

Thus, the risk efficiency of is greatest when A2 5 q, while the range of A2 is smaller for the MSE-matrix comparison. PT

Comparison of ,On The MSE difference of

,

a,, and ,h,

p, and p,. PT is given by

MI@,) - M3(,hET)= u2 (C-lH’(HC-’H’)-’HC-’) Gq+2,m(la; A’) -

SS’{ 2Gq+2,,(la; A’)

- Gq+qm(!;;A’)}. (7.5.8)

The MSE difference is p.s.d. whenever for any nonzero vector t we have

and consequently, we may write

-

~’{c-~H’(Hc-~H’)Hc-~}~ u2t’c - 1t

(7.5.10)

Hence, taking the maximum over all nonzero vectors t we obtain

(7.5.11)


364 This means

PT

p,

p,

is superior t o

..PT

p,

in the range of A’ is given by (7.5.11),

p,

. PT

otherwise is superior t o pn . The range of domination of p, over is bigger in the case of risk analysis. For W = C, the range of A’ is 5 q. The PT risk-based efficiency of p, relative to is given by

p,

-1

x {2Gq+z,,(&; A’)

-

Gq+4,m(f:;A”)}]

. (7.5.12)

For W = C5(7.5.12) equals

[

P P - qGq+z,m(&;A’)

+ A’

{ 2Gq+~,m(e,;A’)

-1 -

Gq+4,m(G; A’)}]

(7.5.13)

. PT Thus, the risk efficiency of p, is greatest when

. PT

The MSE-based efficiency of p, MRE =

1,

relative to

.

p, is given by

(SET: p,) - { H ’ ( H C - ~ H ’ ) - ~ H C - ~ } G , +A’) ~,~(~~;

+ ~-‘C66’{2G,+~,,(!,; = [1 - Gq+z,m(&; A’)

A’) - Gq+4,m(t;; A’)}

I-”’

+ A ’ { ~ G , + Z , ~ (A’) ~ , ; - Gq+4+(!;; A’)}]-l

x {I - G,+z,,(e,;

(7.5.15)

..PT is greatest when

Thus, MRE of p,

which is smaller than (7.5.14). PT To compare p, and f i n we consider the MSE difference .L

~2

(Pn)

- ~3

(bET)

= -u’ {C-lH’(HC-lH’)-lHC-’} {I - Gq+2,m(!,;A’)}

+ 66’ { 1 - 2Gq+z,m(L;A’) + Gq+i,m(t;;A’)} .

(7.5.17)

7.5. MSE-Matrix Analysis of the Estimators

365

The MSE difference is negative-semidefinite (n.s.d.) whenever for any nonzero vector i! we have

L

e{ C- l ~ ’H(C - ~ H ’-) l ~ ~ - l } e e’C-le

which implies that the MSE difference is n d . whenever

(7.5.19) Thus, in this range

6,

,.PT

6,

also the range of superiority of risk comparison. PT The MRE(P, ; f i n ) is given by

..PT

p, ; otherwise, p, is superior. Here . PT over p, is smaller than the range for the

is superior to

bi, p, and bn -s We compare p, and B,. The MSE difference in this case is M1 (B,) - M4 (6:) = dqa’ { C-lH’(HC-lH’)-lHC-l} Comparison of

x

{ ( 4 - 2)E [X;:2(A2)l

--dq(q

+ 2A2E [XqS44(A2)11

+ 2 W ’ E [xqS44(aZ)l.

(7.5.20)

The difference is p.s.d. whenever for a nonzero vector 4 we have

dqa2{e’{C-1H’(HC-’H’)-’HC-1e} { ( q - ~)E[X;:~(A’)] +~A’E[x&(A’)]}

- dq(q

+ 2)(e’ss’e)E[~,S4~(A’)]2 0.

(7.5.21)

This implies that (after simplification)

qA2E [x;:4(A2)]

L

(Q - 2)E [X,s“’(A”,]

,

(7.5.22)


366

-s

which does not hold for all A2. Thus, P, is not uniformly better than under the MSEscriterion. The MRE(pn;P,) in this case is given by

p,

-s -

MRE(Pn; Pn) =

/Ip- dqH'(HC-'H')-'HC-'(

(q - 2)E [ x , S ~ ~ ( A ~ ) ]

I-'"

+ 2A2E [ x ~ : ~ ( A ~ )}]+ dq(q + 2)0-~C66'E[x;$~(A~)] =

(1 - dq{ (4 - 2)E[x,:2(A2)]

+ dq(q + 2)A2E [x;:~(A~)] X

+ 2A2E[~L:2(A2)]}

)-' +

(1 - dq { (q-2)E [x,S42(A2)] 2A2E [X;:~(A~)]})

Similarly, the comparison of ference

M2

P---llP

.

(7.5.23)

-s b, and Pn is obtained through the MSE dif-

(bn)- M4 ( b f ) a2C-' H'(HC-'H')-lHC-'

--

X {

1 - dq(q - 2)E [xi:2(A2)] - 2dqA2E [x;:,(A2)]

+ as'{ 1 - dq(q + 2 ) E [x,&(A2)]

}.

Under Ho, the MSE difference is n.s.d. Therefore, ..s

Pn

Pn.

} (7.5.24)

b,, performs better than -s

at A2 = 0. Except for small intervals around 0, P, performs better than

7.5. hlSE-Matrix Analysis of the Estimators

. PT

367

(7.5.27) whenever A2 satisfies (7.5.27); otherwise,

-s

Thus, 0, performs better than 0, -s PT 0,performs better than p, . Neither estimator dominates the other. . PT Similarly, we can find the range of A2 for which 0, performs better than ., S+ ., PT . s+ 0, using the expression (7.5.26) and observe that neither 0, nor 0, dominates the other. MRE expressions can be obtained by straightforward computation. Next, we consider the MSE differences under Ho for the comS+ parison of 0, and n

p,: M5 (X+) - M2 (p,)=

o2 [C-lH'(HC-lH')-lHC-']

{ 1- d -

[

E (1-d1F;:2,m(0))2 I (Fq+2,m(0)< dl)]}. (7.5.28) This is p.s.d., since (1 - 4 2 E [(I - dlF;:2,m(0))2

6,

I (Fq+2,m(0)< d d ] .

(7.5.29)

. S+

This implies that is superior t o 0, under Ho. . PT A similar comparison of with 0, under Ho shows that

M5

(bt')

- M3 (PET)

= o~C-'H'(HC-'H')-'HC-I{ Gq+2,m(t,;A2) - d

(7.5.30)

368


whenever there exists a set A of a such that

{

A = a : Gq+2,m(ta; A2) L d

+ E [ (1 - d1F>12,m(0))2I (Fq+2,m(0)< d l ) ] } .

(7.5.31)

..S+

PT

-Sf.

Thus, 0, is superior t o pn whenever (7.5.31) is satisfied; otherwise, 0, IS ,.S+ PT superior. Therefore, pn does not always dominate pn when the hypothesis Ho : HP = h holds. Hence, we can order the MSE matrices as ~i

(P.)

L ~4

(a:)

2

~

(XT)L 3

M (Bn) ~ , -s

(7.5.32) . S+

where 2 stands for M2(pn) - Md(bZ) 2 0, M4(Pn) - M5(pn ) 2 0,

Ms(B:+)

- M3(bET)L 0 , and

A

PT

M3(Pn

-

M2(bn) 2 0 .

The position of PTE may be changed to the order

M I (Pn) 2 M3

(BET) L M4 (Bf)

2 M5

(Bf')

2 M2 ( P n )

7

(7.5.33)

depending on the size of a.

Comparison of Finally, we compare

M4

-s

p, , p, and p, A

S+

B:+

and

-s

p,. In this case, MSE difference yields

(b:) - M5 (8')

= n2

{ [C- 'H' (HC-'H' )-'HC - '1

[

x E (1 - d l F q + ~ , m ( A I~ )(Fq+2,m(A2) )~ 22,77z(A2))

(Fq+2,m(A2)

(2F$4,m(A2) - d2F$4,m(A2)) I

> fa)

} }

> f:)

(Fg+4,m(A2)

-2E (f7>12,m(A2)1(Fq+2,7n(A2) > fa))]

(7.6.4)

and R6

(BET+;w)

= R3

. PT (Pn ;W) - a2dl tr[WC-lH'(HC-lH')-lHC-l~

E{ -

(2Fi22,m(A2> - dlF[-,22,m(A2))

d1(6'W6)

[E( (2F$4,,(A2) -2E (F,-l2,,(A2)1

I (Fq+2,m(A2) > ea)

}

- dzF$4,m(A2)) I (J'q+4,m(A2) > e:) ( F q + 2 , m ( A 2 )> fa))].

}

(7.6.5)

Chapter 7. Multiple R.egression Model

370

. PT+

For W = C, the percentage improvements of p, the tabular values in Table 5.6.1 of Chapter 5.

7.7 7.7.1

over

- PT are similar to

0,

Multiple Regression Model: Nonnormal Errors Introduction

Consider the multiple regression model

y , = X,P

+en,

(7.7.1)

where Y , = (y1,. . . ,y,)’ is the vector of observable response, X , is an n x p matrix of known constants, p = (PI,.. . ,pp)‘is the vector of regression parameters, generally unknown, and e , = ( e l , .. . ,en)’ is the vector of i.i.d. random errors assumed to follow a distribution, F ( e ) such that E ( e , ) = 0

-

and E(e,ek) = a21, (a2< 00). Further, assume that max X ; ( X ~ X , ) - ~ X ~ o as n 4 00, (7.7.2) (i) ljijn where xi is the a th row of X,; (ii) lim (n-’(X; X , ) ) = C, finite and positive-definite matrix. n-Dc)

The basic problem of this section is the estimation of the regression parameter p when it is suspected that p belongs to the subspace defined by Hp = h, where H is a known q x p matrix and h is a q x 1 vector of known constants. Accordingly, we consider the unrestricted, restricted, preliminary test and Stein-type estimators of p.

7.7.2 Estimation of Regression Parameters and Test of the Hypothesis Based on the least squares principle, the unrestricted and the restricted estimators of p are given by

where C, = ( X k X , ) . We define the estimator of u2 by

~ 1 =4 ( n- P ) - ’ ( Y , - x , P , ) ’ ( y ,

-

LP,)

(7.7.4)

One may show that sz -+ o2 almost surely (see for example Sen and Singer (1993, p. 281)). For testing the null hypothesis Ho : Hp = h, we consider test-statistic L, defined by L, = sL2(Hp,

-

h)’(HC,’H’)-’(Hp,

- h).

(7.7.5)

Under Ho, L, follows a central chi-square distribution with q d.f. as n (see Sections 7.8.1 and 7.8.2).

4

co

7.8. Asymptotic Distribution of the Estimators

7.7.3

371

Preliminary Test and Stein-Type Estimation

As usual (see Chapters 5 and 6), we combine the unrestricted and the restricted estimators to obtain the PTE and Stein-type estimators of P as follows:

,.PT = P ,

P,

-

(P, - P , ) G

< C,,,),

(7.7.6)

where is the upper a-level critical value of the exact distribution of under Ho. The James-Stein-type estimator (JSE) is defined by

C,

(7.7.7) The positive-rule Stein-type estimator (PRSE) is defined by

- S+ = P , + (1

P,

-

kJ,l)wn > W ( P , - P,)

since C,

# 0 with

7.8

Asymptotic Distribution of the Estimators

probability one.

In order to find the asymptotic distributional bias (ADB), quadratic bias (ADQB), MSE-matrices (ADMSE), and quadratic risks (ADQR) of the estimators of 0, we need the asymptotic distributions of the various estimators and of the test-statistic, C,. As in Section 5.8, we consider the generic estimator P: and a positive-semidefinite matrix W defining the loss function

m:;P ) = 40:

- P>'w(P: - P>

= tr[W{n(P: - P)(P: - P ) / } ] *

Let

M(PT,)denote the

0: is given by

(7.8.1)

- P)(P:

- P)'}. Then the ADQR of

R ( P 3 = t"WM(P:,)I.

(7.8.2)

E{ n(P:

In many situations, the asymptotic distribution of & s,'(p: -P) is equivalent to the fio-'(P, - P ) distribution as n --+ 00 under fixed alternatives

Kc : HP = h + (.

Then, to obtain a meaningful asymptotic distribution of &s,'(P,', consider the class of local alternatives, {K(,)} defined by

K(,) : HP = h + T Z - * / ~ ( .

(7.8.3) - P ) , we

(7.8.4)

372


Let the asymptotic cdf of &s,l(P:

- P ) under

G p ( x )= n-o;, lim pq,,{ f i s , l ( P :

-

{K(,)} be

P) 5 x } .

(7.8.5)

If the asymptotic cdf exists, then the ADB and the ADQB are given by

b(P:) and

= n-m lirri

E [ f i ( P : -P)] =

/

xdG,(x),

w:) .-“b(P:)I’Cjb(P:)i,

(7.8.6b)

=

respectively, where a 2 C 1is the MSE-matrix of

(7.8.6a)

P,

as n

-+ 03.

Defining (7.8.7)

we have the weighted risk of ,B: given by

Asymptotic Distribution of the Estimators under Fixed Alternatives First, we consider the asymptotic distribution of f i s , ’ ( P , - p) under the 7.8.1

regularity conditions given by 7.7.1(i) and (ii). Theorem 1. Under the regularity conditions 7.7.1(i) and (ii) together with o2 < co,as n -+ 00, fiS,’(Pn

- 0)

N,(O, c-1).

(7.8.9)

For proof readers are referred to Sen and Singer (1993, p. 280). Next, we consider the asymptotic distribution of the test-statistic Ln for testing Ho : HP = h under some fixed alternative hypothesis of the form

K t : HP = h + E, where

(7.8.10)

6 is a fixed vector in Rg.Then

+ (HP - h)} = sZ1H(Pn- p) + s i l t ,

sll(HPn - h) = S;’{H(P,

- 0)

and the statistic Ln can be written as

c,

=

s;2(pn - P)’H’(Hc;~H’)-~H(P,- p) + s;2(’(~~;1~’)-1t + 2 s i 2 ( P n- P ) ’ H ’ ( H C ; ~ H ’ ) - ~ ~ . (7.8.11)

7.8. Asymptotic Distribution of the Estimators

373

Under the assumed regularity conditions and the CLT, we obtain as n

J ; ~ ' ~ ; ~ ( H-B11)~ 2 N , ( J ; E w ~ ,( H C - ~ H ~ ) )

--f

CO,

(7.8.12)

and nal2 (xp(y)+ ~,(a))~, then

(BET))

pp (CPT

=I

X;(Y)

Pv1{ llw 112

’xi(4;Il77l -

W1

/I2 L

(x;(y)

Note that xp(r)

+ X,(Q) < A = 1177111.

-

t)+}dHp-&, 0).

7.9. Confidence Set Estimation

389

Since

The reader is referred to Tables 4.8.2a and 4.8.2b for the tabular values of the coverage probabilities against A2-values for p = 5(2)15. Clearly, the coverage probability attains its maximum a t A’ = 0. Then it decreases t o A2 = x;(y) and drops to a minimum value (there is a discontinuity at A’ = x;(y)). As A’ > x:(r), the coverage probability increases monotone toward the value 1 - y as A2 4 cx). The picture of the coverage probability is similar to the risk of the PTE as a function of A2.

for 0 I d 5 2d by the fact that w l ( 1 - dllwlll-2) is the Stein function, while w1 is the usual q-variate normal with mean q 1 and covariance I,.


390 S+

Cs+(/3, ):

In this case, we have the following theorem:

Theorem 1. If q 2 3 and A2 > 0, ~p

{ c+(p,S+)} L P ~ ( C O ( P , ) )for all A’ E ( 0 ,cm)

provided that 0

(7.9.27)

< d < do where do is the minimum of the two unique

(7.9.28)

Proof. The theorem is proved if we can show that for every 0 < b2 5 x$(y),

Pvl{ 11771 - Wl(1 - dlI~lll-2)~(llW1112 > 4112I b 2 } > PvJIl771 - 4 l 2 5 b”.

(7.9.29) Substituting the right-hand side of (7.9.28) for the integrand below, we get

-

Note that w1 Nq(ql, Iq);hence, it is sufficient t o establish that G$’)(d, b) 2 1 and Gi2’(d,b) 2 1 for 0 < d < do and 0 < b2 I xg(y).

Let us prove Gil)(d,xp(y)) 2 1 for 0 < b2 I x;(y). Note that for each b, Gil’(d,b) is decreasing in d where d* satisfies G F ) ( d * , x p ( ~=) 1. ) Hence, it is sufficient to prove that there exists a d* such that @)(d*,b) 2 1. Note that

7.10. Asymptotic Theory of ConfidenceSets

391

Gq(1)( d , b) is strictly decreasing in b. As a result, Gil)(d*,b) either (1) strictly decreases to zero in b or (2) strictly increases t o a unique maximum and then strictly decreases to zero. Since Gq(1)( d * , 0) = Gil'(d; x p ( y ) )= 1, then (1) is not true, and in the case of (2) we get Gil)(d*,b) > 1 for 0 < b2 < xi(y). The proof with &)(d,x;(r)) is similar. In our case where d* = q - 2, computational results show that there is no significant difference in the coverage probabilities. Notice that

[I1171 - W l ( 1

2

PTlI[I1171

-

( 4 - 2)11~111-21 I(lIw11l2 > Q - 2)lI2

- Wl(1 - ( 4 - 2)11~111-2} < X&)l-

< x;(Y)l (7.9.31)

s+

Thus, the confidence sets C"(p,") and Cs+(p, ) satisfy the dominance coverage probability

cs+(p,s+) 1 C"(P,").

(7.9.32)

Consequently,

cs+(P:+) L CS(P,) 2 CO(bn).

(7.9.33)

The reader is referred to Tables 5.9.2 and 5.9.3 for some values of coverage probabilities.

7.10

Asymptotic Theory of Confidence Sets

In Section 7.9 we discussed the finite sample properties of the confidence sets centered a t the unrestricted, restricted, preliminary test, James-Stein-type, and positive-rule Stein estimators when o2 is known. When o2 is unknown, the closed form answer cannot be obtained. Hence, we consider the asymptotic theory, noting the fact that the estimator s: of o2 converges almost surely t o o2 as n -+ co. This leads us to use the results of Section 7.9 for moderate to large sample sizes. However, when the error distribution of the regression model is nonnormal, we consider the asymptotic theory as discussed in Section 7.7.

7.10.1 Confidence Sets Again, consider the model

Y , = X,P

+ e,

(7.10.1)


392

Pnl ,.

..s+

and the estimators P,, P, , P,, and p, of p given in Sections 7.7.1 and 7.7.2 and Sections 7.8.1 and 7.8.2. In this section, we consider the asymptotic theory of confidence sets defined in Section 7.6 repeated below: (i)

-PT

AS

c0(Pn) = { P : n s i ’ i i ~ PnI%=I x;(Y)>. -

(7.10.2)

where $(y) is y-level upper critical value of a central chi-square distribution with p d.f. We can write the confidence sets compactly as

C’(P3

=

{P : ns,’IlP

-

P:ll&,

< x;(Y)l,

(7.10.3)

where

P:

=

P, + (Pn -P n ) g ( L )

(7.10.4)

as before. We study the properties of these sets in the next section and show that asymptotically they are similar to that of Section 7.9.

7.10.2

Asymptotic Properties of Confidence Sets

From Section 7.8.1, we observe that the asymptotic coverage proFability of all the confidence sets is 1 - y under fixed alternative except C$(Pn). By Theorem 3 of Section 7.8.2, we obtain the asymptotic distribution of APT - S the statistics fis;’(p,’, - P ) , where P: = P, ,P,, and /?If+ under local alternatives

P,,p,,

K(n, : HP = h + n-1’2t.

(7.10.5)

First, we note that lim

n-m

PK(,,)(~S,~IIP -

PnII2C,, I x ; ( Y ) > = 1 - Y,

(7.10.6)

The rest of the expressions are obtained using the basic transformation and technique given in Section 7.9.2.

7.10. Asymptotic Theory of ConfidenceSets

393

(7.10.9)

and

respectively. Since the asymptotic coverage probabilities of the five sets are similar to those given in Section 7.9, the properties are same as before.


394

7.11

Nonpararnetric Methods: R-Estimation

In this section, we consider nonparametric methods of estimation, namely the R-estimation of the regression parameters, /3 = (PI , . . . ,&)’ in the usual model

Y =Xp+&,

(7.11.1)

where X is a n x p design matrix of the known regression constants ( n 2 p 2 1) and the components of the random vector E are i.i.d.r.v., with continuous distribution function I?(€,) defined on the real line R1. Further, we consider the following possible restriction on P:

HP = h,

(7.11.2)

where H is a y x p matrix of known constants (rank y) and h is a y x 1 vector of known constants as well. Our problem is the robust estimation of ,f3 when it is suspected but one is not sure that (7.11.2) holds. We consider R-estimation of 0. Let be the unrestricted and be the restricted (under (7.11.2)) R-estimators of P respectively. Further, let LTtbe the rank-test of the null hypothesis Ho : HP = h against the alternative H A : HP # h. Then following previous sections, we can consider three more R-estimators of P, namely, PRE, SRE, and PRSRE given by

p,

b,,

- PT = P , l ( L 2

PTRE : P,

(7.11.3)

SRE :

(7.11.4)

B: = p,(l- d C L 1 I ( L , > c,)) ( d = p - a), PRSRE : b;’ = p, + (p, - B,)(1 - dCL1)l(C,> d ) ,

(7.11.5)

where C, -+ 0 as n + 03, and is the a-level critical value from the null distribution of C,. In this section, we compare and study various properties of the R-estimators of /3 in an asymptotic setup.

7.11.1 Linear Rank Statistics, R-Estimators and Confidence Sets Let Ri(b) be the rank of Y , - (xi-R,)b among y1- (XI-%,)b, . . . ,y, R,)b, where xi is the ith row of X = (xi,. . . ,xi)’and Rn = n-’ Now, consider the LRS,

L ( b ) = (Li,(h), . . . , L p n ( b p ) ) ’ =

n

C(xi- %)an(Ri(b)),

+

- (x, -

cr=lxi. (7.11.6)

i=l

where (i) a n ( k ) = E&(Uk,,) or 4 ( k / n . 1) for square integrable and nondecreasing score generating function &(-) defined on ( 0 , l ) as before (see Chapters 4,5, and 6 ) .

395

7.11. Nonparametric Methods: R-Estimation Let C, = ‘j& (xi - %,)(xi- Z,)’, and assume that

;c,

(i)

=

c,

(ii) maxlsis, {(xi - X,)C,’(X~

I(f) =

-

Zn)} = ~ ( n ) . and A$ = ~ ‘ Q 2 ( ~ ) - ( ~ ’ Q ( U ) d z ~ ) ’ .

{f’(.)/f(.)}2f(.)d. 0 and E > 0, where w is a pdimensional column vector and denotes the pdimensional Euclidean norm. We obtain

&(D,

-P)

Np(0,o2C-l),

cT2

To define the restricted R-estimator of mator of p under HP = h as

= A2,/r2(+,

4).

11 . 11

(7.11-10)

P, we mimic the least square esti-

b, = p, - CG1H’(HCi1H’)(HB,- h).

(7.11.11)

In order to test the null hypothesis Ho : HP = h against H A : HP consider the rank-test

# h, we (7.11.12)

13, = A,2[L,(b,)I’C,1 [L,(b,)l, where

A: = (n - 1)-’ C ( u n ( k )k=l

sin = n-l C u n ( k ) .

(7.11.13)

k=l

We obtain the following theorem on the null distribution of

en:

Theorem 1. Assume (7.11.10). Then, under the null hypothesis HO : HP = h as n 4 00, 13, approximately follows the central chi-square distribution with q d.f.


396

Proof.

By the asymptotic linearity result (7.11.9) we have two relations: (i)

and

(ii)

L,(D,)

+ r(+,$)cfi(b, - 0)’ -Is, 0 L(P) - r(+,+)cfi(P, - P)’ 5 0 . - L(P)

(7.11.14)

From (i) and (ii) of (7.9.13), it follows that

C,

=

r2(+,$)A,’n(P,

- b,)’C(P,

= r2(+,4)A,’n(HP,

-

a,, +

OP(1)

- h)’(HC-lH’)-’(HP,

- h)

+ op(1).

(7.11.15)

After combining (7.11.10) and (7.11.14), we prove of the theorem. We note that the test based on 13, is consistent against fixed P such that HP # h. Thus limn-w P(C, > k) -+ 1 for all k E R1, and the PTRE, SRE, under a fixed alternaand PRSRE of p is asymptotically equivalent t o tive. Hence, we consider the properties of the five estimators under the local alternatives:

P,

7.11.2 Asymptotic Distributional Properties of the R-es t imators By the asymptotic linearity results of JureEkova (1969) (see also Puri and Sen (1986)), we have the following theorem:

Theorem 2. Under {K(,)} and the conditions (7.11.7) as n clude the following:

- a2C-1), A;/?($, 4), &(a, P ) - Np(-6, 6 C-’H’(HC-lH’)< A C-’ C-lH1(HC~HC-~.

6) f i ( P , - P ) (ii)

Np(0,

0’

=

a2A), where

-

=

(iii)

-+ CQ,

-

J;E(P, - j?,)

- N,(s,

=

we con(7.11.17)

and

IH/)-

a2(c-l - A)),

(iv) l i m p ( & 5 KIK(,))= H q ( z ; A 2 ) ,where A’ = a-’d’C6’ - a - z E I (HC-~H’)-~E. -

+

+

- P ) i KIK(,)) = H q ( x ? ( a ) ; A 2 ) @ , ( x6,0,02A) @ , ( x - c-~H’(Hc-~H’)z; 0,o ~ A ) ~ @ , 0, ( za;2 ( ~ ~ - 1 ~ i ) - 1 ) where E( + OP(l),


397

Theorem 3. Under {K(,)} and conditions (7.11.7), as n 4 00, the ADB, ADQB, ADMSE, and ADQR of the five estimators are given by

(i) UR.E(&: bl(&) = 0

(7.11.18) and B1(&) = 0;

MI(&) = a2C-l and R1(&; W) = o2t r ( C - l W ) .


398

where d = q - 2. Notice that these expressions are similar to the expressions of Section 7.8.2. Hence, the properties of the five estimators are asymptotically similar.

7.11. Nonparametric hfethods: R-Estimation

399

7.11.3 Asymptotic Properties of the Recentered Confidence Sets Based on R-Estimators We consider the recentered confidence sets based on the R-estimators, namely

where ut = At/$(+, bY

4 ) and $($, 4) is a consistent estimator of T($, 4) given

C$) is the j t h row of C , , and ej is the unit-vector with 1 at the j t h place and zero elsewhere. We can, again, write the confidence sets compactly as

(p,

- bn)g(L,). We study the asymptotic ( n -+ m) propwhere P: = ,bn+ erties of the confidence sets under the local alternative

First, we note that

Chapter 7. Multiple R,egression Model

400

Next, we consider the coverage probability

by suitable transformation as in (7.9.9) and (7.9.10). Thus, we can write the asymptotic coverage probability as

This expression is the same as (7.9.13). Correspondingly, all conclusions derived in Section (7.9.3) hold. This expression is the same as (7.9.13). Hence, all analyses and conclusions are the same as given in Section 7.9.3 hold.

7.12

Conclusions

In this chapter, we studied the estimation of the regression parameters when it is suspected that the parameters belong to a subspace. Accordingly, the unrestricted, the restricted, the preliminary test, the Stein-type, and the positiverule Stein-type estimators are defined, and their properties are studied in small as well as in large samples. In addition, we discuss the confidence set estimation of the parameters in both small and large samples, which include the nonparametric R-estimation methods.

7.13 1. 2. 3. 4. 5. 6.

Problems

Derive the expression for f i n given in (7.1.4). Verify remark 2 of Section 7.1.1. Prove Theorem 2 of Section 7.1.2. APT Verify the expression for MRE(Pn ; Prove Theorem 3 of Section 7.3. Find the expressions for the covariance matrices of

on).

..s+

Pn . 7. Prove that the risk-difference of -S

-s on and @,

fin, 0, , on,and -PT

-5'

is given by (7.4.17).

8. Verify that the risk of 0, can be expressed as the r.h.s. of (7.4.19). APT

9. Verify that the MRE(Pn

;on) is given by the R.H.S of (7.5.15). PT+

10. Verify the expression for M~(fl:") tions 7.6.4 and 7.6.5.

L.

and R6(Pn

: W) given in Sec-

401

7.13. Problems

-s

11. Prove that the risk R4(Pn : W) may be expressed as R1(Pn : W) - W A n ) { ( q - 2)E[X,;22(A2)l

given by (7.8.33). 12. Prove that n-cc lim P K + )

{ nsi211P- P2I&, < x;(d}

= %{11171 - w7(ll~1112)112 + 11172

- w21I2

5 x2,(r)}

by suitable transformations and analysis, and verify (7.10.2) through (7.10.5). 13. Prove that

q 1 P - P$

&(p,

14. Prove that under K(nl.

-

= 11771

P)

-

fi(pn - bn)

15. Prove that under K(n). 16. 17. 18. 19.

-

-

-

wlg(l/wll12)J/2 + 1%

- w2ll2.

Np(0,a2(C-1)) where a2 =

N p ( - 6 , a2(C-l - A)) of Theorem 7.9.2

N p ( - 6 , a 2 A ) under K ( n ) . Prove that fi(bn - 0,) Show that limn-cc P { L , < zIK(,)} = Hg(z; A2). Verify Theorem 7.9.3. Numerically evaluate

for q = p - 1.


Chapter 8

Multiple Regression Model: Stochastic Subspace Hypothesis Outline 8.1 The Model, Estimation and Test of Hypothesis 8.2 Bias, Quadratic Bias, MSE Matrices, and the Quadratic Risks of the Estima-

tors 8.3 Estimation of Regression Parameters with Prior Information from a Previous

Sample 8.4 Multiple Regression Model and Stochastic Subspace Hypothesis: Nonnormal

Errors

8.5 Asymptotic Distribution of the Estimators

8.6 8.7 8.8 8.9

Confidence Set Estimation: Stochastic Hypothesis &Estimation: Stochastic Hypothesis Conclusions Problems

In Chapter 7, we considered the estimation of the regression parameters P in the multiple regression model Y = XP + e when it is a priori suspected that belongs to the subspace defined by HP = h. In this chapter, we consider the estimation of P when P belongs to the stochastic subspace defined by h = HP + v, where v is normally distributed with mean 6 and covariance matrix a2s2,with s2 being a known ( q x q ) positive-definite matrix. In addition, we consider the estimation of the regression parameters when s2 is unknown based on a sample from previous study. Asymptotic theory is also provided for nonnormal errors in the models. Here we consider the idea of mixed estimation involving combined sample information from the model and an independent prior stochastic subspace information. The methodology is 403

404

Chapter 8. Regression Model: Stochastic Subspace

due to Theil and Goldberger (1961), Theil (1963), and Nagar and Kakwani (1964) among others. Further, we provide the confidence set analysis involving various estimators of p for finite as well as for large sample.

8.1

The Model, Estimation, and Test of Hypothesis

In this section, we formally present the model, estimation of the parameters, and related test of hypothesis results that will be used in the discussions of the multiple regression model.

8.1.1 The Model Formulation Consider the estimation of the regression parameters in the model given by

Y=Xp+e when it is suspected that the parameter model, namely

h = HP + V,

v

(8.1.1)

p belongs to the stochastic subspace

- N,(S,

u2Q),

(8.1.2)

where h is a known q x 1 vector, H is a known q x p matrix of constants with rank g, and v is a q x 1 vector of random errors. In the model above Q is a known matrix but c2is unknown. This type of model appears in many econometric analyses. Reader may see Theil and Goldberger (1961) and Theil (1963). For a suitable analysis, consider the sample information and model (8.1.1) together with the stochastic subspace model (8.1.2) to obtain the following mixed model: (8.1.3) where (8.1.4) Thus, our problem reduces to the estimation of when it is suspected, but not certain, that

p

for the model (8.1.3)

(8.1.5) Rewriting (8.1.3) and (8.1.5), we have the problem of estimating 8 in the model

Y "= Z 8 + E ,

(8.1.6)

8.1. The Model, Estimation, and Test of Hypothesis

405

when it is suspected that 8 belongs to the subspace RB = 6 = 0, where

and

(8.1.8)

8.1.2

Mixed Model Estimation

Applying the least squares principle to the model (8.1.6), we have the unrestricted mixed estimator of 8 for

(8.1.9) Similarly, the mixed restricted estimator of 6 subject to RB = 0 is given by

8,

=

8,

-

C;~R’(RC;~R’)-~R~,, c, = z’z,

1

+ n)-’(HB, - h) . + a)-’ (Hp, - h)

C-lH’(HC-’H’ -n(HC-’H’

Let

a, denote the restricted estimator of ,O; then a, p, C-lH’(HC-’H’ + SZ)-l(Hp, =

-

(8.1.10)

- h)

from (8.1.10). Let us now consider the estimation of P based on the formulations (8.1.3) and (8.1.4) using the generalized least squares principle (GLSE). The mixed restricted estimator of P of the mixed model (8.1.3) is given by

= (C

+ H’W1H)-’(CP, + H’St-Ih).

(8.1.11)

Since

(C + H’n-’H)-’

=

C-’

-

C-lH’(HC-lH’+ O)-’HC-’

(8.1.12)

by Theorem 2.6.8 of Chapter 2 , we have

$f = (p,+ C-’H’n-’h)

-

C-lH’(HC-lH’+ n)-’H(p,

+ C-’H’O-’h).

(8.1.13)

406


Is there any difference between difference

b and p- nR ? To find the answer, consider the

-R

an

=

-Pn

[Pn-c'H' (HC-'H/ + fl))-'(Hp, - h)] - [(p,+ C-'H'SI-'h)

+ O)-'HC-'(Cp + H'W'h)] (8.1.14) = C-'H'[O-' - (HC-'H' + SI)-'(HC-'H' + fl)SI-']h = 0 , which shows that 6 = 8:. Therefore, the restricted mzxed estimator of 8 can C-'H'(HC-'H'

-

be rewritten as

en =

( 3, )

(8.1.15)

+

If we premultiply 6, and 6n by the p x ( p q ) matrix ( I p , O p x q ) , we obtain and respectively. In the next section, we consider the test of the hypothesis Ho : 6 = R8 = 0 against the alternative H A : 6 # 0.

p,

a,,

8.1.3 Test of Hypothesis The joint distribution of

(on,p,)' -I

..I

is given by

(8.1.16) where An = C-' - C-'H'(HC-'H'+ SI)-'HC-'. For the estimation of o', we consider s: = m-'(Y*- Z6,)'(Y* - Zen) = Y*[I - z(z'z)-'z']y* = m-'(Y

- xpn)I(Y- XP,), m = 72 - p .

(8. I. 17)

Further,

(6, - e n ) ' C Z ( 6 -, 6,)

= eLR'[RC,lR']-lRen

= (HP, - h)'(HC,'H'

+ fl)-'(HPn - h).

(8.1.18)

Hence, we can define the statistics .Cz for testing the hypothesis HO : Re 6=Oby

=

.C* = ~;R'(RC;~R')- 1 ~ 6 , qs2 -

(Hp,

- h)'(HC-'H'

4 2

+ fl)--'(HBn- h) 1

(8.1.19)

8.1. The h'lodel, Estimation, and Test of Hypothesis

407

which is similar to the statistic C, given in (7.1.6) except that it contains a. The distribution of Cc under Ho : 6 = R6 = 0 follows the central Fdistribution with ( 4 ,m ) d.f. Under H A : 6 = R6 # 0, it follows the noncentral F-distribution with (q, m ) d.f. and noiicentrality parameter A2/2 where

A2 = a-2[6'R'(RC,1R')-1R6]

= K26'

(HC-lH'

+ a)-' 6.

(8.1.20)

These statements are proved in the same way as in Theorem 7.1.1.

8.1.4 Preliminary Test and Stein-type Mixed Estimators Following Section 7.2.1, we define the PTE of 6 as . PT

6, = 6, - (6, - 6,)l(C; < Fq,,(a)),

(8.1.21)

where F,,,(a) is the upper cu-level critical value from the central F-distribution with (q,m)d.f. Similarly, the James-Stein-type estimator of 6 is given by

-s 6, = 6 ,

-

d(8, - 6,)C;-',

(8.1.22)

and the PRSE of 6 is given by

-s+ 6, - 6,

+ (1 - dC;-'}l(L;

> d)(6, - k),

(8.1.23)

where (8.1.24) Explicitly, the expressions for PTE, SE, and PRSE are given by (8.1.25) where

a,

=

p, - C-'H'(HC-'H' + a)-lHC-l(H,h,

- h).

(8.1.26) and (8.1.27) -PT - S . S+ If we premultiply 6, , 6,, and 6, by the p x ( p + q ) matrix ( I p , O p x g ) , we APT - S ..S+ obtain the corresponding expressions for P, , P,, and P, (see also Khan, 1997).

408

Chapter 8. R,egression Model: Stochastic Subspace

Bias, Quadratic Bias, MSE Matrix, and the Quadratic Risk of the Estimators

8.2

In this section, we provide the bias, quadratic bias, MSE matrices, and risk expressions of the five estimators of 6 .

8.2.1

Bias and Quadratic Bias Expressions

We consider the bias and quadratic bias expressions of the estimators of 6 . They are given by

&(a,)

(i) bl(8,) = 0 and (ii)

(8.2.1)

b2(6n)

= -C,1R’(RC,1R’)-1R6

B2&)

= a-26;C,6,

PT

(iii) b3(6, ) (iv)

= 0;

-S b4(@,)

=

=

,.S f

(v) b 5 ( 6 n )

= A2;

-6,Gq+2,,(&; A’) and

-6, (say), then

- PT

-dq6,E[~;:~(A’)] and B4(6:)

=

)

= A2{Gq+2,m(t,;A2)}2;

= d2q2A2{E[~,S22(A2)1}2;

- 6 z { d 1 E [ F 4 ; ’ , , ~ ( A ~ ) r ( ~ q + ~'2,m (A2))2I(Fq+2,m(A2) < dl)]

+ a-26,6:E[(1 - dzF,-:,,,(A'))21(Fq+4,~(A2)< d z ) ] } + N , ~ : E [ ( ~ I F [ ~ ~ , -~ 1)I(Fq+2,m(A2) (A~) < dl)] 2 0.

(8.2.17)

,.S+ The difference between the MSE matrices is p.s.d. for all A'. Hence, 6, -S

-

-s

dominates On uniformly. But, 6, does not dominate 8,. Hence, we can write ., S+

M4(6:) 2 Ms(6, ).

(8.2.18)

414

Chapter 8. R,egression hiodel: Stochastic Subspace

Next, we consider the difference between the MSE matrices, which is

and S+

~ ~ ( )6- ~, - &(q -

PT

~ ( 8= ,~ZC;’R’(RC;~R’)-’RC;’( -

~)E[X;:~(A’)] - 2dqA2E[~&(A2)]

E“1 - ~ l F ~ 1 z , , ( A 2 ) ) 2 ~ ( F q + z , m. Accordingly, the five confidence sets of (i) CO(P,)= (ii)

P are given by

{ P : o-211P - P,lE 5 x;cr,>,

cR(B,) = { P : 0-'11b - P,II&I x;(r)>, ..PT

(iii) CPT(B;'(~)) (iv)

(8.6.7)

c@)

=

{ P :a

- 2 -~P,~ 11%~

= { P : a-211P -

F X;(Y>>,

b h F xgcr,>,

and S+

I x;(r)>,respectively.

(v) CS+(bEf)= { P : u - ~ I ~-PP,

(8.6.8)

Following Chapter 7, Section 7.9, we obtain the expressions after the coverage probabilities of these confidence sets by

P{a-2/lP- P;I& =

1772

L x;(Y)}

- Wl2 + 1711 - ~ l ~ ( l l ~ 1 1 1 2I )j2

x;cr>>.

(8.6.9)

As given in Section 7.9,

(see Problem 11). Similar conclusions hold for the nonnormal case considering the asymptotic setup.

8.7

R-Est imation: Stochastic Hypothesis

In this section, we consider the problem of R-estimation of the regression parameters P in the model (as in Section 8.1)

Y , = P O L+ X,P when it is suspected but not certain that defined by

h = hp

+ e,

(8.7.1)

belongs t o the stochastic subspace

+ 6 + vq,

(8.7.2)

where h is a q-vector of random variables, H is a q x p matrix of known constants, 6 is a q-vector of constants (unknown), and vg is a q-vector of

431

8.7. R-Estimation: Stochastic Hypothesis

random variables such that E ( v q ) = 0 and Cov(v,) = $a; here 0 is a known q x q matrix of constants. In order to obtain the R-estimator of 0, we first concentrate on the model (8.7.1) and consider the following toward estimating 0: Let R,(b) be the rank of Y , - (x,- %,)b among Yl - (x,- x,)b, . . . ,Y, (x, - R,)b, where x, is the ith row of X = (xi,.. . ,xk)' and X, = n-' c,"=l x,. Now, consider the LRS:

L,(b)

=

. . . Lpn(bp))' =

(h2(h),

7

c n

,=I

(x,- %L)GL(Rz(b)).

(8.7.3)

(i) For a square integrable and nondecreasing score generating function 4(.) defined on ( 0 , l ) as before (see Chapters 4, 5, and 6), a n ( k ) = E#(Uk,,) or

4(&>.

(ii) Let C, = C,"=l(xz - %,)(x, - R,)', and assume that limn-m C, = C , and max12zjn{ (x,- %,)C;'(x, - Xn)} = o ( n ) .

(8.7.4) Further, if we put [/all = ck=lp("),and a = (a('), . . . ,a(?')), then the unrestricted R-estimator (URE), 0, of p is defined by any central point of the set

S = { b : llL,(b)/l = minimum}.

(8.7.5)

Then, by JureEkova's (1969) linearity result (see Section 2.8.4), we have

for any k > 0 and E > 0, where w is a pdimensional column vector and denotes the pdimensional Euclidean norm. Then we obtain

h ( P , -P) -~p(o,O2c-'),

= A;/r2($,4).

I] . [I

(8.7.7)

To define the restricted R-estimator of p, we mimic the least square estimator of P under (8.5.2) as the restricted rank estimator (RRE) of P given by

b, = p, - Ci'H''

n

(8.7.8)

432


where Cov(v,) = $fl and 0’ = A$/y’($J, 4) as defined in (8.7.4) and fl is a krlowr~q x q matrix of constants. In order t o test the hypothesis Ho : 6 = 0 against the alternative H.4 : 6 # 0, we consider the rank-test given by

13,

=

(~~(+,4)A,’}n(Hp,- h)’(nHC,’H’

where A: in($,

and

= ( n - 1)-’ CE=l(a,(k) - En)’,

+ fl)-’(HD, E n = n-l

4) = P-’ C,”= Crjn(47 4)

(8.7.9)

- h),

C;=lUn(k), and

(8.7.10) with the following two constraints: (i) C ( j )is the j t h row of C.

(ii) ej is the unit-vector with 1 at the j t h place and 0 elsewhere. Here, Tn($, 4) is a consistent estimator of ~ ( “ ~ 4 ) . Thus, by (8.7.7) as n + co, 13, converges in distribution to the central chi-square with q d.f. under Ho : 6 = 0. Further, under the fixed alternative, HE : 6 = 6, C, -+ 03 as n 4 03, since

C,

where 8, = HP, Note that (i)

6;’fiSn

-

n(8,

(8.7.11)

h.

- N{na-’ k) 4 1 for all k E R. However, under the null hypothesis Ho : 6 = 0 , L, approximately follows a central chi-square distribution with q d.f. Now, we consider the mixed rank estimators of 8 = URE of 6 is then given by

8,

=

(h

6,

=

( Hi+&

>-

The

( % ), and the RRE of 8 is given by

Pn

+

+O(HC;’H’ $fl)-’(H& - h) >. Let x:(a) be the upper a-level critical value from a central chi-square distribut:on with q d.f. Then we define the PTRE of 8 as -

-. PT

on

= 6, - (6, - 8,)1(Cn

< &a)).

(8.7.13a)

8.7. R-Estimation: Stochastic Hypothesis

433

Similarly, we define the SRE and PRSRE of 6 as the following:

-s

-

+ 03,

and d = 1 - 3 and

6, = 6, - d(6, - 6,)LL11(L, > E where

E,

+0

as n

e:+

6, + (1 - dLi1)I(Ln > d)(6, - 6,)

=

~ ) ,

(8.7.13b)

(8.7.13~)

as in Section 8.4.3. Further, we can show, as in Section 8.7.1, that under the fixed alternative H3 : 6 = 6, we have (i) 6;1&(6ET

= a-1+(~,

-S

(ii) a;’&(~~

-

- 6)+op(l),

+ op(l), - 6 )+op(l),

0) = a-lfi(6, - 6 )

(iii) 6;’&(On -s+- 6 ) = a-’&(Gn

(8.7.14)

while the asymptotic distribution of &6,(6, - 6 ) is degenerate as n Thus, as in Section 8.5.1, we consider the local alternatives

K(,) : 6(,) = n-l/’E,

6 E R”

-+

00.

(8.7.15)

to obtain the following theorem: = nP1/’< and the assumed regularity condiTheorem 1. Under K(,] : tions of (8.6.4), we have the following as n -+ 03:

(i)

vJ~;= &(a,

(ii) VJ:;

=

&(an

A, = C;l(iii) VA ;:

- 6) - 6)

- N,+~(o, -

Nn+q(-Sx,a2AZ),where

C;’R’(RC;lR’)-’RC;’

= &(6, - 6,)

(8.7.16)

a2c;1).

-

and 6, = C;lR’(RC;lR’)R d)P,.

In the next section, we define the ridge regression estimators of p corresponding to each of the estimators, beginning with the unrestricted ridge regression estimator (URRE).

9.1. Ridge Regression Estimators

9.1 9.1.1

441

Ridge Regression Estimators (RRE) Ridge Regression with Normal Errors

In this section, we consider the ridge regression version of the four estimators of p. Accordingly, the unrestricted ridge regression estimator (URRE) of p is defined by the ridge matrix, R(k) as follows:

p , ( k ) = (C + kIp)-lX'Y = C-l(k)X'Y

= R(k)P,,

(9.1.1)

p,

where R ( k ) = (IP+kC-')-', k 2 0, is the unrestricted estimator of p and k is the ridge parameter. This is the well-known Hoerl and Kennard (1970) estimator of p. Further, due to Rao (1975), a generalized ridge regression estimator of p can be defined by choosing a positive-semidefinite matrix G, instead of kip, and writing the estimator as

j,(G) = (C + G)-lX'Y = (I, + GC-l)-'p,.

(9.1.2)

Under the null hypothesis Ho : p = 0 , the restricted ridge regression estimator (RRRE) of p is defined by p,(k) = 0, where 0 is the restricted estimator of p. Next, we consider the preliminary test ridge regression estimator (PTRRE) of p defined by

,bET(k) = R(k)bET= R(k)p,

- R(k)P,I(L,

5 F p , n ( ~ ) ) , (9.1.3)

where C, is the test-statistic for testing p = 0. These kinds of estimators were studied by Saleh and Kibria (1993) for the normal regression model and by Kibria (1996a) for the Student's t regression model. Parallel to PTR.RE, we define the Stein-type ridge regression estimator (SRRE) of as

Finally, we consider the positive-rule Stein-type ridge regression estimator (PRSRRE) of p defined by

s+

a t ' ( k ) = R ( k ) P , = R(k)bf - (1 - dL;')I(C, < d ) R ( k ) P , .

(9.1.5)

These kinds of estimators are considered by Kibria and Saleh (2004a, b) for the normal regression model and by Tabatabaey, Kibria, and Saleh (2005) for the Student's t regression model. -PT - S . St The four estimators 0, p,, and p, therefore give rise to four corresponding ridge regression estimators that are of interest in practical work in applied statistics.

p,,

442

Chapter 9. Ridge Regression

Nonparametric Ridge Regression Estimators

9.1.2

In this section, we consider ridge regression estimators based on rank statistics. As in Chapter 7,we consider the model

(9.1.6)

Y=Xp+E,

where X is a n x p matrix of known constants such that C, = cy=l(xi%,)(xi- ?in)' (where xi is the ith row of X and X, = ~ ~ ~ = lis xill-i ) conditioned. This means that some of the characteristic roots of C, are small, which makes some of the asymptotic variances very large, and as a result makes the inference problem difficult. Thus, as in Section 7.11.1,consider the linear rank statistics required for testing Ho : p = 0: n

(9.1.7)

Ln(b) = C ( X i - %Jan(&(b)) i=l

as given by (7.11.6). We assume (7.11.7i-iv). In particular, we mention limit,-+w(n-lC,) = C and max { (xi - %,)'C,'(xi - X,)} = o(n).

pr)

Let be the unrestricted rank-estimator of point of the set

p by choosing the central (9.1.8)

S = { b : I IL, (b)I I = minimum}. Then, as in Section 7.11.7,

h@r)P ) -

Jyp(o7

A;/?(+,

(9.1.9)

4c-').

We define the ridge matrix R ( k ) as

R ( k ) = lim (I, + k(n-'C,)-' new

= (I,

+ kc-')-'.

(9.1.lo)

-

Then the nonparametric ridge regression estimator of ,B is defined by p, ( k ) =

- (.)

R(k)P, , and under the local alternatives, f l ( n ) = n-l/'d, 6

&(&;- p)

N

= ($1,.

Np{ - kC-'(k)G, A;/r'[R(k)C-'R(k)]}.

. . ,a,)',

(9.1.11)

Thus, the asymptotic distributional bias, covariance, and MSE matrix can be computed. We mention that the asymptotic distribution of the least square theory based ridge regression estimator R(k)p,, where is the LSE of 0, under the local alternative K(,) : p(n)= n-'/'G is again

p,

N,{

- kc-'(k)6;

2(~(k)c-~~(k))}.

(9.1.12)

Hence, the asymptotic efficiency of the rank-based ridge regression estimator is +Y2(47,

&A,',

(9.1.13)

9.2. Ridge Regression as Bayesian Regression Estimators

443

which is the same as the efficiency of the R-estimator relative to the LSE of

P.

Based on the rank ridge regression estimator of more estimators, as before:

p, we can define the three (9.1.14)

(ii) fi,"(k) = ~ t ' ( k ) { l- ( p - ~ ) L ; ' } I ( L ,> En) where (iii)

E,

---$

0 as n -+ 03, and

S+

p, ( I C ) = P:'(~){I

- ( p-~

) L ; ~ } I ( L>, ( p - a)),

respectively, where

L, = A,2[Lh(0)C,1L,(O)]

(9.1.15)

EL=,

with A: = ( n- 1)-' C ; = l ( a n ( k ) a, = n-' a n ( k ) is used to test the null hypothesis Ho : = 0. These estimators parallel those given in (9.1.3through 9.1.5). The bias, covariance matrix, MSE matrix and risks of the estimators are given in Problems 10 and 11 and are left to the reader to verify.

9.2

Ridge Regression as Bayesian Regression Estimators

We justify the ridge regression estimator (RRE) as a Bayesian estimator of the regression parameter p. be Let Y given /3 be N,(XP,a21n), and let the prior distribution of N p ( y ,r2V). Then the posterior distribution of p given Y = y is y

+ (C+ -07V2-')-'X'(y

02 + -V-')-' 72

- Xy),a2(C

Thus, the Bayes estimator of /3 given Y = y under a quadratic loss function is the posterior mean

E [ P ( y ]= y

+ ( C + kV-l)-'x'(y - Xy),

(9.2.2)

where k = a 2 / r 2 This . estimator is the generalized ridge estimator of p. If V = I,, and y = 0, then E(PIy) is equal to the ridge estimator of /3 defined by Hoerl and Kennard (1970). Thus, the Bayes estimator of /3 in this case is given by

p,(k) = R(k)P,,

R(k) = (I,

+ kc-')-',

k 2 0.

(9.2.3)

The remaining three estimators are obtained by transforming the basic estimators given in the introduction. In addition, if k is estimated by some function,

444


k ( y ) of the sample observations, then we obtain the so-called adaptive ridge estimators of P. The parameter k = c?/-r2 is called the ridge parameter of the ridge estimator. There exists a number of methods in the literature for the estimation of k. The interested reader is referred to Newhouse and Oman (1971), Farebrother (1975), Hoerl and Kennard (1975), Hoerl, Kennard, and Baldwin (1975), Marquardt and Snee (1975), McDonald and Galarneau (1975), Obenchain (1975), Dempster et al. (1977), Lawless and Wang (1976), Lawless (1978), Hemmerle and Brantle (1978), and Vinod and Ullah (1981) among others. More discussion on the ridge regression is available in Gruber (1998) and Montgomery and Peck and Vining (2001). Hoed and Kennard (1970) suggested use of the ridge trace t o find an appropriate value of k . The ridge trace is a plot of @,(k) versus k between 0 to 1. If the multicollinearity in the data is severe, the ridge trace will show the obvious instability in the regression coefficients. However, more work needs to be done on the ridge trace. An excellent review and study of this problem is given by Kibria (2003), details of which are provided in Section 9.6.

9.3

Bias Expression of Various Ridge Estimators

The bias vectors of various ridge estimators are given in this section. Bias Vector of The bias vector of the unrestricted RRE is given by

p,(lc).

bl(P,@)) =

=

9.3.1

E(P,(k)

-[Ip

-

-P) = ~

~ R ( k ) -PPI,

R ( k ) ] P= -kC-’(k)P.

(9.3.1)

Bias Vector of f(’(k) PT

Similarly, the bias vector of PTRRE, p, ( k ) can be written as

b Z ( P 3 k ) )= E(P%

- P)

= E ( P , ( k ) - PI - E { R ( k ) P , G

IFp,m(4)}

= - k c - ’ ( k ) P - R(k)PGp+2,,(&;

A2),

where e, = &Fp,,(a)

(9.3.2)

and A’ = (P’CP)O-~.

-s

-s

Bias Vector of P,(k). In a similar way, the bias vector of SRR.E, P,(k) can be written as b3

( P h ) = E (a%

- P ) = E(P,(k)

-

P ) - PdR(w(P,.C,l)

= - k C - l ( k ) P - p d R ( k ) P E [X,S~~(A’)] .

(9.3.3)

9.3. Bias Expressions

445

..S+

. S+

Bias Vector of P, (k). Finally, the bias vector of PRR.RE, P, ( k ) can be written as

bdP:+(w= E ( P : + ( k ) - P ) = E(p:(k) - P ) - qdR(k)E{B,(l - d L i l ) I ( L , =

- ~ c - W-Pdl~[~;&,@2)1~ +dlR(k)PE

=-kC-WP

A 2 2 ... A, > 0 are the eigenvalues of C . It is easy to see that the eigenvalues of R ( k ) = (I, k c - ’ ) - ’ and of C ( k ) = ( C kI,) are X z + k , . .. , and ( X i k , XZ k , . . . ,A, k ) , respectively. Then

(x1xz

&)

+

+

+

+

+

we obtain the following identities: P

t r ( R ’ ( k ) C - ’ R ( k ) )=

i=l (Az

Xz

+ k ) 2’

(9.5.1)

(9.5.2)

For the risk comparison, we let W = I,.

C o m p a r i s o n of 6, and p , ( k ) . The risk function of p , ( k ) is given by (9.5.3)

If k = 0, then the first term equals u2Cr=lA,’, and the second term equals 0. The first term is a continuous monotonically decreasing function of k , and its derivative w.r.t. k approaches 00 as k -+ O+ and X p + Of as k -+ 0. The second term is also a continuous monotonically increasing function of k , and its derivative w.r.t. k approaches 0 as k -+ O+. Note that the second term approaches 11/3112 as k 00. Thus, differentiating (9.5.3) with respect to k , we get ---f

Next we define (9.5.5) where amaxis the largest element of a. We see that a sufficient condition for (9.5.4) t o be negative is that there exists a k E ( 0 , k l ) such that URRE, p , ( k ) has smaller risk than that of unrestricted estimator, Similarly, we can show that a sufficient condition for R1(fin(k);I,) t o be smaller than R l ( p , ; I , ) = u2trC-’ is that there exists a A’ E ( 0 , A 2 ( k ) where ]

p,.

A 2 ( k )=

tr[C-l - R ( k ) C - ’ R ( k ) ] . k2Ch,i, [C-2( k )c - 1 1

Combining (9.5.5) and (9.5.6), we have the following theorem:

(9.5.6)

45 1

9.5. Performance of Estimators

Theorem 1. A sufficient condition for the superiority of @,(k) that there exists a pair (A2,k) such that

(A2$)

E

(0,A2(kl)lx (Olkll.

Further, the MSE difference between

u2(c-l =

-

6,

over

@,

is

(9.5.7)

and B ] , ( k )is given by

R(~)c-~R’( ~k2c-1(k)pp’c-1(k) ))

(9.5.8)

r(h+ k ~ , ) - l {(..’(A + ~ I ~ ) A - ~+(ICI) A - 2~- k 2 a a ’ } ( + ~ krP)-T

= I’(A

+ kip)-' [ka2(21p + k A - ’ ) - k 2 a a ’ ] ( A+ kIP)-’I’’.

(9.5.9)

The r.h.s. of (9.5.8) is an n.n.d. matrix if and only if (1) a belongs to the range of (21, kA-’) and (2) a’(2IP k A - ’ ) - ’ a 5 a 2 / k , holds true (see Baksalary and Kala, 1983). Using Courant’s theorem, we have

+

+

[ (

1 A’ 5 - max ___ k 1SiSp 2Ai + k

)I-’

= A2(kl).

(9.5.10)

Comparison of PTRRE with PTE and URRE. Case 1: Null hypothesis Ho : p = 0 . In this case, the risk-difference of the two estimators, namely u’tr[C-’

-

PT

*

PT

p, and p, ( k ) ,is given by

R(k)C-’R’(k)](l - Gp+~,m(t,; 0)) 2 0.

PT

(9.5.11)

PT

Hence, p, (k)dominates p, uniformly for all k ( 2 0) and.,! Case 2: Alternative hypothesis HA : p # 0 . The risk-difference in this case is given by 0’

tr[C-’ - R(k)C-’R’(k)](l - Gp+2,m(!a; A2))

-p’k -k{I

- CC-2(k)C{2Gp+2,m(!,;

A’)

+ C-’(k)C}G,+z,,(!,;

- CC2(lc)

-

Gp+4,m(c;A’)} - k 2 C 2 ( k )

A’)]p.

(9.5.12)

The expression (9.5.12) is nonnegative if and only if

P’[k2C-2(k)+ (1- CC-2(k)C>{2Gp+~,m(L; A2) - Gp+~,rn(!;; A’)}

+k{CC-’(k)

5 u2tr[C-’

-

+ C-2(k)C}Gp+z,m(tcr; A2)]p R(k)C-’R’(k)](l - Gp+2,m(!a;A2)).

(9.5.13)

By standard calculations using Courant’s theorem, we obtain

A2 I

u’ tr[C-’ - R(k)C-’R’(k)](l - Gp+P,m(!,; A2)) = A’@, a ) , Chmi, [AC-’1 (9.5.14)

Chapter 9. R.idge Regression

452

where

A

+ { I - CC-2(k)}{2Gp+2,m(t,;A2)- Gp+4,,(t;; +k{CC-’(k) + C-’(k)C)G,+z,,(l,; A’).

= k2C-’(k)

Now, differentiating the risk of

PT

p, *

(9.5.15)

( k ) w.r.t. k , we obtain

-a2(1 - Gp+2,m(&;A’)) -Gp+4,m(f:,;

A’)}

-

X,2~?(2G~+z,,(&; A’)

A’))}.

(9.5.16)

A sufficient condition for (9.5.16) to be negative is that there exists a k E (0, k z ) where k’ = l min d. -s 6,PT ( k ) is superior to p,(k);

-s

otherwise, p , ( k ) is superior t o

(9.5.51) . PT

0, ( k ) .

9.5. Performance of Estimators

459

460


The risk-difference of the two estimators is given by

(9.5.62)

9.6. Estimation of the Ridge Parameter

46 1

+ (1 - ( P2a2A2 i 2 ) h i a f ) ( ~ A ’ E [P+4 x - ~(A2)]))

and

fi(a2) = lmax 2

-s

b4(P,(Pn)) = 4 9 - 2)6,E[X,S22(A;)l and B4(P,"(Pn))= qE[X;:2(A;)ll2 b5

-s+ -

(P,

(Pn)) = b4

-s+

Bs(1 - P,

-s

(P,(Pn)) - ~ , E W- ( 4 - 2 ) X , S 2 2 ( a ; ) ) I ( X ; + z ( A ; ) < q-2)l

-

(Pn)) = A;{(4

-

2)E[X,-:2(A;>lEI(1-(4-2)X;~,(A;>>

xI(x:+z(A;)

< 4 - 211 l2

(ii) For ADMSE and ADQR of P,(P,), -s+ -

P,

a,(&),b,"(P,),Bz(Pn),and

(Pn),

M1@,(&)) = a,"C,' and Rl(P,(P,)) = 0,"tr[WC,'];

M2(b,(Pn)) = o,"[CP' - C,lH'(HC,'H')-lHC,']

R2(Bn(Cn) : W) = 0," tr[WCp'] +6bW6,;

(10.4.2)

+ 6,Sb

- c,"tr[WC,'H'(HC,'H')-'HC,']

(10.4.3)

484

Chapter 10. Regression Models with Autocorrelated Errors

10.5. Estimation of Regression Parameters When p Is Near Zero

10.5

485

Estimation of Regression Parameters When p Is Near Zero

Judge and Bock (1978) considered the properties of the preliminary test estimator of p for the model (10.2.1) with autocorrelation p , when it is suspected that p is zero. We attempt to improve on the procedure in this section. To define the preliminary test estimator of p, Judge and Bock used the DurbinWaston test (1950, 1951), &, defined by (10.5.1) where Ell is the residual error -Xp,(P,), t = 1 , 2 , . . . ,n for testing HO : p = 0 against H A : p > 0. In this case, the critical region is of the form {&I& < DL( a ) } and , the decision is reached as follows: If f i w < D L( a ) ,HOis rejected, and if & > D,(a), the hypothesis HO is not rejected. If DL(CY)< f i w < D,(a) then Ho is inconclusive. Durbin and Watson provided the upper and lower critical values D U ( a )and DL(cY), respectively, with various sample sizes. A second test due to Berenblutt and Webb (1973) is indicated by the 91test, defined by =

where B is the matrix

B=

[

Y’[B - BX(X’BX)-lX’B]Y Y”I - X(X’X)-lX’]Y ’

2 -1 0 0 0

-1 2 0 0 0

0 -1 0

0 0 0

... -1 ... 0

.’. ..’ ... 2 -1

(10.5.2)

(10.5.3) -1 -1

test is generally higher than the power of the D-W-test The power of the for higher values of p; otherwise, 61 test is similar to the D-W-test. So the test can be reduced to the same canonical form as the D-W-test. Both tests have been obtained by approximating the likelihood function for p in two different ways. (Durbin and Watson, 1950, 1951, and Berenblutt and Webb, 1973, for details; for some related problems, see Kmenta and Gilbert 1968, 1970).

10.5.1 Preliminary Test and Stein-Type Estimators (Chen and Saleh, 1993) If p = 0, then the LSE is

p, = (X’X)-lXY. The estimator of p is given by

p,(pn) = (x’c;nlx)-lx’c;nlY,

(10.5.4)

486

Chapter 10. R,egression Models with Autocorrelated Errors

if p is unknown. Let L, be a suitable test-statistic for testing Ho : p = 0 against p > 0. Also, let Ln,a be the a-level critical value. Then the PTE of p is defined by

P,“(P~)

= P , ( P ~ )- ( P , ( P ~ ) - P

By the Durbin-Watson test, we can write the P T E of PT(D)

p,

( f i n )= P n ( P n ) I ( - 2

+p,qo

(10.5.5)

< Ln,a)*

n ) ~ ( f n

< fiw

< Dw

as

- ~ u ( a d)(&

-

en),

( 10.6.13)

respectively. See also Saleh (1992).

10.6.2

Asymptotic Theory of the Estimators and the Test-Statistics

Our main purpose is the comparison of the five estimators based on the asymptotic theory of the estimators and loss function

L(6:, 6 ) = n(6: - 6)'W(O:

- 6),

( 10.6.14)

where W is a positive-semidefinite weight matrix. It can be shown that C, is a consistent test, and for any fixed alternative H6 : H6 = h 6,all the estimators will be asymptotically risk equivalent t o 6, except that 6, tends to be degenerate as n -+ co. Thus, we resolve this issue by considering the class of local alternatives (as in Chapter 7) t o be

+

K(,] : H6 = h + n-1/2[,[ # 0. First, we note that the following theorem holds as n

( 10.6.15) -+

co under {K(,]}:

Theorem 1. Under {K(,)} and the assumed regularity conditions for the model (10.6.1), we have as n --$ co

[ '(1)

(i) lim PK(,,,{

B=

&(en

P(0)

-6)5

x} = a P ( x 0; , a:B),

P(1)

P(2)

.'. P ( P - 1) ... P ( P - 2 )

P ( P - 1) P ( P - 2 ) ... P(0) is the Toeplitz matrix associated with sp.d.f.;

( 10.6.16)


496

{&(in - 0) 5 XI = a P ( x , 6,o; , c,”V),

(ii) lim PK(,,

where V = B-’ - B-’H’(HB-lH’)-lHB-’ 6 =B-~H’(HB-~H’)-~~;

(10.6.17)

and

(iii) limPK(,)(C, 5 X) = HQ(x; A’), A’ = O;’~’(HB-’H’)-~~;(10.6.18) - 0) I x} = Hg(x;(cr);A2)@’p(x6;O;o;V)

(iv) h P K ( , l { & ( 6 y

+ J @,(x- Z; 0 ;a,”V)d@,(Z,0;u;(HB-lH’)-l),

+

(10.6.19)

+

where E ( t ) = {Z : (HZ t)’(HB-lH’)-’(HZ t ) 2 x:(cr)} and where Q P ( x0; ; E) stands for the pvariate normal cdf with mean 0 and covariance matrix E, and Hg(x; A’) is the cdf of a noncentral chi-square distribution with q-D.F. and noncentrality parameter A2/2.

-s

(v) the asymptotic representation of fi(6,

J;;;(’:

-

where Z

V

0) --t

N

0) given by

1

z - d { b , . ’ ( H z ~ ~ ) . ( H B - l H , ) - l ~ ~ ~ ~ ~ ) (10.6.20) B-’H’(HB-’H’)-’(Hz+

)

7

NP(O,0;B-l) and d = q - 2;

’

and (vi)

-

fi(e? -

{ o , ’ ( H z + t ) , ( H*8 - l H ’(HZ+ ) - l ~ H ~ + t )1 B- H‘(HB- H’)-

-

)

+ ’(HB-lH’)-l(HZ+ t)}-’ x I{(HZ + E)’(HB-’H’)-I(HZ + t )< d } x { B - ~ H ’ ( H B - ~ H ’ ) - ~ (+ HtZ) } ] . (10.6.21)

- [I - oi2d{(HZ

Proof.

P

(i) Since B;+B as n

3,

-+

03,

we can write

= B;-~P;(T) = B-’P;(T)

+ 0,(1).

( 10.6.22)

V

- O)=Np(O,a;B-l), being a PPMLE (see DzhaFurther as n -+ 03, paridze, 1986). Hence, we set Z = &(en - 6 ) = &B-’(~;(T) - P(T))~N,(O, a;B-’), as n

-+

00.

(10.6.23)

(ii) Restricted estimator

&(in

- 6)=

=

Asn-+m,

&{(an

- 6) - ~B;-lH’(HB~-lH’)-l(HB,- h)}

z - B-~H’(HB-’H’)-~(Hz+ 0), almost everywhere.


500

B. Q is nondecreasing and differentiable with the derivative Q‘being uniformly continuous on [O, 11. Then, for every 0 < k < 00, (10.7.5)

n

y=

i=j+l

1

f d 4 ( F ) and E = ( ( p ( i- j ) ) ) is the Toeplitz matrix with elements

P ( k ) = COV(X0, Xk), 1 I k I P.

(10.7.6)

The theorem above covers Wilcoxon-type scores but not the normal score. For a proof, see Koul and Saleh (1993) and Problem 14. As a consequence of Theorem 1, we can write

n1/2(6,- e) = yc-lL,(o) + op(i).

( 10.7.7)

Observe ,... that Ln is a vector of square integrable mean zero martingales with E[L,Lk] = u;E, where 0: = Var(Q(V)). Thus, by a routine Cramer-Wold device and Corollary 3.1 of Hall and Heyde (1980), we obtain

L,

-

NP(O,o p )

n1/2(6, - 0)

and

N

Np(0,~-2u$E-1).

(10.7.8)

The proof is left to the reader; see Problem 15.

10.7.2

Tests of Hypothesis and Improved &Estimators of e

Since our objective is to improve upon 6, when 6 belongs to subspace defined by HB = h, we need to test the null hypothesis H,-, : HO = h against the alternative H A : He # h. For this, we first consider the restricted R-estimators of 0 as

a, where C,

= ((&,ij))

= 8, - $nH’(HE,1H)-’(H6, - h),

(10.7.9)

consistent estimator of X with

n-rnax(t,j)

u^ n. t’ ~-

(Yk-i k=l

- %)(Yk-j

-

F)1

15

2,

j

5 72,

(10.7.10)

10.7. R-Estimation of the Parameters of the AR[p]-Models P

since E(V?)< 03 and the ergodic theorem implies ;e,-+X Now, we consider the test-statistic C,, defined by

c,

=

n&;~;(b,)k;'~,(ij,)l

(A) 6,)

as n

+ co.

(10.7.11)

6,

2

501

(&).

where 6; = n-l ($ , and = n-' 4 It can be shown (see Problem 16) that under Ho, C, has asymptotically the central chi-square distribution with q D.F. Let x:(a) be the upper 100a% critical value of the central chi-square distribution with q D.F. Then we define the preliminary test rank estimator (PTRE) as (10.7.12) The Stein-type rank estimator (SRE) is defined by

-s

-

8, = 6 ,

+ (1- C L ; ' ) ( ~ ,

- b,), c =

-

( 10.7.13 )

2,

and the positive-rule Stein-type rank estimator (PRSR.E)is defined by

ij:+

10.7.3

=

6, + (1 - CC,')I(L, > C ) ( B n

-

bn),

q

2 3.

(10.7.14)

Asymptotic Bias, MSE Matrix, and Risks of the R-Estimators

In order to obtain the expressions of the ADB, ADQB, ADMSE, and ADQR of the five R-estimators, we follow Saleh and Sen (1986), Sen and Saleh (1987), and Saleh (1992) and use the sequence of total alternatives. K(,) : H8 = h n-'/' c) -+ 0

in probability.

Hence, limn.-+mEF(C;l) = 0, and lirn,,,EF[C;'I(C, result,

> c ) ] = 0. As a

(11.2.17)

11.2. U-statistics Approach to Estimation

519

-

APT

-s

S+

in the first mean as n + 00, and we conclude that On, 8, , On, and 8, equivalent in quadratic mean under fixed alternatives, 0 # 0 . Hence,

,/G@~

,/K(G; (iii)

e) = &(en e) + op(i), e) = &(en - e) + Op(i),

-

-

are

-

,,%(en e) = ,,%(en - e) + op(l), . S+

(1 1.2.18)

-

and

(1 1.2.19) where

(1 1.2.20) and

< &,a)

g(L,) =

= (1 - C L Z I ) = CL,’

+ (1 - cL,’)I(L,

< c),

(1 1.2.21)

respectively. Also, the asymptotic distributional MSE and risks are

M(8:) = E

11.2.2

and

R(0:; Q) = t r ( E Q ) .

(1 1.2.22)

Asymptotic Properties of the Point and Set Estimation under Local Alternatives

As before, we consider a class of local alternatives

q,,: e(,) = R.- 1/26,

6 = (iil,. . . ,iip)’,

(11.2.23)

fixn-1/2

to obtain meaningful- differences in the asymptotic distributions of (6;e,,)), where 0: = 0, - 8,g(L,) as n + 00. Accordingly, consider the point I

APT

-s

AS+

estimators 8, ( a ) ,O,, and 0,

(c) under K(,) as follows:

520

Chapter 11. Multivariate Models

It is known that and

%n

--f

X a.s. as n . -1/2

J;E2,'/2en= &En V

-+

+

a~and

-

+ a,;

(6, - 6(,))

Z + 6' as n -+ 00, 6'

. -1/2

fix,

-

-

(6n-6(n)> Np(O,Ip)

(6; = 9 y 2 6 )

= Plim

a,;

Hence, we have the following theorem:

Theorem 2. Under {K(,)} and the assumed regularity conditions

11.2. U-statistics Approach to Estimation (i) Unrestricted estimator b,(&) = 0 and Bl(8,) = 0, M1(8,) = E and RI(8, : Q) = tr(QE). (ii) PTE

521

(11.2.28a)


522

These results may be obtained by using the random variable

where Z

11.3

- Np(O,Ip)

and g ( . ) is defined by (11.2.21).

Nonparametric Methods: R-estimation

Consider the model Y j = 6 + ~ j ,j = 1 ,

. . . , N,

(11.3.1)

where Y1,.. . ,Y Nare i.i.d. r.v. having pvariate continuous cdf, FO(Y), defined on the Euclidean space R p that is diagonally symmetric about its location parameter, 6 = (61,.. . , BP)’,

F e ( Y )= F ( Y

-

6 ) , Y E RP,

(11.3.2)

where F is diagonally symmetric about 0. Let F[jl be the marginal cdf corresponding to F and F [ i , j ]i, # j be the bivariate marginal cdf corresponding to F . Further, let Frjl possess absolute continuous pdf frjl, and let

In addition, for every N ( 2 1) and j (= 1,. . . ,p)’ define N scores

aftij(h) = E[4T(UNk)] or 4;

(-)N k+ l

for IC = I,.. . ,N,

(11.3.4)

where UNk is the kth order statistic from sample size N from U ( 0 , l ) and for every u E (0, 1),consider

(11.3.5) where the functions { & ~ ( z L ) } are square integrable and nondecreasing. To define the rank statistics, let Rf(b) be the rank of l y Z j - bl among /Y&b(.. . . , 1 Y ~ j bl, for i = 1,. . . ,N and j = 1,. . . , p , and consider the vector of LRS

where

11.3. Nonparametric i'vfethods: R-estimation

523

Notice that T N( b~) \ in b (see Puri and Sen, 1986), and define the unrestricted estimator of 6, as

1 & = ~ ( s u p ( b : T ~ ~>O}+inf{b;T~,(b) c)(Z + s')112 < X2,(T)}

= Hp(c; A

2 ) W < xg(Y))

+Q*{ jlZ - c(Z + 6*jl-2112 < x;4(r);jjz + 6*jj2 > c},

(11.3.45)

which is the same as (4.8.32) in Theorem 4.8.7. The analysis of the confidence sets is the same as in Chapter 4 and therefore is not repeated.

11.4 Simple Multivariate Linear Regression Model In this section, we consider a simple multivariate linear regression model that is an extension of the simple linear model of Chapter 3 in a multivariate setup. In this model, one sample {(XI,Y I ) .,. . ,(ZN, Y N ) }of size N is drawn from the pvariate normal distributions {Np(8 Pz,, X ) I a = 1 , 2 , . - . ,N } , where 8 = (61,. . . ,6,)' and ,B = (PI,. . . ,Pp)' are the intercept and slope vectors and X is the unknown covariance matrix. The main object of this section is the study of the properties of several estimators of 8. As a special case, we study the two-sample problem involving two pvariate normal distributions. For more details, the reader is referred t o Ahmed and Saleh (1990, 1999) and Sen and Saleh (1979).

+

11.4.1 Model, Estimation and Tests Model. Let Y,

=8

+ pz, + E,,

Q

= 1,2,'. . , N ,

(11.4.1)

where Y , = (Y,l,-.- ,Yap)' is the response pvector, z, is a scalar, E = (€,I, '. . ,cap)' is the pvector of errors, 8 = (61,.. , 6,)' is the intercept vector, and P = (PI,... ,&)' is the slope-vector of the model. We assume E, Np(O,E). Our problem is to estimate 8 when it is suspected that p may be null, meaning, 0 without loss of generality.

-

Estimation. Using LSE/MLE methods, we obtain the unrestricted estimators of p and 8 as

11.4. Simple Multivariate Linear Regression Model

h(1"~)~

531

with x = ( X I , - .,.X N ) ' and Z N = 51"~. If where Q N = x'x p = 0, then the restricted estimator of 8 is given by -

8N = Y N =

-1k N1

[ ' 1.

(11.4.3)

Y N

Test of Hypothesis. For testing the null-hypothesis Ho : p = 0 against H A : # 0 , we use the likelihood ratio statistic given by (11.4.4) where N

Sa = (N-2)Se = C((Ya-Y~)-PN(z, -ZN)}{

(Y,- V N ) - ~ Z ~ N ( Z , - Z N ) } ' .

ff=l

(11.4.5) It can be shown that the likelihood ratio statistic is equivalent to the Hotelling's T2-statistic, given by

&-G-';

(11.4.6)

(N -21-1~; = QN&S;'P~

T$ as the test-statistic equivalent to 5";. We define C N = Under H A , C N follows the noncentral F-distribution with (p,m) d.f. ( m= N - p - 1) and noncentrality parameter A2/2 with A2 = &~p'I=-'p.

(11.4.7)

11.4.2 Preliminary Test and Stein-Type Estimators Let F p , m ( ~be) the cr-level critical value of the CN-statistic under Ho. Then we define the preliminary test and the Stein-type estimators of p as follows: A

PT

PN

=P N -PNwN

6; = P N and

-s+

PN

-

d P N c i1i

-s

=p N -pN(1

(p - 2 ) L > 4 + (1 - (p - 2)L;V41(&*, < (p - 2))

d q v ) = 0,

1

and L*, = Q N @ ~ C - ' ~ Nwhere , E is known. Similarly, the confidence sets based on the five estimators of 6 can be written as

where

e;

= 8,

+ (8,

-e N > g ( ~ k ) .

The coverage probabilities may be calculated as in Chapter 5 for the parallelism problem.

11.5. R-estimation and Confidence Sets for Simple Multivariate hdodel 539

11.5

R-estimation and Confidence Sets for the Simple Multivariate Linear Model

11.5.1

Introduction

Let {X, = ( X , l , . . . , XZp)’, i 2 l} be a sequence of independent random vectors with p ( 2 1)-variant distribution functions {F?, z 2 l}, where for every z 2 1,

F,(Y)= F(Y

-6 -

OX,),Y E RP . . ,Pp)’ are the unknown

(11.5.1)

where 6 = (81,. . . ,Op)’ and p = (/?I,. intercept and slope parameters, respectively, and z,(z 2 1) are sequences of known constants. As in Section 11.14, we are interested in the R-estimation of 6 and p when it is suspected that /3 may be null vector ( 6 ) based on the rank method. Let R,,(a,b) and R t ( a ,b) be the rank of yZ, - a - bx, (or /l’iJ - a - bz,l) among Y1, - a - bz,, . . . ,Yn, - a - bx, (or IY1, - a - bzl 1,. . . , lYnl - a - bz,) for z = 1 , . . . ,n; J = 1 , . . . , p , where a , b are real numbers. Then define the LRSs: Tn(a,b)= ( T n l ( a l , b l ) , . .

,Tnp(ap,bp))’

and Ln(b) = (Lnl(h), . .. , L n p ( b p ) ) ’ , (11.5.2)

where n

T ~ , ( u , ~=) n-’ C ~ X ( ~ : ( a , b ) s i g n ( ~ ,-, a - bz,) 2=1

n

and

Lnj(b) = n-l

C(xz - zn)an(RzJ(b)),

( 11.5.3)

i=l

respectively, with scores defined by a,’ (k)= E#j(Unk)

(11.5.4)

a n ( k ) = E#:(Unk)

for i = 1,. . . , n and u n k is the order statistic with rank k among n independent samples from U(0,l). The score functions #j = {#j(u);0 < ZL < 1) is absolutely continuous, nondecreasing skew-symmetric, meaning q 5 j ( ~ ~+ ) #j(l - Z L ) = 0 V ZL E ( 0 , l ) and are square-integrable inside (0,l) whereas = #j u E (0, l), j = 1,... , p . Assume that

#T(ZL)

(y),

c(zi n

Qn =

- Z,)

and

i=l

lim Z n = 3: (121 < m)

n-ca

lim n-’Q, = Q’

,-a,

(11.5.5)

540


both exist. F E 3,where 3 is the class of all pvariate continuous distribution functions that are diagonally symmetric about 0 having a finite Fisher information matrix. (11.5.6) Note that Ti,j(a,b ) \ a for fixed b and Lnj(a,b) \ in b independent of a for every ;I = 1,. . . ,p . Also, under HO : 8 = P = 0 , [TL(O,0 ) ,Lk(O)]’is symmetrically distributed around ( 0 , O ) . Hence, as in Sen and Saleh (1979), P’)’ as follows: define the unrestricted estimators of (O’, and

a n = ( & , . - . ,ap)’

Under Ho :

P n = ( P 1 , . . * ,Pp>’t

= 0, the restricted estimator of B is given by

en = ( 6 1 , . . . ,ep)’,

+

(11.5.7)

where 6, = ${ sup[a, : Tn,(a,;O)> 01 inf[a, : Tn,(a,;O)< 01) since the restricted estimator of is 0 . Then enis a translation-invariant, robust, and consistent estimator of 8 when P = 0, while as well as are similar types of R-estimators when P # 0. In order to define the rest of the modified R-estimators, we consider first the test of hypothesis HO : P = 0 based on the rank-test Cn given by

an

Ln

=

an

[nLL(O)M,Ln(o)]Qn,

(11.5.8)

where

Mn = ((m,e)), m,e = n

c,=l

-1

n

C[an,(Rz,) - ajl[ane(&e)

-

at],

(11.5.9)

2=1

n

an,(i); j , L! = 1 , . . . , p , and M, is the generalized inverse with ii, = n-l of M,. Now, conditionally, C, is a distribution-free statistic, and for large n, Cn is approximately distributed as a central chi-square with p d.f. when M, is of full rank. Let Ln,a be a upper a-level critical value from the distribution of C,, and then Ln,@-+ $ ( a ) as n -+ co. Based on this information, we define the three estimators of P given by ,. PT

Pn

-s

0,

= P n - PnI(Ln < C n p )

=

Pn - CP,L,~I(C~> E ~ ) ,

E,

-+

o as n -+

co; c = p - 2, (11.5.10)

Similarly, we have four additional estimators of 8 as follows:

11.5. R-estimation and Confidence Sets for Simple Multivariate Model 541

6 , the restricted estimator of 8 when P = 0. . PT

= 8, -

8,

-s -

(en e , ) q ~ ,< L,,J -

8, = 8, - c(6, - 6,)L;'I(Ln

and -s+ 8, = 8, n

2 E,),

E,

-+ 0 as n --f

00

and c = ( p - 2),

+ (1 - C L ; ~ ) I ( . C>, c)(en-&I,

(11.5.11)

respectively.

11.5.2

Asymptotic Properties of the R-estimators

Let 4,l and Fjel be the marginal and the joint-distribution of the j t h and (j,!)th variable corresponding to F in (11.5.1) as described in Section 11.3.1 and consider the matrices defined by A and E as in (11.3.9). We assume that they are of full rank and M, is a consistent estimator of E. Then, one may show that under fixed alternative P # 0, the following results hold

JnM,- l / 2 ( j I n &M;'/'(fIZ and

PT

-P) =

-P) =

fiM,1/2(/?lZ+

J;EE-'/2(p, - P ) + o p ( l ) ,

@-1/2(p,

-P) =

-

P ) + op(l),

&E-1/2(pn - 0) + op(1).

(11.5.12)

Thus, we consider the class of contiguous alternatives

K(,) : P(,) = n- 12 6 , 6 = (61,. . . ,bp)'.

(11.5.13)

Now, from the basic theorems of Chapter 8 (Puri and, Sen 1985), it follows that under 8 = P = 0 and (11.5.2)

(11.5.14) Further, from theorems of Chapter 6 (Puri and Sen, 1985), we have the linearity results, where k (0 < k < 00):

+

l~,,(n-1/2(a,,b , ) ) - T,,(o,o) n'/2(a, sup < k2

la31

Ib,/

+ b , ~ ) y , j50 as n

+

m,

< kl

(11.5.15)

so

f' where y, = 1 +J(u)4,(zi)duand +,(a) = -LLfor j = 1 , . . . , p . Then we fb1 ( u ) have the following theorems based on (11.5.14) and (11.5.15).

542


Theorem 1. Under the assumed regularity conditions given in (11.5.5) through (11.5.15), as n + 00, we have

and when p

= 0,

h(6, - 6 ) 5 Np(O,E).

( 11.5.16b)

Theorem 2. Under {K(,)} and assumed regularity conditions, as n

r*Q*

( % 0. Hence, Rz(6,; Q)
Hence, the risk-difference above is nonnegative whenever PT

-S

s.

(12.1.56)

Chapter 12. Discrete Data Models

562

Since (12.2.53b) is equivalent to the density inequality

2 2 2hp+1(xp-1(4;0) 5 -+ a , P L 4 1 P-1 *

PT

we note that for a range of C Y , 6 , as given below:

A

s+

dominates 6 ,

(12.1.57)

and satisfies the ordering

. S+

. PT

Rz(6,; Q) i R3(6, ; Q) I R5(6, ; Q) I R4(6:; Q) I Ri(6,; Q).

(12.1.58)

. S+ -s The next theorem gives the dominance of 6 , and 8, over

6,

for all A’.

Theorem 4. Let p 2 4 and let the matrix Q in the loss function, L(6:; 6 ) = n(6: - 8)’Q(6: - 6)be positive-semi-definite, satisfying (12.1.59) -Sf

Then, under {K(,]}, the ADQR of 6 ,

,6- ,s and 6,

R s ( e + )I R4(6:; Q) I

&(a,;

can be ordered as

Q)

(12.1.60)

uniformly in A’. -S

Proof. The ADQR expression for 6 , can be written as -S

R4(6,;

Q) = tr[QXo] - ( P - 3)tr[QEoJ’]{

( p - ~)E[X;:~(A’)]

-S

The risk-difference R1 (8,; Q) - R4(6,; Q) is nonnegative for all Q , satisfyiilg (12.1.59), and for all A’. Hence, R4(62; Q) I & ( e n ; Q) b’ (A’, Q).

(12.1.62)

Similarly, the risk-difference AS+.

R5(8, =

1

Q) -

&(a:;

Q)

-trlQxoJ’lE[(1 - ( P - ~ ) X ~ ~ ~ ( A ’ ) ) ~ ~ ( < X p~ -+ 3)] ~(A’) f d’Qd{2E[(1-

-

( P - ~ ) X ~ ~ ~ ( A ’ ) ) I ( X ~X,-t23(A2))21(x~+3(A2) ),

then 9: = 9, = 9,- (&, - 6 , ) l ( D n < x ~ - ~ ( Q ) ) . ( 12.2.14~) *

PT

-s (iv) If g(Dn) = dD;l, then 9: = Q, = Q, - d(\k, - 80,)D;l,

d = Ic - 3.

(v) If g(D,) = 1 - (1 - d D ; l ) I ( D , > d ) ,

+ (9,- @on)(l - dD;l)I(D, > d ) , -s = 9, - (9, - 90,)(1- ~ D ; ' ) I ( D , < d ) .

then Q: = 8,

(12.2.144

or

( 12.2.14e)

*:

12.2.3 Asymptotic Theory of the Estimators and the Test-Statistics Note that &, and 80,are consistent estimators of Q and 90= ( $ 0 , . . ,&)', respectively. Similarly, since D, is a consistent test-statistic, under fixed a1t erna tives , 9 6 = $Olk

we obtain

D , = n4;2($~- 9 o n ) ' f i ,

..-I

(9- 90,)

Consequently, for fixed alternatives

\E6

for all

5

--+ m

= $elk

P96{D,>z)-+1

( 12.2.15)

f6 ,

as n

+ m.

+ 6 , we have ( 12.2.17)

as n - + m

2 0. Further, for fixed alternatives,

( 12.2.16)

9 6 .

n4O;a2Il(8Y-*,)1I2= D J ( D n <XE-I(Q)) fin

I x2-&)I(Dn

<xE-&)).

Then, taking expectation on both sides, we have

P[D, < X ~ - ~ ( C Y+ ) ] 0 as n

m by (12.2.17)

(12.2.18a)

E I D i l ] + 0 as n -+ co,

(12.2.1813)

-+

Similarly,

n$G:lI&E - *,/I&

= d 2 D i 1 and

and

n?ZO-,2 II 9,7 .2s+ -

= d2D,11(Dn

= d2D,1

> d ) + D,'I(D,

+ (1 - d2)D,11(Dn

> d) < d).

(12.2.18~)

The r.h.s. of (12.2.18a,18b) tends to 0 in the first mean as n + 03. Hence, under fixed alternatives * 6 , the asymptotic distribution of the estimators are equivalent t o that of &,. Further,

12.2. Product Binomial Distributions (90, -~

P

) - = q (&lk

573

- S ) as n

so that n&2l1&on - \kilh,

+ 0;)

---f

(12.2.19a)

0;)

as n -+ m.

(12.2.19b)

Also,

E[n(Gn- \k)(Gn- S)’]= +in-’, S2 = Diag(w1, ... which is bounded,

n7j;2(Gn - \k)R(Gn - \k) < 53

(12.2.21)

as n + 0;).Thus, we have the following theorem:

Theorem 1. Under fixed alternatives (i)

4onlk APT

S 6

= &lk

+6, ( 12.2.22)

has unbounded risk, -s

s+

(ii) S n , \kn and 9, are all asymptotically equivalent to tion and have bounded risks.

Gn in distribu-

In order to compare the five estimators, we consider the asymptotic setup along with the quadratic loss function

L ( @ ; ;S ) =

n(&i- S)’Q(@Z- \k),

(12.2.23)

where 9;is any of the estimators for different choices of the g-function given a t (12.2.13) and Q is a positive-semidefinite (k x k) matrix. Then the asymptotic risk of @: is given by

R(\k: : Q) = tr(QVG),

( 12.2.24)

where Vz is the asymptotic distributional dispersion matrix of &(S; - S ) . Further, \k: will be called an asymptotically inadmissible estimator of S if there exists an alternative estimator S: such that

R ( 9 : : Q) > R(@E: Q ) VA2,

(12.2.25)

where A2 is the departure function of the parameters of the models from the null hypothesis, with strict inequality for some A2. The problem is the computation of the asymptotic distributional quadratic risks (ADQRs) of estimators, \k;. To enable such a computation, we consider a sequence {K(,)) of local alternatives, namely

K(,) : \k(n)= $ O l k

+ Y$ trIQn-'IE[l-

g(x:+i (A2))121

which is a increasing function of E[1 - ~ ( x E + ~ ( A ~Hence, ) ) ] ~ .we get

and the assertion of the theorem holds. The next theorem deals with dominance of the Stein-type estimators. Theorem 4. Let k > 4 and the matrix Q satisfy the condition

( 12.2.36)

-s

- s+ outperform G , with respect

Then under {K(,l}, the estimators, Q, and Q, to ADQR and

R E , ( G ~Q) + ;I &($:;

Q) I Ri(*,; Q).

(12.2.37)


578 Proof. R4(&:

The ADQR of

&:

can be written as

: Q ) = $: tr[Qn-'] - ( k - 3)$:w-l

tr[Qn-'J']{ ( k - ~ ) E [ x ~ : ~ ( A ~ ) ] (12.2.38)

The risk-difference Rl(G,; Q) - Rq(4,; Q) (12.2.36). Hence,

2 0 for all (A2,Q), Q satisfying

R,(e:; Q) i RI(*,; Q) v(A2,Q).

(12.2.39)

Similarly, the risk-difference R5(*:+;

=

Q) - R4(&:: Q)

-& tr[Qn-'J']E[(l -a'Qa{W(1 - (k

- (k -3)~i$(A'))~1(xg+~(A < ~k )- 3)]

-

3 ) x i ~ l ( A 2 ) ) I ( x i + 1 ( A 2 3. If, in particular, W = I,, or J, then ChmaX(W)= 1, and the rank of I,,A is m and the rank of JA 2 m- 1. In this


598

case, W E W . The dominance may not hold for arbitrary choice of W because the condition (12.3.59) may not hold simultaneously for all A (i.e., K ) , and -S

in such cases, the Stein-type estimator 8, works with some modification due to Berger et al. (1977) as . MS+

8,

=

6, + (1 - cdnD,lw-lE:,1}(6,

- 6,),

(12.3.61)

where c is the shrinkage constant and d, = C I L ~ ~ , ( W - ~ / ~ E ~ ~ ) .

As for the Stein-type estimator, en,it has smaller risk than 8, uniformly for all A2 by (12.3.59). However, for the comparison of risk of the restricted -S estimator en with that of 8,, we first note that -S

+ tr[WA]

R4(6: : W) = &(en : W)

-

(S‘J*’WJ*d)

- (m - 2) tr[WAl{(m - 2)E[X;:2(A2)j

1

+

+

[l-

(m 2)(S*J*’WJ*S) ~A’E[X;:~ (A2)]. (12.3.62) 2(S’J*S) t r [WA]

Under Ho, (12.3.62) becomes

R4(6: : W) = R2(6,, : W) + - tr[WA] L 0. 2 m

-S

Hence, asymptotic risk of 8, is greater than that of

6,

(12.3.63)

under Ho. However, -s

if A2 diverts from 0, the risk of 6 , grows bigger while risk of 8, becomes -S

6,

smaller and 8, dominates -S

neither 8, nor

6,

outside an interval around the origin 0. Thus,

dominate each other completely.

-5-

c.

PT

In the case of 8, and 8,

we have under Ho,

..PT

-s

+

R 4 ( 8 , : W) = Rs(8, : W) tr[WA]{l - H,+z(x:(a) 2 . PT + -tr[WA] 2 R3(8, : W ) V m 2 3. m

: 0))

(12.3.64)

The PTE has smaller ADQR than that of 6; at A’ = 0, and 6: does not - PT dominate 8, when independence hypothesis holds. Clearly, under Ho the risk ordering is as follows:

R2(6, : W) 5

: W)

5 &(6:

: W)

5 R1(8, : W).

(12.3.65)

The picture changes as soon as A2 moves away from 0, namely (1) the risk of 6 , remains constant, (2) the risk of 6 , becomes unbounded, (3) the risk . PT of 8, grows t o a maximum and then drop toward the risk of 6,, and (4)

599

12.4. Conclusions -S

the risk of 8, merges to the risk of 6 , as A' -+ 00 from below. It may be -S ,. PT calculated that the risk of 8, is greater than that of 8, whenever 2hrn+z(x;(a)

: 0)

2

I+ a, m

(12.3.66)

where hm+z(-;0) is the pdf of Hrn+2(.: 0) with (m + 2) d.f. and Q is the level of the preliminary test of significance. As soon as (12.3.66) is violated, the ordering of the risks becomes

~ ' ( 6 , w) I ~ ~ ( :6w) : 5 R3(e, : w). *

PT

-s

(12.3.67)

,.PT

Finally, we conclude that for m 2 3, 8, is a prefered estimator than 8, , PT while for m 5 2, 8, is preferable.

-s+ -s

As regards, 8, , 8 , and

a,, the asymptotic risks may be ordered as follows:

Rs(h:+ : W) 5 R4(6: : W) 5 Rl(6, : W) 'd A'. Hence, among the five estimators of 8,6;'

12.4

(12.3.68)

is the most preferred for m 2 3.

Conclusions

In this chapter, we considered three discrete models related to the baseball data analysis, meta-analysis and T x c contingency tables. Asymptotic properties of the point estimators were studied, and the usual conclusions on the dominance properties of the estimators were arrived at.

12.5 Problems 1. (Refer to Section 12.1.2). Verify that the joint density of (8',Y')'given (K177)is

2. (Continuation.) Verify that the marginal distribution of Y given ( K ,77) is

3. (Continuation.) Verify that the posterior distribution of 8 given (Y',K)' is

cl'

.(e,Ylh',77)ml(YI~,71)~7-]( 77 1)1-1d77,

where C is the constant of the integration.


600 4. Verify, based on Problem 3, that

and

=E

+ Var[E(OzIY,77)1

[ss)+

(b) Var(0,IY) = E[Var(4IY, q)l

(1 -

(n, K - 1)-1]

+ V a rn,+K [ m

I

.

5. Show that as K -+ co,the posterior distributioii of 77 is approximately a beta distribution with parameters ng and n(1 - jj). 6. Verify that limK,, E(0,IY) = IimK,, G(1-G). limK,, Var(qlY) = n+l

E(q1Y) = jj and limK,,

Var(0,IY) =

7. Verify that limK-o E(0,IY) = limK-o E(q1Y) = g and 1imK-o Var(0,jY) = limK-0 Var(7jY) = G(1-8). n+l

8. Show that the empirical Bayes estimators of 8 can be written as

where g(Dn) = min(1, ( p - 3)D;').

9. Prove Theorem 12.1.2. 10. Refer t o Section 12.1.4. Verify the expressions for ADB, ADQB, ADMSE, and ADQR from (i) through (v). 11. Prove that the asymptotic variance of the three estimators given at (12.2.8) iS $:W-l, W = W1 f * . ' + wk.

12. Prove Theorem 12.2.2. 13. Refer to Section 12.2.4. Verify the expressions for ADB, ADQB, ADMSE, and ADQR from (i) through (v).

14. Refer t o Section 12.1.7. Verify the expressions for the asymptotic coverage probabilities of C*(8:).

15. Show that M ( n l K ) N Crm/2exp {

-

i G i } as n + co,where

See formula a t (12.3.33).

16. Refer to (12.3.27b). Show that X(K) 17. Prove Theorem 12.3.2.

(1-dDZ1)I(D, > d) as n -+ ca.

References 1. Adichie, J. N. (1967). Estimates of regression parameters based on ranks. An. Math. Statist. 38:894-904. 2. Agresti, A. (1990). Categorical Data Analyszs. Wiley, New York. 3. Ahmed, S. E., and Saleh, A. K. Md. E. (1988). Estimation strategy using a preliminary test in some normal models. Soochow J. of Math. 14:135165. 4. Ahmed, S. E., and Saleh, A. K. Md. E. (1990). Estimation strategies for intercept vector in a simple multivariate regression model. Comput. Statist. Data Anal., 10:193-206. 5. Ahmed, S. E., and Saleh, A. K. Md. E. (1993). Improved estimation for the component mean vector. J. of the Jpn. Statist. SOC.43:177-195. 6. Ahmed, S. E., and Saleh, A. K. Md. E. (1999). Improved nonparametric estimation of location vectors in multivariate regression models. J. Nonparumetric Statist. 11:51-78. 7. Ahsanullah, M. (1971), On the estimation of means in a bivariate normal distribution with equal marginal variances. Biometrika 58:23@233. 8. Ahsanullah, M., and Saleh, A. K. Md. E. (1972), Estimation of intercept in a linear regression model with one dependent variable after a preliminary test on the regression coefficient, Int. Statist. Rev. 40:139-145. 9. Akritas, M. G., and Johnson, R. A. (1982a). Efficiencies of tests and estimators in autoregression under onnormal error distribution. An. Znst. Math. Statist, A34:579-589. 10. Akritas, hl. G., and Johnson, R. A. (1982b). Asyptotic inference in continuous time, diffusions and Gaussian processes with known covariance. J. Mult. Anal. 12:123-135. 11. Akritas, M., Saleh, A. K. Md. E. and Sen, P. K. (1985). Nonparametric estimation of intercepts after a preliminary test on parallelism of several regression lines. Baostatistics: Statistics in Bzomedical, Public Health and Environmental Sciences Ed. P. K. Sen, Elsevier Science, North-Holland, pp. 221-235. 12. Albert, J. H. (1984). Empirical Bayes estimation of a set of binomial probabilities. J. Statist. Comput. Szmulation 20:129-144. 13. Albert, J. H. (1987). Empirical Bayes estimation in contingency tables. Commun. Statist.-Theory Meth., 16:2459-2485. 14. Albert, J.H., and Gupta, A.K. (1981). Bayesian methods for binomial data with applications to a nonresponse problem. Technical report, Department of Mathematics and Statist., Bowling Green State University.

601

602

References

15. Albert, J. H., and Gupta, A. K. (1982). Mixture of Dirichlet distributions and estimation in contingency tables. A n . Statist. 10:1260-1268. 16. Albert, J. H., and Gupta, A. K. (1983a). Bayes estimation methods for 2 x 2 contingency tables using mixtures of Dirichlet distributions. J. Amer. Statist. ASSOC.,78:708-717. 17. Albert, J. H., and Gupta, A. K. (198313). Estimation in contingency tables using prior information. J . Roy. Statist. SOC.B 45:60-69. 18. Ali, M. Abdunnabi. (1990). Interface of preliminary test approach and empirical Bayes approach to shrinkage estimation. Ph.D. thesis. Carleton University, Ottawa, Canada. 19. Ali, M. Abdunnabi, and Saleh, A. K. Md. E. (1991). Estimation of means and treatment effects in a one-way ANOVA model. Soochow J. of Math. 17:287309. 20. Ali, hl. Abdunnabi, and Saleh, A. K. Md. E. (1991a). Asymptotic Theory for Simultaneous Estimation of Binomial Means. Statist. Sinica 1:271-294. 21. Ali, A.M., and Saleh, A.K.Md.E. (1991b). Preliminary test and empirical Bayes approach to shrinkage estimation of regression parameters. J. Jpn. Statist. SOC.,21(1):401-416. 22. Anderson, T. W. (1984). Introductzon to Multivariate Analysis. Wiley, New York. 23. Anderson, D. R., Sweeney, D. J., and Williams, T. A. (1993). Statistics for Business and Economica, 5th ed., West Publishing, Boulder, CO. 24. Arnold, Steven F. (1981). The Theory of Linear Models and Multivariate Analysis. Wiley, New York. 25. Asano, C., and Sato, S. (1962). A Bivariate Analogue of Pooling Data. Bull, Math. Statzst. 10:39-59. 26. Baksalary, J. K., and Kala, R. (1983). Partial ordering between matrices one of which is of rank one. Bull. Polish Acad. of Sci., Math., 31:5-7. 27. Bancroft, T. A. (1944). On biases in estimation due to the use of preliminary tests of significance. An. Math. Statist., 15:190-204. 28. Bancroft, T. A. (1964). Analysis and inference for incompletely specified models involving the use of preliminary test(s) of significance. Biometn’cs 20:427-442. 29. Bancroft, T. A. (1965). Inference for incompletely specified models in the physical sciences (with discussion). Bull. ISI, Proc. 35th Session, 41(1):497-515. 30. Bancroft, T.A., and Han, C.-P. (1977). Inference based on conditional specification: A note and a bibliography. ISI Rev. 45:117-127. 31. Baranchik, A. (1970). A family of minimax estimators of the mean of a multivariate normal distribution. An. Math. Stat. 41542-645. 32. Bends, N. (1996). Pre-test estimation and design in linear model. J. Statist. Plan and Infer. 52:225-240. 33. Bennett, B. M. (1952). Estimation of means on the basis of preliminary tests of significance. An. Inst. Stat. Math. 4 31-43. 34. Bennett, B.M. (1956). On the use of preliminary tests in certain statistical procedures, An. Inst. Stat. Math. 8:45-57. 35. Berenblutt, I. I., and Webb, G. I. (1973). A new test for autocorrelated errors in the linear regression model. J. Roy. Statist. SOC.B35:33-50.

R,eferences

603

36. Berger, J. 0. (1976). Admissible minimax estimation of a multivariate normal mean with arbitrary quadratic loss. An. Statist. 4:223-226. 37. Berger, J. 0. (19804. Statistical Decision Theory. Springer-Verlag, New York. 38. Berger, 3 . 0 . (1980b). A robust generlized Bayes estimator and confidence region for a multivariate normal mean. An. Statist. 8:716-761. 39. Berger, J. 0. (1985). Statistical Decision Theory and Bayesian Analysis, (2nd edition). Springer-Verlag, New York. 40. Berger, J.O., Bock, M E . , Brown, L.D., Casella, G., and Gleser, L. (1977). Minimax estimation of a normal mean vector for arbitrary quadratic loss and unknown covariance matrix. An. Statist. 5:736771. 41. Berry, J. C. (1994). Improving the James-Stein estimator using the Stein variance estimator. Statist. Prob. Lett. 20:241-245. 42. Bickel, P. J., and Doksum, K. A. (2001). Mathematical Statistzstics: Basic Ideas, Vol. 1. Holden-Day, Oakland, CA 43. Bishop, Y. M. M., Fienberg, S. E., and Holland, P. W. (1975). Discrete Multivariate Analysis. MIT Press, Cambridge. 44. Bock, M. E. (1988). Shrinkage estimators: pseudo-Bayes rules for normal mean vectors, Vol 1. Proc. Fourth Purdue Symp. Stat. Dec. Theo. Rel. Topics, (eds. S. S . Gupta and J. Berger) Springer Verlag, New York. 45. Bock, M. E., Yancey, T. A., and Judge, G. G. (1973). The statistical consequence of preliminary test estimators in regression. J. Amer. Statist. Assoc. 68:109-1 16. 46. Bolfaine, H., and Zacks, S. (1992). Prediction Theory f o r Finite Population. Springer Verlag, New York. 47. Bozivich, H., Bancroft, T.A., and Hartley, H.O. (1956). Power of analysis of variance test procedure for certain incompletely specified models. An. Math. Statist. 27:1037-1043. 48. Brewster, J.F., and Zidek, J. V. (1974). Improving on equivariant estimators. An. Statist. 2:21-38. 49. Brockwell, P. J., and Davis R.A. (1996). Introduction to Times Series and Forecasting. Spinger, New York. 50. Brown, L.D. (1966). On the admissibility of invariant estimators of one or more location parameters. An. Math. Statist. 37:1087-1136. 51. Brown, L. D. (1988). The differential inequality of a statistical estimation problem. Proc. Fourth Purdue Symp. Stat. Dec. Theo. Rel. Topics, Vol 1 (eds. S.S. Gupta and J. Berger). Springer Verlag, New York. 52. Casella, G. (1985). An introduction to empirical Bayes data analysis. J. Amer. Statist. Assoc. 39 (2):83-87. 53. Casella, G., and Berger, R.L. (1990). Statist. inference. Duxbury Press. Belmount, California. 54. Casella, G., and Hwang, J. T. (1983). Empirical Bayes confidence sets for the mean of a multivariate normal distribution, J. of Amer. Statist. Assoc., 78:688698. 55. Casella, G., Hwang, J.T. (1986). Confidence sets and the Stein-effect, Com. Stat. Theo. Meth. 15:2043-2063. 56. Casella, G., and Hwang, J.T. (1987). Employing vague prior information in the construction of confidence sets. J . Mult. Anal., 21:7%104.

604

R,eferences

57. Cellier, D., Fourdrinier, D., and Robert, C. (1989). Robust shrinkage estimators of the location parameter for elliptically symmetrical distributions. J . Mult. Anal. 29:39-52. 58. Chen, J., and Hwang, J. T. (1988). Improved set estimators for the coefficients of a linear model when the error distribution is spherically symmetric with unknown variances. Can. J. Statist. 16:293-299. 59. Chen E.J., and Saleh, A. K. Md. E. (1993). Estimation of regression parameters when the errors are autocorrelated. Proc. 3rd Pacific Area Stat. Conference Stat. Sci. and Data Analysis. VSP Utrect, The Netherlands (eds. Matrusita, Puri and Hawakawa) 61-76. 60. Chiou, P. C., and Saleh, A. K. Md. E. (2002). Preliminary test confidence sets for the mean of a multivariate normal distribution. J. Prop. Prob. Statist. 2: 177-189. 61. Cochrane, D., and Orcutt, G. H. (1949). Application of least squares regression to relationships containing autocorrelated error terms. J. Amer. Statist. Assoc., 44:32-61. 62. Cohen, A. (1965). Estimates of linear combinations of the parameters in the mean vector of a multivariate distribution. An. Math. Statist. 36:78-87. 63. Cohen, A,, and Strawderman, W. (1973). Admissibility implications for different criteria in confidence estimation. An. Stat. 1:363-366. 64. Cramer, H. (1946). Mathematical Methods of Statzstics. Princeton University Press, Princeton. 65. Deely, J., and Lindley, D. (1981). Bayes empirical Bayes. J. Amer. Statist. ASSOC.76:833-841. 66. Dempster, A.P., Schatzoff, M., and Wermuth, N. (1977). A simulation study of alternatives to ordinary least squares. J. Amer. Statist. Assoc. 72:77-91. 67. Doob, J.L. (1953). Stochastic Processes. Wiley, New York. 68. Durbin, J. (1960). Estimation of parameters in time series regression models. J . Roy. Statist. SOC.B22:139-153. 69. Durbin, J., and Watson, G.S. (1951). Testing for serial correlation in least squares regression. I. Biometrzka 37409-428. 70. Durbin, J., and Watson, G.S. (1950). Testing for serial correlation in least squares regression. 11. Biornetrika 38:159-177. 71. Dzhaparidze, K. (1986). Parameter Estimation and Hypothesis Testing in Spectral Analysis of Stationary Time Series. Springer Verlag, New York. 72. Efron, B. (1975). Bias versus unbiased estimation. Adv. Math. 16:259-277. The Statistical Century. RSS News 22(5):1-2. 73. Efron, B., and Morris, C. (1972). Limiting the risk of Bayes and empirical Bayes estimators - Part 11: The empirical Bayes case. J . Amer. Statist. Assoc. 67:130-139. 74. Efron, B., and Morris, C. (1973). Stein’s estimation rule and its competitors An empirical Bayes approach. J. Amer. Statist. Assoc., 68:117-130. 75. Efron, B., and Morris, C. (1975). Data analysis using Stein’s estimator and its generalizations. J. Amer. Statist. Assoc. 70:311-319. 76. Efron, B., and Morris, C. (1977). Stein’s paradox in statistics. Sci. A m . 236(5):119-127.

References

605

77. Faith, R. E. (1976). Minimax Bayes set and point estimators of a multivariate normal mean. Tech. Report 66, University of Michigan, Ann Arbor. 78. Farebrother, R. W. (1975). The minimum mean square error linear estimator and ridge regression. Technometrics, 17:127-128. 79. Feller, W. (1954). A n Introduction to Probability Theory and its Applications. Wiley, New York. 80. Ferguson, T. S. (1967). Mathematical Statistics: A Decision Theoretical Approach. Academic Press, New York. 81. Ferguson, T. S. (1996). A course in Large Sample Theory. Chapman and Hall, New York. 82. Fuller, W. (1976). Introduction to time series. Wiley, New York. 83. Gart, J. J. (1962). On the combination of relative risks. Biometrica 18:471-475. 84. Gart, J. 3. (1992). Pooling 2 x 2 tables: Asymptotic moments of estimators. J . Roy. Statist. SOC.B54:531-539. 85. Ghosh, M., Hwang, J., and Tsui, K. (1983). Construction of improved estimators in multiparameter estimation for discrete exponential families. Discussion by J. 0. Berger, H. Malcolm Hudson and Carl Morris. An. Statist. 11:351-376. 86. Ghosh, M., Saleh, A. K. Md. E., and Sen, P. K. (1989). Empirical Bayes subset estiation in regression models. Stat. Dec., 7:15-35. 87. Gibbons, D. G. (1981) A simulation study of some ridge estimators. J. Amer. Stat. ASSOC.,76:131-139. 88. Graybill, F. A. (1976). Theory and Application of the Linear Model. Duxbury Press. 89. Griliches, Z., and Rao, P. (1969). Small sample properties of several two-stage regression methods in the context of autocorrelated errors. J. Amer. Stat. AsSOC., 64:253-272. 90. Gruber, hl.H.J. (1998). Zmprowing Eficiency b y Shrinkage: The James-Stein and Ridge Regression Estimators. hlarcel Dekker, New York. 91. Gupta, A.K., Saleh, A. K. Md. E., and Sen, P. K. (1989). Improved estimation in a contingency table: Independence structure. J. Amer. Stat. Assoc., 84:525532. 92. Gupta, A.K., and Saleh, A. K. Md. E. (1997). Estimation Odds-Ratio: Homogeneity Constraints. J. Ital. Stat. SOC.6(1):67-81. 93. Hajek, J. (1969). Nonparametric Statistics. Holden Day, San Francisco. 94. Hajek, J., and Sidak, Z. (1967). Theory of Rank Tests. Academic Press, New York. 95. Hajek, J., Sidak, Z., and Sen, P.K. (1999). Theory of Rank Tests. Academic Press, New York. 96. Hald, A. (1952). Statistical Theory with Engineering Applications. Wiley, New York. 97. Hall, P., and Heyde, C. C. (1980). Martingale limit theory and its applications. Academic Press, New York. 98. Hallin, M., and Puri, M. L. (1988). Optimal rank-based procedures for timeseries analysis: testing an ARMA model against other ARMA models. An. Stat&. 16:402-432. 99. Hannan, E. J. (1970). Multiple Time Series. Wiley, New York.

606

References

100. Han, C. P., and Bancroft, T. A. (1968). On pooling means when variance is unknown. J . Amer. Stat. Assoc., 63 1333-1342. 101. Haq, M.S., and Kibria, B.M.G. (1996). A shrinkage estimator for the restricted linear regression model: Ridge regression approach. J. Appl. Stat. Sci. 3(4):301-316. 102. Hemmerle, W. J., and Brantle, T. F. (1978). Explicit and constraint generalized ridge estimator. Technometrics 2: 109-120. 103. Hocking, R. R., Speed, F. M., and Lynn, M.J. (1976). A class of biased estimators in linear regression. Technometrics 18:425-438. 104. Hodges, J. L., and Lehmann, E. L. (1963). Estimates of location based on rank tests. An. Math. Stat. 34:598-611. 105. Hoeffding, W. (1948). A class of statistics with asymptotically normal distribution. An. Math. Stat. 19:293-325. 106. Hoerl, A. E., and Kennard, R. W. (1970). Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12:55-67. 107. Hoerl, A.E., Kennard, R.. W., and Baldwin, K.F. (1975). Ridge regression: Some simulation. Commun. Statist. 4:105-123. 108. Hoffmann, K. (1992). Improved Estimation of Distribution Parameters: Steintype Estimators. Teubner Verlagsgesellschaft, Stuttgart. 109. Hogg, R. V., McKean, J. W. and Craig, A. T. (2005). Introduction to Mathematical Statistics (7th ed.). Prentice Hall, NJ. 110. Huntsberger, D. V. (1955). A generalization of a preliminary test procedure of pooling data. An. Math. Stat. 26:734-743. 111. Hugkova, M. (1971). Asymptotic distribution of rank statistics used for multivariate testing symmetry. J. Mult. Anal. 1:461-484. 112. Hwang, J . T. (1985). Universal domination and stochastic domination: Estimation simultaneously under a broad class of loss functions. An. Stat. 13:295-315. 113. Hwang, J. T., and Casella, G. (1982). Minimax confidence sets for the mean of a multivariate normal distribution. An. Stat. 10:868-881. 114. Hwang, J. T., and Casella, G. (1984). Improved set estimators for a multivariate normal mean. Stat. Decision, suppl. 1:3-16. 115. Hwang, J. T., and Chen, J. (1986). Improved confidence sets for the coefficients of a linear model with spherically symmetric errors. An. Stat. 14:444-460. 116. Hwang, J.T., and Ullah, A. (1989). Confidence sets recentered at James-Stein estimators - a surprise concerning the unknown variance case. Technical Report, Mathematics Department, Cornell University. 117. Inoue, T. (2001). Improving the “HKB” ordinary type ridge estimators. J. Jpn. Stat. SOC.31(1):67-83. 118. James, W. and Stein, C. (1961). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. on Math. Statist., and Probability, University of California, V ~ l . ’ pp. l 361-379. 119. Joshi, V. M. (1967). Inadmissibility of the usual confidence set for the mean of a multivariate normal population. An. Math. Stat. 38:1868-1875. 120. Joshi, V. M. (1969). Admissibility of the usual confidence sets for the mean of a univariate or bivariate normal population. An. Math. Stat. 40:1042-1067. 121. Johnson, N., Kotz, S., and Balakrishnan, N. S. (1994). Continuous univariate distributions - 1. Wiley, New York.

References

607

122. Judge, G. G., and Bock, M. E. (1978). The Statistical Implications of Pre-test and Stein-rule Estimators in Econometrics. North-Holland, Amsterdam. 123. JureEkovA, J. (1969). Asymptotic linearity of a rank statistic in regression parameter. An. Math. Statist. 40:1889-1900. 124. JureEkovA, J., and Sen, P. K. (1996). Robust Statistical Procedures: Asymptotics and Interrelations. Wiley, New York. 125. Kakwani, T. (1968). Note on the unbiased of a mixed regression estimator. Econometrics 36:610-611. 126. Kale, B. K., and Bancroft, T. A. (1967). Inference for some incompletely specified models involving normal approximations to discrete data, Biometrics 23:335-348. 127. Kendall, M. G., and Stuart, A. (1963). The Advanced Theory of Statistics, 2 Vols. Hafner Publishing, New York. 128. Khan, Bashir U1. (1997). Some contribution to positive part shrinkage estimation in various model. Ph.D. thesis. University of Regina, Regina, Canada. 129. Khan, Bashir U1. and Saleh, A.K.Md. E. (2005). Estimation of Regression Parameters: Parallelism R.estriction. J . Statist. Theory Appl. 4:91-107. 130. Khan, S., and Saleh, A. K. Md. E. (1997). Shrinkage pre-test estimator of the intercept parameter for a regression model with multivariate Student’s t-errors. Biomed. J . 2:131-147. 131. Khan, S., and Saleh, A. K. Md. E. (2001). On the comparison of pre-test and shrinkage estimators for the univariate normal mean. Stat. papers 42(4):451473. 132. Khan, S., Hoque, Z., and Saleh, A. K. Md. E. (2002). Estimation of the slope parameter for linear regression model with uncertain prior information. J. Stat. Res. 36(2):55-74. 133. Ki, F., and Tsui, K. (1985). Improved confidence set estimators of a multivariate normal mean and generalizations. An. Inst. Statist. Math. 37:487-498. 134. Kibria, B. M.G. (1996a). On preliminary test ridge regression estimator for the restricted linear model with non-normal disturbances. Commun. Stat. Theory Meth. 25:2349-2369. 135. Kibria, B. M. G. (1996b). On shrinkage ridge regression estimators for restricted linear models with multivariate t disturbances Students, 1 (3) 177-188. 136. Kibria, B. M. G. (2003). Performance of some new ridge regression estimators. Commun. Statist. Simul. Comput. B32:429-435. 137. Kibria, B. M. G. and Saleh, A. K. Md. E. (1993). Performance of shrinkage preliminary test estimator in regression analysis. Jahangirnagar Review A17: 133148. 138. Kibria, B.M. G. and Saleh, A.K. Md. E. (2003). Effect of W, LR, and LM tests on the performance of preliminary test ridge regression estimators. J . J p n . Stat. SOC. 33:119-136. 139. Kibria, B. M. G. and Saleh, A. K. Md. E. (2004a). Preliminary test ridge regression estimators with Student’s t errors and conflicting test-statistics, Metrilca 59:105-124. 140. Kibria, B. M. G. and Saleh, A. K. Md. E. (2004b). Performance of Positive Rule Ridge Regression Estimators for the Ill-Conditioned Gaussian Regression Models. Calcutta Stat. Assoc. Bull. 55 to appear.

608

References

141. Kim, P.T. (1987). Recentered confidence sets for the mean of a multivariate distribution when the scale parameter is unknown. Ph.D. thesis. Department of Mathematics, University of California a t San Diego. 142. Kim, H. M., and Saleh, A. K. Md. E. (2003). Preliminary test estimators of the parameters of simple linear model with measurement errors. Metrika 57:223251. 143. Kim, H. M., and Saleh, A. K. Md. E. (2004). Improved estimation of regression parameters in measurement error model. J . Mult. Anal. 95:273-300. 144. Kitagawa, T. (1963). Estimation after preliminary tests of significance. UC Publ. Stat. 3:147-186. 145. Kmenta, J. and Gilbert, R. F. (1968). Small sample properties of alternative estimates of seemingly unrelated regressions. J. Amer. Stat. Assoc. 63:11801200. 146. Kmenta, J. and Gilbert, R.F. (1970). Estimation of seemingly unrelated regressions with autoregressive disturbances. J . Amer. Stat. Assoc., 65:186-197. 147. Koul, H.L. (1977). Behavior of robust estimators in the regression model with dependent errors. A n . Statist. 5:681-699. 148. Koul, H.L., and Saleh, A. K. Md. E. (1993). R-Estimation of the parameters of autoregressive AR[p] models. A n . Statist. 21:534-551. 149. Koul, H. L., and Saleh, A. K.Md. E. (1995). Autoregression quantiles and related rank-scores processes. A n . Statist. 23:670-689. 150. Kubokawa, T. (1991). An approach to improving the James-Stein estimator. J. Mult. Anal. 36:121-126. 151. Lambert, A.,Saleh, A. K. Md. E. and Sen, P. K. (1985). On least squares estimation of intercept after a preliminary test on parallelism of regression lines. Commun. Statist. Theory Math. 14(4):793-807. 152. Lawless, J.F. (1978) Ridge and related estimation procedure. Commun. Statist., A7:139-164. 153. Lawless, J.F., and Wang, P. (1976). A simulation study of ridge and other regression estimators. Commun. Statist. A5:307-323. 154. Lindley, D. (1962). Discussion of Professor Stein’s paper, ”Confidence sets for the mean of a multivariate normal distribution”. J . Roy. Stat. SOC.B24:265296. 155. Lindley, D., and Smith, A.F. M. (1972). Bayes estimates for the linear model (with discussion) J . Roy. Stat. SOC.B34:l-41. 156. Lu, K., and Berger, J. (1989). Estimated confidence procedures for multidimensional normal means. J. Stat. Plann. Inf. 23( 1):l-20. 157. Maatta, J. M., and Casella, G. (1990). Variance estimation. Statist. Sci. 5:107109. 158. Malthouse, E. C. (1999). Shrinkage estimation and direct marketinmg scoring model. J . Interactive Market. 13(4):10-23. 159. Marquardt, D. W., and Snee, R. D. (1975). Ridge regression in practice. A m . Statistician 29:3-20. 160. McDonald, G.C.,Galarneau, D. I. (1975). A monte carlo evaluation of some Ridge-type estimators. J. A m . Stat. Assoc., 70:407-416. 161. Montgomery, D. C.,Peck, E. A., and Vining, G. G. (2001). Introduction to Linear Regression Analysis. 3rd ed. Wiley, New York.

References

609

162. Morris, C. N. (1983). Parametric empirical Bayes inference: Theory and applications. J . A m . Stat. Assoc. 78:47-65. 163. hfosteller, F. (1948). On pooling data. J. A m . Stat. Assoc., 43:231-242. 164. Nagar, A. L., and Kakwani, N. C. (1965). Note on the use of prior information in statistical estimation of economic relations. Sankhya A27:105-112. 165. Newhouse, J. P., and Oman, S. D. (1971). An Evaluation of Ridge Estimators. Rand Corporation, P-716-PR. 166. Obenchain, R.L. (1975). Ridge analysis following a preliminary test of the shrunken hypothesis. Technometrics 17:431-441. 167. Prais, S. J., and Winsten, J. A. (1954). Tren estimators and series correlation. Cowles Commission Discussion Paper, 383. Chicago. 168. Puri, M.L., and Sen, P.K. (1971). Nonparametric Methods i n Multivariate Analysis. Wiley, New York. 169. Puri, M. L., and Sen, P. K. (1986). Nonparametric methods in general linear models. Wiley, New York. 170. Rao, C. R.. (1973). Linear Statistical Inference and its Applications. Wiley, New York. 171. Rao, C. R. (1975). Simultaneous estimation of parameters in different linear models with applications to biometric problems. Biometrics, 31:545-554. 172. Randles, R. H., and Wolfe, D. A. (1979). Introduction to the Theory of Nonparametric Statistics. Wiley, New York. 173. RCnyi, A. (1970). Foundations of Probability. Holden-Day, San Francisco. 174. Robbins, H.E. (1955). An empirical Bayes approach to statistics. Proc. 3rd Berkeley Symp. Meth. Statist. 35:l-20. 175. R.oberg, S. J., and Stegeman, J. W. (1991). Pooling data for stability studies: testing the equality of batch degradation slopes. Bzometrics 47:1059-1069. 176. Robert, C. and Casella, G. (1987). Improved confidence sets in spherically symmetric distributions. J. Mult. Anal. 22:300-315. 177. Robert, C. P., and Saleh, A. K. Md. E. (1991). Point estimation and confidence set estimation in a parallelism model: An empirical Bayes approach. An. Econ. Statist. 23:65-89. 178. Rodrigue, J. (1987). Some shrunken predictors in finite population with a multinormal superpopulation model. Stat. Prob. 5:347-351. 179. R.ohatgi, V. K., and Saleh, A. K. Md. E. (2001). Introduction to Probability and Statistics. Wiley, New York. 180. Saleh, A. K. Md. E. (1992). On shrinkage estimation of the parameters of an autoregressive Gaussian process. Theory Prob. Appl. 37:290-300. 181. Saleh, A. K. Md. E. (2003). Asymptotic properties of Stein-type confidence sets - a U-statistics Approach. Sankhya 11, 65:l-11. 182. Saleh, A. K. Md. E. and Han, C.-P. (1990). Shrinkage estimation in regression analysis. Estadistica 4 2 40-43. 183. Saleh, A. K. Md. E. and Hassanein, K.M. (1986). On various F-estimation in a parallelism problem. Soochow J . Math. 12:83-94. 184. A. K. Md. E., Hassanein, K.M., Hassein, R.S., and Kim, H.M. (2006). Quasiempirical Bayes methodology for improving meta-analysis. J. Biopharmaceutical Stat. 16:77-90.

610

References

185. Saleh, A.K.Md.E., and Kibria, B.M.G. (1993). Performances of some new preliminary test ridge regression estimators and their properties. Commun. Stat. Theory Meth. 22:2747-2764. 186. Saleh, A. K. Md. E. and Sen, P. K. ( 1 978). Non-parametric estimation of location parameter after a preliminary test on regression. A n . Statist. 6:154-168. 187. Saleh, A. K. Md. E., and Sen, P. K. (1983). Nonparametric tests of location after a preliminary test on regression in the multivariate case. Commun. Statist. Theory Meth. 12(16):1855-1872. 188. Saleh, A. K. Md. E., and Sen, P. K. (1984). Least squares and rank order preliminary test estimation in general multivariate linear models. Proc. ISI, Golden Jubilee Conf. on Statist.: Application and New Directions (Dec. 16-19, 1981), pp. 237-253. 189. Saleh, A. K. Md. E., and Sen, P. K. (1984). Nonparametric preliminary test inference. Handbook of Statist. 4 (eds. P. R. Krishnaiah and P. K. Sen) NorthHolland, Amsterdam, pp. 275-297. 190. Saleh, A. K. Md. E., and Sen, P. K. (1985). On shrinkage M-estimators of location parameters. Commun. Statist. Theory Meth. 1 4 (10):2313-2329. 191. Saleh, A. K. Md. E. and Sen, P. K. (1985a). Nonparametric shrinkage estimation in a parallelism problem. Sankhya 47A:156-165. 192. Saleh, A. K. Md. E., and Sen, P. K. (1985b). Preliminary test predicted in general multivariate linear models. Proc. Pacific Area Statistical Conf. (ed. M. Matusita) (Dec. 15-19, 1982), North-Holland, Amsterdam, pp. 619-638, 193. Saleh, A. K. Md. E., and Sen, P. K. (1985~).Shrinkage least squares estimation in a general multivariate linear model. Proc. 5th Pannonian Symp. on Math. Statistics (eds. J. Mogyorodi, I. Vincze, and W. Wertz), pp. 307-325. 194. Saleh, A. K. Md. E. and Sen, P. K. (1985). On shrinkage least squares estimation in a parallelism problem. Commun. Statist. Theory Meth. 15:1451-1466. 195. Saleh, A. K. Md. E. and Sen, P. K. (1986). On shrinkage R-estimation in a multiple regression model. Commun. Statist. Theory Meth. 15(7):2229-2244. 196. Sarker, N. (1992). A new estimator combining the risge regression and the restricted least squares method of estimation. Commun. Statist. Theory Meth. 21:1987-2000. 197. Sclove, S. L., Morris, C., and Radhakrishnan, R. (1972). Non-optimdity of preliminary test estimators for the mean of a multivariate normal distribution. An. Math. Statist. 45:1481-1490. 198. Sen, P.K. (1969). On a class of rank order tests for parallelism of several regression lines. An. Math. Statist. 40:166&1683. 199. Sen, P. K. (1970). On some convergence properties of one sample rank order statistics. A n . Statist. 41:2 140-2 143. 200. Sen, P. K. (1980). On almost sure linearity theorems for signed rank statistics. A n . Statist. 8:313-321. 201. Sen, P. K. (1981). Sequential Nonparametrics: Invariance principle and Statistical Inference. Wiley, New York. 202. Sen, P. K. (1984). A James-Stein type detour of U-Statistics. Commun. Statist. Theory Meth. 13(22):2725-2747.

References

611

203. Sen, P. K. (1986). On the asymptotic distributional risks of shrinkage and preliminary test versions of maximum likelihood estimators. Sankhya A48:354371. 204. Sen, P. K., and Saleh, A. K. Md. E. (1979). Nonparametric estimation of location parameter after a preliminary test on regression in multivariate case. J . Mult. Anal. 9(2):322-331. 205. Sen, P.K., and Saleh, A.K.Md.E. (1985). On some shrinkage estimators of multivariate location. An. Statist. 13:172-281. 206. Sen, P. K., and Saleh, A. K. Md. E. (1987). On preliminary test and shrinkage M-estimation in linear models. An. Statist. 15(4):1580-1592. 207. Sen, P.K., and Singer, J.M. (1993). Large Sample Methods in Statist.: A n Introduction with Applications, Chapman and Hall, New York. 208. Shinozaki, N. (1989). Improved confidence sets for the mean of a multivariate normal distribution. An. Inst. Stat. Math. 41(2):331-346. 209. Singh, S., and Tracy, D. S. (1999). Ridge-regression using scrambled responses. Metrika, 147-157. 210. Snedecor, G. W. (1938). Statistical Methods. Collegiate Press, Iowa. 211. Srivastava, M. S., and Khatri, C. B. (1979). A n Introduction to Multivariate Statistics. North Holland, Amsterdam. 212. Srivastava, M. S., and Saleh, A. K. Md. E. Saleh (2004). Estimation of the Mean Vector of a Multivariate Normal Distribution: Subspace Hypothesis. J. Mult. Anal. 9655-72. 213. Stein, C. (1955). Inadmissibility of the usual estimator of mean of a multivariate normal distribution. Proc. 3rd Berkeley Symp. Math. Statist. Prob. 1 pp. 197206. 214. Stein, C. (1955). A necessary and sufficient condition for admissibility. An. Math. Statist. 26:518-522. 215. Stein, C. (1956). Inadmissibility of the usual estimator of the mean of a multivariate normal distribution. Proc. 3rd Berkeley Symp. Math. Statist. Probability 1:197-206. 216. Stein, C. (1962). Confidence sets for the mean of a multivariate normal distribution. J. Roy. Stat. SOC.B24:265-296. 217. Stein, C. (1964). Inadmissibility of the usual estimators of variance of a normal distribution with unknown mean. An. Znst. Stat. Math. 16:155-160. 218. Stein, C. (1966). An approach to the recovery of inter-block information in balanced incomplete block designs. Research Papers in Statistics. 219. Stein, C. (1973). Estimation of the mean of a multivariate distribution. Proc. Prague Symp. Asymptotic Statist. pp. 345-381. 220. Stein, C. (1981). Estimation of the mean of multivariate normal distribution. A n . Statist. 93135-1151. 221. Strawdermann, W. E. (1971). Proper Bayes minimax estimators of the multivariate normal mean. An. Math. Statist. 47:365-388. 222. Tabatabaey, S. M. (1995). Preliminary test approach estimation: regression model with spherically symmetric errors. Ph.D. thesis. Carleton University, Ottawa. 223. Tamura, R. (1967a). Small sample properties of a sometimes posted estimate in the nonparametric case. Bull. Math. Statist. 12:75-83.

References 224. Tamura, R. (1967b). Some estimation procedures with a nonparametric preliminary test 11. Bull. Math. Statist. 12:47-59. 225. Tamura, R. (1967~).Some weighted estimates in the nonparametric case. Bull. Math. Statist. 12:61-73. 226. Theil, H. (1963). On the use of incomplete prior information in regression analysis. J. Am. Stat. Assoc. 58:401-414. 227. Theil, H. (1971). Principle of Econometrics. Wiley, New York. 228. Theil, H and Goldberger, A. S. (1961). On pure and mixed statistical estimation in economics. Int. Econ. Rev. 2:65-78. 229. Thompson, S. G., and Pocock, S. J. (1991). Can meta-analysis be trusted? Lancet 338:1127-1130. 230. Vinod, H. D., and Ullah, A. (1981). Recent advances in regression methods. Marcel Dekker, New York. 231. Wencheko, E. (2000). Estimation of the signal-t-noise in the linear regression model, Statist. Papers 41:327-343. 232. Whittle, D. (1952). Estimation and information in time series analysis. Skand. A ~ W U T35:48-60. .

Glossary UE

Unrestricted estimator

UMLE

Unrestricted maximum likelihood estimator

ULSE

Unrestricted least-squares estimator

UR E

Unrestricted rank estimator

UPPE

Unrestricted principal part estimator

UR RE

Unrestricted ridge regression estimator

RE

Restricted estimator

RMLE

Restricted maximum likelihood estimator

RLSE

Restricted least-squares estimator

RRE

Restricted rank estimator

RPPE

Restricted principal part estimator

RUE

Restricted unbiased estimator

R(.; .)

Relative efficiency

RRE(.; .)

Risk-based relative efficiency

PTE

Preliminary test estimator

PThlLE

Preliminary test maximum likelihood estimator

PTLSE

Preliminary test least-squares estimator

PTPPE

Preliminary test principal part estimator

PTRE

Preliminary test rank estimator

PTR RE

Preliminary test ridge regression estimator

PPRSE

Positive-rule Stein-type estimator

PTPPE

Positiverule Stein-type principal part estimator

PTRRE

Positive-rule Stein-type ridge regression estimator

JSE

James-Stein estimator

SE

Stein-type estimator

SMLE

Stein-type maximum likelihood estimator

SLSE

Stein-type least-squares estimator

613

Glossary

614 SRE

Stein-type rank estimator

SRRE

Stein-type ridge regression estimator

SPPE

Stein-type principal part estimator

LSE

least-squares estimator

MLE

maximum likelihood estimator

MSE

Mean square error

MRE(., .)

Mean square error based relative efficiency

ADB

Asymptotic distributional bias

ADQB

Asymptotic distributional quadratic bias

ADMSE

Asymptotic distributional mean square error

ADQR

Asymptotic distributional quadratic Risk

D.F.

Degrees of freedom

C.L.T.

Central limit theorem

EBE

Empirical Bayes estimation

Symbols and Meaning UE RE PTE

SE PRSE

Bias (vector) Quadratic bias MSE

Risk

D.F

CDF of a noncentral chi-square distribution with

Y

CDF of a noncentral F distribution with

D.F.

test-statistic noncentrality parameter

(YI, 2 4 )

Authors Index Adichie, J.N. 50, 56, 110, 111 Agresti, A. 22, 38, 595, 594 Ahmed, S.E. 55, 58, 530 Ahsanullah, M. 55, 58 Akritas, M.G. 272 Albert, J.H. 553, 587 Ali, A.M. 348, 551, 555, 557 Anderson, T.W. 12, 34, 40 Anderson, D.R. 463 Arnold, Steven F. 512 Asano, C. 2 Baldwin, K.F. 444, 462 Baksalary, J.K. 451 Bancroft, T.A. 2, 55, 56, 129, 136, 550 Baranchik, A. 140 Benda, Norbert 344 Bennett, B.M. 2 Berenblutt, 1.1. 485 Berger, J.O. 3, 140, 183, 186, 510 Berger, R.L. 30 Berry, J.C. 171 Bickel, P.J. 30 Bishop, Y.M.M. 22, 38, 585 Bock, M.E. 18, 30, 32, 58, 485, 487 Bolfaine, H. 77, 78 Bozivich, H. 2 Brantle, T.F. 444 Brewster, J.F. 140, 182 Brockwell, P.J. 494 Brown, L.D. 140, 183, 186 Casella, G. 5, 30, 103, 140, 186, 192, 253, 256, 257, 258 Cellier, D. 140 Chen, J. 186 Chen E.J. 487 Chiou, Paul C. 193 615

616

Authors Index

Cochrane, D. 475, 476, 487 Cohen, A. 2, 140, 186 Craig, A.T. 30 Cramer, H. 30, 38 Davis R.A. 494 Deely, J. 156 Dempster, A.P. 444, 462 Doksum, K.A. 30 Doob, J.L. 494 Durbin, J. 474, 485, 504 Dzhaparidze, K. 494 Efron, B. 2, 3, 4, 5, 140, 151, 154, 158, 510, 512, 549, 550, 553 Faith, R.E. 186 Farebrother, R.W. 444 Feller, W . 40 Ferguson, T.S. 40, 594 Fienberg, S.E. 38, 594 Fourdrinier, D. 140 Fuller, W. 474 Galarneau, D.1 444 Gart, J.J. 21, 570 Ghosh, M. 5, 348 Gibbons, D.G. 440,462 Gilbert, R.F. 485 Goldberger, A.S. 404 Graybill, F.A. 18 Griliches, Z. 487 Gruber, M.H.J. 18, 137, 440, 444 Gupta, A.K. 553, 587 Hajek, J. 45, 48, 111, 199, 200, 263, 500 Hall, P. 500 Hallin, M. 500 Han, C.-P. 56, 129 Hannan, E.J. 494 Hartley, H.O. 2 Hassanein, K.M. 272 Hassanein, R.S. 584 Hemmerle, W.J. 444 Heyde, C.C. 500 Hocking, R.R. 462 Hodges, J.L. 50, 110 Hoeffding, W. 516, 517 Hoed, A.E. 16, 440, 444, 462 Hoffmann, K. 30, 140, 152 Hogg, R.V. 30 Holland, P.W. 38, 594

Authors Index

Hoque, Z. 83 Huntsberger, D.V. 2 HuSkov6, M. 526 Hwang, J.T. 5, 183, 186, 192, 253, 256, 257, 258 Inoue, Takakatsu 440 James, W. 125, 151 Joshi, V.M. 186 Judge, G.G. 18, 30, 32, 58, 485, 487 JureEkov6, J . 52, 112, 263, 397, 431 Kakwani, T . 404 Kale, B.K. 550 Kendall, M.G. 38 Kennard, R.W. 16, 440, 444, 462 Khan, S. 78, 83 Khan, Bashir 242, 419 Khatri, C.B. 12, 34, 40, 511 Ki, F. 186 Kibria, B.M.G. 440, 441, 462, 463 Kim, H.M. 584 Kitagawa, T . 2 Koul, H.L. 500, 504 Kubokawa, T. 140, 184 Kmenta, Jan 476, 485 Lambert, Annick 272 Lawless, J.F. 444, 462 Lehmann, E.L. 50, 110 Lindley, D. 156 Lu, K. 510 Lynn, M.J. 462 Maatta, J.M. 103 Malthouse, E.C. 440 Marquardt, D.W. 444 McDonald, G.C. 444 Montgomery, D.C. 440 Morris, C.N. 2, 3, 5, 131, 140, 151, 154, 158, 510, 512, 549, 550, 553 Mosteller, F . 2 Nagar, A.L. 404 Newhouse, J.P. 444 Obenchain, R.L. 444 Oman, S.D. 444 Orcutt, G.H. 475, 476, 487 Peck, E.A. 444 Pocock, S.J. 581 Prais, S.J. 475, 476, 487 Puri, M.L. 45, 110, 111, 199, 200, 397, 500, 516, 517, 523, 541 Radhakrishnan, R. 131

617

618

Authors Index

Randles, R.H. 45, 516 Rao, C.R. 12,40, 511 Rao, P. 487 Renyi, A. 36, 38, 40 Robbins, H.E 5, 154 Robert, Christian P. 140, 193, 272 Rodrigue, J. 77, 78 Rohatgi, V.K. 8, 30, 40, 516 Saleh, A.K.Md. Ehsanes 5, 8, 30, 40, 52, 55, 56, 58, 78, 83, 109, 113, 129, 140, 192, 193, 272, 348, 440, 441, 500, 501, 510, 516, 517, 524, 525, 530, 551, 555, 557, 573, 579, 564 Sarker, N. 440 Sato, S. 2 Schatzoff, M. 444, 462 Sclove, S.L. 36, 131 Sen, P.K. 5, 38, 40, 45, 52, 56, 109, 110, 111, 113, 115, 129, 140, 152, 199, 200, 272, 348, 397, 500, 501, 510, 516, 517, 524, 525, 530, 532, 541 Sidak, Z. 45, 48, 111, 199, 200, 263, 500 Singer, J.M. 38, 40 Shinozaki, N. 186 Singh, S. 440 Smith, A.F.M. 156 Snedecor, G.W. 2 Snee, R.D. 444 Speed, F.M. 462 Srivastava, M.S. 12, 34, 40, 511, 516 Stein, C. 2, 56, 125, 127, 137, 151, 186, 509 Strawdermann, W.E. 140, 183, 186 Stuart, A. 38 Tabatabaey, S.M. 440, 441 Tamura, R. 109 Theil, H. 404 Thompson, S.G. 581 Tracy, D.S. 440 Tsui, K. 5, 186 Ullah, A. 440, 444 Vining, G.G. 444 Vinod, H.D. 440, 444 Wang, P. 444, 462 Watson, G.S. 485 Webb, G.I. 485 Wencheko, E. 440 Wermuth, N. 450, 469 Whittle, D. 494 Winsten, J.A. 475, 476, 487 Wolfe, D.A. 45, 516, 525

Authors Index Yancey, T.A. 493,495 Zacks, S. 77, 798 Zidek, J.V. 140, 182

619


Subject Index ANOVA model:

2, 6, 13, 14, 23, 213

Asymptotic

distribution 22, 43, 45, 51, 53, 297, 375, 424, 427, 557 distributional bias 92, 176, 248, 377, 426, 512, 558 distributional MSE 94, 177, 248, 378, 426, 428, 513, 559, 566 distributional risk 177, 180, 248, 378, 426, 428, 513, 559 distributional quadratic risk 180, 378, 426, 428, 513, 559 Auto correlation 469, 470, 474, 478 Bayes risk 7, 8 Bayes rule 7, 8 Courant theorem

39

Convergence in distribution 41 in mean 41 in probability 41 almost sure 42 CLT 43 Hajek-Sidak CLT 44 Law of large numbers 42 Chebycheff’s inequality 43 Borel-Cantelli Lemma 43 Markov Inequality 43 Contiguity 44 Decision rule 7, 8 Decision theory 6 Distribution

Bernoulli 20, 36, 549, 550 Beta 29 Binomial 22, 36, 549 Chi square 29, 30, 31 F-distribution 10, 12, 14, 15 Multinomial 22, 37 621

622 Multivariate normal normal 29 Wishart 33

Subject Index 33, 34

Estimators Admissible 180 Bayes 154, 278, 344, 443 BLUE 8, 446 Confidence set 185, 250, 311, 386 Empirical Bayes 154, 162, 278, 344, 511 James-Stein 139, 140, 218, 276, 344, 407, 441 Maximum likelihood 127, 510 Meta analysis 580 minimax 127 mixed 405 order statistics 46 Preliminary test 129, 218, 264, 343, 407, 441 Positive-rule Stein 140, 264, 344, 407, 441 P T E of variance 130 Quasi-empirical Bayes 157, 175, 343 Restricted 340, 341, 405, 532 R-estimator 111, 198, 264, 395, 430, 522, 539 Ridge regression 439 Sclove modified 149 sign statistics 45, 46 Stochastic restricted 403 unrestricted 340, 341, 405, 532 Gauss-Markoff 8

Hypothesis likelihood ratio test 216, 274, 342 test statistic 2, 10, 12, 14, 16, 18, 19, 21, 22 inadmissible 2 loss function 8 Linex function 8 Linear model 56 Mean 2, 3, 8 Mean square error 8, 127 Non-centrality parameter 10, 11, 12, 14, 16, 19, 21, 22 Odds-ratio 21, 579, 580 Prior distribution 552, 586 Prior information 1 Posterior distribution 552, 586

WILEY SERIES IN PROBABILITY AND STATISTICS ESTABLISHED BY WALTER A. SHJWHART AND SAMUEL S. WILKS Editors: David J. Balding, Noel A. C. Cressie, Nicholas I. Fisher, lain M. Johnstone, J. B. Kadane, Geert Molenberghs. Louise M. Ryan, David W. Scott, Adrian F. M. Smith, Jozef L. Teugels Editors Emeriti: Vic Barnett, J. Stuart Hunter, David G. Kendall The Wiky Series in Probability and Statistics is well established and authoritative. It covers many topics of current research interest in both pure and applied statistics and probability theory. Written by leading statisticians and institutions, the titles span both state-of-the-art developments in the field and classical methods. Reflecting the wide range of current research in statistics, the series encompasses applied, methodological and theoretical statistics, ranging from applications and new techniques made possible by advances in computerized practice to rigorous treatment of theoretical approaches. This series provides essential and invaluable reading for all statisticians, whether in academia, industry, government, or research.

t

*

* *

ABRAHAM and LEDOLTER . Statistical Methods for Forecasting AGRESTI . Analysis of Ordinal Categorical Data AGRESTI . An Introduction to Categorical Data Analysis AGRESTI . Categorical Data Analysis, Second Edition ALTMAN, GILL, and McDONALD . Numerical Issues in Statistical Computing for the Social Scientist AMARATUNGA and CABRERA . Exploration and Analysis of DNA Microarray and Protein Array Data ANDBL . Mathematics of Chance ANDERSON . An Introduction to Multivariate Statistical Analysis, Third Edition ANDERSON . The Statistical Analysis of Time Series ANDERSON, AUQUIER, HAUCK, OAKES, VANDAELE, and WEISBERG . Statistical Methods for Comparative Studies ANDERSON and LOYNES . The Teaching of Practical Statistics ARMITAGE and DAVID (editors) . Advances in Biometry ARNOLD, BALAKFUSHNAN, and NAGARAJA . Records ARTHANARI and DODGE . Mathematical Programming in Statistics BAILEY . The Elements of Stochastic Processes with Applications to the Natural Sciences BALAKRISHNAN and KOUTRAS . Runs and Scans with Applications BARNETT . Comparative Statistical Inference, Third Edition BARNETT and LEWIS . Outliers in Statistical Data, Third Edition BARTOSZYNSKI and NIEWIADOMSKA-BUGAJ . Probability and Statistical Inference BASILEVSKY . Statistical Factor Analysis and Related Methods: Theory and Applications BASU and RlGDON . Statistical Methods for the Reliability of Repairable Systems BATES and WATTS . Nonlinear Regression Analysis and Its Applications BECHHOFER, SANTNER, and GOLDSMAN . Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons BELSLEY . Conditioning Diagnostics: Collinearity and Weak Data in Regression

*Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-Interscience Paperback Series.

f BELSLEY, KUH, and WELSCH . Regression Diagnostics: Identifying Influential Data and Sources of Collinearity BENDAT and PIERSOL . Random Data: Analysis and Measurement Procedures, Third Edition BERRY, CHALONER, and GEWEKE . Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner BERNARD0 and SMITH . Bayesian Theory BHAT and MILLER . Elements of Applied Stochastic Processes, Third Edition BHATTACHARYA and WAYMIRE ’ Stochastic Processes with Applications f BIEMER, GROVES, LYBERG, MATHIOWETZ, and SUDMAN . Measurement Errors in Surveys BILLINGSLEY . Convergence of Probability Measures, Second Edition BILLINGSLEY . Probability and Measure, Third Edition BIFXES and DODGE . Alternative Methods of Regression BLISCHKE AND MURTHY (editors) . Case Studies in Reliability and Maintenance BLISCHKE AND MURTHY . Reliability: Modeling, Prediction, and Optimization BLOOMFIELD . Fourier Analysis of Time Series: An Introduction, Second Edition BOLLEN . Structural Equations with Latent Variables BOLLEN and CURRAN . Latent Curve Models: A Structural Equation Perspective BOROVKOV . Ergodicity and Stability of Stochastic Processes BOULEAU . Numerical Methods for Stochastic Processes BOX . Bayesian Inference in Statistical Analysis BOX . R. A. Fisher, the Life of a Scientist BOX and DRAPER . Empirical Model-Building and Response Surfaces * BOX and DRAPER . Evolutionary Operation: A Statistical Method for Process Improvement BOX, HUNTER, and HUNTER . Statistics for Experimenters: Design, Innovation, and Discovery, Second Editon BOX and LUCERO . Statistical Control by Monitoring and Feedback Adjustment BRANDIMARTE . Numerical Methods in Finance: A MATLAB-Based Introduction BROWN and HOLLANDER . Statistics: A Biomedical Introduction BRUNNER, DOMHOF, and LANGER . Nonparametic Analysis of Longitudinal Data in Factorial Experiments BUCKLEW . Large Deviation Techniques in Decision, Simulation, and Estimation CAIROLI and DALANG . Sequential Stochastic Optimization CASTILLO, HADI, BALAKRISHNAN, and SARABIA . Extreme Value and Related Models with Applications in Engineering and Science CHAN . Time Series: Applications to Finance CHARALAMBIDES . Combinatorial Methods in Discrete Distributions CHATTERJEE and HADI . Sensitivity Analysis in Linear Regression CHATTERJEE and PRICE . Regression Analysis by Example, Third Edition CHERNICK . Bootstrap Methods: A Practitioner’s Guide CHERNICK and FRIIS . Introductory Biostatistics for the Health Sciences CHILES and DELFINER . Geostatistics: Modeling Spatial Uncertainty CHOW and LIU . Design and Analysis of Clinical Trials: Concepts and Methodologies, Second Edition CLARKE and DISNEY . Probability and Random Processes: A First Course with Applications, Second Edition * COCHRAN and COX . Experimental Designs, Second Edition CONGDON . Applied Bayesian Modelling CONGDON . Bayesian Statistical Modelling CONOVER . Practical Nonparametric Statistics, Third Edition COOK . Regression Graphics


*

* *

* *

*

t

*

t

COOK and WEISBERG . Applied Regression Including Computing and Graphics COOK and WEISBERG . An Introduction to Regression Graphics CORNELL . Experiments with Mixtures, Designs, Models, and the Analysis of Mixture Data, Third Edition COVER and THOMAS . Elements of Information Theory COX . A Handbook of Introductory Statistical Methods COX . Planning of Experiments CRESSIE . Statistics for Spatial Data, Revised Edition CSORGO and HORVATH . Limit Theorems in Change Point Analysis DANIEL . Applications of Statistics to Industrial Experimentation DANIEL . Biostatistics: A Foundation for Analysis in the Health Sciences, Eighth Edition DANIEL . Fitting Equations to Data: Computer Analysis of Multifactor Data, Second Edition DASU and JOHNSON . Exploratory Data Mining and Data Cleaning DAVID and NACARAJA . Order Statistics, Third Edition DEGROOT, FIENBERG, and KADANE . Statistics and the Law DEL CASTILLO . Statistical Process Adjustment for Quality Control DEMARIS . Regression with Social Data: Modeling Continuous and Limited Response Variables DEMIDENKO . Mixed Models: Theory and Applications DENISON, HOLMES, MALLICK and SMITH . Bayesian Methods for Nonlinear Classification and Regression DETTE and STUDDEN . The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis DEY and MUKERJEE . Fractional Factorial Plans DILLON and GOLDSTEIN . Multivariate Analysis: Methods and Applications DODGE . Alternative Methods of Regression DODGE and ROMIG . Sampling Inspection Tables, Second Edition DOOB . Stochastic Processes DOWDY, WEARDEN, and CHILKO . Statistics for Research, Third Edition DRAPER and SMITH . Applied Regression Analysis, Third Edition DRYDEN and MARDIA . Statistical Shape Analysis DUDEWICZ and MISHRA . Modem Mathematical Statistics DUNN and CLARK . Basic Statistics: A Primer for the Biomedical Sciences, Third Edition DUPUIS and ELLIS . A Weak Convergence Approach to the Theory of Large Deviations ELANDT-JOHNSON and JOHNSON . Survival Models and Data Analysis ENDERS . Applied Econometric Time Series ETHIER and KURTZ . Markov Processes: Characterization and Convergence EVANS, HASTINGS, and PEACOCK . Statistical Distributions, Third Edition FELLER . An Introduction to Probability Theory and Its Applications, Volume I, Third Edition, Revised; Volume 11, Second Edition FISHER and VAN BELLE . Biostatistics: A Methodology for the Health Sciences FITZMAURICE, LAIRD, and WARE . Applied Longitudinal Analysis FLEISS . The Design and Analysis of Clinical Experiments FLEISS . Statistical Methods for Rates and Proportions, Third Edition FLEMING and HARRINGTON . Counting Processes and Survival Analysis FULLER . Introduction to Statistical Time Series, Second Edition FULLER. Measurement Error Models GALLANT . Nonlinear Statistical Models GEISSER . Modes of Parametric Statistical Inference GEWEKE . Contemporary Bayesian Econometrics and Statistics


GHOSH, MUKHOPADHYAY, and SEN . Sequential Estimation GIESBRECHT and GUMPERTZ . Planning, Construction, and Statistical Analysis of Comparative Experiments GIFI . Nonlinear Multivariate Analysis GIVENS and HOETING . Computational Statistics GLASSERMAN and YAO . Monotone Structure in Discrete-Event Systems GNANADESIKAN . Methods for Statistical Data Analysis of Multivariate Observations, Second Edition GOLDSTEKN and LEWIS . Assessment: Problems, Development, and Statistical Issues GREENWOOD and NIKULM . A Guide to Chi-Squared Testing GROSS and HARRIS . Fundamentals of Queueing Theory, Third Edition t GROVES . Survey Errors and Survey Costs * HAHN and SHAPIRO . Statistical Models in Engineering HAHN and MEEKER . Statistical Intervals: A Guide for Practitioners HALD . A History of Probability and Statistics and their Applications Before 1750 HALD . A History of Mathematical Statistics from 1750 to 1930 HAMPEL . Robust Statistics: The Approach Based on Influence Functions HANNAN and DEISTLER . The Statistical Theory of Linear Systems HEIBERGER . Computation for the Analysis of Designed Experiments HEDAYAT and SINHA . Design and Inference in Finite Population Sampling HELLER . MACSYMA for Statisticians HMKELMANN and KEMPTHORNE . Design and Analysis of Experiments, Volume 1: Introduction to Experimental Design HINKELMANN and KEMPTHORNE . Design and Analysis of Experiments, Volume 2: Advanced Experimental Design HOAGLM, MOSTELLER, and TUKEY . Exploratory Approach to Analysis of Variance HOAGLIN, MOSTELLER, and TUKEY . Exploring Data Tables, Trends and Shapes * HOAGLIN, MOSTELLER, and TUKEY . Understanding Robust and Exploratory Data Analysis HOCHBERG and TAMHANE . Multiple Comparison Procedures HOCKNG Methods and Applications of Linear Models: Regression and the Analysis of Variance, Second Edition HOEL . Introduction to Mathematical Statistics, Fifih Edition HOGG and KLUGMAN . Loss Distributions HOLLANDER and WOLFE . Nonparamemc Statistical Methods, Second Edition HOSMER and LEMESHOW . Applied Logistic Regression, Second Edition HOSMER and LEMESHOW . Applied Survival Analysis: Regression Modeling of Time to Event Data f HUBER . Robust Statistics HUBERTY . Applied Discriminant Analysis HUNT and KENNEDY . Financial Derivatives in Theory and Practice HUSKOVA, BERAN, and DUPAC ' Collected Works of Jaroslav Hajekwith Commentary HUZURBAZAR . Flowgraph Models for Multistate Time-to-Event Data IMAN and CONOVER . A Modem Approach to Statistics JACKSON . A User's Guide to Principle Components JOHN . Statistical Methods in Engineering and Quality Assurance JOHNSON . Multivariate Statistical Simulation JOHNSON and BALAKRISHNAN . Advances in the Theory and Practice of Statistics: A Volume in Honor of Samuel Kotz JOHNSON and BHATTACHARYYA . Statistics: Principles and Methods, FzPh Edition *Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-Interscience Paperback Series.

*

JOHNSON and KOTZ . Distributions in Statistics JOHNSON and KOTZ (editors) . Leading Personalities in Statistical Sciences: From the Seventeenth Century to the Present JOHNSON, KOTZ, and BALAKRISHNAN . Continuous Univanate Distributions, Volume 1, Second Edition JOHNSON, KOTZ. and BALAKRISHNAN . Continuous Univanate Dlstributions, Volume 2, Second Edition JOHNSON, KOTZ. and BALAKRISHNAN . Discrete Multivariate Disrnbutions JOHNSON, KEMP. and KOTZ Univariate Discrete Distributions, Third Edition JUDGE, GRIFFITHS, HILL, LUTKEPOHL, and LEE The Theory and Practice of Ecpometrics, Second Edition JURECKOVA and SEN Robust Statistical Procedures: Aymptotics and Interrelations JUREK and MASON . Operator-Limit Distributions in Probability Theory KADANE . Bayesian Methods and Ethics in a Clinical Trial Design KADANE AND SCHUM . A Probabilistic Analysis of the Sacco and Vanzetti Evidence KALBFLEISCH and PRENTICE . The Statistical Analysis of Failure Time Data, Second Edition KASS and VOS . Geometrical Foundations of Asymptotic Inference KAUFMAN and ROUSSEEUW Finding Groups in Data: An Introduction to Cluster Analysis KEDEM and FOKIANOS . Regression Models for Time Series Analysis KENDALL, BARDEK, CARNE, and LE . Shape and Shape Theory KHURI Advanced Calculus with Applications in Statistics. Second Edition KHURI, MATHEW, and SINHA ' Statistical Tests for Mixed Linear Models KISH ' Statistical Design for Research KLEIBER and KOTZ Statistical Size Distributions in Economics and Actuarial Sciences KLUGMAN, I'ANJER, and WILLMOT . Loss Models: From Data to Decisions, Second Edition KLUGMAN, PANJER, and WILLMOT. Solutions Manual to Accompany Loss Models: From Data to Decisions, Second Edition KOTZ. BALAKRISHNAN, and JOHNSON Continuous Multivariate Distributions, Volume I , Second Edition KOTZ and JOHNSON (editors) ' Encyclopedia of Statistical Sciences: Volumes 1 to 9 with Index KOTZ and JOHNSON (editors) ' Encyclopedia of Statistical Sciences: Supplement Volume KOTZ. READ, and BANKS (editors) ' Encyclopedia of Statistical Sciences: Update Volume 1 KOTZ. READ, and BANKS (editors) . Encyclopedia of Statistical Sciences: Update Volume 2 KOVALENKO, KUZNETZOV, and PEGG Mathematical Theory of Reliability of Time-Dependent Systems with Practical Applications LACHIN Biostatistical Methods: The Assessment of Relative Risks LAD . Operational Subjective Statistical Methods: A Mathematical, Philosophical, and Historical Introduction LAMPERTI . Probability: A Survey ofthe Mathematical Theory, Second Edition LANGE, RYAN, BILLARD, BRILLINGER, CONQUEST, and GREENHOUSE ' Case Studies in Biometry LARSON . Introduction to Probability Theory and Statistical Inference. Third Edition LAWLESS ' Statistical Models and Methods for Lifetime Data, Second Edition LAWSON ' Statistical Methods in Spatial Epidemiology LE . Applied Categorical Data Analysis LE . Applied Survival Analysis

"Now available i i i a lower priced paperback edition in the Wiley Classics Library. available i n a lower priced paperback edition in the Wiley-lntersciencc Paperback Series.

tYow

LEE and WANG Statistical Methods for Survival Data Analysis, Third Edition LEPAGE and BILLARD Exploring the Limits of Bootstrap LEYLAND and GOLDSTEIN (editors) . Multilevel Modelling of Health Statistics LIAO . Statistical Group Comparison LINDVALL . Lectures on the Coupling Method LINHART and ZUCCHINI Model Selection LITTLE and RUBrli . Statistical Analysis with Missing Data, Second Edition LLOYD . The Statistical Analysis of Categorical Data LOWEN and TEICH ' Fractal-Based Point Processes MAGNUS and NEUDECKER Matrix Differential Calculus with Applications in Statistics and Econometrics, Revised Edition MALLER and ZHOU . Survival Analysis with Long Term Survivors MALLOWS . Design, Data, and Analysis by Some Friends of Cuthbert Daniel MANN, SCHAFER, and SINGPURWALLA . Methods for Statistical Analysis of Reliability and Life Data MANTON, WOODBURY, and TOLLEY . Statistical Applications Using Fuzzy Sets MARCHETTE . Random Graphs for Statistical Pattern Recognition MARDIA and JUPP . Directional Statistics MASON, GUNST, and HESS . Statistical Design and Analysis of Experiments with Applications to Engineering and Science, Second Edition McCULLOCH and SEARLE . Generalized, Linear, and Mixed Models McFADDEN ' Management of Data in Clinical Trials * McLACHLAN . Discriminant Analysis and Statistical Pattern Recognition McLACHLAN, DO, and AMBROISE . Analyzing Microarray Gene Expression Data McLACHLAN and KRISHNAN The EM Algorithm'and Extensions McLACHLAN and PEEL Finite Mixture Models McNEIL Epidemiological Research Methods MEEKER and ESCOBAR ' Statistical Methods for Reliability Data MEERSCHAERT and SCHEFFLER . Limit Distributions for Sums of Independent Random Vectors: l l e a w Tails in Theory and Practice MICKEY, D U " , and CLARK . Applied Statistics: Analysis of Variance ano Regression, Third Edition * MILLER . Survival Analysis, Second Edition MONTGOMERY, PECK, and VINING Introduction to Linear Regression Aralysis, Third Edition MORGENTHALER and TUKEY ' Configural Polysampling: A Route to Practical Robustness MUIRHEAD . Aspects of Multivariate Statistical Theory MULLER and STOY AN . Comparison Methods for Stochastic Models and Risks MURRAY . X-STAT 2.0 Statistical Experimentation, Design Data Analysis, and Nonlinear Optimization MURTHY, XIE, and JIANG Weibull Models MYERS and MONTGOMERY . Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Second Edition MYERS, MONTGOMERY, and VINING . Generalized Linear Models. With Applications in Engineering and the Sciences NELSON . Accelerated Testing, Statistical Models, Test Plans, and Data Analyses t NELSON . Applied Life Data Analysis NEWMAN ' Biostatistical Methods in Epidemiology OCHI . Applied Probability and Stochastic Processes in Engineering and Physical Sciences OKABE, BOOTS, SUGIHARA. and CHIU . Spatial Tesselations: Concepts ar;d Applications of Voronoi Diagrams, Second Edition OLIVER and SMITH ' Influence Diagrams, Belief Nets and Decision Analysir

'

*Now

available in a lower priced paperback edition in the Wiley Classics Library.

'Now available in a lower pnced paperback edition in the Wiley-Interscience Paperback Series.

PALTA . Quantitative Methods in Population Health: Extensions of Ordinary Regressions PANKRATZ . Forecasting with Dynamic Regression Models PANKRATZ . Forecasting with Univariate Box-Jenkins Models: Concepts and Cases * P A F E N . Modem Probability Theory and Its Applications PENA, TIAO, and TSAY .A Course in Time Series Analysis PIANTADOSI . Clinical Trials: A Methodologic Perspective PORT . Theoretical Probability for Applications POURAHMADI . Foundations of Time Series Analysis and Prediction Theory PRESS . Bayesian Statistics: Principles, Models, and Applications PRESS . Subjective and Objective Bayesian Statistics, Second Edition PRESS and TANUR . The Subjectivity of Scientists and the Bayesian Approach PUKELSHEIM . Optimal Experimental Design PURI, VILAPLANA, and WERTZ . New Perspectives in Theoretical and Applied Statistics t PUTERMAN . Markov Decision Processes: Discrete Stochastic Dynamic Programming QIU . Image Processing and Jump Regression Analysis * RAO . Linear Statistical Inference and Its Applications, Second Edition RAUSAND and H0YLAND . System Reliability Theory: Models, Statistical Methods, and Applications, Second Edition RENCHER . Linear Models in Statistics RENCHER . Methods of Multivariate Analysis, Second Edition RENCHER . Multivariate Statistical Inference with Applications * RIPLEY . Spatial Statistics RIPLEY . Stochastic Simulation ROBINSON . Practical Strategies for Experimenting ROHATGI and SALEH . An Introduction to Probability and Statistics, Second Edition ROLSKI, SCHMIDLI, SCHMIDT, and TEUGELS . Stochastic Processes for Insurance and Finance ROSENBERGER and LACHIN . Randomization in Clinical Trials: Theory and Practice ROSS . Introduction to Probability and Statistics for Engineers and Scientists t ROUSSEEUW and LEROY . Robust Regression and Outlier Detection * RUBIN . Multiple Imputation for Nonresponse in Surveys RUBINSTEIN . Simulation and the Monte Carlo Method RUBINSTEIN and MELAMED . Modem Simulation and Modeling RYAN . Modem Regression Methods RYAN . Statistical Methods for Quality Improvement, Second Edition SALEH . Theory of Preliminary Test and Stein-Type Estimation with Applications * SCHEFFE . The Analysis of Variance SCHIMEK . Smoothing and Regression: Approaches, Computation, and Application SCHOTT . Mamx Analysis for Statistics, Second Edition SCHOUTENS . Levy Processes in Finance: Pricing Financial Derivatives SCHUSS . Theory and Applications of Stochastic Differential Equations SCOTT . Multivariate Density Estimation: Theory, Practice, and Visualization SEARLE . Linear Models for Unbalanced Data SEARLE . Matrix Algebra Useful for Statistics SEARLE, CASELLA, and McCULLOCH . Variance Components SEARLE and WILLETT . Matrix Algebra for Applied Economics SEBER and LEE . Linear Regression Analysis, Second Edition 7 SEBER . Multivariate Observations 7 SEBER and WILD . Nonlinear Regression SENNOTT . Stochastic Dynamic Programming and the Control of Queueing Systems * SERFLING . Approximation Theorems of Mathematical Statistics SHAFER and VOVK . Probability and Finance: It’s Only a Game! *Now available in a lower priced paperback edition in the Wiley Classics Library. ?Now available in a lower priced paperback edition in the Wiley-Interscience Paperback Series.

*

SILVAPULLE and SEN . Constrained Statistical Inference: Inequality, Order, and Shape Restrictions SMALL and McLEISH . Hilbert Space Methods in Probability and Statistical Inference SRIVASTAVA . Methods of Multivariate Statistics STAPLETON . Linear Statistical Models STAUDTE and SHEATHER . Robust Estimation and Testing STOYAN, KENDALL, and MECKE . Stochastic Geometry and Its Applications, Second Edition STOYAN and STOYAN . Fractals, Random Shapes and Point Fields: Methods of Geometrical Statistics STYAN . The Collected Papers ofT. W. Anderson: 1943-1985 SUTTON, ABRAMS, JONES, SHELDON, and SONG . Methods for Meta-Analysis in Medical Research TAKEZAWA . Introduction to Nonparametric Regression TANAKA . Time Series Analysis: Nonstationary and Noninvertible Dismbution Theory THOMPSON . Empirical Model Building THOMPSON . Sampling, Second Edition THOMPSON . Simulation: A Modeler’s Approach THOMPSON and SEBER . Adaptive Sampling THOMPSON, WILLIAMS, and FINDLAY . Models for Investors in Real World Markets TIAO, BISGAARD, HILL, PESA, and STIGLER (editors) . Box on Quality and Discovery: with Design, Control, and Robustness TIERNEY . LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics TSAY . Analysis of Financial Time Series, Second Edition UPTON and FINGLETON . Spatial Data Analysis by Example, Volume 11: Categorical and Directional Data VAN BELLE . Statistical Rules of Thumb VAN BELLE, FISHER, HEAGERTY, and LUMLEY . Biostatistics: A Methodology for the Health Sciences, Second Edition VESTRUP . The Theory of Measures and Integration VIDAKOVIC . Statistical Modeling by Wavelets VINOD and REAGLE . Preparing for the Worst: Incorporating Downside Risk in Stock Market Investments WALLER and GOTWAY . Applied Spatial Statistics for Public Health Data WEERAHANDI . Generalized Inference in Repeated Measures: Exact Methods in MANOVA and Mixed Models WEISBERG . Applied Linear Regression, Third Edition WELSH . Aspects of Statistical Inference WESTFALL and YOUNG . Resampling-Based Multiple Testing: Examples and Methods for p-Value Adjustment WHITTAKER . Graphical Models in Applied Multivariate Statistics WINKER . Optimization Heuristics in Economics: Applications of Threshold Accepting WONNACOTT and WONNACOTT . Econometrics, Second Edition WOODING . Planning Pharmaceutical Clinical Trials: Basic Statistical Principles WOODWORTH . Biostatistics: A Bayesian Introduction WOOLSON and CLARKE . Statistical Methods for the Analysis of Biomedical Data, Second Edition WU and HAMADA . Experiments: Planning, Analysis, and Parameter Design Optimization YANG . The Construction Theory of Denumerable Markov Processes ZELLNER . An Introduction to Bayesian Inference in Econometrics ZHOU, OBUCHOWSKI, and McCLISH .Statistical Methods in Diagnostic Medicine


Theory of Preliminary Test and Stein-Type Estimation with Applications

Nonparametric curve estimation: methods, theory and applications

Nonparametric Curve Estimation: Methods, Theory and Applications

Nonparametric Curve Estimation Methods Theory and Applications

Nonparametric Curve Estimation: Methods, Theory and Applications

Theory of Point Estimation

Theory of Point Estimation

Theory of Point Estimation

Theory of Point Estimation

Theory of Vibration with Applications

Estimation with applications to tracking navigation

Nonlinear Filters: Estimation and Applications

Bargaining Theory with Applications

Graph Theory with Applications

Extrapolation Theory With Applications

Optimization theory with applications

Bargaining theory with applications

Bargaining Theory with Applications

Optimization theory with applications

Optimization Theory with Applications

Graph theory with applications

Cost of Capital: Estimation and Applications

Financial Risk Management with Bayesian Estimation of GARCH Models Theory and Applications (Springer)

Discrete inverse and state estimation problems: with geophysical fluid applications

Automata theory with modern applications

Automata Theory with Modern Applications

Simulated Annealing, Theory with Applications

Chemical Thermodynamics and Information Theory with Applications

Probability Theory and Stochastic Processes with Applications

Chemical Thermodynamics and Information Theory with Applications

Microeconomics: Theory and Applications with Calculus

Theory of Preliminary Test and Stein-Type Estimation with Applications

Nonparametric curve estimation: methods, theory and applications

Nonparametric Curve Estimation: Methods, Theory and Applications

Nonparametric Curve Estimation Methods Theory and Applications

Nonparametric Curve Estimation: Methods, Theory and Applications

Theory of Point Estimation

Theory of Point Estimation

Theory of Point Estimation

Theory of Point Estimation

Theory of Vibration with Applications

Estimation with applications to tracking navigation

Nonlinear Filters: Estimation and Applications

Bargaining Theory with Applications

Graph Theory with Applications

Extrapolation Theory With Applications

Optimization theory with applications

Bargaining theory with applications

Bargaining Theory with Applications

Optimization theory with applications

Optimization Theory with Applications

Graph theory with applications

Cost of Capital: Estimation and Applications

Financial Risk Management with Bayesian Estimation of GARCH Models Theory and Applications (Springer)

Discrete inverse and state estimation problems: with geophysical fluid applications

Automata theory with modern applications

Automata Theory with Modern Applications

Simulated Annealing, Theory with Applications

Chemical Thermodynamics and Information Theory with Applications

Probability Theory and Stochastic Processes with Applications

Chemical Thermodynamics and Information Theory with Applications

Microeconomics: Theory and Applications with Calculus

Recommend Documents