Encyclopedia of Statistical Sciences

ENCYCLOPEDIA OF STATISTICAL SCIENCES Volume 15 Wagr Test to Zyskind–Martin Models ENCYCLOPEDIA OF STATISTICAL SCIENCES...

Author: Samuel Kotz

281 downloads 2676 Views 11MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

ENCYCLOPEDIA OF STATISTICAL SCIENCES Volume 15 Wagr Test to Zyskind–Martin Models

ENCYCLOPEDIA OF STATISTICAL SCIENCES Second Edition

Volume 15 Wagr Test to Zyskind–Martin Models

The Encyclopedia of Statistical Sciences is available Online at http://www.mrw.interscience.wiley.com/ess

A John Wiley & Sons, Inc., Publication

CONTRIBUTORS Charles E. Antle, Pennsylvania State University, University Park, PA. Weibull Distribution David Appleton, University of Newcastle, Newcastle upon Tyne, United Kingdom. Woolf’s Test Steven F. Arnold, Pennsylvania State University, University Park, PA. Wishart Distribution Lee J. Bain, University of Missouri, Rolla, MO. Weibull Distribution K. S. Banerjee, University of Maryland, Baltimore, MD. Weighing Designs David A. Belsley, Boston College, Chestnut Hill, MA. Weak Data R. J. Beran, University of California, Berkeley, CA. Weak Convergence, Statistical Applications of R. J. Bhansali, University of Liverpool, Liverpool, United Kingdom. Wiener–Kolmogorov Prediction Theory Lennart Bondesson, Swedish University of Agricultural Sciences, Umea, Sweden. Wargentin, Pehr Wilhelm Ralph A. Bradley, University of Georgia, Athens, GA. Wilcoxon, Frank Kenneth G. Brown, Chapel Hill, NC. Williams’ Test of Trend William P. Cleveland, Federal Reserve Board, Washington, DC. X-11 Method D. R. Cox, Imperial College, London, United Kingdom. Yoke Morris H. DeGroot, Carnegie Mellon University, Pittsburgh, PA. Well-Calibrated Forecasts A. S. Dennis, Bureau of Reclamation, US Department of Interior, Boulder, CO. Weather Modification—II Max Eengelhardt, University of Missouri, Rolla, MO. Weibull Processes J. A. Flueck, Environmental Sciences Group, National Oceanic and Atmospheric Administration, Boulder, CO. Weather Modification—II

K. R. Gabriel, University of Rochester, Rochester, NY. Weather Modification—II; Weather Modification—I Christopher R. Genovese, Carnegie Mellon University, Pittsburgh, PA. Wavelets Jean Dickinson Gibbons, University of Alabama, Tuscaloosa, AL. Wilcoxon-Type Scale Tests William S. Griffith, University of Kentucky, Lexington, KY. Wear Processes E. J. Hannan, Australian National University, Canberra, Australia. Wiener, Norbert H. Hauptman, Medical Foundation of Buffalo, Buffalo, NY. X-Ray Crystallography Thomas P. Hettmansperger, Pennsylvania State University, University Park, PA. Walsh Averages; Weighted Least-Squares Rank Estimators W. G. S. Hines, University of Guelph, Guelph, ON, Canada. Wandering-Quarter Sampling Myles Hollander, Florida State University, Tallahassee, FL. Wilcoxon, Frank; Wilcoxon-Type Tests for Ordered Alternatives in Randomized Blocks H. K. Hsieh, University of Massachusetts, Amherst, MA. Weibull Populations, Schafer–Sheffield Comparison Test for J. P. Imhof, University of Geneva, Geneva, Switzerland. Wiener Measure Iain Johnstone, Stanford University, Stanford, CA. Wald’s Decision Theory A. H. Kroese, Agricultural Mathematics Group, Wageningen, The Netherlands. Weather Forecasting, Brier Score in; Weather Forecasting, Epstein Scoring Rule in Anant M. Kshirsagar, University of Michigan, Ann Arbor, MI. Wilks’s Lambda Criterion M. Eileen Magnello, The Wellcome Institute, London, United Kingdom. Weldon, Walter Frank Raphael v

vi

CONTRIBUTORS

Nancy R. Mann, University of California, Los Angeles, CA. Weibull Distribution, Mann–Fertig Test Statistic for J. S. Marron, University of North Carolina, Chapel Hill, NC. Window Width P. W. Mielke, Colorado State University, Fort Collins, CO. Weather Modification—II Richard Mojena, University of Rhode Island, Kingston, Rhode Island. Ward’s Clustering Algorithm Robb J. Muirhead, University of Michigan, Ann Arbor, MI. Zonal Polynomials John Nelder, Imperial College, London, United Kingdom. Yates, Frank G. P. Patil, Pennsylvania State University, University Park, PA. Weighted Distributions Yudi Pawitan, University College, Dublin, Ireland. Whittle Likelihood Ronald H. Randles, Florida State University, Gainesville, FL. Weighted Symmetry; Wilcoxon Signed Rank Test C. R. Rao, University of Pittsburgh, Pittsburgh, PA. Weighted Distributions Campbell B. Read, Southern Methodist University, Dallas, TX. Zipf’s Law; Wald’s Equation; Weighted Least Squares Martin G. Ribe, Statistics Sweden, Stockholm, Sweden. Wargentin, Pehr Wilhelm Arthur J. Roth, Ciba-Geigy Pharmaceuticals, Summit, NJ. Welch Tests Louise M. Ryan, Harvard School of Public Health, Boston, MA. Weighted Normal Plots W. Schaafsma, University of Groningen, Groningen, The Netherlands. Weather Forecasting, Brier Score in; Weather Forecasting, Epstein Scoring Rule in

A. R. Sen, University of Calgary, Calgary, AB, Canada. Wildlife Sampling P. K. Sen, University of North Carolina, Chapel Hill, NC. Weighted Empirical Processes, Genesis and Applications of E. Seneta, University of Sydney, Sydney, Australia. Yanson (Jahnson), Yulii Eduardovich M. Sibuya, Takachiho University, Tokyo, Japan. Wicksell’s Corpuscle Problem Andrew F. Siegel, University of Washington, Seattle, Washington. Zero Degrees of Freedom R. D. Snee, E. I. du Pont de Nemours & Co. Inc., Wilmington, DE. Window Plot Ralph G. Stanton, University of Manitoba, Winnipeg, MB, Canada. Witt Designs M. A. Stephens, Simon Fraser University, Burnaby, BC, Canada. Watson’s U 2 Stephen M. Stigler, University of Chicago, Chicago, IL. Wilson, Edwin Bidwell Murad S. Taqqu, Cornell University, Ithaca, NY. Weak Stationarity G. S. Watson, Princeton University, Princeton, NJ. Wheeler and Watson’s Test G. Weiss, National Institutes of Health, Bethesda, MD. Wald’s Identity, Applications of L. Weiss, Cornell University, Ithaca, NY. Weiss Test of Independence; Weiss-Type Estimators of Shape Parameters; Wolfowitz, Jacob; Wald, Abraham Svante Wold, Umea University, Umea, Sweden. Wold, Herman O. A. Marvin Zelen, Harvard School of Public Health, Boston, MA. Weighted Distributions

W WAGR TEST

where Pr[X x(F)] = F.

This is another name for the sequential t-test (see SEQUENTIAL ANALYSIS). The initials stand for Wald, Anscombe, Girshick, and Rushton, four statisticians who were instrumental in the formulation of the test.

For Wakeby distributions, x(F) = ξ + θβ −1 {1 − (1 − F)β } −γ δ −1 {1 − (1 − F)δ }.

See also SEQUENTIAL T 2 TEST.

(1)

For β = 0(δ = 0) the second (third) term on the right-hand side of (1) is taken as log(1 − F). The five parameters ξ , θ , β, γ , δ are constrained by the conditions:

´ WAITING TIMES. See TAKACS PROCESS

(a) (b) (c) (d)

WAKEBY DISTRIBUTIONS These distributions were defined by H. A. Thomas in 1976. He gave them the name Wakeby distributions because he conceived them at his summer home overlooking Wakeby Pond in Cape Cod, Massachusetts. According to Hosking [1], the widespread and successful use of Wakeby distributions in hydrology is due to certain general properties of the distributions.

γ 0, θ + γ 0. Either β + δ > 0 or β + γ = δ = 0. If θ = 0, then β = 0. If γ = 0, then δ = 0.

For δ > 0 and γ > 0, as x → ∞, F(x) ∼ 1 − (δx/γ )−1/δ ,

f (x) = F (x) ∼ γ 1/δ (δx)−1−1/δ The upper tail of the distribution behaves like that of a Pareto distribution∗ . The moments of all orders exist provided δ 0. If δ is positive, E[X r ] exists for 0 r < δ −1 . When the rth moment exists,

(i) By suitable choice of parameter values, it is possible to mimic the extremevalue∗ , log-normal∗ , and log-gamma distributions. (ii) There are five parameters, more than most of the common systems of distributions. This allows for a wider variety of shapes. (iii) The distributions have finite lower bounds, which is physically reasonable for many hydrological observations. (iv) Some Wakeby distributions have heavy upper tails and can give rise to occasional high outliers∗ , a phenomenon often observed in hydrology. (See Makalas et al. [4].) (v) The form of the distribution (see below) is well suited to simulation∗ . (See also Houghton [2].)

E[{x − (ξ + θβ −1 − γ δ −1 )}r ] r r (θ/β)i (γ /δ)r−j (−1)r−j = j j=0

×{1 − rδ + j(β + δ)}−1 . The probability moment αr = E[X{1 − F(X)}r ] has the simple expression (r + 1)−1 {ξ + θ (r + β + 1)−1 + γ (r − δ + 1)−1 },

r 0.

(Note that α0 = ξ + θ (β + 1)−1 + γ (−δ + 1)−1 = E[X].)

The distributions are defined in terms of the quantile∗ function (inverse CDF) x(F), 9017

9018

WALD, ABRAHAM

Given an ordered random sample of n values x1 · · · xn , the value of αr is unbiasedly estimated by ar = n−1

n n−j n−1 xj . r r j=1

See Landwehr et al. [3], who recommend the biased, but more stable, estimator a˜ r = n−1

n

{1 − n−1 (j − 0.35)}r xj .

j=1

The estimators ar and a˜ r are asymptotically equivalent. Hosking [1] has shown that as n → ∞ the joint distribution of {n1/2 (ar − αr ), r = 0, 1, . . . , m − 1}—and so also that of {n1/2 (a˜ r − αr ), r = 0, 1, . . . , m − 1}—is multinormal with zero mean. REFERENCES 1. Hosking, J. R. M. (1986). The Wakeby Distribution. IBM Research Report RC 12302, IBM, Yorktown Heights, NY. 2. Houghton, J. C. (1978). Water Resour. Res., 14, 1105–1109. 3. Landwehr, J. M., Matalas, N. C., and Wallis, J. R. (1979). Water Resour. Res., 15, 1055–1064, 1361–1379. 4. Matalas, N. C., Slack, J. R., and Wallis, J. R. (1975). Water Resour. Res., 11, 815–826. See also EXTREME-VALUE DISTRIBUTIONS; LAMBDA DISTRIBUTIONS; and PARETO DISTRIBUTION.

WALD, ABRAHAM Born: October 31, 1902, in Cluj, Hungary (now Romania). Died: December 13, 1950, in Travancore, India. Contributed to: decision theory, sequential analysis, geometry, econometrics. Abraham Wald was born in Cluj on October 31, 1902. At that time, Cluj belonged to Hungary, but after World War I it belonged to Romania. Menger [2] states that when he first met Wald in Vienna in 1927, Wald spoke ‘‘fluent German, but with an unmistakable

Hungarian accent.’’ A short time after Wald emigrated to the United States, he spoke fluent English, with an accent. Wald would not attend school on Saturday, the Jewish sabbath, and as a result he did not attend primary or secondary school, but was educated at home by his family. On the basis of this education, he was admitted to and graduated from the local university. He entered the University of Vienna in 1927 and received his Ph.D. in mathematics in 1931. Wald’s first research interest was in geometry, and he published 21 papers in that area between 1931 and 1937. Two later papers, published in 1943 and 1944, were on a statistical generalization of metric spaces, and are described in [4]. A discussion of Wald’s research in geometry by Menger [2] describes it as deep, beautiful, and of fundamental importance. During the 1930s, economic and political conditions in Vienna made it impossible for Wald to obtain an academic position there. To support himself, he obtained a position as tutor in mathematics to Karl Schlesinger, a prominent Viennese banker and economist. As a result of this association, Wald became interested in economics and econometrics∗ , and published 10 papers on those subjects, plus a monograph [6] on seasonal movements in time series∗ . Morgenstern [3] describes this monograph as developing techniques superior to all others. Wald’s first exposure to statistical ideas was a result of his research in econometrics. This research in economics and econometrics is described by Morgenstern [3] and Tintner [5]. Once again, his contributions to these areas are characterized as of fundamental importance. Austria was seized by the Nazis early in 1938, and Wald came to the United States in the summer of that year, as a fellow of the Cowles Commission. This invitation from the Cowles Commission probably saved Wald’s life, for almost all of the members of his family in Europe were murdered by the Nazis. In the fall of 1938, Wald became a fellow of the Carnegie Corporation and started to study statistics at Columbia University with Harold Hotelling∗ . Wald stayed at Columbia as a fellow of the Carnegie Corporation until 1941, lecturing during the academic year 1939–1940. In 1941, he joined the Columbia

WALD, ABRAHAM

faculty and remained there for the rest of his life. During the war years, he was also a member of the Statistics Research Group at Columbia, doing war-related research. The techniques he developed for estimating aircraft vulnerability are still used and are described in a reprint [16] published in 1980. See also MILITARY STATISTICS. Wald’s first papers in statistics were published in 1939, and one of them [7] is certainly one of his most important contributions to statistical theory. J. Wolfowitz, who became a close friend and collaborator of Wald soon after Wald arrived in New York, in [22] describes this 1939 paper by Wald as probably Wald’s most important single paper. In this paper, Wald points out that the two major problems of statistical theory at that time, testing hypotheses and estimation, can both be regarded as simple special cases of a more general problem—known nowadays as a ‘‘statistical decision problem.’’ This generalization seems quite natural once it is pointed out, and the wonder is that nobody had thought of it before. Perhaps it needed a talented person with a fresh outlook to see it. Wald does much more than merely point out the generalization in this paper. He defines loss functions, risk functions∗ , a priori distributions, Bayes decision rules, admissible decision∗ rules, and minimax decision rules, and proves that a minimax decision rule has a constant risk under certain regularity conditions. It is interesting that in this paper Wald states that the reason for introducing a priori distributions on the unknown parameters is that it is useful in deducing certain theorems on admissibility∗ and in the construction of minimax decision rules: That is, he is not considering the unknown parameters to be random variables. This paper did not receive much attention at first, but many other papers, by Wald and others, have extended statistical decision theory∗ . Wald’s 1950 book [13] contains most of the results developed up to that year. This book is accessible only to those with a strong background in mathematics. The 1952 paper [14] summarizes the basic ideas and is more easily read. Wald’s other great contribution to statistical theory is the construction of optimal statistical procedures when sequential sampling is permitted. (Sequential sampling is any

9019

sampling scheme in which the total number of observations taken is a random variable.) Unlike statistical decision theory, the concept of sequential sampling is not due to Wald and is not included in the 1939 paper on decision theory. Just who first thought of sequential sampling is apparently not known. In [11] it is stated that Captain G. L. Schuyler of the U.S. Navy made some comments that alerted M. Friedman and W. Allen Wallis to the possible advantages of sequential sampling, and Friedman and Wallis proposed the problem to Wald in March 1943. Wald’s great contribution to sequential analysis was in finding optimal sequential procedures. He started by considering the problem of testing a simple hypothesis against a simple alternative using sequential sampling and conjectured that the subsequently famous sequential probability ratio test is optimal for this problem, in the sense that among all test procedures with preassigned upper bounds on the probabilities of making the wrong decision, the sequential probability ratio test minimizes the expected sample size under both the hypothesis and the alternative. Early in his investigation, he was able to show that this conjecture is at least approximately true, but it was not until his 1948 paper with Wolfowitz [21] that a proof that it is exactly true was given. This proof was based on the ideas that Wald had developed in his work on statistical decision theory and thus united his two major contributions to statistical theory. The 1947 book [11] describes the results on sequential analysis known up to that time in an elementary manner and is accessible to anyone with a knowledge of elementary probability theory. The 1950 book [13] incorporates sequential sampling into statistical decision theory. Besides statistical decision theory and sequential analysis, Wald made many other fundamental contributions to statistical theory, some of which will be described briefly. In [9] he derived the large-sample distribution of the likelihood ratio test∗ under alternatives to the hypothesis being tested and proved the asymptotic optimality of the test. In collaboration with Wolfowitz, he made fundamental contributions to nonparametric statistical inference in [17–20]. He wrote a pioneering paper on the optimal design of

9020

WALD—BARTLETT SLOPE ESTIMATION

experiments [8], a field which became very active a few years later. In [10] Wald generalized a theorem of von Neumann on zerosum two-person games. In collaboration with Mann [1], he developed statistical inference for stochastic difference equations. [12] contains a new proof of consistency of maximum likelihood∗ estimators. Wald was an excellent teacher, always precise and clear. He was a master at deriving complicated results in amazingly simple ways. His influence on the teaching of statistics extended far beyond the students who actually attended his classes, because, with his permission, the Columbia students reproduced the notes they took in his classes. These reproduced notes were only supposed to be circulated to other Columbia students, but they had a much wider circulation than that. In late 1950, Wald was in India, at the invitation of the Indian Government, lecturing on statistics. He was accompanied by his wife. On December 13, 1950, both were killed in in a plane crash. A fuller account of Wald’s research and a list of 104 of his publications are contained in [15]. This list is complete except for the 1980 reprint [16].

12. Wald, A. (1949). Ann. Math. Statist., 20, 595–601. 13. Wald, A. (1950). Statistical Decision Functions. Wiley, New York. 14. Wald, A. (1952). Proc. Intern. Congress of Mathematicians. Harvard University Press, Cambridge, MA. 15. Wald, A. (1955). Selected Papers in Statistics and Probability. McGraw-Hill, New York. 16. Wald, A. (1980). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Center for Naval Analyses, Washington, D.C. 17. Wald, A. and Wolfowitz, J. (1939). Ann. Math. Statist., 10, 105–118. 18. Wald, A. and Wolfowitz, J. (1940). Ann. Math. Statist., 11, 147–162. 19. Wald, A. and Wolfowitz, J. (1943). Ann. Math. Statist., 14, 378–388. 20. Wald, A. and Wolfowitz, J. (1944). Ann. Math. Statist., 15, 358–372. 21. Wald, A. and Wolfowitz, J. (1948). Ann. Math. Statist., 19, 326–339. 22. Wolfowitz, J. (1952). Ann. Math. Statist., 23, 1–13. See also DECISION THEORY; GAME THEORY; SEQUENTIAL ANALYSIS; and WALD’S DECISION THEORY.

L. WEISS

REFERENCES 1. Mann, H. B. and Wald, A. (1943). Econometrica, 11, 173–220. 2. Menger, K. (1952). Ann. Math. Statist., 23, 14–20. 3. Morgenstern, O. (1951). Econometrica, 19, 361–367. 4. Schweizer, B. and Sklar, A. (1983). Probabilistic Metric Spaces. North-Holland, New York. 5. Tintner, G. (1952). Ann. Math. Statist., 23, 21–28. 6. Wald, A. (1936). Berechnung und Ausschaltung von Saisonschwankungen. Springer, Vienna. 7. Wald, A. (1939). Ann. Math. Statist., 10, 299–326. 8. Wald, A. (1943). Ann. Math. Statist., 14, 134–140. 9. Wald, A. (1943). Trans. Amer. Math. Soc., 54, 426–482. 10. Wald, A. (1945). Ann. Math., 46, 281–286. 11. Wald, A. (1947). Sequential Analysis. Wiley, New York.

WALD—BARTLETT SLOPE ESTIMATION Consider a situation in which variables Xi and Yi are related by the equation Yi = α + βXi ,

i = 1, . . . , N,

with both variables subject to random measurement errors∗ i and ηi , respectively. It can be assumed that E[ i ] = E[ηi ] = 0. It is known that if simple least-squares∗ procedures are used to estimate β, measurement errors in xi will attenuate the estimate. Wald [11] derived an estimator of β calculated from measured values xi = Xi + i and yi = Yi + ηi that is consistent provided that the following assumptions have been met: (i) The random variables i , . . . , N each have the same distribution and they are uncorrelated, that is, E[ i j ] = 0 for i = j. The variance of i is finite.

WALD’S DECISION THEORY

(ii) The random variables ηi , . . . , ηN each have the same distribution and they are uncorrelated, that is, E[ηi ηj ] = 0 for i = j. The variance of ηi is finite. (iii) The random variables i and ηj , i = 1, 2, . . . , N; j = 1, 2, . . . , N, are uncorrelated, that is, E[ i ηj ] = 0. (iv) A single linear relation holds between the true values Xi and Yi , that is to say, Yi = α + βXi , i = 1, 2, . . . , N. (v) The limit inferior of {(X1 + · · · + Xm ) − (Xm+1 + · · · + XN )}N −1 (N = 2, 4, . . ., ∞), is positive, where the total number of cases N is even and m = N/2. Wald’s estimator b is formed as follows: b= =

y1 − y2 x1 − x2 (y1 + y2 + · · · + ym ) − (ym+1 + · · · + yN ) , (x1 + x2 + · · · + xm ) − (xm+1 + · · · + xN )

where there are two groups G1 and G2 with means x1 and y1 and x2 and y2 , respectively. The elements are ordered in increasing magnitude on X. G1 contains half of the observations (1, 2, . . . , m) and G2 contains the remainder (m + 1, m + 2, . . . , N). In addition, it is assumed that the grouping is unrelated to the error terms (which essentially implies that measurement errors in xi are negligible). We then face the practical problem of finding a method of grouping the scores is independent of the error terms i and ηi . Bartlett [1] modified Wald’s procedure and suggested dividing the distribution ordered on xi into thirds. The smallest third becomes G1 and the largest G2 . Additional related modifications into quarters (the smallest quarter being G1 and the largest G2 ) and an iteration that requires calculations of two slope estimates were developed by Nair and Banerjee [8] and Lindley [5], respectively. Carter and Blalock [2] investigated in some detail robustness of Wald-type estimators, given varying degrees of departure from the assumption that the grouping procedure is independent of measurement errors. They conclude, inter alia, that for normally distributed parent values the quartergrouping method provides less biased slope

9021

estimates than least squares∗ . However, for skewed parent distributions, the grouping techniques produce estimates that are even more biased than least-squares estimates. Related investigations were carried out by Madansky [6] in connection with optimum cutoff points for the groups in Bartlett’s procedure. It should be noted that some confusion exists in the literature concerning these procedures especially in economics and econometrics∗ (see, e.g., Pakes [9] for a detailed, although not completely accurate, discussion of this problem). See also Moran [7] (for a more general treatment), Gupta and Aman [3], Theil and Van Ijzeren [10], and Lancaster [4] (for the case of heteroscedastic data). REFERENCES 1. Bartlett, M. S. (1949). Biometrics, 5, 207–212. 2. Carter, L. F. and Blalock, H. M. (1970). Appl. Statist., 19, 34–41. 3. Gupta, Y. P. and Aman, U. (1970). Statist. Neerlandica, 24, 109–123. 4. Lancaster, T. (1968). J. Amer. Statist. Ass., 63, 182–191. 5. Lindley, D. V. (1947). J. R. Statist. Soc. Suppl., 9, 218–225. 6. Madansky, A. (1959). J. Amer. Statist. Ass., 54, 173–205. 7. Moran, P. A. P. (1971). J. Multivariate Anal., 1, 232–255. 8. Nair, K. R. and Banerjee, K. S. (1943). Sankhya, 6, 331. 9. Pakes, A. (1982). Int. Econ. Rev., 23, 491–497. 10. Thiel, H. and Van Ijzeren, J. (1956). Rev. Inst. Int. Statist., 24, 17–26. 11. Wald, A. (1940). Ann. Math. Statist., 11, 284–300. See also LINEAR REGRESSION.

WALD DISTRIBUTION. See INVERSE GAUSSIAN DISTRIBUTION; SEQUENTIAL ANALYSIS

WALD’S DECISION THEORY The centerpiece of the work of Abraham Wald∗ in mathematical statistics was his

9022


creation of a unified mathematical basis for statistical decision making. This sketch of his formulation and results should be read in conjunction with DECISION THEORY (DT). It is, however, helpful to note that Wald worked from a ‘‘frequentist’’ perspective, rather than from the more ‘‘Bayesian’’∗ point of view taken in DT. FEATURES OF THE THEORY Wald sought a flexible and general framework within which a unified theory could be developed. There were a number of important features of his approach. First, the traditional problems of hypothesis testing∗ , point and interval estimation∗ are all regarded as particular cases of a general multi-decision problem. Second, the theory allows for multistage or sequential experimentation. Third, the question of design of experiments∗ is regarded as part of the general decision problem. Fourth, the general theory is most conveniently cast in terms of randomized decision∗ rules; though a considerable effort is made to determine when nonrandomized rules will suffice. ELEMENTS OF THE THEORY The statistician has (potentially) available observations on a sequence X = (X1 , X2 , . . .) of (not necessarily independent) random variables having joint distribution F. At the outset, the statistician is assumed to know nothing except that F belongs to a family {Fθ : θ ∈ }. There is a space D of decisions d, one of which the statistician has (eventually) to make. If Fθ is the actual distribution governing the data, and if decision d is taken, then a loss L(θ , d) is incurred. The total loss is the sum of the loss due to the decision made and the cost of the observations. Experimentation can, in general, be conducted sequentially. A statistical decision function, in its most general form, is a rule, which at the mth stage (m = 0, 1, 2, . . ., the zeroth stage being at the outset of the experiment before any observations have been taken) tells the statistician whether or not to take further observations (at the zeroth stage, whether to take any observations), on

which random variables to take observations (if at all), and which decision to take (if no further observations are to be taken). At the mth stage, the decision function δm (d|x1 , . . . , xm ) is a function of the preceding observations, and is, in general, a probability distribution over the available possibilities. For such a randomized decision rule, the actual decision is made by an independent chance mechanism governed by the distribution δm (d|x1 , . . . , xm ). A nonrandomized decision rule restricts the values of δm (d|x1 , . . . , xm ) to 0 and 1, and the (statistically objectionable) use of the independent chance experiment is avoided. The rule δ can then be more simply described in terms of a stopping rule τ = τ (x1 , . . . , xm ) ∈ {1, 2, . . .} and terminal decision rules dm (x1 , . . ., xm ) ∈ D, which are defined whenever τ = m. Wald assesses the merit of a candidate decision rule δ by averaging the loss plus cost incurred by the prescriptions {δm } with respect to the distribution Fθ of the data X, obtaining a risk function r(θ , δ). In the special case of a fixed sample size experiment with n cost-free observations x = (x1 , . . . , xn ), the risk of a nonrandomized decision rule dn (x) would be given explicitly by r(θ , dn ) = Eθ L(θ , dn (X)) = L(θ , dn (x))Fθ (dx). Comparison of two decision rules is made on the basis of their risk functions. A decision rule δ1 is (strictly) better than a decision rule δ2 if r(θ , δ1 ) r(θ , δ2 ) for all θ ∈ (with strict inequality holding for at least one θ ). The rule δ1 is called admissible if there is no uniformly better decision rule (see ADMISSIBILITY). Wald’s risk function depends on the decision rule δ and the parameter θ . It should be distinguished from the (Bayes) risk function R(ξ , δ) discussed below and in DT, in which a further average is taken with respect to a (prior) probability ξ (dθ ) on θ . BAYES AND MINIMAX∗ DECISION RULES Wald’s theory studies these rules both for their intrinsic interest and for their role in the construction and characterization of complete classes of decision rules.


If ξ (dθ ) is a (prior) probability distribution on , the Bayes or integrated risk of a decision rule is given by R(ξ , δ) = r(θ , δ)ξ (dθ ), and if R(ξ , δ ∗ ) = inf R(ξ , δ), δ

(1)

then δ ∗ is termed a Bayes decision rule with respect to ξ . More generally, δ ∗ is called a wide-sense Bayes rule if property (1) is approximately true. That is, if there exist prior distributions {ξi }, i = 1, 2, . . ., such that lim [R(ξi , δ ∗ ) − inf R(ξi , δ)] = 0.

i→∞

δ

A minimax decision rule∗ is one for which supθ ∈ r(θ , δ) is a minimum. A prior distribution ξ0 is least favorable∗ (from the point of view of the statistician) if the minimum average risk relative to ξ0 is larger than for any other prior distribution: inf δ R(ξ0 , δ) inf δ R(ξ , δ) for all ξ . In the extreme generality of Wald’s formulation, it is important to know whether decision functions satisfying the above criteria exist. A (weak) notion of convergence is introduced for decision rules so that the class of possible rules forms a compact space. Under appropriate restrictions on loss and cost functions, Wald then shows that: (a) Minimax rules exist, as do Bayes rules, for any choice of ξ . (b) Minimax rules are wide-sense Bayes. (c) If is compact in the appropriate topology, then a least-favorable distribution ξ0 exists, and a minimax rule is always Bayes with respect to ξ0 . Wald’s decision theory is intimately related to game theory∗ , in particular, to two-person zero-sum games in which Nature takes the role of player 1 and the statistician is player 2. Wald notes that parts of the theory of minimax solutions (for example, the existence of a minimax ‘‘value’’ for a game) is of no particular intrinsic interest in the statistical setting. Wald’s attention to minimax theory derives from two sources: its role in deriving the complete class results and the idea that a minimax solution may be reasonable in a decision problem in which a prior distribution does not exist or is unknown to the experimenter.

9023

COMPLETE CLASS THEOREMS A class of decision rules is called complete if for any rule δ not in C, there exists a strictly better rule δ ∗ belonging to C. C is called essentially complete if δ ∗ is only required to be better than δ, and not strictly so. In principle, if C is complete or at least essentially complete, then one can ignore all decision rules not belonging to C. In this way the choice of a decision rule is reduced to the choice of a member of C. Thus, in Wald’s theory, the discovery and description of complete classes occupies a central position. (Of course, even if a minimal complete class can be identified in a given problem, a further criterion needs to be employed to choose a particular rule, or group of rules, for use.) The basic complete class theorems assert, under boundedness conditions on loss, cost, and risk functions, that the class of Bayes rules is essentially complete. If the parameter space is compact, then Bayes rules actually form a complete class. In general, however, a complete class is obtained by extending the class to include wide-sense Bayes rules. Wald notes that the ideas of prior distributions and Bayes rules are used as mathematical tools in describing complete classes. The theory does not require the statistician to adopt a Bayesian approach and to postulate the existence of a specific prior distribution. The obituary article by Wolfowitz [15, pp. 4 ff] contains a useful perspective on Wald’s attitude to the theory generally, to the use of Bayes solutions in particular, and on contemporary reaction to Wald’s results. As an example of a success of decision theory within Wald’s lifetime, one can cite his proof (jointly with Wolfowitz [14]) of the optimum character of the sequential probability ratio test (SPRT). Introduced as a basic element of his work on sequential analysis∗ , the conjectured optimality of the SPRT was only proved after the framework of decision theory was extended to cover sequential experimentation and manipulations of Bayes solutions exploited. In general, however, the results of Wald’s theory are not cast in a form ready for application to specific questions. Explicit description of complete classes and determination of minimax rules is often technically challenging

9024


and has been the object of much subsequent research. (See, for example, [3, 6], or the survey in [1].) Happily, contemporary availability of powerful and cheap computing may provide a convenient alternative to explicit analytic solutions or approximations in the analysis of competing decision rules. FURTHER DEVELOPMENTS Wald’s work in statistical decision theory began with his 1939 paper [11] and continued intensively in the 1940s. It was presented in detail in his 1950 book [12], and would have undoubtedly continued but for his early death. A brief and accessible account of his ideas is given in the address [13], on which, together with [15], the present entry is partly based. Ferguson [4] has given an historical account of the development of the decision model, going back to Daniel Bernoulli. Since 1950, Wald’s theory has been extended and refined by many workers, including L. Brown, J. Kiefer, L. LeCam, E. Lehmann, and C. Stein. This work is too extensive to discuss here; some sources are listed in the references to DT and in the References and Bibliography that follow. Major themes have included the development of necessary and sufficient conditions for admissibility, the inadmissibility of maximum likelihood procedures in multiparameter contexts, the role of invariance in decision problems with natural group structures, decision theoretic study of confidence sets and conditional confidence procedures, model selection, and properties of sequential procedures. The conceptual framework established by Wald (and its antecedents in NeymanPearson∗ theory and game theory) has been influential in a number of fields of mathematical statistics, including asymptotic theory [9] simultaneous or ‘‘Stein’’ estimation [1, 10], robust estimation∗ [5], optimal experimental design∗ [7], multiple comparisons∗ , and robust Bayesian analysis [2]. REFERENCES 1. Berger, J. O. (1988). Multivariate Estimation—A Synthesis of Bayesian and Frequentist Approaches. SIAM, Philadelphia, PA. (To appear.)

2. Berger, J. O. (1980, 1985). Books on Statistical Decision Theory (see listing under Bibliography). 3. Brown, L. D. (1971). Admissible estimators, recurrent diffusions and insoluble boundary value problems. Ann. Math. Statist., 42, 855–903. 4. Ferguson, T. S. (1976). Development of the decision model. in On the History of Statistics and Probability, D. B. Owen, ed. Marcel Dekker, New York, Basel (Switzerland). 5. Huber, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Statist., 35, 73–101. 6. Johnstone, I. M. (1986). Admissible estimation, Dirichlet principles and recurrence of birth-death chains on ZD + . Prob. Theor. Rel. Fields, 71, 231–271. 7. Kiefer, J. (1974). General equivalence theory for optimum designs (approximate theory). Ann. Statist., 2, 849–879. 8. LeCam, L. (1955). An extension of Wald’s theory of statistical decision functions. Ann. Math. Statist., 26, 69–81. 9. LeCam, L. (1986). Asymptotic Methods in Statistical Decision Theory. Springer, New York. 10. Stein, C. (1956). Inadmissibility of the usual estimator of the mean of a multivariate normal distribution. Proc. Third Berkeley Symp. Math. Statist. Prob. 1, 197–206. University of California Press, Berkeley, CA. 11. Wald, A. (1939). Contributions to the theory of statistical estimation and testing hypotheses. Ann. Math. Statist., 10, 299–326. 12. Wald, A. (1950). Statistical Decision Functions. Wiley, New York. 13. Wald, A. (1952). Basic ideas of a general theory of statistical decision rules. Proc. Intern. Congress of Mathematicians, 1, 231–243. 14. Wald, A. and Wolfowitz, J. (1948). Optimum character of the sequential probability ratio test. Ann. Math. Statist., 19, 326–339. 15. Wolfowitz, J. (1952). Abraham Wald, 1902–1950. Ann. Math. Statist., 23, 1–13.

BIBLIOGRAPHY (This list supplements that given in DECISION THEORY.) Berger, J. O. (1980). Statistical Decision Theory: Methods and Concepts, Springer, New York. Berger, J. O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer, New York. (Two editions of an introductory graduate-level text presenting both frequentist and Bayesian

WALD’S EQUATION concepts. There is some difference of perspective between the two editions.) Brown, L. D. (1986). Foundations of Exponential Families. IMS Lecture Notes—Monographs Series. Volume 9. Chentsov, N. N. (1972). Statistical Decision Rules and Optimal Inference (translation). American Mathematical Society, Providence, RI. (An advanced monograph that develops Wald’s theory in the context of modern differential geometry.) Diaconis, P. and Stein, C. (1983). Lecture Notes on Statistical Decision Theory. (Unpublished, Stanford University, Stanford, CA.) (Graduatelevel course taught many times by Stein, one of the principal developers of Wald’s theory.) Ferguson, T. (1967). Mathematical Statistics: A Decision Theoretic Approach. Academic, New York. (A widely used introductory graduatelevel textbook presenting mathematical statistics from the point of view of Wald’s theory.) Lehmann, E. L. (1959, 1986). Testing Statistical Hypotheses, 1st and 2nd ed. Wiley, New York. Lehmann, E. L. (1981). Theory of Point Estimation. Wiley, New York. (Popular introductory graduate-level textbook containing introductions to mathematical statistics using the concepts of the Neyman-Pearson-Wald school.) See also ADMISSIBILITY; BAYESIAN INFERENCE; DECISION THEORY; DESIGN OF EXPERIMENTS; ESTIMATION, CLASSICAL; GAME THEORY; HYPOTHESIS TESTING; LEAST FAVORABLE DISTRIBUTIONS; MINIMAX DECISION RULES; MINIMAX ESTIMATION; MINIMAX TESTS; and SEQUENTIAL ANALYSIS.

IAIN JOHNSTONE

9025

This does not always hold, but Wald∗ [4, p. 53] was motivated to derive it so that he could establish important properties of the sequential probability ratio test in sequential analysis∗ . Various sets of conditions under which (1) holds have been obtained. The most useful of these, in addition to those already mentioned, are as follows. (i) {Xi } is an independent and identically distributed (i.i.d.) sequence of variables and N is independent of {Xi }. (ii) X1 , X2 , . . . are mutually independent; E[|Xi |] A < ∞ for all i and some finite A. The event ‘‘N i’’ depends only on X1 , X2 , . . . , Xi−1 [3,5]. (In sequential analysis, the Xi are frequently the loglikelihood ratios of observations {Yi } in hypothesis testing and are independent; see RANDOM SUM DISTRIBUTIONS and SEQUENTIAL ANALYSIS.) (iii) [1, Theorem 2.3]. {Xn } is a martingale∗ sequence, where Xn is Fn -measurable and Fm ⊂ Fn ⊂ F in a probability space (, F, P) for all m < n, 1 n < ∞. E[X1 ] = 0 and N is a stopping variable. E[XN ] exists and lim inf |xn | dP = 0. (2) n→∞

[N>n]

Under these conditions E[XN ] = 0, or more generally, E[XN |Fn ] = Xn

on [N n], n = 1, 2, . . . .

WALD’S EQUATION Let X1 , X2 , . . . be a sequence of random variables and N a stopping number∗ ; then N is positive integer-valued and provides a stopping rule so that sampling ceases after N variables in the sequence have been observed. A stopping variable by definition requires that the event ‘‘N = n’’ depends upon X1 , X2 , . . ., Xn only, for n = 1, 2, . . ., i.e., that this event belongs to the σ -field generated by X1 , . . ., Xn . Suppose that E[Xi ] = µ and that E[N] < ∞. Then Wald’s equation states that E[X1 + X2 + · · · + XN ] = µE[N].

(1)

This result is the most general. In order for E[XN ] < ∞ and (2) to hold, it suffices either that N is bounded almost surely (a.s.) that a.s. on [N > n], for some c and n, or |Xn | < c 2 2 that E[ N n=1 σn ] < ∞ [where σn = var(Xn )], or that E

N

E[|Xn − Xn−1 |]|Fn−1 ) < ∞.

n=1

Chow et al. [1] and Woodroofe [5] refer to (1) as Wald’s lemma, but the latter term is commonly applied to another result; see FUNDAMENTAL IDENTITY OF SEQUENTIAL ANALYSIS.

9026

WALD’S IDENTITY, APPLICATIONS OF

Example. The sequence {Xn } is mutually independent, defined by Pr(Xn = −2n ) = Pr(Xn = 2n ) = 0.50, n = 1, 2, . . . , so that E[Xn ] = 0 for all n. Let N be the first n for which ni=1 Xi −2. Then E[ N i=1 Xi ] −2 a.s. and E[N] < ∞, but Wald’s equation (1) does not hold. This indicates that the condition E[|Xi |] A < ∞ in (ii) is necessary. An extension of Wald’s equation to variances is frequently linked to (1). Suppose that condition (ii) holds, and, additionally, that var(Xn ) = σ 2 < ∞ for all n, that E[(Xn − µ)2 |N i] B < ∞, for all n < i, and that E[N 2 ] < ∞. Then [3], if for all n, Xn = Xn − µ and Sn = ni=1 Xi , var(SN ) = E[S N ] = σ 2 E[N]. 2

(3)

See also Chow et al. [1, p. 24]. Equation (3) thus holds for i.i.d. variables X1 , X2 , . . . having mean 0, but no such result holds for an i.i.d. sequence X1 , X2 , . . . having mean µ = 0, for which E[(SN − Nµ) ] = σ E[N] < ∞, 2

WALD’S IDENTITY, APPLICATIONS OF Wald’s fundamental identity of sequential analysis∗ is one relating the generating function∗ of a random walk∗ in one dimension to statistical properties of the absorption time and absorption point, in the presence of one or two absorbing barriers. It was derived by Wald in this development of sequential analysis (see SEQUENTIAL ANALYSIS) and plays a central role in that field [14–16] (see POWER; AVERAGE SAMPLE NUMBER (ASN)). A precise statement of the identity is given in FUNDAMENTAL IDENTITY OF SEQUENTIAL ANALYSIS and we use the notation introduced there. Let the individual steps of the random walk be identically distributed random variables with the common generating function M(t), let Sn be the position of the random walk at the nth step, and let N be the first step at which the condition b < Sn < a,

where b < 0, a > 0,

(1)

is violated. Wald’s identity states that E[etSN {M(t)}−N ] = 1.

(2)

2

but where var(SN ) may even be infinite. Chow and Teicher [2] discuss this apparent paradox. REFERENCES 1. Chow, Y. S., Robbins, H., and Siegmund, D. (1971). Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, Boston. 2. Chow, Y. S. and Teicher, H. (1966). Ann. Math. Statist., 37, 388–392. 3. Johnson, N. L. (1959). Ann. Math. Statist., 30, 1245–1247. Correction (1961), ibid., 32, 1344. 4. Wald, A. (1947). Sequential Analysis. Wiley, New York. 5. Woodroofe, M. (1975). Probability with Applications. McGraw-Hill, New York, pp. 338–339. See also OPTIMAL STOPPING RULES; RANDOM SUM DISTRIBUTIONS; SEQUENTIAL ANALYSIS; STOPPING NUMBERS AND STOPPING TIMES; and WALD’S IDENTITY, APPLICATIONS OF.

CAMPBELL B. READ

This identity can be used quite trivially to relate moments of SN to moments of N by differentiating (2) with respect to t and setting t = 0. If µ is the (finite) mean of a single step of the random walk and σ 2 its (finite) variance, application of this procedure leads to the identities E[SN ] = µE[N], E[(SN − Nµ)2 ] = σ 2 E[N].

(3)

Higher moments are readily generated by this means. The results obtainable using Wald’s identity can be applied in a variety of fields, and the key results in sequential analysis are valid for more general random walks. The results cited in FUNDAMENTAL IDENTITY OF SEQUENTIAL ANALYSIS allow one to generate an approximation to the probability of absorption at a particular end of a line. A similar technique leads to an approximate value for the generating function of the stopping time [3].

WALD STATISTICS AND WALD TESTS

APPLICATIONS AND GENERALIZATIONS The largest number of applications has come from the area of sequential analysis. Wald’s identity can be used to furnish an approximation for both the average sample number (ASN) and operating characteristic∗ (OC) for a sequential design, and it was for this purpose that it was devised. In these applications it is customary to make the approximation that SN = a or b in order to use (3). Many authors have tried to improve the degree of approximation by removing or modifying the assumptions made on SN . Kemp [5] has produced more accurate approximations for the ASN and OC, which are relatively simple and appear to work well. Tallis and Vagholkar [13] have also produced an effectively more precise approximation to the same quantities. Both of these papers apply their results to normally distributed steps, finding a marked improvement over results furnished by the application of Wald’s identity. A second area of generalization is that of sums of dependent random variables. Miller [3, 10] appears to have been the first to study this problem, and Kemperman has related material in [6]. Arjas [1, 2] motivated by a study of the semi-Markov∗ queue has also derived a generalized Wald’s identity. A quite general version of Wald’s identity for dependent random variables has been given by Franken and Lisek [4]. Applications of Wald’s identity have appeared in many contexts. The theory of queues∗ and dams∗ has seen several useful applications [11], and for this purpose Wald’s identity has been generalized to processes that evolve in continuous time [3]. Other applications have been made to models for neutron firing [7] and reliability [12].

(A clear account of Wald’s identity using a method developed by Miller of proving it.) 4. Franken, P. and Lisek, B. (1982). Zeit. Wahrscheinlichkeitsth. Verwand. Geb., 60, 143–150. 5. Kemp, K. W. (1958). J. R. Statist. Soc. B, 20, 379–386. 6. Kemperman, J. H. B. (1961). The Passage Problem for a Stationary Markov Chain. University of Chicago Press, Chicago. (Overlaps some of the cited work by H. D. Miller on the derivation of Wald’s identity.) 7. Kryukov, V. I. (1976). Adv. Appl. Prob., 8, 257–277. 8. Miller, H. D. (1961). Ann. Math. Statist., 32, 549–560. 9. Miller, H. D. (1962). Proc. Camb. Philos. Soc., 58, 268–285. (A derivation of the fundamental identity used to prove generalized versions of Wald’s identity.) 10. Miller, H. D. (1962). Proc. Camb. Philos. Soc., 58, 286–298. (Absorption probabilities for sums of random variables defined on a Markov chain.) 11. Phatarfod, M. R. (1982). Stoch. Proc. Appl., 13, 279–282. 12. Serfozo, R. F. (1973). Manag. Sci., 20, 1314–1315. 13. Tallis, G. M. and Vagholkar, M. K. (1965). J. R. Statist. Soc. B, 27, 74–81. 14. Wald, A. (1944). Ann. Math. Statist., 15, 283–296. (The original statement of Wald’s identity.) 15. Wald, A. (1946). Ann. Math. Statist., 17, 493–497. 16. Wetherill, G. B. and Glazebrook, K. D. (1986). Sequential Methods in Statistics, 3rd. ed. Chapman and Hall, London. See also AVERAGE SAMPLE NUMBER (ASN); CHARACTERISTIC FUNCTIONS; DAM THEORY; FUNDAMENTAL IDENTITY OF SEQUENTIAL ANALYSIS; GENERATING FUNCTIONS; PASSAGE TIMES; QUEUEING THEORY; RANDOM WALKS; and SEQUENTIAL ANALYSIS.

G. WEISS

REFERENCES 1. Arjas, E. (1972). Adv. Appl. Prob., 4, 258–270. (A general derivation of Wald’s identity for semi-Markov processes.) 2. Arjas, E. (1972). Adv. Appl. Prob., 4, 271–284. (Application of the generalized Wald’s identity to the semi-Markov queue.) 3. Cox, D. R. and Miller, H. D. (1965). The Theory of Stochastic Processes. Wiley, New York.

9027

WALD’S IDENTITY EQUATION

LEMMA. See FUNDAMENTAL OF

SEQUENTIAL

ANALYSIS;

WALD’S

WALD STATISTICS AND WALD TESTS. See WALD’S W-STATISTICS

9028

WALD’S W -STATISTICS

WALD’S W -STATISTICS Let i : Np (ξ i , ), i = 1, 2, be two p-variate [X = (X1 , . . . , Xp )] multinormal∗ populations with common variance-covariance matrix . It is required to assign an individual to 1 or 2 on the basis of observed values x. When all parameters are known, an optimal classification rule (minimizing the probability of incorrect decision) is ‘‘assign to 1 if u0 = {x − 12 (ξ 1 + ξ 2 )} −1 (ξ 1 − ξ 2 ) > K,

(1)

where K depends on prior probabilities of 1 and 2 . This is also the minimum distance rule∗ . Note that u0 = x −1 (ξ 1 − ξ 2 )+ constant; the first term on the right-hand side is Fisher’s linear discriminant function∗ [3]. When ξ 1 , ξ 2 and are not known, but are estimated from random samples of sizes n1 , n2 from 1 , 2 , respectively, by the sample arithmetic means X1 , X2 and the pooled sample variance-covariance matrix [with divisor (n1 + n2 − 2)] S, Wald [12] proposed using the classification statistic W = {x − 12 (X1 + X2 )} S−1 (X1 − X2 ),

(2)

obtained by ‘‘plugging in’’ the sample estimators in place of the (unknown) parameters. Wald showed that the limiting distribution of W as n1 , n2 → ∞ is the same as that of u0 . Specifically, for samples from 1 the distribution of W tends to normal with mean 1 2 2 2 and variance ; for samples from 2 it tends to normal with expected value − 12 2 and variance 2 , where

2 = (ξ 1 − ξ 2 ) −1 (ξ 1 − ξ 2 ). (This is the Mahalanobis distance∗ .) More generally, if T is an estimator of a vector parameter θ , and it is desired to test the hypothesis that g(θ ) = 0 for some vectorvalued twice-differentiable function g, then quadratic forms∗ W = [g(T)] D−1 g(T)

(3)

may be used as test statistics, rejecting the hypothesis for large values of W, where D is

an estimator of the covariance matrix of the vector g(T) − g(θ ). Test statistics of this kind are called Wald statistics, after their introduction by Wald [11,12]. He introduced them with T as a maximum likelihood∗ estimator (MLE) of θ based on independent observations from a family of distributions indexed by θ, satisfying some regularity conditions. Conditions more general than those of Wald under which W has an asymptotic chi-square or noncentral chi-square distribution∗ are derived in Stroud [10]. When T is a MLE of θ , it is natural to compare the large-sample performance of W with that of the likelihood-ratio∗ (LR) statistic −2 log and of Rao’s score statistic∗ S, since all three have the same asymptotic chisquare distribution; for definitions, assumptions, and basic properties, see Rao [9, Sec. 6e]. Now let θ be scalar and g(θ ) ≡ θ . Then [3,4] the test based on S is more powerful in the neighborhood of θ = 0 than either Wald’s test or the LR test, under certain conditions; furthermore, Wald’s test under these conditions is the least powerful of the three. However, the three tests perform almost equally near θ = 0 for families with small statistical curvature∗ . In testing a restricted model in which the restriction on θ is nonlinear in the parameters against an unrestricted model, a transformation to a form that is algebraically equivalent under the null hypothesis could alter the numerical value of the Wald statistic in finite samples, so that a set of data could conceivably lead to contrary conclusions based on one such form of W or another; the numerical values of −2 log and of S in these circumstances, however, would be unchanged [6]. This shortcoming of W does not arise, for example, in testing general linear restrictions on coefficients in the multivariate general linear model∗ . The exact finite sample distribution of W in the latter case has been obtained by Phillips [8]. In multidimensional contingency-table∗ analysis, Wald statistics can be produced via weighted least-squares∗ techniques if g(θ) is expressible in terms of linear functions of some unknown vector parameters φ; see refs. 2 and 7. See also MARGINAL SYMMETRY, and ref. 1 for applications in tests of marginal symmetry and quasisymmetry∗ .

WALLIS’ FORMULA

More general conditions than those of Wald under which W has an asymptotic central or noncentral chi-square distribution are derived in Stroud [10]. REFERENCES 1. Bhapkar, V. P. (1979). Biometrics, 35, 426. 2. Bhapkar, V. P. and Koch, G. G. (1968). Technometrics, 10, 107–123. 3. Chandra, T. K. and Joshi, S. N. (1983). Sankhya¯ A, 45, 226–246. 4. Chandra, T. K. and Mukerjee, R. (1985). Sankhya¯ A, 47, 271–284. 5. Fisher, R. A. (1936). Ann. Eugen. (Lond.), 7, 179–188. 6. Gregory, A. W. and Veall, M. R. (1985). Econometrica, 53, 1465–1468. 7. Grizzle, J. E., Starmer, C. F., and Koch, G. G. (1969). Biometrics, 25, 489–504. 8. Phillips, P. C. B. (1986). Econometrica, 54, 881–895. 9. Rao, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd ed. Wiley, New York. [The first edition (1965) includes a conjecture (omitted here), that the test based on the score statistic is locally more powerful than the LR or Wald test. Rao’s conjecture was largely justified in ref. 3.] 10. Stroud, T. W. F. (1971). Ann. Math. Statist., 42, 1412–1424. (This paper contains a very readable background discussion of Wald’s original test procedure and its asymptotic optimality properties.) 11. Wald, A. (1943). Trans. Amer. Math. Soc., 54, 426–482. 12. Wald, A. (1945). Ann. Math. Statist., 16, 117–186.

9029

WALLIS AND MOORE PHASE-FREQUENCY TEST This test is aimed at detecting departure from randomness∗ in a sequence of values X1 , X2 , . . . , Xn (such as a time series∗ ). It is based on the signs of the successive differences X1 − X0 , X2 − X1 , . . . , Xn − Xn−1 . Wallis and Moore [1] term a sequence of like signs a phase; their test is based on the total number of phases (whether of + or − sign). Omitting the first and last phases in the sequence, the number of phases H is determined. If the sequence is indeed random (and the X’s are identically distributed), then E[H] = 13 (2n − 7) and var(H) =

1 90 (16n

− 29).

For n 10, a reasonably good test can be based on the assumption that H is normally distributed, if a continuity correction∗ is used. The correction is not needed if n 25. Exact tables (based on a chi-squared distribution∗ ) are given in [1]. REFERENCE 1. Wallis, W. A. and Moore, G. H. (1941). J. Amer. Statist. Ass., 36, 401–409. See also RUN LENGTHS, TESTS OF.

WALLIS’ FORMULA See also CHI-SQUARE TEST —I; DISCRIMINANT ANALYSIS; DISTANCE FUNCTIONS; LIKELIHOOD RATIO TESTS; MULTIVARIATE ANALYSIS; SCORE STATISTICS; and WELCH’S v-CRITERION.

2 2 4 2k 2k 1 π = · · ··· · ··· 2 1 3 3 2k − 1 2k + 1 =

WALD TEST OF MARGINAL SYMMETRY. See MARGINAL SYMMETRY

∞

{4j2 (4j2 − 1)−1 }. j=1

The formula is named after John Wallis [1] (1616–1703).

WALD TESTS. See WALD’S W-STATISTICS REFERENCE

WALD-WOLFOWITZ TEST. See RUNS

TWO-SAMPLE

1. Wallis, J. (1655). Oxford, England.

Arithmetica

Infinitorum,

9030

WALSH AVERAGES

WALSH AVERAGES Hodges and Lehmann [2] discussed a general method for deriving a point estimate of location from a test statistic. Given a sample X1 , . . . , Xn from a continuous, symmetric population, the Hodges–Lehmann estimator, derived from the Wilcoxon signed rank statistic∗ , is the median of the n(n + 1)/2 pairwise averages (Xi + Xj )/2, 1 i j n. In fact, the Wilcoxon statistic, which is the sum of ranks of the positive observations when ranked among all the absolute values, is precisely equal to the number of positive pairwise averages. Walsh [5,6] proposed various tests and confidence intervals based on the pairwise averages, and Tukey [4] established the connection between the Wilcoxon signed rank statistic and the pairwise averages and dubbed them the Walsh averages. The Walsh averages are fundamental in describing statistical inference based on general linear rank tests in the one-sample location model. Let a(1) · · · a(n) be a nonconstant sequence of scores and define S(θ ) =

n

New York Times, September 17, 1982, Section B. The prices in thousands of dollars are 25, 42, 50, 63, and we will suppose this is a sample from a symmetric population of prices. We will use the absolute normal scores∗ statistic with a(i) = −1 [(i + n + 1)/{2(n + 1)}] and . −1 (u) = 4.91[u0.14 − (1 − u)0.14 ], where −1 is the inverse of the standard normal distribution function. The approximation, based on Tukey’s λ-distribution, is sufficient for most practical purposes and is easy to use on a calculator. (See Joiner and Rosenblatt [3].) The absolute normal scores for n = 4 are 0.25, 0.52, 0.84, and 1.28. The table shows the Walsh average (and its rank), the jump in S(θ ), and the value of S(θ ) just to the right of the Walsh average. Note that max S(θ ) = ai = 2.89.

a[Ri (θ )]sign(Xi − θ ),

i=1

where Ri (θ ) is the rank of |Xi − θ | among |X1 − θ |, . . . , |Xn − θ |, and sign(z) = 1, 0, −1 as z > 0, = 0, < 0, respectively. Bauer [1] showed that S(θ ) is a decreasing step function, which has steps only at some or all of the Walsh averages. Let X(1) · · · X(n) denote the ordered sample, then, for i j, the step size at (X(i) + X(j) )/2 is 2[a(j − i) − a(j − i + 1)], where a(0) ≡ 0. Hence the value of S(θ ) at a value just to the right of a Walsh averages is 1n a(i), less the accumulated jumps. The Hodges–Lehmann estimate of θ , derived from S(θ ), is the solution θ = θˆ of S(θ ) = 0, where equality is interpreted to mean the Walsh average where S(θ ) steps across 0 or the average of the two Walsh averages that determine the interval of zeros. If Pθ (S(θ ) < k) = α/2, then the acceptance region of the test determines a (1 − α)100% nonparametric confidence interval∗ for θ . This interval has appropriately chosen Walsh averages as its end points. To illustrate the calculations, we consider the first four used Rolls Royce prices in the

25

42

50

63

25 25(1) −0.50 2.39 33.5(2) −0.54 1.85 37.5(3) −0.64 1.21 44(5) −0.88 −0.17

42

50

63

42(4) −0.50 0.71 46(6) −0.54 −0.71 52.5(8) −0.64 −1.85

50(7) −0.50 −1.21 56.5(9) −0.54 −2.39

63(10) −0.50 −2.89

Hence the Hodges–Lehmann estimate of θ is the Walsh average, 44. If we had used the Wilcoxon signed rank statistic, the estimate is the median of the Walsh averages, (44 + 46)/2 = 45. Finally, if a(i) ≡ 1 so we have the sign statistics, then the estimate is the sample median, (42 + 50)/2 = 46, itself a Walsh average. REFERENCES 1. Bauer, D. F. (1972). J. Amer. Statist. Ass., 67, 687–690.

WANDERING-QUARTER SAMPLING 2. Hodges, J. L., Jr. and Lehmann, E. L. (1963). Ann. Math. Statist., 34, 598–611. 3. Joiner, B. L. and Rosenblatt, J. R. (1971). J. Amer. Statist. Ass., 66, 394–399. 4. Tukey, J. W. (1949). Memorandum Report No. 17, Statistical Research Group, Princeton University, Princeton, NJ. 5. Walsh, J. E. (1949a). Ann. Math. Statist., 20, 64–81. 6. Walsh, J. E. (1949b). J. Amer. Statist. Ass., 44, 342–355. See also HODGES —LEHMANN ESTIMATORS; LOCATION PARAMETER; L-STATISTICS; NONPARAMETRIC CONFIDENCE INTERVALS; ORDER STATISTICS; and WILCOXON SIGNED RANK TEST.

THOMAS P. HETTMANSPERGER

WALSH–FOURIER TRANSFORMS Walsh functions∗ {W(n, x), n = 0, 1, 2, . . ., 0 x < 1} are defined as follows: (i) W(0, x) = 1, 0 x < 1. i (ii) Let n be dyadic, i.e. n = ∞ i=0 xi 2 , where xi = 0 or xi = 1; and xi = 0 for i > m. Then W(n, x) =

n

9031

stationary processes, i.e., stationary processes unchanged under dyadic shifts. For further properties of Walsh functions, see Révész [6] and (with special reference to the law of iterated logarithm∗ ) Révész and Wschebor [7]. REFERENCES 1. Chrestenson, N. E. (1955). Pacific J. Math., 5, 17–31. 2. Fine, N. J. (1949). Trans. Amer. Math. Soc., 65, 372–414. 3. Fine, N. J. (1950). Trans. Amer. Math. Soc., 69, 66–77. 4. Fine, N. J. (1957). Trans. Amer. Math. Soc., 86, 246–255. 5. Morettin, P. A. (1981). Proc. 43rd Session of the ISI, 49, Book 3, 1211–1230. (Detailed list of references.) 6. Révész, P. (1968). The Laws of Large Numbers, Academic, New York. 7. Révész, P. and Wschebor, M. (1965). Publ. Math. Inst. Hung. Acad. Sci., 9A, 543–554. 8. Selfridge, R. G. (1955). Pacific J. Math., 5, 451–480. 9. Walsh, J. L. (1923). Amer. J. Math., 45, 5–24. See also RADEMACHER FUNCTIONS.

{rmi (x)},

i=1

where m1 , . . . , mr correspond to the coefficients xmi = 1. These functions were introduced by Walsh [9] and studied extensively by Fine [2–4]. If f is any function of period 1, Lebesgue integrable on [0, 1], then it can be expanded in a Walsh–Fourier series f (x) ∼

∞

an W(n, x),

n=0

1 with coefficients an = 0 f (x)W(n, x)dx. Generalized Walsh functions are defined for nonnegative real arguments. See, e.g., Chrestenson [1] and Selfridge [8] for details. For applications of Walsh–Fourier transforms in timeseries∗ analysis, see, e.g., Morettin [5] and the references therein. The natural class of stationary processes∗ , which is analyzed by Walsh functions, is the class of dyadic

WALSH FUNCTIONS. See WALSH –FOURIER TRANSFORMS

WALSH INDEX. See INDEX NUMBERS WANDERING-QUARTER SAMPLING A practical method of sampling inter-item distances of spatial distributions of hidden or unmapped (point) items, proposed by Catana [1]. Wandering-quarter sampling is a variant of T-square sampling∗ , which in turn is a variant of nearest neighbor sampling∗ . In each of these sampling schemes, sampling sites O1 , O2 , . . . , On are chosen randomly in the region of interest, for example, by choosing exact coordinates on a map of the region. Each sampling site is then visited, and the

9032

WARD’S CLUSTERING ALGORITHM

agglomerative hierarchical methods. These methods apply a routing strategy to reproduce a hierarchical or treelike structure among n objects. Starting with n clusters, where each object is a cluster, an agglomerative hierarchical method proceeds in a stagewise manner to reduce the number of clusters one at a time until all n objects are in one cluster. See also HIERARCHICAL CLUSTER ANALYSIS. WARD’S ALGORITHM Figure 1.

location of the nearest item to that sampling site is then found, say at Pi for sampling site Oi . In wandering-quarter sampling, the nearest item to that at Pi is then found, subject to the condition that the direction from the new item to Pi lies within 45◦ of the direction from Pi to Oi . From this new item, say at Ri , a new search for its nearest item can again be conducted (once more subject to the condition that the direction from the new nearest item lies within 45◦ of the direction from Pi to Oi ), and so on. See Fig. 1. In Catana’s original version of wandering-quarter sampling, Catana sampling, the original search direction for the location of the original nearest point at Pi , is itself restricted to lie within a prespecified direction. REFERENCE 1. Catana, A. J. (1963). Ecology, 44, 349–360.

BIBLIOGRAPHY Diggle, P. J. (1983). Statistical Analysis of Spatial Point Processes. Academic, London. Ripley, B. D. (1981). Spatial Statistics. Wiley, New York. See also SPATIAL PROCESSES and SPATIAL SAMPLING.

W. G. S. HINES

WARD’S CLUSTERING ALGORITHM Ward’s clustering algorithm is a popular procedure within the set of algorithms called

The following steps describe the usual implementation of an (agglomerative) hierarchical cluster analysis: 1. Define a triangular matrix that shows a measure of similarity or proximity between each pair of n objects (see PROXIMITY DATA). This matrix has n(n − 1)/2 entries and is often constructed by computing proximity measures such as Euclidean distances or correlations based on an original n × m data matrix of n objects and m attributes or variables. 2. Search the proximity matrix for the most similar pair of clusters and join these two clusters. The proximity value between the two merged clusters is called the criterion or objectivefunction value for stage k, zk . 3. Update the proximity matrix by recomputing proximity values between the new cluster and all other clusters. The new proximity matrix has one less row (or column) than the preceding proximity matrix. 4. Repeat steps 2 and 3 until all objects reside in one cluster. The result is a treelike structure that shows which two clusters were merged at each stage k, k = 1, . . . , n − 1, and the corresponding criterion values zk for each stage, where stage k corresponds to n − k clusters. Differences among hierarchical methods primarily center around two procedural steps: the definition of the ‘‘most similar’’ pair of clusters (step 2) and the method of updating


similarity measures from one stage to the next (step 3). Lance and Williams [4] developed a generalized transformation model that elegantly defines the measure of proximity in step 3 for six popular hierarchical models. Later, this transformation model was extended by Wishart [12] to include Ward’s method, which is sometimes called the minimum variance or Ward’s error sum of squares method. Interestingly, the original article by Ward [11] described a generalized hierarchical method similar to, but no less general than, the four-step description above. In particular, Ward specified that the ‘‘loss’’ from joining two groups (i.e., the criterion value in step 2) is best expressed by whatever objective function makes sense to the investigator and then described various objective functions that he used in his research for the Air Force [11, p. 237]. Indeed, in a subsequent communication, Ward’s own preference for naming his model was the MAXOF (MAXimize an Objective Function) clustering model. In his numerical example, Ward used the ‘‘sum of squared deviations about the group mean’’ or ‘‘error sum of squares,’’ which in multidimensional Euclidean space is defined for cluster c as ESSc =

nc m

onehalf of the squared Euclidean distance between two joined single-object clusters and proved that the use of a squared Euclidean distance proximity matrix is functionally equivalent to Ward’s ESS example and implementable through the transformation function first described by Lance and Williams. Thus Ward’s early choice of an ESS example, Wishart’s link to Euclidean distance, and the Lance and Williams transformation function, the attractive but not necessarily valid conceptualization of clusters as swarms in Euclidean space, the ‘‘closeness’’ of clusters based on the proportionality between increase in the ESS and the squared Euclidean distance separating merged-cluster centroids [1, p. 143], and subsequent implementations by commercial computer packages all came together to transform Ward’s perfectly general algorithm into an algorithm with an exclusive, distancebased, minimum variance focus. See Ward [11, p. 241] and Anderberg [1, p. 43] for numerical examples of Ward’s method based on the ESS criterion. Anderberg [1] and Everitt [3] are excellent sources for descriptions of the various hierarchical algorithms. STOPPING RULES

(xcij − xcj )2 ,

(1)

j=1 i=1

where m is the number of attributes; nc is the number of objects in cluster c; xcij is the measure (raw, standardized, etc.) of attribute j on object i within cluster c; and xcj is the mean of the jth attribute in cluster c. The overall error sum of squares objective function in stage k is then given by ESSk =

9033

n−k

ESSc ,

(2)

c=1

and the ‘‘loss’’ or increase in ESS based on the fusion of two clusters in stage k is given by zk = ESSk − ESSk−1 ,

(3)

which defines an ‘‘error sum of squares’’ or ‘‘minimum variance’’ criterion for step 3. Subsequently, Wishart [12, p. 167] showed that the criterion in (3) is equivalent to

Users of hierarchical clustering algorithms often wish to determine the ‘‘best’’ number of clusters. Mojena [7] and Mojena and Wishart [8] proposed and evaluated three statistical rules for this task, based on the behavior of the criterion vector z as a monotonically increasing function. These rules predict a ‘‘significant’’ increase from zk to zk+1 , if any, which implies an undesirable fusion, and the stage with n − k clusters as ‘‘best.’’ Ward’s method, together with a simple uppertail rule, gave consistently good results across Monte Carlo∗ data sets that conceptualized clusters as compact swarms in Euclidean space. Morey et al. [9] further confirmed successful results with this rule, along with an alternative rule based on an adaptation of Cattell’s scree test. Binder [2] has proposed a Bayesian approach to estimating the best number of clusters, but its usefulness is restricted to small problems. See [10] for other procedures.

9034


SOFTWARE PACKAGES There are many sources of computer software for implementing cluster analyses, but two commercially available packages stand out, both of which include Ward’s method: SAS and CLUSTAN (see STATISTICAL SOFTWARE). SAS [10] is a comprehensive system for data analysis that is widely used in universities and other research-oriented groups. It implements seven clustering procedures, including four common hierarchical methods. It uses the n × m multivariate data matrix as input, has procedures for printing dendrograms∗ , and prints reports that include a criterion for estimating the best number of clusters. CLUSTAN [13] is by far the most comprehensive clustering package available. It includes 28 clustering procedures, 10 of which are hierarchical. Input data options include multivariate data matrices and user-defined similarity matrices. Users have a choice of 40 proximity measures, depending on data types (numeric or binary) and user needs. The package also includes relocation routines for improving an initial clustering, various forms of graphical output, stopping-rule procedures from Mojena [7], the ability to read data files previously created through the SPSS software system, and a conversational preprocessor. EVALUATION Ward’s method figures prominently in the literature that addresses the evaluation of clustering algorithms. The effectiveness of Ward’s method as a clustering procedure can be viewed from various perspectives. First, does it give an optimal solution with respect to minimum error sum of squares? Second, just how good is it in recovering cluster structure, i.e., in identifying both the correct number of clusters and the correct membership of objects? Finally, how does it compare to other clustering procedures? Ward’s method is a heuristic rather than an optimization algorithm. As such, it does not ensure that the resulting clustering yields an overall minimum variance solution. Indeed, it would be very surprising if it

were to yield an optimal or even near-optimal solution except for trivial data sets. Optimal solutions to the ESS clustering problem have been generated by dynamic and 0–1 integer programming∗ formulations, but these severely limit the size (n) of the problem due to storage and computational constraints. An attractive strategy suggested by Wishart [13] and implemented by Morey et al. [9] is to generate an initial solution by Ward’s method and then systematically reassign objects by using relocation techniques. The evaluation literature primarily reports on the recovery performances of clustering techniques and on their comparisons. The literature tends to favor Ward’s method, although results are mixed. It would now appear that some tentative conclusions are emerging based on work by Morey et al. [9] and Milligan et al. [6], and on the thorough review by Milligan [5]: 1. Ward’s method performs quite well across a variety of data sets that include Monte Carlo mixtures, ultrametric∗ data, and real data; however, performance can vary widely depending on the selection of clustering parameters such as proximity measures, and on certain data-set characteristics such as cluster size and cluster overlap. 2. The ESS focus of Ward’s method dictates the use of squared Euclidean distance as a measure of proximity; yet this measure of association may not be warranted for all studies, as it mixes together object associations due to shape, scatter, and height. If only shape is of interest, then correlationtype measures are more appropriate. In this case, an algorithm such as the group average method based on a correlation criterion can give more legitimate clustering results. 3. The extent of cluster overlap affects the performance of various algorithms. Ward’s method appears to give the best recovery as overlap increases, but the group average method seems to outperform Ward’s method with nonoverlapping structures. 4. Ward’s method tends to fuse small clusters and appears to favor the creation of

WARGENTIN, PEHR WILHELM

clusters having roughly the same number of observations. The group average method is as good or better when clusters are of unequal size. 5. Ward’s method and other hierarchical algorithms are not very robust with respect to various types of error perturbations, such as outliers∗ . 6. Ward’s method is sensitive to profile elevation, with a tendency to give distinct, but not necessarily valid, clusters along the principal component of a multivariate distribution. If the elevation component is pervasive, then the solution can be valid, as in the alcohol abuse study by Morey et al. [9]. REFERENCES 1. Anderberg, M. R. (1973). Cluster Analysis for Applications. Academic, New York. 2. Binder, D. A. 275–285.

(1981).

Biometrika,

68,

3. Everitt, B. S. (1980). Cluster Analysis, 2nd ed. Heinemann, London, England. 4. Lance, G. N. and Williams, W. T. (1967). Computer J., 9, 373–380. 5. Milligan, G. W. (1981). Multivariate Behav. Res., 16, 379–407. 6. Milligan, G. W., Soon, S. C., and Sokol, L. M. (1983). IEEE Trans. Pattern Analysis Machine Intelligence, PAMI-5, 40–47. 7. Mojena, R. (1977). Computer J., 20, 359–363. 8. Mojena, R. and Wishart, D. (1980). COMPSTAT 1980 Proc. Physica-Verlag, Vienna, Austria, pp. 426–432. 9. Morey, L. C., Blashfield, R. K., and Skinner, H. A. (1983). Multivariate Behav. Res., 18, 309–329. 10. SAS USER’s GUIDE: Statistics (1982). SAS Institute, Cary, NC. 11. Ward, J. H. (1963). J. Amer. Statist. Ass., 58, 236–244. 12. Wishart, D. (1969). Biometrics, 22, 165–170. 13. Wishart, D. (1982). CLUSTAN User Manual. Program Library Unit, Edinburgh University, Edinburgh, Scotland. See also DENDROGRAMS; HIERARCHICAL CLUSTER ANALYSIS; PROXIMITY DATA; and RECURSIVE PARTITIONING.

RICHARD MOJENA

9035

WARGENTIN, PEHR WILHELM Born: September 11, 1717 [Julian Calen¨ dar], in Sunne (Jamtland), Sweden. Died December 13, 1783, in Stockholm, Sweden. Contributed to: astronomy, demography, population statistics, promotion of science. Pehr Wargentin was born in 1717 in the beautiful country near Lake Storsjön in the county ¨ of Jamtland in northern Sweden. His father, Wilhelm Wargentin, was of German descent ˚ and had studied science and theology in Abo, Uppsala, and Lund. He was appointed vicar in Sunne on the condition that he married his deceased predecessor’s widow, who after a year gave birth to Pehr Wargentin. Wargentin was taught first by his father, then at a well-regarded primary school near his home (Frösö trivialskola) and at a secondary ¨ osand. school in Harn¨ At the age of eleven Wargentin saw a lunar eclipse, which fascinated him. After finishing school in 1735 he went to Uppsala University where he studied astronomy and other sciences. One of his teachers was Anders Celsius, known for his geodesic measurements and for the centigrade thermometer scale. As suggested by Celsius, Wargentin wrote a master thesis (magisterarbete) in 1741 on the motion of the moons of Jupiter, which was a topic of great international scientific interest at that time, useful for the determination of longitudes of geographical places. In his master’s thesis Wargentin succeeded in constructing a table of unprecedented accuracy. Later he further improved on these contributions, and he continued his work in astronomy to the end of his life. In 1749 Wargentin was appointed secretary of the Royal Swedish Academy of Sciences in Stockholm, which had been set up in 1739 with the Royal Society in London as a model, when Linnaeus returned from England. Wargentin retained this position until his death more than three decades later; during this period he made a great and successful effort to develop the young Academy into a powerful platform for promotion of the new sciences and their utilization in the

9036


country. He had an important role in several different projects such as the erection of a new observatory in Stockholm (where he became superintendent), the establishment of regular meteorological observations in Stockholm, the editing of an annual almanac, the latecoming transition to the Gregorian calendar in his country, a canal construction project, and population statistics. Wargentin’s contributions to statistics dealt with the development of a national system for population statistics. At the death of King Charles XII in 1718, the Parliament (consisting of the Four Estates) seized power from the Monarch, while the country suffered the consequences of a long, devastating, and finally lost war against its neighbors, notably Russia. There was concern about the sparsity of the Swedish population, and it was thought that a strongly increased population and a better distribution of people over regions and occupations would bring wealth and power to the country. Partly inspired by the ideas of political arithmetic in England and other countries, there was a growing urge for what is today known as population statistics. Under the Church Act of 1686, the Swedish State Church had already conducted a nationwide continuous population registration, which turned out to be an excellent basis for population statistics. The Parliament initiated a Royal Decree of 1748 on the annual compilation of statistical data, which became effective the following year. The compiled data comprised a tabular summary of the following items: 1. The number of infants baptized per month, with distribution by sex and by marital status of the mother. 2. The number of persons buried per month, with distribution by sex and by subdivision into three categories, viz., children under 10 years old, adolescents and other single persons, and married persons. 3. The number of marriages per month, and that of marriages dissolved by death. 4. The number of deaths per month, with distribution by sex and by age in fiveyear groups, children under one year of age stated separately. Furthermore, causes of death were given in 33 categories.

5. The size of the population at the end of the year, with distribution by sex and by age in five-year groups, children under one year of age stated separately. 6. The size of the population at the end of the year, with distribution by sex and by estate (stand) ˚ or occupation. The latter were given in 61 categories. A unique feature of this tabular system, known as Tabellverket, was that it contained both vital statistics∗ and census∗ data, i.e., data on both the changes and the size of the population, thus providing the basis for a complete demographic description of the population. The data on the size of the population were kept secret during the first years, since it was feared that enemies of the country might take advantage of knowing how small its population was (little more than two million in Sweden, including Finland). Forms for the tables were sent to the clergy all over the country, to be filled in by them for each parish. From these tables, summary tables were compiled for each deanery and then for each diocese. The latter summaries were sent to the County Governors, who forwarded their county summaries to the Royal Chancellery in Stockholm. Though the data collection was annual from the start, in view of the burden on the clergy the census data (items 5 and 6) were from 1752 collected only every three years, and later only every five years. From 1753 Wargentin played a key role in organizing and developing the system. The plan for it had been worked out by Pehr Elvius, Wargentin’s predecessor as Secretary of the Academy. Wargentin became the key figure in the Royal Commission on Tabulation, which was set up for the work in 1756, and which was the predecessor of today’s Statistics Sweden∗ . He wrote long reports to the Parliament in 1755–1765, and several essays (in Swedish) on demography∗ in the Proceedings of the Academy in 1754–1782. Especially notable are a series of six essays from 1754–1755 entitled ‘‘The importance of annual summaries on births and deaths,’’ and one essay from 1766 entitled ‘‘Mortality in Sweden.’’ The 1754–1755 essays, still easy to read and free from theological speculations, treat many different topics, such as the


importance of an increase in the population, the regularity of different phenomena in the demographic field, life insurance and mortality, and discussion of Halley’s method of calculating the size of a population by using only data about deaths and births. Other topics were causes of death and steps to reduce mortality. The population pyramid∗ was described. The papers take up ideas of well-known writers on political arithmetic and related fields, such as Graunt, Petty, Hal¨ ley, Sussmilch, Deparcieux, and Kersseboom, who are extensively cited. Wargentin’s 1766 paper ‘‘Mortality in Sweden’’ is his most important one. He made use of data on deaths for the years 1755–1763 which he related to the size of the population data in 1757, 1760, and 1763. His tables show the relative number of deaths for each age group and sex. This was the first time an accurate national mortality table was given. He demonstrated how the mortality numbers could be used for several interesting mortality comparisons between regions, such as Stockholm vs. the countryside. He also gave long discussions on the reliability of the data. By the efforts of an English insurance mathematician, Richard Price, Wargentin’s tables became transformed to life tables∗ and known ¨ abroad. Furthermore, Sussmilch, who laid the foundation for demographic methodology in his work on the ‘‘Divine Order’’ (Göttliche Ordnung), made essential use of Wargentin’s mortality tables in his developments in later editions of that work. Later papers by Wargentin deal with seasonal variations in births, showing that conceptions peaked in December, and with the extent of emigration, which was shown to be substantially smaller that had been previously thought. Like contemporary writers in other countries, Wargentin had advocated a prohibition of emigration, but his new findings made him change his opinion. Political interest in population statistics faded in time, especially after King Gustav III restored much of the Monarch’s former power in 1772; the latter action actually had the sympathy of Wargentin. After Wargentin’s death and toward the end of the century the tabular system decayed somewhat. But it survived, so that there are complete time series∗ of Swedish population statistics from

9037

1749 until today. An edited presentation of historical statistics, with some reconstruction to fill some gaps in the set of variables, ¨ in the was later given by Gustav Sundbarg journal Statistisk Tidskrift around 1900; cf. ¨ [1]. Sundbarg Wargentin was a highly respected man of an honest and faithful character, with modest and cheerful manners, and with an extraordinary working capacity. He took an important role in many pioneering endeavors of his time. He had an intense correspondence on scientific matters with scholars in many countries, and although his published statistical papers are in Swedish, some of them were translated into German and French and became internationally known. During the last months of his life he suffered from illness but still continued his hard work and daily observations and planned new projects. In 1783 he died quietly in his home in the observatory, at the age of 66. REFERENCE ¨ 1. Sundbarg, G. (1907). Bevölkerungsstatistik Schwedens 1750–1900. SCB [Statistics Sweden], Stockhlm. (In German.) Reprinted, 1970 (with preface and vocabulary in English), Urval No. 3, SCB [Statistics Sweden], Stockholm.

BIBLIOGRAPHY Hofsten, E. and Lundström, H. (1976). Swedish Population History: Main Trends from 1750 to 1970, Urval No. 8. SCB [Statistics Sweden], Stockholm. Nordenmark, N. V. E. (1939). Pehr Wilhelm Wargentin, Kungl. Vetenskapsakademins Sekreterare och Astronom, Almqvist & Wiksell, Uppsala. (In Swedish, with a summary in French.) Pearson, K. (1978). The History of Statistics in the 17th and 18th Centuries. Charles Griffin, London. ˚ (1983). Pehr Wargentin, Statistiska Centralbyran den Svenska Statistikens Fader. SCB [Statistics Sweden], Stockholm. (In Swedish, with a summary in English. Contains reprints of seven papers by Wargentin.) Westergaard, H. (1932). Contributions to the History of Statistics. King & Son, London. Reprinted, 1970, New York.

LENNART BONDESSON MARTIN G. RIBE

9038

WARING-HERDAN MODELS

WARING-HERDAN MODELS

probability measures P and Q is defined as

This is a term used in linguistic literature, for models of vocabulary distribution based on Waring distributions. See Herdan [1] and a survey by Holmes [2] for details. REFERENCES 1. Herdan, G. (1964). Quantitative Linguistics. Butterworth, London. 2. Holmes, D. I. (1985). J. R. Statist. Soc. A, 148, 328–341. See also FACTORIAL SERIES DISTRIBUTIONS; LINGUISTICS, STATISTICS IN; and LITERATURE AND STATISTICS —I.

W(P, Q) = inf E[ρ(ξ , η)], where the inf is taken over all possible pairs of random variables ξ and η with distributions P and Q, respectively (Wasserstein [9]). For the case of one-dimensional Euclidean space X = R1 , with usual Euclidean metric W(P, Q) =

|F(x) − G(x)| dx,

1

|F −1 (u) − G−1 (u)| du.

0

WARING’S DISTRIBUTION. See DISTRIBUTIONS;

−∞

where F and G are distribution functions from P and Q, respectively. ¨ Ruschendorf [6] presents a general definition of the form D(P, Q) =

SERIES FACTORIAL DISTRIBUTION

∞

YULE

WARING LIMITS. See CONTROL CHARTS WARNING LINES These are lines on a control chart∗ indicating a mild degree of departure from a desired level of control. More or less, conventionally, a deviation likely to be exceeded (in absolute value) with probability of about 5%, if production is in the desired state of control, is used as the basis for warning lines. In an X chart∗ , if the target average is µ and the population standard deviation (assumed known) is σ , the warning√lines for a sample of size the n are at µ ± 2σ/ n. ‘‘Action’’ lines, on √ other hand, are customarily at µ ± 3σ/ n. Of course, in practical use, ‘‘warning’’ and ‘‘action’’ can take various forms. See also CONTROL CHARTS and QUALITY CONTROL, STATISTICAL.

WASSERSTEIN DISTANCE For a metric space X with metric ρ, the Wasserstein distance W(P, Q) between

This form of distance was studied as early as 1956 by Dall’Aglio [1], and in 1958 by Kantoroviˇc and Rubinstein [4]. Calculations of Wasserstein distances are given in Vallander [8]. For the relation between Prohorov’s distance and Wasserstein distance, see Dobrushin [2] and Strassen as cited ¨ by Ruschendorf [6]. Using the concept of multivariate quantile transformation, ¨ Ruschendorf [6] obtained explicit formulas for the Wasserstein distance between multivariate distributions in certain cases. These may be compared with the Fréchet distance between multinormal distributions∗ [3,5]. A detailed discussion of Wasserstein distance and its applications in the theory of empirical processes∗ is given in Shorack and Wellner [7]. REFERENCES 1. Dall’Aglio, G. (1956). Annali Scuola Normale Sup. di Pisa, 10, 35–74. 2. Dobrushin, R. L. (1970). Theory Prob. Appl., 15, 458–486. 3. Dowson, D. C. and Landau, B. V. (1982). J. Multivariate Anal., 12, 450–455. 4. Kantoroviˇc, L. and Rubinstein, G. (1958). Vestn. Leningrad Univ. Mat., 13(7), 52–59. 5. Olkin, I. and Pukelsheim, F. (1982). Linear Algebra Appl., 48, 257–263.

WATSON’S U 2

9039

¨ 6. Ruschendorf, L. (1985). Zeit. Wahrscheinlichkeitsth. verwand. Geb., 70, 117–130.

REFERENCES

7. Shorack, G. B. and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York.

1. Box, G. E. P. (1954). Ann. Math. Statist., 25, 290–302.

8. Vallander, S. S. (1973). Theory Prob. Appl., 18, 784–786. 9. Wasserstein, L. N. (1969). Prob. Peredachi Informatsii, 5(3), 64–73 (in Russian). See also METRICS AND DISTANCES ON PROBABILITY SPACES.

This is the distribution of the ratio k

λj Wj

k

j=1

µj Wj ,

j=1

where µj > 0 for all j, λ1 /µ1 > λ2 /µ2 > · · · > λn /µn , and W1 , W2 , . . . , Wk are mutually independent random variables, each having the standard exponential distribution∗ with PDF fW (w) = e−w ,

=

m

(λi − rµi

for Pr[R r] =

k i=m

See also QUADRATIC FORMS; SERIAL CORRELATION; and SERIAL CORRELATION, DURBIN –WATSON TEST FOR.

The Watson U 2 -statistic [23] for testing goodness of fit∗ is a statistic derived from the empirical distribution function (EDF)∗ of a given random sample. It measures the discrepancy between the EDF and the hypothesized distribution, which for the present is assumed to be continuous; the distribution may be completely specified, or may contain unknown parameters. Suppose Fn (x) is the EDF of a sample of x-values of size n, and F(x; θ ) is the hypothesized distribution, with θ , the vector of parameters, fully known; then

)k−1

,

kj=1,j=i {(λi − λj ) − r(µi − µj )}

i=1

3. Watson, G. S. (1955). Austral. J. Phys., 8, 402–407.

0 < w.

It is a special case of the distribution of the ratio of quadratic forms in normal variables (Box [1]). Watson [3] obtained the (equivalent) formulas Pr[R r]

λm λm+1 r , µm µm+1

(rµi − λi

)k−1

kj=1,j=i {(µi − µj )r − (λi − λj )}

(1974).

WATSON’S U 2

WATSON’S DISTRIBUTION

R=

2. Dent, W. T. and Broffitt, J. D. J. R. Statist. Soc. B, 36, 91–98.

U2 = n −

∞

−∞

∞

Fn (x) − F(x; θ )

−∞

2

{Fn (x) − F(x; θ )}dF(x; θ )

(1) For practical computations, U 2 can be calculated as follows. Let x(1) < x(2) < · · · < x(n) be the order statistics∗ of the sample and let z(i) = F(x(i) ; θ ), i = 1, . . . , n; the z(i) will also be in ascending order and will lie between 0 and 1. Then U2 =

,

λm−1 λm for r . µm−1 µm For computations, one would use the formula with the smaller number of terms. The mean and variance of the distribution have been obtained by Dent and Broffitt [2].

dF(x; θ ).

n {z(i) − (2i − 1)/(2)}2 i=1

−n(z − 0.5)2 + 1/(1).

(2)

U 2 is a modification of the well-known Cramér-von Mises statistic∗ , now usually called W 2 ; it was introduced by Watson for use with observations Pi recorded on the circumference of a circle (see DIRECTIONAL DATA ANALYSIS). For such observations, x is the

9040

WATSON’S U 2

arc length on the circumference, from an origin O to Pi . The value of W 2 depends on the choice of origin O, but U 2 is invariant. Thus different statisticians, presented with test values on a circle and choosing different origins, will find the same value of the test statistic U 2 , whereas values of W 2 will differ. In fact, U 2 is the minimum value of W 2 as the origin is varied. Kuiper’s statistic V is the corresponding invariant modification of the Kolmogorov-Smirnov∗ D. DISTRIBUTION THEORY AND TABLES Suppose the null hypothesis is H0 : a sample of x-values, x1 , x2 , . . . , xn comes from a continuous distribution F(x; θ ), with θ specified. This situation will be called case 0. The asymptotic distribution of U 2 on H0 was given by Watson [23]; as for other Cramérvon Mises statistics, the asymptotic distribution is that of a sum of weighted independent chi-square variables, each with d.f. = 1. For U 2 the weights occur in equal pairs, so that the distribution is a sum of weighted exponentials, and percentage points are easily calculated. An interesting result is that the asymptotic distributions of π 2 U 2 and of nD2 are the same; this seems difficult to explain on intuitive grounds. Also, the asymptotic distribution of U 2 is a convolution of the asymptotic distribution of W 2 /4 with itself. For small samples, Stephens [12,13] gave exact distribution theory for U 2 on H0 ; this is based on the fact that the values zi = F(xi ; θ ) will be uniformly distributed between limits 0 and 1, written U(0, 1). Moments of U 2 were also found and percentage points were calculated by fitting Pearson curves. The distribution of U 2 was also discussed by Pearson and Stephens [9] and by Tiku [22]. Subsequently, Stephens [15] found a technique to make U 2 available for case 0 tests using only the asymptotic points. Large values of U 2 lead to rejection of H0 . The value of U 2 is found from (2), then modified to U ∗ given by 0.1 0.1 0.8 U∗ = U2 − + 2 , 1.0 + n n n and U ∗ is referred to the upper-tail asymptotic percentage points. These are 0.152,

0.187, 0.222, and 0.268 for α = 0.10, 0.05, 0.025, and 0.01, respectively. In a later work, Pettitt and Stephens [10] have given asymptotic points for right-censored data of either type 1 or type 2. For n finite, percentage points for type 1 censored data∗ converge so rapidly to the asymptotic points that no further tables are necessary; for type 2 censored data Stephens [21] has given points found by Monte Carlo methods. POWER FOR CASE 0 Although U 2 was designed for values on a circle, it can be a useful statistic also for values on a line. U 2 will detect if the zvalues, which should be U(0, 1), are tending to lie in a single cluster somewhere along the line, or in two clusters at 0 and 1 (which, wrapped around a circle of circumference 1, would make a single cluster). In contrast, EDF statistics∗ W 2 and A2 will be significant if z-values have moved strongly toward 0 or toward 1. In practical terms, this means that if vector θ contains location parameter α and scale parameter β, U 2 will detect if β has been wrongly specified in F(x; θ ), and W 2 and A2 will detect if α has been wrongly specified. Asymptotic power of U 2 has been discussed by Stephens [16], using the technique of decomposing the statistic into orthogonal components, by a Fourier √ series expansion of n{Fn (z) − z}, where Fn (z) is the EDF of the z(i) above. Monte Carlo comparisons of tests for uniformity, including U 2 , have been given by Stephens [17], Locke and Spurrier [3], and Miller and Quesenberry [8]. TESTS WITH UNKNOWN PARAMETERS EDF statistics, including U 2 , may be adapted to test for F(x; θ ) with unknown components in the parametric vector θ . These components must then be estimated from the given sample; let θˆ be the vector with estimated components where necessary, and ˆ 2 be the definition of U 2 as in (1) let U but with F(x; θˆ ) replacing F(x; θ ). Similarly, zˆ i = F(x; θˆ ) replaces zi in the computational ˆ 2 . Asymptotic distriformula (2), to give U ˆ 2 , when the estimators of bution theory for U

WATSON’S U 2

unknown components of θ are maximum likelihood or other asymptotically efficient estimators, has been given by Stephens [17]–[21] for test for the normal, exponential, extremevalue, logistic, and Cauchy distributions; for some of these distributions, modifications of the case 0 type above have been devised so that the tests can be made with a complete sample of size n, using only the asymptotic percentage points. Pettitt and Stephens [11] and Lockhart and Stephens [4,5] have given percentage points for tests for the gamma distribution∗ and the von Mises distribution∗ . A test for the Weibull distribution∗ , when only the scale and/or shape parameters are unknown, can be reduced to a test for extreme-value distribution∗ ; Lockhart and Stephens [6] have given points to make the test, when, in addition, the location (origin) of the Weibull distribution is unknown. It would be difficult to develop the asymptotic theory in analytic form for censored data, and points for tests with such data are not available. Most of the tables mentioned above are collected together in Stephens [21]. When statistics such as W 2 , A2 , and U 2 are used with estimated parameters, the variation in ability to detect different types of alternative tends to disappear. Power studies indicate that in many situations (except perhaps for the Cauchy distribution), U 2 appears marginally less powerful than W 2 or A2 when used for points on a line; again, of course, only U 2 should be used for tests for the von Mises distribution, which is a distribution on a circle. For further discussion, see Stephens [21]. TEST FOR DISCRETE DISTRIBUTIONS Freedman [2], has adapted the U 2 -test for a discrete distribution. The null distribution now depends on the number of cells n and on the probabilities pi of falling into the cells; Freedman shows how the distribution may be approximated when these parameters are specified. The discrete version of U 2 is suitable for data such as measurements of angles, which have been grouped; also for counted data, recorded, say, monthly, and displayed in 12 cells around a circle denoting one year. Freedman [2] gives an example.

9041

MULTISAMPLE TESTS 2 for testing that Watson [24] developed UM,N two samples of sizes N and M have the same (unspecified) continuous distribution and showed that the null asymptotic distribution is the same as for the one-sample statistic. Burr [1] gave percentage points and Stephens [14] gave moments and percentage 2 points for UM,N . Maag [7] extended Watson’s work to a k-sample U 2 -test. These multisample tests are essentially based on the ranks of the observations in their own samples and in the combined sample; although not many studies have been made, these tests appear to be comparable in power to other rank tests∗ .

REFERENCES 1. Burr, E. J. (1963). 1091–1098. 2. Freedman, L. S. 708–711.

Ann.

(1981).

Statist.,

34,

Biometrika,

68,

3. Locke, C. and Spurrier, J. D. (1978). Commun. Statist. A. 7, 241–258. 4. Lockhart, R. A. and Stephens, M. A. (1985a). Biometrika, 72, 647–653. 5. Lockhart, R. A. and Stephens, M. A. (1985b). Goodness-of-Fit Tests for the Gamma Distribution. Tech. Rep., Dept. of Mathematics and Statistics, Simon Fraser University, British Columbia, Canada. 6. Lockhart, R. A. and Stephens, M. A. (1986). Tests for the Weibull Distribution Based on the Empirical Distribution Function. Tech. Rep., Dept. of Mathematics and Statistics, Simon Fraser University, British Columbia, Canada. 7. Maag, U. R. (1966). Biometrika, 53, 579–583. 8. Miller, F. L. and Quesenberry, C. P. (1979). Commun. Statist. B, 8, 271–290. 9. Pearson, E. S. and Stephens, M. A. (1962). Biometrika, 49, 397–402. 10. Pettitt, A. N. and Stephens, M. A. (1976). Biometrika, 63, 291–298. 11. Pettitt, A. N. and Stephens, M. A. (1983). EDF Statistics for Testing for the Gamma Distribution. Tech. Rep., Dept. of Statistics, Stanford University, Stanford, CA. 12. Stephens, M. A. 303–313.

(1963).

Biometrika,

50,

13. Stephens, M. A. 393–397.

(1964).

Biometrika,

51,

9042

WAVE-CUTTING INDEX

14. Stephens, M. A. 661–663.

(1965).

Biometrika,

52,

15. Stephens, M. A. (1970). J. R. Statist. Soc. B, 32, 115–122. 16. Stephens, M. A. (1974a). Ann. H. Poincaré Sect. B, 10, 37–54.

Inst.

17. Stephens, M. A. (1974b). J. Amer. Statist. Ass., 69, 730–737. 18. Stephens, M. A. (1976). Ann. Statist., 4, 357–369. 19. Stephens, M. A. 583–588.

(1977).

Biometrika,

64,

20. Stephens, M. A. 591–595.

(1979).

Biometrika,

66,

21. Stephens, M. A. (1986). In Goodness-of-fit Techniques, R. B. D’Agostino and M. A. Stephens, eds. Dekker, New York, Chap. 4. 22. Tiku, M. L. (1965). Biometrika, 52, 630–633. 23. Watson, G. S. 109–114.

(1961).

Biometrika,

48,

24. Watson, G. S. (1962). Biometrika, 49, 57–63. See also EMPIRICAL DISTRIBUTION FUNCTION (EDF) STATISTICS; KOLMOGOROV –SMIRNOV STATISTICS; and RENYI-TYPE DISTRIBUTIONS.

For all graduation formulas satisfying the basic condition that u∗x = θ if uy = θ for all y, the sum of the coefficients is 1, so the index represents the ‘‘proportion’’ of the graduated value contributed by observed values near x. The lower the value of the index, the greater the wave-cutting power. For Spencer’s 21-point formula∗ , for example, the value of the wave-cutting index is 0.766; for Hardy’s wave-cutting formula, u∗x = {[5][3]{[3] + [5] − [7]}/65}ux (where [·] is the summation operator [2r + 1]ux = rj=−r ux+j ), the index value is 0.415. For further examples, see [1, p. 287]. REFERENCE 1. Benjamin, B. and Pollard, J. H. (1980). The Analysis of Mortality and Other Actuarial Statistics, 2nd ed. Heinemann, London, England. See also GRADUATION; SUMMATION [n]; and TWICING.

M. A. STEPHENS

WAVELETS WAVE-CUTTING INDEX This is an index reflecting the effectiveness of a linear m-point graduation∗ formula u∗x = an ux−n + · · · + a1 ux−1 + a0 ux +a1 ux+1 + · · · + an ux+n , m = 2n + 1, or u∗x = an ux−n+1/2 + · · · + a1 ux−1/2 +a1 ux+1/2 + · · · + an ux+n−1/2 , m = 2n, in reducing the effect of local variation (e.g., of wild values near ux ). The index is a0 + 2a1 + 2a2 ,

for m = 2n + 1,

2a1 + 2a2 + a3 ,

for m = 2n.

A wavelet is a function that, roughly speaking, looks like a localized wiggle. Special collections of wavelets can be used to obtain representations of functions and signals by separating information at different scales. Such representations have proven to be powerful tools with application to diverse disciplines including quantum physics, fluid mechanics, astronomy∗ , seismology, acoustics, communications, computer vision, signal and image processing∗ , numerical analysis∗ , approximation theory, functional analysis, operator theory, and the study of complex functions. The stunning success of wavelets in these areas has spurred an explosive growth of research on the development of methodology and generalizations, but only recently, with the work of Donoho and Johnstone, have wavelets entered the mainstream of statistical research. Their influence on statistical practice is still evolving. In its simplest form, a wavelet representation begins with a single function ψ, often called a mother wavelet. We define

WAVELETS

a collection of wavelets—called a wavelet basis—by dilation and translation of the mother wavelet ψ: ψj,k (t) = 2j/2 ψ(2j t − k)

(1)

for j, k ∈ Z, the set of integers. Each ψj,k has a characteristic resolution scale (determined by j) and is roughly centered on the location k/2j . It turns out that if ψ is properly chosen, any ‘‘reasonable’’ function (in a sense to be described) can be represented by an infinite linear combination of the ψj,k ’s. Much of wavelets’ potential in applications stems from the conjunction of three key features: 1. Wavelets provide sparse representations of a broad class of functions and signals. 2. Wavelets can achieve very good localization in both time and frequency. 3. There are fast algorithms for computing wavelet representations in practice. Here, sparsity refers to the fact that most of the coefficients in the linear combination are nearly zero. A single wavelet basis can provide sparse representations of many spaces of functions at the same time. Good localization means that both the wavelet function ψ and its Fourier transform have small support; i.e., the functions are essentially zero outside some compact domain. Regarding speed, the benchmark for a fast algorithm is the fast Fourier transform (FFT), which is O(n log n) in the number of data (see O, o NOTATION). And indeed, wavelet representations can be computed in O(n log n) and sometimes O(n) operations. These features also underlie much of the power of wavelets for statistical applications, so to impart a full understanding of wavelet methods, we must explore their implications and advantages. We will begin by addressing three critical questions in the next three sections. First, we show how to construct a wavelet basis and carry out the so-called discrete wavelet transform (DWT), which is at the heart of wavelet methods. Second, we describe the mathematical reasons why wavelets are effective. Third, we carry these ideas to a statistical context and explain the

9043

power of wavelet methods in statistical problems. Then, we turn to more concrete examples and describe some wavelet methods relevant to specific statistical problems. Although these methods are still under active development, they paint a useful picture of both the potential and the limitations of wavelets in statistics. Finally, we briefly discuss generalizations and extensions of the basic wavelet formulation. The possibilities are just beginning to be explored, but the theoretical and practical successes of wavelets and their generalizations suggest that they will be a part of statistical research and practice for the foreseeable future. MAKING WAVELETS We begin with a mathematical description of a smooth localized wiggle. Let Lp (R) denote the space of measurable complex-valued functions f on the real numbers R such that ||f ||p < ∞ where

( |f |p )1/p 1p J0 as representing the fine structure. We approximate the fine-resolution structure of a function by a linear combination of the ψj,k at j J0 , and we approximate the coarseresolution structure by a combination of the functions φ0,k (t) = φ(t − k) for k ∈ Z, where φ is called the father wavelet (or scaling function). The inhomogeneous representation for f ∈ L2 (R) is then f , φ0,k φ0,k + f , ψj,k ψj,k . (5) f = jJ0 k∈Z

k∈Z

This function φ is intimately related to ψ as described below. We can choose φ so that φ, ψ = 0, and it must satisfy the first two requirements of a basic wavelet with the same indices of regularity as does ψ. The difference between the inhomogeneous and homogeneous representations lies in the first term above; the coarse structure of f can be described either by wavelets at every resolution level or by aggregating the information in the low-resolution levels in terms of the specially constructed function φ. The prime advantage of the inhomogeneous representation is that, despite the notational complexity, the coarse–fine dichotomy is intuitively appealing and useful. In the example of the Haar basis, the father wavelet is the indicator of the unit interval, φ = 1[0,1) . When J0 = 0, for example, the representation (5) takes the form f =

k∈Z

αk φ0,k +

βj,k ψj,k ,

(6)

j0 k∈Z

where αk = f , φ0,k and βj,k = f , ψj,k . The coefficients αk are just integrals of f over intervals of the form [k, k + 1); the remaining

structure in f is represented as fluctuations within those intervals. Note that φ is in L2 (R), and consequently can be approximated by the ψj,k ; indeed, φ=

∞ 1 1 −j/2 2 ψ−j,0 . ψ−1,0 + 2 2

(7)

j=2

That this uses only the coarsest of the ψj,k ’s verifies the dichotomy between coarse and fine structure. The relationship between ψ and φ goes the other way as well: ψ = 1[0,1/2) − 1[1/2,1) by definition, which is a linear combination of translated and dilated φ’s [indeed, a combination of the φ1,k ’s, defined as in (1)]. MULTIRESOLUTION ANALYSIS AND SUCCESSIVE REFINEMENT A recurring idea in the theory of wavelets is that of successive refinement. The underlying idea is simple: If we can approximate a function space at several levels of accuracy, then the differences between successive approximations characterize the refinements that are required to move from one level of approximation to another. A multiresolution analysis (MRA) formalizes this notion for approximations that are related by a translation and dilation. An MRA is a progression of nested approximation spaces for a containing class of functions. When the containing class is L2 (R), we define an MRA as a nested sequence of closed subspaces of L2 (R), · · · ⊂ V−2 ⊂ V−1 ⊂ V0 ⊂ V1 ⊂ V2 ⊂ · · · , (8) that satisfy   clos  Vj  = L2 (R),

(9)

j∈Z

Vj = {0},

(10)

j∈Z

f ∈ Vj ⇔ f (2−j ·) ∈ V0 ∀j ∈ Z,

(11)

f ∈ V0 ⇔ f (· − k) ∈ V0 ∀k ∈ Z.

(12)

The condition (9) ensures that the approximation spaces Vj are sufficient to approximate any function in L2 (R).

WAVELETS

There are many sequences of spaces satisfying (8), (9), and (10); the name ‘‘multiresolution’’ here is derived from (11), which implies that all the Vj are dyadically scaled versions of a common space V0 . By (12), this V0 is invariant under integer translations. When generating orthonormal wavelet bases, we also require that the space V0 of the MRA contain a function φ such that the integer translations {φ0,n } form an orthonormal basis for V0 . For simplicity, we will focus almost exclusively on orthonormal bases here. Since V0 ⊂ V1 , we can define its orthogonal complement W0 in V1 , so V1 = V0 ⊕ W0 . We can do likewise for every j, where Vj+1 = Vj ⊕ Wj . The sequence of spaces {Wj }j∈Z is a mutually orthogonal one; they inherit the property (11) from the Vj ; i.e., f ∈ Wj if and only if f (2−j ·) ∈ W0 . By themselves, these spaces provide a homogeneous representation of L2 (R), since L2 (R) = clos {∪j∈Z Wj }. They are the building blocks for successive refinement from one approximating space to another. Given a function f ∈ L2 (R), the best approximation to f in any Vj is given by Pj f , where Pj is the orthogonal projection onto Vj . It follows that Qj = Pj − Pj−1 is the orthogonal projection onto Wj . Given a coarse approximation P0 f , we can refine to the finer approximation PJ f for any J > 0 by adding the details from the successive Wj spaces. Specifically, PJ f = P0 f +

J

(Pj f − Pj−1 f )

j=1

= P0 f +

J−1

Qj f ,

j=0

and as J → ∞, f = P0 f +

∞

Qj f .

(13)

j=0

9047

As an example of an MRA, suppose that Vj is the set of functions in L2 (R) that are piecewise constant on intervals of the form [k2−j , (k + 1)2−j ). In particular, V0 is generated by integer translations of the function φ = 1[0,1) , which is the father wavelet for the Haar basis. For a given f ∈ L2 (R), φj,k , f . Then let αj,k be the coefficients P0 f = k α0,k φ0,k and P1 f = k α1,k φ1,k are the approximations to f that are piecewise constant on integer and half-integer intervals respectively. What must we do to refine the coarse approximation P0 f to the next higher resolution level? We have that 1 α0,k = √ (α1,2k + α1,2k+1 ), 2 but we also need to know the difference 1 β0,k = √ (α1,2k − α1,2k+1 ) 2 between the integrals of f over the half intervals [k, k + 12 ) and [k + 12 , k + 1). This β0,k is just the coefficient ψ0,k , f of the (0, k) Haar wavelet ψ0,k ; indeed, translations of the Haar mother wavelet form an orthonormal basis for the space W0 . But given a general MRA for L2 (R), how do we find the corresponding ψ? It turns out that ψ and φ determine each other through the refinement relations among the spaces derived from an MRA. In particular, suppose that we can construct a function ψ whose integer translations [i.e., ψ0,k (t) = ψ(t − k)] yield an orthonormal basis of W0 . By construction, both ψ and φ are in V1 , and the ψ0,k and φ0,n are orthogonal. It follows that both φ and ψ can be expressed as a linear combination of the φ1,n with some constraints on the coefficients. This reasoning leads to the following two-scale identities: φ(t) =

√ 2 gn φ(2t − n),

(14)

n

This successive refinement exactly mimics the inhomogeneous wavelet representation: the coarse approximation P0 f corresponds to a linear combination of the φ0,k , and each Qj f for j 0 corresponds to a linear combination of the ψj,k .

ψ(t) =

√ 2 hn φ(2t − n).

(15)

n

The sequences {hn } and {gn } satisfy 2 2 n |hn | = 1 and n |gn | = 1. The orthogonality among the φ0,k (t) and ψ0,k and the fact

9048

WAVELETS

that V1 is a direct sum of V0 and W0 induce relationships among the sequences {gn } and {hn }. Given a specific MRA, these relationships imply enough conditions to identify specific mother and father wavelets φ and ψ. In the Haar √ case, √ the sequences are {gn } = (. .√ . , 0, 1/ √ 2, 1/ 2, 0, . . .) and {hn } = (. . . , 0, 1/ 2, −1/ 2, 0, . . .), as we derived above by successive refinement. In general, it is typically more convenient to work with the two-scale identities in the Fourier domain so as to characterize the Fourier transforms φˆ ˆ See ref. 14 for details and examples. and ψ. The algorithmic value of the two-scale identities lies in the connection they require among the wavelet coefficients. By the twoscale identities, φj,k (t) = 2(j+1)/2 gn φ(2j+1 t − 2k − n), n

ψj,k (t) = 2

(j+1)/2

hn φ(2j+1 t − 2k − n),

n

and hence, φj,k , f =

gn φj+1,2k+n , f

n

=

gn−2k φj+1,n , f ,

(16)

n

ψj,k , f =

hn φj+1,2k+n , f

n

=

hn−2k φj+1,n , f .

(17)

n

Each of these is the result of convolving the sequence of higher-resolution inner products with the reversed sequences {gn } and {hn } and then retaining only the even-numbered components of that convolution (this is called decimating the sequence by factor 2). Consequently, if we start with the coefficients φJ,k , f for k ∈ Z at some fixed resolution level J, we can compute the coefficients at all coarser levels by successively filtering and decimating the sequences. This observation is at the heart of the discrete wavelet transform (DWT). EMPIRICAL WAVELET COEFFICIENTS AND THE DISCRETE WAVELET TRANSFORM The convolution identities in (19) and (21) suggest a procedure for computing the

wavelet coefficients of a function f . Start with the coefficients αJ1 +1,k = φJ1 +1,k , f from a fine-resolution approximation to f at level J1 + 1. By applying the convolutions in (19) and (21), we compute αJ1 ,k = φJ1 ,k , f and βJ1 ,k = ψJ1 ,k , f . The βJ1 ,k are the wavelet coefficients corresponding to level J1 , which we save; then we apply the same procedure to the αJ1 ,k to compute the αJ1 −1,k and the βJ1 −1,k . We continue in this way until we reach some coarser level J0 , at which point we have the wavelet decomposition for the fine-resolution approximation PJ1 f : PJ1 f =

k∈Z

αJ0 ,k φJ0 ,k +

βj,k ψj,k .

J0 jJ1 k∈Z

(18) At each stage, we move to a coarser level of approximation by peeling off the detailed fluctuations at the given resolution level. Let L denote the operation of (low-pass) filtering with the sequence {gn } followed by decimation by factor 2, and let H denote the operation of (high-pass) filtering with the sequence {hn } followed by decimation by factor 2. The sequences {gn } and {hn } yield complementary filters, corresponding to different bandwidths. The filtering reduces the amount of information in the result by a factor of 2 (since the result is effectively bandlimited); this is the information that is removed by the decimation operation [14,55]. Thus, given an input sequence {νn }, L and H effectively partition the frequency information in {νn } into two overlapping but complementary pieces. The coarse wavelet coefficients αJ0 ,. are computed by successive applications of L (αJ0 ,. = LJ1 +1−J0 αJ1 +1,. ), whereas the fine coefficients βj,k are obtained by extracting the high-frequency information at the final stage (βj,. = HLJ1 −j αJ1 +1,. ). This highpass term makes sense in the context of the successive refinement interpretation: the β’s describe the fluctuations that are required to refine the approximation from one resolution level to the next. Given the final coefficients, we can reconstruct the input by appropriately reversing the procedure. At each stage, we interleave both the corresponding α and β coefficients with zeros, apply the adjoint filters, and add together the results to obtain the α coefficients at the next finer resolution

WAVELETS

level. Mathematically, if L∗ and H ∗ represent the adjoint operators (e.g., g∗n = g−n ), we have the relationship I = L∗ L + H∗ H [38,42], where I is the identity operator on the space of sequences 2 (R). Hence, αj+1,. = L∗ αj,. + H∗ βj,. . One weak link in this procedure is that we assumed the coefficients φJ1 +1,k , f to be known, which is generally not so. However, if J1 is sufficiently large, we might approximate αJ1 +1,k ≈ c2−(J1 +1)/2 f (k/2J1 +1 ) for a constant c = 0. The thinking behind this is that if (D, M) is the regularity of the wavelets induced by the MRA, then xm φ(x)dx = 0 for m = 1, . . . , M, and thus a Taylor expansion∗ argument suggests that this is a reasonable approximation for f sufficiently smooth that higher-order derivatives are small. Thus, given samples of the function f on the fine dyadic grid k/2J1 +1 for k ∈ Z, we can use the samples on the fine grid to start the iterative filtering and compute the wavelet coefficients as above. These are the empirical wavelet coefficients of the function f based upon samples on a dyadic grid. The empirical wavelet coefficients are only approximately the coefficients of f corresponding to inner products with the basis elements; they are, however, the true wavelet coefficients of f with respect to a slightly modified basis. The precise relationship between the empirical wavelet coefficients and the actual inner products is discussed in detail in refs. [26,31]. Another issue is that in practice one does not have an infinite sequence of samples, however good the approximation to the αJ1 +1,k . With only a finite number of samples on the function f , the boundary effects can become important. The simplest solution is to treat f as a periodic function over the given interval and to filter using circular convolution. The case in which the initial samples are at points k/2J1 +1 for k = 0, . . . , 2J1 +1 − 1 is particularly instructive. Here, the DWT takes the form of an orthogonal matrix U that carries the input vector to its empirical wavelet coefficients. The matrix U T inverts the transform and returns the original input. Here, the wavelet coefficients are stored as a vector of the form (α J0 ,. , β J0 ,. , . . . , β J1 ,. ) where each subvector (α J0 ,. , β J0 ,. , and so forth) at resolution level j has length 2j . The matrix

9049

U is obtained by stacking the matrices for the operators LJ1 +1−J0 and HLJ1 −j for j = J0 , . . . , J1 . If implemented as a matrix multiplication, the algorithm is effectively O(n2 ), but by taking advantage of the structure of the successive filtering operations, the DWT can be computed in O(n) time. Figure 3 shows the empirical wavelet coefficients for a given function using two different wavelet bases, the Haar wavelets and the so-called Symmlet (8) wavelet [13]. Panel (a) plots samples of the function. Panel (b) plots the mother wavelet for the Symmlet; the Haar mother wavelet was shown in Fig. (1). Panels (c) and (d) show the wavelet coefficients for the Haar and Symmlet wavelets respectively. These coefficient plots identify the coefficients by resolution level (along the y-axis) and location (each spike is located roughly at the center of the corresponding wavelet’s support). The Symmlet coefficients are much sparser for this function. Panels (e) and (f) show successive approximations to the function at different resolution levels, using the Haar and Symmlet wavelets, respectively. The periodic, orthonormal DWT was used to produce these results. The restriction to periodic functions poses serious practical problems, so in general more sophisticated handling of the boundary is required. One approach is to modify the structure of the MRA to allow for the edges, while maintaining equivalence to the periodic MRA ‘‘inside’’ the interval. An algorithm based on this idea is given in ref. 13; it preserves the orthogonality, vanishing moments, smoothness, and compact support of the wavelets on the real line or the circle. The algorithm uses special filters to condition the data on the edges before and after the standard filtering iterations. See refs. [1,13,14,57] for a discussion of this algorithm and others. NONORTHOGONAL WAVELET BASES Orthonormal wavelet bases are convenient both for mathematical analysis and for computation, but there are other ways to specify wavelets that are advantageous in many applications. One example is frames [2,3,14], a collection of nonorthogonal wavelets with specified redundancies. Another important

Figure 3. 9050

Figure 3. (continued) 9051

Figure 3. (continued) 9052

WAVELETS

example is a biorthogonal wavelet basis [11,12,14]. The idea here is to construct two bases, {ψj,k } and {ψ˜ jk }, that are not themselves orthogonal but are rather biorthogonal: ψj,k , ψ˜ j k = δjj δkk . The (homogeneous) wavelet representation of a function f then takes the form f , ψj,k ψ˜ jk = f , ψ˜ jk ψj,k . (19) f = j,k∈Z

j,k∈Z

The primary advantage of biorthogonal representations is that the two bases can have very different properties. For instance, if the ψj,k are smooth with small support and the ψ˜ jk integrate to zero for all polynomials up to some fixed degree, then the second representation in (23) yields a high-order approximation of f in terms of smooth, concentrated elements. Other interesting examples of alternative wavelet families include the nonorthogonal interpolating wavelets of refs. [35,16] and the average interpolating wavelets of refs. [31,75]. WHY WAVELETS WORK Wavelets have a long history and have appeared in many different guises under many different names within many different fields. Recognition of these commonalities across disciplines has spurred unification of the underlying ideas, and the apparent empirical success of wavelet methods in applications has fueled interest. For a full mathematical history of the development of wavelets, see refs. [55,56]; ref. 14 gives references to other accounts, particularly in physics. Wavelets are natural descendents of ‘‘waves,’’ e.g., sinusoids, so in order to understand them, it helps to go back to these roots—Fourier analysis [81]. Given a function f ∈ L2 [0, 2π ), we can represent f by a Fourier series: ∞

f (x) ∼

ck eikx ,

(20)

k=−∞

where ck =

1 2π

2π 0

f (x)e−ikx dx.

(21)

9053

The ∼ indicates that the Fourier series need not converge pointwise to f , although it does converge in L2 . Many key properties of the function f are revealed by the sequence of Fourier coefficients∗ . For instance, 1. If f is integrable, then |ck | → 0 as |k| → ∞. 2. If f ∈ Hölder (α) for 0 < α < 1, then |ck | = O(k−α ). The space Hölder (α) contains the functions f such that |f (x) − f (y)| C|x − y|α for some constant C > 0 and all x, y. 3. If f has m continuous derivatives, then |ck | = o(k−m ) and ||Dl f ||2 = |ck |2 k2l , where D is the derivative operator. These results allow us to ‘‘read off’’ certain properties of f from its sequence of Fourier coefficients. The Fourier series is particularly well adapted to L2 measures of smoothness (i.e., L2 norms of derivatives); for instance, the expression for ||Dl f ||2 in terms of the Fourier coefficients implies that shrinking the Fourier coefficients towards zero yields a ‘‘smoother’’ function, in some overall sense (see SHRINKAGE ESTIMATORS). However, other measures of smoothness and other properties of f are not as easily accessible from the Fourier representation. For example, a singularity in f affects all of the Fourier coefficients and so cannot be localized by examining the coefficients alone. Similarly, we cannot determine the continuity of f from the Fourier coefficients: There is a uniformly continuous function f on [0, 2π ) with Fourier coefficients {ck } and a discontinuous function g with singularities on a dense subset of [0, 2π ) whose Fourier coefficients {dk } satisfy |dk | |ck | [50]. A related question is whether examination of the Fourier coefficients allows us to bound the Lp (R) norm of a function. The answer is a qualified yes, but direct examination of the coefficients is insufficient for the task. It is possible, for example, to construct two functions f1 and f2 where / Lp , and yet the Fourier coeffif1 ∈ Lp , f2 ∈ cients of these functions satisfy |c2k | |c1k |. The qualification above is that the characterization depends on looking at the coefficients in groups rather than individually.

9054

WAVELETS

This grouping of coefficients is an early incarnation of wavelet analysis [54,41]. Suppose that the function f ∈ Lp has Fourier coefficients {ck }; then define Bj f (x) = ck eikx , j 0. (22) 2j |k| 0 such that |αi | |βi | for all i implies || i αi fi || C|| i βi fi ||. Hence, if a function has finite norm in the space, any function with coefficients of smaller magnitude must as well. Any conditions on the coefficients {αi } with respect to an unconditional basis that are necessary and sufficient for the sequence k αi fi to converge in the space must depend only on the magnitudes {|αi |}. The Fourier basis is only an unconditional basis for Lp [0, 1] when p = 2. Indeed, for p = 2, the examples discussed above show that membership in Lp depends on very delicate phase relationships among the Fourier coefficients. Very special cancellation among the terms must result for a Fourier series to have nice properties. On the other hand, the Haar basis provides an unconditional basis for Lp [0, 1] for all p > 1. More general wavelet bases can be constructed that are simultaneously unconditional bases for an even broader class of function spaces, including the Besov [63,17] and Triebel [76] families. The Besov and Triebel families include all of the commonly used function classes (e.g., Lp , Sobolev, Hölder, and Zygmund spaces) and many other spaces containing functions with interesting and quite distinctive structure. The work of Donoho and Johnstone, described in the next section, contains a discussion of the importance of these spaces. The unconditional basis property explains many of the most important properties of wavelet bases, in particular the tendency of wavelet expansions to be sparse and the effectiveness of procedures that act on individual wavelet coefficients. Shrinking the coefficients of an object with respect to an unconditional basis can never take the object out of the space, and tends to lead to a smoother object. Moreover, the sequence of coefficients is highly compressible [27]. The unconditional basis property has another striking implication: wavelets per se are not special. In any given application, the goal is to find an unconditional basis for the function spaces that represent the objects of interest. Sometimes wavelets will be the basis of choice, but there are many other possibilities, as indicated in the last section, for spaces to which wavelets are not well-adapted.

WAVELETS

WAVELETS AND FUNCTION ESTIMATION The statistical potential of wavelets was convincingly demonstrated and thoroughly explored in the seminal work of Donoho and Johnstone [18,19,20,21,22,23,24,25,26,27, 28,29,30,31,32,33,34]. This work showed that, in the nonparametric regression∗ problem, a simple nonlinear estimator obtained by shrinking the empirical wavelet coefficients is essentially optimal (in the sense of minimax estimation∗ ) over a broad range of prior function classes. These results provide a unifying thread for almost two decades of research on the problem of function estimation. Donoho and Johnstone develop three general themes that are at the heart of the ‘‘wavelet miracle’’: (1) spatial adaptation, (2) simultaneous near-optimality, and (3) sparseness and coordinatewise shrinkage. Spatial adaptation refers to the ability of wavelet estimators to automatically adapt to local features of the estimated function. Simultaneous near-optimality means that a single wavelet procedure is very close to optimal over a broad range of prior constraints. Finally, because of the unconditional basis property, wavelet bases provide a sparse representation of many function spaces for which nonlinear shrinkage of the wavelet coefficients separately yields effective denoising. Donoho and Johnstone’s work is reviewed in ref. 22, which places their themes in a historical perspective. The Nonparameteric Regression Problem Classical nonparametric regression, like density and spectral density estimation∗ , is a canonical problem that has served as a foundation for the evaluation and development of new statistical methods. It is simply stated: we observe data yi = f (ti ) + i ,

i = 1, . . . , n,

(25)

where the ti are equally spaced points in the unit interval, f is a function in an infinitedimensional function class F, specified a priori, and the i are independent and identically distributed (i.i.d.) N(0, σ 2 ). The problem is to estimate f from these data.

9055

The function class F serves as the prior information in the problem; the conditions defining F describe the types of structure that may be represented in the function to be estimated. It is common to take F to be a ball in a familiar space such as an Lp , Sobolev∗ , or Hölder∗ space. For example, the Sobolev space L2 2 consists of the functions with at least two continuous derivatives sat isfying ||f ||22,2 = |f |2 + |f |2 < ∞. A ball in this space is the set of functions for which ||f ||2,2 C for some C > 0. Roughly speaking, this prior constraint prescribes limits on the average magnitude and curvature of the function. These standard spaces most often arise because the constraints induced are mathematically tractable. All of the commonly used spaces belong to the Besov or Triebel family of function spaces [63,76]. The latter spaces contain a wide range of different structures, including and beyond those previously considered in the nonparametric regression literature. A large body of research [70,65,66,46,72, 33,37,52,61] has been devoted to developing optimal procedures for the nonparametric regression problem. Optimality here is in the minimax∗ sense with respect to a specified loss function L(fˆ , f ); a minimax optimal estimator minimizes the worst-case loss over the set F. For any given sample size, the optimal procedure attains the minimax risk: Rn (F) = inf sup L(fˆn , f ), fˆn f ∈F

(26)

where the infimum is over all procedures for estimating f from the data and fˆn is an instantiation of the procedure with n data points. However, it is typically very difficult to establish minimax optimality for a fixed sample size, so it is common instead to use an asymptotic criterion. For example, a procedure that achieves the optimal rate of convergence in the minimax sense is one for which lim sup n

supF L(fˆn , f ) < ∞. Rn (F)

(27)

Optimal procedures (mostly with respect to rate, but see refs. [65]) have been constructed for many specific combinations of

9056

WAVELETS

loss function (e.g., integrated squared error) and function class (e.g., Sobolev). There is little coherence in the set of available results, as the optimal procedures for different lossfunction–function-class pairs are usually distinct. Indeed, no uniformly minimax optimal procedure exists. For most of the solved cases, the optimal procedure is linear in the data. Spaces for which a linear procedure is optimal are in some sense homogeneous; they do not exhibit functions with qualitatively different behavior in different parts of the domain. If we move to different spaces in the Besov or Triebel families, however, we encounter spaces for which linear procedures are decidedly suboptimal, because they cannot adapt to the required spatial variations. These latter spaces are more than curiosities; they embody prior constraints that are important in a variety of real problems [22,29].

of the unknown function. When there are n observations, there are n empirical wavelet coefficients. The statistical problem can now be cast as that of estimating the sequence (θjk ) from the empirical coefficients. The wavelet transform thus serves to move the estimation problem from function space to sequence space, which is more familiar territory. And here the sparseness of wavelet coefficient sequences (by virtue of the unconditional basis property) comes to the foreground. Most of the energy in the θjk is concentrated in relatively few coefficients. We would thus obtain a reasonable estimate of f if we could extract the coefficients of largest magnitude accurately, even if we set the rest of zero. Consider the general problem of estimating θ1 , . . . , θn , given data of the form

The Wavelet Shrinkage Paradigm

where zi are i.i.d. N (0, 1) and σ > 0 is known. It is well known [47,71] that for estimating the θi ’s, shrinkage of the yi ’s yields an advantage in mean square error over the linear estimator θî = yi that increases with the dimension of the problem. The admissibility∗ of generalized Bayes estimators in this context [5,6,7] is a manifestation of a similar phenomenon. Now suppose we use the following simple shrinkage scheme: set θˆˆ i = yi 1{|yi |>τ } for a fixed threshold τ > 0 (this is hard thresholding). If we choose τ so that P{maxi |zi | > τ/σ } is small, then with high probability, the thresholding kills all the noise while preserving the large coefficients (with their noise terms). The sparser the representation, the more effective this procedure. This is the idea behind wavelet shrinkage. In general, we allow for a broader class of thresholding schemes,

Given a suitable wavelet basis, a general scheme for applying wavelets to the nonparametric regression problem is as follows: 1. form the empirical wavelet coefficients from the data via the DWT, 2. apply a univariate nonlinear shrinkage to every wavelet coefficient, and 3. invert the DWT on the shrunken coefficients to obtain the estimate fˆn . This is the wavelet shrinkage paradigm introduced by Donoho and Johnstone. This tunable procedure is computationally efficient and yields asymptotically (near-)optimal estimators over a wide variety of prior constraints. The empirical wavelet coefficients of the data vector y are given by Uy for a unitary transformation U. When the noise terms in the model are i.i.d. N(0, σ 2 ), the empirical wavelet coefficients satisfy θˆjk = θjk + zjk ,

J0 j J1 ,

0 k < 2j ,

(28)

where the zjk are themselves i.i.d. N(0, σ 2 ) and the θjk are the ‘‘true’’ wavelet coefficients

yi = θi + σ zi ,

i = 1, . . . , n,

θˆˆ j,k = η(θˆj,k , τnjk ),

(29)

(30)

for a nonlinear function η and thresholds that depend on sample size, resolution level, and location. Common choices of η include hard thresholding [η(y, τ ) = y1{|y|>τ } ], soft thresholding [η(y, τ ) = sgn(y)(|y| − τ )+ ], and various Bayes estimators. Donoho and Johnstone refer to the special scheme described above as

WAVELETS

universal thresholding; they develop a number of other schemes, including one based on Stein’s unbiased risk estimate [23,71]. In some variants, the coefficients of the coarse terms (the φ0,k ) are not altered; there are usually few of these coefficients, and in general the coarse term is not as sparsely represented as the detail terms. Nason [59] uses a cross-validation approach for threshold selection in regression problems, and Vidakovic [78] develops another approach based on Bayesian inference∗ . The relative performance of these different schemes is analyzed in refs. 23, 39. Since normal noise behaves nicely under the DWT, the analysis of this general thresholding procedure is tractable. The concepts are the same but the analysis is harder when there is correlated noise or other distributions are appropriate. Figures 4, 5, and 6 show, respectively, the function f , the noisy data, and a wavelet shrinkage estimator for several canonical examples popularized by Donoho and Johnstone. (These figures were adapted from those in ref. 28 using the Stanford TeachWave software.) These examples demonstrate the spatial adaptiveness of the wavelet procedure over a range of prior constraints: exactly the same basis and procedure is used in each example, and the wavelet estimator adapts to the particular structure of each space. The power of this property is in the fact that it is automatic and in the fact that functions showing a great degree of spatial heterogeneity are often poorly estimated by linear procedures. Moreover, the denoising is effective enough here to preserve the interesting structure in the object: the spikes remain spiky, the discontinuities are visible, and the high-frequency structure is maintained to a good degree. See ref. 23 for a full discussion of this adaptation property, and refs. [28,29] for a detailed discussion of these and similar examples. The noise level in Figs. 4–6 is low relative to the magnitude of signal changes (signalto-noise ratio: 7 to 1). Figures 7 and 8 show samples and estimates of the same functions in the high-noise case (signal-to-noise ratio: 2 to 1). The estimator understandably does not perform as well as before, but many of the features of the objects are

9057

still preserved, in contrast to the case of linear estimators. The estimates also exhibit marked Gibbs effects and are not necessarily visually appealing. However, these estimates were based on simple universal thresholding; it is possible to tune performance to different criteria (e.g., visual appearance) by appropriate selection of a thresholding scheme. While all these examples are revealing, they still leave a number of open questions regarding the small-samples performance of the wavelet shrinkage estimators, particularly relative to the other sophisticated methods that are currently available. In support of the practical advantages of the wavelet procedure, Donoho and Johnstone put forward an elegant body of theory to characterize the asymptotic performance of the wavelet shrinkage paradigm. In particular, the Donoho–Johnstone estimator fˆ exhibits the following properties [22]: 1. fˆ is with high probability about as smooth as the truth (simultaneously in many different senses); 2. fˆ is spatially adaptive; 3. fˆ is simultaneously nearly optimal in a minimax sense over a range of function classes and loss functions. The first property indicates that even though a particular wavelet procedure has the flexibility to model a wide range of functions, including those with singularities and spatial heterogeneity, it will tend to achieve about the right level of smoothness in the estimate. (In particular, it will tend not to produce spurious structure by seriously undersmoothing.) The second property indicates that the wavelet estimator can model functions that change qualitatively across their domain, without strong prior information about the location or nature of the change points. The third property indicates that a single procedure deviates from optimality by only a logarithmic term, simultaneously over a range of Besov and Triebel spaces whose smoothness is only a little greater than that of the wavelets being used. A striking aspect of these results is the simplicity of the shrinkage procedure. Coordinatewise shrinkage is all that is required

Figure 4.

Figure 5. 9058

Figure 6.

Figure 7. 9059

9060

WAVELETS

Figure 8.

to achieve these properties. This is a consequence of the unconditional basis property. Since wavelets form an unconditional basis for these spaces, shrinkage of the coefficients does not move the estimator outside the space (and often produces a smoother object as measured by the norm). Because of the unconditional basis property, the set of wavelet coefficient sequences corresponding to functions in a ball in one of these spaces is bracketed between two very symmetric sets. The symmetry of these bracketing sets of sequences renders the coordinates effectively ‘‘independent’’ and thus makes coordinatewise estimators nearly optimal. See ref. 27 for a full discussion. These results have strong implications for the relevance of the minimax paradigm as a way of thinking about function estimation. A common criticism of the minimax estimator is that it is driven by the worst case over the parameter space, even though that worst case might not be representative of a ‘‘reasonable’’ value for the parameter. Here,

however, a single procedure achieves nearoptimality over a range of different spaces, all of which have, by their very construction, a qualitatively different worst case. See ref. 22 for further elaboration of this heuristic.

STATISTICAL APPLICATIONS OF WAVELETS While the nonparametric regression problem arises in a variety of statistical applications, the potential scope of wavelet methods is broader. Most of the methods that have been developed to attack other problems rely on the wavelet shrinkage paradigm or a related perspective, but these are all still active areas of research. For density estimation∗ , ref. 25 uses an orthogonal series estimator (wavelets) with shrunken coefficients, and ref. 64 uses a Bayesian approach for estimating the square root of a density within a wavelet basis. For spectral estimation, most of the effort has focused on adapting

WAVELETS

wavelet shrinkage to periodogram∗ smoothing. Here, ref. 29 computes a ‘‘log-o-gram’’ by smoothing the log-periodogram with universal thresholding. Others [43,58] use a similar idea but apply more careful resolutionlevel-dependent thresholding to take account of the (asymptotic) chi-square distribution of the periodogram ordinates. Also, Flandrin and Goncalves [40] consider the modeling of transient signals with time-varying frequencies. Wavelet methods have been proposed for discriminant analysis∗ [67,8] and factor analysis∗ [80]. Various heuristics have also been proposed for using wavelets with count data, including the use of a variancestabilizing∗ transformation with a normal approximation to facilitate wavelet shrinkage [29]. In fitting a generalized additive model [45], the smoothing stage in an additive models fit can be implemented as a wavelet shrinkage. For solving inverse problems, a method has been developed [34] for attacking various homogeneous problems, including the Radon transform, with wavelet bases; some of these ideas have been applied [51] to positron emission tomography. In image analysis, wavelets have been used for compression, registration, fitting boundaries, segmentation, and modeling temporal changes. Wavelets have been proposed as a natural basis for Bayesian function estimation [78]; and in [79] the equivalence of the Pólya tree prior (see DIRICHLET PROCESSES) on the interval [53] and a particular prior on the Haar wavelet coefficients is shown. For many applications, especially in statistics, various outstanding issues remain. The analysis of wavelet methods has proceeded via some simplifying assumptions, including i.i.d. normal noise with a common known variance, a regular sampling grid in a Euclidean interval, and boundary-free transformations. Extensions of the analysis to other cases include results on correlated normal noise [49] and stationary mixing noise [4], development of a general method for irregular grids [74], and work on the wavelet transform in bounded domains [12]. In many problems, the natural domain of the functions of interest is not a standard Euclidean interval but a more general manifold (e.g., a sphere, the surface of a

9061

wing); see also refs. [73, 68, 69] for a generalized construction of wavelets that applies to such domains. BEYOND WAVELETS The role of the unconditional basis property in explaining the power wavelet bases shows that wavelets are not unique in their ability to provide effective representations of functions and signals; rather, the appropriate representation depends on the properties of the class of objects being described. Wavelets are in some sense ideal for representing functions with an unknown number of singularities. This insight is related to the way that the wavelet basis partitions the time–frequency plane, with each basis function essentially concentrated on a time interval of the appropriate resolution and roughly into an octave in frequency. Other bases and procedures have been developed to take advantage of the structure of different classes of functions and signals, including wavelet packets, cosine packets, and chirplets and brushlets [14,56]. All of these partition the time–frequency plane differently and yield unconditional bases for the appropriate spaces. Thus, they tend to provide sparser representations of the corresponding objects than do standard wavelets. Building on these ideas [9] is an adaptive method called basis pursuit by which a sparse representation can be chosen in polynomial time from among a ‘‘library of bases.’’ The method requires an additive measure of sparseness or information and a tree structure relating the different bases in the library. Other generalizations of wavelets include interpolating wavelets, using redundant translations [60], and nonlinear multiresolution analysis. In higher dimensions, image segmentation and analysis with wavelets must contend with higherdimensional boundaries and structures and thus a loss of sparseness. Various alternatives are being developed; these involve adaptive segmentation and textures. The most definitive text on the definition, construction, and practical application of wavelets is ref. 14. The most useful entry points into Donoho and Johnstone’s body of work are refs. [22, 24, 29]. The history and

9062

WAVELETS

mathematical applications of wavelets are comprehensively explored in ref. 55. WEB RESOURCES Wavelet resources can be found at http:// www.mathsoft.com/wavelets.html on the World Wide Web. Back issues of the Wavelet Digest, an e-mail newsletter edited by Wim Sweldens, can be obtained at http:// www.wavelet.org/wavelet/index.html. Additional software, papers, technical reports, ideas, and links can be found at the Wave-Lab website at Stanford University, http:// stat.stanford.edu/ ∼ wavelab/. There are a variety of introductions to wavelets available: 1. Burrus, C. S. and Gopinath, R. A., A tutorial overview of wavelets, filter banks and Interrelationships. 2. Edwards, T., Discrete wavelet transforms: theory and application. 3. Gootman, E. and Wickerhauser, M., Elementary wavelets. 4. Graps, A., An introduction to wavelets. 5. Heil, C. and Walnut, D., Continuous and discrete wavelet transforms. 6. Jawerth, B. and Sweldens, W., An overview of wavelet-based multiresolution analysis. 7. Lewalle, J., Tutorial on wavelet analysis of experimental data. 8. Schröder, P. and Sweldens, W., Building your own wavelets at home. 9. Strang, G., Wavelets. 10. Strang, G. Wavelets and dilation equations: a brief introduction. 11. Vidakovic, B., Wavelets for kids. 12. Chui, C. K., An Introduction to Wavelets. Academic Press, 1992. All but the last of these introductions are available on line at the resources web page given above. REFERENCES 1. Andersson, L., Hall, N., Jawerth, B., and Peters, G. (1994). Wavelets on closed subsets

of the real line. In Recent Advances in Wavelet Analysis, L. L. Schumaker and G. Webb, eds. Academic Press, Boston, pp. 1–62. 2. Benedetto, J. J. (1994). Frame decompositions, sampling, and uncertainty principle inequalities. In Wavelets: Mathematics and Applications, J. J. Benedetto and M. W. Frazier, eds. Studies in Advanced Mathematics, CRC Press, Boca Raton, Fla., pp. 247–304. 3. Benedetto, J. J. and Walnut, D. F. (1994). Gabor frames for L2 and related spaces. In Wavelets: Mathematics and Applications, J. J. Benedetto and M. W. Frazier, eds. Studies in Advanced Mathematics. CRC Press, Boca Raton, Fla., pp. 97–162. 4. Brillinger, D. R. (1994). Some Asymptotics of Wavelet Fits in the Stationary Error Case. Tech. Rep., Department of Statistics, University of California, Berkeley. 5. Brown, L. D. (1979). A heuristic method for determining admissibility of estimators—with applications. Ann. Statist., 7, 960–994. 6. Brown, L. D. (1980). A necessary condition for admissibility. Ann. Statist., 8, 540–544. 7. Brown, L. D. (1981). A complete class theorem for statistical problems with finite parameter spaces. Ann. Statist., 9, 1289–1300. 8. Buckheit, J. and Donoho, D. L. (1995). Improved Linear Discrimination Using TimeFrequency Dictionaries. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 9. Chen, S. and Donoho, D. L. (1995). Atomic Decomposition by Basis Pursuit. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 10. Chui, C. K. (1992). An Introduction Wavelets. Academic Press, Boston.

to

11. Cohen, A. (1992). Biorthogonal wavelets. In Wavelets: A Tutorial in Theory and Applications, C. K. Chui, ed. Academic Press, Boston, pp. 123–152. 12. Cohen, A., Daubechies, I., and Feauveau, J. C. (1992). Biorthogonal bases of compactly supported wavelets. Commun. Pure Appl. Math., 45, 485–560. 13. Cohen, A., Daubechies, I., Jawerth, B., and Vial, P. (1992). Multiresolution analysis, wavelets, and fast algorithms on an interval. C. R. Acad. Sci. Paris A, 316, 417–421. 14. Daubechies, I. (1992). Ten Lectures on Wavelets. Society for Industrial and Applied Mathematics, SIAM, Philadelphia.

WAVELETS 15. Daubechies, I. (1994). Two recent results on wavelets: wavelet bases for the interval, and biorthogonal wavelets diagonalizing the derivative operator. In Recent Advances in Wavelet Analysis, L. L. Schumaker and G. Webb, eds. Academic Press, Boston, pp. 237–258. 16. Deslauriers, G. and Dubuc, S. (1987). Interpolation dyadique. Ann. Sci. Math. Québec, 11, 25–44. 17. DeVore, R. A. and Popov, V. A. (1988). Interpolation of Besov spaces. Trans. Amer. Math. Soc., 305, 397–414. 18. Donoho, D. L. and Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81, 424–455. 19. Donoho, D. L. and Johnstone, I. M. (1994). Ideal Time Frequency De-noising. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 20. Donoho, D. L. and Johnstone, I. M. (1996). Neoclassical Minimax Problems, Thresholding, and Adaptation. Bernoulli, 2, 39–62. 21. Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over p balls for q error. Probab. Theory Rel. Fields, 99, 277–303. 22. Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., and Picard, D. (1995). Wavelet shrinkage: Asymptopia? (with discussion). J. R. Statist. Soc. B, 57, 301–369. 23. Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Ass., 90, 1200–1224. 24. Donoho, D. L. and Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Ann. Statist., 26, 879–921. 25. Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., and Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Statist., 24, 508–539. 26. Donoho, D. L. (1995). Interpolating Wavelet Transforms. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 27. Donoho, D. L. (1993). Unconditional bases are optimal bases for data compression and statistical estimation. Appl. Comput. Harmonic Anal., 1, 100–115. 28. Donoho, D. L. (1993). Wavelet shrinkage and W. V. D.—A ten-minute tour. In Progress in Wavelet Analysis and Applications, Y. Meyer and S. Roques, eds. Gif-sur-Yvette, France, pp. 109–128. 29. Donoho, D. L. (1993). Nonlinear wavelet methods for recovery of signals, images, and

9063

densities from noisy and indirect data. In Different Perspectives on Wavelets, I. Daubechies, ed. American Mathematical Society, Providence, RI, pp. 173–205. 30. Donoho, D. L. (1994). On minimum entropy segmentation. In Wavelets: Theory, Algorithms and Applications, C. K. Chui, L. Montefusco, and L. Puccio, eds. Academic Press, Boston. 31. Donoho, D. L. (1994). Smooth wavelet decompositions with blocky coefficient kernels. In Recent Advances in Wavelet Analysis, L. L. Schumaker, and G. Webb, eds. Academic Press, Boston, pp. 259–308. 32. Donoho, D. L. (1996). De-noising by softthresholding. IEEE Trans. Inf. Theory, 41, 613–627. 33. Donoho, D. L. (1997). CART and Best-OrthoBasis: A Connection. Ann. Statist., 25, 1870–1911. 34. Donoho, D. L. (1995). Nonlinear solution of linear inverse problems by wavelet–vaguelette decomposition. Appl. Comput. Harmonic Anal., 2, 101–126. 35. Dubuc, S. (1986). Interpolation through an iterative scheme. J. Math. Anal. Appl., 114, 185–204. 36. Efromovich, S. and Pinkser, M. (1981). Estimation of square-integrable density on the basis of a sequence of observations. Prob. Inf. Transmission, 17, 182–195. 37. Efromovich, S. and Pinkser, M. (1982). Estimation of square-integrable probability density of a random variable. Prob. Inf. Transmission, 18, 175–189. 38. Esteban, D. and Galand, C. (1977). Application of quadrature mirror filters to split band voice coding systems. Int. Conf. on Acoustics, Speech, and Signal Processing, pp. 191–195. 39. Fan, J., Hall, P., Martin, M., and Patil, P. (1993). Adaption to High Spatial Inhomogeneity Based on Wavelets and on Local Linear Smoothing. Tech. Rep. CMA-SR18-93, Centre for Mathematics and Its Applications, Australian National University, Canberra. 40. Flandrin, P. and Goncalves, P. (1994). From wavelets to time-scale energy distributions. In Recent Advances in Wavelet Analysis, L. L. Schumaker and G. Webb, eds. Academic Press, Boston, pp. 309–334. 41. Frazier, M., Jawerth, B., and Weiss, G. (1991). Littlewood–Paley Theory and the Study of Function Spaces, Regional Conference Series in Mathematics 79. American Mathematical Society, Providence, RI.

9064

WAVELETS

42. Galand, C. (1983). Codage en Sous-Bandes: Théorie et Applications a` la Compression Numérique du Signal de Parole. Ph.D. thesis, University of Nice, France. 43. Gao, H. (1993). Spectral Density Estimation Via Wavelet Shrinkage. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 44. Haar, A. (1910). Zur Theorie der orthogonalen Funktionensysteme. Math. Ann., 69, 331–371. 45. Hastie, T. and Tibshirani, R. (1990). Generalized Additive Models. Chapman and Hall, London. 46. Ibragimov, I. A. and Khas’minskii, R. Z. (1982). Bounds for the risks of non-parametric regression estimates. Theory Probab. Appl., 27, 84–99. 47. James, W. and Stein, C. (1960). Estimation with quadratic loss. Proc. Fourth Berkeley Symp. on Math. Statist. and Probab., Vol. 1, pp. 361–380. 48. Jawerth, B. and Sweldens, W. (1994). An overview of wavelet based multiresolution analyses. SIAM Rev., 36, 377–412. 49. Johnstone, I. M. and Silverman, B. W. (1994). Wavelet Threshold Estimators for Data with Correlated Noise. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 50. Kahane, J. P., Katznelson, Y., and De Leeuw, K. (1977). Sur les coefficients de Fourier des fonctions continues. C. R. Acad. Sci. Paris, 285, 1001–1003. 51. Kolaczyk, E. (1995). Wavelet Methods for the Inversion of Certain Homogeneous Linear Operators in the Presence of Noisy Data. Tech. Rep., Department of Statistics, Stanford University, Stanford, CA. 52. Korostelev, A. (1994). An asymptotically minimax regression estimator in the uniform norm up to exact constant. Theory Probab. Appl., 38, 737–743. 53. Lavine, M. (1992). Some aspects of Pólya tree distributions for statistical modelling. Ann. Statist., 20, 1222–1235. 54. Littlewood, J. E. and Paley, R. (1931). Theorems on Fourier series and power series. J. London Math. Soc., 6, 230–233. 55. Meyer, Y. (1992). Wavelets and Operators. Cambridge University Press, New York. 56. Meyer, Y. (1993). Wavelets: Algorithms & Applications. Society for Industrial and Applied Mathematics, Philadelphia. 57. Meyer, Y. (1991). Ondelettes sur l’intervalle. Rev. Mat. Iberoamer., 7, 115–133.

58. Moulin, P. (1993). Wavelet thresholding techniques for power spectrum estimation. IEEE Trans. Signal Process., 42, 3126–3136. 59. Nason, G. P. (1995). Wavelet regression using cross-validation. In Wavelets and Statistics, A. Antoniadis and G. Oppenheim, eds. Lecture Notes in Statistics 103. Springer-Verlag, New York, pp. 261–280. 60. Nason, G. P. and Silverman, B. W. (1995). The stationary wavelet transform and some statistical applications. In Wavelets and Statistics, A. Antoniadis and G. Oppenheim, eds. Lecture Notes in Statistics 103. SpringerVerlag, New York, pp. 281–301. 61. Nemirovskii, A. (1986). Nonparametric estimation of smooth regression function. J. Comput. System Sci., 23, 1–11. 62. Nussbaum, M. (1985). Spline smoothing in regression models and asymptotic efficiency in L2 . Ann. Statist., 13, 984–997. 63. Peetre, J. (1975). New Thoughts of Besov Spaces, vol. I. Mathematics Dept., Duke University, Durham, N.C. 64. Pinheiro, A. and Vidakovic, B. (1995). Estimating the Square Root of a Density via Compactly Supported Wavelets. Tech. Rep., Institute of Statistics and Decision Sciences, Duke University, Durham, N.C. 65. Pinsker, M. (1980). Optimal filtering of square integrable signals in Gaussian white noise. Prob. Inf. Transmission, 16, 120–133. 66. Sacks, J. and Ylvisaker, D. (1981). Asymptotically optimum kernels for density estimation at a point. Ann. Statist., 9, 334–346. 67. Saito, N. (1994). Feature Extraction Using Local Discriminant Basis. Ph.D. thesis, Yale University, New Haven, Conn. 68. Schröder, P. and Sweldens, W. (1995). Spherical wavelets: texture processing. In Rendering Techniques ’95, P. Hanrahan and W. Purgathofer, eds. Springer-Verlag, New York, pp. 252–263. 69. Schröder, P. and Sweldens, W. (1995). Spherical wavelets: efficiently representing functions on the sphere. Comput. Graphics Proc. (SIGGRAPH 95), pp. 161–172. 70. Speckman, P. (1979). Minimax estimates of linear functionals in a Hilbert space. Unpublished. 71. Stein, C. (1981). Estimating the mean of a multivariate normal distribution. Ann. Statist., 9, 1135–1151. 72. Stone, C. (1982). Optimal global rates of convergence for nonparametric estimators. Ann. Statist., 10, 1040–1053.

WEAK CONVERGENCE, STATISTICAL APPLICATIONS OF 73. Sweldens, W. (1996). The lifting scheme: a custom-design construction of biorthogonal wavelets. Appl. Comput. Harmon. Anal., 3, 189–200. 74. Sweldens, W. and Schroder, P. (1996). Building your own wavelets at home. In Wavelets in Computer Graphics, ACM. 75. Tchamitchian, P. (1987). Biorthogonalité et théorie des opérateurs. Rev. Mat. Iberoamer., 3, 163–190. 76. Triebel, H. (1992). Theory of Function Spaces. ¨ Birkhauser, Basel. 77. Vidakovic, B. and Muller, P. (1994). Wavelets for Kids: A Tutorial Introduction. Tech. Rep., Institute of Statistics and Decision Sciences, Duke University, Durham, N. C. 78. Vidakovic, B. (1996). Nonlinear wavelet shrinkage with Bayes rules and Bayes factors. Submitted. 79. Viele, K. (1996). Nonparametric Bayesian Methods for Evaluating Fit in Hierarchical Models. Ph.D. thesis, Department of Statistics, Carnegie Mellon University, Pittsburgh. 80. Wickerhauser, M. V. (1992). Fast Approximate Factor Analysis. Tech. Rep., Department of Statistics, Washington University, St. Louis. 81. Zygmund, A. (1959). Trigonometric Series, 2nd ed. Cambridge University Press, Cambridge, England. See also MINIMAX ESTIMATION; NONPARAMETRIC REGRESSION; RADEMACHER FUNCTIONS; SHRINKAGE ESTIMATORS; STEIN EFFECT; and WALSH –FOURIER TRANSFORMS.

CHRISTOPHER R. GENOVESE

WEAK CONVERGENCE, STATISTICAL APPLICATIONS OF Weak convergence, also known as convergence in law, is a way of describing how a sequence of probability distributions Qn converges to a limit distribution Q as n increases. Formally, Qn converges weakly to Q if Qn (A) converges to Q(A) for every Borel set A whose boundary has Q-probability 0. For distributions in a Euclidean space, weak convergence amounts to pointwise convergence of the cumulative distribution functions, at every continuity point of the limit cumulative distribution function (see Billingsley [3] and CONVERGENCE OF SEQUENCES

9065

OF RANDOM VARIABLES). The importance of weak convergence stems from the central limit theorem∗ , which gives conditions under which the distribution of a normalized average converges weakly to a normal, Poisson, or other limit distribution. Statistical applications of the theory of weak convergence are widespread. Suppose xn = (X1 , . . . , Xn ) is a random sample with distribution Pθ ,n , which belongs to a specified family of distributions {Pφ,n : φ ∈ n }. The parameter value θ is unknown. The parameter space n can be finite or infinite dimensional and may vary with sample size n. Such a probability model amounts to a mathematically tractable way of generating hypothetical samples, which are intended to mimic important features of the data. The behavior of a statistical procedure Tn = Tn (Xn ) under the model {Pφ,n : φ ∈ n } can often be approximated by studying the limit distribution of Tn . Weak convergence results for triangular arrays (see Gnedenko and Kolmogorov [5]) are important tools for this purpose. Several illustrations follow. Since θ may be infinite dimensional, the discussion below also extends to time-series∗ analysis and nonparametrics. Sequential analysis∗ also depends heavily on weak convergence theory.

ESTIMATION Consider the situation where the parameter space n is a fixed subset of the real line, θˆn = θˆn (xn ) is an estimate of θ , and the distribution Hn (θ ) of the centered estimate n1/2 (θˆn − θ ) converges weakly to a N(0, σ 2 (θ )) distribution. This pointwise limit can be deceptive. In an example Hodges (see SUPEREFFICIENCY, HODGES) the components of xn are independent identically distributed N(θ , 1) random variables. The estimate θˆn = X n , the sample mean, if |X n | > n−1/4 ; otherwise θˆn = cX n , with 0 < |c| < 1. The asymptotic variance σ 2 (θ ) is then 1 if θ = 0, but is c2 if θ = 0. At first glance, this result appears to contradict the minimax∗ property of the sample mean as an estimate of θ . However, if θn = n−1/2 h, where h is fixed, then Hn (θn ) converges weakly to the N((c − 1)h, c2 ) distribution. The bias and mean squared error of the Hodges estimate may thus be substantial

9066

WEAK CONVERGENCE, STATISTICAL APPLICATIONS OF

for θ near 0; and the weak convergence of its distribution Hn (θ ) to the normal limit is not uniform over any compact neighborhood of θ = 0. The key idea here—the need to study weak convergence of {Hn (θn ); n 1} for all sequences {θn } that converge to θ at an interesting rate—was developed by Stein, ´ Rubin, Le Cam, Hajek, and others (see Chernoff [4] and Ibragimov and Has’minskii [8]). It applies equally well to infinite-dimensional parameters. The concepts of locally asymptotically minimax estimate and of leastdispersed regular estimate are two important outcomes, which provide asymptotic justifications for maximum likelihood∗ and related estimates. Robustness∗ of the estimate θˆn can be studied by embedding the model {Pφ,n : φ ∈ n } into a larger model, usually nonparametric, and by then considering triangular array convergence of the distribution or risk of θˆn within this supermodel (see Rieder [11]). CONFIDENCE SETS AND THE BOOTSTRAP∗ Suppose the asymptotic variance σ 2 (θ ) of the estimate θˆn is a continuous function of θ and suppose Hn (θn ) converges weakly to the N(0, σ 2 (θ )) distribution for every sequence {θn = θ + n−1/2 h}. An asymptotic confidence interval for θ is An = {θ : n1/2 |θˆn − θ | σ (θˆn )−1 (1 − α/2)}, where is the standard normal CDF. The coverage probability of An converges to 1 − α in a locally uniform way: lim sup{|Pn,φ [φ ∈ An ] − (1 − α)| :

n→∞

n1/2 |φ − θ | c} = 0, for every θ and every finite positive c. Let Kn (θ ) denote the distribution of n1/2 (θˆn − θ )/σ (θˆn ). A natural estimate of Kn (θ ) is Kn (θˆn ), which Efron has termed a bootstrap estimate [see RESAMPLING PROCEDURES and BOOTSTRAP]. Given the sample xn , suppose x∗n is an artificial sample drawn from the fitted model Pθˆn ,n . Let θˆn∗ denote the value of the estimator θˆn , recomputed from the bootstrap

sample x∗n . Then Kn (θˆn ) is the conditional distribution of n1/2 |θˆn∗ − θˆn |/σ (θˆn∗ ), given xn . This interpretation of the bootstrap distribution leads to Monte Carlo algorithms for approximating it numerically. A boot-strap analog to confidence interval An is Bn = {θ : n1/2 |θˆn − θ | σ (θˆn )dn }, where dn is a (1 − α) th quantile∗ of the bootstrap distribution Kn (θˆn ). The coverage probability of Bn is typically 1 − α + O(n−3/2 ), locally uniformly in θ , whereas that of An is only 1 − α + O(n−1 ) (see Hall [7]). The bootstrap approach to confidence sets makes no overt use of asymptotic theory, but often relies on triangular array weak convergence results for its justification (see Beran [1]). The bootstrap approach is particularly valuable when the limit law is intractable.

POWER FUNCTIONS Suppose θ has the partition θ = (ξ , η). A test ψn = ψn (xn ) for the null hypothesis ξ = ξ0 has power function βn (ξ , η) = E(ψn |Pθ ,n ). Typically, limn→∞ βn (ξ , η) is α when ξ = ξ0 and is 0, α, or 1 when ξ = ξ0 . Calculating limn→∞ βn (ξ0 + n−1 h, η) as a function of h—the asymptotic power function—often yields a better approximation to the power function of ψn . Introduced by Neyman [9], this triangular array approach was developed by Wald [12] and others. Contiguity arguments together with the central limit theorem often provide a way to derive the ´ asymptotic power function (see Hajek and ˇ ak ´ [6] and CONTIGUITY). The concept of a Sid locally asymptotically maximin test, which stems from this work, provides asymptotic justification for likelihood ratio∗ and related tests. Robustness of a test’s level and power can be assessed by studying triangular array convergence of its power function within a supermodel (see Rieder [10]). The power function of a test of a composite hypothesis can be estimated directly from the sample by bootstrap methods (see Beran [2]). Since ξ , η can be infinite-dimensional parameters, the discussion here includes goodness-of-fit∗ tests.

WEAK DATA

9067

WEAK DATA

REFERENCES 1. Beran, R. (1984). Jb. Dt. Math. Ver., 86, 14–30. (Surveys bootstrap asymptotics through 1982.) 2. Beran, R. (1986). Ann. Statist., 14, 151–173. (Describes bootstrap tests and bootstrap estimates of power functions.) 3. Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York. (An influential account of weak convergence theory.) 4. Chernoff, H. (1956). Ann. Math. Statist., 27, 1–22. (A fine survey of asymptotic theory in the early 1950s.) 5. Gnedenko, B. V. and Kolmogorov, A. N. (1954). Limit Distributions for Sums of Independent Random Variables. Addison-Wesley, Reading, MA. (A definitive account of the topic.) ˇ ak, ´ ´ Z. (1967). Theory of Rank 6. Hajek, J. and Sid Tests. Academic, New York. (A fine mathematical account of rank tests, contiguity, and asymptotic power.) 7. Hall, P. (1985). Unpublished preprint. (Analyzes the error in coverage probability of bootstrap confidence intervals.) 8. Ibragimov, I. A. and Has’minskii, R. Z. (1981). Statistical Estimation: Asymptotic Theory. Springer, New York. (A modern, highly mathematical treatment.) 9. Neyman, J. (1937). Skand. Aktuarietidskr., 20, 149–199. (Introduces asymptotic power in studying smoothed tests.) 10. Rieder, H. (1978). Ann. Statist., 6, 1080–1094. (One possible shrinking neighborhood analysis of robust tests.) 11. Rieder, H. (1983). Trans. Ninth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes. Academia, Prague, pp. 77–89. (Surveys shrinking neighborhood theories of robust estimation.) 12. Wald, A. (1943). Trans. Amer. Math. Soc., 54, 426–482. (A definitive early study of the likelihood ratio test and its asymptotic optimality.) See also BOOTSTRAP; CONTIGUITY; CONVERGENCE OF SEQUENCES OF RANDOM VARIABLES; LARGE-SAMPLE THEORY; LIMIT THEOREM, CENTRAL; NEYMAN’S AND OTHER SMOOTH GOODNESS-OF-FIT TESTS; RESAMPLING PROCEDURES; and SUPEREFFICIENCY, HODGES.

R. J. BERAN

The collinearity diagnostics given in Belsley [1] (see also CONDITIONING DIAGNOSTICS) are able to determine for linear regression y = Xβ + when collinearity is present among the columns of the n × p data matrix X and which of the least-squares estimates b = (X X)−1 X y is being degraded by its presence. But they are not able to determine if this degradation, which is manifest in inflated parameter variances, is enough to be considered harmful. This is because the collinearity diagnostics examine only the X data, while the parameter variances V(b) = σ 2 (X X)−1 depend on both X and σ 2 , the common variance of the error terms in . Serious degradation due to collinearity in X could be mitigated by a low value for σ 2 . A diagnostic that removes this shortcoming combines the collinearity diagnostics with a measure of adequate signal/noise (s/n) that reflects the degree to which the magnitudes of the regression parameters β exceed the variances V(b) of their estimates: a given degree of variance inflation is clearly more tolerable for large βi than for small ones. If, then, a parameter estimate is found to be degraded by collinearity in conjunction with inadequate s/n, we have harmful collinearity, while collinearity otherwise is nonharmful. Collinearity of any sort constitutes one form of data weakness. This situation of inadequate s/n without collinearity for a parameter known a priori to be nonzero defines a second form of data weakness known as short data. Let us partition the OLS estimator b as [b 1 , b2 ] , with sizes p1 and p2 , and focus on the s/n available for the estimation of b2 . Tests for this s/n can be based on the statistic ◦

φ2 ≡

◦

(b2 − β 2 ) (X2 M1 X2 )(b2 − β 2 ) p2 s2 p

2 ∼ Fn−p (τ 2 ),

where s2 ≡ e e/(n − p), e is the vector of OLS residuals, and M1 ≡ I − X1 (X1 X1 )−1 X1 . This statistic is distributed as a noncentral F with p2 and n − p degrees of freedom and a noncentrality parameter τ 2 ≡ ◦ ◦ (β 2 − β 2 ) V−1 (b2 )(β 2 − β 2 ), which is the s/n ◦ of β 2 relative to β 2 (usually chosen as 0).

9068

WEAK DATA

Figure 1. Diagnostic situations resulting from combining the collinearity diagnostics with the test for adequate signal-to-noise.

Thus, to test the hypothesis that s/n exceeds some assumed level of adequacy τ∗2 at the size α, we need only calculate φ 2 and compare it p2 (τ∗2 ), the (1 − α)-critical value for with 1−α Fn−p the noncentral F with p2 and n − p degrees of freedom and noncentrality parameter τ∗2 . The practical difficulty with this test is determining an appropriate magnitude for τ∗2 , since it depends upon the unknown parameters β 2 and the unknown σ 2 . This is accomplished by noting that s/n can also be viewed as a probabilistic distance relative to the central chi-square distribution. Letting 2 γ χp2 be the (1 − γ )-critical value of a central chi-square with p2 degrees of freedom, it is shown in Belsley [1] that large values of τ∗2 (producing stringent tests for φ 2 ) correspond to critical values with γ ’s chosen near 1, while small values (less stringent tests for φ 2 ) correspond to critical values with γ ’s chosen near 0. A practical test for s/n, then, picks a level of adequacy for τ∗2 corresponding to a γ near 1, and uses φ 2 as a test that this level is achieved. Tables for conducting this test of adequate signal-to-noise for various values of α, γ , n, and p2 are available in Chap. 7 of Ref. [1]. There are four reasons the level of s/n relative to the estimation of β 2 can be small, as can be seen from the following reformulation: ◦

τ 2 ≡ (β 2 − β 2 ) [X2 X2 (I − P)] ◦

× (β 2 − β 2 )/σ 2 ,

ˆ ˆ X ˆ where P ≡ (X2 X2 )−1 X 2 2 , and X2 is the fitted values of X2 regressed on X1 . These are: ◦ (i) β 2 is close to β 2 , (ii) σ 2 is large, (iii) X2 X2 is small in norm, and (iv) I − P is small in norm. Condition (i) is assumed not true for this test, and (ii) deals with a model characteristic fixed for the regression situation. Thus, the data can cause small s/n because of either (iii) or (iv). The first of these is associated with the lengths of the data variates in X2 being small, a condition called short data, and the second is associated with collinearity, for P is recognized as a generalized R2 of the regression of X2 on X1 . The worse the collinearity, the closer P will be to I. We see here, then, two equal causes of data weakness in OLS regression: collinearity and short data (inadequate data variation). Data with either problem are said to be weak data. Collinearity and short data are actually two sides of the same coin, for the two problems can be transformed into one another. It has been suggested, for example, that the strong collinearity between income Y and consumption C can be alleviated simply by using Y along with the more nearly orthogonal variate savings S ≡ Y − C. But here one has simply transformed the collinearity between Y and C into the short data of S; both data sets will be equally weak data and present OLS estimation with the same problems. The test for harmful collinearity now consists of combining the collinearity diagnostics with the above-mentioned test for adequate

WEAR PROCESSES

s/n. Figure 1 summarizes the four possible outcomes of this procedure. REFERENCE 1. Belsley, D. A. (1991). Conditioning Diagnostics. Wiley, New York. See also CONDITIONING DIAGNOSTICS.

(1 + ρ 2 )−1/2 , then Xt is weakly stationary. It is strictly stationary if the t are also Gaussian. In time-series∗ analysis, one focuses on the covariance structure of the underlying process and not on its finite-dimensional distributions. When weak stationarity holds, one has +∞ eitλ dF(λ), r(t) = −∞

DAVID A. BELSLEY

WEAKLY ERGODIC CHAIN. See MARKOV PROCESSES

WEAK STATIONARITY A stochastic process∗ {Xt , t ∈ E} is weakly stationary if its mean and variance are the same for all t, and if the covariance between Xt1 and Xt2 depends only on the absolute difference |t1 − t2 |. Formally, EXt = µ, EXt1 Xt2 − EXt1 EXt2 = r(|t1 − t2 |), for all t, t1 , t2 in E. The space E can be, for example, (−∞, +∞) or {0, 1, 2, . . .}. The process Xt is strictly stationary if the probability structure is invariant under time shifts, i.e., if the distribution of (Xt1 +t , . . ., Xtn +t ), n 1, does not depend on t. Strict stationarity implies weak stationarity (provided Xt has a finite variance). The converse, in general, is false. For Gaussian processes∗ , however, strict and weak stationarity are equivalent. The Ornstein-Uhlenbeck process∗ is an example of a stationary process {Xt , −∞ < t < +∞} that is Gaussian (and Markovian). As a second illustration, consider the following discrete-time process: X0 = a0 , Xt = ρXt−1 + t ,

t = 1, 2, . . . ,

where t , t = 0, 1, 2, . . ., are uncorrelated random variables with mean 0 and variance 1. The process {Xt , t = 0, 1, 2, . . .} is AR (1) (autoregressive of order 1). It is not weakly stationary if |ρ| 1. If |ρ| < 1 and if a =

9069

where F(λ) is a spectral distribution, and therefore the time series can be analyzed in the spectral domain (see SPECTRAL ANALYSIS). In fact, if F(λ) is absolutely continuous, i.e., if dF(λ) = f (λ)dλ, then the covariance function r(t) is simply the Fourier transform (see INTEGRAL TRANSFORMS) of the spectral density f (λ), r(t) =

+∞

eitλ f (λ)dλ.

∞

See also SPECTRAL ANALYSIS; STATIONARY PROCESSES; STOCHASTIC PROCESSES; and TIME SERIES.

MURAD S. TAQQU

WEAR PROCESSES Wear processes refer to a class of reliability∗ models in which the failure of an item is modeled as resulting from an accumulation of wear over time with failure occurring when this wear has exceeded some threshold (either fixed or random). This model has two special cases of sufficient interest to merit separate entries. These are shock models∗ and cumulative damage models.∗ We shall only briefly comment on these; the remainder of our effort here will be in those aspects of wear processes that have not been covered in these entries. Shock models deal with a macroscopic analysis of the failure process. An item is presumed to be subject to shocks that occur over time. Various assumptions can be made on this counting process. The item is generally assumed to have an inherent ability to survive a random number of these shocks. A cumulative damage model deals with a more microscopic perspective on failure. The shocks are assumed to cause random damages, which accumulate in some fashion,

9070

WEAR PROCESSES

usually additively. Failure occurs when the accumulated damage exceeds some threshold (fixed or random). These two classes of models can be seen to be equivalent in a mathematical sense. In any cumulative damage model, there is some random number of shocks before the accumulated damage exceeds the threshold. Conversely, one can view a shock model as a cumulative damage model in which the damages are degenerate random variables taking the value 1 with probability 1 and the threshold is the random number of shocks before failure. In the cumulative damage model, one may think of the accumulated damage as being a stochastic process∗ {W(t) : t 0} in which W(t) = N(t) i=1 Zi , where the collection {Zi } represents the random damages caused by the shocks and where the process {N(t) : t 0} is the counting process governing the shocks or times at which these random damages occur. The sample paths of such a cumulative damage process thus are typically step functions. We shall now turn our attention to processes in which the sample paths of the process {W(t) : t 0} describing the accumulated wear by time t are more general. Namely, we consider the case in which wear accumulates continuously over time, instead of, or in addition to, wear which accumulates in the form of random damages at random points in time. We also permit the possibility of repair (i.e., a decrease in the accumulated wear). In an early paper on shock models, cumulative damage models, and wear processes, Esary et al. [6] consider a wear process that begins at 0, has nonnegative increments, is a Markov process∗ , and satisfies a technical condition that, for fixed time, the greater the wear the more prone the item is to further wear and that for equal amounts of wear, an older device is more susceptible to wear than a younger device. There are a number of stochastic processes that satisfy these conditions, for example, processes starting at the origin and having nonnegative, stationary, and independent increments. These include the Poisson process∗ , compound Poisson process, and the infinitesimal renewal process∗ . For such wear processes in which the threshold is deterministic, the distribution of the time until failure belongs to

the class of increasing failure-rate average (IFRA) distributions. In a sequence of two papers, AdbelHameed [1,2] studied the gamma wear process, in which the wear sustained by time t has a gamma distribution∗ with probability density function f (x, t) =

λe−λx (λx)t−1 , (t)

t 0.

When the threshold is random and G(x) represents the probability that the device survives x units of wear, the reliability function of the device can be represented as F(t) = λ 0

∞

e−λx (λx)t−1 G(x) dx. (t)

In [1], Abdel-Hameed shows that the lifedistribution properties of G(x), such as increasing failure rate (IFR), increasing failure-rate average (IFRA), and new better than used (see HAZARD RATE AND OTHER CLASSIFICATIONS OF DISTRIBUTIONS and RELIABILITY, PROBABILISTIC) are inherited as corresponding properties of the reliability function F(t). In [2], he studies optimal replacement policies for such devices. C ¸ inlar [4] studies shock and wear models using Markov additive processes. He investigates the failure-time distribution under random-threshold models and multiplicative killing-type failure mechanisms. One result of interest is that when the deterioration process is assumed to be a gamma process, whose shape parameter varies as a function of a Brownian motion∗ process, the lifetime distribution is Weibull∗ . Another paper by C ¸ inlar [5] is expository in nature; some general models for deterioration processes and lifetime distributions are discussed. Deterioration processes are modeled by continuous Markov, continuous semi-Markov, rightcontinuous Markov, Markov additive, and semi-Markov processes∗ . Gottlieb [7] has investigated sufficient conditions on a wear process for the life distribution to be IFR. He describes various classes of stochastic processes for which these sufficient conditions are satisfied, and he investigates conditions that lead to an IFRA result. In a 1984 paper, Gottlieb and Levikson [8]

WEATHER FORECASTING, BRIER SCORE IN

investigate optimal replacement policies for a wear process resulting from a self-repairing shock model in which random damages accumulate additively. However, between shocks, the cumulative damage process decreases. Conditions under which a control limit policy is optimal are given. Pieper and Tiedge [10] apply stochastic processes, including Wiener processes with drift and related multiplicative processes to modeling wear, in the engineering setting. Various distributions arising from such models with either a fixed or a random threshold are discussed including the BirnbaumSaunders and inverse Gaussian∗ distributions. Related inference questions are also considered. Processes in which the time until the accumulated wear reaches a fixed threshold has a new better-than-used distribution are called NBU processes. They have been studied by Marshall and Shaked [9] and by Shanthikumar [11]. If wear results by independent, stochastically increasing random damages accumulating additively, if the times between damages are NBU and independent of the damages, and if the cumulative damage decreases deterministically in a certain technical fashion between damages, then the wear process is a NBU process [9]. A second result of Marshall and Shaked assumes that the times between damages are exponential and that there is a deterministic increase in wear between damages. A NBU process also results from this model. Shanthikumar has generalized this in various ways including allowing the times between damages to be DFR and allowing the damage and the times between damages to be dependent. Work on various multivariate extensions of this has been carried out. Other research has been conducted in which the damage process remains constant between damages. One model, which does deserve mention, is that of Assaf et al. [3]. In their cumulative damage model, unlike others, damages may be negative. Under suitable conditions, the first-passage time to any threshold has a Pólya type 2∗ (PF2 ) density. REFERENCES

2. Adbel-Hameed, M. S. (1977). In The Theory and Applications of Reliability, Vol. 1, C. P. Tsokos and I. N. Shimi, eds. Academic, New York, pp. 397–412. 3. Assaf, D., Shaked, M., and Shanthikumar, J. G. (1985). J. Appl. Prob., 22, 185–196. 4. Cinlar, E. (1977). In The Theory and Applications of Reliability, Vol. 1, C. P. Tsokos and I. N. Shimi, eds. Academic, New York, pp. 193–214. 5. C ¸ inlar, E. (1984). In Reliability Theory and Models., M. S. Abdel-Hameed, E. C ¸ inlar, and J. Quinn, eds. Academic, New York, pp. 3–41. 6. Esary, J. D., Marshall, A. W., and Proschan, F. (1973). Ann. Prob., 1, 627–649. 7. Gottlieb, G. (1980). J. Appl. Prob., 17, 745–752. 8. Gottlieb, G. and Levikson, B. (1984). J. Appl. Prob., 21, 108–119. 9. Marshall, A. W. and Shaked, M. (1983). Adv. Appl. Prob., 15, 601–615. 10. Pieper, V. and Tiedge, J. (1983). Math. Operationsforschung, Ser. Statist., 14, 485–502. 11. Shanthikumar, J. G. (1984). Adv. Appl. Prob., 16, 667–686. See also CUMULATIVE DAMAGE MODELS; RELIABILITY, PROBABILISTIC; SHOCK MODELS; and SURVIVAL ANALYSIS.

WILLIAM S. GRIFFITH

WEATHER FORECASTING, BRIER SCORE IN In Brier [2] this score is introduced as a means of evaluating (weather) forecasts expressed in terms of probability. Brier gave an example where forecasts of rain or no rain are available and a forecast consists of a probability of rain. The Brier score is used to evaluate these forecasts after the occurrence of rain or no rain has been observed. Hitherto the score has been applied frequently in meteorology∗ and fields such as medical diagnosis.∗ Suppose that on each of n occasions exactly one out of r events can occur and that the forecast probabilities are fi1 , fi2 , . . . , fir on the ith occasion ( j fij = 1). The Brier score is defined as 1 (fij − Eij )2 , n n

1. Adbel-Hameed, M. S. (1975). IEEE Trans. Rel., R-24, 152–154.

9071

r

i=1 j=1

9072

WEATHER FORECASTING, EPSTEIN SCORING RULE IN

where Eij takes the value 1 or 0 according to whether or not event j occurred on the ith occasion. The score is the average squared distance between the forecast probability distribution and the probability distribution of the perfect forecast, i.e., having probability one in the realized event. The score has a minimum value of zero (perfect forecasting) and a maximum value of 2 (worst possible forecasting). To illustrate the formula, suppose that on ten occasions probability forecasts have been given of rain or no rain (i.e. n = 10 and r = 2). The forecasts are 0.7, 0.9, 0.8, 0.4, 0.2, 0, 0, 0, 0, 0.1, respectively, and it turned out to rain only on the second, third, and fourth occasions. The score for these forecasts 1 × 2 × (0.72 + 0.12 + 0.22 + 0.62 + 0.22 + is 10 02 + 02 + 02 + 02 + 0.12 ) = 0.19 [2]. In some papers the Brier score is called ‘‘probability score’’ or ‘‘quadratic scoring rule.’’ The latter term is somewhat confusing, as in Staël von Holstein and Murphy [6] a family of quadratic scoring rules is introduced, of which the Brier score (as well as the Epstein scoring rule∗ ) is only a special case. An attractive property of the Brier score is that it is strictly proper, i.e., a forecaster minimizes his expected score only by honestly expressing his personal probability assessments. Brier noted this and wrote that using this score ‘‘cannot influence the forecaster in any undesirable way.’’ For a more detailed description of properness see DISTRIBUTIONAL INFERENCE. The Brier score also can be written explicitly as the sum of calibration (‘‘validity,’’ ‘‘reliability’’) and refinement (‘‘sharpness,’’ ‘‘resolution’’) components (Murphy [3], Sanders [5]). The calibration component measures the extent to which forecast probabilities and observed frequencies correspond. The refinement component measures the extent to which, in a sequence of events receiving the same forecast probability, the occurrence of the event is uniquely determined (i.e. always or never). A graphical exposition of the relationship between calibration, refinement, and Brier score can be found in Blattenberger and Lad [1]. A different decomposition of the Brier score is given in Murphy [4].

REFERENCES 1. Blattenberger, G. and Lad, F. (1985). Separating the Brier score into calibration and refinement components: a graphical exposition. Amer. Statist., 39, 26–32. (A graphical exposition of decomposing the Brier score and some related statistical results.) 2. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Rev., 78, 1–3. (First paper on properness. The Brier score is introduced.) 3. Murphy, A. H. (1972). Scalar and vector partitions of the probability score: Part II. N-state situation, J. Appl. Meteorol., 11, 1183–1192. 4. Murphy, A. H. (1986). A new decomposition of the Brier score: formulation and interpretation. Monthly Weather Rev., 114, 2671–2673. (Decomposition of the Brier score by conditioning on the event that occurred.) 5. Sanders, F. (1963). On subjective probability forecasting. J. Appl. Meteorol., 2, 191–201. (Decomposition of the Brier score by conditioning on the probability assessments given.) 6. Staël von Holstein, C.-A. and Murphy, A. H. (1978). The family of quadratic scoring rules. Monthly Weather Rev., 106, 917–924. See also DISTRIBUTIONAL INFERENCE; METEOROLOGY, STATISTICS IN; and WEATHER FORECASTING, EPSTEIN SCORING RULE IN.

A. H. KROESE W. SCHAAFSMA

WEATHER FORECASTING, EPSTEIN SCORING RULE IN In Epstein [1] this scoring rule is introduced as a means of evaluating (weather) forecasts in the form of a probability distribution on a set of ranked events. Epstein gave an example where probability forecasts are available when the temperature is in one of the classes T 0◦ F, 0◦ F < T 20◦ F, 20◦ F < T 40◦ F, or T > 40◦ F. The Epstein scoring rule is used to evaluate these forecasts after the temperature has been observed. The rule complies with the ordering, so that in situations like Epstein’s example it is more attractive than the Brier score, for example, which does not depend on the ordering. Epstein’s derivation is in terms of decision making, costs, and losses. His final formulation of the score is not very transparent,

WEATHER MODIFICATION—I

and a different formulation (Murphy [3]) is presented here. Suppose that on each of n occasions exactly one out of r ranked events can occur, and on one such situation, i, the forecast probabilities are fi1 , . . . , fir . The Epstein scoring rule can then be defined as 1 (Gij − Dij )2 , n n

r

i=1 j=1

j where Gij = k=1 fik (the cumulative probability function), and where Dij = 1 if one of the events 1, . . . , j happened on the ith occasion, and Dij = 0 otherwise. The Epstein scoring rule is the average squared distance between the forecast cumulative distribution function and the cumulative distribution function of the perfect forecast, i.e., having probability one in the realized event. In some papers the Epstein scoring rule is called ranked probability score. An attractive property is that, like the Brier score∗ , it is strictly proper, i.e., a forecaster minimizes his expected score only by honestly expressing his personal probability assessments. In Epstein [1] this was stated; a formal proof was given in Murphy [2]. The Epstein scoring rule belongs to the family of quadratic loss functions introduced in Staël von Holstein and Murphy [4]. In the entry DISTRIBUTIONAL INFERENCE a loss function is described that can be seen as the generalization of the rule to continuous probability distributions. REFERENCES 1. Epstein, E. S. (1969). A scoring system for probability forecasts of ranked categories. J. Appl. Meteorol., 8, 985–987. (Introduction of the Epstein scoring rule in terms of decision making, costs, and losses.) 2. Murphy, A. H. (1969). On the ranked probability score. J. Appl. Meteorol., 8, 988–989. (Proof that the scoring rule is strictly proper.) 3. Murphy, A. H. (1971). A note on the ranked probability score. J. Appl. Meteorol., 10, 155–156. 4. Staël von Holstein, C.-A. and Murphy, A. H. (1978). The family of quadratic scoring rules. Monthly Weather Rev., 106, 917–924.

9073

See also DISTRIBUTIONAL INFERENCE; METEOROLOGY, STATISTICS IN; and WEATHER FORECASTING, BRIER SCORE IN.

A. H. KROESE W. SCHAAFSMA

WEATHER MODIFICATION—I Weather-modification experimentation has been largely statistical; the study of the effectiveness of cloud seeding has mostly dealt with statistical aggregates. After some 40 years, the results are still equivocal: A few significant findings are balanced by a long series of nonsignificant ones. Why is the issue still moot? What has gone wrong with the process of experimentation and analysis that such an accumulation of data has not provided definitive information on the effectiveness of seeding? Is there a lesson here for statistics? THE PHYSICAL BASIS [7] The physical basis for weather modification is the introduction of chemical nucleating agents into clouds that contain super-cooled (below freezing level) water. Under laboratory conditions this was shown to produce ice nuclei and droplets, and it was therefore expected to induce precipitation in actual clouds, whether by forming new nuclei, by augmenting existing drops, or by enhancing cloud buoyancy. Actual seeding with dry ice indeed produced visible changes in individual clouds. Verification of effects on precipitation was more difficult, since the weather is notoriously variable. Some physicists had hoped to isolate meteorological phenomena that act with greater regularity than rainfall or hail on the ground (see METEOROLOGY, STATISTICS IN). But it was well understood that a large stochastic component would remain in any weather data. The occasional enthusiast will produce impressive series of photographs showing some cloud that changed shape after being seeded. But as Samuel Johnson said as early as 1759 [11], ‘‘Might not some other cause produce this occurrence? The Nile does not always rise on the same day [9].’’ Clearly, weather experimentation needs a statistical design in which randomization∗ of treatment

9074


must play a central role. Clearly also, the results must be evaluated statistically and tested for significance. COMMERCIAL OPERATIONS AND EARLY EVALUATION An early statistical experiment was Langmuir’s 1950 week-on-week-off cloud seeding from Alamagordo; it was largely inconclusive [14]. Private enterprise did not wait for verification, and a budding cloud-seeding industry developed in the 1950s, especially in the United States. At a relatively low price per acre, farmers and growers bought future rainfall, to be induced mostly by ground burners that generated smoke containing silver iodide. Rainfall was often observed following seeding and acclaimed as evidence of success; lack of rainfall on other occasions raised doubts. The evidence was equivocal. As Fisher∗ [9] remarked, ‘‘Attempts to seed clouds remind me of the burnt offerings of old; on the ground there was an impressive display of fire and smoke which ascended to the heavens: whether it had any effect on Heaven was, however, more doubtful.’’ In an attempt to assess cloud seeding, a special advisory committee sponsored a statistical analysis of past operations [11] and came to cautiously optimistic conclusions. Its report was forcefully criticized in the statistics literature, principally on the grounds that the available evidence had come from commercial operations without randomized allocation of treatments. The criticism pointed out the likelihood of bias and insisted that valid inferences could only be drawn from properly randomized experiments [3,6]. The analysis of data from commercial seeding had compared data from operations, with historical records of precipitation on the same areas and on surrounding areas. Influential statisticians argued that these analyses were of no possible value, because of the demonstrated possibility of bias. Their attitude discouraged any further analysis of commercial seeding, even of operations that had been carefully controlled and well documented. And yet evaluation of data from such sources is in principle no different from observational studies∗ in other areas, such as medicine [4]. Though it is generally agreed

that randomized experimentation is preferable, this total dismissal of observational studies in weather modification is surprising; it may have been an overreaction to the sanguine claims of commercially motivated cloud-seeding operators. RANDOMIZED EXPERIMENTS Insistence on randomization∗ won the day and was incorporated in the cloud-seeding experiments that were started in the 1960s and 1970s in many countries. Most of these were designed with the help of statisticians, and involved randomly allocating some units to be seeded and other units to serve as controls. Many of them also used data from nearby, upwind, areas as concomitant variables and employed various designs to reduce error variability. The requirements of rigorous design led to careful definition of units and variables [10,13]. Meteorologists would have liked to experiment with physically meaningful units such as clouds, storms, systems, or fronts, but the state of the art did not allow these to be precisely defined in real time. What, indeed, is a cloud? It may be reasonably well defined at any one moment, but a seeding effect takes time and so a cloud unit has to be defined over time. Clouds, however, grow or dissipate, split or merge within short time spans. In many weather situations there seem to be no individual clouds at all: rather, a more or less continuous mass of moisture over hundreds of miles. Since clouds were difficult to define visually, there were attempts to define them operationally in terms of radar echoes of ice nuclei—which surely are relevant to the precipitation-forming mechanism. But the definition remained elusive, especially if it had to be done in real time so that experimenters could proceed to seed the particular cloud that is identified on the radar screen, and then have response data collected specifically for that cloud. If a cloud was difficult to define operationally, what of smaller units such as towers, or larger units such as ‘‘storms’’ or ‘‘systems?’’ Efforts have been made to define these in terms that can be used in real time and analyzed objectively. But the units remain vague in definition and impossible to treat


or observe independently. Thus, the seeding material from one tower may well be carried over to nearby towers, and the dynamic effects on one cloud’s build up may draw moisture from other clouds nearby. Since experimentation with such ‘‘natural’’ units was found to be impractical, units were usually defined in time and space, i.e., a given area over a number of hours, a day, a year, etc. Correspondingly, the response, instead of being some property of a cloud, was defined in terms of precipitation on the given area in that period of time. Thus the experimental unit might have been an afternoon from 2 to 8 P.M. and the response precipitation in central Illinois. Units of this kind were suitable for a statistical experiment as they permitted a randomized decision to seed or not to seed (or to seed in different ways if that was the object of the experiment) to be made ahead of time and the observations to be collected in a predefined objective manner. Interaction between meteorologists and statisticians led to a series of well-designed experiments, which were monitored carefully. The statisticians insisted on adherence to rigid experimental protocol, designed to avoid possible biases and to permit testing the significance of results. Experimenters were not always happy with the conservative attitude, but would rather have incorporated their new ideas into the experiment as it continued. A small illustration of the difference in the professions’ outlooks will illustrate this. The target for seeding in a well-known experiment had been defined a priori but, several months into the experiment, pilots and meteorologists wished to reduce the target area. They claimed that parts of it were difficult to fly over and they cited the evidence of their flight paths. The statistician suspected they might have been flying where they found suitable clouds, so that redefining, ex post facto, the area to fit the treatment would have introduced a bias. The statistician was formally right, but the others did not appreciate his rigid attitude as they thought it would reduce the experiment’s sensitivity. The results of these experiments on inducing precipitation over fixed time–space units were tested for significance of seeding effects. Most of the results were inconclusive, but a few indicated significant effects. The validity

9075

of these tests was generally accepted, though there was some acrimonious controversy on whether randomization had been properly carried out and whether some data might have been doctored. It was curious to see the statisticians who had advocated randomization so fervently now questioning its practical application [5]. EXPLORATORY ANALYSES AND MULTIPLICITY About the time of the analysis of these experiments, John Tukey’s ideas and terminology of ‘‘exploratory’’ vs. ‘‘confirmatory’’ data analysis∗ came into vogue. The experiments had been designed to test precise hypotheses of null seeding effects against hopeful alternatives of positive effects on rainfall and snow, or negative effects on hail. Statisticians had rightly been concerned to insist on ‘‘confirmatory’’ analyses of this kind. In that period, it will be recalled, the profession was very squeamish about any other type of analysis at all, and statisticians may sometimes have discouraged any study of experimental results beyond tests of significance∗ . Now, finally, statistical practice began to be liberated from the exclusive dominance of testing for significance. It is perhaps not surprising that this liberation was accompanied by confusion regarding the proper role of confirmation and exploration, and that some experimenters seized upon the kind of statistical ideas that most strongly supported their own expectations and hopes. Where new hypotheses were suggested by exploration of data, they were embellished by tests of significance carried out meaninglessly on the same data. From the initial priggish insistence that the only proper statistics were the single tests of significance, the stage now moved to promiscuous testing of every new proposition. It is more disturbing that many statisticians were also confused between the two modes of analysis, as is evident by their readiness to compute multiple significance levels, or P-values∗ , from the data of any single experiment. They attached P-values to subsets of experimental data, to redefined responses, to readjustments for newly proposed concomitant variables, and to a variety

9076


of transformations and alternative tests. An instance is known of eminent statisticians testing as many as 180 subhypotheses of an experiment and then publishing the most striking of the many P-values. These are prime examples of the problem of multiplicity [1]. The statisticians’ own confusion cannot have contributed to a better understanding of weather-modification experimentation. There also was a reverse form of multiplicity. Where experiments resulted in significant seeding effects, some meteorologists proceeded to reanalyze the data meticulously by checking a variety of concomitant variables. Some of these reanalyses obviously suggested that the ‘‘significant’’ effect might have resulted from an unfortunate randomization. In other words, they suggested that a ‘‘type I error∗ ’’ had occurred in that a more favorable sample had been selected for seeding and a less favorable one for control. The multiplicity of these reanalyses was generally as unproductive as the multiplicity of analyses [11]. This stage of analysis, subanalysis, and reanalysis was accompanied by often intemperate debate on the interpretation of the results [20]. Some statisticians saw fit to stretch their legitimate role of skeptical watchdogs of unbiased scientific procedure and adopted an inquisitorial attitude in casting doubt on the honesty or competence of experimenters and data analysts. They also preferred to test their own hypotheses, rather than ones proposed by subject matter specialists. At the same time, some meteorologists chafed at the constraints of a rigorous experimental protocol, which did not allow the use of physically meaningful experimental units and had no mechanism for evolutionary adaptation to the new insights into cloud physics and improvements in technology. Since most experiments did not produce the hoped-for results, many meteorologists despaired of the ‘‘statistical’’ approach and wished to revert to simpler exploration of clouds and seeding. RECENT EXPERIMENTS A few experiments on time–space weather modification continued, mostly outside the United States, but by the middle 1970s it

was generally agreed that more elaborate experiments were required. Some such projects were organized and included collection of radar data and in-cloud observations by means of specially instrumented aircraft. It was hoped that this would show how seeding affected the formation of precipitation. The execution of such experiments and the collection of adequate samples of data did not turn out to be as straightforward as had been hoped for, and the results obtained were largely inconclusive. With the paucity of positive findings and the difficulty of funding, weathermodification experimentation has ground to a halt in the middle 1980s. Any future studies are likely to be dependent on new insights into the physics of precipitation. The experiments that will then be designed are likely to be much more complex, both in terms of the physics and the statistics [20]. The disappointing results of this 40-year effort of field trials with cloud seeding must probably be ascribed to the unexpectedly complex nature of precipitation formation, rather than to shortcomings of the investigators and their studies. And yet, one may ask what the statisticians’ contribution to this research effort has been and where it might have failed. THE CONTRIBUTION OF STATISTICIANS On the whole, there is little doubt that statisticians have made a crucial contribution to weather experimentation by providing good designs with randomized controls and allowing valid testing of significance. No less important was their insistence on workable definitions of units and measurements and on clear experimental protocol. Their insistence on rigid adherence to protocol and to predetermined methods of evaluation, as well as their proverbial professional skepticism, have not always endeared them to experimenters who lived with the challenge of ‘‘taming the weather.’’ Carrying out operations and observations from bouncing airplanes, with the exhilaration of seeming to control the clouds, did not always foster patience with earth-bound skeptics and their tables of random numbers. And yet the cooperation between the two professions, with their


very different outlooks, was successful on the whole. The contribution made by formal mathematical statistics, on the other hand, has been rather minor. Much effort has been spent by statisticians in devising ‘‘optimal’’ analyses of cloud-seeding experiments. Using the classical approach of mathematical statistics, they based their derivations on various distributional assumptions about the response variable and its relation to the concomitants, on assuming that the units were independent of each other, and that the effect of seeding was fixed from unit to unit, either additively to natural precipitation or multiplicatively. They then derived optimal tests mathematically. At times, they relaxed some of the distributional assumptions and proposed nonparametric∗ tests instead. At other times they studied multivariate∗ tests and a variety of other techniques and compared them with each other to select the ‘‘best.’’ They never seemed to deal with the wellknown dependence of precipitation on successive units, presumably because it would have made the mathematics intractable. It is doubtful whether this effort did much to improve the evaluation of weathermodification experiments. As a rule, confirmatory analyses by different statistical techniques gave essentially the same results. And power studies did not indicate that the ‘‘optimal’’ method was much better than most other methods. The underlying assumptions were rarely tested but much debated. An ironic example is the discussion of whether seeding effects would be additive or multiplicative, a discussion based on neither physical considerations nor experimental data, but entirely on the statisticians’ predilections. It is curious that all this effort and controversy took place while a valid approach was available, in a source as well known as Fisher’s The Design of Experiments [8], and studied extensively by Kempthorne and his students. Testing by rerandomizing the treatment allocation did not require tenuous assumptions about distributions, false assumptions about independence, or idiosyncratic assumptions about the form of the effects. Isolated early uses of such tests [12] did not convince most statisticians and meteorologists to adopt them until a 1975 report

9077

by Brillinger et al. [2] reintroduced them with a novel terminology. Then, rather belatedly, they became fashionable. One cannot but wonder why it took so long. Some insight on the role of statistics in the assessment of experimental evidence may be obtained from a review article by a senior British meteorologist [17] who evaluated the results of three randomized weather experiments. In reviewing tabulated data and tests, he used the quoted P-values∗ incidentally to give an idea of strength of evidence. He did not use the common statistics textbook recipe of choosing a single statistic to provide a unique decision about each experiment’s outcome. Instead, he assessed each experiment’s results in terms of how well its various subanalyses supported each other and how much meteorological sense he could make of them. How very different from the statisticians’ meta-analyses, which pooled the P-values of a collection of experiments! Statistical theory does not seem to reflect the way experimental scientists view statistical evidence. REFERENCES 1. Braham, R. R., Jr. (1979). J. Amer. Statist. Ass., 74, 57–68. (With discussion, especially J. A. Flueck, K. R. Gabriel, W. Kruskal, and P. W. Mielke.) 2. Brillinger, D. R., Jones, L. V., and Tukey, J. W. (1978). The Management of Weather Resources: Report of the Statistical Task Force to the Weather Modification Advisory Board, vol 2. U.S. Government Printing Office, Washington, D.C. 3. Brownlee, K. A. (1960). J. Amer. Statist. Ass., 55, 446–453. 4. Changnon, S. A., Hsu, C-F., and Gabriel, K. R. (1981). J. Weather Modification, 13, 195–199. 5. Commun. Statist. A, 8, 953–1153. (Special 1979 Issues on Weather Modification, including contributions by K. R. Gabriel, P. W. Mielke, and E. J. Smith.) 6. Cohen, N. M. (1987). Personal communication. 7. Dennis, A. S. (1980). Weather Modification by Cloud Seeding. Academic, New York. 8. Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd, London. 9. Fisher, R. A. (1962). Personal communication. 10. Flueck, J. A. and Mielke, P. W. (1978). Amer. Meteor. Soc. Monograph, 38, 225–235.

9078

WEATHER MODIFICATION—II

11. Gabriel, K. R. (1980). Bull. Amer. Meteorol. Soc., 62, 62–69. 12. Gabriel, K. R. and Feder, P. (1969). Technometrics, 11, 149–160. 13. Gabriel, K. R. and Petrondas, D. (1983). J. Climate Appl. Meteorol., 22, 626–631. 14. Howell, W. (1978). J. Appl. Meteorol., 17, 1753–1757. 15. Johnson, S. (1959). The History of Rasselas, Prince of Abyssinia. (Reissued in 1962 by Appleton-Century-Crofts, New York.) 16. LeCam. L. and Neyman, J. (1967). Proc. Fifth Berkeley Symp. Math. Statist. Prob., 5. University of California Press, Berkeley, CA (Especially contributions by J. Neyman and E. M. Scott.) 17. Mason, J. (1980). Meteorol. Mag. 109, 335–344. 18. Silverman, B. A. (1986). ‘‘Weather modification as a water management-tool,’’ Preprint, Tenth Conf. Water. Manage., American Meteorological Society, Boston, MA. 19. Thom, H. C. S. (1957). In Final Report of the Advisory Committee on Weather Control, Vol. 2. U.S. Government Printing Office, Washington, DC, pp. 25–50. 20. Wegman, E. J. and DePriest, D. J. eds. (1980). Statistical Analysis of Weather Modification Experiments. Marcel Dekker, New York. (Including contributions by R. A. Bradley, O. Kempthorne, and J. Neyman.) See also GEOGRAPHY, STATISTICS IN; HYPOTHESIS TESTING; METEOROLOGY, STATISTICS IN; MULTIPLE COMPARISONS; OBSERVATIONAL STUDIES; P-VALUES; RANDOMIZATION SIGNIFICANCE TESTS, HISTORY AND LOGIC OF; and WEATHER MODIFICATION —II.

K. R. GABRIEL

WEATHER MODIFICATION—II Weather modification studies have been carried out all over the world since about 1950, and quite a number have incorporated randomized allocation of treatments. The common feature of all these studies is that on some occasions clouds were seeded by introducing an agent (silver iodide or dry ice) intended to increase precipitation on a welldefined target area, while on other occasions clouds were left unseeded. The effect of seeding was assessed by statistical comparison of target precipitation on seeded occasions with

that on unseeded occasions. Most studies concentrated on testing the null hypothesis of zero effect, though many also tried to provide effect estimates and confidence bounds. Design problems were mostly concerned with definition of the ‘‘occasions,’’ or experimental units, and their allocation to be seeded or unseeded. Meteorology∗ does not provide well-defined ‘‘natural’’ units, since entities such as storms are very elusive to unequivocal a priori definition that allows randomized allocation of seeding. Time units were commonly used because they are well defined. Other problems concerned precipitation measurements, which varied from averages of a few target rain gauge readings to elaborate integrations of radar observations of rain water reflectivity. Since the variability of all such measurements was very high—coefficients of variation of about 1/3 even when the units were entire years—sample sizes needed to be large (usually five years or more) to yield reasonable power. Attempts to reduce variability centered on introducing concomitant variables∗ , but the only effective ones were precipitation data in nearby control areas which seemed out of the range of likely seeding effects. Analyses consisted basically of comparing the seeded and unseeded samples, possibly after adjusting for concomitants and stratifying the units into some meaningful classifications. Some effort went into the choice of test statistics, which required assumptions about precipitation distributions and type of effect. The issue of multiplicity arose in the wide selection of strata, in the variety of measurements and statistics, and in the choice of covariates. It is an issue of paramount importance in experiments that last a number of years, and which inevitably generate novel ideas and techniques as the study progresses. Valid probabilistic evaluation, on the other hand, demands rigid adherence to a predetermined protocol and analysis. It is no mean matter to reconcile these claims. A few of the better known rainfall stimulation studies are described here, chosen because they illustrate the statistical issues of design and analysis, and not because they were thought to be ‘‘representative.’’ They are written by workers involved in the original analyses of these studies, and tend to sound


Figure 1. Map of Santa Clara cloud seeding project. Twenty-five rain gauges in target area and 18 control stations were selected by mutual agreement of operator and sponsor in 1955 for evaluation purposes. Two of the control stations were eventually dropped, leaving the 16 shown here (C1–C16). From A. S. Dennis (1980). Weather Modification by Cloud Seeding. Academic, New York.

rather sanguine. That is one of the problems encountered by statisticians involved in such studies. For overall appraisals of weather modification studies by meteorologists, see refs. 1, 4, and 15. Collections of papers on designs, analyses, and methodological issues appear in refs. 1 (discussion), 14, 22, and 23, which include, respectively, experimental results for Whitetop in Missouri, several Australian experiments, Israeli I and the Swiss Grossversuch III, and Santa Barbara II, and the Tasmanian experiment. SANTA CLARA COUNTY SEEDING OPERATIONS [5] Commercial seeding operations were directed at a target area in California’s Santa Clara county during 10 winters. It was agreed at

9079

the outset to evaluate success by comparing rainfall at 25 target stations and 18 control stations located around the target but not downwind of it (for prevailing winds on rainy days); Fig. 1 shows the stations, two of the control stations having been omitted for the actual analysis. The operational winters were compared statistically with historical data on earlier winters. The analysis compared each winter’s precipitation on an average target station with that on an average control station. All winters were defined as December–March, irrespective of exact seeding dates. Cloud seeding had actually been carried out on a more flexible schedule, but analyses used that objectively defined unit to avoid obvious biases and excluded years where special operations could have introduced bias. The statistical analysis compared the (target, control) scatter for the operational winters with the corresponding scatter for the historical winters. Linear regressions (without transformations) estimated precipitation on target in terms of precipitation on control during the unseeded winters, and also during the seeded winters. For further details see ref. 5. Applying normal theory would have shown significance at roughly 10%, as would have rerandomization tests and nonparametric tests. The test statistic Double ratio =

(Target seeded total) (Control seeded total) (Target unseeded total) (Control unseeded total)

,

on the other hand, would have given a onesided P-value of 0.018. Distributional properties of this statistic are discussed in ref. 9. Because of concern with validity, the original paper [5] did not publish significance or confidence statements. Indeed, this method of historical regression is fraught with risks [2], because the weather on successive sets of seasons is unlikely to be distributed like two independent random samples. (The bias introduced by such assumptions was studied later [11].) Other possible sources of bias are related to the initiation of commercial seeding operations as a result of preceding droughts and to the influences that determine the termination of such operations [3].

9080


THE RANDOMIZED EXPERIMENTS IN ISRAEL [9,10,13] The two experiments, Israeli I (1961–1966/ 1967) and Israeli II (1969/1970–1974/1975), randomly assigned seeding to 24-hour experimental units in the fall–winter seasons. A changeover design∗ was chosen for Israeli I with alternate target areas—Fig. 2—in the north and center of Israel. On each day, seeding was assigned to one of the targets, the other serving as that day’s control. This was a highly efficient design because (i) each day’s target area could be compared with its control area with which it was highly correlated and (ii) seeding was expected to affect the target—control difference on each day, whereas in an ordinary design there would only have been half the effect since half the days would have remained unseeded (see Gabriel in ref. 14). A buffer zone was introduced between the two areas to minimize the contamination of either target by seeding in the other. The analysis was restricted to 391 rainy days and excluded the other two-thirds of the winter days as those were days with very little rain on the targets (only about 2% of all rain), and they provided essentially no seeding opportunities in either area. The definition of ‘‘rainy days’’ was in terms of any rain in a nearby area that was never seeded. The analysis included all rainy days, even those which were not actually seeded (except for a small number of days when a pilot’s strike grounded the aircraft); in order to avoid possible biases, days without seeding opportunities on the target were not excluded from the analysis. The definition of the 24-hour day was altered during the experiment (from a period commencing at 8 P.M. to one commencing at 8 A.M.). This decision caused concern in certain quarters [20], even though the analysis was always consistent with the randomization and seeding of each day, so that no bias was possible [8,10]. The target areas were defined in the original design and adhered to in the analysis. Analyses of interior subareas of the target were later considered preferable, but were not used for the critical analysis because that might have introduced bias.

The Mann–Whitney–Wilcoxon statistic∗ was chosen early on for the crucial test ∗ of significance . No changes were allowed in the test—even though the results suggested that a nonparametric test was not optimal—because data-directed choice of the test statistic would have invalidated any significance statement. The a priori chosen test statistic suggested a positive effect of seeding. Various other analyses, albeit data driven, supported such a finding even more strongly. Detailed follow-up analyses within a variety of strata gave even stronger support for such effects on days when the clouds were of intermediate temperatures. Israeli II was designed principally to replicate Israeli I, but the analyses published so far relate to a single target—the Lake of Galilee catchment area—and use an upwind control area near Haifa on the coast [13]. The target was highly correlated with the control, so that target—control comparisons were considerably less noisy than the target data itself. The potential benefits of a changeover design could not be obtained with this control because it was upwind of the target and therefore could not be seeded without contaminating the catchment area. An alternative target area farther south was also not suitable for such a design because the catchment area was too far away and too poorly correlated with it. The analyses of Israeli II were focused on a narrower target than those of Israeli I and on days with intermediate cloud-top temperatures. These choices resulted from the findings of Israeli I, which also made sense in terms of cloud physics. The double ratio statistic was chosen for the analysis, again because it had been found sensitive in Israeli I. It attained a rerandomization P-value∗ of 0.017, confirming the positive results of Israeli I. The estimates of increase of precipitation due to seeding were 22(±7)% for the interior areas in Israeli I and 18(±8)% for the target of Israeli II. Later exploratory analyses suggest the effect is strongest at cloud-top temperatures of about −12 to −21◦ C and consists mostly of an increase in the duration of precipitation [12].


9081

Figure 2. Map of Israel showing both experimental areas and the interior areas (shaded). Dots indicate rain gauges used in analysis of 1964–1965. From ref. 14. Table 1. Various Analyses of the Israeli Experiments: One-Sided P-Values∗ against the Alternative of a Positive Effect Target

Mann–Whitney–Wilcoxon

Normal Two-Sample

0.054a 0.013

0.009 0.002

Israeli I Entire areas Interior subareas Israeli II Catchment area a The

Double Ratio Rerandomized 0.025 0.002 0.017a

a priori chosen test statistic.

A variety of detailed analyses of the data from both experiments by regression methods, poststratifications, and multivariate analyses, fleshed out the picture of the apparently successful experiments. However, these were described as exploratory analyses. The pitfalls of multiplicity were largely avoided, and the indications from Israeli I were verified by means of the replication in Israeli II.

CLIMAX I AND II EXPERIMENTS IN COLORADO [17–19] Climax I and Climax II were replicated wintertime orographic cloud seeding experiments, intended to examine if seeding clouds with silver iodide could increase snowfall amounts over what is naturally expected. Climax I ran during 1960–1965 and Climax II during 1965–1970. The target area

9082


of Climaxes I and II was the summit of Fremont pass, a few miles north-northeast of Leadville, Colorado. For complete experimental design descriptions, see ref. 18 and Grant and Mielke in ref. 14. The experimental unit for both experiments was a 24-hour period and the treatment involved seeding clouds with silver iodide from ground generators positioned at different locations to account for various wind directions. A specified 24-hour period was declared an experimental unit if certain wind conditions were satisfied and the U.S. Weather Bureau duty forecaster at Denver, Colorado, forecast at least 0.01 in. of precipitation at Leadville, Colorado. The decision to seed or not seed each experimental unit was randomized. Snowboard measurement sites were placed at specified intervals over Fremont pass and also over the neighboring Vail and Hoosier passes. Two precipitation measurement sites termed CRG (climax recording gauge) and HAO (high altitude observatory snowboard) were independently operated by the Weather Bureau (i.e., both the CRG and HAO measurements were collected by the same operator) and were located within 1 m of each other. A recently noted blunder in the recording of the data by Colorado State University personnel was that the CRG and HAO measurements were accidentally reversed in four of the five years the Climax II experiment occurred. A pooled measurement termed TGM (target mean) that involved the average of the nonmissing measurements from CRG, HAO, and seven other target area measurement sites near the summit of Fremont pass, was not affected by this blunder. The results of the Climax experiments were analyzed several times. (A1) The initial joint analysis, reported in 1971 [19], was based primarily on twosample Mann–Whitney–Wilcoxon tests for the target area measurements, and did not take into account the measurements from eight gauges at sites in the control area. (A2) Reanalyses reported in 1981 [18], had been initiated because of a concern of Mielke (in the discussion of ref. 1) that the 1971 results may have been the consequence of a type I statistical error, which accounted for the measurements of the

eight previously defined gauges in the control area. (A3) Further reanalyses involving metric-based statistical methods carried out in 1982 [17], were prompted by concern about the nonmetric nature of the two-sample Mann–Whitney–Wilcoxon tests∗ of (A1) and (A2) and the least-squares method used in (A2). Comparisons of the three analyses for two important meteorological strata are given in Table 2 for the TGM measurements [17,18]. These results imply that the 1979 concern of Mielke that questioned the credibility of analysis (A1) was unjustified. Any doubts involving the Climax I and II experimental results must be attributed to the inadequacy of (A1). That is surely the wrong reason for doubting the conclusions of a 10-year project involving two carefully replicated 5-year experiments. Raising concerns regarding any complicated scientific experiment is natural and demands attention, but the present situation demonstrates how an unjustified minor concern could have falsely discredited the findings of a major scientific study. THE FLORIDA AREA CUMULUS EXPERIMENTS [24–26] The Florida Area Cumulus Experiments (FACE) comprised a two-stage program for investigating the potential of ‘‘dynamic seeding’’ to enhance summertime convective rainfall over a sizable target area (1.3 × 104 km2 ) in south Florida. In its time, it was the only program ever conducted in the United States whose stated objective was to increase areal precipitation by altering cloud dynamics. The first stage of the program was an exploratory experiment (FACE-1, 1970–1976); the second stage was a fully planned and implemented confirmatory experiment (FACE-2, 1978–1980). FACE-1 The FACE-1 experiment utilized the following: (1) A design document, (2) a single fixed target area (i.e., a quadrilateral extending from Fort Myers and Naples on the west coast of Florida to West Palm Beach and Fort Lauderdale on the east coast), (3) some


9083

Table 2. Climax Experiments: One-Sided P-Values of the Target Group Mean Measurements for Two Strata and Three Successive Analyses Analysis Stratum

(A1)

(A2)

(A3)

0.1314 0.1271

0.0244 0.0110

0.0339 0.0048

0.0084 0.0367

0.0344 0.0075

0.0471 0.0064

◦

500-mb temperature (−20 to −11 C) Climax I Climax II 700-mb wind direction (190–250) Climax I Climax II

‘‘screening’’ criteria (e.g., a one-dimensional numerical model for estimating seedability, airborne assessment of cloud fields, etc.) for selection of operational days, (4) a randomized assignment of treatment (e.g., either pyrotechnic AgI flares or placebo sand flares) to each selected operational day, and (5) hourly adjusted radar estimated rainfall volumes in the target area (e.g., the hour before treatment, the six hours after treatment initiation, etc.). The design and implementation of FACE1 changed markedly during the period of the experiment [25]. Its implementation during the summers of 1970, 1971, 1973, 1975, and 1976, resulted in operational changes (e.g., reformulation of the AgI treatment flares, changes in the gauge network used to adjust the radar estimation of rainfall, yearly changes in the randomization plan, the distinction of A and B operational days, where B days were those in which the cloud field received 60 or more flares). A number of these changes were due to the lack of consistent yearly support and funds. The term ‘‘exploratory’’ was therefore well deserved [6]. The five summers of operations produced a total of 104 operational days with 75 B days (39 seeded and 36 unseeded). The exploratory analyses, back-to-back stem-and-leaf displays∗ of treated vs. nontreated daily rainfall volumes for the B days, produced indications of a positive treatment effect in radar estimated rainfall for both the floating target (FT) and total target (TT), and also showed that relatively few days were responsible for the indicated difference. Comparisons of means, medians,

and interquartile ranges suggested a treatment effect of about 45% for the FT and about 10–20% for the TT with supporting Mann–Whitney–Wilcoxon one-tailed Pvalues∗ less than 0.05 and 0.18, respectively [24,25]. Similar results appear to hold for the A and B days combined. However, due to their highly exploratory nature, all calculated P-values were viewed as only suggestive (if they should have been presented at all). Finally, an analysis of covariance∗ attempted to account for a few potentially interfering meteorological factors (e.g., prewetness, wind speed, etc.). A further linear ‘‘sweepout’’ analysis (more consistent with the data analysis ideas of Tukey) used the same four covariates and explored the residuals for evidence of treatment. Point estimates of treatment in both analyses increased to about 60 and 30% for the FT and TT, respectively. Additional analyses suggested that the greater rainfall on treated days was due to larger rain areas and higher rain rates [25]. FACE-2 Given the encouraging results of FACE-1 and its exploratory nature, the NOAA eventually undertook a confirmatory experiment. A detailed design was prepared with the basic characteristics unchanged from FACE1. The confirmatory analyses were further clarified prior to the disclosure of the treatment decisions and the commencement of analysis [24,25]. The implementation was carefully conducted in each of three summers (1978, 1979,

9084


1980) according to the design and operations documents, and no deviations or changes were allowed. This resulted in a total of 75 operational days with 25 treated and 26 nontreated B days. The exact criteria for confirmatory analysis of FACE-2 were subject to some debate by scientists and statisticians both within and outside the program. The majority opted for testing a ‘‘nested’’ set of three hypotheses by means of six rerandomization [9] analyses of single and double ratios [24,25] and ‘‘adjusted’’ P-values. A minority preferred two linear models (i.e., FT and TT variables), which adjusted for the same four covariates that FACE-1 analyses had used. In each of these analyses, none of the three null hypotheses could be rejected at any reasonable level. Thus the FACE-2 results did not confirm the FACE-1 indications of a positive treatment effect [26]. Subsequently, a linear model cross-validation∗ analysis, with covariates guided by meteorological considerations, indicated that FACE-1 had a treatment effect of about 45–50% and FACE-2 of about 15% [7]. However, although the 95% confidence interval for the FACE-1 result lay entirely above the null value, the FACE-2 interval clearly straddled it. This analysis did sharpen the estimate of treatment effects in both FACE-1 and -2, but did not provide generally acceptable evidence of a treatment effect in FACE-2. The FACE scientists and statisticians concluded that the failure to confirm a seeding effect was probably due to three factors: ‘‘1) an unknown and possibly intermittent seeding effect, 2) inadequate predictor equations, and 3) a limited sample size’’ [26]. This highlights the need for careful thought (e.g., conceptual models), planning, and exploratory analyses before undertaking confirmatory experiments. HIPLEX-1 EXPERIMENT [16,21] The purpose of HIPLEX-1, the first in a sequence of high plains weather modification experiments, was to investigate the effect of dry ice cloud seeding on a chain of physical events associated with small cumulus clouds. In that, HIPLEX-1 differed from earlier experiments that had largely considered

the events in the clouds as a ‘‘black box’’ and concentrated on measuring the clouds’ precipitation ‘‘output.’’ The choice of dry ice was to avoid residual effects of seeding agents such as silver iodide. This randomized cloud seeding experiment was conducted by the U.S. Bureau of Reclamation in the vicinity of Miles City, Montana. The experimental units were individual clouds belonging to three classes, and randomized separately within each class. A double-blind protocol governed the randomized cloud-seeding operation from a jet aircraft. Since this experiment was concerned with a chain of physical events represented (as a rational approximation) by a sequence of random variables, multivariate analyses for 12 primary response variables and 11 secondary response variables were essential. In order to avoid intractable multivariate distributional problems, multiresponse permutation procedures∗ (MRPP) were used to analyze the HIPLEX-1 data [16]. HIPLEX-1 was conducted only during the summers of 1979 and 1980, after which it was terminated by federal budget cuts. By that time only 20 experimental units had been obtained. While substantial changes were noted with the response variables measured during the first 5 min after the seeding treatment, the more subtle changes associated with response variables measured more than 5 min after seeding were not obvious. Descriptions of other aspects of HIPLEX-1 are presented in refs. 16 and 21. REFERENCES 1. Braham, R. R., Jr. (1979). J. Amer. Statist. Ass., 74, 57–68; discussion, pp. 68–104. (A meteorologist’s carefully reasoned account of weather modification experimentation and of cooperation, or otherwise, with statisticians. Comments by some of the latter.) 2. Brier, G. W. and Enger, I. (1952). Bull. Amer. Meteor. Soc., 33, 208. (On biases of historical comparisons.) 3. Court, A. (1960). J. Amer. Soc. Civil Eng., Irrig. Drain. Div., 86, 121–126. 4. Dennis, A. S. (1980). Weather Modification by Cloud Seeding. Academic, New York. (Currently the only text covering the entire topic written by a meteorologist.)

WEBER FUNCTION 5. Dennis, A. S. and Kriege, D. F. (1966). J. Appl. Meteor., 5, 684–691. (Analysis of the Santa Clara operations.) 6. Flueck, J. A. (1986). In Precipitation Enhancement—A Scientific Challenge, Vol. 21, No. 43, R. R. Braham, Jr., ed. Amer. Meteor. Society, Boston, MA, Chap. 16. 7. Flueck, J. A., Woodley, W. L., Barnston, A., and Brown, T. (1986). J. Climate Appl. Meteor., 25, 546–564. 8. Gabriel, K. R. (1980). Commun. Statist. A, 9, 1963–1973; 1997. 9. Gabriel, K. R. and Feder, P. (1969). Technometrics, 11, 149–160. (Rerandomization of the double statistic.) 10. Gabriel, K. R. and Neumann, J. (1978). J. Appl. Meteor., 17, 552–554. (Comments on Israeli I.) 11. Gabriel, K. R. and Petrondas, D. (1983). J. Climate Appl. Meteor., 22, 626–631. (On correcting for bias of historical comparisons.) 12. Gagin, A. and Gabriel, K. R. (1986). J. Appl. Meteor., 28, 913–921. 13. Gagin, A. and Neumann, J. (1981). J. Appl. Meteor., 20, 1301–1311. (Analysis of Israeli II.) 14. Le Cam, L. and Neyman, J., eds. (1967). Proc. Berkeley Symp. Math. Statist. Prob.: Weather Modification, 5, University of California Press, Berkeley, CA. (Papers on experiments in Colorado, Israel, Australia, Switzerland, and other places, as well as comments on methodology, especially by Neyman and Scott.)

21. Smith, P. L., Dennis, A. S., Silverman, B. A., Super, A. B., Holroyd, E. W., Cooper, W. A., Mielke, P. W., Berry, K. J., Orville, H. D., and Miller, J. R. (1984). J. Climate Appl. Meteor., 23, 497–512. (Discussion of HIPLEX-1.) 22. Wegman, E. J. and DePriest, D. J., eds. (1980). Statistical Analysis of Weather Modification Experiments. Dekker, New York. (Mostly on statistical methodology and logic of experimentation.) 23. Wierkowski, J. J. and Odell, P. L., eds. (1979). Commun. Statist. Theor. Meth., 8, Nos. 10, 11. (Special Issues on statistical analysis of weather modification experiments. Papers on analysis and methodology, mostly by statisticians.) 24. Woodley, W. L., Jordan, J., Simpson, J., Biondini, R., Flueck, J. A., and Barnston, A. (1982). J. Appl. Meteor., 21, 139–164. (This and the next two references provide detailed analyses of FACE.) 25. Woodley, W. L., Flueck, J. A., Biondini, R., Sax, R. I., Simpson, J., and Gagin, A. (1982). Bull. Amer. Meteor. Soc., 63, 263–276. 26. Woodley, W. L., Barnston, A., Flueck, J. A., and Biondini, R. (1983). J. Climate Appl. Meteor., 22, 1529–1540. See also CHANGEOVER DESIGNS; CONCOMITANT VARIABLES; DESIGN OF EXPERIMENTS; GEOSTATISTICS; METEOROLOGY, STATISTICS IN; RAINFALL, LANDFORMS, AND STREAMFLOW; RANDOMIZATION; SPATIAL DATA ANALYSIS; SPATIAL SAMPLING; TIME SERIES; and WEATHER MODIFICATION —I.

K. R. GABRIEL A. S. DENNIS J. A. FLUECK P. W. MIELKE

15. Mason, J. (1980). The Meteorological Magazine, 109, 335–344. (A meteorologist’s assessment of the evidence for cloud seeding effects in Florida, Israel, and Tasmania.) 16. Mielke, P. W., Berry, K. J., Dennis, A. S., Smith, P. L., Miller, J. R., and Silverman, B. A. (1984). J. Climate Appl. Meteor., 23, 513–522. (Final analysis of HIPLEX-1.) 17. Mielke, P. W., Berry, K. J., and Medina, J. G. (1982). J. Appl. Meteor., 21, 788–792. (Geometrically consistent analysis of Climax.) 18. Mielke, P. W., Brier, G. W., Grant, L. O., Mulvey, G. J., and Rosenzweig, P. N. (1981). J. Appl. Meteor., 20, 643–659. (Reanalysis of Climax.)

9085

WEBER FUNCTION This is a solution of the nonhomogeneous Bessel equation z2 y + zy + (z2 − ν 2 )y = −(z + ν)π −1 − (z − ν)π −1 cos νπ , ν > 0.

19. Mielke, P. W., Grant, L. O., and Chappell, C. F. (1971). J. Appl. Meteor., 10, 1198–1212; Corrigendum, 15, 801. (Analysis of Climax.)

It can be expressed as 1 π Eν (z) = sin(νφ − z sin φ) dφ. π 0

20. Neyman, J. (1979). In Applications of Statistics, P. R. Krishnaiah, ed. North-Holland, Amsterdam, Netherlands, pp. 1–25.

This is also known as the Bessel function Yν (z) of the second kind of order ν (see BESSEL

9086

WEDDLE’S RULE

FUNCTIONS). Usually the term ‘‘Weber function’’ is reserved for the case when v = n is an integer. In this case an explicit expression for this function is cos(pπ )Jp (x) − J−p (x) sin(pπ ) 2 x Jn (x) = − ln π 2 n−1 1 (n − r − 1)! 2 n−2r − π r! x

Yn (x) = lim

p→n

It gives the exact value of the integral if f (x) is polynomial of degree 5 or less. If f (x) is a polynomial of degree 6, the error is only h 6 δ f (a + 3h) 140 h h = [{f (a) + f (a + 6h)} 140 − 6{f (a + h) + f (a + 5h)} + 15{f (a + 2h) + f (a + 4h)} − 20f (a + 3h)],

r=0

−

∞ 1 (−1)r x n+2r π r!(n + r)! 2 r=0

× [(r + n) + (r)],

n 1,

which is likely to be very small. (δh denotes ‘‘central difference’’—see FINITE DIFFERENCES, CALCULUS OF.) The remainder is − 2.6 × 10−8 (6h)7 f (6) (ξ1 )

with (x) =

r

− 6.4 × 10−10 (6h)9 f (8) (ξ2 ) −1

s

,

(0) = 0,

= −0.00728h7 f (6) (ξ1 )

s=1

− 0.00645h9 f (8) (ξ2 ),

and Jn (x) =

∞ ν=0

x n+2ν (−1)ν ν!(n + ν + 1) 2

is the Bessel function of the first kind of order n.

for some ξ1 , ξ2 between a and a + 6h. The formula a+6h h f (x) dx [41{f (a) + f (a + 6h)} 140 a + 216{f (a + h) + f (a + 5h)} + 17{f (a + 2h) + f (a + 4h)}

BIBLIOGRAPHY Weber, H. F. (1879). Zurich Vierteljahresschrift, 24, 33–76. See also BESSEL FUNCTIONS.

WEDDLE’S RULE This is the following quadrature∗ formula, using values of the integrand at seven equally spaced values of the variable,

a+6h

f (x) dx a

h[f (a) + f (a + 2h) + f (a + 4h) + f (a + 6h) + 5{f (a + h) + f (a + 5h)} + 6f (a + 3h)].

+ 272f (a + 3h)] gives the exact value of the integrals if f (x) is a polynomial of degree 6 or less, though the coefficients are not so simple as for Weddle’s rule. BIBLIOGRAPHY Chataravarti, I. M., Laha, R. G., and Roy, J. (1967). Handbook of Methods of Applied Statistics, Vol. 1, Wiley, New York, pp. 38–41. Milne-Thomson, L. M. (1933). Calculus of Finite Differences. Macmillan, London, p. 172–. Weddle, T. (1854). Cambridge and Dublin Math. J., 9 (Cambridge Math. J., 13), 79–80. See also NUMERICAL INTEGRATION; SHOVELTON’S FORMULA; SIMPSON’S RULE; THREE-EIGHTHS RULE; and TRAPEZOIDAL RULE.

WEIBULL DISTRIBUTION

WEDGE ESTIMATOR This is an estimator of the partial regression∗ coefficients β in the multiple linear regression∗ model Y = Xβ + (see MULTIPLE LINEAR REGRESSION), obtained by modifying the ridge regression∗ estimator. It is

9087

developed and applied rather extensively in the 1950s and the Weibull began to be seriously considered as a competing model in the 1960s [6,29,47,73], especially in problems in which the time to failure was the response of interest [27,32,47,59]. A random variable W follows the threeparameter Weibull distribution [denoted by W ∼ WE3(a, b, c)] if its cumulative distribution is given by

bW = (Z Z)−1 ZY,

FW (w) = 1 − exp{−[(w − a)/b]c }, a < w < ∞,

where Z = X + kX(X X)−1 with k arbitrary. The estimator bW was developed in Riddell and von Hohenbalkan [1].

where b, c > 0. The parameters a and b are location and scale parameters and c is a shape parameter. The density function is given by fW (w) =

REFERENCE 1. Riddell, W. C. and von Hohenbalkan, B. (1978). Unpublished manuscript, Dept. of Economics, University of Alberta, Edmonton, Canada.

c b

w−a b

c−1

× exp{−[(w − a)/b]c }, for a < w < ∞. The reliability function is given by

See also RIDGE REGRESSION.

RW (w) = exp{−[(w − a)/b]c },

WEDGELETS. See

BEAMLETS

AND

MULTISCALE MODELING

WEIBULL DISTRIBUTION The Weibull distribution was named for Professor Waloddi Weibull of Sweden who suggested it as a distribution for a variety of applications [75,76]. The agreement he demonstrated between his observations and those predicted with the fitted Weibulls was very impressive. He considered the problems of yield strength of a Bofors steel, fiber strength of Indian cotton, length of syrtoideas, fatigue life of an St-37 steel, statures of adult males born in the British Isles, and breadth of beans of Phaseolus vulgaris. The distribution was used as early as 1933 by Rosin and Rammler [69] in describing the ‘‘laws governing the fineness of powdered coal.’’ The Weibull includes the exponential distribution∗ as a special case and is sometimes thought of as a generalization of the exponential distribution. The exponential model was

for a < w < ∞. The hazard function, also known as the failure rate, is given by h(w) = c(w − a)c−1 /bc ,

for w > a.

For values of c less than 1, the hazard function is a decreasing function, for c = 1 it is constant, and for c > 1 it is an increasing function of w. The value of c, the shape parameter, is thus an important parameter and often has a characteristic or predictable value depending upon the fundamental nature of the problem being studied. It is fairly common to assume the value of the location parameter (a) is known, often 0. Setting a = 0 or letting X = W − a leads to the common two-parameter Weibull distribution [denoted by X ∼ WE2(b, c)]. RELATION TO EXTREME-VALUE DISTRIBUTIONS The Weibull distribution is one of the extreme-value distributions∗ (a type 3) derived

9088


by Fisher and Tippett [22]. It is also related to their type 1 extreme-value distributions∗ . A random variable Y has a type 1 distribution for minima if its cumulative distribution is given by FY (y) = 1 − exp{− exp[(y − α)/β]}, − ∞ < y < ∞, where β > 0 and α is unrestricted. α is a location parameter and β is a scale parameter and we denote this distribution by EVS(α, β). It is easily seen that if X ∼ WE2(b, c) and if we let Y = ln X, then Y ∼ EVS(ln b, 1/c). It thus follows that the two models are in effect the same model and any statistical procedure developed for one applies to the other. In particular, all general results for location and scale distributions may be called upon when developing procedures for the two-parameter Weibull model [15,23,50,51]. A random variable Z is said to have a type 1 distribution for maxima if FZ (z) = exp[− exp[(z − α )/β ]], −∞ < z < ∞, where β > 0 and α is unrestricted. We denote this distribution by EVL(α , β ) and note that α and β are location and scale parameters. It is easily seen that if Y ∼ EVS(α, β) and we let Z = −Y, we have Z ∼ EVL(−α, β). Thus we also have that if W ∼ WE2(b, c) and Z = − ln W, then Z ∼ EVL(− ln b, 1/c). It follows then that all results for type 1 for maximum distributions also have corresponding results for Weibull distributions (see, e.g., [5] and [50]). PROPERTIES (a) Moments. E[(W − a)r ] = br (1 + r/c) for r > 0. (b) If W ∼ WE3(a, b, c), then [(W − a)/b]c is a standard exponential random variable. (c) If X1 , X2 , . . . , Xn are independent and identically distributed (i.i.d.) random variables, then Xm = min(X1 , . . . , Xn ) has a Weibull distribution if and only if the Xi ’s are Weibull random variables [16]. If Xi ∼ WE3(a, b, c), then Xm ∼ WE3(a, bn−1/c , c).

(d) Let T be the time to first occurrence of a nonhomogeneous Poisson process∗ with intensity V(t) = ctc−1 /bc . Then T ∼ WE2(b, c). (e) Moments of order statistics∗ are available in the literature. Means, variances, and covariances of Weibull order statistics are given by Weibull [77] for c−1 = 0.1(0.1)1.0 and n = 5(5)20, and also by Govindarajulu and Joshi [23] for c = 1, 2, 2.5, 3(1)10 and n = 2(1)10. Harter [27] gives means for c = 0.5(0.5), 4.0(1)8 and n = 1(1)40. Means, variances, and covariances of logWeibull and thus extreme-value order statistics are provided by Mann [47] for n = 2(1)25 (see also [50]). White [78] also gives means for n = 1(1)100. STATISTICAL INFERENCE FOR THE TWO-PARAMETER WEIBULL Early users [60,76] of the two-parameter Weibull model were mainly engineers; they used graphical methods for estimation of the parameters. Such methods served them well and also provided a subjective method for judging the adequacy of the model. As noted earlier the log transform of a Weibull random variable has a type 1 extreme-value distribution, which is a location and scale parameter distribution. Thus the methods of estimation available for such families can be applied to log-Weibull data. Lieblein and Zelen [45] and White [78] worked in this way in developing best linear unbiased estimates. Later, Mann [48,49,51] suggested the use of best linear invariant estimators (BLIEs) and gave tables of weights to be used for samples of size 25 or less. All of these methods are actually developed for estimating the parameters in a type 1 extreme-value distribution; for example, Mann’s estimators are invariant under location and scale transformation of the logWeibull data, and it is in the log-Weibull data that the estimators are linear. Approximate best linear estimators are considered by McCool [58], Hassanein [31], Chan and Kabir [12], Mann and Fertig [56], and Downton [15]. In a notable paper,


Johns and Lieberman [32] used Monte Carlo simulation to obtain lower confidence bounds on reliability based upon estimators that are asymptotic approximations to the BLIEs. Simple linear estimators that have high relative efficiency, especially for heavily censored samples, have been developed for complete and censored samples. See, for example, Bain [5] and Engelhardt and Bain [19]. Maximum likelihood∗ estimation of the parameters has been considered by Harter and Moore [30], Cohen [14], and Leone et al. [44], among others. The maximum likelihood estimators (MLEs) based upon a complete sample of size n are given by the solution to the equations ln Xi 1 Xicˆ ln Xi + − = 0, cˆ cˆ n Xi 1/ˆc Xicˆ . bˆ = n The existence and uniqueness of the solution to these equations was shown by Pike [65] and McCool [59]. They are easily solved by using Newton’s method to solve first for cˆ and ˆ then using this to obtain b. For the two-parameter Weibull the MLEs are asymptotically efficient and they are asymptotically √ normally distributed. Let U = √ ( n(bˆ − b), n(ˆc − c)) so that asymptotically U ∼ MVN(0, V), where 1.109b2 /c2 0.257b . V= 0.257b 0.608c2 Confidence intervals for the parameters based upon maximum likelihood∗ estimates (MLEs) are given by Thoman et al. [73] and by McCool [60–62]. These make use ˆ of the fact that the pivotal quantities b/b and cˆ (ln bˆ − ln b) have densities that do not depend upon the values of b and c. This property is true for complete samples and also for samples with censoring on the order statistics. The necessary tables for certain censoring patterns are given by Billman et al. [11]. Confidence intervals for the reliability at some time of interest, based upon a maximum likelihood estimate of reliability, are given by Thoman et al. [74] for complete samples and by Billman et al. [11] for certain censored

9089

samples. These were developed by Monte Carlo∗ methods, using the fact that the distribution of the MLE of reliability depends only upon the true value of the reliability. Tables of these and other methods for obtaining confidence intervals in the Weibull setting may be found in Bain [5] and Mann and Fertig [56]. Kingston and Patel [36] consider interval estimation of the largest reliability of K populations and simultaneous confidence intervals for K reliabilities. Sample sizes needed to choose the better of two Weibull populations are given by Rademaker and Antle [67]. Qureishi [66] and Thoman and Bain [72] considered the problem of choosing the better of two Weibull populations assuming a common shape parameter. Kingston and Patel [35] consider the problem of choosing m or less populations from K populations, which contain the best l populations in terms of reliability with prescribed probability of correct selection. Lawless [39,40] considers inference procedures based upon MLEs conditional upon the ancillary∗ statistics. The resulting confidence intervals are just slightly shorter than the simpler ones based only upon the MLEs. Approximations to the tables necessary for statistical inference based upon MLEs are given by Bain and Engelhardt [9]. They show that approximately (0.822)nc2 /ˆc2 ∼ χ 2 (0.822(n − 1)), √ n − 1ˆc(ln bˆ − ln b)/1.053 ∼ t(n − 1), where χ 2 and t denote the usual chi-squared∗ and Student’s t-distributions∗ . These lead to convenient tests of hypotheses and confidence intervals. Bain and Engelhardt [9] also give approximate tolerance limits and confidence limits on reliability. Prediction limits based upon MLEs have been studied by Antle and Rademaker [3], Lawless [38], and Mann [52]. Bayesian prediction procedures are considered by Evans and Nigin [21] and Ashour and Raswan [4]. Many other simplified estimators based upon two or more observations have been proposed. A number of these are reviewed by Johnson and Kotz [33] and Mann and Fertig [56]. Bain and Antle [6] review some least-squares-type estimators. They have the advantage of being applicable with various

9090


censoring or missing observations; however, their efficiency is not as good as maximum likelihood estimation. The discussion in this section assumes that the two unknown parameters are b and c. Another two-parameter Weibull of interest has a and b unknown. This particular model, a location- and scale-parameter distribution, has received little attention. A characteristic value of c may indeed be known in many situations. If this is the case it was shown by Rockette et al. [68] and by Peto and Lee [64] that the maximum likelihood estimates for a and b always exist and are unique. Weibull probability paper may be obtained from TEAM, Box 25, Tamworth, NH 03886. STATISTICAL INFERENCES FOR THE THREE-PARAMETER WEIBULL As noted earlier, if either a or c is assumed known the maximum likelihood estimates of the remaining parameters always exist and are unique. When all three are unknown, the likelihood function is unbounded as a → min Xi . However, if one assumes c 1, then the likelihood function is bounded and often has a maximum. However, one should not rush into solving the likelihood equations because if they have one solution, they will for sure have at least two solutions [68]. A picture that is useful for the likelihood function is obtained by fixing the location parameter a at several values near and to the left of min Xi and solving for the respective bˆ and cˆ and the maximized likelihood as a function of a. A plot of this maximized likelihood vs. a will show that as a moves left from Xi , the maximized likelihood will decrease and then either (i) continue decreasing, or (ii) have a minimum very near min Xi and then a relative maximum farther to the left. If the maximized likelihood has no relative maximum to the left of min Xi , then one has ˆ as the aˆ = min Xi , cˆ = 1, and bˆ = (Xi − a)/n maximum likelihood estimates in the space c 1.

If the maximized likelihood as a function of a has a relative maximum to the left of min Xi , then its value there should be compared with its value at the corner point described above (c = 1, etc.) and the point at which the likelihood is larger should be chosen as the maximum likelihood estimate. The above process is not at all difficult to employ because with a known the likelihood equations for b and c are easily solved. It is possible that there may indeed be more than two solutions to the likelihood equations when all three parameters are unknown. However, this does not seem to occur in practice [68]. Lemon [43] studies the three-parameter Weibull as does Cohen [14], Zanakis [80], and Wyckoff et al. [79]. The last paper gives confidence limits for c and a and b unknown. Also see Harter and Dubey [28], Bain and Thoman [10], Bain and Antle [7], and Mann and Fertig [55], and the discussions on this distribution in WEIBULL DISTRIBUTION, MANN –FERTIG TEST STATISTIC FOR and MAXIMUM LIKELIHOOD ESTIMATION. COMPETING MODELS Models that generally compete with the Weibull are the exponential∗ , lognormal∗ , and the gamma∗ . For testing H0 : oneparameter exponential vs. H1 : WE2(b, c), one should simply test H0 : c = 1 vs. H1 : c = 1 by use of the maximum likelihood estimate of c and the tables given by Thoman et al. [73]. For testing H0 : two-parameter exponential vs. H1 : WE3(a, b, c), one should use either the method in Antle et al. [2] or Engelhardt and Bain [18]. For choosing between a Weibull and a lognormal model, the ratio of maximized likelihoods [17] provides a very good test. The Weibull and the lognormal are very difficult to distinguish and sample sizes of 50 or more are needed in order to have a large probability of correct selection. Mann and Fertig [55] give a test for H0 : a = a0 in the three-parameter Weibull setting. This provides a useful test of whether the three-parameter Weibull is needed. Bain and Engelhardt [8] consider discrimination between the Weibull and gamma distributions∗ on the basis of probability of


correct selection. Chandra et al. [13] provide percentage points for goodness-of-fit∗ tests for the extreme-value or Weibull distributions with unknown parameters based on the Kolmogorov–Smirnov∗ or Kuiper∗ statistics. Some results for mixed or compound Weibull distributions are provided by McNolty et al. [63] and Harris and Singpurwalla [26]. Multivariate Weibull distributions are considered by Lee [42] and Johnson and Kotz [34]. Klein and Basu [37] consider accelerated life-testing∗ under competing causes of failure and Shaffer [71] studies confidence bands for minimum fatigue life under K stress levels. REFERENCES 1. Antle, C. E. (1972). Choice of Model for Reliability Studies and Related Topics. Tech. Reps. ARL 72-0108 and AD 751340, ARL, WrightPatterson AFB, OH. 2. Antle, C. E., Klimko, L. A., Rockette, H. E., and Rademaker, A. W. (1975). Upper bounds for the power of invariant tests for the exponential distribution with Weibull alternatives. Technometrics, 17, 357–360. 3. Antle, C. E. and Rademaker, A. (1972). Confidence intervals for the maximum of a set of future observations from the extreme value distribution. Biometrika, 59, 475–477. 4. Ashour, S. K. and Rashwan, D. R. (1981). Bayesian predictions for compound Weibull model. Commun. Statist. A, 10, 1613–1624. 5. Bain, L. J. (1978). Statistical Analysis of Reliability and Life-Testing Models. Marcel Dekker, New York. 6. Bain, L. J. and Antle, C. E. (1967). Estimation of parameters in the Weibull distribution. Technometrics, 9, 621–627. 7. Bain, L. J. and Antle, C. E. (1970). Inferential Procedures for the Weibull and Generalized Gamma Distributions. Tech. Rep. ARL 700266, ARL, Air Force Systems Command, USAF, Wright-Patterson AFB, OH. 8. Bain, L. J. and Engelhardt, M. (1980a). Probability of correct selection of Weibull versus gamma based on likelihood ratio. Commun. Statist. A, 9, 375–381. 9. Bain, L. J. and Engelhardt, M. E. (1980b). Simple approximate distributional results for confidence and tolerance limits for the Weibull distribution based on maximum likelihood estimators. Technometrics, 23, 15–20.

9091

10. Bain, L. J. and Thoman, D. R. (1968). Some tests of hypotheses concerning the three-parameter Weibull distribution. J. Amer. Statist. Ass., 63, 853–860. 11. Billman, B. R., Antle, C. E., and Bain, L. J. (1972). Statistical inference from censored Weibull samples. Technometrics, 14, 831–840. 12. Chan, L. K. and Kabir, A. B. M. L. (1969). Optimum quantities for the linear estimation of the parameters of the extreme-value distribution in complete and censored samples. Naval Res. Logist. Quart., 16, 381–404. 13. Chandra, M., Singpurwalla, N. D., and Stephens, M. A. (1981). Kolmogorov statistics for tests of fit for the extreme-value and Weibull distributions. J. Amer. Statist. Ass., 76, 729–731. 14. Cohen, A. C. (1974). Multi-censored sampling in three-parameter Weibull distribution. Technometrics, 17, 347–352. 15. Downton, F. (1966). Linear estimates of parameters in the extreme-value distribution, Technometrics, 8, 3–17. 16. Dubey, S. D. (1966). Characterization theorems for several distributions and their applications. J. Indust. Math., 16, 1–22. 17. Dumonceaux, R. H. and Antle, C. E. (1973). Discrimination between the lognormal and the Weibull distribution. Technometrics, 15, 923–926. 18. Engelhardt, M. and Bain, L. J. (1975). Tests of two-parameter exponentiality against threeparameter Weibull alternatives. Technometrics, 17, 353–356. 19. Engelhardt, M. and Bain, L. J. (1977). Simplified statistical procedures for the Weibull or extremevalue distribution. Technometrics, 19, 323–331. 20. Engelhardt, M. and Bain, L. J. (1979). Prediction limits and two-sample problems with complete or censored Weibull data. Technometrics, 21, 233–237. 21. Evans, I. G. and Nigin, A. M. (1980). Bayesian prediction for two-parameter Weibull lifetime models. Commun. Statist. A, 9, 659–672. 22. Fisher, R. A. and Tippett, L. M. C. (1928). Limiting forms of the frequency distribution of the largest or smallest member of a sample. Proc. Camb. Philos. Soc., 24, 180–190. 23. Govindarajulu, Z. and Joshi, M. (1968). Best linear unbiased estimation of location and scale parameters of Weibull distribution using ordered observations. Statist. Appl. Res., JUSE, 15, 1–14.

9092


24. Gross, A. J. and Clark, V. A. (1975). Survival Distributions: Reliability Applications in the Biomedical Sciences. Wiley, New York. 25. Hager, H. W., Bain, L. J., and Antle, C. E. (1971). Reliability estimation for the generalized gamma distribution and robustness of the Weibull model. Technometrics, 13, 547–557. 26. Harris, C. M. and Singpurwalla, N. (1968). Life distributions derived from stochastic hazard functions. IEEE Trans. Rel., R-17, 70–79. 27. Harter, H. L. (1969). Order Statistics and Their Use in Testing and Estimation: Estimator Based on Order Statistics from Various Populations, Vol 2. U.S. Government Printing Office, Washington, D.C. 28. Harter, H. L. and Dubey, S. D. (1967). Theory and Tables for Tests of Hypotheses Concerning the Mean and the Variances of a Weibull Population. Tech. Rep. ARL 67-0059, WrightPatterson AFB, OH. 29. Harter, H. L. and Moore, A. H. (1967). Asymptotic variances and covariances of maximumlikelihood estimators, from censored samples, of the parameters of Weibull and gamma populations. Ann. Math. Statist., 38, 557–570. 30. Hartr, H. L. and Moore, A. H. (1968). Maximum-likelihood estimation, from doubly censored samples, of the parameters of the first asymptotic distribution of extreme values. J. Amer. Statist. Ass., 63, 889–901. 31. Hassanein, K. M. (1972). Simultaneous estimation of the parameters of the extreme value distribution by sample quantiles. Technometrics, 14, 63–70. 32. Johns, M. V. and Lickerman, G. J. (1966). An exact asymptotically efficient confidence bound for reliability in the case of the Weibull distribution. Technometrics, 8, 135–175. 33. Johnson, N. L. and Kotz, S. (1970). Distributions in Statistics: Continuous Univariate Distributions—1. Wiley, New York. 34. Johnson, N. L. and Kotz, S. (1975). A vector multivariate hazard rate. J. Multivariate Anal., 5, 53–66. 35. Kingston, J. V. and Patel, J. K. (1980). A restricted subset selection procedure for Weibull populations. Commun. Statist. A, 9, 1371–1383. 36. Kingston, J. V. and Patel, J. K. (1981). Interval estimation of the largest reliability of K Weibull populations. Commun. Statist. A, 10, 2279–2298. 37. Klein, J. P. and Basu, A. P. (1981). Weibull accelerated life tests when there are competing causes of failure. Commun. Statist. A, 10, 2073–2100.

38. Lawless, J. F. (1973). On the estimation of safe life when the underlying life distribution is Weibull. Technometrics, 15, 857–865. 39. Lawless, J. F. (1975). Construction of tolerance bounds for the extreme-value and Weibull distributions. Technometrics, 17, 255–261. 40. Lawless, J. F. (1978). Confidence interval estimation for the Weibull and extreme value distributions. Technometrics, 20, 355–364. 41. Lawless, J. F. and Mann, N. R. (1976). Tests for homogeneity of extreme-value scale parameters, Commun. Statist. A, 5, 389–405. 42. Lee, L. (1979). Multivariate distributions having Weibull properties. J. Multivariate Anal., 9, 267–277. 43. Lemon, G. H. (1975). Maximum likelihood estimation for the three parameter Weibull distribution based on censored samples. Technometrics, 17, 247–254. 44. Leone, F. C., Rutenberg, Y. H., and Topp, C. W. (1960). Order Statistics and Estimators for the Weibull Population. Tech. Reps. AFOSR TN 60-489 and AD 237042, Air Force Office of Scientific Research, Washington, D.C. 45. Lieblein, J. and Zelen, M. (1956). Statistical investigations of the fatigue life of deep groove ball bearings. Research Paper 2719. J. Res. Natl. Bur. Stand., 57, 273–316. 46. Littell, R. C., McClave, J. T., and Offen, W. W. (1979). Goodness-of-fit tests for the twoparameter Weibull distribution. Commun. Statist. B, 8, 257–269. 47. Mann, N. (1965). Point and Interval Estimates for Reliability Parameters when Failure Times have the Two-Parameter Weibull Distribution. Ph.D. dissertation, University of California at Los Angeles, Los Angeles, CA. 48. Mann, N. (1967a). Results on Location and Scale Parameter Estimation with Application to the Extreme-Value Distribution. Tech. Rep. ARL 67-0023, Office of Aerospace Research, USAF, Wright-Patterson AFB, OH. 49. Mann, N. (1967b). Tables for obtaining the best linear invariant estimates of parameters of the Weibull distribution. Technometrics, 9, 629–645. 50. Mann, N. (1968). Results on Statistical Estimation and Hypothesis Testing with Application to the Weibull and Extreme-Value Distributions. Tech. Reps. ARL 68-0068 and AD 672979, ARL, Wright-Patterson AFB, OH. 51. Mann, N. (1971). Best linear invariant estimation for Weibull parameters under progressive censoring. Technometrics, 13, 521–533.


9093

52. Mann, N. R. (1976). Warranty periods for production lots based on fatigue-test data. Eng. Fracture Mech., 8, 123–130.

66. Qureishi, A. S. (1964). The discrimination between two Weibull processes. Technometrics, 6, 57–75.

53. Mann, N. (1977). An F-approximation for twoparameter Weibull and log-normal tolerance bounds based on possibly censored data. Naval Res. Logist. Quart. 9, 187–196.

67. Rademaker, A. W. and Antle, C. E. (1975). Sample size for selecting the better of two populations. IEEE Trans. Rel., R-24, 17–20.

54. Mann, N. and Fertig, K. W. (1973). Tables for obtaining confidence bounds and tolerance bounds based on best linear invariant estimates of parameters of the extreme-value distribution. Technometrics, 15, 87–101. 55. Mann, N. R. and Fertig, K. W. (1975). A goodness-of-fit test for the two-parameter vs. three parameter Weibull, confidence bounds for threshold. Technometrics, 17, 237–246.

68. Rockette, H., Antle, C. E., and Klimko, L. A. (1974). Maximum likelihood estimation with the Weibull model. J. Amer. Statist. Ass., 69, 246–249. 69. Rosen, P. and Rammler, B. (1933). The laws governing the fineness of powdered coal. J. Inst. Fuels, 6, 29–36. 70. Schafer, R. E. and Sheffield, T. S. (1976). On procedures for comparing two Weibull populations. Technometrics, 18, 231–235.

56. Mann, N. and Fertig, K. W. (1977). Efficient unbiased quantile estimators for moderatesize complete samples from extreme-value and Weibull distributions, confidence bounds and tolerance and prediction intervals. Technometrics, 19, 87–93.

71. Shafer, R. B. (1974). Confidence bands for minimum fatigue life. Technometrics, 16, 113–123.

57. Mann, N. R., Schafer, R. E., and Singpurwalla, N. D. (1974). Methods for Statistical Analysis of Reliability and Life Data, Wiley, New York.

73. Thoman, D. R., Bain, L. J., and Antle, C. E. (1969). Inferences on the parameters of the Weibull distribution. Technometrics, 11, 445–460.

58. McCool, J. I. (1965). The construction of good linear unbiased estimates from the best linear estimates for a smaller sample size. Technometrics, 7, 543–552.

74. Thoman, D. R., Bain, L. J., and Antle, C. E. (1970). Reliability and tolerance limits in the Weibull distribution. Technometrics, 12, 363–371.

59. McCool, J. I. (1970). Inferences on Weibull percentiles and shape parameter from maximum likelihood estimates. IEEE Trans. Rel., R-19, 2–9.

75. Weibull, W. (1939). A statistical theory of the strength of materials. Ing. Vetenskaps Akad. Handl., 151, 1–45.

60. McCool, J. I. (1974). Inferential Techniques for Weibull Populations. Tech. Reps. 74-0180 and AD A-009645, ARL, Wright-Patterson AFB, OH. 61. McCool, J. I. (1975). Inferential Techniques for Weibull Populations II. Tech. Rep. ARL 760233, ARL, Wright-Patterson AFB, OH. 62. McCool, J. I. (1979). Analysis of single classification experiments based on censored samples from the two-parameter Weibull distribution. J. Statist. Plann. Inf., 3, 39–68. 63. McNolty, F., Doyle, J., and Hansen, E. (1980). Properties of the mixed exponential failure process. Technometrics, 22, 555–566. 64. Peto, R. and Lee, P. N. (1973). Weibull distributions for continuous carcinogenesis experiments. Biometrics, 29, 457–470. 65. Pike, M. (1966). A suggested method of analysis of a certain class of experiments in carcinogenesis. Biometrics, 22, 142–161.

72. Thoman, D. R. and Bain, L. J. (1969). Two sample tests in the Weibull distribution. Technometrics, 11, 805–815.

76. Weibull, W. (1951). A statistical distribution function of wide applicability. J. Appl. Mech., 18, 293–297. 77. Weibull, W. (1967). Estimation of Distribution Parameters by a Combination of the Best Linear Order Statistic Method and Maximum Likelihood. Tech. Rep. AFML 67105, Air Force Materials Laboratory, WrightPatterson AFB, OH. 78. White, J. S. (1967). The Moments of logWeibull Order Statistics. General Motors Research Publication GMR-717, General Motors Corporation, Warren, Michigan. 79. Wyckoff, J., Bain, L. J., and Engelhardt, M. E. (1980). Some complete and censored sampling results for the three-parameter Weibull distribution, J. Statist. Comp. Simul., 11, 139–152. 80. Zanakis, S. H. (1979). A simulation study of some simple estimators for the threeparameter Weibull distribution. J. Statist. Comp. Simul. 9, 101–116.

9094

WEIBULL DISTRIBUTION, MANN–FERTIG TEST STATISTIC FOR

Editorial Note The double Weibull distribution, with PDF fX (x) = 12 c|x|c−1 exp(−|x|c ),

c > 0,

which is found by reflecting the standard Weibull about the origin, is studied by N. Balakrishnan and S. Kocherlakota in Sankhya¯ B, 47 (1985), 161–178. See also EXTREME-VALUE DISTRIBUTIONS; HAZARD RATE AND OTHER CLASSIFICATIONS OF DISTRIBUTIONS; LIFE TESTING; MAXIMUM LIKELIHOOD ESTIMATION; MULTIVARIATE WEIBULL DISTRIBUTIONS; WEIBULL DISTRIBUTION, MANN –FERTIG TEST STATISTIC FOR; and WEIBULL PROCESSES.

CHARLES E. ANTLE LEE J. BAIN

WEIBULL DISTRIBUTION, MANN–FERTIG TEST STATISTIC FOR The Mann—Fertig [1] statistic was designed to test that a set of data came from a two-parameter Weibull distribution∗ with unknown parameters, under an alternative hypothesis that the distribution is threeparameter Weibull. Thus the test is designed for data which, because of theory (perhaps a weakest-link or other extreme-value consideration) are assumed to have been selected randomly from a Weibull distribution with distribution function given by FZ (z) = 1 − exp{−[(z − λ)/δ]β }. Here λ 0 is a threshold before which, in failure analysis, failure occurs with probability zero, δ > 0 is a scale parameter, and β > 0 is a shape parameter. ∗ The Mann—Fertig goodness-of-fit test essentially tests the hypothesis H0 : (λ = 0) versus the alternative λ > 0 by determining whether the left tail is too short. A variation of the test statistic can be used to obtain a median unbiased∗ estimate of λ. The statistic for testing H0 is a slight modification of the S statistic of Mann et al. [2], designed earlier to test the hypothesis of a two-parameter Weibull against some more general alternatives. It is based on an

asymptotic result of Pyke [3] concerning the approximate independence and approximate exponentiality of spacings∗ , i.e., differences of successive order statistics, from continuous distributions. If Z is a Weibull variate with λ = 0 and if X = ln Z, then X has a Type I distribution of the smallest extreme having distribution function

Fx (x) = 1 − exp − exp (x − η)/ξ with location parameter η and scale parameter ξ = β −1 . The S statistic is based on normalized spacings from the distribution of X, li = (Xi+1,n − Xi,n )

E(Xi+1,n ) − E(Xi,n ) ,

i = 1, . . . , m − 1.

(1)

Here E is the expectation operator, Xi,n is the ith smallest observation of X in a sample of size n, and Xm,n with m n, is the largest value of X observed. It was van Montfort [5] who observed that for a Type I extreme value distribution∗ the asymptotic result of Pyke pertaining to continuous distributions applies for very small samples. For samples as small as three or four the result applies nearly exactly at the lower end of the distribution and roughly at the upper end. Since the Type I extreme-value distribution is a member of the location-scale parameter∗ family, one can observe that li is independent of the unknown location parameter η. If E(Yi,n ), the expected value of the reduced parameter-free ith order statistic, is used in li , i = 1, . . . , m − 1, in place of the unknown E(Xi,n ), i = 1, . . . , m, then each li is proportional to the unknown scale parameter ξ . Or, E(Xi+1,n ) − E(Xi,n ) = ξ E{(Xi+1,n − η)/ξ } − ξ E{(Xi,n − η)/ξ } = ξ {E(Yi+1,n ) − E(Yi,n )}. One determines a value for k as discussed presently and forms the test statistic Pk,m =

m−1 i=k+1

li

m−1 i=1

li ,

(2)

WEIBULL DISTRIBUTION, MANN–FERTIG TEST STATISTIC FOR

as the ratio of sums of li s.Pk,m is therefore parameter free, even though the E(Yi,n )’s are used in place of the E(Xi,n )’s. Moreover, the distribution of each 2li , as defined in (1), is approximately chi-square with 2 degrees of freedom. Hence the distribution of Pk,m is essentially Beta∗ with parameters m − k − 1 and k when H0 is true. The S statistic of Mann et al. is of the form Pk,m with k = [m/2] and [r] the greatest integer less than or equal to r. A Monte Carlo∗ generation of critical values of S resulted essentially in percentiles of appropriate Beta distributions. When k is smaller than [m/2], as it tends to be for testing a three-parameter Weibull, the Beta approximation is enhanced. For Pk,m large, H0 is rejected. The appropriate values for k for small sample sizes are given in Mann and Fertig [1]. If the sample size n is 15 or larger, k equal to the integer nearest to m/3 tends to be optimal for significance levels ranging from 0.01 to 0.25. Before using the goodness-of-fit test, it is useful and, in fact, important to plot the data on Weibull probability paper. It may be, for example, that the data, rather than exhibiting a threshold, are from a mixture of two Weibull distributions reflecting two different phenomena operating at low and high levels of the random variable, respectively. Thus, early and late failures may result from a mixture of two or more types of hardware. In this case, the goodness-of-fit test would tend to reject the hypothesis of a two-parameter Weibull, but it would be incorrect to blindly accept the alternative of a three-parameter Weibull. A probability plot of such data on Weibull paper will ordinarily reveal two or more intersecting straight lines, rather than a smooth curve, concave downward, exhibited by three-parameter Weibull data. CONFIDENCE BOUNDS AND POINT ESTIMATES The theory used to develop the Mann—Fertig statistic Pk,m can be used also to obtain iteratively confidence bounds and a median unbiased estimate of a threshold parameter λ. Note that if Z has a three-parameter Weibull distribution with threshold parameter λ, then Xi,n = ln(Zi,n − λ) is the ith Type

9095

I extreme-value distribution order statistic and P∗k,m (λ) =

m−1 i=k+1

l∗i

m−1

l∗i ,

i=1

with l∗i =

ln(Ti+1,n − λ) − ln(Ti,n − λ) , E(Yi+1,n ) − E(Yi,n ) i = 1, . . . , m − 1,

has the distribution of Pk,n (defined in (2)) under H0 . Mann and Fertig demonstrate that P∗k,m (λ) is monotonically decreasing in λ for 0 < λ < Z1,n . Therefore a lower confidence bound at level γ can be determined for λ if H0 is rejected at significance level α and γ 1 − α. Somerville [4] discovered the necessary condition of rejection of H0 . He also determined that if α = 0.5, as in obtaining a median unbiased estimate of λ, then the optimal k is the integer nearest to m/5. Somerville’s results demonstrate that the power of a test based on Pk,m is bounded above, even though λ may be greatly larger than zero. This bound increases as m and the significance level α increase and as the shape parameter β = 1/ξ decreases. Thus, the maximum power of a test of H0 based on Pk,m is 1.00 if m = 10, β = 0.5, and α = 0.5 or if m = 25, β = 1.0, and α = 0.10. If n = 25 and β = 2, then the maximum power is 0.70 if α = 0.05. Example As an example of the use of this statistic to obtain a median unbiased estimate of λ, we consider a sample of 96 magnitudes of largest California earthquakes occurring in six-month periods over 48 years. Reference to EXTREME-VALUE DISTRIBUTIONS shows that if magnitude M has a Type III distribution of largest values with upper threshold µ, then −(M − µ) = −M − (−µ) = µ − M has a Weibull distribution with lower threshold zero. Or, −M has a Weibull distribution with lower threshold λ = −µ. The presence of an upper threshold is corroborated by physical theory. Hence, a Type III distribution is appropriate for the analysis of the data.

9096

WEIBULL-EXPONENTIAL DISTRIBUTION

A moment estimate of λ is λˆ = −12.3. Using λˆ to calculate P∗k,m with k = 19 yields 0.803. The 50th percentile of a Beta distribution with parameters 76 and 19 is 0.802 and the 10th percentile is 0.745. The very close agreement of the two independent estimates, the moment estimate and the median unbiased estimate, of λ = −µ gives statistical corroboration of the Type III distribution and the upper threshold specified by physical theory. A plot of the data on Weibull probability paper yields a straight line for Z = 12.3 − M. Acknowledgment Research supported by the Office of Naval Research, Contract N00014-82-K-0023, Project 047-204.

REFERENCES 1. Mann, N. R. and Fertig, K. W. (1975). Technometrics, 17, 237–245. 2. Mann, N. R., Scheuer, E. M., and Fertig, K. W. (1973). Commun. Statist., 2, 383–400. 3. Pyke, R. (1965). J. R. Statist. Soc. B, 27, 395–449. 4. Somerville, P. N. (1977). In Theory and Applications of Reliability, Vol. I, P. Tsokos and Shimi, I. N. eds. 5. van Montfort, M. A. J. (1970). J. Hydrology, 11, 421–427. See also WEIBULL DISTRIBUTION.

HAZARD RATE AND OTHER CLASSIFICATIONS OF DISTRIBUTIONS): h(t; λ, α, τ ) λ, = λ + λα α(t − τ )α−1 ,

if 0 t < τ , if τ < t,

which is a superposition (for t τ ) of a Weibull∗ hazard rate on a constant hazard. As τ → ∞, the distribution approaches an exponential distribution∗ with parameter λ. The family is suitable for modeling systems that, after a certain length of time, enter a wear-out phase in which components of the system have an increasing hazard rate. Zacks [1] also develops an adaptive Bayesian∗ estimator for τ (λ and α being assumed to be known), when τ has a prior CDF of the form ξ (τ ) = {p + (1 − p)[1 − e− (τ −τ0 ) ]}, for τ τ0 , where 0 < p < 1, 0 < < ∞, and τ0 is a time point chosen so that the true change point∗ τ exceeds it with prior probability∗ 1. REFERENCE 1. Zacks, S. (1984). Operat. Res., 32, 741–749. See also ADAPTIVE METHODS.

NANCY R. MANN

WEIBULL PARAMETERS, MENON ESTIMATORS FOR

WEIBULL-EXPONENTIAL DISTRIBUTION Zacks [1] introduced a three-parameter Weibull-exponential distribution having a CDF F(x; λ, α, τ ) = 1 − exp{−λx − (λ(x − τ )+ )α }, x 0, where Y+ = max(0, Y), λ > 0 (scale parameter), α 1 (shape parameter), and τ 0 (change-point parameter). This family possesses the following nondecreasing failure-rate function (see

These are estimators for the parameters θ , c of the Weibull distribution∗ with density function f (t) =

ctc−1 exp(−t/θ )c , θ t 0,

θ > 0,

k > 0.

Menon’s estimators [2] for d = 1/c and α = log θ based on n independent values T1 , T2 , . . ., Tn are d = [(6/π 2 )s2Y ]1/2

and

α = Y + 0.5772,

where Yi = log Ti i = 1, . . . , n, and Y and sY are the ‘‘usual’’ sample mean and standard

WEIBULL POPULATIONS, SCHAFER–SHEFFIELD COMPARISON TEST FOR

deviation of Y1 , . . . , Yn . The number 0.5772 is Euler’s constant truncated at the fourth significant figure. The Menon estimators of c and θ are cˆ = 1/α, ˆ

θˆ = exp(α), ˆ

respectively. Although these estimators are computationally attractive, recent investigations of Engeman and Keefe [1] indicate both the generalized least squares∗ and the maximum likelihood∗ estimator of Weibull parameters are substantially more efficient. REFERENCES 1. Engeman, R. M. and Keefe, T. J. (1982). Commun. Statist. A, 11, 2181–2193. 2. Menon, M. V. (1963). Technometrics, 5, 175–182. See also MAXIMUM LIKELIHOOD ESTIMATION and WEIBULL DISTRIBUTION.

WEIBULL-POISSON PROCESS This is a nonhomogeneous Poisson∗ process with intensity function λ(t) = (β/θ )(t/θ )

β−1

,

t > 0; β, θ > 0.

It has been studied by Bain and Engelhardt [1,5], Bain et al. [2]. Crow [3,4], Finkelstein [6], Lee and Lee [7], and Saw [8], among others. There has been special interest in constructing tests of hypotheses on the value of β, with θ regarded as a nuisance parameter∗ [3]. (If β = 1, we have a simple homogeneous Poisson process.) Powers of such tests have been studied in [3].

4. Crow, L. H. (1982). Technometrics, 24, 67–72. 5. Engelhardt, M. E. and Bain, L. J. (1978). Technometrics, 20, 167–169. 6. Finkelstein, J. (1976). Technometrics, 18, 115–117. 7. Lee, L. and Lee, S. K. (1978). Technometrics, 20, 41–45. 8. Saw, J. G. (1975). Commun. Statist., 4, 777–782. See also POISSON PROCESSES; WEIBULL DISTRIBUTION; and WEIBULL PROCESSES.

WEIBULL POPULATIONS, SCHAFER–SHEFFIELD COMPARISON TEST FOR The Schafer—Sheffield test [7] is a procedure for comparing two Weibull* populations. Let x1 , x2 , . . . , xn and y1 , y2 , . . . , yn be independent random samples, both being of size n, from two Weibull distributions F1 (x) = 1 − exp[−(x/b1 )c1 ],

x>0

F2 (y) = 1 − exp[−(y/b2 )c2 ],

y > 0,

respectively. The scale parameters b1 , b2 and the shape parameters c1 , c2 are positive but unknown. The problem is to test the null hypothesis H0 : b1 = b2 , c1 = c2 against the alternative H1 : b1 > b2 , c1 = c2 . The test statistic suggested by Schafer and Sheffield is based on maximum likelihood* estimates of the parameters. Since c1 = c2 is included in both hypotheses, one may set c1 = c2 = c. With no restrictions on b1 and b2 , the maximum likelihood estimates c∗ , b∗1 , and b∗2 of c, b1 , and b2 , respectively, are given by solving the equations ∗

REFERENCES 1. Bain, L. J. and Engelhardt, M. E. (1980). Technometrics, 22, 421–426. 2. Bain, L. J., Engelhardt, M. E., and Wright, F. M. (1985). J. Amer. Statist. Ass., 80, 419–422. 3. Crow, L. H. (1974). In Reliability and Biometry, F. Proschan and R. J. Serfling, eds. SIAM, Philadelphia, pp. 379–410.

9097

∗

(2n/c∗ ) − n[xci ln xi /(xci ) ∗

∗

+ yci ln yi /(yci )]+ ln xi + ln yi = 0, ∗

∗

b∗1 = (xci /n)1/c ,

∗

∗

b∗2 = (yci /n)1/c .

Here c∗ is the positive root of the first equation, and the range of the summation is from i = 1 to n. The solutions (b∗1 , b∗2 , c∗ ) to the above equations uniquely maximize the joint likelihood function with c1 = c2 = c. The

9098

WEIBULL PROCESSES

statistic for testing H0 against H1 is T(c∗ , b∗1 , b∗2 ) = c∗ [ln b∗1 − ln b∗2 ]. The null hypothesis is rejected if the observed value of T is too large. The random variable T(c∗ , b∗1 , b∗2 ) has a distribution that does not depend on b1 , b2 , and c when H0 is true. Nevertheless, the null distribution of T is not available in an explicit form. Schafer and Sheffield obtained some quantiles of T under H0 by the Monte Carlo method, and provided a table containing quantiles for n = 5(1)20, 24(4)40, 50(10)100. In terms of power, the Schafer—Sheffield test is an improvement of a similar test considered by Thoman and Bain [8] for the same problem. But it still relies on the assumption that the populations concerned follow Weibull distributions. Any departure from this assumption may lead to a misleading significance level, should the Schafer—Sheffield test be used. Although failures of many kinds of mechanical devices can be well described by Weibull distributions [2, 3, 9], it is advisable to check whether or not data of interest are from the assumed distributions. One method often used to determine whether or not a set of observed data came from a Weibull distribution* is to perform graphical plotting on a piece of Weibull probability paper (see, e.g., Mann et al. [4, pp. 214–217]). Another method is to perform a goodness-of-fit∗ test, such as the test suggested by Mann et al. [5] (see also [4, Sec. 7.1.2] and WEIBULL DISTRIBUTION, MANN –FERTIG TEST STATISTIC FOR). If data show evidence of departure from the family of Weibull distributions, one should either try some transformations to adjust them or consider nonparametric procedures, such as the Savage test∗ or its modifications (see Basu [1]). A discussion of recent developments in nonparametric procedures, especially for censored data∗ , is given in Miller [6]. REFERENCES 1. Basu, A. P. (1968). Ann. Math. Statist., 39, 1591–1604. (A modified Savage test for type II censored data.)

2. Kao, J. H. K. (1958). IRE Trans. Rel. Quality Control, 13, 15–22. (Vacuum-tube failures and the Weibull distribution.) 3. Lieblein, J. and Zelen, M. (1956). J. Res. Nat. Bur. Stand. 57, 273–316. (The fatigue life of deepgroove ball bearings, an example of the Weibull distribution.) 4. Mann, N. R., Schafer, R. E., and Singpurwalla, N. D. (1974). Methods for Statistical Analysis of Reliability and Life Data. Wiley, New York. (Excellent reference book.) 5. Mann, N. R., Scheuer, E. M., and Fertig, K. W. (1973). Commun. Statist. A, 2, 383–400. 6. Miller, R. G., Jr. (1981). Survival Analysis. Wiley, New York. (A book worth reading.) 7. Schafer, R. E. and Sheffield, T. S. (1976). Technometrics, 18, 231–235. 8. Thoman, D. R. and Bain, L. J. (1969). Technometrics, 11, 805–815. 9. Weibull, W. (1951). J. Appl. Mech., 18, 293–297. (Derivation of the Weibull distribution as a probabilistic characterization for the breaking strength of materials.) See also SAVAGE TEST; WEIBULL DISTRIBUTION; and WEIBULL DISTRIBUTION, MANN –FERTIG TEST STATISTIC FOR.

H.K. HSIEH

WEIBULL PROCESSES A Weibull process is a useful model for phenomena that are changing over time. Essentially, all of the work on Weibull processes has been motivated by applications in which occurrences are failures of a repairable system, and it will be discussed in this framework, although it could be applied more generally. Improvement of a system, or reliability∗ . growth, may occur if the system is in a developmental program. On the other hand, if only minimal repairs are made each time a failure occurs, the system will be deteriorating with time. As noted by Ascher [1], care should be taken to distinguish between a Weibull process, which models a repairable system, and a Weibull distribution, which models a nonrepairable system. Other terms, such as ‘‘Duane model’’ are sometimes used. It was discovered by Duane [6], while examining data on the reliability growth of various repairable systems, that the

WEIBULL PROCESSES

number of system failures vs. operating time appeared to be approximately linear when plotting the logarithms of these quantities. Subsequently, Crow [4] proposed a stochastic analog in which the number of system failures is assumed to occur according to a nonhomogeneous Poisson process∗ , {N(t) : t 0}, with mean value function of the form m(t) = E[N(t)] = λtβ , and intensity function of the form v(t) = dm(t)/dt = λβtβ−1 . This is consistent with the empirical work of Duane since ln m(t) = β ln t + ln λ is a linear function of ln t. Another common parameterization is m(t) = (t/θ )β . β < 1 corresponds to improvement of the system, and β > 1 corresponds to deterioration. An alternative characterization of a Weibull process is given by the sequence of successive failure times T1 , T2 , . . . , where Tn represents the time until the nth failure. The primary reason for the terminology ‘‘Weibull process’’ is the fact that the time to first failure T1 has the Weibull distribution∗ with hazard rate function∗ v(t) = (β/θ )(t/θ )β−1 . It is also true that the conditional failure time Tn , given T1 = t1 , . . . , Tn−1 = tn−1 , follows a Weibull distribution, which is truncated below the point tn−1 . In order to estimate or test hypotheses about the parameters of a Weibull process, it is necessary to adopt a method of gathering data. The most common way to obtain data is ‘‘failure truncation,’’ in which the process is observed for a fixed number n of failures. This leads, in a natural way, to an ordered set of data 0 < t1 < t2 < · · · < tn . Another way to obtain data is ‘‘time truncation,’’ in which the process is observed for a fixed length of time t. In this case, the data have one of the following forms: Either (1) N(t) = 0, or (2) N(t) = n > 0 and 0 < t1 < t2 < · · · < tn < t. Notice that, with time truncation, the observed number of occurrences is part of the data set. It will be advantageous to consider the cases of failure and time truncation separately. FAILURE TRUNCATION Estimation and inference procedures, in this case, are discussed by numerous authors (see [2,4,5,7,8,12]). Suppose T1 , T2 , . . . , Tn are the first n successive failure times of a Weibull

9099

process. The joint probability density function (PDF) is f (t1 , t2 , . . . , tn ) = (β/θ )n

n

(ti /θ )β−1

i=1

× exp[−(tn /θ )β ], where β and θ are, respectively, shape and scale parameters. The joint maximum likelihood∗ estimators (MLEs) are

n−1 βˆ = n ln(Tn /Ti ) i=1

and ˆ θˆ = Tn /n1/β .

The variable Z = 2nβ/βˆ is a pivotal quantity∗ (i.e., its distribution is free of unknown parameters), and it has the chisquare distribution∗ with 2(n − 1) degrees of freedom. Thus confidence limits can be derived. Lower 1 − confidence limits will be given, although upper limits and two-sided confidence intervals can be obtained in a similar manner. A lower 1 − confidence limit for β is given by 2 ˆ βL = [β/(2n)]χ 2(n−1), , 2 2 is the 100 percentile of χ2(n−1) . where χ2(n−1), ˆ β ˆ The variable W = (θ /θ ) is also a pivotal quantity and its cumulative distribution function (CDF) is given by

∞

FW (w) =

H(2(nw)z/2n ; 2n)

0

×h(z; 2(n − 1)) dz, where H(z; r) and h(z; r) represent the CDF and PDF of χr2 . A lower 1 − confidence limit for θ is given by 1/βˆ θL = θˆ /w1− ,

where w1− is the 100(1 − ) percentile of W. Tabulated percentiles of W, obtained by Monte Carlo simulation∗ , are given in [8]. A normal approximation is discussed in [2].

9100

WEIBULL PROCESSES

It is also possible to construct confidence limits on the current system reliability following truncation of the process. Suppose a system is in a developmental program until a time at which changes in the system cease. If the changes cease at the time of the nth failure Tn = tn , and if it is assumed that the intensity v(tn ) remains constant thereafter, then the subsequent times between failures of the system will be independent and exponentially distributed with failure rate v(tn ). The ‘‘current system reliability’’ for some specified length of time t0 would be R(t0 ) = exp[−v(tn )t0 ]. Confidence limits can be based on the pivotal quantity ˆ n . The Q = v(Tn )/ˆv(Tn ), where vˆ (Tn ) = nβ/T CDF of Q is ∞ H(4n2 q/z; 2n)h(z; 2(n − 1)) dz. FQ (q) = 0

A lower 1 − confidence limit for v(Tn ) is vL = vˆ (Tn )q , where q is the 100 percentile of Q, and a lower 1 − confidence limit for R(t0 ) is RL = exp[−ˆv(Tn )q1− t0 ]. This is also related to the instantaneous mean time between failure (MTBF) M(t) = 1/v(t). The corresponding lower 1 − confidence limit is ML = 1/{ˆv(Tn )q1− }. Tabulated values ρ1 and ρ2 are given in [5] ˆ n ) < M(Tn ) < ρ2 M(T ˆ n ) is a such that ρ1 M(T 1 − confidence interval for M(Tn ), where ˆ n ) = Tn /(nβ). ˆ A normal approximation is M(T also discussed. It is also possible to construct prediction limits for future failure times. A lower 1 − prediction limit for the (n + k)th future failure time Tn+k is a statistic TL = TL (n, k, 1 − ) such that P[TL < Tn+k ] = 1 − . Such a statistic can be based on the pivotal quantity Y = (n − 1)βˆ ln(Tn+k /Tn ). The resulting prediction limit is ˆ TL = Tn exp[y /{(n − 1)β}],

where y is the 100 percentile of Y. The CDF of Y is FY (y) =

k

mi {1 − [1 + (n + i − 1)

i=1

×y/{n(n − 1)}]−(n−1) }, with mi =

(−1)i−1 (n + k − 1)! . (n − 1)!(k − i)!(i − 1)!(n + i − 1)!

For the case k = 1, an explicit form is obtained, ˆ TL = Tn exp{[(1 − )−1/(n−1) − 1]/β}. A convenient approximation for the general case is

TL = Tn exp[rfr,2(n−1), / 2(n − 1)cβˆ ], where fr,2(n−1), is the 100 percentile of the F-distribution with r= n+k−1 n+k−1 1/i)2 /( i=n 1/i2 ) and 2(n − 1) 2( i=n degrees of freedom, and c=

n+k−1

n+k−1 2 1/i n 1/i .

i=n

i=n

TIME TRUNCATION Estimation and inference procedures, in this case, are discussed in [2,4, and 5]. Suppose N(t) = n is the number of failures in the time interval (0, t]. If n = 0, then a limited amount of statistical analysis is possible on m(t) = E[N(t)]. In particular, the MLE is m(t) ˆ = 0, and an upper 1 − confidence limit 2 . Suppose n > 0. on m(t) is given by 12 χ2,1− The joint PDF of the successive failure times T1 , T2 , . . . , Tn and N(t) is F(t1 , t2 , . . . , tn , n) = (β/θ )n

n (ti /θ )β−1 exp[−(t/θ )β ], i=1

for 0 < t1 < t2 < · · · < tn < t. The joint MLEs are

n ˆ β=n ln(t/Ti ) i=1

WEIBULL PROCESSES

A conservative lower 1 − confidence limit for v(t) is given by the largest solution vL = v1 of the inequality

and 1/βˆ

θˆ = t/n

9101

.

Since N(t) is sufficient for θ , when β is ˆ given fixed, the conditional distribution of β, N(t) = n, is free of θ . Furthermore, since the ˆ given conditional distribution of Z = 2nβ/β, N(t) = n, is chi-square with 2n degrees of freedom, a conditional lower 1 − confidence limit for β is given by

∞

fN(t)|y (k; v1 ) .

k=n

A conservative lower 1 − confidence limit for R(t0 ) is given by RL = exp(−v2 t0 ),

2 ˆ βL = [β/(2n)]χ 2n, .

Due to the time truncation, θ is no longer a scale parameter, and the pivotal property does not hold. An approximate confidence limit for θ is given in [2]. Suppose wn+1,1− is the 100(1 − ) percentile of W for a fixed number of failures n + 1, as discussed in the case of failure truncation. If we now let n represent the observed number of failures in the time truncation case, an approximate lower 1 − confidence limit is given by

where v2 is the smallest solution of the inequality

ˆ + 1)wn+1,1− ]−n/(n+1) }1/β . θL = θ{n[(n

ML = 1/v2 .

In the present case, the current system reliability would be given by R(t0 ) = exp[−v(t)t0 ]. It should be noted that these results will be applicable only when the entire time interval (0, t] has been observed. In order to construct lower confidence limits for R(t0 ), it is necessary to reparameterize the model in terms of v(t) and β. Since it is necessary to have n > 0 to draw inferences on any parameter except m(t), it is desirable to condition on N(t) > 0. The resulting conditional PDF is

Tabulated values π1 and π2 are given in [5] ˆ ˆ < M(t) < π2 M(t) is a 1 − such that π1 M(t) ˆ confidence interval for M(t), where M(t) = ˆ t/(nβ). Normal approximations are also discussed in [2] and [5].

n

The corresponding conservative lower 1 − confidence limit for instantaneous MTBF M(t) is

ˆ

fc (t1 , t2 , . . . , tn , n) [v(t)]n (ti /t)β−1 , exp{(t/β)v(t)} − 1 n

=

i=1

for 0 < t1 < t2 < · · · < tn < t and n 1. In this form, Y = N(t)/βˆ is sufficient∗ for β, when v(t) is fixed, so the conditional distribution of N(t) given Y = y is free of β. The conditional PDF of N(t) given Y = y is fN(t)|y (n; v(t))

∞ −1 {tv(t)y}n {tv(t)y}k , = n!(n − 1)! k!(k − 1)! k=1

for n = 1, 2, . . ..

fN(t)|y (k; v2 ) .

k=1

OTHER TOPICS Multisample procedures are considered by Crow [5], who gives a test of equality of shape parameters for several independent systems. Related work is given by Lee [11], who provides a method for comparing rate functions of several independent processes. Goodness of fit∗ is also considered by Crow [5], who proposes a test based on the Cramér–von Mises statistic∗ . Lee [10] also considers goodness of fit by a different approach. A more general model is developed, which includes the Weibull process and loglinear rate models as special cases. Tests are derived, in this more general framework, for testing adequacy of the simpler Weibull and log-linear rate models. Sequential∗ probability ratio tests for the shape parameter of one or more independent Weibull processes are provided by Bain and Engelhardt [3]. The resulting tests are

9102

WEIERSTRASS APPROXIMATION THEOREM

expressed in terms of the MLEs of the shape parameters in the usual failure truncation situation. A modification of the Weibull process is proposed by Finkelstein [9]. The modified model yields finite and nonzero instantaneous MTBFs at the start and end of development testing.

Parzen, E. (1962). Stochastic Processes. Holden Day, San Francisco, CA. (There is a good discussion of nonhomogeneous Poisson processes and the connection between the counting process and occurrence times.) See also RELIABILITY, PROBABILISTIC; STOCHASTIC PROCESSES; and WEIBULL DISTRIBUTION.

MAX EENGELHARDT REFERENCES 1. Ascher, H. (1979). IEEE Trans. Rel., R-28, 119. 2. Bain, L. J. and Engelhardt, M. E. Technometrics, 22, 421–426.

(1980).

3. Bain, L. J. and Engelhardt, M. E. (1982). IEEE Trans. Rel., R-31, 79. 4. Crow, L. H. (1974). In Reliability and Biometry, Statistical Analysis of Life Lengths, F. Proschan and R. J. Serfling, eds., SIAM, Philadelphia, pp. 379–410. 5. Crow, L. H. (1982). Technometrics, 24, 67– 72. 6. Duane, J. T. (1964). IEEE Trans. Aerospace, 2, 563–566. 7. Engelhardt, M. and Bain, L. J. (1978). Technometrics, 20, 167–169. 8. Finkelstein, J. M. (1976). Technometrics, 18, 115–117. 9. Finkelstein, J. M. (1976). IEEE Trans. Rel., R-28, 111–113. 10. Lee, L. (1980). Technometrics, 22, 195–200.

WEIERSTRASS APPROXIMATION THEOREM If f (x) is a continuous real-valued function of x for 0 x 1, then, asn → ∞, Pn (x) =

n j n j x (1 − x)n−j → f (x) f j n j=0

uniformly over 0 x 1. The polynomials Pn (x) are Bernstein polynomials∗ . See also INTERPOLATION and MATHEMATICAL FUNCTIONS, APPROXIMATIONS TO.

WEIGHING DESIGNS INTRODUCTION

11. Lee, L. (1980). Technometrics, 22, 427–430. 12. Lee, L. and Lee, S. K. (1978). Technometrics, 20, 41–45.

BIBLIOGRAPHY Ascher, H. and Feingold, H. (1984). Repairable Systems Reliability. Dekker, New York. (There is a good discussion of the confusion that exists between the Weibull process and the Weibull distribution.) Bain, L. J. (1978). Statistical Analysis of Reliability and Life-Testing Models. Dekker, New York. (There is a good summary of statistical methods for the Weibull process.) Kempthorne, O. and Folks, L. (1971). Probability, Statistics, and Data Analysis. The Iowa State University Press, Ames, IA. (There is a derivation that the first failure time of a Weibull process is Weibull distributed.)

The problem of finding the suitable combinations in which several light objects have to be weighed in a scale to determine their individual weights with maximum possible precision is known as the weighing problem, and the suitable combinations as the weighing designs. The weighing problem has its origin in a casual illustration furnished by Yates [48]. This illustration subsequently attracted the notice of Hotelling [20] who recast Yates’ example in an alternative setting, viewing Yates’ illustration as a problem in weighing. Thus came into being Hotelling’s (or the Yates-Hotelling) weighing problem. Besides being useful in routine weighing operations to determine the weights of light objects, the results of research in this area may be useful in chemical, physical, biological, economic, and other sciences. Weighing

WEIGHING DESIGNS

designs should have applications in any problem of measurements, where the measure of a combination is expressible as a linear combination of the separate measures with numerically equal coefficients. Dr. Sloane (see [6]) once wrote to the present author as follows: ‘‘You might be interested in the enclosed, which gives an application of weighing designs (only we didn’t call them by that name!) in measuring frequency spectra.’’ The following lines from Youden [49, p. 118] may also be of interest in this context: ‘‘I confess, with some embarrassment that for the last three investigations over the past years at the NBS (National Bureau of Standards∗ ), I suggested individual and special programs and overlooked the general character and wide applicability of these weighing designs.’’ Viewing the potential of the subject against the above perspective, a precise formulation of the problem and some of its developments are provided in the following pages in the form of a brief outline.

STATISTICAL FORMULATION OF THE WEIGHING PROBLEM The weighing problem may be formulated in the least-squares∗ setup as follows: The results of N weighing operations to determine the individual weights of p light objects fit into the general linear hypothesis model Y = Xβ + , where Y is an N × 1 random observed vector of the records of weighings, X = (xij ), i = 1, 2, . . . , N; J = 1, 2, . . . , p, is an N × p matrix of known quantities, with xij = +1, −1, or 0, if in the ith weighing operation the jth object is placed, respectively, in the left pan, in the right pan, or in none: β is a p × 1 vector (p N) representing the weights of the objects; is an N × 1 unobserved random vector of errors such that E[] = 0, and E[ ] = σ 2 I. Consistent with the signs that the elements xij can take, yi , i = 1, 2, . . . , N, is taken as positive or negative, depending on whether the balancing weight is placed in the right pan or the left. As is well known, the least-squares estimates of the weights will be given by βˆ =

9103

(X X)−1 X Y, where X is the transpose of ˆ = the full rank design matrix X, with cov(β) σ 2 (X X)−1 = σ 2 (cij ).cii is the variance factor of the ith estimated weight. When X is a square matrix of full rank, βˆ = (X X)−1 X Y = X−1 Y. If, in particular, X is orthogonal, βˆ = X−1 Y = (X Y)/N. Also, if we solve for β in Y = Xβ, when X is square and of full rank, β = X−1 Y. Two types of problems would arise in practice, one with reference to the spring balance (one-pan balance) and the other with reference to the chemical balance (two-pan balance). In the spring balance, the elements xij are restricted to the values of +1 or 0, whereas in the chemical balance, these elements are either +1, −1, or 0. EFFICIENCY OF WEIGHING DESIGNS In weighing designs, we search for the elements xij in such a way that each cii be the minimum possible. But, since all the estimates might not have equal variance, a weighing design would be called the most efficient, if (i) the average variance were the minimum (A-optimal), or (ii) the det|cij | were the minimum (equivalently, det|X X| were the maximum) (D-optimal), or (iii) the minimum eigenvalue of X X were the maximum (E-optimal). In some cases, any two of these criteria may lead to equivalent results. (For details, see [6].)

DEVELOPMENTS OF THE SUBJECT In response to Hotelling’s call for further mathematical research, the following authors came out with their contributions: Kishen [25], Mood [29], Banerjee (at least 19 papers, summarized in [6]), Rao [39], Kempthorne [24], Raghavarao (at least five papers, summarized in [36]), Zacks [50], Beckman (at least two papers, summarized in [6]), Sihota and Banerjee [40], Hazra and Banerjee [19], Hazra (see [6]), Dey [14,15], Kulshrestha and Dey [27], Lese and Banerjee [28], Moriguti [30], Sloane and others (several papers, two referred to in [6]), Bhaskararao [10], Youden [49], Swamy [44–46], Bose and Cameron [11], Chakravarti and Suryanarayana [12], Cheng [13], Federer et al. [16], Galil and Kiefer (at

9104

WEIGHING DESIGNS

least 2 papers [17,18]), Jacroux and Wong [22], Kounias and Farmakis [26], Mukherjee and Saha Ray [32], Mukherjee and Huda [31], Raghavarao and Federer [38], Suryanarayana [43], Sinha [41], and others. The work of the above authors includes the construction of efficient spring balance and chemical balance weighing designs of different dimensions, the treatment of designs under autocorrelation of measurement errors∗ , determination of total weight, orthogonal designs∗ , singular weighing designs, fractional weighing designs (randomized and nonrandomized), repeated spring balance designs, biased estimation in weighing designs, etc. The papers written prior to 1975 are mostly summarized in [6]. EFFICIENT WEIGHING DESIGNS Hadamard matrices provide the most efficient chemical balance designs. (For construction of Hadamard matrices∗ , see Plackett and Burman [33] and Raghavarao [36].) When, for the number of objects to be weighed, a suitable Hadamard matrix does not exist, we would look for other efficient design matrices of appropriate dimensions. Some research has also been carried out to cover such situations (see the references cited in [6] and also the section entitled Present Directions of Research). Banerjee [1] has pointed out that the arrangements of a balanced incomplete block design∗ (BIBD) provide efficient (and in some cases, most efficient) spring balance designs. (Incidentally, a BIBD is an arrangement of v distinct objects into b blocks such that each block contains exactly k distinct objects, each object occurs in exactly r different blocks, and every pair of distinct objects occurs together in exactly λ blocks. In weighing designs, v takes the place of p, the number of objects to be weighed, and b that of the number of weighing operations.) A BIBD does not, in general, provide the estimates as orthogonal linear functions of the observations. But, when such a BIBD is used in a slightly adjusted form, the estimates turn out to be mutually orthogonal (see [2]). Incidentally, Yates’ original scheme provides the most efficient spring balance design for the problem

under consideration with mutually orthogonal estimates. There exists only one more spring balance design of the same dimension with the same maximum possible efficiency as pointed out by Banerjee [5]. SINGULAR WEIGHING DESIGNS Despite the best of intentions, a design matrix X may not be of full rank, resulting in what may be called a singular weighing design. It is well known in least-squares theory that in such a situation, it may not be possible to provide unique unbiased estimates of the individual weights. But, it may be possible to provide unique unbiased estimates of certain linear functions (perhaps, total weight) of the weights. What linear functions will be estimable will depend upon the structure of the design matrix X (see ESTIMABILITY). When a singular weighing design is encountered, one way to make the design work may be to augment the design matrix suitably by the inclusion of additional weighing operations (i.e., by the addition of rows to the design matrix) to make up for the deficiency in rank. Raghavarao [35] gave a start in this direction. Subsequently, Banerjee [3,7] and later, Hazra and Banerjee [19] generalized the procedure of augmentation in matrices, bringing, in the sequel, some additional results. FRACTIONAL WEIGHING DESIGNS A singular weighing design might also result when, instead of the full design matrix X, fewer rows of X (a fraction of the design) were used for reasons of economy, time, or other considerations. Zacks [50] visualized the possible use of such fractional weighing designs under randomized procedures, which would provide an unbiased estimate of any linear function of the weights, i.e., of any λ β with the minimum possible variance. The probability vector of the randomization procedure would, of course, depend upon the linear functional λ (For details, see Zacks [50].) Some analogous results were later developed by Banerjee [4] with reference to such fractional weighing designs under nonrandomized procedures.

WEIGHING DESIGNS

CONNECTION WITH FACTORIAL EXPERIMENTS∗ Kempthorne [24] discussed the weighing problem from the viewpoint of factorial experiments and indicated how fractional replicates of factorial experiments could provide the most efficient weighing designs. Connection of weighing designs with factorial experiments could also be traced by way of Hadamard matrices∗ , which, on one hand, provide the most efficient chemical balance designs, and, on the other hand, saturate orthogonal main effect plans. The relationship of weighing designs with factorial experiments∗ could also be traced on a more explicit basis (see Raghavarao [37]). SPECIFIC APPLICATIONS Bose and Cameron [1] introduced balanced weighing designs in the context of tournaments and testing experiments. Later, Suryanarayana [43] and Chakravarti and Suryanarayana [12] generalized these ideas to partially balanced weighing designs. Raghavarao and Federer [38] have found an application of spring balance weighing designs to elicit information on sensitive questions in sample surveys. Also, spring balance weighing designs have been used by Federer et al., [16] to determine the proportion of legume, weed, and grass contents of hay in crop experiments. Sloane et al. ([42], and other papers) have applied the principles of weighing designs to problems in optics, with a continued research interest in this direction. PRESENT DIRECTIONS OF RESEARCH Sinha and Saha [41] have studied the problem of constructing optimal weighing designs with a string property (a design is said to have a string property, if the design matrix has a row of ones). Swamy [45,46] has constructed some efficient chemical balance weighing designs, given a specific number of weighing operations, as alternatives to repeated designs suggested by Dey [14,15]. She has also studied [44] the problem of estimating the total weight, using both spring balance and

9105

chemical balance weighing designs under certain restrictions. Mukherjee and Saha Ray [32] and Mukherjee and Huda [31] have developed some asymptotically D-optimal designs under certain restrictions. Construction of optimum weighing designs in different situations, especially when n ≡ 2 (mod 4) or n ≡ 3 (mod 4), is still being actively pursued, as is reflected in the published papers of Galil and Kiefer [17,18], Jacroux and Wong [22], Kounias and Farmakis [26], Cheng [13], and possibly others. REFERENCES 1. Banerjee, K. S. (1948). Ann. Math. Statist., 19, 394–399. (Weighing designs and balanced incomplete blocks.) 2. Banerjee, K. S. (1950). Biometrika, 37, 50–58. (How balanced incomplete block designs may be made to furnish orthogonal estimates in weighing designs.) 3. Banerjee, K. S. (1966). Ann. Math. Statist., 37, 1021–1032. (Singularity in Hotelling’s weighing design and a generalized inverse.) (A correction note appears in Ann. Math. Statist., 40, 719.) 4. Banerjee, K. S. (1966). Ann. Math. Statist., 37, 1836–1841. 5. Banerjee, K. S. (1974). Commun. Statist., 3, 185–190. 6. Banerjee, K. S. (1975). Weighing Designs. Dekker, New York. (This monograph includes a summary of Banerjee’s 19 papers as well as a list of other authors’ contributions.) 7. Banerjee, K. S. (1972). J. Amer. Statist. Ass., 67, 211–212. (Singular weighing designs and a reflexive generalized inverse.) 8. Beckman, R. J. (1969). Randomized Spring Balance Weighing Designs. Ph.D. dissertation, Kansas State University, Manhattan, KS. 9. Beckman, R. J. (1972). Commun. Statist., 1, 561–565. 10. Bhaskar Rao, M. (1966). Ann. Math. Statist., 37, 1371–1381. (Weighing designs when n is odd.) 11. Bose, R. C. and Cameron, J. M. (1968). J. Res. Natl. Bur. Stand. B69, 323–332. 12. Chakravarti, I. M. and Suryanarayana, K. V. (1972). J. Comb. Theory, A13, 426–431. 13. Cheng, C. S. (1980). Ann. Statist, 8, 436–446. (Optimality of some weighing and 2n fractional factorial designs.)

9106

WEIGHT BIAS

14. Dey, A. (1969). Ann. Inst. Statist. Math., Tokyo, 21, 343–346. 15. Dey, A. (1972). J. Indian Soc. Agric. Statist., 24, 119–126. 16. Federer, W. T., Hedayat, A., Lowe, C. C., and Raghavarao, D. (1976). Agron. J., 68, 914–918. 17. Galil, Z. and Kiefer, J. (1980). Ann. Statist. 8, 1293–1306. [D-optimum weighing designs for n ≡ 3 (mod 4).] 18. Galil, Z. and Kiefer, J. (1982). Ann. Statist., 10, 502–510. 19. Hazra, P. K. and Banerjee, K. S. J. Amer. Statist. Ass., 68, 392–393.

(1973).

20. Hotelling, H. (1964). Ann. Math. Statist., 15, 297–306. (Some improvements in weighing and other experimental techniques.) 21. Jacroux, M. and Notz, W. (1983). Ann. Statist., 11, 970–978. 22. Jacroux, M. and Wong, C. S. (1983). J. Statist. Plann. Inf., 8, 231–240. (On the optimality of chemical balance weighing designs.) 23. Kageyama, S. and Saha, G. M. (1983). Ann. Inst. Statist. Math., Tokyo, 35, 447–452. 24. Kempthorne, O. (1949). Ann. Math. Statist., 19, 238–248. 25. Kishen, K. (1945). Ann. Math. Statist., 16, 294–300. 26. Kounias, S. and Farmakis, N. (1984). J. Statist. Plann. Inf., 10, 177–187. [A construction of D-optimal weighing designs when n ≡ 3 (mod 4).] 27. Kulshrestha, A. C. and Dey, A. (1970). Austral. J. Statist., 12, 166–168. 28. Lese, N. G., Jr. and Banerjee, K. S. (1972). Orthogonal estimates in weighing designs. Proc. 18th Conf. on Designs in Army Research and Testing. (Originally, Lese’s PhD. dissertation, University of Delaware, Newark, DE.) 29. Mood, A. M. (1946). Ann. Math. Statist., 17, 432–446. (On Hotelling’s weighing problem.) 30. Moriguti, S. (1954). Rep. Statist. Appl. Res. JUSE, 3, 1–24. (Optimality of orthogonal designs.) 31. Mukherjee, R. and Huda, S. (1985). Commun. Statist. A, 14, 669–677. (D-optimal statistical designs with restricted string property.)

34. Raghavarao, D. (1959). Ann. Math. Statist., 30, 295–303. (Some optimum weighing designs.) 35. Raghavarao, D. (1964). Ann. Math. Statist., 35, 673–680. (Singular weighing designs.) 36. Raghavarao, D. (1971). Constructions of Combinatorial Problems in Design of Experiments, Wiley, New York. 37. Raghavarao, D. (1975). Gujarat Statist. Rev., 2, 1–16. (Review article.) 38. Raghavarao, D. and Federer, W. T. (1979). J. R. Statist. Soc. B, 41, 40–45. (Block total response as an alternative to the randomized response method in surveys.) 39. Rao, C. R. (1946). Sankhya, 7, 440. 40. Sihota, S. S., and Banerjee, K. S. (1974). Sankhya B, 36, 55–64. (Biased estimation in weighing designs.) 41. Sinha, B. K. and Saha, R. (1983). J. Statist. Plann. Inf., 8, 365–374. 42. Sloane, N. J. A., Fine, T., Phillips, P. G., and Harwit, M. (1969). Appl. Opt., 8, 2103–2106. (Codes for multiplex spectrometry.) 43. Suryanarayana, K. V. (1971). Ann. Math. Statist., 42, 1316–1321. 44. Swamy, M. N. (1980). Commun. Statist. A, 9, 1185–1190. (Optimum spring balance weighing design for estimating the total weight.) 45. Swamy, M. N. (1981). J. Indian Agric. Statist., 33, 23–28. 46. Swamy, M. N. (1981). J. Indian Statist. Ass., 19, 177–181. 47. Swamy, M. N. (1982). Commun. Statist. A, 11, 769–785. 48. Yates, F. (1935). J. R. Statist. Soc. Suppl., 3, 181–247. (Complex experiments.) 49. Youden, N. J. (1962). Technometrics, 4, 111–123. (Systematic errors in physical constants.) 50. Zacks, S. (1966). Ann. Math. Statist., 37, 1382–1395. (Randomized fractional weighing designs.) See also DESIGN OF EXPERIMENTS; FACTORIAL EXPERIMENTS; FRACTIONAL FACTORIAL DESIGNS; and OPTIMAL DESIGN OF EXPERIMENTS.

K. S. BANERJEE

WEIGHT BIAS

32. Mukherjee, R. and Saha Ray, R. (1983). Tech. Rep. No. 19/83; Indian Statistical Institute, Calcutta, India.

A bias attributable to the weighting used in calculating an index number.

33. Plackett, R. L. and Burman, J. P. Biometrika, 33, 305–325.

See also INDEX NUMBERS.

(1946).

WEIGHTED DISTRIBUTIONS

9107

of X ∗ is called the size-biased distribution of PDF

WEIGHTED DISTRIBUTIONS The concept of weighted distributions can be traced to the study of the effect of methods of ascertainment upon estimation of frequencies by Fisher [7]. The initial idea of length-biased sampling appears in Cox [4]. In extending the basic ideas of Fisher, Rao [26] saw the need for a unifying concept and identified various sampling situations that can be modeled by what he called weighted distributions. Within the biomedical context of cell kinetics and the early detection of disease, Zelen [36] introduced weighted distributions to represent what he broadly perceived as length-biased sampling∗ . Patil [16] has discovered weighted distributions as stochastic models in the equilibrium study of populations subject to harvesting and predation. See the bibliography by Patil et al. [21] for a comprehensive survey of the literature.

f ∗ (x; θ ) =

xf (x; θ ) , µ

(2)

where µ = E[X]. The PDF f ∗ is called the length-biased or size-biased version of f , and the corresponding observational mechanism is called length- or size-biased sampling. The concept of weighted distributions has been used during the last 25 years as a useful tool in the selection of appropriate models for observed data, especially when samples are drawn without a sampling frame that enables random samples to be drawn. In many situations the model given in (1) is appropriate, and the statistical problems that arise are the determination of a suitable weight function w(x, β) and drawing inference on θ . The following examples from Patil and Rao [18] illustrate a few situations generating weighted distributions.

UNIVARIATE WEIGHTED DISTRIBUTIONS Suppose X is a nonnegative observable random variable (RV) with PDF f (x; θ ), where θ is a parameter. Suppose a realization x of X under f (x; θ ) enters the investigator’s record with probability proportional to w(x, β), so that Pr(Recording|X = x) = w(x, β). Here the recording (weight) function w(x, β) is a nonnegative function with parameter β representing the recording (sighting) mechanism. Clearly, the recorded x is not an observation on X, but on the RV X w , say, having PDF f w (x; θ , β) =

w(x, β)f (x; θ ) , ω

(1)

where ω = E[w(X, β)] is the normalizing factor, making the total probability equal to unity. The RV X w is called the weighted version of X, and its distribution in relation to that of X is called the weighted distribution with weight function w. An important weighted distribution corresponds to w(x, β) = x, in which case, X ∗ = X w is called the size-biased version of X. The distribution

(i) Truncation. The distribution of a random variable truncated to a set T is a weighted distribution with weight function w(x) = 1 for x ∈ T and 0 elsewhere. (ii) Missing Data. If the sampling mechanism results in a proportion 1 − w(x), 0 w(x) 1, of the observations having the value x being omitted, the PDF to use for the analysis of the observed data is with the weight function w(x). (iii) Damaged Observations. Consider a damage model∗ , where an observation X = x is reduced to y by a destructive process with PDF d(y|x). See Rao [26]. Then the probability that the observation X = x is undamaged is d(x|x), and the distribution of the undamaged observation is the weighted distribution with w(x) = d(x|x). For example, under the binomial survival model, d(x|x) = θ x , 0 < θ < 1. An investigator recording only undamaged observations will need to work with a corresponding weighted distribution. (iv) Analysis of Family Data. This is an example of size-biased sampling. The discussion is based on Rao [26]. Consider the data in Table 1, which relates to brothers and sisters in families of 104 boys

9108


Table 1. Family Data Family size 1 Number of families 1 Brothers 1 Sisters 0

2 6 8 4

3 6 12 6

4 13 34 18

5 12 34 26

6 7 29 13

who were admitted to a post-graduate course at the Indian Statistical Institute. Let us assume that in families of given size n, the probability of a family with x boys coming into the record is proportional to x. Also, suppose that the number of boys in a family follows a binomial distribution with probability parameter π . Then n f (x; π ) = π x (1 − π )n−x , x w(x) = x, E[w(X)] = ω = nπ , n − 1 π x−1 (1 − π )n−x , f w (x; π ) = x−1 E[X w /n] = π + (1 − π )/n > π , E[(X w − 1)/(n − 1)] = π. If k boys representing families of size n1 , n2 , . . ., nk report x1 , x2 , . . ., xk boys, an unbiased estimate of π is ni − k) π˜ = ( xi − k)/( = (414 − 104)/(726 − 104) 1/2, whereas if one wrongly treats xi ’s as observations on k randomly drawn families with at least one boy, i.e., as arising from a truncated binomial, then the estimate of π will have a serious upward bias. WEIGHT FUNCTIONS AND PROPERTIES The following forms of weight functions w(x) have appeared in the scientific and statistical literature: 1. w(x) = xβ for β = 1, 2, 3, 1/2, 2/3, 0 < β < 1. β 2. w(x) = 2x for β = 1, 1/2, 0 < β < 1. 3. 4. 5. 6. 7. 8.

w(x) = (x)(β) = x(x − 1) · · · (x − β + 1). w(x) = eβx , θ x . w(x) = x + 1, αx + β. w(x) = 1 − (1 − β)x for 0 < β < 1. w(x) = (αx + β)/(δx + γ ). w(x) = Pr(Y x) for some RV Y.

7 14 59 39

8 11 50 38

9 12 54 54

10 8 46 34

11 6 32 34

12 5 31 29

13 2 16 10

15 1 8 7

Total 104 414 312

9. w(x) = Pr(Y > x) for some RV Y. 10. w(x) = r(x), where r(x) is the probability of ‘‘survival’’ of observation x. An important case arises where T is a subset of the real line and r(x) is defined by r(x) = 1 if x ∈ T and r(x) = 0 if x ∈ / T. The resulting weighted distribution is said to be a truncated version of the original distribution. If, for an original discrete distribution for example, r(0) = 0 and r(x) = 1 for x = 0, the resulting distribution is called a zerotruncated distribution. The weight functions 1–10 are monotone increasing or decreasing functions of x. The following results provide useful comparisons of X w with X. Result 3.1 (Patil et al. [20]). The weighted version X w is stochastically greater or smaller than the original RV X according as the weight function w(x) is monotone increasing or decreasing in x. As a consequence, the expected value of the weighted version X w is greater or smaller than the expected value of the original RV X according as the weight function w(x) is monotone increasing or decreasing in x. Result 3.2 (Zelen [36] and Patil and Rao [19]). The expected value of the sizebiased version X ∗ is E[X ∗ ] = µ[1 + σ 2 /µ2 ], where E[X] = µ and V(X) = σ 2 . Furthermore, the harmonic mean of X ∗ is equal to the mean of the original RV X when it is positive, i.e., E[1/X ∗ ] = 1/µ. Another way of expressing these results is that E[X ∗ ]E[1/X ∗ ] = 1 + σ 2 /µ2 . Result 3.3 (Patil and Ord [17]). Let the RV X have PDF f (x; θ ) and have size bias with weight function xβ . Then a necessary and sufficient condition for f w (x; θ , β) = f (x; η), where η = η(θ , β), is that f (x; θ ) = xθ a(x)/m(θ ). In this case, f w (x; θ , β) = f (x; θ + β). This result holds under certain mild regularity conditions.


Result 3.4 (Mahfoud and Patil [13]). Consider X w to be a RV X subject to a size bias with the weight function xβ . Then X has a lognormal distribution∗ if and only if V(log X) ≡ V(log X w ) for β > 0, where V stands for variance. Thus the invariance of the logarithmic variance under size bias of order β characterizes the log-normal distribution. Example. In sedimentology and various other fields, the distribution of particles is usually analyzed by mass rather than frequency. Sieve analysis is a good example, which provides data consisting of sizes of sieves and corresponding masses of all particles retained by those sieves. It is interesting to note that the mass–size density is nothing but the weighted version of size bias of order 3 of the PDF of the particle size. It can be verified that, if X is lognormal with parameters µ and σ 2 , then X w with β = 3 is lognormal with parameters µ + 3σ 2 and σ 2 . This property has been empirically noticed and utilized for inference in sedimentology literature. See, for example, Krumbein and Pettijohn [12] and Herdan [11]. BIVARIATE WEIGHTED DISTRIBUTIONS Let (X, Y) be a pair of nonnegative RVs with a joint PDF f (x, y) and let w(x, y) be a nonnegative weight function such that E[w(X, Y)] exists. The weighted version of f (x, y) is f w (x, y) =

w(x, y)f (x, y) . E[w(X, Y)]

The corresponding weighted version of (X, Y) is denoted by (X, Y)w . The marginal and conditional distributions of (X, Y)w are f w (x) = f w (y|x) =

E[w(x, Y)|x]f (x) , E[w(X, Y)] w(x, y)f (y|x) . E[w(x, Y)|x]

Clearly, both are weighted versions of the corresponding marginal and conditional distributions of (X, Y). Special cases of weight functions of practical interest are 1. w(x, y) = xα ;

2. 3. 4. 5. 6.

9109

w(x, y) = w(y); w(x, y) = x + y; w(x, y) = xα yβ ; w(x, y) = max(x, y); and w(x, y) = min(x, y).

The following results are of some interest. Result 4.1 (Patil and Rao [19]). Let (X, Y) be a pair of nonnegative RVs with PDF f (x, y). Let w(x, y) = w(y), as is the case in sample surveys involving sampling with probability proportional to size∗ . Then the random variables X and X w are related by f w (x) =

E[w(Y)|x]f (x) . E[w(Y)]

Note that X w is a weighted version of X, and the regression of w(Y) on X serves as the WF. Result 4.2 (Mahfoud and Patil [14]). Let (X, Y) be a pair of nonnegative independent RVs with joint PDF f (x, y) = fX (x)fY (y) and let w(x, y) = max(x, y). Then the RVs (X, Y)w are dependent. Furthermore, the regression of Y w on X w by E[Y w |X w = x] is a decreasing function of x. Example. Let (X, Y) be statistically independent lifetimes of two components in parallel forming a kit with lifetime w = max(x, y). Consider a renewal system of kits. If at time t, one records the lifetimes of the two components of the kit in action, their joint PDF is the weighted version of their natural PDF with weight function w = max(x, y). The result shows that whereas there is underlying independence in the true lifetimes of the two components/organs, the data so obtained will not reveal that; actually the data will show negative dependence.

POSTERIOR AND WEIGHTED DISTRIBUTIONS There is a Bayesian analog to the theory of weighted distributions. (See Mahfoud and Patil [14] and Patil et al. [20].) Result 5.1 (Mahfoud and Patil, [13]). Consider the usual Bayesian inference∗ in conjunction with (X, θ ) having joint PDF f (x, θ ) = f (x|θ )f (θ ) = f (θ |x)f (x). The

9110


posterior PDF f (θ |x) = f (x|θ )f (θ )/f (x) = l(θ |x)f (θ )/E[l(θ |X)] is a weighted version of the prior∗ PDF f (θ ). The weight function is the likelihood∗ function of θ for the observed x. Result 5.2 (Patil et al. [20]). Consider the usual Bayesian inference in conjunction with (X, θ ) with PDF f (x, θ ) = f (x|θ )f (θ ) = f (θ |x)f (x). Let w(x, θ ) = w(x) be the weight function for the distribution of X|θ , so that the PDF of X w |θ is

population size x, but independent of time t, and γ (t) is the environmental part of the per capita growth rate dependent on time t, but independent of the population size x. Let γ (t) be a white-noise process with environmental unpredictability parameter σ 2 . The population size x(t) is then a stochastic integral, and when it exists, its equilibrium PDF const M(x) f (x) = exp 2 dx V(x) V(x) = exp[a log x + b(x) + c],

w(x)f (x|θ )/ω(θ ), where ω(θ ) = E[w(X)|θ ]. Then the original and the weighted posteriors are related by f (θ |x) =

ω(θ )f w (θ |x) . E[ω(θ )|X w = x]

say,

is a member of the log-exponential family, where M(x) = xg(x) and V(x) = σ 2 x2 . Furthermore, if the population is subjected to exploitation (harvesting, predation, etc.) with per capita exploitation rate h(x), the equilibrium PDF fh (x) of the exploited population size interestingly simplifies to fh (x) =

w(x)f (x) , E[w(X)]

Furthermore, the weighted posterior∗ RV (θ w |X w = x) is stochastically greater or smaller than the original posterior RV θ |X = x according as ω(θ ) is a monotonically decreasing or increasing function of θ .

where f (x) is the natural population equilibrium PDF and xh(x) dx . w(x) = exp −2 σ 2 x2

Examples. Table 2 provides a convenient format.

For further discussion, see Patil [16] and Dennis and Patil [5].

WEIGHTED DISTRIBUTIONS IN STOCHASTIC POPULATION DYNAMICS

APPLICATIONS OF WEIGHTED DISTRIBUTIONS

Consider the stochastic differential equation∗

A vast number of situations arise in which weighted distributions find their application. For lack of space, we indicate rather than discuss some of these applications.

1 dx = r(x, t) x dt = g(x) + γ (t), where x(t) is the population size at time t, r(x, t) is the per capita growth rate of population of size x at time t, g(x) is the biological part of the per capita growth rate dependent on

1. Cell cycle analysis and pulse labeling, Zelen [36]. 2. Efficacy of early screening for disease and scheduling of examinations, Zelen [35, 36].

Table 2. Posterior and Weighted Distributions X| = θ Poisson(θ) Binomial(n, θ) Neg-Bin(k, θ) Exponential(θ)

|X = x

w(x)

Gamma(k, λ) Beta(a, b) Beta(a, b) Gamma(k, λ)

Gamma(k + x, 1/(λ + 1)) Beta(x + a, n − x + b) Beta(k + a, x + b) Gamma(k + 1, λ/(λx + 1))

x x x x

|X w = x Gamma(k + x − 1, 1/(λ + 1)) Beta(x − 1 + a, n − x + b) Beta(k + 1 + a, x − a + b) Gamma(k + 2, λ/(λx + 1))


3. Cardiac transplantation, Temkin [33]. 4. Estimation of antigen frequencies, Simon [29]. 5. Ascertainment studies in genetics, Rao [26, 27] and Stene [30]. 6. Renewal theory and reliability, Cox [4] and Zelen [36]. 7. Nonrenewable natural resource exploration, Barouch et al. [1]. 8. Traffic research, Brown [2]. 9. Word association analysis, Haight and Jones [9]. 10. Marketing and resource utilization, Morrison [15]. 11. Analysis of spatial pattern, Pielou [23]. 12. Species abundance and diversity, Engen [6]. 13. Transect sampling, Cook and Martin [3], Patil and Rao [19], and Quinn [25]. 14. Forest products research, Warren [34]. 15. Income inequality and species inequitability, Hart [10] and Taillie [31]. 16. Canonical hypothesis in ecology, Preston [24] and Patil and Taillie [22]. 17. Particle-size statistics, Gy [8] and Krumbein and Pettijohn [12]. 18. Mass–size distributions, Herdan [11] and Schultz [28]. 19. Quality of Swiss cheese, Tallis [32]. REFERENCES 1. Barouch, E., Chow, S., Kaufman, G. M., and Wright, T. H. (1985). Stud. Appl. Math., 73, 239–260. 2. Brown, M. (1972). Adv. Appl. Prob., 4, 177–192. 3. Cook, R. D. and Martin, F. B. (1974). J. Amer. Statist. Ass., 69, 345–349. 4. Cox, D. R. (1962). Renewal Theory. Methuen, New York. 5. Dennis, B. and Patil, G. P. (1984). Math. Biosci., 68, 187–212. 6. Engen, S. (1978). Stochastic Abundance Models. Chapman and Hall, London. 7. Fisher, R. A. (1984). Ann. Eugen., 6, 13–25. 8. Gy, P. M. (1982). Sampling of Particulate Materials. Elsevier, New York. 9. Haight, F. A. and Jones, R. B. (1974). J. Math. Psych., 11, 237–244.

9111

10. Hart, P. E. (1975). J. R. Statist. Soc. A, 138, 423–434. 11. Herdan, G. (1960). Small Particle Statistics. Elsevier, New York. 12. Krumbein, W. C. and Pettijohn, F. J. (1938). Manual of Sedimentory Petrography. Appleton-Century-Croft, New York. 13. Mahfoud, M. and Patil, G. P. (1981). In Statistics in Theory and Practice: Essays in Honor of Bertil Matern, B. Ranneby, ed. Swedish Uni˚ Sweden, versity Agricultural Science, Umea, pp. 173–187. 14. Mahfoud, M. and Patil, G. P. (1982). In Statistics and Probability: Essays in Honor of C. R. Rao, G. Kallianpur et al., eds. NorthHolland, Amsterdam, pp. 479–492. 15. Morrison, D. G. (1973). Amer. Statist., 27, 226–227. 16. Patil, G. P. (1984). In Proceedings of the Indian Statistical Institute Golden Jubilee International Conference on Statistics: Applications and New Directions, J. R. Ghosh and J. Roy, eds. Statistical Publishing Society, Calcutta, India, pp. 478–503. 17. Patil, G. P. and Ord, J. K. (1975). Sankhya, ¯ 38, 48–61. 18. Patil, G. P. and Rao, C. R. (1977). Weighted distributions and a survey of their applications. In Applications of Statistics, P. R. Krishnaiah, ed. North-Holland, Amsterdam, pp. 383–405. 19. Patil, G. P. and Rao, C. R. (1978). Biometrics, 34, 179–184. 20. Patil, G. P., Rao, C. R., and Ratnaparkhi, M. V. (1986). Comm. Statist.-Theor. Meth., 15, 907–918. 21. Patil, G. P., Rao, C. R., and Zelen, M. (1986). A computerized bibliography of weighted distributions and related weighted methods for statistical analysis and interpretations of encountered data, observational studies, representativeness issues, and resulting inferences. Center for Statistical Ecology and Environmental Statistics, Pennsylvania State University, University Park, PA. 22. Patil, G. P. and Taillie, C. (1979). Bull. Int. Statist. Inst., 44, 1–23. 23. Pielou, E. C. (1977). Mathematical Ecology. Wiley, New York. 24. Preston, F. W. (1962). 185–215–410–432.

Ecol.,

43,

25. Quinn, T. J. (1979). In Contemporary Quantitative Ecology and Related Econometrics, G. P. Patil and M. Rosenzweig, eds. International Co-operative, Fairland, MD, pp. 473–491.

9112

WEIGHTED EMPIRICAL PROCESSES, GENESIS AND APPLICATIONS OF

26. Rao, C. R. (1965). In Classical and Contagious Discrete Distributions, G. P. Patil, ed. Statistical Publishing Society, Calcutta, India and Pergamon Press, New York, pp. 320–332. 27. Rao, C. R. (1985). In Celebration of Statistics: The ISI Centenary Volume, A. C. Atkinson and S. E. Fienberg, eds., International Statistical Institute, The Hague, Netherlands, pp. 543–569. 28. Schultz, D. M. (1975). In Statistical Distributions in Scientific Work, G. P. Patil, S. Kotz, and J. K. Ord, eds. Reidel, Dordrecht, Netherlands and Boston, MA, pp. 275–288. 29. Simon, R. (1980). Amer. J. Epidemiology, 111, 444–452. 30. Stene, J. (1981). In Statistical Distributions in Scientific Work, .6, C. Taillie, G. P. Patil, and B. Baldessari, eds. Reidel, Dordrecht, Netherlands and Boston, MA, pp. 233–264. 31. Taillie, C. (1979). In Ecological Diversity in Theory and Practice, J. F. Grassle et al., eds. International Co-operative, Fairland, MD, pp. 51–62. 32. Tallis, G. M. (1970). Biometrics, 26, 87–104. 33. Temkin, N. (1976). Interactive Information and Distributional Length Biased Survival Models. Ph.D. dissertation, University of New York at Buffalo, University Microfilms, Ann Arbor, Michigan. 34. Warren, W. G. (1975). In Statistical Distributions in Scientific Work, G. P. Patil, S. Kotz, and J. K. Ord, eds. Reidel, Dordrecht, Netherlands and Boston, MA, pp. 369–384. 35. Zelen, M. (1971). Bull. Int. Statist. Inst., Proc., 38, Session I, pp. 649–661. 36. Zelen, M. (1974). In Reliability and Biometry, Statistical Analysis of Life Lengths, F. Proschan and R. J. Serfling, eds. SIAM, Philadelphia, pp. 701–726. See also DAMAGE MODELS and TRUNCATED DATA.

G. P. PATIL C. R. RAO MARVIN ZELEN

WEIGHTED EMPIRICAL PROCESSES, GENESIS AND APPLICATIONS OF Weighted empirical processes (WEPs) arise typically in the context of rank∗ and robust∗ statistics (both tests and estimates) in linear models. One version of the WEP may be regarded as a direct extension of the

classical empirical (distributional) process∗ to the regression∗ setup, a second version corresponds to a stochastic process∗ with regression parameters taking the role of time parameters, whereas a general class of the WEP may be defined in terms of a twodimensional time-parameter process involving the distributional process as well as the regression parameters. We shall introduce these WEPs in this order. For a given set cn1 , . . ., cnn of known (regression) constants and independent random variables X1 , . . ., Xn having distributions F1 , . . ., Fn , respectively (not necessarily all the same), the regression WEP is defined by Wn (x) =

n

c∗ni {I(Xi x) − Fi (x)},

i=1

− ∞ < x < ∞,

(1)

where I(A) stands for the indicator function of the set A and 1/2 

 n  c2nj , for i = 1, . . . , n. c∗ni = cni   j=1

(2) Often, by means of a transformation Xi → Yi = Hn (Xi ), 1 i n, where Hn (x) is nonnegative and monotone (in x) with Hn (−∞) = 0 and Hn (∞) = 1, we may reduce (1) to Wn0 =

n

c∗ni {I(Yni t) − Gni (t)}, t ∈ [0, 1],

i=1

(3) where Gni is the distribution of Yni [i.e., Gni (t) = Fi (Hn−1 (t)), t ∈ [0, 1]], for i = 1, . . . , n. {Wn0 } is termed the reduced weighted empirical process (RWEP). In the particular case of cni = 1, for every i = 1, . . . , n, the RWEP relates to the usual empirical distributional process (in the comparatively more general setup of possibly nonidentically distributed random variables), whereas the classical empirical distributional process corresponds to the case where the Gni are all the same [namely, Gni (t) = t, ∀t ∈ [0, 1], i = 1, . . . , n]. A detailed account of the basic (small- as well as large-sample) properties of these empirical processes is given in EMPIRICAL PROCESSES. Because of some important statistical uses of the WEP and


RWEP, a more application-oriented treatise of them is considered here. Whereas the early adaptation of the classical empirical process can be traced to the study of the asymptotic distribution theory of the one-sample Kolmogorov–Smirnov statistics∗ , the WEP may similarly be identified with the two-sample goodness-of-fit∗ problem. Suppose that X1 , . . . , Xn1 have the common distribution F and Xn1 +1 , . . . , Xn , n = n1 + n2 , have the common distribution G, and we want to test for the equality of F and G. For this problem, the classical twosample Kolmogorov–Smirnov statistic corresponds to (1) with c1 = · · · = cn1 = 1/n1 and cn1 +1 = · · · = cn = −1/n2 . In general, in the one-sample models, in various nonparametric (or robust) statistical inference problems, estimators and/or test statistics are expressed as functionals of the classical empirical processes, and in a similar manner, in the multisample models, these can be expressed in terms of the WEP or RWEPs. This representation greatly simplifies the treatment of the related asymptotic distribution theory, and constitutes by far the most important statistical use of the WER and RWEPs. This important application was considered in a more general regression setup ´ by Hajek [6], although the earlier works of Dwass [4] and Chernoff and Savage [3] (in the particular case of two-sample models) bear close scrutiny. Consider the simple regression model: Xi = βci + ei , where the ci are known constants, β is the (unknown) regression parameter, and the errors ei are indepen´ dent with a common distribution F. Hajek [6] used the WEP in the following manner to test the null hypothesis H0 : β = 0 against β = 0. Corresponding to known regression constants c1 , . . . , cn , let us take cni = ci − n cn , i = 1, . . . , n, where cn = n−1 j=1 cj , and we ∗ define the cni as in (2). Also, let Zn:1 , . . . , Zn:n be the ordered values corresponding to the X1 , . . . , Xn , and define the vector of anti-ranks D = (D1 , . . . , Dn ) by letting Zn:k = XDX for k = 1, . . . , n (ties are neglected with probability 1 by assuming the distribution F to ´ be continuous). Then Hajek’s extensions of the Kolmogorov–Smirnov statistics∗ to the

9113

regression setup are Kn+

= max

 k 

1kn 

j=1

 

c∗nD j

and

k ∗ cnD . Kn = max j 1kn j=1

(4)

If in (1), we let F1 = · · · = Fn = F, then it is easy to show that Kn+ = max{Wn (Zn:k ); k n} = sup{Wn (x) : x ∈ (−∞, ∞)},

(5)

and a similar representation holds for Kn . Thus the asymptotic theory for the WEP in (1) [viz., weak convergence∗ , law of iterated logarithm∗ , etc.] can be incorporated to derive parallel results for the Kolmogorov–Smirnov statistics for regression alternatives. Similar results have also been worked out for the Rényi as well as the Cramér–von Mises statistics∗ , and the WEP plays the basic role ˇ ak ´ ´ [9], in this context (viz., Hajek and Sid Chaps. 5 and 6). The statistics in (4) and (5) have important applications in time-sequential∗ analysis arising typically in clinical trials∗ and other life-testing∗ experimentations, where the observations are gathered sequentially over time. In the context of interim analysis in clinical trials, it is common to monitor the accumulating data set with a view to achieving an early termination on sound statistical grounds. In this progressive censoring∗ setup, at the successive failure points (Zn:k ), one may look at the Wn (Zn:k ), and hence the problem reduces to that of drawing a conclusion based on Kn or Kn+ (or more generally some weighted versions of them). An extension of (4) to the case of a multiple regression model where the ci are p-vectors (so is β), for some p 1, has been considered by Sinha and Sen [37,38]. For the simple as well as multiple regression model, Sinha and Sen [35–38] have incorporated the WEP in a slightly more general setup and considered appropriate time-sequential∗ tests arising typically in the progressive censoring schemes. The WEP (or RWEP) have been very appropriately incorporated in the study of

9114


the asymptotic distribution theory of rankorder statistics (and derived R-estimators) in linear models. As has been mentioned earlier, the two-sample rank statistics may be expressed in terms of the WEP, and taking this lead, Hoeffding [10] suggested a simple way of incorporating the WEP in the study of the asymptotic distribution theory of linear rank statistics in a regression setup; for some further work in this direction, we may refer to Koul and Staudte [26] and Ghosh ´ and Sen [5]. The earlier works of Hajek [7] and Pyke and Shorack [29,30] deserve mention in this context. We shall discuss this in more detail later on. Let us consider next the second type of WEP. In the context of robust (M-) estimation∗ in linear models, the following type of WEP arises. For given cn1 , . . . , cnn and X1 , . . . , Xn , one may consider a suitable score function ψ = {ψ(x), −∞ < x < ∞}, and corresponding to another set {dn1 , . . . , dnn } of real constants, define a process Wn (b) =

n

c∗ni ψ(Xi − bdni ),

i=1

b ∈ B = [−K, K],

for some K ∈ (0, ∞).(6)

For example, taking dni = cni , i = 1, . . . , n, and equating Wn (b) to 0 (in a meaningful way), one gets the usual M-estimator of the regression coefficient. The weak convergence of Wn = {Wn (b); b ∈ B} to some (drifted) Gaussian function provides the key to the study of asymptotic distribution theory of Mestimators∗ of regression parameters. Actually, in (6), one may easily use for cni and dni some p-vectors (so also for b), and this will lead to a p-variate WEP with a pdimensional time parameter b ∈ Bp . Such general WEPs have been studied systematically by Jureˇckova´ and Sen [16–18], where applications to some problems in sequential analysis∗ have also been stressed. Let us consider the third type of WEPs, which are more general than the others referred to earlier and which play a very significant role in the asymptotic distribution theory of rank and other nonparametric statistics. Typically, a linear rank statistic∗ is defined as Ln = Ln (Xn ) = ni=1 c∗ni an (Rni ), where an (1), . . . , an (n) are suitable scores∗ ,

the c∗ni are defined as before, and Rni is the rank of Xi among X1 , . . . , Xn for i = 1, . . . , n. Suppose that in the above definition, we replace Xn by Xn − bdn , where dn = (dn1 , . . . , dnn ) is a vector of known constants and b is real; the resulting ranks are then denoted by Rni (b). For real x and b, we define Sn (x, b) =

n

c∗ni I(Xi x + bdni ),

i=1

−∞ < x < ∞, −∞ < b < ∞.

(7)

Also, the usual empirical distribution based on Xn − bdn is defined as Hn (x, b) = n−1 ni=1 I × (Xi x + bdni ), −∞ < x < ∞, −∞ < b < ∞. If the scores an (k) are generated by a score-generating function φ, such that an (k) = φn (k/n), k = 1, . . ., n, φn (·) → φ(·), then we have Ln (Xn − bdn ) =

∞ −∞

φn (Hn (x, b))dSn (x, b),

for b ∈ (−∞, ∞).

(8)

This representation enables us to use the two-dimensional time-parameter WEP in (7) in the study of asymptotic distribution theory of rank statistics and more general rank processes. In particular, if in (7), we take b = 0, we obtain the stochastic component of Wn (x) in (1), hence Wn (x) in (1) can be used [along with the usual expansion of (8) around the true distributions when b = 0] to express the normalized form of Ln (Xn ) in terms of WEPs, and this has been worked out in detail by Hoeffding [10], Pyke and Shorack [29,30], and others. More generally, we may consider a rank process defined by {Ln (Xn − bdn ) − Ln (Xn ) + bγ n ; b ∈ B = [−K, K]},

(9)

where K is a finite positive number, n depends on the c∗ni and dni , and γ is a functional of the underlying F (and φ). By using (8), we are able to express this rank process in terms of the general WEP in (7), so that (7) provides a very convenient way of studying the asymptotic behavior of such


processes. Actually, on any compact B, (9) converges to a null process, in distribution, as n increases; in the literature, this is known as the Jureˇckova´ [11] linearity of linear rank statistics in the regression parameter. Parallel results on signed WEP and related signed rank statistics (or processes) were obtained by Koul [19–21], van Eeden [39], and Jureˇckova [12], among others. The relations in (7) and (8) also prompted Sen and Ghosh [32] and Ghosh and Sen [5] to incorporate directly the WEP in the study of the almost-sure convergence of rank statistics and to improve the Jureˇckova´ result to an almost-sure convergence result too. To stress further the role of WEP, we also mention its adaptation in rank estimation theory. If the scores an (k) are monotone increasing (in k) and c∗ni and dni are concordant, then Ln (Xn − bdn ) is monotone non-increasing in b, and this fact was tacitly used by Adichie [1] in providing an Restimator∗ of the regression coefficient by equating Ln (Xn − bdn ) to 0. Equation (9) may be used to express the normalized form of this R-estimator in terms of Ln (Xn − βdn ), and deeper results in this direction (based on the WEP) are due to Jureˇckova´ [14,15] and others. In this context (as well as in others), it has been observed that the asymptotic behavior of the WEP for local (contiguous) alternatives provides a basic framework for parallel results on various nonparametric tests and estimates. This approach has been elegantly developed by the Czechoslovakian school under the pioneering guidance of the ´ late professor J. Hajek; a detailed account of some of these developments is given in Sen [31, Chaps. 4–6]; see also the recent book by Shorack and Wellner [34]. The general WEP in (7) has also been extended to the case of vector c∗ni , dni (and b); for some nice use of these WEPs, we may refer to Jureˇckova´ [13] and Sen and Puri [33], among others. In linear models, it is not uncommon to encounter nuisance scale or other parameters. In this context, (9) provides a convenient way of eliminating these nuisance parameters by using their R-estimators and ´ the related aligned rank statistics. Hajek [8] pointed out the rationality of this approach (and the inaccuracy of earlier ones). Koul

9115

and Sen [25] have shown that this alignment principle works out well for the multiparameter linear models when the error distributions are symmetric. The WEPs play a basic role in this study too. Other notable uses of the WEP include the following: (i) Linear combinations of order statistics∗ for the regression model (viz., Bickel [2]), where the ordering of the observations is (essentially) done by assigning weights based on the regression constants. (ii) Minimum distance∗ type estimator of the regression parameter (viz., Koul and DeWet [24], Koul [23], and Millar [27]), where a quadratic functional of the WEP is directly involved in the minimization process. (iii) Goodness of fit∗ in linear models in the presence of nuisance parameters (viz., Koul [22] and Pierce and Kopecky [28]), where estimators of these nuisance parameters are incorporated in the definition of residuals, and WEPs are then constructed for these residuals. These are often called weighted residual empirical processes (WREP). Faced with this genesis background and variety of applications of the WEP, RWEP, and WREP, several workers have studied the asymptotic theory of these processes under increasing degrees of generalizations. Some detailed accounts of this theory are given in Shorack and Wellner [34], where other references are also cited. REFERENCES 1. Adichie, J. N. (1967). Ann. Math. Statist., 38, 894–904. 2. Bickel, P. J. (1973). Ann. Statist., 1, 597–617. 3. Chernoff, H. and Savage, I. R. (1958). Ann. Math. Statist., 29, 972–994. 4. Dwass, M. (1957). Ann. Math. Statist., 28, 424–431. 5. Ghosh, M. and Sen, P. K. (1972). Sankhya¯ A, 34, 33–52.

9116

WEIGHTED LEAST SQUARES

´ 6. Hajek, J. (1965). In Bernoulli–Bayes– Laplace Seminar, Berkeley. University of California Press, Berkeley, CA, pp. 45–60. ´ 7. Hajek, J. (1968). Ann. Math. Statist., 39, 325–346. ´ 8. Hajek, J. (1970). In Nonparametric Techniques in Statistical Inference, M. L. Puri, ed. Cambridge University Press, pp. 1–17. ˇ ak, ´ ´ Z. (1967). Theory of Rank 9. Hajek, J. and Sid Tests. Academic, New York. 10. Hoeffding, W. (1970). In Nonparametric Techniques in Statistical Inference, M. L. Puri, ed. Cambridge University Press, London, England, pp. 18–19. ´ J. (1969). Ann. Math. Statist., 40, 11. Jureˇckova, 1889–1900. ´ J. (1971a). Sankhya¯ A, 33, 1–18. 12. Jureˇckova, ´ J. (1971b). Ann. Math. Statist., 42, 13. Jureˇckova, 1328–1338. ´ J. 14. Jureˇckova, 1046–1060.

(1973).

Ann.

Statist.,

1,

´ 15. Jureˇckova, 664–672.

(1977).

Ann.

Statist.,

5,

J.

´ J. and Sen, P. K. (1981a). Sankhya¯ 16. Jureˇckova, A, 43, 190–210. ´ 17. Jureˇckova, J. and Sen, P. K. J. Statist. Plann. Inf., 5, 253–266.

(1981b).

´ J. and Sen, P. K. (1984). Statist. 18. Jureˇckova, Dec. Suppl., 31–46.

31. Sen, P. K. (1981). Sequential Nonparametrics. Wiley, New York. 32. Sen, P. K. and Ghosh, M. (1971). Ann. Math. Statist., 42, 189–203. 33. Sen, P. K. and Puri, M. L. (1977). Zeit. Wahrscheinlichkeitsth. Verwand. Geb., 39, 175–186. 34. Shorack, G. and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York. 35. Sinha, A. N. and Sen, P. K. (1979a). Commun. Statist. A, 8, 871–898. 36. Sinha, A. N. and Sen, P. K. (1979b). Calcutta Statist. Ass. Bull., 28, 57–82. 37. Sinha, A. N. and Sen, P. K. (1982). Sankhya¯ B, 44, 1–18. 38. Sinha, A. N. and Sen, P. K. (1984). Indian Statist. Inst. Golden Jubilee Vol., 237–253. 39. van Eeden, C. (1972). Ann. Math. Statist., 43, 791–802. ´ –VON See also ANTIRANKS; CLINICAL TRIALS; CRAMER MISES STATISTIC; EMPIRICAL PROCESSES; GOODNESS OF FIT; KOLMOGOROV –SMIRNOV STATISTICS; LAW OF THE ITERATED LOGARITHM; LINEAR RANK TESTS; L-STATISTICS; M-ESTIMATORS; MINIMUM DISTANCE ESTIMATION; PROGRESSIVE CENSORING SCHEMES; RANK STATISTICS; SEQUENTIAL ANALYSIS; TIME-SEQUENTIAL INFERENCE; and WEAK CONVERGENCE, STATISTICAL APPLICATIONS OF.

P. K. SEN

19. Koul, H. L. (1969). Ann. Math. Statist., 40, 1950–1979. 20. Koul, H. L. (1970). Ann. Math. Statist., 41, 1768–1773.


21. Koul, H. L. (1971). Ann. Math. Statist., 42, 466–476.

Suppose that a multiple linear regression∗ model is given by

22. Koul, H. L. (1982). Colloq. Math. Soc. Janos Bolyai, 32, 537–565. 23. Koul, H. L. (1985). Sankhya¯ A, 47, 57–84. 24. Koul, H. L. and DeWet, T. Statist., 11, 921–932.

(1983).

i = 1, . . . , n,

Ann.

25. Koul, H. L. and Sen, P. K. (1985). Statist. Prob. Lett., 3, 111–115. 26. Koul, H. L. and Staudte, R. G., Jr. (1972). Ann. Math. Statist., 43, 832–841. 27. Millar, P. W. (1981). Zeit. Wahrscheinlichkeitsth. Verwand. Geb, 55, 73–89. 28. Pierce, D. A. and Kopecky, K. J. Biometrika, 66, 1–5.

yi = β1 x1i + · · · + βk xki + i ,

(1979).

29. Pyke, R. and Shorack, G. (1968a). Ann. Math. Statist., 39, 755–771. 30. Pyke, R. and Shorack, G. (1968b). Ann. Math. Statist., 39, 1675–1685.

(1)

with responses y1 , . . ., yn , regression coefficients β1 , . . ., βk , independent variables x1i , . . ., xki , i = 1, . . ., n, and errors 1 , . . ., n . Ordinary least squares∗ (OLS) estimates βˆ1 , . . ., βˆk of β1 , . . ., βk are chosen to minimize n {yi − β1 x1i − · · · − βk xki }2 . i=1

This estimation procedure presupposes that the errors are uncorrelated with equal variances. If var( i ) = σ 2 /wi , i = 1, . . . , n, for


known weights w1 , . . . , wn , then weighted least-squares (WLS) estimates of β1 , . . . , βk are obtained by minimizing n

wi (yi − β1 x1i − · · · − βk xki )2 .

(2)

i=1

This can be done by constructing pseudovariates∗ y∗i =

√ wi yi ,

x∗ji =

√ wi xji ,

where i = 1, . . . , n and j = 1, . . . , k, and calculating OLS estimates via the model y∗i = β1 x∗1i + · · · + βk x∗ki + ∗i , where ∗1 , . . . , ∗n now have equal variances. See Box et al. [4] for an elementary discussion and examples. Suppose that the i ’s are also correlated with positive-definite covariance matrix σ 2 V, where V is known, and write eq. (1) in matrix notation as Y = Xβ + ,

(3)

where now X is an n × k matrix of rank k and E[] = 0. Since a nonsingular matrix P exists satisfying V = PP , the transformation Y∗ = P−1 Y = X∗ B + ∗ leads to uncorrelated identically distributed errors ∗1 , . . . , ∗n with common variance σ 2 . The sum of squares to be minimized is then

9117

As with OLS, WLS estimators constructed with suitable weights in the classical general linear model∗ lead to best linear unbiased estimators of the parameters β. This includes models with heterogeneous variances leading to minimization of (2), grouped data, replicates with unequal group sizes, and multivariate regression estimators as in (5). In practice, the covariance matrix σ 2 V is usually unknown. When the responses are independent, σ 2 V is diagonal, with var(Yi ) = σ 2i , say. Replacing σ 21 , . . . , σ 2n by sample variances s21 , . . . , s2n in the preceding leads to esti∗∗ mated WLS estimators βˆ , which have the ∗ same large-sample properties as βˆ (Schmidt [10, p. 71]). Following a Monte Carlo study, Kleinjen et al. [7] gave recommendations for estimating β with small to moderate samples in experimental design situations. In logistic (or logit) regression, k binomial populations with parameters p1 , . . . , pk , are analyzed via the model pi =

exp(α + β xi ) , 1 + exp(α + β xi )

i = 1, . . . , k, (6)

with explanatory variables xi = (x1 , . . . , xq ), say, so that the logit log{pi /(1 − pi )} = α + β xi .

(7)

If ni independent observations are made on the ith group, resulting in Yi occurrences of the event of interest (so that pi is estimated by pˆ i = Yi /ni ), the corresponding observed logit is

∗ ∗ = (Y∗ − X∗ β) (Y∗ − X∗ β)

(4a)

li = log[pˆ i /(1 − pˆ i )].

= (Y − Xβ) V−1 (Y − Xβ)

(4b)

[If any yi = 0, replace it by 0.5; if any yi = ni , let pˆ i = ni /(ni + 1), in order to avoid infinite values of li .] Then WLS estimates of α, β1 , . . . , βq are obtained by minimizing

= (Y − E[Y]) V

−1

(Y − E[Y]).

(4c)

The OLS solution to minimizing (4a) is the WLS solution to minimizing (4b) and leads to WLS estimators ∗

−1 −1 βˆ = (X V X)−1 X V Y.

k

(9)

i=1

(5)

If the space spanned by the columns of V−1 X coincides with that spanned by the columns of X, then the WLS estimates βˆ coincide with ˆ see Seber [11, Sec. 3.6] the OLS estimates β; for details.

(li − α − β xi )2 /vi ,

(8)

where vi = (1/yi + 1/(ni − yi )) estimates the variance of li , i = 1, . . . , k. Under the model (6) the minimized weighted

9118


sum of squares in (9) is Berkson’s minimum logit chi-square statistic, used to test the goodness of fit∗ of the model if the ni ’s are not too small. An excellent elementary presentation, with an example using National Football League field goal kicking data, is given by Morris and Rolph [9, Sec. 8.2]. For a detailed discussion, see LOGISTIC REGRESSION and Bishop et al. [3, p. 355]. WLS estimation in logit regression illustrates an approach to estimating parameters in log-linear models arising from multidimensional contingency tables∗ (see also LOGLINEAR MODELS IN CONTINGENCY TABLES and Bishop et al. [3, pp. 352–357]). The general WLS approach in this setup was developed by Grizzle et al. [6]. With discrete or grouped data∗ Y, minimization of quadratic forms (4c), where E(Y) is a function of a vector of parameters θ , frequently leads to WLS estimates θˆ of θ that are best asymptotically normal, and the substitution of θˆ for θ in the quadratic form yields a statistic with an asymptotic chi-square distribution. In (4c) V may be replaced by a matrix M, which converges in probability to V or to σ 2 V (such as the covariance matrix of the responses), or minimization of (4c) may be subject to certain constraints, or the responses may be transformed. In such cases WLS leads to minimum chi-square∗ estimates; Pearson’s chi-square and minimum logit chi-square are two such instances. Berkson [2] contended that ‘‘the basic principle of estimation is minimum chisquare, not maximum likelihood.’’ Berkson’s contention is not a defense of WLS estimation, but MINIMUM CHI-SQUARE indicates the key role of WLS in many of the minimizing procedures. See also CHI-SQUARE TEST —I; GROUPED DATA; LOGISTIC REGRESSION; MOST PROBABLE NUMBER; POISSON REGRESSION. Where data in the form of time series∗ are used to predict future observations, WLS forecasts based on exponential smoothing can be obtained. The procedure is to discount past observations by giving greater weight to recent observations. Under certain conditions such discounted least-squares estimates lead to minimum mean square error forecasts [1, Chap. 2, 3, 7]; see PREDICTION AND FORECASTING.

In some circumstances the weights (or the matrix of weights) involved in computing WLS estimates are themselves functions of the fitted values. In such cases, initial estimates of the parameters are used to calculate appropriate weights and in turn improved estimates, the process being repeated iteratively until some desired degree of accuracy has been reached. Thus the estimation problem for certain generalized linear models∗ reduces to iteratively reweighted least squares∗ ; see McCullagh and Nelder [8, Sec. 2.5] for a comprehensive and detailed discussion. In robust regression∗ points further away from a regression line or surface receive less weight than points closer to the line; see ITERATIVELY REWEIGHTED LEAST SQUARES for an overview and references. Other fields in which WLS techniques figure are listed among the related entries; see also the Current Index to Statistics∗ . Rather than comprising a unified methodology on its own, WLS is tailored to the requirements of the specific problem at hand, as are the computer programs (in SAS and BMDF for the classical linear model, GLIM∗ for generalized linear models, EM algorithms for certain iterative procedures, etc.) that derive the estimates. There is some difference in nomenclature in the literature. Searle (see GENERAL LINEAR MODEL) and Seber [11] call the minimization of (4b)—and analogous estimation problems in which the rank of V is less than full—the generalized least-squares (GLS) problem, and Seber further restricts the use of the term ‘‘weighted least squares’’ to the case in which V is a diagonal matrix. Among sources covering classical estimation problems with normally distributed errors, Draper and Smith [5, Sec. 2.11] use the WLS terminology to cover all of these cases, as we have done here. The same is true of writers dealing with models having nonnormally distributed errors. In one class of generalized linear model problems, however, the GLS terminology seems to be more appropriate, namely, that in which the covariance matrix V in (4c) is not a matrix of constants but (for example) a matrix whose elements are known functions of E(Y). For further discussions, see QUASILIKELIHOOD FUNCTIONS and McCullagh and Nelder [8, Chap. 8].

WEIGHTED LEAST-SQUARES RANK ESTIMATORS

REFERENCES

9119

statistic

1. Abraham, B. and Ledolter, J. (1983). Statistical Methods for Forecasting. Wiley, New York. 2. Berkson, J. (1980). Ann. Statist., 8, 457–487. (Includes discussion by B. Efron, J. K. Ghosh, L. LeCam, J. Pfanzagl, and C. R. Rao.) 3. Bishop, Y. M. M., Fienberg, S. E., and Holland, P. W. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, MA. 4. Box, G. E. P., Hunter, W. G., and Hunter, J. S. (1978). Statistics for Experimenters. Wiley, New York. 5. Draper, N. R. and Smith, H. (1981). Applied Regression Analysis, 2nd ed. Wiley, New York. 6. Grizzle, J. E., Starmer, C. F., and Koch, G. G. (1969). Biometrics, 25, 489–504. 7. Kleinjen, J., Brent, R., and Brouwers, R. (1981). Commun. Statist. B, 10, 303–313. 8. McCullagh, P. and Nelder, J. A. (1983). Generalized Linear Models. Chapman and Hall, London. 9. Morris, C. N. and Rolph, J. E. (1981). Introduction to Data Analysis and Statistical Inference. Prentice-Hall, Englewood Cliffs, NJ. 10. Schmidt, P. (1976). Econometrics. Dekker, New York. 11. Seber, G. A. F. (1977). Linear Regression Analysis. Wiley, New York. See also CHI-SQUARE TEST —I; GENERALIZED LINEAR MODELS; GENERAL LINEAR MODEL; GROUPED DATA; ITERATIVELY REWEIGHTED LEAST SQUARES; LEAST SQUARES; LINEAR REGRESSION, SCHWEPPE-TYPE ESTMATORS; LOGISTIC REGRESSION; MINIMUM CHI-SQUARE; MOST PROBABLE NUMBER; ORDINAL DATA; POISSON REGRESSION; PREDICTION AND FORECASTING; PROBABILITY PLOTTING; PSEUDO-VARIATES; QUASI-LIKELIHOOD FUNCTIONS; RESISTANT TECHNIQUES; ROBUST REGRESSION; and WALD’S W-STATISTICS.

CAMPBELL B. READ

Sn (t) =

a[Ri (t)]sign(Xi − t),

(1)

i=1

where Ri (t) is the rank of |Xi − t| among |X1 − t|, . . . , |Xn − t|. When a(i) ≡ 1, a(i) = i, or a(i) = −1 [(i + n + 1)/{2(n + 1)}] where is the standard normal CDF, we have the sign statistic, the Wilcoxon signed rank statistic∗ , or the van der Waerden one-sample normal scores∗ statistic, respectively. Note that Sn (t) is a decreasing step function and ESn (θ ) = 0. Hence a natural estimator of θ is θˆ defined as a solution t = θˆ of the estimating equation Sn (t) = 0.

(2)

Since Sn (t) is a step function, we take as our solution either the point at which Sn (t) steps across zero or the midpoint of the interval of values for which Sn (t) is identically zero. When a(i) = 1, θˆ is the median of the sample, and when a(i) = i, θˆ is the median of the n(n + 1)/2 pairwise Walsh averages∗ (Xi + Xj )/2, 1 i j n. The estimator, derived from a nonparametric rank statistic, was first proposed by Hodges and Lehmann [2] (see HODGES —LEHMANN ESTIMATORS). For other scores, such as the one-sample normal scores, an explicit representation of the estimator is not available. Hence numerical methods are needed to solve (2). Even in the case a(i) = i, numerical methods can be useful since it may be time consuming to find the median of the n(n + 1)/2 averages. A simple iterative procedure can be based on the method of weighted least squares∗ . We rewrite (1) in the following form: Sn (t) =

WEIGHTED LEAST-SQUARES RANK ESTIMATORS Suppose X1 , . . . , Xn represents a random sample from an absolutely continuous∗ distribution. Suppose, further, that the density function f (x − θ) is symmetric about θ , so that the location parameter θ is the median (and mean if it exists). Let a(1) · · · a(n) be a sequence of nonconstant scores and consider the rank∗

n

n

wi (t)(Xi − t),

(3)

i=1

where

wi (t) =

     0,

a[Ri (t)]/|Xi − t|, if |Xi − t| > 0, otherwise.

Now, using θˆ 0 to denote an initial estimate, generally the sample mean or median, a

9120

WEIGHTED MEAN

one-step solution of (3) is given by wi (θˆ 0 )Xi Sn (θˆ 0 ) θˆ 1 = = θˆ 0 + . ˆ wi (θ 0 ) wi (θˆ 0 )

(4)

The estimator θˆ k is defined by performing k iterations and is called the k-step weighted least-squares rank estimator. This approach extends quite easily to estimation in the general linear model∗ . The properties of θˆ k are related to the properties of the initial estimate θˆ 0 and to the properties of θˆ , the true solution to (2). As k increases, θˆ k is more like θˆ and the effects of θˆ become negligible. Under mild regularity conditions, θˆ k has an approximately normal distribution with mean θ and variance σk2 for large n. The formula for σk2 is rather complicated; see Theorem 2 of Cheng and Hettmansperger [1]. However, in many circumstances, σk2 is essentially the same as the asymptotic variance of θˆ after a few iterations. Thus θˆ k and θˆ have essentially the same efficiency properties. Furthermore, after 4 or 5 iterations, even starting with the nonrobust θˆ 0 = X, the breakdown values and the influence curve∗ for θˆ k are very similar to those for θˆ . Hence, for most practical purposes, after a few iterations θˆ k may be used ˆ in place of θ. As k increases convergence of θˆ k to θˆ can be guaranteed by incorporating an intervalhalving procedure with the iteration of (4). Generally, however, we can treat θˆ k as an estimator in its own right with properties ˆ Simply iterating very similar to those of θ. (4) a few times yields numerical values quite close to θˆ . As a simple example we consider Example 12.4 of Noether [5, p. 130]. Nine examination scores are given in order as follows: 62, 70, 74, 75, 77, 80, 83, 85, and 88. If we take a(i) = i, then θˆ = 77.5, the median of the 45 pairwise averages. A Minitab program [6] was written to iterate (4) using an extreme starting value of θˆ 0 = 50. We find for k = 1, 2, . . . , 5, θˆ k = 79.39, 78.08, 77.71, 77.36, and 77.46, respectively. Hence, even with a poor starting value, θˆ k was close to θˆ after only a few iterations. Using an appropriate linear approximation to (1), Newton-Raphson∗ -types of iterations can be performed to define another

k-step estimator. This approach requires the estimation of the slope of the linear approximation and is a bit more complicated than a weighted least-squares method. However, good numerical results are also achieved with this method. For a discussion of these linearized rank estimators in the linear model see Kraft and van Eeden [3] and McKean and Hettmansperger [4]. REFERENCES 1. Cheng, K. S. and Hettmansperger, T. P. (1983). Commun. Statist. A, 12, 1069–1086. (This reference contains the details of the weighted least-squares rank estimates for location as well as linear models.) 2. Hodges, J. L., Jr. and Lehmann, E. L. (1963). Ann. Math. Statist., 34, 598–611. 3. Kraft, C. H. and van Eeden, C. (1972). Ann. Math. Statist., 43, 42–57. 4. McKean, J. W. and Hettmansperger, T. P. (1978). Biometrika, 65, 571–579. 5. Noether, G. E. (1976). Introduction to Statistics, A Nonparametric Approach, 2nd ed. Houghton Mifflin, Boston. 6. Ryan, T. A., Jr., Joiner, B. L., and Ryan, B. F. (1982). Minitab Reference Manual. Minitab Project, Inc., University Park, PA. See also INFLUENCE FUNCTIONS; RANK STATISTICS; SCORE STATISTICS; and WEIGHTED LEAST SQUARES.

THOMAS P. HETTMANSPERGER

WEIGHTED MEAN. See TUKEY’S INEQUALITY FOR OPTIMAL WEIGHTS

WEIGHTED NORMAL PLOTS Weighted normal plots, proposed by Dempster and Ryan [2], provide a model-checking technique that is particularly sensitive to nonnormality of the random effects in a simple one-way∗ comparisons model. Suppose Y1 , . . . , Yn are independent observations from the model Yi = µi + i ,

(1)

where the pairs (µi , i ) are independent and normally distributed with means µ and

WEIGHTED NORMAL PLOTS

9121

0, and variances σ 2 and σi2 , respectively. The outcomes Y1 , . . . , Yn might be summary statistics from grouped data∗ of unequal sample sizes or with different sampling variances in each group, as illustrated later with an example. Overall goodness of fit∗ may be assessed by a normal probability plot∗ of the standardized variables

for x ∈ (Z(i−1) , Z(i) ], where Z(i) is the ith order statistic∗ , and

Zi = (Yi − µ)/(σ 2 + σi2 )1/2 .

Fn (x) = (i − 3/8)/(n + 1/4),

This approach involves plotting Zi against −1 [Fn (Zi )], where (·) is the standard normal cumulative distribution function (CDF) and Fn (·) is an empirical CDF, which, ignoring end-point adjustments for now, can be written as Fn (x) =

n

I(x − Zi )/n,

(2)

i=1

where I(x) = 1 for x 0 and 0 otherwise. Endpoint adjustments are ignored for the present. For the purpose of checking the normality of the random effects in model (1), a weighted normal plot may be more efficient. The idea is to weight each Zi according to the amount of information∗ contained about the random effects µi . For instance, a variable Zi should be downweighted if the error variance σi2 is relatively large. Suitable weights are given by Wi = (σ 2 + σi2 )−1 and may be incorporated by replacing Fn (·) in (2) by Fn∗ (x) =

n

I(x − Zi )Wi

n

i=1

Wi .

S = [(9/8)W(1) + · · · + W(i) + · · · + (9/8)W(n) ]. In the case of equal weights, Fn∗ (·) yields the widely used plotting positions recommended by Blom [1]:

x ∈ (Z(i−1) , Z(i) ]. Dempster and Ryan [2] show that weighted and unweighted plots behave similarly, except that the pointwise variation of the weighted plot increases by the factor (1 + v/m2 ), where m and v are the mean and variance of the weights. However, the extra variability of the weighted plot is countered by increased sensitivity to detect violations of the normality assumption on the random effects in model (1). Weighted normal plots are illustrated here by an example involving a comparison of stomach surgery rates in 21 regions of Vermont (Miao [3]). For each of the 21 regions, a logistic model was used to provide estimates λˆ i of age-adjusted surgery rates in each region. To smooth these estimators and to model the interregional variation, model (1) was applied using the estimators λˆ i as Yi and the estimated sampling variance of each

(3)

i=1

In practice, the weighted empirical CDF (3) must be adjusted at the end points, otherwise the function −1 [Fn∗ (·)] cannot be calculated at the largest Zi . The following formula performs well: Fn∗ (x) = (5/8)W(1) /S, for x Z(1) , and Fn∗ (x) = [(9/8)W(1) + W(2) + · · · +(1/2)W(i) ]/S,

Figure 1. Weighted normal plot of stomach surgery rates in Vermont.

9122

WEIGHTED QUASILINEAR MEAN

λˆ i as σi2 . The corresponding weighted normal plot, displayed in Fig. 1, suggests that model (1) fitted quite well. For this example, the factor (1 + v/m2 ) was 1.12. REFERENCES 1. Blom, G. (1958). Statistical Estimates and Transformed Beta Variables. Wiley, New York. 2. Dempster, A. P. and Ryan, L. M. (1985). J. Amer. Statist. Ass., 80, 845–850. 3. Miao, L. L. (1977). An Empirical Bayes Approach to Analysis of Interarea Variation: A study of Vermont Surgical Mortality Rates, Ph.D. Thesis, Statistics Department, Harvard University, Cambridge, MA. See also HALF-NORMAL PLOTS and PROBABILITY PLOTTING.

LOUISE M. RYAN

WEIGHTED QUASILINEAR MEAN Fishburn [1] has defined the quasilinear mean of a random variable X with CDF FX (x), relative to a function f (x), as ∞ −1 −1 f (E[f (X)]) = f f (x)dF(x) . −∞

He also defines the weighted quasilinear mean relative to f (x), with weight function g(x), as f −1 (E[f (x)g(x)]/E[g(x)]) ∞ −∞ f (x)g(x)dF(x) −1 ∞ . =f −∞ g(x)dF(x) These concepts have applications in utility theory∗ .

for every t > 0. Assuming a density (or probability function) exists, it must therefore satisfy f (θ + t) = f (θ − t), for every t > 0. Thus symmetry around θ means the density is the same t units above θ as it is t units below θ . Weighted symmetry extends this characterization to cover cases in which the probability above θ is λ(> 0) times the probability below θ . Therefore f (θ + t) = λf (θ − t), for every t > 0, that is, the density (probability function) t units above θ is the same as λ times the density (probability function) t units below θ . Thus the density above and below θ has the same shape but potentially differs in total probability content. More generally, X is weighted symmetric around θ if for some λ > 0, P(X > θ + t) = λP(X < θ − t), for every t > 0. See Wolfe [2] for tests using weighted symmetry. Ref. 1 contains an interesting application. REFERENCES 1. Parent, E. A. (1965). Tech. Rep. No. 80, Dept. of Statistics, Stanford University. (Uses weighted symmetry to perform a sequential signed-rank test.) 2. Wolfe, D. A. (1974). J. Amer. Statist. Ass., 69, 819–822. (Characterizes weighted symmetry and uses these characterizations to perform tests of hypotheses.)

RONALD H. RANDLES

REFERENCE 1. Fishburn, P. C. 1197–1205.

(1986).

Econometrica,

54,

See also GEOMETRIC MEAN and UTILITY THEORY.

WEIGHT OF AN ARRAY. See STRENGTH OF AN

ARRAY

WEIGHT OF EVIDENCE. See STATISTICAL EVIDENCE

WEIGHTED SYMMETRY A random variable X is said to be symmetric about the value θ if P[X > θ + t] = P[X < θ − t],

WEILER’S INDEX OF GOODNESS OF FIT Weiler [3] developed an ‘‘index of discrepancy,’’ which measures the deviation of a

WEISS TEST OF INDEPENDENCE

distribution of frequencies from a hypothetical one. Suppose a random event can have s possiprobble outcomes A1 , . . . , As with unknown abilities P1 , . . . , ps , respectively ( sj=1 pj = 1). To test the hypothesis H0 : pj = πj ,

j = 1, . . . , s,

using a random sample of size n with Aj observed nj times (j = 1, . . . , s; sj=1 nj = n), we can use the sample version of Weiler’s index, which is φˆ 2 =

X2

π0 · , 1 − π0 n

where π0 = min(π1 , . . . , πs ) and X2 =

s

(nπj )−1 (nj − nπj )2

j=1

is the familiar chi-squared∗ statistic. The sample version is a (positively biased) estimator of the population Weiler index φ2 =

s π0 (pj − πj )2 . 1 − π0 πj j=1

The modified sample version   s nj π0 1 2 2 X + 1 −  φˆ 1 = 1 − π0 nπj n − 1 j=1

is an unbiased estimator of φ 2 , but it can take negative values. In the special case π1 = π2 = · · · = πs = s−1 (= π0 ), we have φ 2 = {n(s − 1)}−1 X 2 , which is closely related to Simpson’s diversity index D (see DIVERSITY INDICES). [The index I = s(s − 1)−1 D was introduced independently by Mueller and Schuessler [2].] Agresti and Agresti [1] give a survey of these and related indices. REFERENCES 1. Agresti, A. and Agresti, B. F. (1978). In Sociological Methodology, K. F. Schuessler, ed. Jossey-Bass, San Francisco, pp. 204–237.

9123

2. Mueller, J. H. and Schuessler, K. F. (1961). Statistical Reasoning in Sociology. Houghton Mifflin, Boston. 3. Weiler, H. (1966). Technometrics, 8, 327–334. See also DIVERSITY INDICES; GOODNESS OF FIT; LOG-LINEAR MODELS IN CONTINGENCY TABLES; MULTINOMIAL DISTRIBUTIONS; SOCIOLOGY, STATISTICS IN; and SURPRISE INDEX.

WEISS TEST OF INDEPENDENCE A familiar example of testing independence is a contingency table∗ , in which we test the independence of two or more criteria of classification, each containing a finite number of classes. The classes in the table are sometimes formed by grouping continuous random variables. The boundaries of these classes must be nonrandom for the usual statistical analysis to be valid. But in testing the independence of a set of continuous random variables, whose joint distribution is unknown, using nonrandom boundaries (that is, boundaries set before observations are taken) can result in a situation where it is impossible to carry out a meaningful analysis. For example, every observed point might fall in just one of the cells of the contingency table, and then no light would be shed on whether or not the variables are independent. In the tests described below, we use order statistics∗ as boundaries. Also, the number of cells may increase as the sample size increases. In deciding how many cells there should be, there are two conflicting considerations. The larger the number of cells, the smaller is the amount of information lost due to the grouping. But if the number of cells is too large, the joint distribution of the cell frequencies will no longer approach a normal distribution as the number of observations increases, and the analysis becomes too complicated to be of any practical use. We use the following notation. n is a positive integer, and we observe n independent and identically distributed s-tuples (X1 (i), . . ., Xs (i); i = 1, . . ., n) of continuous random variables. The joint distribution of the s components is unknown, and the problem is to test the hypothesis that the s components are mutually independent. Kn and Ln are positive integers, with

9124

WEISS TEST OF INDEPENDENCE

n/(Kn + 2) lim {Ln − n/(Kn + 2)} =

n→∞

< Ln , lim K 3s /n n→∞ n

= 0.

(Note that these assumptions allow Kn to approach infinity as n increases, but do not require it.) Let Yj (1) < · · · < Yj (n) denote the ordered values of Xj (1), . . . , Xj (n) and define Yj (0) as −∞ and Yj ((Kn + 2)Ln ) as +∞. If h1 , . . . , hs are integers, each between 1 and Kn + 2 inclusive, we define Nn (h1 , . . . , hs ) as the number of values of i for which the s-tuple (X1 (i), . . . , Xs (i)) satisfies the conditions (Yj ((hj − 1)Ln ) < Xj (i) < Yj (hj Ln )) for j = 1, . . . , s. The (Kn + 2)s quantities (Nn (h1 , . . ., hs )) are analogous to the cell frequencies in an s-dimensional contingency table, and they will be used to construct tests of independence. Let Fj (x) denote the marginal cumulative distribution function for Xj (i). The values of the (Kn + 2)s quantities (Nn (h1 , . . . , hs )) would not be affected if Xj (i) were replaced by Fj (Xj (i)) for j = 1, . . . , s and i = 1, . . . , n. Of course, Fj (x) is unknown, so we cannot actually replace Xj (i) by Fj (Xj (i)), but for the analysis below it will be useful to imagine that this has been done. Then we can assume that the marginal distribution of Xj (i) is the uniform distribution over (0, 1), and we can write the joint probability density function of the s components of (X1 (i), . . . , Xs (i)) as 1 + r(x1 , . . . , xs ) over the s-dimensional unit cube Cs , and 0 outside Cs , where r(x1 , . . . , xs )dx1 · · · dxs = 0. η = ··· Cs

Then the hypothesis of independence is equivalent to the hypothesis that r(x1 , . . . , xs ) = 0 for all (x1 , . . . , xs ) in Cs . A statistic analogous to the usual statistic used for testing independence in a contingency table is  K n +2 −1  (Kn + 2)s ··· n hs =1

×

K n +2



{Nn (h1 , . . . , hs ) − n(Kn + 2)−s }2  ,

h1 =1

which we denote by Wn . Assuming that Kn approaches infinity as n increases, subject to the restrictions on Kn given above,

the distribution of Wn is approximately a non-central chi-square distribution∗ with (Kn + 1)s degrees of freedom and noncentrality parameter r2 (x1 , . . . , xs )dx1 · · · dxs . λ = n ··· Cs

If the desired level of significance is α, the hypothesis of independence is rejected if Wn is greater than the (1 − α)th quantile of the central chi-square distribution∗ with (Kn + 1)s degrees of freedom. This test is consistent against any alternative for which λ = 0. We can find much better tests than the one based on Wn , if we are willing to make some mild assumptions about the joint distribution of (X1 (i), . . . , Xs (i)). Since the test based on Wn is consistent against any fixed alternative, to describe the improvement we use the familiar device of letting the alternative approach the hypothesis being tested as the sample size n increases. Specifically, we assume that the joint probability density function of the s components of (X1 (i), . . ., Xs (i)) is 1 + n−1/2 r(x1 , . . ., xs ) over Cs , with η = 0 and λ < ∞. The test based on Wn is useless against such alternatives: The power of the test approaches the level of significance as n increases. We expand r(x1 , . . . , xs ) as a Fourier cosine series over Cs , ∞

···

j1 =0

∞

A(j1 , . . . , js )2D(j1 ,...,js )

js =0

s

cos(ji xi ),

i=1

where D(j1 , . . . , js ) is equal to one-half the number of positive integers among j1 , . . . , js . Since η = 0, it follows that A(0, . . . , 0) = 0. Then the hypothesis of independence is equivalent to the hypothesis that A(j1 , . . . , js ) = 0 for every s-tuple (j1 , . . . , js ) of nonnegative integers containing at least one positive integer. We can estimate A(j1 , . . . , js ) by the following function of the quantities (Nn (h1 , . . . , hs )): n1/2 2D(j1 ,...,js ) ×

K n +2

···

is =1

×

s & b=1

K n +2

{Nn (i1 , . . . , is ) − n/(Kn + 2)s }

i1 =1

' cos π jb ib − 12 n−1 Ln .

WEISS-TYPE ESTIMATORS OF SHAPE PARAMETERS

Denote this estimator by A∗n (j1 , . . . , js ). It is shown in [4] that if (j1(i) , . . . , js(i) ; i = 1, . . . , m) are m different sets of non-negative integers, each set containing at least one positive integer, then the joint distribution of (A∗n (j1(i) , . . . , js(i) ); i = 1, . . . , m) approaches the distribution of m independent normal random variables, each with unit variance and with respective means (A(j1(i) , . . ., js(i) ); i = 1, . . ., m), as n increases. This fact can be used to construct a variety of tests of the hypothesis of independence. For example, from the theory of Fourier series∗ , if r(x1 , . . . , xs ) has partial derivatives that are bounded in absolute value, then A(j1 , . . . , js ) will be small in absolute value if at least one of the s values j1 , . . . , js is large. Suppose there is a positive integer T such that we are willing to assume that A(j1 , . . . , js ) is close to 0 if any of the values j1 , . . . , js is above T. Define T

V(T) =

T

···

j1 =0

(A∗n (j1 , . . . , js ))2 ,

js =0

where A∗n (0, . . . , 0) is defined to be 0. If n is large, the distribution of V(T) is approximately noncentral chi-square∗ with (T + 1)s − 1 degress of freedom and noncentrality parameter T

···

j1 =0

T

(A(j1 , . . . , js ))2 .

js =0

If the desired level of significance is α, the hypothesis of independence is rejected if V(T) is greater than the (1 − α)th quantile of the central chi-square distribution with (T + 1)s − 1 degrees of freedom. The test based on V(T) is an all-purpose test, in the sense that it has reasonably good power against the very wide class of alternatives for which the noncentrality parameter ζ is a respectable distance above 0. Suppose we are particularly concerned about an alternative to the hypothesis for which r(x1 , . . . , xs ) has known Fourier coefficients (A(j1 , . . . , js ); ji = 0, . . . , T). Define R(T) =

T j1 =0

···

T

A(j1 , . . . , js )

js =0

×A∗n (j1 , . . . , js ).

9125

The optimal test against this particular alternative is to reject the hypothesis if R(T) is greater than ( ) T T ) ) ··· (A(j1 , . . . , js ))2 G−1 (1 − α), * j1 =0

js =0

where G denotes the standard normal cumulative distribution function. We can combine the special-purpose test based on R(T) and the all-purpose test based on V(T) into a single test, as follows: Reject the hypothesis if either V(T) > c1 or R(T) > c2 , where c1 and c2 are chosen to give the desired level of significance. By varying the relative values of c1 and c2 , we can increase the power of the test against the special alternative at the expense of power against other alternatives, or vice versa. Details are given in [4]. Historical Note. There are many tests of independence of continuous random variables. Some of the better known ones are Hoeffding’s test∗ for s = 2 [3], a general purpose test for general s based on the empirical distribution function∗ [2], and rank tests∗ for s = 2 which are optimal against specific alternatives [1]. REFERENCES 1. Bell, C. B. and Doksum, K. A. (1967). Ann. Math. Statist., 38, 429–446. 2. Blum, J., Kiefer, J., and Rosenblatt, M. (1961). Ann. Math. Statist., 32, 485–498. 3. Hoeffding, W. (1948). Ann. Math. Statist., 19, 546–557. 4. Weiss, L. (1985). Naval Res. Logist. Quart., 32, 337–346. See also CONTINGENCY TABLES: DIACONIS-EFRON CONDITIONAL VOLUME TEST; DEPENDENCE, TESTS FOR; FOURIER COEFFICIENTS; HOEFFDING’S INDEPENDENCE TEST; HYPOTHESIS TESTING; and LOG-LINEAR MODELS IN CONTINGENCY TABLES.

L. WEISS

WEISS-TYPE ESTIMATORS OF SHAPE PARAMETERS The random variable X is said to have a threeparameter Weibull distribution∗ if its cumulative distribution function is 1 − exp{−[(x −

9126

WELCH’S v -CRITERION

α)/β]1/γ } if x α and 0 if x < α. Here β and γ are positive. α is known as the location parameter, β as the scale parameter, and γ as the shape parameter. Suppose X1 , . . . , Xn are independent and identically distributed random variables, the common distribution being a Weibull distribution with all three parameters unknown. The problem of estimating the parameters is difficult, largely because the form of a good estimator of α changes greatly as γ crosses from below 1 to above 1. A consistent estimator of γ can be constructed as follows. Let t1 , t2 be fixed values with 0 < t1 < t2 < 1, and let [A] denote the largest integer not greater than A. Let X1 < X2 < · · · < Xn denote the ordered values of X1 , X2 , . . . , Xn . Then log(X[nt ] − X1 ) − log(X[nt ] − X1 ) 2

1

log log(1/(1 − t2 )) − log log(1/(1 − t1 )) is a consistent estimator of γ . Weiss [1] investigated the following problem. Suppose X1 , . . . , Xn are independent and identically distributed with common probability density function f (x), given as follows: f (x) = 0, 1/γ −1

f (x) = c(x − α)

if x < α, [1 + r(x − α)],

for x > α,

where c, α, γ are unknown parameters with c > 0 and γ > 0; r(y) is an unknown function except that we know that |r(y)| KyA for all y in some interval [0, ]; where K, A, and are unknown positive values. That is, in a neighborhood of α, this density function is like a Weibull density, which itself satisfies these conditions. A wide class of densities, which also satisfies the conditions, consists ∗ on the left at α. That of densities truncated ∞ is, f (x) = g(x)/ α g(x)dx if x > α, and f (x) = 0 if x < α, where g(x) is a density satisfying mild conditions. Let U1 , U2 , . . . denote independent random variables, each with CDF 1 − e−u for u 0. Let {k(n); n = 1, 2, . . .} be any sequence of positive integers such that limn→∞ k(n) = ∞ and limn→∞ k(n)/nδ = 0 for every δ > 0. [For example, k(n) = [log n].] Then Weiss showed that for all asymptotic (as n → ∞) probability calculations, we can represent Xi as

α+

1 (U1 + · · · + Ui ) cγ n

γ ,

for i = 1, . . . , k(n). Based on this, Weiss showed that − X1 ) − log(X[k(n)/2] − X1 ) log(Xk(n)

log 2 is a consistent estimator of γ . Weissman [2] investigated the consistent estimators of γ of the form − X1 ) − log(Xm(n) − X1 ) log(Xk(n)

log(k(n)/m(n))

,

for all integers m(n) < k(n), and showed that the value of m(n) that minimizes the asymptotic variance of the estimator is m(n) = [0.2032k(n)]. This choice gives an asymptotic variance about 25% lower than m(n) = [ 21 k(n)], this latter being the one used in the estimator constructed by Weiss. The estimators constructed by Weiss and Weissman use only three order statistics. No doubt they could be improved by using more, but the computations become very difficult. Even in the simpler case when we are sampling from a Weibull distribution, the maximum likelihood∗ estimators are so complicated that they have not yet been satisfactorily analyzed. REFERENCES 1. Weiss, L. (1971). Naval Res. Logist. Quart., 18, 111–114. 2. Weissman, I. (1981). Naval Res. Logist. Quart., 28, 603–605. See also CONSISTENCY; ORDER STATISTICS; and WEIBULL DISTRIBUTION.

L. WEISS

WELCH’S v -CRITERION k Suppose we have n = t=1 nt p-variate observations, classified into k groups. Denoting the random variable corresponding to the jth observation in the tth group by the p × 1 column vector Xtj , the assumed model is

WELCH TESTS

Xtj = µ + α t + Etj ·, (t = 1, . . . , k; j = 1, . . . , nt ), where µ and α t are each constant p × 1 vectors, such that k

and Etj is a random p × 1 error vector. The joint distribution of the p-variates of Etj does not depend on j and the expected value of each is 0. The common distribution is often assumed to be multinormal∗ , and the Etj ’s are assumed mutually independent. In multivariate analysis of variance∗ (MANOVA), the null hypothesis (of no difference among group means) H0 : α 1 = α 2 = · · · = α k = 0

(1)

is tested against the class of alternative hypotheses that at least one of the equalities in (1) is violated. For a general model with p > 1 and k > 2 and with the variance-covariance matrices of the distributions of the Etj ’s unknown, James [1] proposed the test statistic k (Xt − X) Wt (Xt − X), t=1

where Xt = n−1 t

nt

Xtj ,

j=1

St = (nt − 1)−1

nt

(Xtj − Xt )(Xtj − Xt ) ,

j=1 −1 Wt = (n−1 t St ) ,

W=

k

Wt ,

t=1

X = W−1

k

See also BEHRENS –FISHER PROBLEM; MULTIVARIATE ANALYSIS OF VARIANCE (MANOVA); WALD’S W-STATISTICS; and WELCH TESTS.

WELCH TESTS

nt α t = 0,

t=1

Tυ2 =

9127

Wt X t .

t=1

See related entries and James [1] for further details. REFERENCE 1. James, G. S. (1954). Biometrika, 41, 19–43.

In the one-way∗ layout to compare the means of k normally distributed populatons, it may not be valid in some cases to assume homogeneous variances. Hence the ANOVA∗ F-test∗ is not applicable, and the Welch [19] test was proposed to fill this void. An important special case (k = 2) is the famous Behrens–Fisher∗ problem. This special case was solved by Welch [18] several years earlier than the general case. His solution for k = 2 was refined and tabled by Aspin [1,2] and has become known as the Aspin–Welch test (AWT). Further tables were later provided by Trickett et al. [17]. Competing solutons to the Behrens–Fisher problem have been suggested by Fisher [8], Lee and Gurland [11] (denoted LG), Cochran [6], and Welch himself [18; 2, Appendix]. All these tests depend on normality, and Yuen [21] and Tiku and Singh [16] attempt more robust solutions. Some competing procedures for general k are due to Brown and Forsythe [5], James [9], and Bishop and Dudewicz [3]. The Welch and Brown-Forsythe tests have been extended by Roth [13] to the case where the k populations have a natural ordering (e.g., different dosages of the same drug) and a trend test∗ is desired to detect differences in the means that are monotone as a function of this ordering. Another (unrelated) Welch [20] test was designed in mixed or random effects models to provide confidence intervals for variance components∗ , whose estimators are often distributed as linear combinations of chi-squared variates. Basically, Welch provides correction terms to the confidence limits obtained via the Satterthwaite [14,15] approximation∗ , which is based on a single chi-squared variate. These corrections were long among the most widely advocated methods (see, e.g., Mendenhall [12, pp. 352–354]), with no perceived major drawbacks except tedious computations. However, Boardman’s [4] simulations showed that the Welch corrections are actually detrimental

9128

WELCH TESTS

to achieving the nominal confidence coefficients. Hence they have fallen justifiably into disfavor and will not be discussed further. We now explore the details and properties of the Welch tests described in the first paragraph, and we begin with the AWT. The test statistic is t = +

x1 − x2

,

(1)

s21 /n1 + s22 /n2

and Welch [18], for a slightly more general problem, derives the percentage points of t as a power series in 1/fi = 1/(ni − 1) for i = 1, 2. The P fractile (0 < P < 1) of t is explicitly given to order (1/fi )2 by ,

(1 + α)2 (s4i /n2i fi ) α 1+ 4 ( s2i /ni )2 (3 + 5α 2 + α 4 ) (s6i /n3i fi2 ) + 3 ( s2i /ni )3 (15 + 32α 2 + 9α 4 ) (s4i /n2i fi )2 − , (2) 32 ( s2i /ni )4

on (3); the latter two are widely used due to the computational simplicity of referring to ordinary t-tables, (3) being far more accurate. This accuracy is evaluated from the tables of LG [11], whose general method for this whole class of size and power calculations revealed that the AWT operates closest by far to the nominal level from among a set of seven competing tests. LG then proposed their own test, which is almost identical to the AWT in both size and power, and recommended it on the grounds of greater simplicity. However, it requires five constants that depend on the sample sizes and the nominal level, which are provided only for 5 n1 n2 10 at the 0.05 level. The Welch [19] test for general k compares the statistic

W∗ =

wj (xj − µ) ˆ 2 /(k − 1) , 1 + [2(k − 2)/(k2 − 1)] hj

(4)

to the F(k − 1, f ) distribution, where wj = nj /s2j ,

µˆ =

wj xj /W,

W=

wj ,

hj = (1 − wj /W) /(nj − 1), hj ). f = (k2 − 1)/(3 2

where α = −1 (P) and is the standard normal CDF. Note that the constant term in (2) reflects simply the normal approximation to t . Welch also suggests a method that refers to ordinary t-tables. This is done by equating the first two moments of t to those of a tdistribution∗ with f degrees of freedom. The solution for f is (1 − c)2 c2 1 + , = f f1 f2

(3)

where c = (s21 /n1 )/(s21 /n1 + s22 /n2 ). Welch [18] originally suggested replacing fi in (3) by fi + 2, i = 1, 2, and blank, but he later repudiated this suggestion [2, Appendix]. He showed that critical values based on (3) agree with the correct ones based on (2) to order (1/fi ), but they differ in the (1/fi )2 term. The AWT is more powerful (i.e., has lower critical values) to varying degrees, and hence gives narrower confidence intervals, in general, than the essentially Bayesian∗ Behrens–Fisher solution, the Cochran method, or the Welch test based

It and the Brown–Forsythe [5] test both reduce to the Welch test based on (3) when k = 2. The derivation of W ∗ , like that of the AWT, stems from a power series in (1/fi ). Welch shows that W ∗ agrees to order 1/fi , but not to order (1/fi )2 , with the James [9] test, which is based on a chi-squared (not F) approximation. Brown and Forsythe [5] demonstrate via simulations that, in general, their procedure and W ∗ both outperform the James test; furthermore, W ∗ tends to be better than their procedure when extreme means are associated with small variances, and vice versa. Importantly, both procedures lose little power in the equal variance case relative to the ‘‘optimal’’ ANOVA F-test, which is hence NOT recommended for the one-way layout. Dykstra and Werter [7] refine the James test and claim from their simulations that this refinement is on balance superior to the other tests; however, their numerical tables seem to support this conclusion only mildly

WELCH TESTS

when k = 6 and not at all when k = 4. In any case, the Welch test is quite competitive. Incidentally, Johansen [10] rederives the Welch test as a special case of a more general result on residuals∗ from a weighted linear regression∗ . Roth’s [13] extension of W ∗ to the Welch trend test (WT) for ordered populations is basically obtained by first amalgamating the population means using isotonic regression∗ for simple order with weights wj = nj /s2j . Conditionally on the results of the amalgamation process, the statistic W ∗ (when applied to the amalgamated populations) is multiplied by an appropriate constant so that its conditional distribution is similar 2 to that of E , which is the trend analog of the ANOVA F-test. Roth also developed the Brown–Forsythe trend test (BFT), and his simulations showed that WT is generally (but by no means uniformly) the better of the two, tending to have larger type I error rates but compensating for this with gains in power too great to be explained merely by the differences in level. Conditions under which WT is superior to BFT (and vice versa) are analogous to the above-mentioned findings of Brown and Forsythe in the nontrend situa2 tion. Analogously as well, the E -test does not seem to gain much power (and hence is not recommended) even when the variances are equal, unless the sample sizes are as small as 2 or 3. REFERENCES 1. Aspin, A. A. (1948). Biometrika, 35, 88–96. (Refines the AWT and tables some critical values.) 2. Aspin, A. A. (1949). Biometrika, 36, 290–296. [Contains an appendix by Welch commenting on Aspin’s work and proposing the test based on (3).] 3. Bishop, T. A. and Dudewicz, E. J. (1978). Technometrics, 20, 419–430. 4. Boardman, T. J. (1974). Biometrics, 30, 251–262. (Simulations and references involving 12 procedures for confidence intervals for variance components.) 5. Brown, M. B. and Forsythe, A. B. (1974). Technometrics, 16, 385–389. 6. Cochran, W. G. (1964). Biometrics, 20, 191–195. (Points out important drawbacks to his own procedure.)

9129

7. Dykstra, J. B. and Werter, P. S. P. J. (1981). Commun. Statist. B, 10, 557–569. 8. Fisher, R. A. (1941). Ann. Eugen., Lond., 11, 141–172. [The original fiducial (essentially Bayesian) solution to the Behrens–Fisher problem.] 9. James, G. S. (1951). Biometrika, 38, 324–329. 10. Johansen, S. (1980). Biometrika, 67, 85–92. 11. Lee, A. F. S. and Gurland, J. (1975). J. Amer. Statist. Ass., 70, 933–941. [Extensive bibliography on the entire subject. Also outlines general method for computing size and power of Welch-type tests (numerically tabled for several tests), and proposes a new test for k = 2.] 12. Mendenhall, W. (1968). Introduction to Linear Models and the Design and Analysis of Experiments. Wadsworth, Belmont, CA. 13. Roth, A. J. (1983). J. Amer. Statist. Ass., 78, 972–980. (Defines WT and BFT with detailed numerical examples.) 14. Satterthwaite, F. E. (1941). Psychometrika, 6, 309–316. 15. Satterthwaite, F. E. (1946). Biometrics Bull., 2, 110–114. 16. Tiku, M. L. and Singh, M. (1981). Commun. Statist. A, 10, 2057–2071. (Robust statistic for k = 2 that outperforms Yuen’s.) 17. Trickett, W. H., Welch, B. L., and James, G. S. (1956). Biometrika, 43, 203–205. (More tables of critical values for the AWT.) 18. Welch, B. L. (1947). Biometrika, 34, 28–35. [Proposes both the AWT and the test based on (3) with fi replaced by fi + 2.] 19. Welch, B. L. (1951). Biometrika, 38, 330–336. (Welch’s k-sample test.) 20. Welch, B. L. (1956). J. Amer. Statist. Ass., 51, 132–148. 21. Yuen, K. K. (1974). Biometrika, 61, 165–170. [Obtains level of Welch test based on (3) for many nonnormal distributions and proposes a more robust statistic.]

Editorial Note In more recent papers, Aucamp (1986) (J. Statist. Comp. Simul., 24, 33–46) proposes the critical region ˆ 2 f −1 + 2(1 − C) ˆ 2 f −1 ]1/2 , |t | > z1−α/2 [1 + 2C 1 2 with (z1−α/2 ) = 1 − α/2 and 2 ˆ = s1 C n1

,

s21 s2 + 2 n1 n2

-−1 ,

9130

WELDON, WALTER FRANK RAPHAEL

and Matuszewski and Sotres (1986) (Comp. Statist. Data Anal., 3, 241–249) propose rejection of the null hypothesis if the 80% confidence intervals for the two individual means do not overlap—giving a significance level of approximately 5%. See also BEHRENS –FISHER PROBLEM; ISOTONIC INFERENCE; TREND; and TREND TESTS.

ARTHUR J. ROTH

WELDON, WALTER FRANK RAPHAEL Born: Highgate, London, England, March 15, 1860. Died: Oxford, England, April 13, 1906. Contributed to: Biometrics, evolutionary biology, zoology. W. F. R. Weldon was the second child of the journalist and industrial chemist Walter Weldon and his wife Anne Cotton. His father changed residences so frequently that Weldon’s early education was desultory until he became a boarder in 1873 at Caversham near Reading. Weldon matriculated at University College London (UCL) in the autumn of 1876 with the intention of pursuing a medical career. During his time at UCL, he acquired a respectable knowledge of mathematics from the Danish mathematician Olaus Henrici, and attended the lectures of the zoologist E. Ray Lankester. In the following year he transferred to Kings College, London, and in April 1878 he entered St. John’s College, Cambridge, as a bye-term student. Once at Cambridge, he met the zoologist Francis Maitland Balfour, and subsequently gave up his medical studies for zoology. In 1881, he gained a first-class degree in the Natural Science Tripos; in the autumn he left for the Naples Zoological Station to begin the first of his studies in marine biological organisms. Upon returning to Cambridge in 1882, Weldon was appointed university lecturer in invertebrate morphology. In the following year he married Florence Tebb. He became a founding member of the Marine Biological Station in Plymouth in 1884 and resided there until 1887.

From 1887 until his death in 1906, Weldon’s work was centered around the development of a fuller understanding of marine biological phenomena and, in particular, the examination of the relationship between various organs of crabs and shrimps, to determine selective death rates in relation to the laws of growth. During his first five years at the Marine Biological Station, Weldon’s investigations were directed to the study of classification, morphology, and the development of decapod crustacea. His only work on invertebrate morphology contained an account of the early stage of segmentation and the building of the layers of shrimp. Weldon was both a master of histological techniques and a powerful and accurate draftsman. In 1889 he succeeded E. Ray Lankester in the Joddrell Chair of Zoology at University College London. During this time Weldon read Francis Galton’s Natural Inheritance. In this book Galton∗ had shown that the frequency distributions of the average size of certain organs in man, plants, and moths were normally distributed. Similar investigations had been pursued by the Belgian statistician, Adolphe Quetelet∗ , whose work was confined to ‘‘civilized man.’’ Weldon was interested in investigating those variations in organs in a species living in a wild state, acted upon by natural selection and other destructive influences. Writing on heredity in 1889, Galton had predicted that selection would not change the shape of the normal distribution∗ ; he expected that his frequency distributions would remain normally distributed in all cases, whether or not animals were under the action of natural selection. Around this time, Weldon began to study the variation of four organs in the common shrimp (Crangon vulgaris), and he collected five samples from waters fairly distant from Plymouth. His statistical analysis, published in 1890, confirmed Galton’s prediction. Shortly after the paper was published, Weldon was elected a Fellow of the Royal Society. During the Easter vacation of 1892, Weldon and his wife collected 23 measurements from 1000 adult female shore crabs (Carcinus mœnas) from Malta and the Bay of Naples. Weldon discovered that all but one of the 23 characters he measured in the Naples

WELDON, WALTER FRANK RAPHAEL

group were normally distributed; he found that this one character (the frontal breadth of the carapace) was instead a ‘‘double-humped’’ (i.e., bimodal) curve. His first attempt to interpret the data involved breaking up the curve into two normal distributions as Galton had advocated. Weldon then approached Karl Pearson∗ (who had been appointed as Professor of Mechanics and Applied Mathematics at UCL in 1884) for assistance with interpreting his data. At that time, Pearson was teaching applied mathematics to engineering students at UCL and was also giving his Gresham lectures on geometry at Gresham College. (This ancient educational foundation, located in the City of London, offered lectures to members of the public on an annual basis.) By the end of 1892 Pearson began to devise a probability system of curve fitting for Weldon’s data, and he used this material in his Gresham lectures in the following year. From his analysis of Weldon’s data, Pearson concluded that two separate species had arisen. Up until the middle of the nineteenth century, species were defined in terms of types or essences. Charles Darwin’s recognition that species comprised different sets of ‘‘statistical’’ populations, rather than types or essences, prompted a reconceptualization of statistical populations by Pearson and Weldon. Moreover, this required the use of new statistical methods. Following his work on curve fitting of asymmetrical distributions, in his Gresham lecture of November 23, 1893, Pearson devised a goodness-of-fit test for asymmetrical distributions for Weldon’s data. These statistical innovations formed much of the basis of Pearson’s work in the nineteenth century, and in 1900 he devised the chisquare (χ 2 ) goodness-of-fit∗ test (his single most important contribution to modern statistical theory). When Pearson was working out the mathematical properties of simple correlation∗ and regression∗ in 1896, Weldon suggested to Pearson the idea of a negative correlation (Galton had used positive correlations only.) Pearson regarded Weldon as ‘‘one of the closest friends he ever had.’’ Their relationship can be characterized as an emotional and intellectual intimacy that engendered a

9131

symbiotic alliance. It is thus not surprising that one the most extensive sets of letters in Pearson’s archives are those of Weldon and his wife Florence, which consist of nearly 1000 pieces of correspondence. Though not a statistician by training, it was Weldon’s interest in finding statistical tools to demonstrate empirical evidence of Darwin’s theory of natural selection in marine organisms that provided the impetus to Pearson’s development of the modern theory of statistics. Weldon also provided Pearson with the basis of a program that underpinned the construction of his statistical innovations in the 1890s, which in turn provided the infrastructure for Pearson’s statistical developments in the twentieth century. Weldon’s influence, which exceeded that of any other person in the emergence and development of Pearsonian statistics, arose from the following factors: he provided the stimulus for new statistical methods by asking Pearson biological and statistical questions that could be answered only by devising a new statistical approach, he offered continual moral support as well as encouragement, and he promulgated the Pearsonian corpus of statistics to ‘‘serious students of animal evolution’’ throughout the 1890s and until his death in 1906. Though some of the statistical work of John Vennfand Francis Ysidro Edgeworth∗ played a role in Pearson’s early work on probability, there seems to have been no other person whose influence on the emergence and development of Pearsonian statistics was as rapid and immediate in its impact as that of Weldon. When Weldon went up to Oxford in 1899, to take up the Linacre Chair of Comparative Anatomy, he carried on the biometric tradition by gathering a number of students who began to look for empirical evidence of natural selection acting upon various animals and plants. Despite Weldon’s move, he and Pearson made arrangements to be together every year during the Easter and Christmas vacations and throughout the summer months. They continued their collaborative work, investigating biometrica problems such as natural selection, inheritance, and, in particular, Mendelian inheritance. These joint biometrical projects

9132

WELL-CALIBRATED FORECASTS

were pursued until Weldon’s untimely death in 1906. Weldon’s death was for Pearson the single greatest loss in his life. Pearson had not only lost the closest friend he ever had, but he found himself, at once, alone in the scientific community with no one to guide him with such problems as Mendelian inheritance and no one to share his enthusiasm for his statistical work. The intensity of Pearson’s sense of bereavement, which remained poignant for the remaining thirty years of his life, led to an emotional and intellectual reconstruction of his life in the absence of his colleague. Though Pearson’s statistical work continued to be prodigious, he never fully regained the creative force he had developed from his partnership with Weldon. BIBLIOGRAPHY Bourne, G. C. (1906). Walter Frank Raphael Weldon (1860–1906). In Dictionary of National Biography. Oxford University Press, pp. 629–630. Cowan, R. S. Walter Frank Raphael Weldon (1860–1906). In Dictionary of Scientific Biography, C. C. Gillispie, ed. vol. XIV, pp. 251–252. Magnello, M. E. (1993). Karl Pearson: Evolutionary Biology and the Emergence of a Modern Theory of Statistics. Doctoral dissertation, University of Oxford. Magnello, M. E. (1996). Karl Pearson’s Gresham lectures: W. F. R. Weldon, speciation and the origins of Pearsonian statistics. Brit. J. Hist. Sci., 29, 43–63. Magnello, M. E. (1998). Karl Pearson’s mathematization of inheritance. From ancestral heredity to Mendelian genetics (1895–1909). Ann. Sci., 55, 35–94. Pearson, K. (1906). Walter Frank Raphael Weldon. 1860–1906. Biometrika, 5, 1–52. Reprinted in E. S. Pearson and Maurice Kendall, Studies in the History of Statistics and Probability, vol. 1. Griffin, London, 1970. See also ENGLISH BIOMETRIC SCHOOL and PEARSON, KARL.

M. EILEEN MAGNELLO

WELL-CALIBRATED FORECASTS Consider a forecaster who at the beginning of each period n in a sequential∗ process

n = 1, 2, . . . must specify the probability that some particular event An will occur during that period. It is assumed that when the forecaster specifies the probability of An , he or she is aware of the values of various variables which may be relevant to the occurrence of An . In particular, the knowledge of which of the previous events A1 , . . . , An−1 actually occurred will usually be part of the forecaster’s information at the beginning of period n. For example, the forecaster might be a meteorologist, who at the beginning of each day must specify the probability that it will rain during that day at a particular location, or an economist, who at the beginning of each weekly or quarterly period must specify the probability that a particular interest rate or stock market average will rise during the period. In another context, the forecaster might be a medical diagnostician, who must specify the probability that a patient has a particular disorder on the basis of an examination and who subsequently learns the patient’s true condition. See METEOROLOGY, STATISTICS IN and MEDICAL DIAGNOSIS, STATISTICS IN. In this article the discussion will be presented in the context of a weather forecaster, who day after day must specify the probability x that there will be at least a certain amount of rain at some given location during a specified time interval of the day. We refer to the occurrence of this wellspecified event simply as ‘‘rain.’’ The probability x specified by the forecaster on any particular day is called the prediction for that day. To begin, we make the assumption that x is restricted to a given finite set of values 0 = x0 < x1 < · · · < xk = 1, and let X = {x0 , . . . , xk } denote the set of possible predictions. The forecaster’s predictions might, in general, be based on mathematical or stochastic models of the weather, computer simulations, subjective judgments, or a combination of these methods. The notion of the calibration∗ of a forecaster’s predictions pertains to a comparison of the predicted probabilities with the actual outcomes of the events being predicted. Consider the forecaster’s predictions over a large number of days, and for each value xj ∈ X, let ρ(xj ) denote the proportion of days on

WELL-CALIBRATED FORECASTS

which it actually rained among all those days for which the prediction was xj . The function ρ(xj ) is called the forecaster’s calibration curve. Loosely speaking, the forecaster is said to be well calibrated if ρ(xj ) is approximately equal to xj for each xj ∈ X. In other words, the forecaster is well calibrated if among all those days for which the predicted probability of rain was xj , the proportion of days on which it actually rained was approximately xj . In evaluating a forecaster’s performance, it is usually regarded as desirable that he or she be well calibrated. However, as the following argument indicates, it is typically not difficult for the forecaster to make sure that this criterion will be satisfied. Suppose, for example, that the forecaster calculates a calibration curve ρ(xj ) after a certain number of days, and notes that ρ(xj ) < xj for some particular value of xj . Then, to increase the value of ρ(xj ), the forecaster need only wait until there is a day when he or she is almost certain that it will rain, and to state the value x = xj as the prediction for that day. Similarly, if ρ(xj ) > xj for some value of xj , then the forecaster can decrease the value of ρ(xj ) by making the prediction x = xj on days when he or she is almost certain that it will not rain. In this way, by stating predictions other than actual subjective probabilities of rain, the forecaster can usually manipulate ρ(xj ) so that it is approximately equal to xj for all values of xj ∈ X. These considerations throw doubt on the usefulness, and even on the meaning, of the forecaster’s predictions when the property of being well calibrated is regarded as a primary criterion. Even if a forecaster’s ‘‘honest’’ subjective probabilities make him or her well calibrated, the predictions may not be very useful. Consider, for example, the extreme case of a forecaster who knows nothing about meteorology and simply states the same prediction x = µ every day, where µ is the long-run relative frequency of rain. Then the forecaster will be well calibrated, but the predictions will obviously be useless to anyone who already knows the value of µ. At the other extreme is a forecaster whose prediction each day is either x = 0 or x = 1, and who is always correct. That forecaster is also well calibrated and the predictions convey perfect information

9133

about whether or not it will rain each day. Methods of comparing and evaluating forecasters based on concepts of refinement and scoring rules are presented by DeGroot and Fienberg [3,4] and DeGroot and Eriksson [2]. Empirical studies of calibration are described by Murphy and Winkler [9] and Lichtenstein et al. [6]. For any fixed finite number of days, the property of being well calibrated can only be stated, as we have done, in terms of approximate equality between ρ(xj ) and xj . Consider now an infinite sequence of days, and let X ∗ denote the subset of X containing the values of xj for which the prediction x = xj is made on an infinite number of days in the sequence. For each value of xj ∈ X ∗ , let ρ(xj ) denote the limiting relative frequency of rain among all those days for which the prediction was x = xj . In this context, a forecaster is said to be well calibrated if ρ(xj ) = xj for each value of xj ∈ X ∗ . A more stringent definition of being well calibrated over an infinite sequence of days, but one that is more general in the sense that it does not require that the set X of possible predictions be finite, has been given by Dawid [1]. His approach is to select an infinite subsequence of days and to compare the limiting relative frequency of rain over this subsequence with the limit of the average of the predicted probabilities for the subsequence. More precisely, let ρ denote the limiting relative frequency of rain and let π = lim

k→∞

k 1 πni , k i=1

where πni is the prediction made on the ith day of the subsequence. Then the forecaster is said to be well calibrated if ρ = π for every subsequence that is selected in accordance with certain conditions. The selection of a subsequence is constrained only by the requirement that the decision to include or not include any particular day in the subsequence must be based on the information that is available to the forecaster at the time that the prediction for that day must be made. Thus, the selection of days can be made sequentially, but the decision to include a particular day cannot be based on whether or

9134

WESTENBERG TEST OF DISPERSION

not it actually rained on that day or on any other future observations. Under these conditions, Dawid shows that if the forecaster specifies a joint probability distribution of all of the variables that will be observed over the entire infinite sequence of days, and if the prediction made by the forecaster each day is the conditional probability of rain given all of the past data, as calculated from this joint distribution, then the forecaster will be well calibrated with probability 1 (where this probability is again calculated under the forecaster’s joint distribution). Another approach to the concept of calibration—one that is completely subjective—takes into account both the forecaster and a decision maker, who will learn (and presumably use) the forecaster’s prediction. In this context, a forecaster is said to be well calibrated for a particular decision maker if, after learning the prediction x on any given day, the decision maker’s subjective probability of rain on that day also becomes x (Lindley [7]). This approach has the advantage that it is based on individual days rather than on limiting frequencies over some hypothetical infinite sequence of ‘‘similar’’ days. However, it introduces a dependence between the forecaster and the decision maker that may impede its applicability (French [5]). The usefulness of concepts of calibration in problems of combining expert opinion is discussed in Morris [8] and Schervish [10]. REFERENCES 1. Dawid, A. P. (1982). J. Amer. Statist. Ass., 77, 605–613. 2. DeGroot, M. H. and Eriksson, E. A. (1985). In Bayesian Statistics 2, J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, eds. Elsevier Science Publishers (NorthHolland), Amsterdam, pp. 99–118. 3. DeGroot, M. H. and Fienberg, S. E. (1982). In Statistical Decision Theory and Related Topics III, S. S. Gupta and J. O. Berger, eds. Academic, New York, pp. 291–314. 4. DeGroot, M. H. and Fienberg, S. E. (1983). The Statistician, 32, 12–22. 5. French, S. (1983). Tech. Rep. 114, Department of Decision Theory, University of Manchester, Manchester, England (unpublished).

6. Lichtenstein, S., Fischhoff, B., and Phillips, L. D. (1977). In Decision Making and Change in Human Affairs, H. Jungerman and G. de Zeeuw, eds. Reidel, Dordrecht, Netherlands, pp. 275–324. 7. Lindley, D. V. (1982). J. R. Statist. Soc. A, 145, 117–126. 8. Morris, P. A. (1977). Manag. Sci., 29, 24–32. 9. Murphy, A. H. and Winkler, R. L. (1977). J. R. Statist. Soc. Ser. C, 26, 41–47. 10. Schervish, M. J. (1984). Tech. Rep. 294, Department of Statistics, Carnegie-Mellon University, Pittsburgh, PA (unpublished). See also BAYESIAN INFERENCE; CALIBRATION; DECISION THEORY; and PREDICTION AND FORECASTING.

MORRIS H. DEGROOT

WESTENBERG TEST OF DISPERSION. See DISPERSION, WESTENBERG TEST OF

WESTERGAARD, HARALD LUDVIG Born: April 19, 1853, in Copenhagen, Denmark. Died: December 13, 1936, in Copenhagen, Denmark. Contributed to: mathematical economics, data analysis, demography. Although Harald Westergaard was best known as an economist, he made substantial contributions to the statistical assessment of demographic and economic data. It is with this latter aspect (the ‘‘most original,’’ according to Brems [1]) of his work that we are primarily concerned. Westergaard was born and died in Copenhagen, and worked all his life in that city, apart from studying in England and Germany in 1877–1878, after obtaining degrees in mathematics (in 1874) and economics (in 1877) from the University of Copenhagen. While he was in England, he met the British economist W. S. Jevons and became interested in the application of mathematics to problems in economics. (See ref. [2].) In 1880–1882 he worked for a lifeinsurance company and developed interests in demography and statistical method. From this there came a work [3] on mortality

WHITE TESTS OF MISSPECIFICATION

laws. In 1883 he joined the faculty at the University of Copenhagen as a docent. He remained there for the remainder of his working life, becoming professor in 1886 and retiring in 1924. During the period 1886–1935—nearly 50 years—he was a director of the Copenhagen Fire Insurance Company. Westergaard was in advance of his time in regard to the application of mathematics to economic theory, but somewhat behind it in appreciation of the use of tools of statistical inference—notably correlation analysis. On the other hand, he had a superb ability to perceive rational groupings and subgroupings underlying data, without the assistance of formal testing and searching procedures. ‘‘In the fine art of listening to the voice of numbers, few practitioners have had an ear as sensitive as Westergaard’s’’[1]. Later in life, he became interested in the historical study of ‘‘statistics,’’ culminating in the classic book [4]. As was natural, this concentrated on historical development from ‘‘Staatenkunde’’ (‘‘comparative description of States’’ was Westergaard’s description) through ‘‘political arithmetic’’ to demography and other applications. There is, indeed, a chapter on calculus of probability, but this was treated as an interesting digression from the main discussion. The book presents the accumulated results of many years of research and though, very clearly expressed, and even today it is a mine of information on the growth of statistical method, particularly in the context of its use in government and economics. Westergaard married Thora Alvilda Koch in 1881. Subsequent to her death in 1891, he married Lucie Bolette Blaedel in 1892. His son, Harald Malcolm (1888–1950), had a distinguished career as an engineer, emigrating to the United States and becoming a professor at the University of Illinois in 1927, and then professor of civil engineering and dean of the graduate school of engineering at Harvard University in 1937. His daughter, Bodil, from his second marriage, was born in 1893. She married Erik Rydberg (of Swedish origin), who was later a professor of medicine in the University of Copenhagen.

9135

Westergaard was a deeply religious person, active for many years in the evangelical wing of the Church of Denmark, and was President of the Danish Bible Society from 1927. He was very generous and helpful to those needing financial and other assistance, especially students and the elderly. REFERENCES 1. Brems, H. (1988). Westergaard, Harald Ludvig (1853–1936). In The New Palgraves Dictionary of Economic Science, vol. 4, pp. 898–899. ˚ 2. Kaergard, N. (1995). Cooperation not opposition: Marginalism and socialism in Denmark, 1871–1924. In Socialism and Marginalism in Economics, I Steedman, ed. Routledge, London and New York, pp. 87–101. 3. Westergaard, H. (1882). Die Lehre von der Mortalitat. ¨ Fischer, Jena, Germany (2nd ed., enlarged, 1901). 4. Westergaard, H. (1932). Contributions to the History of Statistics. King, London. [Reprint, 1969, published by Mouton Publishers (The Hague and Paris) and S. R. Publishers (Wakefield, England).]

WEYL INEQUALITY If A and B are symmetric n × n matrices with eigenvalues∗ (characteristic roots) λ1 (A) λ2 (A) · · · λn (A) and λ1 (B) λ2 (B) · · · λn (B), respectively, then λi (A) + λj (B) λi+j−1 (A + B). This is Weyl’s inequality (Bellman [1]). Applications in the theory of multivariate analysis∗ are described in Seber [2]. REFERENCES 1. Bellman, R. (1960). Introduction to Matrix Analysis. McGraw-Hill, New York. 2. Seber, G. A. F. (1984). Multivariate Analysis. Wiley, New York.

WHITE NOISE. See NOISE WHITE TESTS OF MISSPECIFICATION. See MISSPECIFICATION, WHITE TESTS OF

9136

WHITTAKER-HENDERSON GRADUATION FORMULAS

WHITTAKER-HENDERSON GRADUATION FORMULAS. See GRADUATION, WHITTAKER –HENDERSON

WHITTAKER-TYPE DISTRIBUTIONS Apart from the inverse Gaussian distribution∗ , no other stable distributions∗ with characteristic exponent α ∈ (0, 1) are known to have a density that can be expressed in terms of elementary functions. However, series expansions of such densities are available for rational values of α. In the special case α = 2/3 the series can be expressed in terms of Whittaker functions (see Zolotarev [4], Pollard [3] and Bar-Lev and Enis [1] for further details). Explicitly, a stable distribution with characteristic exponent ρ = 2/3 has density 1 1 −2/(27x2 ) e g2/3 (x) = 2(3π )1/2 x −4 × W−1/2,−1/6 , 27x2

(1)

given explicitly by (−2µ)

Mλ,µ (z)

( 12

− λ − µ)

+

(2µ) ( 21

− λ + µ)

1. Bar-Lev, S. K. and Enis, P. (1986). Ann. Statist., 14, 1507–1522. 2. Morris, C. (1982, 1983). Ann. Statist., 10, 65–80; 11, 515–529. 3. Pollard, H. (1946). Bull. Amer. Math Soc., 52, 908–910. 4. Zolotarev, V. M. (1961). Selected Trans. Math. Statist. Probab., 1, 163–167. See also NATURAL EXPONENTIAL FAMILIES and STABLE DISTRIBUTIONS.

WHITTEMORE’S

Mλ,−µ (z),

where Mλ,µ (z) = zµ+1/2 e−z/2 ×F1 (µ − λ + 12 ; 2µ + 1; z) [F1 (·; ·; ·) being a confluent hypergeometric function]. For λ = 0, equation (1) reduces to a Bessel equation. Whittaker-type distributions are natural exponential families∗ (see, e.g., Morris [2]). They possess power variance functions of the form V(µ) = αµγ generated by stable distributions; Bar-Lev and Enis [1] present recent applications.

COLLAPSIBILITY.

See MULTIDIMENSIONAL CONTINGENCY TABLES

WHITTINGHILL–POTTHOFF See HOMOGENEITY, TESTS OF

where Wλ,µ (z) is a solution of the Whittaker equation , 1 − µ2 λ 1 W = 0, W + 4 2 + − z z 4

Wλ,µ (z) =

REFERENCES

TESTS.

POTTHOFF-WHITTINGHILL

WHITTLE LIKELIHOOD We may view the Whittle likelihood function for a time series∗ from two angles. From one perspective we may see it simply as a way to compute the likelihood function∗ for stationary Gaussian processes∗ . On observing a zero-mean time series x1 , . . . , xT , the quantity −2 × (log-likelihood function of the unknown parameter θ) is, up to a constant term, given by L0 = log |Cθ | + X C−1 θ X,

(1)

where X = (x1 , . . . , xT ) and Cθ is the covariance matrix of X. The parameter θ typically specifies the dynamics of the time series; for example, θ may be the autoregressive∗ (AR) or moving average∗ (MA) parameters in standard parametric modeling of time series. Estimating θ from the data X is in principle quite straightforward: simply minimize L0 to find the maximum likelihood∗ estimate of θ . In practice this is far from simple, since Cθ is a T × T matrix and, in general, there is no explicit formula to compute ∗ |Cθ | and C−1 θ . For stationary processes Cθ is ∗ a Toeplitz matrix , since Cθ (j, k) = cθ (j − k), where cθ (·) is the autocovariance function of

WHITTLE LIKELIHOOD

X t . As T gets large, Toeplitz matrices have an approximate eigenvalue decomposition of the form Cθ ≈ VFθ V , where the jth eigenvector is T −0.5 exp(−2π ijk/T)fork = 0, . . . , T − 1, and Fθ is a diagonal matrix of eigenvalues∗ [2, p. 73]. These eigenvalues are the Fourier transform of cθ (·) and known as the spectrum of Xt : fθ (ω) =

∞

cθ (k) exp(−2π ikω)

k=−∞

(see SPECTRAL ANALYSIS). Since the determinant is equal to the product of eigenval−1 ues and C−1 θ ≈ VF θ V , we may approximate L0 by L1 =

T

log fθ (ωt ) +

t=1

I(ωt ) , fθ (ωt )

(2)

where ωt = t/T and I(ω) is the periodogram∗ defined by 1 I(ω) = T

T 2 xt exp(−2π iωt) t=1

(see PERIODOGRAM ANALYSIS). We can intuitively see that, for large T, L1 converges to L2 =

0.5 −0.5

log fθ (ω) +

I(ω) dω. fθ (ω)

(3)

Whittle [11,12] originally suggested (3) as an asymptotic approximation to L0 . The standard theoretical results (e.g., [5, Theorem 2.1] and [6]) state that T −1/2 (L0 − L2 ) → 0 in probability as T → ∞ and the estimates of θ based on L0 and L2 are asymptotically equivalent. Given a spectrum fθ (·), the likelihood function L1 may be computed efficiently using the fast Fourier transform∗ algorithm. Whittle [12] noted that the process under study does not have to be Gaussian, even if we are using the Gaussian likelihood. In this sense the Whittle likelihood is a quasi-likelihood∗ [8], where only the second-order properties are used to define the objective function for inferential purposes. Basing the inference on the second-order properties is akin to using the generalized least squares∗ approach. An optimality property of the Whittle likelihood is studied in [7].

9137

The second perspective is more illuminating. Under quite general assumptions (e.g., [2, Theorem 4.4.1]), the scaled periodograms 2I(ω)/f (ω) are asymptotically distributed as χ 2 with 2 degrees of freedom [except at ω = 0 mod 12 , whereI(ω)/f (ω) is asymptotically χ 2 with 1 degree of freedom because in this case exp(−2π iωt) is real rather than complex]. Moreover, periodograms from different frequencies are asymptotically independent. This heuristic leads to L1 above as the plausible likelihood function for the time series. It also motivates a generalized linear modeling of the second-order properties using the periodograms I(ωt ) as the basic observations [3, §4.9]. The Whittle likelihood approximation is an important analytical tool for the asymptotic theory of parametric time-series modeling (e.g., [4], [5], and [6]); however, it is not the method of choice for practical work. Historically, the traditional division in timeseries analysis between the timedomain∗ and frequency-domain∗ methods may have prevented the use of Whittle likelihood for practical ARMA∗ estimation (see AUTOREGRESSIVE –MOVING AVERAGE (ARMA) MODELS): Almost all parametric time-series modeling is of ARMA type, performed by time-domain analysts, but the form of Whittle likelihood is more familiar and friendly to frequency-domain analysts, who are much more concerned with nonparametric modeling. In their seminal works Box and Jenkins [1] developed recursive methods to compute the error sum of squares∗ for ARMA estimation, thus avoiding the need to compute the Gaussian likelihood. In recent years the advent of the statespace methodologies has produced very fast algorithms to compute exact Gaussian likelihood for ARMA processes [9], so there is no longer any practical need to approximate the likelihood. The use of Whittle likelihood has been extended to nonparametric spectral estimation using the penalty approach and shown to have good properties [10]. This approach allows a straightforward statement of the statistical problem without reference to any algorithmic detail (see MAXIMUM PENALIZED LIKELIHOOD ESTIMATION).

9138

WHOLESALE PRICE INDEX

REFERENCES 1. Box, G. E. P. and Jenkins, G. M. (1976). Time Series Analysis: Forecasting and Control. Holden Day, San Francisco. (The classic reference for parametric time-series modeling.) 2. Brillinger, D. (1981). Time Series: Data Analysis and Theory. Holden Day, San Francisco. (A graduate-level textbook of spectral analysis of time series.) 3. Diggle, P. (1990). Time Series: A Biostatistical Introduction. Oxford Science Publication, Oxford. (An applied time-series text with lots of real examples. Chapter 4 has an interesting development of generalized linear models based on periodograms or Whittle likelihood.) 4. Duinsmuir, W. and Hannan, E. J. (1978). Vector linear time series models. Adv. Appl. Probab., 8, 339–364. 5. Dzhaparidze, K. (1986). Parameter Estimation and Hypothesis Testing in Spectral Analysis of Time Series. Springer-Verlag, New York. Translated by S. Kotz (A highly theoretical treatment of parametric inference using Whittle likelihood.) 6. Hannan, E. J. (1973). Asymptotic theory of linear time series models. J. Apl. Probab., 10, 130–145. (A definitive theory for parametric estimates for scalar processes based on Whittle likelihood.) 7. Kulperger, R. (1985). On an optimality of Whittle’s Gaussian likelihood estimate of the parameter of the spectrum of a time series. J. Time Ser. Anal., 6, 253–259. 8. McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models. Chapman & Hall, London. (The main text on many generalizations of linear models.) 9. Mélard, G. (1984). A fast algorithm for the exact likelihood of autoregressive moving average models. Appl. Statist., 33, 104–114. 10. Pawitan, Y. and O’Sullivan, F. (1994). Nonparametric spectral density estimation using penalized Whittle likelihood. J. Amer. Statist. Ass., 89, 600–610. 11. Whittle, P. (1952). Estimation and information in time series analysis. Skand. Aktuar., 35, 48–60. 12. Whittle, P. (1962). Gaussian estimation in stationary time series. Bull. Inst. Int. Statist., 39, 105–129. See also AUTOREGRESSIVE –MOVING AVERAGE (ARMA) MODELS; GAUSSIAN PROCESSES; KALMAN FILTERING;

NONPARAMETRIC REGRESSION; PENALIZED LIKELIHOOD; PERIODOGRAM ANALYSIS; TIME SERIES; and TOEPLITZ MATRICES.

YUDI PAWITAN

WHOLESALE PRICE INDEX. See PRODUCER PRICE INDEXES

WICKSELL’S CORPUSCLE PROBLEM Spherical particles are scattered in an opaque medium, and only sectional circles of the spheres cut by a plane are observed. The problem is to estimate, especially in a nonparametric way, the distribution of the sphere size from the observed sectional circles. Wicksell [8] named it the corpuscle problem and obtained the relation between the probability density functions of the distributions. The estimation, or unfolding, is important in metallurgy, medical science, and other fields. The problem is formulated in two ways. First, the sectional plane is placed at random. Second, the placement of the centers of the spheres is assumed to be a Poisson process∗ , and their size distribution is assumed to be independent of the process, which is thus a marked point process∗ . The size should be small compared with the spacing of the centers, so that overlapping of the spheres is negligible. Wicksell measured the size of the spheres and the circles by their diameter. However, both in practice and in theory, it is convenient rather to measure them by the area, say Y, of the great circle of spheres and that, say W, of the sectional circles. The survival functions of these areas are related as follows: ∞ 1 (y − w)−1/2 SY (y)dy, SW (w) = 2m w

SY (y) =

2m π

0 < w < ∞,

∞

(w − y)−1/2 dSW (w),

y

0 < y < ∞, where m = E(Y 1/2 ) = π/[2E(W −1/2 )]. The exponential distribution∗ is a unique invariant distribution in these transformations.

WIENER–KOLMOGOROV PREDICTION THEORY

Our concern is to estimate SY or related quantities from a random sample from SW . This is a simple and typical problem in stereology (see STEREOLOGY for the classical approach, and ref. 6 for the modern approach using the sampling theory for spatial processes∗ ). However, since it is illposed [5], it is long-standing, and many procedures for it have been proposed. References are provided in ref. 6, and in ref. 3; the latter reviews some earlier papers, classifying them in seven categories, and discussing further the estimation of the distribution function of the volume or weight of the spheres. A numerical quadrature∗ for the inverse integral transform∗ is studied by Mase [4]. An optimal isotonic estimator of the distribution function of Y has been proposed and its convergence rate found [2]. Extreme-value theory∗ in Wicksell’s corpuscle problem and its application have been studied [1,7]. REFERENCES 1. Drees, H. and Reiss, R. -D. (1992). Tail behavior in Wicksell’s corpuscle problem. In Probability Theory and Applications: Essays to the Memory of József Mogyoródi, J. Galambos and ´ I. Katai, eds. Kluwer, Dordrecht, pp. 205–220. 2. Groeneboom, P. and Jongbloed, G. (1995). Isotonic estimation and rates of convergence in Wicksell’s problem. Ann. Statist., 23, 1518–1542. 3. Hoogendoorn, A. W. (1992). Estimating the weight undersize distribution for the Wicksell problem. Statist. Neerland., 46, 259–282. 4. Mase, S. (1992). Stereological estimation of particle size distribution. Adv. Appl. Probab., 27, 350–366. 5. O’Sullivan, F. (1986). A statistical perspective on ill-posed inverse problems. Statist. Sci., 1, 502–527. 6. Stoyan, D., Kendall, W. S., and Mecke, J. (1995). Stochastic Geometry and Its Applications, 2nd ed. Wiley, New York, Section 11.4. 7. Takahashi, R. and Sibuya, M. (1996). The maximum size of the planar sections of random spheres and its application to metallurgy. Ann. Inst. Statist. Math., 48, 127–144. 8. Wicksell, S. D. (1925). The corpuscle problem. Biometrika, 17, 84–99. See also PARTICLE-SIZE STATISTICS and STEREOLOGY.

M. SIBUYA

9139

WIEAND’S THEOREM. See BAHADUR EFFICIENCY, APPROXIMATE

WIENER CHAIN An intuitively appealing discrete analog of the Wiener process (see BROWNIAN MOTION) suggested by Kreith [1]. It is defined as a stationary Markov process∗ W0 , W1 , W2 , . . . with state space consisting of all integers for which (i) W0 = 0; (ii) (Wn + n)σ −1 has a binomial distribution with parameters (2n, 12 )—a ‘‘(nσ 2 /2)-centered binomial’’; (iii) if n1 < n2 < n3 < · · · then Wn2 − Wn1 , Wn3 − Wn2 , · · · are mutually independent. If W0 , W1 , W2 , . . . is a Wiener chain then Wm+n − Wm has a nσ 2 /2-centered binomial distribution, for all m. Kreith [1] gives details of motivation and background, and discusses a stochastic difference equation yielding a continuous version, corresponding to a Wiener process. REFERENCE 1. Kreith, K. (1985). Amer. Math. Monthly, 92, 281–284. See also BROWNIAN MOTION and MARKOV PROCESSES.

WIENER–HOPF EQUATION. See PREDICTION NORBERT

AND

FILTERING, LINEAR; WIENER,

WIENER–KOLMOGOROV PREDICTION THEORY A principal question examined by Wiener [17] and Kolmogorov [10] is how to predict the unknown future values of a time series∗ on the basis of the known past. Suppose that the observations Xt , t = 1, . . . , n, on a single time series are made at consecutive time periods, and it is desired to estimate a future observation Xn+h , h 1. The observed time

9140


series may be viewed as a (part) realization of a stochastic process∗ , {xt }, t = 0, ±1, . . .. Wiener and Kolmogorov assumed that {xt } is weakly stationary∗ with mean 0, the infinite past {xt , t n} has been observed and considered only the linear least-squares∗ predictors. Thus xn+h is estimated by a linear function of known values xˆ n (h) =

∞

infinite past. If (2) does not hold, then xt admits the Wold decomposition and it may be written as xt = wt + υt , where υt is purely nondeterministic while wt , uncorrelated with υt , is deterministic. When assumptions (1) hold, xt admits the infinite moving-average∗ representation xt =

δh (j)xn−j ,

∞

b(j)t−j ,

b(0) = 1,

(3)

j=0

j=0

and an infinite autoregressive representation

where the δh (j) are chosen so that the mean square error of prediction

∞

a(j)xt−j = t ,

a(0) = 1,

(4)

j=0

V(h) = E[{ˆxn (h) − xn+h }2 ] is minimized. In this case all one need know of the {xt } process to determine the prediction constants, δh (j), from the least-squares principle is its covariance function, R(s) = E[xt xt+s ], or equivalently its spectral density function f (µ) = (2π )−1

∞

in which {t } is a sequence of uncorrelated random variables with mean 0 and variance σ 2 , say, and the a(j) and b(j) are absolutely summable. If f (µ) is known exactly, then σ 2 and the a(j) and b(j) may be determined by the Wiener–Hopf factorization of f (µ). One sets σ 2 = 2π exp{c(0)}, π b(j) = (2π )−1 B(µ) exp(ijµ) dµ,

R(s) exp(−isµ).

s=−∞

Note that if {xt } is a Gaussian process∗ , then the linear least-squares predictor reduces to the unrestricted, and so possibly nonlinear, least-squares predictor. Also, although unrealistic, the assumption that the infinite past has been observed is made for convenience. A full mathematical derivation of the Wiener–Kolmogorov prediction theory, in a Hilbert space setting, has been given by Doob [8]; see also PREDICTION AND FILTERING, LINEAR. A simplified derivation is given below by assuming that ∞

|R(s)| < ∞,

f (µ) > 0,

all µ. (1)

s=−∞

a(j) = (2π )−1 where B(µ) = exp

(5)

−π π

A(µ) exp(ijµ) dµ, −π

∞

c(υ) exp(−iυµ)

υ=1

and A(µ) = {B(µ)}−1 give the transfer functions of the b(j) and a(j), respectively, and π −1 c(υ) = (2π ) log f (µ) exp(iυµ) dµ. −π

These assumptions are more restrictive than is necessary for deriving the linear leastsquares predictor. They however ensure that π log f (µ)dµ > −∞ (2) −π

so that the process is purely nondeterministic and free from deterministic components which could be predicted exactly from the

The c(υ) introduced above are known as cepstral correlations (Parzen [12]). They are also the parameters of an exponential model proposed by Bloomfield [6] for f (µ), with c(υ) = 0, |υ| > p, say. This model provides an alternative to the standard autoregressive–moving-average process∗ ; see BOX –JENKINS MODEL, for which f (µ) is a rational function.


As in Whittle [16], let xˆ n (h) =

∞

δh (j)xn−j =

j=0

∞

ϕh (j)n−j .

j=0

The generating functions, Dh (z) and h (z) of the δh (j) and ϕh (j), respectively, are connected by the relation ∞

Dh (z) =

δh (j)zj = B(z)h (z),

(6)

In words, the rule for determining xˆ n (h) is to replace the n+j , j = 1, . . . , h − 1, which have not yet happened by their mean value of 0, to replace the xn+j j = 1, . . . , h − 1, which have not been observed by their optimal linear predictor xˆ n (j), and the xn−j , j = 0, 1, . . ., which have already been observed are left unchanged. As an example, suppose that xt follows a moving-average process of order 1, xt = t − βt−1 ,

j=0

where B(z) denotes the corresponding generating function of the b(j). Now,

+

j=0

= σ2

h

xˆ n (h) = 0,

2    (b(j + h) − ϕh (j))n−j  

b2 (j)

j=0

+

∞

{b(j + h) − ϕh (j)}2

(7)

j=0

Dh (z) = {B(z)}−1

∞

b(j + h)zj ,

(8)

j=0

V(h) = σ 2

h

b2 (j).

(9)

j=0

Equations (8) and (9) completely determine the linear least-squares predictor when f (µ) is known exactly. Moreover, in view of (4), we may write ∞

a(j)xn−j ,

j=1

xˆ n (h) = −

h−1 j=1

β j xn+1−j , h > 1,

and for h > 1 the observed past conveys no information on xn+h . In general, for a movingaverage process of order q, xˆ n (h) = 0, h > q. Similarly, if xt is an autoregressive process of order m, then xˆ n (h) depends only on the last m observed x’s, alone. Note that once an additional observation, xn+1 , becomes available, the xˆ n (h) may be updated by the formula (Box and Jenkins [7, p. 134]) xˆ n+1 (h) = xˆ n (h + 1) + b(h){xn+1 − xˆ n (1)}.

and attains its smallest value when ϕh (j) = b(j + h). Thus

xˆ n (1) = −

∞ j=1

j=0

∞

|β| < 1.

Then xˆ n (1) = −

V(h) = E[{xn+h − xˆ n (h)}2 ]  h  b(j)n+h−j = E 

9141

a(j)ˆxn (h − j) −

∞

a(j)xn+h−j .

j=h

(10)

The basic linear least-squares prediction theory as described above has been extended in several directions. First, to the problem of linear interpolation. Suppose that the entire realization of {xt } has been observed except for a single value, x0 , say. The linear leastsquares interpolator of x0 can be determined by an analogous argument; the mean squared interpolation error is the harmonic mean of f (µ) whereas that of one-step prediction is the geometric mean of f (µ). Applications include the problems of estimating missing values and outlier detection in time series, and analysis of one-dimensional spatial data (Battaglia and Bhansali [2]). Second, to the problem of predicting one time series yt from only the past of xt . This is the problem of Wiener filtering and includes as its special case that of estimating a signal in the presence of noise, the transfer function modeling, and estimation of a distributed

9142


lag relationship; see Bhansali and Karavellas [4]. Finally, to a continuous time process, see PREDICTION AND FILTERING, LINEAR. In practice, f (µ) is invariably unknown and the prediction constants need to be estimated from an observed realization of {xt }. A ‘‘parametric’’ and a ‘‘nonparametric’’ approach is available for estimating the Wiener–Kolmogorov predictor. In the parametric approach, f (µ) is postulated to be a known function of a finite number of unknown parameters, which are then estimated from the observed time series. A popular model is the autoregressive–moving-average process; see PREDICTION AND FORECASTING. In the nonparametric approach, f (µ) is explicitly recognized to be a function of an infinite number of parameters, but since only n observations are available, a model with k parameters is fitted such that k → ∞ as n → ∞. This approach may be implemented by fitting an autoregressive model of order k; see Parzen [12]. An alternative is to factorize a ‘‘window’’ spectral estimate by numerically adapting the procedure described earlier; see Bhansali [3]. When an estimated model is used for prediction, expression (9) for the mean squared error of prediction needs to be modified to allow for the effect of parameter estimation. Bloomfield [5] and Yamamoto [18], amongst others, show that the effect is to increase the mean square error of prediction by a term of order n−1 , and develop expressions for evaluating the increase. Another problem of considerable interest is that of model selection, i.e., which model to use for prediction. In this context, the use of a model selection criterion has received considerable attention. It is pertinent to note that a derivation of Akaike’s [1] final prediction criterion, which is the forerunner of his information criterion∗ , is essentially based on the idea of selecting the model so as to minimize the expected one-step mean square error prediction when the parameters of each candidate model have also been estimated. For an autoregressive model, expressions for evaluating the mean square error of prediction when the order as well as the parameters have been estimated are derived by Shibata [14,15] and when the order is unknown

by Bhansali [3a]. Hannan and Nicholls [9], amongst others, consider the question of directly estimating σ 2 from (5) by replacing f (µ) by the raw, or slightly smoothed, periodogram∗ ; this question is of interest in assessing the adequacy of a fitted model and for model selection. The stationarity assumption made so far is not realistic from the point of view of practical applications. A standard approach is to try and transform the observed nonstationary series to one that is approximately stationary, e.g., by differencing or by estimating and removing a ‘‘trend’’ by linear regression, see PREDICTION AND FORECASTING and BOX –JENKINS MODEL. A number of widely used ad hoc forecasting procedures provide optimal linear forecasts for special cases of the former procedure, see Box and Jenkins [7]. Priestley [13] considers a class of nonstationary processes, called oscillatory processes, with time-varying spectra and develops a theory analogous to that already described for their linear least-squares prediction. For simplicity, the discussion has so far been confined to univariate processes. An exposition of the multivariate prediction theory may be found in Whittle [16]; see also PREDICTION AND FILTERING, LINEAR. REFERENCES 1. Akaike, H. (1969). Ann. Inst. Statist. Math., 21, 243–247. 2. Battaglia, F. and Bhansali, R. J. (1987). Biometrika, 74, 771–779. 3. Bhansali, R. J. (1974). J. R. Statist. Soc. B, 36, 61–73. 3a. Bhansali, R. J. (1981). J. Amer. Statist. Ass., 76, 588–597. 4. Bhansali, R. J. and Karavellas, D. (1983). In Time Series in the Frequency Domain, Handbook of Statistics, Vol. 3, D. R. Brillinger and P. R. Krishnaiah, eds. North-Holland, Amsterdam, Netherlands, pp. 1–19. 5. Bloomfield, P. (1972). Biometrika, 59, 501–508. 6. Bloomfield, P. (1973). Biometrika, 60, 217–226. 7. Box, G. E. P. and Jenkins, G. M. (1970). Time Series Analysis: Forecasting and Control. Holden Day, San Francisco.

WIENER MEASURE 8. Doob, J. L. (1953). Stochastic Processes. Wiley, New York. 9. Hannan, E. J. and Nicholls, D. F. (1977). J. Amer. Statist. Ass., 72, 834–840. 10. Kolmogorov, A. N. (1939). C. R. Acad. Sci. Paris, 208, 2043–2045. 11. Parzen, E. (1969). In Multivariate Analysis—II, P. R. Krishaniah, ed. Academic, New York, pp. 389–409. 12. Parzen, E. (1983). In Time Series in the Frequency Domain, Handbook of Statistics, D. R. Brillinger and P. R. Krishnaiah, eds. North-Holland, Amsterdam, Netherlands, pp. 221–247. 13. Priestley, M. B. (1981). Spectral Analysis and Time Series, Vol. 2. Academic, New York. 14. Shibata, R. (1976). Biometrika, 63, 117–126. 15. Shibata, R. (1980). Ann. Statist., 8, 147–164. 16. Whittle, P. (1963). Prediction and Regulation by Linear Least Squares Methods. English Universities Press, London, England. 17. Wiener, N. (1949). Extrapolation, Interpolation and Smoothing of Stationary Time Series. Wiley, New York. 18. Yamamoto, T. (1976). Appl. Statist., 25, 123–127. See also AUTOREGRESSIVE –INTEGRATED MOVING AVERAGE (ARIMA) MODELS; AUTOREGRESSIVE –MOVING AVERAGE (ARMA) MODELS; BOX –JENKINS MODEL; FORECASTING; KOLMOGOROV, ANDREI NIKOLAYEVICH; MODEL SELECTION: AKAIKE’S INFORMATION CRITERION; PREDICTION AND FORECASTING; STATIONARY PROCESSES; STATISTICAL MODELING; STOCHASTIC PROCESSES; and WIENER, NORBERT.

R. J. BHANSALI

WIENER MEASURE Early studies of the behavior of a particle performing the random type motion first described by the naturalist Robert Brown date back to the first years of the century (Bachelier, Einstein, Smoluchowski; references to authors mentioned in the text but not in the list in fine can be found in the bibliography of Knight’s monograph [3]). The first mathematically satisfactory description of the resulting stochastic process∗ was given by Wiener (1923). Assuming that this Brownian motion∗ {B(t), t 0}, where B(t) represents displacement at time t in a chosen direction, has continuous trajectories (with

9143

probability 1), one is led to consider the space C of continuous functions ω(t), t 0, and seek to determine for C a probability measure W which assigns to sets of continuous functions the very probability that a trajectory of B be one of the functions in the set. Basic are cylinder sets of the form S(t1 , . . . , tn ; A1 , . . . , An ) = {ω : ω(ti ) ∈ Ai , i = 1, . . . , n}, where 0 = t0 < t1 < · · · < tn and the Ai are Borel sets of the line. To such a set W one must assign the weight

dx1 · · · A1

dxn An

n

p(ti − ti−1 ; xi − xi−1 ),

i=1

where p(t; x) = (2π t)−1/2 exp{−x2 /(2t)} is the normal density with mean zero and variance t, at the argument x. This expresses the fact that B passes through the successive ‘‘gates’’ A1 , . . . , An by independent normal increments of variances ti − ti−1 , as postulated for standard Brownian motion. Wiener showed both that B has continuous paths and that there exists exactly one such probability measure W defined on the σ -algebra C generated by the cylinder sets. This W is called Wiener measure. The fundamental importance of this work only became widely appreciated after Kolmogorov, a decade later, set forth the general notion of a probability space (see AXIOMS OF PROBABILITY). The probability space (C, C , W) on which B is defined as the coordinate process B(t, ω) = ω(t), ω ∈ C, constitutes the prototype for the canonical description of a stochastic process∗ , one in which the space of trajectories of the process is probabilized in accordance with the law of that process, which is then realized as a coordinate process. Considering C rather than the subspace of those functions ω with ω(0) = 0 permits assigning to Brownian motion arbitrary initial value B(0) = x. This is done by using instead of W the probability W x defined by W x (A) = W(A + x) with A + x = {ω ∈ C : ω − x ∈ A} and ω − x the function (ω − x)(t) = ω(t) − x. The effective evaluation of a Wiener integral f (ω)W(dω) C

9144

WIENER, NORBERT

is in general difficult. Some classes of integrands f have been studied by Cameron and Martin (1944–1945), a typical example being T for g(ω) = 0 ω2 (t) dt, exp{−λg(ω)}W(dω), λ > 0. (1) C

The Laplace transform of the functional g of the Brownian path is thus determined. Brownian motion is the prime example of a Gaussian process∗ . A Gaussian measure on C is the law of a Gaussian process with continuous trajectories, so Wiener measure constitutes the basic example. Brownian motion over a finite time interval [0, T] has the representation B(t) = 1∞ Zn hn (t), where the Zn are independent standard normal variables and the functions hn are obtained from an arbitrarily chosen complete orthonormal sequence over [0, T]. Using this, Shepp [5] has characterized Gaussian measures on C, equivalent to W (having the same null sets), and computed certain integrals generalizing (1). The random Fourier series for B(t), given when T = 1 by hn (t) = 21/2 sin(n − 12 )π t/[(n − 12 )π ], is 1 well suited2 because it transforms g(ω) = 0 (ω(t) − µ(t)) dt, for square-integrable µ, into a weighted series of noncentral chisquare∗ variables from which (1) is easy to compute. Rothman and Woodroofe [4] have used this for a one-sample test of symmetry relative to 0. Wiener measure is invariant with respect to many 1-1 maps of C. A simple one is ω(t) → tω(1/t), t > 0. Grintsyavicius [2] describes classes of such maps and deduces some Wiener integrals. Diffusions generalize Brownian motion in another direction, being Markov processes with continuous paths. Laws of diffusions are induced on C, from W, by suitable timechange maps, well explained by Freedman [1] (see also DIFFUSION PROCESSES). REFERENCES 1. Freedman, D. (1983). Brownian Motion and Diffusion. Springer, New York. (A good reference to start with.)

2. Grintsyavicius, A. K. (1982). Litov. Math. Sbornik, 22(3), 55–66 (English transl.). (Research paper on transformations preserving W.) 3. Knight, F. (1981). Essentials of Brownian Motion and Diffusion. Amer. Math. Soc., Providence, RI. (A reference monograph.) 4. Rothman, E. D. and Woodroofe, M. (1972). Ann. Math. Statist., 43, 2035–2038. 5. Shepp, L. A. (1966). Ann. Math. Statist., 37, 321–354.

BIBLIOGRAPHY Norbert Wiener special issue (1966). Bull. Amer. Math. Soc., 72(No. 1), Part II. (Review and discussion of Wiener’s work by leading specialists.) See also AXIOMS OF PROBABILITY; BROWNIAN MOTION; GAUSSIAN PROCESSES; DIFFUSION PROCESSES; and STOCHASTIC PROCESSES.

J. P. IMHOF

WIENER, NORBERT Born: November 26, 1894, Columbia, Missouri. Died: March 19, 1964, Stockholm, Sweden. Contributed to: cybernetics, stochastic processes, mathematical physics, communication theory. Norbert Wiener was born in 1894 in the United States. His father, descended from a family of rabbinical scholars, had migrated from Russia and, without a university education, became a professor of Slavic languages at Harvard. Under his father’s influence Norbert Wiener became a child prodigy, entering Tufts College in Boston at 11 and graduating with a Ph.D. from Harvard at 18. His early interest was in natural science, particularly biology, rather than mathematics, but failure in the laboratory led his father to suggest philosophy and his Ph.D. thesis was on mathematical logic. In 1913 he went to Cambridge, England, and was influenced by Bertrand Russell and G. H. Hardy; Wiener claimed the latter was the ‘‘master in my mathematical training.’’ Russell pointed out to Wiener that

WIENER, NORBERT

a mathematical logician might well learn something of mathematics. He published his first mathematical work soon after and his first substantial paper [10] in 1914. After a period as an instructor in philosophy at Harvard and in mathematics at Maine he became an instructor in mathematics at Massachusetts Institute of Technology. Motivated partly by Einstein’s work on Brownian motion he developed the idea of representing each Brownian path as a point in a function space on which a probability measure is defined (see BROWNIAN MOTION and WIENER MEASURE). He showed that almost all paths were continuous but not differentiable. His ideas were presented fully [11] in 1923. They have had an influence on modern probability of the most profound kind, as can be seen from the following. If X(t) denotes a one-dimensional Wiener process, then the increment X(t + δ) − X(t) is independent of X(s), s t, and has a distribution independent of t. As a consequence of this (and the continuity of sample paths) X(t) is Gaussian. If Yk is a sequence of sums of independent identically distributed (i.i.d.) random variables with zero mean and finite variance, then the sequence has the same probabilistic structure as Xk = X(T1 + T2 + · · · + Tk ) for suitable random variables, Tk , that are functions of the Xj , j k, but i.i.d. (see INVARIANCE PRINCIPLES AND FUNCTIONAL LIMIT THEOREMS). Thus much of the limit theory of probability can be studied via X(t). Moreover X(t) itself represents chaotic behaviour, as the previous comment about differentiability shows. Thus dX(t) can intuitively be thought of as the driving term (or input) to a stochastic differential equation∗ (or system) out of which comes a solution with the relatively organised behaviour expected of physical phenomena. In this sense, the phenomena might be regarded as having been explained. In 1926 he began his work on generalized harmonic analysis, which was fully developed in ref. 13. Beginning from the notion of a measurable function f (t), he introduced the autocorrelations 1 γ (t) = lim T→∞ 2T

9145

assumed to exist. From these he constructed what would now be called the spectral distribution function (see SPECTRAL ANALYSIS). Moreover he showed how the mean square γ (0) could be represented as a linear superposition of contributions from every oscillatory frequency, each contribution being the squared modulus of the contribution of that frequency to f (t) itself. Wiener’s characteristically constructive method is here demonstrated and can be contrasted with the axiomatic approach commencing from a probability space and a Hilbert space of square integrable functions, over that space, generated by the action of a unitary group of translations on an individual function. (See ref. 1, pp. 636–637, for some history of this.) Out of his work on generalized harmonic analysis grew his work on Tauberian∗ theory [12], one aspect of which is his celebrated theorem that if f (ω), ω ∈ [−π , π ], has an absolutely convergent Fourier series and f (ω) is never zero, then {f (ω)}−1 also has an absolutely convergent Fourier series. The Wiener–Lévy theorem [4, p. 280] is the natural generalisation of this. His work from 1926 to 1930 was gathered together in his book [14]. In 1933, with Paley named as co-author, he produced ref. 7, Paley being dead at the time of writing. Apart from the (socalled) Paley-Wiener theory (see ref. 4, pp. 173–177, for a discussion), this contained also mathematical results relating to the Wiener-Hopf equation (see below) necessary for Wiener’s later work on linear prediction (see PREDICTION AND FILTERING, LINEAR). For the latter Wiener sought to discover a weight function K(s), that, for a stationary process∗ f (t) with finite variance, minimised T 1 f (t + a) lim T→∞ 2T −T 2 ∞ f (t − s)dK(s) dt, − 0

i.e., that minimised the mean square error in predicting f (t + a) from f (s), s t. He reduced the solution of this to that of the Wiener-Hopf equation ∞ γ (t − s)dK(s) = γ (t), t 0, 0

T −T

f (s + t)f (s) ds,

which had previously arisen in connection with the distribution of stellar atmosphere

9146

WIENER, NORBERT

temperature. Wiener solved this by the methods mentioned in connection with ref. 7. This work was published in ref. 15, publication having been delayed due to restrictions because of a supposed need for military secrecy. At much the same time Kolmogoroff [5, 6] had been working on the same problem. (See ref. 15, p. 59, for a discussion by Wiener of the question of priority.) Though perhaps less constructive, Kolmogoroff’s work, which began from the Wold decomposition [20] of a discrete time stationary process into a purely nondeterministic part and a perfectly predictable part, was in some ways more general. Wiener commenced from a direct representation of f (t + 1) in terms of f (s), s t (i.e., an autoregressive representation), whereas Kolmogoroff commenced from the representation of f (t) in terms of the prediction error or innovation sequence (i.e., moving average∗ representation). Kolmogoroff made use of fundamental results due to Szegö[9], that, for example, express the variance of the innovations as the geometric mean of the spectral density. See also WIENER –KOLMOGOROV PREDICTION THEORY (Supplement). Wiener became well known to the general scientific public for his basically philosophical work on cybernetics [16]. This was concerned with the analogy between man as a selfregulating system, receiving sensory data and pursuing certain objectives, and mechanical or electrical servomechanisms. He made some attempt with Siegel [8] to use his differential space as a basis for a theory of quantum systems. With Masani [18,19] he extended prediction theory to the multivariate case. In ref. 17 he sought to describe a general class of random processes obtained from a Brownian motion as a sum of homogeneous multilinear functionals of that process, of various degrees. This has led to much further research. (See ref. 2.) The breadth of Wiener’s work means that his name occurs throughout mathematics and probability. An example over and above those already mentioned arises in connection with a test for recurrence in a random walk∗ . This is turn relates to potential theory and to diffusion. (See ref. 3, p. 257.) Norbert Wiener died in 1964.

REFERENCES 1. Doob, J. L., (1953). Wiley, New York.

Stochastic

Processes.

2. Hida, T. (1970). Stationary Stochastic Processes. Princeton University Press, Princeton, NJ. 3. Itô, K. and McKean, H. P., (1965). Diffusion Processes and their Sample Paths. SpringerVerlag, Berlin. 4. Katznelson, Y. (1968). An Introduction to Harmonic Analysis. Wiley, New York. 5. Kolmogoroff, A. N. (1939). Sur l’interpolation et extrapolation des suites stationnaires, C.R. Acad. Sci., Paris, 208, 2043–2045. 6. Kolmogoroff, A. N. (1941). Interpolation and extrapolation of stationary random sequences, Izv. Akad. Nauk SSSR, Ser. Math. 5, 3–14. 7. Paley, R. E. A. C. and Wiener, N. (1934). Fourier Transforms in the Complex Domain. Amer. Math. Soc., Providence, RI. 8. Siegel, A. and Wiener, N. (1955). The differential space of quantum systems, Nuovo Cimento (10), 2, 982–1003. ¨ 9. Szegö, G. (1920). Beitrage zur theorie der Toeplitzschen formen. Math. Z. 6, 167–202. 10. Wiener, N. (1914). A simplification of the logic of relations. Proc. Camb. Philos. Soc., 27, 387–390. 11. Wiener, N. (1923a). Differential J. Math. Phys., 2, 131–174.

space,

12. Wiener, N. (1923b). Tauberian theorems, Ann. Math., 33, 1–100. 13. Wiener, N. (1930). Generalized harmonic analysis, Acta Math., 55, 117–258. 14. Wiener, N. (1933). The Fourier Integral and Certain of its Applications. C.U.P., New York. 15. Wiener, N. (1949). Extrapolation, Interpolation and Smoothing of Stationary Time Series. The MIT Press, Cambridge, Mass. and Wiley, New York. 16. Wiener, N. (1950). The Human Use of Human Beings. Houghton Mifflin, Boston. 17. Wiener, N. (1958). Nonlinear Problems in Random Theory. The MIT Press, Cambridge, Mass. and Wiley, New York. 18. Wiener, N. and Masani, P. (1957). The prediction theory of multivariate stochastic processes. I. The regularity condition, Acta Math., 98, 111–150. 19. Wiener, N. and Masani, P. (1958). The prediction theory of multivariate stochastic processes, II. The linear predictor. Acta Math., 99, 93–137.

WILCOXON, FRANK 20. Wold, H. (1938). A Study in the Analysis of Stationary Time Series. Almqvist and Wiksell, Uppsala, Sweden.

Bibliographical Notes Wiener wrote a most interesting two volume autobiography that serves to give some impression of the greatness of his mind. The volumes are: Wiener, N. (1953). Ex-prodigy: My Childhood and Youth. Simon and Schuster, New York. Wiener, N. (1956). I Am a Mathematician. The Later Life of a Prodigy. Doubleday, Garden City, NY. In 1966 the American Mathematical Society devoted a special edition of its Bulletin (Vol. 72, No. 1, Part II) to Wiener. This contains articles by distinguished scholars about Wiener and his work, including a very interesting article about the man himself by N. Levinson and a complete bibliography of his writing.

E. J. HANNAN

WIENER PROCESS. See BROWNIAN MOTION

WIJSMAN’S REPRESENTATION WIJSMAN’S THEOREM. See INVARIANTS, REPRESENTATION

WILCOXON, FRANK Born: September 2, 1892, County Cork, Ireland. Died: November 18, 1965, Tallahassee, Florida. Contributed to: (statistics) rank tests, multiple comparisons, sequential ranking, factorial design; (chemistry) fungicidal action, synthesis of plant growth substances, insecticides research (pyrethrins, Parathion, Malathion). Frank Wilcoxon was born in Glengarriffe Castle, near Cork, Ireland, to wealthy American parents on September 2, 1892. His father was a poet, outdoorsman, and hunter. Wilcoxon spent his boyhood at Catskill, New York and developed his lasting love of nature and water there. Adolescence seems to have been difficult, with a runaway period during

9147

which he was briefly a merchant seaman in New York harbor, a gas pumping station attendant in an isolated area of West Virginia, and a tree surgeon. As this period ended, he was enrolled in Pennsylvania Military College and, although the school’s system did not agree with his ideas of personal freedom, he received the B.S. degree in 1917. Wilcoxon entered Rutgers University in 1920 after a World War I position with the Atlas Powder Company at Houghton, Michigan and received the M.S. degree in chemistry in 1921. He continued his education at Cornell University and received the Ph.D. degree in physical chemistry in 1924. At Cornell, he met Frederica Facius and they were married on May 27, 1926. Frank and Freddie later became well known and loved in the statistical community, particularly through their regular participation in the Gordon Research Conference on Statistics in Chemistry and Chemical Engineering. From 1924 to 1950, Frank Wilcoxon was engaged in research related to chemistry. In 1925 he went to the Boyce Thompson Institute for Plant Research under a Crop Protection Institute fellowship sponsored by the Nichols Copper Company and worked on colloid copper fungicides until 1927. He then (1928–1929) worked with the sponsoring company in Maspeth in Queens. In 1929 he returned to the Boyce Thompson Institute and remained there until 1941, working on the chemistry and mode of action of fungicides and insecticides. With a leave of absence from 1941 to 1943, he designed and directed the Control Laboratory of the Ravenna Ordnance Plant operated by the Atlas Powder Company. Wilcoxon joined the American Cyanamid Company in 1943 and continued with that company until his retirement in 1957, first with the Stamford Research Laboratories as head of a group developing insecticides and fungicides and then as head of the statistics group of the Lederle Division in Pearl River, New York. He served as a consultant to various organizations, including the Boyce Thompson Institute, until 1960, when he joined the faculty of the new Department of Statistics at Florida State University. There he was active in research and teaching and contributed to the development of the department until his death in 1965.

9148

WILCOXON, FRANK

Frank Wilcoxon made many contributions to chemistry and biochemistry with some 40 publications in the field. His first paper [13] on acidimetry and alkalimetry was published in 1923 in Industrial and Engineering Chemistry with van der Meulen. A series of papers with S. E. A. McCallan was written on the mode of action of sulphur and copper fungicides. A series of papers with Albert Hartzell dealt with extracts of pyrethrum flowers and resulted in a mercury reduction method for the determination of pyrethrin I. Also, in his work with the Boyce Thompson Institute, he synthesized a number of plant growth substances, including alphanaphthaleneacetic acid, which were studied with P. W. Zimmerman and led to a series of papers on the action of these growth substances on plants. In his work with the American Cyanamid Company he led the research group that studied the insecticide Parathion and which developed the less toxic Malathion. Wilcoxon’s interest in statistics began in 1925 with a study of R. A. Fisher’s∗ book, Statistical Methods for Research Workers. This study was done in a small reading group in which his colleague at the Boyce Thompson Institute, W. J. Youden∗ , participated also. Wilcoxon became increasingly interested in the application of statistics in experimentation and this is apparent in a number of his papers on spore germination tests. His first publication in a statistics journal [14] dealt with uses of statistics in plant pathology. His research contributions in statistics were to range over rank tests∗ , multiple comparisons∗ , sequential ranking∗ (see RANK STATISTICS), factorial experiments∗ , and bioassay∗ . Throughout his research, he sought statistical methods that were numerically simple and easily understood and applied. Wilcoxon introduced his two rank tests, the rank-sum test for the two-sample problem and the signed-rank test for the pairedsamples problem, in his 1945 paper [15]. This paper and a contemporary one by Mann and Whitney [10] led the way to the extensive development of nonparametric statistics, including Wilcoxon’s own further contributions. (See WILCOXON SIGNED RANK TEST; MANN –WHITNEY –WILCOXON

STATISTIC; LINEAR RANK TESTS; WILCOXONTYPE TESTS FOR ORDERED ALTERNATIVES IN RANDOMIZED BLOCKS.) There is no doubt that this paper was Wilcoxon’s most important contribution. Although there were independent proposals of the two-sample statistic (see Kruskal [9] for historical notes), the paper became a major inspiration for the development of nonparametric methods. In addition, the statistical methodology introduced has had a major impact in applied statistics, particularly for applications in the social sciences, becoming one of the most popular of statistical tools. Research on Wilcoxon test theory continued. He, himself, provided additional tables [16] in 1947 and again, with Katti and Wilcox [20], in 1963. Serfling [12] studied the properties of the two-sample ranksum test in a setting where dependence is allowed within samples. Hollander et al. [8] studied the properties of the rank-sum test in a model where the assumption of independence between samples is relaxed. Hettmansperger [7] provides a modern treatment of nonparametric inference based on ranks. Wilcoxon was interested in extensions of his basic rank procedures to new situations. He was an initiator of research in nonparametric sequential methods. Since nonparametric techniques were so successful in fixed sample-size situations, he felt that their good properties would naturally carry over to the sequential setting. This idea led to a number of sequential rank procedures developed under the leadership of Wilcoxon and Bradley [2,3,17,18]. Other researchers continued the development of nonparametric sequential methods after this early work by Bradley and Wilcoxon and a comprehensive development of the theory is given by Sen [11]. Wilcoxon was interested also in the problem of testing whether two p-variate populations, p 2, are equivalent and two of his proposals for this problem motivated refs. 1 and 4. In experiments in the natural sciences and in the behavioral sciences, typically there are more than just one or two conditions (treatments, etc.) under investigation. Often multiple hypotheses need to be tested. In such settings, it is important to control

WILCOXON, FRANK

the overall or experimentwise error rate. Wilcoxon recognized this and had a strong interest in multiple comparisons. In particular, his 1964 revision of the booklet [19] (joint with Roberta A. Wilcox) Some Rapid Approximate Statistical Procedures, features multiple comparison procedures based on Wilcoxon rank sums for the one-way layout and multiple comparison procedures based on Friedman rank sums for the two-way layout. The booklet played a significant role in the (now) widespread use of nonparametric multiple comparison procedures. In the early 1960s Wilcoxon also suggested and largely directed a dissertation on multiple comparisons by Peter Dunn-Rankin, part of which was published as a joint paper [6]. Other areas of Wilcoxon research seem less well known. From his research on insecticides and fungicides, he, with J. T. Litchfield, Jr. and K. Nolen, developed an interest in and a series of papers on a simplified method of evaluating dose-effect experiments. Daniel and Wilcoxon [5] devised fractional factorial designs∗ robust against linear and quadratic trends, anticipating to some degree the concept of trend-free block designs. While Frank Wilcoxon was not an academician for much of his career, he was a teacher and a student throughout his life. His enthusiasm for statistics and his encouragement of others led many to more intensive study of the subject. It was typical of the man that, prior to visits to Russia in 1934 and 1935, he undertook a study of the language and retained a remarkably proficient reading knowledge throughout his life. Frank Wilcoxon was recognized by his associates. He was a Fellow of the American Statistical Association and of the American Association for the Advancement of Science. He was a leader in the development of the Gordon Research Conference on Statistics in Chemistry and Chemical Engineering and a past Chairman of that Conference. Karas and Savage list the publications of Wilcoxon. This and other material are listed in the bibliography. REFERENCES 1. Bradley, R. A. (1967). Proc. Fifth Berkeley Symp., Vol. 1, L. LeCam and J. Neyman,

9149

eds. Univ. of Calif. Press, Berkeley, CA, pp. 593–605. 2. Bradley, R. A., Martin, D. C., and Wilcoxon, F. (1965). Technometrics, 7, 463–483. 3. Bradley, R. A., Merchant, S. D., and Wilcoxon, F. (1966). Technometrics, 8, 615–623. 4. Bradley, R. A., Patel, K. M., and Wackerly, D. D. (1971). Biometrics, 27, 515–530. 5. Daniel, C. and Wilcoxon, F. (1966). Technometrics, 8, 259–278. 6. Dunn-Rankin, P. and Wilcoxon, F. (1966). Psychometrika, 31, 573–580. 7. Hettmansperger, T. P. (1984). Statistical Inference Based on Ranks. Wiley, New York. 8. Hollander, M., Pledger, G., and Lin, P. (1974). Ann. Statist., 2, 177–181. 9. Kruskal, W. H. (1957). J. Amer. Statist. Ass., 52, 356–360. 10. Mann, H. and Whitney, D. R. (1947). Ann. Math. Statist., 18, 50–60. 11. Sen, P. K. (1981). Sequential Nonparametrics. Wiley, New York. 12. Serfling, R. J. (1968). Ann. Math. Statist., 39, 1202–1209. 13. van der Meulen, P. A. and Wilcoxon, F. (1923). Ind. and Eng. Chem., 15, 62–63. 14. Wilcoxon, F. (1945). Biometrics Bull., 1, 41–45. 15. Wilcoxon, F. (1945). Biometrics Bull., 1, 80–83. 16. Wilcoxon, F. (1947). Biometrics, 3, 119–122. 17. Wilcoxon, F. and Bradley, R. A. (1964). Biometrics, 20, 892–895. 18. Wilcoxon, F., Rhodes, L. J., and Bradley, R. A. (1963). Biometrics, 19, 58–84. 19. Wilcoxon, F. and Wilcox, R. A. (1964). Some Rapid Approximate Statistical Procedures. Stamford Research Laboratories, Pearl River, New York. (Revision of a 1947, revised 1949, booklet by Wilcoxon.) 20. Wilcoxon, F., Katti, S. K., and Wilcox, R. A. (1970). Selected Tables in Mathematical Statistics, Vol. 1, H. L. Harter and D. B. Owen, eds. Markham, Chicago, IL, pp. 171–259. (Originally prepared and distributed in 1963, revised 1968, by Lederle Laboratories, Pearl River, New York and the Department of Statistics, Florida State University, Tallahassee, Florida.)

BIBLIOGRAPHY Anon. (1965). New York Times, Nov. 19, p. 39, col. 2.

9150

WILCOXON RANK-SUM TEST

Bradley, R. A. (1966). Biometrics, 22, 192–194. Bradley, R. A. (1966). Amer. Statist., 20, 32–33.

WILCOXON SIGNED RANK TEST

Bradley, R. A. and Hollander, M. (1978). International Encyclopedia of Statistics, Vol. 2, W. H. Kruskal and J. M. Tanur, eds. The Free Press, New York, pp. 1245–1250. Dunnett, C. W. (1966). Technometrics, 8, 195–196. Karas, J. and Savage, I. R. (1967). Biometrics, 23, 1–11. McCallan, S. E. A. (1966). Boyce Thompson Institute for Plant Research, Contributions, 23, 143–145.

Wilcoxon [33] proposed a simple yet powerful test for the location of a symmetric population. (It is also applicable to paired data, as will be illustrated later.) Suppose Z1 , . . . , Zn denotes a random sample from a population that is continuous and symmetric around θ . We want to test whether or not the population is located at some specific number θ0 , i.e., the null hypothesis is H0 : θ = θ0 . The first step is to subtract θ0 from every Zi , forming

See also BIOASSAY, STATISTICAL METHODS IN; FACTORIAL EXPERIMENTS; FISHER, RONALD AYLMER; LINEAR RANK TESTS; MANN –WHITNEY –WILCOXON STATISTIC; MULTIPLE COMPARISONS; RANK STATISTICS; RANK TESTS; SEQUENTIAL RANK ESTIMATORS; SIGNED-RANK STATISTICS; TREND-FREE BLOCK DESIGNS; WILCOXON SIGNED RANK TEST; WILCOXON-TYPE SCALE TESTS; and WILCOXON-TYPE TESTS FOR ORDERED ALTERNATIVES IN RANDOMIZED BLOCKS.

Z 1 − θ0 , Z2 − θ0 , . . . , Z n − θ0 .

RALPH A. BRADLEY MYLES HOLLANDER

WILCOXON RANK-SUM TEST. See MANN –WHITNEY –WILCOXON STATISTIC

Now find the absolute values of these differences and rank the absolute values from smallest to largest. (Assign the rank of 1 to the smallest, rank 2 to the second smallest, . . ., rank n to the largest.) The test statistic T + is the sum of the ranks which correspond to positive values of Zi − θ0 . To test H0 : θ = θ0 against Ha : θ > θ0 (θ < θ0 or θ = θ0 ), we reject H0 whenever T + kα [T + n(n + 1)/2 − kα or |T + − n(n + 1)/4| kα/2 − n(n + 1)/4]. Here kα represents the value satisfying

WILCOXON SCORES These are used in the construction of linear rank statistics∗ estimating location. The scores are the coefficients an1 , an2 , . . . , anm of the order statistics∗ X1 X2 · · · Xn in the statistic n

ani Xi .

i=1

For Wilcoxon scores ani = i. REFERENCE 1. Randles, R. H. and Wolfe, D. A. (1979). Introduction to the Theory of Nonparametric Statistics. Wiley, New York. See also L-STATISTICS; NORMAL SCORES TESTS; and SAVAGE SCORES.

α = P[T + kα when H0 is true]. Values of kα may be found, for example, in Table A.4 of Hollander and Wolfe [14]. When the sample size n is large, kα may be aproximated by means of . n(n + 1) kα = 4 n(n + 1)(2n + 1) 1/2 +Zα , 24 where Zα is the point on a standard normal distribution with probability α above it. This test procedure applies to paired data also. Let (X1 , Y1 ), . . . , (Xn , Yn ) denote a random sample of paired observations from some bivariate population. For instance, Xi might denote a measurement on the ith person before treatment and Yi , the same type of measurement made after treatment. Analysis of these settings often focuses on the magnitude of the treatment effect which can


be measured by forming Z1 = Y1 − X1 , Z2 = Y2 − X2 , . . . , Zn = Yn − Xn . The Z’s are analyzed in the fashion described earlier. The assumption of symmetry on the distribution of Zi is very appropriate in these cases, since it is often assumed that, aside from the treatment effect θ , the two measurements would be interchangeable. That is, the pair (Xi , Yi − θ ) would have the same distribution as (Yi − θ , Xi ). Typically, in these problems the null hypothesis corresponds to no treatment effect, that is, θ0 = 0. We illustrate the Wilcoxon signed rank test on data collected by Faria and Elliott [6] while investigating the existence of a 23-day physical biorhythm pattern. Eight female gymnasts had their physical performance recorded during periods which were predicted to be ones of high and low physical performances according to the individual athlete’s biorhythm sine curve pattern. The study was double blind in that neither the athlete nor the person recording performance knew the state of the athlete’s biorhythm pattern. The data in Table 1 show the maximum oxygen uptake (VO2 max) for each athlete while on a treadmill test (averaged over two such measurements) during both high and low biorhythm periods. In this example the null hypothesis is ‘‘no biorhythm pattern differences between high and low periods, H0 : θ = 0’’, and the alternative would be increased oxygen uptake during high biorhythm periods, i.e., Ha : θ > 0. With a significance level of α = 0.098, Table A.4 in Hollander and Wolfe [14] yields kα = 28. Thus, we reject H0 , if T + 28. Table 1 shows that the sum of the ranks corresponding to positive differences is T + = 7 + 8 + 6 + 4 = 25. Therefore the data are not sufficiently convincing that VO2 max levels during high biorhythm periods exceed those during low periods. The Wilcoxon signed rank test is distribution-free∗ in that its null distribution is appropriate under a very weak set of assumptions about the Zi population, namely

9151

that the population is symmetric and continuous. Extensive tables for the null distribution are given by Wilcoxon et al. [34]. Other large sample approximations for the null distribution are discussed by Fellingham and Stoker [7] and Bickel [3]. The one-sided test is locally most powerful for detecting a shift in a logistic distribution among tests based on the signs and the ranks of the absolute values. (See Fraser [8].) It is also asymptotically locally efficient in the same setting among all ´ α-level tests. (See Hajek and Sidak [10], p. 279.) Its tolerance to bad data is also good. (See Hettmansperger [11], p. 124.) The test has excellent efficiencies, as derived by Pitman [19]. Hodges and Lehmann [12] show that for continuous symmetric distributions, its Pitman efficiency∗ relative to the one-sample t-test∗ has a lower bound of 0.864. This efficiency for some common underlying distributions takes on the value 1.0 for a uniform distribution, 3/π for a normal, π 2 /9 for a logistic, 1.5 for a double exponential, and +∞ for a Cauchy. The test’s Bahadur efficiency∗ was established by Klotz [16]. Small sample efficiencies were investigated by Klotz [16] and Arnold [1]. Examples of its performance in Monte Carlo∗ studies may be found in Randles and Wolfe [23] and Nath and Duran [18]. Handling ties and zeros is studied by Pratt [20] and Cureton [5]. Also see Putter [21]. The performance under serial correlation∗ is investigated by Gastwirth and Rubin [9]. Extensions of this method to other settings have been investigated by many authors. The confidence interval corresponding to the Wilcoxon signed rank test was proposed by Tukey [30] and described by Moses in Chapter 18 of Walker and Lev [31]. Also, see Sen [26] and Lehmann [17] for further properties and extensions of this interval. Hodges and Lehmann [13] develop the point estimator corresponding to the test. Quade [22] and Salama and Quade [24] extend the method to randomized block∗ experiments. Two-stage and sequential versions have also been proposed. (See Weed and Bradley [32], Spurrier and Hewett [29], and Sen [27].) Multivariate extensions were proposed by Bickel [2], Sen and Puri [28], Killeen and Hettmansperger [15], and Brown and

9152


Table 1. VO2 max of Athletes During High and Low Biorhythm Periods Person

Yi High Period

Xi Low Period

Zi = Yi − Xi Difference

Abs. Value of Diff.

Rank of Abs. Value

Sign

A B C D E F G H

50.89 40.37 43.60 40.75 41.23 41.38 53.01 45.04

44.12 42.21 35.77 43.09 41.83 42.69 48.84 42.87

6.77 −1.84 7.83 −2.34 −0.60 −1.31 4.17 2.17

6.77 1.84 7.83 2.34 0.60 1.31 4.17 2.17

7 3 8 5 1 2 6 4

+ − + − − − + +

Hettmansperger [4]. Extensions to censored data∗ have been investigated by Woolson and Lachenbruch [35] and Schemper [25]. REFERENCES 1. Arnold, H. (1965). Ann. Math. Statist., 36, 1767–1778. (Small sample power of signed rank test.) 2. Bickel, P. J. (1965). Ann. Math. Statist., 36, 160–173. (Marginal signed rank statistics used to construct a multivariate test.) 3. Bickel, P. J. (1974). Ann. Statist., 2, 1–20. (Uses Edgeworth expansion to approximate the null distribution of rank statistics.) 4. Brown, B. M. and Hettmansperger, T. P. (1985). Tech. Report, Dept. of Statist., Penn. State Univ., University Park, PA. (Use bivariate quantiles to develop an affine invariant analogue to the signed rank test for the bivariate location problem.) 5. Cureton, E. E. (1967). J. Amer. Statist. Ass., 62, 1068–1069. (Large sample normal approximation for the signed rank test in the presence of zeros and ties.) 6. Faria, I. E. and Elliott, T. L. (1980). J. Sports Med. Phys. Fitness, 20, 81–85. (Source of the data for the example.) 7. Fellingham, S. A. and Stoker, D. J. (1964). J. Amer. Statist. Ass., 59, 899–905. (Approximates the null distribution of the signed rank test.) 8. Fraser, D. A. S. (1957). Ann. Math. Statist., 28, 1040–1043. (Shows the locally most powerful property of the signed rank test among tests based on the signs and ranks of the absolute values.) 9. Gastwirth, J. L. and Rubin, H. (1971). J. Amer. Statist. Ass., 66, 816–820. (Effects of serial correlation among data on the signed rank test.)

´ 10. Hajek, J. and Sidak, Z. (1967). Theory of Rank Tests. Academic, New York. 11. Hettmansperger, T. P. (1984). Statistical Inference Based on Ranks. Wiley, New York. (Advanced text mixing applications and theory of rank tests and estimators.) 12. Hodges, J. L., Jr. and Lehmann, E. L. (1956). Ann. Math. Statist., 27, 324–35. (Discusses properties of the Pitman efficiency of the signed rank test compared to the one-sample t-test.) 13. Hodges, J. L., Jr. and Lehmann, E. L. (1963). Ann. Math. Statist., 34, 598–611. (Source of point estimators of θ corresponding to signed rank test.) 14. Hollander, M. and Wolfe, D. A. (1973). Nonparametric Statistical Methods. Wiley, New York. (Introductory text containing description and tables for the Wilcoxon signed rank test, as well as references to related work.) 15. Killeen, T. J. and Hettmansperger, T. P. (1972). Ann. Math. Statist., 43, 1507–1516. (Proposes a multivariate signed rank test and compares tests via Bahadur efficiency.) 16. Klotz, J. (1963). Ann. Math. Statist., 34, 624–632. (Small sample power and efficiency of the signed rank test.) 17. Lehmann, E. L. (1963). Ann. Math. Statist., 34, 1507–1512. (Develops the asymptotic properties of the confidence interval corresponding to the signed rank test.) 18. Nath, R. and Duran, B. S. (1984). J. Statist. Comp. Simul., 20, 235–260. (t tests compared to T + in a Monte Carlo.) 19. Pitman, E. J. G. (1948). Notes on Nonparametric Statistical Inference, Columbia Univ., New York. (Finds the local asymptotic efficiency of the Wilcoxon signed rank test relative to the test based on the sample mean.) 20. Pratt, J. W. (1959). J. Amer. Statist. Ass., 54, 655–667. (Discusses methods for handling zeros in the signed rank test.)

WILCOXON-TYPE SCALE TESTS 21. Putter, J. (1955). Ann. Math. Statist., 26, 368–386. (Asymptotic comparison of the randomized and nonrandomized treatments of ties, in the contents of the sign and two-sample rank sum tests.) 22. Quade, D. (1979). J. Amer. Statist. Ass., 74, 680–683. (Uses interblock information in the fashion of the signed rank test but in a manypopulation setting.) 23. Randles, R. H. and Wolfe, D. A. (1979). Introduction to the Theory of Nonparametric Statistics. Wiley, New York. (Develops the theory behind rank tests and estimates.) 24. Salama, I. A. and Quade, D. (1981). Commun. Statist. A, 10, 385–399. (Tests for ordered alternatives in complete blocks with interblock information included.) 25. Schemper, M. (1984). Commun. Statist. A, 13, 681–684. (Generalizes the signed rank test to interval and censored data.) 26. Sen, P. K. (1963). Biometrics, 19, 532–52. (Applies the estimation structure corresponding to the signed rank test to the estimation of relative potency in dilution assays.) 27. Sen, P. K. (1981). Sequential Nonparametrics. Wiley, New York. (Describes sequential rank tests, estimators, and corresponding theory.) 28. Sen, P. K. and Puri, M. L. (1967). Ann. Math. Statist., 38, 1216–1228. (Develops theory for a class of permutationally distribution-free rank order statistics for the multivariate onesample location problem.) 29. Spurrier, J. D. and Hewett, J. E. (1976). J. Amer. Statist. Ass., 71, 982–987. (Two stage signed rank test proposed.) 30. Tukey, J. W. (1949). Memo Rep. 17, Statistical Research Group, Princeton Univ., Princeton, NJ. (Source of confidence interval for θ from the signed rank test.) 31. Walker, H. M. and Lev, J. (1953). Statistical Inference, 1st ed. Holt, Rinehart and Winston, New York. (Chapter 18, written by L. Moses, includes a description of the confidence interval corresponding to the signed rank test.) 32. Weed, H. D., Jr. and Bradley, R. A. (1971). J. Amer. Statist. Ass., 66, 321–326. (Sequential grouped signed rank tests are proposed.) 33. Wilcoxon, F. (1945). Biometrics, 1, 80–83. (Signed rank and rank sum tests first introduced.) 34. Wilcoxon, F., Katti, S. K., and Wilcox, R. A. (1970). Selected Tables in Mathematical Statistics, Vol. I, Harter, H. L. and Owen, D. B. eds. Markham, Chicago, IL, pp. 171–259. (Null distribution critical values for n 50.)

9153

35. Woolson, R. F. and Lachenbruch, P. A. (1980). Biometrika, 67, 597–600. (Rank tests for location are discussed for censored matched pairs.) See also DISTRIBUTION-FREE METHODS; RANK STATISTICS; SIGNED-RANK STATISTICS; and TUKEY’S CONFIDENCE INTERVAL FOR LOCATION.

RONALD H. RANDLES

WILCOXON-TYPE SCALE TESTS The Wilcoxon (or Mann-Whitney∗ ) test is well known as a distribution-free∗ test of location for two mutually independent random samples. However, the basic idea of this test can also be used to produce tests which are sensitive to scale∗ differences; these are frequently called Wilcoxon-type scale tests. The test applied to absolute values of deviations from central tendency is an exact analog of the Wilcoxon test and the Sukhatme [10] test separates the deviations into two groups according to sign and then applies the Wilcoxon test to each group separately. A distinct advantage of these scale tests over most others is that they have a corresponding confidence interval procedure to estimate the ratio of scale parameters (see Noether [9]). These test and estimation procedures are rarely covered in books on nonparametric methods; Gibbons [4,5] and Noether [8] are the exceptions. NOTATION AND ASSUMPTIONS Let X1 , X2 , . . . , Xm and Y1 , Y2 , . . . , Yn denote mutually independent random samples from continuous distributions Fx and Fy with known medians Mx and My but unknown scale parameters σx and σy , respectively. Assume the scale distribution model that (X − Mx )/λ and (Y − My ) are identically distributed with λ = σx /σy . The tests covered here are for H0 : λ = λ0 where λ0 can be any positive constant but is frequently equal to 1. The corresponding confidence interval procedures give estimates of λ. TEST PROCEDURES Both test procedures use the transformed variables Vi = (Xi − Mx )/λ0 and Wj = (Yj − My ).

9154

WILCOXON-TYPE SCALE TESTS

For the first Wilcoxon-type test, we pool the m + n absolute values |V1 |, |V2 |, . . . , |Vm | and |W1 |, |W2 |, . . . , |Wn |, array them from smallest to largest, and assign ranks 1, 2, . . . , m + n according to their relative magnitudes. The test statistic is TV , the sum of the ranks assigned to the m absolute values of V’s. If we assume that X and Y have symmetric distributions, the test based on TV is consistent against scale alternatives. A large value of TV supports the alternative λ > λ0 and a small value supports the alternative λ < λ0 . Under H0 , the distribution of TV is the same as the Wilcoxon rank sum test for location and exact tables are widely available. The most extensive tables are in Wilcoxon et al. [14], but most nonparametric textbooks also have tables. For larger samples we can use the fact that TV is asymptotically normal with mean m(m + n + 1)/2 and variance mn(m + n + 1)/12 under H0 . A continuity correction of 0.5 can be incorporated in the test statistic. The asymptotic relative efficiency of this test relative to the F test is 6/π 2 for normal distributions. For the Sukhatme [10] test, the positive V’s and W’s are separated from the negative V’s and W’s. The positives are arrayed and ranked, and the absolute values of the negatives are arrayed and ranked. Let T+ be the sum of the ranks of the V’s in the positive group and T− the sum of the ranks of the V’s in the negative group. Then the test statistic is TV = T+ + T− . A large value of TV supports the alternative λ > λ0 and a small value supports λ < λ0 . For large samples, the normal approximation can be used with mean m(m + n + 2)/4 and variance mn(m + n + 7)/48. For small samples, the distribution of TV depends on the numbers of positive and negative deviations, but Laubscher and Odeh [7] have generated the null distribution of a statistic which is a linear function of TV and is in fact the original form of the Sukhatme [10] statistic. This statistic is

where

1 Dij = 0

if Wj < Vi < 0 or 0 < Vi < Wj , otherwise.

This form shows clearly that T is a modified form of the Mann-Whitney∗ statistic while TV corresponds to the Wilcoxon statistic. Sukhatme [10] showed that T is consistent against the scale alternative without any assumption of symmetry. For the statistic T, large values support the alternative λ < λ0 and small values support λ > λ0 . Laubscher and Odeh [7] give tables of the critical values for 2 m, n 10 and the corresponding complete null distributions are available from these authors. For large samples we can use the normal approximation with mean mn/4 and variance mn(m + n + 7)/48. Laubscher and Odeh [7] show that this normal approximation is adequate for m and n larger than 10 if a continuity correction of 0.5 is used. The asymptotic relative efficiency∗ of this test relative to the F-test∗ is 6/π 2 for normal distributions. TEST PROCEDURES FOR MEDIANS UNKNOWN If the medians Mx and My are unknown, a natural approach would be to substitute the respective sample medians in calculating V and W and then proceed as before. Fligner and Killeen [3] showed that the first Wilcoxon-type test with sample medians substituted remains distribution-free, has the same consistency properties, and the same asymptotic relative efficiency as when the medians are assumed known. Sukhatme [11] showed that his test statistic T˜ computed from sample medians is asymptotically distribution-free for bounded and symmetric distributions and found the test highly efficient for exponential distributions. Laubscher and Odeh [7] used a simulation study to assess the accuracy of using their tables of T with the statistic T˜ and found this procedure quite conservative for the normal∗ , uniform∗ , and Cauchy∗ distributions. CONFIDENCE INTERVAL PROCEDURES

T=

m n i=1 j=1

Dij ,

A confidence interval estimate of λ that assumes symmetry and corresponds to the

WILCOXON-TYPE SCALE TESTS

first Wilcoxon-type test is constructed by forming the mn ratios |Xi − Mx |/|Yj − My |. These ratios are then arranged from smallest to largest and the confidence interval endpoints are the k th smallest and k th largest ratios. For m and n small, k is the rank of a left-tail critical value at level α/2 of the Wilcoxon test statistic found in a table that gives the complete null distribution. The confidence coefficient is then 1 − α. For larger samples with m, n 12, we can estimate k by / mn mn(m + n + 1) k = 0.5 + − Zα/2 , 2 12 where Zα/2 satisfies (Zα/2 ) = 1 − α/2. If k is not an integer it should always be rounded downward for a conservative result. The confidence interval estimate of λ that corresponds to the Sukhatme test is constructed by forming the M ratios (Xi − Mx )/(Yj − My ) which are positive because both the numerator and denominator have the same sign. Of course, M mn and approaches (m/2)(n/2). These positive ratios are then arrayed and the confidence interval endpoints are the kth smallest and kth largest of the M. The correct value of k can be found from the tables in Laubscher and Odeh [7]. For large samples the value of k can be approximated by / mn(m + n + 7) mn − Zα/2 . k = 0.5 + 4 48 A third type of confidence interval estimate of λ which is also a Wilcoxon-type procedure can be used if the X and Y variables can take on only positive values. Here σx and σy should be regarded as measures of variability from zero as opposed to spread about a central value. The procedure is to form the mn ratios Xi /Yj and array them. The confidence interval endpoints are the kth smallest and kth largest ratio where k is the same as in the first Wilcoxon-type confidence procedure. MODIFICATIONS AND EXTENSIONS Sukhatme [12] proposed another modification of his statistic. More recently, Deshpande and Kusum [1] proposed a generalization of Sukhatme’s test for distributions with

9155

an equal quantile which is not necessarily the median. The Kruskal-Wallis test∗ is the generalization of the Wilcoxon test for more than two samples. It could be used to detect scale differences if the data are transformed to absolute values of deviations from their respective medians. The performance of this procedure is compared with some other non-parametric c sample scale tests in Tsai et al. [13]. An excellent comprehensive survey of nonparametric tests for scale is given in Duran [2]. REFERENCES 1. Deshpande, J. V. and Kusum, K. (1984). Austral. J. Statist., 26, 16–24. 2. Duran, B. S. (1976). Commun. Statist. A, 5, 1287–1312. (Comprehensive survey of nonparametric tests for scale.) 3. Fligner, M. A. and Killeen, T. J. (1976). J. Amer. Statist. Ass., 71, 210–213. 4. Gibbons, J. D. (1985a). Nonparametric Methods for Quantitative Analysis. American Sciences Press, Columbus, OH. (Scale tests and confidence interval procedures are covered in Chap. 5.) 5. Gibbons, J. D. (1985b). Nonparametric Statistical Inference. Dekker, New York. (Scale tests and confidence interval procedures are covered in Chap. 10.) 6. Kochar, S. C. and Gupta, R. P. (1986). Commun. Statist. A, 15, 231–239. 7. Laubscher, N. F. and Odeh, R. E. (1976). Commun. Statist. A, 5, 1393–1407. 8. Noether, G. E. (1967). Elements of Nonparametric Statistics. Wiley, New York. 9. Noether, G. E. (1972). Amer. Statist., 26, 39–41. 10. Sukhatme, B. V. (1957). Ann. Math. Statist., 28, 188–194. 11. Sukhatme, B. V. (1958a). Ann. Math. Statist., 29, 60–78. 12. Sukhatme, B. V. (1958b). Biometrika, 45, 544–548. 13. Tsai, W. S., Duran, B. S., and Lewis, T. O. (1975). J. Amer. Statist. Ass., 70, 791–796. 14. Wilcoxon, F., Katti, S. K., and Wilcox, R. A. (1972). Selected Tables in Mathematical Statistics, Vol. I. American Mathematical Society, Providence, RI, pp. 171–259. See also DISTRIBUTION-FREE METHODS; KLOTZ TEST; MANN –WHITNEY –WILCOXON STATISTIC; RANK TESTS;

9156

WILCOXON-TYPE TESTS FOR ORDERED ALTERNATIVES IN RANDOMIZED BLOCKS

SCALE, CAPON TEST OF; SCALE TESTS; SCALE TESTS, ANSARI —BRADLEY; WILCOXON SCORES; WILCOXON SIGNED RANK TEST; and WILCOXON-TYPE TESTS FOR ORDERED ALTERNATIVES IN RANDOMIZED BLOCKS.

JEAN DICKINSON GIBBONS

section can be applied by simply relabeling the treatments so that the X values corresponding to treatment j1 are called the Xi1 ’s, the X values corresponding to treatment j2 are called the Xi2 ’s, and so forth.


SOME NONPARAMETRIC METHODS BASED ON RANKS

INTRODUCTION

Tests Based on Average Rank Correlation Coefficients

Consider the typical two-way layout consisting of n blocks, k treatments, and one observation per cell. The data can be expressed as

Let rij , i = 1, . . . , n, j = 1, . . . , k, be the rank of k X ijn in the joint ranking of {Xiα }α=1 . Let Rj = i=1 rij , j = 1, . . . , k. Then Page’s [15] statistic can be written as

Xij = βi + θj + eij , i = 1, . . . , n; j = 1, . . . , k,

L=

(1)

k

jRj .

(4)

j=1

where βi denotes the ith block effect, θj , the jth treatment effect, and the {eij } are independent and identically distributed (i.i.d.) random variables according to the unknown continuous distribution F. In this entry, we review some competing nonparametric tests of H0 : θ1 = θ2 = · · · = θk

(2)

Ha : θ1 θ2 · · · θk ,

(3)

versus

where at least one of the inequalities is strict. Procedures discussed include (i) Page’s∗ [15] and Jonckheere’s∗ [10] tests based on average rank correlation coefficients, (ii) Hollander’s [8] and Doksum’s [4] tests based on generalizations of Wilcoxon’s paired-replicates signed-rank statistic∗ , and (iii) tests based on weighted rankings due to Tukey [23], Moses [14], and Quade [19]. Examples of treatments which exhibit a natural ordering and where there may be prior reasons to suggest a deviation from H0 will be in the direction Ha (or some other specific direction such as θk θk−1 · · · θ1 ) include amounts of practice, levels of fertilizer, intensities of a stimulus, angles of knee flexion for conditioning, and doses of radiation. If there is a priori evidence that a deviation from H0 is in the direction θj1 θj2 · · · θjk , for some permutation (j1 , j2 , . . . jk ) of (1, 2, . . . , k), the tests described in the next

H0 is rejected in favor of Ha for large values of L. L is distribution-free∗ under H0 and Page gives upper percentiles of the null distribution of L in the α = 0.001, 0.01, and 0.05 regions for k = 3, n = 2(1)20 and k = 4(1)8, n = 2(1)12. These tables are reproduced as Table A.16 of Hollander and Wolfe [9]. The large sample approximation (n → ∞) treats L∗ = =

L − E0 (L) {Var0 (L)}1/2 L − nk(k + 1)2 /4 + 1)2 (k − 1)/144}1/2

{nk2 (k

(5)

as an approximate N(0, 1) random variable under H0 . The test based on L is equivalent to one based on ρ = ni=1 ρi where    k   ρi = 1 −  6 (rij − j)2 {k3 − k} (6)   j=1

is Spearman’s rank order correlation coefficient∗ between postulated order and observation order in the ith block. The relationship between L and ρ is L = (k3 − k)ρ/12 + nk(k + 1)2 /4. For the model Xij = βi + (j − 1)θ + eij ,

(7)


where the {eij } are i.i.d. according to F, Hollander [8] derived the Pitman asymptotic relative efficiency (ARE) (see PITMAN EFFICIENCY) of L with respect to the likelihood ratio∗ statistic t [for testing θ = 0 when the common distribution of the e’s is N(0, σ 2 )] for the alternatives Hn : Pn (Xij x) = F(x − βi − (j − 1)cn−1/2 ). The ARE is E(L, t)

,

= k{(k + 1)−1 } 12σ 2

Y=

f 2 (x)dx

k

Tuv .

(8)

u 0.864 for all F. When F is normal, E(Y, t) = 0.963 for k = 3 and → 0.989 as k → ∞. These values compare favorably with the corresponding values of Page’s test (0.716, 0.955). See Table 4.1 of ref. 8 for more values. A much simpler procedure than estimating µ(F) and λ(F) is to use upper bounds for µ(F) and λ(F) (see refs. 8 and 9, Section 7.6). Lehmann [12] showed that for all continuous F, λ(F) 7/24 and Hollander [8] showed that for all continuous F, µ(F) {(21/2 + 6)/24} = 0.3089. For all distributions for which they have been calculated, both µ(F) and λ(F) are quite close to their upper bounds (see Table 1). Thus a slightly conservative test is to treat YU =

Y − {k(k − 1)n(n + 1)/8} VU

(14)

as an approximate N(0,1) random variable under H0 , where 2 = VU

n(n + 1)(2n + 1)k(k − 1) 144 n } ×{3 + 2(k − 2)ρU

(15)

and n2 + 2n(21/2 − 1) + 3 − 2(21/2 ) n = ρU (n + 1)(2n + 1)

(16)

n is obtained by replacing µ(F) and λ(F) by ρU their upper bounds in (10). Lehmann [12] obtained the value of λ(F) for F uniform∗ , normal∗ , and Cauchy∗ . Hollander [7] obtained the values of λ(F) for F exponential∗ and the values of µ(F) for F uniform, normal, and exponential. Mann and Pirie [13] obtained the value of λ(F) for F logistic and the values of µ(F) for F logistic∗ and Cauchy. Incidentally, the known lower bounds for λ(F) and µ(F) are not nearly as good as the upper bounds. Lehmann showed

that for all continuous F, λ(F) 1/4 and Mann and Pirie improved the lower bound to λ(F) 5/18 for all continuous F. Mann and Pirie also showed that for all continuous F, µ(F) 1/4. Doksum [4] proposed a test that is very similar to the test based on Y. Doksum uses (i) , the random variables Uuv = Tuv − ni=1 ψuv k and considers the statistic D = u 0. For a given N and c, s1 > s. When population density varies over areas, stratified sampling with quadrats selected at random from each area is recommended. Stratified quadrat sampling with optimum allocation of sampling effort was used in aerial surveys of Alaska caribou [31] which reduced variance by more than half over that of simple random sampling. Where it is difficult to count all animals in all the sampled quadrats, two-phase sampling may be adopted. To estimate the number of beavers in a tract, we count only the number of beaver lodges per quadrat from a preliminary sample of quadrats; also, the ratio of the number of beavers to total beaver lodges can be estimated from a subsample of the quadrats. The two estimates can be combined to estimate the number of beavers in the tract. For a specified cost, double-sampling also proved more efficient [30] than single sampling for estimation of age-composition of commerical fish by species. Techniques have been developed for estimating total catch using ratio-estimates based on poststratification of sampled trips by market categories and subsampling clusters of a given weight for each category. Improvements in quadrat sampling in aerial surveys were proposed in refs. 5 and 6 by developing models for estimating the magnitude of visibility. There are basically three transect methods: the line intercept∗ , the strip, and the line transect∗ . The line intercept is generally used for estimating population total (N1 say) of inanimate objects, e.g., den sites of animals. If wi is the width of a den site parallel to the baseline W intersecting a random transect of length L and if m is the number of den sites intersecting n randomly selected transects, unbiased estimates of N1 and its

WILDLIFE SAMPLING

error are given by 1 ˆ1 = 1 , N n pi m

i

m (1 − pi ) ˆ 1) = 1 , v(N n2 p2i i=1

(1)

where pi = wi /W. This method is also known as length biased sampling [8]. An excellent review is given in ref. 10. A further treatment of the subject is in LINE INTERCEPT SAMPLING. In the strip-transect method all animals within a strip of fixed width are counted. Parallel lines, one strip width (2W, say) apart determine the population of strips. All the n animals observed within the sampled strips are counted. The estimate of total number of animals (N2 , say) is given by N2 =

A·n , 2LW

(2)

where L and A represent the length and population area of the strip. Estimation of marine mammals using strip transects is discussed in ref. 13. When the terrain is difficult and the population area is large, strip sampling will be more appropriate than quadrats. When population areas are irregular in shape, probability proportional to size∗ (PPS) selection of strips with replacement was adopted in making aerial censuses of wildlife populations in East Africa [17]. The line-transect method is generally used to measure the rare or fast moving terrestrial mammals and birds that are difficult to locate in a specified area. The method utilizes data on all subjects seen as these are flushed on either side of a transect line of length L across a transect of area A for estimating the ‘‘effective width’’ W of the strip covered by the observer as he moves along the transect line. Different methods based on various models proposed for estimating the population are of the form N = AD, where D, the population density, is estimated by ˆ = n 1 . D ˆ L W Assumptions underlying the models are given in LINE TRANSECT SAMPLING. References 15 and 27 deal with parametric estimators and with consequences for departures from the

9165

assumptions, particularly for moving animals; nonparametric estimators are dealt with in refs. 3 and 12. Line-transect methods based on elliptic flushing curves are discussed in ref. 2. Refs. 11 and 22 deal with their relative efficiencies. Substantial work has been done on moving populations using radiotelemetry [27]. CAPTURE-RECAPTURE∗ A number of M animals from a population are caught, marked, and released. On a second occasion, a sample of n animals is captured. If m be the number of marked animals in the sample, then a biased estimate of its size (N3 ) and its variance for a closed population are ˆ 3 = n M, N m ˆ2 ˆ ˆ N ˆ 3 ) = 3 (N3 − M)(N3 − n) . v(N ˆ 3 − 1) Mn(N

(3)

This was first given by Petersen [23] using ˆ 3 ) m−1/2 , we tagged plaice. Since C.V. (N should have sufficient recaptures in the ˆ 3 to be efficient. For second sample for N scarce populations, inverse sampling [27] is adopted, in which the second sampling proceeds until a specified m is recovered; for a given M, m can be chosen beforehand to give the desired C.V. CAPTURE –RECAPTURE METHODS —I provides a recent review for ‘‘closed’’ and ‘‘open’’ populations when marking is done over time. Reference 24 provides a design robust to unequal probability of capture and ref. 25 relates the capture probability to auxiliary variables. Reference 21 provides confidence limits for the parameter values by the Jolly-Seber method. A rationale of the procedures in capture-recapture methods, with illustrations from terrestrial invertebrates and small mammals, is provided in ref. 1. CHANGE-IN-RATIO The method provides estimates of the population size by removing individual animals if the change-in-ratio∗ of some attribute of the animal, e.g., sex or age composition is known. The method was first noted in ref. 18 for estimating deer and other wildlife populations. Consider a population of animals and

9166

WILDLIFE SAMPLING

assume that a differential in the number of males and females occurs before and after hunting. Then the maximum likelihood∗ estimator (MLE) of total (Nt ) and the number of males (Mt ) of the population at times t = 1, 2 based on certain assumptions are given by ˆ t = Rm − Rpt , N p1 − p2 ˆ t, ˆ t = pt N M

(4)

where pt = mt /nt , nt is the sample size at the beginning (t = 1) and end (t = 2) of the ‘‘harvest period,’’ mt is the number of males in nt , and R = Rm + Rf , where Rm and Rf are the number of males and females caught between t = 1, 2. The number of females Ft is estimated by subtraction, Ft = Nt − Mt . The formulas (4) assume (i) a closed population, (ii) all animals have the same probability of capture in the tth sample, and (iii) Rm and Rf are known exactly. A detailed discussion of the method when these assumptions are violated is given in ref. 27, Section 9.1.2. CATCH-EFFORT In the catch-effort method one unit of sampling is assumed to catch a fixed proportion of the population. If N is the initial population size, nt is the size of the sample removed during the tth time period, et is the effort applied in the tth time period, kt and Et are, respectively, the cumulative catch and effort through time period (t − 1), and Ct = nt /et is the catch per unit effort in the tth time period, we have, from refs. 9 and 20, E[Ct /kt ] = K(N − kt ),

(5)

where K is the catchability coefficient. The values of Ct plotted against kt will be a straight line with intercept KN and slope K, whence N can be estimated. If (5) holds, it can be shown [9] that E[log Ct ] = log(KN) − K(log e)Et .

(6)

Estimates of K and N can also be obtained if the points (log Ct , Et ) lie on a straight line. Both (5) and (6) have been widely used in fishery work. INDICES Indices are estimates of animal populations derived from counts of animal signs, breeding

birds, nests, etc. The results do not give estimates of absolute populations, but do indicate trends in populations over years and habitats. Stratified random sample surveys are being annually conducted in the U.S. and Canada for detecting and measuring changes in abundance of nongame breeding birds at the height of the breeding season [14,26]. Data on counts of birds heard or seen at predetermined stops on predetermined routes are collected by volunteer observers who make roadside counts of birds heard or seen according to a specified sampling scheme. An estimate of change and its standard error for a particular species between two successive years is given by ˆ = (y − x)/x, R Var y Var x ˆ = y se(R) + x y2 x2 2 cov(x, y) 1/2 − , xy

(7)

where x and y are, respectively, mean number of birds per route based on the first and second year. MEASUREMENT ERRORS∗ A very important source of variation arises from errors in the measurement of a quantity. For example, aerial surveys may be biased due to errors in sighting, recognition, and reliable counting of animals by observers. Mail surveys may be subject to high nonsampling errors [28,29] due to nonresponse bias, prestige bias (e.g., hunter exaggerating his kill), memory bias, or other factors. Caughley [3] showed that in aerial censusing of large mammals the observer missed a significant number of animals on the transect and that the number missed increased with increasing width of transect, cruising speed, and altitude. He recommended measurement and correction of the bias using regression∗ models. Air counts of ducks are also subject to visibility bias. Among prairie breeding species about two-thirds are missed [16]. Besides, observers may be biased for the most colourful species, e.g., mallards and canvasbacks as against pintails, which are conspicuous. Also, they may fail to determine

WILDLIFE SAMPLING

the edge of the transect accurately. Consequently, each year apart from the operational transects, air-ground comparison transects were selected purposively between operational transects and surveyed by the same aerial crew during the same time to provide a correction factor for the estimate based on a larger sample of air counts. Aerial photographs were used (scale 1 : 10,000) to correct for bias in aerial counts for estimating snow goose populations in Hudson Bay [19]. A subsample of the sampled areas was rephotographed using a scale of 1 : 3000. The 1 : 10,000 photographs yielded wide coverage but resulted in a biased count of total geese owing to difficulty in differentiating between geese and topographical features. The 1 : 3000 photographs yielded more accurate counts as well as discernibility between the white and blue phases of the species. Population estimates of both phases were obtained using regression techniques [32]. REFERENCES 1. Blower, J. G., Cook, L. M., and Bishop, J. A. (1981). Estimating the Size of Animal Populations, Allen and Unwin, London. 2. Burnham, K. P. 587–596.

(1979).

Biometrics,

35,

3. Burnham, K. P. and Anderson, D. R. (1976). Biometrics, 32, 325–336. 4. Caughley, G. (1974). J. Wildl. Manag., 38, 921–933. 5. Cook, R. D. and Jacobsen, J. O. (1979). Biometrics, 35, 735–742. 6. Cook, R. D. and Martin, F. B. (1974). J. Amer. Statist. Ass., 69, 345–359. 7. Cormack, R. M. (1968). Ocean. Marine Biol., 6, 455–506. 8. Cox, D. R. (1969). In New Developments in Survey Sampling, N. L. Johnson and H. Smith, eds. Wiley, New York, pp. 120–140. 9. DeLury, D. B. (1947). Biometrics, 3, 145–167. 10. DeVries, P. G. (1979a). In Sampling Biological Populations, R. M. Cormack, G. P. Patil, and D. S. Robson, eds. Satellite Program in Statistical Ecology. International Cooperative Publishing House, Fairland, MD, pp. 1–70. 11. DeVries, P. G. (1979b). Biometrics, 35, 743–748. 12. Eberhardt, L. L. (1978). J. Wildl. Manag., 42, 1–31.

9167

13. Eberhardt, L. L., Chapman, D. G., and Gilbert, J. R. (1979). Wildl. Monogr. No. 63. 14. Erskine, A. J. (1973). Can. Wildl. Ser. Prog. Note, 32, 1–15. 15. Gates, C. E. (1969). Biometrics, 25, 317–328. 16. Hanson, R. C. and Hawkins, A. S. (1975). Naturalist, 25, 8–11. 17. Jolly, G. M. (1969). East Afr. Agric. For. J., 34, 46–49. 18. Kelker, G. H. (1940). Proc. Utah Acad. Sci., Arts and Lett., 17, 65–69. 19. Kerbes, R. H. (1975). Can. Wildl. Serv. Rep. Series, 35. 20. Leslie, P. H. and Davis, D. H. S. (1939). J. Animal. Ecol., 8, 94–113. 21. Manly, B. F. J. (1984). Biometrics, 40, 749–758. 22. Otten, A. and DeVries, P. G. (1984). Biometrics, 40, 1145–1150. 23. Petersen, C. E. J. (1896). Rep. Dan. Biol. Statist., 6, 1–48. 24. Pollock, K. H. and Otto, M. C. (1983). Biometrics, 39, 1035–1049. 25. Pollock, K. H., Hines, J. E., and Nichols, J. D. (1984). Biometrics, 40, 329–340. 26. Robbins, C. S. and Vanvelzen, W. T. (1967). Spec. Sci. Rep. Wildl. No. 102, U.S. Forestry and Wildlife Service. 27. Seber, G. A. F. (1980). The Estimation of Animal Abundance and Related Parameters, Vol. 2. Griffin, London, England. 27a. Seber, G. A. F. (1986). Biometrics, 42, 267–292. (Reviews techniques developed beyond those in ref. 27. Contains six sections with 339 references including wildlife (61%), fisheries (21%), and insects and plants (4%) as major groups.) 28. Sen, A. R. (1972). J. Wildl. Manag., 36, 951–954. 29. Sen, A. R. (1973). J. Wildl. Manag., 37, 485–491. 30. Sen, A. R. (1986). Fishery Bull., 84, 409–421. 31. Siniff, D. B. and Skoog, R. O. (1964). J. Wildl. Manag., 28, 391–401. 32. Smith, G. E. J. (1975). Appendix to Kerbes, R. H. (1975). Can. Wildl. Service. Rep. Series, 35. See also ANIMAL SCIENCE, STATISTICS IN; CAPTURE –RECAPTURE METHODS; ECOLOGICAL STATISTICS; FISHERIES RESEARCH, STATISTICS IN; OPTIMUM STRATIFICATION; SURVEY SAMPLING; and TRANSECT METHODS.

A. R. SEN

9168

WILD SHOT

WILD SHOT. See OUTLIERS WILD VALUE. See OUTLIERS WILKS, SAMUEL STANLEY Born: June 17, 1906, in Little Elm, Texas, USA. Died: March 7, 1964, in Princeton, New Jersey, USA. Contributed to: multivariate analysis, nonparametric methods, statistical education in the USA, Institute of Mathematical Statistics Samuel Wilks was the eldest of three sons of Chance C. Wilks and his wife, Bertha May Gammon Wilks. His father farmed a 250-acre ranch outside Little Elm, Texas. His early education was in a one-room school-house. In high school, he boarded in nearby Denton. After graduating from high school, he attended North Texas Teachers’ College, obtaining a B.A. degree in architecture in 1926. During the years 1926 to 1929 he was associated with the Department of Mathematics at the University of Texas, obtaining an M.A. degree in 1928. He studied statistics from 1929 to 1931 at the University of Iowa under the prominent American statistician (the ‘‘father of American mathematical statistics’’) H. L. Rietz. He obtained a Ph.D, degree in 1931 with a dissertation on the distributions of statistics in samples from a bivariate normal population with matched sampling of one variable. This was published in Metron (9, 87–126) in 1932. Wilks married Virginia Orr in September, 1931. In the years 1931–1932, Wilks was working with Harold Hotelling∗ at Columbia University, New York on problems in multivariate analysis, and became acquainted with Walter Shewhart, the pioneer of statistical quality control. Wilks spent 1932–1933 in England, first at University College, London, collaborating with E. S. Pearson,∗ and later at Cambridge University, with J. Wishart. The Wilks’ only child, a son, Stanley, was born in London in October, 1932. In 1933 Samuel

was offered a post at Princeton University. From 1933 to 1936, he was an instructor in mathematics, rising to associate professor in 1938. His lectures in the late thirties led to his famous notes on Statistical Inference. After World War II began, he was working with the National Defense Research Committee, devoting more and more of his time to the war effort as the demand increased. In 1944 he was promoted to professor of mathematics at Princeton, and became director of the Mathematical Statistics Section in 1945, remaining in that post until his untimely death in 1964. In this period, Wilks published his lecture notes, more or less informally, as Statistical Inference (1937—the ‘‘little orange book’’), Mathematical Statistics (1943—gray paperbound), and Elements of Statistical Analysis (1948—the ‘‘little blue book’’). Mathematical Statistics was finally published, hardbound, after considerable revisions (notably during a sabbatical leave at Cambridge University), by Wiley in 1962. Wilks was prominent in organizing the Institute of Mathematical Statistics (IMS) in 1935, and was the founding editor of the Annals of Mathematical Statistics for over eleven years (1938–1949). By the end of his editorship, this journal had become the leading periodical devoted to mathematical statistics in the world. He was president of the IMS in 1940, and presented the Rietz lecture in 1959. He assisted in bringing together various American statistical societies to form the Committee of Presidents of Statistical Societies (COPSS). After Wilks’ death, this committee established the prestigious Wilks Award. Wilks was a fellow of the Econometric Society and the Royal Statistical Society, and a member of the International Statistical Institute. He was president of the American Statistical Association in 1950 and a member of the Russell Sage Foundation’s’ board of trustees. His activities in statistical education are reflected in five books. The last one—Introductory Engineering Statistics (with I. Guttman, published by Wiley) appeared posthumously in 1965. Wilks established, in cooperation with W. A. Shewhart, the highly esteemed Wiley Publications in Statistics series, which, at the

WILKS’S LAMBDA CRITERION

time of his death, contained about thirty titles in mathematical statistics and about the same number in applied statistics. A man of exceptional energy and dedication, Wilks was involved in numerous committees, trained many graduate students, and was active in application of statistical methods in industry. His main research, in the field of multivariate analysis, started with derivation of maximum-likelihood estimators and proceeded to development of likelihood criteria for testing various hypotheses on multivariate normal distributions (including the famous Wilks A-criterion). He introduced the concept of ‘‘generalized variance’’ and studied multivariate versions of intraclass correlation models—his name, along with those of T. W. Anderson, P. C. Mahalanobis,∗ and J. Wishart, is familiar to even occasional users of these techniques. Wilks pioneered research on tolerance intervals and order statistics. His further fields of versatile endeavor include design of experiments, statistical inference in geology, and the history of statistics. His book Mathematical Statistics (referred to above), together with M. G. Kendall’s Advanced Theory of Statistics, vol. 1 (1943) and H. Cramér’s Mathematical Methods in Statistics (1946), forms a triad which ushered in a golden age of statistical science. The second version (1962, published by Wiley) serves, even today after 30 years, as an evergreen standard text and reference. (It was translated into Russian in 1966.) Wilks’ friendly personal interest in his students and colleagues, his sociability, and his sense of humor endeared him to his contemporaries. His untimely death was a blow to the development of statistical science, but his life was an inspiration for it. REFERENCES 1. Anderson, T. W. (1965). Samuel Stanley Wilks: 1906–1964. Amer. Math. Statist., 34, 1–26. 2. Mosteller, F. (1964). Samuel S. Wilks: Statesman of statistics. Amer. Statist., 18(2), 11–17. 3. Pearson, E. S. (1964). Obiturary: Samuel Stanley Wilks, 1906–1964. J. R. Statist. Soc. A. 127, 597–599.

9169

WILKS’S LAMBDA CRITERION WILKS’S AND ITS ROLE Let A and B be two independent Wishart matrices of order p, with probability density functions (PDFs) W(A|n − q|) and W(B|q|), respectively, where W(A|n − q|) stands for constant |A|[(n−q)−p−1]/2 exp[− 21 tr −1 A]. (n − q) and q are the degrees of freedom (d.f.) of A and B, respectively, and is the parameter matrix of the Wishart distributions∗ . Denoting the determinant of A by |A|, we define =

|A| , |A + B|

called Wilks’s lambda. Its distribution depends on only the three parameters n, p, and q; n is the sum of the d.f.’s of A and B, p is the order of the matrices, and q is the d.f.’s of the matrix B. We will denote the PDF of by (n, p, q) and ∼ (n, p, q) denotes that has the (n, p, q) distribution. Wilks’s criterion plays the same role in multivariate analysis∗ as the F statistic plays in univariate analysis. In univariate analysis, if we have a general linear model∗ representing the means of independent normal observations with the same but unknown variance σ 2 , many important linear hypotheses about the parameters in the model are tested by dividing the total sum of squares (s.s.) of the observations into two parts, one of which has the χ 2 σ 2 distribution, irrespective of the values of the parameters, and the other has an independent χ 2 σ 2 distribution only if the hypothesis is true; otherwise, it has a noncentral χ 2 σ 2 distribution. The F statistic then compares these two independent χ 2 σ 2 variables to test the hypothesis. Wilks’s extends this idea to multivariate analysis, where we have p multinormal variables with a variance–covariance matrix and independent observations on these variables with a linear model representing the structure of the mean vectors of these

9170


observations. The matrix of the total s.s. and sum of products (s.p.) of these observations is then split into two parts A and B, where A will have the W(A|n − q|) distribution irrespective of the values of the parameters in the model and B will have an independent W(B|q|) distribution, only if a certain hypothesis about the parameters is true, but otherwise it will have a noncentral Wishart distribution. The hypothesis is then tested by using the Wilks’s criterion defined above. The distribution of does not involve the nuisance parameter∗ , just as F is free of σ 2 . DISTRIBUTION OF WILKS’S Explicit expressions for the exact PDF of are available in the literature [7,30,44,46], but what is really useful in practice is the Bartlett approximation [4] that W = −m loge ,

2. For p = 2, {(1 − 1/2 )/1/2 }{(n − q − 1)/q} has the F distribution with 2q and 2(n − q − 1) d.f.’s. All the results above are derived from the hth moment of , which is obtained from the distributions of A and B (see, e.g., Kshirsagar [27]). Box [6] has given an extended version of Bartlett’s approximation stated earlier. ALTERNATIVE EXPRESSIONS FOR Many important statistical problems can be viewed as problems of relationship (or lack of relationship) of one p-vector x of variables with another q-vector y and Wilks’s is useful as a measure of lack of association of x with y. If Cxx represents the matrix of s.s. and s.p. of observations on x, Cxy represents the matrix of s.p. of observations on x with those on y, the matrix B = Cxy C−1 yy Cyx

where m = n − 12 (p + q + 1), is asymptotically a χ 2 with pq d.f.’s for large n. The exact percentage points of W are given by Schatzoff [44] and Pillai and Gupta [37] and are also available in Biometrika Tables for Statisticians, Vol. 2 [35]. What is tabulated in all these sources is not the 100α% percentage points directly but only a multiplying factor Cα (p, q, M), which depends on α, p, q, and M = n − p − q + 1. To obtain the 100α% point of the distribution of W, we have to multiply the 100α% point of the χ 2 distribution∗ with pq d.f.’s by this multiplying factor Cα (p, q, M). The extra accuracy provided by Cα (p, q, M) is, however, often not necessary if n is sufficiently large. It has been proved (see, e.g., Kshirsagar [27]) that (n, p, q) is the same as (n, q, p) or that p and q are interchangeable. For the particular cases p = 1 or 2 (and due to interchangeability, q = 1 or 2), one may use, alternatively, the exact results: 1. For p = 1, {(1 − )/}{(n − q)/q} has the F distribution∗ with q and n − q d.f.’s.

is the ‘‘regression s.s. and s.p. matrix’’ of x on y and the matrix A = Cxx − Cxy C−1 yy Cyx = Cxx·y is called the ‘‘error matrix.’’ The d.f.’s of B are q and of A are, say, n − q. Then under assumption of normality for x, y, A will have the W(A|n − q|) distribution but B will have an independent W(B|q|) distribution only if x and y are independent. Wilks’s is then also expressible as |Cxx·y | |Cxx | and interchanging y and x, it is also |Cyy·x | . |Cxx | If p q and r21 , r22 , . . . , r2p are the squares of the canonical correlations∗ between x and y, is also expressible as b (1 − r2i ) 1


and yet another expression for is |L|, where the matrix L is defined by A = CLC ,

A + B = CC .

In a sense, L can be described as the ratio of A, the error s.s. and s.p. matrix to A + B, the total s.s. and s.p. matrix of x and hence it is a generalization of 1 − r2 , where r is the correlation coefficient∗ and R is the multiple correlation coefficient∗ . It should also be noted that is invariant for linear transformation of x and y. USES OF WILKS’S Wilks’s is useful in the following problems. 1. It can be used in a test of equality of the variance–covariance matrices of two independent multivariate normal populations. If samples of sizes n1 and n2 are available from these two p-variate normal populations, one can compute A and B, the matrices of the corrected s.s. and s.p. of the observations for the two samples, and then as defined earlier will have the (n, p, q) distribution with n = n1 + n2 − 2,

p, q = n2 − 1

as the parameters if the hypothesis is true. The hypothesis will be rejected at the 100(1 − α)% point level of significance if W, as defined earlier, exceeds the 100α% point of W. 2. It can also be used in a test of independence of x and y, when both of them have a multivariate normal distribution or one of them, say x, has a multivariate normal distribution, the other is a fixed vector, and the regression of x on y is linear. In this case Cxx , Cxy , and Cyy are calculated and the form |Cxx·y | = |Cxx | is useful. The parameters n, p, and q for this situation are already given. This is also a test for the hypothesis that the

9171

true canonical correlations between x and y are all null; and this is also a test for the hypothesis that the matrix of true regression coefficients of x and y is null. 3. If a test of significance of only a subset of columns of the matrix of regression coefficients of x and y is desired, Wilks’s is useful there, too, and this is done by splitting B = Cxy C−1 yy Cyx suitably (see, e.g., Kshirsagar [27]). 4. If there are k = q + 1 independent pvariate normal populations with the same variance–covariance matrix but different mean vectors µ1 , µ2 , . . . , µk , a test of the ‘‘multivariate analysis of variance∗ (MANOVA) hypothesis’’ µ1 = µ2 = · · · = µk is provided by Wilks’s , by taking B as the ‘‘between-groups’’ s.s. and s.p. matrix of q = k − 1 d.f.’s and A as the ‘‘within-groups’’ s.s. and s.p. matrix of d.f.’s n − q, where n = N − 1 and N = n1 + · · · + nk , where ni is the size of the sample from the ith group or population. This is a typical one-way classification∗ problem for multivariate analysis and is useful for testing appropriate hypotheses for the general multiway classification for multivariate analysis of variance also. Usefulness of in assessing the degree of separation of the groups is also considered by Gau [13]. 5. Wilks’s is useful in tests associated with the growth curve model, introduced by Potthoff and Roy [41], namely E(X) = Pξ Q, where X is the p × n matrix of observations on p multinormal variables, corresponding to m different groups, such that every column of X has the same variance–covariance matrix , which is unknown. P and Q are, respectively, p × 1 and m × n matrices of known

9172


elements, representing design matrices within and across individuals [14]. ξ is the matrix of unknown parameters of the growth curves of the m groups. Khatri [17] shows how Wilks’s can be used to test the hypothesis Cξ V = 0 in such a situation. Grizzle and Allen [14] illustrate this method with some interesting practical problems. This test is useful in analyzing longitudinal data, in general. FACTORS OF WILKS’S AND APPLICATION IN DISCRIMINANT ANALYSIS Apart from the factorization = (1 − r2i ) described earlier, there is another, more useful, factorization, = t211 · t222 · . . . · t2pp , where tii are the diagonal elements of a lower triangular matrix T defined by L = TT ,

A = CLC ,

A + B = CC ,

with C also lower triangular. It is shown that the t2ii are independent beta variables [27,31]. The usefulness of these factors t2ii , their meaning and significance, and their use in a step-down analysis of the relationship of x and y are discussed in Kshirsagar [27]. McHenry [32] and Farmer and Freund [10] have utilized this factorization of for variable selection purposes in multivariate analysis of variance. Rao [42] has utilized this factorization to construct a modified , when observations on one of the p variables are missing. Bartlett [5] and Williams [48] consider the use of canonical variables for discriminating among several multivariate populations and obtain a test of goodness of fit of a single hypothetical discriminant function using Wilks’s . Bartlett has factorized into a direction factor and a ‘‘partial’’ collinearity factor to test the two aspects of the overall goodness-of-fit∗ hypothesis, namely, whether the hypothetical function agrees with the true one and whether a single function is adequate

at all for discrimination. Reference may be made to Kshirsagar [28] for an expository article on this topic and for various related references to work of Kshirsagar [21–27] and Williams [49] where several distributional results, extensions, and generalizations (as, for example, to contingency tables) are considered. POWER OF WILKS’S AND OTHER MISCELLANEOUS RESULTS The nonnull distribution of is the distribution when the matrix A, defined in the first section, is Wishart as before, but the other matrix B has an independent noncentral Wishart distribution. This distribution will obviously be useful in investigating the power of the Wilks’s test in the various problems considered. It is, however, too complicated. It is studied by Khatri and Pillai [18], Sugiara and Fujikoshi [45], Pillai et al. [40], and Gupta [15]. Reference may also be made to Pillai and Jayachandran [39], Michail [33], and Ito [16], for comparison of power of with other criteria, in some special cases. Das Gupta and Perlman [8] have shown that the power of Wilks’s strictly increases with p and q in the linear case (i.e., when the noncentrality matrix in the distribution of B is of rank 1) and Fujikoshi [12] proves the monotonicity property of the power function. Wilks’s for complex normal variables has been studied by Young [50]. REFERENCES 1. Anderson, T. W. and Das Gupta, S. (1964). Ann. Math. Statist., 35, 206–211. 2. Bartlett, M. S. (1934). Proc. Camb. Philos. Soc., 30, 327–355. 3. Bartlett, M. S. (1938). Proc. Camb. Philos. Soc., 34, 33–48. 4. Bartlett, M. S. (1939). Proc. Camb. Philos. Soc., 35, 180–190. 5. Bartlett, M. S. (1951). Ann. Eugen. (Lond.), 16, 109–127. 6. Box, G. E. P. (1949). Biometrika, 36, 317–346. 7. Consul, P. C. (1965). Bull. Acad. R. Belg. (Cl. Sci.), 51, 683–691. 8. Das Gupta, S. and Perlman, M. D. (1973). J. Multivariate Anal., 3, 220–225.

WILLCOX, WALTER FRANCIS 9. Das Gupta, S., Anderson, T. W., and Mudholkar, G. S. (1964). Ann. Math. Statist., 35, 200–220. 10. Farmer, J. H. and Freund, R. J. (1975). Commun. Statist., 4, 87–98. 11. Fujikoshi, Y. (1970). J. Sci. Hiroshima Univ., Ser. A-I, 34, 73–144. 12. Fujikoshi, Y. (1973). Ann. Statist., 1, 388–391. 13. Gau, G. W. (1978). Decision Sci., 9, 341–345. 14. Grizzle, J. and Allen, D. M. (1969). Biometrics, 25, 357–381. 15. Gupta, R. D. (1979). Statistica, 39, 333–342. 16. Ito, K. (1962). Biometrika, 49, 455–462. 17. Khatri, C. G. (1966). Ann. Inst. Statist. Math., Tokyo, 18, 75–86. 18. Khatri, C. G. and Pillai, K. C. S. (1965). Ann. Math. Statist., 36, 1511–1520. 19. Kiefer, J. (1966). In Multivariate Analysis: Proceedings of an International Symposium, P. R. Krishnaiah, ed. Academic Press, New York. 20. Krishnaiah, P. R. (1977). Some Recent Developments on Real Multivariate Distributions. Monogr., Dept. Mathematics and Statistics, University of Pittsburgh, Pittsburgh, Pa. 21. Kshirsagar, A. M. (1961). Ann. Math. Statist., 32, 104–111. 22. Kshirsagar, A. M. (1964). J. Indian Statist. Ass., 2, 1–20. 23. Kshirsagar, A. M. (1964). Proc. Camb. Philos. Soc., 60, 217–222. 24. Kshirsagar, A. M. (1969). Aust. Math. Soc., 10, 269–273. 25. Kshirsagar, A. M. (1970). Ann. Inst. Statist. Math. Tokyo, 22, 295–305. 26. Kshirsagar, A. M. (1970). Calcutta Statist. Ass. Bull., 19, 123–130. 27. Kshirsagar, A. M. (1971). J. R. Statist. Soc., B, 33, 111–116. 28. Kshirsagar, A. M. (1976). In The Search for Oil: Some Statistical Methods and Techniques, D. B. Owen, ed. Marcel Dekker, New York, pp. 147–168. 29. Lawley, D. N. (1959). Biometrika, 46, 59–65. 30. Mathai, A. M. and Rathie, P. N. (1969). The Exact Distribution of Wilks’s Criterion. Preprint No. 1969-28, Dept. of Mathematics, Queen’s University, Kingston, Ontario. 31. McHenry, C. (1976). Commun. Statist., 5, 1047–1053. 32. McHenry, C. (1978). Appl. Statist., 27, 291–297.

33. Michail, M. N. 149–152.

(1965).

9173

Biometrika,

52,

34. Narain, R. D. (1950). Ann. Math. Statist., 21, 293–300. 35. Pearson, E. S. and Hartley, H. O. (1971). Biometrika Tables for Statisticians, Vol. 2. Cambridge University Press, Cambridge. 36. Pillai, K. C. S. (1977). Canad. J. Statist., 5, 1–62. 37. Pillai, K. C. S. and Gupta, H. K. Biometrika, 56, 109–118.

(1969).

38. Pillai, K. C. S. and Jayachandran, K. (1967). Biometrika, 54, 195–203. 39. Pillai, K. C. S. and Jayachandran, K. (1968). Biometrika, 55, 335–342. 40. Pillai, K. C. S., Al-Ani, and Jouris, G. M. (1969). Ann. Inst. Statist. Math. Tokyo, 21, 309–320. 41. Potthoff, R. R. and Roy, S. N. Biometrika, 51, 313–326.

(1964).

42. Rao, C. R. (1956). J. R. Statist. Soc. B, 9, 259–264. 43. Rencher, A. C. and Larson, S. F. (1980). Technometrics, 22, 349–356. 44. Schatzoff, M. (1966). Biometrika, 53, 347–358; correction, ibid, 54, 688 (1966). 45. Sugiura, N. and Fujikoshi, Y. (1969). Ann. Math. Statist., 40, 942–952. 46. Wald, A. and Brookner, R. J. (1941). Ann. Math. Statist., 12, 137–152. 47. Wilks, S. S. (1932). Biometrika, 24, 471–494. 48. Williams, E. J. (1951). Biometrika, 38, 17–35. 49. Williams, E. J. 274–282.

(1952).

Biometrika,

39,

50. Young, J. C. (1971). Some Inference Problems Associated with the Complex Multivariate Normal Distribution. Tech. Rep. No. 102, Dept. of Statistics, Southern Methodist University, Dallas, Tex. See also MULTIVARIATE ANALYSIS; MULTIVARIATE ANALYSIS OF VARIANCE (MANOVA); and WISHART DISTRIBUTION.

ANANT M. KSHIRSAGAR

WILLCOX, WALTER FRANCIS Born: March 22, 1861 in Reading, Massachusetts. Died: October 30, 1964 in Ithaca, New York.

9174

WILLIAMS’ TEST OF TREND

Contributed to: American Statistical Association, history of statistics, vital statistics. In the course of his remarkably long life, Walter Francis Willcox was influential in the development and structuring of statistics, and especially vital statistics∗ , as an independent subject in the general field of scientific method. He taught ‘‘an elementary course in statistical methods with special treatment of vital and moral statistics’’ at Cornell University in 1892–1893, soon after joining the faculty in 1891. He served at Cornell for 40 years, retiring in 1931. However, his work was by no means confined to university teaching. He was active in working on the 12th United States Census (1900) and 1899 to 1901, and initiated and supervised a large volume of Supplementary Analysis and Derivative Tables. Willcox was in steady demand as a consultant on collection and analysis of demographic statistics, especially in regard to international migration. A topic attracting much of his attention was the basis of apportionment of seats in the House of Representatives of the U.S. Congress. Much interesting detail is available in ref. 1, which was produced in connection with Willcox’s 100th birthday, and in the obituary [2], which contains personal reminiscences. Willcox was President of the American Statistical Association∗ in 1912 and of the International Statistical Institute∗ in 1947. He was very active in the work of both of these organizations for many years, and compiled the first extensive list [3] of definitions of statistics, from G. Achenwall in 1749 to R. A. Fisher∗ in 1934. In his later years he interested himself in the history of statistics—in particular of demographic statistics—and produced a number of insightful analyses in this field. REFERENCES 1. Leonard, W. R. (1961). Amer. Statist., 15(1), 16–19. 2. Rice, S. A. (1964). Amer. Statist., 18(5), 25–26. 3. Willcox, W. F. (1935). Int. Statist. Inst. Review, 3, 388–399.

See also DEMOGRAPHY; KIAER, ANDERS NICOLAI; MIGRATION; and VITAL STATISTICS.

WILLIAMS’ TEST OF TREND A problem in isotonic inference∗ that has received considerable attention arises when there are k + 1 independent samples from normal distributions with equal variances and means known a priori to be monotonically ordered. For example, in an animal bioassay we believe that µ0 · · · µk , where µ0 is the mean of a response of interest in a control group, and µ1 , . . . , µk are the means corresponding to increasing exposures of the test agent. Williams [5] suggests a sequential testing procedure to find the largest index j such that µ0 = µj . The null hypotheses µ0 = µk , µ0 = µk−1 , . . . , µ0 = µ1 are tested sequentially until an hypothesis is not rejected, or all hypotheses have been tested. Knowledge of the ordering of the means is utilized by basing the tests on x0 , µ˜ 1 , . . . , µ˜ k , where x0 is the sample mean for the control group and µ˜ i is the maximum likelihood estimate (MLE) of µi , i = 1, . . . , k (see discussion of the amalgamation process under ISOTONIC INFERENCE or in ref. 1). For a monotonically nondecreasing order of the means the test of µ0 = µk is rejected for large values of Williams’ statistic tk = (µ˜ k − x0 )(2s2 /r)−1/2 ,

(1)

where r is the common sample size and s2 is an unbiased estimate of the common error variance of the sampled distributions. Replacing µ˜ k in (1) by the sample mean xk would yield the familiar Student’s t-statistic. Hence the notation tk . A table of critical values for tk is given in ref. 5 for the case where all sample-sizes are equal and for α = 0.01, α = 0.05, k = 1, . . . , 10. In ref. 6, Williams extends his procedure to allow for a control group that is larger than the common sample size of the treatment groups. He also discusses the optimal relative ratio of the control to treatment sample size, provides tables of critical values for increased control replication, and gives approximate critical values for moderate differences between the number of replications within treatment groups.

WILLIAMS’ TEST OF TREND

If the test statistic tk given in (1) exceeds the tabled value tk,α , then it is suggested [5] that to test the next hypothesis, µ0 = µk−1 , the subscript k in (1) should be replaced by k − 1. The size of the latter test conditional on having rejected µ0 = µk , however, exceeds the nominal size α. It is incorrectly stated [5] as equal to α, but the correction is noted in ref. 7. How much larger the test size is than the nominal size α at any of the tests subsequent to the first one has apparently not been established. It is also noted in ref. 7 that the proof at the beginning of Section 6 in ref. 6 that Pi < α is incorrect as stated, but the result remains valid. A result in ref. 4 applies to the nominal type I error rate. If µ0 · · · µk , then the null hypotheses sequentially tested, Hk : µ0 = µk , Hk−1 : µ0 = µk−1 , . . . , H1 : µ0 = µ1 , are nested with Hk ⊂ · · · ⊂ H1 . If each sequential test that is conducted is of size α, then the probability that no type I error is made is at least 1 − α. This is because for no type I error to occur the first true null hypothesis in the sequential order must be tested and not rejected. But this event stops the sequential testing. Due to the monotonic ordering of the means, a test of µ0 = µk is equivalent to a test that all the means are equal (µ0 = · · · = µk ) against the alternative of at least one strict inequality in the ordering. Power comparisons that include Williams’ test and others for treating this problem may be found in refs. 2,3, and 5. The results do not suggest that a single test is preferable in all circumstances. Williams’ test, or the modified Williams test [3,6] wherein x0 is replaced by its MLE µ˜ 0 in (1), appears to perform particularly well when k is small (k 2) and in cases where the means are equal except at the smallest and/or largest index values. As an example, suppose that mice are exposed to an ether compound for 18 weeks in a laboratory. Four groups of 16 each are exposed to concentrations of 0, 30, 100, and 300 parts per million (ppm). One indicator of a potential detrimental health effect from the exposure is a reduction in body weight. The mean body weights at the end of 18 weeks are 544, 514, 532, and 488 at 0, 30, 100, and 300 ppm, respectively.

9175

The estimated variance of the normal distributions is s2 = 1426 with 60 degrees of freedom. In this case we have a priori knowledge that the means should be nonincreasing as the exposure increases, µ0 µ1 µ2 µ3 . The MLEs of the means are 544, 523, 523, and 488 and x0 = 544. The value of (2s2 /r)−1/2 is 0.075, so t3 = (488 − 544)0.075 = −4.2 < −t3,0.05 = −1.77 and t2 = (523 − 544)0.075 = −1.58 > −t2,0.05 = −1.75. Thus we conclude that there is a significant reduction in body weight at 300 ppm, but not at 30 or 100 ppm. REFERENCES 1. Barlow, R. E., Bartholomew, D. J., Bremner, J. M., and Brunk, D. (1972). Statistical Inference under Order Restrictions. Wiley, New York. 2. Chase, G. R. (1974). Biometrika, 61, 569– 578. 3. Marcus, R. (1976). Biometrika, 63, 177–183. 4. Marcus,R., Peritz,E., and Gabriel,K. R. (1976). Biometrika, 63, 655–660. 5. Williams,D. A. (1971). Biometrics, 27, 103–117. 6. Williams,D. A. (1972). Biometrics, 28, 519–531. 7. Williams, D.A. (1986). Correction note, included in correspondence from Williams to the author.

BIBLIOGRAPHY Abelson, R. P. and Tukey, J. W. (1963). Ann. Math. Statist., 34, 1347–1369. (Utilizes prior information on spacing of means for a test of equality against an ordered alternative.) House, D. E. (1986). Biometrics, 42, 187–190. (Provides a nonparametric version of Williams’ test for a randomized block design.) Shirley,E. (1977). Biometrics, 33, 386–389. (Derives a nonparametric equivalent of Williams’ test where observations are replaced by their ranks.) Shorack, C. R. (1967). Ann. Math. Statist., 38, 1740–1752. (Normal and nonparametic theory for tests in the one-way analysis of variance against an ordered alternative.) Williams, D. A. (1977). Biometrika, 64, 9–14. (Determines a limiting distribution of the estimated maximum and range of a set of monotonically ordered normal means when all means are in fact equal.)

9176

WILSON, EDWIN BIDWELL

Williams, D. A. (1986). Biometrics, 42, 183–186. (Suggests a modification to Shirley’s test to improve its power.) See also ISOTONIC INFERENCE; MONOTONE RELATIONSHIPS; ORDER-RESTRICTED INFERENCES; and TREND TESTS.

KENNETH G. BROWN

WILSON, EDWIN BIDWELL Born: April 25, 1879, in Hartford, Connecticut. Died: December 28, 1964, in Brookline, Massachusetts. Contributed to: mathematics, physics, statistical inference, biostatistics. Edwin Bidwell Wilson was a scientific generalist. As a mathematician, he published two of the most influential advanced texts of the early part of this century, and he criticized David Hilbert on the foundations of geometry. As a physicist, he did pathbreaking work on the mathematics of aerodynamics, and he criticized Albert Einstein on relativity. As a statistician, he anticipated Neyman∗ on confidence intervals, he devised one of the earliest normalizing transformations, he founded the biostatistics program at the Harvard School of Public Health, and he criticized Ronald Fisher∗ on inference. Wilson’s lifelong commitment to science and education was presumably instilled in him at a very early age: his father was a teacher and superintendent of schools in Middleton, Connecticut. Wilson graduated from Harvard College in 1899, majoring in mathematics, and he then went to Yale to continue his mathematical study as a student of J. Willard Gibbs. In 1901, the same year he received his Ph.D., Wilson published Vector Analysis, Founded upon the Lectures of J. Willard Gibbs, a text that, with Wilson’s later Advanced Calculus (1912), provided a significant portion of the upper-level mathematics curriculum in America for the first third of the century. From 1900 to 1907, Wilson taught at Yale, with a year off in 1902–1903 for study in Paris. In 1907 he moved from Yale to a faculty position at MIT, where his interests evolved to mathematical physics and academic administration. From 1920 to 1922

he served as one of a committee of three, functioning collectively as interim president of MIT. In 1922 he moved to Harvard as professor and head of Vital Statistics in the Harvard School of Public Health, where he remained until his retirement in 1945. Wilson was managing editor of the Proceedings of the National Academy of Sciences for fifty years, from its first issue in January 1915 until his death in December 1964. He was active on nearly every national committee involved with social science over the last half of his life. A student of his, Paul Samuelson, has described him as ‘‘the only intelligent man I ever knew who liked committee meetings.’’ [6] WORK IN STATISTICS Wilson’s most important contribution to statistics was arguably as an institution builder, founding a program that still flourishes under the title of biostatistics in Harvard’s School of Public Health. He also brought his sharp critical intelligence and knowledge of quantitative social science to bear on methodological issues on many national committees and in a far-ranging national correspondence (see, e.g., ref. [7]). He had a keen sense of data analysis, as indicated in his reinvestigation with Margaret Hilferty of C. S. Peirce’s extensive data on reaction times [9], a study still cited in the literature on robustness. But he also made a number of important technical contributions. In work with Jane Worcester on quantal response [15], he advanced the study of the estimation of the median lethal dose, which they called ‘‘LD 50.’’ Wilson’s 1931 paper with Margaret Hilferty [10] introduced what has become known as the Wilson–Hilferty transformation, a device that allowed the use of the normal approximation for chi-square∗ probabilities over a wide range of degrees of freedom (see NORMAL DISTRIBUTION). In 1927 Wilson published a short note that anticipated (albeit for a very limited class of problems) the concept behind Neyman’s confidence intervals. Wilson clearly described in that paper how the confidenceinterval idea could be invoked for inference about the binomial parameter, explaining the difference between the confidence

WINCKLER–GAUSS IEQUALITIES

idea and the common use of standard errors as an approximate way of doing inverse probability∗ (or Bayesian) inference [8,12]. In 1941 Wilson published a note in Science that contradicted R. A. Fisher on the analysis of two-by-two tables∗ [11]. Fisher’s polite reply [2] elicited a published retraction by Wilson [13,14], and the two enjoyed a long correspondence on statistical issues [1]. Wilson wrote several general articles on statistical inference and scientific methodology, articles that show an acute sensitivity to both similarities among and differences between measurement problems in the social and physical sciences [3]. He also made important contributions to mathematical utility∗ theory, showing that Pareto’s derivation of the law of demand held under more general conditions than had been previously believed [5]. Wilson’s work on statistics, whether writing on bioassay, contingency tables, factor analysis, population growth, or the foundations of inference, showed a keen and acutely perceptive intelligence that was unusual among writers at that time. Although he only rarely achieved originality of concept, his knowledge and critical assessment of contemporary advances was far ahead of most of his contemporaries’.

9177

6. Samuelson, P. (1989). Gibbs in economics. Proc. Gibbs Symp. Amer. Math. Soc., pp. 255–267. 7. Stigler, S. M. (1994). Some correspondence on methodology between Milton Friedman and Edwin B. Wilson; November–December 1946. J. Econ. Lit., 32, 1197– 1203. 8. Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. J. Amer. Statist. Ass., 22, 209–212. 9. Wilson, E. B. and Hilferty, M. M. (1929). Note on C. S. Peirce’s experimental discussion of the law of errors. Proc. Nat. Acad. Sci. U.S.A., 15, 120–125. 10. Wilson, E. B. and Hilferty, M. M. (1931). The distribution of chi-square. Proc. Nat. Acad. Sci. U.S.A., 17, 684–688. 11. Wilson, E. B. (1941). The controlled experiment and the four-fold table. Science, 93, 557–560. 12. Wilson, E. B. (1942). On confidence intervals. Proc. Nat. Acad. Sci. U.S.A., 28, 88–93. 13. Wilson, E. B. (1942). On contingency tables. Proc. Nat. Acad. Sci. U.S.A., 28, 94– 100. 14. Wilson, E. B. and Worcester, J. (1942). Contingency tables. Proc. Nat. Acad. Sci. U.S.A., 28, 378–384. 15. Wilson, E. B. and Worcester, J. (1943). The determination of LD 50 and its sampling error in bioassay. Proc. Nat. Acad. Sci. U.S.A., 29, 79–85, 114120, 257262.

STEPHEN M. STIGLER REFERENCES

WINCKLER–GAUSS IEQUALITIES 1. Bennett, J. H., ed. (1990). Statistical Inference and Analysis: Selected Correspondence of R. A. Fisher. Clarendon Press, Oxford. 2. Fisher, R. A. (1941). The interpretation of experimental four-fold tables. Science, 94, 210–211. (Reprinted in vol. 4 of Fisher’s Collected Papers as No. 183.) 3. Hunsaker, J., and Mac Lane, S. (1973). Edwin Bidwell Wilson 1979–1964. Biog. Mem. (National Academy of Science), 43, 285–320. (Includes a complete bibliography of Wilson’s work.) 4. Irwin, J. O. and Worcester, J. (1965). Edwin Bidwell Wilson, 1879–1964. J. R. Statist. Soc. A, 128, 616–618. 5. Milgate, M. (1987). Wilson, Edwin Bidwell. In The New Palgrave Dictionary of Economics, vol. 4. Macmillan, London, pp. 922–923.

These are inequalities on values of absolute moments of any distribution. The rth absolute moment∗ of a random variable about a (a fixed number) is a νr

= E[|X − a|r ].

The Winckler-Gauss inequalities are {(n + 1)a νn }1/n {(r + 1)a νr }1/r or, more generally 1/n 1/r {(r + 1)[k] , {(n + 1)[k] a νn } a νr }

where g[b] = g(g + 1) · · · (g + b − 1) for all 0 < n < r. They were first stated by Gauss [2] for

9178

WINCKLER–VON MISES TYPE INEQUALITIES

a special case. The general case was stated by Winckler [3], who gave an incorrect proof, and later corrected by Beesack [1]. REFERENCES 1. Beesack, P. A. (1984). J. Math. Anal. Appl., 98, 435–457. 2.

Gauss, C. F. (1821). Theoria Combinationis Observationem Erroribus Minimis Obnoxiae. ¨ Drucksache. In Werke. Dieterische Universitats 3. Winckler,A. (1866). Sitzungsber. Math.-Natur. Kl. Kon. Akad. Wiss., Wien, Zweite Abt., 53, 6–41. See also BERNSTEIN’S INEQUALITY; CAMP –MEIDELL INEQUALITY; CHEBYSHEV’S INEQUALITY; MARKOV INEQUALITY; and WINCKLER –VON MISES TYPE INEQUALITIES.

WINCKLER–VON MISES TYPE INEQUALITIES In 1866, Winckler [3] derived a large number of inequalities relating to cumulative distribution functions (CDFs) of distributions of random variables (X) possessing a continuous CDF, which is unimodal and symmetrical. Similar inequalities, under less restrictive conditions, were obtained by von Mises∗ [2] in 1938. More recently, these inequalities have been refined by Beesack [1]. Typical inequalities of this kind are of form: ‘‘For any real a, and under certain conditions on the CDF, Pr[|X − a| x] if x

r r+1

r

a νr xr

r {(r + 1)a νr }1/r r+1

and Pr[|X − a| x] 1 − if x
0, i.e., Xi has a p-dimensional multivariate normal distribution with mean vectorµi and covariance matrix , and let W = Xi X i , δ = µi µ i . We say that W has a p-dimensional Wishart distribution with n degrees of freedom on the covariance matrix with noncentrality parameter δ and write W ∼ Wp (n, , δ). Note that W is a p × p matrix and that W 0. From this definition, it is apparent that

9185

Muirhead [11], pp. 258–262). If δ = 0, then 1 0 F1 2 n, 0 = 0, so that the central Wishart density is given by K||−n/2 |w|(n−p−1)/2 exp{− 12 tr( −1 w)}, w > 0.

(4)

In the next section we present some basic results about the Wishart distribution. In later sections we indicate how the Wishart distribution occurs in two multivariate models, the multivariate one-sample model and the one-way multivariate analysis of variance∗ model. BASIC FACTS

EW = n + δ, AWA ∼ Wq (n, AA , AδA ),

(1)

where A is q × p of rank q. If δ = 0, we say that W has a central Wishart distribution and write W ∼ Wp (n, ). If δ = 0, we say that W has a noncentral Wishart distribution. The rank of W is the minimum of n and p. Therefore, W > 0 (and hence invertible) if and only if n p. If n p, we say that W has a nonsingular Wishart distribution and if n < p, we say W has a singular Wishart distribution. The characteristic function∗ of the Wishart distribution is φ(T) = E exp(i tr(TW)) = |I − 2iT|−n/2 exp(− 21 tr D) with D = −1 δ[I − (I − 2iT)−1 ],

(2)

where T is a symmetric p × p matrix such that −1 − 2T > 0. The singular Wishart distribution does not have a density function. The density function of the nonsingular Wishart distribution is K||−n/2 |w|(n−p−1)/2 × exp[− 21 tr{ −1 (w + δ)}] ×0 F1 ( 12 n; 14 −1 δ −1 w),

(3)

w > 0, where K −1 = 2−np/2 π p(p−1)/4 i=1 ((n + 1 − i)/2) p

& ' and 0 F1 12 n, 14 −1 δ −1 w is a hypergeometric function of matrix argument∗ (e.g., see

We first present some facts about the Wishart distribution which are straightforward generalizations of well-known facts about the χ 2 distribution: A. Let the Wi s be mutually indepen Wi ∼ dent, Wi ∼ W p (ni , , δ i ). Then Wp ( ni , , δ i ). B. Suppose that nVn ∼ Wp (n, ). Then Vn converges in probability to . C. Let X = (X1 , . . . , Xn ) where Xi are independent, Xi ∼ Np (µi , ). Let µ = EX. Let A and B be symmetric n × n matrices and let C be q × n. 1. X AX ∼ Wp (k, , µ Aµ) if and only if A2 = A, k = rank(A). 2. X AX and X BX are independent if and only if AB = 0. X AX and CX are independent if and only if CA = 0. (Note that X is an n × p matrix and that the independent replication is represented by the rows of X, as in univariate models.) Result C1 implies that X AX has a Wishart distribution in the multivariate case if and only if X AX/σ 2 has a χ 2 distribution in the univariate case and that the degrees of freedom are the same for the multivariate case as for the univariate case. Similarly C2 implies that two quadratic forms∗ are independent in the multivariate case if and only if they are independent in the univariate case. Using these facts, for example, we could immediately generalize Cochran’s theorem to the multivariate case.

9186

WISHART DISTRIBUTION

We have seen that the Wishart distribution is a generalization of the χ 2 distribution. We now state other relationships between these distributions. Let a ∈ Rp , a = 0. D. a Wa/a a ∼ χn2 (a δa/a a). E. If δ = 0, n p, then a −1 a/a W−1 a ∼ 2 . χn−p+1 2 ( −1 δ). F. tr( −1 W) ∼ χnp

G. If δ = 0, n p, then |W|/|| = | −1 W| ∼ p 1 Ui , where the Ui are independent, 2 . Ui ∼ χn−i+1 We now discuss one of the most important properties of the Wishart distribution. Let X and W be independent, X ∼ Np (µ, ), W ∼ Wp (n, ), n p. Let F = (n − p − 1)p−1 X W−1 X. Then F ∼ Fp,n−p+1 (µ

−1

µ).

(5)

(Note that when p = 1, F is the square of an obvious t random variable.) np(n − p − 1)−1 F = T 2 is called Hotelling’s T 2∗ , whose distribution can be determined from (5). Our next topic is the Bartlett decomposition of the Wishart distribution. Let W ∼ Wp (n, I), n p. Let W = T T, where T is upper triangular with positive diagonal elements. Let tij be the (i, j) component of T. Then the tij are independent, t2ii ∼ χn2 , tij ∼ N(0, 1). The last result in this section is one which is useful in determining the distribution of sample partial correlation∗ coefficients. Partition W and as W11 W12 , W= W21 W22 11 12 = , 21 22 with W11 and 11 s × s. and W11 − W12 W−1 22 W21 12 −1 22 21 . Then

Let W11.2 = 11.2 = 11 −

H. W11 ∼ Ws (n, 11 ), W22 ∼ Wp−s (n, 22 ). If 12 = 0, then W11 and W22 are independent. I. W11.2 is independent of W12 and W22 , and W11.2 ∼ Ws (n − p + s, 11.2 ).

THE MULTIVARIATE ONE-SAMPLE MODEL We now look at the multivariate one-sample model as an example of how the results in the previous section are applied. In this model, we observe X1 , . . . , Xn independent, Xi ∼ Np (µ, ), where µ and > 0 are unknown parameters. Let X, X = n−1 i −1 S = (n − 1) (Xi − X)(Xi − X) . X is called the sample mean vector and S is called the sample covariance matrix. Note that S is a p × p matrix whose kth diagonal element is just the sample variance computed from the kth components of the Xi and whose (j, k)th off-diagonal element is just the sample covariance between the jth and kth components of the Xi . Result C2 in the last section implies that X and S are independent, (n − 1)S ∼ Wp (n − 1, ). By well-known results about the normal distribution, X ∼ Np (µ, n−1 ). Therefore, X and S are unbiased estimators of µ and [see (1)] and are consistent (see B). (X, S) is a complete sufficient statistic∗ for this model, so that X and S are minimum variance unbiased estimators∗ . We also note that S > 0 as long as n > p, which we henceforth assume. Now, consider testing that µ = 0. Using the fact that X and S are independent, n1/2 X ∼ Np (n1/2 µ, ) and (n − 1)S ∼ Wp (n − 1, ), we see from (5) that

F = c(n, p)X S−1 X ∼ Fp,n−p (nµ

−1

µ),

where c(n, p) = (n − p)np−1 (n − 1)−1 . A sensiα . [Note that ble test is to reject if F > Fp,n−p when p = 1, F = (n1/2 X/S)2 = t2 so that this test is a natural generalization of the univariate one sample t-test.] This is called the one-sample Hotelling’s T 2 test and is uniformly most powerful invariant, the likelihood ratio∗ test, unbiased, and admissible. It can also be derived from the unionintersection principle∗ . We now show how (5) can be used to generate simultaneous confidence intervals∗ for the set of all t µ, t ∈ Rp . Let F ∗ = c(n, p)(X − µ) S−1 (X − µ) 2 3 = c(n, p) sup {t (X − µ)}2 /t St . t

WISHART DISTRIBUTION

By (5), F ∗ ∼ Fp,n−p . Therefore,

P(t µ ∈ t X ± (t

α StFp,n−p /c(n, p))1/2 )

αi = 0. Let N=

for all t)

∗

= P(F

X.. = N −1

for all t = 0)

α Fp,n−p )

= 1 − α.

Any procedures for drawing inferences about the covariance matrix would be based on the sample covariance matrix, and, therefore, properties of these procedures would be based on the nonsingular central Wishart distribution whose density is given in (4). In particular, various sample correlation coefficients (simple, multiple, partial, canonical) are computed from S [or equivalently from (n − 1)S] and hence their joint and marginal densities can be determined from the density in (4). Let rn (ρ) be the distribution of the sample (simple) correlation coefficient∗ computed from a sample of size n with (true) correlation coefficient ρ. Let r∗ be a sample partial correlation coefficient computed from a sample of size n, conditionally on q variables, with (true) partial correlation coefficient ρ ∗ . Result I with s = 2 implies that r∗ ∼ rn−q (ρ ∗ ), so that any distribution theory developed for sample correlation coefficients can be immediately extended to sample partial correlation coefficients∗ . In addition, principal component analysis∗ and factor analysis∗ also use the sample covariance matrix S, so that their properties are also determined from the Wishart distribution. The determinant || is called the generalized variance∗ . It is often used as a real-valued measure of the variability in the sample. |S| is called the sample generalized variance. Its distribution is given in G.

MULTIVARIATE ANALYSIS OF VARIANCE In the one-way multivariate analysis of variance∗ (MANOVA) model, we observe Xij independent, Xij ∼ N(µ + α i , ), where µ and α i areunknown p-dimensional vectors such that ni α i = 0, and > 0 is an unknown p × p matrix. We want to test that the

ni , Xi. = n−1 i

i

= P((t (X − µ))2 /t St α /c(n, p) Fp,n−p

9187

i

Xij ,

i

Xij

j

and T1 =

ni (Xi. − X.. )(Xi. − X.. ) ,

i

T2 =

(Xij − Xi. )(Xij − Xi. ) , i

δ=

j

ni α i α i .

i

We are testing that δ = 0. The procedures that are used for this problem are based on R = (r1 , . . . , rb ), where r1 r2 · · · rb are the nonzero eigenvalues of T−1 2 T1 . (b is the minimum of k − 1 and p.) From C, we see that T1 and T2 are independent, T1 ∼ Wp (k − 1, , δ), T2 ∼ Wp (N − k, ). We must assume that N − k p, so that T2 is invertible. Typically, we do not assume that k − 1 p, so that T1 may have a singular Wishart distribution. T2 has a central Wishart distribution, but T1 has a possibly noncentral Wishart distribution. We are testing that the distribution is a central Wishart distribution. The joint distribution of R is rather complicated, depending on the non-singular central Wishart distribution of T2 and the possibly singular, possibly noncentral Wishart distribution of T1 . Invariance considerations can be used to show that the distribution depends only on the eigenvalues∗ of −1 δ. In particular, the null distribution does not depend on any unknown parameters, so that the null distribution of any test statistic based on R would be completely specified. FURTHER COMMENTS In the first two sections, we presented the definition of the Wishart distribution together with some of its properties. In the third

9188

WITCH OF AGNESI

section, we looked at the multivariate onesample model and saw that the sample covariance matrix has a Wishart distribution. For this model, the only relevant Wishart distribution is the nonsingular central Wishart distribution. However, in the one-way MANOVA model, we see that we need the singular Wishart distribution even for finding null distributions of statistics, and need the noncentral Wishart distribution for dealing with power functions (e.g., to show unbiasedness∗ or admissibility∗ of tests). Muirhead [11] presents a detailed treatment of both the central and noncentral Wishart distributions. It is the best book to read on this subject. Other textbooks which present the basic facts about the central Wishart distribution (together with some simple facts about the noncentral case) include Anderson [2], Arnold [3], Eaton [5], Giri [7], and Kshirsagar [9]. We now cite some historical papers. Fisher [6] derives the central Wishart density when p = 2. Wishart [13] finds the density function for general p in the central case. Anderson [1] derives the noncentral Wishart density when the noncentrality matrix has rank 1 or 2. Weibull [12] extends this result to the case of rank 3. Herz [8], James [9], and Constantine [4] derive different forms of the noncentral Wishart density for general rank. REFERENCES 1. Anderson, T. W. (1946). The non-central Wishart distribution and certain problems of multivariate statistics. Ann. Math. Statist., 17, 409–431. 2. Anderson, T. W. (1958). An Introduction to Multivariate Analysis. Wiley, New York. 3. Arnold, S. F. (1981). The Theory of Linear Models and Multivariate Analysis. Wiley, New York. 4. Constantine, A. G. (1963). Some non-central distribution problems in multivariate analysis. Ann. Math. Statist., 34, 1270–1285. 5. Eaton, M. L. (1983). Multivariate Statistics, A Vector Space Approach. Wiley, New York. 6. Fisher, R. A., (1915). Frequency distribution of the values of the correlation coefficient in samples from an infinitely large sample. Biometrika, 10, 507–521. 7. Giri, N. C. (1977). Multivariate Statistical Inference. Academic, New York.

8. Herz, C. S. (1955). Bessel functions of matrix argument. Ann. Math., 61, 474–523. 9. James, A. T. (1961). The distribution of noncentral means with known covariance. Ann. Math. Statist., 32, 874–882. 10. Kshirsagar, A. M. (1972). Multivariate Analysis. Dekker, New York. 11. Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, New York. 12. Weibull, M. (1953). The distribution of t- and F-statistics and of correlation and regression coefficients in stratified samples from normal populations with different means. Skand. Aktuar. (Suppl.), 36, 1–106. 13. Wishart, J. (1928). The generalized product moment distribution in samples from a normal multivariate population. Biometrika, 20, 32–52. See also CHI-SQUARE DISTRIBUTION; GENERALIZED VARIANCE; HOTELLING’S T 2 ; MULTIVARIATE ANALYSIS; MULTIVARIATE NORMAL DISTRIBUTIONS; and WILKS’S LAMBDA CRITERION.

STEVEN F. ARNOLD

WITCH OF AGNESI This is the name given to a curve with parametric equations x = a cot φ;

y = a sin2 φ.

This results in a curve with y proportional to (a2 + x2 )−1 . It is generated by a point P moving in the way indicated in Fig. 1. The curve is, in fact, in the form of a Cauchy distribution∗ PDF (x representing the random variable and y the PDF). The area under the curve is ∞ (x2 + a2 )−1 dx a3 −∞

= =

π/2 −π/2 π/2 −π/2

(cot2 φ + 1)−1 (sin2 φ)−1 dφ dφ = π

so the Y scale needs to be multiplied by π −1 to produce a PDF. The curve seems to have first appeared in the works of Fermat∗ in the middle of the seventeenth century. Stigler [1] gives a fascinating historical discussion of the

WITCH’S HAT DISTRIBUTIONS

9189

Figure 1.

relations between the curve and the Cauchy distribution. The name of the curve derives from Marie Gaetena Agnesi (1718–1799), who discussed the curve in 1748. She called the curve La Versiera (The Witch). Stigler [1] noted that this name had been used previously (in 1718), and discussed its etymology in some detail. REFERENCE 1. Stigler, S. M. (1974). Biometrika, 61, 375–380. See also CAUCHY DISTRIBUTION and POISSON, ´ SIMEON -DENIS.

WITCH’S HAT DISTRIBUTIONS The name of the distribution stems from the fact that, in two dimensions, it resembles a witch’s hat with a broad flat brim and a high conical peak. These distributions could be traced to Mathews [3] who uses it as a counterexample to the Gibbs mixture sampler. (The mixing time of the Gibbs’ sampler increases exponentially with dimension since all but one coordinate must be lined up with the peak before the Gibbs’ step can move from the brim to the peak and this has exponentially small probability.) Formally, the densities of the family are of the mixture form f (·, α) =

1−α α − αd u(·, α) + u(·, 1), 1 − αd 1 − αd

where u(·, α) is a uniform density (with respect to the Lebesgue measure) on [0, α]d , which has probability α under f (·, α), and

u(·, 1) is a uniform density on the ddimensional hypercube [0, 1]d , which has probability 1 − α under f (·, α). Note that α will be larger than the volume α d , and this is what slows down the convergence of the Gibbs’ sampler or any similar singlecoordinate Markov chain updating scheme. For more details, one may refer to Reference 2. Geyer and Thompson [1] considered a simplified witch’s hat distribution as follows. Let α and β be real numbers such that 0 α 1 and β 0. On the unit hypercube [0, 1]d , the nonnormalized density is 1 + β on the smaller hypercube [0, α]d and is 1 everywhere else in [0, 1]d . The same idea has been used in defining nicked and notched distributions on [0, 1]2 . By the way, the part of the distribution over the small hypercube is referred to as the ‘‘peak” and the rest as the ‘‘brim.” Various choices of α and β result in different members of the simplified witch’s family. Clearly, for α = 1 and β = 0, we obtain the uniform distribution on the whole unit hypercube [0, 1]d , which is referred to as a hot distribution. A typical distribution used in Gibbs’ sampler is a cold distribution with d = 30, α = 13 and β ≈ 1014 chosen so as to obtain the probability of the peak to be exactly 13 . In this case, the peak is practically an atom resulting in a regenerating Gibbs’ sampler with the mean regenerating time being Pr (1Peak) = 3. Here, the probability of leaving the peak in one scan of the sampler is 6 · 10−13 and the Gibbs’ sampler needs at least 329 ≈ 7 · 1012 scans to get close to mixing and about 102 times that number to obtain an accurate answer. REFERENCES 1. Geyer, C. J. and Thompson, E. A. (1995). Annealing Markov chain Monte Carlo with

9190

WITT DESIGNS

applications to ancestral inference. J. Am. Stat. Assoc., 90, 909–920. 2. Madras, N. and Piccioni, M. (1999). Importance sampling for families of distributions. Ann. Appl. Probab., 9, 1202–1225. 3. Mathews, P. (1993). A slowly mixing Markov chain with implications for Gibbs sampling. Stat. Probab. Lett., 17, 231–236.

Seminar der Hansisischen Universitaet, 12, 256–264. Sprott, D. A. (1955). Balanced incomplete block designs and tactical configurations. Ann. Math. Statist., 26, 752–758. See also BLOCKS, BALANCED INCOMPLETE; CYCLIC DESIGNS; DESIGN OF EXPERIMENTS; and t DESIGNS.

RALPH G. STANTON

WITT DESIGNS Witt designs are balanced incomplete block∗ designs with super balance. They are determined by three parameters, t, k, and v. They have also been called t designs∗ with unit t-set balance. In a Witt design there are v varieties arranged in blocks of k elements such that every t-set occurs exactly once. It is easy to see that these are special balanced incomplete block designs with y k b= , t t v−1 k−1 r= t−1 t−1 v−2 k−2 λ= . t−2 t−2 Not only are these designs balanced on pairs, they are balanced on i-sets (i = 2, 3, . . . , t) and the number of occurrences of each i-set is given by v−i k−i . t−i t−i The smallest practical design with t > 2 is the (3, 4, 14) design generated by cycling (∞124) and (3567), modulo 7. As a balanced incomplete block design, this design has parameters (8, 14, 7, 4, 3); every pair occurs three times, and every triple occurs once. Sprott has discussed the features of the particularly interesting Witt design on 24 symbols, block size 8, and t = 5; it has parameters (24, 759, 253, 8, 77). To date, the extra balance features of Witt designs have not been exploited statistically. BIBLIOGRAPHY Witt, E. (1938). Ueber Steinersche Systems. Abhandlungen aus dem Mathematischen

WOLD, HERMAN O. A. Born: December 25, 1908, in Skien, Norway. Died: February 16, 1992, in Uppsala, Sweden. Contributed to: time-series analysis, econometrics, multivariate analysis. Herman Ole Andreas Wold was born in southern Norway in 1908 as the sixth child of Edvard and Betsy Wold. Due to hard economic times in Norway, the family, except the two oldest children, moved to Lidköping in southwestern Sweden in 1912, and became Swedish citizens. Here Edvard Wold started a small fur and hide business, which still operates today under the ownership of Torbjörn Wold, a nephew of Herman Wold. Herman Wold grew up and went to elementary school in Lidköping, but at that time there was no high school in town, and he therefore went to high school in Skara 40 kilometers (25 miles) away. In 1927 Wold enrolled in the University of Stockholm, where he graduated in 1930 with a degree in mathematics, economics, and physics. Harald Cramér∗ was one of his teachers, and Wold become very interested in Cramér’s work and in statistics. After some years’ work in an insurance company, Wold enrolled as Cramér’s graduate student, and in 1938 he presented his thesis ‘‘A study in the analysis of stationary time series.’’ In the thesis he proved a theorem about the decomposition of a time series∗ into two components, the Wold decomposition, which forms an essential element in the foundation of time-series analysis and forecasting. Basically, Wold showed that any stationary time series can be separated into a deterministic component precisely predictable

WOLD, HERMAN O. A.

from its past, plus a random component which can be modeled as a weighted sum of innovations. Wold’s decomposition showed that the three classic time-series models–the model of hidden periodicities, the moving-average model, and the autoregressive model–could be seen as different cases of the same general time-series model. This theorem is also referred to as the Cramér–Wold device, and had an immediate impact on the fields of statistics and economics. Ragnar Frisch was the first opponent at the dissertation, and was very critical of the decomposition theorem. After a long and lively dissertation Wold passed with honors, however, and became Docent of statistics and insurance mathematics at Stockholm University. After a few more years in Stockholm, Wold became the first professor of statistics at Uppsala University in 1942, where he stayed until 1970. He then moved to Gothenburg as professor of statistics, staying until his retirement in 1975, when he moved back to Uppsala. He married Anna-Lisa Arrhenius in 1940, and they had three children: Svante, Maria, and Agnes. The three children all became scientists—Agnes an immunologist at Gothenburg University, Maria a data analyst at Pharmacia AB in Uppsala, and Svante a chemometrician at Umea˚ University. Svante Wold collaborated with Herman Wold on multivariate modeling and analysis of chemical systems and processes, and continues the development of the partial leastsquares∗ methodology as applied in natural science and technology. Wold liked to travel, and visited many statistics and econometrics departments all over the world for both short and long periods, among them Columbia University in New York, the Wharton School in Philadelphia, the University of Wisconsin at Madison, and the University of Geneva, Switzerland. After the dissertation, Wold’s research was centered on demand analysis and econometrics; his book Demand Analysis written in collaboration with his assistant Lars Jureén and published in 1952, became a classic in the field. He continued to work with the modeling of complicated econometric systems, and was very active in the international discussion

9191

about how best to formulate and estimate these models. In the 1960s he developed the so-called fixpoint method∗ of estimation for econometric systems. This solved some problems for systems with many variables and collinear variables. This, in turn, led him to his final domain of interest, multivariate analysis using ‘‘soft’’ modeling and projection methods [principal-components analysis and its extension, partial least-squares (PLS) projection to latent structures]. Wold saw these methods as least squares counterparts of multivariate maximum likelihood (ML) models such as ML factor analysis and LISREL∗ . The latter class of methods were developed by his pupil Jöreskog, who together with Wold edited two volumes on the PLS and ML approaches to multivariate analysis, namely Systems under Indirect Observation (NorthHolland, Amsterdam, 1982). Wold was a very practical man, and wanted estimation and modeling methods to work with a minimum of assumptions, for incomplete data, with many variables and collinear variables, etc.; and he developed PLS accordingly. This had made PLS of great use for the analysis of large collinear data sets in the social sciences, business economics, and market research. PLS, in its simplest two-block form, is also extensively employed in chemistry, chemicalengineering, chemometrics, and parts of biology and medicine for applications such as multivariate calibration, composition–property and structure–property modeling, image analysis, environmental analysis, and the monitoring, modeling, and optimization of chemical and biological processes. The Swedish Chemical Society in 1995 established the Herman Wold Medal in memory of his contributions to chemistry and chemometrics, to be given to a distinguished chemist who creatively uses statistics in her/his research. Wold saw teaching as equally important as research, and spent much time on developing undergraduate and graduate courses of statistics; he also wrote two undergraduate textbooks (in Swedish). Besides writing and editing the books mentioned above, Wold was the editor or coeditor of a few other books, and author or coauthor of around 40 scientific papers, mainly in statistics and econometrics.

9192

WOLFOWITZ, JACOB

Wold was a Fellow of the Institute of Mathematical Statistics∗ , the American Statistical Association∗ , and the Swedish Academy of Sciences, a Honorary Fellow of the Royal Statistical Society∗ , and an honorary member of the American Economic Association and of the American Academy of Arts and Sciences. He served as vice president of the International Statistical Institute∗ in 1957–1961, and as president of the Econometric Society in 1966. He was awarded honorary doctorates by the Technical University of Lissabon ˚ and the University of Abo, Finland. SVANTE WOLD

WOLFOWITZ, JACOB Born: March 19, 1910 in Warsaw, Poland. Died: July 16, 1981 in Tampa, Florida. Contributed to: statistical inference, sequential analysis, inventory theory, queuing theory, information theory, decision theory. Jacob Wolfowitz was born in Warsaw, Poland on March 19, 1910, and came to the United States with his family in 1920. He received the baccalaureate from the College of the City of New York in 1931. Positions were scarce in 1931, a year of severe economic depression, and he supported himself as a high school teacher while studying for the doctorate in mathematics at New York University. He received the Ph.D. degree in 1942. Wolfowitz had met Abraham Wald∗ in the autumn of 1938, when Wald came to Columbia University to study statistics with Harold Hotelling∗ . Wald and Wolfowitz quickly became close friends and collaborators, their first joint paper [18] appearing in 1939. During the period of United States involvement in World War II, they worked together on war-related research at the Statistics Research Group of Columbia University (see MILITARY STATISTICS). In 1945 Wolfowitz became Associate Professor at the University of North Carolina at Chapel Hill. In 1946 he joined the faculty of Columbia University, leaving in 1951 to join the Department of Mathematics at Cornell University. In 1970 he became Professor of Mathematics at the University of Illinois in Urbana.

After retiring from the University of Illinois in 1978, he became Distinguished Professor of Mathematics at the University of South Florida in Tampa, a position he held until his death following a heart attack, on July 16, 1981. He had held visiting professorships at the University of California at Los Angeles, at the Universities of Paris and Heidelberg, and at the Technion-Israel Institute of Technology in Haifa. Wolfowitz’s research is remarkable for its combination of breadth and depth. He made important contributions to all of the major areas of mathematical statistics, and also to inventory theory∗ , queuing theory∗ , and information theory∗ . Several of his papers make contributions to several different areas simultaneously. Wolfowitz’s earliest research interest was nonparametric inference. His first two published papers, written jointly with Wald, were on nonparametric inference: ref. 18 constructs a confidence band∗ for an unknown continuous cumulative distribution function based on a random sample from the distribution; ref. 19 proposes and analyzes the celebrated two-sample test based on runs∗ . Wolfowitz wrote several other papers on the theory and application of runs, ref. 27 containing an application to quality control∗ . The term ‘‘nonparametric’’ was originated by Wolfowitz in ref. 26. His interest in nonparametric inference did not end with these early papers. In ref. 7, with Dvoretzky and Kiefer, he proved that the empirical cumulative distribution function (see EMPIRICAL DISTRIBUTION FUNCTION (EDF) STATISTICS) is an asymptotically minimax∗ estimator of the population cumulative distribution function for a variety of reasonable loss functions. In refs. 13 and 15 Kiefer and Wolfowitz extended these results to the problem of estimating joint cumulative distribution functions. Wolfowitz’s research on the minimum distance∗ method is an application of techniques developed in nonparametric inference to parametric inference. The method estimates the unknown parameters by those values of the parameters that minimize a distance between the empirical cumulative distribution function and the parametric family of cumulative distribution functions. This method gives consistent estimators in some

WOLFOWITZ, JACOB

very complicated problems. The papers in refs. 32–35 and the joint paper [9] with Kac and Kiefer cover the development, analysis, and applications of the minimum distance method. Starting with ref. 28, which discusses the sequential estimation∗ of a Bernoulli parameter, Wolfowitz made many important contributions to sequential analysis∗ . In ref. 29 he developed a Cramér–Rao∗ type of lower bound for the variance of an estimator based on sequential sampling, under certain regularity conditions (see WOLFOWITZ INEQUALITY). In ref. 21 he and Wald studied the structure of Bayes decision rules when sequential sampling is used. One of the papers [20] he was proudest of was written with Wald and proves the optimum character of the Wald sequential probability ratio test. In ref. 8 Dvoretzky, Wald, and Wolfowitz showed that randomization can be eliminated in sequential decision problems under certain conditions. In ref. 22 Wald and Wolfowitz showed that under mild conditions, in sequential decision problems if randomization is used after each observation, we get the same class of risk functions as when randomization is used only once, to choose a nonrandomized decision rule at the start of the process. Dvoretzky, Kiefer and Wolfowitz [4,5] solved sequential decision problems when observation is continuous over time. In ref. 31 Wolfowitz showed that the optimal sequential estimator of a normal mean when the variance is known is essentially a fixed sample size estimator. In ref. 24 Weiss and Wolfowitz constructed an asymptotically efficient sequential equivalent of Student’s t-test∗ , and in ref. 23 these authors used an adaptive sequential scheme to construct optimal fixed length nonparametric estimators of translation parameters. Kiefer and Wolfowitz [10] modified the Robbins–Monro stochastic approximation∗ procedure to estimate the point at which an unknown regression function achieves its maximum (see REGRESSION FUNCTION: KIEFER –WOLFOWITZ MINIMIZATION). In a regression∗ model, a particular choice of the values of the independent variables is called a ‘‘design.’’ An optimal design is a design which enables the user of the model

9193

to estimate given functions of the unknown regression coefficients as efficiently as possible. Kiefer and Wolfowitz [14,16,17] made important contributions to the theory underlying the construction of optimal designs (see OPTIMUM DESIGN, KIEFER –WOLFOWITZ EQUIVALENCE THEOREM FOR). The inventory∗ problem is the problem of deciding how much inventory to hold during each of a sequence of time periods, when there are penalties for holding either too much or too little inventory and demand for the product is random. Dvoretzky, Kiefer and Wolfowitz [2,3,6] made pioneering contributions to this subject and really started the subject known nowadays as ‘‘dynamic programming’’; this is the theory of which sequence of nonsampling decisions is optimal, when a decision must be made in each time period in a sequence of time periods. In ref. 6 the authors showed that under certain circumstances the well-known (s, S) policy is optimal: this policy is to order enough to make the total inventory equal to S as soon as the stock on hand goes below s. Wolfowitz’s research on maximum likelihood∗ estimators started with ref. 30 and led to the development, in collaboration with Weiss, of maximum probability estimators∗ . For large samples, these estimators have the highest probability of being close to the true unknown parameters, among a wide class of estimators they often coincide with maximum likelihood estimators, but exist in cases where the latter do not. The monograph in ref. 25 describes most of the results in this area. Kiefer and Wolfowitz [11,12] made fundamental contributions to the theory of queues with many servers, by showing the existence of limiting distributions of waiting times and queue lengths as time approaches infinity. Starting in 1957, Wolfowitz [36] devoted a rapidly increasing proportion of his time to what he called coding theorems of information theory∗ , describing how rapidly information can be sent when random errors occur in the transmission and the probability of correct decipherment must be at least equal to a preassigned value. This problem can be considered as a generalization of statistical decision theory∗ , in the

9194

WOLFOWITZ, JACOB

following sense. In statistical decision theory, there is an unknown parameter with a given set of possible values, and based on observed random variables whose distribution depends on the parameter, we must guess the value of the parameter. In coding theory, we have the additional choice of the set of possible values of the parameter: Each value in the set we choose becomes one of the entries in our codebook, the codebook being simply a list of the words which we are allowed to transmit over the channel. We want to choose as many words as possible, but in such a way as to achieve the desired lower bound on correct decipherment. Wolfowitz proved both direct coding theorems, which state that the codebook can contain at least a certain number of words, and converse theorems which state that the codebook cannot contain more than a certain number of words. His work in this area represents deep generalizations of the theory which existed before he started his research. Most of his results are contained in a monograph [39]. In addition to the mathematical theory, Wolfowitz was interested in practical and philosophical issues. Reference 37 contains a criticism of a set of axioms used to support the Bayesian∗ approach to statistical decision theory. Reference 38 contains an interesting criticism of the theory of testing hypotheses∗ for not having practical application. Wolfowitz was a renowned teacher and lecturer, unsurpassed in his ability to clarify the intuition underlying the most complicated results. He was selected as Rietz Lecturer and as Wald Lecturer by the Institute of Mathematical Statistics, and as Shannon Lecturer by the Institute of Electrical and Electronic Engineers. His list of other academic honors is a long one: an honorary doctorate from the Technion; election to the U.S. National Academy of Sciences and to the American Academy of Arts and Sciences; election as a Fellow of the International Statistics Institute, the Econometric Society, the American Statistical Association, and the Institute of Mathematical Statistics; a term as President of the Institute of Mathematical Statistics; Visiting Professorships at several universities; and selection as a Guggenheim Fellow.

Wolfowitz’s reading was not confined to mathematical subjects. He read detective stories for relaxation and kept up with political and social conditions in all of the large nations of the world and many of the smaller ones. He was a man of strong opinions, with a particular detestation of tyranny. He took a leading part in organizing protests against Soviet repression of minorities and dissidents, and was able to aid several victims of such repression. A fuller account of Wolfowitz’s research can be found in ref. 1. A complete list of his 120 publications is given in ref. 40. REFERENCES 1. Augustin, U., Kiefer, J., and Weiss, L. (1980). In Jacob Wolfowitz: Selected Papers, J. Kiefer, ed. Springer, New York, pp. ix–xxi. 2. Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1952a). Econometrica, 20, 187–222. 3. Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1952b). Econometrica, 20, 450–466. 4. Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1953a). Ann. Math. Statist., 24, 254–264. 5. Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1953b). Ann. Math. Statist., 24, 403–415. 6. Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1953c). Econometrica, 21, 586–596. 7. Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1956). Ann. Math. Statist., 27, 642–669. 8. Dvoretzky, A., Wald, A., and Wolfowitz, J. (1951). Ann. Math. Statist., 22, 1–21. 9. Kac, M., Kiefer, J., and Wolfowitz, J. (1955). Ann. Math. Statist., 26, 189–211. 10. Kiefer, J. and Wolfowitz, J. (1952). Ann. Math. Statist., 23, 462–466. 11. Kiefer, J. and Wolfowitz, J. (1955). Trans. Amer. Math. Soc., 78, 1–18. 12. Kiefer, J. and Wolfowitz, J. (1956). Ann. Math. Statist., 27, 147–161. 13. Kiefer, J. and Wolfowitz, J. (1958). Trans. Amer. Math. Soc., 87, 173–186. 14. Kiefer, J. and Wolfowitz, J. (1959a). Ann. Math. Statist., 30, 271–294. 15. Kiefer, J. and Wolfowitz, J. (1959b). Ann. Math. Statist., 30, 463–489. 16. Kiefer, J. and Wolfowitz, J. (1960). Canad. J. Math., 12, 363–366. 17. Kiefer, J. and Wolfowitz, J. (1965). Ann. Math. Statist., 36, 1627–1655.

WOOLF’S TEST

9195

18. Wald, A. and Wolfowitz, J. (1939). Ann. Math. Statist., 10, 105–118.

WOLFOWITZ INEQUALITY

19. Wald, A. and Wolfowitz, J. (1940). Ann. Math. Statist., 11, 147–162.

A generalization of the Cramér-Rao inequality∗ to any sequential unbiased estimator φ(X) of a parametric function g(θ ), which states that

20. Wald, A. and Wolfowitz, J. (1948). Ann. Math. Statist., 19, 326–339. 21. Wald, A. and Wolfowitz, J. (1950). Ann. Math. Statist., 21, 82–99. 22. Wald, A. and Wolfowitz, J. (1951). Ann. Math., 53, 581–586. 23. Weiss, L. and Wolfowitz, J. (1972a). Z. Wahrsch. verw. Geb., 24, 203–209. 24. Weiss, L. and Wolfowitz, J. (1972b). J. R. Statist. Soc. B, 34, 456–460. 25. Weiss, L. and Wolfowitz, J. (1974). Maximum Probability Estimators and Related Topics. Springer, New York. 26. Wolfowitz, J. (1942). Ann. Math. Statist., 13, 247–279. 27. Wolfowitz, J. (1943). Ann. Math. Statist., 14, 280–288. 28. Wolfowitz, J. (1946). Ann. Math. Statist., 17, 489–493. 29. Wolfowitz, J. (1947). Ann. Math. Statist., 18, 215–230. 30. Wolfowitz, J. (1949). Ann. Math. Statist., 20, 601–602. 31. Wolfowitz, J. (1950). Ann. Math. Statist., 21, 218–230. 32. Wolfowitz, J. (1952). Skand. Aktuar., 35, 132–151. 33. Wolfowitz, J. (1953). Ann. Inst. Statist. Math., 5, 9–23. 34. Wolfowitz, J. (1954). Ann. Math. Statist., 25, 203–217. 35. Wolfowitz, J. (1957a). Ann. Math. Statist., 28, 75–88. 36. Wolfowitz, J. (1957b). Illinois J. Math., 1, 591–606. 37. Wolfowitz, J. (1962). Econometrica, 30, 470–479. 38. Wolfowitz, J. (1967). The New York Statistician, 18, 1–3. 39. Wolfowitz, J. (1978). Coding Theorems of Information Theory, 3rd ed. Springer, New York. 40. Wolfowitz, J. (1980). Springer, New York.

Selected

Papers.

See also ESTIMATION, CLASSICAL; INFORMATION THEORY AND CODING THEORY; SEQUENTIAL ANALYSIS; and WALD, ABRAHAM.

L. WEISS

Varθ (φ(X)) (g (θ ))2 /(Eθ [N]I(θ )) for every θ ∈ H (the space of the values of the parameter). This unequality is valid under similar regularity conditions as its fixed-size sample analog. Here N is the (random) sample size and I(θ ) is the amount of information∗ , I(θ ) = Eθ

∂ log f (X, θ ) ∂θ

2 -

corresponding to density fX (·). REFERENCE 1. Wolfowitz, J. (1947). Ann. Math. Statist., 18, 215–230. ´ –RAO LOWER BOUND; SEQUENTIAL See also CRAMER ANALYSIS; and SEQUENTIAL ESTIMATION.

WOOLF’S TEST Woolf’s test considers an unordered set of two-by-two tables∗ and tests for heterogeneity of odds ratios (see ODDS RATIO ESTIMATORS). The calculations are all carried out using the logarithm of the odds ratio. If there are k tables of the form ai bi , ci di ˆ i = ai di /(bi ci )(i = each giving an estimate 1, . . . , k) for the odds ratio, with sampling ˆ i , then, on the null variance 1/wi for ln hypothesis of homogeneity of odds ratios,

ˆ i )2 − V wi (ln

&

î wi ln

'2

,

where V = 1/ wi , is treated as a χ 2 statistic with k − 1 degrees of freedom∗ . If it is thought appropriate to find an overall estimate for ln (for example, if the above test yields

9196

WRAPPED DISTRIBUTIONS

î a nonsignificant result), then V wi ln is calculated, and its sampling variance is V. Hence confidence intervals may be found for ln , and these may be exponentiated to give (asymmetrical) confidence intervals for itself. In common with other methods based on the odds ratio, this technique requires large samples, and in particular there can be no ˆ i is zeros in the tables, as the variance of ln estimated as 1 1 1 1 1 = + + + . wi ai bi ci di In such cases the Mantel–Haenszel statistic∗ is to be preferred. Woolf’s paper [3] was published four years before that of Mantel and Haenszel, who did not refer to it, though they did recognize the possibility of ‘‘weighting the several subclassification estimates according to their relative precisions’’ [1]. To some extent the two methods have been in competition, and Woolf’s has proved the less popular, appearing in the Science Citation Index about a quarter as often over the last 30 years. It is interesting to speculate on some of the possible reasons, apart from the intrinsic features of the methods themselves, such as ease of calculation—for example, where the authors worked, the lengths of the original articles, and the journals in which they appeared. Barnet Woolf was an applied statistician at the University of Edinburgh, and the method was first presented in the context of combining data from several centers in Britain to evaluate the strength of association between blood groups and disease. He also made an early contribution to stepwise regression∗ [2]. Because Woolf provided a test for homogeneity as well as a way of combining oddsratio estimates, his method forms a useful pedagogical link from the estimation of the odds ratio for a single two-by-two table∗ to the analysis of multiway contingency tables∗ by generalized linear models∗ . REFERENCES 1. Mantel, N. and Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. J. Nat. Cancer Inst., 22, 719–478.

2. Woolf, B. (1951). Computation and interpretation of multiple regression. J. R. Statist. Soc. B., 13, 100–119. 3. Woolf, B. (1955). On estimating the relation between blood group and disease. Ann. Hum. Genet., 19, 251–253. See also LOG-LINEAR MODELS IN CONTINGENCY TABLES; MANTEL –HAENSZEL STATISTIC; ODDS RATIO ESTIMATORS; and TWO-BY-TWO (2 × 2) TABLES.

DAVID APPLETON

WRAPPED DISTRIBUTIONS Suppose a random variable X is measured on a scale with scale interval α, so that the possible measured values are {Nα}, where N is an integer (positive, negative, or zero). Then the value of N scaled to X is nα (X), where αnα (X) − 12 α < X αnα (X) + 12 α. The deviation Uα = X − nα (X)α has PDF fUα (u) =

∞

fX (nα − u)

n=−∞

(− 21 α u 12 α), where f (·) is the PDF of X. Distributions with PDFs of this type are called wrapped (or wrapped-up) distributions. They appear naturally in models of directional data∗ (see [1]). Stadje [2] gives a discussion of wrapped distributions in the context of measurement error. REFERENCES 1. Mardia, K. V. (1972). Statistics of Directional Data. Academic, New York. 2. Stadje, W. (1984). Metrika, 31, 303–317. See also CIRCULAR NORMAL DISTRIBUTION; DIRECTIONAL DISTRIBUTIONS; MEASUREMENT ERROR; and WRAPPED-UP CAUCHY DISTRIBUTION.

WRAPPED-UP CAUCHY DISTRIBUTION This is a distribution obtained by wrapping a Cauchy distribution∗ around a circle and

WU–HAUSMAN SPECIFICATION TEST

adding up the probability densities (PDFs) coinciding at each point. With mean angle θ and mean vector length ρ, the PDF of a variable T with this distribution is 1 − ρ2 1 , 2 2π 1 + ρ − 2ρ cos(t − θ )

0 t < 2π.

The distribution is unimodal and symmetric. Further details are available in [1].

9197

REFERENCE 1. Batschelet, E. (1981). Circular Statistics in Biology. Academic, New York. See also DIRECTIONAL DISTRIBUTIONS and WRAPPED DISTRIBUTIONS.

WN2 -TEST. See GOODNESS OF FIT WU–HAUSMAN SPECIFICATION TEST. See HAUSMAN SPECIFICATION TEST

X X-11 METHOD The X-11 method refers to a computer program for seasonal adjustment of quarterly or monthly economic time series∗ maintained by the U.S. Bureau of the Census∗ and to the methodology employed by the program. The program is the eleventh and last in a sequence of programs developed in the late 1950s and early 1960s at the Census Bureau under the direction of Julius Shiskin. Technical Paper No. 15 [12] remains the authoritative document on the contents of the program and contains references on the statistical methodology. The X-11 method is widely used by government agencies and private businesses in the United States and many foreign countries. The paradigm used in formulating the seasonal adjustment problem is that of unobservable components. Let Mt represent the observed monthly series in Fig. 1 of demand deposits at commercial bnks, with t indexing the months. The series is viewed as containing seasonal variation St , trend and business cycle variation Pt , and random or irregular variation It . Whereas the additive representation Mt = Pt + St + It may be used, one frequently observes seasonal movements proportional to the level of the series, as in Fig. 1. In such cases the alternative representation Mt = Pt St It is adopted, and the St are called seasonal factors. The seasonally adjusted series Mta is Mt with St removed. For the additive representation Mta = Mt − St , whereas for the multiplicative formulation Mta = Mt /St . The quality of the seasonal adjustment process rests on the validity of the component representation and the accuracy of the estimate of St . The techniques for estimating St contained in X-11 make use of smoothing or graduation∗ formulas developed in the early 1900s [9,14]. These provide several sets of moving average weights or filters, which are applied to the data. The series is first detrended to center it about 0 (additive) or 1 (multiplicative) and then averaged by month (all Januarys, etc.) to get the seasonal deviation from center. This 9198

Figure 1.

Figure 2.

process is repeated with moving averages∗ of different widths and with downweighting of selected observations based on the size of irregular estimates. The seasonal factor estimate for demand deposits is shown in Fig. 2. The use of local symmetric averages of each month to estimate St , rather than a uniform average over all occurrences of the month, allows for evolving seasonal patterns. Asymmetric versions of these averages are used near the ends of the series. Some series, e.g., retail sales, reflect a definite daily pattern related to the day of the week. When the daily values are summed to get a monthly value, the effect of having five of some weekdays and four of others is

X-RAY CRYSTALLOGRAPHY

observable. An optional regression procedure is contained in X-11 to handle such series. Although X-11 is a nonlinear procedure because of its treatment of outliers∗ , linear approximations excluding this feature have been published and studied [3,13,15] in addition to studies of observed results [10,11]. A recent modification, known as X-11 ARIMA∗ , uses forecasting∗ models to improve adjustments near the end of the series and to obtain seasonal factor projections [5]. The SABL computer program [4] adopts a philosophy similar to X-11, but uses robust versions of moving averages rather than the X-11 outlier procedure. The proceedings of a 1976 conference on seasonal adjustment [16] gives an excellent overview of issues in seasonal adjustment and how they relate to X-11 and the other methodologies. Alternative strategies for seasonal adjustment based on statistical time series models are described there and elsewhere [1,2,6–8].

12. Shiskin, J., Young, A., and Musgrave, J. C. (1965). The X-11 Variant of Census Method II Seasonal Adjustment Program. Technical Paper 15, U.S. Department of Commerce, Bureau of the Census, Washington, D.C. 13. Wallis, K. F. (1974). J. Amer. Statist. Ass., 69, 18–31. 14. Whittaker, E. and Robinson, G. (1944). The Calculus of Observations. Blackie and Son, London, England. 15. Young, A. (1968). J. Amer. Statist. Ass., 63, 445–471. 16. Zellner, A., ed. (1976). Seasonal Analysis of Economic Time Series. U.S. Department of Commerce, Bureau of the Census, Washington, D.C. See also AUTOREGRESSIVE –INTEGRATED MOVING AVERAGE (ARIMA) MODELS; FORECASTING; GRADUATION; MOVING AVERAGES; SEASONALITY; and TIME SERIES.

WILLIAM P. CLEVELAND

X-BAR CHART. See CONTROL CHARTS

REFERENCES 1. Burman, J. P. (1980). J. R. Statist. Soc. A, 143, 321–337. 2. Cleveland, W. P. and Dempster, A. P. (1980). Proc. Bus. Econ. Statist. Sect. Amer. Statist. Ass., 30–36. 3. Cleveland, W. P. and Tiao, G. C. J. Amer. Statist. Ass., 71, 581–587.

9199

(1976).

4. Cleveland, W. S., Dunn, D. M., and Terpenning, I. J. (1976). In Seasonal Analysis of Economic Time Series, A. Zellner, ed. U.S. Department of Commerce, Bureau of the Census, Washington, D.C., pp. 201–231. 5. Dagum, E. B. (1980). The X-11 ARIMA Seasonal Adjustment Method. Statistics Canada, Ottawa. 6. Gersch, W. and Kitagawa, G. (1983). J. Bus. Econ. Statist., 1, 253–264. 7. Havenner, A. and Swamy, P. A. V. B. (1981). J. Econometrics, 15, 177–209. 8. Hillmer, S. C. and Tiao, G. C. (1982). J. Amer. Statist. Ass., 77, 63–70. 9. Macaulay, F. R. (1931). The Smoothing of Time Series. National Bureau of Economic Research. 10. Nerlove, M. (1965). J. Amer. Statist. Ass., 60, 442–491. 11. Rosenblatt, H. M. (1968). J. Amer. Statist. Ass., 63, 472–501.

X-RAY CRYSTALLOGRAPHY THE PHASE PROBLEM The atomic arrangement in the unit cell of a crystal, i.e., the crystal structure, is determined once the intensities of a sufficient number of x-ray diffraction maxima have been measured. The number of these intensities usually exceeds by far the number of parameters required to describe the structure. From these intensities a set of numbers |EH | can be derived, one corresponding to each intensity. However the elucidation of the crystal structure requires also a knowledge of the complex numbers EH = |EH | exp(iφH ), of which only the magnitudes |EH | can be determined from experiment. Thus a ‘‘phase’’ φH must be assigned to each |EH |, and the problem of determining the phases when only the magnitudes |EH | are known is called ‘‘the phase problem.’’ Owing to the known atomicity of crystal structures and the redundancy of observed magnitudes |EH |, the phase problem is solvable in principle. It is the redundancy of the system of equations relating the maginitudes |E| with

9200


the desired phases φ, as well as errors in the observed |E|’s, which makes possible, even indispensable, the use of probabilistic techniques in the solution of the phase problem. THE NORMALIZED STRUCTURE FACTORS E The relationship between the (complex) normalized structure factors E and the crystal structure is given by the pair of equations EH = |EH | exp(iφH ) =

N 1 1/2

σ2

Zj exp(2π iH · rj ),

(1)

j=1

< EH exp(−2π iH · r) >H

=

Zj 1/2 σ2

=0

 if r = rj  if f = r  j

(2) where H is an arbitrary reciprocal lattice vector, Zj is the atomic number and rj is the position vector of the atom labeled j, N is the number of atoms in the unit cell, and σ2 =

N

Z2j .

(3)

j=1

Clearly (2) shows that the crystal structure (i.e., the position vectors rj ) is determined in terms of the normalized structure factors EH . However, it turns out that although the magnitudes |EH | may be determined, at least approximately, from experiment, the phases φH , which are also needed if the crystal structure is to be found via (2), cannot be determined experimentally. Nevertheless, because the number of equations (1) usually exceeds by far the number of unknowns rj , the available data, i.e., the known |EH |’s, are in general more than sufficient to determine crystal structures uniquely. In fact, from (1) one naturally formulates the problem as the determination of the N position vectors rj which minimize the weighted sum of squares H

2 N 1 Wj |EH | − 1/2 Zj exp(2π iH · rj ) , σ2 j=1 

(4) in which the sum is taken over all reciprocal lattice vectors H for which magnitudes

|EH | are available, and the WH ’s are a suitably chosen set of weights. This formulation clearly calls for a probabilistic approach. In practice the problem of finding the global minimum of (4) is too intractable to be solved ab initio. Instead the unknown position vectors rj are eliminated from the system (1) to yield relationships among the EH ’s having probabilistic validity. These in turn lead to approximate values of the unknown phases φH which can then be used in (2) to determine a trial structure, i.e., approximate values of the unknowns rj . Employing standard iterative techniques (4), or something similar, is then used to obtain refined values for the rj . The techniques which employ the phases to determine crystal structures are known as direct methods, since the phases φH are determined directly from the observed magnitudes |EH | [rather than from a presumed known structure via (1)]. THE STRUCTURE INVARIANTS Equation (2) implies that the normalized structure factors EH determine the crystal structure. However (1) does not imply that, conversely, the crystal structure determines the values of the normalized structure factors EH since the position vectors rj depend not only on the structure but on the choice of origin as well. It turns out nevertheless that the magnitudes |EH | of the normalized structure factors are in fact uniquely determined by the crystal structure and are independent of the choice of origin, but that the values of the phases φH depend also on the choice of origin. Although the values of the individual phases depend on the structure and the choice of origin, there exist certain linear combinations of the phases, the socalled structure invariants, whose values are determined by the structure alone and are independent of the choice of origin. The most important structure invariants are the linear combinations of three phases (triplets): φH + φK + φL ,

(5)

H + K + L = 0;

(6)

where


and the linear combination of four phases (quartets): φH + φK + φL + φM ,

H + K + L + M = 0;

magnitudes |EH |, |EK |, |EL |.

(7)

where (8)

Two structures related by reflection through a point are said to be enantiomorphs of each other. The x-ray diffraction experiment is not capable of distinguishing the enantiomorphs when they are distinct. For this reason the fundamental principle of direct methods is formulated as follows. For fixed enantiomorph the observed magnitudes |E| determine, in general, unique values for all the structure invariants. The latter, as certain well defined linear combinations of the phases, lead in turn to unique values for the phases, φ. In short, the structure invariants serve to link the observed magnitudes |E| with the desired phases φ (the fundamental principle of direct methods). THE NEIGHBORHOOD PRINCIPLE It has been seen that for fixed enantiomorph the values of the observed magnitudes |E| determine the values of all the structure invariants. A major recent insight is that, for fixed enantiomorph, the value of any structure invariant T is primarily determined, in favorable cases, by the values of one or more small sets of magnitudes |E|, the neighborhoods of T, and is relatively insensitive to the values of the great bulk of remaining magnitudes |E| (the neighborhood principle). The conditional probability distribution of T, assuming as known the magnitudes |E| in any of its neighborhoods, yields an estimate for T which is particularly good in the favorable case that the variance of the distribution happens to be small.

(9)

The first neighborhood of the quartet (7), where (8) holds, consists of the four magnitudes

etc. THE FUNDAMENTAL PRINCIPLE OF DIRECT METHODS

9201

|EH |, |EK |, |EL |, |EM |.

(10)

The second neighborhood of the quartet consists of the four magnitudes (10) in the first neighborhood plus the three additional magnitudes |EH+K |, |EK+L |, |EL+H |,

(11)

i.e., seven magnitudes |E| in all. CONDITIONAL PROBABILITY DISTRIBUTIONS OF THE STRUCTURE INVARIANTS Triplets Suppose that a crystal structure consisting of N atoms per unit cell is fixed. Denote by W the collection of all reciprocal lattice vectors H and by φH the phase of the normalized structure factor EH . Assume also that R1 , R2 , and R3 are fixed nonnegative numbers. Suppose finally that the primitive random variable (vector) is the ordered triple (H, K, L) of reciprocal vectors H, K, L which is assumed to be uniformly distributed over the subset of the threefold Cartesian product W × W × W defined by |EH | = R1 ,

|EK | = R2 ,

|EL | = R3

(12)

and H + K + L = 0.

(13)

Then the structure invariant φ3 = φH + φK + φL

(14)

THE NEIGHBORHOODS OF THE STRUCTURE INVARIANTS

is a function of the primitive random variables H, K, L and therefore is itself a random variable. Denote by

The first neighborhood of the triplet (5), where (6) holds, consists of the three

P1|3 = P(|R1 , R2 , R3 )

(15)

9202


the conditional probability distribution of φ3 , given the three magnitudes (12), the first neighborhood of φ3 . Then [1,3] P1|3 ≈

1 exp(A cos ), 2π I0 (A)

(16)

where I0 is the modified Bessel function∗ , A is defined by A=

2σ3 3/2

σ2

R1 R2 R3 ,

(17)

and H + K + L + M = 0.

In view of (20), the linear function of four phases φ4 = φH + φK + φL + φM

σn =

N

Znj .

(18)

j=1

Graphs of the distribution (16) for A = 2.316 and A = 0.731 are shown in Figs. 1 and 2. Clearly this distribution has a unique maximum at = 0 in the interval (−π , π ) so that the most probable value of φ3 is zero. The larger the value of A the smaller is the variance of the distribution and the more reliable is the estimate of φ3 , zero in this case. Quartets As before, suppose that a crystal structure consisting of N atoms per unit cell is fixed. Two distributions will be described. The first, in strict analogy with the preceding section (‘‘Triplets’’), is the conditional probability distribution of the quartet, given the four magnitudes in its first neighborhood; the second is the conditional probability distribution of the quartet, assuming that the seven magnitudes in its second neighborhood are known. THE FOUR-MAGNITUDE DISTRIBUTION Assume that R1 , R2 , R3 , and R4 are fixed nonnegative numbers. Next suppose that the primitive random variable (vector) is the ordered quadruple (H, K, L, M) of reciprocal vectors H, K, L, M which is assumed to be uniformly distributed over the subset of the fourfold Cartesian product W × W × W × W defined by |EH | = R1 ,

|EK | = R2 ,

|EL | = R3 ,

|EM | = R4

(19)

(21)

is a structure invariant which, as a function of the primitive random variables H, K, L, M, is itself a random variable. Denote by P1|4 = P(|R1 , R2 , R3 , R4 )

and

(20)

(22)

the conditional probability distribution of φ4 , given the four magnitudes in its first neighborhood (19). Then [3] P1|4 ≈

1 exp(B cos ), 2π I0 (B)

(23)

where B is defined by B=

2σ4 R1 R2 R3 R4 σ22

(24)

and σn by (18). Thus P1|4 is identical with P1|3 , but B replaces A. Hence similar remarks apply to P1|4 . In particular (23) always has a unique maximum at = 0, so that the most probable value of the structure invariant (21) is zero, and the larger the value of B the more likely it is that φ4 ≈ 0. Since B values, of order 1/N, tend to be less than A values, or order 1/N 1/2 , the estimate (zero) of the quartet (21) is in general less reliable than the estimate (zero) of the triplet (14). Hence (23) is no improvement over (16) and the goal of obtaining a reliable nonzero estimate for a structure invariant is not realized by (23). The decisive step in this direction is made in the next section. THE SEVEN-MAGNITUDE DISTRIBUTION If one assumes as known not only the four magnitudes (19), but the additional three magnitudes |EH+K |, |EK+L |, and |EL+H | then, in favorable cases, one obtains a more reliable estimate for the quartet (21), and, furthermore, the estimate may lie anywhere in the interval 0 to π . Assume that the seven nonnegative numbers R1 , R2 , R3 , R4 , R12 , R23 , R31 are fixed.

Figure 1. The distribution P1|3 , (16), for A = 2.316.

Figure 2. The distribution P1|3 , (16), for A = 0.731. 9203

9204


Suppose next that the ordered quadruple of reciprocal vectors (H, K, L, M) is a random variable which is uniformly distributed over the subset of the fourfold Cartesian product W × W × W × W defined by |EH | = R1 , |EL | = R3 , |EH+K | = R12 ,

|EK | = R2 , |EM | = R4 ; |EK+L | = R23 ,

|EL+H | = R31 ;

(25)

(26)

and H + K + L + M = 0.

(27)

Then the quartet (21) is a structure invariant which, as a function of the primitive random variable (H, K, L, M) is itself a random variable. Denote by P1|7 = P(|R1 , R2 , R3 , R4 , R12 , R23 , R31 ) (28) the conditional probability distribution of the quartet (21), given the seven magnitudes in its second neighborhood, (25) and (26). The explicit formula for P1|7 has been found [2,3].

Figures 3 to 5 show the distribution (28) (solid line ) for typical values of the seven parameters (25) and (26). For comparison the distribution (23) (broken line – – –)) is also shown. Since the magnitudes |E| have been obtained from a real structure, comparison with the true value of the quartet is also possible. As already emphasized, the distribution (23) always has a unique maximum at = 0. The distribution (28), on the other hand, may have a maximum at = 0, or π , or any value between these extremes, as shown by Figs. 3 to 5. Roughly speaking the maximum of (28) occurs at 0 or π according as the three parameters R12 , R23 , R31 are all large or all small, respectively. These figures also clearly show the improvement which may result when, in addition to the four magnitudes (25), the three magnitudes (26) are also assumed to be known. Finally, in the special case that R12 ≈ R23 ≈ R31 ≈ 0

(29)

the distribution (28) reduces to P1|7 ≈

1 exp(−2B cos ), L

(30)

Figure 3. The distributions (28) (—) and (23) (– – –) for the values of the seven parameters (25) and (26) shown. The mode of (28) is 0, of (23) always 0.

Figure 4. The distributions (28) (—) and (23) (– – –) for the values of the seven parameters (25) and (26) shown. The mode of (28) is 105◦ , of (23) always 0.

Figure 5. The distributions (28) (—) and (23) (– – –) for the values of the seven parameters (25) and (26) shown. The mode of (28) is 180◦ , of (23) always 0. 9205

X − Y PLOTTER

9206

where B =

1 (3σ32 − σ2 σ4 )R1 R2 R3 R4 , σ23

(31)

and L is a suitable normalizing parameter. Clearly (31) has a unique maximum at = π (Fig. 5). SUMMARY Major emphasis has been placed on the neighborhood principle. The conditional probability distribution of a structure invariant T, given the magnitudes |E| in any of its neighborhoods, yields a reliable estimate for T in the favorable case that the variance of the distribution happens to be small. Since the structure invariants are the essential link between magnitudes |E| and phases φ, probabilistic methods are seen to play the central role in the solution of the phase problem. Owing to limitations of space, only the simplest cases have been treated in this article. However a much larger class of distributions is presently available. Not only have these distributions already proven to be of great value in the applications, particularly for complex structures when data sets have been limited in number and quality, but preliminary calculations strongly suggest that the distributions of the higher order structure invariants, particularly if one takes into

account whatever elements of symmetry may be present, will play a vital role in future applications. Acknowledgments This work was supported by Grant No. CHE8203930 from the National Science Foundation and a grant from the James H. Cummings Foundation, Inc.

REFERENCES 1. Cochran, W. (1955). Acta Crystallogr., 8, 473–478. 2. Hauptman, H. (1975). Acta Crystallogr., A31, 680–687. 3. Hauptman, H. (1976). Acta Crystallogr., A32, 877–882. See also CRYSTALLOGRAPHY, STATISTICS IN; PERIODOGRAM ANALYSIS; and TIME SERIES.

H. HAUPTMAN

X − Y PLOTTER Synonymous with data plotter—a unit providing a visual display in the form of a graph on paper. See also GRAPHICAL REPRESENTATION OF DATA.

Y YANSON (JAHNSON), YULII EDUARDOVICH

Yanson was elected to the International Statistical Institute∗ in 1885 and became corresponding member of the Russian Academy of Science in 1892. He was active in social reform in the manner of the liberal intelligentsia of his milieu, and practically, apart from peasant economics, in the careful planning of censuses and epidemiological investigations. A photograph may be found in [4]. There is a good obituary by A. I. Chuprov [1]; and the encyclopedia entry [2] gives extensive information.

Born: November 5, 1835 (o.s.), in Kiev, Russian Empire. Died: January 31, 1892 (o.s.), St. Petersburg, Russian Empire. Contributed to: official statistics, economics, demography. Y. E. Yanson’s initial tertiary training was in the historico-philological faculty at Kiev University. In 1861, he was appointed to an academic position in agricultural statistics and political economy and, after several such posts, first taught at St. Petersburg University in 1865. Yanson’s role was within the historical development of Russian statistical presentation to important socioeconomic issues, particularly problems of agricultural economics. Sometimes considered a father of the discipline of statistics in the Russian Empire, he is best known for two works: (1) Comparative Statistics of Russian and Western-European States, a 2-volume work, the first volume of which appeared in 1878; (2) The Theory of Statistics, which appeared in five editions between 1885 and 1913 [5]. Of the latter work, Yanson’s contemporary A. I. Chuprov, the father of A. A. Chuprov∗ and himself a leading figure in the same areas as Yanson, said [1]: ‘‘For statistical methodology we have nothing superior to Yanson’s book; in regard to the description of statistical establishments and the applications of statistics, it seems there is little comparable to be found in Western-European literature.’’ This book was used as a text for the course of statistics at St. Petersburg University (where Yanson was full professor from 1873), and was studied diligently by Lenin prior to his examination in 1891 as external student by a commission of which Yanson was a member [4]. Lenin was to refer later to Yanson’s statistical data in his writings and may be regarded as having gained his statistical technology [3] from this book.

REFERENCES 1. Chuprov, A. I. (1893). Yulii Eduardovich Yanson. Obituary—1893 (in Russian). In Rechi i Stati, Vol. 1. Sabashnikov, Moscow (1909), pp. 518–525. 2. Entsyklopedicheskii Slovar (1904), Vol. XLIA . Brokhaus and Efron, St. Petersburg. (The Yanson, Yu. E. entry extends over pp. 681–684.) 3. Il’in, V. (1908). Razvitie Kapitalizma v Rossii, 2nd ed. ‘‘Pallada,’’ St. Petersburg. [In English as: Lenin, V. I. (1964). The Development of Capitalism in Russia. Progress Publishers, Moscow. V. Il’in was a pseudonym used by Vladimir Il’ich Ulianov, later to become known as Lenin.] 4. Sipovska, I. V. and Suslov, I. P., eds. (1972). Istoriia Prepodavaniia i Razvitiia Statistiki v Peterburgskom-Leningradskom Universitete (1819–1971). Leningrad University, Leningrad, pp. 22–36. 5. Yanson, Yu. E. (1887). Teoriia Statistiki, 2nd ed. Schröder, St. Petersburg (5th ed., 1913).

E. SENETA

YARNOLD’S CRITERION A criterion proposed by Yarnold [2] for using the chi-squared distribution∗ with k − 1 degrees of freedom as an approximation to the distribution of X2 =

k j=1

9207

(Nj − npj )2 (npj )−1 ,

9208

YASTREMSKIˇI, BORIS SERGEYEVICH

where N1 , N2 , . . . , Nk (with kj=1 Nj = n) have a multinomial distribution∗ with parameters n; p1 , . . . , pk . According to this criterion, the approximation can be expected to be adequate if npj 5× (number of h’s for which nph < 5). Eaton [1] devised a systematic procedure for selecting, in a few steps, the minimum sample size consistent with Yarnold’s criterion. REFERENCES 1. Eaton, P. W. (1978). Amer. Statist., 32, 102–103. 2. Yarnold, J. K. (1970). J. Amer. Statist. Ass., 65, 864–886. See also APPROXIMATIONS TO DISTRIBUTIONS and CHI-SQUARE TESTS: NUMERICAL EXAMPLES.

YASTREMSKIˇI, BORIS SERGEYEVICH Born: May 9, 1877, in Dergach, near Kharkov, Ukraine. Died: November 28, 1962, in Moscow, USSR. Contributed to: time series, applied statistics. Son of the well-known Russian revolutionary S. V. Yastremskiˇı, B. S. Yastremskiˇı was the leader of the dogmatic, strictly Marxist—materialistic approach to statistical sciences in the USSR and had a substantial influence on the development of statistics in that country. He published a total of 97 papers on both the theoretical and practical problems of statistics. He started his career in 1913 by criticizing the theory of stability of statistical series, which was developed by W. Lexis∗ , and continued by criticizing the ‘‘idealistic treatment’’ of the law of large numbers∗ and the ‘‘law of averages.’’ He also wrote extensively against Pearson’s system of distributions∗ and the concept of spurious correlation∗ . He co-authored (with A. Ya. Boyarskiˇı and others) two ‘‘Marxist’’ textbooks on statistics in 1931 and 1936. A summary of his statistical ideas is contained in his last book Mathematical Statistics, published in Moscow in 1956.

Further details are given in an article [1] commemorating his 90th birthday. REFERENCE 1. Boyarskiˇı, A. Ya. and Kil’disher, G. (1967). Vestnik Statist., 5, 35–40.

YATES’ ALGORITHM This (also called Yates’ technique) is a system introduced by Yates [3] for facilitating the calculation by hand of estimates of main effects∗ and interactions∗ from data of factorial experiments∗ . The algorithm was first proposed for 2k experiments (k factors at 2 two levels each). Here we will first describe the analysis for a 24 experiment with factors A, B, C, D; and then indicate how the method can be extended to include factors with more than two levels. We will denote the level of a factor by a subscript attached to the corresponding lower-case level. Thus a0 , a1 denote the lower and higher levels of A, respectively; a0 b0 c1 d0 , for example, denotes the factor level combination in which A, B, and D are at lower, and C at higher level. The experiment gives rise to 24 = 16 observed values, one for each of the 16 possible factor level combinations. The calculations are set out in the form of a Yates table (see Table 1). The rows of the table correspond to the factor level combinations in standard order. Starting from a0 b0 c0 d0 , this is achieved by increasing the level of each factor in turn (the order of factors is immaterial), in combination with all preceding combinations of levels of the other factors. Thus following a0 b0 c0 d0 , we have a1 b0 c0 d0 , and then a0 b1 c0 d0 , and so on. See the first column of Table 1. The second column gives the observed values for the 16 factor level combinations. The calculations are simply addition or subtraction of values in successive rows. The top half of each of the next four columns is obtained by summing successive pairs of values in the preceding column. The lower half is formed by subtracting the upper item of each pair from the lower one. The procedure is repeated four times. (For a 2k experiment

YATES’ ALGORITHM

9209

Table 1. Application of Yates’ Algorithm to a 24 Experiment

Factor Level Combination

Summation/Subtraction Operation

Observed Value

I

II

III

IV

Effect or Interaction

a0 b0 c0 d0 a1 b0 c0 d0 a0 b1 c0 d0 a1 b1 c0 d0 a0 b0 c1 d0 a1 b0 c1 d0 a0 b1 c1 d0 a1 b1 c1 d0

4 6 7 8 3 4 6 5

10 15 7 12 19 22 16 19

25 19 41 35 3 0 2 1

44 76 3 3 10 6 −3 −1

120 6 16 16 −12 −4 0 −6

Total A B A×B C A×C B×C A×B×C

a0 b0 c0 d1 a1 b0 c0 d1 a0 b1 c0 d1 a1 b1 c0 d1 a0 b0 c1 d1 a1 b0 c1 d1 a0 b1 c1 d1 a1 b1 c1 d1

9 9 10 12 7 9 10 9

2 1 1 −1 0 2 2 −1

5 5 3 3 −1 −2 2 −3

−6 −6 −3 −1 0 0 −1 −5

32 0 −4 2 0 2 0 4

D A×D B×D A×B×D C×D A×C×D B×C×D A×B×C×D

it would be repeated k times and there would be k columns, in place of I-IV.) Column IV contains estimators of the main effects and interactions (or some multiple thereof) as indicated in the final column. The corresponding ‘‘sum of squares’’ (with 1 degree of freedom) is 2

(amount in final column) /4 [generally (amount in final column)2 /k]. EXTENSION TO FACTORS WITH MORE THAN TWO LEVELS For illustrative purposes, we will consider results from a 2 × 32 experiment (factor A at two levels; factor B and C each at those levels). Extension to general numbers of levels is straightforward. For factor A we have the single comparison a1 − a0 , but for each of B and C there are two orthogonal∗ comparisons possible (2 ‘‘degrees of freedom’’). Provided the conditions, of orthogonality to each other and to the sum (b0 + b1 + b2 , or c0 + c1 + c2 , as the case may be), hold, the 2 degrees of freedom may be split up arbitrarily. For our illustration we will take the ‘‘linear’’ components b2 − b0 , c2 − c0 and the ‘‘quadratic’’ components

b2 − 2b1 + b0 , c2 − 2c1 + c0 (which would be appropriate if the levels of B and C were each equally spaced and we were using a polynomial regression∗ model). As before, we start with the factor level combination a0 b0 c0 and introduce the higher levels of each factor in turn, as for 2k experiments (cf. Table 1). This leads to the first column of Table 2. We now have three columns I–III of linear operations, corresponding to three factors. The first column (corresponding to A, which has two levels) is obtained exactly the same way as in Table 1. That is, we form sums (corresponding to a0 + a1 ) and then differences (corresponding to a1 − a0 ) of successive rows in the preceding column. But, in proceeding from column I to column II (corresponding to B, which has three levels), we divide the 18 rows into six sets of three. We first form sums (corresponding to b0 + b1 + b2 ), then differences between the last and first item in each set of three rows (corresponding to b2 − b0 ); and then the sum of the first and last, minus twice the middle item (corresponding to b2 − 2b1 + b0 ). This operation is repeated in going from column II to column III (corresponding to the three level factor C). The final column shows the main effects and interactions estimated in column III.

9210

YATES’ CORRECTION FOR CONTINUITY Table 2. Yates’ Algorithm for a 2 × 32 Experiment Observed Value

Linear Function Operations I(A)

II(B)

III(C)

Main Effect or Interaction

a0 b0 c0 a1 b0 c0 a0 b1 c0 a1 b1 c0 a0 b2 c0 a1 b2 c0

2 2 4 5 8 9

40 9 17 3 6 11

30 20 44 2 0 4

94 6 28 0 4 0

Total A BL A × BL BQ A × BQ

a0 b0 c1 a1 b0 c1 a0 b1 c1

1 2 3

11 15 18

13 8 7

14 2 −6

CL A × CL BL × CL

a1 b1 c1 a0 b2 c1 a1 b2 c1

3 6 5

0 1 1

1 −2 1

0 −4 2

A × BL × CL BQ × CL A × BQ × CL

a0 b0 c2 a1 b0 c2 a0 b1 c2 a1 b1 c2 a0 b2 c2 a1 b2 c2

5 6 7 8 8 10

1 0 −1 1 1 2

3 2 −1 −1 0 1

34 6 4 6 −2 0

CQ A × CQ BL × CQ A × BL × CQ BQ × CQ A × BQ × CQ

Factor Level Combination

The letters L and Q are used to indicate ‘‘linear’’ and ‘‘quadratic’’ components, respectively. The BL × CQ estimator, for example, is, in fact, the linear combination obtained by formal multiplication from (a0 + a1 )(b2 − b0 )(c2 − 2c1 + c0 ). The divisor for the square of this quantity, in order to obtain the sum of squares for the analysis of variance table, is the sum of squares of the coefficients, which is conveniently calculated as (12 + 12 )(12 + 12 )(12 + 22 + 12 ) = 2 × 2 × 6 = 24. For the A × BQ × CQ estimator (a1 − a0 )(b2 − 2b1 + b0 )(c2 − 2c1 + c0 ),

REFERENCES 1. Hicks, C. R. (1973). Fundamental Concepts in Design of Experiments, 2nd ed. Holt, Rinehart, and Winston, New York. 2. Johnson, N. L. and Leone, F. C. (1977). Statistics and Experimental Design: In Engineering and Physical Sciences, 2nd ed., Vol. 2. Wiley, New York. 3. Yates, F. (1937). The Design and Analysis of Factorial Experiments. Imperial Bureau of Soil Science, Harpenden, England. See also ANALYSIS OF VARIANCE; CONFOUNDING; FACTORIAL EXPERIMENTS; FRACTIONAL FACTORIAL DESIGNS; INTERACTION; and MAIN EFFECTS.

YATES’ CORRECTION FOR CONTINUITY. See CONTINUITY CORRECTIONS

the divisor would be (12 + 12 )(12 + 22 + 12 )(12 + 22 + 12 ) = 2 × 6 × 6 = 72. Yates’ technique can be extended to analysis of fractional factorial∗ experiments and to experiments involving confounding∗ . For details, see [1, p. 215] and [2, p. 800].

YATES–DURBIN RULE This is a rule for calculating estimates of sampling error for multistage designs of any degree of complexity. The original version was given by Yates [2] for the case of equal probability of selection. It was subsequently generalized by Durbin [1] to read:

YATES, FRANK The estimate of variance in multi-stage sampling is the sum of two parts. The first part is equal to the estimate of variance calculated for the assumption that the first-stage values have been measured without error. The second part is calculated as if the first-stage units selected were fixed strata, the contribution from each first-stage unit being multiplied by the probability of that unit’s inclusion in the sample.

Compare with Horvitz and Thompson’s∗ estimator, originally given for the case of singlestage sampling, and with the alternative estimator given by Yates and Grundy [3]. For an illustration of this principle and further discussion, see, e.g., Durbin [1]. REFERENCES 1. Durbin, J. (1953). J. R. Statist. Soc. B, 15, 262–269. 2. Yates, F. (1949). Sampling Methods for Censuses and Surveys. Griffin, London, England. 3. Yates, F. and Grundy, P. (1953). J. R. Statist. Soc. B, 15, 253–261. See also STRATIFIED DESIGNS; STRATIFIED MULTISTAGE SAMPLING; and SURVEY SAMPLING.

YATES, FRANK Born: May 12, 1902, in Manchester, UK. Died: June 17, 1994, in Harpenden, UK. Contributed to: design and analysis of experiments, survey sampling, statistical computing. FY (as he was universally known) went to school first in Manchester, where he grew up, then to Clifton College, and to university at St. John’s College, Cambridge, from which he graduated with first-class honors in 1924. His first important job (1927–1931) was as research officer and mathematical adviser to the Geodetic Survey of the Gold Coast (now Ghana). Here he became acquainted with least squares∗ , the applications of which he greatly extended when he went to Rothamsted in 1931 as assistant to R. A. (later Sir Ronald) Fisher∗ . Within two years Fisher had left Rothamsted and Yates became head of

9211

the Statistics Department, a post which he held for 35 years, until he retired in 1968. Yates’s career belongs to Rothamsted, and he built upon Fisher’s legacy to make the Statistics Department there a continuing source of important ideas, which, although originating in agriculture, spread to many other fields. Fisher had founded the subject of design of experiments∗ , and propagated the ideas of replication, randomization∗ , and blocking (to reduce error) together with factorial∗ designs, where several treatment factors are varied together, rather than one at a time. Yates clarified the ideas of orthogonality, confounding∗ , and balance [2], and in his famous TC35 monograph, ‘‘The Design and analysis of Factorial Experiments’’ (1937) [6], he dealt with factorial experiments having factors with two, three, or four levels, and the use of split plots∗ with both complete and partial confounding∗ . In TC35 will be found the Yates algorithm∗ for calculating in two-level factorial experiments the main effects∗ and interactions∗ from the original yields. The computing trick here is the same as was later to underlie the fast Fourier transform. The inverse algorithm, for calculating fitted values from a subset of the effects, is also given. The other major new idea in design was Yates’s introduction of balanced incomplete block∗ designs, whereby only a subset of treatments occurs in each block, but treatments are balanced in such a way that any two occur together in an equal number of blocks [4]. He also realized that where the number of blocks is substantial, information about treatment effects can also be extracted from block totals, the so-called interblock information. For variety trials with many varieties he introduced the lattice designs, where varietal effects can be mapped on to a pseudofactorial structure [5]; this greatly simplified the analysis. There is now a huge combinatorial literature, spreading as far as coding theory, that derives from this work. Curiously, Yates is probably most widely known for his continuity correction∗ to chi-square in tables of counts [3], something which has become of much less importance now that the exact test statistic can be easily computed. By 1939, when the Second World War began, Yates had become an authority on

9212

YATES, FRANK

sampling schemes, and this knowledge was put to work in assessing the effect of bombing. This work, in close collaboration with Zuckerman, showed, for example, that the bombing of vital points of the railway system during the Sicily campaign of 1943 had been more effective than attacks on cities and factories. Here we see the introduction of quantitative methods that led in time to operational research as a subject in its own right. At the same time Yates continued his work at Rothamsted, where, in a major study with E. M. Crowther [7], he drew together all the past work on the effect of fertilizers on crops; this was to form the basis of government policy on fertilizer imports during the war years. This work is an example of the combination of information from many diverse experiments, and exhibits a skill that many statisticians seem only recently to have appreciated the need for. In 1950 the United Nations was responsible for a survey of world agriculture, and Yates was commissioned to write a sampling manual for it. Sampling Methods for Censuses and Surveys [8] was indeed a manual and not a textbook. It described, with numerous examples, different sampling schemes, and how estimates should be made of population values and their uncertainties calculated. The fourth edition is still in print. In Yates and Grundy [9] he introduced sampling without replacement with probability proportional to size, an early example of what is now known as rejective sampling. He also gave the rule for calculating estimates of sampling error for multistage designs of any degree of complexity (see YATES –DURBIN RULE). The first analyses of surveys were done with Hollerith sorters and tabulators. Yates managed to invert symmetric matrices using Hollerith cards! However, in 1954 the world changed, and Yates secured for Rothamsted one of the first commercial electronic computers, the Elliott 401. It had 512 words of memory on a rotating disk, paper tape for I/O, no floating-point arithmetic, and not even integer division. FY and his colleagues showed remarkable ingenuity in doing useful work with this machine. What must have been the first routines for operations on multiway tables were among the products of this work, and Yates himself began optimizing program

speed by the positioning of instructions on the disc. The 401 was succeeded by the 402 and then by the Ferranti Orion, perhaps the first modern computer in the series. Program development went on steadily under Yates’s direction, resulting, for example, in Genfac, for the analysis of factorial experiments [10], and RGSP, the Rothamstead General Survey Program [12], a particular interest of his. Yates never lost touch with agriculture; he was a member of the committee that supervised all research work on the experimental farm at Rothamsted for 35 years, being chairman for 13. He was appointed a deputy director in 1958, and after retirement he kept a room at Rothamsted and completed 60 years of work there in 1993. His published work began in 1929; the last paper appeared in 1988. The statistical tables [1] that he published with Fisher (first edition, 1936) were widely used throughout the world, and in 1970 appeared a book of selected papers on experimental design [11] with his additional comments. He was elected a Fellow of the Royal Society in 1948 and gained the Society’s Royal Medal in 1966. He was president of the British Computer Society in 1960–1961 and of the Royal Statistical Society∗ in 1967–1968. In 1953 he was awarded the Weldon Memorial Prize of Oxford University for work in biometrical science. I believe that Yates regarded himself as a scientist whose subject was statistics. He believed, like Fisher, that the interesting problems came from the real world, and that their solution would aid the processes of science and technology. REFERENCES 1. Fisher, R. A. and Yates, F. (1936). Statistical Tables for Biological, Agricultural and Medical Research. Oliver and Boyd, Edinburgh (subsequent editions 1942, 1948, 1953, 1957, 1963). (The introduction contains much of interest historically.) 2. Yates, F. (1933). The principles of orthogonality and confounding in replicated experiments. J. Agric. Sci., 23, 108–145. 3. Yates, F. (1934). Contingency tables involving small numbers and the chi-squared test. J. R. Statist. Soc. Suppl. 1, 217–235.

YOKE 4. Yates, F. (1936). Incomplete randomized blocks. Ann. Eugen., 7, 121–140. 5. Yates, F. (1936). A new method of arranging variety trials involving a large number of varieties. J. Agric. Sci., 26, 424–455. 6. Yates, F. (1937). The Design and Analysis of Factorial Experiments. Tech. Commun. 35, Commonwealth Bureau of Soil Science. (The classical exposition of factorial experiments.) 7. Yates, F. and Crowther, E. M. (1941). Fertiliser policy in war-time. The fertilizer requirements of arable crops. Emp. J. Exp. Agric., 9, 77–97. 8. Yates, F. (1949). Sampling Methods for Censuses and Surveys. Griffin, London (subsequent editions 1953, 1960, 1981). 9. Yates, F. and Grundy, P. M. (1953). Selection without replacement from within strata with probability proportional to size. J. R. Statist. Soc. B, 15, 253–261. 10. Yates, F. and Anderson, A. J. B. (1966). A general computer programme for the analysis of factorial experiments. Biometrics, 22, 503–524. 11. Yates, F. (1970). Experimental Design: Selected Papers. Griffin, London. 12. Yates, F. (1973). The analysis of surveys on computers—features of the Rothamsted Survey Program. Appl. Statist., 22, 161–171.

9213

via the maximum likelihood estimate and an ancillary statistic, a; in such cases we suppress the dependence on a. The general properties of score functions and of maximum likelihood estimates are described in most books on statistical theory; see, for example, Rao [5] and Cox and Hinkley [3]. Now the score function or log-likelihood ˆ where the suffix denotes derivative is lr (θ , θ), differentiation with respect to the rth component of the first argument θ . The maximum likelihood estimating equation is ˆ θˆ ) = 0. lr (θ, Thus considered as a function of both its arguments, the score function satisfies lr (θ , θ ) = 0, the name of the argument being immaterial. Moreover the matrix of second derivatives with respect to the first argument is negative definite because the log-likelihood function is maximized. In general a function g(θ , θ ) is called a yoke if gr (θ , θ ) = 0

BIBLIOGRAPHY Dyke, G. V. (1995). Obituary: Frank Yates. J. R. Statist. Soc. A, 158, 333–338. (Evokes Rothamsted background and personality well.) Healy, M. J. R. (1995). Obituary: Frank Yates, 1902–1994. Biometrics, 51, 389–391. (Another well-written and Enlightening Account.)

JOHN NELDER

YOKE In a parametric family the log-likelihood function for a parameter θ from data y can be written l(θ ; y). For a full exponential family the log likelihood depends essentially only on the canonical sufficient statistic, and this in turn is in (1, 1) correspondence with the ˆ Thus the maximum likelihood estimate θ. log likelihood can be written l(θ , θˆ ). More generally, for example in a transformation model, the log likelihood involves the data

and the matrix of second derivatives with respect to the first argument at (θ , θ ) is negative definite. In this context the log likelihood is called the observed likelihood yoke. The corresponding expected likelihood yoke is the function E[l(θ ) − l(θ )], the expectation being evaluated at θ . The expected yoke is in other contexts used as a measure of distance between two distributions. The reason for considering yokes is essentially mathematical rather than statistical in the narrow sense. They arise in defining certain higher derivatives, called tensorial derivatives, which are central to developing invariant forms of Taylor expansion, i.e. versions of Taylor series in which the expansion to a given number of terms of (say) f (x) is unchanged if the argument x is replaced by a monotonic function of x.

9214

YOUDEN SQUARE

The idea is due to Barndorff-Nielsen [1,2]. For a different approach to invariant Taylor series via jet bundles, see Murray and Rice [4]. REFERENCES

Figure 2. Construction of a Youden square.

1. Barndorff-Nielsen, O. E. (1987). Differential geometry and statistics: some mathematical aspects. Indian J. Math., 29, 335–350. (Ramanujan Centenary Volume.) 2. Barndorff-Nielsen, O. E. (1989). Contribution to discussion of paper by R.E. Kass. Statist. Sci., 4, 222–227. 3. Cox, D. R. and Hinkley, D. V. Theoretical Statistics. Chapman and Hall, London. 4. Murray, M. K. and Rice, J. W. (1993). Differential Geometry and Statistics. Chapman and Hall, London. 5. Rao, C. R. (1973). Linear Statistical Inference. Wiley, New York. See also DIFFERENTIAL GEOMETRY IN STATISTICS and SCORE STATISTICS.

D. R. COX

YOUDEN SQUARE These experimental designs are not really ‘‘squares’’ at all. A Youden square is a balanced incomplete blocks∗ design with the additional property that it can be arranged in columns (corresponding to blocks) in such a way that each treatment appears once in each row. An example with seven treatments and seven blocks each of three plots is shown in Fig. 1. This is a balanced incomplete blocks∗ design with parameters b = 7, t = 7, k = 3, r = 3, and λ = 1. Note that in a Youden square the number of blocks b must equal the number of treatments t, and the number of plots per block k must equal the number of replications r of each treatment.

The design does not form a ‘‘square’’ but the name ‘‘Youden square’’ (apart from referring to their proponent Youden [3]) may be based on the fact that the design may be regarded as an incomplete Latin square∗ . (This name was given to them by F. Yates [2].) Indeed, if the last (or any one) row of a m × m Latin square is deleted, the result is a Youden square with b = t = m; k = r = m − 1; λ = m − 2. Figure 2 shows such a construction, with m = 4. However, omission of two or more rows from a Latin square does not, in general, produce a Youden square. The standard analysis of variance∗ for data from an experiment designed as a Youden square is the same as for a balanced incomplete randomized block design of the same dimensions, except that a ‘‘Between Rows’’ sum of squares is computed and subtracted from the standard Residual sum of squares. The degrees of freedom are correspondingly reduced by (r − 1), giving Residual degrees of freedom = bk − 1 − (b − 1) − (t − 1) − (r − 1) = bk − b − t − r + 2 = (t − 1)(r − 2), since b = t and k = r. On the assumption of the parametric model observed value = constant + (Block effect) +(Row effect) +(Treatment effect) +(Residual),

Figure 1. A Youden square.

with independent residuals having constant variance σ 2 , the Residual mean square will have expected value t2 ; and if normal variation can be assumed, the Rows sum of squares will be independent of the Residual and the

YULE DISTRIBUTION

Blocks (or Blocks adjusted) and Treatments (or Treatments adjusted) sums of squares. There is a detailed descripted of construction and analysis of Youden squares in Natrella [1]. REFERENCES 1. Natrella, M. G. (1963). Experimental Statistics. U.S. Natl. Bur. Stand. Handbk. 9, Government Printing Office, Washington, D.C. 2. Yates, F. (1936). J. Agric. Sci., 26, 301–315. 3. Youden, W. J. (1937). Contrib. Boyce Thompson Inst., 9, 41–48. See also ANALYSIS OF VARIANCE; BLOCKS, BALANCED INCOMPLETE; and LATIN SQUARES, LATIN CUBES, LATIN RECTANGLES

9215

as a compound geometric distribution. The probability generating function∗ is G(z) =

ρz 2 F1 [1, 1; ρ + 2; z], ρ+1

that of a beta-negative binomial variable on the support {1, 2, . . .} [3 (Sec. 6.10.3)], where 2 F1 (·, ·; ·; z) is Gauss’s hypergeometric function∗ . The mean and variance are µ = ρ/(ρ − 1),

provided ρ > 1,

σ = ρ /[(ρ − 1) (ρ − 2)], 2

2

2

provided ρ > 2.

If ρ = 1, (1) takes the form f1 (x) = [x(x + 1)]−1 ,

x = 1, 2, . . . ;

(2)

Yukich [1] proposed the following metric on the k-dimensional space Rk . The distance between probability measures P(·) and Q(·), with respect to probability density g(·), is

however, the expected value is infinite. For large values of x, f1 (x) ∼ = x−2 , and is thus nearly equivalent to a zeta distribution∗ with ρ = 1, an approximation that has been applied to word frequency distributions [4]. Yule’s distribution (1) on support {1, 2, . . .} is also so named on support {0, 1, 2, . . .} with pmf

dg (P1 Q) = sup{|g(x − y)d{P(y) − Q(y)}| :

Pr(X = x) = ρ(ρ!)x!(x + ρ + 1)!,

YUKICH METRIC

x = 0, 1, 2, . . . ,

x, y ∈ Rk }. See Yukich [1] for details, and for potential applications.

(3)

where x in (1) is replaced by x + 1; the rth ascending factorial moment is E[X(X + 1) . . . (X + r − 1)]

REFERENCE

= ρr![(ρ − r)(ρ − r + 1)],

1. Yukich, J. E. (1985). Math. Proc. Camb. Philos. Soc., 98, 533–540. See also METRICS AND DISTANCES ON PROBABILITY SPACES.

YULE DISTRIBUTION Yule∗ [4] described the frequencies X of biological species in families by the distribution with probability mass function (pmf) of form Pr(X = x) = f (x) = ρ(ρ!)(x − 1)!/(x + ρ)! = ρB(x, ρ + 1),

x = 1, 2, . . . ,

(1)

where ρ! = (ρ + 1), (x − 1)! = (x), B(·, ·), is the beta function and ρ > 0. Yule derived (1)

provided r < ρ. The distribution (1) is a generalized Waring distribution (see FACTORIAL SERIES DISTRIBUTIONS and Irwin [2]). It is a discrete analog of the Pareto distribution∗ . See Xekalaki and Panaretos [8] for a discussion of this interrelation. Xekalaki [6,7] describes applications of Yule and modified Yule distributions. The Yule distribution has also been employed [1] in a stochastic model of superstardom. REFERENCES 1. Chung, K. H. and stochastic model

Cox, R. A. K. (1994). A of superstardom: An

9216

YULE, GEORGE UDNY

application of the Yule distribution, Rev. Econ. Statist., 76, 771–775. 2. Irwin, J. O. (1975). J. R. Statist. Soc. A, 138, 18–31. (See also ibid., pp. 204–227 and 374–384, for a full discussion of generalized Waring distributions.) 3. Johnson, N. L., Kotz, S. and Kemp, A. W. (1992). Univariate Discrete Distributions (2nd ed.). Wiley, New York. (See Chap. 6, Sec. 10.3.) 4. Simon, H. A. (1954). Biometrika, 42, 425–440. 5. Yule, G. U. (1924). Philos. Trans. R. Soc. Lond. B, 213, 21–87. 6. Xekalaki, E. (1983). Commun. Statist. A, 12, 1181–1189. 7. Xekalaki, E. (1984). J. Econ., 24, 397–403. 8. Xekalaki, E. and Panaretos, J. (1988). Teor. Veroyat. Primen., 33, 206–210.

FURTHER READING Dimaki, C. and Xekalaki, E. (1997). Additive and multiplicative distortion of observations, J. Appl. Statist. Sci., 5, 113–127. See also FACTORIAL SERIES DISTRIBUTIONS; ZETA DISTRIBUTIONS; and ZIPF’S LAW.

YULE, GEORGE UDNY Born: February 18, 1871, in Beech Hill, near Haddington, Scotland. Died: June 26, 1951, in Cambridge, England. Contributed to: correlation theory, distribution theory, stochastic processes, statistics of literary vocabulary. George Udny Yule was a member of a Scottish family steeped in literary and administrative traditions. After schooling at Winchester, he proceeded at the age of 16 to study engineering at University College, London. His first published work was on research into electric waves under Heinrich Hertz during his sojourn at Bonn 1892. He wrote six papers on electromagnetic theory, but, after taking up a demonstratorship offered him in 1893 by Karl Pearson∗ (then a professor of applied mathematics at University College), he turned his attention to statistical problems, starting with his 1895 paper, ‘‘On the Correlation of Total Pauperism with Proportion of Outrelief.’’ There is a fascinating discussion of

this paper in Selvin [3], describing how Yule introduced correlation∗ coefficients in studying two-way tables in the earlier volumes of the monumental work of Booth [2]. For Yule, Pearson was an inspiring teacher, and Yule made fundamental contributions in 1897 and 1899 to the theory of statistics of regression and correlation. In 1899, Yule left University College for a post at the City and Guilds of London Institute. Between 1902 and 1909, he also gave the Newmarch lectures in statistics at University College. These lectures formed the basis of his famous Introduction to the Theory of Statistics, which, during his lifetime, ran to 14 editions [the 11th (1937), 12th (1940), 13th (1945), and 14th (1950) being joint with M.G. Kendall]. He continued to publish papers on association and correlation (1900, 1901, 1903), and was awarded the Guy Medal in Gold in 1911 by the Royal Statistical Society∗ . His theoretical works were accompanied by contributions to various economic and sociological subjects (1906, 1907, 1910). In 1912, Yule was appointed a lecturer in statistics at Cambridge University (later raised to the status of a readership). The years 1920–1930 were the most productive in his career. During this period he introduced the correlogram∗ and laid the foundations of the theory of autoregressive∗ series. He was president of the Royal Statistical Society from 1924–1926. In one of his last publications in the Journal of the Royal Statistical Society∗ , in late 1933, he showed that German authorities had overestimated the number of Jews entering Germany from Poland and Galicia during and after World War I by a factor of about 5. In 1931, he retired from his post at Cambridge University. However, he ‘‘felt young enough to learn to fly. Accordingly, he went through the intricacies of training, got a pilot’s license, and brought a plane. Unfortunately, a heart attack cut short his flying and, to a considerable degree, his scholarly work’’ (Bates and Neyman [1]). The death of Karl Pearson in 1936 affected him deeply. However, according to M. G. Kendall, ‘‘the publication of the revised Introduction in 1937 gave him a new lease of life.’’ In the later years, his main work was related to frequency of occurrence of words (particularly nouns) in

YULE’S Q

various texts (see ZIPF’S LAW). This research found expression in his last book (1944) on The Statistics of Literary Vocabulary. A great deal of Yule’s contributions to statistics resides in the stimulus he gave to students, in discussion with his colleagues, and advice he generously tendered to all who consulted him. His work on correlation and regression is now so standard that only history buffs would consult the original sources; he invented the correlogram and the autoregressive series; he also paved the way for Fisher’s derivation of the distributions of partial correlations∗ . The terms Yule process and Yule distribution (see Supplement) are now firmly established in the literature. REFERENCES 1. Bates, G. E. and Neyman, J. (1952). Univ. California Publ. Statist., 1, 215–254. 2. Booth, C. E., ed. (1889–1893). Life and Labour of the People in London. Macmillan, London, England (first 4 of 17 volumes). 3. Selvin, H. C. (1976). Archives Europ. J. Sociol., 17, 39–51.

Works by G. U. Yule (1895). Econ. J., 5, 477–489. (1897). Proc. R. Soc. Lond., 60, 477–489. (1899). J. R. Statist. Soc., 62, 249–286. (1900). Philos. Trans. R. Soc. Lond., 197A, 91–133. (1901). Philos. Trans. R. Soc. Lond., 197A, 91–133. (1903). Biometrika, 2, 121–134. (1906). J. R. Statist. Soc., 69, 88–132. (1907). J. R. Statist. Soc., 70, 52–87. (1910). J. R. Statist. Soc., 73, 26–38. (1933). J. R. Statist. Soc., 96, 478–480. (1944) The Statistics of Literary Vocabulary. Cambridge University Press, London, England. A list of Yule’s publications is included in the obituary: Kendall, M. G. (1952). J. R. Statist. Soc. A, 115, 156–161.

9217

A list of publications is also given in: Stuart, A. and Kendall, M. G., eds. (1971). Statistical Papers of George Udny Yule. Hafner, New York.

YULE SCHEME A linear autoregressive∗ scheme of the form ut = α1 ut−1 + α2 ut−2 + t , where α1 , α2 are parameters and the t ’s are mutually independent random variables with zero expected value and common variance. BIBLIOGRAPHY Yule, G. U. (1921). J. R. Statist. Soc. A, 84, 497–526. Yule, G. U. (1927). Philos. Trans. R. Soc. Lond., 226, 267–298. (Section III, pp. 280 et seq.) See also AUTOREGRESSIVE –INTEGRATED MOVING AVERAGE (ARIMA) MODELS; AUTOREGRESSIVE –MOVING AVERAGE (ARMA) MODELS; and TIME SERIES.

YULE’S Q Let P(Ai Bj ) = Pij , i, j = 1, 2, be the population probability of falling into the cell defined by the ith row and jth column of a 2 × 2 contingency table∗ , with fixed column totals. Yule’s Q is defined as Q=

P11 P22 − P12 P21 . P11 P22 + P12 P21

This measure of association∗ is widely used in social sciences. It can be expressed as Q = (α − 1)/(α + 1), where α = (P11 /P21 )/(P12 /P22 ) is the odds ratio∗ . The range of possible values of Q is from −1.0 to 1.0 with 0 corresponding to independence. Yule’s Q is estimated by ˆ = f11 f22 − f12 f21 , Q f11 f22 + f12 f21 where fij , i, j = 1, 2, is the observed frequency ˆ (under corresponding to Pij . The variance of Q

9218

YULE’S Y

the assumption that fij > 0 for all i, j = 1, 2) is estimated by 1 ˆ 2 fij−1 . (1 − Q) 4 2

σˆ (2Q) ˆ =

2

i=1 j=1

See also ASSOCIATION, MEASURES OF; LOG-LINEAR MODELS IN CONTINGENCY TABLES; TETRACHORIC CORRELATION COEFFICIENT; and YULE’S Y.

It is estimated by replacing each Pij by fij . The variance of this estimator of Y (under the assumption that fij > 0, i, j = 1, 2) is estimated by 1 ˆ 2 fij−1 . (1 − Y) 16 2

σˆ (2Y) ˆ =

2

i=1 j=1

For interpretation of Y and further details, see, e.g., Bishop et al. [1] and Reynolds [2].

YULE’S Y REFERENCES Yule’s Y (also known as coefficient of colligation) is defined in a 2 × 2 contingency table as √ √ √ P11 P22 − P12 P21 α−1 = √ Y= √ √ α+1 P11 P22 + P12 P21 (see YULE’S Q for notation). This measure of association has the same properties as Yule’s Q, but the absolute value of Y is less than the absolute value of Q unless the ‘‘categories’’ are independent or completely associated.

1. Bishop, Y. M., Fienberg, S. E., and Holland, P. W. (1975). Discrete Multivariate Analysis. MIT Press, Cambridge, MA. 2. Reynolds, H. T. (1975). The Analysis of CrossClassifications. Free Press-Macmillan, New York. See also ASSOCIATION, MEASURES OF; DEPENDENCE, MEASURES AND INDICES OF; LOG-LINEAR MODELS IN CONTINGENCY TABLES; TETRACHORIC CORRELATION COEFFICIENT; TWO-BY-TWO (2 × 2) TABLES; and YULE’S Q.

Z ZAREMBA

TEST

STATISTIC. See

MANN –WHITNEY –WILCOXON MODIFIED

TEST,

2. Zelen, M. (1954). J. Res. Natl. Bur. Stand., 53, 377–381.

ZAREMBA’S

See also CAMP –MEIDELL INEQUALITY; CANTELLI’S INEQUALITY; CHEBYSHEV’S INEQUALITY; GAUSS –WINCKLER INEQUALITY; and PROBABILITY INEQUALITIES FOR SUMS OF BOUNDED RANDOM VARIABLES.

¨ ZEITSCHRIFT FUR WAHRSCHEINLICHKEITS THEORIE UND VERWANDTE GEBIETE. See MANN –WHITNEY –WILCOXON MODIFIED

TEST,

ZAREMBA’S

ZELEN’S INEQUALITIES

ZELEN’S RANDOMIZED CONSTANT DESIGNS. See CLINICAL TRIALS, ZELEN’S

An explicit form of the Chebyshev—Markov inequalities [1] for the case when the first four moments (µ1 , µ2 , µ3 , µ4 ) of the distribution of a random variable X are known, given by Zelen [2]. For a distribution with support∗ on (−∞, +∞) the inequalities are √ √ 3/2 (with σ = µ2 , β1 = µ3 /µ2 , β2 = µ4 /µ22 ): √ For t < − 12 {| (β1 + 4)| − β1 },

RANDOMIZED

ZELLNER ESTIMATOR. See SEEMINGLY UNRELATED REGRESSION, ZELLNER ESTIMATOR

ZERO

CONTROL

METHOD. See

EDITING STATISTICAL DATA

Pr[X µ1 + tσ ] A(t); √ for − 12 {| (β1 + 4)| − β1 } √ t 12 {| (β1 + 4)| + β1 },

ZERO DEGREES OF FREEDOM

B(t) Pr[X µ1 + tσ ] A(t) + B(t); √ for t > 12 {| (β1 + 4)| + β1 }, Pr[X µ1 + tσ ] 1 − A(t); where β2 −β1 − 1 , √ (1 + t2 )(β2 −β1 − 1) + (t2 − t β1 − 1)2 √ 1 + 12 t{| (β1 + 4)| + β1 } . B(t) = √ | (β1 + 4)| 12 {| (β1 + 4)| + β1 } − t A(t) =

√ Note that when t2 − t β1 − 1 = 0 [i.e., t = √ 1 2 −1 2 {− β1 ± | (β1 + 4)|}], A(t) = (1 + t ) . When the range of variation (support) is not unlimited, there are more complex formulas; see ref. 2 for details. REFERENCES

We usually think of ‘‘degrees of freedom∗ ’’ as a positive integer representing the number of independent pieces of information in a given situation. We might therefore expect the case of ‘‘zero degrees of freedom’’ to represent the uninteresting case in which there is no information. Whereas this is typically true for central distributions, there exist noncentral distributions with zero degrees of freedom that have rich, interesting, and useful structures. We will discuss the chi-squared∗ , t∗ , and F ∗ distributions. In particular, the noncentral chi-squared distribution∗ with zero degrees of freedom (a mixture of exact zero values with positive continuous variation) is useful in modeling data containing exact zeroes, in queueing∗ , and as an asymptotic distribution in time-series∗ analysis and geometric probability∗ . DEFINITIONS AND PROPERTIES

1. Shohat, J. A. and Tamarkin, J. D. (1943). The Problem of Moments. American Mathematical Society, New York.

The central chi-squared distribution with zero degrees of freedom χ02 is a degenerate 9219

9220

ZERO DEGREES OF FREEDOM

distribution that always takes the value zero. The noncentral chi-squared distribution with zero degrees of freedom χ02 (λ) is a mixture∗ of a discrete distribution degenerate at zero with a continuous positive distribution. As the noncentrality parameter λ approaches zero, the discrete component occurs with high probability and the continuous component (which occurs rarely) is approximately exponentially distributed. As the noncentrality parameter becomes large, the discrete component occurs rarely and the continuous component tends towards a Gaussian distribution. A random variable X ∼ χ02 (λ) may be represented as a Poisson mixture of central chi-squared variates with even numbers of degrees of freedom according to the following two-stage procedure. First, choose K from a Poisson distribution∗ with mean λ/2, and 2 . Whenever K = 0, we then choose X ∼ χ2K will have X = 0; otherwise X will be a positive continuous random variable. The χ02 (λ) distribution has the following properties: Cumulative distribution function: ∞ (λ/2)k

Pr[X x] = 1 − e−(λ+x)/2

k!

k=1

×

k−1 (x/2)j j=0

j!

,

x > 0,

Pr[X 0] = e−λ/2 , and Pr[X x] = 0,

x < 0.

Characteristic function: exp[itλ/(1 − 2it)].

distribution is degenerately zero if n2 > 0, whereas F(n1 , 0) is always undefined due to division by zero. The noncentral cases are more interesting: The t0 (λ) distribution, a standard Gaussian divided by an independent χ02 (λ), takes on real values whenever the denominator is nonzero, i.e., with probability 1 − exp(−λ/2). Properties of the singly and doubly noncentral F distribution follow similarly by considering the noncentral chisquared variates from which it is formed, paying close attention to zeroes in the denominator. APPLICATIONS The χ02 (λ) distribution is a natural choice for model building in situations where a Poisson number of basic events each contributes an independent exponentially distributed quantity to a total. For example, in queueing theory if customers arrive according to a Poisson process∗ and if the service times are exponential, then the total service time required for all customers arriving during a fixed period would follow this distribution. Fitting the χ02 (λ) distribution to data by maximum likelihood∗ has been explored by Siegel [4]. The χ02 (λ) distribution has also been found useful as an asymptotic limiting distribution in testing for periodicity in a time series∗ (Siegel [3] and Siegel and Beirlant [5]) when tests are considered which are more powerful than Fisher’s test in the case of strong periodicity at multiple periods. The χ02 (λ) distribution is also the limiting distribution of the amount of a circle left uncovered (the ‘‘vacancy’’ in geometrical probability) by randomly placed arcs on the circumference of a circle (Siegel [2]).

Moments:

k m λ m−1 k! . E[X ] = 2 m! k−1 2 m

m

k=1

Further details and graphs may be found in Siegel [1]. Properties of t and F distributions with zero degrees of freedom follow easily from the chi-squared case. The central cases are trivial: the t0 distribution is undefined because it always involves division by zero. The F(0, n2 )

REFERENCES 1. Siegel, A. F. (1979a). The noncentral chisquared distribution with zero degrees of freedom and testing for uniformity. Biometrika, 66, 381–386. (Definitions, properties, and graphs of the χ02 (λ) distribution.) 2. Siegel, A. F. (1979b). Asymptotic coverage distributions on the circle. Ann. Prob. 7, 651–661. (The χ02 (λ) distribution as the asymptotic limit of the random amount of a circle uncovered by randomly placed arcs.)

ZERO-ONE LAWS 3. Siegel, A. F. (1980). Testing for periodicity in a time series. J. Amer. Statist. Ass., 75, 345–348. (Extends Fisher’s test for periodicity in a time series to make it more powerful in detecting multiple periodicity; the asymptotic distribution of the test statistic follows the χ02 (λ) distribution.) 4. Siegel, A. F. (1985). Modelling data containing exact zeroes using zero degrees of freedom. J. R. Statist. Soc. B, 47, 267–271. [Establishes existence and uniqueness of maximum likelihood estimators for the χ02 (λ) distribution. An example fitting the model to snowfall data is given in which data values are either exactly zero (when it didn’t snow) or positive (when it did snow).] 5. Siegel, A. F. and Beirlant, J. (1987). Periodicity testing based on spacing. In Contributions to the Theory and Applications of Statistics, A Volume in Honor of Herbert Solomon, A. E. Gefand, ed. Academic, New York, pp. 179–196. [Provides further asymptotic results and tables for using the χ02 (λ) distribution in testing for periodicity in a long time series.]

9221

and n(r) = n(n − 1) · · · (n − r + 1), respectively. By convention, the ‘‘zeroth’’ ascending and descending factorials are each equal to 1: n[0] = 1 = n(0) . This is analogous to the formula x0 = 1(x = 0) for zero powers. See also COMBINATORICS.

ZERO-ONE LAWS A zero-one law refers to events for which the probability of occurrence is equal to 0 or 1. Let A1 , A2 , . . . , An , . . . be a sequence of inde∞ pendent events and define A = ∩∞ n=1 ∪k=n Ak , so that A is the tail event in which infinitely many Ak occur. Borel [1] noted that

See also NONCENTRAL DISTRIBUTIONS (Various entries).

P(A) = 0

or P(A) = 1.

ANDREW F. SIEGEL

ZERO FACTORIAL

The Borel–Cantelli lemma* (also called the Borel–Cantelli zero-one law) specifies further that

The symbol n!, where n is a positive integer, stands for n × (n − 1) × (n − 2) × · · · × 2 × 1. This definition does not apply if n = 0, but conventionally the value of 0! is taken to be 1. This is formally consistent, for example, with the combinatorial (binomial∗ coefficient) formula n! n = , r r!(n − r)! since n! n = 0 0!n! is equal to 1. The rth ascending and descending factorials of n are n[r] = n(n + 1) · · · (n + r − 1)

P(A) = 0

if

∞

P(An ) < ∞

n=1

and P(A) = 1

if

∞

P(An ) = ∞.

n=1

A general theorem on zero-or-one probabilities was stated by Kolmogorov [4–6] as follows: Let X1 , X2 , . . . , Xn , . . . be a sequence of random variables and f (X1 , X2 , . . . , Xn , . . .) be a Baire function* of variables X1 , X2 , . . . , Xn , . . . such that the conditional probability P{f (X1 , X2 , . . . , Xn , . . .) = 0 |X1 , . . . , Xn } is equal to the unconditional probability P{f (X1 , X2 , . . . , Xn , . . .) = 0}

(1)

for each n. Then the probability (1) equals 0 or 1. For the case of a sequence of independent and identically distributed random variables,

9222

ZERO-ORDER CORRELATION

Hewitt and Savage [2] have shown that the probability of not only the tail events but of any event invariant with respect to permutations of a finite number of terms of the sequence is either 0 or 1. For the case of Brownian motions, the zero-one law was formulated by R. M. Blumenthal in 1975. REFERENCES 1. Bauer, H. (1996). Probability Theory. Walter de Gruyter, Berlin. 2. Borel, E. (1909). Rend. Circolo Math. Palermo, 27, 247–271. 3. Hewitt, E. and Savage, L. J. Symmetric measures on Cartesian products.(1955). Trans. Am. Math. Soc., 80, 470–501. 4. Kolmogorov, A. N. (1928). Math. Ann., 99, 309–319. 5. Kolmogorov, A. (1930). Sur la loi forte de grandes nombres. C. R. Acad. Sci. Paris, 191, 910–912. 6. Kolmogorov, A. N. (1974). Fundamental Concepts of Probability Theory, 2nd ed. Moscow (in Russian).

ZERO-ORDER CORRELATION In behavioral sciences the simple correlation between two variables is sometimes called the zero-order correlation (as opposed to partial correlations). See also CORRELATION.

Abramowitz and Stegun [1, p. 811] for s = 2(1)42. The kth moment, µk = ζ (ρ − k + 1)/ζ (ρ + 1),

is finite only for k < ρ, and is infinite for k ρ. If (X1 , . . . , Xn ) is an independent random sample based on (1), the maximum likelihood∗ estimator ρˆ of ρ is a solution of the equation [5] n−1

n

log Xi = −

i=1

∂ {ζ (ρ + 1)}/ζ (ρ + 1). (3) ∂ρ

Values of the right-hand side of (3) for 0.1 ρ 4 are tabulated in Johnson and Kotz [3, p. 242]. For ρ > 4, ∂ {ζ (ρ + 1)}/ζ (ρ + 1) ∼ = (1 + 2ρ+1 )−1 loge 2. ∂ρ The zeta distribution appears as the size— frequency form of Zipf’s law∗ , and, in particular, has been used to describe word frequency in sentence construction. Nanopoulos [4] establishes a weak law of large numbers under (1). For further discussion of (1) see ref. 3. The distribution with PMF Pr[X = x] = g(x) =

1 1 − , α (2x − 1) (2x + 1)α x = 1, 2, . . . ; α > 0,

ZERO-TRUNCATED DISTRIBUTION. See DECAPITATED DISTRIBUTIONS

distribution∗

The zeta or discrete Pareto probability mass function (PMF)

has

x = 1, 2, . . . ; ρ > 0,

= (2Z)

Pr[X = x] = f (x) = [ζ (ρ + 1)]−1 r−(ρ+1) ,

(4)

and having mean value (1 − 2−1/α )ζ (1/α), was called a ‘‘zeta distribution’’ by Haight [2], but it differs from (1), and was derived as the limit of a generalized harmonic distribution∗ (as Z → ∞) with PMF Pr[X = x] = h(x)

ZETA DISTRIBUTIONS

(2)

−

1/α

2Z 2x + 1

2Z 2x − 1

1/α

1/α

,

Z > 0, (5)

(1)

where ζ (·) is the Riemann zeta function, −s defined by ζ (s) = ∞ r=1 r , and tabulated in

where [ ] denotes the greatest integer value. Here α is usually close to 1; when α = 1, g(x) = 2(4x2 − 1)−1 , and h(x) is proportional to the

ZIPF’S LAW

number of quantities Z/j (j being a positive integer) for which the nearest integer is x. Equation (5) was also derived to describe word frequencies (see ref. 3, p. 247), and is related to the Yule distribution∗ . REFERENCES

N(X) =

∞

nf (u)du x

= rank of an object of size x. Under Zipf’s law (1), xN(x) = constant or N(x) = K/x. Hence f (x) = −n−1 N (x) = K /x2 ,

1. Abramowitz, M. and Stegun, I. A., eds. (1964). Handbook of Mathematical Functions. Appl. Math. Series No. 55, National Bureau of Standards, Washington, D.C. 2. Haight, F. A. (1966). J. Math. Psychol., 3, 217–233. 3. Johnson, N. L. and Kotz, S. (1969). Distributions in Statistics: Discrete Distributions. Wiley, New York. 4. Nanopoulos, P. (1977). C. R. Acad. Sci., Paris, A & B, 285, 875–876. 5. Seal, H. L. (1952). J. Inst. Actuar., 78, 115–121. See also PARETO DISTRIBUTION; YULE DISTRIBUTION; and ZIPF’S LAW.

ZIPF’S LAW Consider a set of data values, ordered as x(1) x(2) · · · x(n) , in the reverse of the conventional arrangement, having the largest value ranked first, and so on. We may think of r as the rank and x(r) as the size of the rth data value in the ordered set. Zipf [15] noticed that the relationship rx(r) = constant

(1)

9223

(2)

where K = K/n. Equation (2) is the sizefrequency relation corresponding to (1). Zipf attempted to explain the origins of (1) in the nature of human behavior, through the so-called principle of least effort. Some writers [4,8] have criticized (1) and (2) adversely, for one or more of three reasons. 1. Zipf’s explanation [15] in terms of human behavior is of doubtful relevance to many of its manifestations and gives no clue to any underlying statistical process. 2. The value of the constant K in (2) depends on the number n of objects in the study. 3. A statistical rationale for the phenomena observed by Zipf leads to a family of distributions—(4), discussed next— which only includes (2) as a special case. This last criticism is not entirely fair, because Zipf proposed a generalization of the ranksize relation, namely, rq x(r) = constant,

q > 0,

(3)

which leads to the size-frequency form with discrete density function f (r) = Ar−(1+a) ,

r = 1, 2, . . . ,

(4)

where a > 0, and seemed to hold for various kinds of objects, including cities in the United States by population, books by number of pages, words in an essay by their frequency of occurrence, and biological genera by number of species. The rank-size relation (1) is known as Zipf’s law; its graph is a rectangular hyperbola. Let x be the size of an object and f (x) its

∞ relative frequency of occurrence, where 0 f (x)dx = 1. If n is the number of objects in the data set or collection and N(x) the number of objects with size greater than x, then

A = ζ (1 + a) =

∞

r−(1+a)

r=1

is the zeta function; ζ (u) is tabulated in Abramowitz and Stegun [1, p. 811] for u = 1(1)42. Equation (4) defines the discrete Pareto or zeta distribution∗ [7], which includes (2) as the case a = 1. Fox and Lasker [4] fitted parameter values to (4) by maximum likelihood estimation∗ (see Seal [11] for the frequency of surnames in nine districts near

9224

ZIPF’S LAW

London in England), and found that the data gave acceptable fits to the model, with estimates of a lying between 1.76 and 2.88. The nomenclature is no longer well defined. Hill [5] uses the term ‘‘Zipf’s law’’ to denote Zipf’s generalization (3) and the family (4) derived from it. He derives (3) from an urn model with two important features. The first is a twofold classification of cities (genera) into regions (families) in a country, and then of cities (genera) within regions (families); the second is a Bose-Einstein scheme of allocating the population (species) to cities (genera) within regions (families); see FERMI-DIRAC, MAXWELL-BOLTZMANN, AND BOSE-EINSTEIN STATISTICS. The allocation schemes are independent between regions (families). In the city-region context, let L(Ri ) be the size of the Ri th largest city in the ith region. Then under some regularity conditions, the ordered values of L(Ri ) across regions should yield a Zipf rank-size curve (3), asymptotically. The same should hold if we select a city at random from each region. Finally, a rank-size curve of all cities in the country should be of form (3), approximately. For further discussion, see Hill [5]; the derivation of convergence in probability to the size-frequency form (4) is given in ref. 6, and necessary and sufficient conditions for the same in ref. 13. These results address the strong form of Zipf’s law; in this, the proportion of cities, etc., with population r (or grouped around r) has a probability distribution given by (4). In the weak form of the law, it is only the expected value of this proportion that has the form (4). Chen [2] discusses general urn models leading to the weak form and gives further references. Rouault [9] shows that sequences of values taken by certain finite Markov chains that exclude passage at any stage to the same state (see MARKOV PROCESSES) may follow the distribution (4) for 0 < a < 1. Simon [12] describes a stochastic process that leads to a stationary distribution with discrete density function g(r) ∝ B(r, a + 1), where B(·, ·) is the beta function. For large r, g(r) ∼ (a + 1)r−(a+1) ,

so that the Yule distribution∗ (5) approximates (4) in the tails. If a = 1, (4) gives g(r) = [r(r + 1)]−1 , ∼ 1/r2 for large r. Yule [14] also explained the distribution of genera by numbers of species by means of (5) as a limiting distribution. For applications of Zipf’s law to bibliographic data bases see ref. 3 and the references listed therein, and for applications to prediction in geological studies see ref. 10. Rapoport [8] gives an interesting discussion of early papers anticipating Zipf’s law. REFERENCES 1. Abramowitz, M. and Stegun, I. A. (1964). Handbook of Mathematical Functions: Appl. Math. Series No. 55, National Bureau of Standards, Washington, D.C. 2. Chen, W.-C. 611–622.

(1980).

J. Appl.

Prob.,

17,

3. Fedorowicz, J. (1982). J. Amer. Soc. Inf. Sci., 33, 285–293. 4. Fox, W. R. and Lasker, G. W. (1983). Int. Statist. Rev., 51, 81–87. 5. Hill, B. M. (1974). J. Amer. Statist. Ass., 69, 1017–1026. 6. Hill, B. M. and Woodroofe, M. (1975). J. Amer. Statist. Ass., 70, 212–219. 7. Johnson, N. L. and Kotz, S. (1969). Distributions in Statistics: Discrete Distributions, Vol. 1. Wiley, New York, p. 240. 8. Rapoport, A. (1978). International Encyclopedia of Statistics, W. H. Kruskal and J. M. Tanur, eds. Free Press, New York, pp. 847–854. 9. Rouault, A. (1978). Ann. Inst. H. Poincaré, B, 14, 169–188. 10. Rowlands, N. J. and Sampey, D. (1977). J. Int. Ass. Math. Geol. 9, 383–392. 11. Seal, H. L. (1952). J. Inst. Actuaries, 78, 115–121. 12. Simon, H. A. (1955). Biometrika, 42, 425–440. 13. Woodroofe, M. and Hill, B. (1975). J. Appl. Prob., 12, 425–434. 14. Yule, G. U. (1944). The Statistical Study of Literary Vocabulary. Cambridge University Press, Cambridge, England. 15. Zipf, G. K. (1949). Human Behavior and the Principle of Least Effort. Addison-Wesley, Reading, MA.

ZONAL POLYNOMIALS See also FACTORIAL SERIES DISTRIBUTIONS; URN MODELS; and YULE DISTRIBUTION.

CAMPBELL B. READ

ZONAL POLYNOMIALS Many distributions and moments in multivariate analysis∗ based on the multivariate normal distribution can be expressed as power series in symmetric functions∗ of m variables. Often, but not always, these power series may be written in terms of hypergeometric functions of matrix argument. James [13] gives a survey of such distributions for random matrices and their latent roots. The power series involved in multivariate distribution theory can be expanded in terms of one of the many types of symmetric polynomials. For a given basis of the symmetric polynomials, the individual homogeneous polynomials of degree k are usually indexed by partitions

we can define a linear transformation of the space Vk by

φ → T(L)φ : (T(L)φ)(X) = φ(L−1 XL−1 ). This transformation defines a representation of the real linear group Gl(m, R) in the vector space Vk —i.e., the mapping L → T(L) is a homomorphism from Gl(m, R) to the group of linear transformations of Vk . A subspace V ⊂ Vk is invariant if T(L)V ⊂ V , for all L ∈ Gl(m, R). If, in addition, V contains no proper invariant subspaces, it is called an irreducible invariant subspace. It can be shown that the space Vk decomposes into a direct sum of irreducible invariant subspaces Vκ , Vk = ⊕ Vκ , κ

κ = (k1 , k2 , . . . , km ), k1 k2 · · · km 0, m

ki = k,

i=1

of k into not more than m parts. One particular class of homogeneous symmetric polynomials, namely the class of zonal polynomials, yields an enormous simplification of the coefficients in these power series. These polynomials are derived from the group representation theory of Gl(m, R), the general linear group of m × m real nonsingular matrices, and their study was initiated independently by Hua [9] and James [10]. The general theory of zonal polynomials was developed in a series of papers by James [10–16] and Constantine [3]. We now turn to the group-theoretic definition of zonal polynomials. (An alternate but closely related definition that leads more directly to an algorithm for calculation will be given in the next section). Let Vk be the vector space of homogeneous polynomials φ(X) of degree k in the m(m + 1)/2 different elements of the symmetric m × m matrix X. Corresponding to any congruence transformation X → LXL ,

L ∈ Gl(m, R),

9225

where κ = (k1 , k2 , . . . , km ), k1 k2 · · · km 0, runs over all partitions of k into not more than m parts. The polynomial (trX)k ∈ Vk then has a unique decomposition (trX)k =

Cκ (X),

(1)

κ

into polynomials Cκ (X) ∈ Vκ , belonging to the respective invariant subspaces. The polynomial Cκ (X) is the zonal polynomial corresponding to the partition κ; it is a symmetric homogeneous polynomial of degree k in the latent roots of X. When m = 1, (1) becomes xk = C(k) (x) so that the zonal polynomials of a matrix are analogous to powers of a single variable. Equation (1) holds for all m, with Cκ (X) ≡ 0 if the partition κ has more than m parts. For detailed discussions of the group-theoretic construction of zonal polynomials, the reader is referred to Farrell [6], Kates [17], and the papers of James referenced earlier, particularly James [12,13]. Another approach to zonal polynomials has been given by Saw [27] and an essentially combinatoric approach has been given by Takemura [32]; a useful survey paper has been written by Subrahmaniam [28].

9226

ZONAL POLYNOMIALS

CALCULATION OF ZONAL POLYNOMIALS No general formula for zonal polynomials is known and methods for calculating them have been given by James [11,13,15], Saw [27], and Kates [17]. The discussion in this section is based on the papers by James [14,15]; see also Muirhead [24]. Let k be a positive integer and order the partitions of k lexicographically—i.e., if κ = (k1 , . . . , km ) and λ = (l1 , . . . , lm ) are two partitions of κ, then κ > λ if ki > li for the first index i for which the parts are unequal. If κ and λ are two partitions of k with κ > λ and y1 , . . . , ym are m variables, the monok mial y11 , . . . , ykmm is said to be of higher weight l than the monomial y11 , . . . , ylmm . A definition of zonal polynomials, which leads to a general algorithm for their calculation follows. Definition. Let Y be an m × m symmetric matrix with latent roots y1 , . . . , ym and let κ = (k1 , . . . , km ) be a partition of k into not more than m parts. The zonal polynomial of Y corresponding to κ, Cκ (Y), is a symmetric homogeneous polynomial of degree k in the latent roots y1 , . . . , ym satisfying the following three conditions: (i) The term of highest weight in Cκ (Y) k k is y11 , . . . , ykmm ; i.e., Cκ (Y) = dκ y11 , . . . , km ym + terms of lower weight, where dκ is a constant. (ii) Cκ (Y) is an eigenfunction of the differential operator Y given by Y =

m i=1

y2i

m m y2i ∂2 ∂ + , 2 y − y ∂y ∂yi i j i i=1 j=1 j=i

i.e., Y Cκ (Y) = ακ Cκ (Y), where ακ is a constant. (iii) As κ runs over all partitions of k the zonal polynomials have unit coefficients in the expansion of

(trY)k ; i.e., k k (trY) = (y1 + · · · + ym ) = κ Cκ (Y). This definition of zonal polynomials is intimately related to the definition in the first section. Because of its group-theoretic nature, it is known that Cκ (Y) must be an eigenfunction of a differential operator called the

Laplace-Beltrami operator. The differential operator Y in (ii) is derived from this operator. Using conditions (i) and (ii), it can be readily shown that the constant (or eigenvalue) ακ in (iii) is ακ = ρκ + k(m − 1), where ρκ =

m

ki (ki − i),

(2)

i=1

so that Cκ (Y) satisfies the second-order partial differential equation Y Cκ (Y) = [ρκ + k(m − 1)]Cκ (Y).

(3)

This forms the basis of an algorithm developed by James [14] for calculating the coefficients of the terms in Cκ (Y). Basically what happens is that condition (i), along with the condition that Cκ (Y) is a symmetric homogeneous polynomial of degree k, establishes what types of terms appear in Cκ (Y). The differential equation (3) for Cκ (Y) then gives recurrence relations between the coefficients of these terms, which determine Cκ (Y) uniquely up to some normalizing constant. The normalization is provided by condition (iii). Zonal polynomials can be conveniently expressed in terms of the monomial symmetric functions∗ . If κ = (k1 , . . . , km ), the monomial symmetric function of y1 , . . . , ym corresponding to κ is defined as Mκ (Y) =

k

k

kp

yi 1 yi 2 , . . . , yip , 1

2

where p is the number of nonzero parts in the partition κ and the summation is over the distinct permutations (i1 , . . . , ip ) of p different integers from the integers 1, . . . , m. Condition (i) and the fact that Cκ (Y) is symmetric and homogeneous of degree k show that Cκ (Y) can be expressed in terms of the monomial symmetric functions as cκ,λ Mλ (Y), (4) Cκ (Y) = λκ

where the cκ,λ are constants and the summation is over all partitions λ of k with λ κ, i.e., λ is below or equal to κ in the lexicographical ordering. Substituting (4) in the partial differential equation (3) and equating

ZONAL POLYNOMIALS

coefficients of like monomial symmetric functions on both sides leads to a recurrence relation for the coefficients, namely, cκ,λ =

[(li + t) − (lj − t)] cκ,µ , ρκ − ρλ λ0

etr(−XZ)(det X)a−(m+1)/2 Cκ (X−1 Y)(dX) X>0

= and

m (a) = π m(m−1)/4

m

(a − 12 (i − 1))

(11)

j=1

(sometimes called the ‘‘multivariate gamma function’’), and (a)κ is given by (6). In (8) and (9), Z is a complex symmetric m × m matrix with Re(Z) > 0, Y is a symmetric m × m matrix and the integrations are over the space of all positive definite m × m matrices X; (8) is valid for Re(a) > 12 (m − 1) and (9) is valid for Re(a) > k1 + 12 (m − 1), where κ = (k1 , k2 , . . . , km ). In (10), Y is a symmetric m × m matrix, the integration is over all positive definite matrices X with I − X positive definite, and Re(a) > 12 (m − 1) and Re(b) > 12 (m − 1). The integrals (8) and (10) show that a zonal polynomial has a reproductive property under expectations taken with respect to the Wishart and matrix-variate beta distributions∗ . Specifically, if A has the Wm (n, ) distribution (Wishart∗ with n degrees of freedom and covariance matrix ) with n > m − 1 and B is an arbitrary symmetric (nonrandom) m × m matrix, then (8) shows that E[Cκ (AB)] = 2k ( 21 n)κ Cκ (B). If A has a beta (a, b) distribution with density function proportional to (det A)a−(m+1)/2 det(Im − A)b−(m+1)/2 , 0 < A < Im ,

and B is a fixed m × m symmetric matrix, (9) (8) shows that

= (a)κ m (a)(det Z)−a Cκ (YZ−1 ),

In these integrals

E[Cκ (AB)] =

(a)κ Cκ (B). (a + b)κ

Many other integrals involving zonal poly−a (det Z) C (YZ), (9) κ nomials are known. For an excellent survey (−a + 12 (m + 1))κ of these the interested reader is referred to Subrahmaniam [28] and the extensive list of references there. (det X)a−(m+1)/2 (−1)k m (a)

0<X 0 and Cκ (Y) is increasing in each yi , i = 1, . . . , m (James [14] and Chattopadhyay and Pillai [2]). (ii) If Y is a positive definite m × m matrix, then Cκ (Y)Cκ (Y−1 ) Cκ (I)2 . (iii) A generalization of the binomial expansion, due to Constantine [4], is k Cκ (Im + Y) κ Cσ (Y) = , Cκ (Im ) σ Cσ (Im ) σ s=0

where the inner summation is over all partitions σ of the integer s. This defines the generalized binomial coefficients σκ , which have been tabulated to k = 4 by Constantine [4] and to k = 8 by Pillai and Jouris [26]. For further results on these coefficients, see Bingham [1] and Muirhead [23]. (iv) It is sometimes useful to express a product of two zonal polynomials in terms of other zonal polynomials, namely, gκσ τ Cκ (Y), Cσ (Y)Cτ (Y) = k

where σ is a partition of s, τ is a partition of t, κ runs over all partitions of k = s + t, and the gκσ τ are constants, some of which have been tabulated by Khatri and Pillai [19]. (v) A second-order partial differential equation satisfied by Cκ (Y) has already been given in (3). A fourth-order partial differential equation has also been established by Sugiura [30]. (vi) A number of sums involving zonal polynomials have been evaluated by Sugiura and Fujikoshi [31], Fujikoshi [7], Muirhead [22], and Sugiura [29]. We now list some of these, in which etr(Y) ≡ exp tr Y, si = tr Yi , and the constants ρκ and πκ are given by ρκ =

m i=1

ki (ki − i),

9229

and πκ =

m

ki (4k2i − 6iki + 3i2 ),

i=1 ∞ Cκ (Y) = sl1 etr(Y), (k − l)! κ k=0

∞ ρκ Cκ (Y) = s2 etr(Y), k! κ k=0

∞ 2 ρκ Cκ (Y) k! κ k=0

= (s22 + 4s3 + s21 + s2 )etr(Y), ∞ kρκ Cκ (Y) = (s1 s2 + 2s2 )etr(Y), k! κ k=0

∞ πκ Cκ (Y) k! κ k=0

= (4s3 + 3s21 + 3s2 + s1 )etr(Y), ρκ Cκ (Y) = k(k − 1)s2 sk−2 1 ,

κ

ρκ2 Cκ (Y) = k(k − 1)[(s2 + s21 )sk−2 1

κ

+ 4(k − 2)s3 sk−3 1 + (k − 2)(k − 3)s22 sk−4 1 ], πκ Cκ (Y) κ

= k[sk1 + 3(k − 1)(s2 + s21 )sk−2 1 + 4(k − 1)(k − 2)(k − 3)s3 sk−3 1 ]. (vii) The effects of a number of differential operators on Cκ (Y) have been studied, primarily to establish partial differential equations for hypergeometric functions of matrix argument. For work in this direction, see Muirhead [22], Constantine and Muirhead [5], James [14,16], Fujikoshi [8], and the survey paper of Subrahmaniam [28]. REFERENCES 1. Bingham, C. (1974). An identity involving partitional generalized binomial coefficients. J. Multivariate Anal., 4, 210–223.

9230

ZONAL POLYNOMIALS

2. Chattopadhyay, A. K. and Pillai, K. C. S. (1970). On the Maximization of a Matrix Function over the Group of Orthogonal Matrices. Mimeograph Series No. 248, Statistics Dept., Purdue University, Lafayette, IN. 3. Constantine, A. G. (1963). Some noncentral distribution problems in multivariate analysis. Ann. Math. Statist., 34, 1270–1285. 4. Constantine, A. G. (1966). The distribution of Hotelling’s generalized T02 . Ann. Math. Statist., 37, 215–225. 5. Constantine, A. G. and Muirhead, R. J. (1972). Partial differential equations for hypergeometric functions of two argument matrices. J. Multivariate Anal., 3, 332– 338. 6. Farrell, R. H. (1976). Techniques of Multivariate Calculation. Springer, New York. 7. Fujikoshi, Y. (1970). Asymptotic expansions of the distributions of test statistics in multivariate analysis. J. Sci. Hiroshima Univ. Ser. A-I, 34, 73–144. 8. Fujikoshi, Y. (1975). Partial differential equations for hypergeometric functions 3 F2 of matrix argument. Canad. J. Statist., 3, 153–163. 9. Hua, L. K. (1959). Harmonic Analysis of Functions of Several Complex Variables in the Classical Domains. Moscow (in Russian). 10. James, A. T. (1960). The distributions of the latent roots of the covariance matrix. Ann. Math. Statist., 31, 151–158. 11. James, A. T. (1961a). The distributions of noncentral means with known covariance. Ann. Math. Statist., 32, 874–882. 12. James, A. T. (1961b). Zonal polynomials of the real positive definite symmetric matrices. Ann. Math., 74, 456–469. 13. James, A. T. (1964). Distributions of matrix variates and latent roots derived from normal samples. Ann. Math. Statist., 35, 475–501. 14. James, A. T. (1968). Calculation of zonal polynomial coefficients by use of the LaplaceBeltrami operator. Ann. Math. Statist., 39, 1711–1718. 15. James, A. T. (1973). The variance information manifold and the functions on it. In Multivariate Analysis III, P. R. Krishnaiah, ed. Academic, New York, pp. 157–159. 16. James, A. T. (1976). Special functions of matrix and single argument in statistics. In Theory and Applications of Special Functions, R. A. Askey, ed. Academic, New York, pp. 497–520.

17. Kates, L. K. (1980). Zonal Polynomials. Ph.D. Thesis, Princeton University, Princeton, NJ. 18. Khatri, C. G. (1966). On certain distribution problems based on positive definite quadratic functions in normal vectors. Ann. Math. Statist., 37, 468– 479. 19. Khatri, C. G. and Pillai, K. C. S. (1968). On the noncentral distributions of two test criteria in multivariate analysis of variance. Ann. Math. Statist., 39, 215–226. 20. Krishnaiah, P. R. (1976). Some recent developments on complex multivariate distributions. J. Multivariate Anal., 6, 1–30. 21. McLaren, M. L. (1976). Coefficients of the zonal polynomials. Appl. Statist., 25, 82–87. 22. Muirhead, R. J. (1970). Partial differential equations for hypergeometric functions of matrix argument. Ann. Math. Statist., 41, 991–1001. 23. Muirhead, R. J. (1974). On the calculation of generalized binomial coefficients. J. Multivariate Anal., 4, 341–346. 24. Muirhead, R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley, New York. 25. Parkhurst, A. M. and James, A. T. (1974). Zonal polynomials of order 1 through 12. In Selected Tables in Mathematical Statistics, H. L. Harter and D. B. Owen, eds. American Mathematical Society, Providence, RI, pp. 199–388. 26. Pillai, K. C. S. and Jouris, G. M. (1969). On the moments of elementary symmetric functions of the roots of two matrices. Ann. Inst. Statist. Math., 21, 309–320. 27. Saw, J. G. (1977). Zonal polynomials: An alternative approach. J. Multivariate Anal., 7, 461–467. 28. Subrahmaniam, K. (1976). Recent trends in multivariate normal distribution theory: On the zonal polynomials and other functions of matrix argument. Sankhya A, 38, 221– 258. 29. Sugiura, N. (1971). Note on some formulas for weighted sums of zonal polynomials. Ann. Math. Statist., 42, 768–772. 30. Sugiura, N. (1973). Derivatives of the characteristic root of a symmetric or a Hermitian matrix with two applications in multivariate analysis. Commun. Statist., 1, 393– 417. 31. Sugiura, N. and Fujikoshi, Y. (1969). Asymptotic expansions of the non-null distributions

ZYSKIND–MARTIN MODELS of the likelihood ratio criteria for multivariate linear hypothesis and independence. Ann. Math. Statist., 40, 942–952. 32. Takemura, A. (1984). Zonal Polynomials, Lecture Notes–Monograph Series 4, Institute of Mathematical Statistics, Hayward, CA. See also LATENT ROOT DISTRIBUTIONS; MATRIX-VALUED DISTRIBUTIONS; MATRIX-VARIATE BETA DISTRIBUTION; MULTIVARIATE ANALYSIS; SYMMETRIC FUNCTIONS; and WISHART DISTRIBUTION.

ROBB J. MUIRHEAD

1/3



 2   2   xα − xα /(n − 1) 

,

α=i

i = 1, . . . , n, are calculated, and the product moment correlation r between xi and yi computed, using the n pairs of values (xi , yi ). Then if X has a normal distribution,

Z=

1 1+r log 2 1−r

has approximately a normal distribution with mean zero and variance 3n−1 (see FISHER’S ZTRANSFORMATION). A better approximation to the upper 100α% point of the distribution of Z is Z1−α u1−α +

1. Lin, C. C. and Mudholkar, G. S. Biometrika, 67, 455–461.

(1980).

TRANSFORMATION

Lin and Mudholkar [1] proposed a test of normality based on a random sample of size n, giving sample values x1 , x2 , . . . , xn of a variable X. The statistics

α=i

REFERENCE

Z -TRANSFORMATION. See FISHER’S Z-

Z -TEST OF NORMALITY

yi = n

The power of the test is comparable (although less) than that of the Shapiro-Wilk∗ test. The Z-test was constructed primarily for detecting asymmetric alternatives to normality but it compares quite well with tests for departure from normality∗ against longtailed symmetrical alternatives.

See also DEPARTURES FROM NORMALITY, TESTS FOR.

ZONE CHART. See STRATA CHART

−1

9231

1 3 − 3u1−α )γ2n , (u 24 1−α

where σn2 = 3n−1 − 7.32n−2 + 53.005n−3 , γn = −11.70n−1 + 55.06n−2 , and u1−α is the upper 100α% point of the standard normal distribution. This approximation is adequate for n as small as 5. Large values of |Z| are regarded as significant.

ZYSKIND–MARTIN MODELS Consider general linear models∗ of the form Y = Xβ + Z. Y is a n × 1 vector of observed random variables; X is a n × p matrix of rank r, r p n, of concomitant variables∗ (with known values); β is a p × 1 vector of regression coefficients∗ (parameters with unknown values); and Z is a n × 1 vector of random residuals with E[Z] = 0 and

Var(Z) = E[ZZ ] = σ 2 V,

where V is known. Zyskind-Martin models [4] have the additional property that the space spanned by columns of X is contained in the space spanned by columns of V. These models have the property that the best linear unbiased estimator of any estimable linear function a β of the regression coefficients is the same linear function a β of the solution β˜ of the general normal equations∗ X V∗ Xβ˜ = X V∗ Y, where V∗ is any conditional inverse of V. The name Zyskind-Martin models was coined by Oktaba [1], who studied some special cases. Estimation of β˜ in such models with missing values is discussed in Oktaba

9232

ZYSKIND–MARTIN MODELS

et al. [2]. There are some interesting historical remarks in [3]. REFERENCES

3. Shelton, J. T. and Lowe, V. W. (1986). Amer. Statist., 40, 78–79. 4. Zyskind, G. and Martin, F. B. (1969). SIAM J. Appl. Math., 17, 1190–1202.

1. Oktaba, W. (1984). Biomed. J., 26, 415– 424. 2. Oktaba, W., Kornacki, A., and Wawrzosek, J. (1985). Biomed. J., 27, 733–740.

See also ESTIMABILITY; GENERALIZED INVERSES; and GENERAL LINEAR MODEL.

Encyclopedia of Statistical Sciences

Encyclopedia of Forensic Sciences

Encyclopedia of forest sciences

Encyclopedia of hydrological sciences

Encyclopedia of Ocean Sciences,

International Encyclopedia of Statistical Science

International Encyclopedia of Statistical Science

International Encyclopedia of Statistical Science

Encyclopedia of the Philosophical Sciences

Encyclopedia of the Cognitive Sciences

The Encyclopedia of Occult Sciences

Statistical Methods for Health Sciences

Encyclopedia of Solid Earth Geophysics (Encyclopedia of Earth Sciences Series)

Encyclopedia of Geobiology (Encyclopedia of Earth Sciences Series)

Statistical methods for health sciences

Encyclopedia of Forensic Sciences 3. Volumes

International Encyclopedia of Social & Behavioral Sciences

International Encyclopedia of Social & Behavioral Sciences

Encyclopedia of Dairy Sciences, Four-Volume Set

The MIT Encyclopedia of the Cognitive Sciences

Historical Encyclopedia of Natural and Mathematical Sciences

International encyclopedia of the social sciences volume2

International Encyclopedia Of The Social Behavioral Sciences

Computational Invariant Theory (Encyclopedia of Mathematical Sciences)

Cell Death (Encyclopedia of Life Sciences)

International Encyclopedia of Social & Behavioral Sciences

Historical encyclopedia of natural and mathematical sciences

Historical Encyclopedia of Natural and Mathematical Sciences

International Encyclopedia of Social & Behavioral Sciences

International Encyclopedia of Social & Behavioral Sciences

The MIT Encyclopedia of the Cognitive Sciences

Encyclopedia of Statistical Sciences

Encyclopedia of Forensic Sciences

Encyclopedia of forest sciences

Encyclopedia of hydrological sciences

Encyclopedia of Ocean Sciences,

International Encyclopedia of Statistical Science

International Encyclopedia of Statistical Science

International Encyclopedia of Statistical Science

Encyclopedia of the Philosophical Sciences

Encyclopedia of the Cognitive Sciences

The Encyclopedia of Occult Sciences

Statistical Methods for Health Sciences

Encyclopedia of Solid Earth Geophysics (Encyclopedia of Earth Sciences Series)

Encyclopedia of Geobiology (Encyclopedia of Earth Sciences Series)

Statistical methods for health sciences

Encyclopedia of Forensic Sciences 3. Volumes

International Encyclopedia of Social & Behavioral Sciences

International Encyclopedia of Social & Behavioral Sciences

Encyclopedia of Dairy Sciences, Four-Volume Set

The MIT Encyclopedia of the Cognitive Sciences

Historical Encyclopedia of Natural and Mathematical Sciences

International encyclopedia of the social sciences volume2

International Encyclopedia Of The Social Behavioral Sciences

Computational Invariant Theory (Encyclopedia of Mathematical Sciences)

Cell Death (Encyclopedia of Life Sciences)

International Encyclopedia of Social & Behavioral Sciences

Historical encyclopedia of natural and mathematical sciences

Historical Encyclopedia of Natural and Mathematical Sciences

International Encyclopedia of Social & Behavioral Sciences

International Encyclopedia of Social & Behavioral Sciences

The MIT Encyclopedia of the Cognitive Sciences

Recommend Documents