Encyclopedia of Statistical Sciences, Preference Mapping to Recovery of Interblock Information (Volume 10)

ENCYCLOPEDIA OF STATISTICAL SCIENCES Volume 10 Preference Mapping to Recovery of Interblock Information

ENCYCLOPEDIA OF STATISTICAL SCIENCES Second Edition

Volume 10 Preference Mapping to Recovery of Interblock Information

The Encyclopedia of Statistical Sciences is available Online at http://www.mrw.interscience.wiley.com/ess

A John Wiley & Sons, Inc., Publication

CONTRIBUTORS J. R. Abernathy, University of North Carolina, Chapel Hill, NC. Randomized Response Robert J. Adler, The Technion, Haifa, Israel. Random Fields Munir Ahmad, KFUPM, Dhahran, Saudi Arabia. PValues, Combination of James H. Albert, Bowling Green State University, Bowling Green, OH. Probit Models, Bayesian Analysis of Erling B. Andersen, University of Copenhagen, Copenhagen, Denmark. Rasch, Georg J. R. Ashford, University of Exeter, Exeter, United Kingdom. Probit Analysis R. A. Bailey, Rothamsted Experimental Station, Harpenden, United Kingdom. Randomization, Constrained N. Balakrishnan, McMaster University, Hamilton, ON, Canada. Record Values and Record Statistics L. A. Baxter, State University of New York, Stony Brook, NY. Quasi Life Tables Betsy Jane Becker, Michigan State University, East Lansing, MI. P-Values, Combination of M. L. Berenson, Baruch College, New York, NY. Puri’s Expected Normal Scores Test Matilde Bini, Universita degli Studi di Firenze, Firenze, Italy. Raked (Adjusted) Kappa ¨ , Institut fur ¨ Statistik und Okonometrie, Fred Boker Gottingen, Germany. P-Values and Power of Tests K. R. W. Brewer, Commonwealth Schools Commission, Woden, Australia. Primary Sampling Unit; Proportional Allocation Jacob Burbea, University of Pittsburgh, Pittsburgh, PA. Rao Distance S. Cambanis, University of North Carolina, Chapel Hill, NC. Radon–Nikodym Theorem

H. Caussinus, Universit´ee, Paul Sabatier, Toulouse, France. Principal Component Analysis, Generalized David Chappell, University of Sheffield, Sheffield, United Kingdom. Randomness, BDS Test for V. Chew, University of Florida, Gainesville, FL. Radial Error Raj S. Chhikara, National Aeronautics and Space Center, Houston, TX. Proportion Estimation in Surveys using Remote Sensing Siddhartha Chib, Washington University, St. Louis, MO. Probit Models, Bayesian Analysis of Norman Cliff, Carnegie Mellon University, Pittsburgh, PA. Psychometrika Clifford C. Clogg, Pennsylvania State University, University Park, PA. Quasi-Independence Marc-David Cohen, SAS Institute, Cary, NC. PseudoRandom Number Generators Peter Coughlin, University of Maryland, College Park, MD. Probabilistic Voting Models Keren L. Coxe, Washington, DC. Principal Components Regression Analysis ´ Csorg ˝ o˝ , Carleton University, Ottawa, ON, Miklos Canada. Quantile Processes J. N. Darroch, The Flinders University, Bedford Park, South Australia. Quasi-Symmetry C. E. Davis, University of North Carolina, Chapel Hill, NC. Quantile Estimation A. P. Dawid, University College, London, United Kingdom. Prequential Analysis; Probability Forecasting Morris H. Degroot, Carnegie Mellon University, Pittsburgh, PA. Record Linkage and Matching Systems W. E. Deming, Washington, DC. Principles of Professional Statistical Practice B. Dennis, Pennsylvania State University, University Park, PA. Profiles of Diversity v

vi

CONTRIBUTORS

D. Disch, Rose-Hulman Institute of Technology, Terre Haute, IN. Ramsey’s Prior Constantin A. Drossos, University of Patras, Patras, Greece. Probability and Logic Acheson J. Duncan, Johns Hopkins University, Baltimore, MD. Quality Control, Statistical John F. Early, US Bureau of Labor Statistics, Washington, DC. Producer Price Indexes Eugene S. Edgington, University of Calgary, Calgary, AB, Canada. Randomization Tests A. W. F. Edwards, University of Cambridge, Cambridge, United Kingdom. Problem of Points Regina C. Elandt-Johnson, University of North Carolina, Chapel Hill, NC. Rates, Standardized; Rates Eva Elvers, Statistics Sweden, Stockholm, Sweden. Quality Concept for Official Statistics A. K. Md. Eshanes Saleh, Carleton University, Ottawa, ON, Canada. Rank Tests, Grouped Data in Randall L. Eubank, Southern Methodist University, Dallas, TX. Quantile Brian Everitt, Institute of Psychiatry, London, United Kingdom. Psychology, Statistics in Jean-Claude Falmagne, New York University, New York, NY. Psychophysics, Statistical Methods in Robert E. Fay, US Bureau of the Census, Washington, DC. Rao and Scott (Type) Tests Stephen E. Fienberg, Carnegie Mellon University, Pittsburgh, PA. Rasch Model Peter D. Finch, Monash University, Clayton, Australia. Randomization—I Terrence L. Fine, Cornell University, Ithaca, NY. Probability, Foundations of—II; Probability, Foundations of—I David Firth, University of Oxford, Oxford, United Kingdom. Quasiscore Bernard D. Flury, Indiana University, Bloomington, IN. Principal Components; Principal Points G. H. Freeman, University of Warwick, Coventry, United Kingdom. Quasi-Factorial Designs Janos Galambos, Temple University, Philadelphia, PA. Probabilistic Number Theory; Raikov’s Theorem J. L. Gastwirth, George Washington University, Washington, DC. Pyramid Scheme Models Jean Dickinson Gibbons, University of Alabama, Tuscaloosa, AL. Randomized Tests; Randomness, Tests of; Ranking Procedures; P-Values; Rank Tests Richard Gill, University of Utrecht, Utrecht, The Netherlands. Product Integration Michael Goldstein, University of Hull, Hull, United Kingdom. Prevision B. G. Greenberg, University of North Carolina, Chapel Hill, NC. Randomized Response

Raymond S. Greenberg, Emory University, Decatur, GA. Prospective Studies Kenneth H. Gross, California Survey Research, Sherman Oaks, CA. Random-Digit Dialing, Sampling Methods for Emil Grosswald, Temple University, Philadelphia, PA. Rademacher Functions Rudolph Haag, University of Hamburg, Hamburg, Germany. Quantum Mechanics, Statistical Interpretation of C. C. Heyde, University of Melbourne, Parkville, Australia. Probability Theory: An Outline; Quantile Transformation Methods; Random Matrices; Random Sum Distributions Katuomi Hirano, Institute of Statistical Mathematics, Tokyo, Japan. Randomized Gamma Distribution; Rayleigh Distribution Bruce Hoadley, Bell Communications Research, Holmdel, NJ. Quality Measurement Plan (QMP) David C. Hoaglin, Harvard University, Cambridge, MA. Random Number Generation W. Hoeffding, University of North Carolina, Chapel Hill, NC. Probability Inequalities for Sums of Bounded Random Variables; Range-Preserving Estimators D. G. Horvitz, Research Triangle Institute, Research Triangle, NC. Randomized Response J. R. M. Hosking, Institute of Hydrology, Wallingford, United Kingdom. Quenouille’s Test A. Glen Houston, University of Houston, Houston, TX. Proportion Estimation in Surveys using Remote Sensing Nuri T. Jazairi, York University, Toronto, ON, Canada. Productivity Measurement; Purchasing Power Parity Lyle V. Jones, Center for Advanced Study in the Behavioral Sciences, Stanford, CA. Psychological Scaling Eleanor W. Jordan, University of Texas, Austin, TX. Problem Solving in Statistics Brian Junker, Carnegie Mellon University, Pittsburgh, PA. Psychometrika John D. Kalbfleisch, University of Western Ontario, London, ON, Canada. Pseudo-Likelihood Graham Kalton, University of Michigan, Ann Arbor, MI. Question-Wording Effects in Surveys D. Kannan, University of Georgia, Athens, GA. Processes, Discrete ´ , Adam Mickiewicz University, PozMichal Karonski nan, Poland. Random Graphs Richard Kay, University of Sheffield, Sheffield, United Kingdom. Proportional Hazards Model, Cox’s Oscar Kempthorne, Iowa State University, Ames, IA. Randomization—II

CONTRIBUTORS D. G. Kendall, University of Cambridge, Cambridge, United Kingdom. Quantum Hunting; Random Sets of Points K. B. Kulasekera, Clemson University, Clemson, SC. Pseudo Residuals; Quasi Residuals James M. Landwehr, AT & T Bell Laboratories, Murray Hill, NJ. Projection Plots Louis Lefeber, York University, Toronto, ON, Canada. Productivity Measurement A. M. Liebetrau, Batelle Pacific Northwest Laboratories, Richmond, WA. Proportional Reduction in Error (PRE) Measures of Association Frederic M. Lord, Educational Testing Service, Princeton, NJ. Psychological Testing Theory David W. Matula, Southern Methodist University, Dallas, TX. Random Graphs P. McCullagh, Imperial College, London, United Kingdom. Quasi-Likelihood Functions Paul W. Mielke, Jr., Colorado State University, Fort Collins, CO. Quantit Analysis R. E. Miles, Australian National University, Canberra, Australia. Random Tessellations David Moher, University of Ottawa, Ottawa, ON, Canada. Quality Assessment for Clinical Trials Stephen G. Nash, Johns Hopkins University, Baltimore, MD. Quasi-Random Sampling; QuasiRandom Sequences Rajeshwari Natarajan, Southern Methodist University, Dallas, TX. Reciprocal Inverse Gaussian Distribution Bent Natvig, University of Oslo, Oslo, Norway. Priority Queue Valery Nevzorov, St. Petersburg State University, St. Petersburg, Russia. Record Values and Record Statistics J. V. Noble, University of Virginia, Charlottesville, VA. Quantum Mechanics and Probability W. R. S. North, Northwick Park Hospital, Harrow, United Kingdom. Quangle David J. Pack, Union Carbide Corporation, Oak Ridge, TN. Prior Distributions; Prior Probabilities Anthony G. Pakes, University of Western Australia, Nedlands, Western Australia. Queueing Theory S. Panchapakesan, Southern Illinois University, Carbondale, IL. Ranking and Selection Procedures G. P. Patil, Pennsylvania State University, University Park, PA. Profiles of Diversity B. F. Pendleton, University of Akron, Akron, OH. Ratio Correlation A. N. Pettitt, University of Technology, Loughborough, United Kingdom. Rank Likelihood E. C. Pielou, University of Lethbridge, Lethbridge, AB, Canada. Quadrat Sampling

vii

Harry O. Posten, University of Connecticut, Storrs, CT. Quincunx Albert Prat, Universitat Politecnica de Catalunya, Barcelona, Spain. Questii ¨ o´ Sandra Pruzansky, AT & T Bell Laboratories, Murray Hill, NJ. Proximity Data C. P. Quesenberry, North Carolina State University, Raleigh, NC. Probability Integral Transformations Damaraju Raghavarao, Temple University, Philadelphia, PA. Random Balance Designs James O. Ramsay, McGill University, Montreal, PQ, Canada. Principal Differential Analysis C. Radhakrishna Rao, University of Pittsburgh, Pittsburgh, PA. Rao’s Axiomatization of Diversity Measures J. N. K. Rao, Carleton University, Ottawa, ON, Canada. Ratio Estimators—I Poduri S. R. S. Rao, University of Rochester, Rochester, NY. Ratio Estimators—II Amnon Rapoport, University of Haifa, Haifa, Israel. Psychological Decision Making Campbell B. Read, Southern Methodist University, Dallas, TX. Ranges; Rao–Blackwell Theorem Bengt Ros´en, Statistics Sweden, Stockholm, Sweden. Quality Concept for Official Statistics James L. Rosenberger, Pennsylvania State University, University Park, PA. Recovery of Interblock Information William D. Rowe, American University, Washington, DC. Rare-Event Risk Analysis Bernhard Ruger ¨ , University of Munich, Munich, Germany. Profile-a Tests A. Ruiz-Gazen, Universite Toulouse 1-Sciences Sociales, Toulouse, France. Principal Component Analysis, Generalized Harold Sackrowitz, Rutgers University, Piscataway, NJ. Rationality Criterion Ester Samuel-Cahn, Hebrew University, Jerusalem, Israel. Prophet Inequalities Hans T. Schreuder, U.S. Forest Service, Fort Collins, CO. Ratio Estimators, Tin’s; Ratio and Regression Estimators, Mickey’s; Ratio Estimator, Pascual’s P. K. Sen, University of North Carolina, Chapel Hill, NC. Progressive Censoring Schemes; Progressively Censored Data Analysis; Rank Statistics E. Seneta, University of Sydney, Sydney, Australia. Probability, History of L. R. Shenton, University of Georgia, Athens, GA. Quasibinomial Distributions Andrew F. Siegel, University of Washington, Seattle, WA. Rarefaction Curves Gerald L. Sievers, Western Michigan University, Kalamazoo, MI. Probability Plotting

viii

CONTRIBUTORS

Barry Simon, California Institute of Technology, Pasadena, CA. Quantum Physics and Functional Integration C. J. Skinner, University of Southampton, Southampton, United Kingdom. Probability Proportional to Size (PPS) Sampling Grant W. Somes, University of Tennessee, Memphis, TN. Proportions, Cochran’s Q-Statistic for S. M. Steinberg, University of North Carolina, Chapel Hill, NC. Quantile Estimation Stephen M. Stigler, University of Chicago, Chicago, IL. Quetelet, Adolphe S. Lynne Stokes, University of Texas, Austin, TX. Ranked Set Sampling David J. Strauss, University of California, Riverside, CA. Random Utility Models Heinrich Strecker, University of Tuubingen, ¨ Tuubingen, ¨ Germany. Quotient Method Donna F. Stroup, University of Texas, Austin, TX. Problem Solving in Statistics Seymour Sudman, University of Illinois, Urbana, IL. Public Opinion Polls Jiayang Sun, Case Western Reserve University, Cleveland, OH. Projection Pursuit

W. Y. Tan, Memphis State University, Memphis, TN. Quadratic Forms Thaddeus Tarpey, Wright State University, Dayton, OH. Principal Points Rasesh Thakkar, York University, Toronto, ON, Canada. Purchasing Power Parity ¨ Tubingen, ¨ Gerhard Tintner, University of Tuubingen, Germany. Quotient Method R. L. Trader, University of Maryland, College Park, MD. Quasi-Bayesian Inference Brent M. Troutman, U.S. Geological Survey, Lakewood, CO. Rainfall, Landforms, and Streamflow Dietrich Von Rosen, Uppsala University, Uppsala, Sweden. Profile Analysis Gernot Wassmer, Universitat zu Kooeln, ¨ Kooeln, ¨ Germany. Profile-a Tests D. S. Watkins, Washington State University, Pullman, WA. QR Algorithm George H. Weiss, National Institutes of Health, Bethesda, MD. Random Walks D. G. Wellington, Environmental Protection Agency, Washington, DC. Rai and Van Ryzin Dose–Response Model

P

(Continued)

PREFERENCE MAPPING. See GRAPHICAL

consider various properties, such as consistency of our estimator, as n → ∞. However, study of the properties of prequential schemes in the original context of probability forecasting [4,6] suggested that it could be fruitful to reformulate many such nonprequential activities in the prequential setting. The general framework for such ‘‘prequential inference’’ was laid down by Dawid [5]. In it, estimation, for example, is treated merely as an aid to prediction of the next observation. Success at estimation is then measured by the quality of the associated prequential forecasting system, which, in turn, is judged directly by comparing forecasts with actual outcomes. Apart from the pleasing mathematical properties that, it turns out, are possessed by such procedures, it is intuitively more satisfying to formulate inferences in terms of future observables rather than, say, unobservable parameters: for only in such a case can we ever know whether or not we are being successful. Prediction of observables has always been a practically important and theoretically challenging task for probability and statistics: see the Encyclopedia of Statistical Science articles by Kallianpur [15], Ledolter [18], and Geisser [13]. Prequential analysis shares the general concerns of all predictive analysis, but with special emphasis on the assessment of predictive performance through comparison of outcomes with their predictions. In this, it is superficially similar to assessment by cross-validation, or predictive sample reuse [12,26], in which, for a fixed sample, a forecast is made for each data point on the basis of an analysis of all the other points in the sample omitting that one, this forecast then being compared with the omitted value. In prequential analysis, by contrast, the forecast is based on previous values only. This avoids any problems as further data accrue and, more important, eliminates dependence between the assessments for different data points, thus simplifying analysis and interpretation, and permitting the development of a very general and powerful underlying theory.

REPRESENTATION OF DATA

PREFERENCES. See BAYESIAN

INFERENCE;

DECISION THEORY

PREGIBON’S LINK TEST. See LINK TESTS PREMATURE AGGREGATION. See HALF-NORMAL PLOTS

PREQUENTIAL ANALYSIS There are many applied problem areas where data are observed sequentially, and where at any time it is required to issue a forecast, make an inference, or take an action, related to the next observation, on the basis of past and current data. Examples include weather forecasting, playing the stock market, or thermostatic control of a refrigerator. Such problems may be termed prequential, i.e., predictive sequential. In cases where only a point forecast, or some action, is required, it is nonetheless often useful to think of the forecaster as first formulating a probability distribution, Pi+1 say, representing uncertainty about the next observable Xi+1 in the light of the accumulated data Xi = xi [where Xi denotes the initial sequence (X1 , . . . , Xi ) of the variables, and xi := (x1 , . . . , xi ) are their observed values], and then applying this to solve the specific decision problem at hand. A variety of methods is available to assess the quality of such sequential probability forecasts, or their implied decisions, in the light of the actual outcomes [8]. Much of the impetus for these arose from the specific application to meteorology [2,19]. In probability and statistics more generally, we often meet problems in which variables (X1 , X2 , . . .) arise sequentially, but our goals are typically not expressed in this prequential form. For example, we might think of estimating an underlying parameter θ of the joint distribution of X := (X1 , X2 , . . .) on the basis of data Xi = xi , and would then 6391

6392

PREQUENTIAL ANALYSIS

PREQUENTIAL ASSESSMENT OF PROBABILITY FORECASTS Suppose that, for a realized sequence x := (x1 , x2 , . . .) of outcomes of variables X := (X1 , X2 , . . .), a string of probability forecasts P := (P1 , P2 , . . .) has been issued prequentially. Thus Pi+1 is a distribution for Xi+1 , formulated when the data Xi = xi are in hand. One might think of Pi+1 as being formed, as the appropriate conditional distribution for Xi+1 given Xi = xi , from some underlying joint distribution P for X; but this is not necessary. Such an underlying P could be termed a probability forecasting∗ system (PFS), since it determines the forecast for Xi+1 for any i and any data string xi . The general methodology of probability forecasting (see ref. [8], which could usefully be read in conjunction with this article) supplies a number of ways in which we can assess the performance of the issued forecasts P in the light of the actual observations x. In the case of binary (Xi ), we might assess ‘‘calibration’’ up to time
n by examining quantities of the form νn−1 {(xi − pi ) : 1
i
n; i ∈ ξ }, where pi is the assessed probability Pi (Xi = 1); ξ is some appropriately constructed subsequence (which could be the whole sequence, or, for example, contain just those terms i for which |pi − π0 | < , for some fixed π0 and ); and νn is the number of terms in the summation. For a sequence P to be ‘‘valid,’’ such quantities should tend to zero as n → ∞, since if P is a distribution underlying P this property must hold with P-probability 1 [4]. By considering enough subsequences simultaneously, we can show that valid forecasts are essentially uniquely determined, in the sense that, if P and P are both valid forecast sequences for the same outcome sequence x, we must have pi − pi → 0 as i → ∞ [6]. For continuous (Xi ), we can form, from P and x, the conditional probability integral transform (CPIT) ui := Pi (Xi
xi ). Then P may be considered a valid forecast sequence, for data x, if u := (u1 , u2 , . . .) looks like a random sample from the uniform distribution U[0, 1]: again, if P underlies P, this must hold with P-probability 1, thus justifying such a criterion. In particular, for any t and suitable subsequences, we could require that the relative frequency, in the subsequence,

of occurrence of the events Ai : ui
t should converge to t; again forecasts valid under this criterion will be essentially unique. Another approach to forecast assessment is to award a ‘‘performance score’’ si ≡ si (xi , Pi ) to the forecast Pi for Xi when in fact Xi = xi , and to compare different sequences Pn of forecasts for the first n outcomes xn by means of their cumulative scores Sn (x, P) :=
n s (x i i , Pi ). A simple and important special i=1 case has si ≡ log pi (xi ), with
pi (·) the density of Pi . Then Sn (x, P) = ni=1 log pi (xi ) = log p(xn ), the overall log likelihood based on data xn for an underlying joint distribution having joint density p(·). We call it the prequential log likelihood for Pn based on xn . Thus, in this case, comparison is simply likelihood-based. If X is generated from a joint distribution P, and Q is any other distribution for X, then Sn (X, P) − Sn (X, Q) is, with P-probability one, bounded below; typically it will tend to infinity. In this sense, forecasts constructed from the true distribution will be optimal. PREQUENTIAL PRINCIPLES The preceding validity criteria have the property that they can be calculated knowing only the realized sequences x and P; in particular, no other features of any joint distribution, or PFS, P underlying the actual forecasts issued are required. This is clearly desirable in applications such as weather forecasting, where it would be unreasonable to expect that such a P has even been formulated. Any criterion with this property will be said to satisfy the weak prequential principle (WPP). The justifications we have given for using such measures, however, have rested on their probabilistic properties under an underlying distribution P (i.e., when the same P used for constructing forecasts is assumed to have generated the data). Nevertheless, these properties, in fact, did not depend on the detailed specification of P. Such probabilistic properties may be said to satisfy the strong prequential principle (SPP). Although indeed strong, it turns out that this is satisfiable for a wide range of interesting problems. Suppose, for example, that, for binary (Xi ), data sequence x, and forecast sequence given by p = (pi ), where pi = Pi (Xi = 1), we wish to

PREQUENTIAL ANALYSIS

assess whether the observed value
of the simple calibration measure, n−1 ni=1 (xi − pi ), is ‘‘significant.’’ A natural test statistic, satisfying WPP, is 1/2   n n  (xi − pi ) pi (1 − pi ) . yn := i=1

i=1

This is the observed value of 1/2   n n  (Xi − pi ) pi (1 − pi ) . Yn := i=1

i=1

If we consider the specific underlying P for which the (Xi ) are independent, with P(Xi = 1) = pi , then Yn has mean 0 and variance 1, and (so long as its denominator tends to ∞) is asymptotically standard normal. For a different underlying P, however, we might have dependent (Xi ); then pi will not be a constant but random, being a function of Xi−1 . Nevertheless, it turns out that the asymptotic standard normal distribution for Yn will still hold, essentially under any such P [24]. So a significance test that refers the observed yn to tables of the standard normal distribution satisfies, asymptotically, (WPP and) SPP. Similarly, many other tests with this property can be constructed. Some understanding of such phenomena can be grounded in the theory of global probability, introduced by Vovk [28], which is essentially determined by realized forecast sequences alone, and is consistent with ordinary Kolmogorov probability under any underlying distribution.

6393

on Xi , and define Pi+1 = Pθ˜ ,i+1 . This is the i plug-in SFS determined by the estimator sequence (θ˜i ). Alternatively, we might assign to θ a prior distribution, say with density  π0 (·), construct the mixture P = Pθ π0 (θ )dθ , and use this to generate forecasts. This  gives a Bayes SFS. It has Pi+1 = Pθ ,i πi (θ )dθ , where πi (·) is the posterior density of θ given Xi . Example 1. Suppose that, under Pθ , the (Xi ) are independent exponential E(θ ) variables, with density pθ (x) = θ e−θ x (x > 0, θ > 0). The maximum likelihood
estimator based on xi is θˆi = i/ti , with ti = ij=1 xj , yielding the MLE plug-in SFS under which Xi+1 ∼ Pi+1 = E(i/ti ). If we give θ a gamma (a, b) prior distribution, (a + i, b + ti ) is the posterior given xi , and the Bayes SFS forecast distribution Pi+1 for Xi+1 is [(b + ti )/(a + i)] × F2,2(a+i) , F·,· denoting the F-distribution. In either case, the underlying P incorporates dependence between the Xi , since Pi+1 depends on Xi
through Ti := ij=1 Xj . As i, ti → ∞, the two SFSs produce essentially identical forecasts. An alternative ‘‘fiducial’’ SFS may be constructed by noting that, for any θ , Xi+1 /Ti ∼ (1/i)F2,2i , suggesting Pi+1 = (ti /i)F2,2i , a limiting form of the above Bayes SFS as a, b → 0. (Note that both this and the MLE plug-in SFS require ad hoc modification for i = 0.) As the construction of an SFS is essentially arbitrary, we need to identify those that are ‘‘good,’’ in the sense of having desirable behavior under any Pθ ∈ P. PREQUENTIAL CONSISTENCY

STATISTICAL FORECASTING SYSTEMS Now suppose we have a parametrized family of forecast distribution sequences Pθ = (Pθ ,i , i = 1, 2, . . .), Pθ ,i+1 being the probability forecast made for Xi+1 , in the light of past data xi , under parameter value θ . These might be constructed from an underlying statistical model P = (Pθ ). The statistician is allowed to make use of this model, or its forecasts, in forming her own forecast Pi+1 . A system for so doing, for any data string xi , is termed a statistical forecasting system (SFS) for P. A simple class of SFSs may be constructed as follows. Let θ˜i be an estimator of θ based

In Example 1, under Pθ , Ti → θ −1 almost surely. It follows that any of the forecasts Pi+1 constructed above will be asymptotically indistinguishable from the true forecast distribution E(θ ) under Pθ , no matter what θ is. This desirable property is termed prequential consistency. Typically it will hold even in cases where consistent estimation of θ , in the usual parametric sense, is impossible: see ref. [5, Example 3]. PREQUENTIAL EFFICIENCY Let P be a Bayes SFS for P, with everywhere positive prior density π0 (·), and let Q

6394

PREQUENTIAL ANALYSIS

be an arbitrary PFS. The likelihood ratio in favor of Q as against P remains bounded (and will typically tend to 0) as the number of observations tends to infinity, with probability 1 under Pθ ∈ P, for almost every θ . In this sense, if P is a valid model, a Bayes SFS P cannot be beaten. This property may be termed prequential efficiency. It will also typically (but not universally) hold for the MLE plug-in SFS. Any prequentially efficient SFS P provides, in an optimal way, a single joint distribution which, for predictive purposes, can replace the full family P. MODEL TESTING Suppose we wish to test the validity of the family P of exponential distributions in Example 1. As a simple and concrete example, we might construct, for each i, the discrepancy Xi − µi , where µi is the expectation of Xi under the forecast distribution Pi made by some prequentially efficient SFS; and then assess overall
departure,
up to time n, by the statistic ni=1 (Xi − µi )/( ni=1 σi2 )1/2 , where σi2 is the variance of Xi under Pi . Using, for simplicity, the MLE plug-in SFS (and omitting the problematic but ignorable term for P1 ) yields
n−1
1/2 . It can be (X − T /i)]/[ [ n−1 i+1 i i=1 i=1 (Ti /i)] shown that, for any θ , this will be asymptotically standard normal when the (Xi ) are independent E(θ ) variables. We thus obtain a simple test of the model. In more general models, such test statistics, and more complex variations on them, will again be asymptotically standard normal whenever prequentially efficient forecasts are utilized, under some regularity conditions [25]. Cowell et al. [3] have applied such tests to assess the performance of probabilistic expert systems∗ . MODEL CHOICE [11, § 6.4] set of foreIf (Pk ) is a finite or countable
P casting systems, and P = α k k , with αk >
0, αk = 1, is a Bayes SFS for (Pk ), then P is prequentially efficient and consistent for (Pk ). Given data xn , P’s forecast for Xn+1 is
αkn Pk,n+1 , where αkn ∝ αk pk (xn ) is the posterior probability of Pk for priors (αk ). Typically, the (Pk ) will, as distributions over sequence

space, be mutually singular. In this case αkn → 1 as n → ∞ with Pk -probability 1, so P will ‘‘home in’’ on the true distribution. Then ˆ where Pˆ n+1 = Pk ,n+1 if kn the plug-in SFS P, n maximizes αkn over k, will likewise be consistent. Now consider a collection (Pk ) of parametric statistical models, where Pk = (Pk,θk ) has finite-dimensional parameter θk with prior density πk (·) > 0. Let Pk be
a prequentially efficient SFS for Pk and P = αk Pk , as above. Under mutual singularity of the (Pk ), which will typically hold for distinct models, we then have αkn → 1 with Pk,θk -probability 1 for almost all Pk,θk ∈ Pk . Hence choosing k to maximize the ‘‘adjusted prequential likelihood’’ (or posterior probability) αk pk (xn ) (where we may omit αk if the number of models is finite) will yield a consistent strategy for model selection. Even when the various derived (Pk ) are not mutually singular, the
plug-in SFS, or the full mixture SFS αk Pk , will give asymptotically correct forecasts, no matter which Pk,θk holds. In regular models, the prequential log likelihood log pk (xn ) can be approximated by the Jeffreys–Schwarz criterion [26] log pk,θˆk (xn ) − 12 dk log n, where θˆk is the maximum likelihood estimate of θk in Pk based on xn , and dk is the dimensionality of θk ; more refined approximations are given by Phillips and Ploberger [21]. Note that, in the direct prequential approach, any required ‘‘penalization’’ of the likelihood is effected automatically. ROBUSTNESS [9, § 8] Suppose that, although we are using a model P = (Pθ ) to aid prediction, the actual unknown joint distribution generating X is / P. We can still find a SFS which will P∗ ∈ perform effectively as well as will any Pθ ∈ P, with probability 1 under P∗ , for any P∗ . However, the construction of this will depend on the specific decision problem at hand. Suppose that, just before time i + 1, we have to choose an act a, and will suffer a loss depending on a and the value xi+1 of Xi+1 . Without loss of generality we can assume that the act a involves specifying a probability distribution Q for Xi+1 , and that the loss, if in fact Xi+1 = x, is given by a proper scoring rule

PREQUENTIAL ANALYSIS

S ≡ S(x, P) [8]; that is, when X ∼ P, S(P, Q) := E{S(X, Q)} is minimized, in Q, by quoting the true distribution, Q = P. The excess loss in quoting another Q rather than Q = P is then D(P, Q) := S(P, Q) − S(P, P), and this can be used as a measure of discrepancy between distributions, appropriate for the decision problem at hand. For the logarithmic scoring rule S(x, P) ≡ − log p(x), for example, this yields the Kullback–Liebler∗ discrepancy between P and Q. Alternatively, a quadratic loss function yields S(x, P) = [x − EP (X)]2 and the discrepancy D(P, Q) = [EP (X) − EQ (X)]2 . For any infinite data sequence x, we can consider, for each θ , the
limiting average discrepancy, limn→∞ n−1 ni=1 D(P∗i , Pθ ,i ), between the ‘‘true’’ forecasts, based on P∗ , and those based on Pθ ∈ P. A value θ ∗ minimizing this quantity may be regarded as giving the closest approximation to P∗ obtainable from within P (note that θ ∗ may depend on the data x). Since P∗ is unknown, so is θ ∗ . Howthe minimizer of the actual ever, consider θn† ,
cumulative score ni=1 S(xi , Pθ ,i ) for the initial data sequence xn . Then θn† is calculable from the data, and moreover, under weak conditions we can show that we have θn† → θ ∗ with P∗ -probability 1. That is, choosing θ = θn† by minimizing the actual total score will yield, asymptotically, the closest approximation to the true, unknown P∗ ; and the plug-in SFS formed from the estimator sequence (θn† ) will deliver, asymptotically, the same forecasts as this closest approximation. STRATEGIES In attempting to describe a data sequence x, we might undertake various more or less complex modeling exercises. Almost any standard statistical technique, for example factor analysis∗ , nonlinear time-series∗ analysis, or nonparametric density estimation∗ , may be applied to the partial data xn and used to generate a suitable probability forecast Pn+1 for Xn+1 . Thus any data analysis strategy may be converted into a PFS, and its predictive success assessed accordingly. We can feed this assessment back into the strategy in various ways, to produce a more complex strategy. This in its turn can be converted into a PFS, and its performance assessed.

6395

Suppose, for example, we have fitted a Gaussian AR(1) model, leading to an SFS P. At time n we could examine the CPIT values ui = Fi (xi ), i = 1, . . . , n, with Fi the distribution function of Xi under Pi , for uniformity and independence. Suppose we judge that independence and identical distribution are not at issue, but that the (ui ) do not appear uniformly distributed. Our analysis leading to P thus appears, empirically, to have been unsuccessful. We might however try to estimate the actual distribution of the (ui ), say by nonparametric density estimation, obtaining the estimated distribution function Gn (u). The past data are thus reasonably well described by taking Fi (Xi ) ∼ Gn , independently. If we now apply this ‘‘recalibration’’ correction to the next observation Xn+1 , we might take Fn+1 (Xn+1 ) ∼ Gn (rather than U[0, 1], as under Pn+1 ); this assertion can be transformed into a new predictive distribution, Qn+1 say, for Xn+1 . By continual reassessment of and correction for past performance, we can thus produce a new PFS Q, which we hope will describe the data better. However, we cannot be certain that it will do so, since the correction at any time is based on the past, but applied to the future, which might not continue the same pattern. (Dawid [7] considers some philosophical implications of this lack of guarantee.) In a variety of economic applications, Kling and Bessler [16] found that recalibration generally led to improved performance; but Bessler and Kling [1] gave examples of performance degrading with recalibration. As another application, we might have a collection of candidate models or strategies, as in the section ‘‘Model Choice,’’ and use some strategy based on their forecasts and the outcomes to data to select from or combine these, so as to produce a forecast for the next observation. (Bayes or maximum adjusted likelihood strategies might seem attractive, although their performance need not necessarily be good when the data cannot be regarded as arising from any of the candidate models.) Phillips [20] has applied this approach to 14 historical time series of the U.S. economy, the candidate models being autoregressions of order up to 6, with or without additional linear trend. For 10 of the series, an approximate maximum

6396

PREQUENTIAL ANALYSIS

prequential likelihood strategy had higher prequential likelihood than the generally favored AR(3)+linear trend model, often substantially. Hart [14] uses what he calls ‘‘time-series cross-validation,’’ essentially a prequential least-squares criterion, to choose the bandwith∗ for nonparametric kernel estimation of trend, in the presence of autocorrelated errors, so as to optimize forecast performance. The use of prequential assessment to choose, at each step, among models or strategies is close in spirit to cross-validation [27] or stacked generalization [29], but with the important difference that its output is itself simply another forecasting strategy. This, in turn, can be assessed prequentially and compared with others; and so on to ever deeper levels. At any level, prequential assessment supplies a valid measure of performance, unaffected by overfitting or data dredging. For further discussion, examples, and applications, see ref. [10]. CONNECTIONS WITH OTHER THEORIES Stochastic Complexity The total negative logarithmic prequential likelihood—log p(xn ) of a PFS P, for any data xn , can be regarded as defining the length function of a prefix code for encoding xn ; moreover, this will be, essentially, the shortest possible code when xn can be regarded as generated by the distribution P [11]. When P is prequentially efficient for a model P, this code length defines the stochastic complexity∗ (ref. [23]: see also ref. [22]) of xn relative to P. The theory of stochastic complexity may thus be treated as a special case of prequential analysis. Algorithmic Complexity Kolmogorov and Uspenskii [17] review this approach to probability theory, which attempts to explicate the idea that a given data string x can be considered ‘‘random’’ with respect to some joint distribution P. Equivalently, we might reexpress this as requiring that P be a ‘‘valid’’ PFS for x, as considered above. This reformulation then renders the insights and techniques of each approach, prequential analysis and algorithmic complexity, available to the other.

REFERENCES 1. Bessler, D. A. and Kling, J. L. (1990). Prequential analysis of cattle prices. Appl. Statist., 39, 95–106. 2. Brier, G. W. (1950). Verification of forecasts expressed in terms of probability. Mon. Weather Rev., 78, 1–3. 3. Cowell, R. G., Dawid, A. P., and Speigelhalter, D. J. (1993). Sequential model criticism in probabilistic expert systems. IEEE Trans. Pattern Recognition and Machine Intell., 15, 209–219. 4. Dawid, A. P. (1982). The well-calibrated Bayesian (with discussion). J. Amer. Statist. Ass., 77, 605–613. 5. Dawid, A. P. (1984). Statistical theory. The prequential approach (with discussion). J. Roy. Statist. Soc. A, 147, 278–292. 6. Dawid, A. P. (1985). Calibration-based empirical probability (with discussion). Ann. Statist., 13, 1251–1285. 7. Dawid, A. P. (1985). The impossibility of inductive inference. (Comments on ‘‘Selfcalibrating priors do not exist,’’ by David Oakes.) J. Amer. Statist. Ass., 80, 340–341. 8. Dawid, A. P. (1986). Probability forecasting. In Encyclopedia of Statistical Sciences, Vol. 7, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley-Interscience, pp. 210–218. 9. Dawid, A. P. (1991). Fisherian inference in likelihood and prequential frames of reference (with discussion). J. R. Statist. Soc. B, 53, 79–109. 10. Dawid, A. P. (1992). Prequential data analysis. In Current Issues in Statistical Inference: Essays in Honor of D. Basu, M. Ghosh and P. K. Pathak, eds. IMS Lecture Notes–Monograph Ser. 17, pp. 113–126. 11. Dawid, A. P. (1992). Prequential analysis, stochastic complexity and Bayesian inference (with discussion). In Bayesian Statistics 4, J. M. Bernardo, J. Berger, A. P. Dawid, and A. F. M. Smith, eds. Oxford University Press, pp. 109–125. 12. Geisser, S. (1975). The predictive sample reuse method with applications. J. Amer. Statist. Ass., 70, 320–328. 13. Geisser, S. (1980). Predictive analysis. In Encyclopedia of Statistical Sciences, Vol. 7, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley-Interscience, pp. 158–170. 14. Hart, J. D. (1994). Automated kernel smoothing of dependent data using time series crossvalidation. J. R. Statist. Soc. B, 56, 529–542.

PREVISION 15. Kallianpur, G. (1986). Prediction and filtering, linear. In Encyclopedia of Statistical Sciences, Vol. 7, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley-Interscience, pp. 139– 148. 16. Kling, J. L. and Bessler, D. A. (1989). Calibration-based predictive distributions: an application of prequential analysis to interest rates, money, prices, and output. J. Business, 62, 477–499. 17. Kolmogorov, A. N. and Uspenskii, V. A. (1987). Algorithms and randomness. Theory Probab. Appl., 32, 389–412. 18. Ledolter, J. (1986). Prediction and forecasting. In Encyclopedia of Statistical Sciences, Vol. 7, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley-Interscience, pp. 148–158. 19. Murphy, A. H. and Epstein, E. S. (1967). Verification of probabilistic predictions: a brief review. J. Appl. Meteorol., 6, 748–755. 20. Phillips, P. C. B. (1995). Bayesian model selection and prediction with empirical applications. J. Econometrics, 69, 289–331. 21. Phillips, P. C. B. and Ploberger, W. (1995). An asymptotic theory of Bayesian inference for time series. Econometrica, 64, 381–412. 22. Rissanen, J. (1985). Minimum-descriptionlength principle. In Encyclopedia of Statistical Sciences, Vol. 5, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley-Interscience, pp. 513–527. 23. Rissanen, J. (1989). Stochastic Complexity in Statistical Inquiry. World Scientific Publishing, Singapore. 24. Seillier-Moiseiwitsch, F. and Dawid, A. P. (1993). On testing the validity of sequential probability forecasts. J. Amer. Statist. Ass., 88, 355–359. 25. Seillier-Moiseiwitsch, F., Sweeting, T. J., and Dawid, A. P. (1992). Prequential tests of model fit. Scand. J. Statist., 19, 45–60. 26. Smith, A. F. M. (1988). Schwarz criterion. In Encyclopedia of Statistical Sciences, Vol. 8, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley-Interscience, pp. 289–290. 27. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions (with discussion). J. R. Statist. Soc. B, 36, 111– 147. 28. Vovk, V. G. (1993). A logic of probability, with application to the foundations of Statistics (with discussion). J. R. Statist. Soc. B, 55, 317–351. 29. Wolpert, D. H. (1992). Stacked generalization. Neural Networks, 5, 241–259.

6397

See also CONSISTENCY; EFFICIENCY, SECOND-ORDER; LIKELIHOOD; PREDICTION AND FORECASTING; PREDICTIVE ANALYSIS; and PROBABILITY FORECASTING.

A. P. DAWID

PREVAILING ITEM An obsolete term for mode (see MEAN, MEDIAN, AND MODE).

PREVISION Prevision is a concept introduced by B. de Finetti to establish the notion of expectation as the fundamental element of the subjectivist theory of probability. Your prevision, P(X), for the random quantity X may be thought of, intuitively, as the ‘‘fair price’’ for the random gain X [i.e., so that you are indifferent between the random gain X and the certain gain P(X)]. (Prevision, as a subjectivist concept, always expresses the judgment of a specified individual, here termed ‘‘you.’’) To make this precise, P(X) must be defined in terms of an operational criterion. One such criterion is as follows. For a general random quantity X, your prevision for X, P(X), is the value x that you would choose if you were subsequently to suffer a penalty L given by L = k(X − x)2 , where k is a constant defining the units of loss. Your previsions x1 , . . . , xm for the random quantities X1 , . . . , Xm are said to be coherent if there is no other choice x∗1 , . . . , x∗m which would lead to a reduction in your overall penalty for all possible combinations of values of X1 , . . . , Xm . (Thus, if you are not coherent, you are a ‘‘sure loser.’’) Coherence is here used in the same intuitive sense as for the axiomatic development of inference and decision-making (see COHERENCE), namely as a criterion to identify intrinsically contradictory beliefs. For any random quantities X and Y, coherence imposes the following requirements on your specifications: (1) infX
P(X)
sup X, and (2) P(X + Y) = P(X) + P(Y). These properties reflect the ‘‘fair price’’ interpretation. (Notice that prevision is finitely additive but not necessarily countably additive.) [(1) and (2) are special cases of the general result that a specification P(Yi ) = yi , i = 1, . . . , m, is

6398

PRIESTLEY’S TEST FOR HARMONIC COMPONENT(S)

coherent if and only if the point (y1 , . . . , ym ) lies in the closed convex hull of the set of possible values of the random vector (Y1 , . . . , Ym ) in m-dimensional Euclidean space.] Prevision can be considered to be expectation, but defined directly as a primitive quantity, rather than in terms of a prespecified probability distribution. For any event H, if you also use H to denote the indicator function of the event (i.e., H = 1 if H is true, H = 0 if false), then you may interpret P(H) as the probability of H. In this development there is no distinction between probability and expectation, and the notation is chosen to emphasize this. If you have specified a full probability distribution for X, your prevision for X will agree with the usual definition of expectation (at least if X is bounded; there are subtleties for unbounded X). The essential difference arises when you do not want to assess probabilities for all the possible outcomes, in which case P(X), being assigned directly, is well defined, as opposed to your expectation for X, which is defined only through the associated probability distribution for X. Thus you may consider ‘‘average’’ properties without exhaustively considering some limiting partition of possibilities, so that you are free to choose the level of detail at which you wish to describe your beliefs. Using H interchangeably as the event and the indicator function of your event, your prevision of the random quantity X, conditional on the event H, P(X|H), is defined exactly as for P(X), but with the penalty L replaced by the ‘‘called off’’ penalty HL. Coherence imposes the following requirement: (3) P(HX) = P(H)P(X|H). The essential reference on prevision is de Finetti [1,2], where all the foregoing material is developed. Much of the importance of prevision lies in the depth and thoroughness of the treatment which de Finetti provides in these two volumes. Some statistical implications of this formulation are developed in Goldstein [3] as follows. In the usual Bayesian formulation, your beliefs are collected into a prior probability distribution which is modified by Bayes’ theorem in the light of new information to yield a posterior∗ probability distribution. When working with prevision rather than probability, it is shown

that your beliefs should be collected into a prior inner product space, which is modified by a linear transformation, in the light of new information, to yield a posterior inner product space. The relationship between the conditional probability approach and the linear transformation approach is discussed in detail in Goldstein [5]. These developments rely on the condition that if PT (X) is the actual value that you will assign for P(X) at the future time T, then your current previsions should satisfy the relation: (4) P(PT (X)) = P(X). [This result would not, in general, be true if PT (X) were replaced by conditional prevision.] A coherence argument for (4) is given in Goldstein [4]. REFERENCES 1. de Finetti, B. (1974). Theory of Probability, Vol. 1. Wiley, London. 2. de Finetti, B. (1975). Theory of Probability, Vol. 2. Wiley, London. 3. Goldstein, M. (1981). J. R. Statist. Soc. B, 43, 105–130. 4. Goldstein, M. (1983). J. Amer. Statist. Ass., 78, 817–819. 5. Goldstein, M. (1984). In Bayesian Econometrics and Statistics, Vol. 3 (in honour of B. de Finetti), Prem. K. Goel and A. Zellner, eds. North-Holland, Amsterdam (in press) See also COHERENCE; CONDITIONAL PROBABILITY AND EXPECTATION; EXPECTED VALUE; and SUBJECTIVE PROBABILITIES.

MICHAEL GOLDSTEIN

PRIESTLEY’S TEST FOR HARMONIC COMPONENT(S). See TIME SERIES, PRIESTLEY’S HARMONIC COMPONENT TEST FOR

PRIMARY SAMPLING UNIT This is a term used in multistage sampling. At the first stage of sampling it is necessary to divide the entire population or universe into a convenient number of primary sampling units (PSUs). In large-scale area samples∗ it is usual to form a few thousand PSUs, from which a few hundred would be selected

PRIMARY SAMPLING UNIT

to constitute the first-stage sample. Thus in area samples of the United States it is not uncommon for the PSUs to be counties. The second stage of selection in multistage sampling takes place only within those PSUs which form the first-stage sample. Secondary sampling units, for instance census tracts, need to be listed only within those selected PSUs. If the number of selected PSUs is held constant, the consequences of forming large PSUs are that many second-stage units need to be listed and that those selected are scattered over relatively large areas. Conversely, if the PSUs are made small, the task of selecting at the second stage involves forming relatively short lists of second-stage units, and those selected are geographically concentrated. It is usual, particularly in area sampling, for primary sampling units to have unequal probabilities of selection or of inclusion in a sample. These selection or inclusion probabilities are usually proportional to some measure of size, such as the number of persons or dwellings recorded in the PSU at the latest available census. There are mathematical and operational advantages in selecting PSUs ‘‘with replacement.’’ To select with unequal probabilities with replacement, the PSUs within each (usually geographical) stratum are assigned selection probabilities which sum to unity. A series of n sample draws is then made using these probabilities. It is possible for a PSU to be selected more than once, hence the expression ‘‘with replacement.’’ (Because the same probabilities are used at each draw, the distribution of the numbers of times each PSU is selected in a sample is multinomial∗ . Since ‘‘sampling with replacement’’ does not necessarily imply that the same probabilities of selection are used at each draw, this type of sampling is occasionally referred to as ‘‘multinomial.’’) If sampling is to be without replacement at the first stage, the probabilities of inclusion in the sample are usually chosen to sum to n, an integer, which is the required sample number or sample size. A procedure then has to be specified for selecting the sample in accordance with these inclusion probabilities. Either this procedure itself will be complicated, or the variance estimation will be

6399

complicated, inefficient or biased, or the sample must be limited to two units. Sampling ‘‘with replacement’’ (multinomial sampling) is less efficient but can be distinctly more convenient. In particular, if the PSUs are selected ‘‘with replacement,’’ then regardless of the selection methods used at subsequent stages, it is possible to obtain an unbiased estimator of variance (of the unbiased estimator of mean or total) by comparing the estimates based on the individual PSUs. Since these were selected independently, the estimates based on them are also independent. This applies only to estimation of the variance over all stages of sampling combined. To obtain simple unbiased estimators of the variances at the individual stages, sampling must be ‘‘with replacement’’ at the subsequent stages as well as the first (see STRATIFIED MULTISTAGE SAMPLING). In the event that a PSU is selected twice or three times, it is usual to double or triple the number of second-stage sample units to be selected within it. This may in some cases involve multiple selection of second- or even lower-stage units. If final-stage units are selected more than once, unequal weighting must be used in the sample estimator. Since area sampling effects a great simplification in the sampling procedure and the resulting clustered samples∗ are much cheaper to enumerate than unclustered samples, this technique is widely used in largescale household surveys such as the Current Population Survey∗ and those of the major private polling organizations in the United States and other developed countries. Although the term ‘‘primary sampling unit’’ is seldom used outside the context of area sampling, it is not confined to it. Thus if a two-stage sample is used in which the first stage consists of a sample of boxes and the second of cards within those boxes, the boxes would be the PSU’s. BIBLIOGRAPHY Barnett, V. (1974). Elements of Sampling Theory. Hodder & Stoughton, London, Chap. 5. (Concise treatment at an intermediate mathematical level.)

6400

PRIMITIVE ROOT

Sudman, S. (1976). Applied Sampling. Academic Press, New York. Chap. 7. (For survey users with limited statistical backgrounds.)

properties of their proposals. One of these proposals consists in letting T=

See also AREA SAMPLING; STRATIFIED MULTISTAGE SAMPLING; and SURVEY SAMPLING.

K. R. W. BREWER

PRIMITIVE ROOT In any finite field (Galois field∗ ), there is at least one element in the field, different powers of which give all the nonzero elements of the field. Such an element is called a primitive root of the field. For example, if the number of elements in the field is p = 7, then 3 is a primitive root since (modulo p) 31 = 3,

32 = 2, 35 = 5,

33 = 6, and

34 = 4,

36 = 1.

(Actually, X p−1 = 1 when X is any element of the field.) See also CONFOUNDING; FRACTIONAL FACTORIAL DESIGNS; and GALOIS FIELDS.

PRINCIPAL COMPONENT ANALYSIS, GENERALIZED Let V be the variance matrix of the pdimensional data X 1 , . . . , X n and T be a positive definite p × p matrix. While classical principal component analysis∗ (PCA) rests on the spectral decomposition of V, generalized PCA rests on the eigen analysis of VT −1 and the T −1 -orthogonal projection (instead of the canonical projection) of the centered data on the subspace Eq of Rp spanned by the q (q < p) eigenvectors associated with the q largest eigenvalues. Various choices of T give rise to interesting projections of the data. In order to display clusters, Art et al. [1] consider a withincluster dispersion matrix T estimated without knowledge of the clusters. In a similar approach displaying outliers∗ [4], the matrix T is a robust estimate of the total variance. Caussinus and Ruiz-Gazen [2,3] propose to simplify these methods and give some


n−1
n i=1

exp[−(β/2)||X i −X j ||2 −1 ](X i −X j )(X i −X j ) j=i+1 V ,
n−1
n exp[−(β/2)||X i −X j ||2 −1 ] j=i+1 i=1 V

where the squared norm ||x||2V −1 = x V −1 x for any column vector x of Rp . The use of an exponential weighting of the pairwise differences in the definition of T permits the authors to develop theoretical results. These results are obtained for  large samples from a mixture distribution∗ Np (x, W)dP(x), where P, the nonnormal distribution of interest, is concentrated on an unknown k-dimensional subspace Fk of Rp (k < p), and Np (x, W), the p-dimensional normal density with mean x and covariance W, is the noise. For instance, Ek tends to Fk when n goes to infinity if P is a discrete distribution defining clusters, W is small enough with respect to the variance of P, and β is suitably chosen (in practice, 1
β
2). Since k is unknown, means are also provided to assess an appropriate dimension for a useful projection. All the displays are affine invariant. These techniques can be considered as alternatives to projection pursuit∗ . REFERENCES 1. Art, D., Gnanadesikan, R., and Kettenring, J. R. (1982). Data based metrics for cluster analysis. Utilitas Math., 21, 75–99. 2. Caussinus, H. and Ruiz-Gazen, A. (1994). Projection pursuit and generalized principal component analyses. In New Directions in Statistical Data Analysis and Robustness, S. Morgen¨ thaler et al., eds. Birkhauser, Basel, pp. 35–46. 3. Caussinus, H. and Ruiz-Gazen, A. (1995). Metrics for finding typical structures by means of principal component analysis. In Data Science and its Applications, Y. Escoufier et al., eds. Academic Press, pp. 177–192. 4. Yenyukov, I. S. (1988). Detecting structures by means of projection pursuit. COMPSTAT 88, Physica-Verlag, Heidelberg, pp. 47–58. See also COMPONENT ANALYSIS; PRINCIPAL COMPONENTS; and PROJECTION PURSUIT.

H. CAUSSINUS A. RUIZ-GAZEN

PRINCIPAL COMPONENTS

PRINCIPAL COMPONENTS This entry is an update to COMPONENT ANALYSIS. It focuses on developments during the past decade, highlighting mostly modelbased generalizations of principal component analysis. The topics discussed are: (1) generalizations to several groups; (2) nonlinear generalizations; (3) principal components in classification and regression; (4) patterned covariance matrices; and (5) varia, including testing problems and robust estimation. Other recent developments are treated in three books [9,19,23] that deal exclusively with principal component analysis. GENERALIZATIONS TO SEVERAL GROUPS When multivariate data fall into natural groups, such as males and females of the same species, the investigator may be interested in comparing principal components across groups. A descriptive method achieving this goal consists of computing angles between the linear subspaces spanned by the first q (out of p) principal components in each group [27]. In a model setup, assume that the principal component directions (eigenvectors of the covariance matrices) are identical across groups, while the variances may differ. Specifically, let
i denote the covariance matrix of a p-variate random vector X in the ith group, and let
i = Bi i Bi be its spectral decomposition, where the columns of the orthogonal matrix Bi are the eigenvectors of
i , and i is a diagonal matrix with the eigenvalues of
i as diagonal entries. The common principal component (CPC) model [9] assumes that all Bi are equal, i.e., all
i can be diagonalized simultaneously by a single orthogonal matrix B. With B = (β 1 · · · β p ) and i = diag(λi1 , . . . , λip ), the CPC model can be written as
i = Bi B =

p 

λij β j β j .

6401

estimates of the parameters, based on k sample covariance matrices Si , can be computed using a Jacobitype algorithm for simultaneous diagonalization [10]. The first principal component of logtransformed measurements is often used in morphometrics to estimate constants of allometric growth [22,24]. In the case of several related species, the investigator may want to estimate identical constants of allometric growth for all groups under consideration [1,9], without imposing the constraint of equality of all eigenvectors of the k covariance matrices. A modification of the CPC model, the partial CPC model, assumes that
i = λi1 β 1 β 1 +

p 



(i) λij β (i) j βj ,

i = 1, . . . , k.

j=2

Here, the orthogonal matrices Bi = (β 1 β (i) 2 ··· β (i) ) have one vector in common, while the p remaining ones are allowed to differ between groups. In a variation of this model, q  2 eigenvectors of each covariance matrix span the same subspace [9,31,34]. Another modification of the CPC model imposes constraints λij = ci λj on the diagonal matrices i ; this results in the model of proportional covariance matrices
i = ci
1 ,

i = 2, . . . , k,

for proportionality constants c2 , . . . , ck [8,9,21,30]. Related approaches in the psychometric literature are called three-mode∗ principal component analysis [26]. They are based on three- and higher-way data matrices (e.g., data arranged by variables, objects, and experimental conditions), and originated in the psychometric literature, but have been further developed in chemometrics∗ [17,18,42,43]. These methods rely on algebraic decompositions of higherorder data matrices and are not explicitly based on parametric models (see LINEAR ALGEBRA, COMPUTATIONAL). Solutions are usually obtained by a least squares criterion.

j=1

NONLINEAR GENERALIZATIONS The CPC model holds trivially if all covariance matrices are equal or proportional. Normal-theory maximum likelihood

A principal curve, or self-consistent curve, of a p-variate random vector X is a smooth

6402

PRINCIPAL COMPONENTS

curve in Rp such that each point x on the curve is the conditional mean of X in the subset of all points in Rp that are closer to x than to any other point on the curve [16]. For elliptical distributions, principal-component axes are self-consistent curves; thus principal curves are a genuine generalization of ordinary linear principal components. For a bivariate spherically symmetric random vector X, the circle of radius E[ X ] centered at the mean of X is also self-consistent. Self-consistent curves other than lines can typically only be found by iterative computations similar to a k-means algorithm∗ . In addition, for estimation of principal curves based on observed data one needs a scatterplot smoothing technique [2,16,39]. Principal curves can be generalized to principal surfaces, i.e., smooth manifolds∗ that generalize subspaces spanned by subsets of eigenvectors in linear principal component analysis [16]. While principal curves generalize principal components using self-consistency∗ as the defining criterion, smallest additive principal components [7] generalize the principal components associated with the smallest eigenvalues by constructing additive func
p tions i=1 i , where the i (Xi ) are nonlinear functions of the original variables, such that
var[ 
i ] is a minimum subject to the constraint var[i ] = 1. PRINCIPAL COMPONENTS IN CLASSIFICATION AND REGRESSION In discriminant analysis∗ , principal components are sometimes used to reduce the dimensionality of the data, but the success of such a procedure depends on the assumption that discarding principal components with small eigenvalues will not, at the same time, discard valuable information on differences between the groups. A different application of principal components in discriminant analysis imposes constraints like the ones discussed above in connection with the covariance matrices [3,14,30]. This leads to classification∗ rules that are quadratic but reduce the number of parameters estimated, which may improve the expected error rates. Principal components are also used frequently in cluster analysis∗ to detect

homogeneous subsets of data; in particular, a scatterplot of the first two components often reveals structure. Principal points of a p-variate random vector X are points that approximate the distribution of X optimally in the least squares sense; if the distribution of X is elliptical, then principal points lie in the subspace spanned by the first eigenvectors of the covariance matrix of X, thus justifying the use of principal components [36,37]. The principal components of regressor variables may be used in multiple regression to combat the effects of collinearity∗ ; see PRINCIPAL COMPONENTS REGRESSION ANALYSIS. By omitting components with small eigenvalues the stability of parameter estimates is improved while a bias may be introduced, similarly to ridge regression∗ . Nonlinear functions of the principal components, including their products and powers, may be used to improve prediction [29]. PATTERNED COVARIANCE MATRICES Suppose a pk-variate random vector X is partitioned into k vectors X1 , . . . , Xk of dimension p each; for instance, Xi may be a set of p variables measured in the ith stadium of growth of an insect. The pk × pk covariance matrix
of X, partitioned into submatrices
ij of dimension p × p each, may then display some distinct structure, and models similar to those of the first section can be defined [12]. For instance, one may assume that the same orthogonal p × p matrix B diagonalizes all
ij simultaneously; this leads to the CPC model for dependent random vectors [25] used in studies of static and ontogenetic allometry∗ . For example, for k = 2 the model is       
11
12 B O 11 12 B O = , O B
21
22 21 22 O B where B is an orthogonal p × p matrix, and all ij are diagonal. If this model holds, then the canonical correlations between X1 and X2 are the same as the correlations between the common principal components [12]. Assuming that B is nonsingular but not necessarily orthogonal leads to the common canonical variates model [28]. See also the references on three-way principal component analysis given at the end of the first section.

PRINCIPAL COMPONENTS

VARIA Robust estimation∗ of principal components is discussed in refs. 5, 20; for common principal components, in ref. [13]. References [6, 43] use projection pursuit∗ to define robust principal components, and to detect clusters and outliers∗ . The influence function∗ [4,20] provides a tool to assess the effect of single observations on estimates of eigenvalues and eigenvectors. See ref. [4] for a comprehensive review. Various tests for hypotheses are developed in [32,33,40,41]. The T 2 statistic for equality of the mean vectors of two p-variate populations can be decomposed into p parts corresponding to principal components [35]. This decomposition can be generalized by assuming that the two populations differ in location and in variability, but only in q < p common principal component directions [11]. The same approach allows us to test the model of allometric extension. The biometric, chemometric, and psychometric literature abounds with applications of principal component analysis and its various extensions, for example, to process monitoring in paper production [42], water chemistry [18], soil contamination [43], and regional variation in growth of insects [24]. Future directions for research should include models for high-dimensional data, in particular data measured in continuous time, and Bayesian estimation. On the latter topic there is only a very limited amount of literature available [15,38], and none related to the various generalizations presented in this update. REFERENCES 1. Airoldi, J. P. and Flury, B. (1988). An application of common principal component analysis to cranial morphometry of Microtus californicus and M. ochrogaster (Mammalia, Rodentia). J. Zool. (London), 216, 21–36. 2. Banfield, J. D. and Raftery, A. E. (1992). Ice floe identification in satellite images using mathematical morphology and clustering about principal curves. J. Amer. Statist. Ass., 87, 7–15. 3. Bensmail, H. and Celeux, G. (1996). Regularized Gaussian discriminant analysis through eigenvalue decomposition. J. Amer. Statist. Ass., 91, 1743–1748.

6403

4. Brooks, S. P. (1994). Diagnostics for principal components: influence functions as diagnostic tools. Statistician, 43, 483–494. 5. Campbell, N. A. (1980). Robust procedures in multivariate analysis I: robust covariance estimation. Appl. Statist., 29, 231–237. 6. Caussinus, H. and Ruiz-Gazen, A. (1993). Projection pursuit and generalized principal component analysis. In New Directions in Statistical Data Analysis and Robustness, S. Morgenthaler, E. Ronchetti, and W. A. Stahel, eds. ¨ Birkhauser, Basel, pp. 35–46. 7. Donnell, D. J., Buja, A., and Stuetzle, W. (1994). Analysis of additive dependencies and concurvities using smallest additive principal components. Ann. Statist., 22, 1635–1673. 8. Eriksen, P. S. (1987). Proportionality of covariance matrices. Ann. Statist., 15, 732–748. 9. Flury, B. (1988). Common Principal Components and Related Multivariate Models. Wiley, New York. (Model-based introduction to principal component analysis, with extensions to common principal components, partial common principal components, and proportional covariance matrices, including distribution theory and numerical algorithms.) 10. Flury, B. and Gautschi, W. (1986). An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. SIAM J. Sci. Statist. Comput., 7, 169–184. 11. Flury, B., Nel, D., and Pienaar, I. (1995). Simultaneous detection of shift in means and variances. J. Amer. Statist. Ass., 90, 1474–1481. 12. Flury, B. and Neuenschwander, B. (1994). Principal component models for patterned covariance matrices, with applications to canonical correlation analysis of several sets of variables. In Descriptive Multivariate Analysis, W. J. Krzanowski, ed. Oxford University Press, pp. 90–112. 13. Flury, B. and Novi Inverardi, P. L. (1992). Robust estimation of common principal components. Quad. Statist. e Mat. Appl. Sci. Econ.–Soc., 14, 49–80. 14. Flury, B., Schmid, M., and Narayanan, A. (1994). Error rates in quadratic discrimination with constraints on the covariance matrices. J. Classification, 11, 101–120. 15. Geisser, S. (1965). Bayesian estimation in multivariate analysis. Ann. Math. Statist., 36, 150–159. 16. Hastie, T. J. and Stuetzle, W. (1989). Principal curves. J. Amer. Statist. Ass., 84, 502–516.

6404

PRINCIPAL COMPONENTS

17. Henrion, R. (1993). Body diagonalization of core matrices in three-way principal component analysis: theoretical bounds and simulation. J. Chemometrics, 7, 477–494. 18. Henrion, R. (1994). N-way principal component analysis: theory, algorithms, and applications. Chemometrics and Intell. Lab. Systems, 25, 1–23. 19. Jackson, J. E. (1991). A User’s Guide to Principal Components. Wiley, New York. (An easyto-read introduction to principal component analysis and its applications, including many real and synthetic data sets, and connections to other multivariate methods.) 20. Jaupi, L. and Saporta, G. (1993). Using the influence function in robust principal component analysis. In New Directions in Statistical Data Analysis and Robustness, S. Morgenthaler, E. Ronchetti, and W. A. Stahel, eds. ¨ Birkhauser, Basel, pp. 147–156. 21. Jensen, S. T. and Johansen, S. (1987). Estimation of proportional covariances. Statist. Probab. Lett., 6, 83–85. 22. Jolicoeur, P. C. (1963). The multivariate generalization of the allometry equation. Biometrics, 19, 497–499. 23. Jolliffe, I. T. (1986). Principal Component Analysis. Springer, New York. (A very readable introduction to principal-component analysis with many illustrative examples, including applications to factor analysis and regression.) 24. Klingenberg, C. P. (1996). Multivariate allometry. In Advances in Morphometrics, L. F. Marcus et al., eds. Plenum, New York, pp. 23–49. 25. Klingenberg, C. P., Neuenschwander, B., and Flury, B. (1996). Ontogeny and individual variation: analysis of patterned covariance matrices with common principal components. Systematic Biol., 45, 135–150. 26. Kroonenberg, P. M. (1983). Three-Mode Principal Component Analysis. DSWO Press, Leiden, The Netherlands. (A comprehensive review of the psychometric literature on extensions of principal component analysis to threeway data up to 1983.) 27. Krzanowski, W. J. (1979). Between-groups comparison of principal components. J. Amer. Statist. Ass., 74, 703–707. Correction, 76, 1022. 28. Neuenschwander, B. and Flury, B. (1995). Common canonical variates. Biometrika, 82, 553–560. 29. Oman, S. D., Næs, T., and Zube, A. (1993). Detecting and adjusting for non-linearities

in calibration of near-infrared data using principal components. J. Chemometrics, 7, 195–212. 30. Owen, A. (1984). A neighbourhood-based classifier for LANDSAT data. Can. J. Statist., 12, 191–200. 31. Schott, J. R. (1988). Common principal component subspaces in two groups. Biometrika, 75, 229–236. 32. Schott, J. R. (1988). Testing the equality of the smallest latent roots of a correlation matrix. Biometrika, 75, 794–796. 33. Schott, J. R. (1991). A test for a specific principal component of a correlation matrix. J. Amer. Statist. Ass., 86, 747–751. 34. Schott, J. R. (1991). Some tests for common principal component subspaces in several groups. Biometrika, 78, 771–777. 35. Takemura, A. (1985). A principal decomposition of Hotelling’s T 2 statistic. In Multivariate Analysis—VI, P. R. Krishnaiah, ed. Elsevier, Amsterdam, pp. 583–597. 36. Tarpey, T. (1995). Principal points and selfconsistent points of symmetric multivariate distributions. J. Multivariate Anal., 53, 39–51. 37. Tarpey, T., Li, L., and Flury, B. (1995). Principal points and self-consistent points of elliptical distributions. Ann. Statist., 23, 103–112. 38. Tiao, G. C. and Fienberg, S. (1969). Bayesian estimation of latent roots and vectors with special reference to the bivariate normal distribution. Biometrika, 56, 97–108. 39. Tibshirani, R. (1992). Principal curves revisited. Statist. and Comput., 2, 183–190. 40. Tyler, D. E. (1981). Asymptotic inference for eigenvectors. Ann. Statist., 9, 725–736. 41. Tyler, D. E. (1983). A class of asymptotic tests for principal component vectors. Ann. Statist., 11, 1243–1250. 42. Wold, S. (1994). Exponentially weighted moving principal components analysis and projections to latent structures. Chemometrics and Intell. Lab. Systems, 23, 149–161. 43. Xie, Y., Wang, J., Liang, Y., Sun, L., Song, X., and Yu, R. (1993). Robust principal component analysis by projection pursuit. J. Chemometrics, 7, 527–541. See also ALLOMETRY; CLASSIFICATION —I; COMPONENT ANALYSIS; DISCRIMINANT ANALYSIS; PRINCIPAL COMPONENTS REGRESSION ANALYSIS; and THREE-MODE ANALYSIS.

BERNARD D. FLURY

PRINCIPAL COMPONENTS REGRESSION ANALYSIS

r = 1, 2, . . . , p, where a r = (a1r , a2r , . . . , apr ) with constraints a r as = δrs , where δrs = 1 if r = s and is zero otherwise.

PRINCIPAL COMPONENTS REGRESSION ANALYSIS Principal components (PC) regression is often used to solve the problem of multicollinearity∗ [3–5, 8], especially when deleting independent variables to break up the multicollinearity is not a feasible option [3]. One also uses PC regression when there are more independent variables than observations [5]. Basically, the way that PC regression works is that one first finds the set of orthonormal eigenvectors of the correlation matrix of the independent variables. The matrix of PCs is then the product of the eigenmatrix with the matrix of independent variables. The matrix of PCs is such that the first PC will have the maximum possible variance, the second the maximum possible variance among those uncorrelated with the first, and so on. The set of PCs then becomes the set of new regressor variables. Upon completion of this regression, one transforms back to the original coordinate system. Let the ordinary least squares∗ (OLS) model be defined (the vector and matrix dimensions are given in parentheses) as Y =

(n×1)

+

γ0 1 (1×1)(n×1)

Xγ (n×p)(p×1)

+  . (n×1)

  i

(Xij − X j )2

λ1  λ2  · · ·  λp  0.

(5)

Then, if A = (a1 , . . . , ap ), A x xA = Z Z = 

and

x x = AA , (6)

since A is orthogonal, where Z = xA is the PC matrix and Z Z is the new design matrix. Then r = 1, . . . , p, (7) var(zr ) = z r zr = λr ,     tr(x x) = p = z r zr = λr implies that λi /p represents the proportionate contribution of the ith PC to the total variation of the x’s. McCallum [8] has noted that y = xβ +  = ZA β + ,

(8)

which implies that h = A β

1/2

√ = (Xij − X j )/(sj n − 1)  

1/2 (Yi − Y)2 yi = (Yi − Y)

 = diag[λ1 , λ2 , . . . , λp ] such that

(1)

To compute the correlation matrix∗ , it is useful to standardize the variables: xij = (Xij − X j )

6405

(2)

or

β = hA,

(9)

where h is the coefficient vector of the regression on the PCs. Hence the transformed coefficients for model (1) are obtained from (9) and (3) as γi =

p 

√ aij hj /(si n − 1),

i = 1, . . . , p, (10)

j=1

i

√ = (Yi − Y)/(sy n − 1).

where i j

Then (1) becomes y = xβ + ,

√ βj = γj sj n − 1.

γi (3)

The correlation matrix x x has eigenvalues and eigenvectors defined by |x x − λr I| = 0

and

(x x − λr I)ar = 0, (4)

aij hj

si

corresponds to the independent variable number, corresponds to the PC number is the transformed coefficient, model (1) are elements of the eigenmatrix are the coefficients obtained from the regression on the PCs is the standard deviation of independent variable i

6406

PRINCIPAL COMPONENTS REGRESSION ANALYSIS

(In ridge regression∗ , h is referred to as the vector of canonical coefficients.) If a subset of the PCs is selected, some of the hj are zero. The constant term in the transformed equation can be obtained from γ0 = Y −

p 

γi X i .

(11)

i=1

McCallum [8] has also observed that var b = σ 2 (x x)−1 = σ 2 (A−1 A ),

(12)

where b is the least squares estimator of β. Therefore, the standard errors of the estimators of the coefficients for model (1) (but not the constant term) are, from (12) and (3),  SE(γi ) =

σ

p  a2ij

1/2

  √ si n − 1 j=1 λj

,

i = 1, . . . , p.

(13)

When a subset of the PCs is selected, the appropriate aij and λj are deleted from (13). One does not need to be concerned with the interpretation of the PCs or the regression on the PCs, because this is just an intermediate step and one can transform back to the original coordinate system (as shown above). Three selection rules for picking PCs are [2,3,8]: 1. Keeping all p PCs yields zero bias [because this yields OLS, as shown in (8)]. 2. Keeping those PCs with the largest eigenvalues minimizes the variances of the estimators of the coefficients. (The proof is given in Greenberg [3] and Coxe [2, p. 226]. One simply needs to substitute A for Z and Z for W.) 3. Keeping those PCs which are highly correlated with the dependent variable tends to minimize the mean square errors of the estimators of the coefficients. (Coxe [2] has given an intuitive argument why this rule holds, but it has yet to be proven.)

The reasons given are: (a) McCallum [8] has shown that one can obtain smaller MSEs by eliminating some of the PCs. (b) Since rule 1 yields zero bias and rule 2 yields minimum coefficient variances, minimum MSE should fall somewhere in between. (c) Selecting those PCs that are strongly correlated with the dependent variable is similar to selecting independent variables by stepwise regression∗ in OLS; that is, both use F-values to determine which terms to retain. It should be noted that selection rules 2 and 3 do not necessarily yield the same PCs model, and hence the resulting coefficients in model 1 can be quite different, especially when the degree of multicollinearity is severe. There is no definitive source for PC regression analysis. In going from paper to paper, one must be alert to the different standardizations of the variables. Some √ use sj in the denominator, others use sj n − 1, and still others do not standardize at all. REFERENCES 1. Anderson, T. W. (1958). An Introduction to Multivariate Statistics. Wiley, New York, pp. 272–287. (A rigorous derivation of PC.) 2. Coxe, K. (1982). Amer. Statist. Ass., Proc. Bus. & Econ. Statist. Sec., pp. 222–227. (Selection rules for PC regression. Comparison with latent root regression. Long on math symbols, short on words. Good list of references.) 3. Greenberg, E. (1975). J. Amer. Statist. Ass., 70, 194–197. [Deals with selection rule (2). Good, fairly technical paper.] 4. Gunst, R. F. and Mason, R. L. (1980). Regression Analysis and Its Application. Marcel Dekker, New York, pp. 317–328. (One of the better treatments of the subject in a textbook.) 5. Johnston, J. (1972). Econometric Methods, 2nd ed. McGraw-Hill, New York, pp. 322–331. (A good, simple derivation of PC regression.) 6. Karson, M. J. (1982). Multivariate Statistical Methods. Iowa State University Press, Ames, IA, pp. 191–224. (Many numerical examples. Excellent list of references.) 7. Massy, W. F. (1965). J. Amer. Statist. Ass., 60, 234–256. (Derivation of PC regression. Fairly technical paper and one of the first.) 8. McCallum, B. T. (1970). Rev. Econ. Statist., 52, 110–113. (Good paper. Easy to read.) 9. Morrison, D. F. (1967). Multivariate Statistical Methods. McGraw-Hill, New York, pp.

PRINCIPAL DIFFERENTIAL ANALYSIS 221–258. (A rather simple approach. Long list of references.) 10. Pidot, G. B., Jr. (1969). Rev. Econ. Statist., 51, 176–188. (Suggests using a combination of the X’s and the PC’s.) See also COMPONENT ANALYSIS; DISCRIMINANT ANALYSIS; LATENT ROOT REGRESSION; MULTICOLLINEARITY; and MULTIVARIATE ANALYSIS.

KEREN L. COXE

PRINCIPAL COMPONENTS, ¨ SCHONEMANN’S THEOREM FOR The following theorem by Sch¨onemann [7] dealing with principal components* has been used by various authors [1,4,5] in connection with the so-called Spearman’s hypothesis (see, e.g., ref. 3) of the positive relationship between the black–white differences in means on cognitive tests and the loading of these tests on the general common factor identified as ‘‘general cognitive ability’’. Sch¨onemann [6,7] asserts that this relationship is ascribable to a statistical artifact and is simply just a mathematical necessity. The statement of the theorem is as follows: If the range Rp —a p-dimensional Euclidean space—of a p-variate normal vector y ∼ N(0, ) is partitioned into a
high set (H) and a low set (L) by the plane k yk = 0 and both within-covariance matrices  H and  L remain positive-definite, then (i) the mean difference vector d = E(y|H) − E(y|L) is collinear with the largest principal components (pc’s) of , and (ii) d is also collinear with the pc1’s (eigenvectors corresponding to the largest eigenvalues) of  L and  H . In other words, if the test correlations are positive in the parent population, they remain so in both selected subpopulations. Parts (i) and (ii) of the theorem do not generally hold for the samples created by
selection on the basis of a variable k yk [2]. (It may be noteworthy that Americans are not usually sorted into categories of ‘‘white’’ and ‘‘black’’ by an observed selection variable that is a sum of test scores. Moreover, principal components analysis is not well suited to assess cognitive abilities, since it does not distinguish between factors common to the tests under consideration and factors specific to the tests.)

6407

REFERENCES 1. Braden, J. P. (1989). Fact or artifact. An empirical test of Spearman’s hypothesis. Intelligence, 13, 149–155. 2. Dolan, C. V. (1997). A note on Sch¨onemann’s refutation of Spearman’s hypothesis. Multivariate Behav. Res., 32, 319–328. 3. Jensen, A. R. (1987). Further evidence for Spearman’s hypothesis concerning black-white differences on psychometric tests. Brain Behav. Sci., 10, 512–519. 4. Jensen, A. R. (1992). Spearman’s hypothesis: methodology and evidence. Multivariate Behav. Res., 27, 225–233. 5. Loehlin, J. C. (1992). On Sch¨onemann, on Guttman, on Jensen, via Lewontin. Multivariate Behav. Res., 27, 261–263. 6. Sch¨onemann, P. H. (1989). Some new results on the Spearman hypothesis artifact. Bull. Psychon. Soc., 27, 462–464. 7. Sch¨onemann, P. H. (1992). Extension of Guttman’s results from g to PC1. Multivariate Behav. Res., 27, 219–223. See also COMPONENT ANALYSIS; PRINCIPAL COMPONENTS; and PSYCHOLOGICAL TESTING THEORY.

PRINCIPAL DIFFERENTIAL ANALYSIS Principal differential analysis (PDA) is used in the analysis of functional data to estimate a linear differential equation (linear DIFE) that describes the observed functions. The main References are 4, 8, and 9, and the entry FUNCTIONAL DATA ANALYSIS can be consulted for a discussion of functional data.

DIFFERENTIAL EQUATION MODELS It is often natural to describe a function of time or space in terms of a relationship between the function’s rate of change and its value. More generally, a relation may be specified among derivatives of various orders, the function itself, and other functions. Differential equations are powerful tools for modeling functional data that offer capabilities difficult to achieve with conventional parametric and nonparametric approaches. Because a DIFE involves rates of change, it is often called a

6408

PRINCIPAL DIFFERENTIAL ANALYSIS

dynamic equation, and is said to offer a model for the dynamics of a process rather than just its static behavior. In this article, lower case Greek and Roman letters such as x and β indicate functions of a single argument t unless otherwise indicated, instead of the usual notation β(t) and x(t). This simplifies expressions considerably. Also, we use Dx to indicate the first derivative of function x, Dm x for the derivative of order m, and specifically, D0 x = x. The article will only consider functions of a single argument, but principal differential analysis can also be applied to functions of multiple variables. There are many famous examples of differential equation models. Newton’s Second Law is D2 x = βf , where x is the position of a moving body through time, f is a force applied to the body, and the constant β is the reciprocal of the body’s mass. The differential equation for harmonic motion is D2 x = −ω2 x, where spring constant ω2 determines the motion’s period. Exponential growth, decay, and approach to an asymptote are all described by D2 x = βDx when β is a constant, and if β is a function, this simple equation describes any smooth strictly monotone curve [5]. See reference 6 for other examples of differential equations that are useful in statistics. A linear differential equation of order m has the form Dm x = β0 x + β1 Dx + · · · + βm−1 Dm−1 x + f , (1) and the equation is said to be homogeneous or nonhomogeneous depending on whether the function f is zero or not respectively. The examples given above are all linear DIFEs because of their linearity in the coefficients βj , j = 0, . . . , m − 1. The forcing function f is often considered to represent external or exogenous input to a system whose closed state is described by the homogeneous equation. Linear DIFEs share with other types of linear equations a mathematical structure that is relatively easy to describe and techniques for solving them that are fairly routine. Moreover, a linear differential equation has a solution for any value of t, a feature that

does not always hold for nonlinear DIFEs. This article will focus on linear DIFEs, but nonlinear DIFEs are playing an increasing role in statistics because of the much richer range of functional behaviors that they can define (see Ref. 1 for a readable account). You are encouraged to regard (1) as an equation defining a multiple regression model. The dependent variable is Dm x and the independent variables or covariates are Dj x. The forcing function f can, if not observed directly, be viewed as the error or residual term. The main distinction between a conventional linear model and a linear DIFE is that each variable Dj x and each regression coefficient βj is a function of t. That is, Equation 1 is a t-indexed multiple regression model. In fact, if a sample of functions xi , i = 1, . . . , N are available, then performing a multiple regression for each of a fine mesh of values of t is a reasonable strategy for estimating the coefficient functions βj . It is not surprising, therefore, that the theory associated with linear DIFEs uses the same matrix theory that underlies regular regression (see Ref. 2). Describing functional observations by differential equations brings several useful features that are not provided by conventional approaches. By conventional we mean parametric models; expansions in terms of Fourier series, B-splines, and other basis systems; or other methods that smooth the data. For example, exponential behavior in a function x can either be expressed by the equation x(t) = c1 + c2 eβt where c1 and c2 are arbitrary constants, or more compactly as the DIFE D2 x = βDx. First of all, a DIFE simultaneously defines the behavior of the m derivatives Dj x and the behavior of the function itself, while conventional methods focus all the attention on estimating x. In applications where rates of change are of theoretical interest, as is the case in mechanics, a DIFE requires that derivatives also be well estimated because of their explicit role in the model. Secondly, a single differential equation model can simultaneously fit a whole sample of distinct functions xi . Homogeneous linear DIFEs of order m have an m-dimensional space of solutions, like the exponential two-dimensional exponential equation given

PRINCIPAL DIFFERENTIAL ANALYSIS

above. That is, any solution x can be
represented as a linear combination x = m j cj uj of m linearly independent specific solutions uj , where the coefficients cj are arbitrary constants. The homogeneous case for Newton’s Law, D2 x = 0, has as its solution x(t) = c1 + c2 t. Solutions of the DIFE D3 x = −ω2 Dx are all linear combinations of 1, sin ωt, and cos ωt. Moreover, nonhomogeneous equations have the same solution space translated by a functional of the forcing function f . Finally, as was noted above, specific constraints on x can often be expressed by a specialized DIFE. For example, Dm x = βDm−1 x constrains x to entirely positive/negative, strictly monotonic, or strictly convex/concave depending on whether m is 1, 2, or 3 respectively. The solutions to the nonlinear DIFE Dx = βx(1 − x) are the set of probability functions taking values in (0,1).

ESTIMATING A LINEAR DIFFERENTIAL OPERATOR L The basic idea behind PDA is to estimate a linear differential equation of order m from one or more observed functions xi . The mdimensional solution space of the estimated DIFE is designed to contain most of the variation in these functions. PDA has much in common with principal components analysis (PCA) since, if the DIFE is successful, a set of m linearly independent functions uj will be identified that can be combined linearly to reproduce well the original functional data. But unlike PCA, PDA aims to give a good account of each of m − 1 derivatives at the same time. In PDA, we seek a linear differential operator L of the form Lx =

m−1 

βj Dj x + Dm x.

(2)

Now, viewing the forcing function f as an unknown residual term, we can ask whether it is possible to use the data to estimate an operator L such that the least squares fitting criterion N  

(Lxi )2 (t)dt =

i=1

N  

fi2 (t)dt

(3)

i=1

is minimized. That is, we are aiming to get as close as possible to a homogeneous model Lx = 0 that has solutions that well describe the data. Similarly, in PCA, we essentially define the analysis as a least squares problem in which a linear projection operator Q is defined that minimizes the variation in the residual vectors Qxi . The projection of the observations into the m-dimensional space spanned by the principal components is defined by m eigenvalues. But the linear differential operator L is also defined by the m coefficient functions βj , j = 0, . . . , m − 1; and the essential problem in PDA is also to estimate these coefficient functions by minimizing a least squares criterion. A number of extensions of PDA are possible. We can augment Lx by adding to it terms αk zk where each zk , k = 1, . . . , K, is a known function, and each coefficient function αk also needs to be estimated. If we do so, we do not change the nature of the problem in any qualitative way; the result is still an operator that is linear in the unknown coefficient functions βj and αk . We can also work with systems of linear differential equations, where we have an equation for each of a set of variables, x ,  = 1, . . . , L. The actual estimation of the weight coefficients βj and αk requires the use of a basis function expansion. That is, when we have a set of K basis functions φk , which are often terms in a Fourier series, or a set of B-splines, then

j=0

The operator L defines the homogeneous part of Equation 1, and we can think of it as a generalization of the notion of the specific derivative Dm . Using this definition, we can express the DIFE model (1) more compactly as Lx = f .

6409

βj =

K 

cjk φk = c j φ.

(4)

k=1

The fitting criterion (3) is then quadratic in the coefficients of the expansion, and its minimization merely requires the solution of a set of linear equation. PDA also works fine

6410

PRINCIPAL DIFFERENTIAL ANALYSIS

if there is only a single observed function x provided that the numbers of basis functions in the weight coefficient expansions are small relative to that of x. Finally, we can easily incorporate both specific constraints on certain βj ’s, such as being zero, and can also apply roughness penalties to them that ensure that they are as smooth in some sense as is required. The functions xi being analyzed are usually the result of some smoothing process applied to a set of discrete observed values. It is essential that the smoothing procedure gives a reasonable estimate of the derivatives up to order m at the same time, and consequently PDA is most effective for functional observations where the observational error is fairly small relative to the size of the variation in the function that the smoothing technique is trying to estimate. Methods are under development, however, that will permit using the discrete noisy data directly rather than passing through a smoothing stage. These are based on the work of Reference 3. Because a linear DIFE is essentially a tindexed regression equation, the inferential and diagnostic tools of regression analysis can be adapted to the results of a PDA. We can, for example, examine squared multiple correlation and F-ratio functions R2 and F 0.30

D3 x = βxx1 Dx + βxx2 D2 x + βxy1 Dy + βxy2 D2 y + βxz1 Dz + βxz2 D2 z D3 y = βyx1 Dx + βyx2 D2 x + βyy1 Dy + βyy2 D2 y + βyz1 Dz + βyz2 D2 z D3 z = βzx1 Dx + βzx2 D2 x + βzy1 Dy + βzy2 D2 y + βzz1 Dz + βzz2 D2 z.

(5)

This is a second-order linear system where pen velocities Dx, Dy, and Dz are the

•• •• • •

••••••••••••••• •••• •••• •••• ••• • •• ••••••• ••• ••• ••• ••••••••••••••• ••• • • •• •••••••••••••• • •• •••••••••••••••• ••• •••••••• • • • • • •• • • •• • • ••• • • ••• •• •• • • • •• • •• • •• ••• •• •••••• • • •• • ••••• •• •••••••••••••• • • • • • • • • • •• ••• •• • • • • • • • • • • • • ••• •• • • ••• • ••••••• • • • ••••• • ••••••••• • • • • • • • ••••• •• • ••••••••••• ••• • • • ••• • • • • •• • • • • • • • • • • •• ••• • • • • •••••••••• •• • ••• • • •• • • • •••••••• • • •• •• • • •• •• • •• ••• • •••• ••••••••••••••• • • • • • •• • • • • •• • • ••• ••• • •••••• • ••••• • •• •• • • • • ••••••• •• • •••••• •••• • • • • • •• • • • • • • • • • • • • • • • •• •• •• ••• • •• •• •••••• • • • • •••• • • • ••• ••• • •• ••• •••• •• ••• • ••••

0.25

Y (m)

that are themselves functions of time so that we can identify where the linear DIFE fits the data well and where it does not. As in regression analysis, an inspection of the residual functions fi can offer many useful insights. Examples of PDA can be found in references 8 and 9. Figure 1 is taken from reference 7 reporting an analysis of 50 replications of writing ‘‘statistical science’’ in Mandarin. The dynamics of handwriting involve sequences of high-energy strokes occurring about every 120 milliseconds with levels of acceleration that would send a rocket into orbit in a few minutes. In this application, we have as functional observations the three coordinate functions xi , yi , and zi , i = 1, . . . , 50. The differential equation model was the system

0.20

0.15

0.10 −0.2

−0.1

0.0 X (m)

0.1

0.2

Figure 1. A sample of Mandarin script for the phrase ‘‘statistical science.’’ The dots are pen positions at 200-ms intervals, and the solid line is the reconstruction of the script using the second-order DIFE [5].

PRINCIPAL POINTS

functional observations. It is remarkable that such a simple equation can reconstruct the static trace of the original script so well as well as describing accurately its first and second time-derivative. Acknowledgments The author’s work was prepared under a grant from the Natural Science and Engineering Research Council of Canada.

REFERENCES 1. Alligood, K. T., Sauer, T. D., and Yorke, J. A. (1996). Chaos: An Introduction to Dynamical Systems. Springer, New York. 2. Coddington, E. A. and Carlson, R. (1997). Linear Ordinary Differential Equations. SIAM, Philadelphia, Pa. 3. Heckman, N. and Ramsay, J. O. (2000). Penalized regression with model-based penalties. Can. J. Stat., 28, 241–258. 4. Ramsay, J. O. (1995). Principal differential analysis: data reduction by differential operators. J. R. Stat. Soc., Ser. B, 58, 495– 508. 5. Ramsay, J. O. (1998). Estimating smooth monotone functions. J. R. Stat. Soc., Ser. B, 60, 365–375. 6. Ramsay, J. O. (2000). Differential equation models for statistical functions. Can. J. Stat., 28, 225–240. 7. Ramsay, J. O. (2000). Functional components of variation in handwriting. J. Am. Stat. Assoc., 95, 9–15. 8. Ramsay, J. O. and Silverman, B. W. (1997). Functional Data Analysis. Springer, New York. 9. Ramsay, J. O. and Silverman, B. W. (2002). Applied Functional Data Analysis. Springer, New York.

JAMES O. RAMSAY

PRINCIPAL MOMENTS An obsolete term for central moments∗ (moments about the mean).

PRINCIPAL POINTS A set of k principal points of a distribution is the best k-point approximation of the distribution in terms of a quadratic loss. Principal

6411

points can be motivated by the following practical problem. Suppose a clothing manufacturer wishes to produce a new type of swimsuit in four different sizes. Some relevant measurements such as stature, bust size, etc., are taken on a large number of people. The statistician is asked to identify four ‘‘typical’’ people whose dimensions will be used to model the swimsuits. The term principal points was defined in a similar problem of determining sizes and shapes of new gas masks [9]. Another problem to which principal-point theory has been applied is determining a few representative curves from a large collection of curves [11,15].

DEFINITION AND PROPERTIES OF PRINCIPAL POINTS A set of k points ξ 1 , . . . , ξ k are called k principal points of a random vector X if E[minj=1,...,k X − ξ j 2 ]
E[minj=1,...,k X − yj 2 ] for any set of k points {y1 , . . . , yk } ⊂ Rp . Here · denotes the Euclidean norm. Thus principal points provide a least squares∗ approximation by projecting the values of a random vector on a finite number of points, similar to Pearson’s definition of principal components∗ [22]. Principal points are a theoretical analogue to k-cluster means [12] (k-MEANS ALGORITHMS), and have also been called k-centers [21]. For k = 1, the single principal point is the mean of the distribution. The k  2 principal points of a distribution are a generalization of the mean from one to several points. Usually numerical methods are required to determine the principal points except in some simple cases. For instance, for the N (µ, σ 2 ) distribution, the two principal points are  µ ± σ 2/π [2,9]. For any k  1, the k principal points of the uniform distribution in [a, b] are a + (2j − 1)(b − a)/2k, j = 1, . . . , k [31]. Determining the principal points of a distribution is aided by studying self-consistent points, particularly for multivariate distributions. For k points y1 , . . . , yk in Rp , let Sj ⊂ Rp denote the set of all points in Rp closer to yj than to all other yi ’s. If E[X|X ∈ Sj ] = yj for j = 1, . . . , k, then the yj are called k self-consistent points of X [10]. Self-consistent points have also been called ‘‘unbiased’’ [19].

6412

PRINCIPAL POINTS

Principal points are self-consistent [10]. Multivariate distributions have typically several sets of k self-consistent points but only one set of k principal points. For example, consider the bivariate normal distribution∗ centered at the origin with covariance matrix diag (σ 2 , 1), where σ > 1. Then there exist two sets of two self-consistent  points: {(0, 2/π ) , (0, − 2/π ) } and {(σ 2/π , 0) , (−σ 2/π , 0) }. The second set corresponds to two principal points [9]. An important property of self-consistent points (and hence principal points) is the following: if y1 , . . . , yk are k self-consistent points of X, then E[X] lies in the convex hull of the yj [28]. See also SELF-CONSISTENCY.

UNIVARIATE RESULTS Principal points for the standard normal distribution have appeared in numerous publications, e.g., refs. 2, 9, 18, and for k up to 31 in ref. [6]. Computational methods for finding principal points of univariate continuous distributions are given in refs. 18, 25, 30. Symmetric univariate distributions always have a set of k symmetric selfconsistent points [31], but the k principal points may not be symmetric to the mean even if the distribution is continuous and unimodal [9]. However, if the distribution has a log-concave density, then there exists a unique set of k self-consistent points which must be the k principal points of the distribution [8,16,26,29]. Under rather general conditions on the density f of a random variable X, asymptotically as k → ∞, the k principal points tend towards the k quantiles of the distribution whose density is proportional to f 1/3 [23].

MULTIVARIATE RESULTS Suppose X follows a p-variate elliptical distribution∗ such as a multivariate normal with mean zero. If k principal points of X span a subspace of dimension q < p, then this subspace is also spanned by q eigenvectors associated with the q largest eigenvalues of the covariance matrix [28]. Similar results relating principal component subspaces to

principal points and self-consistent points hold for a larger class of symmetric multivariate distributions [27]. For the bivariate normal, numerical results indicate that principal points and self-consistent points form highly symmetrical patterns [9,10,17]. For the uniform distribution on the unit square, asymptotically the optimal configuration of points as k → ∞ corresponds to the tessellation by regular hexagons with principal points at their centers ([7,20]; see RANDOM TESSELLATIONS). ESTIMATION OF PRINCIPAL POINTS To apply the theory of principal points to real data, various estimators have been defined, ranging from normal theory maximum likelihood∗ [10] to the nonparametric k-means clustering algorithm∗ [12]. The cluster means from the k-means algorithm are by construction a set of k self-consistent points of the empirical distribution. Under rather general conditions, the cluster means from the kmeans algorithm are consistent estimators of principal points, and are asymptotically normal [1,13,19,24]. Convergence results have also been established for the more general case of distributions on Banach spaces [3,21]. Semiparametric∗ estimators of principal points have been defined under the constraint that they form particular symmetric patterns in the subspace of the first few principal components∗ , and by applying the k-means algorithm to data reduced to the first few principal components [10], as suggested by the theory of principal points of elliptical distributions [28]. EXAMPLE Principal point methodology has been applied in a problem of determining shapes and sizes of new gas masks for the Swiss Army [10]. Due to the considerable variability in human heads, a single mask would not be sufficient to fit all its users. The dimensions of k = 2 to 5 ‘‘typical’’ heads were requested for constructing masks of different sizes and shapes. Six head measurements were taken on a sample of Swiss soldiers. Assuming multiˆ denote the variate normality, let µ ˆ and


PRINCIPAL POINTS

maximum likelihood estimates of the mean vector and the covariance matrix respectively. Then the k principal points of the ˆ distribution are maximum likeliˆ
) N6 (µ, hood estimates of k principal points. Figure 1 shows the normal theory maximum likelihood estimates of k = 4 principal points for the head dimension data. The points are plotted in the subspace spanned by the first two principal components. These estimates were based on the assumption that the four principal points form a ‘‘cross’’ pattern in the subspace spanned by the first two principal components. RELATED TOPICS Since a set of k principal points partition the space into k Voronoi regions or strata according to minimal distance, the mathematical problem of determining principal points is related to the problem of optimum stratification∗ [4] and optimal grouping [2]. For a unified approach to these and several other related problems that share a common mathematical framework, see [5]. Principal points are also related to quantization in signal processing∗ . A common form of quantization is rounding off, which occurs in all digital systems. A quantizer partitions

6413

the sample space of a random variable and maps the random variable to representative points so as to minimize the average distortion. Determining an optimal quantizer is often equivalent to finding a set of representative (principal) points of a distribution [14]. IEEE published a special issue on the topic of quantization (IEEE Transactions on Information Theory, 1982). Several of the articles that appear in that issue are noted in the references below. REFERENCES 1. Bryant, P. (1991). Large-sample results for optimization-based clustering methods. J. Classification, 8, 31–44. 2. Cox, D. (1957). Note on grouping. J. Amer. Statist. Ass., 52, 543–547. 3. Cuesta, J. and Matran, C. (1988). The strong law of large numbers for k-means and best possible nets of Banach valued random variables. Probab. Theory and Related Fields, 78, 523–534. 4. Dalenius, T. (1950). The problem of optimum stratification. Skand. Aktuarietidskrift, 33, 203–213. 5. Eubank, R. (1988). Optimal grouping, spacing, stratification, and piecewise constant approximation. SIAM Rev., 30, 404–420.

Figure 1. Normal theory maximum likelihood estimates of four principal points (denoted •) for the head dimensions data. Scatter plot shows 50 observations in the coordinate system of the first two principal components. Broken lines indicate the boundaries of the Voronoi regions associated with the principal points.

6414

PRINCIPLE OF PARSIMONY

6. Fang, K. and He, S. (1982). The problem of selecting a given number of representative points in a normal population and a generalized Mills’ ratio. Tech. Rep., Department of Statistics, Stanford University. 7. Fejes T´oth, L. (1953). Lagerungen in der Ebene, auf der Kugel und im Raum. SpringerVerlag, Berlin. 8. Fleischer, P. (1964). Sufficient conditions for achieving minimum distortion in a quantizer. IEEE Int. Convention Record Part I, 12, 104–111. 9. Flury, B. (1990). Principal points. Biometrika, 77, 33–41. 10. Flury, B. (1993). Estimation of principal points, Appl. Statist., 42, 139–151. 11. Flury, B. and Tarpey, T. (1993). Representing a large collection of curves: a case for principal points, Amer. Statist., 47, 304–306. 12. Hartigan, J. (1975). Clustering Algorithms. Wiley, New York. 13. Hartigan, J. (1978). Asymptotic distributions for clustering criteria. Ann. Statist., 6, 117–131. 14. Iyengar, S. and Solomon, H. (1983). Selecting representative points in normal populations. In Recent Advances in Statistics: Papers in Honor of Herman Chernoff on his 60th Birthday. Academic Press, New York, pp. 579– 591.

22. Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Phil. Mag., 2, 559–572. 23. P¨otzelberger, K. and Felsenstein, K. (1994). An asymptotic result on principal points for univariate distributions. Optimization, 28, 397–406. 24. Pollard, D. (1981). Strong consistency of kmeans clustering. Ann. Statist., 9, 135–140. 25. Rowe, S. (1996). An algorithm for computing principal points with respect to a loss function in the unidimensional case, Statist. and Comput., 6, 187–190. 26. Tarpey, T. (1994). Two principal points of symmetric, strongly unimodal distributions. Statist. and Probab. Lett., 20, 253–257. 27. Tarpey, T. (1995). Principal points and selfconsistent points of symmetric multivariate distributions. J. Multivariate Anal., 53, 39–51. 28. Tarpey, T., Li, L., and Flury, B. (1995). Principal points and self-consistent points of elliptical distributions. Ann. Statist., 23, 103–112. 29. Trushkin, A. (1982). Sufficient conditions for uniqueness of locally optimal quantizer for a class of convex error weighting functions. IEEE Trans. Inf. Theory, IT-28, 187–198. 30. Zopp`e, A. (1995). Principal points of univariate continuous distributions, Statist. and Comput., 5, 127–132.

15. Jones, M. and Rice, J. (1992). Displaying the important features of large collections of similar curves. Amer. Statist., 46, 140–145.

31. Zopp`e, A. (1997). On uniqueness and symmetry of self-consistent points of univariate continuous distributions, J. Classification, 14, 147–158.

16. Kieffer, J. (1982). Exponential rate of convergence for Lloyd’s method I. IEEE Trans. Inf. Theory, IT-28, 205–210.

See also COMPONENT ANALYSIS; k-MEANS ALGORITHMS; SELF-CONSISTENCY; and SIZE AND SHAPE ANALYSIS.

¨ 17. Kipper, S. and Parna, K. (1992). Optimal kcentres for a two-dimensional normal distribution. Acta et Comment. Univ. Tartuensis, 942, 21–27. 18. Lloyd, S. (1982). Least squares quantization in PCM. IEEE Trans. Inf. Theory, IT-28, 129–136. 19. MacQueen, J. (1966). Some methods for classification and analysis of multivariate observations. Proc. Fifth Berkeley Symp. Math. Statist. and Probab. University of California Press, vol. 1, pp. 281–297.

BERNARD D. FLURY THADDEUS TARPEY

PRINCIPLE OF PARSIMONY. See PARSIMONY, PRINCIPLE OF ; and SCIENTIFIC METHOD STATISTICS

AND

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

20. Newman, D. (1982). The hexagon theorem. IEEE Trans. Inf. Theory, IT-28, 137–139.

PURPOSE AND SCOPE

¨ 21. Parna, K. (1990). On the existence and weak convergence of k-centres in Banach spaces. Acta et Comment. Univ. Tartuensis, 893, 17–28.

1. The statistician’s job. The statistician’s responsibility, whether as a consultant on a part-time basis, or on regular salary is to

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

find problems that other people could not be expected to perceive. The statistician’s tool is statistical theory (theory of probability). It is the statistician’s use of statistical theory that distinguishes him from other experts. Statistical practice is a collaborative venture between statistician and experts in subject matter. Experts who work with a statistician need to understand the principles of professional statistical practice as much as the statistician needs them. Challenges face statisticians today as never before. The whole world is talking about safety in mechanical and electrical devices (in automobiles, for example); safety in drugs; reliability; due care; pollution; poverty; nutrition; improvement of medical practice; improvement of agricultural practice; improvement in quality of product; breakdown of service; breakdown of equipment; tardy buses, trains and mail; need for greater output in industry and in agriculture; enrichment of jobs. These problems cannot be understood and cannot even be stated, nor can the effect of any alleged solution be evaluated, without the aid of statistical theory and methods. One cannot even define operationally such adjectives as reliable, safe, polluted, unemployed, on time (arrivals), equal (in size), round, random, tired, red, green, or any other adjective, for use in business or in government, except in statistical terms. To have meaning for business or legal purposes, a standard (as of safety, or of performance or capability) must be defined in statistical terms. 2. Professional practice. The main contribution of a statistician in any project or in any organization is to find the important problems, the problems that other people cannot be expected to perceive. An example is the 14 points that top management learned in Japan in 1950. The problems that walk in are important, but are not usually the important problems (4). The statistician also has other obligations to the people with whom he works, examples being to help design studies, to construct an audit by which to discover faults in procedure before it is too late to make alterations, and at the end, to evaluate the statistical reliability of the results.

6415

The statistician, by virtue of experience in studies of various kinds, will offer advice on procedures for experiments and surveys, even to forms that are to be filled out, and certainly in the supervision of the study. He knows by experience the dangers of (1) forms that are not clear, (2) procedures that are not clear, (3) contamination, (4) carelessness, and (5) failure of supervision. Professional practice stems from an expanding body of theory and from principles of application. A professional man aims at recognition and respect for his practice, not for himself alone, but for his colleagues as well. A professional man takes orders, in technical matters, from standards set by his professional colleagues as unseen judges, never from an administrative superior. His usefulness and his profession will suffer impairment if he yields to convenience or to opportunity. A professional man feels an obligation to provide services that his client may never comprehend or appreciate. A professional statistician will not follow methods that are indefensible, merely to please someone, or support inferences based on such methods. He ranks his reputation and profession as more important than convenient assent to interpretations not warranted by statistical theory. Statisticians can be trusted and respected public servants. Their careers as expert witnesses will be shattered if they indicate concern over which side of the case the results seem to favor. ‘‘As a statistician, I couldn’t care less’’ is the right attitude in a legal case or in any other matter. LOGICAL BASIS FOR DIVISION OF RESPONSIBILITIES 3. Some limitations in application of statistical theory. Knowledge of statistical theory is necessary but not sufficient for successful operation. Statistical theory does not provide a road map toward effective use of itself. The purpose of this article is to propose some principles of practice, and to explain their meaning in some of the situations that the statistician encounters. Statisticians have no magic touch by which they may come in at the stage of tabulation

6416

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

and make something of nothing. Neither will their advice, however wise in the early stages of a study, ensure successful execution and conclusion. Many a study launched on the ways of elegant statistical design, later boggled in execution, ends up with results to which the theory of probability can contribute little. Even though carried off with reasonable conformance to specifications, a study may fail through structural deficiencies in the method of investigation (questionnaire, type of test, technique of interviewing) to provide the information needed. Statisticians may reduce the risk of this kind of failure by pointing out to their clients in the early stages of the study the nature of the contributions that they themselves must put into it. (The word client will denote the expert or group of experts in a substantive field.) The limitations of statistical theory serve as signposts to guide a logical division of responsibilities between statistician and client. We accordingly digress for a brief review of the power and the limitations of statistical theory. We note first that statistical inferences (probabilities, estimates, confidence limits, fiducial limits, etc.), calculated by statistical theory from the results of a study, will relate only to the material, product, people, business establishments, and so on, that the frame was in effect drawn from, and only to the environment of the study, such as the method of investigation, the date, weather, rate of flow and levels of concentration of the components used in tests of a production process, range of voltage, or of other stress specified for the tests (as of electrical equipment). Empirical investigation consists of observations on material of some kind. The material may be people; it may be pigs, insects, physical material, industrial product, or records of transactions. The aim may be enumerative, which leads to the theory for the sampling of finite populations∗ . The aim may be analytic, the study of the causes that make the material what it is, and which will produce other material in the future. A definable lot of material may be divisible into identifiable sampling units. A list of these sampling units, with identification of each unit, constitutes a frame. In some physical

and astronomical investigations, the sampling unit is a unit of time. We need not detour here to describe nests of frames for multistage sampling. The important point is that without a frame there can be neither a complete coverage of a designated lot of material nor a sample of any designated part thereof. Stephan introduced the concept of the frame, but without giving it a name (7). Objective statistical inferences in respect to the frame are the speciality of the statistician. In contrast, generalization to cover material not included in the frame, nor to ranges, conditions, and methods outside the scope of the experiment, however essential for application of the results of a study, are a matter of judgment and depend on knowledge of the subject matter (5). For example, the universe in a study of consumer behavior might be all the female homemakers in a certain region. The frame might therefore be census blocks, tracts or other small districts, and the ultimate sampling unit might be a segment of area containing households. The study itself will, of course, reach only the people that can be found at home in the segments selected for the sample. The client, however, must reach generalizations and take action on the product or system of marketing with respect to all female homemakers, whether they be the kind that are almost always at home and easy to reach, or almost never at home and therefore in part omitted from the study. Moreover, the female homemakers on whom the client must take action belong to the future, next year, and the next. The frame only permits study of the past. For another example, the universe might be the production process that will turn out next week’s product. The frame for study might be part of last week’s product. The universe might be traffic in future years, as in studies needed as a basis for estimating possible losses or gains as a result of a merger. The frame for this study might be records of last year’s shipments. Statistical theory alone could not decide, in a study of traffic that a railway is making, whether it would be important to show the movement of (for example) potatoes, separately from other agricultural products in

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

the northwestern part of the United States; only he who must use the data can decide. The frame for a comparison of two medical treatments might be patients or other people in Chicago with a specific ailment. A pathologist might, on his own judgment, without further studies, generalize the results to people that have this ailment anywhere in the world. Statistical theory provides no basis for such generalizations. No knowledge of statistical theory, however profound, provides by itself a basis for deciding whether a proposed frame would be satisfactory for a study. For example, statistical theory would not tell us, in a study of consumer research, whether to include in the frame people that live in trailer parks. The statistician places the responsibility for the answer to this question where it belongs, with the client: Are trailer parks in your problem? Would they be in it if this were a complete census? If yes, would the cost of including them be worthwhile? Statistical theory will not of itself originate a substantive problem. Statistical theory cannot generate ideas for a questionnaire or for a test of hardness, nor can it specify what would be acceptable colors of dishes or of a carpet; nor does statistical theory originate ways to teach field workers or inspectors how to do their work properly or what characteristics of workmanship constitute a meaningful measure of performance. This is so in spite of the fact that statistical theory is essential for reliable comparisons of questionnaires and tests. 4. Contributions of statistical theory to subject matter. It is necessary, for statistical reliability of results, that the design of a survey or experiment fit into a theoretical model. Part of the statistician’s job is to find a suitable model that he can manage, once the client has presented his case and explained why statistical information is needed and how the results of a study might be used. Statistical practice goes further than merely to try to find a suitable model (theory). Part of the job is to adjust the physical conditions to meet the model selected. Randomness∗ , for example, in sampling a frame, is not just a word to assume; it must be created by use of random numbers for the selection of sampling units. Recalls to find people at home,

6417

or tracing illegible or missing information back to source documents, are specified so as to approach the prescribed probability of selection. Statistical theory has in this way profoundly influenced the theory of knowledge. It has given form and direction to quantitative studies by molding them into the requirements of the theory of estimation∗ , and other techniques of inference. The aim is, of course, to yield results that have meaning in terms that people can understand. The statistician is often effective in assisting the substantive expert to improve accepted methods of interviewing, testing, coding, and other operations. Tests properly designed will show how alternative test or field procedures really perform in practice, so that rational changes or choices may be possible. It sometimes happens, for example, that a time-honored or committee-honored method of investigation, when put to statistically designed tests, shows alarming inherent variances between instruments, between investigators, or even between days for the same investigator. It may show a trend, or heavy intraclass correlation∗ between units, introduced by the investigators. Once detected, such sources of variation may then be corrected or diminished by new rules, which are of course the responsibility of the substantive expert. Statistical techniques provide a safe supervisory tool to help to reduce variability in the performance of worker and machine. The effectiveness of statistical controls, in all stages of a survey, for improving supervision, to achieve uniformity, to reduce errors, to gain speed, reduce costs, and to improve quality of performance in other ways is well known. A host of references could be given, but two will suffice [2,3]. 5. Statistical theory as a basis for division of responsibilities. We may now, with this background, see where the statistician’s responsibilities lie. In the first place, his specialized knowledge of statistical theory enables him to see which parts of a problem belong to substantive knowledge (sociology, transportation, chemistry of a manufacturing process, law), and which parts are statistical. The responsibility as logician in a study falls to him by

6418

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

default, and he must accept it. As logician, he will do well to designate, in the planning stages, which decisions will belong to the statistician and which to the substantive expert. This matter of defining specifically the areas of responsibility is a unique problem faced by the statistician. Business managers and lawyers who engage an expert on corporate finance, or an expert on steam power plants, know pretty well what such experts are able to do and have an idea about how to work with them. It is different with statisticians. Many people are confused between the role of theoretical statisticians (those who use theory to guide their practice) and the popular idea that statisticians are skillful in compiling tables about people or trade, or who prophesy the economic outlook for the coming year and which way the stock market will drift. Others may know little about the contributions that statisticians can make to a study or how to work with them. Allocation of responsibilities does not mean impervious compartments in which you do this and I’ll do that. It means that there is a logical basis for allocation of responsibilities and that it is necessary for everyone involved in a study to know in advance what he or she will be accountable for. A clear statement of responsibilities will be a joy to the client’s lawyer in a legal case, especially at the time of cross-examination. It will show the kind of question that the statistician is answerable for, and what belongs to the substantive experts. Statisticians have something to learn about professional practice from law and medicine. 6. Assistance to the client to understand the relationship. The statistician must direct the client’s thoughts toward possible uses of the results of a statistical study of the entire frame without consideration of whether the entire frame will be studied or only a small sample of the sampling units in the frame. Once these matters are cleared, the statistician may outline one or more statistical plans and explain to the client in his own language what they mean in respect to possible choices of frame and environmental conditions, choices of sampling unit, skills, facilities and supervision required. The statistician will urge the client to foresee

possible difficulties with definitions or with the proposed method of investigation. He may illustrate with rough calculations possible levels of precision to be expected from one or more statistical plans that appear to be feasible, along with rudimentary examples of the kinds of tables and inferences that might be forthcoming. These early stages are often the hardest part of the job. The aim in statistical design is to hold accuracy and precision to sensible levels, with an economic balance between all the possible uncertainties that afflict data—built-in deficiencies of definition, errors of response, nonresponse, sampling variation, difficulties in coding, errors in processing, difficulties of interpretation. Professional statistical practice requires experience, maturity, fortitude, and patience, to protect the client against himself or against his duly appointed experts in subject matter who may have in mind needless but costly tabulations in fine classes (fiveyear age groups, small areas, fine mileage brackets, fine gradations in voltage, etc.), or unattainable precision in differences between treatments, beyond the capacity of the skills and facilities available, or beyond the meaning inherent in the definitions. These first steps, which depend heavily on guidance from the statistician, may lead to important modifications of the problem. Advance considerations of cost, and of the limitations of the inferences that appear to be possible, may even lead clients to abandon a study, at least until they feel most confident of the requirements. Protection of a client’s bank account, and deliverance from more serious losses from decisions based on inconclusive or misleading results, or from misinterpretation, is one of the statistician’s greatest services. Joint effort does not imply joint responsibility. Divided responsibility for a decision in a statistical survey is as bad as divided responsibility in any venture—it means that no one is responsible. Although they acquire superficial knowledge of the subject matter, the one thing that statisticians contribute to a problem, and which distinguishes them from other experts, is knowledge and ability in statistical theory.

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

SUMMARY STATEMENT OF RECIPROCAL OBLIGATIONS AND RESPONSIBILITIES 7. Responsibilities of the client. The client will assume responsibility for those aspects of the problem that are substantive. Specifically, he will stand the ultimate responsibility for: a. The type of statistical information to be obtained. b. The methods of test, examination, questionnaire, or interview, by which to elicit the information from any unit selected from the frame. c. The decision on whether a proposed frame is satisfactory. d. Approval of the probability model proposed by the statistician (statistical procedures, scope, and limitations of the statistical inferences that may be possible from the results). e. The decision on the classes and areas of tabulation (as these depend on the uses that the client intends to make of the data); the approximate level of statistical precision or protection that would be desirable in view of the purpose of the investigation, skills and time available, and costs. The client will make proper arrangements for: f. The actual work of preparing the frame for sampling, such as serializing and identifying sampling units at the various stages. g. The selection of the sample according to procedures that the statistician will prescribe, and the preparation of these units for investigation. h. The actual investigation; the training for this work, and the supervision thereof. i. The rules for coding; the coding itself. j. The processing, tabulations and computations, following procedures of estimation that the sampling plans prescribe. The client or his representative has the obligation to report at once any departure from instructions, to permit the statistician

6419

to make a decision between a fresh start or an unbiased adjustment. The client will keep a record of the actual performance. 8. Responsibilities of the statistician. The statistician owes an obligation to his own practice to forestall disappointment on the part of the client, who if he fails to understand at the start that he must exercise his own responsibilities in the planning stages, and in execution, may not realize in the end the fullest possibility of the investigation, or may discover too late that certain information that he had expected to get out of the study was not built into the design. The statistician’s responsibility may be summarized as follows: a. To formulate the client’s problem in statistical terms (probability model), subject to approval of the client, so that a proposed statistical investigation may be helpful to the purpose. b. To lay out a logical division of responsibilities for the client, and for the statistician, suitable to the investigation proposed. c. To explain to the client the advantages and disadvantages of various frames and possible choices of sampling units, and of one or more feasible statistical plans of sampling or experimentation that seem to be feasible. d. To explain to the client, in connection with the frame, that any objective inferences that one may draw by statistical theory from the results of an investigation can only refer to the material or system that the frame was drawn from, and only to the methods, levels, types, and ranges of stress presented for study. It is essential that the client understand the limitations of a proposed study, and of the statistical inferences to be drawn therefrom, so that he may have a chance to modify the content before it is too late. e. To furnish statistical procedures for the investigation—selection, computation of estimates and standard errors, tests, audits, and controls as seem warranted for detection and evaluation of important possible departures from specifications, variances between

6420

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

investigators, nonresponse, and other persistent uncertainties not contained in the standard error; to follow the work and to know when to step in. f. To assist the client (on request) with statistical methods of supervision, to help him to improve uniformity of performance of investigators, gain speed, reduce errors, reduce costs, and to produce a better record of just what took place. g. To prepare a report on the statistical reliability of the results. 9. The statistician’s report or testimony. The statistician’s report or testimony will deal with the statistical reliability of the results. The usual content will cover the following points: a. A statement to explain what aspects of the study his responsibility included, and what it excluded. It will delimit the scope of the report. b. A description of the frame, the sampling unit, how defined and identified, the material covered, and a statement in respect to conditions of the survey or experiment that might throw light on the usefulness of the results. c. A statement concerning the effect of any gap between the frame and the universe for important conclusions likely to be drawn from the results. (A good rule is that the statistician should have before him a rough draft of the client’s proposed conclusions.) d. Evaluation (in an enumerative study) of the margin of uncertainty, for a specified probability level, attributable to random errors of various kinds, including the uncertainty introduced by sampling, and by small independent random variations in judgment, instruments, coding, transcription, and other processing. e. Evaluation of the possible effects of other relevant sources of variation, examples being differences between investigators, between instruments, between days, between areas.

f. Effect of persistent drift and conditioning of instruments and of investigators; changes in technique. g. Nonresponse and illegible or missing entries. h. Failure to select sampling units according to the procedure prescribed. i. Failure to reach and to cover sampling units that were designated in the sampling table. j. Inclusion of sampling units not designated for the sample but nevertheless covered and included in the results. k. Any other important slips and departures from the procedure prescribed. l. Comparisons with other studies, if any are relevant. In summary, a statement of statistical reliability attempts to present to readers all information that might help them to form their own opinions concerning the validity of conclusions likely to be drawn from the results. The aim of a statistical report is to protect clients from seeing merely what they would like to see; to protect them from losses that could come from misuse of the results. A further aim is to forestall unwarranted claims of accuracy that a client’s public might otherwise accept. Any printed description of the statistical procedures that refer to this participation, or any evaluation of the statistical reliability of the results, must be prepared by the statistician as part of the engagement. If a client prints the statistician’s report, it will be printed in full. The statistician has an obligation to institute audits and controls as a part of the statistical procedures for a survey or experiment, to discover any departure from the procedures prescribed. The statistician’s full disclosure and discussion of all the blemishes and blunders that took place in the survey or experiment will build for him a reputation of trust. In a legal case, this disclosure of blunders and blemishes and their possible effects on conclusions to be drawn from the data are a joy to the lawyer.

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

A statistician does not recommend to a client any specific administrative action or policy. Use of the results that come from a survey or experiment are entirely up to the client. Statisticians, if they were to make recommendations for decision, would cease to be statisticians. Actually, ways in which the results may throw light on foreseeable problems will be settled in advance, in the design, and there should be little need for a client or for anyone else to reopen a question. However, problems sometimes change character with time (as when a competitor of the client suddenly comes out with a new model), and open up considerations of statistical precision and significance of estimates that were not initially in view. The statistician may describe in a professional or scientific meeting the statistical methods that he develops in an engagement. He will not publish actual data or substantive results or other information about a client’s business without permission. In other words, the statistical methods belong to the statistician: the data to the client. A statistician may at times perform a useful function by examining and reporting on a study in which he did not participate. A professional statistician will not write an opinion on another’s procedures or inferences without adequate time for study and evaluation. SUPPLEMENTAL REMARKS 10. Necessity for the statistician to keep in touch. A statistician, when he enters into a relationship to participate in a study, accepts certain responsibilities. He does not merely draft instructions and wait to be called. The people whom he serves are not statisticians, and thus cannot always know when they are in trouble. A statistician asks questions and will probe on his own account with the help of proper statistical design, to discover for himself whether the work is proceeding according to the intent of the instructions. He must expect to revise the instructions a number of times in the early stages. He will be accessible by mail, telephone, telegraph, or in person, to answer questions and to listen to suggestions.

6421

He may, of course, arrange consultations with other statisticians on questions of theory or procedure. He may engage another statistician to take over certain duties. He may employ other skills at suitable levels to carry out independent tests or reinvestigation of certain units, to detect difficulties and departures from the prescribed procedure, or errors in transcription or calculation. It must be firmly understood, however, that consultation or assistance is in no sense a partitioning of responsibility. The obligations of a statistician to his client, once entered into, may not be shared. 11. What is an engagement? Dangers of informal advice. It may seem at first thought that statisticians ought to be willing to give to the world informally and impromptu the benefit of their knowledge and experience, without discussion or agreement concerning participation and relationships. Anyone who has received aid from a doctor of medicine who did his best without a chance to make a more thorough examination can appreciate how important the skills of a professional man can be, even under handicap. On second thought, most statisticians can recall instances in which informal advice backfired. It is the same in any professional line. A statistician who tries to be helpful and give advice under adverse circumstances is in practice and has a client, whether he intended it so or not; and he will later find himself accountable for the advice. It is important to take special precaution under these circumstances to state the basis of understanding for any statements or recommendations, and to make clear that other conditions and circumstances could well lead to different statements. 12. When do the statistician’s obligations come to a close? A statistician should make it clear that his name may be identified with a study only as long as he is active in it and accountable for it. A statistical procedure, contrary to popular parlance, is not installed. One may install new furniture, a new carpet, or a new dean, but not a statistical procedure. Experience shows that a statistical procedure deteriorates rapidly when left completely to nonprofessional administration.

6422

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

A statistician may draw up plans for a continuing study, such as for the annual inventory of materials in process in a group of manufacturing plants, or for a continuing national survey of consumers. He may nurse the job into running order, and conduct it through several performances. Experience shows, however, that if he steps out and leaves the work in nonstatistical hands, he will shortly find it to be unrecognizable. New people come on the job. They may think that they know better than their predecessor how to do the work; or they may not be aware that there ever were any rules or instructions, and make up their own. What is even worse, perhaps, is that people that have been on a job a number of years think that they know it so well that they cannot go wrong. This type of fault will be observed, for example, in a national monthly sample in which households are to be revisited a number of times; when left entirely to nonstatistical administration, it will develop faults. Some interviewers will put down their best guesses about the family, on the basis of the preceding month, without an actual interview. They will forget the exact wording of the question, or may think that they have something better. They will become lax about calling back on people not at home. Some of them may suppose that they are following literally the intent of the instructions, when in fact (as shown by a control), through a misunderstanding, they are doing something wrong. Or they may depart wilfully, thinking that they are thereby improving the design, on the supposition that the statistician did not really understand the circumstances. A common example is to substitute an averagelooking sampling unit when the sampling unit designated is obviously unusual in some respect (1). In the weighing and testing of physical product, people will in all sincerity substitute their judgment for the use of random numbers. Administration at the top will fail to rotate areas in the manner specified. Such deterioration may be predicted with confidence unless the statistician specifies statistical controls that provide detective devices and feedback. 13. The single consultation. It is wise to avoid a single consultation with a commercial

concern unless satisfactory agenda are prepared in advance and satisfactory arrangements made for absorbing advice offered. This requirement, along with an understanding that there will be a fee for a consultation, acts as a shield against a hapless conference which somebody calls in the hope that something may turn up. It also shows that statisticians, as professional people, although eager to teach and explain statistical methods, are not on the lookout for chances to demonstrate what they might be able to accomplish. Moreover, what may be intended as a single consultation often ends up with a request for a memorandum. This may be very difficult to write, especially in the absence of adequate time to study the problem. The precautions of informal advice apply here. 14. The statistician’s obligation in a muddle. Suppose that fieldwork is under way. Then the statistician discovers that the client or duly appointed representatives have disregarded the instructions for the preparation of the frame, or for the selection of the sample, or that the fieldwork seems to be falling apart. ‘‘We went ahead and did so and so before your letter came, confident that you would approve,’’ is a violation of relationship. That the statistician may approve the deed done does not mitigate the violation. If it appears to the statistician that there is no chance that the study will yield results that he could take professional responsibility for, this fact must be made clear to the client, at a sufficiently high management level. It is a good idea for the statistician to explain at the outset that such situations, while extreme, have been known. Statisticians should do all in their power to help clients to avoid such a catastrophe. There is always the possibility that a statistician may be partly to blame for not being sufficiently clear nor firm at the outset concerning his principles of participation, or for not being on hand at the right time to ask questions and to keep apprised of what is happening. Unfortunate circumstances may teach the statistician a lesson on participation. 15. Assistance in interpretation of non-probability samples. It may be humiliating, but statisticians must face the fact

PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE

that many accepted laws of science have come from theory and experimentation without benefit of formal statistical design. Vaccination for prevention of smallpox is one; John Snow’s discovery of the source of cholera in London is another (6). So is the law F = ma in physics; also Hooke’s law, Boyle’s law, Mendel’s findings, Keppler’s laws, Darwin’s theory of evolution, the Stefan-Boltzmann law of radiation (first empirical, later established by physical theory). All this only means, as everyone knows, that there may well be a wealth of information in a nonprobability sample. Perhaps the main contribution that the statistician can make in a nonprobability sample is to advise the experimenter against conclusions based on meaningless statistical calculations. The expert in the subject matter must take the responsibility for the effects of selectivity and confounding∗ of treatments. The statistician may make a positive contribution by encouraging the expert in the subject matter to draw whatever conclusions he believes to be warranted, but to do so over his own signature, and not to attribute conclusions to statistical calculations or to the kind of help provided by a statistician. 16. A statistician will not agree to use of his name as advisor to a study, nor as a member of an advisory committee, unless this service carries with it explicit responsibilities for certain prescribed phases of the study. 17. A statistician may accept engagements from competitive firms. His aim is not to concentrate on the welfare of a particular client, but to raise the level of service of his profession. 18. A statistician will prescribe in every engagement whatever methods known to him seem to be most efficient and feasible under the circumstances. Thus, he may prescribe for firms that are competitive methods that are similar or even identical word for word in part or in entirety. Put another way, no client has a proprietary right in any procedures or techniques that a statistician prescribes. 19. A statistician will, to the best of his ability, at his request, lend technical assistance to another statistician. In rendering this assistance, he may provide copies of procedures that he has used, along with

6423

whatever modification seems advisable. He will not, in doing this, use confidential data. REFERENCES 1. Bureau of the Census (1954). Measurement of Employment and Unemployment. Report of Special Advisory Committee on Employment Statistics, Washington, D.C. 2. Bureau of the Census (1964). Evaluation and research program of the censuses of population and housing. 1960 Series ER60. No. 1. 3. Bureau of the Census (1963). The Current Population Survey and Re-interview Program. Technical Paper No. 6. 4. Center for Advanced Engineering Study. (1983). Management for Quality, Productivity, and Competitive Position. Massachusetts Institute of Technology, Cambridge, Mass. 5. Deming, W. E. (1960). Sample Design in Business Research. Wiley, New York, Chap. 3. 6. Hill, A. B. (1953). Observation and experiment. N. Engl. J. Med., 248, 995–1001. 7. Stephan, F. F. (1936). Practical problems of sampling procedure. Amer. Sociol. Rev., 1, 569–580.

FURTHER READING See the following works, as well as the references just given, for more information on the topic of principles of professional statistical practice. Brown, T. H. (1952). The statistician and his conscience. Amer. Statist., 6(1), 14–18. Burgess, R. W. (1947). Do we need a ‘‘Bureau of Standards’’ for statistics? J. Marketing, 11, 281–282. Chambers, S. P. (1965). Statistics and intellectual integrity. J. R. Statist. Soc. A, 128, 1–16. Court, A. T. (1952). Standards of statistical conduct in business and in government. Amer. Statist., 6, 6–14. Deming, W. E. (1954). On the presentation of the results of samples as legal evidence. J. Amer. Statist. Ass., 49, 814–825. Deming, W. E. (1954). On the Contributions of Standards of Sampling to Legal Evidence and Accounting. Current Business Studies, Society of Business Advisory Professions, Graduate School of Business Administration, New York University, New York. Eisenhart, C. (1947). The role of a statistical consultant in a research organization. Proc. Int. Statist. Conf., 3, 309–313.

6424

PRIOR DENSITY

Freeman, W. W. K. (1952). Discussion of Theodore Brown’s paper [see Brown, 1952]. Amer. Statist., 6(1), 18–20. Freeman, W. W. K. (1963). Training of statisticians in diplomacy to maintain their integrity. Amer. Statist., 17(5), 16–20. Gordon, R. A. et al. (1962). Measuring Employment and Unemployment Statistics. President’s Committee to Appraise Employment and Unemployment Statistics. Superintendent of Documents, Washington, D.C. Hansen, M. H. (1952). Statistical standards and the Census. Amer. Statist., 6(1), 7–10. Hotelling, H. et al. (1948). The teaching of statistics (a report of the Institute of Mathematical Statistics Committee on the teaching of statistics). Ann. Math. Statist., 19, 95–115. Jensen, A. et al. (1948). The life and works of A. K. Erlang. Trans. Danish Acad. Tech. Sci., No. 2 (Copenhagen). International Statistical Institute (1985). Declaration on Professional Ethics, website www.cbs.nl/isi/ethics.htm Molina, E. C. (1913). Computation formula for the probability of an event happening at least c times in n trials. Amer. Math. Monthly, 20, 190–192. Molina, E. C. (1925). The theory of probability and some applications to engineering problems. J. Amer. Inst. Electr. Eng., 44, 1–6. Morton, J. E. (1952). Standards of statistical conduct in business and government. Amer. Statist., 6(1), 6–7. Shewhart, W. A. (1931). Economic Control of Quality of Manufactured Product. D. Van Nostrand, New York. Shewhart, W. A. (1939). Statistical Method from the Viewpoint of Quality Control. The Graduate School, Department of Agriculture, Washington, D.C. Shewhart, W. A. (1958). Nature and origin of standards of quality. Bell Syst. Tech. J., 37, 1–22. Society of Business Advisory Professions (1957). Report of Committee on Standards of Probability Sampling for Legal Evidence. Graduate School of Business Administration, New York University, New York. Zirkle, C. (1954). Citation of fraudulent data. Science, 120, 189–190. See also CONSULTING, STATISTICAL.

W.E. DEMING

PRIOR DENSITY. See PRIOR DISTRIBUTIONS

PRIOR DISTRIBUTIONS Prior distributions are probability distributions or probability density functions (PDFs) that summarize information about a random variable or parameter known or assumed at a given point in time, prior to obtaining further information from empirical data, and so on. The use of the designation ‘‘prior distribution’’ occurs almost entirely within the context of statistical methodology generally labeled ‘‘Bayesian inference’’∗ . Bayes’ theorem∗ and a 1763 paper by Thomas Bayes [2] are generally cited as the beginnings of this methodology. However, Stigler [12] cites evidence that the littleknown English mathematician Saunderson may have been the true builder of the foundations of what is now termed Bayesian inference. Suppose that Y and X are two random variables with PDFs fY (y) and fX (x) (Y and X may be discrete or continuous). Denoting the joint distribution of Y and X by fX,Y (x, y) and conditional distributions of Y given X and X given Y by fY|X (y|x) and fX|Y (x|y), respectively, it is known by definition that fY|X (y|x)fX (x) = fX,Y (x, y) = fX|Y (x|y)fY (y). Then, for example, fX|Y (x|y) =

fY|X (y|x)fX (x) . fY (y)

This result is one representation of the socalled ‘‘Bayes’ theorem,’’ true by definition and not an object of debate among statisticians. fX (x) tells us what is known about X before or prior to the observation of Y, and is thus termed the prior distribution of X. Alternatively, fX|Y (x|y) tells us what is known about X after or posterior to the observation of Y (see POSTERIOR DISTRIBUTIONS). The representation of Bayes’ theorem that is associated with Bayesian inference and the typical reference to prior distribution is f|Y (θ |y) =

fY| (y|θ)f (θ ) , fY (y)

PRIOR DISTRIBUTIONS

where Y = (Y1 , Y2 , . . . , Yn ) is a vector of n independent Yi representing sample observations, each with probability distribution fYi | (yi |θ) dependent on the value of m parameters in the vector . For example, for normally distributed Yi ,     1 1 yi − µ 2 fYi | (yi |θ) = √ , exp − 2 σ 2π σ −∞ < yi < ∞, where  = (µ, σ ), the mean  and standard deviation, and fY| (y|θ) = fYi | (yi |θ). The focus of this representation is inference about the typically unknown parameter vector , where the prior distribution f (θ) is modified by the empirical data y to form the posterior distribution f|Y (θ|y). The prior distribution f (θ) is an object of debate among statisticians. Classical statisticians perceive parameters as fixed, even if unknown, and reject the idea of any ‘‘distribution’’ for . They also complain that even if one accepts the concept of such a distribution, its exact nature is almost inevitably subjective, based only on some vague degree-of-belief∗ probability concept rather than the scientifically sound relative-frequency-in-repeatedexperiments probability concept. Alternatively, Bayesian statisticians are comfortable using prior distributions for unknown parameters, even if there is considerable subjectivity involved. There have been attempts to reduce subjectivity by developing procedures for estimating prior distributions. Martz and Waller [9] discuss associating distribution parameters with characteristics of the distribution that one may feel relatively well known, such as chosen percentiles, the distribution mean, and so on. O’Bryan and Walter [10] and Walter [13] discuss specific prior distribution estimation procedures. Some commonly encountered categorizations of prior distributions are: Improper Prior: Distribution does not sum or integrate to 1. Otherwise, the prior is proper. Improper priors are often used as approximations to proper prior distributions which are nearly flat in the range of  indicated to be relevant by the empirical data y. Akaike [1]

6425

and Eaton [5] discuss the interpretation and evaluation of improper priors. Reference Prior: A prior distribution used as a standard in a particular area of study [3, p. 22]. Locally Uniform Prior: Typically refers to a prior density function for a continuous parameter which is at least approximately constant over a limited relevant parameter range, thus expressing equal probability of the parameter falling in any equal-length subrange. Noninformative Prior: A prior distribution which expresses little knowledge about the parameter relative to the information expected in a size n sample of empirical data. Locally uniform priors are commonly used as noninformative priors. There is a problem, however, in selecting the metric (i.e.,  itself, log , −1 , etc.) in which the prior f (θ ) is locally uniform to best reflect true noninformativeness. Box and Tiao [3, p. 25] discuss this in depth. Jeffrey’s rule [7] is a rule guiding the choice of noninformative prior distributions. See JEFFREYS’ PRIOR DISTRIBUTION. Conjugate Prior: A prior distribution f (θ ) which combines fY| (y|θ) to produce a posterior distribution f|Y (θ |y) which is in the same family as the prior distribution; for example, both are normal distributions. See Martz and Waller [9, p. 226], De Groot [4, p. 164], Raiffa and Schlaifer [11, p. 53], and CONJUGATE FAMILIES OF DISTRIBUTIONS. Exchangeable Prior: A prior distribution f (θ ) which is not altered by any permutation in the multiparameter . Exchangeability∗ has been examined in the context of parameters from multiple populations and multinomial parameters of a single population. See Lindley and Smith [8] and Eaves [6]. REFERENCES 1. Akaike, H. (1980). J. R. Statist. Soc. B, 42, 46–52. (On the interpretation of improper prior distributions as limits of data-dependent proper prior distributions.) 2. Bayes, T. (1958). Biometrika, 45, 298–315. (Reprint of the famous 1763 paper generally

6426

3.

4. 5.

6.

7. 8.

9.

10. 11.

12.

13.

PRIORITY QUEUE

credited with giving birth to Bayesian statistical inference.) Box, G. E. P. and Tiao, G. C. (1973). Bayesian Inference in Statistical Analysis. AddisonWesley, Reading, Mass. (An excellent general statistical reference on Bayesian inference.) DeGroot, M. H. (1970). Optimal Statistical Decisions. McGraw-Hill, New York. Eaton, M. L. (1982). In Statistical Decision Theory and Related Topics III, Vol. 1, S. Gupta and J. Berger, eds. Academic Press, New York, pp. 1329–1352. (Evaluating improper prior distributions.) Eaves, D. M. (1980). J. R. Statist. Soc. B, 42, 88–93. (Good reference list for work with exchangeable priors.) Jeffreys, H. (1961). Theory of Probability, 3rd ed. Clarendon Press, Oxford, England. Lindley, D. V. and Smith, A. F. M. (1972). J. R. Statist. Soc. B, 34, 1–41. (Exchangeable priors in the context of the usual linear statistical model.) Martz, H. F. and Waller, R. A. (1982). Bayesian Reliability Analysis. Wiley, New York. (Chapters 5 and 6 contain good general knowledge about Bayesian inference.) O’Bryan, T. and Walter, G. (1979). Sankhya A, 41, 95–108. Raiffa, H. and Schlaifer, R. (1961). Applied Statistical Decision Theory. Harvard University Press, Cambridge, Mass. Stigler, S. M. (1983). Amer. Statist., 37, 290–296. (A witty search into who really discovered Bayes’ theorem.) Walter, G. (1981). Sankhya A, 43, 228–245.

BIBLIOGRAPHY See the following works, as well as the references just given, for more information on the topic of prior distributions. Lindley, D. V. (1965). Introduction to Probability and Statistics from a Bayesian Viewpoint, Part 1: Probability; Part 2: Inference, Cambridge University Press, Cambridge, England. Savage, L. J. (1954). The Foundations of Statistics. Wiley, New York. (General reference with philosophical basis for Bayesian statistical inference.) See also BAYESIAN INFERENCE; BAYES’ THEOREM; BELIEF, DEGREES OF; CONJUGATE FAMILIES OF DISTRIBUTIONS; POSTERIOR DISTRIBUTIONS; POSTERIOR PROBABILITIES; and PRIOR PROBABILITIES.

DAVID J. PACK

PRIORITY QUEUE As a general background regarding ideas and notation, we refer the reader to QUEUEING THEORY. The theory for priority queueing systems is the branch of queueing theory dealing with systems where high-priority customers get faster through the system at the expense of others. The basic reference seems still to be ref. 4, giving a series of different models, with a variety of priority disciplines, and a solid mathematical treatment. In ref. 5, priority queueing systems of special interest in computer applications are considered. Since the applicability of queueing theory has often been questioned, we will here give reviews of three papers having specific applications in mind. We are not claiming that these are either the best or the most general in the area. One should also remember that even a complex queueing system is more sympathetic to mathematical modeling than is a series of other real-life systems. Hence queueing theory has been and will still be an experimental area to develop tools which are useful in other areas of applied probability. CLASSICAL PRIORITY QUEUEING MODELS Assume that the customers are divided into p different priority classes each having a priority index 1, 2, . . . , p. The index 1 corresponds to the highest-priority class and p corresponds to the lowest. For simplicity we assume a single server and an infinite waiting-room capacity. Within each class we have a first-in-first-out (FIFO) queueing discipline. Denoting the interarrival-time distribution and the service-time distribution within the ith class by the symbols Ai and Bi , respectively, we can use the notation A1 / · · · /Ap /B1 / · · · /Bp /1 for this class of models. However, we have to specify the queueing discipline between customers from different priority classes. Compare the ith and jth classes, where 1
i < j
p. In the queue all customers from the ith class are standing in front of all customers from the jth class and will hence be served first. If, on the other hand, a customer from the jth class is being served on the arrival of a customer from the

PRIORITY QUEUE

ith class, there are different alternatives. The priority discipline is preemptive∗ if the current service is immediately interrupted and service is started for the arriving customer. It is nonpreemptive if service is continued to completion, and it is discretionary if the server may use his or her discretion to decide which of the two former strategies to use in each case. For instance, he or she may use the nonpreemptive discipline if the amount of service received exceeds a certain level [4], or if remaining service time is sufficiently small [3]. APPOINTMENT SYSTEM Consider the GI/M/1 queueing model with infinite waiting-room capacity. The customer arriving at t = 0 will find k − 1 customers waiting. The latter customers belong to the second priority class, whereas the ones arriving in [0, ∞) belong to the first. The priority discipline is nonpreemptive, whereas within each class we have a FIFO queueing discipline. As a motivation for studying the present model, consider the following specialization of the arrival pattern above which is realistic when, for example, doctors, dentists, or lawyers are consulted. Let the intervals between possible arrivals have fixed length 1/λ and let the probability of a customer not turning up be 1 − p. Customers turn up or not independently of each other. The number of intervals of length 1/λ between the arrivals of two customers are then geometrically distributed with parameter p. The k − 1 customers in the second priority class do not have an appointment but are allowed to queue up, for instance, either before the office is opened in the morning or before it is reopened after the lunch break. One is now interested in: 1. Waiting times for customers from both priority classes not to be too long. 2. The initial busy period (starting with k customers in the system) not to be too short. Small values of k will satisfy 1, whereas large values satisfy 2.

6427

In ref. 1 the transient waiting times for customers belonging to both priority classes are arrived at for the general model. Using these results on the special arrival pattern above, optimal values for k have been tabulated for various values of p, λ and service intensity. COMPUTER TIME SHARING In ref. 2 a modification of the so-called roundrobin priority discipline is treated in an M/M/ 1 queueing model. Each program receives a quantum q of service at a time from a single central processor. If this completes its service requirement, it leaves the system. If not, and there is a new arrival during the service of the quantum, it is given an additional quantum. Otherwise, the program joins the end of the queue to await its next turn. The model is analyzed under the assumption of a constant, nonzero overhead when the processor swaps one program for another. Obviously, during periods of high arrival rates, this algorithm has the effect of reducing the system’s swapping activities. On the other hand, during periods of low arrival rates the discipline is similar to the conventional round robin, which automatically gives priority to programs with lesser service time requirements. Reference 2 arrives at expressions for the mean waiting time in queue as a function of the quanta required and for the mean system cost due to waiting. Numerical comparisons with the conventional round-robin discipline are performed. TELEPHONE EXCHANGE In ref. 6 a telephone exchange is considered, handling the calls to and from a minor group of subscribers. Let the latter group belong to the second priority class, whereas people from the rest of the world belong to the first class. The Electronic Selector Bar Operator (ESBO) serves customers from both classes. The model one wished to be able to analyse is a modified version of GI/G/M/G/ 1 with a nonpreemptive priority discipline. However, customers of the higher priority are just

6428

PRIOR PROBABILITIES

allowed to wait a fixed length of time before they hear the busy signal and are lost. This is not the case for customers of lower priority. Second, whereas we have a FIFO queueing discipline within the first class, it is random within the second (i.e., all customers waiting have equal chance of being served next). Unfortunately, there seems no way of arriving at the stationary waiting-time distributions for customers of the two classes by using queueing theory of today. As a first approximation ref. 6 starts out from an M1 /G/M2 /G/1 model where Laplace transforms of the distributions above are well known. Furthermore, in this case it is possible to get a look behind the ‘‘Laplace curtain.’’ REFERENCES 1. Dalen, G. and Natvig, B. (1980). J. Appl. Prob., 17, 227–234. 2. Heacox, H. and Purdom, P. (1972). J. ACM, 19, 70–91. 3. Hokstad, P. (1978). INFOR, 16, 158–170. 4. Jaiswal, N. K. (1968). Priority Queues. Academic Press, New York. 5. Kleinrock, L. (1976). Queueing Systems, Vol. 2: Computer Applications. Wiley, New York. 6. Natvig, B. (1977). On the Waiting-Time for a Priority Queueing Model. Tech. Rep. No. 3, Dept. of Mathematical Statistics, University of Trondheim, Trondheim, Norway. Prepared for the Norwegian Telecommunications Administration Research Establishments. See also PREEMPTIVE DISCIPLINE and QUEUEING THEORY.

BENT NATVIG

PRIOR PROBABILITIES Prior probabilities are probabilities that are applicable at a point in time which is prior to one obtaining additional information provided by empirical studies, experiments, surveys, and so on, yet to be executed. In the context of a particular random variable or unknown parameter, prior probabilities follow directly from the choice of a prior distribution∗ , as the assumption of a probability distribution or probability density function in any statistical endeavor automatically leads to the ability to determine

probabilities of various events of interest. See PRIOR DISTRIBUTIONS for a full discussion and references. Suppose for the sake of illustration that you are concerned with the probability of the event E that a particular person known to you only by name is a college graduate. Knowing nothing else, you might reason that about 1 in 20 people in the general population is a college graduate, and thus your probability for E is Pr[E] = 0.05. This is a prior probability. It is based on your knowledge at this time and is not conditioned on any other event (explicitly). It is a probability that you would use prior to getting any other information about the person. If, for example, you knew the person read Newsweek or a similar news magazine, you might feel the chances of the person being a college graduate were more than the 1 in 20 in the general population. The illustration above suggests an element that is frequently present in discussions of prior probabilities—subjective probability assessment. Pr[E] = 0.05 is a statement of degree of belief∗ , not an objective statement based on repeated experiments or some scientific fact. Prior probabilities may have some objectivity, particularly when they are the posterior probabilities from previous empirical observations (see POSTERIOR DISTRIBUTIONS), but in most cases there is inevitably a significant subjective component. The discipline called applied statistical decision theory∗ as discussed by Raiffa and Schlaifer [11] and Raiffa [10] is one that particularly illustrates the utilization of subjective prior probabilities. There is a substantial literature on the quantification of degree-of-belief information in a prior probability. Martz and Waller [8] contains a good bibliographic overview. Hampton et al. [3] and Hogarth [4] are good general technical references. The issue of encoding prior probabilities from one expert source is dealt with in Lichtenstein et al. [6] and Lindley et al. [7]. One may also be concerned with forming the prior probabilities from a number of expert sources into a consensus probability, as discussed by DeGroot [2] and Morris [9]. Adams [1] and Kazakos [5] provide other perspectives on the

PROBABILISTIC NUMBER THEORY

general problem of quantifying prior probabilities. REFERENCES 1. Adams, E. W. (1976). In Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. 1, W. L. Harper and C. A. Hooker, eds. D. Reidel, Boston, pp. 1–22. 2. DeGroot, M. H. (1974). J. Amer. Statist. Ass., 69, 118–121. (Consensus prior probabilities via prior distributions which are mixtures of individual expert priors.) 3. Hampton, J. M., Moore, P. G., and Thomas, H. (1973). J. R. Statist. Soc. A, 136, 21–42. (General reference on the measurement of subjective probability.) 4. Hogarth, R. M. (1975). J. Amer. Statist. Ass., 70, 271–289. (General reference on the measurement of subjective probability.) 5. Kazakos, D. (1977). IEEE Trans. Inf. Theory, 23, 203–210. (Estimation of prior probabilities using a mixture.) 6. Lichtenstein, S., Fischhoff, B., and Phillips, L. D. (1977). In Decision Making and Change in Human Affairs, H. Jungerman and G. de Zeeuv, eds. D. Reidel, Dordrecht, The Netherlands, pp. 275–324. (Encoding prior probabilities.) 7. Lindley, D. V., Tversky, A., and Brown, R. V. (1979). J. R. Statist. Soc. A, 142, 146–180. (Reconciling inconsistent prior probability assessments.) 8. Martz, H. F. and Waller, R. A. (1982). Bayesian Reliability Analysis. Wiley, New York. [Section 6.4 gives an overview of prior probability (distribution) assessment.] 9. Morris, P. A. (1977). Manag. Sci., 23, 679–693. (Consensus prior probabilities.) 10. Raiffa, H. (1968). Decision Analysis, Introductory Lectures on Choices under Uncertainty. Addison-Wesley, Reading, Mass. (Introduction to simple statistical decision theory.) 11. Raiffa, H. and Schlaifer, R. (1961). Applied Statistical Decision Theory. Harvard University Press, Cambridge, Mass. See also BELIEF, DEGREES OF; POSTERIOR DISTRIBUTIONS; POSTERIOR PROBABILITIES; and PRIOR DISTRIBUTIONS.

DAVID J. PACK

PROBABILISTIC GRAMMAR. See LINGUISTICS, STATISTICS IN

6429

PROBABILISTIC NUMBER THEORY Some questions and results of number theory can easily be expressed in probabilistic terms; for others, proofs can be simplified by probabilistic arguments. Probabilistic number theory can therefore be defined as a branch of number theory in which questions are of interest to number theorists, but the proofs utilize techniques of probability theory. There are two major subfields of probabilistic number theory: (a) distribution of values of arithmetical functions, and (b) metric (measure theoretic) results concerning expansions of real numbers. These are discussed in detail in the first two sections. The present article is special in that it is somewhat outside the expertise of most readers. Therefore, a detailed survey of the subject is not aimed at; rather, an expository introduction is given in which the major trends, but not the fine points, are presented. DISTRIBUTION OF VALUES OF ARITHMETICAL FUNCTIONS A function f (n) on the positive integers n = 1, 2, . . ., is called an arithmetical function. In other words, an arithmetical function is a sequence in which the emphasis on the variable (index) n is its arithmetical structure rather than its magnitude. Let us start with an example. Let f (n) be the number of all (distinct) prime divisors of n. Hence, to determine the value of f (n), one first has to express n as the product of powers of primes, and then the number of factors in this product is f (n). One can hardly find any regularity of this f (n), because it is frequently equal to 1 (at prime numbers), but it also exceeds any prescribed value infinitely many times (e.g., when n is a factorial). Here is where probabilistic number theory comes in: One can prove, by an appeal to the Chebyshev inequality∗ , that, for ‘‘most numbers n,’’ f (n) is approximately loglog n. More precisely, if A(N) denotes the number of positive integers n
N such that |f (n) − log log n| < (log log n)3/4 , then A(N)/N → 1 as N → +∞. To prove this claim, let us introduce the probability spaces

6430

PROBABILISTIC NUMBER THEORY

where p < q are prime numbers and [y] signifies the integer part of y, we have

that log log n ∼ log log N for all n satisfying √ N
n
N, say, the Chebyshev inequality now implies our claim on A(N) concerning the closeness of f (n) to log log n. This computation represents well the essence of probabilistic number theory. The question, namely the behavior of f (n), originates from number theory; then the basic idea of applying the Chebyshev inequality is a tool of probability theory, but finally, in computing E(f (n)), the application of the prime number theorem is unavoidable. This procedure is generally the case in connection with number-theoretic functions. Only the basic ideas come from probability theory; in the actual computations number-theoretic results are frequently utilized. Let us now consider a whole family of functions f (n) satisfying

PN (ep (n) = 1) = 1/p + O(1/N)

f (nm) = f (n) + f (m)

(N , AN , PN ), where N = {1, 2, . . . , N}, AN is the set of all subsets of N , and PN is the uniform distribution on N . Now the function fN (n) = f (n), 1
n
N, is a random variable,
which can be represented as f (n) = ep (n), where the summation is over primes p
N, and ep (n) = 1 if p divides n and ep (n) = 0 otherwise. The expectation and the variance of f (n) can easily be computed by means of this representation. Namely, because PN (ep (n) = 1) = [N/p]/N and PN (ep (n) = eq (n) = 1) = [N/(pq)]/N,

and PN (ep (n) = eq (n) = 1) = 1/(pq) + O(1/N). Thus E(f (n)) =



1/p + O(π (N)/N),

p
N

where π (N) is the number of primes not exceeding N. However, from number theory we know that π (N) ∼ N/ log N (the so-called prime number theorem), which yields E(f (n)) =

(1)

for all n and m which are relatively prime (i.e., they do not have any common divisor other than 1). Functions satisfying (1) are additive arithmetical functions. If, in addition, the additive function f (n) satisfies f (pk ) = f (p) for all primes p and integers k  1, then f (n) is strongly additive. In what follows there is no difference between additive and strongly additive functions. However, computations are much simpler with strongly additive functions, so we restrict ourselves to these. Now, for such functions f (n), (1) implies that

  N  π (m) − π (m − 1) 1 +O m log N

f (n) =



f (p)ep (n),

(2)

m=1 N 



1 1 π (m) = − m m+1 m=1   1 +O log N =

N  m=1



1 + O(1) (m + 1) log m

= log log N + O(1). The variance of f (n) can be computed similarly; it also equals log log N. Finally, observing that log log N is so slowly increasing

where summation is again over all primes p (when n is restricted to N , then p
N). One easily gets from (2) that E(f (n)) ∼ AN and V(f (n)) ∼ B2N , where AN =

 f (p) p

p
N

and

B2N =

 f 2 (p) . (3) p

p
N

Repeating the argument with the Chebyshev inequality, we get that f (n) ∼ AN for ‘‘most n’’ whenever BN → +∞ with N. More accurate results can also be deduced from (2). Because the indicator variables ep (n)

PROBABILISTIC NUMBER THEORY

are almost independent, that is, for arbitrary primes p1 < p2 < · · · < pk , PN (ep1 (n) = ep2 (n) = · · · = epk (n) = 1) = 1/(p1 p2 · · · pk ) + O(1/N), (4) the sums in (2) and (3) suggest that the distribution of (f (n) − AN )/BN converges weakly under conditions similar to sums of independent random variables. Although most results for sums of independent random variables are indeed valid for strongly additive functions, unfortunately these cannot be deduced directly from probability theory because the error term O(1/N) in (4) is more troublesome than it looks at first. Namely, if √ only two primes in (4) are around N, the major term is already smaller than the error term, and it is not easy to show that one can limit the primes in (2) to remain below powers of N. This is why one is forced to consider  f (p)ep (n) fr (n) = p
r

in place of f (n), where r = r(N) is some slowly increasing function of N, and then one uses probability theory to determine the asymptotic behavior of fr (n), and number theory (so-called sieve methods) to show that the difference f (n) − fr (n) is small compared with BN . This line of attack proves very successful. A few basic results will be quoted as we now turn to the history of the subject. The references are far from complete, even when one adds all references in the survey papers and books listed later. The probabilistic theory of arithmetic functions started with the paper of ` [43], who gave a new proof, essentially Turan through the Chebyshev inequality (although without reference to probability theory), for the asymptotic relation between the number of prime divisors and log log n. The result itself had been known in number theory, but it was obtained by analytic methods. Both P. Erd¨os and M. Kac were influenced ´ by Turan’s proof. They, first independently, and later jointly, contributed at the early stages the most significant results of the distribution theory of additive functions. It was Kac who approached the subject from

6431

a purely probabilistic point of view by recognizing in the representation at (2) the sum of almost independent random variables. Kac’s early work is well set out in his popular monograph [24]. Erd¨os [7] and Erd¨os and Wintner [10] found necessary and sufficient conditions for the weak convergence of PN (f (n) < x) for arbitrary additive functions f (n). Erd¨os and Kac [9] established that if the sequence f (p) is bounded (over primes p), then, for the strongly additive function f (n), PN (f (n) < AN + BN x) is asymptotically standard normal, where AN and BN are defined at (3). It turned out that the boundedness of f (p) is not necessary for asymptotic normality∗ . In a series of papers, J. Kubilius developed a probabilistic theory of distributions for the truncated functions fr (n), in which his only requirement on r is that r → +∞ with N and log r/log N should converge to zero as N → +∞. If in addition Br /BN → 1 and BN → +∞ as N → +∞, then f (n) and fr (n) have the same asymptotic laws. Later, Kubilius published his work in a book which has had several editions; we quote here the English language edition [30]. His contribution to the theory of distributions of arithmetical functions is immeasurable because he not only published significant papers and books on the subject, but established a school at Vilnius which produced a large number of scientists working on the subject and also made a considerable influence on both individual scientists all over the world and the literature. Some of the work of this school is presented in the survey papers of Galambos [11,14], Kubilius [31], Manstavicius [34], and Schwarz [41], and in the books of Kubilius [30] and Elliott [6]. In the Kubilius model, to show that (f (n) − fr (n))/BN converges to zero (‘‘in probability’’), quite deep number theoretic results are employed. This induced investigations aimed at reobtaining at least some of the results of Kubilius by purely probabilistic arguments. Several models have been set up, which to various degrees have achieved the desired results, but none of them would have come close to the generality of the Kubilius model. However, through the models of Novoselov [35], Paul [36], Galambos [12], and Philipp [37], interesting probabilistic proofs are given for one result or another in the distribution theory of additive

6432

PROBABILISTIC NUMBER THEORY

functions. The model of Galambos has the added advantage that it presents a unified treatment of additive functions when the variable is not limited to the set of successive integers. Such investigations are extensive ` in the literature; see Kubilius [30], Katai [26,27], and Indlekofer [21]. For the truncated function fr (n) Kubilius was able to use the theory of sums of independent random variables. However, for extending his results to additive functions for which a truncation is not possible, other methods are necessary. The method of characteristic functions∗ proved fruitful, although it required new deep number-theoretic investigations. Namely, since PN is the uniform distribution over a finite set, the characteristic function of an arithmetical function f (n) is the arithmetical mean of the function g(n) = exp(itf (n)). Now, if f (n) is additive, then, in view of (1), g(n) satisfies the equation g(mn) = g(m)g(n)

(5)

for all m and n which are relatively prime. A function satisfying (5) is multiplicative. Thus asymptotic mean value theorems were needed for multiplicative functions whose absolute value is bounded by 1. Such results were established by Delange [5] and ` [18], who employed tools of analytic Halasz number theory (see also R´enyi [40]). These mean value theorems gave new life to the distribution theory of arithmetical functions. In a series of papers, Elliott, C. Ryavec, ` Delange, Halasz, and the members of the Vilnius school brought the theory of additive functions to a very advanced level. The twovolume set of books by Elliott [6] is based on this new development. The distribution theory of multiplicative functions has also been studied in detail. Because multiplicative functions which can take both positive and negative values cannot be related to real-valued additive functions, their distribution theory requires a completely different approach. The combined result of Bakstys [1] and Galambos [13] gives necessary and sufficient conditions for the weak convergence of PN (g(n) < x) for multiplicative functions g(n), and the weak convergence of the normalized multiplicative function (g(n) − CN )/DN is characterized in Levin et al. [33].

For arithmetical functions which are neither additive nor multiplicative, approximation methods have been developed: by additive functions in Galambos [15] and Erd¨os and Galambos [8], and by so-called Ramanujan sums in a series of papers by W. Schwarz and his collaborators; see the survey [41]. The distribution theory of arithmetical functions has several applications within number theory. For example, for estimating certain sums, see De Koninck and Ivic [4], and for its role in characterizing log n among all additive functions, see Indlekofer [22]. For how much joy the study of this theory can give, one should read the recollections of Kac [25]. METRIC THEORY OF EXPANSIONS OF REAL NUMBERS When a sequence of ‘‘digits’’ is assigned to real numbers by a specific algorithm, we speak of an expansion. The best known expansion is the decimal, or more generally, the q-adic expansions, in which, when q is an integer, the digits are independent and identically distributed random variables with respect to Lebesgue measure. Hence, although no regularity of the digits of irrational numbers seems to hold, the law of large numbers implies that these digits are quite regular for almost all x. Other theorems of probability theory are directly applicable as well. The situation is completely different for most other series expansions. As an example, let us look at the so-called Engel series. For a given real number 0 < x
1, the ‘‘digits’’ dj are defined by the following algorithm: Every dj is an integer such that 1/dj < xj
1/(dj − 1), where x = x1 and xj+1 = dj xj − 1. This algorithm leads to the series representation x = 1/d1 + 1/(d1 d2 ) + 1/(d1 d2 d3 ) + · · · , where 2
d1
d2
d3
· · · . The digits dj are strongly dependent in view of their monotonicity. However, the ratios

PROBABILISTIC NUMBER THEORY

dj /(dj−1 − 1) can well be approximated by independent and identically distributed random variables, which can be utilized for discovering general properties of the sequence dj , valid for almost all x. Similar situations arise in connection with most series expansions. That is, the digits are dependent but after some transformation an approximation by independent random variables becomes possible. This is not obvious, though, in most instances, and the challenge in these problems usually is probabilistic: to extend known theorems of probability theory to some weakly dependent random variables. These investigations did indeed contribute to probability theory itself. Most results on series expansions are contained in the books of Galambos [17] and Vervaat [44], although for some expansions there are newer results as well (see, e.g., Deheuvels [3] concerning the Engel series). Among the expansions which do not result in a series most attention was paid to continued fraction expansions. This is obtained by repeated application of the transformation Tx = x − [1/x], where, once again, [y] signifies the integer part of y. As far back as in the nineteenth century, Gauss recognized that, if instead of length (i.e., Lebesgue measure), an absolutely continuous measure is assigned to the unit interval whose derivative is 1/{(log 2)(1 + x)}, then the sequence an , defined by the algorithm aj = [1/xj ], where x = x1 and Txj = xj+1 , is stationary. More recent studies revealed that this sequence is stationary mixing, which fact induced intensive studies of mixing sequences of random variables. This benefited both probability theory and number theory; see Ibragimov and Linnik [20], Billingsley [2], and Philipp [37]. For a more classical approach, see Khintchine [29]. On the speed of convergence of the distribution of Txn , see Kuzmin [32] and ¨ [42]. Galambos [16], Philipp [38], and Szusz Iosifescu [23] deal with the stochastic behavior of the largest of the digits an . The work of R´enyi [39] shows the connection between series expansions and continued fractions. We conclude by noting that there are several works in probabilistic number theory which do not belong to either topic covered in the preceding two sections. These are, however, individual problems rather than a

6433

part of a systematic study. We mention two ` of these. Katai [28] is probably the purest among all papers in probabilistic number theory in that he shows that for certain sequences of integers, the number of divisors √ of n in the neighborhood of n is normally distributed, in which argument no number theory is applied. The other area we wish to mention is the proof of the existence of certain types of numbers by showing that their probability is positive. Although most of the results are due to Erd¨os, the most convenient source for consultation is Halberstam and Roth [19]. REFERENCES 1. Bakstys, A. (1968). Litovsk. Mat. Sb., 8, 5–20. 2. Billingsley, P. (1965). Ergodic Theory and Information. Wiley, New York. 3. Deheuvels, P. (1982). C. R. Acad. Sci. Paris, 295, 21–24. 4. De Koninck, J. -M. and Ivic, A. (1980). Topics in Arithmetical Functions. North-Holland, Amsterdam. ´ 5. Delange, H. (1961). Ann. Sci. Ecole Norm. Sup., 78, 273–304. 6. Elliott, P. D. T. A. (1979/1980). Probabilistic Number Theory, Vols. I and II. SpringerVerlag, Berlin. 7. Erd¨os, P. (1938). J. Lond. Math. Soc., 13, 119–127. 8. Erd¨os, P. and Galambos, J. (1974). Proc. Amer. Math. Soc., 46, 1–8. 9. Erd¨os, P. and Kac, M. (1940). Amer. J. Math., 62, 738–742. 10. Erd¨os, P. and Wintner, A. (1939). Amer. J. Math., 61, 713–721. 11. Galambos, J. (1970). Ann. Inst. Henri Poincar´e B, 6, 281–305. 12. Galambos, J. (1971). Z. Wahrscheinl. verw. Geb., 18, 261–270. 13. Galambos, J. (1971). Bull. Lond. Math. Soc., 3, 307–312. 14. Galambos, J. (1971). In The Theory of Arithmetic Functions, A. A. Gioia et al., eds. Springer-Verlag, Berlin, pp. 127–139. 15. Galambos, J. (1973). Proc. Amer. Math. Soc., 39, 19–25. 16. Galambos, J. (1974). Acta Arith., 25, 359–364. 17. Galambos, J. (1976). Representations of Real Numbers by Infinite Series. Springer-Verlag, Berlin.

6434

PROBABILISTIC VOTING MODELS

` 18. Halasz, G. (1968). Acta Math. Acad. Sci. Hung., 19, 365–403. 19. Halberstam, H. and Roth, K. F. (1966). Sequences 1. Oxford University Press, Oxford, England. 20. Ibragimov, I. A. and Linnik, Yu. V. (1971). Independent and Stationary Sequences of Random Variables. Wolters-Noordhoff, Groningen. 21. Indlekofer, K. -H. (1974). In Proc. Coll. Bolyai Math. Soc. Debrecen, B. Gyires, ed. NorthHolland, Amsterdam, pp. 111–128. 22. Indlekofer, K. -H. (1981). Ill. J. Math., 25, 251–257. 23. Iosifescu, M. (1978). Trans. 8th Prague Conf. Inf. Theory, Vol. A. D. Reidel, Dordrecht, The Netherlands, pp. 27–40. 24. Kac, M. (1959). Statistical Independence in Probability, Analysis and Number Theory. Carus Monographs. Wiley, New York. 25. Kac, M. (1979). Probability, Number Theory and Statistical Physics. MIT Press, Cambridge, Mass. ` 26. Katai, I. (1968). Compositio Math., 19, 278–289. ` 27. Katai, I. (1969). Acta Math. Acad. Sci. Hung., 20, 69–87. ` 28. Katai, I. (1977). Publ. Math. Debrecen, 24, 91–96. 29. Khintchine, A. (1964). Continued Fractions. Wolters-Noordhoff, Groningen, The Netherlands. 30. Kubilius, J. (1964). Probabilistic Methods in the Theory of Numbers. Transl. Math. Monogr., Vol. 11. American Mathematical Society, Providence, R.I. 31. Kubilius, J. (1974). In Current Problems of Analytic Number Theory, Izd. Nauka i Techn., Minsk, pp. 81–118. 32. Kuzmin, R. O. (1928). Rep. Pol. Acad. Sci. (A), 375–380. 33. Levin, B. V., Timofeev, N. M., and Tuljaganov, S. T. (1973). Litovsk. Mat. Sb., 13, 87–100. 34. Manstavicius, E. (1980). Litovsk. Mat. Sb., 20, 39–52. 35. Novoselov, E. V. (1964). Izv. Akad. Nauk SSSR, Ser. Mat., 28, 307–364. 36. Paul, E. M. (1962). Sankhya A, 24, 103–114, 209–212; ibid., 25, 273–280 (1963). 37. Philipp, W. (1971). Mem. Amer. Math. Soc., 114, 1–102. 38. Philipp, W. (1976). Acta Arith., 28, 379–386.

39. R´enyi, A. (1957). Acta Math. Acad. Sci. Hung., 8, 477–493. 40. R´enyi, A. (1965). Publ. Math. Debrecen, 12, 323–330. 41. Schwarz, W. (1976). Jb. Dtsch. Math.-Ver., 78, 147–167. ¨ 42. Szusz, P. (1961). Acta Math. Acad. Sci. Hung., 12, 447–453. ` 43. Turan, P. (1934). J. Lond. Math. Soc., 9, 274–276. 44. Vervaat, W. (1972). Success Epochs in Bernoulli Trials with Applications in Number Theory. Math. Centre Tracts No. 42, Mathematische Centrum, Amsterdam. See also FIRST-DIGIT PROBLEM; LAWS OF LARGE NUMBERS; ¨ FUNCTION; and PROBABILITY THEORY: AN MOBIUS OUTLINE).

JANOS GALAMBOS

PROBABILISTIC VOTING MODELS In many nations and at many levels of government, the question of who should hold various political offices is settled by voting. As a consequence, there are many public officials, and many politicians who would like to become public officials, who are concerned about the implications of their choices for the decisions that voters will make. In particular, they are concerned about which voters will vote in the next election and who these people will vote for. Two choices that elected officials and officeseeking politicians make that are especially important in this regard are: Their positions on the leading policy questions of the day and the way in which they allocate their campaign resources. As a consequence, researchers have developed mathematical models of the relation between these choices and the decisions that public officials and office-seeking politicians can expect the voters to make. These models can be used to address questions such as: In a given situation in which campaign resources have already been allocated, are there ‘‘best’’ policy positions that can be taken? If so, what are they? In a given situation in which policy positions have already been selected, is there a ‘‘best’’ campaign resource allocation that can be chosen? If so, what is it? In a given situation in which both policy positions

PROBABILISTIC VOTING MODELS

and an allocation of resources can be selected, is there a ‘‘best’’ possible combination? If so, what is it? The answers are useful for social scientists who want to know what policy positions and campaign resource allocations can be expected from public officials and officeseeking politicians. They are also potentially useful for campaign strategists who advise political candidates.

TWO EXAMPLES Before stating the features that differ in the two examples in this section, the characteristics that they have in common will be covered: There are two candidates for a particular political office. They will be indexed by the elements in the set C = {1, 2}. There are three voters. They will be indexed by the elements in the set N = {1, 2, 3}. Each candidate has to decide on what policy proposals he or she wants to make. The only characteristic of a candidate’s policy proposals that matters to the voters is the distribution of income that they expect the policies to lead to. For each vector of policy proposals that a candidate can make, each voter has a unique distribution of income that he or she expects the policies to lead to. These expectations are the same for all three voters. As a consequence, the candidate’s decision can be viewed as the choice of an income distribution∗ . It will be assumed that each candidate can, in particular, choose any ψc ∈ X = {(x1 , x2 , x3 ) ∈ R3 : x1 + x2 + x3 = 1, x1  0.01, x2  0.01, x3  0.01} (by a suitable choice of policy proposals). Each particular voter cares only about his or her own income. More specifically, (for any given i ∈ N) for each pair x, y ∈ X : x is at least as good as y for i if and only if xi  yi . This implies that i prefers x to y (i.e., x is at least as good as y for i, but y is not at least as good as x for i) if and only if xi > yi . It also implies that i is indifferent between x and y (i.e., x is at least as good as y for i and y is at least as good as x for i) if and only if xi = yi . Thus i’s preferences on X can be represented by the utility function Ui (x) = xi ; for each pair x, y ∈ X, Ui (x)  Ui (y) if and only if x is at least as good as y for i. In addition, assume

6435

that Ui (x) measures the intensity of i’s preferences. More specifically, the values assigned by Ui (x) have the following interpretations: Ui (x)/Ui (y) = 12 if and only if i likes x only half as much as he or she likes y; Ui (x)/Ui (y) = 2 if and only if i likes x twice as much as he or she likes y; Ui (x)/Ui (y) = 3 if and only if i likes x three times as much as he or she likes y; and so on. This property implies that each voter’s utility∗ function is unique up to multiplication by a positive scalar. (It is, accordingly, called a ratio-scale utility function.) In the examples, Pci (ψ1 , ψ2 ) will be used to denote the probability that the individual indexed by i will vote for candidate c when c = 1 chooses ψ1 and c = 2 chooses ψ2 . Thus [at any given (ψ1 , ψ2 ) ∈ X 2 ] the expected vote for a given c ∈ C can be written as EVc (ψ1 , ψ2 ) =

3 

Pci (ψ1 , ψ2 ).

i=1

Each candidate c is concerned solely about his or her expected plurality. Plc (ψ1 , ψ2 ) = EVc (ψ1 , ψ2 ) − EVk (ψ1 , ψ2 ) (where k is the index for the other candidate), that is, with his or her expected margin of victory (or, phrased differently, how much he or she expects to win or lose by). Furthermore, each candidate wants to maximize his or her expected plurality; in a candidate’s view, the larger the expected margin of victory, the better. This implies that, for any specification of the Pci (·) functions, the decisions that the two candidates have to make can be appropriately modeled as a two-person, noncooperative game, (X, X; Pl1 , Pl2 ), in which (1) the two players are the two candidates, (2) the strategy set for each candidate is X, and (3) the payoff functions are Pl1 : X × X → R1 and Pl2 : X × X → R1 , respectively. By the definitions of Pl1 and Pl2 , Pl1 (ψ1 , ψ2 ) + Pl2 (ψ1 , ψ2 ) = 0∀(ψ1 , ψ2 ) ∈ X 2 . Hence the game is zero-sum. See GAME THEORY or NASH EQUILIBRIUM. Example 1. Consider the case in which, for each i ∈ N and each pair (ψ1 , ψ2 ) ∈ X 2 ,   1 if Ui (ψ1 ) > Ui (ψ2 ) P1i (ψ1 , ψ2 ) = 12 if Ui (ψ1 ) = Ui (ψ2 )   0 if Ui (ψ1 ) < Ui (ψ2 )

6436

PROBABILISTIC VOTING MODELS

and P2i (ψ1 , ψ2 ) = 1 − P1i (ψ1 , ψ2 ). The resulting game, (X, X; Pl1 , Pl2 ), has no Nash equilibrium. Since the game is zerosum, this can also be phrased as: The game has no saddle point. The fact that this game has no Nash equilibrium (or, equivalently, has no saddle point) can be seen quite easily. Choose any (x, y) ∈ X 2 . Since the game is zero-sum, there are three possibilities: (a)

Pl1 (x, y) = Pl2 (x, y) = 0,

(b)

1

(c)

2

Pl (x, y) > 0 > Pl (x, y), or Pl1 (x, y) < 0 < Pl2 (x, y).

Suppose that (a) or (b) holds. Identify a voter, i, who gets at least as much income at x as anyone else (i.e., for whom we have xi  xj , ∀j = i). Let z be the alternative in which this individual gets zi = 0.01 and the others get zj = xj + 12 (xi − 0.01) (i.e., the others split the decrease in i’s income). Since xi > 0.01, we have Uj (z) = zj > xj = Uj (x) for each j = i and Ui (z) = zi < xi = Ui (x). Therefore, P2j (x, z) = 1 and P1j (x, z) = 0 for each j = i and, in addition, P2i (x, z) = 0 and P1i (x, z) = 1. Therefore, Pl2 (x, z) = +1 > 0  Pl2 (x, y). Therefore, if (a) or (b) holds, (x, y) is not a Nash equilibrium. Similar reasoning applies when (c) holds. Example 1 is a game-theoretic version of the voting paradox∗ , the fact that when majority rule is used to make collective decisions, a society can find itself in a situation where, for each social choice that could be made, there is a feasible alternative that a majority of the voters prefer (and, hence, any particular choice can be overturned by a majority vote). More specifically, this example illustrates how easy it is for the paradox of voting (and the corresponding absence of equilibrium policies) to occur when the issues that are to be resolved involve the distribution of income in the society (see also VOTING PARADOX, Arrow [1] and Sen [30]). Example 2. Consider the case in which the probabilities that describe the voters’ choices

satisfy the following version of Luce’s axiom of ‘‘independence from irrelevant alternatives’’ (see CHOICE AXIOM, LUCE’S): For each i ∈ N and (ψ1 , ψ2 ) ∈ X 2 , P1i (ψ1 , ψ2 ) P2i (ψ1 , ψ2 )

=

Ui (ψ1 ) ; Ui (ψ2 )

(1)

assume also (implicit in Example 1) that each voter is going to vote; that is, for each i ∈ N and (ψ1 , ψ2 ) ∈ X 2 , P1i (ψ1 , ψ2 ) + P2i (ψ1 , ψ2 ) = 1.

(2)

Equations (1) and (2) imply that for each i ∈ N and (ψ1 , ψ2 ) ∈ X 2 , P1i (ψ1 , ψ2 ) =

Ui (ψ1 ) , Ui (ψ1 ) + Ui (ψ2 )

P2i (ψ1 , ψ2 ) =

Ui (ψ2 ) . Ui (ψ1 ) + Ui (ψ2 )

This time, the resulting game (X, X; Pl1 , Pl2 ) does have a Nash equilibrium. Since the game is zero sum, this can also be phrased as: The game has a saddle point (see GAME THEORY or NASH EQUILIBRIUM). The fact that this game has a Nash equilibrium (or equivalently, has a saddle point) can be seen quite easily. For each i ∈ N and (x, y) ∈ X 2 , P1i (x, y) = xi /(xi + yi ). Therefore, each P1i (x, y) is a concave function of x = (x1 , x2 , x3 ), and each −P1i (x, y) is a concave function of y = (y1 , y2 , y3 ). This, in turn, implies that EV1 (x, y) is a concave function of x and −EV1 (x, y) is a concave function of y. Similarly, EV2 (x, y) is a concave function of y and −EV2 (x, y) is a concave function of x. Therefore, candidate 1’s payoff function, Pl1 (x, y) = EV1 (x, y) − EV2 (x, y), is a concave function of x and candidate 2’s payoff function, Pl2 (x, y) = EV2 (x, y) − EV1 (x, y), is a concave function of y. By a similar argument, Pl1 (x, y) and Pl2 (x, y) are continuous functions of (x, y). Finally, from its definition, X is a compact, convex subset of R3 . Hence all of the assumptions in the premise of one of the theorems that has been labeled ‘‘Nash’s Theorem’’ are satisfied (see NASH EQUILIBRIUM). Therefore, there is a Nash equilibrium in this example.

PROBABILISTIC VOTING MODELS

Where is (or are) the Nash equilibrium (or equilibria) located? This question can be answered by solving the problem: Find the x ∈ X that maximize(s) U1 (x) · U2 (x) · U3 (x) = x1 · x2 · x3 over the set X. There is a unique x which solves this problem: x = ( 13 , 13 , 13 ). This implies that there is a unique Nash equilibrium in the game: ψ1 = ψ2 = ( 13 , 13 , 13 ). (See Theorem 1 in Coughlin and Nitzan [9].) GENERAL MODEL Models similar to the two examples given above have been studied in the references listed at the end of this entry. They are all special cases of the following general model: There are two candidates for a particular political office. They will be indexed by the elements in the set C = {1, 2}. There is a set of possible ‘‘locations,’’ X, for the candidates. The set of possible locations is the same for both of them. A particular x ∈ X could, for instance, specify a position on the policy issues in the election, an allocation of campaign resources, or both of these. ψc will be used to denote a particular location for c. There is an index set, N, for the individuals who can vote in the election. A particular i ∈ N could be a number (e.g., if the voters are labeled as voters 1, . . . , n), a specification of various characteristics that a voter can have (such as location of residence, income, age, etc.), a vector that specifies the ‘‘ideal’’ positions on the various policy issues for a voter, or something else. For each i ∈ N and c ∈ C, there is a function Pci : X × X → [0, 1] that assigns to each (ψ1 , ψ2 ) ∈ X × X a probability for the event ‘‘a voter randomly drawn from the individuals labeled by i ∈ N will vote for c if candidate 1 is located at ψ1 , and candidate 2 is located at ψ2 .’’ For each (ψ1 , ψ2 ) ∈ X 2 , P0i (ψ1 , ψ2 ) = 1 − P1i (ψ1 , ψ2 ) − P2i (ψ1 , ψ2 ) will be the probability for the event ‘‘a voter randomly drawn from the individuals labeled by i ∈ N will not vote (i.e., will abstain from voting), if candidate 1 is located at ψ1 and candidate 2 is located at ψ2 .’’ These can be objective

6437

probabilities or they can be subjective probabilities that are believed by both of the candidates. There is a probability distribution on N which assigns to each measurable set, B ⊆ N, a probability for the event ‘‘a voter randomly drawn from the individuals who can vote in the election has an index i ∈ B.’’ These probabilities can also be either objective probabilities or subjective probabilities that are believed by both of the candidates. Each candidate is concerned solely about his or her (a) expected vote, EVc (ψ1 , ψ2 ), (b) expected plurality, Plc (ψ1 , ψ2 ), or (c) probability of winning W c (ψ1 , ψ2 ). DETERMINISTIC VOTING MODELS An important special case for the general model given above is the deterministic voting model. This terminology comes from analyses of models in which each candidate wants to maximize his or her expected plurality (as in the examples). When [for a given i ∈ N and (ψ1 , ψ2 ) ∈ X 2 ] P1i (ψ1 , ψ2 ) = P2i (ψ1 , ψ2 ), the expected vote from the individual(s) indexed by ‘‘i’’ is split evenly between the two candidates. When this occurs (since each candidate’s objective is to maximize his or her expected plurality), the expected votes corresponding to the index i cancel each other out—and, therefore, have no effect on Pl1 (ψ1 , ψ2 ) or Pl2 (ψ1 , ψ2 ). For instance, in Example 1, if U3 (ψ1 ) = U3 (ψ2 ), then P13 (ψ1 , ψ2 ) − P23 (ψ1 , ψ2 ) = 12 − 1 2 = 0. Therefore, the expected plurality for candidate 1 when only voters 1 and 2 are counted is the same as when voters 1, 2, and 3 are counted. That is, [P11 (ψ1 , ψ2 ) + P12 (ψ1 , ψ2 )] − [P21 (ψ1 , ψ2 ) + P22 (ψ1 , ψ2 )] = [P11 (ψ1 , ψ2 ) + P12 (ψ1 , ψ2 ) + P13 (ψ1 , ψ2 )] − [P21 (ψ1 , ψ2 ) + P22 (ψ1 , ψ2 ) + P23 (ψ1 , ψ2 )]. Similarly, the expected plurality for candidate 2 when only voters 1 and 2 are counted is the same as when voters 1, 2, and 3 are counted. From the preceding observations it is thus clear that at any given (ψ1 , ψ2 ), the

6438

PROBABILISTIC VOTING MODELS

only voter indices that matter (to expected plurality maximizing candidates) are ones with P1i (ψ1 , ψ2 ) = P2i (ψ1 , ψ2 ). When, in fact, P1i (ψ1 , ψ2 ) = 1 or P2i (ψ1 , ψ2 ) = 1 at a given i ∈ N and (ψ1 , ψ2 ) ∈ X 2 , one of two things must be true: (a) there is one voter with the index i and the candidates believe that his or her decision will be completely determined once they choose the strategies ψ1 and ψ2 , respectively, or (b) there is more than one voter with the index i and the decisions made by all these voters will be completely determined (and the same) once the candidates choose the strategies ψ1 and ψ2 , respectively. Because of this, any model which satisfies the assumptions of the general voting model given above and is such that, at each (ψ1 , ψ2 ) ∈ X 2 and i ∈ N where P1i (ψ1 , ψ2 ) = P2i (ψ1 , ψ2 ), either (i) P1i (ψ1 , ψ2 ) = 1 or (ii) P2i (ψ1 , ψ2 ) = 1, is called a ‘‘deterministic voting model.’’ Example 1, for instance, is such a model, whereas Example 2 is not. The defining characteristic of a deterministic voting model can be restated as: For each (ψ1 , ψ2 ) ∈ X 2 and i ∈ N, either the expected votes corresponding to the index i cancel each other out (and, therefore, have no effect on the candidates’ expected pluralities) or the decision(s) of the voter(s) corresponding to the index i will be completely determined (and identical when candidate 1 chooses ψ1 and candidate 2 chooses ψ2 . A third way of stating this characteristic is: For each (ψ1 , ψ2 ) ∈ X 2 and i ∈ N, (a) P1i (ψ1 , ψ2 ) = 1, (b) P2i (ψ1 , ψ2 ) = 1, or (c) P1i (ψ1 , ψ2 ) = P2i (ψ1 , ψ2 ) = 12 (1 − P0i (ψ1 , ψ2 )). The deterministic voting models that have received the most attention are ones in which (1) each index corresponds to one voter, (2) each voter, i, has a utility function, Ui (x), and (3a) for each voter, i, and each (ψ1 , ψ2 ) ∈ X 2 ,   1 if Ui (ψ1 ) > Ui (ψ2 ), 1 Pi (ψ1 , ψ2 ) = 12 if Ui (ψ1 ) = Ui (ψ2 ),   0 if Ui (ψ1 ) < Ui (ψ2 ), P2i (ψ1 , ψ2 ) = 1 − P1i (ψ1 , ψ2 ) (as in Example 1) or (3b) for each voter, i, and each (ψ1 , ψ2 ) ∈ X 2  1 if Ui (ψ1 ) = Ui (ψ2 ), P0i (ψ1 , ψ2 ) = 0 if Ui (ψ1 ) = Ui (ψ2 ),

P1i (ψ1 , ψ2 )

 1 if Ui (ψ1 ) > Ui0 (ψ2 ), = 0 if Ui (ψ1 )
Ui (ψ2 ),

P2i (ψ1 , ψ2 ) = 1 − P1i (ψ1 , ψ2 ) − P0i (ψ1 , ψ2 ). An exception is McKelvey [26], where it is assumed that (a) for each index, α, all of the voters who are labeled by this index have the same utility function, Uα (x), and (b) for each index α and each (ψ1 , ψ2 ) ∈ X 2 , (P0α (ψ1 , ψ2 ), P1α (ψ1 , ψ2 ), P2α (ψ1 , ψ2 )) ∈ {(1, 0, 0), (0, 1, 0), (0, 0, 1)}, P1α (ψ1 , ψ2 ) = 1 ⇒ Uα (ψ1 ) > Uα (ψ2 ), P2α (ψ1 , ψ2 ) = 1 ⇒ Uα (ψ1 ) < Uα (ψ2 ). This formulation provides a deterministic voting model in which abstentions can occur more frequently than just when Uα (ψ1 ) = Uα (ψ2 ). The seminal work on deterministic voting models was done by Hotelling [21], Downs [16], Black [2], and Davis and Hinich [11,12]. Surveys of results that have been derived in these and other analyses [using either Plc (ψ1 , ψ2 ) or W c (ψ1 , ψ2 ) as the objective functions for the candidates] can be found in Plott [29], Kramer [22], and Mueller [27,28]. PROBABILISTIC VOTING MODELS Any model which satisfies the assumptions of the general voting model given above, but is not a deterministic voting model, is called a probabilistic voting model. Example 2, for instance, is such a model. The terminology reflects the fact that in any such model, there is at least one pair of strategies (ψ1 , ψ2 ) ∈ X 2 at which there is at least one index that matters (to expected plurality maximizing candidates) where the candidates’ beliefs about the voter(s) corresponding to the index are probabilistic in a nontrivial way; the random variable that describes them is nondegenerate. The defining characteristic for these models can be restated as: There is at least one (ψ1 , ψ2 ) ∈ X 2 and i ∈ N where P1i (ψ1 , ψ2 ) = P2i (ψ1 , ψ2 ) and 0 < P1i (ψ1 , ψ2 ) < 1 or 0 < P2i (ψ1 , ψ2 ) < 1 (or both).

PROBABILISTIC VOTING MODELS

Any model that satisfies all the assumptions of the general voting model given above will be a probabilistic voting model if there is an index i that corresponds to two or more voters and at least one pair of possible locations (ψ1 , ψ2 ) ∈ X 2 such that the choices of the voters corresponding to index i are completely determined when candidate 1 chooses ψ1 and candidate 2 chooses ψ2 , but (a) they will not all make the same choice, and (b) those who will vote are not split evenly between the candidates. Indeed, almost any deterministic voting model can be converted into a probabilistic voting model of this sort by appropriately regrouping the indices in the deterministic voting model into indices for the probabilistic voting model. For instance, let all three voters in Example 1 have the same index i. Then, for the strategies ψ1 = ( 31 , 13 , 13 ) and ψ2 = (0, 12 , 12 ), we have P1i (ψ1 , ψ2 ) = 13 and P2i (ψ1 , ψ2 ) = 23 . The resulting model is a probabilistic voting model. This approach is the basis for the models analyzed in McKelvey [26]. A voting model that satisfies all the assumptions of the general voting model given above can, alternatively, be a probabilistic voting model if there is at least one voter whose choice of whether to vote and/or which candidate to vote for (if he or she votes) is probabilistic in nature, as in Example 2. (See CHOICE AXIOM, LUCE’S for a discussion of theories of probabilistic choice behavior.) This approach is the basis for the models analyzed in Brams [3], Brams and Davis [4], Comaner [5], Coughlin and Nitzan [9,10], Davis and Hinich [13], Hinich [17], Hinich and Ordeshook [18], Hinich et al. [19,20], and Lake [23]. A voting model that satisfies all the assumptions given above could be a probabilistic voting model because the candidates are uncertain about the choices that voters will make and use subjective probabilities to summarize their expectations about these choices (see DECISION THEORY and UTILITY THEORY). This is the basis for the model analyzed in Ledyard [24,25]. Analyses that apply to models in which any of these three interpretations can arise have been carried out in Coughlin [7,8]. The most closely scrutinized probabilistic voting models can be grouped into three basic categories. The first consists of models

6439

in which the description of the candidates’ expectations about the voters’ decisions to vote (or abstain) are probabilistic, but the description of their expectations about the choices that the voters make between the candidates are deterministic (i.e., they believe that, for each (ψ1 , ψ2 ) ∈ X × X and i ∈ N, when candidate 1 chooses ψ1 and candidate 2 chooses ψ2 , the voters corresponding to i who vote will all vote for the same candidate and the candidate who so benefits is completely determined). This category includes the models in Davis and Hinich [13], Hinich and Ordeshook [18], Hinich et al. [19,20], and McKelvey [26]. The second category consists of models in which there are no abstentions, but the description of the candidates’ expectations about the choices that the voters will make in choosing between them is probabilistic: (a) P0i (ψ1 , ψ2 ) = 0, ∀(ψ1 , ψ2 ) ∈ X × X, ∀i ∈ N and (b) ∃i ∈ N and (ψ1 , ψ2 ) ∈ X × X, where P1i (ψ1 , ψ2 ) = 12 . This category includes Brams [3], Brams and Davis [4], Hinich [17], Coughlin and Nitzan [9,10], and Coughlin [8]. The third category consists of models in which the description of the candidates’ expectations both about voter abstentions and about the choices that the voters will make in choosing between them are probabilistic. This category includes Denzau and Kats [15], Coughlin [6,7], and Ledyard [24,25]. For further discussions, see the references.

REFERENCES 1. Arrow, K. (1963). Social Choice and Individual Values, 2nd ed. Yale University Press, New Haven, Conn. 2. Black, D. (1958). The Theory of Committees and Elections. Cambridge University Press, Cambridge, England. 3. Brams, S. (1975). Game Theory and Politics. MacMillan, London. 4. Brams, S. and Davis, M. (1982). Math. Social Sci., 3, 373–388. 5. Comaner, W. 169–178.

(1976).

J. Public

6. Coughlin, P. 427–433.

(1982).

Public

Econ.,

Choice,

5, 39,

7. Coughlin, P. (1983). Math. Social Sci., 4, 275–292.

6440

PROBABILITIES, ASSESSMENT OF

8. Coughlin, P. (1984). J. Econ. Theory, 34, 1–12. 9. Coughlin, P. and Nitzan, S. (1981). J. Public Econ., 15, 113–122. 10. Coughlin, P. and Nitzan, S. (1981). J. Econ. Theory, 24, 226–240. 11. Davis, O. and Hinich, M. (1966). In Mathematical Applications in Political Science, Vol. II, J. Bernd, ed. Southern Methodist University Press, Dallas, Tex. 12. Davis, O. and Hinich, M. Choice, 5, 59–72.

(1968).

Public

13. Davis, O. and Hinich, M. (1972). In Probability Models of Collective Decision Making, R. Niemi and H. Weisberg, eds. Charles E. Merrill, Columbus, Ohio. 14. Davis, O., DeGroot, M., and Hinich, M. (1972). Econometrica, 40, 147–157. 15. Denzau, A. and Kats, A. (1977). Rev. Econ. Stud., 44, 227–233. 16. Downs, A. (1957). An Economic Theory of Democracy. Harper, New York. 17. Hinich, M. 208–219.

(1977).

J. Econ.

Theory,

16,

18. Hinich, M. and Ordeshook, P. (1969). Public Choice, 7, 81–106. 19. Hinich, M., Ledyard, J., and Ordeshook, P. (1972). J. Econ. Theory, 4, 144–153. 20. Hinich, M., Ledyard, J., and Ordeshook, P. (1973). J. Politics, 35, 154–193. 21. Hotelling, H. (1929). Econ. J., 39, 41–57. 22. Kramer, G. (1977). In Frontiers of Quantitative Economics, Vol. III, M. Intriligator, ed. North-Holland, Amsterdam. 23. Lake, M. (1979). In Applied Game Theory, S. Brams, A. Schotter, and G. Schwodiauer, eds. ¨ Physica-Verlag, Wurzburg, Federal Republic of Germany. 24. Ledyard, J. (1981). In Essays in Contemporary Fields of Economics, G. Horwich and J. Quirk, eds. Purdue University Press, West Lafayette, Ind.

30. Sen, A. (1970). Collective Choice and Social Welfare. Holden-Day, San Francisco. See also CHOICE AXIOM, LUCE’S; DECISION THEORY; GAME THEORY; NASH EQUILIBRIUM; POLITICAL SCIENCE, STATISTICS IN; UTILITY THEORY; and VOTING PARADOX.

PETER COUGHLIN

PROBABILITIES,

ASSESSMENT

OF.

See ASSESSMENT OF PROBABILITIES

PROBABILITY AND LOGIC This term usually deals with formalized theories for probability and more generally the study of relationships between logic and probability. The metamathematical structure of probability theory comes out of the necessity to study the formal linguistic structure which goes into the description of the concepts of probability. The method is a glueing of probability and logic methods, intended to give us a better understanding of the foundations of probability∗ and statistics; it may be a new starting point for incorporating theories such as those of fuzzy sets∗ , probabilistic metric spaces, Boolean and Heyting-valued models, nonstandard analysis, topos theory, and so on. As a result we may also have a general theory of uncertainty not necessarily probabilistic in nature. We review the more mathematical aspects of the subject. The reader who is interested in foundational problems and different axiomatic theories for probability should see ref. 7. The article is divided into three sections: Assigning Probabilities to Logical Formulas, Hyperfinite Model Theory, and Final Remarks.

25. Ledyard, J. (1984). Public Choice, 44, 7– 41. 26. McKelvey, R. 815–844.

(1975).

Econometrica,

43,

27. Mueller, D. (1976). J. Econ. Lit., 14, 395– 433. 28. Mueller, D. (1979). Public Choice. Cambridge University Press, Cambridge, England. 29. Plott, C. (1971). In Frontiers of Quantitative Economics, Vol. I, M. Intriligator, ed. NorthHolland, Amsterdam.

ASSIGNING PROBABILITIES TO LOGICAL FORMULAS Historically this idea goes back to the Cambridge philosopher W. E. Johnson, who influenced both Keynes and Jeffreys in their wellknown treatises on probability [10]. Also, one can find similar ideas in Koopman’s comparative logical probability [7, pp. 183–186].

PROBABILITY AND LOGIC

Carnap was the first to study probabilities on formalized sentences [7, pp. 187–204]. His study, however, was restricted to a portion of a formalized language and to symmetric probabilities. It was Gaifman [8] who first used the full power of a first-order language and arbitrary probabilities. His work was extended considerably by Scott and Krauss [19] and others [4–6], most recently Gaifman and Snir [9]. Let L be a first-order language and let C be the set of its constants with C ⊆ T. A probability model for L(T) is an ordered pair (T, µ) where µ : S0 → [0, 1] is a probability on the quantifier-free sentences S0 (T) such that, for φ, ψ ∈ S0 (T), (i) µ(φ) = 1 if φ is provable in L and µ(φ) = µ(ψ) if φ and ψ are logically equivalent in L, (ii) µ(φ ∨ ψ) + µ(φ ∧ ψ) = µ(φ) + µ(ψ), (iii) µ(φ) = 1 − µ(ϕ). Gaifman [8] first proved the following interesting extension property. Theorem. Let (T, µ) be a probability model. Then there exists a unique probability µ∗ on the set S(T), of all closed sentences of L(T), which extends µ and satisfies the Gaifman condition:    φ(t) , (G)µ∗ (υφ(υ)) = sup µ∗ F∈T (ω)

t∈F

where T (ω) is the set of all finite subsets of T. It follows that Doob’s separability condition is a special case of the (G) condition. Another important aspect of assigning probabilities to logical formulas is that it is a special case of assigning truth values in a complete Boolean algebra B. More precisely, if we take as the Boolean algebra B the quotient algebra of the σ -field A of a probability space (, A, P), modulo the σ -ideal of sets of probability zero, then assigning probabilities to logical formulas is the same as assigning truth values in B. This gives us a connection to Boolean-valued models [1,23] and finally to more general Heytingvalued models [13] and topos theory. These structures are generalizations toward covering uncertainties which are not necessarily probabilistic in nature.

6441

HYPERFINITE MODEL THEORY Hyperfinite model theory was introduced by Keisler [16] and is aimed at models that arise in probability and applied mathematics, in contrast with classical model theory which is more appropriate for problems in algebra and set theory. Hyperfinite models are appropriate for the study of the limiting behavior of finite models through the use of finite combinatorial arguments in infinite settings. Keisler’s work [16], along with Hoover’s [14,15], constitutes a novel approach to the metamathematics of probability. It is based on hyperfinite probability [21], which in turn is based on Robinson’s infinitesimal analysis. A hyperfinite model is a structure A = A, µ, ai , Sj , i ∈ I, j ∈ J, where A, µ is a hyperfinite probability space, aj ∈ A, and each Sj is an internal relation on A. The logics LωP , Lω1 P involved are similar to the ordinary logics but instead of the quantifiers ∃x and ∀x, one uses the probability quantifiers (Px > r), (Px  r), where (Px > r) means that the set {x, φ(x)} has probability greater than r in the Loeb measure involved. An interpretation of a hyperfinite model can be given in terms of the so-called opinion poll model [16]. We can think of A as a very large set of people, Sj as a statement about the tuple y, and Sj (a, b) is true if the person a agrees with statement Sj about the n-tuple of people b. Thus (Px > 12 )(Py > 12 )S(x, y) means that a majority of x’s agree with S about a majority of y’s. The sentence (Py > 12 )(Px > 12 )S(x, y) means that for a majority of y’s, a majority of x’s agree with S about y. Neither of the foregoing sentences implies the other. In addition to the above logics, an internal logic L is introduced with hyperfinite truth values  and three quantifiers (max), (min), and . . . dx. This internal logic is needed to get deeper model-theoretic results about the probability logic Lωp, but it is also of interest in its own right. An interesting discovery is the connection between apparently unrelated theorems from model theory and

6442

PROBABILITY AND LOGIC

probability theory. Downward L¨owenheim Skolem theorems for L are closely related to the weak law of large numbers∗ , whereas the elementary chain theorem is related to the strong law of large numbers. In the opinion-poll model an elementary submodel B of A is a sample which is good in the sense that, for any statement Sj and any n-tuple b ∈ An , the percentage of a ∈ B who agree with Sj about b is infinitely close to the percentage of a ∈ A who agree with Sj about b. In a similar way one can interpret the L¨owenheim-Skolem theorem. FINAL REMARKS In addition to the above two mainstreams of metamathematics of probability there are interesting connections between logic and probability. Hyperfinite Stochastic Analysis This is based on Robinson’s infinitesimal analysis and is a formal way of reducing continuous problems to discrete ones. For example, Brownian motion∗ is treated as a hyperfinite random walk∗ , stochastic differential equations∗ as stochastic difference equations, stochastic integrals∗ as nonstandard Stieltjes integrals, etc. For details, extensions and further references, see ref. 21. Statistical Model Theory This idea starts with Suppes [22], who studies models of (statistical) theories, models of experiments, and models of data. Then we ´ ´ have the book of Hajek and Harvanek [11], ´ and Harvanek’s work [12] toward a model theory of statistical theories. These works can be considered a starting point for a metamathematical structure of statistics. From a Bayesian viewpoint, Gaifman and Snir [9] study a language L which has also an empirical part L0 , consisting of finitely many so-called empirical predicates and/or empirical function symbols. Recursion Theory and Randomness The basic issue here is what it means for a sequence to be random. R. von Mises∗ based his axiomatic treatment of probability on the notion of ‘‘kollektiv,’’ a notion of

randomness∗ for a sequence; see ref. 7, pp. 92–100. Church [2] was the first to apply recursion theory to von Mises’ concept of a kollektiv. The names of Ville, Kolmogorov, Martin-L¨of, Knuth, Schnorr, and others [17] are connected with the efforts to develop a consistent theory of random sequences, with the help of recursion theory. The most recent developments of the notion of randomness can be found in Gaifman and Snir [9]. For details and exact references that are not mentioned here, see refs. 7, 9, 12, and 17. Fuzzy Sets, Heyting-Valued Models, and Topos Theory Examining the work of Higgs [13] and other developments of topos theory one can easily convince oneself that Heyting-valued models and topos theory constitute a formalization of the theory of fuzzy sets. The logic involved is of course intuitionistic. On the other hand some of the work on elementary topoi was conducted much in the spirit of nonstandard analysis. The elementary topoi is the appropriate frame to do nonstandard mathematics. The development of nonstandard models (not only Robinson’s infinitesimal models [1,13,21,23]), has shown that they contain the concept of ‘‘randomness’’ or more generally the concept of ‘‘uncertainty’’ as a basic element. For example, in Takeuti [23, Chap. 2] the nonstandard reals are random variables, and in Solovay [20] the concept of a ‘‘random real’’ is introduced for the adoption of Cohen’s forcing into measure-theoretic problems. Furthermore, Boolean algebras and probabilities on them represent the qualitative and quantitative aspects of ‘‘randomness’’ respectively, whereas Heyting algebras represent more general aspects of uncertainty, not necessarily probabilistic in nature, e.g., fuzzy sets [3]. It is believed that fuzzy sets∗ [3] and probabilistic metric spaces [18] can be expressed formally using Heyting-valued and Booleanvalued models. REFERENCES 1. Bell, J. L. (1977). Boolean-Valued Models. Clarendon, Oxford, England. (A standard textbook on the subject. Presupposes acquaintance with mathematical logic and axiomatic set theory.)

PROBABILITY AND MATHEMATICAL STATISTICS 2. Church, A. (1940). Bull. Amer. Math. Soc., 46, 130–135. 3. Dubois, D. and Prade, H. (1980). Fuzzy Sets and Systems: Theory and Applications. Academic Press, New York. 4. Eisele, K. Th. (1976). Booleschwertige Modelle mit Wahrscheinlichkeitsmasses und Darstellungen Stochastischer Prozesse. Ph.D. Thesis, University of Heidelberg, Germany. 5. Fenstad, J. E. (1967). In Sets, Models and Recursion Theory, J. N. Crossley, ed. NorthHolland, Amsterdam, pp. 156–172. 6. Fenstad, J. E. (1968). Synth`ese, 18, 1–23. 7. Fine, T. (1973). Theories of Probability. Academic Press, New York. (Examines the foundations of probability. Perhaps the only textbook of this sort.) 8. Gaifman, H. (1964). Israel J. Math., 2, 1–18. (Original work; deals with finitary formulas.) 9. Gaifman, H. and Snir, M. (1982). J. Symbolic Logic, 47, 495–548. (A recent landmark paper which introduces new concepts; results from a Bayesian viewpoint and summarizes and extends the works in refs. 4, 5, 16, and 14.) 10. Good, I. J. (1965). The Estimation of Probabilities. MIT Press, Cambridge, MA. ´ ´ 11. Hajek, P. and Havranek, T. (1978). Mechanizing Hypothesis Formation: Mathematical Foundations for a General Theory. Universitexte, Springer-Verlag, Heidelberg. ´ 12. Havranek, T. (1977). Synth`ese, 36, 441–458. (Technical paper, worth reading by those interested in model theory of statistical theories. Also, there are further references on this subject.) 13. Higgs, D. (1973). A Category Approach to Boolean-Valued Set Theory. University of Waterloo, preprint. (Excellent technical paper for Heyting-valued models; can be considered as a formalization of the theory of fuzzy sets.) 14. Hoover, D. N. (1978). Ann. Math. Logic, 14, 287–313. (Technical paper on probability logic extending Keisler’s work in ref. 11.) 15. Hoover, D. N. (1982). J. Symbolic Logic, 47, 605–624. (Recent extensions of refs. 11 and 13.) 16. Keisler, H. J. (1977). In Logic Colloquium 76. North-Holland, Amsterdam, pp. 5–110. (Source paper; develops hyperfinite model theory, based on hyperfinite probabilities; see ref. 16.) 17. Schnorr, C. P. (1971). Zufalligkeit ¨ und Wahrscheinlichkeit. Lecture Notes in Math., No.

218, Springer, Berlin. (Studies sequences using recursion theory.)

6443 random

18. Schweizer, B. and Sklar, A. (1983). Probabilistic Metric Spaces. North-Holland, Amsterdam. (The only introductory text to this very interesting subject.) 19. Scott, D. and Krauss, P. (1966). In Aspects of Inductive Logic, J. Hintikka and P. Suppes, eds. North-Holland, Amsterdam, pp. 219–264. (Extends ref. 7 to infinitary languages.) 20. Solovay R. M. (1970). Ann. Math., 92, 1–59. (Source paper; adopts forcing to measure theoretic problems.) 21. Stroyan, K. D. and Bayod, J. M. (1983). Foundations of Infinitesimal Stochastic Analysis. North-Holland, Amsterdam. (A graduate introduction to hyperfinite probability and infinitesimal stochastic analysis.) 22. Suppes, P. (1962). In Logic, Methodology and Philosophy of Science; Proceedings of the 1960 International Congress, E. Nagel, P. Suppes, and A. Tarski, eds. Stanford University Press, Stanford, CA, pp. 252–261. 23. Takeuti, G. (1978). Two Applications of Logic to Mathematics. Iwanami Shoten, Tokyo; Princeton University Press, Princeton, NJ. (A technical book that develops Boolean analysis.) See also AXIOMS OF PROBABILITY; BELIEF FUNCTIONS; PROBABILITY, FOUNDATIONS OF —I; FUZZY SET THEORY; NONADDITIVE PROBABILITY; and SUBJECTIVE RANDOMNESS.

CONSTANTIN A. DROSSOS

PROBABILITY AND MATHEMATICAL STATISTICS This journal is published by the Institute of Mathematics, Polish Academy of Sciences, Wroclaw University, Poland. The Editor-inChief is Kazimierz Urbanik. The main purpose of the journal is to foster development in fields of probability theory of special relevance to statistics. Volume 7 appeared in 1986. Papers may be in English or French. Most papers are in English. The editorial address is Pl. Grunwaldzki 2 / 4, 50-384 Wroclaw, Poland. The website is www.math.uni.wroc.pl/mathbank/ pms.html

6444

PROBABILITY BOUNDS, WORSLEY–HUNTER

PROBABILITY BOUNDS, WORSLEY–HUNTER This is an improved Bonferroni upper bound∗ proposed by Worsley [8] in 1982. A similar result was obtained independently by Hunter [3]. A somewhat different approach for obtaining Bonferroni-type bounds was developed by Galambos [1]. In its general formulation, the Worsley bound is given by the following result: Let events A1 , . . . , An be represented as vertices υ1 , . . . , υn of a graph∗ G, where υi and υj are joined by an edge eij if and only if the events Ai and Aj are not disjoint. Let T be a spanning tree of G. Then n  n    Ai
Pr(Ai ) − Pr(Ai ∩ Aj ). Pr i=1

i=1

(i,j):eij ∈T

(1) The sharpest bound is obtained by finding the tree for which the second term on the right hand side of (1) is maximum. [Note that the upper Bonferroni bound simply states that n  n   Ai
Pr(Ai ).] Pr i=1

i=1

Ai = Ap1 ∪ (Ap2 \Ap2 ) ∪ · · ·

i=1

∪ (Apn \Apn ).

(2)

Applying the Bonferroni bound to (2) we obtain (1). If T ≡ G, then the events on the right-hand side of (2) are disjoint and (1) becomes equality (by the well-known properties of the union operator ∪). As a corollary, letting T = (e12 , e23 , . . . , en−1,n ), we have n  n n−1    Ai
Pr(Ai ) − Pr(Ai ∩ Ai+1 ). Pr i=1

i=1

i=1




n 

Pr(Ai ) − (2/n)

i=1



Pr(Ai ∩ Aj )

i<j

is a particular case of the Worsley bound, as is the more general Kounias bound [4]: n   Pr Ai i=1




n  i=1

Pr(Ai ) −

m  

Pr(Ai ∩ Ajk ),

k=1 i∈Jk \jk

where (J1 , . . . , Jm ) is a partition of {1, . . . , n} and jk ∈ Jk , k = 1, . . . , m. See Worsley [8] for more details and applications. Also compare with the Galambos [1] approach. There are enlightening discussions of the Worsley-Hunter bound in [2], [6] and, more recently [7]. In the latter paper, Seneta presents a sharpening of the bounds, and an illuminating numerical example. REFERENCES

If T is a tree it is always possible to find a permutation p1 , . . . , pn of {1, . . . , n} so that υpi is joined to some υpj = υpi say, for some j <  i, i = 2, . . . , n, and ni=1 Ai can be written as n 

Kwerel’s [3] refinement of the Bonferroni bound given by n   Pr Ai

i=1

1. Galambos, J. (1977). Ann. Prob., 5, 577–581. 2. Hochberg, Y. and Tamhane, A. C. (1987). Multiple Comparison Procedures, Wiley, New York. 3. Hunter, D. (1976). J. Appl. Prob., 13, 597–603. 4. Kounias, E. G. (1968). Ann. Math. Statist., 39, 2154–2158. 5. Kwerel, S. M. (1975). J. Amer. Statist. Ass., 70, 472–479. ˇ aritescu, ˇ 6. Marg E. (1986). Biom. J., 28, 937–943. 7. Seneta, E. (1988). Austral. J. Statist., 30A, 27–38. 8. Worsley, K. J. (1982). Biometrika, 69, 297–302. See also BONFERRONI INEQUALITIES AND INTERVALS and GRAPH THEORY.

PROBABILITY DENSITIES, COMPATIBILITY OF This concept was introduced by Abrahams and Thomas [1]. Let p1 (x|y) and p2 (y|x) be

PROBABILITY FORECASTING

given conditional density functions. If there exists a bivariate density a (x, y) with the given conditional densities, these densities are said to be compatible. A necessary and sufficient condition for compatibility of a pair of conditional densities over the region of the (x, y) space where these densities are nonzero is that g(x) p1 (x|y) is of the form , p2 (y|x) h(y) where g and h are nonnegative (integrable) functions satisfying 

 g(x)dx =

h(x) dy.

Examples of beta (yielding bivariate Dirichlet density), gamma, exponential, and Gaussian conditional densities are provided in ref. [1]. This result has been used and refined later by subsequent authors for characterization of bivariate densities in terms of their conditionals. See BIVARIATE DISTRIBUTIONS, SPECIFICATION OF.

REFERENCE 1. Abrahams, J. and Thomas, J. B. (1984). A note on the characterization of bivariate densities by conditional densities. Commun. Statist. Theory Methods, 13, 395–400.

PROBABILITY FORECASTING At its simplest, probability forecasting refers to the process of attaching a numerical probability to an uncertain event. Surprisingly, this most basic requirement for any realworld application of probability theory was for long ignored by most statisticians. The motivation for its development as a discipline in its own right has largely arisen from the needs of the field of probabilistic weather forecasting∗ , and much of the theoretical work underpinning it originally appeared in the meteorological literature. This special application also forms a convenient framework on which to hang a discussion of general principles.

6445

In many countries the weather service still issues its weather forecasts in purely verbal terms, either as a categorical prediction (‘‘It will be dry and sunny’’) or with imprecise indications of uncertainty (‘‘A chance of light showers’’). But many decisions can often only be taken rationally when the uncertain nature of the weather is admitted and quantified as precisely as possible. To meet this need, the National Weather Service in the United States has, since 1965, formulated and issued probability of precipitation (PoP) forecasts. Such a forecast is a number between 0 and 1, intended to be interpreted as the probability that measurable precipitation ( 0.01 inch) will occur during a specific period (generally 12 hours) at a particular point in the area of concern. These PoP forecasts are now popularly accepted by the American public as meaningful and informative. The forecasts themselves are usually arrived at intuitively by the weather forecaster, after taking account of all the information available. This information may include, as a decision aid, an ‘‘objective’’ probability produced by a computerized statistical analysis of weather records, but the forecast issued is usually interpreted, in the last analysis, as a personal or subjective probability∗ statement for that forecaster. Questions thus arise concerning the motivation and integrity of forecasters in forming and stating their subjective probabilities, the empirical validity of the forecasts issued, the extent of the agreement between different forecasters, and the way in which the public should use the forecasts. This article will concentrate on the first two issues. A very readable guide is that of Sta¨el von Holstein [30]. The work of Savage [27] on elicitation of subjective probabilities is particularly elegant. ENCOURAGING HONESTY [28] To motivate a forecaster F to divulge honestly his personal probability p for the uncertain event A, we can present him with a decision problem having economic consequences to him which depend on his decision∗ d and the outcome a of A (where a = 1 if A occurs, a = 0 if not). These consequences may be expressed

6446

PROBABILITY FORECASTING

by a loss function L(a, d). If L is measured on a utility∗ scale, or is small overall in relation to the forecaster’s assets, the theory of coherence∗ implies that he will choose d = dp , where dp minimizes his expected loss pL(1, d) + (1 − p)L(0, d) = L(p, d), say. Suppose now that F is asked to quote a value q ∈ [0, 1], on the understanding that this will commit him to the decision dq appropriate if q were his ‘‘true’’ probability. Then he clearly has no incentive to quote any q = p, and should definitely quote q = p if dp yields the unique minimum of L(p,d). His loss, if he quotes q and A = a, is S(a, q) = L(a, dq ), so that his expected loss, if he quotes q but believes Pr(A) = p, is S(p, q) = L(p, dq ), which is minimized if q = p. Any such function S(a, q), for which S(p, ˜ q)], where a˜ = 1 with q) (defined as E[S(a, probability p, 0 with probability 1 − p) is minimized in q when q = p, is a proper scoring rule; it is strictly proper if this minimum is unique. Such a rule can always be regarded as arising, as above, from a decision problem, for example that having d ∈ [0, 1] and L(a, d) = S(a, d). Using any such scoring rule as a loss function encourages honesty. (However, difficulties in their use arise when the forecaster’s utility is nonlinear in S, or he is motivated to maximize the probability of getting a bigger score than rival forecasters.) In practice, if the elicitor of F’s forecast intends to use it as his own in a decision problem he faces, presenting F with a ‘‘share’’ in his own business will yield a scoring rule which is likely to provide the best motivation for F to concentrate his attention on good assessment where it matters most. The formula D(p, q) = S(p, q) − S(p, p)(= L(p, dq ) − L(p, dp )) defines a directed distance of q from p associated with the decision problem or scoring rule, from which S(a, q) may be effectively recovered as D(a, q). Then D(p, p) = 0, and D(p, q) is nondecreasing as q departs from p in either direction [27]. Important strictly proper scoring rules are the quadratic Brier score [3], with S(a, q) = (a − q)2 , and the logarithmic score [12] S(1, q) = − log q, S(0, q) = − log(1 − q). The scoring rule S(a, q) = |a − q| is improper, however, having optimal qp = 1 for p > 12 , 0 for p < 12 . Murphy and Epstein [23]

consider the optimal ‘‘hedges’’ qp for other scoring rules. We can introduce an operational definition of F’s personal probability p as the solution of dp = d∗ , where d∗ is F’s chosen decision in his decision problem, and the solution is supposed existent and unique. Equivalently, we take p = q, where q is F’s quote under a strictly proper scoring rule. General coherence arguments (see especially ref. 27) imply that this solution should not depend on the specific decision problem. An early approach along these lines was that of de Finetti [7,8], based on the Brier score. Multipurpose Forecasts The users of a weather forecast face many diverse decision problems. A particularly simple one, the cost-loss problem [31,32], is that of the fruit farmer, say, who can take protective measures (decision d1 ) to guard her plants against frost (event A), at a cost of C; if she does not protect (d0 ), she faces a loss L if there is frost, 0 if not. Her expected loss when Pr(A) = p is L(p, d1 ) = C, L(p, d0 ) = pL, so that dp = d1 if p > k = C/L, dp = d0 if p < k. The associated proper scoring rule S(a, q) = L(a, dq ) is not, however, strictly proper, q being an optimal quote whenever q and p both lie on the same side of k, and thus lead to the same decision. To motivate F to meet the needs of all such decision makers, one might proceed as follows [16,17]. After F states his forecast q, a value k is generated, independently of A, from a known distribution v on [0, 1], and he is faced with the cost-loss decision problem having L = 1, C = k, in which he must use his implied optimal decision (i.e., dq = d1 if k < q, d0 if k  q). His overall expected loss on following this strategy for quote q when A = a is 

q

S(a, q) = 0



1

kdv(k) + a

dv(k). q

This formula must therefore yield a proper scoring rule [18], and this will be strictly proper if and only if v has the whole of [0, 1] as its support (so that p and q have to be on the same side of any k ∈ [0, 1]).

PROBABILITY FORECASTING

With no essential change,  1 we can replace S(1, q) above by S(1, q) − 0 kdv(k), yielding the standard form  q kdv(k), S(0, q) = 

0 1

S(1, q) =

(1 − k)dv(k).

(1)

q

It can, in fact, be shown [27–29], under weak conditions, that any proper scoring rule is equivalent to one having the form (1), where v is an arbitrary measure on [0, 1]. (No additional generality is obtained by allowing L above to vary also, as in [25].) In choosing an ‘‘all-purpose’’ scoring rule to elicit a probability in practice, it makes sense to use one whose associated v gives strongest weight to values of k regarded as most typical of the decision problems faced by users. Taking v to be the uniform distribution on [0, 1], dv(k) = dk, effectively recovers the Brier score [16]. FORECASTING CONTINUOUS AND MULTISTATE QUANTITIES Many of the ideas above extend to the more general case in which the forecaster has to assess a whole distribution Q for a quantity (or quantities) X. Again we can consider a decision problem with loss function L(x, d), define dQ to minimize EQ [L(X, d)], and so construct a proper scoring rule S(x, Q) = L(x, dQ ). One important strictly proper scoring rule is the logarithmic score S(x, Q) = − log q(x), where q(·) is the density of Q with respect to a dominating measure µ. It may be shown [1,15,27] that whenever X can take more than two values, this is essentially the only proper scoring rule which is local [i.e., depends on q(·) only through its value at the point x which materializes]. The associated distance function is  D(P, Q) = p(x) log[p(x)/q(x)]dµ(x), the Kullback–Leibler information distance of Q from P. In practice one often confines attention to a finite set A = (As : s = 1, 2, . . . , k) of events

6447

determined by X, and their corresponding probability forecasts q = (qs : s = 1, 2, . . . , k). Let these be judged by a strictly proper scoring rule S(a, q) when A = a. It may be that S is defined for values of q that violate the probability axioms. However, considerations of coherence imply [27] that any elicited probabilities should obey the axioms, so that if, for example, A3 is the disjoint union of A1 and A2 , then q3 = q1 + q2 (see also ref. 14 for coherence of deduced probabilities when the loss function is given by an improper scoring rule). The
kBrier score2 in the context above is now s=1 (as − qs ) and is strictly proper. It is normally applied in the case that the (As ) form a partition [e.g., wind-speed rating (A9 = gale, force 9, etc.)]. Then q is a ‘‘multistate’’ forecast. However, when, as in this example, the various states have a natural ordering, the Brier score appears somewhat unsatisfactory, since no credit is given for assessing high probabilities for states close to that materializing. This end can be accomplished by an analog of the cost-loss decision problem: A suitable loss function would have the property that if ds is the optimal decision when As occurs, then the loss in using dr instead of ds increases with the distance between r and s. Suitable random selection of such a decision problem will yield a strictly proper scoring rule which is ‘‘sensitive to distance.’’ Epstein [11] used this argument to derive his ‘‘ranked probability score.’’ EMPIRICAL VALIDITY: RELIABILITY A single noncategorical probability forecast (i.e., not 0 or 1) can never be said to have been ‘‘right’’ or ‘‘wrong.’’ But when a forecaster has issued a long string of such forecasts, it becomes possible to apply checks of external validity. Suppose that of n sequential forecasts, the ith is pi and the realized outcome of the associated event Ai (‘‘rain on day i’’) is ai (= 0 or 1). Then we can compare the overall aver
age forecast probability pn = n−1 ni=1 pi with the overall
relative frequency of occurrence an = n−1 ni=1 ai . If pn  an , the set of forecasts may be regarded as approximately valid on an overall basis. Murphy and Epstein [22]

6448

PROBABILITY FORECASTING

refer to this property as ‘‘unbiasedness in the large.’’ A more incisive test looks at that subset of occasions i for which the forecast probability pi was at, or suitably close to, some preassigned value p∗ , and compares the observed relative frequency in this subset, a(p∗ ) say, with p∗ . If a(p∗ )  p∗ for all p∗ , the forecasts have been variously termed ‘‘unbiased in the small,’’ ‘‘reliable,’’ ‘‘valid,’’ or ‘‘well calibrated.’’ [Of course, sampling variability will lead to unavoidable departures of a(p∗ ) from p∗ , particularly if based on small numbers.] The plot of a(p) against p is the forecaster’s reliability diagram or Calibration curve. Consider, for illustration, the following probability forecasts and outcomes: Outcome, ai 0 0 1 0 1 0 1 Probability, pi 0.4 0.6 0.3 0.2 0.6 0.3 0.4

Outcome, ai 1 1 0 1 0 0 1 Probability, pi 0.5 0.6 0.2 0.6 0.4 0.3 0.5 Table 1 details the calculation of a(p∗ ) for these data. For example, of the three occasions on which a probability forecast of 0.3 was given, just one of the three associated events occurred, yielding a(0.3) = 13 . The reliability diagram for these forecasts and outcomes is plotted in Fig. 1. The diagonal line corresponds, in the absence of sampling fluctuations, to perfect reliability. With such small numbers, there can be little strong evidence against this. Figure 2 displays the reliability diagram for a sequence of 2820 12-hour PoP forecasts issued by a single weather forecaster in Chicago during the period 1972–1976. These results, which are typical of experienced forecasters, clearly indicate excellent reliability.

Figure 1. Reliability diagram corresponding to Table 1.

For assessing multistate forecasts (pi ), with pi = (pi1 , · · · , pik ), one might concentrate on reliability for each state separately, or for all states jointly (by extracting the subset for which pi  p∗ ). The latter (‘‘vector reliability’’) is more common, and provides a more demanding test. Another possibility, not generally recommended, is ‘‘scalar reliability,’’ in which the whole set (pis : i = 1, . . . , n; s = 1, . . . , k) is regarded as a single sequence, thus ignoring the identity of the state being forecast. For a continuous quantity X, such as maximum temperature over 12 hours, credible interval forecasting involves specifying an interval (l, u) with the interpretation that Pr(l < X < u) is some fixed preassigned value γ (e.g., 75%). Reliability of a set of such forecasts is equivalent to requiring that a proportion γ of the intervals turn out to contain their specified quantity. If one also requires Pr(X
l) = Pr(X  u) = 1 − γ /2, say, vector

Table 1. Calibration Calculations Probability, p∗ 0.2 0.3 0.4 0.5 0.6

Number of Instances

Number of 1’s

Relative Frequency of 1’s, a(p∗ )

2 3 3 2 4

0 1 1 2 3

0 0.33 0.33 1 0.75

PROBABILITY FORECASTING

Figure 2. Reliability diagram of a weather forecaster. Reproduced from [24] with the permission of the authors. The number by each point is the number of occasions on which the associated forecast probability was issued.

reliability can be applied to this multistate problem. EMPIRICAL VALIDITY: RESOLUTION Even when the calibration criterion above is satisfied, the forecasts can be clearly unsatisfactory. Suppose that the weather in fact alternates—dry, wet, dry, wet, . . .. Consider two forecasters assigning precipitation probabilities. F1 assigns pi = 12 always, while F2 ’s forecast probabilities alternate 0, 1, 0, 1, . . .. Then both F1 and F2 are well calibrated; but F2 ’s forecasts are perfect, whereas those of F1 are almost useless. Now the assignment of probability forecasts to a sequence of events can be considered as comprising two separate tasks, sorting and labeling [26]. In sorting, the sequence of events is divided into disjoint subsequences, such that all the events in any given subsequence are considered (approximately) equally probable. Labeling then assigns a numerical value to the common probability in each subsequence. Reliability measures how well the labeling has been carried out but does not address the sorting aspect at all. In the example above, F1 and F2 were both

6449

completely reliable, but F2 was clearly the more successful sorter. Another forecaster F3 , who assigns alternating probabilities 1, 0, 1, 0, . . . is also sorting as well as F2 , although the reliability of these forecasts is abominable. In many ways, good sorting is more important than good labeling: F3 ’s forecasts are more valuable than those of F1 once we have obtained enough data to see how to recalibrate them. In effect, ability at sorting depends on substantive (e.g., meteorological) expertise, and ability at labeling on normative expertise (i.e., skill at expressing uncertainty) [34]. The quality of the sorting process, termed resolution, might be judged by the forecaster’s ability to create a set of subsequences in which the relative frequencies are close to 0 or 1 rather than near the overall relative frequency in the whole sequence. However, it is unreasonable, in general, to expect perfect sorting, which is equivalent to absolutely correct (or absolutely incorrect!) categorical forecasting. When, then, can we say that sorting has been carried out successfully? Curtiss [4] suggests that this has been achieved when within each subsequence of events corresponding to some fixed forecast value, the sequence of actual outcomes exhibits no pattern, but is essentially a ‘‘random sequence,’’ or collective in the sense of von Mises [33]. Informally, this means that it should not be possible to extract, from the subsequence, a sub-subsequence in which the relative frequency differs from that in the subsequence. Forecaster F1 above has clearly failed, and F2 and F3 succeeded, by this criterion. Dawid [5] introduced an extended calibration criterion. Consider a subsequence Sn of (1, 2, . . . , n), extracted arbitrarily, subject to the constraint that the decision on whether or not i is to be included in Sn should be determined only by the previous outcomes (a1 , . . . , ai−1 ), not by ai or any later outcomes. Let pn be the average forecast probability, and an the empirical relative frequency, for the events in Sn . Then we require that pn  an . This is justified by showing that if the (pi ) are constructed sequentially as appropriate conditional probabilities from a joint distribution  for the events (A1 , A2 , . . .), so that pi = (Ai |a1 , a2 , . . . , ai−1 ), then pn − an → 0 except on a set of -probability zero.

6450

PROBABILITY FORECASTING

Consequently, if pn and an are not close enough together, there is a suggestion that, in fact, pn − an →0, which serves to discredit ’s probability assignments. This extended calibration criterion subsumes the earlier one, since the decision to include i in Sn whenever pi is close enough to some preassigned value p∗ depends only on pi , which in turn depends only on (a1 , a2 , . . . , ai−1 ). It also allows us to extract a suitable sub-subsequence of this subsequence and still require matching of forecasts and frequencies, as demanded by Curtiss. A sequence of forecasts which satisfies this criterion for every one of a large enough collection of subsequences (e.g., all computable subsequences) is called completely calibrated. Such a sequence has both perfect reliability and maximum attainable resolution. It is shown by Dawid [6] that if (p1i ) and (p2i ) are each completely calibrated computable forecast sequences, then p1i − p2i → 0(i → ∞). Consequently, this criterion of complete calibration is strong enough to exclude all but one limiting assignment of probabilities, as being empirically invalid. Unfortunately, there is no general way of constructing the asymptotically unique valid probability assignments. In practice, one must allow departures from complete calibration ascribable to sampling variation in finite sequences, so that the criterion is not overrestrictive. Refinement. DeGroot and Fienberg [9,10] introduce a partial ordering between forecasters, which is closely related to comparative resolution. Each combination of forecast p and observed outcome a can be formally regarded as one observation from a bivariate distribution for two random variables, p˜ ˜ From a sequence of such forecasts and and a. their outcomes we can build up an empirical ˜ a): ˜ Q may be bivariate distribution Q for (p, regarded as constructed by randomly selecting one of the occasions in the sequence, with equal probability for each, and setting ˜ a) ˜ to the values (pi , ai ) associated with the (p, selected occasion. Then Q serves as a partial description of the sequence. (To be usable, this empirical distribution may need smoothing or discretisation.) Nature determines the ˜ The forecaster’s empirical distribution of a. input into Q is thus contained in the con˜ given a˜ = 0 and ditional distributions of p,

given a˜ = 1. (For good resolution, these two distributions should be very different.) We can now apply concepts from the theory of comparison of experiments [2]. Forecaster A is sufficient for, or more refined than, forecaster B if B’s conditional distribution for p˜ given a˜ can be obtained by first generating p˜ ∗ from A’s conditional distribution and then generating p˜ from some fixed conditional distribution given p˜ ∗ . In other words, B’s forecast distributions can be produced by passing those of A through a noisy channel, independently of the actual outcome. Consequently, A’s forecasts might be considered more valuable than B’s. ECONOMIC EVALUATION OF FORECASTS Another approach to assessing the validity of a sequence of forecasts p = (p1 , p2 , . . . , pn ) in the light of outcomes a = (a1 , a2 , . . . , an ) is in terms of the economic consequences which would have arisen from using these forecasts to solve a sequence of decision problems. Suppose that before each event Ai is observed, a decision di must be made, leading to loss L(ai , di ) if Ai = ai (we suppose the decision space and loss function independent of i, for simplicity). Using the forecasts above corresponds to taking di = dpi , the decision optimal under Pr(Ai ) = pi , and thus leads to loss S(ai , pi ) on the ith event, where S is the proper scoring rule associated with this decision problem. Comparisons between different forecasters might thus
be based on their average scores S = n−1 ni=1 S(ai , pi ). (Since any suitably regular proper scoring rule can be regarded as having arisen from choosing a random cost-loss decision problem, forecaster A will do at least as well as forecaster B for any decision problem if he does so for every simple cost-loss problem.) The average score S can be split into components of independent interest. Thus letting E denote expectation under the (perhaps discretized or smoothed) joint empirical ˜ a) ˜ based on the observed distribution Q for (p, ˜ p˜ = forecasts and outcomes, define ρ(p) = E(a| p) = Q(a˜ = 1|p˜ = p). This is the ‘‘recalibrated’’ version of p, giving the proportion of events occurring when the forecast p is quoted. We ˜ = E(a| ˜ p). ˜ Then S can be introduce r˜ = ρ(p)

PROBABILITY FORECASTING

˜ p)]. ˜ But expressed as E[S(a,

a proportion ρj results in the event occurring. Then

˜ p)] ˜ = E[E{S(a, ˜ p)| ˜ p}] ˜ E[S(a, ˜ p), ˜ p)] ˜ = E[S(˜r, p)]. ˜ = E[S(E(a| Recalling that the directed distance D(p, q) ≡ S(p, q) − S(p, p)( 0), ˜ we obtain S = S1 + S2 , where S1 = E[D(˜r, p)] and S2 = E[S(˜r, r˜ )]. Note that S1  0, with equality (if S is strictly proper) only when ρ(p) ≡ p (almost surely under Q)—that is, when the forecasts are empirically calibrated. Consequently, S1 can be regarded as a penalty score for poor calibration. On the other hand, S2 is just the average score accruing to the recalibrated forecasts. Since S(r, r) is concave in r [27], S2 will tend to be smaller when these are concentrated near the endpoints of [0, 1]. If forecaster A is more refined than forecaster B, A will have a smaller value for S2 than B for any proper scoring rule [10]. Thus S2 may be regarded as a penalty for poor resolution. Schervish [28] points out that A will be more refined than B if and only if the recalibrated forecasts of A perform at least as well as those of B in every simple cost-loss problem. ˜ = Q(a˜ = 1) denote the Now let π = E(a) empirical relative frequency of 1’s in the outcome sequence (called the climatological probability in meteorology). Then ˜ p)] ˜ = E(a) ˜ = π . So E[S(˜r, π )] = E(˜r) = E[E(a| S(E(˜r), π ) = S(π , π ). Hence S2 = E[S(˜r, r˜ )] = E[S(˜r, π ) − D(˜r, π )] = S0 − S3 , with S0 = S(π , π ), S3 = E[D(˜r, π )]  0. Thus S = S0 + S1 − S3 . In this decomposition, S0 is the score attained by the least refined wellcalibrated forecaster, who always gives probability forecast π ; S3 measures the improvement in resolution over these constant forecasts. Again, S1 penalizes poor calibration. As an example, let S be the Brier score, so that S(a, p) = (a − p)2 , D(p, q) = (p − q)2 . Suppose that the probability forecasts are confined to a set (pj : j = 1, 2, . . . , k) of values, and that forecast pj is issued on nj occasions, of which

6451

nS =

k j=1

nj {ρj (1 − pj )2 + (1 − ρj )p2j },

S0 = π (1 − π ) (with nπ =


k j=1

nS1 = nS2 = nS3 =

nj ρj ), k j=1

k j=1

k j=1

nj (ρj − pj )2 , nj ρj (1 − ρj ) nj (ρj − π )2 .

For this case, the decomposition S = S1 + S2 was first obtained by Sanders [26], and the formula S = S0 + S1 − S3 by Murphy [21]. The former decomposition was extended, as above, to general proper scoring rules by DeGroot and Fienberg [10]. The extension of the latter decomposition appears to be new. If a completely calibrated set of forecasts q exists, these will be more refined than any other set p, and lead to the minimum achievable average loss, for any decision problem. For in any subsequence for which pi and qi take on values (approximately) equal, respectively, to some preassigned p∗ and q∗ , the proportion of events occurring will be q∗ . Thus q is sufficient for the pair (p, q) [10], so is more refined than p. The average loss on applying the p’s will be L(q∗ , dp∗ )  L(q∗ , dq∗ ), the average loss for the q’s. Extensions of the decompositions above to multistate forecasts are straightforward, but can be based on either a scalar or a vector approach [20]. Indeed, even in the two-state case, scalar partitions will differ from the foregoing essentially vector partitions [19]. The vector partitions appear more relevant in general. EMPIRICAL EVIDENCE Many studies of actual forecasting performance have been carried out, investigating reliability, average scores, related ‘‘skill scores,’’ and so on. A good review is given by Lichtenstein et al. [13]. The principal finding is that naive forecasters, or experienced

6452

PROBABILITY, FOUNDATIONS OF—I

experts departing even slightly from their usual tasks, find probability assessment difficult and perform poorly, generally showing overconfidence, with ρ(p) much closer to 12 than p. Feedback sessions, in which penalties arising from applying a proper scoring rule are revealed to the forecaster (or docked from his pay!), can sharpen forecasting ability remarkably. With such practice, U.S. weather forecasters, for example, find themselves well able to discern small differences in probability and are very well calibrated. REFERENCES 1. Bernardo, J. M. (1979). Ann. Statist., 7, 686–690. 2. Blackwell, D. (1951). Proc. 2nd Berkeley Symp. Math. Statist. Prob. University of California Press, Berkeley, Calif., pp. 93–102. 3. Brier, G. W. (1950). Monthly Weather Rev., 78, 1–3. 4. Curtiss, J. H. (1968). J. Appl. Meteorol., 7, 3–17. 5. Dawid, A. P. (1982). J. Amer. Statist. Ass., 77, 605–613. 6. Dawid, A. P. (1985). Ann. Statist., 13, 1251–1274. 7. de Finetti, B. (1962). In The Scientist Speculates, I. J. Good, ed. Heinemann, London, 357–364. 8. de Finetti, B. (1975). Theory of Probability (English transl.). 2 vols. Wiley, New York. 9. DeGroot, M. H. and Fienberg, S. E. (1982). In Statistical Decision Theory and Related Topics III, Vol. 1, S. S. Gupta and J. O. Berger, eds. Academic Press, New York, pp. 291–314. 10. DeGroot, M. H. and Fienberg, S. E. (1983). Statistician, 32, 12–22. 11. Epstein, E. S. (1969). J. Appl. Meteorol., 8, 985–987. 12. Good, I. J. (1952). J. R. Statist. Soc. B, 14, 107–114. 13. Lichtenstein, S., Fischhoff, B., and Phillips, L. D. (1982). In Judgment under Uncertainty: Heuristics and Biases, D. Kahneman, P. Slovic, and A. Tversky, eds. Cambridge University Press, Cambridge, England. 14. Lindley, D. V. (1982). Int. Statist. Rev., 50, 1–26. 15. McCarthy, J. (1956). Proc. Natl. Acad. Sci. (USA), pp. 654–655. 16. Murphy, A. H. (1966). J. Appl. Meteorol., 5, 534–537.

17. Murphy, A. H. (1969). J. Appl. Meteorol., 8, 863–873. 18. Murphy, A. H. (1969). J. Appl. Meteorol., 8, 989–991. 19. Murphy, A. H. (1972). J. Appl. Meteorol., 11, 273–282. 20. Murphy, A. H. (1972). J. Appl. Meteorol., 11, 1183–1192. 21. Murphy, A. H. (1973). J. Appl. Meteorol., 12, 595–600. 22. Murphy, A. H. and Epstein, E. S. (1967). J. Appl. Meteorol., 6, 748–755. 23. Murphy, A. H. and Epstein, E. S. (1967). J. Appl. Meteorol., 6, 1002–1004. 24. Murphy, A. H. and Winkler, R. L. (1977). Appl. Statist., 26, 41–47. 25. Pearl, J. (1978). Int. J. Man-Machine Stud., 10, 175–183. 26. Sanders, F. (1963). J. Appl. Meteorol., 2, 191–201. 27. Savage, L. J. (1971). J. Amer. Statist. Ass., 66, 783–801. 28. Schervish, M. J. (1983). A General Method for Comparing Probability Assessors. Tech. Rep. No. 275, Dept. of Statistics, Carnegie-Mellon University, Pittsburgh, Pa. 29. Shuford, E. H., Albert, A., and Massengill, H. E. (1966). Psychometrika, 31, 125–145. 30. Sta¨el von Holstein, C. A. S. (1970). Assessment and Evaluation of Subjective Probability Distributions. Economic Research Institute, Stockholm School of Economics, Stockholm. 31. Thompson, J. C. (1950). Monthly Weather Rev., 78, 113–124. 32. Thompson, J. C. (1952). Bull. Amer. Meteorol. Soc., 33, 223–226. 33. von Mises, R. (1957). Probability, Statistics and Truth. Allen & Unwin, London. 34. Winkler, R. L. and Murphy, A. H. (1968). J. Appl. Meteorol., 7, 751–758. See also METEOROLOGY, STATISTICS IN; SUBJECTIVE PROBABILITIES; WEATHER FORECASTING, BRIER SCORE IN; and WEATHER FORECASTING, EPSTEIN SCORING RULE IN.

A. P. DAWID

PROBABILITY, FOUNDATIONS OF—I Foundations of Probability As A Discipline Any account of the foundations of probability (hereinafter designated F. of P.) must

PROBABILITY, FOUNDATIONS OF—I

be read skeptically. The objectives are to understand and develop probabilistic reasoning. The scale of the effort in the F. of P. is broad, usually conceptual, and generally not concerned with the more narrowly focused technical advances in the familiar numerical probability concept. Efforts in the F. of P. are likely to be critical, expository, and/or argumentative, exhibiting a tradition inherited from the philosophy of science. The F. of P. lacks widely accepted guidelines as to important problems and acceptable solutions; this lack has been hospitable to the development of several schools of thought, each housing a variety of positions. Contributions to the F. of P. come from many disciplines, including philosophy of science, economics, mathematical probability, physics, and engineering, with philosophers of science in the modal but not the majority position. There is no single journal that is a reliable source for current literature. Some journals that occasionally publish articles on the F. of P. are: Annals of Probability∗ , Annals of Statistics∗ , British Journal of the Philosophy of Science, Journal of the Royal Statistical Society∗ (Series B), Synth`ese, Theory and Decision, and Theoria. Mathematical Reviews can be profitably consulted under 60A05. Brief Historical Background While we are likely to learn little to improve the mathematics of numerical probability by consulting the past, history is of importance to the F. of P., for it provides a perspective on the conceptual assumptions that lie at the root of our present-day notions of probability. This perspective can enlighten us as to alternative assumptions, as well as enable us to better perceive the consequences of commitments to particular probability concepts and statistical methodologies. Standard historical works include David [8], Maistrov [34], Todhunter [45], and more recently the work of Hacking [21] and Shafer [43]; these latter two works are of interest for their greater concern with key conceptual issues, although their historical accuracy has been challenged. Probability ideas, as expressed by scholars and not just as implied by gambling∗ or religious practices, can be traced back to

6453

the ancient Egyptians and Greeks and then forward through the Middle Ages (e.g., the writings of Aquinas) to the appearance of recognizably modern probability in the last half of the seventeenth century. Scholarly views up to the onset of the Renaissance held that it did not admit refined degrees and that the basis for a probability assertion was the opinions of qualified individuals. Numerical probability on a scale from 0 to 1 first appears in the seventeenth century (if we omit scattered precursors who were ignored in their time), at a time when a belief in the desirability of mathematical precision is replacing the belief in the possibility of certain knowledge about the world, the latter possibility having been called into question by Cartesian skepticism. At the close of the seventeenth century we find not only numerical probability and expectation but also Bernoulli’s law of large numbers∗ (published posthumously in 1713). The latter supplied the basis for a frequentist view of probability that largely came to supplant the subjective/epistemic view predominant hitherto (see FREQUENCY INTERPRETATION IN PROBABILITY AND STATISTICAL INFERENCE). The eighteenth century saw the introduction of conditional probability∗ and Bayes’∗ formula, great advances in the calculus and limit laws, the elimination of the belief held by Bernoulli∗ that the sum of the probability of an event and the probability of its complement need not be 1, and the beginnings of the scientific use of probability. The use of probability in the physical and social sciences grew greatly in the nineteenth century under the influence of the works of Boltzmann∗ , Laplace∗ , Quetelet∗ , and others, and increasing attention was paid to a frequentist view [48]. Major contributions to the F. of P. in the first part of the twentieth century include the following: 1. A frequentist interpretation by R. von Mises∗ based on a notion of randomness and repeated experiments modeled by the sample space and collective [49] (a posthumous work with references to earlier work) 2. A. N. Kolmogorov’s axiomatization of probability as a normed measure [25]

6454

PROBABILITY, FOUNDATIONS OF—I

3. B. de Finetti’s and L. J. Savage’s development of the subjective∗ view of probability [9–11, 38] 4. R. Carnap’s development of logical probability, as an objective assessment of the degree to which an evidence statement (inductively) supports a hypothesis statement [4] 5. J. M. Keynes’ [24], B. O. Koopman’s [27], and B. de Finetti’s studies of comparative probability. Role of The F. of P. in Statistics The gap between the F. of P. and applied probability∗ and statistics is narrower than might be expected. Knowledge of the different mathematical and interpretive concepts of probability treated in the F. of P. can guide the selection of families of probability models (not necessarily numerical ones) so as to better reflect the indeterminate, uncertain, or chance phenomena being treated, and can clarify a choice among the divergent, conflicting statistical methodologies now current. The relevance of the F. of P. to statistical methodology is established when we realize that the positions of the NeymanPearsonians, Bayesians, structuralists, fiducialists, and maximum entropists are better understood when it is seen that these schools rely on different concepts of probability, albeit this difference is obscured by common agreement on the mathematical structure (excepting countable additivity) of probability. Neyman–Pearsonians postulate that a class of uncertain/indeterminate phenomena (i.e., the ‘‘unknown’’ parameter) cannot be given a probability model. Bayesians/subjectivists in turn insist upon giving this class an overly precise numerical probability model, but allow great latitude in the subjectively based choice of model. Structuralists/fiducialists and maximum entropists carry the modeling process one step further by claiming to provide objective, rational grounds for the selection of a unique numerical probability model to describe, say, the ‘‘unknown’’ parameter. (Barnett [1] provides some of the details of the different statistical methodologies we have mentioned.) The F. of P. enables us to understand the actual claims and

presuppositions of these methodologies (not just the proffered written and verbal explanations of proponents and opponents), and it provides a basis for the improvement and selective utilization of these methodologies. Lest the reader misunderstand, the F. of P. is far from a panacea for our statistical and modeling ills. It is an undeveloped subject that, like philosophy, will always promise more than it delivers. Yet what it does deliver is essential to the development of statistical methodology. Outline of Contents Our presentation is calculated to broaden the reader’s view by first detaching him or her from the notion that probability is only numerical (see AXIOMS OF PROBABILITY) and then by supplying some examples of nonnumerical probability concepts. It is also our intent to expose the reader to a variety of interpretations that are available for probability. In the following section, we comment on probabilistic reasoning, the subject of the F. of P., locate it with respect to science, mathematics, and rationality, and observe that a probability concept lies at the center of probabilistic reasoning. The third section introduces a five-element framework for analyzing and presenting a probability concept. Elements of the framework are illustrated by reference to several current concepts of probability, with details reserved to the references. Most entries in the reference list pertain to specific theories of probability. A broader coverage is available in Black [3], Fine [13], and Kyburg [29]. The references should also assist the reader to establish his or her own perspective on the F. of P. PROBABILISTIC REASONING The F. of P. has probabilistic reasoning as its subject. This is a mode of rational thought, engaged in by diverse entities having various aims. Furthermore, rationality itself evolves in time, varies with the reasoner and with the goals of the reasoner [35]. The diversity of conditions under which probabilistic reasoning is engaged in makes a prima facie case for a parallel diversity in its forms.

PROBABILITY, FOUNDATIONS OF—I

Probabilistic reasoning, like rationality, has the status of a methodology and as such is neither an empirical discipline nor a branch of deductive or mathematical logic. Probability is neither contained within science nor within mathematics; it is in fact part of the methodology of science, although the appropriateness of specific probability models for categories of empirical phenomena is the subject of scientific investigations. Probability is no more a branch of mathematics than is physics, although it owes a great debt to mathematics for its formulation and development. Hence neither science nor mathematics can be relied on to confirm or refute the broad probabilistic ideas that are the concern of the F. of P. This status makes it difficult to determine generally agreed-on canons of acceptability for proposed or familiar modes of probabilistic reasoning, and it partially explains the controversial nature of the F. of P. A concept of probability is the armature around which one constructs a form of probabilistic reasoning. We thus turn to present a framework for the classification and analysis of probability concepts and aspects of the associated system of probabilistic reasoning. FRAMEWORK FOR THE ANALYSIS OF PROBABILITY CONCEPTS We identify five elements (reasoner, domain of application, semantics/interpretation, syntax/mathematical structure, and working basis) that form a framework for the analysis and selection of a probability concept. The Reasoner The first element is the entity having aims or goals that lead it to engage in probabilistic reasoning. Examples of such entities, hereafter referred to as reasoners, are: 1. Individual 2. Organization (e.g., a business planning ahead, a family group) 3. Society (e.g., the inductive reasoning and rhetoric of the medical profession differs from that of the legal profession, which in turn differs from that of engineering).

6455

Goals can include: 1. Summarization of data 2. Understanding (e.g., inference in science, generation/selection of a model) 3. Choice of action (e.g., decision making by an individual or formal organization) 4. Communication (e.g., between individuals or between an individual scientist and his or her colleagues, say). One also needs to take into account the knowledge and information available to the reasoner and the limits to the ‘‘intelligence’’ of the reasoner (bounded rationality). Domain of Application The domain of application over which the reasoner seeks some mastery is a milieu isolated from its surroundings by the reasoner’s interest in the occurrence of events (we shall use ‘‘events’’ to refer both to events as commonly conceived and to propositions describing events) in some collection and by the reasoner’s, perhaps tacit, identification of an available setting or background that has a bearing on the collection of events. Analysis of the domain guides the choice of an appropriate form of probabilistic reasoning; such analyses can point the way to the need for new theories. Examples of backgrounds or settings drawn from the logical, mental, and physical realms include: 1. Logical. Formally presented information corpus 2. Mental. Individual’s or group’s state of knowledge or belief 3. Physical. Category of empirical phenomena having nearly identical characteristics Descriptors associated with the foregoing setting are, respectively: 1. Indeterminate 2. Uncertain 3. Chance or aleatory. The term epistemic is used to jointly refer to indeterminate and uncertain settings; it

6456

PROBABILITY, FOUNDATIONS OF—I

also often suggests that the issue is one of inductive reasoning from the setting to the events, with probability inhering in the link between the two. Aleatory, on the propensity account, suggests that the reasoner is concerned with a physical property of some empirical phenomenon and not just with his or her knowledge about this property; probability inheres in the phenomenon itself. It is also common for ‘‘chance’’ or ‘‘aleatory’’ to refer to a summarization of empirical data through relative frequencies of occurrence. Further classification of domains can be made along the ‘‘subjective’’/‘‘objective’’ axis. Objective domains are those that are in principle available to more than one individual and that can be described independently of the particular observer. Subjective domains are to a significant extent private and cannot be described without reference to a particular individual. In practice, one rarely encounters pure cases of chance, indeterminate, uncertain, objective, or subjective domains (e.g., consider the bases for the choice of a reference class for a frequentist probability model). Examples of domains of Types 1 and 3 often differ more in the view of probability than in the phenomena themselves. For example, the events of interest may concern the outcome of the nth toss of a particular coin when we have recorded the results of the first (n − 1) tosses. The explicit corpus would then be the record of (n − 1) tosses and perhaps some generalities about coins. One can then construe probability either as a physical property of the coin, with the corpus providing relevant measurement data, or one can construe probability as representing the linkage or relation between the given record of (n − 1) tosses and background on coins and the nth toss; the first case is what we mean by type 3 and the second case is of Type 1. Examples of Type 2 domains are common in the subjectivist/personalist/Bayesian decision-making area [11,18,23,33,38,39]. An individual confronted with a choice of actions introspectively and seriously weights his or her beliefs or strength of conviction as to the probabilities of the consequences attendant on each choice of action. He or she then assesses the expected utility∗ for each action and decides on an action with maximal expected utility. It is an important feature of

the subjective account that there need not be formal knowledge of the contents or even of the extent of the domain. Semantics/Interpretations A domain thus contains both a collection of events whose occurrences are of interest to a reasoner and a setting identified by the reasoner as informative about the occurrence of events and as relevant to achieving its goals. In some fashion, the reasoner decides that it can perhaps identify which of the events are, say, probable, or which events are more probable than other events, or even assign a numerical probability to each event. Implicit in this process is an initial determination as to what provides the evaluative basis for the probability concept being invoked (e.g., what weather data and theory can we use to calculate the probability of rain tomorrow). The evaluative basis largely, but not completely, fixes the meaning of the probability concept, which must have meaning extending beyond its evaluative basis if it is to serve a role other than that of data summarization. Nevertheless, we discuss semantics in terms of this basis, for it provides the least controversial access to elements of meaning. Possible evaluative bases for probability are: 1. Past occurrences of other events of the same type 2. The phenomenon/experiment generating the events 3. The ‘‘state of mind’’ of the reasoner as exhibited by his or her behavior 4. The inductive linkage or relation between the setting and the events. The interpretation of probability most closely associated with Basis 1 is that of converging relative frequencies of occurrence for each event A in the collection; typically, the domain type is 3 (see FREQUENCY INTERPRETATION IN PROBABILITY AND STATISTICAL INFERENCE). Reflection on Basis 1, though, reveals that it is less understandable than it seems at first glance. One possibility is that probability is just descriptive of actual occurrences without regard to the random experiment or setup generating the events. Typically, the descriptive approach is elaborated in a frequentist context in terms of the proportion of

PROBABILITY, FOUNDATIONS OF—I

occurrences of an event in repeated performances of a random experiment. However, probability as a description of actual occurrences is an easily discredited view; we have no grounds for determining such a probability in advance of the actual occurrences. Hence, the common frequentist view of proabability is an ensemble one that carefully refrains from committing itself to what will actually occur. Generally, one would ask of a property that it have known relations to other properties and terms. Yet, if probability is just a property of occurrences of events, and not say, of the underlying random experiment, then what is our basis for determining the value of probability, and how is it related to other probabilities and properties? Basis 2 is a somewhat clearer basis for, say, the common converging relativefrequency-of-occurrence interpretation (see Giere [15], Gillies [17], and Mellor [36]). A random phenomenon or experiment is now said to have a ‘‘physical’’ characteristic: namely, its disposition, propensity, or tendency to produce, say, A with probability Pr(A), or its greater tendency to produce A than to produce B, and this characteristic manifests itself through occurrence rates for the events that are the outcomes of the performance of the experiment; i.e., the propensity is large that the observed relative frequency of event A will be close to the propensity of the experiment to produce A. Probability on this account is a property of the apparatus and not of its output, with implications for the outputs that make it easy to travel the road to Basis 1. Basis 3 is the one held by proponents of the subjectivist/personalist/Bayesian interpretation (see BELIEF, DEGREES OF) as well as by some holders of a subjective epistemic theory [16,32,42]. Subjective probability describes the strength of belief of the reasoner (an individual) concerning the truth of propositions or occurrence of events. Strength of belief is determined through a process of introspection and manifests itself through overt choice/betting behavior. Basis 3 might also include Toulmin’s conception of probability as guarding or qualifying an assertion [46]. Although Toulmin’s interpretation has few adherents, it could

6457

have a role in communications; an individual can signal reservations about a statement in a graded manner that can, perhaps, be made interpersonally meaningful. Basis 4 is maintained by proponents of objective epistemic theories [4,5,7,30], who hold that the probability of an event is relative to a given background corpus (domain Type 1), and it represents the rational degree of inductive support lent by the corpus to the truth/occurrence of the event. It is perhaps also held by some subjective epistemic theorists (e.g., Shafer [42]). Semantic position 4 and domain Type 1 also seem to be appropriate as locations for the following: computational complexity-based induction methods and universal probability constructions proposed by Kolmogorov [26], Solomonoff [44], and Chaitin [6] and developed also by Schnorr [40] and others; Fisher’s fiducial inference∗ (see Seidenfeld [41]); Fraser’s structural inference∗ [14]; and some treatments of likelihood-based statistical support [2,12,20]. At any rate, fiducial and structural inference do not seem to directly admit frequentist or subjectivist interpretations. Other issues in the consideration of the semantics of probability include those aspects of the domains referred to in the section ‘‘Domain of Application’’ by the descriptions ‘‘indeterminate,’’ ‘‘uncertain,’’ and ‘‘chance,’’ as well as by the objective/subjective classifications. Furthermore, there is the generally overlooked issue of the extent to which distinctions or degrees of precision are meaningful. The prevalent commitment to numerical probability has been supported by, and in turn supports, a view that at least in principle, degrees of indeterminacy, uncertainty, and chance∗ can be arbitrarily finely graded. This is a remarkable hypothesis having little justification except that of its widespread acceptance. We can address this issue more clearly when we turn to the mathematical structures we adopt for probability. Syntax/Mathematical Structure

Viewpoint of Measurement Theory. Once the reasoner has adopted a concept of probability supported by the domain of application, he or she then wishes to move this system into a formal mathematical domain so as better

6458

PROBABILITY, FOUNDATIONS OF—I

to determine the implications of the position. From the viewpoint of the theory of measurement [28] the domain and the selected probability concept form an ‘‘empirical relational system.’’ The reasoner homomorphically maps this system into a ‘‘mathematical relational system.’’ The collection of events becomes a collection A of subsets A, B, C, . . ., of a sample space . Relations are then identified between elements of the mathematical domain A that homomorphically represent the relational properties holding between events, and these relations characterize the adopted probability concept. For example, if the reasoner has only classified events as to whether or not they are probable, this is represented by partitioning A so as to specify those sets that are probable. If the reasoner has instead considered binary comparisons of the likelihood of events, this is > represented by a binary relation ∼ such that > A ∼ B represents the claim ‘‘the event represented by A is at least as probable as the event > represented by B.’’ One might expect that ∼ is an order relation for a reasonable probability concept. If, finally, the reasoner holds that his or her goals and setting justify a numerical assignment, he or she may adopt the familiar probability measure representation µ : A → [0, 1], with µ(A)  µ(B) if and only if A represents an event at least as probable as the event represented by B. A numerical probability assignment usually also carries the implications that such statements as ‘‘A is x times as probable as B’’ and ‘‘A is more probable than B by x’’ are meaningful in the empirical domain. It is common in discussions of the meaning of probability to reverse the process we have described and start with a well-defined mathematical structure for a probability concept. One then attempts to coordinate the given mathematical structure with an application domain through an interpretation so as to give meaning to what is otherwise merely a collection of symbols. The possible interpretations are constrained by the axiomatic structure of the probability concept but are not determined by it. (See AXIOMS OF PROBABILITY.) This reverse process has

seemed reasonable only because of the prevalent belief that the only mathematical structure for a probability concept is that of Kolmogorov for numerical probability [25]. We hope that our discussion of syntactical variety in the section ‘‘Hierarchy of Probability Concepts’’ will encourage the study and use of new mathematical structures for probability. Event Collections. The events of interest in the domain are represented either by sets or by propositions. The collection of events is typically assumed to generate a Boolean algebra or σ -algebra of sets or propositions, although there is some interest, largely motivated by quantum logic, in lattices∗ . Too little attention has been paid to the mathematical structure of event collection representations [37]. It is generally not possible to enumerate all possible events (complex systems occasionally surprise us by behaving in unforeseen ways), and therefore the sample space is at best a list of practical possibilities. (The device of logically completing the space by introducing a new event, ‘‘everything else,’’ has problems of its own.) Furthermore, it has been argued in discussions of quantum logic [22] that representations of event collections need not be closed under the usual Boolean operations. Nevertheless, almost all work in the F. of P. presumes at least a Boolean algebra of sets or propositions. Hierarchy of Probability Concepts. The recognition that probabilistic reasoning must confront a wide range of domains and levels of information, knowledge, belief, and empirical regularity can lead us to an acceptance of an hierarchy of increasingly precise mathematical concepts of probability. This hierarchy has been little explored, as almost all of the effort, even in the F. of P., has been devoted to the acknowledgedly vital case of familiar numerical probability. That numerical probability may be inadequate to the full range of uses of probabilistic reasoning is suggested by the following observations: 1. There are categories of empirical phenomena (e.g., speech and picture information sources, earthquakes, weather) for which there is no obvious stability of relative frequency for all events of interest.

PROBABILITY, FOUNDATIONS OF—I

2. An information corpus may lack information; the resulting indeterminacy should be respected and not papered over by dubious hypotheses (e.g., ‘‘If you know nothing about the parameter, then adopt a uniform or maximum entropy∗ prior for it.’’). 3. Self-knowledge of individuals is intrinsically limited, and attempts to force belief or conviction to fit the mold of a particular ‘‘rational’’ theory can only yield results of unknown value. An attempt to accommodate to the preceding observations leads to the following hierarchy of concepts: 1. 2. 3. 4.

‘‘Possibly A.’’ ‘‘Probably A’’ or ‘‘not improbably A.’’ ‘‘A is at least as probable as B.’’ ‘‘A has interval-valued probability (P(A), P(A)’’ where ‘‘P(A)’’ is called the lower probability of A and ‘‘P(A)’’ is the upper probability of A. 5. ‘‘A has probability x.’’

Conditional versions of each of the foregoing concepts are also available. The use of a statistical hypothesis (family of probability measures) to specify our knowledge of the domain phenomena can lead to upper and lower probabilities generated by the upper and lower envelopes of the family of measures; a special case of such hypotheses are the so-called ‘‘indeterminate probabilities’’ [31,32]. It is shown in Walley and Fine [50] that the subclass of concept 4 called belief functions∗ can serve to represent concept 3, which in turn can represent concept 2. Furthermore, the belief functions are a subclass of lower probabilities and are therefore representable by them. These representations (e.g., ‘‘probably A’’ iff ‘‘A is at least as probable as not A’’) are generally nonunique but establish the hierarchy of concepts. The probability concepts just introduced must then be given structure through a set of axioms and definitions of significant terms (e.g., independence or unlinkedness, expectation). The most familiar axiom set is that given by Kolmogorov for concept 5. Some discussion of axioms for the other concepts is

6459

given in Walley and Fine [50]. Here we will only present the basic axioms for lower and upper probabilities [19]. Let A be an algebra of subsets of a set  and let A, B denote elements of A. A1. P(A)  0. A2. P() = 1. A3. If A ∩ B = Ø (empty set), then P(A) + P(B)
P(A ∪ B) (superadditivity) P(A) + P(B)  P(A ∪ B) (subadditivity). A4. P(A) + P(Ac ) = 1 (Ac is the complement of A). Elementary consequences of these axioms include: P(Ø) = P(Ø) = 0. P(A)  P(A). P() = 1. A ⊃ B implies that P(A)  P(B), P(A)  P(B). 5. If A ∩ B = Ø, then P(A ∪ B)
P(A) + P(B)
P(A ∪ B).

1. 2. 3. 4.

If P ≡ P or if in A3 either superadditivity or subadditivity is replaced by additivity, A1 to A4 reduce to the Kolmogorov axioms for the finite case. In one is interested in algebras containing infinitely many events, the basis axioms would need to be supplemented. In fact, there is almost no work in the F. of P. that proceeds solely in terms of the basic axiom set we have given (see BELIEF FUNCTIONS). Working Basis While it is the role of interpretation to coordinate the mathematical, axiomatically constrained concept with the domain of events of interest to the reasoner, this coordination is typically idealized and not itself a working basis for probabilistic reasoning. Statistics is the discipline that supplies the working basis for numerical probability with a frequencyof-occurrence interpretation. Statistics is also of value in supplying the basis for numerical probability in the personalist/subjectivist setting. In this case there is also a literature concerned with the actual capabilities of

6460

PROBABILITY, FOUNDATIONS OF—I

humans to supply probabilities and consistently/rationally/coherently use probabilities to choose actions [47]. Any truly rational theory of partial belief or conviction must take account of the capabilities of the agent in question. Psychology and business school students of these capabilities have shown that human performance is somewhat at variance with the idealized requirements of current personal probability theories. Little is known at present about the practical issues connected either with formal concepts of probability other than the numerical one or with interpretations of the objective epistemic variety. Walley and Fine [51] outline the elements of a frequentist basis for interval-valued probabilities. Acknowledgment Preparation of this article partially supported by National Science Foundation Grant SOC-7812278.

REFERENCES 1. Barnett, V. (1973). Comparative Statistical Inference. Wiley, New York. 2. Birnbaum, A. (1977). Synthese, 36, 19–49. 3. Black, M. (1967). Probability. In The Encyclopedia of Philosophy, Vol. 6, P. Edwards, ed. Macmillan, New York, pp. 464–479. 4. Carnap, R. (1962). The Logical Foundations of Probability, 2nd ed. University of Chicago Press, Chicago. 5. Carnap, R. and Jeffrey, R. C. (1971). Studies in Logic and Probability, Vol. 1. University of California Press, Berkeley, Calif. 6. Chaitin, G. J. (1969). J. ACM, 16, 145–149. 7. Cohen, L. J. (1977). The Probable and the Provable. Clarendon Press, Oxford. 8. David, F. N. (1962). Games, Gods and Gambling. Charles Griffin, London. 9. de Finetti, B. (1931). Fundam. Math., 17, 298–329. 10. de Finetti, B. (1937). Foresight: its logical laws, its subjective sources, transl. in H. Kyburg and H. Smokler, eds., Studies in Subjective Probability. Wiley, New York, 1964, pp. 93–158. 11. de Finetti, B. (1974). Theory of Probability, Vols. 1, 2. Wiley, New York. 12. Edwards, A. W. F. (1973). Likelihood. Cambridge University Press, Cambridge.

13. Fine, T. L. (1973). Theories of Probability: An Examination of Foundations. Academic Press, New York. 14. Fraser, D. A. S. (1968). The Structure of Inference. Wiley, New York. 15. Giere, R. N. (1976). In Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. 2, W. L. Harper and C. A. Hooker, eds. D. Reidel, Dordrecht, Holland, pp. 63–101. 16. Giles, R. (1976). In Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. 1. W. L. Harper and C. A. Hooker, eds. D. Reidel, Dordrecht, Holland, pp. 41–72. 17. Gillies, D. A. (1973). An Objective Theory of Probability. Methuen, London. 18. Good, I. J. (1950). Probability and the Weighing of Evidence. Charles Griffin, New York. 19. Good, I. J. (1962). In Logic, Methodology and Philosophy of Science, E. Nagel, P. Suppes, and A. Tversky, eds. Stanford University Press, Stanford, Calif., pp. 319–329. 20. Hacking, I. (1965). The Logic of Statistical Inference. Cambridge University Press, Cambridge. 21. Hacking, I. (1975). The Emergence of Probability. Cambridge University Press, Cambridge. 22. Jauch, J. M. (1968). Foundations of Quantum Mechanics. Addison-Wesley, Reading, Mass. 23. Jeffrey, R. C. (1965). The Logic of Decision. McGraw-Hill, New York. 24. Keynes, J. M. (1921). A Treatise on Probability, reprinted 1962. Harper and Row, New York. 25. Kolmogorov, A. N. (1933). Grundbegriffe der Wahrscheinlichkeitrechnung. Transl. by N. Morrison, Foundations of the Theory of Probability, 2nd ed. Chelsea, New York, 1956. 26. Kolmogorov, A. N. (1965). Problemy peredachii informatsii, 1, 4–7 (Soviet journal translated as Problems of Information Transmission). 27. Koopman, B. O. (1940). Bull. Amer. Math. Soc., 46, 763–774. 28. Krantz, D. H., Luce, R. D., Suppes, P., and Tversky, A. (1971). Foundations of Measurement, Vol. 1. Academic Press, New York. 29. Kyburg, H. E., Jr. (1970). Probability and Inductive Logic. Macmillan, Toronto. 30. Kyburg, H. E., Jr. (1974). The Logical Foundations of Statistical Inference. D. Reidel, Dordrecht, Holland. 31. Levi, I. (1974). J. Philos., 71, 391–418.

PROBABILITY, FOUNDATIONS OF—II 32. Levi, I. (1980). The Enterprise of Knowledge. MIT Press, Cambridge, Mass. 33. Lindley, D. (1965). Introduction to Probability and Statistics from a Bayesian Viewpoint, Vols. 1, 2. Cambridge University Press, Cambridge. 34. Maistrov, L. E. (1974). Probability Theory: A Historical Sketch, 2nd ed., S. Kotz, trans. Academic Press, New York. 35. March, J. (1978). Bell J. Econ., Autumn, 587–608. 36. Mellor, D. H. (1971). The Matter of Change. Cambridge University Press, Cambridge. 37. Randall, C. H. and Foulis, D. J. (1976). In Foundations of Probability Theory, Statistical Inference and Statistical Theories of Science, Vol. 3, W. L. Harper and C. A. Hooker, eds. D. Reidel, Dordrecht, Holland. 38. Savage, L. J. (1954). The Foundations of Statistics. Wiley, New York. 39. Savage, L. J. (1962). The Foundations of Statistical Inference: A Discussion. Wiley, New York. 40. Schnorr, C. P. (1971). Zufalligskeit und Wahrscheinlichkeit. Springer-Verlag, Berlin. 41. Seidenfeld, T. (1979). Philosophical Problems of Statistical Inference. D. Reidel, Dordrecht, Holland. 42. Shafer, G. (1976). A Mathematical Theory of Evidence. Princeton University Press, Princeton, N. J. 43. Shafer, G. (1978). Arch. History Exact Sci., 19, 309–370. 44. Solomonoff, R. J. (1978). IEEE Trans. Information Th. IT-24, 422–443. 45. Todhunter, I. (1865). A History of the Mathematical Theory of Probability: From the Time of Pascal to That of Laplace. Reprinted by Macmillan, London, 1949. 46. Toulmin, S. E. (1958). The Uses of Argument. Cambridge University Press, Cambridge. 47. Tversky, A. and Kahneman, D. (1974). Science, 185, 1124–1131. 48. Venn, J. (1888). The Logic of Chance, 3rd ed. Macmillan, London. 49. von Mises, R. (1964). Mathematical Theory of Probability and Statistics, ed. and complemented by H. Geiringer. Academic Press, New York. 50. Walley, P. and Fine, T. L. (1979). Synthese, 41, 321–374. 51. Walley, P. and Fine, T. L. Statist., 10, 741–761.

(1982).

Ann.

6461

See also AXIOMS OF PROBABILITY; BAYESIAN INFERENCE; BELIEF FUNCTIONS; CAUSATION —I; CHANCE —I; BELIEF, DEGREES OF; FREQUENCY INTERPRETATION IN PROBABILITY AND STATISTICAL INFERENCE; LOGIC OF STATISTICAL REASONING; PROBABILITY, HISTORY OF; and PROBABILITY THEORY: AN OUTLINE.

TERRENCE L. FINE

PROBABILITY, FOUNDATIONS OF—II MathSci, the online version of Mathematical Reviews, lists over 1000 items (several with duplications) under foundations of probability and statistics (AMS codes 60A05, 62A99) in the period 1980–1996, roughly covering the time since the original appearance of PROBABILITY, FOUNDATIONS OF —I in vol. 4 of this Encyclopedia. These papers can be grouped under the headings: Bayesian/subjectivist Decision theory/utility Logical, inductive, knowledge representation Conditioning and updating Complexity-based randomness and frequentist Quantum-mechanical issues Formalist/mathematical—finite additivity, upper—lower Dempster—Shafer belief functions Objectivist and not the above Fuzzy and possibilistic As probability is central to statistics, issues in the foundations of probability are also issues in the foundations of statistics and have potential, when sufficiently matured, to change the conduct of statistics. The brevity of this update and the wide variety of approaches listed above dictate selectivity in our presentation. Our choices reflect our tastes and our judgement as to those issues in the foundations of probability that have developed to where they can have an influence on statistics. We focus below on Kolmogorov complexitybased approaches to randomness* and

6462

PROBABILITY, FOUNDATIONS OF—II

simplicity of explanation, the Bayesian program, and the alternative nonadditive probability model of interval-valued or upper—lower probability. Kolmogorov complexity has provided the strongest focus on frequentist phenomena, and it and intervalvalued probability depart the farthest from the conventional circle of ideas. The Bayesian program, largely subjective but significantly objective as well, is in the conventional center and among the most readily applicable of developments.

FREQUENTIST DEVELOPMENTS BASED UPON COMPLEXITY Defenses and explications of frequentist approaches to probability have included those based on Kolmogorov complexity (see below), the prequential* probability approach of Dawid [9] and Vovk [43], and uses of interval-valued probability. Perhaps the most penetrating efforts to elucidate an objective, frequentist-related conception of probability and to explicate a logical mathematical basis for induction have come from what is now called Kolmogorov complexity although it was founded independently in the 1960s by A. N. Kolmogorov, R. J. Solomonoff, and G. Chaitin [6]. This pioneering work, reported in Fine [12] (see COMPLEXITY), was then pursued by the three founders (that Kolmogorov took it seriously is evident [1]) as well as by a number of other researchers in the 1970s (e.g., P. Gacs, L. Levin, C.P. Schnorr) and developed in detail. The major themes, of which we address only the first, were: a computational complexity basis for determining the randomness of an individual finite or infinite sequence; construction of a universal prior probability* which, when combined with Bayes’ Theorem*, provides an objective foundation for induction; an algorithmic basis for the notion of information. For an excellent overview of this area with a comprehensive citation of results see Li and Vitanyi [26]; a brief survey is given

by Shen [39]. The complexity program has enabled us to understand issues of randomness, induction, and information that have hitherto been opaque. The major flaw in this program has been the intrinsic ineffective computability of the ideas put forward, rendering them not applicable in statistics. Rissanen’s revitalization of the principle of minimum description length* renders accessible to statistics some of the key concepts developed in the complexity program. The notion of randomness∗ is explicated through a notion of the complexity K(x) of, say, a binary-valued string x ∈ {0, 1}∗ , the set of all finite length strings. Let l(x) denote the length of the string x. The complexity K(x) is taken to be the length l(p) of the shortest program p (another finite length, binary-valued string) that can precisely reproduce x when input to an appropriate universal computing machine U (e.g., a universal Turing machine capable of emulating the computation of any other Turing machine or partial recursive function): K(x) = min{l(p) : x = U(p)}. The important proviso added in the 1970s was that such machines be self-delimiting, in that they do not allow a program p that leads to a defined output (the machine eventually halts to signal the completion of the calculation) to be the prefix, i.e., initial segment, of another such program q yielding a different result. Since we expect random sequences to be incompressible, we would expect that K(x) ≈ l(x); one cannot do much better than a 1 : 1 transformation of x itself to represent x. One could then show that the class of infinite length random strings could be well defined in terms of the complexitybased randomness of all of their initial segments. More precisely, if x is an infinite length string and x1:n its initial segment of length n, then we have the following definition.

Definition: Random Sequence. The infinite random sequence x is random if and only if for K a universal prefix complexity measure (∃c finite)(∀n)K(x1:n ) > n − c.

PROBABILITY, FOUNDATIONS OF—II

In effect, an infinite sequence is random if and only if all of its initial segments are also random in a precise sense. The necessary fly in this ointment is that the complexity measure K cannot be effectively computable (recursive). In fact, K is coenumerable, meaning that it can be effectively approximated only from above; e.g., if U(p) = x then K(x)
l(p). There is an interesting axiomatic complement to this approach [40,41]. By allowing defects in incompressibility (deviations of initial segment complexities from the maximal) one can develop families of ‘‘random’’ sequences that have some, but not all of the properties traditionally associated with full randomness. One of the easiest properties to satisfy is that of the strong law of large numbers, and this is taken to hold for any concept worthy of being called random. The property that infinitely often a sequence has an equal number of 0’s and 1’s holds when the complexity deficiency is less than or equal to ( 21 − ) log n, but not for a deficiency of (2 + ) log n [42]. For the familiar law of the iterated logarithm∗ , however, the deficiency cannot exceed log n, but can exceed o(log log n). A reasonable recasting of the complexity approach in an effective and statistically plausible form uses the minimumdescription-length principle∗ (MDL), reinvigorated by the 1980s work of Rissanen in stochastic complexity* [33,34,35] (see also Dawid [9]). Rissanen replaces encoding by short programs that will produce the desired sequence on a universal self-delimiting partial recursive function by computation or encoding of data X (finding a short program) with respect to a parametrized family of probability measures {Pθ : θ ∈ } that need not contain a ‘‘true’’ model generating X. The key is the well-known result (Huffman coding) that, given a probability mass function P(x) for x ∈ χ , we can encode, in a prefix-free manner, the elements of χ using codewords of lengths nearly − log P(x). One way to approximate the MDL calculation is first to estimate (e.g., via maximum likelihood) the parameter θ from X, encoding it as θn with a precision depending upon the sample size n. Then one encodes the sample X using the model Pθn , resulting in an overall short description of the pair X, θn .

6463

BAYESIAN APPROACH Focus The past fifteen years has seen a significant number (about 100 cited in Mathematical Reviews) of articles and books concerned with the Bayesian approach, primarily in its subjective interpretation (see SUBJECTIVE PROBABILITIES; BELIEF, DEGREES OF), but also regarding objective methods for determining priors (e.g., maximum entropy∗ methods), and studies of finitely additive probability, motivated by de Finetti. The needs of artificial intelligence for knowledge representation and processing have been an interesting driver for developments in the Bayesian approach [11,23,27,32,38]. Three developments that stand out are selection of priors; rendering feasible the multidimensional integrations required by Bayesian calculations to evaluate posteriors and parameter estimates through expectations; the dogma of precision and the attempts to more realistically characterize the nature of prior knowledge by intervalvalued alternatives to measures. Choosing Priors The Bayesian program is of no value unless one can specify the required prior probability laws. Carnap made early efforts to identify the unique rationally correct Bayesian prior that would drive his program of logical probability. A more fundamental attack was made by Solomonoff using computational complexity ideas to establish a universal prior [26,8]. Jaynes [18] and his followers, using the socalled maximum entropy principle∗ , have constructed unique priors that have been the basis of useful applications such as estimation of power spectra. You are urged to take as a prior the one having the maximum entropy, subject to the constraints reflecting one’s prior knowledge (e.g., of certain moments). Other approaches with a significant following in the Bayesian community include the use of conjugate and reference priors [2,4].

6464

PROBABILITY, FOUNDATIONS OF—II

Nor can we rest content with toy problems in which priors are assigned to a handful of parameters. Bayesian approaches to applications such as neural networks∗ [27,29] have forced a confrontation with the need to choose complex priors to describe very many parameters (typically in hundreds or thousands). The difficulty of constructing priors on spaces of infinitely many outcomes (even those as small as the set of integers) is addressed by Diaconis and Freedman [10]. They establish that posterior distributions∗ based upon priors that indeed support the ‘‘true’’ underlying distribution need not converge to this distribution. A consistent prior must be chosen with care for the Bayesian program to yield sensible results. The Bayes methodology, while enjoying good properties (e.g., admissibility* and consistency), is peculiar, in that it requires the user to postulate a prior distribution that is basically as complex as the quantities being inferred, if not more so. One must assume at least as much as one wishes to learn! Practical approaches finesse this difficulty either by adopting relatively simply parametrized families of priors (e.g., jointly normal distributions over parameters) or by the device of hyperparameters∗ . A relatively small number of the latter specify a distribution that in turn randomly selects the actual prior. However, there remains the chronic difficulty of rooting the choice of prior in the prior knowledge of the user. Discussion of the essentials and philosophy of the Bayesian position has been plentiful (see BELIEF, DEGREES OF), and here we note only refs. 2, 3, 4, 5, 19, 32, 44, 48. Calculating Posterior Parameter Estimates Calculations of posterior estimates of parameters, given likelihoods and priors, often involve high-dimensional integrations, and the resulting calculations cannot be carried out analytically. Those who feel a need for analytical calculations are constrained to take especially simple (e.g., conjugate) priors that may have little relationship to any prior knowledge. Failing analytical methods, we can estimate a parameter as an expected value of a suitable random variable with respect to

a posterior distribution if we can generate efficiently a large number of (independent, if practicable) samples drawn from this distribution. The expectation is then estimated as an arithmetical average of these results. Thus there has been interest in Monte Carlobased calculations suited to computers. An original discussion of the computation of posterior expectations is provided by Neal [29] in the context of applications to neural network design. Neal introduces a hybrid Markov chain Monte Carlo∗ calculational method drawn from elementary particle physics [29, Chap. 3]. The resulting samples come from a Markov chain, and thus are dependent, but the chain has the desired limiting stationary distribution. The presence of dependence increases the number of samples that need to be taken and may require a large number of iterations of the Markov chain before a good approximation to the limiting distribution appears. While calculations with complex models then become possible, computation times are still enormous. The Dogma of Precision The Bayesian program suffers from what has been called the dogma of precision [24; 44, Section 5.7] The desire to avoid incoherence (e.g., Dutch books) is claimed to force prior knowledge into a representation by a precise probability distribution. That beliefs and our personal knowledge are hardly ever of this kind is evident upon introspection. While the Bayesian position enjoys significant properties of rationality, none of this is to our good if it can only be maintained through self-deception. A welldeveloped alternative to precise probabilistic knowledge, which yet preserves a strong notion of coherence∗ , is available from Walley’s theory of prevision∗ [44], and it is noted below. INTERVAL-VALUED PROBABILITY Classification By interval-valued or upper—lower probability we refer to nonadditive mathematical models of probability that differ significantly from the familiar real-valued probability that

PROBABILITY, FOUNDATIONS OF—II

is either countably additive (in the celebrated formulation by Kolmogorov) or finitely additive (as espoused by de Finetti and many who are his disciples) (see also NONADDITIVE PROBABILITY). The probability of an event A is now represented by the pair [P(A), P(A)], where P(A)
P(A). Since all of these theories share the relationship P(A) = 1 − P(Ac ), it is possible to proceed in terms of just the lower probability P. Interval-valued probability has been used most commonly to represent epistemic probability—probability based upon knowledge/evidence that may either be uniquely determined by this knowledge (as in logical probability), unconstrained by anything more than a form of self-consistency* or coherence* (as in personalist/subjective probability representing degrees of belief of an individual), or lying somewhere in between. Proponents of an unconstrained view of epistemic probability include I.J. Good, C.A.B. Smith, and P. Suppes; those favoring constraints include F. Hampel, H.E. Kyburg, I. Levi, and P. Walley. An objective, frequentist account of interval-valued probability is central to the work of Fine and his students, and possibly also to that of P. Huber. A hierarchical classification of lower probabilities has been given [30,7]. For present purposes a lower probability is said to be dominated, P ∈ D , if there exists a measure µ such that (∀A ∈ A)P(A)
µ(A). If there does not exist a dominating measure, the lower probability is undominated, P ∈ U. Given a family M of finitely additive measures, we can define the lower expectation E and its restriction to the lower envelope P ∈ L through EX = inf Eµ X, µ∈M

EIA = P(A) = inf µ(A). µ∈M

The corresponding upper probability P is derived by replacing inf by sup. Thus, for example, a lower envelope can be derived from the usual relative frequencies rn (A)

6465

for the occurrence of event A in the first n repeated, unlinked random experiments, through P(A) = lim inf rn (A), n→x

P(A) = lim sup rn (A). n→x

The upper and lower probabilities will agree, with a finitely additive measure, if the relative frequencies are all convergent, but in the theory of interval-valued probability we no longer need to postulate this at the outset. A special lower envelope is a 2-monotone lower probability satisfying P(A ∪ B) + P(A ∩ B)
P(A) + P(B). Applications of Interval-Valued Probability We consider briefly applications to problems of representation of belief and knowledge, inference, stochastic modeling. Applications of epistemic interval-valued probability have been concerned primarily with the first two items, and we start with them. Levi [24] and Walley both use lower envelopes but prefer to focus on families of measures and expectation, called prevision∗ by Walley in apparent homage to de Finetti. Levi’s credal probabilities are a special case of lower envelopes in which the family of measures M is required to be convex, whereas Walley allows arbitrary families of measures. The importance of M is that it is not uniquely identifiable from the lower envelope P, although it is identifiable from the lower expectation E. Levi is concerned both with the representation of belief and mechanisms for changing belief [25]. In Walley’s approach to ‘‘statistical reasoning with imprecise probabilities’’ [44] we interpret EX as the highest price we would pay for the gamble with outcome X (assuming the payoffs of X to be linear in utility), and EX as the lowest price we would sell it for. As in the marketplace, buying and selling prices are typically unequal, although in the unrealistic standard Bayesian approach they are required to be equal. Walley introduces a notion of coherence* and proves that lower envelopes and lower expectations are precisely the right representations to characterize coherence. He then extends these concepts to

6466

PROBABILITY, FOUNDATIONS OF—II

include conditioning. In this way he refounds Bayesianism while denying the implausible dogma of precision. Walley [44] is a masterpiece both for its clear and thorough presentation of a wide circle of ideas on the foundations of probability and statistics, and for the alternative it develops carefully and in detail. While he does not see his efforts as restricted to the Bayesian program, anyone seriously concerned with Bayesian methods will benefit from ref. 44. Hampel [14,15] makes use of conditional upper and lower probability in constructing his related notion of successful bets; he interprets them as epistemic and not physical/frequentist concepts. Let X be an observed random variable and Y one we desire to predict, particularly whether Y ∈ A. Assume that both random variables are defined on a parametrized family of measure spaces with {µθ , θ ∈ }, and some unknown θ0 ∈  determines the actual distribution generating X, Y. We interpret P(A|X) as that we are ‘‘just barely willing to bet’’ at odds of P(A|X) : [1 − P(A|X)] on A when we have observed X. Letting EX|θ denote expectation over X using µθ , the lower probability P is successful if it leads to bets (the Xdependent gain for a unit stake being given by −P(A|X)µθ (Ac ) + [1 − P(A|X)]µθ (A)) that satisfy (∀θ ∈ ) EX|θ P(A|X)
µθ (A). As Hampel [15, p. 12] observes, ‘‘The operational interpretation is that with many independent such bets in different situations (different parametric models), my average gain will most probably be  −.’’ Belief functions∗ , whose structure and application owe most to the efforts of Shafer, are also special 2-monotone lower envelopes. An interesting analysis of belief functions in the context of modal logics is provided by Mongin [28]. Belief functions continue to attract interest in the artificial intelligence community [38]. Kyburg [22,23], in a long-standing research program (e.g., ref. 21), develops logical probability, which relates strongly to (reconstructs) frequentist statistics in the form of epistemic probabilities that are interval-valued. The basic setup is that we

have a rational corpus K of sentences that inter alia contains statistical statements giving ranges for the number of instances of, say, a B being a C; thus, in K we may have the sentence ‘‘the frequency of B ∈ C is in [p, q].’’ We can say that the epistemic probability that A ∈ C is [p, q] if we have the additional sentence in K informing us that A ∈ B and A is a random member of B with respect to C. A notion of A being a random member of B is essential to this project. We also want to identify the ‘‘strongest’’ such claims that can be made relative to K. Kyburg asserts that the lower probability is dominated [22, p. 49]. Applications of interval-valued probability to inference problems have been considered systematically by Walley [44,45] using lower envelopes, and indeed this is the raisond eˆ tre for his work. He uses lower envelopes [45] to treat the fundamental problem of inference in a multinomial setting when little is known a priori either about the number of kinds of possible outcomes (the elements of the sample space) or about their likelihoods. In this case we have an urn of unknown composition from which we draw samples with replacement. The vacuous lower probability [(∀A = ) P(A) = 0] provides a plausible starting point, and conditioning on successive observations through an imprecise Dirichlet model leads to a sequence of more informative (narrower intervals) lower envelopes. An early use of 2-monotone lower envelopes for robust statistical models (e.g., -contamination) was made by Huber [16]. Kadane and Wasserman [20,46] have studied the mathematical properties of upper envelopes, calling them coherent capacities. In ref. 46 they develop Bayes tests for 2monotone lower envelopes. In ref. 47 they attempted to extend the maximum entropy principle to the selection of a set of measures, and hence a lower envelope, but concluded that this could not be done. The major attempt at a frequentist-based notion of interval-valued probability has been in the context of time series* modeling by Fine and his students P. Walley, A. Kumar, Y. L. Grize, A. Papamarcou, and A. Sadrolhefazi, in chronological order. The example of a frequentist-based lower envelope given in the preceding subsection suggests that one can make time series

PROBABILITY, FOUNDATIONS OF—II

models that indulge in fewer metaphysical assumptions about the unobservable infinitely long run. The goal was to construct time series models incorporating stationarity (an unvarying random generating mechanism for the series) and a limited ability to infer from the directly observable events (cylinder sets) to some limiting or tail events (those determined by monotone limits of cylinder sets). For bounded random variables, the celebrated stationarity convergence theorem asserts that the event D, say, of the divergence of relative frequencies must be assigned probability 0 by any stationary, continuous (countably additive) measure. Standard probability theory commits us to a position concerning a tail event (D) on the basis of what is seen on the finite-dimensional cylinder sets. We are unable either to suspend judgment or assert the contrary that µ(D) = 1. Papamarcou and Fine [31] established that it is indeed possible to have stationary lower probabilities that are monotonely continuous along the cylinder sets yet assign P(D) = 1, albeit such a lower probability has to be undominated. Sadrolhefazi and Fine [36] then determined that for any  > 0 and n one can find an undominated lower probability P ∈ U that is noncommittal on all tail events, while at the same time agreeing to within  with a conventional countably additive measure on cylinder sets of size no greater than n. Hence, while we might have good evidence for a standard probability model on small enough cylinder sets, there yet exists an unconventional extension using lower probability that is noncommittal with respect to more metaphysical claims.

4. 5.

6.

7.

8.

9.

10.

11.

12.

13.

REFERENCES 1. Bassalygo, L. A., Dobrushin, R. L., and Pinsker, M. S. (1988). Kolmogorov remembered. IEEE Trans. Inf. Theory, IT-34, 175. 2. Berger, J. O. and Bernardo, J. M. (1992). On the development of reference priors. In Bayesian Statistics 4. Oxford Univ. Press, New York, pp. 35–60. 3. Berger, J. O. (1993). Statistical Decision Theory and Bayesian Analysis, corrected reprint of 2nd (1985) ed. Springer-Verlag, New York.

14.

15.

6467

(A good textbook on the subject by an eminent statistician who became a Bayesian only after the first version of this significantly rewritten text.) Berger, J. O. (1994). An overview of robust Bayesian analysis. Test, 3, 5–124. Bernardo, J. M. and Smith, A. F. M. (1994). Bayesian Theory. Wiley, New York. (Another good textbook on the subject by persons committed to Bayesianism.) Chaitin, G. J. (1987). Algorithmic Information Theory. Cambridge Univ. Press, Cambridge. (Papers by a highly original thinker on the subject.) Chateauneuf, A. and Jaffray, J. Y. (1989). Some characterizations of lower probabilities and other monotone capacities through the use of Mobius inversion. Math. Soc. Sci., 17, 263–283. Chuaqui, R. (1991). Truth, Possibility and Probability: New Logical Foundations of Probability and Statistical Inference. NorthHolland/Elsevier, Amsterdam. Dawid, A. P. (1991). Prequential analysis, stochastic complexity and Bayesian inference. In Bayesian Statistics 4. Oxford Univ. Press, New York, pp. 109–125. Diaconis, P. and Freedman, D. (1986). On the consistency of Bayes estimates (including discussion). Ann. Statist., 14, 1–67. Faigin, R. and Halpern, J. (1994). Reasoning about knowledge and probability. J. Ass. Comput. Mach., 41, 340–367. (Two leading figures in artificial intelligence who take epistemological questions seriously and proceed as logicians.) Fine, T. L. (1973). Theories of Probability: An Examination of Foundations. Academic Press, New York. Fine, T. L. (1994). Upper and lower probability. In ref. 17, pp. 109–129. (An overview of efforts to use lower probability in time-series modeling and nonadditive probability in a frequentist setting.) Hampel, F. (1996). On the philosophical foundations of statistics: bridges to Huber’s work and recent results. In Robust Statistics Data Analysis and Computer Intensive Methods: in Honor of Peter Huber’s 60th Birthday, H. Rieder, ed. Lecture Notes in Statistics. Springer, New York, pp. 185–196. Hampel, F. (1997). What can the foundations discussion contribute to data analysis? And what may be some of the future directions in robust methods and data analysis? J. Statist. Plann. Inference, 57, 7–19. (A summary of

6468

PROBABILITY, FOUNDATIONS OF—II

Hampel’s position, noting issues in the foundations of probability such as the need for upper and lower probability to represent prior knowledge and the importance of distinguishing between epistemic and so-called physical probabilities. This paper provides several references to Hampel’s work.) 16. Huber, P. (1980). Robust Statistics. WileyInterscience, New York. 17. Humphreys, P., ed. (1994). Patrick Suppes: Scientific Philosopher, vol. 1. Kluwer, Dordrecht. (A Festchrift for Suppes including many valuable contributions on a range of foundational issues.) 18. Jaynes, E. T. (1989). E.T. Jaynes: Papers on Probability, Statistics, and Statistical Physics, R. D. Rosenkrantz, ed. Kluwer Academic, Dordrecht. (Reprint of 1983 collection of papers by the foremost exponent of the maximum entropy principle for determining priors.) 19. Jeffrey, R. C. (1992). Probability and the Art of Judgment. Princeton Univ. Press, Princeton. (Reprints of several papers by Jeffrey, who has been highly influential as a philosopher concerned with Bayesianism and the issue of updating or conditioning probabilities.) 20. Kadane, J. B. and Wasserman, L. (1996). Symmetric, coherent, Choquet capacities. Ann. Statist., 24, 1250–1264. 21. Kyburg, H. E., Jr. (1974). The Logical Foundations of Statistical Inference. Reidel, Dordrecht. (An extended development of Kyburg’s epistemic approach to probability and its role in statistics; uses formalism that will challenge the reader.) 22. Kyburg, H. E., Jr. (1990). Science and Reason. Oxford Univ. Press. (Contains a brief, but accessible introduction to Kyburg’s formulation of interval-valued epistemic probability.) 23. Kyburg, H. E., Jr. (1997). Combinatorial semantics: semantics for frequent validity. Comput. Intell., 13, 215–257. 24. Levi, I. (1985). Imprecision and indeterminacy in probability judgment. Phil. Sci., 52, 390–409. 25. Levi, I. (1991). The Fixation of Belief and Its Undoing: Changing Beliefs through Inquiry. Cambridge Univ. Press, Cambridge. 26. Li, M. and Vitanyi, P. (1997). An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, New York. (An excellent and comprehensive review of the subject notable for an uncommon interest in the history of and contributors to computational complexitybased approaches to randomness, induction, and information.)

27. MacKay, D. (1992). A practical Bayesian framework for backpropagation networks. Neural Comput., 4, 448–472. 28. Mongin, P. (1994). Some connections between epistemic logic and the theory of nonadditive probability. In ref. 17, pp. 135–171. 29. Neal, R. (1996). Bayesian Learning for Neural Networks. Springer-Verlag, New York. (An accessible treatment of effective Bayesian calculations in the realistic setting of many parameters. While neural networks are the focus, they can be thought of as another parametrized family of flexible regression functions.) 30. Papamarcou, A. and Fine, T. L. (1986). A note on undominated lower probabilities. Ann. Probab., 14, 710–723. 31. Papamarcou, A. and Fine, T. L. (1991). Stationary and almost sure divergence of time averages in interval-valued probability. J. Theor. Probab., 4, 239–260. 32. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo, CA. 33. Rissanen, J. (1984). Universal coding, information, prediction, and estimation. IEEE Trans. Inf. Theory, IT-30, 629–636. 34. Rissanen, J. (1987). Stochastic complexity (with discussion). J. R. Statist. Soc. B, 49, 223–239, 252–265. 35. Rissanen, J. (1989). Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore. (The most comprehensive presentation of stochastic complexity.) 36. Sadrolhefazi, A. and Fine, T. L. (1994). Finitedimensional distributions and tail behavior in stationary interval-valued probability models. Ann. Statist., 22, 1840–1870. 37. Shafer, G. (1987). Probability judgment in artificial intelligence and expert systems. Statist. Sci., 2, 3–16. 38. Shafer, G. and Pearl, J. (1990). Readings in Uncertain Reasoning. Morgan Kaufmann, San Mateo, CA. 39. Shen, A. (1992). Algorithmic complexity and randomness: recent developments. Theory Probab. Appl., 37, 92–97 (Russian original, Teor. Veroyatnost. i Primenen., 37, 124–131). 40. van Lambalgen, M. (1992). Independence, randomness and the axiom of choice. J. Symbolic Logic, 57, 1274–1304. 41. van Lambalgen, M. (1993). Axioms for randomness. Theory Probab. Appl., 38, 562–566 (Russian original, Teor. Veroyatnost. i Primenen., 38, 670–674).

PROBABILITY, HISTORY OF 42. Vovk, V. G. (1988). The law of the iterated logarithm for Kolmogorov random, or chaotic, sequences (in Russian). Teor. Veroyatnost. i Primenen., 32, 413–425. 43. Vovk, V. G. and Vyugin, V. V. (1994). Prequential level of impossibility with some applications. J. R. Statist. Soc. B, 56, 115–123. 44. Walley, P. (1991). Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, London. (A masterful approach to decision making based upon upper and lower expectation that also contains thorough discussion of competing approaches.) 45. Walley, P. (1996). Inferences from multinomial data: learning about a bag of marbles. J. R. Statist. Soc. B, 58, 3–57. (A clear application of the approach.) 46. Wasserman, L. and Kadane, J. B. (1990). Bayes’ theorem for Choquet capacities. Ann. Statist., 18, 1328–1339. 47. Wasserman, L. and Kadane, J. B. (1996). Entropy when probabilities are imprecise. In Bayesian Analysis in Statistics and Econometrics. Wiley, New York, pp. 549–555. 48. Wright, G. and Ayton, P. (1994). Subjective Probability. Wiley, Chichester. (A compendium of recent essays on many aspects of subjective probability.) See also BELIEF, DEGREES OF; BELIEF FUNCTIONS; COHERENCE —I; COMPLEXITY; DECISION THEORY; IMPRECISE PROBABILITIES; MAXIMUM ENTROPY PRINCIPLE: CLASSICAL APPROACH; MINIMUM DESCRIPTION LENGTH PRINCIPLE; NONADDITIVE PROBABILITY; POSTERIOR PROBABILITIES; PREVISION; PRIOR PROBABILITIES; PROBABILITY, HISTORY OF; PROBABILITY THEORY: AN OUTLINE; STOCHASTIC COMPLEXITY; SUBJECTIVE PROBABILITIES; SUBJECTIVE RANDOMNESS; and UTILITY THEORY.

TERRENCE L. FINE

PROBABILITY GENERATING FUNCTIONS. See GENERATING FUNCTIONS PROBABILITY, HISTORY OF The history of probability is in essence separate from the history of statistics∗ , although the statistics of sample data as a mathematical discipline relies on probability as the foundation of quantitative inference. The synthesis of probability with statistics to

6469

form mathematical statistics was ultimately achieved at the beginning of the twentieth century, largely due to the English Biometric School∗ . In this article we adhere to the history of probability proper up to the end of the nineteenth century, with a brief incursion into the twentieth. Thus we shall not mention names as eminent as that of Gauss∗ , more pertinent to the ‘‘theory of errors.’’ The history of probability theory is traditionally taken to begin in France in 1654, the year of the correspondence between Blaise Pascal∗ (1623–1662) and Pierre de Fermat∗ (1601–1665), primarily on the problem of equitable division of stakes in games of chance∗ . Although there is a ‘‘prehistory’’ of probability, it is due to the enormous prestige of these two workers that later developments crystallized round their work. It is also characteristic of the history that until the nineteenth century its motivation remained, very substantially, games of chance. The frequently mentioned significant names of the prehistory of probability in the context of gaming are Cardano∗ , Galileo∗ , Pacioli, and Tartaglia in Italy. In a book of Fra Luca Pacioli (ca. 1445–ca. 1517) printed in 1494, there occurs a simple instance of the problem of points∗ which was later taken up by Tartaglia (Niccol`o Fontana, ca. 1500–1557) and Girolamo Cardano (1501–1576), and then solved by Fermat and Pascal. In Cardano’s manuscript Liber de Ludo Aleae we find (perhaps for the first time) the fundamental notion, in the context of dice games, that the probability of an event in the situation where each of a finite set of possible outcomes has equal weight is the ratio of the favorable cases to the total number of the cases. Galileo Galilei (1564–1642) showed, inter alia, a thorough understanding of this principle in reference to a throw of three fair dice. The keynote of the Fermat–Pascal correspondence is the equitable division of total stake when a game is interrupted before its agreed completion (‘‘probl`eme de partis’’), and a solution to the problem of points is produced by an ingenious extension of the sample space to equally likely points. The correspondence contains a numerical instance of the gambler’s ruin problem (random walk∗ with absorbing barriers). Pascal’s posthumously

6470

PROBABILITY, HISTORY OF

published Trait´e du triangle arithmetique is the corresponding treatise; his Pens´ees and portions, disputably written by Pascal, of the Logique de Port-Royal (1662) contain elements of the calculus of probabilities also. The next landmark is the publication in 1657 of the first text on probability: De Ratiociniis in Ludo Aleae by the Dutch scientist Christiaan Huygens∗ (1629–1695). Huygens had learned of the problem of points on a visit to Paris in 1655, and his systematization of arguments regarding this problem, and some exercises including the gambler’s ruin problem, were inspired directly by the activities of the French mathematicians. His booklet contains a clear allusion to expectation and notions of sampling without and with replacement. Its main function was as a popularizing medium for the calculus of chances until superseded by the very substantial works Essai d’analyse sur les jeux de hasard (1708) of Pierre de Montmort∗ (1678–1719), Ars Conjectandi (1713, posthumous) of Jacob (James) Bernoulli∗ (1654–1705), and The Doctrine of Chances (1718) of Abraham De Moivre∗ (1667–1754). Bernoulli’s book, written perhaps about 1690, incorporated Huygens’s treatise, with notes, in its first part and the ideas in its fourth part were suggested at least partially by the Logique de Port-Royal. In the first part, the gambler’s ruin problem is generalized to a totally algebraic setting. In the whole, the general rules of probability calculus are made clear. The most significant achievement, however, is in the fourth part, where it is proved under some restrictions that if n−1 Sn is the proportion of successes in n binomial∗ trials in which the success probability is p, then as n → ∞ for any  > 0, Pr(|n−1 Sn − p|
) → 1, which has come to be known as Bernoulli’s theorem∗ and is the first version of the weak law of large numbers∗ . In the preface to Ars Conjectandi, James Bernoulli’s nephew Nicholas asks that Montmort and De Moivre turn their attention to further applications of probability. One of the results, in part, was the second enlarged edition in 1713 of Montmort’s treatise; Montmort had been aware at the time of his first edition of the work

of the Bernoullis. Inasmuch as he writes for mathematicians rather than gamblers, using ‘‘modern’’ analysis, this work is historically significant. He worked on the game of Rencontre (matching pairs), resolved the problem of points in general terms, and worked with Nicholas Bernoulli (1687–1759) on the problem of duration of play in the gambler’s ruin problem. The most important contribution of De Moivre is the refinement of Bernoulli’s theorem by obtaining the normal approximation∗ to the binomial distribution∗ for general p (‘‘De Moivre’s theorem’’), thereby establishing the first instance of the central limit theorem∗ . This work appears in the second and third editions of his Doctrine, which also contains the elements of the Poisson approximation to the binomial distribution. The book of the Marquis De Condorcet (1743–1794), Essai sur l’Application de l’Analyse a` la Probabilit`e des D´ecisions Rendues a` la Pluralit´e des Voix (1785), is important in that it is concerned with social questions and in that it stimulated the fundamental work of Pierre Simon de Laplace∗ (1749–1827), Th´eorie analytique des probabilit´es (1812), which is often taken to mark the end of an era. Laplace’s book may be regarded as describing the then state of the art not only for probability but for statistics in a systematic mathematical fashion. Perhaps two of Laplace’s greatest contributions for probability theory proper are recognition of the mathematical power of transforms (generating functions∗ , characteristic functions∗ ) and an early general version of the central limit theorem. The nineteenth century sees work in probability by French mathematicians on the foundations laid by Laplace and the development of a strong probabilistic school in the Russian Empire as the two most subsequently influential streams. (German developments were rather in the direction of statistics.) Of the French school, a significant landmark was the publication in 1837 of the book by Sim´eon Poisson∗ (1781–1840), Recherches sur la Probabilit´e des Jugements en Mati`ere Criminelle et en Mati`ere Civile, Pr´ec´ed´ees des R`egles G´en´erales du Calcul des Probabilit´es (translated into German in 1841), which, as well as taking up the social context treated

PROBABILITY, HISTORY OF

by Condorcet and Laplace, is in part a probability text. Poisson considered in particular binomial trials with probability of success which may vary deterministically from trial to trial. This situation gives rise to Poisson’s law of large numbers∗ for the proportion of successes, and a corresponding normal approximation, possibly the first suggestion of a central limit statement for sums of independent but nonidentically distributed random variables. Poisson also arrived at the Poisson distribution as the limit of a negative binomial∗ , and the probability distribution function makes possibly its first appearance. Augustin Cauchy∗ (1789–1857) in 1853 discovered the symmetric stable laws∗ and supplied almost a complete rigorous proof of the central limit theorem by characteristic function methods within the context of least squares, while Jules Bienaym´e∗ (1796–1878) proved by the simple techniques still used today the Bienaym´e –Chebyshev inequality∗ , used it to prove a general weak law of large numbers for sums of i.i.d. random variables, and gave a completely correct statement of the criticality theorem∗ of branching processes∗ . The book of Antoine Cournot∗ (1801–1877), Exposition de la Th´eorie des Chances et des Probabilit´es (1834) (translated into German in 1849), is largely philosophical and is notable for an early frequentist view of the subject, while that of Joseph Bertrand (1822–1900), entitled Calcul des Probabilit´es (1889), is a significant and sensibly written ‘‘modern’’ textbook on probability and its applications. The flowering of probability theory in particular (as well as mathematics in general) in the Russian Empire is due in part to the return of the young mathematicians V. Ya. Buniakovsky (1804–1889) and M. V. Ostrogradsky (1801–1861) from Paris, where in the early 1820s both had had contact with Laplace, Cauchy, Fourier, and Poisson. Both popularized probability theory in St. Petersburg, together with A. Yu. Davydov (1823–1885) in Moscow. Buniakovsky wrote the first Russian-language text (heavily influenced by Laplace’s Th´eorie) on probability, Osnovania Matematicheskoi Teorii Veroiatnostei, and thereby introduced the still-current mathematical terminology into that language. The stage had been

6471

set for the founding of the St. Petersburg School by P. L. Chebyshev∗ (1821–1894), whose probabilistic contributions, with those of his pupils A.A. Markov∗ (1856–1922) and A. M. Liapunov∗ (1857–1918), in particular, established it as another center of the subject area and gave rise to much of the direction of modern probability theory. Chebyshev proved rigorously Poisson’s weak law of large numbers and popularized the applicability of the Bienaym´e –Chebyshev inequality as a tool for proving weak laws. All three worked on rigorous proofs of the central limit theorem for nonidentically distributed independent summands under general conditions, the culmination being Liapunov’s theorem∗ . Markov developed the concept of Markov chains∗ and thereby gave impetus to the development of stochastic processes∗ . His textbook Ischislenie veroiatnostei (1900, 1908), in its German version in 1912 and in its third (substantially revised) edition of 1913 (and the posthumous edition of 1924), to a large extent laid the foundation for the further development of probability theory. In regard to stochastic processes, further foundations were laid, again in France, by the Calcul des probabilit´es (1896) of Henri Poincar´e (1854–1912), and by Louis Bachelier (1870–1946) through his Th´eorie de la speculation (1900), for the theory of Markov processes. In England some early impetus for probabilistic work was provided by Augustus de Morgan (1806–1871). The Logic of Chance (1876) of John Venn (1834–1923) gave impetus to the study of probability from the standpoint of philosophy, while that of William Whitworth (1840–1905), entitled Choice and Chance, of which the first edition appeared in 1867, even now continues to provide exercises on combinatorial probability. Other eminent contributors to the subject were Robert Ellis (1817–1859), Morgan Crofton (1826–1915), and James Glaisher (1848–1928). In the context of statistical mechanics∗ , important contributors were James Maxwell∗ (1831–1879) and Ludwig Boltzmann∗ (1844–1906). For the prehistory of probability see refs. 3, 6, 9, 10, 13, and 15. The Fermat–Pascal correspondence is thoroughly discussed in refs. 3 and 19, while work from this point up to and including Laplace is well covered in the

6472

PROBABILITY, HISTORY OF

classic ref. 19. Other relevant sources for this period are refs. 1–4, 6, 7, 9, 11–14, and 16. Unlike the preceding period, the history of probability in the nineteenth century is not adequately represented in monograph form. The most comprehensive is Czuber [2], with the supplement, ref. 21. Modern monograph sources are refs. 5, 8, 9 and 20. For specialized aspects, consult also refs. 1, 13, 17, and 18. The modern era in the development of probability theory begins with its axiomatization. The first attempts in this direction are due to Sergei Bernstein∗ (1880–1968), Richard von Mises∗ (1883–1953), and Emile Borel (1871–1956). A. N. Kolmogorov’s Grundbegriffe der Wahrscheinlichkeitsrechnung (1933) introduced the now generally accepted axiomatization suitable not only for classical probability theory but also for the theory of stochastic processes∗ . A few of the significant books which provided an impetus to the development of modern probability are P. L´evy’s Th´eorie de l’addition des variables al´eatoires (1937); H. Cram´er’s Random Variables and Probability Distributions (1937) and Mathematical Methods of Statistics (1946); Limit Distributions for Sums of Independent Random Variables (Russian edition 1949; first English edition 1954) by B. V. Gnedenko and A. N. Kolmogorov; W. Feller’s An Introduction to Probability Theory and Its Applications, Vol. 1 (1950); and M. Lo`eve’s Probability Theory (1955).

REFERENCES 1. Adams, W. J. (1974). The Life and Times of the Central Limit Theorem. Caedmon, New York. (Chapter 2 contains a careful bibliographical analysis of the origin of De Moivre’s theorem.) 2. Czuber, E. (1899). Jb. Dtsch. Math.-Ver., 7, Part 2, 1–279. (Together with W¨olffing [21], the best bibliographical source on probability and statistics in the nineteenth century.) 3. David, F. N. (1962). Games, Gods and Gambling. Charles Griffin, London. 4. Dupont, P. (1979). Atti. Accad. Sci. Torino Cl. Sci. Fis. Mat. Natur., 113, 243–261. 5. Gnedenko, B. V. and Sheynin, O. B. (1978). In Matematika XIX veka (Mathematics of the

nineteenth century). Nauka, Moscow, Chap. 4. (Chapter 4, pp. 184–240, deals with the history of probability, beginning with Laplace.) 6. Gouraud, C. (1848). Histoire du calcul des probabilit´es. A. Durand, Paris. 7. Hacking, I. (1975). The Emergence of Probability. Cambridge University Press, Cambridge, England, Chap. 8. 8. Heyde, C. C. and Seneta, E. (1977). I. J. Bienaym´e: Statistical Theory Anticipated. Springer-Verlag, New York. (A modern source on probability and statistics in the nineteenth century.) 9. Maistrov, L. E. (1967). Teoriya Veroiatnostei: Istoricheskii Ocherk. Nauka, Moscow. (English transl.: Probability Theory. A Historical Sketch, S. Kotz, trans./ed. Academic Press, New York, 1974.) 10. Ore, O. (1953). Cardano, the Gambling Scholar. Princeton University Press, Princeton, N.J. (This book contains a translation by S. H. Gould of Liber de Ludo Aleae.) 11. Ore, O. (1960). Amer. Math. Monthly, 67, 409–419. 12. Pearson, K. (1978). The History of Statistics in the 17th and 18th Centuries. (Lectures by Karl Pearson 1921–1933, edited by E. S. Pearson.) Charles Griffin, London. 13. Pearson, E. S. and Kendall, M. G., eds. (1970). Studies in the History of Statistics. Charles Griffin, London; Hafner, Darien, Conn. (Vol. II, M. G. Kendall and R. L. Plackett, eds., 1977). 14. Seneta, E. (1979). In Interactive Statistics, D. McNeil, ed. North-Holland, Amsterdam, pp. 225–233. 15. Sheynin, O. B. (1974). Arch. History Exact Sci., 12, 97–141. 16. Sheynin, O. B. (1977). Arch. History Exact Sci., 17, 201–259. 17. Sheynin, O. B. (1978). Arch. History Exact Sci., 18, 245–300. 18. Stigler, S. M. 503–517.

(1975).

Biometrika,

62,

19. Todhunter, I. (1865). A History of the Mathematical Theory of Probability from the Time of Pascal to That of Laplace. Cambridge University Press, Cambridge, England. (Reprinted in 1949 and 1961 by Chelsea, New York.) 20. Ulbricht, K. (1980). Wahrscheinlichkeitsfunktionen im neunzehnten Jahrhundert. Minerva, Munich. (Contains a good bibliography and is valuable in presenting a German standpoint, possibly under-represented in the article.)

PROBABILITY INEQUALITIES FOR SUMS OF BOUNDED RANDOM VARIABLES 21. W¨olffing, E. (1899). Math. Naturwiss. Ver. Wurttemberg ¨ (Stuttgart), Mitt., (2), 1, 76–84. (Supplement to Czuber [2].) See also BERNOULLIS, THE; BERNOULLI’S LIMIT THEOREM; BINOMIAL DISTRIBUTION; CHEBYSHEV’S INEQUALITY; ´ ), PAFNUTY LVOVICH; CHEBYSHEV (or TCHEBICHEF COMBINATORICS; CONVERGENCE OF SEQUENCES OF RANDOM VARIABLES; DE MOIVRE, ABRAHAM; DE MOIVRE –LAPLACE THEOREM, GLOBAL; FERMAT, PIERRE DE; GAMES OF CHANCE; LAPLACE, PIERRE SIMON; LIMIT THEOREM, CENTRAL; MARKOV, ANDREI ANDREEVICH; ´ MONTMORT, PIERRE REMOND DE; PASCAL, BLAISE; ´ -DENIS; PROBLEM OF POINTS; POISSON, SIMEON RANDOM WALKS; and STATISTICS, HISTORY OF).

E. SENETA

PROBABILITY INEQUALITIES FOR SUMS OF BOUNDED RANDOM VARIABLES

Pr[|S − ES|  x(var S)1/2 ]
x−2 .

(1)

If S is the sum of n independent, identically distributed random variables, then, by the central limit theorem∗ , as n → ∞, the probability on the left approaches 2(−x), where (x) is the standard normal distribution function. For x large, (−x) behaves as const. x−1 exp(−x2 /2). S. Bernstein∗ [2,3] has shown that under an additional assumption, which is satisfied when the summands of S are uniformly bounded, the upper bound in (1) can be replaced by one which, for n large, behaves as exp(−x2 /2). We first discuss Bernstein’s inequality∗ and related results for sums of independent random variables and random vectors, and then extensions of some of these results to certain sums of dependent random variables. SUMS OF INDEPENDENT RANDOM VARIABLES Let X1 , . . . , Xn be n independent real-valued random variables with finite means, S = X1 + · · · + Xn ,

µ = ES/n.

We begin with inequalities that involve no moments other than µ. Theorem 1. If 0
Xi
1, i = 1, . . . , n, then for 0 < t < 1 − µ, Pr[S − ES  nt]  µ+t  1−µ−t n µ 1−µ
µ+t 1−µ−t

(2)

(3)


exp{−g(µ)nt2 }

(4)


exp{−2nt2 },

(5)

where

g(µ) =

If S is a random variable with finite mean and variance, the Bienaym´e –Chebyshev inequality states that for x > 0,

6473

 1 1−µ   ln    1 − 2µ µ     

1 2µ(1 − µ)

for 0 < µ < 12 , (6)

for

1 2


µ < 1.

(7)

[Note: If t > 1 − µ, the probability in (3) is zero.] The assumption 0
Xi
1 has been made to give the bounds a simple form. If, instead, we assume that a
Xi
b, then µ and t in the statement of Theorem 1 are to be replaced by (µ − a)/(b − a) and t/(b − a), respectively. Upper bounds for Pr[S − ES
−nt] = Pr[−S + ES  nt] and for Pr[|S − ES|  nt] = Pr[S − ES  nt] + Pr[S − ES
−nt] follow from Theorem 1 by appropriate substitutions. Similar remarks apply to the inequalities that follow. Let σ 2 = n−1 var S.

(8)

Under the conditions of Theorem 1, σ 2
µ(1 − µ), with equality holding only in the case Pr[Xi = 0] = 1 − µ,

Pr[Xi = 1] = µ, i = 1, . . . , n.

(9)

6474

PROBABILITY INEQUALITIES FOR SUMS OF BOUNDED RANDOM VARIABLES

For this (binomial) case inequality (3) is implicitly contained in Chernoff [5], and the other bounds of Theorem 1, except for (4) with µ < 12 , are due to Okamoto [7]. In the general case Theorem 1 was proved by Hoeffding [6]. If the inequalities of Theorem 1 are written as Pr[S − ES  nt]
A1
A2
A3 , then each Ai is of the form ani , where ai is a function of µ and t only. Each of these bounds is simpler but cruder than the preceding. We have A1 = A2 if t = 1 − 2µ and A2 = A3 if µ = 12 . If t and µ are fixed as n → ∞, in which case Pr[S − ES  nt] → 0 exponentially fast, and if the inequality Ai < Ai+1 is strict, then the bound Ai is appreciably smaller than Ai+1 when n is large. On the other hand, if we put t = yn−1/2 and hold y and µ fixed, then, as n → ∞,  A1 → exp −

 y2 , 2µ(1 − µ)

A2 = exp(−g(µ)y2 ), A3 = exp(−2y2 ). Note that the limit of A1 is equal to A2 for 1 2
µ < 1. When the central limit theorem applies to S, Pr[S − ES  n1/2 y] → (−y/σ ) 
(−y/ µ(1 − µ)). The following is an extension of bound (5); see Hoeffding [6]. Theorem 2. If X1 , . . . , Xn are independent and ai
Xi
bi (i = 1, . . . , n), then for t > 0, Pr[S − ES  nt]   n
exp −2n2 t2 / (bi − ai )2 . 1

(10)

We now assume that the Xi have a common mean. For simplicity the mean is taken to be zero. Theorem 3. If X1 , . . . , Xn are independent, σ 2 < ∞, EXi = 0, Xi
b(i = 1, . . . , n), then for 0 < t < b,

Pr[S  nt] 


1+

bt σ2

−(1+bt/σ 2 )σ 2 /(b2 +σ 2 )

2 2 2 n  t −(1−t/b)b /(b +σ ) × 1− b    σ2 nt 1+
exp − b bt   !" bt × ln 1 + 2 − 1 σ  " nt2
exp − . 2(σ 2 + bt/3)



(11)

(12) (13)

Here the summands are assumed to be bounded only from above. However, to obtain from Theorem 3 upper bounds for Pr[|S|  t], we must assume that they are bounded on both sides. The bounds in (3) and (11) are related as follows. If the assumptions of Theorem 3 are satisfied and the Xi are also bounded from below, a
Xi
b (where a < 0 < b), then σ 2
−ab. The bound in (11) is an increasing function of σ 2 . If we replace σ 2 by its upper bound −ab, we obtain from (11) the inequality which results from (3) when the appropriate substitutions mentioned after the statement of Theorem 1 are made. Note, however, that Theorem 1 does not require that the Xi have a common mean. The bounds for Pr[S  nt] in (11) to (13) are due to Hoeffding [6], Bennett [1], and Bernstein [3], respectively. For proofs of Bernstein’s inequality, see also Uspensky [8] and Bennett [1]. For an early version of (13), see Bernstein [2]. Bernstein [3] derived an upper bound for Pr[|S|  nt] similar to (13) without assuming that the Xi are bounded. Instead, he assumed that the moments of the Xi satisfy |EXim |
12 (EXi2 )m!cm−2 , m = 3, 4, . . . ;

i = 1, . . . , n,

(14)

with some constant c > 0. If |Xi |
b for all i, then (14) is satisfied with c = b/3. The inequalities of Theorem 3 can be written Pr[S  nt]
B1
B2
B3 .

PROBABILITY INEQUALITIES FOR SUMS OF BOUNDED RANDOM VARIABLES

Each Bi is of the form bni , where bi is a function of t/b and σ/b only. If t, b, and σ are fixed as n → ∞, remarks similar to those concerning Theorem 1 apply. If we put t = xσ n−1/2 and hold x, b, and σ fixed, then, as n → ∞, Bi → exp(−x2 /2),

i = 1, 2, 3.

When the central limit theorem applies to S, Pr[S  xσ n1/2 ] → (−x). Thus in this case the simple bound (13) is nearly as good as the sharper bounds (11) and (12) when n is sufficiently large. For further comparisons between different bounds, see Bennett [1] and Hoeffding [6]. Bernstein’s exponential bound (13) has been extended to the cases of sums of independent random vectors taking values in a Euclidean space, a Hilbert space, and a Banach space. See Yurinskiˇı [9], where references to earlier work are given. SUMS OF DEPENDENT RANDOM VARIABLES Some of the inequalities discussed above have been extended to certain types of dependent random variables. (a) Martingales∗ . Let X1 , . . . , Xn be random variables with finite means, Xi =
j Xi − EXi , Sj = 1 Xi , and suppose that the sequence S1 , . . . , Sn is a martingale, that is,  |X1 , . . . , Xj ] = 0, j = 1, . . . , n − 1, with E[Xj+1 probability 1. Bernstein [4] showed that a version of the inequality named for him holds in the present case. Theorems 1 and 2 remain true if the Sj sequence is a martingale [6, p. 18]. (b) U-statistics∗ . Let X1 , . . . , Xn be i.i.d., and consider the U-statistic U=

1  φ(Xi , . . . , Xir ), n,r n(r)

n  r,

where φ is a measurable function,
n(r) = n(n − 1) . . . (n − r + 1), and the sum n,r is taken over all r-tuples i1 , . . . , ir of distinct positive integers not exceeding n. If 0
φ(x1 , . . . , xr )
1, then the bounds of Theorem 1 with n replaced by [n/r] (the largest integer
n/r) and µ = Eφ(X1 , . . . , Xr )

6475

are upper bounds for Pr[U − EU  t]. If σ 2 = var φ(X1 , . . . , Xr ) exists and φ(x1 , . . . , xr )
EU + b, then the bounds of Theorem 3 with n replaced by [n/r] are upper bounds for Pr[U − EU  t]. Similar results hold for k-sample Ustatistics and for statistics of the form V = n−r

n  i1 =1

···

n 

φ(Xi1 , . . . , Xir ).

ir =1

(c) Sums of finitely dependent random variables. Let the random variables X1 , . . . , Xn be (r − 1)-dependent; that is, the random vectors (X1 , . . . , Xi ) and (Xj , . . . , Xn ) are independent if j − i  r. Suppose also that X1 , . . . , Xn are identically distributed. Let µ = EX1 , S = X1 + · · · + Xn . Under the appropriate boundedness conditions the bounds of Theorems 1 and 3 with n replaced by [n/r], are upper bounds for Pr[S − ES  nt]. (d) Sampling from a finite population. Consider a finite population∗ P = {1, . . . , N} of size N and a real-valued function f defined on P. Let (Y1 , . . . , Yn ) be a simple random sample without replacement from P, Xi = f (Yi ), S = X1 + · · · + Xn . Let µ =
2 EX1 , σ 2 = N −1 N j=1 (f (j) − µ) . If the appropriate boundedness conditions on f (j) are satisfied, the inequalities of Theorems 1 and 3 hold in this case. REFERENCES 1. Bennett, G. (1962). J. Amer. Statist. Ass., 57, 33–45. 2. Bernstein, S. N. (1924). Uˇcen. Zap. Nauc.ˇ Issled. Kafedr Ukrainy, Otd. Mat., 1, 30–49 (in Russian). [Reprinted in S. N. Bernstein, Collected Works, Vol. IV. Nauka, Moscow, 1964 (in Russian).] 3. Bernstein, S. N. (1927). Probability Theory (in Russian). (Referred to in Uspensky [8].) 4. Bernstein, S. N. (1937). Dokl. Akad. Nauk SSSR, 17, 275–277 (in Russian). [Reprinted in S. N. Bernstein, Collected Works, Vol. IV. Nauka, Moscow, 1964 (in Russian).] 5. Chernoff, H. (1952). Ann. Math. Statist., 23, 493–507. 6. Hoeffding, W. (1963). J. Amer. Statist. Ass., 58, 13–30. 7. Okamoto, M. (1958). Ann. Inst. Statist. Math., 10, 20–35.

6476

PROBABILITY INTEGRAL TRANSFORMATIONS

8. Uspensky, J. V. (1937). Introduction to Mathematical Probability. McGraw-Hill, New York. 9. Yurinskiˇı, V. V. (1976). J. Multivariate Anal., 6, 473–499. See also BERNSTEIN’S INEQUALITY; BIRNBAUM-RAYMOND-ZUCKERMAN INEQUALITY; and CHEBYSHEV’S INEQUALITY.

W. HOEFFDING

PROBABILITY INTEGRAL TRANSFORMATIONS The classic probability integral transformation (PIT) theorem can be stated as follows. Theorem 1. If a random variable X has a continuous distribution function F(x), then the random variable U = F(X) has a uniform distribution on the interval (0, 1), that is, is a U(0, 1) random variable (see UNIFORM DISTRIBUTION MODULO 1). This result is given in a number of recent books, including refs. 9 (p. 408) and 19 (p. 203). If the distribution function is assumed to be absolutely continuous, so that a density function exists, then the PIT is easily established using the transformation of densities technique. The earliest use of this result of which we are aware was by R. A. Fisher∗ [4] in his famous 1930 paper in which he introduced the theory of fiducial∗ limits or intervals. Although he did not discuss the PIT explicitly as such, his implicit use of it to construct fiducial intervals leaves no doubt that he understood it (see FIDUCIAL INFERENCE). Let T be a statistic with continuous distribution function Fθ (t), and suppose that we wish to estimate θ . For fixed values of t1 , u1 , and u2 (0
u1
u2
1), put t = {θ : u1
Fθ (t)
u2 }. Then if t is an interval, it is called a 100(u2 − u1 )% fiducial interval for θ . This construction was clearly motivated by the PIT. Fisher [5] again used the PIT in 1932 in the fourth edition of Statistical Methods for Research Workers, when he proposed a method for combining tests of significance. Again, he did not discuss the PIT explicitly, and this omission caused considerable

confusion among readers (see ref. 21). His combined test of significance is developed as follows. Suppose that T1 , T2 , . . . , Tk are k test statistics that are independent when the k corresponding null hypotheses hold. Let p1 , . . . , pk denote the observed significance levels or p values∗ of these tests. Then by Theorem 1, the p values are independent, identically distributed (i.i.d.) U(0, 1) random variables. If it is desired to test that all the null hypotheses are correct simultaneously against the alternative that at least one is incorrect, then an intuitively appealing test can be made by rejecting when the product of the p-values, say Q = p1 · · · pk , is too small, or equivalently, when −2 ln Q is too large. However, since p1 , . . . , pk are i.i.d. U(0, 1) random variables, it is readily shown that −2 ln p1 , . . . , −2 ln pk are i.i.d. χ 2 (2)
random variables; therefore, −2 ln Q = −2 ln pi is a χ 2 (2k) random variable, and the test is easily made by rejecting when −2 ln Q exceeds χα2 (2k). The principal uses of the PIT in statistics consider a random sample X1 , . . . , Xn from a parent distribution function F(·), and the i.i.d. U(0, 1) random variables U1 = F(X1 ), . . . , Un = F(Xn ). If X(1) , . . . , X(n) and U(1) , . . . , U(n) , denote the order statistics∗ from the samples X1 , . . . , Xn and U1 , . . . , Un , respectively, then it follows that U(j) = F(X(j) ) for j = 1, . . . , n, by the monotonicity of F. These results are fundamental to the development of important fields of statistics such as the distribution theory of order statistics∗ (see ref. 3) and tolerance regions∗ theory (see ref. 6). The PIT result has an important ‘‘inverse’’ which we consider next. If X is a random variable with distribution function F(x)—continuous or not—mapping the real number line to the unit interval (0, 1), we define an inverse function, say F − , that maps the unit interval to the real number line, by F − (u) = inf{x : F(x)  u} for 0 < u < 1. If U is a U(0, 1) random variable and if Y = F − (U), then X and Y are identically distributed random variables [see ref. 19 (p. 203)]. This result also has many important applications in statistics. One use is to generate observations from an arbitrary distribution in simulation∗ studies by drawing

PROBABILITY INTEGRAL TRANSFORMATIONS

observations on a U(0, 1) random variable and transforming them with this result. By combining the two transformations above, we can transform a random variable with any particular continuous distribution to one with any other distribution. That is, if X has continuous distribution function F and Y has any distribution function G, then Y and the random variable W = G− (F(X)) are identically distributed. If F and G are both continuous, this is sometimes called a quantile transformation∗ , since it transforms the pth quantile of the distribution of X to the pth quantile of the distribution of Y for 0 < p < 1. Note the generality of these transformation results. They hold for all continuous distributions! We note also a point that will arise later in different contexts. The results hold even if the distribution functions depend on additional variables; those that will be of interest later will be either distribution parameters or random variables that are conditionally fixed.

MULTIVARIATE PROBABILITY INTEGRAL TRANSFORMATION (MPIT) The PIT is sometimes used to transform the elements of a given sample to a sample from a U(0, 1) distribution. We now consider a result due to Rosenblatt [20] that shows how to transform a set of possibly dependent continuous random variables to a set of i.i.d. U(0, 1) random variables. Let (X1 , . . . , Xk ) be a random vector with an absolutely continuous distribution function F(x1 , . . . , xk ). Then put (u1 , . . . , uk ) = H(x1 , . . . , xk ), where the transformation H is defined by u1 = P{X1
x1 } ≡ F1 (x1 ), u2 = P{X2
x2 |X1 = x1 } ≡ F2 (x2 |x1 ), .. . uk = P{Xk
xk |X1 = x1 , . . . , Xk−1 = xk−1 } ≡ Fk (xk |x1 , . . . , xk−1 ). For u = (u1 , . . . , uk ) , put Su = {(u1 , . . . , : uj
uj , j = 1, . . . , k}, and the distribution function of U ≡ H(X1 , . . . , Xk ) is given by uk )

6477

P(Su ) = P{Uj
uj , j = 1, . . . , k}, = P{X1
x1 }P{X2
x2 |x1 } · · · ×P{Xk
xk |x1 , . . . , xk−1 } = u1 u2 . . . uk , and 0
uj
1; j = 1, . . . , k. We have the following result. Theorem 2. If (X1 , . . . , Xk ) is a random vector with absolutely continuous distribution function F(x1 , . . . , xk ), then U1 = F1 (X1 ), U2 = F2 (X2 |X1 ), . . . , Uk = Fk (Xk |X1 , . . . , Xk−1 ) are i.i.d. U(0, 1) random variables. This result can be used in problems in statistical inference for multivariate distributions. If X1 , . . . , Xn is a sample of k-variate independent vector observations from an absolutely continuous multivariate distribution, then Theorem 2 shows how these observations can be transformed to nk i.i.d. U(0, 1) random variables. If, for example, the distribution function is known under a simple goodnessof-fit∗ null hypothesis, then a test for this hypothesis can be constructed by utilizing the i.i.d. U(0, 1) distributions of the nk U’s under the null hypothesis. It should be observed that the random variables U1 , . . . , Uk in Theorem 2 depend on the order of X1 , . . . , Xk . If these X’s are written in a different order, then a set of i.i.d. U(0, 1) random variables will be obtained, but they will, in general, be different random variables. This lack of invariance to permutations of the X’s can be important in some contexts. CONDITIONAL PROBABILITY INTEGRAL TRANSFORMATIONS (CPIT) When an observed sample is at hand, in order to actually apply either the classic PIT of Theorem 1 or the multivariate PIT of Theorem 2, it is necessary to know the distribution function required exactly. This requirement limits the use of such transformations in many statistical problems, since we often can assume only that the distribution function is a member of some specified class of distribution functions. This fact led Johnson and David [7] in an early pioneering

6478

PROBABILITY INTEGRAL TRANSFORMATIONS

paper to replace the parameters in a distribution function (df) by estimators. They showed that for location-scale classes, the transformed sample values, using this df with estimated parameters, have distributions that do not depend on the values of the parameters. In this section we give some transformations which can be used when the parent distribution function for a sample is assumed only to be a member of a parametric class of distribution functions. Although the transformations we consider here generalize to multivariate or more complex families of distributions, such as distributions dependent on structural variables, the initial presentation will be for i.i.d. univariate samples. Let X1 , . . . , Xn be i.i.d. random variables with common distribution function F assumed to be a member of a parametric class F = {Fθ : θ ∈ } of continuous distribution functions. Suppose that the family F admits a v-component sufficient statistic∗ Tn , which we assume, without loss of generality, is a symmetric function∗ of X1 , . . . , Xn , or, equivalently, that Tn is a function of the order statistics of this sample. Let F˜ n (x1 , . . . , xα ) denote the conditional distribution function defined by

is a maximum when Tn is a minimal sufficient statistic. This theorem is most often useful when F is a parametric family with, say, v functionally independent parameters. In many common cases the maximum value of α for which F˜ n (x1 , . . . , xα ) is absolutely continuous is then α = n − v, that is, the sample size reduced by the number of parameters. Even though Theorem 3 gives general transformation results, in particular cases writing out the transformations can itself be a difficult task, so it is worthwhile to consider particular techniques and results that are helpful. The next result is given for this purpose. A sequence of statistics (Tn )n1 is doubly transitive if for each n we can compute Tn+1 from Tn and Xn+1 ; and conversely, if we can compute Tn from Tn+1 and Xn+1 . Many important families of distributions have doubly transitive minimal sufficient statistics. When this is the case, the following result from [10] gives the transformations of Theorem 3 in a form that is easier to find in many problems.

F˜ n (x1 , . . . , xα ) = E{I[X1
x1 ,...,Xα
xα ] |Tn },

U1 = F˜ n−α+1 (Xn−α+1 ), . . . , Uα = F˜ n (Xn )

α ∈ {1, . . . , n}. This is an unbiased estimating distribution function for F(x1 , . . . , xα ); when Tn is complete as well as sufficient, it is the minimum variance unbiased∗ (MVU) estimating distribution function. It is sometimes called the Rao-Blackwell (R-B) distribution function. The basic result is stated using these R-B distribution functions. Theorem 3. If F˜ n (x1 , . . . , xα ) is absolutely continuous for an α ∈ {1, . . . , n}, then the α random variables U1 = F˜ n (X1 ), U2 = F˜ n (X2 |X1 ), Uα = F˜ n (Xα |X1 , . . . , Xα−1 )

···,

are i.i.d. U(0, 1) random variables. This result is proved in O’Reilly and Quesenberry [10], where it is shown also that α

Theorem 4. If F˜ n (x1 , . . . , xα ) is absolutely continuous for α ∈ {1, . . . , n}, and (Tn )n1 is doubly transitive, then

is a set of α i.i.d. U(0, 1) random variables. The R-B distribution functions are known for many classes of distributions, and the transformations of Theorem 4 can be readily obtained explicitly for these classes. Example: Normal Distributions. Suppose that X1 , . . . , Xn is a random sample 2 distribution. Then from a N(µ,
σ ) parent
(X n , S2n ) = ( Xj /n, (Xj − X)2 /n) is a doubly transitive minimal sufficient statistic, since one can compute (X n+1 , S2n+1 , Xn+1 ) from (X n , S2n , Xn+1 ) and conversely. Moreover, the R-B distribution function is given (see ref. 8) for r > 2 by   z − X r < −(r − 1)1/2 Sr , 0, ˜ Fr (z) = 1, (r − 1)1/2 Sr < z − X r ,   Gr−2 (Ar ), elsewhere,

PROBABILITY INTEGRAL TRANSFORMATIONS

(r − 2)1/2 (z − X) Ar =

1/2 , (r − 1)S2r − (z − X r )2 where Gr−2 is a Student t-distribution∗ function with (r − 2) degrees of freedom. Using this result in Theorem 4, we see that the n − 2 random variables given by Ur−2 = Gr−2 (Br ), (r − 2)1/2 (Xr − X r ) Br =

1/2 , (r − 1)S2r − (Xr − X r )2 for r = 3, . . . , n, are i.i.d. U(0, 1) random variables. The conditioning approach discussed above can be used in some cases for models more complex than i.i.d. univariate samples. Multivariate normal distributions were considered in ref. 10 and in more detail in refs. 17 and 18. In these papers the transformations were used to establish goodness-of-fit tests for multivariate normality. The assumption that the original values X1 , . . . , Xn are identically distributed can also be relaxed in some important models such as regression or design models with normal errors. Some design models have been considered in ref. 15, which gives a multiple samples test of normality, and in ref. 16, which gives a test for a one-way analysis of variance∗ model. The independence assumption of the variables X1 , . . . , Xn may also be relaxed in Theorem 3; however, the relatively simple approach of Theorem 4 is not then available, and finding the transformations is in most cases a formidable task. One place where this approach is potentially helpful is the following. It was pointed out earlier that the MPIT is not a symmetric function of X1 , . . . , Xn , and depends on the order of the variables. The same is true for the CPIT, in general; this has a number of practical consequences. First, when the transformations are used to establish a test of fit, it is possible to obtain different test results from different orderings of the same data. The probability of agreement of tests based on different randomly selected permutations of the sample has been considered in ref. 13, where

6479

this probability was found to be high in a large class of cases. Another approach is to replace (X1 , . . . , Xα ) in Theorem 3 by a subset of α order statistics, say (X(1) , . . . , X(α) ), where X(1)
X(2)
· · ·
X(n) . Then the transformed values will be functions only of the order statistics and will therefore be symmetric functions of the original sample. The task of finding the actual transformation is difficult with this approach, and has been solved only for two rather simple cases of uniform classes [12] and scale parameter exponential classes [11]. The transformations obtained for the exponential distribution∗ are characterizations of the distribution that are closely related to other characterizing transformations studied in ref. 2. A second consequence of the order dependence of the CPIT transformations is that it may be exploited to investigate the dependence of a data set on an ordering variable, such as time. FURTHER REMARKS ON CPITS The CPIT approach is useful mainly for parametric classes of distributions that admit fixed-dimension sufficient statistics∗ . Such classes are, under regularity conditions, exponential classes, so that it appears that this method is useful largely just for exponential classes. However, in ref. 12 an extension of the foregoing conditioning approach is considered which gives results for many important families that are not exponential classes because their support is a function of parameters, such as uniform or Pareto distributions∗ . The transformations of Theorem 4 possess interesting and useful properties. They have important invariance properties [14]. For a parent class of distribution functions F that has an associated class of transformations G, say, these transformations will themselves be invariant under the members of G. To illustrate, suppose that F is a location-scale parametric family of distributions. Then the transformations will be invariant under linear transformations: g(xj ) = axj + b; j = 1, . . . , n; a > 0. This invariance can readily be verified for the transformations of the normal distribution in the preceding section. The CPITs possess

6480

PROBABILITY INTEGRAL TRANSFORMATIONS

similar invariance properties quite generally. This invariance property can sometimes be helpful in the derivation of most powerful invariant tests for separate hypothesis testing∗ problems (see ref. 14), or, to obtain optimal invariant selection statistics (see MODEL CONSTRUCTION: SELECTION OF DISTRIBUTIONS). If the sufficient statistic T is also complete, then T = (T1 , . . . , Tv ) and U = (U1 , . . . , Un−v ) are independent, since the distribution of U is the same for every parent distribution in the class F, for which T is sufficient and complete [1] (see also BASU THEOREMS). This result has important implications. T contains all the information about the parameters of F, that is, all information on which inferences can be made within the class F. Thus U can be used to make inferences about the class F, perhaps to make goodness-of-fit tests or conduct model selection procedures, and T can be used to make a parametric test or other inference within F, and the independence of U and T exploited to assess overall error rates. The remarks above suggest that the transformations are useful largely for investigating the correctness of the parent class F, which will usually be posed under a null hypothesis. It is likely that additional applications of the conditioning approach to transformations and inference described here will be developed. REFERENCES 1. Basu, D. (1955). Sankhya, ¯ 15, 377–380. (The now well-known condition for a statistic to be independent of a complete sufficient statistic was given.) 2. Cs¨org¨o, M. and Seshadri, V. (1971). Z. Wahrschlein. verw. Geb., 18, 333–339. (‘‘Characterizing’’ transformations to uniformity are considered for exponential and normal distributions. See also references to other papers in this paper.) 3. David, H. A. (1981). Order Statistics, 2nd ed. Wiley, New York. (A general reference for order statistics.) 4. Fisher, R. A. (1930). Proc. Camb. Philos. Soc., 26, Part. 4, 528–535. (Fisher introduced ‘‘fiducial’’ intervals and bounds for a parameter. The distribution function was used as a ‘‘pivotal’’ function, and this was the first use of the PIT.)

5. Fisher, R. A. (1932). Statistical Methods for Research Workers, 4th ed. Oliver & Boyd, Edinburgh. (Fisher implicitly used the PIT to define his ‘‘combined test for significance.’’) 6. Guttman, I. (1970). Statistical Tolerance Regions: Classical and Bayesian. Hafner, Darien, Conn. (A general reference for tolerance regions.) 7. Johnson, N. L. and David, F. N. (1948). Biometrika, 35, 182–190. (The members of a sample are transformed using a distribution function with parameters replaced by estimates.) 8. Lieberman, G. J. and Resnikoff, G. J. (1955). J. Amer. Statist. Ass., 50, 457–516. 9. Lindgren, B. W. (1968). Statistical Theory, 2nd ed. Macmillan, New York. 10. O’Reilly, F. J. and Quesenberry, C. P. (1973). Ann. Statist., 1, 79–83. (This paper introduces the important general principle of conditioning a set of RVs on a sufficient statistic and then using this conditional distribution to transform the RVs themselves—the conditional probability integral transformation, CPIT. Results that are helpful for performing these transformations are given for the special case when the original RVs constitute a random sample from an exponential class of distributions.) 11. O’Reilly, F. J. and Stephens, M. A. (1982). J. R. Statist. Soc. B, 44, 353–360. (Obtain the CPIT for order statistics for the scale parameter exponential distribution.) 12. Quesenberry, C. P. (1975). Commun. Statist. A, 4, 1149–1155. (This paper extends the CPIT theory for samples to distributions with ranges dependent on parameters.) 13. Quesenberry, C. P. and Dietz, J. (1983). J. Statist. Comp. Simul., 17, 125–131. (Studies agreement probabilities for CPIT-Neyman smooth tests made on randomly selected permutations of a sample.) 14. Quesenberry, C. P. and Starbuck, R. R. (1976). Commun. Statist. A, 5, 507–524. (A theoretical paper that studies optimality properties of CPITs.) 15. Quesenberry, C. P., Whitaker, T. B., and Dickens, J. W. (1976). Biometrics, 32, 753–759. (Gives a test for normality for multiple samples.) 16. Quesenberry, C. P., Giesbrecht, F. G., and Burns, J. C. (1983). Biometrics, 39, 735–739. (Consider testing normality in a one-way ANOVA model.) 17. Rincon-Gallardo, S. and Quesenberry, C. P. (1982). Commun. Statist. Theor. Meth., 11,

PROBABILITY MEASURES, TIGHTNESS OF

18.

19.

20.

21.

343–358. (An applied paper that considers applying the multivariate normality test of ref. 18.) Rincon-Gallardo, S., Quesenberry, C. P., and O’Reilly, F. J. (1979). Ann. Statist., 7, 1052–1057. (Proposes a multiple samples test for multivariate normality.) Rohatgi, V. K. (1976). An Introduction to Probability Theory and Mathematical Statistics. Wiley, New York. Rosenblatt, M. (1952). Ann. Math. Statist., 23, 470–472. (Brief but important paper that gives the multivariate probability integral transformation.) Wallis, W. A. (1942). Econometrica, 10, 229–248. (Interesting historically since it explains why Fisher’s combined test of significance works. Some earlier writers had given fallacious accounts.)

See also DISTRIBUTION-FREE METHODS; GOODNESS OF FIT; MULTIVARIATE NORMAL DISTRIBUTIONS; QUANTILE TRANSFORMATION METHODS; RANDOM NUMBER GENERATION; and UNIFORM DISTRIBUTION MODULO 1.

C. P. QUESENBERRY

PROBABILITY IN THE ENGINEERING AND INFORMATIONAL SCIENCES This journal commenced publication in 1987; the Editor is Sheldon M. Ross and the Editorial Office is Department of Industrial Engineering and Operations Research, University of California, Berkeley, CA 94720. All papers must be in English. ‘‘This journal will be concerned with articles detailing innovative uses of probability in a variety of areas. Among others, we will be especially interested in applications to computer and information science, the engineering sciences, applied statistics, and reliability theory.’’ [From the Introductory Editorial, Vol. 1, No. 1 (1987).] The journal is published by Cambridge University Press, New York. There is a link at www.cambridge.org/uk/journals/

PROBABILITY, INVERSE. See BAYESIAN INFERENCE

PROBABILITY MEASURE Formally, a probability measure is defined as a measure∗ µ on an arbitrary space  such

6481

that µ() = 1. We then define an event as a measurable set and the probability of an event E as µ(E). The space  is called ‘‘space of elementary events’’ or ‘‘sample space.’’ In applications, a probability measure is associated with our conceptions of the likelihoods of occurrences of various physical events. Probability measures which are defined on the real line (R1 ) result in univariate (one-dimensional) probability distributions, while those on ndimensional Euclidean space yield multivariate (n-dimensional) probability distributions. See MEASURE THEORY IN PROBABILITY AND STATISTICS and PROBABILITY THEORY: AN OUTLINE and entries on specific distributions for more details. BIBLIOGRAPHY Cram´er, H. (1946). Mathematical Methods of Statistics. Princeton University Press, Princeton, N.J., Chaps. 4 to 9. Lo`eve, M. (1979). Probability Theory, 4th ed. Springer-Verlag, New York. (See especially Chaps. I to III.) See also MEASURE THEORY IN PROBABILITY AND STATISTICS.

PROBABILITY MEASURES, TIGHTNESS OF A collection of probability measures {µn } on the real line is tight if for all  > 0 there exist a and b, a < b, with µn ([a, b])  1 −  for all n. In other words, all the measures give most of their mass to the same finite interval; the probability mass does not ‘‘escape to infinity.’’ Any finite collection of probability measures is tight. A subcollection of a tight collection is tight and, finally, the union of two tight collections of probability measures must again be tight. The concept is of special importance in continuity theorems for characteristic functions* [1].

6482

PROBABILITY MINIMUM REPLACEMENT SAMPLING DESIGN

REFERENCE

and

1. Rosenthal, J. S. (2000). A First Look at Rigorous Probability Theory. World Scientific Publishers, Singapore. See also MEASURE THEORY IN PROBABILITY AND STATISTICS.

PROBABILITY MINIMUM REPLACEMENT SAMPLING DESIGN We will use the following notation for the parameters of a sampling design: N:

number of sampling units

S(i) :

size measure associated with

sampling unit(i) N S(+) = S(i)

P{n(i) = m(i)} = 1 − f (i). [Observe that PNR sample designs are a special case of PMR designs with all m(i) equal to zero.] A selection algorithm for a sequential procedure for a PMR sampling and associated unbiased estimation procedures was developed by J. R. Chromy [1]. REFERENCE 1. Chromy, J. R. (1981). In Current Topics in Survey Sampling, D. Krewski, R. Platek, and J. N. K. Rao, eds. Academic Press, New York, pp. 329–347. See also PROBABILITY PROPORTIONAL TO SIZE (PPS) SAMPLING and SURVEY SAMPLING.

i=1

n:

total sample size

n(i) :

number of times sampling unit

PROBABILITY MOMENTS AND FREQUENCY MOMENTS

(i)is selected Probability nonreplacement (PNR) sample designs are (usually) restricted to sampling frames with nS(i)/S(+) < 1, i = 1, . . . , N. When this condition holds, a PNR sample design is one for which Pr(n(i) = 1) = nS(i)/S(+)

For a random variable X having a discrete distribution, with Pr[X = xj ] = pj ,

j = . . . , −1, 0, 1, . . . ,

the rth probability moment is ωr =

∞ 

prj .

(1)

j=−∞

and Pr(n(i) = 0) = 1 − Pr(n(i) = 1). Probability minimum replacement (PMR) sample designs, to be defined below, allow any positive size measures S(i). The standardized size measure may be written in terms of two components nS(i)/S(+) = m(i) + f (i), where m(i) is a nonnegative integer, while 0
f (i)
1. Now, using this notation, a PMR sample design is one for which Pr{n(i) = m(i) + 1} = f (i)

If the distribution is continuous with probability density function f (x), it is defined as  r =

∞ −∞

{f (x)}r dx.

(Note that ω1 = 1 = 1.) These quantities were introduced by Sichel [1,2], who used sample analogs of these parameters in fitting distributions. He also defined ratios  r+2 /2r+1 , r = 1, 2, 3, . . . , αr = (2r+1)/2 (2r+5)/2 /3/2 , r = 12 , 32 , 52 , . . . , as ‘‘measures of kurtosis∗ .’’ For continuous distributions he suggested using ωr , as

PROBABILITY PLOTTING

defined in (1) with a ‘‘discretized’’ (grouped) form of the distribution, as an approximation to r , and termed the sample analogs ‘‘working’’ probability moments. The rth working frequency moment for a sample of size n, is nr times the rth working probability moment. Sichel [2] gives an account of accuracy to be expected using probability moments to estimate parameters of a distribution.

Fn (x) = (number of observations
x)/n. It is a step function increasing from 0 to 1 with jumps of size 1/n at each observation (see EMPIRICAL DISTRIBUTION FUNCTION (EDF) STATISTICS). Fn (x) is a natural, nonparametric estimate of F(x) and its graph can be studied informally in making inferences about F(x). A simultaneous (1 − α)100% confidence band for F(x) is Fn (x) − kα
F(x)
Fn (x) + kα

REFERENCES 1. Sichel, H. S. (1947). J. R. Statist. Soc. Ser. A, 110, 337–347. 2. Sichel, H. S. (1949). Biometrika, 36, 404–425. See also ESTIMATION: METHOD OF MOMENTS and PROBABILITY-WEIGHTED MOMENTS.

PROBABILITY PLOTTING Plots and graphs of a wide variety are used in the statistical literature for descriptive purposes. Visual representations have high impact and they provide insight and understanding not readily found by other means. This article will consider methods for the oneand two-sample problems that have plots as their central focus. These methods are useful for informal data analysis and model assessment, while more formal statistical inferences can also be made. Fisher [12] has given an excellent review. See GRAPHICAL REPRESENTATION OF DATA and GRAPHICAL REPRESENTATION, COMPUTER AIDED for more examples of plots. CUMULATIVE DISTRIBUTION FUNCTION PLOTS The cumulative distribution function (CDF) of a random variable X is F(x) = Pr(X
x). Many features of the distribution of X are apparent in the CDF, such as likelihood regions, location, scale, percentiles, and skewness. A random sample of size n on X is a collection of independent random variables, each having CDF F(x). The observed, ordered observations for the sample are denoted by x1
x2
· · ·
xn . The empirical CDF is

6483

for all x,

where Pr(D
kα ) = 1 − α and D = supx |Fn (x) − F(x)|, the Kolmogorov–Smirnov statistic∗ . The confidence band should be plotted with Fn (x) to assess variability. A formal level α test of a hypothesis H0 : F(x) = F0 (x), for a specified CDF F0 (x), can be carried out with the rule: Accept H0 if F0 (x) lies in the confidence band for all x. The confidence band thus contains all CDFs that would be accepted in this testing problem. Doksum [9] has improved on this graphical method by developing a confidence band using sup a
Fn (x)
b

|Fn (x) − F(x)| {F(x)(1 − F(x))}1/2

!

as a pivot, where 0 < a < b < 1. The weight function in the denominator serves to equalize the variance of the empirical CDF and the interval (a, b) restricts the scope of the band. The resulting confidence band is slightly wider in the middle but is much narrower in the tails when compared to the first band. QUANTILE AND PERCENT PLOTS Suppose that two CDFs F(x) and G(x) are given. These may be empirical or population CDFs. A graphical comparison of the two distributions can be made by plotting the quantiles∗ of one distribution against those of the other. The quantile of order p for the CDF F(x) is a number xp which satisfies F(xp ) = p when F(x) is strictly increasing, but to cover other cases define F −1 (p) = inf {x : F(x)  p} and use xp = F −1 (p). Similarly, let yp be the quantile of order p for G(x). Then a quantile plot, called a Q-Q plot, is the graph of (xp , yp ) for 0 < p < 1.

6484

PROBABILITY PLOTTING

If G(x) = F(x), the Q-Q plot will be a 45◦ line through the origin. If the two distributions differ only in location and scale, G(x) = F((x − µ)/σ ) and yp = µ + σ xp . Thus the Q-Q plot is a straight line with intercept µ and slope σ . This is an important prototype case since the eye can readily perceive straight lines and identify slope and intercept. Departures from linearity can be easily spotted and interpreted. In a similar manner, a percent plot, called a P-P plot, is constructed by graphing the points (F(x), G(x)) for −∞ < x < ∞. This plot will trace a path from (0, 0) to (1, 1). If G(x) = F(x), the plot will be a straight line. Unlike the Q-Q plot, the P-P plot will not be linear in the case of only location-scale differences. Differences in the middles of distributions are usually more apparent for P-P plots than for Q-Q plots. Generalization to multivariate cases is easier with percent plots. The basic concepts behind these plots can be used in other creative ways. For example, ref. 16 has some good examples of plots of functions h1 (xp , yp ) versus h2 (xp , yp ) or just p. Probability plots are a useful tool in data analysis. One can assess whether a single random sample arises from a distribution with CDF F0 (x) by constructing a Q-Q or a P-P plot of F0 (x) against the empirical CDF Fn (x) of the sample. The Q-Q plot would be represented by points (F0−1 [(i − 1/2)/n], xi ) plotted for i = 1, . . . , n, where x1
· · ·
xn are the ordered observations. The first coordinates are quantiles of order (i − 12 )/n for F0 (x). Alternatively, quantiles of order i/(n + 1) can be used. If the plot follows a 45◦ line through the origin, the CDF F0 (x) would informally be accepted. Departures from this line would indicate differences between F0 (x) and the actual CDF. Location or scale differences, heavier tails, existence of outliers∗ , regions of poor fit, and other peculiarities may be noticed. The Kolmogorov-Smirnov test could be used for a more formal test. Probability plotting has been widely used for location-scale models, where the observations are assumed to arise from a distribution with CDF F0 ((x − µ)/σ ), with F0 known but µ and σ unknown. A Q-Q plot of F0 (x) against Fn (x), as given in the preceding paragraph, will informally test the adequacy of the model. When F0 (x) is the standard

normal CDF, this is called a normal probability plot. If the location-scale model is correct, the plot should lie along a straight line with intercept µ and slope σ . Departures from linearity would provide insights for choosing an alternative model. See GRAPHICAL REPRESENTATION OF DATA for some examples. Doksum [9] has discussed a formal confidence band that can be graphed and used to test the adequacy of the locationscale model. His procedure is based on a weighted Kolmogorov–Smirnov statistic and has an advantage over the usual Kolmogorov–Smirnov statistic in that µ and σ need not be estimated from the data. OPTIMAL PLOTTING POSITIONS With the probability plot for the locationscale model, the slope σ and intercept µ can be crudely estimated from the graph. To improve on this it is natural to use the leastsquares∗ fitted line. Let the plotted points be denoted by (vi , xi ), where the xi are the ordered observations and the vi are called the plotting positions. The basic Q-Q plot uses quantiles from F0 for the plotting positions. Chernoff and Lieberman [3] considered the problem of determining optimal plotting positions in the sense of minimum mean square error or of unbiased, minimum variance estimates. The resulting optimal plotting positions, given in their paper, are functions of the expectations, variances, and covariances of the order statistics of a sample from F0 (x). An undesirable feature is that the optimal plotting positions need not be in an increasing order. The order statistics∗ used in the plot are correlated and have unequal variances, and as a result a weighted least-squares∗ procedure would be preferable to the ordinary least-squares fit. Cran [4] gave the weighted least-squares solution to the optimal plotting position problem. He also obtained conditions on the weights for the plotting positions to be in increasing order. Barnett [1] noted that the previous results require knowledge of the first and second moments of order statistics, and this information is available only for selected sample sizes and distributions F0 . He suggested an alternative approach which

PROBABILITY PLOTTING

does not require this information and which yields estimates of µ and σ that are nearly optimal. GAMMA PROBABILITY PLOTS The gamma probability distribution is widely used in life testing∗ and reliability∗ models. It is indexed on three parameters: location, scale, and shape. A graphical method for assessing a gamma probability model consists of making an initial estimate of the shape parameter and then constructing a Q-Q plot to check for linearity as in the preceding section. Reference 17 contains several interesting examples of this approach. It illustrates an important application in assessing the assumption of homogeneity of variance in analysis-of-variance∗ problems. Under the usual assumptions of independence, normality, and equal variance, the within-cell sample variances should have a gamma distribution∗ and the gamma plot can be used to spot any discrepancies. HALF-NORMAL PLOTS∗ The half-normal distribution is the distribution of the absolute value of a standard normal random variable; its CDF is 0 (x) = 2(x) − 1 for x > 0, where  is the standard normal CDF (see FOLDED DISTRIBUTIONS). A half-normal probability plot of a sample is a regular Q-Q plot of the quantiles of the absolute values of the data |x1 | < · · · < |xn | against the corresponding quantiles of the half-normal distribution, that is, a plot 1 of (−1 0 [(i − 2 )/n], |xi |) for i = 1, . . . , n. If the sample arises from a normal distribution with mean 0 and variance σ 2 , this plot should follow a straight line through the origin with slope σ . Daniel [5] presented an interesting application of this plot in the analysis of 2p factorial experiments∗ . The usual set of 2p − 1 orthogonal sample contrasts∗ are used for the half-normal plot. Departures from linearity may indicate heteroscedasticity∗ , outliers, or real effects. Formal tests for real effects are given; see also Birnbaum [2]. Zahn [18] gave several improvements for Daniel’s procedure and corrected the critical limits for the test.

6485

GRAPHICAL ANALYSIS FOR TWO-SAMPLE PROBLEMS Plots can be used to detect similarities and differences between two distributions. Interest may focus on locations, scales, shapes, tail weight, symmetries, skewness, stochastic ordering, or other features. Assuming two independent samples, let x1
· · ·
xm be the ordered sample from a distribution with CDF F(x) and let Fm (x) be the empirical CDF of the sample; let y1
· · ·
yn be the ordered sample from a distribution with CDF G(y) and let Gn (y) be the empirical CDF of this sample. The two samples can be compared with a Q-Q plot of the empiricals Fm versus Gn . If the sample sizes are equal, this is simply a plot of the points (xi , yi ) for i = 1, . . . , n. For −1 (p) = unequal sample sizes, recall that Fm inf {x : Fm (x)  p} will take the value xi if (i − 1)/m < p
i/m and similarly for G−1 n . The Q-Q plot is a plot of the points arising from −1 (p), G−1 (p)) for 0 < p < 1 and this will (Fm n consist of points (xi , yi ) for suitable subscripts. THE SHIFT FUNCTION FOR TWO-SAMPLE PROBLEMS The Kolmogorov–Smirnov statistic D = supx |Fm (x) − Gn (x)|, or a weighted version of it, can be used to test the hypothesis of identical CDFs for the two-sample problem. This test can be carried out with an additional emphasis on plotting and graphical analysis to provide more information and insight than the basic accept–reject decision. A discussion of this can be found in Doksum and Sievers [9] and Doksum [6,7,9]. To motivate this approach, consider the widely used shift model F(x) = G(x + ) for all x, where the Y-population is shifted a uniform amount  from the X-population. Comparing populations by a simple shift may often be unrealistic, however. More generally, a shift function (x) can be defined by the equation F(x) = G(x + (x)) for all x. It follows that (x) = G−1 (F(x)) − x (see Fig. 1). Under general conditions (x) is the only function for which X + (X) has the same distribution as Y. We say that (X) is the amount of shift to bring the X variable up to

6486

PROBABILITY PLOTTING

be graphed to explore its properties. The plot of G−1 n (Fm (x)) at the values xi is basically the regular Q-Q plot. A simultaneous confidence band for (x) can be constructed for more formal inferences. One such band is based on the Kolmogorov–Smirnov statistic D . With kα chosen so that Pr(D
kα |F = G) = 1 − α, a simultaneous (1 − α)100% confidence band for (x) is given by yi1 − xi
(x) < yi2 − xi Figure 1. Shift function (x) between two CDFs F and G.

for xi
x < xi+1 , where x0 = −∞, xm+1 = ∞,

the Y. When X is a control response and Y a treatment response, it may be reasonable to interpret (x) as the amount that the treatment adds to a control response x. This effect need not be constant. The set {x : (x) > 0} identifies the part of the population where the treatment has a positive effect. When the pure shift model holds, (x) = , a constant. When a location-scale model holds, G(x) = F((x − µ)/σ ) and (x) = µ + (σ − 1)x, a linear function. A natural, nonparametric estimate of (x) ˆ is (x) = G−1 n (Fm (x)) − x and this function can

ˆ Figure 2. Shift function (x) and an 80% confidence band for simulated samples of sizes n = m = 50, where X and Y are N(0, 1) and N(1, 4), respectively.

i1 = n((i/m) − kα ), i2 = [n((i/m) + kα )] + 1, t is the least integer greater than or equal to t, and [t] is the greatest integer less than or equal to t. An example is given in Fig. 2. The confidence band can be used to compare the two distributions in various ways. Does it contain a horizontal line (a pure shift model is accepted), a line through the origin (0, 0) (a pure scale model is accepted), or any straight line (a location-scale model is accepted)? A confidence set will be visible

PROBABILITY PLOTTING

for the set {x : (x) > 0} and similarly other conclusions can be reached, as many in number as desired because of the simultaneous nature of the band. Doksum [9,10] gave several other nonparametric bands for (x) that can be more efficient than the band discussed here. Also, special bands were developed for the locationscale case. In life testing and reliability studies, multiplicative effects have more direct interest than additive effects. For such cases a scale function θ (x) = G−1 (F(x))/x can be useful. Doksum [8] has developed appropriate graphical analyses and interesting applications for the scale function, including methods for the Weibull∗ and exponential∗ models. PLOTTING FOR SYMMETRY Suppose that X is a random variable with a continuous CDF F(x). When the distribution is symmetric about a point θ , this point is clearly the measure of location or center. In asymmetric cases, the location is not so clear. Many statistical procedures require symmetric distributions and it is useful to have methods to check on this assumption. Doksum [7] has discussed some useful graphical methods. He defined a symmetry function by θ (x) = (x + F −1 (1 − F(x)))/2; it is essentially the only function for which X − θ (X) and −(X − θ (X)) have the same distributions. For symmetric distributions θ (x) = θ , a constant. When the distribution is skewed to the right (left), θ (x) is U-shaped (upsidedown U-shaped). He also discussed a location interval [θ, θ ], where θ = infx θ (x) and θ = supx θ (x), and showed that all reasonable location measures lie in this interval. The empirical CDF Fn (x) of a sample is used to estimate θ (x) with θˆ (x) = (x + Fn−1 (1 − Fn (x)))/2. Using the ordered data x1 < · · · < xn , the graph of θˆ (x) is basically given by a plot of the points (xi , (xi + xn+1−i )/2) for i = 1, . . . , n. This plot and a simultaneous confidence band should be plotted to assess asymmetry and the location interval.

6487

REFERENCES 1. Barnett, V. (1976). Appl. Statist., 25, 47–50. (A paper on plotting positions.) 2. Birnbaum, A. (1959). Technometrics, 1, 343–357. (On theory of hypothesis testing for 2p factorials, half-normal plots.) 3. Chernoff, H. and Lieberman, G. (1956). Ann. Math. Statist., 27, 806–818. (Basic paper on estimation of location and scale; theoretical results.) 4. Cran, G. W. (1975). J. Amer. Statist. Ass., 70, 229–232. (Use of weighted least-squares line.) 5. Daniel, C. (1959). Technometrics, 1, 311–341. (Basic reference on half-normal plots for 2p factorials; readable, examples.) 6. Doksum, K. (1974). Ann. Statist., 2, 267–277. (First paper on shift function, theoretical.) 7. Doksum, K. (1975). Scand. J. Statist., 2, 11–22. (On plots for asymmetry, theoretical, has examples.) 8. Doksum, K. (1975). In Reliability and Fault Tree Analysis. SIAM, Philadelphia, pp. 427–449. (Focus on plotting with life testing, reliability data; lots of examples and applications.) 9. Doksum, K. (1977). Statist. Neerlandica, 31, 53–68. (Good survey on a wide range of plotting methods; examples and applications, readable.) 10. Doksum, K. and Sievers, G. (1976). Biometrika, 63, 421–434. (Basic paper on plotting for shift functions, location/scale model; theory and discussion.) 11. Fisher, N. I. (1981). Austral. J. Statist., 23, 352–359. 12. Fisher, N. I. (1983). Int. Statist. Rev., 51, 25–58. (Good review of many graphical methods; extensive bibliography.) 13. Hoaglin, D. C. (1980). Amer. Statist., 34, 146–149. (Plots for Poisson and other discrete distributions; readable.) 14. Nair, V. N. (1981). Biometrika, 68, 99–103. (Plots for censored data.) 15. Schweder, T. and Spjøtvoll, E. (1982). Biometrika, 69, 493–502. (Plots of p values for many tests.) 16. Wilk, M. B. and Gnanadesikan, R. (1968). Biometrika, 55, 1–17. (Basic paper on Q-Q and P-P plots; lots of examples and applications, readable.) 17. Wilk, M. B., Gnanadesikan, R., and Huyett, M. J. (1962). Technometrics, 4, 1–15. (Basic paper on gamma probability plots.)

6488

PROBABILITY PROPORTIONAL TO SIZE (PPS) SAMPLING

18. Zahn, D. A. (1975). Technometrics, 17, 189–200. (On modification and corrections for Daniel’s half-normal plots.)

BIBLIOGRAPHY (EDITORS’ ADDITION) Harter, H. L. (1984). Commun. Statist. Theor. Meth., 13, 1613–1633. (Contains 107 references.) See also GRAPHICAL REPRESENTATION, COMPUTER AIDED; GRAPHICAL REPRESENTATION OF DATA; HALF-NORMAL PLOTS; KOLMOGOROV –SMIRNOV STATISTICS; and QUANTILE.

GERALD L. SIEVERS

PROBABILITY PROPORTIONAL TO SIZE (PPS) SAMPLING USES PPS sampling is used principally in two contexts. Multi-Stage Sampling Samples are often selected in a multistage manner [16, Chap. 10]. For example, in a two-stage national household survey, a sample of areas, the primary sampling units∗ (PSUs), may be selected within each of which, subsamples of households, the subunits, are drawn. A common procedure is to sample the PSUs with probability proportional to size, that is, the number of subunits within each PSU, and to select a fixed number of subunits within each PSU by simple random sampling∗ (SRS). This yields a self-weighting sample; that is, each subunit has equal probability of selection so that no reweighting is necessary for estimation purposes. The PPS strategy above has practical advantages over other self-weighting procedures, such as SRS of PSUs, and fixed sampling fractions within PSUs, since it yields a fixed overall sample size (which is usually desirable, given budget constraints) and a fixed number of subunits in each PSU, giving equal work loads to PSU-based interviewers. In terms of sampling precision, the PPS strategy will usually not differ greatly from its competitors [8, p. 310]. A further advantage of the PPS strategy occurs for PSUs (e.g.,

schools or hospitals) which are of interest for their own sake, when equal subsample sizes may be preferred for a between-PSU analysis [16, p. 418]. For multistage sampling with more than two stages, the use of PPS sampling at every stage except the last, where SRS of fixed size is used, is again self-weighting and the comments above also apply (see also STRATIFIED MULTISTAGE SAMPLING). Single-Stage Designs For populations of units such as farms or businesses, measures of size xi may be available for each unit i in the population, and xi may be expected to be approximately proportionately related to a variable yi to be measured in a survey. For example, xi may be crop acreage, known for all farms i, and yi may be wheat yield in a given year. The procedure of sampling units by PPS may then permit more efficient estimation of the population total of the yi than is possible using SRS with the same sample size (see the section ‘‘Estimation Theory’’). PPS sampling may here be viewed as a limiting form of size-stratified sampling. If the population is divided into strata within which xi is approximately constant and if the within-stratum standard deviation of yi is proportional to xi , then optimal Neyman allocation∗ for stratified random sampling implies that units are selected by PPS. One problem with PPS sampling is in multiobjective surveys, where, for example, sampling with probability proportional to crop acreage may be more efficient than SRS for estimating wheat yield but may be much less efficient for estimating number of sheep, say. One practical advantage of PPS sampling in this case occurs in area sampling∗ , where areas may be selected simply using random points on a map [20, p. 207]. ESTIMATION THEORY For ‘‘Single-Stage Designs,’’ let x1 , . . . , xN be known size values for the population (xi > 0) and y1 , . . . , yN be the values of the variable of interest, where, without loss of generality, y1 , . . . , yn are the values observed in the sample. In with-replacement PPS sampling

PROBABILITY PROPORTIONAL TO SIZE (PPS) SAMPLING

(PPSWR) the sample is formed from n independent draws, at each of which the ith unit of the population is selected with probability pi = xi /X, where X = x1 + · · · + xN . In this case the usual estimator [14,25] of the population total, Y = y1 + · · · + yN , is Yˆ WR =

n 

6489

where ij is the probability that both units i and j are included in the sample. Assuming that ij > 0, i, j = 1, . . . , N, two unbiased estimators of V(Yˆ HT ) are n 

Vˆ HT (Yˆ HT ) =

2 −2 i (1 − i )yi

i=1

yi /(npi ).

+

i=1

n 

−1 −1 −1 i j ij

i =j

This estimator is unbiased with variance V(Yˆ WR ) = n−1

N 

2 pi (p−1 i yi − Y) .

× (ij − i j )yi yj , (1)

proposed by Horvitz and Thompson [15], and
Yˆ SYG (Yˆ HT ) = ni<j −1 ij (i j − ij )

i=1

An unbiased estimator of V(Yˆ WR ) is ˆ Yˆ WR ) = [n(n − 1)]−1 V(

n 

−1 2 ×(−1 i yi − j yj ) ,

2 ˆ (p−1 i yi − YWR ) .

i=1

(2) In PPSWR sampling the same unit may appear more than once in the sample. This is avoided in without-replacement PPS sampling (PPSWOR), where the ith unit of the population has probability i = nxi /X of inclusion in the sample (assuming that nxmax
X). Here the usual estimator of Y is the Horvitz-Thompson estimator∗ [15] Yˆ HT =

n 

−1 i yi .

i=1

The estimators Yˆ HT and Yˆ WR are formally identical, since i = npi = nxi /X, but are distinguished because they have different distributions under the two sampling schemes. Under PPSWOR, Yˆ HT is still unbiased but now has variance V(Yˆ HT ) =

N 

2 −1 i (1 − i )yi

i=1

+

N 

−1 −1 i j (ij − i j )yi yj

i =j

=

N 

−1 2 (i j − ij )(−1 i yi − j yj ) ,

i<j

(3)

proposed by Sen [24] and Yates and Grundy [26]. Of the two estimators, Vˆ SYG seems to perform better in empirical comparisons [6, p. 8]. In comparing PPSWR and PPSWOR the ratio V(Yˆ HT )/V(Yˆ WR ) is (N − n)/(N − 1) roughly, as in SRS [8, p. 310]. Comparisons between different PPSWOR strategies (see the section ‘‘Methods’’) are given in refs. 2 and 22. The efficiency of the PPS estimators depends on how closely the yi are proportional to the xi , as can be seen by examining (1) and (3). Under the regression g model, E(yi |xi ) = α + βxi , V(yi |xi ) = σ 2 xi , the PPS strategy may be compared with the use of SRS and the ratio estimator∗ Yˆ R = X

n 

yi /

i=1

n 

xi .

i=1

The PPS strategy is generally more efficient if g > 1 and less efficient if g < 1 [8, p. 258]. For example, if g = 2 and α = 0, . then VSRS (Yˆ R )/VPPS (Yˆ HT ) = 1 + C2x , where Cx is the coefficient of variation of the xi . This ratio becomes closer to unity when α = 0. For two-stage sampling (see the section ‘‘Multistage Sampling’’) suppose that n PSUs are selected with probability proportional to size, Mi . Under PPSWR an unbiased estimate of the population total, Y, is Yˆ TS = M0 n−1

n  i=1

yi ,

6490

PROBABILITY PROPORTIONAL TO SIZE (PPS) SAMPLING

where M0 = M1 + · · · + MN and the yi are the PSU subsample means. The variance of Yˆ TS can be expressed as the sum of a betweenPSU component of the form of (1) and a within-PSU component VW [8, p. 307]. An unbiased estimator of V(Yˆ TS ) is ˆ Yˆ TS ) = [n(n − 1)]−1 V(

n  (M0 yi − Yˆ TS )2 . i=1

Under PPSWOR with i = Mi /M0 the estimator Yˆ TS is again unbiased and its variance is the sum of a between-PSU component of the form of (3) and the same withinPSU component, VW [8, p. 309]. Unbiased estimation of this variance is relatively complicated, although simplified procedures have been suggested in refs. 10 and 5. The discussion above is based on the conventional approach of making inferences with respect to sampling distributions induced by the randomized sample design. Alternative discussions of PPS sampling from Bayesian and prediction theory viewpoints are given in refs. 1, 9, and 18. METHODS The implementation of PPSWR is in principle straightforward [14], although a simplified procedure for large N has been suggested by Lahiri [17; 20, p. 202]. For PPSWOR (i.e., the selection of a sample of size n with inclusion probabilities given by i = nxi /X), many methods are available; 50 are listed in ref. 6. The case n = 2 has been especially studied because it corresponds to the common choice of two PSUs per stratum. Methods differ in respect of criteria such as (a) simplicity of implementation, (b) ease of computing the ij , (c) Vˆ SYG (Yˆ HT ) is always nonnegative, and (d) Yˆ HT and Vˆ SYG (Yˆ HT ) are efficient estimators. We assume that i = npi < 1 for i = 1, . . . , N, since any unit for which npi  1 is included in the sample with certainty and for our present purpose can therefore be excluded from the population. The simplest procedures to implement are those of systematic sampling∗ [19,12; 20, p. 215], but these suffer because the ij may be zero and hence no unbiased estimate of V(Yˆ HT ) may exist.

For n = 2 various simple PPSWOR procedures are available for which the ij are nonzero and easy to compute. In Brewer’s method [3] the first unit is selected with probability proportional to pi (1 − pi )/(1 − 2pi ), where pi = xi /X, and the second with probability pi /(1 − pj ), where unit j was drawn first. In durbin’s method [10] the first unit is selected with probability pi and the second with probability pi.j =

pi [(1 − 2pi )−1 + (1 − 2pj )−1 ] ,
−1 1+ N k=1 pk (1 − 2pk ) i = j,

where unit j was drawn first. Rao’s method [21], a case of rejective sampling∗ [13], involves selecting the first unit with probability pi and the second with replacement with probability proportional to pi /(1 − 2pi ). If both units are the same, the sample is rejected and the procedure above begun again until a sample of two distinct units is obtained. For all three procedures above, the joint inclusion probabilities are given by ij = 2pi pj.i . Also in each case Yˆ HT is more efficient than Yˆ WR under PPSWR [3] and Vˆ SYG is always nonnegative [21]. For n > 2 Rao’s method is generalized by Sampford [23], who proposes selecting units 2, 3, . . . , n with replacement with probability proportional to pi /(1 − npi ) and again rejecting samples not consisting of distinct units (see REJECTIVE SAMPLING). Brewer’s method has also been extended in ref. 4. The computation of the ij for these methods, however, does become more complicated as n increases. A straightforward algorithmic approach for general n which obeys at least criteria (a), (b), and (c) above is given by Chao [7]. A method for continuous surveys involving rotation of the PSUs has been suggested by Fellegi [11]. Further methods are reviewed in ref. 6. REFERENCES 1. Basu, D. (1971). In Foundations of Statistical Inference, V. P. Godambe and D. A. Sprott, eds. Holt, Rinehart and Winston, Toronto, pp. 203–242.

PROBABILITY THEORY AND RELATED FIELDS 2. Bayless, D. L. and Rao, J. N. K. (1970). J. Amer. Statist. Ass., 65, 1645–1667.

6491

3. Brewer, K. R. W. (1963). Aust. J. Statist., 5, 5–13.

25. Sukhatme, P. V. and Sukhatme, B. V. (1970). Sampling Theory of Surveys with Applications. Asia Publishing House, London, Chap. 2.

4. Brewer, K. R. W. (1975). Aust. J. Statist., 17, 166–172.

26. Yates, F. and Grundy, P. M. J. R. Statist. Soc. B, 15, 253–261.

5. Brewer, K. R. W. and Hanif, M. J. R. Statist. Soc. B, 32, 302–311.

(1970).

6. Brewer, K. R. W. and Hanif, M. (1983). Sampling with Unequal Probabilities. SpringerVerlag, New York. (The most comprehensive reference on methods and estimators.) 7. Chao, M. T. (1982). Biometrika, 69, 653– 656. 8. Cochran, W. G. (1977). Sampling Techniques, 3rd ed. Wiley, New York. (A good reference for the estimation theory.) 9. Cumberland, W. G. and Royall, R. M. (1981). J. R. Statist. Soc. B, 43, 353–367.

(1953).

See also AREA SAMPLING; POPULATION SIZE, HORVITZ –THOMPSON ESTIMATOR FOR; OPTIMUM STRATIFICATION; PRIMARY SAMPLING UNIT; REJECTIVE SAMPLING; REPRESENTATIVE SAMPLING; SIMPLE RANDOM SAMPLING; STRATIFIED DESIGNS; STRATIFIED MULTISTAGE SAMPLING; and SYSTEMATIC SAMPLING.

C. J. SKINNER

PROBABILITY SPACES, METRICS AND DISTANCES ON. See METRICS AND DISTANCES ON PROBABILITY SPACES

10. Durbin, J. (1967). Appl. Statist., 16, 152–164. 11. Fellegi, I. P. (1963). J. Amer. Statist. Ass., 58, 183–201. 12. Grundy, P. M. (1954). J. R. Statist. Soc. B, 16, 236–238. 13. Hajek, J. (1981). Sampling from a Finite Population. Marcel Dekker, New York. (Includes asymptotic results on various PPS schemes not available elsewhere.) 14. Hansen, M. H. and Hurwitz, W. N. (1943). Ann. Math. Statist., 14, 333–362. (The original PPS paper.) 15. Horvitz, D. G. and Thompson, D. J. (1952). J. Amer. Statist. Ass., 47, 663–685. 16. Kish, L. (1965). Survey Sampling. Wiley, New York, Chap. 7. (Good on the practical context of PPS sampling.) 17. Lahiri, D. B. (1951). Bull. Int. Statist. Inst., 33, Bk. 2, 133–140. 18. Little, R. J. A. (1983). J. Amer. Statist. Ass., 78, 596–604. 19. Madow, W. G. (1949). Ann. Math. Statist., 20, 333–354. 20. Murthy, M. N. (1967). Sampling Theory and Methods. Statistical Publishing Society, Calcutta, Chap. 6. 21. Rao, J. N. K. (1965). J. Indian Statist. Ass., 3, 173–180. 22. Rao, J. N. K. and Bayless, D. L. J. Amer. Statist. Ass., 64, 540–559. 23. Sampford, M. R. 499–513.

(1967).

(1969).

Biometrika,

24. Sen, A. R. (1953). J. Indian Statist., 5, 119–127.

Soc.

54,

Agric.

PROBABILITY THEORY AND RELATED FIELDS The journal Probability Theory and Related Fields (PTRF) is published by Springer Verlag in Heidelberg, Germany, in 12 issues per year, each set of 4 issues constituting one volume. PTRF was founded in 1962 with the title Zeitschrift fur ¨ Wahrscheinlichkeitstheorie und Verwandte Gebiete, which changed to the current title in 1986 with Volume 71. A website link for the journal is http://link. springer.de/link/service/journals/00440. PTRF is directed by a Managing Editor with the assistance of ten or so members of an international Editorial Board. All papers submitted to the journal for publication are refereed. Quoting from the website, ‘‘The first Managing Editor was L. Schmetterer, who wrote thus in the first issue: ‘Until about 30 years ago, the theory of probability and its applications were somewhat isolated mathematical disciplines. Then, following a period of remarkable development that is not yet completed by any means, probability theory became an important domain of mathematics, intimately connected with other domains, especially with analysis and number theory. In addition, it leads a life of its own, all by itself and in its manifold applications.’’’

6492

PROBABILITY THEORY: AN OUTLINE

‘‘The Founding Managing Editor was succeeded by K. Krickeberg in 1971, and by H. Rost in 1985. Subsequent Managing Editors [have been] O. Kallenberg (1991), E. Bolthausen (1994) and G. R. Grimmett (2000).’’ ‘‘Probability Theory and Related Fields publishes research papers in modern probability theory, its relation to analysis, geometry and other areas in mathematics, and its various fields of application. Thus, subjects of interest include: statistical mechanics, ergodic theory, mathematical biology, filtering theory, mathematical statistics, theoretical computer science, optimization and control, stochastic geometry, and stochastic algorithms. Survey papers on emerging areas of importance may be considered for publication. The main languages of publication are English, French and German.’’

PROBABILITY THEORY: AN OUTLINE Probability theory provides a mathematical framework for the description of phenomena whose outcomes are not deterministic but rather are subject to chance∗ . This framework, historically motivated by a frequency interpretation∗ of probability, is provided by a formal set of three innocent-looking axioms and the consequences thereof. These are farreaching and a rich and diverse theory has been developed. It provides a flexible and effective model of physical reality and furnishes, in particular, the mathematical theory needed for the study of the discipline of statistics. THE FORMAL FRAMEWORK Elementary (non-measure-theoretic) treatments of probability can readily deal with situations in which the set of distinguishable outcomes of an experiment subject to chance (called elementary events) is finite. This encompasses most simple gambling situations, for example. Let the set of all elementary events be  (called the sample space∗ ) and let F be the Boolean field of subsets of  (meaning just that (a) A ∈ F implies that A ∈ F, the bar

denoting complementation with respect to  and (b) A1 , A2 ∈ F implies that A1 ∪ A2 ∈ F). Then the probability P is a set function on F satisfying the axioms: A1 P(A)  0, A ∈ F. A2 If A1 , A2 ∈ F and A1 ∩ A2 = Ø (the empty set), then P(A1 ∪ A2 ) = P(A1 ) + P(A2 ). A3 P() = 1. These axioms∗ enable probabilities to be assigned to events in simple games of chance∗ . For example, in the toss of an ordinary six-sided die,  consists of six equiprobable elementary events and for A ∈ F, P(A) = n(A)/6, where n(A) is the number of elementary events in A. The formulation above can be extended straightforwardly to the case where the number of elementary events in the sample space is at most countable, by extending Axiom A2 to deal with countable unions of disjoint sets. However, a full measure-theoretic formulation is necessary to deal with general sample spaces  and to provide a framework which is rich enough for a comprehensive study of limit results on combinations of events (sets); see also MEASURE THEORY IN PROBABILITY AND STATISTICS. The measure-theoretic formulation has a very similar appearance to the elementary version. We start from a general sample space  and let F be the Borel σ -field of subsets of  [meaning that (a) A ∈ F implies that A ∈ F and (b) Ai ∈ F, 1
i < ∞, implies that ∞ i=1 Ai ∈ F]. Then as axioms for the probability measure P we have Axioms A1 and A3 as before, while A2 is replaced by: A2 If Ai ∈ F, 1
i < ∞, and Ai ∩ Aj = Ø, i = j, then P

∞  i=1

 Ai

=

∞ 

P(Ai ).

i=1

The triplet (, F, P) is called a probability space. This framework is due to Kolmogorov [14] in 1933 and is very widely accepted, although some authors reject the countable additivity axiom A2 and rely on

PROBABILITY THEORY: AN OUTLINE

the finite additivity version (A2) (e.g., Dubins and Savage [8]). An extension of the Kolmogorov theory in which unbounded measures are allowed and conditional probabilities are taken as the fundamental concept has been provided by R´enyi [18, Chap. 2]. For A, B ∈ F with P(B) > 0, the conditional probability of A given B is defined as P(A|B) = P(A ∩ B)/P(B). Suppose that the sample space  has points ω. A real-valued function X(ω) defined on the probability space (, F, P) is called a random variable [or, in the language of measure theory, X(ω) is a measurable function] if the set {ω : X(ω)
x} belongs to F for every x ∈ R (the real line). Sets of the form {ω : X(ω)
x} are usually written as {X
x} for convenience and we shall follow this practice. Random variables and their relationships are the principal objects of study in much of probability and statistics. IMPORTANT CONCEPTS Each random variable X has an associated distribution function defined for each real x by F(x) = P(X
x). The function F is nondecreasing, rightcontinuous, and has at most a countable set of points of discontinuity. It also uniquely specifies the probability measure which X induces on the real line. Random variables are categorized as having a discrete distribution if the corresponding distribution function is a step function and as having a continuous distribution if it is continuous. Most continuous distributions of practical interest are absolutely continuous, meaning that the distribution function F(x) has a representation of the form  F(x) =

6493

important are the binomial∗ , which is discrete and for which   n r P(X = r) = p (1 − p)n−r , r 0
r
n,

0 < p < 1,

∗

and the (unit) normal , which is absolutely continuous and for which  x 2 e−(1/2)u du, P(X
x) = (2π )−1/2 −∞

−∞ < x < ∞. The binomial random variable represents the numbers of successful outcomes in n trials, repeated under the same conditions, where the probability of success in each individual trial is p. Normal distributions, or close approximations thereto, occur widely in practice, for example in such measurements as heights or weights or examination scores, where a large number of individuals are involved. Two other particularly important distributions are the Poisson∗ , for which P(X = r) = e−λ λr /r!, r = 0, 1, 2, . . . ,

λ > 0,

and the exponential∗ , for which  x P(X
x) = λ e−λu du, x > 0,

λ > 0;

0

= 0 otherwise. These commonly occur, respectively, in situations of rare events and as waiting times between phenomena. A sequence of random variables {X1 , X2 , . . .} defined on the same probability space is called a stochastic process∗ . For a finite collection of such random variables {X1 , X2 , . . . , Xm } we can define the joint distribution function F(x1 , x2 , . . . , xm )

x

f (u) du −∞

for some f  0; f is called the probability density function. A relatively small number of families of distributions are widely used in applications of probability theory. Among the most

= P(X1
x1 , X2
x2 , . . . , Xm
xm ) for xi ∈ R, 1
i
m, and the random variables are said to be independent∗ if P(X1
x1 , . . . , Xm
xm ) = P(X1
x1 ) · · · P(Xm
xm )

6494

PROBABILITY THEORY: AN OUTLINE

for all x1 , . . . , xm . Independence is often an essential basic property and much of probability and statistics is concerned with random sampling, namely independent observations (random variables) with identical distributions. One step beyond independence is the Markov chain {X1 , X2 , . . .}, in which case P(Xm
xm |X1 = i1 , . . . , Xn = in ) = P(Xm
xm |Xn = in ) for any n < m and all i1 , . . . , in and xm . That is, dependence on a past history involves only the most recent of the observations. Such random sequences with a natural time scale belong to the domain of stochastic processes; see MARKOV PROCESSES. Considerable summary information about random variables and their distributions is contained in quantities called moments. For a random variable X with distribution function F, the rth moment (r > 0) is defined by  EX r =

∞

xr dF(x)

to that of a random variable X (written d Xn →X) if the sequence of distribution functions {Fn (x) = P(Xn
x)} converges to the distribution function F(x) = P(X
x) at all points x of continuity of F. Convergence in distribution does not involve the random variables explicitly, only their distributions. Other commonly used modes of convergence are: (a) {Xn } converges in probability to X (writp ten Xn →X) if, for every  > 0, lim P(|Xn − X| > ) = 0.

n→∞

(b) For r > 0, {Xn } converges in the mean of r order r to X (written Xn →X) if lim E|Xn − X|r = 0.

n→∞

(c) {Xn } converges almost surely to X (writa.s. ten Xn → X) if for every  > 0,   lim P sup |Xk − X| > 

n→∞

−∞

∞ provided that −∞ |x|r dF(x) < ∞. E|X|r is called the rth absolute moment. The operator E is called Expectation and the first moment (r = 1) is termed the mean. Moments about the mean E(X − EX)r , r = 2, 3, . . . are widely used and E(X − EX)2 is termed the variance∗ of X, while (E(X − EX)2 )1/2 is its standard deviation∗ . The mean and variance of X are, respectively, measures of the location and spread of its distribution. CONVERGENCE Much of probability theory is concerned with the limit behavior of sequences of random variables or their distributions. The great diversity of possible probabilistic behavior when the sample size is small is often replaced by unique and well-defined probabilistic behavior for large sample sizes. In this lies the great virtue of the limit theory: statistical regularity emerging out of statistical chaos as the sample size increases. A sequence of random variables {Xn } is said to converge in distribution (or in law)

= 0.

kn

See also CONVERGENCE OF SEQUENCES OF RANDOM VARIABLES. It is not difficult to show that (b) implies (a), which in turn implies convergence in distribution while (c) implies (a). None of the converse implications hold in general. Convergence in distribution is frequently established by the method of characteristic functions∗ . The characteristic function of a random variable X with distribution function F is defined by  ∞ eitx dF(x), f (t) = EeitX = √

−∞

where i = −1. If fn (t) and f (t) are, respectively, the characteristic functions of Xn and d

X, then Xn →X is equivalent to fn (t) → f (t). Characteristic functions are particularly convenient for dealing with sums of independent random variables as a consequence of the multiplicative property Eeit(X+Y) = EeitX EeitY for independent X and Y. They have been extensively used in most treatments of

PROBABILITY THEORY: AN OUTLINE

convergence in distribution for sums of independent random variables (the classical central limit problem). The basic central limit theorem∗ (of which there are many generalizations) deals with independent and identically distributed random variables Xi with 2 mean µ and
variance σ . It gives, upon writing Sn = ni=1 Xi , that lim P(σ −1 n−1/2 (Sn − nµ)
x)  x 2 = (2π )−1/2 e−(1/2)u du,

n→∞

(1)

−∞

the limit being the distribution function of the unit normal law. Convergence in probability results are often obtained with the aid of Markov’s inequality∗ P(|Xn | > c)
c−r E|Xn |r for r > 0 and a corresponding convergence in rth mean result. The useful particular case r = 2 of Markov’s inequality is ordinarily called Chebyshev’s inequality∗ . For example, in the notation of (1), P(|n−1 Sn − µ| > )
 −2 n−2 E(Sn − nµ)2 → 0 (2) as n → ∞, which is a version of the weak law p of large numbers∗ , n−1 Sn →µ. Almost sure convergence results rely heavily on the Borel-Cantelli lemmas∗ . These deal with#a sequence of events (sets) {En } and ∞ E , the set of elements comE= ∞ n r=1 n=r mon to infinitely many of the En . Then
(a) P(En ) < ∞ implies that P(E) = 0.
(b) P(En ) = ∞ and the events En are independent implies that P(E) = 1. With the aid of (a), (2) can be strengthened to the strong law of large numbers∗ , a.s. n−1 Sn → µ

(3)

as n → ∞. The strong law of large numbers embodies the idea of a probability as a strong limit of relative frequencies; if we set Xi = Ii (A), the indicator function of the set A at the ith trial,

6495

then µ = P(A). This property is an important requirement of a physically realistic theory. The law of large numbers, together with the central limit theorem, provides a basis for a fundamental piece of statistical theory. The former yields an estimator of the location parameter µ and the latter enables approximate confidence intervals∗ for µ to be prescribed and tests of hypotheses∗ constructed.

OTHER APPROACHES The discussion above is based on a frequency interpretation of probability, and this provides the richest theory. There are, however, significant limitations to a frequency interpretation in describing uncertainty in some contexts and various other approaches have been suggested. Examples are the development of subjective probability∗ by de Finetti and Savage (e.g., refs. 7 and 19) and the so-called logical probability of Carnap [3]. The former is the concept of probability used in Bayesian inference∗ and the latter is concerned with objective assessment of the degree to which evidence supports a hypothesis (see BAYESIAN INFERENCE and PROBABILITY, FOUNDATIONS OF —I). A detailed comparative discussion of the principal theories that have been proposed is given in Fine [11]. The emergence of probability theory as a scientific discipline dates from around 1650; a comprehensive sketch of its history is given in PROBABILITY, HISTORY OF. THE LITERATURE Systematic accounts of probability theory have been provided in book form by many authors. These are necessarily at two distinct levels since a complete treatment of the subject has a measure theory prerequisite. Elementary discussions, not requiring measure theory, are given for example in Chung [5], Feller [9], Gnedenko [12], and Parzen [16]. A full treatment can be found in Billingsley [1], Breiman [2], Chow and Teicher [4], Chung [6], Feller [10], Hennequin and Tortrat [13], Lo´eve [15], and R´enyi [17].

6496

PROBABILITY-WEIGHTED MOMENTS

REFERENCES 1. Billingsley, P. (1979). Probability and Measure. Wiley, New York. 2. Breiman, L. (1968). Probability. AddisonWesley, Reading, Mass. 3. Carnap, R. (1962). The Logical Foundations of Probability, 2nd ed. University of Chicago Press, Chicago. 4. Chow, Y. S. and Teicher, H. (1978). Probability Theory. Independence, Interchangeability, Martingales. Springer-Verlag, New York. 5. Chung, K. L. (1974). Elementary Probability Theory with Stochastic Processes. SpringerVerlag, Berlin. 6. Chung, K. L. (1974). A Course in Probability Theory, 2nd ed. Academic Press, New York. 7. de Finetti, B. (1974). Theory of Probability, Vols. 1 and 2. Wiley, New York. 8. Dubins, L. E. and Savage, L. J. (1965). How to Gamble If You Must. McGraw-Hill, New York. 9. Feller, W. (1968). An Introduction to Probability Theory and Its Applications, Vol. 1, 3rd ed. Wiley, New York. 10. Feller, W. (1971). An Introduction to Probability Theory and Its Applications, Vol. 2, 2nd ed. Wiley, New York. 11. Fine, T. L. (1973). Theories of Probability. An Examination of the Foundations. Academic Press, New York. 12. Gnedenko, B. V. (1967). The Theory of Probability, 4th ed. (Translated from the Russian by B. D. Seckler.) Chelsea, New York. 13. Hennequin, P. L. and Tortrat, A. (1965). Th´eorie des Probabilit´es et Quelques Applications. Masson, Paris. 14. Kolmogorov, A. (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung. SpringerVerlag, Berlin. (English trans.: N. Morrison, in Foundations of the Theory of Probability. Chelsea, New York, 1956.) 15. Lo`eve, M. (1977). Probability Theory, Vols. 1 and 2, 4th ed. Springer-Verlag, Berlin. 16. Parzen, E. (1960). Modern Probability Theory and Its Applications. Wiley, New York. 17. R´enyi, A. (1970). Probability Theory. NorthHolland, Amsterdam. (Earlier versions of this book appeared as Wahrscheinlichkeitsrechnung, VEB Deutscher Verlag, Berlin, 1962; and Calcul des probabilit´es, Dunod, Paris, 1966.) 18. R´enyi, A. (1970). Foundations of Probability. Holden-Day, San Francisco.

19. Savage, L. J. (1962). The Foundations of Statistical Inference: A Discussion. Wiley, New York. See also AXIOMS OF PROBABILITY; BAYESIAN INFERENCE; CHANCE —II; CONVERGENCE OF SEQUENCES OF RANDOM VARIABLES; FREQUENCY INTERPRETATION IN PROBABILITY AND STATISTICAL INFERENCE; LAWS OF LARGE NUMBERS; LIMIT THEOREM, CENTRAL; NONADDITIVE PROBABILITY; PROBABILITY, HISTORY OF; PROBABILITY, FOUNDATIONS OF —I; and SUBJECTIVE PROBABILITIES.

C. C. HEYDE

PROBABILITY-WEIGHTED MOMENTS The (r, s, t) probability-weighted moment of a random variable X, having a cumulative distribution function (CDF) F(x) is Mr,s,t = E[X r {F(X)}s {1 − F(X)}t ]  1 = {x(F)}r F s (1 − F)t d F, 0

where x(F) is the inverse of the CDF. Equivalently, Mr,s,t is (s!t!)/(s + t + 1)! times the expected value of the rth power of the (s + 1)th order statistic∗ in a random sample of size (s + t + 1). The quantities {Mr,0,0 } are the moments about zero of X. The quantities Mr,s,t were introduced by Greenwood et al. [1]. Equating sample to population values may be used for estimation of parameters. [Note, however, that the sample size (n) need not be equal to (s + t + 1).] Hosking et al. [2] consider use of {M1,s,0 } in this connection. Hosking [3] gives detailed theory. REFERENCES 1. Greenwood, J. A., Landwehr, J. M., Matalar, N. C., and Wallis, J. R. (1979). Water Resources Res., 15, 1049–1054. 2. Hosking, J. R. M., Wallis, J. R., and Wood, E. F. (1985). Technometrics, 27, 251–261. 3. Hosking, J. R. M. (1986). The Theory of Probability-Weighted Moments, IBM Res. Rep. RC 12210 (#54860). See also ESTIMATION, CLASSICAL; ESTIMATION: METHOD OF MOMENTS; ORDER STATISTICS; PROBABILITY MOMENTS AND FREQUENCY MOMENTS; and U-STATISTICS.

PROBIT ANALYSIS

PROBABLE ERROR Francis Galton∗ expressed the values in a normal distribution∗ as a function of probable error [4], which he described as ‘‘the value that one-half of the Errors exceed and the other half fall short of.’’ It is an archaic term for the semi-interquartile range (see QUARTILE DEVIATION), for normal variables equal to 0.67449 × (standard deviation∗ ). Probable error as a measure of spread was usually applied to normal variables [6], the probability of obtaining a value within one probable error of the population mean being 0.50, but it was also used for nonnormal data [1]. The term appeared in the early nineteenth century among German astronomers. It was apparently first used explicitly by F. W. Bessel [2] in 1818, was developed subsequently by Gauss∗ , and was used extensively by Adolphe Quetelet∗ [5]. Its origin, however, can be traced [3] to Abraham De Moivre’s The Doctrine of Chances (1756). Galton [4, pp. 57–58] objected to the name ‘‘probable error’’: ‘‘It is astonishing that mathematicians, who are the most precise and perspicacious of men, have not long since revolted against this cumbrous, slip-shod, and misleading phrase.’’ He then introduced the term ‘‘probable deviation’’ instead, but it was not adopted by others. Although probable error is still used as a measure occasionally in the physical sciences, it has been replaced in modern statistics by the standard deviation. The term ‘‘error’’ reflects the earlier convention of describing the normal law as the error distribution (see LAWS OF ERROR —I, II, III). REFERENCES 1. Aitken, A. C. (1949). Statistical Mathematics, 6th ed. Oliver & Boyd, Edinburgh. (See Chapter II.) ¨ 2. Bessel, F. W. (1818). Uber der Ort des Polarsterns. Berliner Astronomische Jahrbuch ¨ 1818. Berlin. fur 3. De Moivre, A. (1756). The Doctrine of Chances, 3rd (final) ed. A. Millar, London. (Reprinted by Chelsea, New York, 1967. 4. Galton, F. (1889). Natural Inheritance. Macmillan, London, pp. 57–58.

6497

5. Quetelet, L. A. (1849). Letters on the Theory of Probabilities, O. G. Downes, trans. Layton, London. 6. Yule, G. U. and Kendall, M. G. (1937). An Introduction to the Theory of Statistics, 11th ed. Charles Griffin, London. (See Section 19.10.) See also INTERQUARTILE RANGE; NORMAL DISTRIBUTION; QUARTILE DEVIATION; and STANDARD DEVIATION.

PROBABLE ITEM An obsolete term for mode (see MEAN, MEDIAN, AND MODE).

PROBIT ANALYSIS ‘Quantal response analysis is concerned with the relationship between one or more dependent or response variables on the one hand and a set of independent or explanatory variables on the other, in the situation in which the response is assessed in terms of a discrete, qualitative, or categorical∗ variable. The simplest form of quantal variable involves a dichotomy, with two response categories which may be assigned the conventional labels 0 and 1, respectively. More generally, response may be recorded in terms of a polychotomy with several classes or categories, which may or may not reflect some implied underlying ordering (see also NOMINAL DATA). APPLICATIONS In the past, the main applications of quantal response analysis have been in biometry and medical statistics, including pharmacology, toxicology, occupational medicine, and follow-up∗ studies of groups of patients with particular characteristics, conditions, or treatment regimes (see BIOSTATISTICS, CLASSICAL and STATISTICS IN MEDICINE). Quantal models have commonly been concerned with the death of a human being or other biological organism or the appearance of a particular symptom, in response to a stimulus such as a drug, a poison, a period of exposure to a hazardous environment, or merely the passage of time. Many biometric

6498

PROBIT ANALYSIS

applications involve just one response variable and just one stimulus, although some consideration has been given both to multiple quantal responses and mixtures of several stimuli administered jointly. See also BIOASSAY, STATISTICAL METHODS IN. More recently, there have been an increasing number and range of economic applications, in a variety of contexts which include labor force participation, choice of occupation, migration∗ , transportation, housing, education, legislation, and criminology. Many of the variables of interest in this field are either naturally expressed in a discrete or categorical form or are recorded in this way. Because of the complexities of the situation, large numbers of explanatory variables are commonly used; in addition, the growing availability and sophistication of survey data has led to a sharp increase in the raw material for such studies. Both developments reflect the growing use and power of modern computing facilities. There is an extensive literature on the applications of quantal response analysis in biometry; for example, see refs. 6, 8, and 13. Economic analysis is less well served in this context, although Daganzo [7] dealt with one specific application—transport model choice—and Amemiya [1] produced an excellent survey of the use of qualitative response models in both econometrics∗ and biometry. FRAMEWORK FOR QUANTAL RESPONSE MODELS A general framework for the formulation of quantal response models in biometry was provided by Ashford [3], who proposed that a biological organism should be regarded as a system comprising several subsystems, S1 , S2 , . . . , Sk . When a stimulus consisting of d components, D1 , D2 , . . . , Dd , is applied at level z = (z1 , z2 , . . . , zd ), the organism manifests a reaction y = (y1 , y2 , . . . , yk ), where yi , i = 1, 2, . . . , k, refers to the underlying response of subsystem Si ; yi may in principle be expressed in quantitative terms, but is not necessarily observable in this form. For example, human subjects are assessed in terms of several distinct respiratory symptoms, each of which corresponds

to the quantal response of a particular subsystem. The same framework may also be employed in economic analysis, although the interpretation of the components reflects the field of application. The underlying stimulus-response relation may then be represented in terms of an additive model of the form y = x(z) +
.

(1)

In this expression, x(z) is the vector effects function of the stimulus z. The random vector
represents the ‘‘tolerance’’ or ‘‘susceptibility’’ of the individual subject and is distributed with joint cumulative distribution function (CDF) T(
) and joint probability density function (PDF) t(
) in the population of subjects. In general, the tolerance components (
i ,
j ), (i, j = 1, 2, . . . , k; i = j), will be positively correlated, since they reflect different characteristics of the same subject. RELATION BETWEEN QUANTITATIVE AND QUANTAL RESPONSES The value of the underlying reaction y will determine the particular quantal or semiquantal category of the observed system response. Suppose that the k-variate reaction space is divided into mutually exclusive and exhaustive regions corresponding to the various categories. In the simplest situation, just one subsystem S is involved; for such a subsystem quantal response, the two regions correspond to the subdivision of the y-axis at a point which, without loss of generality, may be taken as zero. The probability of subsystem quantal response may then be expressed as P(z) = P{y
0} = P{x(z) +

0} = 1 − T(−x(z)),

(2)

where T(
) is the (marginal) CDF of
. If we make the plausible assumption that T(
) is symmetrical, (2) may be expressed in the form P(z) = T(x(z)).

(3)

PROBIT ANALYSIS

The form of the right-hand side of (3) emphasizes that, on the basis of a quantal response alone [i.e., using only the information contained in P(z)], it is not possible to determine the forms of both the tolerance distribution and the effects function without ambiguity, since for any given P(z) and specified marginal CDF T(
), the form of x(z) is uniquely determined. A semiquantal response involving s ordered categories C1 , C2 , . . . , Cs may also be represented in terms of the response of a single subsystem S. The boundaries of the various categories may be defined as the points η1 , η2 , . . . , ηs−1 on the y-axis, where η1 < η2 < · · · < ηs−1 , the rth response category Cr corresponding to all points y in the interval (ηr−1 , ηr ), where η0 and ηs are taken as −∞ and +∞, respectively. Without loss of generality, any one of the ηi may be set as equal to zero. The probability that the response falls within category Cr is then Pr (z) = P{ηr−1  y1 < ηr } = P{ηr−1 − x(z) 
 ηr − x(z)} = T{ηr − x(z)} − T{ηr−1 − x(z)}. (4) Quantal and semiquantal responses defined in relation to a complete system may also be represented in terms of the underlying reaction of more than one subsystem. Thus a ‘‘weakest link’’ type of system might involve the quantal response of any one of k constituent subsystems. The corresponding probability is P(z) = P{any yi > 0} = 1 − P{all yi  0} = 1 − T(−x1 (z), −x2 (z), . . . , −xk (z)).(5) In general, if the quantal response corresponds to a region R in the underlying reaction space, the corresponding probability is


p(z) = · · · t(x(z) −
)d
1 d
2 . . . d
k . R

(6) Semiquantal responses may similarly be defined in terms of a mutually exclusive and exhaustive set of regions. Such categories will not in general reflect any intrinsic ordering of

6499

the various response variables, although, for example, regions defined by a set of parallel hyperplanes would in some sense imply an underlying gradation. THE TOLERANCE DISTRIBUTION: PROBITS AND LOGITS Various algebraic forms have been proposed for the distribution of the tolerances
in (1). Turning first to the marginal distribution of a particular component, say
, the following types of distribution have been employed in the analysis of quantal data. Linear Probability Model  0,
 0, T(
) =
, 0 <
 1,  1,
> 1.

(7)

This model restricts the range of variation of the tolerance component, which may well be unrealistic in many applications. This deficiency must be balanced against the advantage of computational simplicity. The model has been found useful in the preliminary analysis of economic data. Probit Model The scale used for the measurement of the underlying response is arbitrary. Given that the mean of the tolerance distribution∗ has already been set equal to zero, the variance may be equated to unity and the tolerance may be assumed to be a standardized variable. The most widely used tolerance distribution of this type is the standardized normal,

√ 2 (1/ 2π )
−(1/2)t dt, (8) T(
) = (
) = −∞

which forms the basis of probit analysis (see also BIOASSAY, STATISTICAL METHODS IN). The probit of the proportion p is defined in terms of the inverse of  as p∗ = −1 (p) + 5.

(9)

The factor 5 was originally included in (9) for convenience of manual computation, to avoid the occurrence of negative values of the

6500

PROBIT ANALYSIS

probit. In the more modern usage of the term ‘‘probit,’’ this factor is excluded from the definition (see NORMAL EQUIVALENT DEVIATE). Logistic Model

the extreme minimal value (m1 = 1, m2 → ∞) and the extreme maximum value (m1 → ∞, m2 = 1). MULTIVARIATE TOLERANCE DISTRIBUTIONS

An alternative ‘‘smooth’’ standardized distribution to the normal is the logistic distribution∗ , employed in the form T(
) = L(λ,
) =
−λ
/(1 +
−λ
).

(10)

√ Taking λ = π/ 3, the variance of the tolerance distribution may be set equal to unity. However, some authors consider that closer overall agreement with the standardized normal distribution may be obtained by setting λ = 1.6. The inverse of the relation L(1,
) defines the logit∗ transformation (or the logarithm of the odds ratio; see ODDS RATIO ESTIMATORS) l = ln[p/(1 − p)].

(11)

Cox [6, p. 33] has suggested a modified form of (11) which avoids infinite values of the logit corresponding to p = 0 and p = 1. The logit and probit transformations are similar for values of p that lie toward the center of the range and differ substantially only for values close to zero or unity. In practical applications, extremely large numbers of observations are required to differentiate between the two transformations [5]. The logit transformation has the advantage of greater ease of computation, which must be set against the fact that the probit, by virtue of its derivation from the normal distribution, is closer to the mainstream of statistical analysis. A two-parameter class of tolerance distributions has been proposed [14] which includes both the normal and logistic distributions as special cases, exp(m1
){1 + exp
}−(m1 +m2 ) t(m1 , m2 ,
) = β(m1 , m2 ) (12) where β represents the beta function. The logistic model is given by m1 = m2 = 1, while (12) converges to a normal distribution as m1 , m2 → ∞. Other limiting special cases are

The assumption of a standardized multivariate normal distribution for the joint distribution of several tolerance components corresponds to multivariate probit analysis [2]. This has the advantage of standardized normal univariate marginal distributions, as used in probit analysis, coupled with a representation of the association between pairs of tolerance components in terms of the correlation coefficient. Representations based on a multivariate logistic distribution have also been proposed. Morimune [11] compared the Ashford–Sowden bivariate probit model with the Nerlove–Press [12] multivariate logit model in an application concerned with home ownership and number of rooms in house, and concluded that the probit was to be preferred. ORDERED RESPONSE VARIABLES: RANKITS In circumstances such as sensory tests for degrees of preference, the response is measured by a variable that can be placed in rank order, but cannot be assigned a quantitative value (see ORDINAL DATA). The (subjective) ranks may be replaced by the corresponding rankits [10]. The rankit∗ of the rth in rank order from a sample of n observations is defined as the expected value of the rthorder statistic∗ of a random sample of size n from a standardized normal distribution. This transformation is in a sense a generalization of the probit transformation and produces a response variable that may be regarded as quantitative in form. MIXED DISCRETE-CONTINUOUS RESPONSE VARIABLES: TOBITS Response may be recorded in terms of a variable that is capable of being measured continuously over only part of the real line. For example, household expenditures on luxury goods are zero at low income levels. Individuals available for employment may

PROBIT ANALYSIS

be working full-time or for a certain number of hours part-time or may be unemployed during a particular week. The general ‘‘twolimit’’ Tobit model envisages an underlying quantitative response which is only measurable in categorical form at the lower and upper limits of the range of variation. If one or other of the extreme categories is not present, the Tobit model is called ‘‘right-truncated’’ or ‘‘left-truncated,’’ as appropriate. Tobit models invariably involve a standardized normal distribution of tolerances and the probabilities of obtaining one or other of the categorical responses may be calculated on the basis of (2) and (8); see also ECONOMETRICS. THE EFFECTS FUNCTION The effects function x(z) in (1) represents the average contribution of the stimulus z to the response y of the subject. In general, this function and its individual components will include parameters θ = (θ1 , θ2 , . . . , θs ) which must be estimated. The form of x(z) and its components xi (z) reflects the particular application. In most but not all applications it is reasonable to suppose that any given component of the effects function will be a monotonically increasing function of the level of any particular component of the stimulus, for fixed levels of any other components. Usually, in the absence of any stimulus all components of the effects function will be zero. Linear functions of the form z θ of the components zi of the stimulus z have been commonly used, conforming to the classical general linear model∗ . Nonlinear functional forms may, however, be appropriate in certain circumstances. The simplest situation, commonly used in the biological assay of a single drug, involves a single subsystem S for which the effects function is expressed in terms of the logarithm of the dosage of a single drug. The underlying dosage–response relation then takes the form y = θ1 + θ2 ln z1 +
,

(13)

where
is a standardized variable and the corresponding probability of a quantal response is, from eq. (3), P(z1 ) = T(θ1 + θ2 ln z1 ).

(14)

6501

Both the response y and the stimulus z may be subject to errors of observation. Suppose that the observed value of y (which forms the basis of the observed quantal response) is denoted y = y + φy and that the observed stimulus (ln z1 ) = ln z1 + φz . The underlying dosage–response relation (13) may then be expressed in the form y = θ1 + θ2 ln z1 +
 ,

(15)

where
 =
1 + φy − θ2 φz . Given that φy and φz follow independent normal distributions with zero mean and variances σy2 and σz2 , respectively,
 is itself normally distributed with zero mean and variance (1 + σy2 + θz2 σz2 ). This means that for a standardized normal distribution of tolerances
the corresponding observed dosage–response relation is  (1 + σy2 + θ q2z σz2 ) y = y  = (θ1 + θ2 ln z) (1 + σy2 + θz2 σz2 ) +
. (16) Comparison of (13) and (16) with (14) shows that the effect of errors of observation is to reduce the slope of the quantal response relationship between the inverse of the CDF of the tolerance distribution and the log dose, but the position corresponding to a 50% response, the LD50 (see BIOASSAY, STATISTICAL METHODS IN), is unchanged. In bioassay more complex forms of the effects function have been proposed to represent stimuli with several components, corresponding to mixtures of drugs or poisons, which are becoming increasingly important in both pharmacology and toxicology (see, e.g., ref. 9). An approach based on a simplified representation of the underlying mode of biological action of the stimulus has been put forward by Ashford (see ref. 3), presenting models based on drug-receptor theory that permit both monotonic and nonmonotonic dosage–response relationships for a single-component stimulus and a simple representation of the phenomena of synergism and antagonism. The majority of examples in economics arise from the representation of events whose outcome is determined by the decisions of a customer, a product, or some other economic

6502

PROBIT ANALYSIS

unit. There is an implicit assumption that rational decisions are made which optimize some inherent utility [15]. An effects function x θ is commonly thought to be appropriate. ESTIMATION AND HYPOTHESIS TESTING The main objective of the statistical analysis of quantal response data is the assessment of the parameters θ contained within the effects function, although in certain applications further parameters relating to the tolerance distribution are also of interest, including the correlation coefficients associated with a multivariate normal distribution of tolerances. As far as the raw data are concerned, a distinction is often made between structures in which only one or a few observations correspond to any given level of stimulus and situations in which many observations are involved. If possible, the number of observations at any stimulus level should be sufficient for large-sample theory∗ to apply in order to minimize the effects of sampling variations on the observed proportions of quantal or semiquantal responses. The most commonly used estimation procedure is maximum likelihood∗ (ML), which may be applied to both data structures. Suppose that for any particular value zi of the stimulus or independent variable, there are ni observations, of which ri manifest the quantal response of some subsystem S. (If there is only one observation at any particular stimulus level, ri will be zero or unity.) Assuming that the observations are independent and applying the binomial theorem on the basis of (3), the log likelihood may be written (apart from a constant term) in the form  {ri ln T(x(θ, zi )) + (ni − ri ) L= i

× ln(1 − T(x(θ, zi )))}.

(17)

The ML estimator of θ is given by the solution of the simultaneous equations  {ri − ni T(x(θ , zi ))} ∂L = ∂θk T(x(θ, zi )){1 − T(x(θ, zi ))} i

× t(x(θ, zi )) where k = 1, 2, . . . , s.

∂x(θ, zi ) = 0, ∂θk

(18)

In general, explicit algebraic solutions of (18) do not exist and the ML estimators must be obtained by an iterative method. For the probit or logit models, L is globally concave, which implies that the ML estimator is unique if it is bounded; hence any iterative procedure that is guaranteed to converge to a stationary point must also converge to the global maximum. Various methods of iteration have been used. Powell’s method has the advantage of involving the calculation of the function L, but not of its derivatives, in contrast to the Newton–Raphson method∗ and the method of scoring, which both involve the first derivatives. See ITERATED MAXIMUM LIKELIHOOD ESTIMATES. For a semiquantal response, the log likelihood involves the application of the multinomial distribution∗ in conjunction with expressions of the form (4) in a generalization of (17). For the Tobit model, the log likelihood involves the sum of components corresponding to the discrete and continuous elements. The log likelihoods corresponding to multiple subsystem and system quantal responses (which reflect the state of several subsystems) may also be defined in terms of the binomial or multinomial distributions. Under certain general conditions the ML estimator is consistent and asymptotically normal. The large-sample variance–covariance matrix of the ML estimators may be calculated as the inverse of the matrix [E{−∂ 2 L/∂θk ∂θl }], where k, l = 1, 2, . . . , s. Tests of hypotheses about the parameters θ may be based on the ML estimators using Wold’s test and the likelihood ratio test∗ . Other forms of estimation have been proposed, largely in an attempt to reduce the computational difficulties associated with the ML method. The most widely used is the modified minimum logit χ 2 method [4] (see MINIMUM CHI-SQUARE). REFERENCES 1. Amemiya, T. (1981). J. Econ. Lit., 1483–1536. 2. Ashford, J. R. (1970). Biometrics, 535–546. 3. Ashford, J. R. (1981). Biometrics, 457–474.

19, 26, 37,

PROBIT MODELS, BAYESIAN ANALYSIS OF 4. Berkson, J. (1955). J. Amer. Statist. Ass., 50, 130–162. 5. Chambers, E. A. and Cox, D. R. (1967). Biometrika, 54, 573–578. 6. Cox, D. R. (1970). The Analysis of Binary Data. Methuen, London. 7. Daganzo, C. (1979). Multinomial Probit. Academic Press, New York. 8. Finney, D. J. (1971). Probit Analysis, 3rd ed. Cambridge University Press, Cambridge, England. 9. Hewlett, P. S. and Plackett, R. L. (1979). The Interpretation of Quantal Response in Biology. Edward Arnold, London. 10. Ipsen, J. and Jerne, N. K. (1944). Acta Pathol., 21, 343–361. 11. Morimune, K. (1979). Econometrica, 47, 957–976. 12. Nerlove, M. and Press, S. J. (1973). Univariate and Multivariate Log-Linear and Logistic Models. Mimeograph No. R-1306-EDA/NIH, Rand Corporation, Santa Monica, Calif. 13. Plackett, R. L. (1974). The Analysis of Categorical Data. Charles Griffin, London. 14. Prentice, R. L. (1976). Biometrics, 32, 761–768. 15. Thurstone, L. (1927). Psychol. Rev., 34, 273–286. See also BIOASSAY, STATISTICAL METHODS IN; ECONOMETRICS; ITERATED MAXIMUM LIKELIHOOD ESTIMATES; NORMAL EQUIVALENT DEVIATE; PHARMACEUTICAL INDUSTRY, STATISTICS IN; QUANTIT ANALYSIS; and RANKIT.

J. R. ASHFORD

PROBIT MODELS, BAYESIAN ANALYSIS OF A practical Bayesian fitting procedure for ordinal response regression models has been proposed [2]. This approach relies on the innovative use of latent data, in conjunction with Markov chain Monte Carlo∗ methods [9,12], to simplify the simulation of the posterior distribution. Consider the binary probit regression model in which n independent measurements yi ∈ {0, 1} are modeled in terms of covariates xi and an unknown parameter vector β. The probit model specifies that pi = Pr(yi = 1) = (xi β), where  is the standard normal

6503

cumulative distribution function. Because β appears nonlinearly in this specification, the posterior distribution of β, given data y = (y1 , . . . , yn ), is difficult to simplify by standard methods, for any given prior. The posterior distribution can, however, be analyzed by the following device. Let zi = xi β + εi where εi is N (0, 1), and let yi = I[zi > 0], where I[·] is the indicator function. This two-equation specification is equivalent to the binary probit regression model, since Pr(yi = 1) = Pr(zi > 0) = (xi β). The Albert–Chib algorithm [2] now consists of simulating the joint posterior distribution of Z and β, where Z = (z1 , . . . , zn ). The Markov chain Monte Carlo (MCMC) simulation is implemented by sampling the conditional distributions [β|y, Z] and [Z|y, β]. The first of these distributions depends only on Z (and not on y) and is obtained by standard Bayesian results on normal general linear models∗ . The second factors into a product of independent distributions [zi |yi , β]. The knowledge of yi merely restricts the support of the distribution zi |β; if yi = 0, then zi  0, and if yi = 1, then zi > 0. Suppose that a priori β ∼ Nk (β 0 , B0 ), where β 0 and B0 are known hyperparameters. Then a sample of β and Z from the posterior distribution is obtained by iterating the following steps: 1. Sample βˆ from the distribution ˆ B−1 ), where βˆ = B−1 (B−1 β 0 + N (β, 0 n n −1  i=1 xi zi ), B = (B0 + i=1 xi xi ). 2. Sample zi from T N (−∞,0) (xi β, 1) if yi = 0 and from T N (0,∞) (xi β, 1) if yi = 1, where T N (a,b) (µ, σ 2 ) denotes the N (µ, σ 2 ) distribution truncated to the region (a, b). Example. To illustrate this fitting algorithm, consider the data in Table 1 from [10]. The first column contains the number of days of radio-therapy received by each of 24 patients, and the second column shows the absence (1) or presence (0) of a disse three years after treatment. The probit model pi = (β0 + β1 xi ) is estimated, where pi is the probability that the disease is absent for the ith patient and xi is the number of days of treatment.

6504

PROBIT MODELS, BAYESIAN ANALYSIS OF

Table 1. Distribution of pi Days 21 24 25 25 26 28 29 31 33 34 35 37 43 43 44 46 46 49 51 51 55 55 56 58

Disease

5th

50th

95th

1 1 1 0 1 1 0 1 1 1 1 1 1 0 0 0 0 1 1 0 1 0 0 0

.64 .62 .61 .61 .60 .59 .58 .56 .53 .51 .50 .46 .33 .33 .31 .26 .26 .19 .15 .15 .08 .08 .07 .05

.89 .86 .85 .85 .83 .81 .79 .76 .72 .71 .69 .65 .52 .52 .50 .46 .46 .40 .36 .36 .28 .28 .26 .23

.99 .97 .97 .97 .96 .94 .93 .91 .88 .86 .85 .81 .70 .70 .69 .66 .66 .62 .60 .60 .56 .56 .55 .53

Using a uniform prior, 5000 values were simulated from the joint posterior distribution of β  = (β0 , β1 ). For each simulated value of β, we compute pi = (β0 + β1 xi ). This procedure gives a simulated sample from the marginal posterior distribution of the probability pi . In Table 1, the 5th, 50th, and 95th percentiles of the posterior distribution of pi are displayed for each patient. Consider next the cumulative probit model for independent ordinal data yi ∈ {0, 1, . . . , J}. Under this model, Pr(yi = j) = (γj − xi β) − (γj−1 − xi β) where γ  = (γ0 , γ1 , . . . , γJ ) are ordered category-specific parameters. The posterior distribution of the parameters (β, γ ) can again be simulated by defining a latent variable zi distributed as N (xi β, 1), but now letting yi = j if zi ∈ (γj−1 , γj ). In this case, the posterior simulation of zi is from a normal distribution truncated to the interval that is implied by the observed value of yi . The simulation of β is as in the binary probit model. Finally, γ is sampled from γ |y,

as recommended in refs. 8, 13, using the Metropolis–Hastings algorithm [6]. The correspondence of the probit model with an underlying continuous latent normal model allows for many generalizations. Albert and Chib [2] extend the probit link by letting zi follow a scale mixture of normal distributions. Hierarchical models∗ and residual analysis have been discussed [2,3], as well as the comparison of alternative probit models via Bayes factors [5,7]. Some of the subsequent applications have involved item response models [1,11], longitudinal data models [4,9], and correlated binary responses using the multivariate probit model [7]. REFERENCES 1. Albert, J. H. (1992). Bayesian estimation of normal ogive response curves using Gibbs sampling. J. Educ. Statist., 17, 251–269. 2. Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Ass., 88, 669–679. 3. Albert, J. H. and Chib, S. (1995). Bayesian residual analysis for binary response regression models. Biometrika, 82, 747–759. 4. Albert, J. H. and Chib, S. (1996). Bayesian probit modeling of binary repeated measures data with an application to a cross-over trial. In Bayesian Biostatistics, D. A. Berry and D. K. Stangl, eds. Marcel Dekker, New York, pp. 577–599. 5. Chib, S. (1995). Marginal likelihood from the Gibbs output. J. Amer. Statist. Ass., 90, 1313–1321. 6. Chib, S. and Greenberg, E. (1995). Understanding the Metropolis–Hastings algorithm. Amer. Statist., 49, 327–335. 7. Chib, S. and Greenberg, E. (1998). Analysis of multivariate probit models. Biometrika, 85, 347–361. 8. Cowles, M. K. (1996). Accelerating Monte Carlo Markov chain convergence for cumulative-link generalized linear models. Statist. and Comput., 6, 101–111. 9. Cowles, M. K., Carlin, B. P., and Connett, J. E. (1996). Bayesian tobit modeling of longitudinal ordinal clinical trial compliance data with nonignorable missingness. J. Amer. Statist. Ass., 91, 86–98. 10. Gelfand, A. E. and Smith, A. F. M. (1990). Sampling-based approaches to calculating

PROBLEM OF POINTS

11.

12.

13.

14.

marginal densities. J. Amer. Statist. Ass., 85, 398–409. Johnson, V. E. (1996). On Bayesian analysis of multirater ordinal data: an application to automated essay grading. J. Amer. Statist. Ass., 91, 42–51. Mendenhall, W. M., Parsons, J. T., Stringer, S. P., Cassissi, N. J., and Million, R. R. (1989). T2 oral tongue carcinoma treated with radiotherapy: analysis of local control and complications. Radiotherapy and Oncol., 16, 275–282. Nandram, B. and Chen, M. -H. (1996). Accelerating Gibbs sampler convergence in the generalized linear models via a reparameterization. J. Statist. Comput. Simulation, 54, 129–144. Tierney, L. (1994). Markov chains for exploring posterior distributions. Ann. Statist., 22, 1701–1762. (With discussion.)

See also BINARY DATA; GIBBS SAMPLING; MARKOV CHAIN MONTE CARLO ALGORITHMS; ORDINAL DATA; and PROBIT ANALYSIS.

JAMES H. ALBERT SIDDHARTHA CHIB

PROBLEM OF POINTS The ‘‘problem of points’’ is the name given to a simple gaming problem that came into prominence in the Renaissance and was not correctly solved until the seventeenth-century French mathematicians Fermat∗ and Pascal∗ considered it. Their famous correspondence about the problem, and Pascal’s treatment of it in his Trait´e du Triangle Arithm´etique, are important milestones in the theory of probability. The problem, also known as the ‘‘division problem,’’ involves determining how the total stake should be equitably divided when a game is terminated prematurely. Suppose that two players A and B stake equal money on being the first to win n points in a game in which the winner of each point is decided by the toss of a fair coin, heads for A and tails for B. If such a game is interrupted when A still lacks a points and B lacks b, how should the total stake be divided between them? Particular cases of the problem have been noted in Italian mathematical manuscripts

6505

as early as 1380, and the Renaissance mathematicians Pacioli, Tartaglia, and Peverone all made unsuccessful efforts at solutions. It was brought to Blaise Pascal’s attention by Antoine Gombaud, Chevalier de M´er´e, and led to a correspondence with Pierre de Fermat in the late summer of 1654. Unfortunately, not all the letters have survived, but it is possible to reconstruct the sequence of events from the remainder. Pascal sought Fermat’s opinion, and Fermat replied in a letter which, although missing, like Pascal’s, must have contained solutions to two gaming problems, one of which was the problem of points, for in his reply on July 29, Pascal expressed his agreement with Fermat’s solution to the latter, which had also occurred to him. If A wants a points and B wants b, they argued, then the game must be over in (a + b − 1) further tosses at most, which may occur in 2a+b−1 equiprobable ways. Each of these ways may be seen, on examination, to be a win either for A or for B, and on counting them the probability (as we should now say) of A winning is found, and hence the equitable division of the stakes given that the game is not in fact continued. In other words, the actual game has been embedded in a hypothetical game of fixed length which is easy to solve. In the same letter Pascal offered as an alternative solution an analysis of the possible games in which, working backwards, he uses recursively the idea that if expectations of gain of X and Y units are equally probable, the expectation of gain is 12 (X + Y) units, thereby introducing into probability theory the notion of expectation. Then, having appreciated that with only two players the order of occurrence of heads and tails in the hypothetical game does not matter, he makes use of the fact that the hypothetical game is solved by the partial sum of a binomial distribution∗ for equal chances. In his contemporary Trait´e du Triangle Arithm´etique he proves this result by applying mathematical induction to the method of expectations. The result itself is often attributed to Fermat, and the method of expectations to Huygens∗ . Not only are they Pascal’s but he has also introduced the binomial distribution for equal chances, albeit by

6506

PROBLEM OF THE NILE

reference to the arithmetical triangle rather than algebraically. The remainder of the Fermat–Pascal correspondence is devoted mostly to the case of three players, which Fermat had suggested could readily be solved by the original ‘‘combinatorial’’ method. Pascal thought that by this Fermat meant assigning each of the possible hypothetical games of fixed length to the appropriate player according to the numbers of points each had won without regard to order, as was correct with two players, and he demurred. Fermat reassured him that he had intended, rather, that each possible sequence should be assigned, which leads to the correct solution, and he added for good measure the direct probability solution to an example with three players, in which he simply analyses the possible games in a forward direction (as a modern student would do with the help of a tree diagram) in contrast to Pascal’s backward ‘‘expectation’’ solution. Apart from some minor generalizations introduced by Montmort∗ , the Bernoulli∗ brothers, and De Moivre∗ half a century later, the correspondence between Fermat and Pascal and the account in the latter’s Trait´e du Triangle Arithm´etique essentially disposed of the problem of points and in doing so created an incipient theory of probability where previously there had only been simple enumerations. The important methodological contributions of operating on expectations and of constructing the binomial distribution for equal chances and applying it to the problem of points were Pascal’s alone, however. BIBLIOGRAPHY The account above relies on the primary sources directly, as follows. For the Fermat–Pascal correspondence, see either Oeuvres de Fermat, edited by P. Tannery and C. Henry, Vol. II, Correspondance (Gauthier-Villars, Paris, 1894), or B. Pascal, Oeuvres Compl`etes, edited by J. Mesnard, Volume II, (Descl´ee de Brouwer, Paris, 1970). English translations (which should both be treated with reserve) are available in A Source Book in Mathematics, by D. E. Smith (McGraw-Hill, New York, 1929), and Gods, Games and Gambling, by F. N. David (Charles Griffin, London, 1962). Pascal’s Trait´e du Triangle Arithm´etique was originally published by Desprez in Paris in 1665 and is

contained in his Oeuvres cited above. There is a partial English translation in D. E. Smith (see above) and an even lesser part in A Source Book in Mathematics, by D. J. Struik (Harvard University Press, Cambridge, Mass., 1969), but neither includes the treatment of the problem of points. There are many secondary accounts, but we mention only the invaluable A History of the Mathematical Theory of Probability, by I. Todhunter (Macmillan, Cambridge, 1865; reprinted by Chelsea, New York, 1965). See also FERMAT, PIERRE DE; GAMES OF CHANCE; PASCAL, BLAISE; and PROBABILITY, HISTORY OF.

A. W. F. EDWARDS

PROBLEM OF THE NILE. See FISHER’S PROBLEM OF THE NILE

PROBLEM SOLVING IN STATISTICS A basic approach to problem solving in statistics is to compare sets of numbers: to compare observations with theoretical models, past experience, or other observations in numerical form [9]. Another approach is to make estimations based on models, experience, or observations. Such comparisons and estimations are valid and informative only if the observations and models are appropriately (and measurably) representative of some phenomenon or population. Therefore, a working knowledge of statistics is important in the planning stages of investigations as well as in determining what statistical comparisons and models are appropriate. A common complaint of practicing statisticians is that often statistical consultants are not called in until after the data have been collected [25]. One of the purposes of this article is to stress the full range of the statistical process in order to correct a narrow public image of the discipline. To provide a systematic framework for a discussion of problem solving in statistics, a conceptual model has been adapted from information systems literature, including three distinct phases: (1) study design, (2) data collection∗ , and (3) data analysis. These phases, outlined in Table 1, have been presented as an alternative to an experimental model in a business context [16]; the process

PROBLEM SOLVING IN STATISTICS

6507

Table 1. Phases and Steps of Statistical Problem Solving Phase 1. Study design

2. Data collection

3. Data analysis

Step a. Question definition b. Alternative strategies development c. Strategy evaluation d. Strategy selection a. Sample design b. Measurement a. Statistical analysis b. Reliability assessment c. Report generation

Activity The question is defined in terms of the need for information. Alternative strategies are developed and specified for sampling, data collection, and analysis. The advantages and disadvantages of the feasible alternatives are evaluated. A strategy is selected on the basis of costs and the importance of the information to the organization. Sampling procedures are planned on the basis of work done in step 1b and the selection made in step 1d. Observations are chosen and recorded in a form that facilitates analysis Statistical methods are used for estimating or summarizing. Measures of possible error in results are calculated. The results are reported in a form useful to others.

model has been revised here to encompass the general procedures required in any application area. PHASE 1: STUDY DESIGN The first phase involves determining the optimum strategy to solve the problem. The steps from the beginning of the process until final selection of a strategy are illustrated in Fig. 1. These are important steps since inadequate planning means no data, or worse, meaningless data, to analyze later. The experimenter should consider all the possibilities of misfortune as well as unexpected interactions of the variables, because results are seldom exactly what is anticipated. If a statistical consultant is involved, he or she should take an active part in planning the methodology of the study itself and not wait to help with the analysis of data that may or may not be meaningful [14,15]. Problem Definition In the first step of the study design phase the question or problem must be defined in terms of the need for information. Tukey [28] cautions that ‘‘finding the question is often more important than finding the answer.’’ This question-finding step can be a very

difficult one with none of the tidiness of a well-defined mathematical problem [15,23]; however, it is crucial to the meaningfulness of the subsequent phases and steps. If the scientist, economist, demographer, or whoever who wants the problem solved has the required statistical skills, the first phase is mainly one of defining the question clearly in order to proceed with an efficient design. ‘‘What is the real question I am interested in, and how can it logically be phrased so that it is answerable with data that can be obtained with available resources?’’ If a statistician is assisting one or more experts in a consulting∗ situation, the statistician should aid in the problem definition process by discussing the problem and determining the exact question of interest. Consultants have reported that their clients have found this initial problem definition assistance to be the most valuable service provided by the statistician [3,7,14]. Several articles [3,14,21] specifically focus on the help that statisticians can provide in developing the objectives of a study. Salsburg [23] describes his role in this step as one of asking stupid questions and worrying out loud about everything that could go wrong in the data collection stage. Statistics: A Guide to the Unknown [27] provides

6508

PROBLEM SOLVING IN STATISTICS

Figure 1. Steps in the study design phase.

reasonable examples of statistical designs and analysis to answer such ill-defined, difficult, and interesting questions as: Are police effective? Can people control when they die? In these examples and other real problems, the definition of the logical issues and the experimental design are the main challenge, rather than the choice and execution of analysis techniques. To be effective in the planning stages of an investigation, the statistician must know enough of the background subject (or ask enough questions) to be sure that the study fits the real problem [13,24]. Working with a client to determine the logical issues of a problem also may take special interpersonal skills; suggestions on this topic can be found in a discussion of the use of transaction analysis by statistical consultants [3], an identification of consulting roles [14,25], and a reading list on the training of consultants [2]. Identification of Alternatives After the problem is well defined, the statistician develops alternative procedures for the

experiment, sample survey, or other procedure appropriate for data collection. The value of the information gained in the final phase depends largely on the thoroughness of the data collection and data analysis phases, so the statistician must plan ahead. Failing to allow for dropped test tubes or uncooperative subjects may mean that there will be no usable data for the analysis stage. Evaluation of Alternatives For each alternative procedure the statistician should present a cost estimate and an error range indicating the reliability of the answer. The optimum strategy is not necessarily the one that provides the most information. Within any organization cost factors must be weighed against benefits, so the ‘‘best’’ strategy is one that provides adequate information with a justifiable cost and a reasonable amount of time. (A statistician who does not appreciate this fact is unlikely to enjoy a successful career in industry.) Each feasible alternative must be evaluated. In the final step of phase 1 a strategy is selected on the basis of cost and the

PROBLEM SOLVING IN STATISTICS

importance of the information to the organization. If the statistician has done a good job, the chosen strategy will provide a specified quantity of information at a minimum cost. At the end of phase 1 the project is approved with an understanding of how much time the study will take, how much it will cost, and the quality and type of inferences that can be made from the results. A written statement of the objectives of the study and the time and budget constraints should be prepared by the statistician before the study proceeds to the data collection stage. See CONSULTING, STATISTICAL; INDUSTRY, STATISTICS IN; PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE. PHASE 2: DATA COLLECTION The data collection phase includes two steps: design of the sampling procedure and measurement of observations or collection of data. The time requirements and difficulty level of these steps depend on the problem. The first step consists of planning what will be done in the second step. Time-consuming, complicated observations and measurements will usually require time-consuming, detailed planning in the first step. Sample Design The total group to be studied was determined in the problem definition step of the process. The sample may be birds in a field [13], treated water [5], filtered air [21], or earthworms [26], to name a few examples from the literature that discuss the entire problemsolving process. In a quality control∗ study the target population may be all the machine parts delivered by a particular supplier. Testing millions of machine parts supposedly checked by the original manufacturer would be infeasible for the manufacturer who wishes to use these parts in assembling a product; instead, sampling is used. In a study of the environmental impact of a factory process the population or universe under investigation may be treated water in a lake, filtered air in a plant facility, or shrimp in a bay. Again it would be infeasible to measure impurities in all the air and water or to check every shrimp for contamination. The sampling procedures

6509

for the machine parts, air, water, and shrimp are based on probability theory∗ . The planning for the sample is done in the first step of this data collection phase. Measurement The complexity of the planning step varies considerably with the problem. For projects that require complicated data collection procedures, the second step of this phase requires careful planning. The design to select the air, water, and shrimp samples takes considerable time, as to the actual measurement and observation procedures in the second step of this phase. Accountants rely on sampling to audit large organizations. Courts depend on sampling to determine compensation in law suits involving thousands of plaintiffs. Market research surveys always involve sampling. An effective sampling design can ensure a valid audit, appropriate compensation, and invaluable market forecasts, without sacrificing economy (see MARKETING, STATISTICS IN). In the easiest case the sample includes all company computer records pertinent to the question and available at a particular time. In this situation the planning for data collection requires the choice of which file of information is appropriate for the question of interest; this might be the choice of annual, monthly, or end-of-the-day data on a particular date. The next step, the actual data collection, is completed by one command in a computer job that accesses the chosen data file. For example, airlines regularly analyze route schedules and passenger loads. The data source is the company file that is created and updated in the reservation process that goes on every hour of the day in airports and travel agencies around the world. Making this file of passenger information available with one computer command is all that is necessary before proceeding to the analysis phase. In the other extreme the Bureau of the Census takes years to plan and carry out the data collection for the U.S. census and to make available the results. Demographic studies are used to choose the sample of individuals asked to complete more detailed census forms. The actual collection of data

6510

PROBLEM SOLVING IN STATISTICS

and recording of survey answers involves thousands of full-time and part-time employees of the Bureau of the Census∗ . Issues in Data Collection The role of the statistician in developing sound methodology is stressed in the study design phase. The logical issues of problem solving in the definition of the research question are critical to the value of any study, even though they are not basically a mathematical concern. In the data collection∗ phase, the role of the statistician as a scientific advisor is just as important. The difficulties of designing the sample and problems with the actual measurement are considerable. Insofar as statistics is an applied science, justified by the ‘‘help it can give in solving a problem . . . [reaching] a decision, on a probabilistic basis’’ [18], the problem-solving methodology must be repeatable and reproducible [1]. Such repeatability and reproducibility are dependent on the quality of statistical data. Even the most sophisticated multivariate analysis∗ cannot correct for data that are inconsistent or clearly wrong. Kruskal [17] discusses fascinating problems arising in the public policy application of statistics which are often neglected since they fall in the cracks between disciplinary labels: (1) wrong data (e.g., 1960 census data showing 62 women aged 15 through 19 with 12 or more children) identified by lack of smoothness, logical inconsistencies, or transgressions of general knowledge; (2) ambiguity of classification (when might an unemployed person be said to be in the labor force?); (3) lack of data responsibility (unwillingness to publish the raw data); which relates to (4) confidentiality (can we learn information about specific individuals from a published table of cross classifications?). Traditionally, the ‘‘quality of statistical data’’ has referred to its accuracy [1], which is often judged by insisting on high precision and low bias. In judging this quality, we must realize that smoothness of repeated observations does not guarantee that the estimate is close to the ‘‘truth.’’ When we report ‘‘estimate ± something,’’ we imply an emphasis on precision, which does not necessarily imply accuracy. (See Eisenhart [11] for a comprehensive discussion of

this distinction.) Therefore, a complete and clear disclosure of the measurement process is necessary for those who will evaluate the results. (See CENSUS; MEASUREMENT ERROR; SAMPLING PLANS; SURVEY SAMPLING; TELEPHONE SURVEYS, COMPUTER ASSISTED).

PHASE 3: DATA ANALYSIS The data analysis phase includes three steps: statistical analysis, reliability assessment, and report generation. The sequence of steps is shown in Fig. 2. The methods employed in this phase are the main concern of statistics. Statistical Analysis Articles on specific statistical techniques are a major portion of this encyclopedia, so no attempt at a complete list will be made here. Summary lists of the most frequently used techniques have been complied for industrial statisticians [22], consultants [19,20], and statisticians in the federal government [12]. Basic analysis strategies include hypothesis testing∗ (the approach used in experimentation), estimation∗ techniques, and exploratory data analysis∗ (for a general understanding of relationships that exist in data). Summaries of statistical applications in various fields are also provided in this encyclopedia; for example, see AGRICULTURE, STATISTICS IN; BIOASSAY, STATISTICAL METHODS IN; BIOSTATISTICS, CLASSICAL; CHEMISTRY, STATISTICAL METHODS IN, and so on. Impact of Computers on Analysis The availability and accessibility of computers has had a pervasive impact on statistics as complicated statistical procedures come into routine use in analysis of large data sets (see COMPUTERS AND STATISTICS). Multivariate techniques of stepwise regression∗ , discriminant analysis∗ , and categorical techniques are now feasible as exploratory tools due to the extensive calculating ability of computers. Statistical packages have made data analysis (and misanalysis) widely accessible with multidimensional computer graphics and quick results of data transformation.

PROBLEM SOLVING IN STATISTICS

6511

Figure 2. Steps in the data analysis phase.

In addition to extending the accessibility of established statistical methodology, computers have played a vital role in the development of new statistical techniques. Diaconis and Efron [8] observe that the most commonly used statistical methods were developed when computation was slow and expensive; they point out that many new methods are free of distributional assumptions and are feasible, as a result of the use of computers. Monte Carlo∗ simulation techniques are easily employed to examine the performance of a new estimator or test. Efron [9] asserts that ‘‘computer assisted theory is no less ‘mathematical’ than the theory of the past; it is just less constrained by the limitations of the human brain.’’ The empirical Bayes∗ procedure is an example of a technique feasible only with the advent of computer technology. Empirical Bayes’ inference differs from Bayesian inference∗ in that a prior distribution∗ is assumed but not known, thus allowing the applied statistician to derive from the data a compromise between classical frequentist and Bayesian estimators. This procedure has been shown to provide accurate estimates in practical multivariate applications and is an example of a procedure whose performance has been assessed without formal proof, but with the aid of a computer. The computer makes feasible nonparametric estimators involving massive amounts of computation. Consider, for example, the simple problem of determining whether two

samples have been drawn from a single population. A frequentist addresses this question with a t-test, which uses only a few calculations but assumes the (common) population is Gaussian. Computers make possible the randomization∗ approach of deriving all possible ways of partitioning the combined data into two like-sized samples and assessing the likelihood of the original partition among all possible samples. Additionally, computers make possible the use of the jack-knife∗ and bootstrap∗ procedures to measure accuracy of estimators of population parameters without the Gaussian assumption (see SAMPLE REUSE). Cross-validation, combinations of robustness and efficiency, and analysis of censored data∗ additionally illustrate the impact of computers on previously impossible data analytic problems. Thus an aid to statistical analysis has now begun to shape the statistical methods themselves. Reliability Assessment The second step of the analysis phase is to determine how much error can be expected in the estimations or inferences made from a statistical procedure. Each technique presented in this encyclopedia includes a measure of reliability to indicate how much confidence can be placed in a decision based on the statistical results. In the final step of the analysis phase, the results must be reported in a form that is intelligible to the user. Stating the results of a statistical investigation in terms of the

6512

PROBLEM SOLVING IN STATISTICS

original objectives of the study is analogous to turning the numerical results of an arithmetic word problem back into words. [13]. When complicated statistical procedures have been used, this is a very important step; when controversial issues are involved, it is a very difficult step [22]. Ehrenberg [10] provides tips for effective statistical report writing. Hunter [15] and Salsburg [23] provide motivation. See also GRAPHICAL REPRESENTATION OF DATA. POSTPROCESS RESPONSIBILITIES At the conclusion of any worthwhile study a decision is made based on the results. The decision may be to market a new drug that has been shown to have no adverse effects, reallocate federal funds to states due to shifts in population, continue with present policies of proven worth, or gather more information for further study. This postprocess decision is usually out of the hands of the statistician. Technically, it should not be the responsibility of the statistician to determine what the implications of a 0.05 significance level are in each of the examples just named or even to set the significance level. But, perhaps, professional responsibilities require some continued interest in the actions taken based on statistical results. Kruskal [17] outlines several problems that affect the value of results of statistical studies for which no one will take responsibility—statistician or client. The ethical guidelines proposed in 1983 have also generated some questions about the responsibility of the statistician as opposed to the responsibility of the organization that acts on the study results. For example, should a statistician allow his or her work to be represented as investigating ‘‘cancer cure rates’’ when success was measured as a cancer victim’s failure to die [6]? CONCLUSIONS Statistics is a broad problem-solving process used for organizing and thinking about logical issues as well as modeling data to arrive at inferences about the true state of nature. As educators, statisticians should be concerned about teaching statistical problemsolving skills as well as methods. Managers,

engineers, and anyone else who requires the services of a statistician should at least learn that the proper time to hire is at the beginning of a statistical study, not after the data are gathered. No amount of probability theory or formula solving with clean data will communicate this knowledge effectively. It is necessary to convey a sense of the importance of structuring real problems so that they can be solved, as well as to teach statistical theory and methods. Acknowledgments We are grateful to G. J. Hahn, W. G. Hunter, and M. M. Whiteside for helpful comments on an early draft of this article. We also thank South-Western Publishing Co. for permission to use copyrighted material, including Figs. 1 and 2, from Introduction to Business and Economic Statistics (7th ed.) by C. T. Clark and E. W. Jordan.

REFERENCES 1. Bailar, B. A. (1983). Bull. Int. Statist. Inst., 813–834. (A complete discussion of concepts related to the quality of statistical data.) 2. Baskerville, J. C. (1981). Amer. Statist., 35, 121–123. (Problem solving is discussed briefly in the context of a course in statistical consulting; bibliography for course included.) 3. Bishop, T., Petersen, B., and Trayser, D. (1982). Amer. Statist., 36, 387–390. (Illustrations of types of information that should be obtained during experimental planning stages.) 4. Boen, J. and Fryd, D. (1978). Amer. Statist., 32, 58–60. (Interactions between statisticians and clients are explained in terms of transactional analysis; valuable for both parties.) 5. Box, G. E. P., Hunter, W. G. and Hunter, J. S. (1978). Statistics for Experimenters. Wiley, New York. (Chapter 1 is a readable overview of the role of statistics in experimental design; text is oriented toward scientific and engineering problem solving with topics such as ‘‘lurking variables’’ and reducing data models to ‘‘white noise.’’) 6. Bross, I. D. J. (1983). Amer. Statist., 37, 12–13. (Examples of statistical results that are misrepresented by policy makers; relevant to issue of boundary of responsibilities of statistician in reporting phase.) 7. Daniel, C. (1969). Technometrics, 11, 241–245. (Discussion of elements required

PROBLEM SOLVING IN STATISTICS

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

for successful statistical problem solving in consulting situations.) Diaconis, P. and Efron, B. (1983). Science, Apr., 116–130. (Summary of statistical methods made usable by computers.) Efron, B. (1979). SIAM Rev., 21, 460–480. (Very brief summary of standard pre-1950 statistical problem solving, followed by examples of statistical approaches that have changed dramatically due to computers.) Ehrenberg, A. S. C. (1982). A Primer in Data Reduction. Wiley, New York. (Chapters 16 and 17 cover the use of tables and graphs in reports; Chapter 18 includes recommendations on organizations and writing style.) Eisenhart, C. (1963). J. Res. Nat. Bur. Stand. C. Eng. Instrum., 67, 161–187. (A discussion of issues relating to precision and accuracy.) Eldridge, M. D., Wallman, K. K., Wulfsberg, R. M., Bailar, B. A., Bishop, B. A., Bishop, Y. M., Kibler, W. E., Orleans, B. S., Rice, D. P., Schaible, W., Selig, S. M., and Sirken, M. G. (1982). Amer. Statist., 36, 69–81. (Describes problem solving in a wide variety of positions for statisticians in the federal government.) Hahn, G. J. (1984). Technometrics, 26, 19–31. (Examples of problem solving in engineering and science with recommendations on how to improve the practice and use of statistics in industry.) Hunter, W. G. (1981). Amer. Statist., 35, 72–76. (Discussion of three roles played by statistical consultants; valuable for practicing statisticians and educators in applied statistics programs.) Hunter, W. G. (1981). Statistician, 30, 107–117. (A very enjoyable series of stories about the reality of applied statistics.) Jordan, E. W. (1982). Amer. Statist. Ass., 1982 Proc. Statist. Educ. Sec. (Examples of problem solving in business with recommendations on how to improve education in statistics for managers.) Kruskal, W. H. (1981). J. Amer. Statist. Ass., 76, 1–8. (Discussion of problems, such as data integrity in large data sets, that affect the validity of results of statistical studies, but that are not acknowledged as the responsibility of statisticians.) Mahalanobis, P. C. (1950). Sankhya, ¯ 10, 195–228. (A delightful essay entitled ‘‘Why Statistics?’’) Marquardt, D. W. (1979). Amer. Statist., 33, 102–107. (Very broad view outlined for responsibilities of statisticians in industry.)

6513

20. Marquardt, D. W. (1981). Amer. Statist., 35, 216–219. (Argument for total involvement in problem solving by statisticians in industry.) 21. Price, B. (1982). Amer. Statist. Ass., 1982 Proc. Phys. Eng. Sci. Sec. (The need to clarify objectives and specify realistic achievements for any study is discussed within the context of an environmental regulations problem.) 22. Raiffa, H. (1982). Amer. Statist., 36, 225–231. (Discussion of the difficulty of maintaining statistical objectivity in controversial application areas of risk analysis for problems such as chemical waste disposal; includes examples that could divest anyone of an image of statistics as a cold science.) 23. Salsburg, D. S. (1973). Amer. Statist., 27, 152–154. (Delightful article about how a practicing statistician earns a living.) 24. Snee, R. D. (1982). Amer. Statist., 36, 22–87. (Argument for broader view of the field of statistics, including topics that should be added and roles that should be acknowledged.) 25. Snee, R. D., Boardman, T. J., Hahn, G. J., Hill, W. J., Hocking, R. R., Hunter, W. G., Lawton, W. H., Ott, R. L., and Strawderman, W. E. (1980). Amer. Statist., 34, 65–75. (Includes list of statistical techniques that are used most often in problem solving by statisticians in industry; discussion of personal characteristics and training necessary for industrial statisticians.) 26. Sprent, W. G. (1981). J. R. Statist. Soc. B, 133, 139–164. (Difficulties in consulting are discussed; necessity of consultant being involved in projects is stressed by Sprent; wide variety of comments by members follow, including argument about role of statistician in problem solving.) 27. Tanur, J. M., Mosteller, F., Kruskal, W. H., Link, R. F., Pieters, R. S., Rising, G. R., and Lehmann, E. L., eds. (1978). Statistics: A Guide to the Unknown, 2nd ed. Holden-Day, San Francisco. (Series of examples from all fields of applied statistics.) 28. Tukey, J. W. (1980). Amer. Statist., 34, 23–25. (An argument for both exploratory and confirmatory methods in data analysis.) See also COMPUTERS AND STATISTICS; CONSULTING, STATISTICAL; DATA COLLECTION; DESIGN OF EXPERIMENTS; EXPLORATORY DATA ANALYSIS; HYPOTHESIS TESTING; INDUSTRY, STATISTICS IN; PRINCIPLES OF PROFESSIONAL STATISTICAL PRACTICE; QUALITY CONTROL, STATISTICAL; and SAMPLING PLANS.

ELEANOR W. JORDAN DONNA F. STROUP

6514

PROCESS CAPABILITY INDEX, WRIGHT’S

PROCESS CAPABILITY INDEX, WRIGHT’S

2. Wright, P. A. (1995). A process capability index sensitive to skewness. J. Statist. Comput. and Simul., 52, 195–208.

A process capability index which takes skewness into account was proposed by Wright [2]. The index is defined as

See also CAPABILITY and MEAN, MEDIAN, AND MODE.

min(USL − µ, µ − LSL) Cs = 3 σ 2 + (µ − T)2 + |µ3 /σ | d − |µ − M| , = 3 σ 2 + (µ − T)2 + |µ3 /σ | where USL and LSL are the upper and lower specification limits, µ and σ are the mean and the standard deviation of the process, respectively, T is the target value, d = (USL − LSL)/2 is the half length of the specifications interval, M = (USL + LSL)/2 is the midpoint of this interval, and µ3 = (X − µ)3 is the third central moment. It is implicitly assumed that the target value is at the midpoint. It seems to be the only index of capability which accounts for skewness that is proposed in the literature. A natural estimator of Cs is given by d − |X − T| ˆs =  , C 3 S2 + (X − T)2 + |m3 /S|   where m3 = (Xi − X)3 /n, X = Xi /n, and  S2 = (Xi − X)2 /n. Wright also proposes a somewhat more complicated estimator of Cs to achieve efficient small sample behavior. ˆ s have been Asymptotic properties of C investigated by Chen and Kotz [1]. The index and its estimator are quite sensitive to skewness; the estimator has an asymptotic normal distribution only when µ = T and µ3 = 0, which indicates that when the process is ‘‘deviating’’ in both shape and the target, ˆ s is indeed a natural choice of a meaC sure to assess the process capability. However, if the process stays symmetric, the simpler canonical index Cp could be recommended. REFERENCES 1. Chen, H. F. and Kotz, S. (1996). An asymptotic distribution of Wright’s process capability index sensitive to skewness. J. Statist. Comput. and Simul., 54, 27–38.

PROCESSES, DISCRETE All physical and natural phenomena are subject to environmental and other types of fluctuations or noises. These phenomena are therefore of stochastic nature. A stochastic process∗ arises when one observes the evolution of such a random phenomenon as a function of time. When the process is observed at discrete time points, one obtains a discrete process. More precisely, a discrete process is a sequence Xn , n
1, of random variables (RVs) defined on an appropriate sample space . A space X containing all values of the RVs Xn , n
1, is called the state space of the process. In most practical cases X is a Euclidean space Rm . Consider, for example, the study of a population growth at nonoverlapping generations. Then the population size Xn at generation n is a RV, and Xn , n
1, is a discrete process. Discrete processes arise in the theories of statistical estimation and optimal filtering. For example, let θ (t) be a deterministic signal transmitted at time points t = 1, 2, . . ., and ξ (t, ω) be random noises entering the communication channel at these time points. Then the signal received is given by X(t) = θ (t) + ξ (t). In estimation theory, one needs to determine operations on the signal X that reproduces θ as precisely as possible. We seek an operation that produces a discrete process θ (t) such that E[X(t) − θ (t)] = 0 and E[X(t) − θ (t)]2 is minimum. (We will return to this problem after some mathematical preliminaries.) Let Xt be a discrete process with state space X . Let us first clarify the meaning of the adjective ‘‘discrete’’. We began by assuming that the time parameter set T is discrete (such as T = 1, 2, . . .). In defining discrete processes, some authors let t vary continuously while assuming that the state space is discrete. While noting that a countable state space is a particular case of a general state space, we also see that discrete time is not a particular case of continuous time.

PROCESSES, DISCRETE

To be more precise, consider the cases where T = {0, 1, 2, . . .} = N0 , and T = R+ . While R+ has plenty of limit points, N0 has ∞ as its only limit point. Consequently, in studying the limit behavior of a discrete-time process one considers the asymptotic behavior at ∞, whereas in the continuous-time process case many new properties such as the local path properties arise. Furthermore, most of the results in the discrete-time case concern the asymptotic behavior at ∞ of functions of the process. Therefore, it is more appropriate to name the discrete-time case as the discrete process; of course, the state space X can be discrete or continuous. Let X = {Xn (ω), n
1} be a discrete process. It is common to suppress the variable ω in Xn (ω) and write it only as Xn . If we fix ω ∈ , we get a sequence Xn in X , called a sample path or sample sequence. Thus each sample point in  represents a path in the state space X . To keep the presentation simple we shall assume henceforth that X = R = R1 . For every integer d
1, let us arbitrarily choose time points n1 < n2 < · · · < nd . The joint distribution of the RVs Xn1 , Xn2 , . . . , Xnd is called a d-dimensional distribution of the process X = {Xn }. The collection of all these finite-dimensional distributions for all possible choices of n1 < · · · < nd and d
1 governs the probabilistic laws of X. Stochastic processes are classified according to the special properties of their finite-dimensional distributions. We need not distinguish between two processes X = {Xn } and Y = {Yn } that have the same finite-dimensional distributions; that is, the multivariate distributions of Xn1 , . . . , Xnd and Yn1 , . . . , Ynd are the same for all d
1 and all choices of n1 < · · · < nd ∈ T. The researcher is therefore free to choose, from a class of processes having the same finite-dimensional distributions, any process that he or she wishes to work with. In many practical situations the researcher knows the finite-dimensional distributions of a stochastic phenomenon and would like to associate a discrete process with them. Such an association is possible under Kolmogorov’s extension theorem [13]. Let Fn1 ...nd (ξ1 , . . . , ξd ) denote the finite-dimensional distributions of the RVs Xn1 , . . . , Xnd of a process X. Then the set of all finite-dimensional distributions

6515

satisfies the consistency condition lim Fn1 ...nd (ξ1 , . . . , ξd )

ξk ↑∞

= Fn1 ...nk−1 nk+1 ···nd (ξ1 , . . . , ξk−1 , ξk+1 , . . . , ξd ). Conversely, given a sequence {Fn } of ndimensional distributions satisfying the consistency condition limξn ↑∞ Fn (ξ1 , . . . , ξn ) = Fn−1 (ξ1 , . . . , ξn−1 ), one can find a sample space , an appropriate probability P on , and a process X = {Xn } such that Fn (ξ1 , . . . , ξn ) = P{X1  ξ1 , . . . , Xn  ξn }, n
1. In place of the finite-dimensional distributions Fn of a process X, one can work with the probabilities P{X1 ∈ B1 , . . . , Xn ∈ Bn }, Bk ∈ B, where B is the Borel σ -algebra on R. Kolmogorov’s extension theorem will transform a process X into a probability measure on  = RT . Treating a discrete process as a probability measure finds nice applications in statistics, for example, in maximum likelihood estimation∗ . Let θ = θt be an unknown parameter, and X(t) = θt + ξt , t = 0, ±1, . . ., where ξt is a suitable stationary Gaussian process∗ . Let µθ and µ0 be the distributions or measures corresponding to the processes Xt and ξt , respectively. To obtain the maximum likelihood estimator of θ , one needs to find the Radon–Nikodym∗ derivative dµθ /dµ0 , the density function of µθ with respect to µ0 (see Ibragimov and Rozanov [9]). A discrete process X = {Xn } is a secondorder process if EXn2 < ∞ for all n. For such a process the functions µ(n) = EXn and γij = EXi Xj form the mean sequence and correlation matrix of X, respectively. The covariance matrix of X is defined analogously. Secondorder processes can very profitably be studied since, on one hand, the (equivalence class of) second-order RVs form a Hilbert space L2 (, T , P) and, on the other hand, for each second-order process with mean sequence µ(n) (= 0, without loss of generality) and covariance matrix C = [Cij ], we can find a process Y = {yn }, possibly defined on a different probability space, such that Y has the same mean sequence, the same covariance matrix C, and such that the finitedimensional distributions of Y are all Gaussian (see Ibragimov and Rozanov [9]). Hence,

6516

PROCESSES, DISCRETE

if one has a second-order process to work with, one can assume without much loss of generality that it is a Gaussian process as well. Gaussian processes arise most often in statistical applications. Some of the other useful discrete processes are processes with uncorrelated RVs, processes with orthogonal RVs, processes with independent increments, Markov chains, stationary processes, martingales∗ , and mixing processes. We now present some fairly general examples of discrete processes that arise in statistics and in other applications. Defining a discrete process in a recursive fashion is quite common in statistical estimation theory and learning theory (in psychology). 1. Let us begin with a simple case of Tmaze learning in psychology. On each trial a rat is placed at the bottom of a T-maze. The rat makes a choice, at the fork of the T-maze, and proceeds to the end of the left or right arm (response e0 or e1 ). There the rat may or may not find food. Finding food will reinforce the response just made. Consider now a state space X and a response space E. Corresponding to each response e ∈ E, there is a contraction or distance diminishing function fe : X → X . If the experiment is in state xn at trial n, a response en is determined and xn+1 is given by xn+1 = fen (xn ). The process xn , n
1, is a discrete process roughly describing the learning; for details, see Norman [14] and Cypkin [6]. 2. Some statistical estimates are discrete processes defined as functions of other discrete processes. Let θ be a Gaussian signal N(m, σ 2 ). What one observes is yk = θ + ξk , k
1, where ξk , k
1, are additive noise consisting of independent RVs with a common density f (x). Then the best estimate xn (in the sense of standard error) is xn = E(θ |y1 , . . . , yn ). In principle the estimate xn can be computed by Bayes’ formula. This method is too tedious and does not render itself to practical applications and the study of the solution. For computational purposes one can follow the recursive procedure. For example, yk+1 = ξk+1 + A(y1 , . . . , yk )θ + B(y1 , . . . , yk ). Here A and B are the so-called ‘‘controlling’’ functions. Using such a recursive procedure and appealing to the theorem on normal correlation (stated below) one can obtain the estimates xn , the tracking errors

ek = E(θ − xk )2 , and the conditional variance σk2 = E[(θ − xk )2 |y1 , . . . , yk ]. (Further details are given below.) 3. Finally, we give a random walk∗ model of a collision diffusion process∗ that arises very commonly in the study of particle transport in dilute gas. In particular, the discrete process we define below approximates the random motion of two elastic particles of identical mass which move in such a way that they exchange velocities when they collide. Let P1 and P2 be two particles moving on the same one dimensional lattice such that (a) initially P1 is an even number of units of distance to the left of P2 ; (b) each particle moves independently of each other with steps +1 or −1; (c) if at any time n the two particles are not at the same place, then at time n + 1 each moves +1 or −1 step with probability 12 ; and (d) if at any time n both are at the same point x0 , then at time n + 1 they move according to P(both at x0 − 1) = 14 , P(both at x0 + 1) = 14 , P(P1 at x0 − 1, P2 at x0 + 1) = 12 . If xn and yn , n
0, are two independent simple random walks with probability 12 for steps of +1 or −1 unit, and x(0) − y(0) is a negative even integer, let ξn = min(xn , yn ) and ηn = max(xn , yn ). Then the random walk with collision is given by the discrete process Xn = (ξn , ηn ). As remarked earlier, an important class of results of a discrete process {Xn } is the asymptotic behavior as n → ∞. The sequence Xn can converge to a limit X in several senses: convergence∗ in distribution, in probability, in mean, in mean square, almost sure convergence, and so on. The notions lead to different classification of estimates. For example, let ξ1 , . . . , ξn be independent RVs following the same distribution whose density f (y, x) depends on a parameter x belonging to a class X . Let Xn , n > 1, be a process obtained in the procedure of estimating x. Let Px and Ex be the corresponding probability measure and expectation respectively. An estimate Xn is called asymptotically unbiased (respectively, strongly consistent, consistent, asymptotically normal, strongly asymptotically efficient, or asymptotically efficient) if (as n → ∞), Xn → x in mean (respectively, almost surely, √ n(Xn − x) → N(0, σ 2 (x)) in in probability,

PROCESSES, DISCRETE

law, nEx [Xn − x]2 → I−1 (x), or σ 2 (x) = I−1 (x), where I(x) is the Fisher information∗ ). This classification list clearly indicates the importance of these notions of convergence and the corresponding limit theorems∗ . (See the annotations in the references at the end of this article.) Finally, for completeness, we present some statistical results connected with discrete processes that arise as estimates. We will be sketchy for the simple reason that more general results appear elsewhere. Having found the estimate Xn , let Xn+1 be determined recursively by Xn+1 = g(Xn , ξn+1 ), where ξn+1 is the new observation. The first result is basic in the theory of recursive estimation. (i) Let {ξn , n
1} be independent observations from a population with a common density f (ξ , x0 ); x0 , ξ ∈ R1 . Let m(x) and D(x) be the common mean and variance with x0 = x. If D(x0 ) < ∞ and m(x) is strictly increasing such that |m(x)| increases at most linearly as |x| → ∞, then the recursive estimates defined successively by Xn+1 − Xn = an (ξn+1 − m(xn )), X0 ≡ constant,   with an = ∞, a2n < ∞, give a strongly consistent estimate of x0 . Assume furthermore that m (x0 ) > 0 and 2am (x0 ) > 1. Then the recursive procedure

of recursive estimates or filtering (basic filtering equations are by-products of this theorem); see Liptser and Shiryayev [12]. If (θ , X) = ([θ1 , . . . , θk ], [X1 , . . . , Xm ]) is a Gaussian vector with mθ = Eθ , mX = EX, Dθ = cov(θ , θ ), Dθ X = cov(θ , X), DX = cov(X, X), and if D+ X denotes the pseudo-inverse matrix of DX , then the conditional expectation E[θ |X] and the conditional covariance cov(θ , θ |X) are given by E[θ |X] = mθ + Dθ X D+ X (X − mX ), ∗ cov(θ , θ |X) = Dθ − Dθ X D+ X Dθ X ,

respectively. In particular, let the RVs θ , X1 , . . . , Xm together form a Gaussian vector such that X1 , . . . , Xm are independent and var(Xk ) > 0. Then E[θ |X1 , . . . , Xm ] = Eθ +

m  cov(θ , Xi ) i=1

var(Xi )

An (y) = A(y1 , . . . , yn ),

Xn+1 − Xn = (ξn+1 − m(Xn ))/(n + 1)

Bn (y) = B(y1 , . . . , yn ),

dX(t) =

 − m (x0 ) X(t)dt + D(x0 )dβ(t),

1 2

where β(t) is a standard Brownian motion∗ . (ii) The following theorem on normal correlation is fundamental in the study

(Xi − EXi ).

In example 2 presented above, let the additive noises ξn [occurring with the signal θ ∈ N(m, σ 2 )] be independent unit normal RVs. Then it follows from (ii) that xn = (m + ni=1 yi )/(1 + σ 2 n) and the tracking error en = σ 2 /(1 + σ 2 n). To consider the case of yn defined in terms of the controlling functions A and B, put

X0 = constant,

is asymptotically normal, and for any k > 0, ti > 0, 1  i  k, and n < n1 < · · · < nk such that log[ni /n]√→ ti as n → ∞, the joint dis√ n(Xn − x0 ), n1 (Xn1 − x0 ), . . . , tribution of √ nk (Xnk − x0 ) converges to the finite-dimensional distribution of X(0), X(t1 ), . . . , X(tk ) corresponding to a diffusion process∗ X(t) satisfying the Ito equation

6517

xn = xn+1 − xn ,

x0 = m,

σn = σn+1 − σn ,

σ0 = σ 2 ,

D = 1/(1 + σn A2n (y)). Applying the theorem on normal correlation, we get σn = −DA2n (y)σn2 , xn = Dσn An (y)(yn+1 − Bn (y) − An (y)xn ). For further details, see Liptser and Shiryayev [12, Chap. 13]. (iii) The following result also plays a central role in obtaining a maximum likelihood estimate∗ . Let Y(t), t = 0, ±1, . . ., be a

6518

PROCESSES, DISCRETE

wide-sense stationary process with spectral representation
Y(t) =

π −π

where k(t + 1) = k(t) +

eiut [Pn−1 (eiu )/Qn (Eiu )]dZ(u),

where dZ(u) is an orthogonal random 2 measure with = 0, E|dZ| ), n−1E(dZ) n = du/(2π k Pn−1 (s) = k=0 bk s , Qn (s) = k=1 ak sk , an = 1, ak , bk ∈ R1 , and with the roots of Qn (s) = 0 in the unit circle. Then Y(t) = Y1 (t), where (Y1 (t), . . . , Yn (t)) is an n-dimensional widesense stationary process satisfying the recurrent equations,

9 2 Also, EY12 (t) = 12 5 , EY1 (t)Y2 (t) = − 15 , EY2 (t) 7 12 2 = 5 , E[X(t) − θ ] = 5 , and E[(X(t) − θ )Y2 (t)] 9 . Now the theorem on normal correla= − 15 tion yields

mθ (0) = (θ − X(0))/4,

t−1  i=0

j = 1, . . . , n − 1; Yn (t + 1) = −

n−1 

+ aj Yj+1 (t) + Bn V(t + 1),

j=0




with V(t) = E[Yi (γ )V(t)] = 0,

γ < t,

π

Bj = bn−1 −



3 + 4k(i) X(0) − 2(1 + k(i))

t−1  t−1  

−

i=0 j=i+1

3 + 4k(j) 2(1 + k(j))





X(i) 1 + 2k(i) + X(i + 1) , 2 2(1 + k(i))   t−1  3 + 4k(i) − g(t) = 4−1 2(k + k(i)) ×

eiu(t−1) Z(du),

−π

i = 1, . . . , n;

i=0

B1 = bn−1 , j−1 

k(0) = 54 .

Hence mθ (t) = f (t, x) + g(t)θ , where f (t, x) = 4−1

Yj (t + 1) = Yj+1 (t) + Bj V(t + 1),

1 (1 + 2k(t))2 − . 4 4(1 + k(t))

−

t−1  t−1   i=0 j=i+1

Bi an−j+1 ,

j=1

j = 2, . . . , n.

−

 3 + 4k(j) . 2(1 + k(i))

For t
1, set  {1 + g(i + 1)}2 5 + . 12 1 + k(i − 1) t

As an application, consider X(t) = θ + Y(t), where −∞ < θ < ∞ is an unknown parameter and Y(t) is stationary Gaussian with EY(t) = 0 and spectral density (or power spectrum) f (u) = (eiu + 1)/(e2iu + eiu + 12 ). By the result above, Y(t) = Y1 (t), where

Dt =

i=1

The maximum likelihood estimate θˆ (t) is obtained by maximizing the right side of the Radon-Nikodym derivative dµθX

Y1 (t + 1) = Y2 (t) + V(t + 1), Y2 (t + 1) = − 21 Y1 (t) − Y2 (t) + 12 V(t + 1).

dµ0X

 = exp

+

If mθ (t) = E[Y2 (t)|At (X)], then mθ (t) solves mθ (t + 1) =

5X(0)θ 25θ 2 − 12 248 t 

 [F(i, x)θ − G(i)θ ] , 2

i=1

where

θ − X(t) θ m (t) 2 1 + 2k(t) + 2(1 + k(t))

F(i, x) =

×(X(t + 1) − θ − m (t)), θ

G(i) =

{1 + g(i − 1)}{X(i) − f (i − 1, x)} , 1 + k(i − 1) {1 + g(i − 1)}2 . 2{1 + k(i − 1)}

PROCRUSTES TRANSFORMATION

Thus  θˆ (t) = D−1 t

 t 5X(0)  F(i, x) . + 12 i=1

The books by Cox and Miller [5], Feller [8], Kannan [10], and Karlin and Taylor [11] are of introductory nature and include basic properties and various applications of different types of discrete processes. The first part of Chung [4] is devoted to the theory of discrete Markov chains. For an extensive treatment of theory (and potential theory) of random walks consult Spitzer [16], and for applications, see Barber and Ninham [2]. Even though published in 1953, Doob [7] is a very good advanced level treatise on various types of stochastic processes. Lo`eve [13] and Petrov [15] extensively study various types of limit theorems that are particularly useful in obtaining different types of estimates. Convergence in law is the topic to which Billingsley [3] is devoted; here one can find applications to exchangeable RVs. For detailed statistical study of discrete processes, consult Albert and Gardner [1], Ibragimov and Rozanov [9], and Liptser and Shiryayev [12].

6519

9. Ibragimov, I. A. and Rozanov, Y. A. (1978). Gaussian Random Processes. SpringerVerlag, New York. 10. Kannan, D. (1979). An Introduction to Stochastic Processes. North-Holland, New York. 11. Karlin, S. and Taylor, H. M. (1975). A First Course in Stochastic Processes, 2nd ed. Academic Press, New York. 12. Liptser, R. S. and Shiryayev, A. N. (1977/1978). Statistics of Random Processes, Vols. I and II. Springer-Verlag, New York. 13. Lo`eve, M. (1977/1978). Probability Theory, Vols. I and II, 4th ed. Springer-Verlag, New York. 14. Norman, F. (1972). Markov Process and Learning Model. Academic Press, New York. 15. Petrov, V. V. (1975). Sums of Independent Random Variables. Springer-Verlag, New York. 16. Spitzer, F. (1964). Principles of Random Walk. Van Nostrand, Reinhold, New York. See also DIFFUSION PROCESSES; GAUSSIAN PROCESSES; MARKOV PROCESSES; RANDOM WALKS; and STOCHASTIC PROCESSES.

D. KANNAN

PROCRUSTES TRANSFORMATION REFERENCES 1. Albert, A. E. and Gardner, L. A. (1967). Stochastic Approximation and Nonlinear Regression. MIT Press, Cambridge, Mass. 2. Barber, M. N. and Ninham, B. W. (1970). Random and Restricted Walks. Gordon and Breach, New York. 3. Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York. 4. Chung, K. L. (1967). Markov Chains with Stationary Transition Probabilities, 2nd ed. Springer-Verlag, New York. 5. Cox, D. R. and Miller, H. D. (1970). The Theory of Stochastic Processes. Methuen, London. 6. Cypkin, Ja. Z. (1973). Foundations of the Theory of Learning Systems. Academic Press, New York. 7. Doob, J. L. (1953). Stochastic Processes. Wiley, New York. 8. Feller, W. (1971). Introduction to Probability Theory and Its Applications, Vol. I, 3rd ed. Wiley, New York (Vol. II, 2nd ed., 1966).

A procedure used in factor analysis∗ for transforming a matrix to obtain a matrix approximately of prespecified form, in particular to transform factors so that the factor loading analysis approximates to some desired form (a ‘‘target’’ matrix). Let F denote the r × r factor loading matrix obtained from data and H denote a target matrix, also of order r × r. We seek an s × s ˜ = FS ‘‘best’’ matches H. matrix S such that H ‘‘Best’’ is conventionally defined as meaning ‘‘minimizes the sum of squares of differences between corresponding elements of H and ˜ The required S is H.’’ S = (FF )−1 F H. It represents a regression of each column of H on the corresponding column of F. Cliff [2] restricts transforming matrices (S) to be orthogonal∗ . References 1 and 3 to 6 give further details on construction and generalization; see also ref. 7.

6520

PRODUCER PRICE INDEXES

REFERENCES 1. Browne, M. W. (1967). Psychometrika, 32, 125–132. 2. Cliff, N. (1966). Psychometrika, 31, 33–42. 3. Gower, J. C. (1975). Psychometrika, 40, 33–51. 4. Hurley, J. L. and Cattell, R. B. (1962). Behav. Sci., 7, 258–262. 5. Kristof, W. and Wingersky, B. (1971). Proc. 79th Annu. Conv. Amer. Psychol. Ass., pp. 81–90. 6. Sch¨onemann, P. H. (1966). Psychometrika, 31, 1–10. 7. Sch¨onemann, P. H. and Carroll, R. H. (1970). Psychometrika, 35, 245–256. See also FACTOR ANALYSIS.

PRODUCER PRICE INDEXES The Producer Price Indexes (PPI) are calculated and released by the Bureau of Labor Statistics∗ (BLS). They began publication in 1902 as the Wholesale Price Indexes. They measure the change in prices received by producers for their goods and services. In 1980, BLS published the first installment of these indexes based on a comprehensively revised basis. The January 1986 indexes reflect the final major installment of that revision. The following discussion focuses on the indexes from the revised survey, but the old indexes will be referenced as appropriate (see also INDEX NUMBERS). UNIVERSE AND CLASSIFICATION Historically, these indexes have measured price changes for selected products of agriculture, manufacturing, and mining. The revision systematically measures the changes in prices received for all products sold by establishments classified in mining or manufacturing according to the Standard Industrial Classification (SIC). The basic building block of the survey is the four-digit industry. Generally following the Census of Manufactures product codes, products of each industry are coded to a seven- or nine-digit level. (The old indexes use an eight-digit product coding scheme that is unique to the PPI.)

There are three families of indexes produced from the same basic data base. The first are the output price indexes for each four-digit industry (493 in mining and manufacturing). These indexes reflect the price change of the output which leaves the particular industry. Sales among establishments within the industry are not included. This so-called net-output approach can also be applied to aggregations of these four-digit industries. The stage-of-processing price indexes are the most useful summary-level indexes for macroeconomic analysis of inflation. In the old indexes they are calculated for crude, intermediate, and finished goods with appropriate subindexes for specific product areas (food, energy) or end markets. Each product is classified by its degree of fabrication and type of buyers and may be included in more than one stage-of-processing index. The new stage-of-processing indexes will be constructed from aggregates of industries, each industry being classified in total based on the pattern of its purchases and sales as a producer of crude materials, primary manufactures, intermediate manufactures, or finished goods. The third set consists of price indexes for detailed products. In 1986, there are about 6,000 such detailed indexes published each month. THE INDEXES Producer Price Indexes are calculated using a modified Laspeyres index of the general form   P1,i Q0,i × 100 I1 =  i i P0,i Q0,i where I1 = index for period 1 P1,i , P0,i = prices for item i in periods 1 and 0 respectively Q0,i = quantity of item i sold in period 0 (the base period) These indexes may be viewed as measuring the price change of a constant set of production through time.

PRODUCER PRICE INDEXES

PRICES In order to price the output of a production unit, BLS requests the price which the producer actually received at the time of shipment for the item being priced—the transaction price. The desired price will reflect all applicable discounts and all applicable extras and surcharges. The price is f.o.b. the seller’s freight dock and excludes all direct excise taxes and transportation charges. They seek to price all the different types of transactions and types of customers relevant to a particular item. All the conditions of the sale must be identified and held constant through time. When an item is no longer being sold, it is necessary to replace it in the index. This must be done in such a way that only pure price change is captured by the index, and it remains as close as possible to the concept of pricing a constant set of production through time. This substitution procedure is generally referred to as ‘‘quality adjustment.’’ It consists of two steps: (1) identifying all the changes in the specification of the item being priced, and (2) placing a value on each change. It is the second step which presents the major difficulty. Traditionally, it has been the objective to value specification changes as the sum of the producer’s variable costs, fixed costs, and standard markup associated with the item. Increasingly, there are cases in which the producer cost methodology cannot be applied (e.g., when there is product improvement at reduced cost). Computers, electronic calculators, and other high-technology goods are the prime examples. A technique called hedonic∗ regression, which has been a research tool for some time, is being investigated for some of these cases. This approach views a particular good as composed of a large number of independent characteristics which are bought in bundles. For example, a computer is composed of certain cycle times, add times, memory size, and so on. A regression equation (in any number of possible functional forms) is estimated with the price of the computer as the dependent variable and the characteristics as the independent variables. The coefficients (or transformations of them) are

6521

then implicit prices for the characteristics. The coefficient for memory size may set an implicit value of $100 for each ‘‘K’’ of memory. A new computer with 4K more memory than an old one would thus have a quality increase of $400. Although its application has been somewhat limited, the hedonic approach has yielded encouraging results. Three major limitations are associated with its application. First, all price-determining characteristics must be specified, which makes it a costly, time-consuming process. Second, it is of no assistance when an entirely new characteristic appears (e.g., a smog-control device on a car). Third, it is necessary to assume that the value of a characteristic is independent of the bundle in which it occurs. This assumption is believed to be approximately true for most cases, but requires evaluation.

USES Producer Price Indexes are widely used by the general public as indicators of inflation. They are, in conjunction with the Consumer Price Index∗ , the unemployment rate, and the gross national product∗ , the chief statistical inputs to national fiscal and monetary policy. They are also heavily relied on by the business and research communities for economic and market analyses and projections. Most major measures of real economic activity (GNP, orders, shipments, inventories, productivity) are derived by using appropriate Consumer and Producer Price Indexes to deflate current dollar aggregates. Finally, increasing numbers of contract sales are using price escalation based on one or more Producer Price Indexes. In 1976, more than $100 billion in contracts had such escalators.

SAMPLING The universe of interest is the set of all transactions originating in mining and manufacturing establishments. Each transaction is characterized by the industry of the seller and the product sold. It can be portrayed as follows:

6522

PRODUCT INTEGRATION

Industry/Product 1 2 3 . . . 10,000 1 2 .. . 493 Each four-digit SIC industry (row in the table above) is sampled and estimated independently. A multistage probabilityproportional-to-size∗ sample of transactions is selected for pricing through time. In the first stage, an average of 74 firms in the industry are selected from the unemployment insurance file of all employers in the country. These firms are visited by BLS economists, who use a multistage sampling technique called disaggregation to select an average of five specific items for pricing along with a particular type of transaction if any are sold in more than one way at different prices. Output price indexes can be constructed for each industry with data from the appropriate row in the matrix above. Product indexes (wherever made) can be constructed from data in the appropriate column. Weights of major product groups and for industries are from the Census of Manufacturers. Within major product groups, weights are developed from the sampling process. Net output weights are developed from the input–output tables of the Bureau of Economic Analysis. ERRORS Because the old indexes were not based on probability samples, variances have not been available. Variances are now under development for the revised indexes. A total error profile will also be developed for the program covering such factors of errors in the sampling frame∗ , nonresponse∗ bias, and processing errors. Rigorous error control procedures are being installed at all critical stages in the process. The difficulties associated with quality adjustment may contribute a major portion of the total error in price indexes. This kind of error is unique to estimating price indexes. As a result, the existing literature on errors in surveys is largely not applicable. This topic has received considerable attention from both BLS and academic researchers, but a generally acceptable solution has been elusive.

BIBLIOGRAPHY Archibald, R. B. (1977). Ann. Econ. Social Meas., Winter, 57–72. [Extends Fisher and Schell (below). The direct theoretical basis for the PPI.] Bureau of Labor Statistics (1982). BLS Handbook of Methods, Bulletin 2134, Vol. 1. Bureau of Labor Statistics, Washington, D.C., pp. 43–61. (Official, general description of PPI. Updated editions issued periodically.) Early, J. F. (1978). Monthly Labor Rev., Apr., 7–15. (General audience description of the theory and methods for the revision.) Early, J. F. and Sinclair, J. (1981). In The U.S. National Income and Product Accounts: Selected Topics, M. Foss, ed. Chicago. (Extensive treatment of actual quality adjustment in the PPI. Illustrative example of hedonic approach.) Fisher, F. M. and Schell, K. (1972). The Economic Theory of Price Indexes. New York. (Rigorous mathematical economic theory underlying modern price indexes.) Triplett, J. E. (1971). The Theory of Hedonic Quality Measurement and Its Use in Price Indexes. Bureau of Labor Statistics, Washington, D.C. (A little dated, but the best introduction to the topic.) Triplett, J. E. (1975). In P. H. Earl, Analysis of Inflation. Lexington Books, Lexington, Mass. (Basic, moderately technical discussion of the problems associated with quality adjustment.) See also CONSUMER PRICE INDEX; GROSS NATIONAL PRODUCT DEFLATOR; HEDONIC INDEX NUMBERS; INDEX NUMBERS; INDEX OF INDUSTRIAL PRODUCTION; LABOR STATISTICS; and PAASCHE –LASPEYRES INDEX.

JOHN F. EARLY

PRODUCT INTEGRATION All statisticians arefamiliar  with the sum and product signs and , and with the  integral sign . Also they are aware that there is a certain analogy between summation and integration; indeed the integral sign is in origin a stretched-out s for ‘‘summation’’. Strange therefore that not many people are aware of the existence of the product integral , invented by the Italian mathematician Vito Volterra in 1887, which bears exactly the same relation to the ordinary product as the integral does to summation.

π

PRODUCT INTEGRATION

The mathematical theory of productintegration is not terribly difficult and not terribly deep, which is perhaps one of the reasons it was out of fashion by the time survival analysis∗ came into being in the 1950s. However, it is terribly useful, and it is a pity that E. L. Kaplan and P. Meier [10], the inventors (1956) of the product-limit or Kaplan–Meier estimator∗ (the nonparametric maximum likelihood estimator∗ of an unknown distribution function based on a sample of censored survival times), did not make the connection, and neither did the authors of the classic 1967 and 1974 papers on this estimator, respectively by B. Efron [5] and N. Breslow and J. Crowley [3]. Only with the 1978 paper of O. O. Aalen and S. Johansen [1] was the connection between the Kaplan–Meier estimator and product integration made explicit. It took several more years before the connection was put to use to derive new largesample properties of the Kaplan–Meier estimator (e.g., the asymptotic normality of the Kaplan–Meier mean) with the 1983 paper of R. D. Gill [8]. Modern treatments of the theory of product integration with a view toward statistical applications can be found in refs. 2, 6, 9. The Kaplan–Meier estimator is the product-integral of the Nelson–Aalen estimator∗ (see COUNTING PROCESSES) of the cumulative or integrated hazard function∗ ; these two estimators bear the same relation to one another as the actual survival function∗ (one minus the distribution function) and the actual cumulative hazard function. There are many other applications of product integration in statistics, for instance in the study of multistate processes (connected with the theory of Markov processes∗ ) [1] and in the theory of partial likelihood∗ (see PROPORTIONAL HAZARDS MODEL, COX’S); see ref. [2]. Product integration also turns up in extremevalue∗ theory, where again the hazard rate plays an important role, and in stochastic analysis and martingale∗ theory (stochastic integration∗ ), where it turns up under the name Dol´eans–Dades exponential martingale. Properties of integrals are often easily guessed by thinking of them as sums of many, many (usually very small) terms. Similarly,

6523

product integrals generalize the taking of products, making their properties easy to guess and to understand. Let us define product integration at a slightly higher level of generality than Volterra’s original definition (corresponding to the transition from Lebesgue to Lebesgue–Stieltjes integration). Suppose X (t) is a p × p matrix-valued function of time t ∈ [0, ∞). Suppose also X (or each component of X) is right-continuous with left-hand limits. Let 1 denote the identity matrix. The product integral of X over the interval (0, t] is now defined as t

π [1 + dX(s)] 0

=

lim

max |ti −ti−1 |→0

 {1 + [X(ti ) − X(ti−1 )]},

where the limit is taken over a sequence of ever finer partitions 0 = t0 < t1 < . . . < tk = t of the time interval [0, t]. For the limit to exist, X has to be of bounded variation; equivalently, each component of X is the difference of two increasing functions. A very obvious property of product integration is multiplicativity. Defining the product integral over an arbitrary time interval in the natural way, we have for 0 < s < t t

s

t

0

0

s

π (1 + dX) = π (1 + dX) π (1 + dX). We can guess (for proofs, see ref. [6] or preferably [9]) many other useful properties by looking at various simple identities for finite products. For instance, in deriving asymptotic statistical theory it is often important to study the difference between two productintegrals. If a1 , . . . , ak and b1 , . . . , bk are two sequences of numbers, we have the identity  (1 + ai ) − (1 + bi )   (1 + ai )(aj − bj ) (1 + bi ). =



j

i<j

i>j

This can be easily proved by replacing the middle factor on the right-hand side of the equation, (aj − bj ), by (1 + aj ) − (1 + bj ).

6524

PRODUCT INTEGRATION

Expanding about this difference, the righthand side becomes      (1 + ai ) (1 + bi ) j

ij



−

i>j



(1 + ai )

ij−1

 (1 + bi ) .

i>j−1

This is a telescoping sum; writing out the terms one by one, the whole expression collapses to the two outside products, giving the left-hand side of the identity. The same manipulations work for matrices. It is therefore no surprise, replacing sums by integrals and products by product-integrals, that t

t

π (1 + dX) − π (1 + dY) 0


=

0

t

hazard function A corresponding to S with the Nelson–Aalen estimator ˆ = A(t)

 dj tj t

rj

.

This is a discrete cumulative hazard function, corresponding to the discrete estimated ˆ = 0 for t not an hazard α(t ˆ j ) = dj /rj , with α(t) observed death time. The product integral of ˆ is then A   dj ˆ = ˆS(t) = t (1 − dA) 1− , 0 rj t t j

which is just the Kaplan–Meier estimator of the true survival function S. The Duhamel equation now becomes the identity
t S(t) ˆ ˆ ˆ − S(t) = , S(s−)[d A(s) − dA(s)] S(t) S(s) s=0

s−

π (1 + dX)[dX(s) − dY(s)]

s=0 0

t

×

π (1 + dY). s+

This valuable identity is the Duhamel equation (the name refers to a classical identity for the derivative with respect to a parameter of the solution of a differential equation). As an example, consider the scalar case (p = 1); let A be a cumulative hazard function ˆ the Nelson–Aalen estimator based on and A a sample of censored survival times. Here we have the statistical problem of estimating the survival curve S(t) = Pr{T
t} given a sample of independently censored i.i.d. survival times T1 , . . . , Tn . The cumulative hazard rate A(t) is defined by


t

A(t) = 0

dS(t) ; S(t−)

which can be exploited to get both smallsample and asymptotic results [2,6,7,8,9]. The same identity pays off in studying Dabrowska’s multivariate product-limit estimator [4,9,11], and in studying Aalen and Johansen’s estimator of the transition matrix of an inhomogeneous Markov chain [1,2]. It can be rewritten [9] as a so-called van der Laan identity [11], expressing Sˆ − S as a function-indexed empirical process (see EMPIRICAL PROCESSES), evaluated at a random argument, so that the classical largesample results for Kaplan–Meier (consistency and asymptotic normality∗ ) follow from a two-line proof: without further calculations, simply invoke the modern forms of the Glivenko–Cantelli theorem∗ and the Donsker theorem, i.e., the functional versions of the classical law of large numbers∗ and the central limit theorem∗ , respectively. Taking Y identically equal to zero in the Duhamel equation yields the formula t

A is just the integrated hazard rate in the absolutely continuous case, the cumulative sum of discrete hazards in the discrete case. Let t1 < t2 < . . . denote the distinct times when deaths are observed; let rj denote the number of individuals at risk just before time tj , and let dj denote the number of observed deaths at time tj . We estimate the cumulative

π (1 + dX) − 1 =




0

t

s−

π (1 + dX)dX(s).

s=0 0

This is the integral version of Kolmogorov’s forward differential equation from the theory of Markov processes, of the type


t

Y(t) = 1 +

Y(s−)dX(s) 0

PRODUCTION SCHEDULING

(in unknown Y, given X), that motivated Volterra to invent product integration. Y(t) = t

π (1 + dX) 0

is the unique solution of this

equation.

7.

REFERENCES 1. Aalen, O.O. and Johansen, S. (1978). An empirical transition matrix for nonhomogenous Markov chains based on censored observations. Scand. J. Statist., 5, 141–150. (Introduced counting-process theory and product integration, simultaneously, into the study of nonparametric estimation for Markov processes; the relevance to the Kaplan–Meier estimator was noted by the authors, but not noticed by the world.) 2. Andersen, P. K., Borgan, Ø., Gill, R. D., and Keiding, N. (1993). Statistical Models Based on Counting Processes. Springer-Verlag, New York. (Contains a ‘‘users’ guide’’ to product integration in the context of counting processes and generalized survival analysis.)

8.

9.

10.

3. Breslow, N. and Crowley, J. (1974). A large sample study of the life table and product limit estimates under random censorship. Ann. Statist., 2, 437–453. (First rigorous largesample results for Kaplan–Meier estimation, using the then recently developed Billingsleystyle theory of weak convergence.) 4. Dabrowska, D. (1988). Kaplan–Meier estimate on the plane. Ann. Statist., 16, 1475–1489. (Beautiful generalization of the product-limit characterization of the Kaplan–Meier estimator to higher dimensions. Other characterizations, e.g., nonparametric maximum likelihood, lead to other estimators; see refs. 9, 11.) 5. Efron, B. (1967). The two sample problem with censored data. Proc. 5th Berkeley Symp. Math. Statist. Probab., L. Le Cam and J. Neyman, eds. University of California Press, Berkeley, Calif., pp. 831–853. (This classic introduced the redistribute-to-the-right and selfconsistency properties of the Kaplan–Meier estimator and claimed, but did not prove, weak convergence of this estimator on the whole line in order to establish asymptotic normality of a new Wilcoxon generalization—results established in [7].) 6. Gill, R. D. and Johansen, S. (1990). A survey of product-integration with a view towards application in survival analysis. Ann. Statist.,

11.

6525

18, 1501–1555. (The basic theory, some history, and miscellaneous applications. Reference [9] contains some improvements and further applications.) Gill, R. D. (1980). Censoring and Stochastic Integrals, MC Tract 124. Centre for Mathematics and Computer Science (CWI), Amsterdam. (The author emphasized the counting process approach to survival analysis, using some product limit theory from [1] but not highlighting this part of the theory.) Gill, R. D. (1983). Large sample behaviour of the product limit estimator on the whole line. Ann. Statist., 11, 49–58. (Cf. comments on [7].) Gill, R. D. (1994). Lectures on survival analysis. In Lectures on Probability Theory ´ e de Probabilit´es de Saint Flour (Ecole d’Et´ XXII—1992), SLNM 1581, D. Bakry, R. D. Gill, and S. A. Molchanov, eds. SpringerVerlag, Berlin, pp. 115–241. (Perhaps cryptically brief in parts, but a yet more polished treatment of product integration and its applications in survival analysis.) Kaplan, E. L. and Meier, P. (1958). Nonparametric estimation from incomplete observations. J. Amer. Statist. Ass., 53, 457–481, 562–563. [This classic is second on a list of most cited ever papers in mathematics, statistics, and computer science. The authors never met, but submitted their independent inventions of the product-limit estimator simultaneously to J. Amer. Statist. Ass.; the resulting joint paper was the product of postal collaboration.] van der Laan (1996). Efficient and Inefficient Estimation in Semiparametric Models, CWI Tract 114. Centre for Mathematics and Computer Science, Amsterdam. (Contains a beautiful identity for the nonparametric maximum likelihood estimator in a missing-data problem: estimator minus estimand equals the empirical process of the optimal influence curve  evaluated atˆ the estimator, Fˆ − F = (1/n) ni=1 ICopt (Xi /F). Applications to the bivariate censored-data∗ problem are presented, as well as treatment of other estimators for the same problem.)

See also CENSORED DATA; COUNTING PROCESSES; EMPIRICAL PROCESSES; KAPLAN-MEIER ESTIMATOR —II; PROPORTIONAL HAZARDS MODEL, COX’S; and SURVIVAL ANALYSIS.

RICHARD GILL

PRODUCTION SCHEDULING. See LINEAR PROGRAMMING

6526

PRODUCTIVITY MEASUREMENT

PRODUCTIVITY MEASUREMENT Productivity measurement is the estimation of a broad class of ratios of outputs to one input or a combination of associated inputs in particular industries or the entire economy. The aggregate output Q is associated with n inputs (factors) vi , for i = 1, 2, . . . , n, of production often aggregated into labor (L) and capital (K). The ratios, such as Q/L and Q/K, are partial  factor productivity indices. The ratio Q/ ni=1 αi vi , where the αi are weights, is referred to as the multifactor or total factor productivity (TFP) index. In comparisons over time, the partial index is a measure of the rate of change in total output per unit of a particular input, without separating out the contributions of other factors to the production process. It fails to account for the effects of technological advances, substitution between inputs, scale economies, and changes in input quality and product composition. In contrast, the TFP index is the rate of change in ‘‘productive efficiency,’’ or more accurately, in the ratio of total output to a weighted combination of all inputs. The value of the TFP index must be equal to 1 if all inputs, tangible or intangible, and the specification of their combination are correctly accounted for. If the TFP index exceeds unity, as it usually does in the case of industrial economies, it is because some inputs are left out or because the relationship between output and inputs (production function) is incompletely specified. For example, if Q is assumed to be produced by L and K only, and shows an increase of 4% between two points in time, it may turn out that 25% of this increase is explained by increases in the quantities of L and K, and the remaining 75% of the 4%, which is the value of the TFP index, is due to left-out inputs or misspecification. The identification of the misspecification, the left-out inputs, and the decomposition of the TFP index between them is a complex and as yet controversial problem in productivity measurement. The left-out variables have been interpreted to be among the sources and causes of economic growth and have frequently been identified with such factors as the various forms of technological progress, scale economies, research and development, and changes in the nature and quality of

inputs, institutions, and availability of relatively cheap inputs, especially energy. PARTIAL AND TOTAL FACTOR PRODUCTIVITY INDEXES All productivity measurements before 1942 were of the simple partial factor productivity type, in particular, output per worker, or per hour worked (Q / L). They continue to be published by the statistical agencies of many countries (e.g., ref. 40) and in spite of their evident theoretical limitations, widely used. The reasons are to be found in the general availability of the relevant statistical information and the simplicity of the computational requirements. Furthermore, in interindustry, intercountry, or intertemporal comparisons the average product of labor (output per worker) is an indirect measure of the differences in capital and other resources available for a unit of labor to work with. Thus the partial factor productivity ratios are indicators, however primitive, of differences in technological and economic states, even though the causes of these differences remain unrevealed by them. It is primarily for this latter reason that the main interest in recent years has been in the TFP index.1 In 1942, Tinbergen [37] was the first to estimate TFP indexes by introducing a trend function of the form A(t) = A(0)eλt into a loglinear regression of Q on L and K, where A(t)/A(0) was the TFP index in period t relative to period 0. After the mid-1950s several other investigators independently estimated TFP indexes. In particular, in two seminal contributions Solow [32] and Kendrick [18] provided the foundations of two interrelated but conceptually different approaches—one based on the economic concept of the production function and the other on the statistical concept of index numbers∗ . These are discussed in the following two sections, while the analysis and decomposition of the TFP index are discussed in the final section of this article. The literature on the subject is large. Recent developments in productivity measurements can be found inter alia in refs. 5, 20, 21, 25, and 27. Since 1958, the U.S. Bureau of Labor Statistics∗ has published five bibliographies on productivity; the

PRODUCTIVITY MEASUREMENT

latest [39] covers 1976–1978 and contains 1200 publications. Much of this literature is related to the economic theories of capital, production and returns to productive factors (see, e.g., refs. 2, 4, 13, 29, 31, 34, 35, 38, and 42).

6527

has increased over time (see e.g., refs. 6, 9, and 32). To account for this phenomenon, assume (as in ref. 32) that the production function shifts outward as function of time. Then the function can be written as Q(t) = F[L(t), K(t), t],

THE PRODUCTION FUNCTION APPROACH

(3)

In the neoclassical economic theory of production the entire output is explained by the interaction of homogeneous and constant quality productive factors (inputs) as specified by a constant returns to scale (i.e., first-order homogeneous and additive) production function. In a typical two-input case the latter can be written as

where t designates time and the amounts of the output as well as the constant quality inputs are functions of time. As shown in ref. 26, the production function can take various explicit forms. However, if F[·] is subject to constant returns to scale as in (1), the variable t can be factored out from F and (3) can be restated as

Q = f (K, L),

Q(t) = A(t)f [L(t), K(t)],

(1)

where Q, K, and L represent output, capital, and labor, respectively. If changes in the quantities of these variables are considered in time, the percentage rate of growth of the output, as derived from (1), can be written as ˙ ˙ K L˙ Q = EK + EL , Q K L

(2)

where the dotted variables represent time derivatives, and EK and EL are defined as ∂Q K ∂K f

and

∂Q L . ∂L f

Because of the assumption of constant returns to scale, EK and EL sum to 1 (Euler’s identity), and the percentage rate of growth of the output, as shown in (2), is the weighted (linear) combination of the percentage rates of growth of the inputs. Hence the growth rate of the output must lie between, or be equal to, the growth rates of the two inputs. The empirical evidence is, however, different. In the United States, and in the advanced industrialized countries in general, the growth of labor in the long run has been exceeded by the growth of capital and output, the last two having been approximately equal to each other. In terms of partial productivity indicators, Q/L has grown over time, while Q/K has remained approximately constant. Concomitantly, the capital-labor ratio

(4)

where A(t) is a shift factor which determines the cumulative outward shift in the production function itself. A(t) is neutral with respect to, and independent of, the amounts of L and K utilized in the production process. Hence it can be arbitrarily designated as a measure of technological change of a type which is neutral with respect to input combinations and not embodied in (or independent of) the qualities of the inputs themselves. Accordingly, it has been referred to as ‘‘neutral disembodied technological change.’’ Analogously to (2), the percentage rate of change of the output over time can be obtained from (4) as ˙ ˙ ˙ K L˙ A Q = + EK + EL Q A K L and   ˙ ˙ ˙ K L˙ Q A = − EK + EL , A Q K L

(5)

˙ where A/A represents the percentage rate of growth of the cumulative shift factor, or the difference between the observed growth rate of the output and that part of its growth which is imputable to the rate of growth of the amounts of factors used in production. As ˙ a consequence, A/A can also be thought of as a residual.

6528

PRODUCTIVITY MEASUREMENT

The estimation of the shift factor is based on national income and input statistics requiring the restatement of (5) in value terms. Let the prices of Q, L, and K at time t be P, PL , and PK , respectively, and in line with neoclassical purely competitive production theory assume that the prices or returns to factors are determined by the value of their contributions to output (marginal products), so that PL = A(t)(∂f /∂L)P and PK = A(t)(∂f /∂K)P. Given the assumption of competition, PQ = PL L + PK K at any point in time. We obtain ˙ ˙ Q PL L˙ + PK K˙ A . = − L A Q P L + PK K

(6)

In his original 1957 article Solow [32] assumed that A(t) = A(0)eλt , where λ rep˙ resents [A(t)/A(t)]. In calculation he set A(0) = 1 and used the formula

A(t) A(t) = A(t − 1) 1 + A(t − 1) for t = 1, 2, . . . , T, where  is the first difference operator. Based on U.S. time series for 1909–1949 and using a production function of the specific form Q(t) = L(t)α K(t)β A(t),

THE INDEX NUMBER APPROACH The TFP index can be defined as an index of output divided by an index of input, an approach implicit in Mills [24], Abramovitz [1], Fabricant [14], and Kendrick [18,19]. Kendrick’s TFP index can be defined as Q(T)/Q(0) A(T) = , A(0) α[L(T)/L(0)] + β[K(T)/K(0)]

(8)

where α and β are income shares of labor and capital. The difference between Solow’s TFP index and Kendrick’s is that the latter is not explicitly based on a formal production function. However, as pointed out by Domar [11], Kendrick’s index does imply a production function of the form Q(t) = A(t)[αL(t) + βK(t)]. Also [22], under certain economic assumptions and for ‘‘small’’ changes in the quantities of the input and output, Kendrick’s and Solow’s indices are equivalent, and (8) can be rewritten as a Paasche index of ∂Q/∂L and ∂Q/∂K. Diewert [10] has proven that under certain economic assumptions, different index number formulas (e.g., those of Laspeyres, Paasche, and Fisher) are consistent with (‘‘exact’’ for) different functional forms of (3). For example, if (3) takes the form  Q(t) = 

2  2 

1/2 aij Li (t)Kj (t)

(9)

where α and β are parameters, he found that A(1949)/A(1909) = 1.81; that is, the annual rate of technical progress in the United States over that period was 1.5%. In fact, by setting A(0) = 1, A(T) can be identified as the TFP index. Thus if t is expressed in years and A(2) = 1.0404, total productivity increased by 4.04% or at an ˙ annual rate [A(t)/A(t)] of 2%. It is in this sense that the TFP index is sometimes referred to as a measure of ‘‘technical change,’’ one of several alternative descriptions in the literature (see, e.g., ref. 12). Also, as shown by refs. 15 and 36, we obtain  Q(T) X(T) A(T) = , (7) A(0) Q(0) X(0)

where the constants aij = aji , then, as shown in [10], Fisher’s ideal∗ input index is exact for (9). The problem is of course that the functional form of (3) is unknown. Caves et al. [3] proposed the use of the T¨ornqvist index for the measurement of input, output, and TFP indexes. Maddala [23] compared empirically four of Diewert’s exact indices and concluded that the differences between the underlying four functional forms result in ‘‘negligible’’ differences between the corresponding TFP indices.

where X(T)/X(0) is the Divisia input index. See INDEX NUMBERS and DIVISIA INDICES.

It is the output growth not directly attributable to the factors of production

i=1 j=1

ANALYSIS AND DECOMPOSITION OF THE TFP INDEX

PRODUCTIVITY MEASUREMENT

which is measured by the TFP index. It has also been referred to as ‘‘residual.’’ A large part of the literature on productivity measurement is devoted to attempts to decompose this residual into indices of changes of variables which can account for it. Three approaches to the analysis and decomposition of the residual are described briefly below. Growth Accounting Some studies in this vein are Denison [6,7,8,9] and Jorgensen and Griliches [16]. Denison in his implicit framework substituted a set of ‘‘explanatory variables,’’ the traditionally defined inputs in the aggregate production, such as (1). He also dropped the assumption of constant returns to scale. His productivity index can be expressed as A(t) = Q(t)/f (xi ; yj ),

(10)

where xi = x1 , x2 , . . . , xn and yj = y1 , y2 , . . . , ym , are the explanatory variables. Their rates of change over time are Denison’s ‘‘sources of growth.’’ The xi are disaggregated and quality-adjusted labor, capital and land, the contributions of which can be estimated by their respective income shares. The yj are variables which measure scale effects and improvements in resource allocation. The final residual in the growth of output is Denison’s productivity index, which he attributes to ‘‘advances of knowledge.’’ The method in ref. 16 is similar to Denison’s except that the classification of the data and perception of the required quality adjustments are different and Divisia indices are used in the calculations. Capital Embodied Technological Change A different concept of technological change than the one presented in Solow’s 1957 model is found in refs. 21, 28, 33, and 34, among others. Best known among these is the model based on a production function of the form Q(t) = A(t)L (t)J α

1−α

(t),

(11)

where J(t) is the quality-weighted capital defined by J(t) =

n  i=0

Ki (t)(1 + λ)i ,

(12)

6529

Ki (t) is the gross amount of capital produced in year i and in use in year t, and λ is the rate of improvement in the productivity of new machines over those produced in the previous year. In this model, the rate of increase in the TFP index is given by ˙ ˙ ˙ Q(t) L(t) A(t) = −α A(t) Q(t) L(t) 

 ˙ K(t) ˙ + λ − λa , (13) −(1 − α) K(t)

where a is the average age of capital. Maximum likelihood estimation∗ of the parameters of this model is discussed in ref. 41. Best Practice Technique Salter [30] attributes productivity growth to best practice technique which yields minimum production costs. He decomposes the TFP index into advances in technical knowledge, scale economies, and factor substitution. Salter examines proportionate changes in labor and capital, so as to attribute them to both neutral and nonneutral technical progress as well as to substitution between labor and capital. The discussion of Salter’s measures and their application to U.K. and U.S. industries is also contained in ref. 30. NOTE 1. A survey of partial productivity comparisons between countries can be found in [17]. The weighted mean of the ratio of labor hours worked at two points of time is called the man-hour index. The weights are usually the hourly earnings. REFERENCES 1. Abramovitz, M. (1956). Amer. Econ. Rev., 46, 5–23. 2. Arrow, K. J. (1962). Rev. Econ. Stud., 29, 155–173. 3. Caves, D. W., Christensen, L. R., and Diewert, W. E. (1982). Econometrica, 50, 1343–1414. 4. Champernowne, D. G. (1961). In The Theory of Capital, F. A. Lutz, ed. Macmillan, London.

6530

PRODUCTIVITY MEASUREMENT

5. Cowing, T. G. and Stevenson, R. E. (1981). Productivity Measurement in Regulated Industries. Academic Press, New York. (A collection of useful survey and application papers.)

23. Maddala, G. S. (1979). In ref. 27, pp. 309– 317.

6. Denison, F. (1962). The Sources of Economic Growth in the United States and the Alternatives before Us. Committee for Economic Development, New York.

25. Nadiri, M. I. (1970). J. Econ. 1137–1177. (Survey; elementary.)

7. Denison, F. (1964). Amer. Econ. Rev., 54, 90–94. 8. Denison, F. (1967). Amer. Econ. Rev., 57, 325–332. 9. Denison, F. (1967). Why Growth Rates Differ. The Brookings Institution, Washington, D.C.

24. Mills, F. C. (1952). Productivity and Economic Progress. National Bureau of Economic Research, New York. (Elementary.) Lit.,

8,

26. Nadiri, M. I. (1982). In Handbook of Mathematical Economics, Vol. 2. K. J. Arrow and M. D. Intriligator, eds. North-Holland, Amsterdam, pp. 431–490. (Economic theory of production; mathematical.) 27. National Research Council (1979). Measurement and Interpretation of Productivity. National Academy of Sciences, Washington, D.C. (Elementary.)

10. Diewert, W. E. (1976). J. Econometrics, 4, 115–145. (Economic index numbers and their use in productivity measurement; intermediate.)

28. Nelson, R. R. (1964). Amer. Econ. Rev., 54, 575–606.

11. Domar, E. D. (1961). Econ. J., 71, 709–729. (Concepts and measurement of technical progress; elementary.)

30. Salter, W. E. G. (1960). Productivity and Technical Change. Cambridge University Press, Cambridge, England.

12. Domar, E. D. (1962). J. Pol. Econ., 70, 597–608. (Reviews Kendrick’s approach to productivity measurement; elementary.)

31. Samuelson, P. A. (1962). Rev. Econ. Stud., 29, 193–206.

13. Eckaus, R. S. and Lefeber, L. (1962). Amer. Econ. Rev., 44, 113–122. 14. Fabricant, S. (1959). Basic Facts on Productivity Change. National Bureau of Economic Research, New York. (Elementary.) 15. Hillinger, C. (1970). Econometrica, 38, 773–774. (Defines Divisia form of TFP index; intermediate.) 16. Jorgenson, D. and Grilliches, Z. (1967). Rev. Econ. Stud., 34, 249–284. (Concepts, methods and application.) 17. Kravis, I. B. (1976). Econ. J., 86, 1–44. (Survey; elementary.) 18. Kendrick, J. (1961). Productivity Trends in the United States. Princeton University Press, Princeton, N.J. 19. Kendrick, J. (1973). Postwar Productivity Trends in the United States. National Bureau of Economic Research, New York. 20. Kendrick, J. and Vaccara, B. N. eds. (1980). New Developments in Productivity Measurement and Analysis. University of Chicago Press, Chicago. (Collection of survey and application papers; elementary.)

29. Phelps, E. S. (1962). Quart. J. Econ., 76, 548–567.

32. Solow, R. (1957). Rev. Econ. Statist., 39, 312–320. (Reprinted with correction in Readings in Economic Statistics and Econometrics, A. Zellner, ed. Little, Brown, Boston, 1968.) 33. Solow, R. (1959). In Mathematical Methods in the Social Sciences, K. Arrow et al., eds. Stanford University Press, Stanford, Calif., pp. 89–104. 34. Solow, R. (1962). Amer. Econ. Rev., 52, 76–86. 35. Solow, R. (1963). Capital Theory and the Rate of Return. North-Holland, Amsterdam. 36. Star, S. and Hall, R. E. (1976). Econometrica, 44, 257–263. (Divisia indices of productivity; intermediate.) 37. Tinbergen, J. (1942). In L. H. Klassen, et al. Jan Tinbergen: Selected Papers, 1959, pp. 182–221; first appeared in 1942 in German, published by North-Holland, Amsterdam. 38. Tobin, J. (1961). Amer. Econ. Rev., 52, 26–37. 39. U.S. Bureau of Labor Statistics (1980). Productivity: A Selected Annotated Bibliography, 1976–78. U.S. Government Printing Office, Washington, D.C.

21. Kennedy, C. and Thirlwall, A. P. (1972). Econ. J., 82, 11–72. (Survey paper; intermediate.)

40. U.S. Bureau of Labor Statistics (1982). Productivity Measures for Selected Industries, 1954–80. U.S. Government Printing Office, Washington, D.C.

22. Kleiman, E., Halevi, N., and Levhari, D. (1966). Rev. Econ. Statist., 48, 345–347.

41. Wickens, M. R. (1970). Rev. Econ. Statist., 52, 187–193.

PROFILE ANALYSIS

6531

42. Worswick, G. D. N. (1959). Oxf. Econ. Pap., 11, 125–142.

PROFESSIONAL STATISTICIAN, THE .

See also DIVISIA INDICES; FISHER’S IDEAL INDEX NUMBER; INDEX NUMBERS; INDEX OF INDUSTRIAL PRODUCTION; PAASCHE –LASPEYRES INDEX; and PRODUCER PRICE INDEXES.

PROFILE ANALYSIS

NURI T. JAZAIRI LOUIS LEFEBER

PRODUCT-LIMIT

ESTIMATOR. See

KAPLAN –MEIER ESTIMATOR —I

PRODUCT MOMENT The expected value of a product of random variables (X1 , X2 , . . . , Xm ) possibly raised to different powers. The usual notation is   m  rj  µr1 r2 ...rm (X1 , X2 , . . . , Xm ) = E  Xj  . j=1

(For brevity the symbol µr1 r2 ...rm is often used.) The corresponding central product moment is   m  r j µr1 r2 ...rm = E  {Xj − E[Xj ]}  . j=1

To emphasize the distinction between µr1 r2 ...rm and µr1 r2 ...rm the former is sometimes called a crude product moment. Some of the r’s may be zero, but at least two should differ from zero. Another name is mixed moment. Absolute product moments are defined by   m  vr1 r2 ...rm = E  |Xj |rj  ,  vr1 r2 ...rm = E 

j=1 m 

 |Xj − E[Xj ]|

rj 

.

j=1

See also ABSOLUTE MOMENT and COVARIANCE.

PRODUCT, MULTINOMIAL. See LOG-LINEAR MODELS IN CONTINGENCY TABLES

See STATISTICIAN, THE

‘‘Profile analysis’’ has been used as a statistical term since the fifties. Originally it was used in the social sciences, but there exist many applications in medical, biological, and economic fields. One example concerns biological diurnal rhythms of hormone levels such as cortisol or melatonin levels. One may observe serum samples of hormone levels from human beings over a day-and-night period. Typically melatonin reaches its peak value about midnight and its lowest values during the forenoon, whereas the opposite holds for cortisol. We may be interested in comparing different groups of individuals with respect to some population averages over day-and-night cycles—for instance, between shift workers, who are on duty during at night, and nine-to-five workers. Thus, we may be dealing with data which, for each individual from each population, have been collected every fourth hour during the twenty-four hours. Hence, we may have observations at 4, 8, 12, 16, 20, and 24 o’clock, from which it is possible to construct a graph by joining averages within respective treatment groups. The primary interest is to study if the two graphs are similarly shaped. We suppose that for various reasons the mean values are of less interest than the shape, or, in other words, the profile. The so-called battery of tests may serve as another interesting example. Suppose that one intends to compare students from two schools with regard to their performance in mathematics, languages, and biology. For each of these subjects one tests the students and comes up with some kind of score (see Table 1). In Fig. 1(a) the average scores for school A and school B are plotted and joined with lines. The figure suggests that the shapes of the graphs are the same. In Fig. 1(b) are graphs that clearly differ. Note that one is not interested in the absolute levels of the scores; one is interested in their relative levels. The conclusion could be that one school is focusing more on mathematics and the other on languages. It could also be

6532

PROFILE ANALYSIS

Table 1. Student Scores in Two Schools and Three Subjects School A

School B

Individual

Math.

Lang.

Biol.

Individual

Math.

Lang.

Biol.

1 2 . .. n1

10 11 . .. 10

22 20 . .. 21

11 10 . .. 10

1 2 . .. n2

15 13 . .. 17

22 24 . .. 23

15 14 . .. 16

that the difference in scores has nothing to do with the schools themselves, but rather mirrors the selection of students. Of course, the quality of the teachers may be better in one school, but it is difficult to draw that conclusion, because the students do not constitute random samples. Let us now formalize the concept of statistical profile analysis. The purpose is to present the classical approach first given by Greenhouse and Geisser [3]. However, the analysis below will be put into the framework of multivariate analysis of variance∗ , repeated measurements∗ , and growth curve∗ analysis. It is beyond our scope to give a complete review of the subject; various generalizations can be found via a literature search. For extensions within the framework of classical multivariate linear models see refs. [5,10,13]. Other extensions, where test statistics are based on a nonparametric approach, are given by Chinchilli and Sen [2] and Bhapkar [1]. Even in the classical approach there are several open problems; one is to exploit the connections with multiple testing. Among other things, the significance levels of the tests given below are not as exact as might appear at first sight. Since experimental units are observed (measured) several times, we cannot regard all measurements as independent; hence we start from a repeated-measurements point of view where observations within experimental units are correlated and observations between experimental units are uncorrelated. Suppose that for experimental units within g treatment groups, p repeated observations are obtained, say, Xijk ,

i = 1, 2, . . . , p,

j = 1, 2, . . . , g,

k = 1, 2, . . . , nj .

(1)

We assume that the experimental units constitute random samples from their respective groups. Hence, for every i and j the expectations satisfy E[Xijk ] = µij ,

k = 1, 2, . . . , nj .

For the dispersion it is natural to assume some kind of correlation structure within experimental units, i.e., the covariances satisfy C[Xi1 j1 k1 , Xi2 j2 k2 ] " σi1 i2 = 0

for j1 = j2 , k1 = k2 , otherwise.

As with repeated measurements, the σij are functionally dependent if autocorrelation, intraclass correlation, or equicorrelation structure, etc. applies. In the following, besides symmetry (σij = σji ), we only require positive definiteness of the matrix  = (σij ); no other structure will be imposed. Let us at first consider just one treatment group, i.e., g = 1 in (1). If the observations Xik are arranged into a matrix X = (Xik ) and the mean parameters µi into a p-vector B = (µi ), the model X = BC + F

(2)

emerges, where C is a vector of n ones, i.e. C = 1n = (1, 1, . . . , 1), and F is an error matrix. Since the columns of X are independently distributed, so also are the columns fi , i = 1, 2, . . . , n, of F. Up to now we have not made any assumption about the distributions for F. In the classical approach, we assume a multivariate normal distribution to hold.

PROFILE ANALYSIS

6533

where B1 C1 : B2 C2 : · · · : Bg Cg = (B1 : B2 : · · · : Bg )C and C is a g ×



nj matrix satisfying

 1 ··· 0 · · ·  C = .  .. 0 ···

1 0 .. . 0

0 ··· 1 ··· .. .. . . 0 ···

0 ··· 1 ··· .. . 0 ···

 0 0  ..  . . 1

Let X = (X1 : X2 : · · · : Xg ), B = (B1 : B2 : · · · : Bg ), F = (F1 : F2 : · · · : Fg ), where Fj ∼ Np,nj (0, , Inj ). Then, again, we may write X = BC + F.

(3)

However, Xisp ×

g 

nj , Bisp × g,

j=1

Cisg ×

g  j=1

Figure 1. (a) Average scores for school A (solid lines) and school B (dashed lines). (b) Profiles which are not parallel.

Thus, fi ∼ Np (0, ). In general, when there are g treatment groups, there will be g models of the form (2). Putting these models side by side yields X1 : X2 : · · · : Xg = B1 C1 : B2 C2 : · · · : Bg Cg + F1 : F2 : · · · : Fg ,

nj , Fisp ×

g 

nj .

j=1

At first sight the model in (3) is just the multivariate analysis of variance∗ model (MANOVA). However, next we present what is typical of profile analysis. Consider the special case of two groups, i.e. g = 2, and let us return to the classical battery-of-tests example. Suppose now in Fig. 1(a) that the dots represent the mean values, i.e., for school A we have µA1 , µA2 , µA3 , and for school B, µB1 , µB2 , µB3 . According to the literature on the classical approach, there exist three different tests concerning the mean structure. The first is the so-called test of similarity of profiles, sometimes referred to as the test of parallel profiles. The null hypothesis is     µ1B − µ2B µ1A − µ2A = (4) H01 : µ2A − µ3A µ2B − µ3B

6534

PROFILE ANALYSIS

This is the starting point of the classical approach. Note that the way of ordering the points µ1· , µ2· , µ3· is immaterial for the procedures given below, although the graphs such as the one in Fig. 1(a) will look different. Equivalently, in (4) we may write 



H01 :

µ1A − µ1B µ2A − µ2B



 =

 µ2A − µ2B ; µ3A − µ3B

n

j 1  xj = xkj , nj

k=1

xkj = (X1jk , X2jk , . . . , Xpjk ) ,

j = 1, 2,

and

equality of mean differences is the same as no interaction between tests and schools. The restriction in (4) will be reformulated in terms of matrices, but instead of p = 3 a general p will be considered. Let D be (p − 1) × p, where  1 −1 0 · · · 0 1 −1 · · ·  D = . . ..  .. .. . 0 0 0 ···   1 E= . −1

where

 0 0  ..  , . 1

S=

n1 n2   (xk1 − x1 )2 + (xk2 − x2 )2 . k=1

k=1

The test of similar profiles was first given by Greenhouse and Geisser [3] and extended a result of Rao [7]. If the hypothesis of similar profiles is not rejected the classical approach explores whether the profiles are either identical (coincident) or parallel to the x-axis. The resemblance of profiles is modeled by parametric restrictions on B, given by

(5) BE = 0,

Then the hypothesis in (4) is identical to H01 : E[X] = BC,

DBE = 0.

(6)

where E was defined in (5) (see Fig. 2). However, since the hypothesis of similarity of profiles is not rejected, the testing problem is set up as

(7)

H02 : E[X] = BC, DBE = 0, BE = 0;

The alternative hypothesis will be H11 : E[X] = BC,

i.e., DBE = 0. Hence, the test of similar profiles is a MANOVA with a null hypothesis described via (6). If g = 2, i.e. two treatment groups, the hypothesis H01 is rejected if n1 + n2 − p  E (CC )−1 CX D (DSD )−1 p−1 

 −1

× DXC (CC )



 −1

E[E (CC )

E]


Fp−1,n1 +n2 −p,α ,

−1

H12

: E[X] = BC, DBE = 0.

(8) (9)

Note that H02 is identical to H02 : E[X] = BC, BE = 0. From one point of view the test formulated by (8) and (9) is more powerful than the usual MANOVA test 

H02 : E[X] = BC, DBE = 0, BE = 0;

where



H12 : E[X] = BC, S = X(I − C (CC )−1 C)X .

Observe that E (CC )−1 E =

1 1 + , n1 n2

XC (CC )−1 E = x1 − x2 ,

since in this test no assumption about parallelism is made. However, to properly take into account the pretesting in (6), in order to determine the significance level, is very difficult. If the profiles are assumed to be identical, one may be interested in estimating the

PROFILE ANALYSIS

6535

or expressed in the original quantities,  −1 % = 1p 1 p S (x1 − x2 ) , BE  1 p S−1 1p

since one may choose (D )0 = 1p . Furthermore, since the profiles are supposed to be parallel, we can write BE = δ1p , where δ is an unknown constant estimated by δˆ =

1 p S−1 (x1 − x2 ) . 1 p S−1 1p

From results on growth curves∗ a confidence interval for δ can be constructed: Iδ = δˆ ± tn1 +n2 −p−1,α/2 

−1/2

×

n1 n2 (n1 + n2 − p − 1) n1 + n2

(1

2 + Tp−1 )1/2 (1 p S−1 1p )−1/2

,

where 2 = Tp−1

n1 n2 (x1 − x2 ) D (DSD )−1 n1 + n2 × D(x1 − x2 )

Figure 2. (a) Coincident profiles. (b) Profiles which are not coincident.

difference between them. In terms of a linear models approach, E[X] = BC,

DBE = 0,

the intention would be to estimate BE. Let (D )0 and E0 be any matrices of full rank such   that (D )0 D = 0, r((D )0 ) = p − r(D), E0 E = 0 0, r(E ) = g − r(E), where r(·) denotes the rank of a matrix. Note that E0 and (D )0 form bases for the orthogonal complements to the column spaces of E and D , respectively. By either performing a reparametrization or relying on the theory for multivariate linear models [8] it follows that % = (D )0 [(D )0 S−1 (D )0 ]−1 (D )0 S−1 BE × XC (CC )−1 E,

(10)

(see HOTELLING’S T 2 ). A test that the profiles are identical given that the profiles are parallel can be performed via the confidence interval, i.e., the null hypothesis is rejected if 0 lies outside the interval. Alternatively, the hypothesis is rejected if n1 n2 (n1 + n2 − p − 1)  −1 2 1 p S 1p δˆ2 (1 + Tp−1 )−1 n1 + n2
F1,n1 +n2 −p−1,α . The test and the confidence interval are due to Srivastava [11]. Commonly in use are less powerful alternative tests. For example, to test that the profiles are identical given that they are similar one tests H02 : E[X] = BC, 1 p BC = 0. The test is rejected [4] if n1 n2 (n1 + n2 − 2) (x1 − x2 ) 1p (1 p S1p )−1 1 p n1 + n2 × (x1 − x2 )
F1,n1 +n2 −2,α .

6536

PROFILE ANALYSIS

(n1 + n2 − p + 1)(n1 + n2 )   x D DSD p−1  

−1 1 1 + + D(x1 − x2 )(x1 − x2 ) D n1 n2 × Dx
Fp−1,n1 +n2 −p+1,α , where x = (n1 x1 + n2 x2 )/(n1 + n2 ). Sometimes, as a third step in profile analysis, one tests whether the profiles are identical and parallel to the x-axis against the alternative that the profiles are identical. In this case we have a stagewise procedure. First one tests that the profiles are similar; second, if the profiles are similar, one tests if the profiles are identical; third, if the profiles are identical, one tests if they are parallel to the x-axis. The last is precisely the test of equality of means in one sample. Formally, using the preceding notation, we may write 

H03 : E[X] = BC, BE = 0, DB = 0, 

H13 : E[X] = BC, BE = 0, 

and H03 is rejected if n1 + n2 − p + 1 0 E CX D (DSD )−1 DXC E0 (p − 1)(n1 + n2 ) Figure 3. (a) Profiles parallel to the x-axis. (b) Profiles which are not parallel to the x-axis.

Next we present the test that the profiles are parallel to the x-axis (see Fig. 3). For this test the hypotheses are H03 : E[X] = BC, DBE = 0, DB = 0, H13 : E[X] = BC, DBE = 0, which is equivalent to H03 : E[X] = BC, DB = 0, H13 : E[X] = BC, DBE = 0. pt The test of parallelism with the x-axis is sometimes referred to as a test of condition variation, since it sometimes tests for differences in conditions. The likelihood ratio test∗ states that H03 is rejected if


Fp−1,n1 +n2 −p+1,α . 

where E0 CX = n1 x1 + n2 x2 = (n1 + n2 )x. A mean structure of the form E[X] = ABC for some A and C comes within the growth curve model of Potthoff and Roy [6] (see ref. [9] for a review). All results given above follow from results for the growth-curve model, but by assuming that the profiles are parallel, the tests here usually differ from those applied in growth curve analysis. Moreover, the assumption of similar profiles, i.e. the restriction DBE = 0, can equivalently be written B = 1p 1 + D 2 1 g .

(11)

In this equation 1 and 2 have natural interpretations; (10) presents an extended growth-curve model [8]. Furthermore, to test for identical profiles is the same as to test whether 1 = 0, and in order to test for parallelism with the x-axis we test whether 2 = 0.

PROFILE ANALYSIS

We now present some results for more than two treatment groups, i.e. r(C) > 2. The distribution of the test statistic for the test of parallel profiles is then more complicated. In (6) and (7) the test of similarity of profiles was given; if we replace C in these equations by C given by (3) and the matrix E by the g × (g − 1) matrix   1 0 ··· 0 −1 1 · · · 0      E =  0 −1 · · · 0  ,   . . . ..  ..  .. 0 0 · · · −1 the hypothesis-testing problem once again is H01 : E[X] = BC, DBE = 0, H11 : E[X] = BC. From MANOVA results the hypothesis is rejected if |DSD |/|DMD |
Up−1,g−1,n−g,α , where n=

g 

nj ,

S = X[I − C (CC )−1 C]X ,

j=1

M = X[I − C 1g (1 g CC 1g )−1 1 g C]X , and U·,·,·,α is the α-percentile of the standard test distribution in MANOVA (e.g. ref. [12]; see also WILKS’S LAMBDA CRITERION; HOTELLING’S TRACE; PILLAI’S TRACE). In order to test that the profiles are identical given that they are parallel, the quantity λ=

|S||DMD | |M||DSD |

is used. The hypothesis is rejected if n−g−p+11−λ
Fg−1,n−g−p+1,α . g−1 λ The hypothesis that all profiles are parallel to the x-axis is rejected if n−p+1  1 g CX D (DMD )−1 DXC 1g n(p − 1)
Fp−1,n−p+1,α . For further details see Srivastava [11].

6537

REFERENCES

1. Bhapkar, V. P. (1984). On nonparametric profile analysis of several multivariate samples. J. Statist. Plann. Inference, 9, 285–296. (Procedures based on U-statistics and linear rank statistics are given.) 2. Chinchilli, V. M. and Sen, P. K. (1982). Multivariate linear rank statistics for profile analysis. J. Multivariate Anal., 12, 219–229. (Linear rank statistics are utilized together with Roy’s union–intersection principle.) 3. Greenhouse, S. W. and Geisser, S. (1959). On methods in the analysis of profile data. Psychometrika, 24, 95–112. (The first paper on profile analysis.) 4. Johnson, R. A. and Wichern, D. W. (1998). Applied Multivariate Statistical Analysis, 4th ed. Prentice-Hall, Englewood Cliffs, NJ. (Introductory textbook in multivariate analysis, including numerical illustrations; see Sec. 6.6.) 5. Nel, D. G. (1987). A profile analysis for two groups under the heteroscedastic assumption. Commun. Statist. Theory Methods, 16, 3577–3589. (The Behrens–Fisher problem is considered.) 6. Potthoff, R. F. and Roy, S. N. (1964). A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika, 51, 313–326. (The first presentation of the growth-curve model.) 7. Rao, C. R. (1948). Tests of significance in multivariate analysis. Biometrika, 35, 58–79. (The analysis of dispersion is clearly presented.) 8. von Rosen, D. (1989). Maximum likelihood estimators in multivariate linear normal models. J. Multivariate Anal., 31, 187–200. (General approach for treating multivariate linear models is given.) 9. von Rosen, D. (1991). The growth curve model: a review. Commun. Statist. Theory Methods, 20, 2791–2822. (Papers which have treated the growth curve model are presented.) 10. Sparks, R. and Ballantyne, R. (1990). Interactions explored using an extension of profile analysis. S. Afr. Statist. J., 24, 127–152. (An interesting paper where profile analysis is extended.) 11. Srivastava, M. S. (1987). Profile analysis of several groups. Commun. Statist. Theory Methods, 16, 909–926. (General theoretical results for profile analysis are given.)

6538

PROFILE-a TESTS

12. Srivastava, M. S. and Carter, E. M. (1983). Applied Multivariate Statistics. NorthHolland, New York. (A solid applied work on classical multivariate analysis.) 13. Yokoyama, T. and Fujikoshi, Y. (1993). A parallel profile model with random-effects covariance structure. J. Japan Statist. Soc., 23, 83–89. (Profile analysis with certain covariance structures is considered.) See also MULTIVARIATE ANALYSIS; and MULTIVARIATE ANALYSIS OF VARIANCE (MANOVA).

DIETRICH VON ROSEN

PROFILE-a TESTS In the classical Neyman–Pearson theory of statistical hypothesis testing∗ , the probability of an erroneous rejection of a hypothesis H is controlled by a preassigned significance level α, called the size of the test. The underlying concept involves finding tests with highest power subject to a given size α. That is, size and power are treated unequally, leading to unsymmetrical test results with different interpretative meanings. In particular, not rejecting (‘‘accepting’’) H does not imply the rejection of the alternative hypothesis. This drawback in the classical theory gave rise to the development of profile-a tests, which ¨ were introduced by Ruger [2,3] and further developed by Wassmer [4]. The idea is to measure the evidence of a sample for one hypothesis against the other by the ratio of the respective probabilities of some appropriate region in the sample space∗ . The decisions are based on a level a, which does not—as in the classical theory—consist of a condition for a probability, but rather for a ratio of two probabilities. For the sake of simplicity, consider two simple hypotheses H− and H+ , and let there be given a number a > 1 (usually an integer). If an event is observed which under H+ is at least a times more likely than under H− , we accept H+ (and reject H− ). Conversely, if an event is observed which under H− is at least a times more likely than under H+ , we accept H− (and reject H+ ). That is, we do not already reject a hypothesis—as in the classical theory—if the probability of erroneous rejection is small (α), but only if the probability of a correct decision (i.e., the probability of

accepting H− if H− is true and the probability of accepting H+ if H+ is true, respectively) is at least a times greater than the probability of a false decision. Furthermore, to achieve symmetric test results the ratios of the respective probabilities should be equal. This additional symmetry condition is extracted from the idea that the hypotheses should be treated symmetrically and that a decision on H− should posses the same evidence from the data as a decision on H+ . A profile test is well defined if a no-decision region is introduced which consists of the data points where the sample shows evidence for neither H− nor H+ . Subject to an optimality criterion which maximizes the probabilities for correct decisions, these conditions lead to a unique partition of the sample space. Example 1. Let X1 , . . . , Xn be independent and distributed according to the Gaussian distribution with known σ 2 = 1, and suppose it is desired to test H− : µ = −1 and H+ : µ = +1. For n = 2, the corresponding profile test at level a = 20 becomes: if x < −0.211, accept H− ; if x > 0.211, accept H+ ; otherwise, accept neither H− nor H+ . As is easy to verify, PrH+ (X > 0.211) PrH− (X > 0.211)

=

PrH− (X < −0.211) PrH+ (X < −0.211)

= 20; 0.211 and −0.211 are the lowest and the greatest values, respectively, for which the profile condition is fulfilled. These thresholds consequently give the highest probabilities for correct decisions. For comparison, the classical Neyman–Pearson∗ test at level α = 0.05 rejects H− in favor of H+ if x > 0.170, or rejects H+ in favor of H− if x < −0.170, and hence the profile test approach leads to a more conservative decision rule than the classical test with α = 0.05. The α = 5% level of the classical test, however, ‘‘corresponds’’ with the a = 20 level, since by performing a profile-a test both probabilities of false decision are bounded by 1/a. For n = 10, the profile test becomes:

PROFILE-a TESTS

if x < 0, accept H− ; if x > 0, accept H+ . Here, PrH+ (X > 0) PrH− (X > 0)

=

PrH− (X < 0) PrH+ (X < 0)

> 20,

and 0 is the unique threshold that provides a profile test at level a = 20. In particular, the symmetry condition makes the calculation of the test possible. (Note that a lower threshold for accepting H+ and a higher threshold for accepting H− would keep the profile condition, but would fail to produce a well-defined test.) The classical test (α = 0.05) rejects H− in favor of H+ if x > −0.477, or rejects H+ in favor of H− if x < 0.477, and hence depends on the formulation of the test problem, leading to different test outcomes. Example 2. Let X be binomial (n; p) with n = 20; the hypotheses H− : p = 0.2 and H+ : p = 0.4 are tested at level a = 20. When we choose c to make PrH+ (X > c)/ PrH− (X > c) greater than and as near as possible to a = 20 (i.e., we want to decide on H+ for large values of x), we get c = 8. To get a = 20 exactly here, we have to resort to randomization∗ , yielding the rule: accept H+ with probability 1 if x = 9, . . . , 20; accept H+ with probability 0.777 if x = 8. Analogously, the rule for H− becomes: accept H− with probability 1 if x = 0, . . . , 3; accept H− with probability 0.191 if x = 4. The theory of profile tests therefore uses a test function φ(x) with three components refering to the probabilities of the outcomes ‘‘acceptance of H+ ,’’ ‘‘acceptance of H− ,’’ and ‘‘no decision.’’ The decision rule of a profilea test allows symmetric test results, thus avoiding the difficulty of the (actually unnatural) unsymmetry in the classical test results. There are alternative approaches that overcome this feature of the Neyman–Pearson theory (see MULTIPLE

6539

DECISION PROCEDURES). None of these, however, takes into account the probability ratio of the test outcomes. A second drawback in the classical theory is that the choice lies between only two test outcomes: rejecting H or not rejecting H corresponds to a simple division of the sample space into the critical region and its complement. It would be desirable, however, to find a concept which distinguishes between sample results, allowing hierarchically ordered types of inferences. Indeed, in deciding between two hypotheses H− and H+ , profile-a tests in general are characterized by permitting more than two test outcomes, which are ordered according to the evidence of a sample for either H− or H+ . GENERAL FORMULATION To progress with the general formulation of profile-a tests, consider 2k + 1 test outcomes A−k , . . . , A0 , . . . , A+k , where A−k , . . . , A−1 and A+1 , . . . , A+k are decisions on the (not necessarily simple) hypotheses H− and H+ , respectively, and A0 is the decision on neither H− nor H+ . A+i+1 shows stronger evidence for H+ than A+i , and A−i−1 analogously shows stronger evidence for H− than A−i . Accordingly, the resulting test is given by a test vec(x), . . . , φ0 (x), . . . , φ+k (x)) where tor φ(x) = (φ−k 0  φi (x)  1, φi (x) = 1 for all x in the sample space, and φi (x) refers to the probability of the decision Ai given x. If the distribution of X is Pθ, these probabilities are given by Pr θ (Ai ) = φi (x)dPθ (x), where θ denotes the unknown parameter. In the nonrandomized case such a test consists of a partition of the sample space in disjoint regions B−k , . . . , B0 , . . . , B+k with φi (x) = 1 if x ∈ Bi , / Bi . and φi (x) = 0 if x ∈ Let a = (a1 , . . . , ak ) with 1 < a1 < · · · < ak denote the preassigned (arbitrarydimensional) level. The basic requirement for the test φ to be a profile-a test is that for i = 1, . . . , k, inf θ ∈H+ Pr θ (A+i )
ai sup η∈H− Pr η (A+i ), inf η∈H− Pr η (A−i )
ai sup θ ∈H+ Pr θ (A−i ) (1) should hold. That is, in the nonrandomized case the test results in A+i if an event B+i is observed whose probability under H+ is

6540

PROFILE-a TESTS

at least ai times more likely than under H− , and an analogous condition should hold in the event B−i for the decision A−i . If sup η∈H− Pr η (A+i ) = 0 and sup θ ∈H+ Pr θ (A−i ) = 0, the symmetry condition is written as inf η∈H− Pr η (A−i ) inf θ ∈H+ Pr θ (A+i ) = , (2) sup η∈H− Pr η (A+i ) sup θ ∈H+ Pr θ (A−i ) i = 1, . . . , k. Hence, the requirements for the outcomes A+i are completely analogous to those for the outcomes A−i . A test φ that fulfills (1) and (2) is called a profile test at level a or profile-a test. Simple Hypotheses ¨ Ruger [2,3] introduced profile-a tests for the case of simple hypotheses, i.e., both H− and H+ consist of one distribution, which, without loss of generality, may be parametrized by θ− and θ+ , respectively. In the family Fa of these tests, he defined an optimality criterion for profile-a tests based on the idea of optimizing probabilities for correct test decisions where decisions with higher evidence for H− or H+ take precedence over decisions with lower evidence. More precisely, a profile-a test φ is said to be better in component i than a profile-a test ψ if

φ+i (x)dPθ+ (x)
ψ+i (x)dPθ+ (x),



φ−i (x)dPθ− (x)


ψ−i (x)dPθ− (x),

with strict inequality at least once. A profilea test φ is as good as ψ in component i if equality holds in the above conditions. Using this terminology, a profile-a test φ is better than ψ if there exists a j(1  j  k) with φ as good as or better than ψ in the components i with j < i  k, and φ better than ψ in component j. Consequently, a profile-a test φ ∗ is optimum in Fa if there exists no test in Fa better than φ ∗ . As a consequence of the Neyman–Pearson ¨ [2] showed that φ ∗ has neclemma∗ , Ruger essarily the form of a generalized likelihood ratio test∗ , which in the nonrandomized case is given by  1 if Q(x) < κ−k , ∗ (x) = φ−k 0 if Q(x)
κ−k ,

∗ φ−i (x)

 Q(x)  κ−i−1 , 0 if = 1 if κ−i−1 < Q(x) < κ−i ,  0 if Q(x)
κ−i ,

for i = 1, . . . , k − 1,  0 = 1  0  0 ∗ φ+i (x) = 1  0 φ0∗ (x)

if Q(x)  κ−1 , if κ−1 < Q(x) < κ+1 , if Q(x)
κ+1 , if Q(x)  κ+i , if κ+i < Q(x) < κ+i+1 , if Q(x)
κ+i+1 ,

for i = 1, . . . , k − 1, and ∗ (x) = φ+k

 0 if Q(x)  κ+k , 1 if Q(x) > κ+k ,

where Q(x) denotes the likelihood ratio of H+ to H− , and κi are real numbers with 0  κ−k  · · ·  κ+k  ∞. [In order to permit randomization one has to treat the case Q(x) = κi separately.] Due to the optimality criterion, κ+k and κ−k in φ ∗ are, if possible, chosen such that Pr θ+ (Q(X) > κ+k ) = ak Pr θ− (Q(X) > κ+k ) =

Pr θ− (Q(X) < κ−k ) . Pr θ+ (Q(X) < κ−k )

Having done this, the remaining κi ’s have to be chosen so that Pr θ+ (κ+i < Q(X) < κ+i+1 ) = ai Pr θ− (κ+i < Q(X) < κ+i+1 ) =

Pr θ− (κ−i−1 < Q(X) < κ−i ) . Pr θ+ (κ−i−1 < Q(X) < κ−i )

This is done first, if possible, for i = k − 1, then for i = k − 2, etc. An optimum profile-a test φ ∗ can be definitely calculated in this way, since R(τ , ξ ) :=

Pr θ+ (τ < Q(X) < ξ ) , Pr θ− (τ < Q(X) < ξ ) 0  τ < ξ  ∞,

is (weakly) isotonic in ξ given τ , and in τ given ξ .

PROFILE-a TESTS

Example 3. Let X1 , . . . , Xn be independent and Gaussian (µ, σ ) with known µ = 0, and the hypotheses tested at profile level a = (10, 20, 100) be H− : σ 2 = 1 and H+ : σ 2 = 2. depends on the sample The optimum test φ ∗ x only through s2 = x2i /n, and φ ∗ is given by the regions for s2 . For various n the optimum tests are given in Table 1. Depending on n, not all test outcomes may be realized. For n = 8, outcomes with higher evidence for H− than A−1 are not realizable. For moderate n(n = 16) all outcomes are realizable, whereas with increasing n the number of realizable outcomes decreases. If n is large enough (n = 100), only outcomes in the components with highest intensity are possible. As in this example, a general feature of the profile-a test concept is that the two distributions (under H− and H+ ) produce a specific division of the sample space and ‘‘pick out’’ their typical test outcomes.

Composite Hypotheses In order to derive profile-a tests for composite hypotheses∗ we first mention that in general, for a test to fulfill a specified a,

Table 1. Test results

Region n=8

A+3 A+2 A+1 A0 A−1 A+3 A+2 A+1 A0 A−1 A−2 A−3 A+3 A0 A−1 A−3 A+3 A−3

s2 > 3.217 2.653 < s2 < 3.217 2.436 < s2 < 2.653 0.304 < s2 < 2.436 s2 < 0.304 n = 16 s2 > 2.219 2.062 < s2 < 2.219 1.876 < s2 < 2.062 0.846 < s2 < 1.876 0.773 < s2 < 0.846 0.270 < s2 < 0.773 s2 < 0.270 n = 50 s2 > 1.535 1.238 < s2 < 1.535 1.166 < s2 < 1.238 s2 < 1.166 n = 100 s2 > 1.377 s2 < 1.377

6541

the hypotheses H− and H+ must be positive in distance. That is, assuming an indifference zone I between the hypotheses, the parameter space is partitioned into three sets belonging to H− , H+ , and I, respectively. Parameter values in H+ must differ from those in H− to the extent that the probability requirements (1) can be fulfilled at all and, accordingly, profile-a tests for testing H− and H+ are obtainable. In order to derive optimum profile-a tests for composite hypotheses, an alternative optimality criterion is formulated that demands a hierarchical maximin optimality of an optimum profile-a test. Roughly speaking, this criterion first requires maximizing inf θ ∈H+ φ+k (x)dPθ (x)  (x)dP φ and inf η∈H η (x). Having done this, −k  − inf θ ∈H+ φ+k−1 (x)dP θ (x) has to be maximized, and inf η∈H− φ−k+1 (x)dPη (x), etc. A solution to this maximin problem is obtained by specifying a pair of distributions in the parameter space which is least favorable∗ , i.e., which minimizes the hierarchical power of a profile-a test for the reduced (simple) hypotheses formed with this pair. In specific situations, e.g., by assuming a oneparameter exponential family∗ and by considering hypotheses that specify regions in the parameter space, the pair of least-favorable distributions∗ is given by the pair which gives mass 1 on the points in the parameter spaces of H− and H+ with the smallest distance. Optimum profile-a tests are then given by the generalized likelihood ratio test described in the preceding section. The calculation of the κi ’s is straightforward but generally different from the case where only simple hypotheses are considered. Nonetheless, the determination leads to a unique solution which first makes the regions for A+k and A−k as large as possible, then the regions for A+k−1 and A−k+1 , etc. [4]. Example 4. Suppose X1 , . . . , Xn are independent and Gaussian (µ, σ ) with known σ 2 = 4. By considering the hypotheses H− : −0.5 − c  µ  −0.5 and H+ : +0.5  µ  +0.5 + c, given c
0, for n = 10 the maximin tests φ ∗ [profile level a = (10, 20, 100)] are based on x (see MINIMAX TESTS), and the resulting regions for the test outcomes are given in Table 2. If the specified parameter regions for H− and H+ are small, i.e., if c is

6542

PROFILE-a TESTS

Table 2. Test results A+3 A+2 A+1 A0 A−1 A−2 A−3 A+3 A+2 A0 A−2 A−3 A+3 A0 A−3

Region c = 0.5(c = 0) x > 1.640 0.973 < x < 1.640 0.873 < x < 0.973 −0.873 < x < 0.873 −0.973 < x < −0.873 −1.640 < x < −0.973 x < −1.640 c = 2.0 x > 1.640 1.360 < x < 1.640 −1.360 < x < 1.360 −1.640 < x < −1.360 x < −1.640 c = 2.5 x > 1.640 −1.640 < x < 1.640 x < −1.640

small, then φ ∗ coincides with the optimum test for the corresponding simple hypotheses (c = 0). With larger c the number of realizable outcomes decreases continually. If c is large enough and, in particular, the one-sided hypotheses H− : µ  −0.5 and H+ : µ
+0.5 are accepted, then the differentiation of outcomes in interior components of φ ∗ is no longer possible, and only decisions with highest evidence for H− or H+ are realizable. This last property is unappealing and might seem inappropriate to practitioners. In the light of the conceptual framework of profile-a tests, however, it is obvious and easily interpretable. Namely, in these cases the class of distributions is ‘‘too big’’ to produce a sufficiently fine division of the sample space. Furthermore, typically only rather large regions for H− and H+ make the differentiation in interior components impossible. Nevertheless, it can be argued that with a less extreme sample the intensity of the test result should gradually decrease, too. This issue is under discussion, possibly leading to an alternative profile-a test concept. In this concept, in order to define the tests and their optimality, separate test decisions A+i and A−i as described here are not considered, but rather their cumulative counterparts ∪kj=i A+j and ∪kj=i A−j , respectively. Fortunately, the

derivation of cumulative profile-a tests is straightforward, since it is reducible to the case k = 1. Clearly, in the latter case this concept leads to exactly the same tests as the one previously described. For k > 1, however, it does not. In the case of one-sided hypotheses, cumulative profile-a tests have more different outcomes in general between the two extremes A+k and A−k . Therefore, this concept might be preferred. The question, however, of whether these tests are preferable on the whole is still unanswered. The one-parameter case provides some general solutions, since here the pair of leastfavorable distributions is directly obtainable. In more general cases, e.g., by considering test problems with nuisance parameters∗ , these pairs are available only in specific situations (concerning assumed hypotheses and distributions). On the other hand, restricting the class of permitted tests, as in the classical theory [1], provides solutions for optimum profile-a tests. In particular, restricting the class to tests that are invariant under suitable transformations in the sample and parameter space leads to invariant maximin profile-a tests (see INVARIANCE). Furthermore, conditioning is also an instrument which enables the determination of profilea tests (see CONDITIONAL INFERENCE). These approaches and the way in which classical properties are transferable to profile-a tests are described in detail in Wassmer [4]. APPLICATIONS In practice profile-a tests provide a tool for evaluating and interpreting sample results. For example, these tests can be applied in bioavailability and bioequivalence∗ studies, where the tests can establish both equivalence and nonequivalence of two drugs [5] (see also BIOEQUIVALENCE CONFIDENCE INTERVALS). Among other things, in these studies profile tests display which sample size is necessary so that a decision on the equivalence of the drugs is possible at all, or which sample size is necessary so that any sample result shows evidence for either equivalence or nonequivalence. Hence, these tests provide an alternative method for sample size calculations.

PROFILES OF DIVERSITY

There are other potential fields of application where both the rejection and the acceptance of a hypothesis (in favor of an alternative hypothesis) should be taken into account, and symmetric test decisions are desirable (e.g., quality control∗ ). Furthermore, in all areas where the knowledge of both states of nature should influence the construction of the test, profile-a tests are useful to apply. REFERENCES 1. Lehmann, E. L. (1986). Testing Statistical Hypotheses, 2nd ed. Wiley, New York. ¨ 2. Ruger, B. (1978). Niveauprofiltests: Konstruktionsprinzip von Alternativtests mit endlicher Ergebnismenge (in German). Tech. Rep., Department of Statistics, University of Munich, Germany. ¨ ¨ zwei ein3. Ruger, B. (1980). Alternativtests fur fache Hypothesen mit endlich vielen Ergebnissen (in German). Metrika, 27, 73–90. (The paper that introduced profile tests for the case of simple hypotheses. Proofs of the theorems are given in ref. [2].) ¨ 4. Wassmer, G. (1993). Niveauprofiltests fur zusammengesetzte Hypothesen (in German). Doctoral dissertation, Department of Statistics, University of Munich, Germany. (Generaliza¨ tion of Ruger’s fundamental ideas to composite hypotheses.) 5. Wassmer, G. (1994). Testing equivalence in clinical trials using a new principle for constructing statistical tests. Commun. Statist. Theory and Methods, 23, 1413–1427. See also HYPOTHESIS TESTING; LIKELIHOOD RATIO TESTS; MINIMAX TESTS; and NEYMAN –PEARSON LEMMA.

GERNOT WASSMER ¨ BERNHARD RUGER

PROFILE LIKELIHOOD. See PSEUDOLIKELIHOOD

PROFILES, CIRCULAR. See SNOWFLAKES PROFILES OF DIVERSITY Diversity is an important concept in ecology and, under various names, it appears in several biological, physical, social, and

6543

management sciences∗ . The common theme is the apportionment of some quantity into a number of well-defined categories. The quantity may be in such a form as abundance, time, resource, investment, or energy. The concern in diversity analysis is about the degree of the apportionment being more diverse or less diverse, expressed variously as diversification or concentration, spread or specialization, monopoly or lack of it, and so on. As a simple example, consider the apportionment of time spent on two activities, music and mathematics. Suppose that John and Jane apportion their time as follows: Category Person

Music

Mathematics

John Jane

2 3 1 3

1 3 2 3

If we ask whether John has a different kind of specialization/diversification from Jane, the answer is yes: The subject identity matters. Instead, if we ask whether John has a different degree of specialization/diversification from Jane, the answer is no: The subject identity does not matter. The degree of specialization/diversification, or diversity, is permutation invariant; the identities of the categories are disregarded. Consider a quantity distributed among a finite set of categories labeled i = 1, 2, 3, . . . , s. Examples of such a quantity are (a) total abundance of an ecological community distributed among s species, and (b) total market sales distributed among s competing firms. Let πi be the proportion of the quantity found in the ith category, as, for example, the relative abundance of the ith species in an ecological community. We assume for this discussion that π1
π2
· · ·
πs > 0, and π1 + π2 + · · · + πs = 1. The relative abundance vector, π , is given by π = (π1 , π2 , . . . , πs ). Since the vast bulk of applications of diversity analysis occurs in the field of ecology (see ref. 1), we will here refer to the categories as species and define a community C as the pair C = (s, π ). The concept of diversity of a community has been defined by Patil and Taillie [9–12] independent of any particular diversity indices∗ . In a diverse community, the

6544

PROFILES OF DIVERSITY

typical species is relatively rare. Intuitively, a transfer of abundance either from a common to a rare species or from a common to a new, additional species would increase an observer’s assessment of the rarity of a typical species. Accordingly, a community C = (s , π  ) is defined in [10] to be intrinsically more diverse than a community C = (s, π), written C
C, if C can be constructed from C by a finite sequence of the following two operations: 1. Introducing a species in this manner: (a) subtract an amount h from the relative abundance πi of an existing species, where 0 < h < πi , and (b) add a new species to the community of relative abundance h. 2. Transferring abundance between two existing species in this manner: (a) subtract an amount h from the relative abundance πi of a species, and (b) add h to the relative abundance πj of another species, where 0 < h < πi − πj . Note that the species’ actual labels are ignored in transforming the vector π into the vector π  . It is shown in ref. 12 that C is intrinsically more diverse than C if and only if every right tail-sum of π  is greater than or equal to the corresponding right tail-sum of π , that is,  i>k

πi




πi ,

k = 1, 2, 3, . . . , s.

(1)

i>k

Since the kth right tail-sum is the combined abundance of those species rarer than the kth ranked species, greater diversity in this sense means a greater amount of rarity. A diversity measure of a community C is defined in [10,12] as the average rarity of the community, given by (C) =

s 

πi R(i, π ).

(2)

diversity measures with the following properties: (1) (C )
(C) whenever C
C, and (2) (C) = 0 if C has only one species. Two general categories of diversity measures are used frequently, based on two types of rarity functions: (1) rarity functions of the dichotomous type depend only on the relative abundance of the ith species: R(i, π ) = R(πi ); and (2) rarity functions of the ranking type depend only on the (descending) rank of the ith species: R(i, π ) = R(i). One dichotomous rarity function suggested in refs. 10 and 12 is β

R(πi ) = (1 − πi )/β, where β is a constant chosen by the investigator such that β
−1. The limiting form R(πi ) = − log πi is used for β = 0. The result is a parametric family of diversity measures:   s  β+1 β. (3) πi β (C) = 1 − i=1

Three diversity indices∗ popular in ecological studies are special cases of β : (a) −1 = s −1, the species richness index, (b) 0 = −  πi log πi , the Shannon index, and (c) 1 = 1 − πi2 , the Simpson index. However, any other values of β
−1 also yield diversity indices with the required properties mentioned above. Monotone transformation of β would preserve the ordering properties required of a diversity index; some have been proposed as indices. The transformation Sβ = 1/(1 − ββ )1/β is Hill’s [3] family of indices. It has the interpretation of being the numbers equivalent of β ; that is, Sβ is the number of species a totally even community must have in order for its value of β to equal that of the given community. The function log Sβ is R´enyi’s [15] entropy∗ of order β + 1, proposed by Pielou [13] as a diversity index. (See refs. 3, 13, and 15 and DIVERSITY INDICES for additional information.) Another dichotomous rarity function is R(πi ) = (1 − πi )[1 − (1 − πi )m ]/πi ,

i=1

Here R(i, π ) is a numerical measure of rarity associated with the ith species. Different forms for the function R produce different diversity measures. A natural requirement is to use only those forms of R which yield

where m is a nonnegative integer. It leads to the family of diversity indices studied in refs. 4 and 16, HSG m (C) =

s  (1 − πi )[1 − (1 − πi )m ]. i=1

PROFILES OF DIVERSITY

6545

The quantity HSG + 1 has the interpretam tion of being the expected number of species obtained when m + 1 individuals are randomly selected from the community C. One ranking-type rarity function takes the kth ranked species to be the standard against which a species’ rarity is assigned the value 1 or 0: R(i) =

 1, i > k . 0, i  k

The diversity measure produced, denoted Tk (C), is simply the right tail-sum of the relative abundances: Tk (C) =

s 

πi .

(4)

i>k

Other ranking-type indices are discussed in ref. 12. Diversity is a partial ordering in the sense that (a) C
C, (b) C
C whenever C
C and C
C, and (c) a given pair of communities need not be comparable. This explains the complaint frequently heard among ecologists (e.g., in ref. 4) that different diversity indices may give different orderings. In ref. 12, for example, a simple illustration involving two communities is given. The intrinsic diversity profile of a community C is a plot of the diversity measure Tk (C) (4) against the ranks k = 1, 2, . . . , s. By plotting the profiles of two communities on one graph, one can tell at a glance whether the communities can be ordered by diversity. By (1), C
C ⇔ Tk (C )
Tk (C) for all k = 1, 2, . . . , s. Thus the profile of an intrinsically more diverse community is everywhere above or equal to that of a less diverse community (Fig. 1). The profiles will cross if the communities are not intrinsically comparable. The measure Tk (C) is a family of diversity indices parameterized by k. For larger values of k, Tk is in some sense more sensitive to changes in numbers and abundances of rare species in the community. In ref. 12 and Taillie [20] a formal definition of sensitivity of an index to rare species is provided. In particular, it is shown that the sensitivity of β to rare species is a strictly decreasing

Figure 1. Intrinsic diversity profiles of four communities (data from [11]). It is seen that C1
C2 , C1
C3 , C1
C4 , C2
C3 , C2
C4 , but that C3 and C4 are not intrinsically comparable.

function of β, while that of HSG is a strictly m increasing function of m. Other profiles of diversity can be constructed by plotting a parametric diversity index against a parameter which measures sensitivity of the index to rare species. For example, β (C) can be computed for a given community C using a variety of values of β
−1. The plot of β against β is a type of diversity profile. Similarly, HSG may be comm puted and plotted for a variety of m values. If the diversity profiles of two communities cross, the communities cannot be intrinsically ordered according to diversity. Additional information on diversity analysis may be found in the references following. Ecological studies using diversity profiles include refs. 2, 11, and 8 (this last study refers to the intrinsic diversity profile as a ‘‘kdominance curve’’). In ref. 21 the β , Sβ , and

6546

PROGRESSIVE CENSORING SCHEMES

HSG families are used to explore relationm ships between genetic variation and species diversity. In ref. 19 the β family is applied to forestry. In ref. 17 estimation for diversity index families is studied. In ref. 18 a diversity ordering based on the concept of majorization is proposed. It turns out to be equivalent to the intrinsic diversity ordering (see ref. 12). In ref. 5 the log series distribution is employed as a model of species abundances, and the estimate of the parameter α is used as a diversity index (see also refs. 6 and 7). In ref. 20 it is shown that the parameter α possesses the intrinsic diversity ordering property. Finally, we mention a different approach to diversity analysis taken in ref. 14. See RAO’S AXIOMATIZATION OF DIVERSITY MEASURES. REFERENCES 1. Dennis, B., Patil, G. P., Rossi, O., Stehman, S., and Taillie, C. (1979). In Ecological Diversity in Theory and Practice, J. F. Grassle, G. P. Patil, W. Smith, and C. Taillie, eds. International Co-operative Publishing House, Fairland, Md. (Extensive bibliography of applications of diversity in ecology.) 2. Dennis, B., Patil, G. P., and Rossi, O. (1979). In Environmental Biomonitoring, Assessment, Prediction, and Management, J. Cairns, G. P. Patil, and W. E. Waters, eds. International Co-operative Publishing House, Fairland, Md. 3. Hill, M. O. (1973). Ecology, 54, 427–432. 4. Hurlbert, S. H. (1971). Ecology, 52, 577–586. 5. Kempton, R. A. and Taylor, L. R. (1974). J. Anim. Ecol., 43, 381–399. 6. Kempton, R. A. and Taylor, L. R. (1976). Nature (Lond.), 262, 818–820. 7. Kempton, R. A. and Wedderburn, R. (1978). Biometrics, 34, 25–37. 8. Lambshead, P. J. D., Platt, H. M. and Shaw, K. M. (1983). J. Nat. Hist., 17, 859–874. 9. Patil, G. P. and Taillie, C. (1976). In Proc. 9th Int. Biom. Conf., Vol. II. The Biometric Society, Washington, D.C. 10. Patil, G. P. and Taillie, C. (1979). In Ecological Diversity in Theory and Practice, J. F. Grassle, G. P. Patil, W. Smith, and C. Taillie, eds. International Co-operative Publishing House, Fairland, Md. 11. Patil, G. P. and Taillie, C. (1979). In Contemporary Quantitative Ecology and Related

Econometrics, G. P. Patil and M. Rosenzweig, eds. International Co-operative Publishing House, Fairland, Md. 12. Patil, G. P. and Taillie, C. (1982). J. Amer. Statist. Ass., 77, 548–561. (The key reference on the intrinsic diversity ordering and various diversity index families. Discussion follows, pp. 561–567.) 13. Pielou, E. C. (1975). Wiley, New York.

Ecological

Diversity.

14. Rao, C. R. (1982). Theor. Popul. Biol., 21, 24–43. 15. R´enyi, A. (1961). Proc. 6th Berkeley Symp. Math. Statist. Prob., Vol. 1. University of California Press, Berkeley, CA, pp. 547–561. 16. Smith, W. and Grassle, J. F. (1977). Biometrics, 33, 283–292. 17. Smith, W., Grassle, J. F., and Kravitz, D. (1979). In Ecological Diversity in Theory and Practice, J. F. Grassle, G. P. Patil, W. Smith, and C. Taillie, eds. International Co-operative Publishing House, Fairland, Md. 18. Solomon, D. (1979). In Ecological Diversity in Theory and Practice, J. F. Grassle, G. P. Patil, W. Smith, and C. Taillie, eds. International Co-operative Publishing House, Fairland, Md. 19. Swindel, B. F., Conde, L. F., and Smith, J. E. (1984). For. Ecol. Manag., 7. 20. Taillie, C. (1977). The Mathematical Statistics of Diversity and Abundance. Ph.D. thesis, Pennsylvania State University. 21. Taylor, C. E. and Condra, C. (1979). In Ecological Diversity in Theory and Practice, J. F. Grassle, G. P. Patil, W. Smith, and C. Taillie, eds. International Co-operative Publishing House, Fairland, Md. See also DIVERSITY INDICES and ECOLOGICAL STATISTICS.

B. DENNIS G. P. PATIL

PROGRESSIVE CENSORING SCHEMES In clinical trials∗ , survival analysis∗ , followup∗ studies, and in some problems in reliability∗ , some incompleteness and/or incompatibility of data may arise due to nonsimultaneous entry of the units into the (experimental) scheme, possible withdrawal of the units from the scheme, and more typically, due to intended termination of the

PROGRESSIVE CENSORING SCHEMES

study before all the responses become available. There may be additional complications due to incorporation of interim or repeated analysis schemes for the accumulating data (over the tenure of the study period). For example, in a clinical trial [designed to study the comparative effectiveness or performance of two (or more) drugs (or treatments) relating to a common response] yielding data on the times to failure (response) for the different units (subjects) under study, we may encounter a variety of complications due to one or more of the following factors (these call for special attention for a valid and efficient statistical analysis of the clinical trial data): 1. Censoring/truncation. Due to limitations of time, cost of experimentation, and other considerations, the clinical trial may be planned so that after a prefixed period of time, the trial may be terminated and a statistical analysis be performed on the (partial) data accumulated during this tenure of the experiment. This may be termed a truncated or Type I censored experiment. In some other situations, it may be decided to curtail experimentation after a prefixed number (or proportion) of responses becomes available. This may be termed a Type II censoring scheme. Note that in a Type I censoring scheme, the duration of the trial is prefixed, but the number of responses occurring within this period is random. In a Type II censoring scheme, this number is fixed in advance, but the duration of the trial is a random variable. In either case, at the termination of the experimentation, there may still be some surviving units whose failure times are not totally known (these are, however, known to be larger than the censoring times). These censored observations introduce some incompleteness in the response data. 2. Staggering entry. The subjects (or units) may not all be available at the beginning of experimentation, so that they may enter (possibly in batches) into the scheme at possibly different points of time. If t0 is the time point at which the study begins and t∗ the time point at which the experimentation is stopped, then for an entry at a time point t(
t0 ), the exposure time may be defined as max{0, t∗ − t}. Thus a staggering entry plan for the units results in (possibly) different exposure times for these units

6547

(under a given censoring scheme), and this introduces unequal censoring patterns. In most of the cases, this entry plan may be random in nature (e.g., patients arriving in a clinic with a specific disease), so that one would have a random censoring pattern for a staggering entry plan. 3. Withdrawal. Some of the subjects may have failures due to causes other than the specific one under study, and hence, their failure times may not throw much light on the particular cause one wants to investigate. Also, due to migration or lack of interest in the participation, other subjects may even drop out of the scheme (during the experimentation). These may be regarded as withdrawals. In this setup, withdrawals may be regarded as random, although in some other plans, planned withdrawals may also be considered as a part of the experimentation protocol. For models incorporating various withdrawal patterns (random or nonrandom in nature), some information is lost due to the withdrawal and the statistical analysis scheme for the trial may become more complicated and less efficient, too. 4. Monitoring of clinical trials. In view of the fact that clinical trials involve human subjects, based on medical ethics there may be a need to monitor experimentation from the very beginning: If any significant difference among the treatments (in relation to the responses) is detected at any intermediate stage of the trial, the trial may be stopped at that point and the surviving subjects switched to the better treatment for their benefit. From the statistical point of view, this monitoring of the trial involves interim or repeated statistical analysis on accumulating data, where in a followup study, the setup of the classical sequential analysis∗ may not apply. For a proper interpretation of progressive censoring schemes (PCSs), all the concepts listed under factors 1 to 4 need to be incorporated. The term PCS has been used in the statistical literature in more than one context. Earlier uses generally refer to a (fixed) single-point censoring scheme when there is a staggering entry plan for the units, so that one essentially has an unequal pattern of censoring (see Nelson [6]). Thus, if we have n(
1) units entering into the scheme at time

6548

PROGRESSIVE CENSORING SCHEMES

points t1 , . . . , tn , respectively (where t0  t1  · · ·  tn ), t0 being the starting point of the study, and if t∗ be any (fixed) censoring point, then, at that time point the exposure times for these n units are t∗ − t1 , . . . , t∗ − tn , respectively (assuming that t∗
tn ). For these units and under the fixed-point censoring scheme, one has to work with the minimum of the actual failure time and the corresponding exposure time, along with the information whether this corresponds to a failure or a censored event. When t1 , . . . , tn are random variables, this may also be termed random censoring. The relevance of the term progressive censoring lies in the progressive entry of the units resulting in the unequal censoring pattern for them. In the context of life testing∗ , the term progressive censoring has also been used for a scheme to further reduce the testing time (see Klein and Basu [4]). For n items under life testing, consider a set T1 , . . . , Tm of predetermined censoring times. At time Tj , a fixed number cj (
0) of the items are removed from the test, for i = 1, . . . , m − 1; at time Tm either a fixed number cm are removed from the test or the test is terminated with some units still surviving. Instead of predetermined time points T1 , . . . , Tm , one may also conceive of predetermined numbers r1 , . . . , rm such that at the rj th failure, an additional number cj of units may be removed and testing be continued, for j = 1, . . . , m, the modification for j = m being the same as in the case of Type I censoring. In the context of clinical trials with monitoring in mind, a comprehensive definition of progressive censoring schemes, due to Chatterjee and Sen [1] may be given. This extended definition of PCS incorporates statistical monitoring of clinical trials (i.e., factor 4) in a very natural way where various combinations of 1, 2, and 3 may be blended. For simplicity consider first simultaneous entry without any withdrawal. Here n(
1) units entering into the scheme at a common point of time t0 are followed through time and their failure times (along with other tagging variables) are recorded in order as they occur. Thus at any time point t(> t0 ), one has a composite picture of the failures occurring during (t0 , t], the censored observations [having failure times in (t, ∞)], and other design and

concomitant variables. In a fixed-point censoring scheme, at some t∗ (> t0 ), where t∗ may or may not be random depending on Type II or Type I censoring, based on the picture at t∗ , a statistical analysis of the survival data is to be made and conclusions are to be drawn therefrom. In most practical problems, unless the survival distribution is at least roughly known, a fixed-point censoring scheme may lead to considerable loss of efficiency relative to cost. This drawback is particularly felt in exploratory studies. A too early termination of the experiment may lead to an inadequately small set of observations and thus increases the risk of an incorrect decision, while unnecessary prolongation may lead to loss of valuable time with practically no extra gain in efficiency. Also, in a life-testing situation, continuous monitoring is usually adopted. As such, it seems unrealistic to wait until a given time period is over or a given number of failures has occurred, and then to draw inferences from the study. Naturally, one would like to review the results as the experiment progresses, and thereby have the scope for terminating the experiment at an early stage depending on the accumulated evidence from the observations. Such a scheme is termed a progressive censoring scheme. Here the censoring is made progressively over time; this flexibility would increase the efficiency relative to cost and would be particularly relevant in long-term studies of an exploratory nature. Although in a PCS, monitoring is done continuously, in many studies (particularly, in nonparametric ones) the picture may change only at the failure points, so that it amounts to reviewing statistically the picture at successive failure points. In this simultaneous entry plan, in a PCS, withdrawals can easily be accommodated by considering the response variable as the minimum of the failure time and the withdrawal time (along with the tagging variable whether it is a failure or censored observation). PCS may become somewhat more complicated for a staggering entry plan. Here monitoring may commence before all the subjects have entered into the scheme, and failures may also occur among the entries already in hand. Thus if t0 is the time point at which the study begins, and if t1 , . . . , tn

PROGRESSIVELY CENSORED DATA ANALYSIS

are the points of entries of the n units in the scheme, then at any time point t (not
max{t1 , . . . , tn }) we have n(t) entries, where n(t) is nondecreasing in t and n(t)  n, t
t0 ; for t
max(t1 , . . . , tn ), n(t) = n. Further, at the time point t, for the n(t) units having points of entries prior to t, the exposure times are possibly different. This picture is similar to the first interpretation of PCS, sketched earlier, but with sample size n(t) instead of n. The picture in the extended PCS is then obtained by varying t progressively over t
t0 ; if at any t > t0 , a statistically significant difference among the treatments is detected (based on the picture available at that time), the trial is stopped at that time along with the rejection of the null hypothesis of equality of treatments. Otherwise, the monitoring continues. In this setup, an upper bound for the duration of the trial may also be set in advance. Also, withdrawals may be accommodated in a manner similar to that in the nonstaggering entry plan. For details of this staggering entry PCS, see Majumdar and Sen [5] and Sinha and Sen [8,9]. Statistical analysis for PCS opened up a new area of active research. Although technically allied to classical sequential analysis∗ , lack of independence of the increments over time as well as the lack of stationarity of the related stochastic processes∗ make it somewhat difficult to apply the classical theory in PCS. Some of these results are discussed in Sen [7] and Tsiatis [10].

REFERENCES 1. Chatterjee, S. K. and Sen, P. K. (1973). Calcutta Statist. Ass. Bull., 22, 13–50. 2. Elandt-Johnson, R. C. and Johnson, N. L. (1980). Survival Models and Data Analysis. Wiley, New York. 3. Gross, A. J. and Clark, V. A. (1975). Survival Distributions: Reliability Applications in the Biomedical Sciences. Wiley, New York. 4. Klein, J. P. and Basu, A. P. (1981). Commun. Statist. A, 10, 2073–2100. 5. Majumdar, H. and Sen, P. K. (1978). Commun. Statist. A, 7, 349–371. 6. Nelson, W. A. 945–966.

(1972).

Technometrics,

14,

6549

7. Sen, P. K. (1981). Sequential Nonparametrics: Invariance Principles and Statistical Inference. Wiley, New York. 8. Sinha, A. N. and Sen, P. K. (1982). Sankhya¯ B, 44, 1–18. 9. Sinha, A. N. and Sen, P. K. (1983). Proc. Golden Jubilee Conf. Statist., Indian Statistical Institute, Calcutta. 10. Tsiatis, A. A. 311–315.

(1981).

Biometrika,

68,

See also CENSORED DATA; CENSORING; CLINICAL TRIALS —II; PROGRESSIVELY CENSORED DATA ANALYSIS; and SURVIVAL ANALYSIS.

P. K. SEN

PROGRESSIVELY CENSORED DATA ANALYSIS In the entry PROGRESSIVE CENSORING SCHEMES, some statistical problems associated with progressive censoring schemes (PCS) were presented. Valid and efficient statistical analysis procedures of such progressively censored data are considered here. Consider first a staggered entry but fixedpoint censoring scheme (e.g., Nelson [18]), where n items enter at time points t1  t2  · · ·  tn , respectively, and the study is terminated at a point t∗
tn . Thus the effective exposure times for the n units are Ti = t∗ − ti , i = 1, . . . , n. If Xi stands for the length of life (i.e., failure time) of the ith unit (from the entry point), then this failure occurs during the tenure of the study if ti + Xi  t∗ , and is otherwise censored. Thus, if Xi0 = min(Xi , Ti ) and δi = 1 or 0 according as Xi0 is equal to Xi or Ti , for i = 1, . . . , n, then the observable random vectors are (X10 , δ1 ), . . . , (Xn0 , δn ).

(1)

Note that if the entry points (ti ) are stochastic, then so are the Ti , so that we have a random censoring scheme; generally, in this context, we assume that Ti are independent and identically distributed random variables with a distribution function G, defined on [0, ∞). For a preplanned entry plan, the Ti are nonstochastic, and we have an unequal censoring plan. In either case, when the distributions of the Xi are of specified forms (involving some unknown parameters), and

6550

PROGRESSIVELY CENSORED DATA ANALYSIS

the same is the case with G, then the likelihood function for the observable random elements in (1) can be used to estimate these unknown parameters and to make suitable tests of significance. These parametric procedures are discussed in detail in books by Elandt-Johnson and Johnson [9], Kalbfleisch and Prentice [13], Gross and Clarke [12], and Lawless [15]. In the nonparametric case (typically in the context of testing the equality of two or more survival functions) Gehan [11], Efron [8], Peto and Peto [19], and others have advocated the use of some simple rank tests, while Breslow and Crowley [2] and others have incorporated the product limit (PL) or the socalled Kaplan-Meier∗ [14] estimators of the individual survival functions in the formulation of the test procedures. Under random censoring, if in (1) the information contained in the δi is ignored, the usual nonparametric tests based on the Xi0 remain applicable, although they may entail some loss of efficiency due to this sacrifice of information. Sen [24] has studied conditions under which such rank tests are locally optimal. In this context the proportional hazard (PH) model of Cox [5] has been used extensively (see PROPORTIONAL HAZARDS MODEL, COX’S). Under this model, the hazard function may be quite arbitrary though its dependence on the covariates is of a given parametric form. Thus such models can at best be regarded as quasinonparametric. Cox [5,6] has advocated the use of a partial likelihood∗ function which enables one to employ the classical parametric approach in a general setup. This has been followed up more thoroughly by a host of workers (see Tsiatis [31], Slud [30], Sen [22], Anderson and Gill [1], where other references are cited, and a conference proceedings on survival analysis, edited by Crowley and Johnson [7]). Let us next consider the second interpretation of PCS, due to Cohen [4] and others. Here also, for Type I or Type II censoring schemes, in the parametric case, the likelihood function can be written explicitly. The usual maximum likelihood∗ estimators and likelihood ratio tests∗ may be prescribed in the usual fashion. For exponential∗ , normal∗ , lognormal∗ , and logistic∗ distributions, such procedures are due to Cohen [4] and Gajjar

and Khatri [10], among others. In the nonparametric case, this PCS permits the permutational invariance to generate distributionfree∗ tests. However, closed expressions for the test statistics may be difficult to prescribe. Let us proceed to the broad interpretation of PCS, due to Chatterjee and Sen [3], and consider first nonstaggered entry plans. Essentially, we encounter here a timesequential problem which may be posed as follows. For n items under life testing, corresponding to the actual failure times X1 , . . . , Xn , the observable random vectors are (Zi , Si )i = 1, . . . , n, where Z1 < · · · < Zn stand for the order statistics∗ corresponding to the Xi , and Sn = (S1 , . . . , Sn ) is the vector of antiranks (i.e., XSi = Zi , i = 1, . . . , n). As we monitor the experiment from the beginning, we observe the failure points Z1 , . . . , Zn in order. Thus at the kth failure point Zk , the observed data relate to the set {(Z1 , S1 ), . . . , (Zk , Sk )} for k = 1, . . . , n.

(2)

Unfortunately, for different i, the vectors (Zi , Si ) are neither stochastically independent nor marginally identically distributed. Hence, if one attempts to adapt some sequential procedure based on the partial set in (2), difficulties may arise due to non-homogeneity and lack of independence of the increments. On the other hand, the PCS relates to looking at the data at each failure point (with a view to rejecting the null hypothesis if the outcome at that point so indicates), so that one essentially encounters a repeated significance testing problem on accumulating data where the increments are neither independent nor homogeneous. If we denote the likelihood function of the set of observed random variables {(Z1 , S1 ), . . . , (Zk , Sk )} by pnk (·; θ), where θ stands for the parameters associated with the underlying probability law, then the logarithmic derivative of pnk (·; θ ) (with respect to θ ) forms a martingale∗ array (for every n and k  n); see Sen [21] and Sen and Tsong [25]. Based on this martingale characterization, one may then construct a random function (stochastic process∗ ) from this

PROGRESSIVELY CENSORED DATA ANALYSIS

partial sequence of the derivatives of the loglikelihood function (at the successive failure points). Under fairly general regularity conditions, such a process weakly converges to a Wiener process (in the single-parameter case) or a Bessel process (in the multiparameter case). This weak convergence result enables one to use the distribution theory of suitable functionals (such as the maximum or the maximum absolute value) of the Wiener (or Bessel) process to provide largesample approximations for the critical levels of the test statistics (in the PCS) based on the partial sequence in (2). As an illustration, consider the simple model where θ is real valued (e.g., Sen [21]). Suppose that we want to test the null hypothesis H0 : θ = θ0 against one-or twosided alternatives. Let λ0nk = (d/dθ ) log pnk (·, θ )|θ =θ0 , 0 = Eθ0 (λ02 Jnk nk ), 0 = 0). Also, let k = 0, 1, . . . , n (where λ0n0 = Jn0 0 0 /Jnn , tnk = Jnk

0 −1/2 Wn (tnk ) = λ0nk {Jnn } ,

k = 0, 1, . . . , n, and by linear interpolation we complete the definition of Wn = {Wn (t) : 0  t  1}. Then under H0 and some mild regularity conditions, as n increases, sup{Wn (t) : 0  t  1} [or sup{|Wn (t)| : 0  t  1}] converges in distribution to D+ = sup{W(t) : 0  t  1} [or D = sup{|W(t)| : 0  t  1}], where W = {W(t) : 0  t  1} is a standard Wiener process on (0, 1). Hence if D+ α (or Dα ) is the upper 100α% point of the distribution of D+ (or D), then we have for large n, P{ sup Wn (t) > D+ α |H0 } → α, 0t1

P{ sup |Wn (t)| > Dα |H0 } → α.

(3)

0t1

Operationally, at each failure point Zk , we 0 )−1/2 λ0 and if it lies above D+ compute (Jnn α nk (for one-sided alternatives) or beyond ±Dα (for the two-sided case), we stop at that point along with the rejection of the null hypothesis; if not, we continue the study until the next failure and repeat the test at that point. By virtue of (3), this repeated significance testing∗ scheme has the asymptotic level of significance α. On the other hand, if the λ0nk

6551

are tested repeatedly against their marginal critical levels, the overall significance level may be much higher than the nominal significance level for each of these marginal tests. In the nonparametric case, in the context of the simple regression model containing the two-sample model as a special case, Chatterjee and Sen [3] spotted the same martingale property of a suitable version of linear rank statistics, censored at the successive failure points. Let us denote these censored linear rank statistics by Tnk , k = 0, 1, . . . , n (where Tn0 = 0) and the corresponding array of variances (under the null hypothesis H0 of the equality of all the n distributions) by Vnk , k = 0, . . . , n. Defining tnk = Vnk /Vnn , k = 0, . . . , n, 1/2 and Wn (tnk ) = Tnk /Vnn , k = 0, . . . , n, we complete the definition of Wn = {Wn (t) : 0  t  1} by linear interpolation. As in the parametric case (3) holds, so that under a PCS the overall significance level of the repeated significance test based on the Tnk will be close to α, for large n. Under H0 the Tnk are jointly distribution-free, so that for small values of n, the exact critical value for this procedure may be obtained by direct enumeration. But these computations become prohibitively laborious, so that for large n, one would naturally use (3) as a handy tool. Majumdar and Sen [17] extended the Chatterjee–Sen [3] results to the case of a multiple regression∗ model (containing the several sample model as a special case). They were able to show that for the progressively censored rank test statistics, some Bessel process approximations hold. For local alternatives, asymptotic power properties of these rank tests for the simple and multiple regression models were also studied by Chatterjee and Sen [3] and Majumdar and Sen [16]. A detailed discussion of these results is given in Sen [23]. Instead of using linear rank statistics, one may also look at the empirical distributions for the different samples (or more generally, a weighted empirical process) and employ them for testing purposes. Note that the classical Kolmogorov–Smirnov∗ statistic is usable in a PCS for testing the identity of the two underlying distributions. Sinha and Sen [26,27] have studied this problem in detail and cited additional references. Let us finally consider the staggered entry plan. As in the case of random or unequal

6552

PROGRESSIVELY CENSORED DATA ANALYSIS

censoring (treated earlier), here also the entry pattern may or may not be stochastic in nature. Although analysis based on a specific entry pattern may be made using similar martingale characterizations, one might like to adopt some procedure which remains robust against departures from a specified pattern. With this objective in mind, Sen [20] considered a two-dimensional functional central limit theorem∗ for censored linear rank statistics (where the sample size n is allowed to vary between 1 and N and the failure numbers k between 1 and n). This theorem provides a Brownian sheet approximation for the staggered entry censored linear rank statistics. This result was used by Majumdar and Sen [17] to formulate some (conservative) tests for the two-sample as well as the simple regression model, for the staggered entry PCS; these tests remain insensitive to the entry pattern. Sinha and Sen [28,29] considered the case of weighted empirical processes∗ in the staggered entry plans, and developed the tied-down Brownian sheet approximations for these processes. Their tests are also somewhat conservative in nature (due to the fact that the entry pattern is not assumed to be precisely known in advance). Similar Brownian sheet approximations work out well for the Cox [5] proportional hazard model in the staggered entry plan as well as for the parametric likelihood ratio statistics under fairly general regularity conditions. Thus we may conclude that for the PCS, where independence, homogeneity and simultaneous entry plan may not hold, the basic tool for data analysis is to incorporate martingale theory in the formulation of suitable stochastic processes converging in distribution to appropriate Gaussian ones (in a oneor more-dimensional time parameter), and to employ the distribution theory for these Gaussian processes∗ to provide large-sample approximations for the specific ones under study. This approach is dealt with elaborately in Sen [23, Chap. 11].

2. Breslow, N. and Crowley, J. (1974). Ann. Statist., 2, 437–453. 3. Chatterjee, S. K. and Sen, P. K. (1973). Calcutta Statist. Ass. Bull., 22, 13–50. 4. Cohen, A. C. 327–339.

(1963).

Technometrics,

5,

5. Cox, D. R. (1972). J. R. Statist. Soc. B, 34, 187–220. 6. Cox, D. R. (1975). Biometrika, 62, 269–276. 7. Crowley, J. and Johnson, R. A., eds. (1982). Survival Analysis. IMS Lecture Notes–Monographs, Vol. 2. Inst. Math. Statist., Hayward, CA. 8. Efron, B. (1967). Proc. 5th Berkeley Symp. Math. Statist. Prob., Vol. 4. University of California Press, Berkeley, CA, pp. 831–853. 9. Elandt-Johnson, R. C. and Johnson, N. L. (1980). Survival Models and Data Analysis. Wiley, New York. 10. Gajjar, A. V. and Khatri, C. G. (1969). Technometrics, 11, 793–803. 11. Gehan, E. A. (1965). Biometrika, 52, 650–652. 12. Gross, A. J. and Clark, V. A. (1975). Survival Distributions: Reliability Applications in the Biomedical Sciences. Wiley, New York. 13. Kalbfleisch, J. D. and Prentice, R. L. (1980). The Statistical Analysis of Failure Time Data. Wiley, New York. 14. Kaplan, E. L. and Meier, P. (1958). J. Amer. Statist. Ass., 53, 457–481. 15. Lawless, J. F. (1982). Statistical Models and Methods for Lifetime Data. Wiley, New York. 16. Majumdar, H. and Sen, P. K. J. Multivariate Anal., 8, 73–95.

(1978).

17. Majumdar, H. and Sen, P. K. (1978). Commun. Statist. A, 7, 349–371. 18. Nelson, W. 945–965.

(1972).

Technometrics,

14,

19. Peto, R. and Peto, J. (1972). J. R. Statist. Soc. A, 135, 186–206. 20. Sen, P. K. (1976). Ann. Prob., 4, 13–26. 21. Sen, P. K. (1976). Ann. Statist., 4, 1247–1257. 22. Sen, P. K. (1981). Ann. Statist., 9, 109–121. 23. Sen, P. K. (1981). Sequential Nonparametrics: Invariance Principles and Statistical Inference. Wiley, New York. 24. Sen, P. K. (1984). J. Statist. Plan. Infer., 9, 355–366.

REFERENCES 1. Anderson, P. K. and Gill, R. D. (1982). Ann. Statist., 10, 1100–1120.

25. Sen, P. K. and Tsong, Y. (1981). Metrika, 28, 165–177. 26. Sinha, A. N. and Sen, P. K. (1979). Commun. Statist. A, 8, 871–898.

PROJECTION PLOTS 27. Sinha, A. N. and Sen, P. K. (1979). Calcutta Statist. Ass. Bull., 28, 57–82. 28. Sinha, A. N. and Sen, P. K. (1982). Sankhyn¯ B, 44, 1–18. 29. Sinha, A. N. and Sen, P. K. (1983). Proc. Golden Jubilee Conf. Statist., Indian Statistical Institute, Calcutta. 30. Slud, E. V. (1982). Biometrika, 65, 547–552.

6553

of X form an orthonormal basis for X , then X X = Ir and P = XX . Succinctly, the projection of y onto X is Py for all vectors y in En . If dim(X ) < r, and x1 , . . . , xr span X but are linearly dependent, then the uniqueness property still holds, and P = X(X X)− X ,

31. Tsiatis, A. (1981). Ann. Statist., 9, 93–108. See also CENSORED DATA; GAUSSIAN PROCESSES; KAPLAN –MEIER ESTIMATOR; LINEAR RANK TESTS; PROGRESSIVE CENSORING SCHEMES; PROPORTIONAL HAZARDS MODEL, COX’S; and SURVIVAL ANALYSIS.

P. K. SEN

PROJECTION MATRICES Let X be a vector subspace of n-dimensional Euclidean space En , and let y be any vector in En . Then the vector yx is the projection or orthogonal projection of y on X if a) yx ∈ X , b) x (y − yx ) = 0 for every vector x in X . For y = 0, its projection yx on X is unique, and the vector y − yx lies in the subspace X 1 that is orthogonal to X . An equivalent definition is that every vector y in En can be expressed uniquely via y = yx + (y − yx ) where yx ∈ X is the projection of y on X and (y − yn ) ∈ X 1 . As an illustration, when En is threedimensional Euclidean space and X is a two-dimensional plane through the origin, then y − yx is the line perpendicular to X passing through y [1, Sec. 2.6]. Suppose that x1 , x2 , . . . , xr are linearly independent vectors in En and that they span X . Then the projection yx of y onto X can be expressed as yx = X(X X)−1 X y, where the columns of X(n × r) are x1 , . . . , xr . If P = X(X X)−1 X , then P is the projection matrix for the vector space X in En , and P is unique. If the columns

where (X X)− is a generalized inverse∗ of X X [2, Sec. 8.6]. Properties The projection Py of y onto the vector space X minimizes the Euclidean distance [(y − x) (y − x)]1/2 from y to x for all vectors x in X , that is, (y − Py) (y − Py)  (y − x) (y − x),

x ∈ X.

The projection matrix P is symmetric and idempotent∗ . In the general linear model∗ , y = Xβ + e,

E(e) = 0,

every least squares∗ solution βˆ for β gives rise to predicted responses yˆ = Xβˆ = X(X X)− X y = Py, where P is the projection matrix for the vector space spanned by the columns of X. P is also known as the hat matrix∗ . REFERENCES 1. Schott, J. R. (1997). Matrix Analysis for Statistics. Wiley, New York. (Secs. 2.5, 2.6.) 2. Searle, S. R. (1982). Matrix Algebra Useful for Statistics. Wiley, New York. (Sec. 8.6.) See also GENERAL LINEAR MODEL and HAT MATRIX.

PROJECTION PLOTS One of the most useful statistical techniques for interpreting bivariate data is the scatter plot∗ . It can help to identify interesting features of the data such as clusters of observations, outlying points, and possible relationships between two variables.

6554

PROJECTION PLOTS

For multivariate data with more than two variables, however, an ordinary scatter plot cannot display the entire data. Finding interesting structure in multivariate data is much more difficult; see also MULTIVARIATE GRAPHICS. There are a variety of ways to modify and enhance scatter plots for use with multivariate data. One possibility is to project the data into two dimensions and then examine a scatter plot constructed using these two dimensions (or equivalently, using these two variables). This is called a projection plot. With multivariate data there are many ways to choose the two dimensions, and projecting into them implies some loss of information from the remaining dimensions. One cannot expect interesting features of the data to show up in all projections, and there may be no single projection that reveals everything important. Some choices—such as looking at scatter plots of all pairs of the original variables—can be made without performing any calculations on the data, while other choices of dimensions can be determined from calculations performed on the specific data set. No single method will always reveal the interesting features of the data, and it is often useful to try several of the projection methods described below. PROJECTIONS ON PRECHOSEN VARIABLES Suppose that the analyst has n values (observations) on each of p variables x1 , x2 , . . . , xp , p
3, and the goal is to find scatter plots showing interesting features of the data. Consider first the ordinary scatter plot using just variables x1 and x2 , which ignores the data values on variables x3 , . . . , xp . Conceptually, the n multivariate observations can be thought of as n points lying in a p-dimensional vector space, with axes corresponding to the variables x1 , x2 , . . . , xp . The (x1 , x2 ) scatter plot corresponds to projecting the n points (orthogonally) into the x1 , x2 plane, which can also be thought of as the plane determined by the dimensions (variables) x1 and x2 . Other scatter plots will correspond to different projections. One way to construct a scatter plot that gives information on three variables rather

than just two is to change the character that is plotted. Instead of having all plotted characters being the same—such as a ∗, ·, or + as is typically done—a plot can be constructed in which the location of the character represents the (x1 , x2 ) values of an observation as usual, but the character plotted is L if the value of x3 is in its lowest quartile, M if x3 is in its middle two quartiles, and U if x3 is in its upper quartile. Clearly, there are many choices of characters for plotting and many ways of dividing the distribution on the x3 variable. What is most useful may depend on the specific application. A related idea which is possible with some plotting devices is to use only one symbol as plotting character, such as a circle, but to make its size vary depending on the value of the x3 variable. This can permit an impression of perspective on x3 ; we view the x1 x2 plane at a particular value of x3 , and points closest to this plane are plotted with the largest characters, while points farther away (in terms of the x3 variable) are plotted smaller. For this to work satisfactorily, the character size will generally have to be some nonlinear function of x3 , so that no points are so big as to completely dominate the plot or so small as to be invisible. Typically, iteration is necessary to produce a satisfying and informative plot. Another way of introducing x3 is to plot the points with different colors, depending on the value of x3 . This idea does not seem to have been used or evaluated very much, possibly because the technology permitting it is still new. There is no reason to examine only a single scatter plot. One way to construct multiple plots is to look at two variables x1 and x2 using different subsets of the observations. Using the ideas behind the preceding three paragraphs, we can first partition the observations using variable x3 , for example, into the four quartiles. Then make four scatter plots of x1 and x2 using each of these four subsets; these plots can be arranged on a page from left to right corresponding to the increase in the background variable x3 . A more complicated version of this idea is to partition the observation using two background variables, x3 and x4 , and then arrange the scatter plots of x1 and x2 for the resulting subsets in a rectangular array corresponding

PROJECTION PLOTS

to the value on x3 and x4 . In effect, this is a scatter plot of scatter plots, incorporating four variables in all. A different approach toward constructing multiple plots is to juxtapose scatter plots involving different variables in some useful way, where now each scatter plot includes different variables but all the observations. Several ways to do this, as well as ways to implement the ideas discussed previously, are described by Tukey and Tukey [7, Chap. 10; Chap. 11, pp. 231–232; Chap. 12, pp. 245–260].

6555

One way to arrange the scatter plots from all pairs of the four variables x1 , x2 , x3 , and x4 is shown in Fig. 1. For this example there appear to be positive relationships between the pairs of variables. None of these six plots indicates, however, that these 50 observations may consist of two separated groups. Now consider Fig. 2, constructed using y1 = (x1 + x2 + x3 + x4 )/2 and y2 = (x1 + x2 − x3 − x4 )/2. This is an orthogonal projection into two dimensions that do not correspond to any of the original four variables; it does suggest that these data may separate nicely

Figure 1.

6556

PROJECTION PLOTS

Figure 2.

into two groups, those with y1 < 2 and those with y1 > 2. This example points out that it can be desirable to view many different projections of the data. Simply viewing projections using each variable marginally does not guarantee finding whatever underlying structure that there may be. [The example was constructed to illustrate this point by using two multivariate normal populations with identity covariance matrices, one centered at (0,0,0,0) and the other at (2,2,2,2).] With more variables the number of scatter plots that could be examined increases greatly. Unfortunately, the chance that a prechosen projection will be sufficiently close to the ‘‘right’’ projection (assuming that there is such a projection that will reveal the structure) decreases dramatically with the number of variables. The result is that when there are a moderate to large number of variables—say, four or more—simply relying on some small number of prechosen projections is not likely to suffice to reveal the structure. Fortunately, calculations are available which can help to find useful projections. PROJECTIONS FROM MULTIVARIATE METHODS Calculations from several of the classical multivariate analysis methods give

projections which can be useful for displaying the data, although obtaining the projections need not be the only or main reason for performing these analyses. These methods, which are interrelated, include component analysis∗ (often called principal component analysis), discriminant analysis∗ , and canonical analysis∗ (canonical correlations). Component analysis can be helpful if we are treating the data as one set of observations measured on p variables. Component analysis gives those projections which preserve interpoint distances, based on distances between all pairs of observations using all p variables, as well as possible. Thus a plot of the observations projected on the first two principal components is often used to give a first, overall, two-dimensional picture of multivariate data. If the data set consists of two or more samples which are assumed Gaussian with equal covariance matrices, and the goal is to find the projections which minimize the error rates of classification, then the appropriate theory is that of linear discriminant analysis. If there are three or more groups, a plot of the data projected on the first two linear discriminant functions can help to understand relationships among the groups. Canonical analysis is designed for the situation of two sets of continuous variables and one underlying population. Again, projecting

PROJECTION PLOTS

the observations on pairs of canonical variates can help to indicate relationships. The preceding methods can all be obtained from classical multivariate normality assumptions, and all involve linear algebra eigenvector calculations. With recent advances in computational power and interest in methods that are not so closely tied to specific distributional and model assumptions, several approaches have been investigated which can give useful projections for interpreting data. The basic idea is to define a criterion that measures to what extent a given projection is ‘‘interesting,’’ and then to use numerical methods to seek those projections that maximize the criterion. Both parts of the problem can be approached in different ways [6,3]. In choosing the criterion we do not want a projection in which the observations have a more-or-less uniform scatter, but instead one in which they are clumped (clotted or clustered). There may be several local maxima to the criterion, corresponding to different sorts of structure in the data. These might all be of interest, so it can be desirable to have numerical procedures which find more than a single local maximum. The use of this idea to fit a multiple regression model was given by Friedman and Stuetzle [2]. Work in a similar spirit has been developed for cluster analysis. In clustering, generally some complicated numerical algorithm has been used to group the observations into clusters; the analyst might then need to examine various plots to interpret the results and to study relationships between and within the clusters. But, as with other multivariate data, it is not clear which projections are useful for studying the possible clusters. Gnanadesikan et al. [4] modified some of the component analysis and discriminant analysis calculations in order to produce projection plots which give as uncluttered a view of the observations and clusters as possible, by finding those projections which separate things maximally. Unlike the methods mentioned earlier in this section, these do not require iterative calculations. Instead of having the computer choose one or a small number of projection plots to examine by maximizing some criterion, a next step is to make even more use of increased

6557

computing power by showing many projection plots successively in real time, generating movie-like effects. This can now be done on certain specially designed interactive computer graphic systems. A specific projection (involving two dimensions) is shown on a CRT screen; then a third dimension is considered and the data are rotated within this subspace. This enables the user to perceive a spatial three-dimensional picture of the data, in terms of these dimensions. The rotations and dimensions can be controlled by the user. The analyst can see, quickly and interactively, many views of the data. An early such system was developed by Tukey et al. [8]; some recent work and ideas are described by Donoho et al. [1]. Such systems have not yet seen wide use, partly because of their expense; if they do become widely available, they could have a tremendous impact on analysis of multivariate data. OTHER RELATED METHODS Several graphical ways to display multivariate data have been devised which do not amount to projection plots. Instead, each multivariate data point is mapped into some appropriate symbol, and the symbols are displayed. Nearby data points should be mapped into symbols that are similar. By comparing the symbols, the analyst may be able to discover interesting relationships among the data points and variables. For Chernoff faces∗ , the symbol is a cartoon face; for Andrews function plots∗ , it is a continuous function defined on the interval (−π , π ). Other symbols that have been used include star-shaped figures and trees and castles [5]. Such methods are good for giving general impressions involving all the variables, but it often seems that specific features and details are more apparent in appropriately chosen projection plots. (It can be difficult, of course, to find the appropriate projection plots.) The biplot∗ is another graphical method, useful for displaying certain kinds of multivariate data. REFERENCES 1. Donoho, D., Huber, P. J., and Thoma, H. M. (1981). In Computer Science and Statistics:

6558

2. 3. 4.

5. 6.

7.

8.

PROJECTION PURSUIT

Proceedings of the 13th Symposium on the Interface, W. F. Eddy, ed. Springer-Verlag, New York, pp. 274–278. Friedman, J. H. and Stuetzle, W. (1981). J. Amer. Statist. Ass., 76, 817–823. Friedman, J. H. and Tukey, J. W. (1974). IEEE Trans. Computers, C-23, 881–890. Gnanadesikan, R., Kettenring, J. R., and Landwehr, J. M. (1982). In Statistics and Probability: Essays in Honor of C. R. Rao, G. Kallianpur, P. R. Krishnaiah, and J. K. Ghosh, eds. North-Holland, New York, pp. 269–280. Kleiner, B. and Hartigan, J. A. (1981). J. Amer. Statist. Ass., 76, 260–269. Kruskal, J. B. (1969). In Statistical Computation, R. C. Milton and J. A. Nelder, eds. Academic Press, New York. Tukey, P. A. and Tukey, J. W. (1981). In Interpreting Multivariate Data, V. Barnett, ed. Wiley, Chichester, England. Tukey, J. W., Friedman, J. H., and Fisherkeller, M. A. (1976). Proc. 4th Int. Conf. Stereol., Gaithersburg, Md.

See also ANDREWS FUNCTION PLOTS; BIPLOTS; CHERNOFF FACES; GRAPHICAL REPRESENTATION, COMPUTER AIDED; GRAPHICAL REPRESENTATION OF DATA; MULTIVARIATE ANALYSIS; MULTIVARIATE GRAPHICS; and TRIPLE SCATTER PLOT.

JAMES M. LANDWEHR

PROJECTION PURSUIT Many data sets are high-dimensional. It has been a common practice to use lowerdimensional linear projections of the data for visual inspection. The lower dimension is usually 1 or 2 (or maybe 3). More precisely, if X 1 , . . . , X n ∈ Rp are p-dimensional data, then for k < p, a k-dimensional linear projection is Z1 , . . . , Zn ∈ Rk where Zi = α T X i for some p × k orthonormal matrix α (so that α T α = I k , the k-dimensional identity matrix). When k = 1, α may be called a direction and the structure of the projected data can be viewed through a histogram∗ ; when k = 2, the structure can be inspected through its scatter plot∗ ; when k = 3, it can be comprehended by spinning a three-dimensional scatter plot. Since there are infinitely many projections from a higher dimension to a lower dimension, it is important to have a technique for pursuing a finite sequence of projections that can reveal the most interesting

structures of the data. The idea of combining both projection and pursuit originates from Kruskal [18] and Switzer [27]. However, the first successful implementation of the idea was by Friedman and Tukey [10], who also suggested the felicitous name projection pursuit (PP). A unified mathematical notion of PP was introduced by Huber [14], which provided the basis for further statistical research in the area. More recent papers include [7,11,22,24,25], and also [2,19,23]. TWO ELEMENTS The two basic elements of projection pursuit are a PP index and a PP algorithm. A PP index I(α) is a measure of how interesting the projection of the data by α is, where I(α) = I(α|X) implicitly depends on the data X = (X 1 , . . . , X n ). The larger the index value, the more interesting is the projection. A PP algorithm is a numerical optimization algorithm that maximizes the index over all possible α. Since there is no perfect optimizer, one hopes that the first several maxima found by a PP algorithm will provide the most interesting projections. Using a PP algorithm to seek a sequence α 1 , α 2 , . . . (which locally maximize a PP index) and then viewing structures by projecting the data to the space spanned by the columns of these α i (i = 1, 2, . . .) may be called automatic PP. This is in contrast to manual PP, in which a dynamic graphical system is employed and the interesting projections are sought by human eyes via spinning the projected data, as in Prim-9 (Tukey et al. [28]) and the grand tour (Asimov [1]). Manual PP depends on experience and great patience in a user. It can be infeasible for a nonexpert or if the data dimension is large. A more practical solution may be to combine automatic and manual PP in a suitable way, as attempted in XGobi (Swayne et al. [26]). This entry only concerns the automatic PP, as it is the PP that is used today. Clearly, nonlinear structures such as clusters, separations, and unexpected shapes are interesting. In general, the phrase ‘‘nonlinear structures’’ refers to those structures which cannot be detected easily by using a sample mean and covariance matrix. These summary

PROJECTION PURSUIT

statistics reveal linear structures such as the location, scales, and correlations of a data set. Thus a reasonable PP index should, in the first place, be location-, scale-, and rotationinvariant, i.e., I(·|sAX + β) = I(·|X)

(1)

for any s > 0, β ∈ Rp , and p × p orthonormal matrix A. This invariance can be achieved by sphering data before implementing PP. The ˜ i from X i satisfy sphered data X 1 ˜ X i = 0, n n

i=1

1  ˜ ˜T X iX i = Ip. n n

(2)

i=1

See Friedman [7] for a simple sphering scheme, and Huber [14] on the importance of sphering. How should one choose a PP index? The interestingness of a projection is given by the value of a PP index, which is a sample version (or estimate) of a distance between the distribution of the projected data and an uninteresting distribution. So, heuristically, the maximum of the index corresponds to the most interesting projection. A natural uninteresting distribution is the normal distribution. Indeed, (Diaconis and Freedman [3]), . . ., a projection will be uninteresting if it is random or unstructured. One standard measure of randomness is entropy∗ . [Thus] a projection is interesting if it has small entropy relative to other projections. . . . If the scale is fixed, maximum entropy∗ is attained by the Gaussian distribution. This suggests . . . a projection is interesting if it is far from Gaussian.

Huber [14], Friedman [7], Jones and Sibson [17], Hall [11], Cook et al. [2], and Li and Cheng [19] all considered PP indices that use the standard normal distribution as the uninteresting distribution. (However, the indices can easily be adapted to other uninteresting distributions.) Example 1. Legendre Index. Assume for the moment that E(X i ) = 0 and Cov[X i ] = I p . (Let k = 1 for simplicity.) Friedman [7] constructed his Legendre index based on the L2 distance between the densities of the projected data point Zi = α T X i and a standard

6559

normal random variable Z0 after the transformation R = 2(Z) − 1. Here (z) is the distribution function of Z0 , and the L2 distance between any two functions f and g  is [f (x) − g(x)]2 dx}1/2 . Since polynomials are inexpensive to compute, Friedman expanded the L2 distance by an m-term Legendre polynomial expansion: m 

{Eα [aj (R)]}2 ,

(3)

j=1

where Eα is the expectation with respect to the distribution of R for the fixed α, and aj (r) = (j + 1/2)1/2 Lj (r) is the jth orthonormal Legendre polynomial, i.e., 1 otherwise. −1 ai (r)aj (r)dr = 1 if i = j and 0 In practice, E[X i ] and Cov[X i ] are arbitrary and the distribution of X i is unknown. So the m-term Legendre index is simply an estimate ˜ i from X i : of (3) based on the sphered data X F Im (α)

m  2j + 1 = 2 j=1



1 ˜ i) L j (R n

˜ i ) − 1, ˜ i = 2(α T X R

n

2 ,

i=1

(4)

˜ denotes the transformation calcuwhere R lated from sphered data. (When k = 2, a twodimensional Legendre index can be obtained in the same way.) Example 2. Hermite Index. Hall [11] suggested an index differing from Friedman’s in that he used the first m-term Hermite polynomial expansion of the L2 distance between the densities of Zi = α T X i and the standard normal random variable directly. Again, we have taken k = 1 for simplicity. Most PP indices are derived this way: Find an estimate of some measure of the interestingness assuming E[X i ] = 0, Cov[X i ] = I p , and then take the estimate as a PP index ˜ i . In this way, the but with X i replaced by X invariance in (1) is also enforced. However, the sphered data satisfy the constraints (2), which makes the first two terms of Legendre, Hermite, and other polynomial indices much smaller than the other terms (cf. Sun [25], who suggested using 3  m  6 in polynomial indices).

6560

PROJECTION PURSUIT

Sensitivity to a particular nonlinear structure varies from one index to another. The number of terms m and the lower dimension k play nonnegligible roles in the sensitivity, too. There are some comparisons on the sensitivity in Sun [25], based on Friedman and Hall’s indices, and in Cook et al. [2], based on the population version of the indices. More studies of other indices (taking into account the fact that the indices are functions of sphered data) are yet to come. A beginner is advised to try a couple of indices. Since PP is implemented via an algorithm, the algorithm should do what it is supposed to do. Huber [15] contains an excellent discussion on ‘‘goals versus algorithms.’’ Friedman [7] gives a practical PP algorithm for his PP index, in which he uses a clever idea called structure removal; the idea is to transform the data along an ‘‘interesting’’ direction into standard normal data while keeping the structure along the other orthogonal directions unchanged. It has the effect that interesting directions already found will not be rediscovered, as normal structures are uninteresting and will therefore minimize the index. Sun [25] found that the largest of the first p maxima from Friedman’s PP algorithm is often very close to the global maximum, and based on this, she suggested a more exhaustive search algorithm. SIGNIFICANCE LEVEL After an interesting structure is found, it is useful to make a significance test∗ to decide whether the apparent structure is real or just the effect of noise. Sun [24] derived a theoretical approximation for the significance level (or P-value∗ ) associated with Friedman’s onedimensional PP index [7]:

F (α)
I0 P max Im α:α T α=1

≈ κ1 ψ1 (nI0 ) + κ2 ψ2 (nI0 ),

(5)

where I0 is the observed value of the index, κ1 and κ2 are constants, ψ1 and ψ2 are incomplete gamma functions depending on p and m, and the probability is found under the null hypothesis that the data are Gaussian (i.e., uninteresting). The formula (5) works well

when nI0 is not too small. Only the first term in (5) should be used if n is not large and I0 is small. So a rule of thumb is to calculate the P-value based on this formula and adjust the value if circumstances require. A P-value like this helps ease Miller’s concern in his discussion of Huber [14]: With reference to this new technology [PP], I feel that there is a gap developing between practice and theory. While interesting shapes, clusters, and separations are now more discoverable in high dimensional data, how do we evaluate whether these are real or merely random variation? [20]

P-values for other indices and higher dimensions (k
2) are still to be studied. Zhang et al. [29] and Li and Cheng [19] are good references for bounds to the P-values associated with indices based on Kolmogorov distance (rather than the L2 distance). Based on the heuristics that a projection is uninteresting if it is random or presents normal structure, a PP index essentially amounts to testing nonnormality (cf. Huber [14]). Nevertheless, there are some subtle differences between PP and a test of multivariate normality∗ . PP aims at finding the most informative multiple views of a multivariate data set. The projections obtained from a good PP procedure usually correspond to the first several largest local maxima of the index. In testing multivariate normality, however, only the largest maximum is considered. With regard to a particular projection found by a PP algorithm, the P-value in (5) gives us an indication of whether the direction is interesting. With regard to the whole process, the P-value may be conservative, as the projection by α may only correspond to local maximum of the index. A by-product of the P-value is that it helps a user decide when to stop a PP algorithm. Consecutive large P-values indicate that there might not be much interesting structure left. In summary, with a PP index, an algorithm, and the aid of P-values, a user may be able to identify and view interesting structures by projecting a data set to some lower dimensional spaces determined by α 1 , α 2 , . . . that are found sequentially by the algorithm which maximizes the index. An artificial

PROJECTION PURSUIT

example serves as an illustration of this point. Example 3. Consider a nine-dimensional data set X 1 , . . . , X 1000 of size 1000, where the fifth coordinate is from an interesting bimodal distribution, say test scores of people from different backgrounds, while the other coordinates are noise, say other irrelevant scores and measurements. Suppose a user does not know the structure of the data and applies the PP procedure. If one of the first several α i found by the PP algorithm is close to e5 ( or − e5 ), a basis vector along the fifth coordinate, then the P-value is small in this direction and large in the other α i . Thus the user would be able to make a correct decision in classifying the data and identifying the structure. APPLICATIONS PP is a powerful tool for seeking nonlinear structures of high-dimensional data. Its ability to overcome the curse of dimensionality∗ by lifting one or two dimensional techniques to high dimensions does not stop with exploratory data analysis∗ . For example, Friedman et al. [9] and Friedman and Stuetzle [8] applied PP in density ∗ estimation∗ , classification , and regression∗ problems (PPD, PPC, and PPR, respectively). More recent applications can be found in [5,19,21] and the references therein. Projection Pursuit Regression In a typical regression model, (X, Y) is an observable pair of random variables from a distribution F, where X ∈ Rp is a pdimensional variable (called a predictor) and Y ∈ R is a response. The goal is to estimate the regression function f (x) = E[Y|X = x], or the conditional expectation of Y given X = x, using a random sample (X 1 , Y1 ), . . . , (X n , Yn ) from F. PPR approximates the regression function f (x) by a finite sum of ridge functions fˆ (x) =

m 

gk (α Tk x)

(6)

k=1

where m is often determined adaptively, and the α k are p × k orthonormal matrices.

6561

The PPR model (6) is quite general. It can be used to closely approximate a large class of functions f as m → ∞ with suitable choice of α k and gk . For instance, the following models may be considered as special cases of the PPR model: 1. the simple linear regression: f (x) = α T x with k = p = 1; 2. the single-index model: f (x) = g(aT x) (cf. [12]); model with 3. the neural network∗  T one hidden layer: f (x) = m k=1 gk (ak x), where gk is a known function [usually the sigmoidal function gk (x) = 1/(1 + e−x )]; and p 4. the additive model: f (x) = k=1 gk (xk ), where xk is the kth coordinate of x (cf. [13]). Other standard models also fit into the PPR framework. A mathematical justification for (6) in terms of approximation theory is presented by Diaconis and Shahshahani [4]. A stepwise PPR algorithm can be found in Friedman and Stuetzle [8], where the gk are found by a smoothing operation. Specifically, given g(0) = 0, for j
1, the algorithm iteratively estimates the α j so as to maximize an index and the gj by a low-dimensional nonparametric regression∗ estimate based on the pro(j) (j) (j) (j) , rn ), where Zi jected data (Z1 , r1 ), . . . , (Z(j) j−1 n (j) T T = α j X i and ri = Yi − k=1 gk (α k X i ) are the residuals at the jth step (where an empty sum is to be interpreted as zero), i = 1, . . . , n. This alternating estimation is performed, sequentially, foreach k until the residual sum of squares r2i is less than a predetermined value. A different smoother for gk , or index, or fitting order may be used, yielding a different PPR algorithm. Friedman [6], Johansen and Johnstone [16], and Roosen and Hastie [23] gave various candidates. Projection Pursuit Density Estimation The idea in this application is to approximate a high-dimensional (dimension p) data density f (x) of X ∈ Rp by g(m) (x) = g0 (x)

m  i=1

gi (zi ),

6562

PROJECTION PURSUIT

where zi = α Ti x ∈ Rk for a suitable choice of {α i , i = 1, . . . , m}. Here g0 is an uninteresting density or initial model; the gi are low-dimensional densities based on interesting projections of data. Huber [14, p. 458] shows that if g is any approximation to f , then the biggest possible improvement in relative entropy∗ E(f , g) := Ef log{f (X)/g(X)} that can be achieved through replacing g(x) by g∗ (x) = g(x)g1 (α T x) is g1 (α T x) = fα (α T x)/gα (α T x), where Ef is the expectation with respect to f , and fα and gα are marginal densities of Z = α T X under f and g, respectively. Among all choices of α, the largest improvement results from one maximizing E(fα , gα ). A typical g0 is the normal density with mean and covariance estimated by the sample mean and covariance of a random sample X 1 , . . . , X n from f . This suggests a stepwise PPD algorithm as follows. 1. Let g(0) = g0 . 2. Given g(l−1) , find αˆ that maximizes the index I(α) =

n fˆα (Zi ) 1 , log (l−1) ˆ n gα (Zi )

(7)

i=1

where fˆα is a marginal density estimate a denbased on Z1 , . . . , Zn , and gˆ (1−1) α sity estimate based on the projection to α of a Monte Carlo random sample from g(l−1) .

Friedman [7] approximated the relative entropy E(gα , fα )[not E(fα , gα )] by the weighted L2 distance [fα (z) − gα (z)]2 /gα (z)dz, and (Zi ) by a finite Legendre polynofˆα (Zi )/g(l−1) α mial expansion; he hence developed a different PPD algorithm. Note that there is an asymmetry in relative entropy since E(f , g) = E(g, f ). Friedman’s approximation is valid if gα and fα are close, but not necessarily when they are distant. Thus, the justification above from Huber [14] cannot be carried over to this procedure. The justification for the Friedman’s procedure in terms of approximation theory remains an open problem. CONCLUSIONS Projection pursuit seeks interesting structures of a data set through linear projections of the data. A common feature of PP, PPC, PPD, and PPR is to use an algorithm stepwise in searching over low-dimensional projections to maximize a PP index [e.g., (4) and (7)] that is sensitive to a particular structure. Projection pursuit belongs to the class of modern statistical computing techniques. With the development of statistical software∗ , it has become a useful data analysis tool. As with any modern technique, it should be used in conjunction with (and after) some simple preliminary data analysis; this would find simple structures using the summary statistics (e.g., the mean and covariances), perform a preanalytical dimension reduction via some standard procedure, and (for PP in particular) sphere the data. REFERENCES

3. Set g1 (αˆ x) = T

fˆαˆ (αˆ T x) gˆ (l−1) (αˆ T x) αˆ

,

g(l) (x) = g(l−1) (x)gl (αˆ T x). Then repeat steps 2 and 3 until maxα I(α) is less than a preselected small value. Huber [14] also showed that if f is a convolution∗ of some density f and the standard normal density φ then the g(i) obtained above converges uniformly and in L1 norm (integrated absolute deviation) to f .

1. Asimov, D. (1985). The grand tour: a tool for viewing multidimensional data. SIAM J. Sci. and Statist. Comput., 6, 128–143. (Manual PP.) 2. Cook, D. Buja, A., and Cabrera, J. (1993). Projection pursuit indexes based on orthonormal function expansions. J. Comput. and Graphical Statist., 2, 225–250. (Gives global maxima of the population version of three polynomialbased PP indices. There are some open problems here: Is the global maximum unique? What is the global maximum of the indices, and how is this captured in the sample version?)

PROJECTION PURSUIT 3. Diaconis, P. and Freedman, D. (1984). Asymptotics of graphical projection pursuit. Ann. Statist., 12, 793–815. (Motivates what uninteresting projections/distributions are.) 4. Diaconis, P. and Shahshahani, M. (1984). On nonlinear functions of linear combinations. SIAM J. Sci. and Statist. Comput., 5, 175–191. (A piece of theoretical work validating PPR modeling in terms of approximation theory.) 5. Duan, N. (1990). The adjoint projection pursuit regression. J. Amer. Statist. Ass., 85, 1029–1038. (A variant of PP.) 6. Friedman, J. H. (1984). Smart user’s guide, Tech. Rep., Standford University. (A different PPR algorithm from that in ref. [8].) 7. Friedman, J. H. (1987). Exploratory projection pursuit. J. Amer. Statist. Ass., 82, 249–266. (The first of several PP indices based on orthogonal polynomial expansions. The idea of ‘‘structure removal’’ was proposed here.) 8. Friedman, J. H. and Stuetzle, W. (1981). Projection pursuit regression. J. Amer. Statist. Ass., 76, 817–823. (The original work on PPR.) 9. Friedman, J. H., Stuetzle, W., and Schroeder, A. (1984). Projection pursuit density estimation. J. Amer. Statist. Ass., 79, 599–608. (An application of PP to density estimation.) 10. Friedman, J. H. and Tukey, J. (1974). A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput., C-23, 881–889. (The first successful PP implementation.) 11. Hall, P. (1989). Polynomial based projection pursuit. Ann. Statist., 17, 589–605. (A generalization from Legendre index [7] to Hermite index.) ¨ 12. Hardle, W., Hall, P., and Ichimura, H. (1993). Optimal smoothing in single-index models. Ann. Statist., 21, 157–178. (Discusses how to choose the smoothing parameter in the singleindex model and its difference from that in PPR.) 13. Hastie, T. and Tibshirani, R. (1987). Generalized additive models: some applications. J. Amer. Statist. Ass., 82, 371–386. 14. Huber, P. (1985). Projection pursuit (with discussion). Ann. Statist., 13, 435–475. (An excellent review article which gives theoretical justifications for some seemingly ad hoc PP ideas.) 15. Huber, P. (1992). Goals versus algorithms. In Data Analysis and Statistical Inference, S.

6563

Schach and G. Trenkler, eds. Josef Eul Publishers, Germany, pp. 567–579. 16. Johansen, S. and Johnstone, I. (1990). Hotelling’s theorem on the volume of tubes: some illustrations in simultaneous inference and data analysis. Ann. Statist., 18, 652–684. (Shows how the powerful tube formula can be applied to complicated problems like PPR.) 17. Jones, M. C. and Sibson, R. (1987). What is projection pursuit? (with discussion). J. Roy. Statist. Soc. A, 150, 1–36. (Another review article, containing some parallel findings to that in [14].) 18. Kruskal, J. B. (1969). Toward a practical method which helps uncover the structure of a set of multivariate observations by finding the linear transformation which optimizes a new ‘‘index of condensation’’. In Statistical Computation, R. Milton and J. Nelder, eds. Academic Press, NewYork, pp. 427–440. (This seems to be the earliest work using PP.) 19. Li, G. and Cheng, P. (1993). Some recent developments in projection pursuit in China. Statist. Sinica, 3, 35–51. 20. Miller, R. G., Jr. (1985). Discussion. Ann. Statist., 13, 510–512. (Referes to Huber [14].) 21. Maller, R. A. (1989). Some consistency results on projection pursuit estimators of location and scale. Can. J. Statist., 17, 81–90. (An interesting application.) 22. Morton, S. C. (1990). Interpretable projection pursuit. Computer Science and Statistics: Proc. Symp. on the Interface. American Statistical Association, Alexandria, Virginia, pp. 470–474. (Provides a different perspective on PP—interpretability.) 23. Roosen, C. and Hastie, T. (1994). Automatic smoothing spline projection pursuit. J. Comput. and Graphical Statist., 3, 235–248. (A PPR algorithm.) 24. Sun, J. (1991). Significance levels in exploratory projection pursuit. Biometrika, 78(4), 759–769. (Gives P-values associated with the Legendre index [7]. The method can be generalized to all polynomial-based PP indices.) 25. Sun, J. (1993). Some practical aspects of exploratory projection pursuit. SIAM J. Sci. and Statist. Comput., 14(1), 68–80. (Contains discussions on choosing PP indices and algorithms.) 26. Swayne, D. F., Cook, D., and Buja, A. (1991). XGobi: Interactive dynamic graphics in the X Window system with a link to S. ASA Proc. Section on Statistical Graphics, pp. 1–8. (An

6564

PROJECTION PURSUIT, NASON’S t -INDEX FOR

attempt to mix manual and automatic PP. A fine graphical tool for viewing/analyzing data.) 27. Switzer, P. (1970). Numerical classification. In Geostatistics. Plenum, New York. (Another historical reference.) 28. Tukey, J. W., Friedman, J. H., and Fisherkeller, M. A. (1988). Prim-9, an interactive multidimensional data display and analysis system. In Dynamic Graphics for Statistics, W. S. Cleveland and M. E. McGill, eds. Wadsworth, New York, pp. 91–110. (Manual PP.) 29. Zhang, J., Zhu, L., and Cheng, P. (1993). Exponential bounds for the uniform deviation of a kind of empirical process, II. J. Multivariate Anal., 47, 250–268. (A theoretical bound for the P-value associated with a Kolmogorov-type index.)

a simple quadratic polynomial with no linear term, where Dn is a constant depending only on n. (Recall that the F-divergence from a probability density function f defined on Euclidean p-space Rp to a probability density function g defined on Rp is
FF (f |g) =

f (x)F[f (x)/g(x)]dx,

where F is a function from [0, ∞) into R, which is continuous and convex on [0, ∞), finite on (0, ∞), and strictly convex at some point x, 0 < x < ∞.) REFERENCES

See also COMPUTER-INTENSIVE STATISTICAL METHODS; EXPLORATORY DATA ANALYSIS; GENERALIZED ADDITIVE MODELS; and NONPARAMETRIC REGRESSION.

JIAYANG SUN

PROJECTION PURSUIT, NASON’S t -INDEX FOR [This entry presupposes some familiarity with the content of the entry PROJECTION PURSUIT.] Nason [1,2] points out that the functional
J(f ) = − f (x)1−2/(n+1) dx is minimized among all sphered distributions* by the sphered Student’s t-density tn (·) with n degrees of freedom given by 

1 (n + 1) tn (x) = [(n − 2)π ]−1/2  2 &−(n+1)/2  & 1 x2 , ( n) 1 + 2 n−2 and proposes the t-index of projection pursuit J(f ) − J(tn )(
0), which can also be represented as the sum of two F-divergences (defined presently) for spherical densities f . This is connected with the property of tn (x): tn (x)−2/(n+1) = Dn [1 + (n − 2)−1 x2 ],

1. Nason, G. P. (1992). Design and Choice of Projection Indices. Ph.D. Dissertation. University of Bath, Bath, U.K. 2. Nason, G. P. (2001). Robust projection indices. J. R. Stat. Soc. B, 63, 557–567. See also J-DIVERGENCES AND RELATED CONCEPTS and PROJECTION PURSUIT.

PROJECTION PURSUIT, YENYUKOV’S INDEX Yenyukov [1,2] proposes indices for projection pursuit* based on the observation that in a planar region A, homogeneity of data can be interpreted as either uniformity or complete spatial randomness*. Thus, if A denotes the area of A, a homogeneous planar Poisson process is characterized by the properties that the number N(A) of events in any region A follows a Poisson distribution * with mean λA and that for N(A) fixed, the events in A constitute of a random sample from the uniform distribution over the region A. Under these assumptions, one of Yenyukov’s projection pursuit indices is Ypp = D/d, where D is the mean of the interpoint distances and d the average nearest neighbor* distance. For projections with fractal∗ and

PROPHET INEQUALITIES

cluster structures, the index should be large; it should be small for homogeneous structures. No practical examples have yet been provided. Other Yenyukov indices [2] are based on the inverse studentized range* and on normal scores*. REFERENCES 1. Yenyukov, I. S. (1988). ‘‘Detecting Structures by Means of Projection Pursuit’’. Proceedings of COMPSTAT 1988. Physica-Verlag, Heidelberg, Vienna, pp. 47–58. 2. Yenyukov, I. S. (1989). ‘‘Indices for Projection Pursuit’’. In Data Analysis Learning Symbolic and Numeric Knowledge, Diday, ed. Nova Science Publishers, New York. See also NEAREST-NEIGHBOR METHODS and PROJECTION PURSUIT.

PROPAGATION OF ERROR. See ERROR ANALYSIS; LAWS CONCEPT

OF

ERROR —I: DEVELOPMENT

OF

THE

PROPENSITY SCORE This is a parameter that describes one aspect of the organization of a clinical trial∗ . It is an estimate of the conditional probability of assignment to a particular treatment given a vector of values of concomitant variables∗ . Rosenbaum [1] and Rosenbaum and Rubin [2,3] discuss the use of propensity scores in correcting the analysis of observational∗ data to allow for possible biases in the assignment of individuals to treatments as indicated by the propensity scores.

6565

PROPHET INEQUALITIES Prophet inequalities are related to optimal stopping rules∗ . They compare the best return of an optimal player (the ‘‘mathematician’’, ‘‘statistician,’’ or ‘‘mortal’’) to that of a ‘‘prophet,’’ who can base his decision also on the future. Let n
2 and X n = (X1 , . . . , Xn ), X = (X1 , X2 , . . .), where the X’s have finite expectations and the distribution of X n (in the finite case) or of X (in the infinite case) is completely known to the mortal (and to the prophet). Denote by V(X n ) [V(X), respectively] the value of the sequence to the optimal player, i.e., V(X n ) = sup EXt , t

where t is a nonanticipating stopping rule such that P(t  n) = 1. [A stopping rule satisfies {t = i} ∈ σ (X1 , . . . , Xi ), the σ -field generated by X i .] V(X) is defined similarly, with P(t < ∞) = 1. The value for the prophet is M(X n ) = E(max(X1 , . . . , Xn ))[M(X) = E(sup Xi ), respectively]. For a broad class C of random variables, a ratio prophet inequality is of the following form: For all X n [X] in C, M(X n )  cn V(X n ) [M(X)  cV(X)].

(1)

A difference Prophet inequality has the following form: For all X n [X] in C, M(X n ) − V(X n )  dn

[M(X) − V(X)  d], (2)

and a prophet region is Rn = {(x, y) : x = V(X n ), y = M(X n )

REFERENCES 1. Rosenbaum, P. R. (1984). J. Amer. Statist. Ass., 79, 565–574. 2. Rosenbaum, P. R. and Rubin, H. (1983). Biometrika, 70, 41–55. 3. Rosenbaum, P. R. and Rubin, H. (1984). J. Amer. Statist. Ass., 79, 516–524. See also CLINICAL TRIALS —II and OBSERVATIONAL STUDIES.

for some X n
C},

(3)

with R defined similarly. The first and bestknown prophet inequality is when C is the class of all nonnegative independent random variables. See Krengel and Sucheston [2]. For this class the constants in (1) are cn = c = 2, and these values are sharp (i.e., cannot be improved upon). Thus for this class (1) should be read as follows: Let X1 , X2 , . . . be any finite

6566

PROPORTIONAL ALLOCATION

or infinite sequence of independent nonnegative random variables. The value of this sequence to the prophet is never more than twice its value to the mortal. In the case where C consists of nonnegative i.i.d. sequences, cn is less than 2, depends on n, and can be computed recursively. In the case where C consists of nonnegative random variables which are negatively dependent in the sense that they satisfy P(Xi < ai |X1 < a1 , . . . , Xi−1 < ai−1 )  P(Xi < ai ) for i = 2, . . . , n and all constants a1 , . . . , an , (1) again holds with cn = c = 2. If C is the class of independent random variables taking values in [0, 1], then dn = d = 14 in (2). For this class, Rn of (3) satisfies Rn = R = {(x, y) : 0  x  1, x  y  2x − x2 }. If C is the class of all nonnegative random variables, the best value of cn in (1) is n, and no finite c suffices for the infinite case. The ‘‘extremal distribution’’ is a martingale∗ . If, in addition, Xi , takes values in [0, 1] for all i, then Rn = {(x, y) : 0  x  1, x  y  x + (n − 1)x(1 − x1/(n−1) )}. Other classes of random variables for which prophet inequalities have been considered are sequences of averages of independent nonnegative random variables, discounted independent random variables in [0, 1], independent variables in [0, 1] with cost of observations, and convex functions of sums of independent variables. Hill and Kertz [1] is an excellent reference with an almost complete bibliography of everything published in the field up till 1992. Recently, statistical prophet inequalities have been considered. In this setting, pairs (X1 , Z1 ), . . . , (Xn , Zn ) have a given joint distribution, but the Xi s are not directly observable, and only Xi ’s ‘‘noise corrupted version’’, Zi , is observable. This includes the case Zi = Xi + Yi where the Yi s are any sequence

independent of the Xi s, as well as many other examples. When the pairs are independent, all previously mentioned prophet inequalities valid for independent Xi s remain valid also for the corresponding statistical version. REFERENCES 1. Hill, T. P. and Kertz, R. P. (1992). A survey of prophet inequalities in optimal stopping theory. Contemp. Math., 125, 191–207. 2. Krengel, U. and Sucheston, L. (1978). On semiamarts, amarts and processes with finite value. In Probability on Banach Spaces, J. Kuelbs, ed. Marcel Dekker, New York. See also OPTIMAL STOPPING RULES.

ESTER SAMUEL-CAHN

PROPORTIONAL ALLOCATION This term is used in stratified designs∗ in finite population sampling∗ . When a finite population is divided into homogeneous strata and a separate sample is drawn from each stratum, the sample design is said to be stratified. For instance, a population of persons might be stratified by sex and age, a particular stratum consisting of females aged 55–59; or a population of retail establishments might be stratified by area, description, and size, a particular stratum consisting of metropolitan hardware stores with annual sales of $1,000,000–$1,999,999 at the last retail census. Proportional allocation is one method which can be used to decide how many sample units are to be selected from each stratum, given the sample number or sample size for the entire survey. In proportional allocation, the sample number for each stratum is chosen to be proportional to the number of population units in each stratum i: nh = n(Nh /N) to the nearest integer, where nh = sample number for the hth stratum n = sample number for the whole survey Nh = number of population units in the hth stratum N = number of population units in the population.

PROPORTIONAL HAZARDS MODEL, COX’S

Thus for the population of persons stratified by age and sex, the number of females aged 55–59 selected in the sample would be proportional to the number of females aged 55–59 in the population as a whole. Proportional allocation is often (not invariably) appropriate for samples of persons. It is inefficient in circumstances where the population units differ greatly in size or importance, as is the case with surveys of retail establishments. In such circumstances it is preferable to use optimal allocation (see OPTIMUM STRATIFICATION). Proportional allocation is commonly used in small surveys which do not require multistage sampling. The corresponding techniques used in multistage surveys are known as equal probability selection methods (epsem for short). BIBLIOGRAPHY Sudman, S. (1976). Applied Sampling. Academic Press, New York, pp. 112–114. (Indicates why proportional allocation is often preferred to optimum allocation on grounds of simplicity and versatility.) See also NEYMAN ALLOCATION; OPTIMUM STRATIFICATION; STRATIFIED DESIGNS; and SURVEY SAMPLING.

where β is a p × 1 vector of unknown parameters reflecting the effects of x on survival and λ0 (t) is an unspecified function of time. An extension of model (1) introduced by Kalbfleisch [7] allows strata∗ within the population to have distinct underlying hazard functions λ0j (·).

PARAMETER AND FUNCTION ESTIMATION Usually, survival data of the type envisaged here will be subject to right censoring∗ since, e.g., some individuals will not have failed on termination of the study. It will be assumed throughout that the censoring and failure mechanisms are independent [14]. For the n individuals in the study with independent variable vectors xi , i = 1, . . . , n, let t(1) < t(2) < · · · < t(k) denote the ordered uncensored failure times with corresponding independent variables x(1) , x(2) , . . . , x(n) and denote by R(t(i) ) the collection of individuals with censored or uncensored failure times
t(i) . Following Prentice and Kalbfleisch [14], let Ai be the event that individual (i) fails at t(i) and Bi be the event that describes the pattern of failure and censorings to t(i) and the information that a failure occurs at t(i) . The likelihood factors to produce

K. R. W. BREWER k 

PROPORTIONAL HAZARDS MODEL, COX’S

6567

p(Ai |Bi )

" k 

i=1

p(Bi |Bi−1 , Ai−1 )

i=2



× p(B1 )p(Bk+1 )

THE MODEL In medical and industrial statistics, methods for evaluating the dependence of survival∗ time or response time T on independent variables or covariates x = (x1 , . . . , xp ) have received considerable attention. The independent variables could include, e.g., state of disease, duration of symptoms prior to treatment, or a binary (0/1) variable representing control/treatment group. Cox’s regression model [1] assumes that λ(t; x), the hazard function of the continuous random variable T, is given by 

λ(t; x) = λ0 (t)eβ x ,

(1)

(with t(k+1) = ∞). The first of these terms does not involve λ0 (t) and reduces to

L(β) =

 k   i=1



 



eβ x(i) 

j∈R(t(i) )

eβ xj 

.

(2)

Cox [2], through the concept of partial likelihood∗ , indicates that inferences concerning β, using the usual large-sample likelihood methods, can be based on (2). In the absence of censoring, (2) is the marginal distribution of the ranks of the failure times.

6568

PROPORTIONAL HAZARDS MODEL, COX’S

Estimation of λ0 (t) can be achieved [9] by approximating this ‘‘underlying hazard function’’ as a step function with λ0 (t) = λi ,

t ∈ [bi−1 , bi ),

i = 1, . . . , r,

where b0 = 0 < b1 < . . . < br−1 < br = ∞ is a subdivision of the time scale. Conditional on β = βˆ (the partial likelihood estimator of β), the maximum likelihood estimator∗ of λi is given by  −1 n   ˆ eβxj Dij , i = 1, . . . , r, λˆ i = di  

a collection of censored and uncensored survival times. Choosing τ to ensure a specified proportion, cp are censored. Kay [11] shows, e.g., in the two-group case (p = 1, xi = 0/1) that the Fisher information∗ from the exponential form of model (1) [λ0 (t) ≡ 1] is given by IE (β) =

EFFICIENCY CALCULATIONS The efficiency∗ of βˆ compared to the corresponding estimator based on parametric forms of (1) is clearly of interest. Efron [5] obtains the general element of the Cox model information matrix∗ from (2) for k, l = 1, . . . , p, as &  2 −∂ log L(β) Ikl (β) = E ∂βk ∂βl  
∞   ER(t) yjk yjl λ0 (t) dt (3) =   0 j∈R(t)

where yjr = xjr −

  

i∈R(t)

β  xi

xik e

   

i∈R(t)

β  xi

e

−1  

and ER(t) denotes expectation over the probability distribution of rankings in R (t). A censoring model of interest in the clinical trial∗ setting assumes that individuals enter the trial at random in the interval (0, τ ), termination of the trial at τ giving

1 − 12 [a1 (β, τ ) + a2 (β, τ )]

,

where −β

j=1

where Dij is the length of time spent in the interval [bi−1 , bi ) by individual j and di is the number of uncensored failures in that interval. For estimation in the stratified form of the model, see Kay [10]. In practice data may involve ties. Prentice and Gloeckler [13] have presented some results on the use of grouped form of model (1) to handle ties and have discussed other approaches to this problem.

n[1 − a1 (β, τ )][1 − a2 (β, τ )]

a1 (β, τ ) =

eβ (1 − e−τ e τ

a2 (β, τ ) =

e−β (1 − e−τ e ) τ

)

and β

and (3) reduces to
1 IC (β) = 2n β e−τ e

τ + e−β log u 2β

τ {1 + e2β u(2β−1)e }

du.

The asymptotic efficiency of βˆ under the exponential assumption is then R(β) = IC (β)/IE (β). At β = 0, R(β) = 1. In the uncensored case R(0.2) = 0.963 and R(0.4) = 0.874 while for Cp = 0.3, R(0.2) = 0.988 and R(0.4) = 0.950. Again for p = 1 and in the uncensored case Kalbfleisch [8] shows that 2 in the neighborhood of β = 0, R(β)  e−u2 β , where µ2 is the second central moment of the x-values. MODEL CHECKING Some informal methods, based on residuals∗ and hazard plotting∗ , have been suggested for assessing model (1) assumptions by Kay [10], Prentice and Kalbfleisch [14], and Cox [3], among others. Defining for r = 1, 2, . . .   ˆ ˆ = tj eβ xj + t(rm) eβ xj B(m) r j∈R(t / (rm) )

j∈R(t(rm) )

as the total estimated ‘‘operational’’ time at risk up to the rmth failure, Cox shows that (m) approximately Z(m) = d − log[B(m) r r − B(r−1) ], where d is a constant independent of r and B(m) 0 = 0, has mean log λr , where λr is the average underlying hazard between the failures at t((r−1)m) and t(rm) , and variance

PROPORTIONAL HAZARDS MODEL, COX’S

(m − 12 )−1 . In addition, the Z’s are indepenagainst dent and, for chosen m, plots of Z(m) r the midpoint of that interval will display the form of λ0 (t). To check that a particular independent variable x affects the hazard in the assumed multiplicative way, the sample may be divided into a small number of subsamples on the basis of the x values and the stratified form of model (1) fitted. Plots as above, separately within each stratum, can then be viewed for evidence of trends. These would indicate violation of the multiplicative assumptions with regard to the effect of x. TIME-DEPENDENT COVARIATES A generalization of (1) allows the independent variables to be time dependent and Crowley and Hu [4] and Farewell [6] provide interesting examples of their use. Prentice and Kalbfleisch [14] give a thorough discussion of their application and show how they may be used with model (1) in the analysis of competing risks∗ . AN APPLICATION The data set to be used as an illustration of the methods described above is from the Manchester (U.K.) Regional Breast Study. Further details are given in Lythgoe et al. [12]. The independent variables, recorded at entry into the study, are: " 1 tumor size > 2 cm x1 = log age x2 = 0 tumor size  2 cm " 1 tumor site lateral x3 = 0 otherwise " 1 tumor site central x4 = 0 otherwise " 1 postmenopausal < 3 years x5 = 0 otherwise " 1 postmenopausal
3 years x6 = 0 otherwise " 1 clinical state II x7 = 0 clinical stage I

6569

" 1 local + XRT, stage I y1 = 0 local, stage I " 1 local + XRT, stage II y2 = 0 radical, stage II Of the 881 patients, there were 205 deaths, of which 185 were distinct. The 10 pairs of tied values were broken at random for the purposes of analysis. Table 1 presents Cox model β estimates with estimated standard errors obtained from the inverse of the negative of the matrix of second partial derivative of ˆ Chi-square statistics (2) evaluated at β = β. were obtained by fitting reduced models with each variable omitted in turn and employing the large-sample likelihood ratio test∗ . Stage and menopausal status are the only significant factors and in particular the treatment variables y1 and y2 produce nonsignificant effects. The model checking ideas of §4 are illustrated by investigating the effect of clinical stage in the model. Defining two strata according to the value of x7 , Fig. 1 plots values, for m = 12, having fitted the Z(m) r the stratified form of the Cox model with x7 omitted from the variables listed above. The pattern of the two sets of Z values, particularly in view of the associated standard deviations, are similar, suggesting that clinical stage affects the underlying hazard function in the assumed multiplicative way. REFERENCES 1. Cox, D. R. (1972). J. R. Statist. Soc. B, 34, 187–220. 2. Cox, D. R. (1975). Biometrika, 62, 269–276. 3. Cox, D. R. (1979). Biometrika, 66, 188–190. 4. Crowley, J. and Hu, M. (1977). J. Amer. Statist. Ass., 72, 27–36. 5. Efron, B. (1977). J. Amer. Statist. Ass., 72, 557–565. 6. Farewell, V. T. (1979). Appl. Statist., 28, 136–143. 7. Kalbfleisch, J. D. (1974). Biometrics, 30, 561. 8. Kalbfleisch, J. D. (1974). Biometrika, 61, 31–38. 9. Kalbfleisch, J. D. and Prentice, R. L. (1973). Biometrika, 60, 267–278. 10. Kay, R. (1977). Appl. Statist., 26, 227–237. 11. Kay, R. (1979). Biometrika, 66, 91–96.

6570

PROPORTIONAL REDUCTION IN ERROR (PRE) MEASURES OF ASSOCIATION

Table 1. Cox Model Fit Giving Estimates, Estimated Standard Errors, and χ 21 Statistics for Testing Effects Independent Variable

Estimated Coefficient

Age, Tumor size,



Tumor site,  Menopausal status, Stage,



Treatment,

Estimated Standard Error

χ12 Statistic

x1 x2 x3

0.004 0.236 0.068

0.672 0.169 0.184

0 2.01 0.14

x4 x5

0.015 0.573

0.218 0.293

0 3.56a

x6 x7 y1

0.323 0.553 −0.229

0.283 0.201 0.195

1.31 6.86b 1.39

y2

0.159

0.205

0.61

a 0.05 < p < 0.10 b p < 0.001.

Figure 1. Cox model check for effects of clinical stage. 12. Lythgoe, J. P., Leck, I., (1978). Lancet, 744–747.

and

Swindell, R.

13. Prentice, R. L. and Gloeckler, L. A. (1978). Biometrics, 34, 57–67. 14. Prentice, R. L. and Kalbfleisch, J. D. (1979). Biometrics, 35, 25–39. See also CLINICAL TRIALS; CONCOMITANT VARIABLES; HAZARD PLOTTING; and SURVIVAL ANALYSIS.

RICHARD KAY

PROPORTIONAL REDUCTION IN ERROR (PRE) MEASURES OF ASSOCIATION Proportional reduction in error measures were developed largely because of the need for measures of association∗ between two nominal∗ variables with meaningful interpretations. Unlike measures based upon the chi-square∗ statistic, which are difficult to

PROPORTIONAL REDUCTION IN ERROR (PRE) MEASURES OF ASSOCIATION

interpret, PRE measures have interpretations in terms of probabilities of misclassification which are clear and easily understood. Suppose that a population is classified on the basis of two nominal attributes, A and B. In contingency table∗ notation, let pij be the probability that a member of the population is classified at level i of the (row) variable A and at level j of the (column) variable B, for i = 1, . . . , r  and j = 1, . . . , c. Then the ith row total pi+ = j pij is the marginal∗ probability that an individual falls in class i of attribute A; similarly, p+j is the probability that the individual falls in class j of B. PRE measures are motivated as follows. Some rule is used to classify a randomly selected member of the population with respect to attribute A, first using only the population information at hand, and again when the member’s B class is known as well. The associated PRE measure is the relative reduction in the probability of misclassification: PRE(A|B) = (P(E) − P(E|B))/P(E),

(1)

where P(E) is the probability of error in the first case and P(E|B) is the probability of error in the second. Thus (1) is the decrease in the probability of error in predicting (for some prediction rule) a member’s A class gained by knowledge of its B class, relative to the probability of error when its B class is unknown. If it is a priori possible to classify the member on A with certainty [i.e., P(E) = 0], then (1) is undefined. Moreover, 0  PRE(A|B)  1, with PRE(A|B) = 0 when A and B are independent, and PRE(A|B) = 1 when knowledge of the member’s B class permits errorless prediction of its A class. It can happen that PRE(A|B) = 0 even though A and B are not independent. See ref. 1 or 2 for further discussion. Each classification rule used with (1) yields a PRE measure. One rule (optimal prediction) is to assign the member to the most probable A class, given available information. When the member’s B class is unknown, it is assigned to the A class of maximum probability (i.e., to the class m such that pm+ = max{p1+ , . . . , pr+ }). When it is known to fall into B class (column) j, it is assigned to the A class (row) of maximum probability

6571

in column j (i.e., to the class m such that pmj = max{p1j , . . . , prj }). The resulting PRE measure is    c  pmj − pm+  (1 − pm+ ). (2) λA|B =  j=1

A second rule (proportional prediction) is to assign the member to an A class according to the appropriate (marginal or conditional) probability distribution for A: The member is assigned to class i with probability pi+ when its B class is unknown, and to class i with probability pij /p+j when its B class is known to be j. This rule yields r  c 

τA|B =

i=1 j=1

p2ij /p+j − 1−

r  i=1

r  i=1

p2i+ (3)

.

p2i+

When a classification of the entire population is unavailable, it is necessary to estimate the measures (2) and (3). For a random sample of size n from a population in which xij is the number of sample members in A class i and B class j, the analogs to (2) and (3) are  λˆ A|B = 

c 

 xmj − xm+ 

 (n − xm+ )

(4)

j=1

and (see GAMMA) τˆA|B =

GOODMAN –KRUSKAL

n

r i=1

c

TAU

AND

r

2 j=1 xij /x+j −  n2 − ri=1 x2i+

2 i=1 xi+

.

(5)

Under the multinomial∗ sampling model, (4) and (5) are maximum likelihood estimators∗ of (2) and (3), respectively. Exact sampling distributions of (4) and (5) are not known, but both have asymptotically normal distributions, provided that the sample is not concentrated in a single A class and that (2) and (3) are neither 0 nor 1. Using Gini’s definition of total variability in a sample from a categorical variable, Margolin and Light [4,5] show that, when τA|B = 0, 2 UA|B = (n − 1)(r − 1)τˆA|B

(6)

6572

PROPORTIONAL SAMPLING

is asymptotically distributed as a chi-square variate with (r − 1)(c − 1) degrees of freedom. The PRE measures (2) and (3) treat A and B asymmetrically, with B playing the role of ‘‘independent’’ variable. Versions of (2) to (6) exist which treat A as ‘‘independent’’ and B as ‘‘dependent.’’ Symmetric versions of (2) to (5) have also been defined for cases where no natural asymmetry exists between A and B; see ref. 2 or 3. The measure λ was first proposed by L. Guttman and τ is attributed to W. A. Wallis.

REFERENCES 1. Bishop, Y. M. M., Fienberg, S. E., and Holland, P. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, Mass., pp. 387–393. (An exposition and derivation of the log-linear model for the analysis of contingency tables, with emphasis on iterative proportional fitting. Contains a chapter on measures of association which includes a concise discussion of the properties of λ and τ .) 2. Goodman, L. A. and Kruskal, W. H. (1979). Measures of Associations for Cross Classifications. Springer-Verlag, New York. (A reprint of four classical papers, written between 1954 and 1972, on the construction of measures of association for two-way contingency tables. The original and most extensive source of information on λ and τ ; these papers contain derivations, sample estimates and related asymptotic calculations, examples, and historical references.) 3. Liebetrau, A. M. (1983). Measures of Association. Sage, Beverly Hills, Calif. (A comprehensive and fairly elementary discussion of a large number of widely used measures of association, including λ and τ .) 4. Light, R. J. and Margolin, B. H. J. Amer. Statist. Ass., 66, 534–544.

(1971).

5. Margolin, B. H. and Light, R. J. J. Amer. Statist. Ass., 69, 755–764.

(1974).

See also ASSOCIATION, MEASURES OF; CATEGORICAL DATA; GOODMAN –KRUSKAL TAU AND GAMMA; LOG-LINEAR MODELS IN CONTINGENCY TABLES; and NOMINAL DATA.

A. M. LIEBETRAU

PROPORTIONAL PROPORTIONAL ALLOCATION

SAMPLING. See

PROPORTION ESTIMATION IN SURVEYS USING REMOTE SENSING During 1972–1984 the National Aeronautics and Space Administration (NASA) launched a series of land observatory (LANDSAT) satellites, and developed a remote sensing technology capable of monitoring certain types of earth resources. A major application of LANDSAT remote sensing is the estimation of acreage for agricultural crops. Remotely sensed data consist of radiometric measurements in each of several regions of the electromagnetic spectrum. The sensor system has a radiometer which measures the intensity of radiant energy. Two radiometers used onboard LANDSAT satellites are the multispectral scanner (MSS) and the thematic mapper (TM). Of the five LANDSAT satellites launched, the first three carried only an MSS sensor and the last two carried both the MSS and TM sensors. Both sensors image a 100-nautical mile wide swath of the earth by viewing the orbital ground track through a mirror, periodically scanning strips perpendicular to the orbital path. The analog signals recorded by the sensors are converted into digital counts onboard the satellite and transmitted to a ground station. Hence the LANDSAT coverage of an area results in a scene consisting of scanlines with a certain number of resolution elements per scanline. The TM sensor was designed to provide improved spatial and spectral resolution. It consists of six wavelength bands, compared to four for the MSS sensor. It has a smaller size resolution element, approximately 0.2 acre versus 1.1 acres for MSS. The resolution element is called a pixel. The TM data transmission rate is 85 megabits per second versus 15 megabits per second for MSS. The digitization counts are from 0 to 255 for TM and from 0 to 63 for MSS. The first three LANDSAT satellites were launched into circular, near-polar orbits 920 km (570 miles) in altitude. These orbits are ‘‘sun-synchronous’’; for a given point on the ground, it is the same local time of day when the satellite passes. These satellites pass over a given point on the ground every 18 days. Landsats 4 and 5 were launched into a sun-synchronous, circular, near-polar

PROPORTION ESTIMATION IN SURVEYS USING REMOTE SENSING

orbit 705 km (431 miles) in altitude with repetitive coverage every 16 days. Multidate acquisitions over a single location on the ground can be registered with each other and the radiance measurements can be modeled as a function of time. These measurements are influenced by the vegetation, soil type, and atmospheric conditions, and when these are statistically modeled and correlated with ground features, it is feasible to estimate agricultural crop acreages by acquiring and analyzing the remotely sensed data for an area of interest. Image analysis techniques are used to label spectral classes by crop types. A segment, several square miles in area, is required to delineate discernible patterns and identify possible crop types. A crop can be distinguished from others in a scene by monitoring the temporal development of its fields from planting through harvest and performing multitemporal analysis. This, of course, requires that multitemporal acquisitions of a segment be properly registered since misregistration, especially near field boundaries, can cause measurements to shift from one crop type to another. STATEMENT OF THE PROBLEM Let z be a p × 1 measurement vector for a pixel in an area segment containing a large number of pixels. Suppose that the segment consists of m ground cover classes for which the distribution of z is given by the probability density function fi (z), i = 1, 2, . . . , m. Then one may consider that the vectors z are independent observations sampled from a mixture∗ population with density function f (z) =

m 

λi fi (z),

(1)

i=1

where λi is the proportion of the segment in ground cover class i satisfying λi
0,

m 

λi = 1.

(2)

i=1

The general problem is to estimate the λi (i = 1, 2, . . . , m), given the observed vectors z for a segment without any knowledge of m or fi (z). This problem cannot be solved unless one

6573

is able to obtain information which would permit estimation, if not a complete determination, of m and fi (z). This general mixture problem has been extensively studied. A common approach is to make certain assumptions; one of the following cases is usually considered. 1. The number of classes m is known and fi (z) is from a known identifiable family of distributions, such as multivariate normal. 2. The number of classes m is known, a set of z observations from each class distribution is available, and the densities fi (z) are members of some identifiable family, usually known up to a set of parameters. For a definition of identifiability∗ and further discussion on this finite mixture problem, see Teicher [7]. In the present context, the number of classes was either fixed or estimated. An initial approach was to develop false color imagery of a segment using its spectral data and have an image analyst, trained in photointerpretation techniques, identify the potential number of ground cover classes, and label a set of pixels in the segment corresponding to the various classes. There were problems with this approach. First, the analyst was not in a position to identify all ground cover classes in a segment. Clustering the segment data can provide the possible number of classes represented in the data set, but these may not correspond to the ground cover classes in the segment. Second, labeling of pixels was highly labor intensive and time consuming, and hence not enough pixels could be labeled to estimate the class distributions nonparametrically. Generally, an analyst was trained in identifying features for a few major ground cover classes and his or her labeling of pixels was subject to error. These difficulties play a significant role in the approaches to the estimation of crop proportions using LANDSAT data. CROP PROPORTION ESTIMATION In this section we survey mainly methods of proportion estimation which were used in

6574

PROPORTION ESTIMATION IN SURVEYS USING REMOTE SENSING

practice and evaluated at the NASA Johnson Space Center. These studies were conducted under two major programs, the Large Area Crop Inventory Experiment (LACIE) [6] and the Agriculture and Resources Inventory Surveys Through Aerospace Remote Sensing (AgRISTARS). The initial approach was to assume that fi (z) in (1) represents either a parametric family of multivariate normal distributions or mixtures of multivariate normal distributions with mixing weights assumed known. For example, in LACIE, wheat was the class of interest, and therefore there were only two classes, wheat and nonwheat. Since nonwheat may consist of various ground-cover classes in a segment, its spectral distribution was assumed to be a mixture of multivariate normals. A similar model was assumed for wheat to take into account the different varieties, growth stages, and so on. Thus the density functions for the two classes were of the form f1 (z) =

m1 

f2 (z) =



(3) λ2k f2k (z),

k=1

where fik (z) is the multivariate normal density with mean vector µik and covariance matrix  ik , and λik is the relative size of the kth subclass in class i, i = 1, 2 and k = 1, 2, . . . , mi , satisfying λi =

mi 

fˆi (z) = max fˆj (z), j

(4)

where fˆi (z) is the estimated density function for class i. The class proportion λi was estimated by * Nj , (5) λˆ i = Ni j

where Ni is the number of pixels in the segment classified into the ith class. This estimator is biased, as discussed next, and hence estimation procedures that are relatively unbiased and somewhat insensitive to errors in estimates of class distributions were necessary to obtain reliable crop acreage estimates during LACIE and AgRISTARS. ESTIMATORS

λ1k f1k (z),

k=1 m2

and the sample covariance matrices). All pixels in a segment were classified using the maximum likelihood classification∗ rule (see DISCRIMINANT ANALYSIS); a pixel with measurement vector z is classified as class i if

Let C1 represent the crop of interest and suppose a pixel either belongs to C1 or its complement C2 . Suppose that pixels in a segment are classified according to the maximum likelihood classification rule and let R1 and R2 be the classification regions corresponding to C1 and C2 . Then  & Ri = z : fˆi (z) = max fˆj (z) , i = 1, 2. (6) j

λik .

k=1

The number of subclasses mi and their respective weights λik were determined by clustering the data, labeling the clusters as being wheat or nonwheat, and then finding their relative sizes within each class. If the data were not clustered, an image analyst provided a rough estimate of mi and the subclass weights were assumed equal [i.e., λik = 1/mi (i = 1, 2)]. Spectral data for the set of pixels labeled by an analyst were used to estimate fik (z), and hence fi (z), by replacing the unknown parameters with their corresponding training statistics (i.e., the sample mean vectors

Define the conditional probability pij of a pixel from class j being classified as class i by
fj (z)dz. (7) pij = Ri

Suppose that y is the probability that a pixel belongs to C1 and x is the probability of classifying a pixel as C1 . Then it follows from (7) that x = p11 y + p12 (1 − y). Define the random variable φ by  1, if z ∈ R1 , φ(z) = 0, otherwise,

(8)

(9)

PROPORTION ESTIMATION IN SURVEYS USING REMOTE SENSING

where R1 is the classification region for C1 . Suppose that the segment has N pixels that are classified as C1 or C2 . Then the proportion of pixels classified as C1 , given by yˆ =

N 

φ(zj )/N,

(10)

j=1

provides an estimate of y. In (10), zˆj denotes the measurement vector for pixel j. This is a biased estimator since E(ˆy) = x, where x is given in (8). The bias is given by E(ˆy) − y = p12 − (p12 + p21 ) y, where p12 and p21 are misclassification errors called the commission and omission errors, respectively. If the commission error p12 is large relative to the omission error p21 , a strong possibility when the maximum likelihood classifier is used and the true proportion y is small, then the bias may be substantial. If the classification errors are known, an unbiased estimator can be constructed by modifying (10) as follows: From (8), y = (x − p12 )/(1 − p21 − p12 ),

(11)

provided that 1 − p21 − p12 = 0. Replacing x by yˆ in (11), an unbiased estimator of y is obtained. The classification errors are generally unknown and the preceding simple modification is not possible. However, one may consider estimating p12 and p21 using an independent sample of pixels in the segment, and then estimating y by yˆˆ = (ˆy − pˆ 12 )/(1 − pˆ 21 − pˆ 12 ),

(12)

provided that 1 − pˆ 21 − pˆ 12 = 0, where yˆ is given in (10). If 1 − pˆ 21 − pˆ 12 = 0, then there is no need to modify yˆ (i.e., let yˆˆ = yˆ ). The estimator yˆˆ is then bounded between 0 and 1. Clearly, for yˆˆ to be a reliable and nearly unbiased estimator of y, the estimators pˆ 21 and pˆ 12 should be based on large samples drawn randomly from C1 and C2 , respectively. In the remote sensing application, it was not possible to select pixels randomly from individual classes C1 and C2 , since class labels were unknown for the pixels in a segment, and for a large sample pixel labeling

6575

by an analyst was too labor intensive and subject to error. There was no apparent way to deal with the latter problem; however, the problem of obtaining random samples was solved by sampling pixels from the segment without having to control sample sizes for C1 and C2 , as discussed next. Let n pixels be randomly selected from the segment and be identified for their true class labels. For a pixel with measurement vector z, define the random variable η(z) by η(z) =

 1, if pixel belongs to C1 , 0, otherwise.

(13)

Clearly, E[η(z)] = y and an estimate of y is given by y=

n 

η(zj )/n,

(14)

1

where zj , j = 1, 2, . . . , n, are the measurement vectors for the n sampled pixels. But y is inefficient, as it does not make use of the classification results for the segment. Suppose that S1 and S2 are the two strata of pixels classified as C1 and C2 , respectively. In other words, S1 = {uj : φ(zj ) = 1, j = 1, 2, . . . , N}, S2 = {uj : φ(zj ) = 0, j = 1, 2, . . . , N}, where uj denotes pixel j. Then yˆ given in (10) is the proportion of pixels in S1 and (1 − yˆ ) is the proportion pixels in S2 . In the sampled of n = φ(z pixels, n 1 j ) are from S1 and n2 = 1 n from S2 . Of the n1 pixels 1 [1 − φ(zj )] are  from S1 , n11 = n1 η(zj )φ(zj ) belong  to C1 , and from the n2 pixels in S2 , n12 = n1 η(zj )[1 − φ(zj )] belong to C1 . Thus an estimator of y is obtained by y˜ = n11 yˆ /n1 + n12 (1 − yˆ )/n2 ,

(15)

provided that ni
1, i = 1, 2. Otherwise, let y˜ = y, where y is defined in (14). Since the event that n1 or n2 is zero has nonzero probability, y˜ is in general biased. However, as n becomes large, y˜ is asymptotically unbiased. The estimator in (15) can be rewritten y˜ = y + b(X − x),

(16)

6576

PROPORTION ESTIMATION IN SURVEYS USING REMOTE SENSING

where y is given in (14), x = as defined in (10), and n 

b=

n i

φ(zj )/n, X = yˆ

G(t) = G0 + A(t − T0 )α exp[−β(t − T0 )2 ], (18)

(η(zj ) − y)(φ(zj ) − x)

j=1 n 

behavior of the greenness variable has been approximately modeled by

.

(17)

(φ(zj ) − x)2

j=1

If the denominator in b is zero, take b = 0. Clearly, b is an estimator of the slope in the linear regression∗ of η(z) onto φ(z). Thus y˜ can be thought of as a regression estimator in the case of a finite population where the sample mean y is adjusted to take into account the auxiliary information of the measurement vector z based on the correlation between φ(z) and η(z). Properties of this type of estimator are discussed by Cochran [2]. Heydorn [4] discusses the estimators given above in greater detail, noting further extensions of the estimator in (15). Feiveson [3] describes some of these estimators as well as others, particularly mixture decomposition estimators with mixtures of density functions as in (1), marginal distribution functions, or moments.

where G(t) is the greenness value at time t, G0 is the bare soil greenness value, T0 is the date of emergence, α is a crop specific parameter related to the rate of change of greenness in early season (‘‘green up’’), and β is another crop specific parameter related to the rate of onset of senescence or ripening of the crop (‘‘green down’’). This formulation was extremely important because it related the LANDSAT data to potentially predictable agrophysical parameters such as emergence date, rate of green development, rate of senescence, and total length of growing season. The model in (18), called the greenness profile model, has two inflection points, T1 and T2 , such that 2 = (T2 − T1 )2 = 1/(2β) + α[1 − (1 − 1/α)1/2 ]/(2β) ≈ 1/β,

(19)

and a peak at FEATURE EXTRACTION USING CROP SPECTRAL PROFILE MODELS The LANDSAT multiband data are highly correlated for both the MSS and TM sensors. A principal component analysis∗ of MSS data acquired for a segment in a single LANDSAT pass would show that these data are primarily two-dimensional; one corresponds to a measure of vegetation growth and the other to a measure of background reflectance. The two dimensions are called greenness and brightness, respectively. Recent research has shown that a similar situation holds for the six-band TM data, with the greenness and brightness space augmented by a third dimension related to soil water content. This type of analysis not only reduces the data dimensionality by a factor of 2, but also provides physically interpretable parameters. From investigations of the greenness and brightness variables, it has been observed that their behavior as a function of time is different for different crops. The temporal

t = Tp = T0 + (α/(2β))1/2

(20)

with a maximum greenness value of Gmax = G0 + A(α/(2βe))α/2 .

(21)

For corn and soybeans T1 corresponds closely to the onset of the reproductive phase of these crops and T2 corresponds to the onset of senescence. Since T1 , T2 , and thus , can be predicted using crop phenology models, this result provides an important feature for labeling crop types. Crop phenology models predict the dates of particular phases in a crop’s life cycle. The predictor variables for these models usually include daylength, temperature, and rainfall measurements. Several empirical studies showed that the variables , Tp , and Gmax given in (19) to (21) are good discriminators for crops with different crop phenology and can be used in place of the multitemporal measurement vectors z. Of course, this would not change any of the

PROPORTIONS, COCHRAN’S Q -STATISTIC FOR

methods of proportion estimation discussed earlier. The crop profile model approach to feature extraction was initially proposed by Badhwar [1]. See Houston and Hall [5] for a good review of this approach from a statistical viewpoint. Other references on the use of satellite data for crop surveys are provided in the Bibliography.

6577

Odell, P. L., ed. (1976). Commun. Statist. A, 5. (A special issue on proportion estimation using satellite data.) See also AGRICULTURE, STATISTICS IN; CLASSIFICATION; CROP AREA ESTIMATION, LANDSAT DATA ANALYSIS IN; GEOGRAPHY, STATISTICS IN; METEOROLOGY, STATISTICS IN; SPATIAL DATA ANALYSIS; and SPATIAL SAMPLING.

A. GLEN HOUSTON RAJ S. CHHIKARA

REFERENCES 1. Badhwar, G. D. (1980). Photogramm. Eng. Remote Sensing, 46, 369–377. 2. Cochran, W. G. (1977). Sampling Techniques. Wiley, New York. 3. Feiveson, A. H. (1979). Proc. Tech. Sess., LACIE Symp., Vol. 2, NASA-JSC 16015, pp. 643–646. (An excellent review of crop proportion estimation methods in remote sensing from an analytical viewpoint.) 4. Heydorn, R. P. (1984). Commun. Statist. Theor. Meth., 13, 2881–2903. (An up-to-date review of crop proportion estimation methods used in practice.) 5. Houston, A. G. and Hall, F. G. (1984). Commun. Statist. Theor. Meth., 13, 2857–2880. (Describes LANDSAT MSS and TM data and approaches for relating their temporal behavior to crop growth cycles. Also briefly describes techniques for crop identification and area and yield estimation.) 6. MacDonald, R. B. and Hall, F. G. (1980). Science, 208, 670–679. (An excellent reference on an approach to global crop forecasting using satellite data.) 7. Teicher, H. (1961). Ann. Math. Statist., 32, 244–248.

See the following works, as well as the references just given, for more information on the topic of proportion estimation in crop surveys using remotely sensed data. BIBLIOGRAPHY Bauer, M. E., ed. (1984). Remote Sensing Environ., 14 (1–3). (A special issue devoted to remote sensing research to inventory and monitor crop, soil, and land resources accomplished by the AGRISTARS program.) Chhikara, R. S., ed. (1984). Commun. Statist. Theor. Meth., 13. (A special issue on crop surveys using satellite data.)

PROPORTIONS, COCHRAN’S Q -STATISTIC FOR When comparing proportions in independent samples, a large-sample chi-square∗ statistic is quite well known. Often, however, each member of a sample is matched with a corresponding member of every other sample in order to increase the precision of comparison. When matching occurs, the samples are correlated and the ordinary Pearson chi-square statistic is no longer valid. One way of creating matched samples is to divide nc individuals into n groups of c individuals matched according to one or more characteristics or variables and then to randomize the c treatment assignments independently within each matched group. The individuals should be matched on characteristics that are associated with the response being studied. Matched samples also occur when one observes n individuals under c different treatments (e.g., c different questions or one question at c different times). No matter how the matching occurs, matched ctuples are used for comparing c samples to gain efficiency. McNemar [10] developed a test for comparing two matched samples when the response variable is a dichotomy. Cochran [6] extended McNemar’s results to the case of several matched samples. To derive Cochran’s Qstatistic, consider comparing c matched treatments (i.e., samples) with n observations per treatment. Let Xij = 1 if the outcome for the jth treatment in the ith matched observation is a ‘‘success’’ and Xij = 0 otherwise, for j = 1, 2, . . . , c and i = 1, 2, . . . , n. The total number  of successes for the jth treatment is Tj = ni=1 Xij , and the total number of successes  for the ith matched observation is ui = cj=1 Xij .

6578

PROPORTIONS, COCHRAN’S Q -STATISTIC FOR

In deriving the test statistic, Cochran regarded the ui as fixed. He then argued that under the ‘‘null hypothesis’’ each of the ways of distributing the ui successes among the c treatments is equally likely. Therefore, for the ith matched observation, each treatment has an equal probability of success, and this probability is dependent on the fixed ui and is allowed to vary from observation to observation. Note that Cochran’s Q-statistic reflects the idea that the probability of the response is related to the values of the matching variables. The statistic is given as  c(c − 1) cj=1 (Tj − T)2 ; Q=   c ni=1 ui − ni=1 u2i it provides an exact conditional test and, also, a large-sample chi-square test of the hypothesis of permutational symmetry. Bhapkar [3] formally states this hypothesis as H(c): (Xij1 , Xij2 , . . . , Xijc ) has the same conditional distribution, given uk , as (Xi1 , Xi2 , . . . , Xic ) for every permutation (j1 , j2 , . . . , jc ) of (1, 2, . . . , c), i = 1, 2, . . . , n. Although Cochran [6] never formally states H (c) as the null hypothesis of interest, he does show that Q has a limiting χ 2 (c − 1) distribution under the ‘‘null hypothesis.’’ An application of Cochran’s Q-statistic is seen in the following data originally given by Somes [13]. Shown in Table 1 are the results of observing 37 horses over a 4-day period and noting their responses as either correct or incorrect, represented by 0 or 1. The plausible alternative to rejection of H (c) would be that the horses have learned. To simplify the calculations, Q may be expressed as   c  2 2 Tj − S (c − 1) c j=1 n 

Q=

cS −

i=1

, u2i

where S=

c  j=1

Tj =

n  i=1

ui =

n  c  i=1 j=1

Xij .

For these data, Q has a value of 15.77, which is significant at the 0.005 level in a χ 2 (3) table. Madansky [11] mentions Cochran’s Qstatistic when he deals with a test of an hypothesis of interchangeability. He considers a more general situation of n individuals holding one of s opinions at t successive time intervals. He derives a conditional test statistic which for s = 2 reduces to that given by Cochran by letting c = t. Fleiss [7] demonstrates the applicability of Cochran’s Q-statistic for situations where the ordering of the c treatments is not necessarily random for any of the nc-tuples. Cochran’s Q-statistic has been used to test hypotheses other than the one Cochran had in mind. If the probability of success is assumed to be constant from observation to observation within a sample, then the hypothesis concerning equality of proportions in the samples may be formally stated as H0 : π1 = π2 = · · · = πc , where πj is the probability of success for the jth treatment. It is important to recognize that in H0 the probability of success is assumed to be constant from matched observation to observation, whereas in H (c) it is allowed to vary, conditional on ui . Cochran’s Q-statistic has been used to test H0 and it was erroneously assumed that Q had an asymptotic χ 2 (c − 1) distribution when H0 was true. The conditions necessary for Q to have an asymptotic χ 2 (c − 1) distribution when H0 is true were derived by Bhapkar [2]. This necessary and sufficient side condition may be expressed as H1 : π1,2 = π1,3 = π1,c = π2,3 = . . . = πc−1,c , where πj,k is the probability of simultaneous success for the jth and kth treatments. Berger and Gold [1] obtained the asymptotic distribution for Q under H0 for c = 3, and more recently Bhapkar and Somes [4] derived the asymptotic distributions of Q under H0 for any c and demonstrated that the limiting rejection value of Q under H0 is

PROPORTIONS, COCHRAN’S Q -STATISTIC FOR

always at least as large as the rejection value obtained from the χ 2 (c − 1) distribution and often much larger, depending on the violation of H1 . If the hypothesis of interest is H0 , then Bhapkar [2] offers a Wald statistic [14] which is asymptotically χ 2 (c − 1) under H0 . Even though this test statistic is more cumbersome than Cochran’s Q-statistic, it may be preferable, since the added calculations necessary to arrive at an approximation to the asymptotic distribution of Q under H0 are also considerable. Naturally, in testing H (c) or if H1 is assumed when testing H0 , then Q is the correct test statistic. When Cochran [6] introduced his Qstatistic he also indicated how it may be broken down into components for more detailed tests comparing mutually exclusive subgroups of the c treatments. It can be shown that the statistic given by Miettinen [12] for comparing a case to multiple matched controls is a special case of breaking Q into its components. That is, Miettinen’s statistic∗ [12] is identical to that one given by Cochran [6] for comparing the first treatment (say) with the (c − 1) other treatments. Along these same lines it should be noted (see, e.g., Brownlee [5]) that Cochran’s Q-statistic is a special case of Friedman’s rank test statistic∗ [9] when there are ties in the ranks. For further readings on Cochran’s Qstatistic, one should refer to Cochran’s original article [6] or possibly to the appropriate section of books by Brownlee [5] and Fleiss [8]. The latter is a good reference for many hypotheses dealing with categorical data∗ . For more information regarding the applicability of Cochran’s Q-statistic for hypotheses other than H (c), refer to Bhapkar [2,3] and Bhapkar and Somes [4].

REFERENCES 1. Berger, A. and Gold, R. F. (1973). J. Amer. Statist. Ass., 68, 984–993. 2. Bhapkar, V. P. (1970). In Random Counts in Scientific Work, Vol. 2, Patil, G. P. ed. Pennsylvania State University Press, University Park, Pa., pp. 255–267. 3. Bhapkar, V. P. 341–356.

(1973).

Sankhya¯

A,

6579

Table 1. Responses of Horses as Correct (0) or Incorrect (1) Day

Tj

1

2

3

4

1 1 1 0 1 1 0 1 0 0 0 23

1 1 0 1 1 0 1 0 0 0 0 15

1 0 1 1 0 1 1 0 1 0 0 10

1 1 1 1 0 0 0 0 0 1 0 10

Number of Cases 3 4 1 1 6 1 1 8 3 1 8

4. Bhapkar, V. P. and Somes, G. W. (1977). J. Amer. Statist. Ass., 72, 658–661. 5. Brownlee, K. A. (1965). Statistical Theory and Methodology in Science and Engineering. Wiley, New York, pp. 262–265. 6. Cochran, W. G. (1950). Biometrika, 37, 256–266. 7. Fleiss, J. L. (1965). Biometrics, 21, 1008–1010. 8. Fleiss, J. L. (1973). Statistical Methods for Rates and Proportions. Wiley, New York, pp. 83–87. 9. Friedman, M. (1937). J. Amer. Statist. Ass. 32, 675–701. 10. McNemar, Q. (1947). Psychometrika, 12, 153–157. 11. Madansky, A. (1963). J. Amer. Statist. Ass., 58, 97–119. 12. Miettinen, O. S. 339–355.

(1969).

Biometrics,

25,

13. Somes, G. W. (1975). Some Contributions to Analysis of Data from Matched Samples. Ph.D. dissertation, University of Kentucky. 14. Wald, A. (1943). Trans. Amer. Math. Soc., 54, 426–482. See also CATEGORICAL DATA; CHI-SQUARE TEST —I; FRIEDMAN’S CHI-SQUARE TEST; MATCHED PAIRS; MATCHED SAMPLES, MIETTINEN’S TEST STATISTIC FOR; and TWO-BY-TWO TABLES, MCNEMAR’S TEST FOR.

35,

GRANT W. SOMES

6580

PROSPECTIVE STUDIES

PROSPECTIVE STUDIES In epidemiology, the term ‘‘prospective study’’ usually designates a nonexperimental research design in which all the phenomena under observation occur after the onset of the investigation. This approach is known by a variety of other names, such as follow-up∗ , incidence, longitudinal∗ , and cohort studies. Regardless of the title preferred, the prospective approach may be contrasted with other nonexperimental studies that are based on historical information (see EPIDEMIOLOGICAL STATISTICS and RETROSPECTIVE STUDIES). The usual plan of prospective research is illustrated in Fig. 1. After initiation of the study, subjects are enrolled according to the level of exposure to the main independent variable. Typically, one group is defined as ‘‘exposed’’ and the comparison group consists of ‘‘unexposed’’ subjects. The group assignment in a prospective study is determined by observations on naturally occurring exposures. The reliance on natural exposures may be contrasted with experimental studies (see CLINICAL TRIALS), in which the exposure status is assigned by randomization∗ (but see also HISTORICAL CONTROLS). After the study groups are defined in a prospective investigation, the subjects are followed forward in time for the development of the outcome variable. Then the frequency of the outcome among exposed subjects is compared against the frequency of the outcome among unexposed subjects. Example. One of the most noteworthy examples of the prospective approach was the investigation of atherosclerotic heart disease in Framingham∗ , Massachusetts [1]. This study was started in 1948 by the U.S. Public Health Service for the purpose of evaluating possible determinants of heart disease. Framingham was chosen as the site of investigation because it had a stable, cooperative population, with good access to local and referral medical facilities. A sample of 6,500 persons between the ages of 30 and 60 years was chosen from a population of about 10,000 persons in that age group. Over 5,000 persons without clinical evidence of atherosclerotic heart disease agreed to participate in this research. Each subject was

examined at the outset and was reexamined at intervals of two years. The primary outcome was the development of atherosclerotic heart disease. Over the 20 years of follow-up several risk factors were identified, including hypertension, elevated serum cholesterol, and cigarette smoking [6]. THE SELECTION OF SUBJECTS A variety of factors must be considered in the choice of the study population for prospective research. In particular, the subjects must be accessible, willing to cooperate, and traceable over time. The following sources of subjects are employed commonly in prospective research. 1. General Population. A prospective study can be performed in an entire community, as illustrated by the Framingham Study. The choice of a community for investigation may be governed by the stability and demographic features of the population, public support for the study, and the availability of resources over a sufficient period of time to evaluate the outcome. 2. Special Population. A prospective study can be conducted among persons who are affiliated with a particular group. For example, Doll and Hill [2] studied the relationship between cigarette smoking and lung cancer among physicians on the British Medical Register who were living in the United Kingdom. The choice of physicians as a study population offered the advantages of a group that was interested in health research and could be traced through professional organizations. 3. Clinical Population. A prospective study can be performed among patients at specific clinical facilities. As an illustration, the Collaborative Perinatal Project was a prospective study of more than 54,000 pregnant women who received prenatal care and delivered at one of 12 cooperating hospitals between 1959 and 1966 [5]. This population was used to evaluate risk factors

PROSPECTIVE STUDIES

6581

Figure 1. Schematic diagram of a prospective study (shaded areas represent subjects exposed to the antecedent factor; unshaded areas correspond to unexposed subjects).

for perinatal and infant mortality∗ , as well as for infant morbidity∗ . THE MEASUREMENT OF EFFECT The analysis of a prospective study is based on a comparison of the frequencies of outcomes among exposed and unexposed subjects. The magnitude of effect usually is quantified with one of the following measures. 1. Incidence density ratio (IDR). The IDR is defined as the incidence rate among the exposed divided by the incidence rate among the unexposed. The standard data layout for this type of analysis is portrayed in Table 1. For this analysis, the symbols a and b represent the number of specific outcomes among exposed and unexposed subjects, respectively. The length of observation on each individual, summed over all subjects in the study group are represented by L1 and L0 , respectively. Thus, the incidence density rate of the outcome among exposed

persons is estimated by a/L1 . Similarly, the incidence density of the outcome among unexposed persons is estimated by b/L0 . The IDR is estimated from
= a/L1 . IDR b/L0 When there is no association between the exposure and outcome of interest, the IDR equals unity. An IDR less than unity implies a reduced incidence of the outcome among exposed subjects, compared with the unexposed. Conversely, an IDR greater than unity indicates that exposed subjects have an elevated incidence of the outcome. A largesample test of difference from the null value can be constructed from the normal approximation to the binomial distribution∗ [3]: z=

a − m(L1 /T) . [m(L1 /T)(L0 /T)]1/2

2. Risk ratio (RR). The RR is defined as the risk of the outcome among exposed

Table 1. Data Layout for Calculation of the Incidence Density Ratio (IDR) Exposed Unexposed Subjects Subjects Total Number of Specific Outcome(s) a b m= a+b Population Time of Follow-up

L1

L0

T = L1 + L0

6582

PROSPECTIVE STUDIES Table 2. Data Layout for Calculation of the Risk Ratio (RR) Exposed Unexposed Subjects Subjects Total Number with Outcome a b m1 = a + b Number without Outcome Total

c

d

m0 = c + d

n1 = a + c

n0 = b + d

N =a+b+c+d

divided by the risk among unexposed. The standard data layout for this type of analysis is depicted in Table 2. The symbols a and b have the previously defined interpretations. The symbols c and d represent the number of subjects who do not develop the outcome among exposed and unexposed persons, respectively. The risk of the outcome among the exposed is estimated by a/n1 . Similarly, the risk of the outcome among the unexposed is estimated by b/n0 . Thus, the RR is estimated by  RR

=

a/n1 . b/n0

The interpretation of the value of the RR is identical to that for the IDR. A large sample test for this measure, based on the hypergeometric distribution∗ , uses the Mantel–Haenszel∗ chi-square statistic [4]: 2 χ(1) =

(N − 1)(ad − bc)2 . m1 m0 n1 n0

As indicated, the IDR and RR have the same scale of potential values. Although there are no absolute rules for the interpretation of these measures, Table 3 may serve as a rough

Table 3. Guidelines to the Interpretation of the Incidence Density and Risk Ratios Value

Effect of Exposure

0–0.3 0.4–0.5 0.6–0.8 0.9–1.1 1.2–1.6 1.7–2.5
2.6

Strong benefit Moderate benefit Weak benefit No effect Weak hazard Moderate hazard Strong hazard

guideline. For both measures, the absolute lower limit is zero, which results when there are no outcomes among exposed subjects. The upper limit is positive infinity, which results when there are no outcomes among unexposed subjects. RELATIVE MERITS OF PROSPECTIVE STUDIES When compared with other methods of nonexperimental research, the prospective design offers several advantages. First, incidence rates of the outcome can be measured. Second, there is a clear temporal relationship between the purported cause and effect in prospective research. Third, exposures that are uncommon can be evaluated by selective sampling of exposed persons. Fourth, the amount and quality of information collected is not limited by the availability of historical records or recall of subjects. Fifth, multiple effects of a given exposure can be assessed. Nevertheless, prospective studies also have important limitations. First, large numbers of subjects may be needed to evaluate rare outcomes. Second, a prospective study may involve years of observation to evaluate chronic processes. A prolonged investigation may suffer from logistical problems, such as turnover of the research staff and variations in the methods used to measure the outcome. Third, it is difficult to follow subjects over protracted time periods and these individuals may change their exposure levels over time. Fourth, prospective research is relatively expensive when compared with alternative nonexperimental methods. Ultimately, the decision to conduct a prospective study is governed by the purpose of the investigation, the nature of the phenomena under observation, and the resources available for the study.

PROXIMITY DATA Acknowledgments The preparation of this entry was supported in part by a contract from the National Cancer Institute (N01-CN-61027) to the author, and the manuscript was typed by Ms. Vickie Thomas.

REFERENCES 1. Dawber, T. R., Meadors, G. F., and Moore, F. E. (1951). Amer. J. Public Health, 41, 279– 286. 2. Doll, R. and Hill, A. B. (1964). Brit. Med. J., 1, 1399–1410. 3. Kleinbaum, D. G., Kupper, L. L., and Morgenstern, H. (1982). Epidemiologic Research. Lifetime Learning Publications, Belmont, Calif., p. 286. 4. Mantel, N. and Haenszel, W. (1959). J. Natl. Cancer Inst., 22, 719–748. 5. Niswander, K. and Gordon, M. (1972). The Women and Their Pregnancies. National Institutes of Health Publ. 73–379. U.S. Government Printing Office, Washington, D.C. 6. Truett, J., Cornfield, J., and Kannel, W. (1967). J. Chronic Dis., 20, 511–524. See also CLINICAL TRIALS; EPIDEMIOLOGICAL STATISTICS; FOLLOW-UP; FRAMINGHAM: AN EVOLVING LONGITUDINAL STUDY; HISTORICAL CONTROLS; LONGITUDINAL DATA ANALYSIS; PROBABILITY, FOUNDATIONS OF; and RETROSPECTIVE STUDIES.

RAYMOND S. GREENBERG

PROXIMITY DATA Proximity data are numbers representing measures of relatedness of pairs of objects, for example, the judged similarity between political candidates, the confusability of speech sounds, or the number of tourists traveling between different countries. Proximities among a set of n objects are specified in an n × n matrix whose entry in row i and column j indicates the extent to which objects i and j are related to each other. Proximity matrices may be obtained directly, as in the examples above, or derived from rectangular matrices where each of a set of objects under study is described by a set of variables. Some ways of obtaining proximity data are discussed in the next section.

6583

Proximity matrices are often large and complex. Methods for summarizing these data can help to discover important relationships and patterns within the data. Much work has been reported in the past 20 years on the development of models and methods for the representation of structure in proximity data. Most of these representations fall into one of two families: continuous spatial models and discrete network models. Spatial models embed objects in some coordinate space so that the metric distances between points represent the observed proximities between the respective objects. Network models represent each object as a node in some graph so that the relations among the nodes in the graph reflect the proximities among the objects. Spatial and discrete models that assume the proximity matrices are approximately symmetric are discussed in the sections ‘‘Spatial Representations’’ and ‘‘Discrete Representations.’’ Questions about symmetry and some nonsymmetric models are touched on briefly in the section ‘‘Nonsymmetric Data.’’ WAYS OF OBTAINING PROXIMITY DATA One common procedure for obtaining proximity data is to ask subjects to judge the similarity (or dissimilarity) of the stimulus objects under study. Subjects are presented pairs of stimuli and are asked to rate the proximity of these stimuli on some scale. Although similarity or dissimilarity are most frequently used to elicit the judgments, other words have been used such as association or substitutability. However, specific attributes on which the judgments are to be made are usually not specified to the subjects. When the stimulus set under study is large, say 50 to 100 objects, a useful way of obtaining proximity data is to ask subjects to sort the stimuli into mutually exclusive and exhaustive categories; typically, subjects are asked to cluster the items so that stimuli in the same category are more similar to each other than to stimuli in the other categories. A measure of proximity, then, is the number or proportion of times each pair of stimuli is placed into the same category. Another procedure that requires somewhat less direct judgments about proximity

6584

PROXIMITY DATA

involves measurement of stimulus confusability. Subjects are asked to judge whether two stimuli are the same or different. The number or proportion of times a ‘‘same’’ response is given for two stimuli that are, in fact, different, (i.e., confusability of the stimuli) can be considered to be a proximity measure. Another type of measure for which there is a considerable body of data is the amount of mail, telephone communication, air traffic, and so on, between different locations, people, or organizational units. It is also possible to compute a derived measure of proximity from a rectangular matrix of profile data where the rows correspond to objects and the columns are variables measured on the objects. For example, a square symmetric matrix of profile proximities between variables could result from computing correlations of the variables over objects. This would measure similarity of the variables. Measures of dissimilarity of the objects could be obtained by computing distances between these objects. A variety of derived proximity measures is described in Anderberg [1, Chaps. 4 and 5] for ratio, interval, ordinal, nominal, and binary variables. Gordon [15, Chap. 2] gives a good summary of ways of computing a proximity matrix from an objects-by-variables matrix and discusses difficulties which can arise in the construction of such proximity measures. SPATIAL REPRESENTATIONS A common representation of a matrix of proximities is in terms of a spatial model; objects are represented as points in a multidimensional space and the distance between the points are related in some way to the proximity data. Multidimensional scaling∗ (MDS) is a family of models and associated methods for constructing such a space. The aim of MDS is to discover the dimensions relevant to the objects under consideration and to determine their coordinates on each dimension. There are several different approaches to MDS. The metric approach (e.g., Torgerson [33]) generally assumes that the relationship between the proximities and distances in space is linear. In nonmetric MDS (e.g., Shepard [28,29], Kruskal [21,22]), the relationship need only be monotonic. Two-way

MDS assumes that there is only one proximity matrix. Euclidean distances resulting from a two-way MDS analysis are invariant under rigid or orthogonal rotation of the coordinate system. Therefore, the configuration that results from a two-way MDS analysis that assumes Euclidean distances must usually be rotated in order to find the most interpretable dimensions. There are many different computer programs for implementing two-way MDS. One example of such a program that will do both metric and nonmetric MDS is KYST [24]. Other computer programs for two-way scaling are described in MULTIDIMENSIONAL SCALING. It is possible to have several proximity matrices, for example, one matrix for each of several different subjects or experimental conditions. Three-way MDS procedures will handle these data. The most popular threeway MDS approach, weighted MDS or INDSCAL (Individual Differences multidimensional SCALing [4,8,19]), assumes that there is a set of dimensions common to all individuals (or conditions) but that each individual (or condition) can weight these dimensions differently. Three-way MDS produces a separate space for each individual; these spaces are all related, except that the dimensions are stretched or shrunk according to how salient that dimension is for a particular individual. Unlike two-way MDS, the coordinates from a three-way weighted MDS analysis are uniquely oriented, since the model is not rotationally invariant. There are also many computer programs that implement the threeway MDS model. One example of a metric weighted MDS program for symmetric data is SINDSCAL [25]. See also MULTIDIMENSIONAL SCALING; more general models for three-way MDS are described by Tucker [34] and Harshman [17]. DISCRETE REPRESENTATIONS Spatial distance models assume that continuous dimensions are the underlying basis for the proximities. Tversky [35] and Tversky and Gati [36] question this assumption. They argue that similarity between objects often should be represented in terms of discrete features. One of the most widely used

PROXIMITY DATA

methods for discrete representation of proximity data is hierarchical cluster analysis∗ . This produces a set of clusters such that any two clusters are disjoint or one includes the other. The objects being clustered can be represented as terminal nodes of a tree and the distances between objects are the ‘‘heights’’ of the first nonterminal node at which they meet. This leads to a particular kind of metric defined on a tree, the ultra-metric. See HIERARCHICAL CLUSTER ANALYSIS for further details. Another representation of proximity data in terms of a tree is as a path-length tree, also called an additive similarity tree or free tree (see Carroll [5], Sattath and Tversky [27], and Cunningham [13]). While a hierarchical tree has a natural root, path-length trees do not, so it is not necessary to think of them as being hierarchically organized. In a path-length tree, objects are represented as nodes and there are lengths associated with branches; the distance between any two objects is the sum of lengths of the branches that connect the two nodes. A computer program implementing a path-length tree is ADDTREE [27]. Shepard and Arabie [31] describe a nonhierarchical nonspatial model and method for representing data as overlapping clusters. The model assumes that each of the clusters of objects corresponds to a discrete property shared by all and only those objects within that cluster. MAPCLUS is a computer program [2] that uses a mathematical programming∗ approach for implementing this model. INDCLUS [7] generalizes the Shepard-Arabie model to handle several proximity matrices; it assumes that there is a set of clusters that is common to all individuals, or other data sources, but that each individual can weight these clusters differently. Like MAPCLUS, the INDCLUS computer program uses a mathematical programming procedure to implement the model. In a hierarchical representation, in order for an object to be represented in more than one cluster, one of the clusters must be a subset of a larger cluster. Overlapping clustering procedures allow an object to be represented in more than one cluster. Another way of classifying an object in more than one way is to assume that the objects are organized into

6585

two or more separate hierarchies, not necessarily independent of one another. This can be done by fitting more than one tree to a single proximity matrix; objects are displayed as terminal nodes of one or more different trees (see Carroll [5] and Carroll and Pruzansky [9]). Carroll and Pruzansky [9] describe a hybrid model and fitting procedure for representing structure in proximity data that combines both spatial and discrete representations. They simultaneously fit to the data one or several hierarchical or path-length trees and an N-dimensional space. Given such a wide variety of possible models and methods, how does one select the correct model for a given set of data? Pruzansky et al. [26] investigated some properties of proximity data that might help determine the appropriate model for a particular data set. They found that the skewness and a property that measures departures from the ultrametric inequality∗ were useful in distinguishing between a nonmetric MDS model and an additive tree model. However, one should be cautious about trying to find a single correct model for representing proximities. Shepard [30] points out that different methods of analysis may bring out different but equally important aspects of the underlying structure in the data. NONSYMMETRIC DATA The models and methods described above assume that the matrix (or matrices) of proximities is symmetric. For many sets of data this is unrealistic. Asymmetries may be due to noise or they may be systematic. If it can be determined that the data are symmetric but the asymmetries are due to noise, it is reasonable to symmetrize the matrix by using some technique for averaging sij and sji (e.g., Arabie and Soli [3]). Hubert and Baker [20] describe a method for evaluating the symmetry of a proximity matrix. However, there are data sets where systematic asymmetries might be an inherent characteristic. Several approaches have been described for analyzing nonsymmetric proximity matrices in terms of both spatial and discrete models. For example, if the rows and columns of the matrix are thought of as separate entities, the row and column objects

6586

PROXIMITY DATA

can be displayed separately in the same multidimensional space (e.g., Constantine and Gower [10]). A method of this sort exists for data consisting of several nonsymmetric proximity matrices as well (see DeSarbo and Carroll [14]). Some methods deal with the symmetric and nonsymmetric portions separately [10,16,18,32]. Cunningham [13] uses an extension of a free tree representation to account for nonsymmetric proximities; he introduces bidirectional trees, allowing the length of the branch between node i and node j to be different from the length between node j and node i.

11. Coombs, C. H. (1964). A Theory of Data. Wiley, New York.

LITERATURE

12. Coxon, A. P. M. (1982). The User’s Guide to Multidimensional Scaling. Heinemann, Exeter, N.H.

A book by Coombs [11] contains a general discussion of types of data and types of models as well as the earliest formulation of nonmetric scaling. Carroll and Arabie [6] contains a comprehensive review of the work in MDS through 1979. Since they use a broad definition of MDS, the article includes discussion of discrete models as well as spatial distance models. A monograph on MDS by Kruskal and Wish [23] provides an excellent introduction to spatial distance models for representing proximity data. A book by Coxon [12] is a good introductory text on MDS that includes material about both spatial and discrete representations of proximity data. Additional references are mentioned in HIERARCHICAL CLUSTER ANALYSIS and MULTIDIMENSIONAL SCALING.

REFERENCES 1. Anderberg, M. R. (1973). Cluster Analysis for Applications. Academic Press, New York. 2. Arabie, P. and Carroll, J. D. (1980). Psychometrika, 45, 211–235. 3. Arabie, P. and Soli, S. D. (1980). In Multidimensional Analysis of Large Data Sets, R. G. Golledge and J. N. Rayner, eds. University of Minnesota Press, Minneapolis, Minn. 4. Bloxom, B. (1968). Individual Differences in Multidimensional Scaling. Res. Bull. 68-45, Educational Testing Service, Princeton, N.J. 5. Carroll, J. D. 439–463.

(1976).

Psychometrika,

41,

6. Carroll, J. D. and Arabie, P. (1980). Annu. Rev. Psychol., 31, 607–649. 7. Carroll, J. D. and Arabie, P. (1983). Psychometrika, 48, 157–169. 8. Carroll, J. D. and Chang, J. J. (1970). Psychometrika, 35, 283–319. 9. Carroll, J. D. and Pruzansky, S. (1980). In Similarity and Choice, E. D. Lantermann and J. Feger, eds. Hans Huber, Berne, Switzerland, pp. 108–139. 10. Constantine, A. G. and Gower, J. C. (1978). Appl. Statist., 27, 297–304.

13. Cunningham, J. P. (1978). J. Math. Psychol., 17, 165–188. 14. Desarbo, W. S. and Carroll, J. D. (1981). Proc. 3rd ORSA/TIMS Special Interest Conf. Market Meas. Anal., pp. 157–183. 15. Gordon, A. D. (1981). Classification. Chapman & Hall, London. 16. Gower, J. C. (1977). In Recent Developments in Statistics, J. R. Barra, F. Brodeau, G. Romier, and B. van Cutsem, eds. North-Holland, Amsterdam. 17. Harshman, R. A. (1972). PARAFAC2: Mathematical and Technical Notes. Working Paper in Phonetics No. 22, University of California, Los Angeles, CA, pp. 30–47. 18. Holman, E. W. (1979). J. Math. Psychol., 20, 1–15. 19. Horan, C. B. 139–165.

(1969).

Psychometrika,

34,

20. Hubert, L. J. and Baker, F. B. (1979). Qual. Quant., 13, 77–84. 21. Kruskal, J. B. 1–27.

(1964).

Psychometrika,

29,

22. Kruskal, J. B. 115–129.

(1964).

Psychometrika,

29,

23. Kruskal, J. B. and Wish, M. (1978). Multidimensional Scaling. Sage, Beverly Hills, Calif. 24. Kruskal, J. B., Young, F. W., and Seery, J. B. (1973). How to Use KYST 2: A Very Flexible Program to Do Multidimensional Scaling and Unfolding. Bell Laboratories, Murray Hill, N.J. 25. Pruzansky, S. (1975). How to Use SINDSCAL; A Computer Program for Individual Differences in Multidimensional Scaling. Bell Laboratories, Murray Hill, N.J.

PSEUDOCOMPONENTS

6587

26. Pruzansky, S., Tversky, A., and Carroll, J. D. (1982). Psychometrika, 47, 3–24.

REFERENCE

27. Sattath, S. and Tversky, A. (1977). Psychometrika, 42, 319–345. 28. Shepard, R. N. (1962). Psychometrika, 27, 125–140. 29. Shepard, R. N. (1962). Psychometrika, 27, 219–246.

1. Judge, G. G., Hill, R. C., Griffith, W., ¨ Lutkepohl, H., and Lee, T. C. (1982). Introduction to the Theory and Practice of Econometrics. Wiley, New York.

30. Shepard, R. N. (1980). Science, 210, 390–398. 31. Shepard, R. N. and Arabie, P. (1979). Psychol. Rev., 86, 87–123. 32. Tobler, W. (1976). J. Environ. Syst., 6, 271–301. 33. Torgerson, W. S. (1958). Theory and Methods of Scaling. Wiley, New York. 34. Tucker, L. R. (1972). Psychometrika, 37, 3–27. 35. Tversky, A. (1977). Psychol. Rev., 84, 327–352. 36. Tversky, A. and Gati, I. (1978). In Cognition and Categorization, E. Rosch and B. B. Lloyd, eds. Lawrence Erlbaum, Hillsdale, N.J., pp. 79–98. See also HIERARCHICAL CLUSTER ANALYSIS; MULTIDIMENSIONAL SCALING; and SIMILARITY, DISSIMILARITY AND DISTANCE, MEASURES OF.

SANDRA PRUZANSKY

PROXIMITY

GRAPH. See

CLUSTER

ANALYSIS, GRAPH-THEORETIC

PROXY VARIABLE In stochastic regression models∗ of the type yt = β1 + β2 x∗t + vt , where x∗t is an unobservable true variable and vt is a stochastic term, x∗t is sometimes replaced by an observable ‘‘proxy’’ variable xˆ t representing x∗t . Most commonly xˆ t is an estimated or measured (as opposed to ‘‘true’’) value of x∗t . Formally, xˆ t = x∗t + uˆ t , where uˆ t is a random measurement error. As an example (see, e.g., Judge et al. [1]), in studying the effects of education on income we need to measure the years of education. Instead, the observable proxy variable—years of schooling—is used.

See also STOCHASTIC REGRESSION MODELS.

PSEUDO-BAYESIAN INFERENCE. See QUASI-BAYESIAN INFERENCE

PSEUDOCOMPONENTS The concept of pseudocomponents was introduced explicitly by Piepel [4] and Crosier [2] into the statistical literature. It has been popularized in the quality-control literature by Gorman [3] and Snee and Rayner [5]. In mixture experiments∗ the independent variables are often proportions (X) of the g components in the mixture, having q the natural constraints 0  Xi  1 and i=1 Xi = 1; these reduce the factor space to a regular (q − 1)-dimensional simplex. Additional constraints in the form of lower and upper bounds 0  Li  Xi  Ui  1

(1)

are often placed on the components for various reasons. Whenever the constraints (1) are used, experiments transform original components (X) to a new set of variables called pseudocomponents. A conventional pseudocomponent transformation is Zi =

Xi − Li , q 1 − j=1 Lj

(2)

used mainly to adapt standard designs to a simplex that has been reduced by the constraints (1). Piepel [4] notes that the use of (2) is incorrect in the case of inconsistency of the lower and upper bounds—it may happen that the upper bounds make some of the lower bounds unattainable. Crosier [2] proposes pseudocomponent transformations based on the upper bounds.

PSEUDO-F

6588

These are useful not only for transforming standard designs and reducing illconditioning, but also for solving other problems arising in experiments with mixtures. A general definition of pseudocomponents as linear transformations of the original variables, which allows the use of standard results for unconstrained mixture experiments, is provided in Atkinson and Donev [1] with numerous examples.

(usually K = 1). The value of ani depends on the distribution populations. It is commonly supposed, in applications, that these distributions are normal, under which conditions the distribution of the variance ratio (S22 /S21 ) is that of

Fn1 −1,n2 −1 distribution).

1. Atkinson, A. C. and Donev, A. N. (1992). Optimum Experimental Designs. Clarendon Press, Oxford, England. 2. Crosier, R. B. (1984). Mixture experiments: geometry and pseudocomponents. Technometrics, 26, 207–210. 3. Gorman, J. W. (1970). Fitting equations to mixture data with restraints on composition. J. Qual. Technol., 2, 186–194. 4. Piepel, G. F. (1983). Defining consistent constraint regions in mixture experiments. Technometrics, 25, 97–102. 5. Snee, R. D. and Rayner, A. A. (1982). Assessing the accuracy of mixture model regression calculation. J. Qual. Technol., 14, 67–79. See also MIXTURE EXPERIMENTS.

PSEUDO-F This is a statistic (not a distribution). More precisely, it is a class of statistics formed by replacing the unbiased estimators S2i = ni (Xij − X i )2 of variance σi2 (i = (ni − 1)−1 j=1 1, 2) in the variance-ratio∗ statistic S22 /S21 by some other measure of (spread)2 . The class includes, for example, the ratio of squared mean deviations   2  n2 n1     −1 −1 |X2j − X 2 | n1 |X1j − X 1 | n2   j=1

and the ratio 

ani = Kσi /E[range(Xi1 , . . . , Xini )]

(σ22 /σ12 ) × (variable having a

REFERENCES

j=1

where the factors ani are chosen so that

an2 range(X21 , . . . , X2n2 ) an1 range(X11 , . . . , X1n1 )

2 ,

See F-DISTRIBUTION.

PSEUDO-LIKELIHOOD This is a poor title for a class of procedures in which inferences are based on a derived likelihood for a subset of the parameters. We discuss here marginal, conditional, and profile likelihood; partial likelihood∗ is a fourth general class of procedures and is discussed in a separate entry. Suppose that X = (X1 , . . . , Xn ) is a vector of random variables whose distribution depends on parameters (θ , α). We suppose that θ is of interest and that θ and α may be vectors of parameters. In most cases of interest, α is of very high dimension. We consider the problem of drawing suitable likelihood-based inferences about θ when α is unknown. MARGINAL, CONDITIONAL, AND PROFILE LIKELIHOODS The joint likelihood of (θ , α) based on data X = x is L(θ , α; x) ∝ fX (x; θ , α), where fX represents the joint density function. For convenience, we consider the relative likelihood function R(θ , α; x) = L(θ , α; x)/L(θˆ , α; ˆ x),

(1)

which compares each hypothesized (θ , α) with ˆ the maximum likelihood∗ estimate (θˆ , α). Likelihood-based inferences on θ , α can be based on (1), but when (θ , α) is of high dimension, the examination and comprehension of the full likelihood surface is difficult. One

PSEUDO-LIKELIHOOD

approach to estimating θ would utilize the profile or maximized relative likelihood of θ , RP (θ ; x) = sup R(θ , α; x).

(2)

α

Note that RP (θ0 ; x) is the generalized likelihood ratio∗ statistic for the hypothesis θ = θ0 , and places an upper bound on the relative likelihood of (θ , α)-pairs with θ = θ0 . The profile likelihood can quite generally be used, whereas marginal and conditional likelihoods arise in problems with sufficient structure that the nuisance parameter∗ α can be eliminated by integration over a subset of the sample space (marginal likelihood) or by conditioning on a particular observed event (conditional likelihood). Suppose that there exists a one-to-one transformation of X to U, V such that the joint PDF of U, V factors as fU,V (u, v; θ , α) = fU|V (u|v; θ , α)fV (v; θ ),

(3)

where only the second term depends on θ . The marginal likelihood of θ based on V is LM (θ ; v) ∝ fV (v; θ ).

(4)

In general, the use of (4) for the estimation of θ will entail a loss of information, namely the information about θ in the first term on the right side of (3). When, as for example through invariance arguments, one can argue that the whole of the information about θ (when α is unknown) is contained in V, the likelihood (4) is highly recommended as the basis of likelihood inference about θ . More generally, the residual term may contain some information about θ , but (3) may still be used for inference on account of its simplicity. If, on the other hand, the transformation of X into U, V leads to a joint density which factors as fU|V (u, v; θ , α) = fU|V (u|v; θ )fV (v; θ , α), (5) LC (θ ; u|v) ∝ fU|V (u|v; θ )

(6)

is the conditional likelihood of θ based on U for the given V. Again, in certain instances, it may be argued that V contains little or no information about θ when α is unknown so

6589

that the conditional likelihood is appropriate for the estimation of θ . Marginal likelihoods were first introduced within the structural model∗ by Fraser (see refs. 6 and 7). Conditional likelihoods have been used by many authors. Bartlett [4] suggested their use and extended the argument to allow the conditioning variable V to depend on the parameter θ . Both marginal and conditional likelihoods are discussed and exemplified in refs. 9 and 10. EXAMPLES AND COMMENTS Estimation of a Normal Variance Suppose that duplicate determinations X1i , X2i are made on the ith individual, and that X1i , X2i are independent N(µi , σ 2 ) variates, i = 1, . . . , n. Here µ1 , . . . , µn and σ 2 are unknown parameters and our purpose is to estimate σ 2 . The profile or maximum relative likelihood of σ 2 is    (x1i − x2i )2 , RP (σ 2 ; x) ∝ σ −2n exp − 4σ 2 which is maximized at σˆ 2 = (4)−1



(x1i − x2i )2 .

As n → ∞, σˆ 2 converges in probability to σ 2 /2. In this example, the profile likelihood is concentrated for large n about an incorrect value of σ 2 . The data for the ith pair can be written Ui = X1i − X2i and Vi = X1i + X2i . Note that Ui and Vi are independent and Vi ∼ N(2µi , σ 2 ). This suggests that Vi is uninformative about σ 2 when µi is unknown and that a marginal likelihood could be based on U = (U1 , . . . , Un ). Thus    (X1i − X2i )2 (7) Lm (σ 2 ; u) ∝ σ −n exp − 4σ 2 and the marginal  maximum likelihood esti2 mator is σˆ M = (X1i − X2i )2 /(2n), which is consistent. The marginal likelihood takes account of the degrees of freedom lost through the estimation of µ1 , . . . , µn . The heuristic argument which suggests the use of U1 , . . . , Un for inference about σ 2

6590

PSEUDO-LIKELIHOOD

can be formalized by group invariance∗ arguments. The group of transformations Xji → Xji∗ = Xji + ci (j = 1, 2; i = 1, . . . , n), where −∞ < ci < ∞, acts transitively on Vi and leaves Ui invariant. In the parameter space, the induced group acts transitively on µ1 , . . . , µn and leaves σ 2 invariant. This suggests that U1 , . . . , Un is sufficient for σ 2 when µ1 , . . . , µn are completely unknown. This example illustrates difficulties with the use of the profile likelihood when the dimensionality of the nuisance parameter is large. In other instances, when the nuisance parameter is of small dimension compared to the number of observations, the profile likelihood typically gives a convenient and simple summary. Estimation of β in the Proportional Hazards Model The proportional model of Cox (see ref. 5 and PROPORTIONAL HAZARDS MODEL, COX’S) with fixed covariates Z = (Z1 , . . . , Zp ) specifies the hazard function for the time to failure T as λ(t; Z) = λ0 (t) exp(Z β),

Estimation of a Ratio of Poisson Means Suppose that X and Y are independent Poisson variates with means α and θ α, respectively. Let U = X + Y and V = Y; the joint probability function of U, V factors as fU,V (u, v; θ , α) = fu (u; θ , α)fV|U (v; θ |u)   u = (γ u e−γ /u!) v ×θ v (1 + θ )−u , where u
v = 0, 1, 2, . . . and γ = α(1 + θ ). A conditional likelihood for θ can be based on the binomial distribution of V, given U, and yields LC (θ ; v|u) ∝ θ v (1 + θ )−u ,

0 < θ < ∞. (8)

Further, since the distribution of U depends only on γ = α(1 + θ ), it may be argued that no additional information about θ can be extracted from the residual term when α is unknown. In this example, the conditional likelihood (8) is proportional to the profile likelihood. Estimation of A Common Odds Ratio

where β  = (β1 , . . . , βp ), λ0 (t) is a baseline hazard function which is left completely unspecified, and β  = (β1 , . . . , βp ) is a vector of regression parameters. The model specifies that the rate at which individuals fail at time t is a constant (depending on the measured covariates Z) times a baseline rate. Consider a sample T1 , . . . , Tn with corresponding covariates Z1 , . . . , Zn , let T(1) , . . . , T(n) be the order statistics∗ and Z(1) , . . . , Z(n) the corresponding covariates. A marginal likelihood can be based on the distribution of the ranks or of the antiranks, A = ((1), . . . , (n)). This gives Z(i) β

LM (β; a) ∝ π e

 n

Z(j) β

e

,

j=i

which is identical to the partial likelihood obtained by Cox (see PARTIAL LIKELIHOOD). Group invariance arguments can be used to establish the sufficiency of A for the estimation of β when λ0 (·) is completely unspecified (see ref. 8).

Let X1i and X2i be independent Bernoulli variates with parameters p1i and p2i , respectively, where log{p2i /(1 − p2i )} = log{p1i /(1 − p1i )} + θ , i = 1, . . . , n. Here exp(θ ) represents the odds ratio∗ , which is assumed common for each i. This model might be entertained when pairs of like individuals are assigned at random to two treatments and the response is binary. A conditional likelihood can be based on the distribution of V = (V1 , . . . , Vn ) given U = (U1 , . . . , Un ), where Vi = X2i and Ui = X1i + X2i . If Ui = 0 or 2, then Vi is determined, so only discordant pairs (pairs with Ui = 1) contribute and Pr{Ui = 1|Vi = 1} = 1 − Pr{Ui = 0|Vi = 1} = eθ /(1 + eθ ).

PSEUDO-MEDIAN

If λ = eθ /(1 + eθ ), it follows that the conditional likelihood of λ is proportional to λa (1 − λ)b ,

(9)

where a(b) is the number of discordant pairs with X2i = 1(X1i = 1). The conditional likelihood (9) makes no use of information in U about θ . In ref. 13 it is pointed out that some inferences on θ may be possible given only the value of U. If, for example, almost all the pairs were discordant, this would suggest that θ is not near zero. (If θ = 0, then the probability that a pair is discordant is at most 12 , which occurs when p1i = p2i = 12 .) It is not clear, however, how this information can be incorporated in the inference about θ , nor that it is desirable to do so. REMARKS The problem of assessing the information lost through use of a marginal or conditional likelihood, and of specifying conditions under which a statistic can be said to be sufficient for θ when α is unknown have been considered by several authors. See, for example, refs. 1 to 3 and 13. Both marginal and conditional likelihoods can be generalized to allow V in (3) or (5) to depend on θ . Extension of the argument to this case are considered in refs. 4, 9, and 10. Interesting applications of marginal likelihood can be found in refs. 11 and 12. REFERENCES 1. Barnard, G. A. (1963). J. R. Statist. Soc. B, 25, 111–114. (Some aspects of the fiducial argument.) 2. Barndorff-Nielsen, O. (1976). Biometrika, 63, 567–571. (Nonformation.) 3. Barndorff-Nielsen, O. (1973). Biometrika, 60, 447–455. 4. Bartlett, M. S. (1936). Proc. Camb. Philos. Soc., 32, 560–566. (The information available in small samples.) 5. Cox, D. R. (1972). J. R. Statist. Soc. B, 34, 187–220. [Regression models and life tables (with discussion).] 6. Fraser, D. A. S. (1967). Ann. Math. Statist., 38, 1456–1465. (Data transformations and the linear model.)

6591

7. Fraser, D. A. S. (1968). The Structure of Inference. Wiley, New York. 8. Kalbfleisch, J. D. and Prentice, R. L. (1973). Biometrika, 60, 267–278. (Marginal likelihoods based on Cox’s regression and life model.) 9. Kalbfleisch, J. D. and Sprott, D. A. (1970). J. R. Statist. Soc. B, 32, 175–208. [Application of likelihood methods to models involving large numbers of parameters (with discussion).] 10. Kalbfleisch, J. D. and Sprott, D. A. (1974). Sankhya A, 35, 311–328. (Marginal and conditional likelihood.) 11. Minder, C. E. and McMillan, I. (1977). Biometrics, 33, 333–341. (Estimation of compartment model parameters using marginal likelihood.) 12. Minder, C. E. and Whitney, J. B. (1975). Technometrics, 17, 463–471. (A likelihood analysis of the linear calibration problem.) 13. Sprott, D. A. (1975). Biometrika, 62, 599–605. (Marginal and conditional sufficiency.) See also ANCILLARY STATISTICS —I; LIKELIHOOD; PARTIAL LIKELIHOOD; QUASI-LIKELIHOOD FUNCTIONS; and SUFFICIENT STATISTICS.

JOHN D. KALBFLEISCH

PSEUDO-MEDIAN The pseudo-median of a distribution F is defined as a median of the distribution of 1 2 (X1 + X2 ), where X1 and X2 are independent random variables, each having the distribution F. If F is symmetric, the median and pseudomedian coincide. In general, the pseudomedian may differ from the median. Consistent estimators of pseudo-medians are discussed in Hollander and Wolfe [1]. Pseudomedians should not be confused with quasimedians∗ , which are obtained by extension of the sample median. See also ref. 2. REFERENCES 1. Hollander, M. and Wolfe, D. A. (1973). Nonparametric Statistical Methods. Wiley, New York. 2. Høyland, A. (1965). Ann. Math. Statist., 36, 174–197.

6592

PSEUDOPANELS

PSEUDOPANELS. See PANEL DATA —II PSEUDO-RANDOM NUMBER GENERATORS The need to generate a sequence of numbers {Xi } that behaves like a sample of independent uniform (0, 1) random variables arises in several areas, including discrete event simulation and Monte Carlo∗ studies. The numbers generated are called pseudorandom or quasi-random numbers and are produced from generators that yield integers on [0, m], which are then transformed to [0, 1] by division by m. In theory, a uniform deviate can be transformed to an arbitrary random variable having distribution function F(·). Numerous authors (see Fishman [8], Knuth [18], and Bratley et al. [3]) have compiled algorithms for transformations to commonly required distributions (see also RANDOM NUMBER GENERATION). Several sources of error contaminate the ideal above. The finite word length of the computer can result in errors both in the transformation and in restricting all samples to a finite subset of the rational numbers. However, these are insignificant in comparison to errors resulting from nonuniformity and dependence in the pseudo-random number sequence. The latter source of error is a focal point for research. Early approaches concentrated on physical devices such as white noise∗ in electronic circuits and tables of random numbers compiled externally to the computer; see Niederreiter [28] and Jansson [14]. Reproducing results (a necessity both for assuring that the computer representation is an accurate implementation of the model and for gaining efficiencies in the estimation technique through processes known as variance reduction techniques) when using these techniques is problematic and led researchers to consider algorithms for random number generation. RANDOMNESS Pseudo-random number generators are usually evaluated on two criteria: their speed of

execution and the degree to which the resulting number sequence ‘‘appears’’ random. A generator is acceptable if sequences it produces can pass an assortment of tests for randomness∗ . The specific tests and passing levels have not been formalized in the literature so there are no canonical measures by which to evaluate a proposed generator (see Knuth [18]). In practice, researchers seek a generator that yields a sequence {Xi } that approximates a k-distributed sequence. The infinite sequence {Wi } is k-distributed if for numbers 0  v1 < w1  1, . . . , 0  vk < wk  1, lim

n→∞

n  k 

I(vj  Wi < wj )/n

i=1 j=1

= (w1 − v1 ) · · · (wk − vk ), where I(A) = 1 if A is true and 0 otherwise. Thus a k-distributed sequence behaves as independent samples from the joint distribution of k independent uniform deviates. A sequence is ∞-distributed if it is k-distributed for all k > 0. Tests are performed on a given finite sequence to identify deviations from kdistributed uniformity. The tests fall into two categories: empirical and theoretical. The empirical tests are the standard statistical tests one encounters in hypothesis testing∗ and are based on a sample of {Xi }. Knuth [18] and Niederreiter [28] catalog many of these; Fishman and Moore [9] demonstrate a comprehensive approach. Theoretical tests are based on examining the structure that the pseudo-random number generator imparts to the entire sequence {Xi }. As a result, they test the global properties of the generator; empirical tests evaluate local properties of the generator. The spectral test∗ [6,24] is a theoretical test that is useful for examining the degree to which {Xi } is k-distributed. The lattice test, developed by Marsaglia [25], measures the lattice structure of k-tuples from {Xi }. The degree to which the lattice is orthogonal is measured by the ratio of the lengths of the longest to shortest edges, denoted Lk . For the ideal generator Lk = 1 for all k > 1. The importance of testing a pseudorandom number generator cannot be overstated. Knuth [18] gives several examples

PSEUDO-RANDOM NUMBER GENERATORS

of algorithms which, upon cursory examination, appear to produce highly random sequences that, upon closer examination, are poor approximations to random sequences. LINEAR CONGRUENTIAL METHODS Among the most popular and well-studied methods for generating a pseudo-random sequence are the linear congruential generators. These are based on the recurrence scheme Zi+1 ≡ aZi + c (mod m),

(LCG)

where seed Z0 is an integer 0  Z0 < m, multiplier a is an integer 1 < a < m, increment c is an integer 0  c < m, and modulus m is an integer 0 < m. {Zi } is often referred to as a linear congruential sequence or a random number stream, and LCG is called a mixed congruential generator when c > 0 and a multiplicative congruential generator when c = 0. The modulo arithmetic guarantees that each element in {Zi } is an integer in the interval [0, m − 1]. Consequently, when {Zi } displays sufficiently random behavior, a pseudorandom sequence {Xi } can be obtained by normalizing Zi by the modulus m so that Xi = Zi /m. As a result, the larger the modulus m, the smaller the error that results from the finite word length of the computer. It can also be shown that the sequence {Zi } is periodic in the sense that there exists a P  m such that Zi = ZP+i for all i
0. A large P (and hence a large m) is desirable for achieving apparent randomness of {Xi }. The speed of generation is also a consideration in choosing the modulus m. Consider algorithm ALCG for generating a linear congruential sequence according to LCG: Algorithm Alcg 1. W ← aZi + c. 2. Zi+1 ← W − W/M . 3. Return Zi+1 . The notation Y means the largest integer less than or equal to Y. The division W/m in line 2 (which tends to be a slow operation) can be optimized when m = ze , where

6593

z is the base of the computer’s arithmetic and e is an integer. When m has this form, division is accomplished through a shift operation which is faster than a divide. As a result, 2e (10e ) is often chosen as the modulus, where e is the number of bits (digits) in the computer’s word. The generators RANDU (formerly used in the IBM Scientific Subroutine Package), with a = 65, 539, c = 0, and m = 231 and SUPER-DUPER [25], with a = 69, 069, c = 0, and m = 232 , are two such examples. One drawback to generators having m = ze is that the low-order e bits (digits) of {Zi } form a linear congruential sequence with period at most 2e (10e ). Consequently, the low-order bits (digits) have periods shorter than P and can exhibit extreme departures from randomness. RELATIONSHIP BETWEEN A, C , Z0 , AND M Two theorems help to characterize the length of the period P in terms of a, c, Z0 , and m. The first theorem assumes that c > 0. Theorem 1. The linear congruential sequence {Zi } has full period (P = m) if and only if: (a) c is relatively prime to m. (b) a − 1 is a multiple of every prime factor of m. (c) a ≡ 1 (mod 4) when m ≡ 0 (mod 4). Example 1. When m = 2e , c is relatively prime to m, and a ≡ 1 (mod 4), LCG generates a sequence with full period 2e . A full period is no guarantee that the generator is adequate. Niederreiter [28] presents evidence questioning the use of the following full period generator. Example 2. The LCG generator on the DEC VAX-11 [7] has m = 232 , a = 69, 069, and c = 1. Since these satisfy the conditions of Theorem 1, LCG generates a sequence with full period 232 . When c = 0 there is no need to perform the addition in line 1 of ALCG. As a result, LCG with c = 0 may execute somewhat faster than one with c > 0. Theorem 2 gives the conditions on a, m, and Z0 to maximize the period of {Zi } when c = 0.

6594

PSEUDO-RANDOM NUMBER GENERATORS

Theorem 2. When c = 0 the linear congruential sequence {Zi } has maximal period λ(m) if: (a) Z0 is relatively prime to m. (b) a is a primitive root modulo m. The function λ(m) is the order of a primitive element modulo m. Knuth [18] defines λ(m) explicitly and gives conditions for a to be a primitive root modulo m. In the often encountered case when p = 2 and e > 3, they simplify to the single requirement that a ≡ 3 or 5 (mod 8). Example 3. The generator with m = 232 , a = 69, 069, and c = 0 can generate linear congruential sequences with maximal period λ(232 ) = 230 . It achieves this maximal period when Z0 is relatively prime to m, which in this case is equivalent to Z0 being odd. When this holds, every element in the sequence {Zi } is odd. Marsaglia [25] favors this multiplier on the basis of its performance on the lattice test. He claims that for m = 232 , L2 = 1.06, L3 = 1.29, L4 = 1.30, and L5 = 1.25. However, Niederreiter [28] cautions against using this generator solely on the basis of its performance on the lattice test. Example 4. The RANDU generator with m = 231 , a = 65, 539, and c = 0 has maximal period λ(231 ) = 229 . Several authors note deficiencies in the performance of this generator on the spectral and lattice tests [25,18]. PRIME MODULUS MULTIPLICATIVE GENERATORS When m is prime and c = 0, LCG is a prime modulus multiplicative generator. Theorem 2 shows that for these generators the maximal period is λ(m) = m − 1, an almost full period. Furthermore, since all integers less than m are relatively prime to m, there are no additional restrictions on the seed Z0 . Example 5. On a 32-bit machine the largest prime that can be directly represented in one word is 231 − 1. Generators with this modulus have period 231 − 2. As a result, there are several such generators.

These include (a) LLRANDOM [19,22] with a = 16, 807, used in IMSL [13], APL [15], and SIMPL/I and GPSS [12]; (b) the SIMSCRIPT II simulation language generator for which a = 630, 360, 016 (Kiviat et al. [17]); and (c) the RANUNI generator [31] and the LLRANDOMII generator [21] with a = 397, 204, 094. From an empirical point of view Fishman and Moore [9] study the performance of a multiplicative congruential generator with modulus 231 − 1 and 16 alternative multipliers; the three multipliers mentioned above are included. Although a direct implementation of the prime modulus multiplicative congruential generator following ALCG is feasible, some of the efficiencies associated with substituting a shift operation for the division in line 2 when m = ze can be gained when m is a Mersenne prime. A prime is a Mersenne prime if it can be written in the form 2e − 1. An algorithm attributed to Payne et al. [30] is such an implementation. GENERAL LINEAR CONGRUENTIAL GENERATORS A generalization of the LCG is obtained by making Zi+1 a function not only of Zi but also of Zi−1 , Zi−2 , . . . , Zi−p+1 , as Zi+1 ≡

p 

aj Zi−j+1

(mod m). (GLCG)

j=1

By an extension of Theorem 2, for prime modulus m a set {aj |j = 1, . . . , p} can be found such that the linear congruential sequence generated with GLCG has period mp − 1. The coefficients {aj } must be such that the polynomial f (x) = xp + a1 xp−1 + a2 xp−2 + · · · + ap has a root that is a primitive element of the field with mp elements. Then f (x) is a primitive polynomial modulo m, which we refer to as condition PPM(m). Knuth [18] gives criteria for testing for primitivity modulo m. The additive congruential generators are the most popular of the GLCG form. These have aj = 0 or 1 for j = 1, . . . , p.

PSEUDO-RANDOM NUMBER GENERATORS

Example 6. The Fibonacci generator with p = 2, a1 = a2 = 1, and large m has been shown to produce a sequence with poor statistical properties and should be avoided; see Bratley et al. [3]. Example 7. A class of generators, which, Knuth [18] reports, produces sequences with good statistical properties but need further testing, has m ≡ 0 (mod 2), p = 55, and a24 = a55 = 1, aj = 0 for all other j. These generators have period at least as large as 255 − 1. The linear congruential generators modulo 2 are a special case of GLCG with m = 2. This restriction implies that w.l.o.g. the coefficients {aj } can be limited to {0, 1}. As a result, these generators are also a special case of additive congruential generators. When the {aj } are chosen to satisfy condition PPM(2) then {Zi } is a sequence of bits that is called a pseudo-noise sequence or maximallength shift register sequence. Because the {aj } satisfy PPM(2) and m = 2 is a prime, the sequence {Zi } has period 2p − 1. Example 8. Because the trinomial f (x) = x55 + x24 + 1 satisfies PPM(2) the generator Zi ≡ Zi−55 + Zi−24 (mod 2) has period 255 − 1. This generator is one of those in the class shown in Example 7. GENERALIZED FEEDBACK SHIFT REGISTER GENERATORS Tausworthe [34] suggests that a pseudo-noise sequence {Zi } with period 2p − 1 be used as a source of pseudo-random numbers. He examines the properties of the sequence of L-bit numbers {Xi } defined by Xi =

L 

2−j Zsi+r−dj

j=1

for i = 0, . . . , (2p − 1)/s ,

(GFSRG)

where {Zi } is a pseudo-noise sequence with period 2p − 1, s
L, d = 1, p  L, and r is a randomly selected integer on (0, 2p − 1). This transformation of {Zi } is called the L-wise decimation of {Zi }. When s and 2n − 1 are relatively prime {Xi } has period (2p − 1)/s . In this case the transformation of {Zi } is known as proper L-wise decimation of {Zi }.

6595

The sequence {Xi } is sometimes referred to as a Tausworthe sequence and the method of generation as a Tausworthe generator or feedback shift register generator. The restriction s
L is important; Knuth [18] states that for s = 1 a sequence with poor statistical properties results. In practice the sequence {Zi }is usually generated using GLCG with j aj = 2, so that Zi ≡ Zi−p + Zi−q

(mod 2).

As a result one need only determine integers p and q for which f (x) = xp + xq + 1 satisfies PPM(2). Primitive trinomials to degree 1000 are given by Zierler and Brillhart [38]. Tausworthe [34] shows that in the limit, as p → ∞, the properly decimated sequence {Xi } has the mean, variance, and autocorrelation structure that one expects from independent samples of a uniform deviate. He also shows that for k  p/s the sequence {(Xi+1 , Xi+2 , . . . , Xi+k )} is approximately uniformly distributed over the unit k-hypercube. However, Tootill et al. [35] have shown that {Xi } performs better for some values of p and q than others. They recommend that s = L = q, q p/q be relatively prime to 2p − 1 and that q < p/2 be neither too small nor too close to p/2. They claim if these conditions are satisfied, then GFSRG produces sequences with good runs and uniformity properties. Example 9. Tootill et al. [36] propose a generator with p = 607 and q = 273. For d = 1, L = 23, and s = 512, {Xi } has a period of 2607 − 1. They claim that for simulations with no fixed dimensionality requirements on the pseudo-random sequence, this generator possesses 26-distributed uniformity. Lewis and Payne [20] studied sequences {Xi } generated by GFSRG with d = 100p, s = 1, and L = q. They suggest that d relatively prime to the period 2p − 1 improves the degree to which {Xi } approximates a kdistributed sequence for 1  k  p/L . Furthermore, they present an algorithm that

6596

PSEUDO-RANDOM NUMBER GENERATORS

is both extremely efficient and machine independent. Its greatest drawback is an elaborate initialization process. However, Arvillias and Maritsas [1] suggest changes in the algorithm that simplify initialization when q is an integral power of 2. In Bright and Enison [4], Enison shows that with proper initialization the Lewis and Payne [20] scheme results in sequences that are 1-distributed. Fushimi and Tezuka [10] have extended this work to give necessary and sufficient conditions on GFSRG for {Xi } to be k-distributed. Example 10. Bright and Enison [4] recommend that p be a Mersenne exponent so that 2p − 1 is prime. They propose the generator having p = 521, q = 32, L = 64, d = 52, 100, and s = 1. They claim that this generator produces sequences that are 8-distributed. The attraction of these generators comes from the fact that their implementation can be machine independent. The Lewis and Payne [20] algorithm generates an L-bit integer for any value of s. Thus sequences with extremely large periods can be generated on small-word machines. However, the algorithm requires memory for pL-bit integers and a time-consuming initialization. Although initial results from these generators are encouraging, more empirical and structural study is needed before they are adopted widely [8].

The XOR indicates the Boolean exclusiveor operation and the X  e(X e) indicates left (right) shift of X by e bits. In practice, methods (b) and (c) are performed on segments of the sequence {Wi } or {Vi }. Knuth [18] presents algorithms and recommends strongly the use of method (c). Brown and Solomon [5] show that method (a) results in sequences with improved k-distributed properties. They warn, however, that improvements in the k-distribution do not necessarily imply improvements in the marginal distributions of subsets. Fishman [8] reports that the LCG generator SUPERDUPER (a = 69, 069, c = 0, m = 232 ) uses method (d) to lengthen the short period evident in its low-order bits. Example 11. The combination of three multiplicative congruential generators has been suggested by Wichmann and Hill [37]. Their goal is to use three generators with short period that can be implemented on a 16bit machine to produce a pseudo-random sequence with long period. Their scheme uses the generators with multipliers 170, 171, and 172, and moduli 30,323, 30,269, and 30,307, respectively. The fractional part of the sum of three uniform deviates is also a uniform deviate; they use this result to combine the three generators. The resulting sequence has period in excess of 244 . They claim that the sequence has good statistical properties, and exhibit a FORTRAN implementation.

SHUFFLING AND SHIFTING: COMBINATIONS OF GENERATORS

LITERATURE

In an attempt to improve the degree to which a pseudo-random sequence approximates a truly random sequence, several authors suggest methods for shuffling the sequence {Zi } or shifting the bits in each Zi . If {Vi } and {Wi } are two sequences, preferably generated from independent generators, then the following schemes have been suggested:

Knuth [18] gives an excellent summary of pseudo-random number generation with emphasis on the congruential generators. Fishman [8], Kennedy and Gentle [16], and Bratley et al. [3] also contain overviews. Bratley et al. [3] go into some detail on feedback shift register generators. Bibliographies can be found in refs. 11, 27, and 29.

(a) (b) (c) (d)

Zi Zi Zi Zi

≡ (Wi + Vi ) (mod m) [18]. = Wk where k = Vi [23]. = Wk where k = Wi [2]. = (Y  e1 ) XOR Y, where Y = Wi XOR (Wi e2 ) [8].

REFERENCES 1. Arvillias, A. C. and Maritsas, A. E. (1978). J. ACM, 25, 675–686. (A technical article about GFSR generators.)

PSEUDO-RANDOM NUMBER GENERATORS

6597

2. Bays, C. and Durham, S. D. (1976). ACM Trans. Math. Software, 2, 59–64. (Shifting to improve randomness.)

Mass. (A comprehensive guide to LCG, testing, and implementation, less information on GFSRG.)

3. Bratley, P., Fox, B. L., and Schrage, L. E. (1983). A Guide to Simulation. SpringerVerlag, New York. (An introductory guide that focuses on generator implementation. Several FORTRAN programs are given.)

19. Learmonth, G. and Lewis, P. A. W. (1973). Naval Postgraduate School Random Number Generator Package LLRANDOM. Rep. No. NPS55-LW-73061-A, Naval Postgraduate School, Monterey, Calif.

4. Bright, H. S. and Enison, R. L. (1979). Computing Surv., 11, 357–370.

20. Lewis, T. G. and Payne, W. H. J. ACM, 20, 456–468.

5. Brown, M. and Solomon, H. (1979). Ann. Statist., 7, 691–695.

21. Lewis, P. A. W. and Uribe, L. (1981). The New Naval Postgraduate School Random Number Package LLRANDOMII. Rep. No. NPS55-81005, Naval Postgraduate School, Monterey, Calif.

6. Coveyou, R. R. and MacPherson, R. D. (1967). J. ACM, 14, 100–119. (The basis of the spectral test.) 7. Digital Equipment Corp. (1982). VAX 11 Runtime Library Ref. Manual. AA-D036C-TE. Digital Equipment Corp., Maynard, Mass. 8. Fishman, G. S. (1978). Principles of Discrete Event Simulation. Wiley, New York. (An overview of the area with attention to implementation details, empirical and theoretical tests.) 9. Fishman, G. S. and Moore, L. R. (1982). J. Amer. Statist. Ass., 77, 129–136. (A comprehensive study of alternative prime modulus LCG.) 10. Fushimi, M. and Tezuka, S. (1983). Commun. ACM, 28, 516–523. (k-distributivity of GFSRG.) 11. Hull, T. E. and Dobell, A. R. (1962). SIAM Rev., 4, 230–254. (Early bibliography.) 12. IBM Corp. (1981). PL / I General Purpose Simulation System Program Description / Operations Manual. SH20-6181-0, IBM Corp., White Plains, N.Y. 13. International Mathematical and Statistical Libraries, Inc. (1977). IMSL Library, Houston, Tex. 14. Jansson, B. (1966). Random Number Generators. Almqvist & Wiksell, Stockholm. (Early approaches.) 15. Katzan, H., Jr. (1971). APL User’s Guide. Van Nostrand Reinhold, New York. 16. Kennedy, W. J. and Gentle, J. E. (1980). Statistical Computing. Marcel Dekker, New York. (An overview of the area.) 17. Kiviat, P. J., Villanueva, R., and Markowitz, H. (1969). The Simscript II Programming Language. Prentice-Hall, Englewood Cliffs, N.J. 18. Knuth, D. (1981). The Art of Computer Programming, Vol. 2: Seminumerical Algorithms, 2nd ed. Addison-Wesley, Reading,

(1973).

22. Lewis, P. A. W., Goodman, A. S., and Miller, J. M. (1969). IBM Syst. J., 2, 136–145. 23. MacLaren, M. D. and Marsaglia, G. (1965). J. ACM, 12, 83–89. 24. Marsaglia, G. (1968). Proc. Natl. Acad. Sci. USA, 61, 25–28. 25. Marsaglia, G. (1972). In Applications of Number Theory to Numerical Analysis, S. K. Zaremba, ed. Academic Press, New York, pp. 249–286. 26. Marsaglia, G. and Bray, T. A. (1968). Commun. ACM, 11, 757–759. 27. Nance, R. E. and Overstreet, C. (1972). Computing Rev., 13, 495–508. (Bibliography.) 28. Niederreiter, H. (1978). Bull. Amer. Math. Soc., 84, 957–1041. (Monte Carlo technique and number-theoretic aspects of linear congruential generators.) 29. Page, E. S. (1967). In The Generation of Pseudo-Random Numbers in Digital Simulation in Operational Research, S. H. Hollingsdale, ed. American Elsevier, New York, pp. 55–63. (Bibliography.) 30. Payne, W. H., Rabung, J. R., and Bogyo, T. P. (1969). Commun. ACM, 12, 85–86. 31. SAS Institute, Inc. (1982). SAS User’s Guide: Basics, 1982 Edition. SAS Institute, Inc., Cary, N.C. 32. Sowey, E. R. (1972). Int. Statist. Rev., 40, 355–371. (Bibliography.) 33. Sowey, E. R. (1978). Int. Statist. Rev., 46, 89–102. (Bibliography.) 34. Tausworthe, R. C. (1965). Math. Comp., 18, 201–209. 35. Tootill, J. P. R., Robinson, W. D., and Adams, A. G. (1971). J. ACM, 18, 381–399. 36. Tootill, J. P. R., Robinson, W. D., and Eagle, D. J. (1973). J. ACM, 20, 469–481.

6598

PSEUDO-RANDOM VARIABLE GENERATION, TAUSWERTHE METHOD FOR

37. Wichmann, B. A. and Hill, I. D. (1982). Appl. Statist., 31, 188–190.

REFERENCES

38. Zierler, N. and Brillhart, J. (1969). Inf. Control, 14, 566–569.

1. Kennedy, W. J. and Gentle, J. E. (1980). Statistical Computing. Dekker, New York. 2. Tauswerthe, R. C. (1965). Math. Comp., 19, 201–209. 3. Toothill, J. P. P., Robinson, W. D., and Adams, A. G. (1971). J. Amer. Soc. Comp. Mach., 18, 381–399. 4. Toothill, J. P. P., Robinson, W. D., and Eagle, D. J. (1973). J. Amer. Soc. Comp. Mach., 20, 469–48.

See also GENERATION OF RANDOM VARIABLES, COMPUTER; MONTE CARLO METHODS; NUMBER-THEORETIC METHODS; QUASI-RANDOM SEQUENCES; and RANDOM NUMBER GENERATION.

MARC-DAVID COHEN

PSEUDO-RANDOM VARIABLE GENERATION, TAUSWERTHE METHOD FOR

See also PSEUDO-RANDOM NUMBER GENERATORS; RANDOM NUMBER GENERATION; and RANDOM VARIABLES.

This method (Tauswerthe [2]) is based on the sequence {ak } of 0’s and 1’s produced by a linear recurrence relation

PSEUDO RESIDUALS Yi = f (xi ) +  i ,

ak = cp ak−p + cp−1 ak−p+1 + · · · + c1 ak−1 (mod2)

(1)

based on a set (c1 , . . . , cp ) of 0’s and 1’s with cp = 1. The corresponding random integers (to base 2) are formed by successive L-tuples of the a’s, namely Y1 ≡ a1 a2 · · · aL

(base2),

Y2 ≡ aL+1 aL+2 · · · a2L

(base2),

and generally Yj ≡ a(j−1)L+1 a(j−1)L+2 · · · ajL

Consider a regression model of the form

(base2).

This is said to form a L-wise decimation of {ak }. If L is prime relative to 2p − 1, the decimation is said to be proper. A generator using this method is called a Tauswerthe generator. Toothill et al. [3] showed that if nL is prime relative to 2p − 1, then the sequence {Uj } with Uj = 2−L Yj has n-dimensional uniformity. Tauswerthe generators are examples of ‘‘feedback shift register’’ methods. For further details see refs. 3 and 4 and especially ref. 1.

i = 1, . . . , n,

(1)

where  i (i = 1, . . . , n) are independent random errors, f is the underlying mean function, and the x’s are assumed to be ordered. Let Var( i ) = σ 2 , i = 1, . . . , n. It is customary to assume some conditions on f in the analysis of this model. Information about one of the terms on the right-hand side of (1) will be helpful in obtaining information on the other. At the outset, when often there is not much is known about f , knowledge of the error variance is helpful in the process of model building. The estimation of σ 2 in classical models has been done through an averaged squared errors of the residuals∗ of a least-squares∗ fit. However, if no parametric assumption is made on f , then the classical averaged-squared-error estimator leads to large biases, and an alternative estimator of σ 2 with smaller bias and variance is needed. Gasser et al. [1] have proposed such an estimator based on pseudo residuals, which are defined in the following manner. Let the error variance σ 2 be constant. Pseudo residuals ˜ i are obtained by taking consecutive triples of design points xi−1 , xi , xi+1 and setting   xi+1 − xi xi − xi−1 ˜ i = Yi−1 + Yi+1 xi+1 − xi−1 xi+1 − xi−1 − Yi

(2)

PSEUDO-t STATISTIC, BIRNBAUM’S

This amounts to joining the two outer observations by a straight line at each triplet and computing the difference between this straight line and the middle observation. Thus ˜ i can be written as ˜ i = ai Yi−1 + bi Yi+1 − Yi , i = 1, . . . , n − 1, where ai =

xi+1 − xi , xi+1 − xi−1

bi =

xi − xi−1 . xi+1 − xi−1

If f is twice continuously differentiable, then E(˜ 2i ) = (a2i + b2i + 1)σ 2 + O(n−2 ) and |E(˜ i )|  sup{|f  |}/n. Hence pseudo residuals have small bias and mean squared errors of order n−1 , in contrast to O(n−4/5 ) for classical residuals obtained from a nonparametric regression curve estimator with an optimal bandwidth∗ . It was then suggested [1] that n−1 1  2 ci ˜ i , S = n−2 i=2

where ci = (a2i + b2i + 1)−1 , be used as an estimator of σ 2 . Closely related estimators of σ 2 have been proposed by Rice [3] and Hall et al. [2]. REFERENCES 1. Gasser, T., Sroka, L., and Jennen-Steinmetz, C. (1986). Residual variance and residual pattern in nonlinear regression. Biometrika, 73, 625–633. 2. Hall, P., Kay, J. W., and Titterington, D. M. (1990). Asymptotically optimal difference based estimation of variance in nonparametric regression. Biometrika, 77, 521–528. 3. Rice, J. (1984). Bandwidth choice for nonparametric regression. Ann. Statist., 12, 1215–1230. See also QUASI RESIDUALS and RESIDUALS.

K. B. KULASEKERA

6599

√ to an independent χ / ν variable (see CHI DISTRIBUTION). It applies (with ν = n − 1) to √ − example, the distribution of n(X n µ)/S, for 2 = (n − 1)−1 2 and X = (X − X) when S i i=1  n−1 ni=1 Xi , X1 , . . . , Xn being independent normal variables with expected value µ and standard deviation σ . A pseudo-t statistic is obtained by replacing S by some other estimator of σ . This estimator need not be an unbiased estimator of σ (indeed, S itself is not unbiased), but it is often felt desirable—perhaps for aesthetic purposes—to arrange that if there is any bias, it is small. Most commonly, S is replaced by an estimator based on the sample range W = max(X1 , . . . , Xn ) − min(X1 , . . . , Xn ). The corresponding pseudo-T is √ n(X − µ)/(an W), where an is chosen so that an E[W] = σ if the population is normally distributed. It is possible to use other denominators, based on the mean deviation∗ or the interquartile range r, for example. Similar modifications can be used in modifying the denominator of the two-sample t. Approximations to the distribution of pseudo-t (often by a regular t with suitable degrees of freedom, usually fractional) have been developed. See also PSEUDO-F and t-DISTRIBUTION.

PSEUDO-t STATISTIC, BIRNBAUM’S Birnbaum [1] proposes a pseudo-t statistic based on three order statistics. Let X(1)  X(2)  . . .  X(2m+1) be the order statistics from a sample of odd size 2m + 1 of a random variable X, and consider u = X(m+1−r) , V = X(m+1) , W = X(m+1+r)

PSEUDO-t See also PSEUDO-F. The regular Student’s t∗ is the ratio of a unit normal variable

for some integer r, 1  r  m. Here V is the sample median and W − U is an interquantile range between two sample quantiles.

6600

PSEUDO-VARIANCE

REFERENCES

The Birnbaum statistic is S=

V −µ , W−U

(1)

where µ is the expectation of X. A class of similar statistics was considered by David and Johnson [2]. The statistic S is invariant under linear transformations. Thus, the probability distribution of S for given m and r is the same for any bell-shaped probability distribution defined by a probability density f (x) satisfying 1. f (µ − x) = f (µ + x), x
0, and 2. f (µ + x) is nonincreasing for x  0. Birnbaum suggests using Equation 1 for the family of normal distributions, tabulating the probability distribution of S ‘‘for a reasonable range of r and m’’, and applying S in those situations in which Student’s t is used. He points out that when some of the smallest and some of the largest observations are censored, S is still valid, as long as the values of the sample median X(m+1) and of two order statistics X(m+1−r) and X(m+1+r) are known. He also shows that for any bellshaped probability density,   2m + 1 2r r m−r  −r −(m+r) × λ(λ − 1) 2 , λ > 1, 

Pr{|S| > λ} 

which allows tabulation of the appropriate (approximate) critical values. Exact values of Pr{X > λ} are given [1]. Numerical computation of the distribution of S is quite straightforward. The definition of S can be extended using two-order statistics X(m+1−r) and X(m+1+r ) , where r and r are not necessarily equal, and which may be of practical importance when only censoring from above is considered. In this case, the numerical computation of the distribution of S is somewhat more difficult but not prohibitive. For onesided conclusions about µ, a simpler statistic S = (V − µ)/(W − V) can be used.

1. Birnbaum, Z. W. (1970). ‘‘On a Statistic Similar to Student’s t’’. In Nonparametric Techniques in Statistical Inference, M. L. Puri, ed. Cambridge University Press, Cambridge, UK. 2. David, F. N. and Johnson, N. L. (1956). Some tests of significance with ordered variables. J. R. Stat. Soc., 18, 1–20.

PSEUDO-VARIANCE The 100p% pseudo-variance of a continuous cumulative distribution function (CDF) G(x) is the square of the ratio of the difference between its upper and lower 100p% points to the corresponding difference for the standard normal CDF (x). It is thus equal to  v(p) =

G−1 (1 − p) − G−1 (p) −1 (1 − p) − −1 (p)

2 .

For normal distributions, v(p) equals the variance for all p. For heavy-tailed∗ symmetric distributions, v(p) increases as p decreases to zero. Andrews et al. [1] noted that the 4.2% pseudo-variance is remarkably stable for distributions in the Pearson system∗ (see also Pearson and Tukey [3]) and so may serve as a satisfactory index for comparing variability of estimators. Further details can be found in refs. 1 and 2.

REFERENCES 1. Andrews, D. F., Bickel, P. J., Hampel, F. R., Huber, P. J., Rogers, W. H., and Tukey, J. W. (1972). Robust Estimators of Location: Survey and Advances. Princeton University Press, Princeton, N.J. 2. Huber, P. J. (1971). Robust Statistical Procedures. SIAM, Philadelphia. 3. Pearson, E. S. and Tukey, J. W. Biometrika, 52, 533–546.

(1965).

See also HARMONIC VARIABILITY and ROBUST ESTIMATION.

PSYCHOLOGICAL DECISION MAKING

PSEUDO-VARIATES A term used in the theory of weighted least squares∗ . Given a linear model Yu =

p 

βi xiu + u

i=1 2 with var( u ) = w−1 introduction of u σ , the √ pseudo-variates Yu∗ = Yu wu and x∗iu = √ xiu wu results in a model

Yu∗ =

p 

βi x∗iu + u∗

i=1

with var( u∗ ) = σ 2 . If the u ’s are mutually uncorrelated, so are the ∗ ’s, and the second model can be analyzed by ordinary least squares. Of course, it is necessary to know the wu ’s (at least, their ratios) to apply this method. For more details, see Box et al. [1]. REFERENCE 1. Box, G. E. P., Hunter, W. G., and Hunter, J. S. (1978). Statistics for Experimenters. Wiley, New York.

PSI-FUNCTION. See DIGAMMA FUNCTION; POLYGAMMA FUNCTIONS

PSYCHOLOGICAL DECISION MAKING Decision making is a complex, ill-defined body of knowledge developed and studied by researchers from various disciplines, including mathematics, statistics, economics, political science, management science, philosophy, and psychology. The theory has two interrelated facets, normative and descriptive. The normative part of the theory [4,28,29] is concerned with optimal rather than actual choices. Its main function is to prescribe which action should be taken, given the structure of the decision problem, the consequences of each action, the objective of the decision maker (DM), and the information available to him. As such, normative decision

6601

theory∗ is a purely deductive discipline. The aim of the descriptive part [15,20,39] is to account for the DM’s values, opinions, and goals, the way in which they are affected by information, perception of the decision task, or the personality of the DM, and the manner by which the DM incorporates them into decisions. As such, descriptive decision theory is a proper branch of psychology. The distinction between the normative and descriptive parts of the theory is elusive; the two theories are interrelated in most applications. On the one hand, the criteria for optimality or rationality are necessarily based on human judgment and agreement about the criteria is typically not easily reached. On the other hand, in a variety of decision problems, such as most economic investments, goal-oriented DMs try hard to behave rationally. When faced in such problems with obvious errors of judgment (e.g., intransitivity in preference) or calculation, DMs often admit their mistakes and readily reverse their decisions. TASKS The experimental tasks employed to study decision behavior fall into two classes. A single-stage decision task is characterized by a triple (E, D, r), where E is a finite nonempty set of possible events, D is a finite nonempty set of possible decisions available to the DM, and r is an outcome or reward uniquely associated with the joint occurrence of a decision and an event. The DM constitutes a ‘‘system’’ that may reside in one of several states, each of which consists of variables assumed to affect decisions. These variables may include the DM’s utility function, financial status, or the information he or she has acquired. The latter is often expressed as a subjective probability distribution over the uncertain outcomes associated with each decision. Given that the system is in state s, the DM chooses a course of action d, receives the reward determined by the joint occurrence of the action and the event that obtains, and the task terminates. A multistage decision task [4] is characterized by a quadruple (E, D, p, r) where E and D are defined as above, p associates

6602

PSYCHOLOGICAL DECISION MAKING

with each pair (s, d) a probability distribution on E known as a transformation rule or a transition function, and r is now the outcome associated with the transition from one state of the system to another. At the beginning of each stage of the decision process the system is at state s; the DM makes a decision d, the system moves to state s , selected according to the transformation rule, and the DM receives the reward r(s, d, s ) associated with this transition. The process continues for N stages; generally the DM is assumed to maximize some criterion function such as the total subjectively expected reward (or utility∗ ) over all stages or the subjectively expected reward for the last stage only. Research on multistage decision behavior (multistage betting, inventory control where the demand fluctuates from one stage to another) has been meager [25,32]. Although human judgment is a continuous, interactive process as individuals cope with their environment, most behavioral decision research has focused on discrete incidents [12]. Because of their simplicity and more fundamental nature, single-stage decision tasks have been the major focus of experimental research. A typical instance is a choice between two gambles: A: win either $a1 with probability p1 , $a2 with probability p2 , . . ., or $an with probability pn ; B: win either $b1 with probability q1 , $b2 with probability q2 , . . ., or $bm with probability qm ; where ai (i = 1, . . . , n) and b j (j = 1, . . . , m) are  real numbers, and pi = qj = 1. A variety of experiments have been conducted in the last 30 years or so, frequently with m = n = 2, in which gambles A and B were varied from each other in their expected value, variance, or both. In other tasks, the DMs have been required to rank order several gambles of this type in terms of their attractiveness or riskiness. Other experimental tasks have been constructed to assess individual utility functions, to measure the subjective probability of both simple and compound events, to measure the perceived riskiness of gambles and relate it to the properties of the gambles, and in

particular, to test the implications of theories that have been proposed to describe human decision behavior. THEORIES Classifications of psychological theories of decision making depend on the assumptions made about the characteristics of the system. A useful three-way dichotomization is due to Luce and Suppes [23]. The first distinction is whether the theory employs algebraic or probabilistic tools. The second is whether each decision determines a certain outcome (decision making under certainty) or a probability distribution over the outcomes (decision making under risk or uncertainty). The third is whether the theory provides a complete ranking of all available courses of action or merely specifies which one will be selected. The most intensive study has been of the algebraic theory of maximizing subjectively expected utility (SEU). A characteristic of all utility theories, whether under certainty or uncertainty, is that one can assign numerical quantities (utilities) to alternatives in such a way that alternative a is chosen from a set of alternatives T if and only if the utility of a is larger than that of any other alternative in T. When it is possible to assign utilities to alternatives, it is said that the DM behaves optimally relative to his utility scale. When the decision situation is under UNDER uncertainty (see JUDGMENTS UNCERTAINTY), then in addition to assigning utilities to alternatives—risky as well as sure—numbers can also be assigned to events. These numbers are called subjective probabilities∗ and are interpreted as the DM’s evaluation of the likelihood of the event e occurring. These two numerical scales are interlocked [22] as follows: The expected utility of a risky alternative is the sum of the utilities of its component outcomes, each weighted according to the subjective probability of its occurring. Axiomatic systems that justify this representation have been provided by von Neumann and Morgenstern [38], Savage [28], and Krantz et al. [19]. Several experiments have demonstrated that subjects are not always consistent in

PSYCHOLOGICAL DECISION MAKING

their choices and that some exhibit regular patterns of inconsistency. Rather than treating these results as errors of judgment, probabilistic choice theories regard them as inherent in the choice mechanism (see CHOICE AXIOM, LUCE’S). Of the various postulates of these theories, the three that have received much empirical attention are weak, moderate, and strong stochastic transitivity. Probabilistic choice theories that are built on these or related postulates have been proposed and developed by Block and Marschak [2], Restle [27], Luce [21], and Tversky [35]. Another class of theories was developed to study diagnosis, which is conceptualized as the process of revising a subjective probability distribution over a set of events on the basis of some data. These theories have employed Bayes’ rule or modified it to yield a psychological theory of revision of opinion. Consider the simplest case of two events e0 and e1 . One form of Bayes’ rule, which yields the posterior odds in favor of e1 relative to e0 , given the data, is (e1 |X) = L(X; e1 ) (e1 ), where X denotes the data, the posterior odds (e1 |X) is the ratio of conditional probabilities p(e1 |X)/p(e0 |X), the likelihood ratio L(X; e1 ) is the ratio p(X|e1 )/p(X|e0 ), and if p0 (ei ) is the prior probability of event ei (i = 0, 1), then (e1 ), the prior odds, is the ratio p0 (e1 )/p0 (e0 ). FINDINGS The utility of money or other valuable objects has been assessed either by direct estimation or via some model. Direct estimation techniques have provided some evidence for power utility functions. Indirect measurements of utility, typically conducted via the SEU model, have yielded more variable results. Early studies concluded that it is feasible to measure utility experimentally. Later studies, which were particularly successful in inducing their subjects to be consistent, yielded both linear and nonlinear utility functions. More recent experiments have shown that experimentally derived utility functions depend to a large extent on

6603

the values of the objects, the DM’s financial status, and in particular, the properties of the utility assessment technique. Based on a series of simple demonstrations, Tversky and Kahneman [37] proposed that what they call a value function is commonly S-shaped, concave about the DM’s reference point and convex below it. Consequently, risk-averse decisions are common when DMs must choose between a sure win and a substantial probability of a larger loss, whereas risk-seeking decisions are common when choices between a sure loss and a substantial probability of a larger loss have to be made. For reviews, see Edwards [5,6], Fishburn [9], Lee [20], and Rapoport and Wallsten [26]. Subjective probabilities may also be measured either by direct psychophysical judgment methods or by inference from the SEU model. Direct psychophysical methods, which usually require the DM to estimate the proportion of one type of element in a display with stimulus elements of two or more types, show that DMs perform the task well and that subjective probability thus measured is roughly equal to observed proportion. Indirect assessment methods [33], which have been used more frequently, have shown more diverse and complex results. There is strong evidence [10] that because of their limited information processing capacity, people do not regard the world in ‘‘intuitively probabilistic’’ terms; rather, they substitute ‘‘certainty equivalents’’ for probability distributions over uncertain outcomes. People do not have satisfactory intuitive notions of the concepts of randomness∗ , variability, and probability; the subjective probability distributions derived from their decisions are often too ‘‘tight’’; and probability assessment is highly susceptible to task characteristics such as response mode, payoff, order of information presentation, and feedback. Several experiments have tested the descriptive power of the SEU model [1,7,13,16,17,40]. The bulk of the early findings showed that SEU might serve as a first-order approximation to human decisions in experiments involving a choice between a small number of gambles with objective probabilities that are easily discriminable, are quantifiable, and have outcomes that are given in monetary units. In more recent

6604

PSYCHOLOGICAL DECISION MAKING

experiments, many of which concentrated on directly testing the axioms of SEU, the proponents of SEU have been greatly outnumbered by its critics [24,32]. Several experiments showed that utility and subjective probability interact, in violation of SEU theory, or developed counterexamples (e.g., the paradoxes due to Allais and Ellsberg) to the fundamental axioms of the theory. With judicious selection of the experimental design, subjects have been enticed to persistently violate the axioms of SEU, including transitivity [34], the sure-thing principle, and the dominance principle [31]. Consequently, several researchers have proposed replacing SEU with other nonaxiomatic, algebraic theories that describe how probabilities and payoffs are integrated into decisions. Prominent among these are dimensional theories that rely on correlational and functional measurement approaches [30], and prospect theory [14]. The correlational approach, which uses multiple regression∗ techniques, postulates that the DM’s numerical responses in a choice between gambles constitute some linear combination of the available dimensions of the choice alternatives (i.e., the probabilities and payoffs). Functional measurement relies on factorial designs, quantitative response measures, and monotone transformations for rescaling these measures in attempting simultaneously to scale the stimulus attributes and response measures and to determine the combination rule relating the two. Prospect theory is a descriptive model proposed to modify SEU theory. It does so by replacing the traditional utility function over absolute amounts by a value function over gains and losses, and the probability function by a decision weighting function. The shapes of both functions are based on generalizations from several psychological studies. Although all these approaches have received some empirical support, they lack the generality and refutability of the axiomatic SEU theory. In investigating how opinions are revised, a considerable number of studies compared the DM’s numerical estimates of conditional probabilities or posterior odds to p(ei |X) or (ei |X), respectively. The general but far

from unanimous result has been that the estimates were monotonically related to these quantities but were too evenly distributed among the ei ’s; i.e., they were conservative relative to the predicted Bayesian values. Assuming the Bayesian model, three alternative explanations of the pervasive conservatism phenomenon have been offered; misperception of the data’s diagnostic impact, misaggregation of the data, or response bias. Experimental support of each explanation has been obtained, suggesting again that, as with the SEU theory, the focus of empirical research should not be whether the Bayesian theory is right or wrong, but under what empirical conditions does it hold and in what ways is it systematically violated. TRENDS Reviews of the literature on psychological decision making (e.g., Slovic et al. [32]; Einhorn and Hogarth [8]) have noted two general trends. One is a change in orientation of the experimental work. Whereas past studies mainly compared observed and predicted decisions, recent research has focused on the psychological underpinnings of observed decision behavior by attempting to determine how the underlying cognitive processes are modeled by the interaction between the limitations of the DM and the demands of the task. Much of the impetus for this change in orientation can be attributed to demonstrations [15,36] of several judgmental heuristics (e.g., representativeness, availability, and anchoring), which affect probabilistic judgments in a variety of decision tasks. Other studies (e.g., estimation of probabilities via fault trees∗ , coding processes in risky choice, alternative framings of the same decision problem, and causal schema in probability judgments) demonstrated that the process of the cognitive representation of the decision task, and the factors that affect it, are of major importance in judgment and choice. A second and related trend was a growing interest in studying the process of information search, acquisition, and storage [10,39], since evaluation and search strategies are interdependent. Concern for how information is acquired raised questions about the role of

PSYCHOLOGICAL DECISION MAKING

memory and attention in decision behavior and necessitated the use of different research methodologies to account for the dynamics of the process. Both trends reflect the connection between psychological decision making and more traditional branches of psychology. REFERENCES 1. Becker, G. M. and McClintock, C. G. (1967). Annu. Rev. Psychol., 18, 239–286. (A review of the literature, 1961–1967.) 2. Block, H. D. and Marschak, J. (1960). In Contributions to Probability and Statistics, I. Olkin, S. Ghurye, W. Hoeffding, W. Madow, and H. Mann, eds. Stanford University Press, Stanford, Calif., pp. 97–132. 3. Davidson, D., Suppes, P., and Siegel, S. (1957). Decision Making: An Experimental Approach. Stanford University Press, Stanford, Calif. (An extensive experimental study.) 4. DeGroot, M. H. (1970). Optimal Statistical Decisions. McGraw-Hill, New York. (A thorough course in the theory and methodology of optimal statistical decisions.) 5. Edwards, W. (1954). Psychol. Bull., 51, 380–417. (The first thorough review of the literature up to 1954.) 6. Edwards, W. (1961). Annu. Rev. Psychol., 12, 473–498. (A review of the literature 1954–1961.) 7. Edwards, W. and Tversky, A., eds. (1967). Decision Making. Penguin, London. (A collection of early readings, mostly experimental.) 8. Einhorn, H. J. and Hogarth, R. M. (1981). Annu. Rev. Psychol., 32, 53–88. (A review of the literature, 1977–1981.) 9. Fishburn, P. C. (1968). Manag. Sci., 14, 335–378. 10. Hogarth, R. M. (1975). J. Amer. Statist. Ass., 70, 271–294. (An attempt to consider the implications of research on cognitive processes for the practical problem of subjective probability assessments.) 11. Hogarth, R. M. (1980). Judgment and Choice: The Psychology of Decision. Wiley, New York. (A textbood that emphasizes the unstructured, natural way in which people make decisions.) 12. Hogarth, R. M. (1981). Psychol. Bull., 90, 197–217. (A discussion of the theoretical and methodological implications of considering judgment as a continuous rather than discrete process.)

6605

13. Jungerman, H. and de Zeeuw, G., eds. (1977). Decision Making and Change in Human Affairs. D. Reidel, Dordrecht, The Netherlands. (A collection of papers on judgment and choice.) 14. Kahneman, D. and Tversky, A. (1979). Econometrica, 47, 263–291. 15. Kahneman, D., Slovic, P., and Tversky, A., eds. (1982). Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press, Cambridge, England. (A collection of papers on the effects of heuristics and biases of intuitive judgments.) 16. Kaplan, M. F. and Schwartz, S., eds. (1975). Human Judgment and Decision Processes. Academic Press, New York. (A collection of papers on judgment and choice.) 17. Kaplan, M. F. and Schwartz, S., eds. (1977). Human Judgment and Decision Processes in Applied Settings. Academic Press, New York. (A collection of papers on judgment and choices in applied settings.) 18. Kleinmuntz, B., ed. (1968). Formal Representations of Human Judgment. Wiley, New York. (A collection of papers on judgment and choice.) 19. Krantz, D. H., Luce, R. D., Suppes, P. and Tversky, A. (1971). Foundations of Measurement, Vol. I. Academic Press, New York. (An excellent textbook that presents and discusses various axiomatic theories of preference and choice, including SEU, conjoint measurement, and other theories.) 20. Lee, W. (1971). Decision Theory and Human Behavior. Wiley, New York. (An elementary textbook that describes both theoretical and experimental research.) 21. Luce, R. D. (1959). Individual Choice Behavior. Wiley, New York. (A research monograph that presents Luce’s probabilistic theory of choice.) 22. Luce, R. D. (1962). In Social Science Approaches to Business Behavior, G. B. Strother, ed. Richard D. Irwin, Homewood, Ill., pp. 141–161. 23. Luce, R. D. and Suppes, P. (1965). In Handbook of Mathematical Psychology, Vol. III, R. D. Luce, R. R. Bush, and E. Galanter, eds. Wiley, New York, pp. 249–410. (An excellent and thorough review of psychological theories of decision making developed prior to 1965.) 24. March, J. G. (1978). Bell. J. Econ., 9, 587–608. (A critique of the normative theory of choice.)

6606

PSYCHOLOGICAL SCALING

25. Rapoport, A. (1975). In Utility, Probability, and Human Decision Making, D. Wendt and C. A. J. Vlek, eds. D. Reidel, Dordrecht, The Netherlands, pp. 349–369. (A review of multistage decision-making experiments.) 26. Rapoport, A. and Wallsten, T. S. (1972). Annu. Rev. Psychol., 23, 131–175. (A review of the literature, 1967–1972.) 27. Restle, F. (1961). Psychology of Judgment and Choice. Wiley, New York. (A research monograph.) 28. Savage, L. J. (1947). The Foundations of Statistics. Wiley, New York. 29. Simon, H. (1957). Models of Man: Social and Rational. Wiley, New York. (A collection of papers about decision making and bounded rationality.) 30. Slovic, P. and Lichtenstein, S. C. (1971). Organizat. Behav. Hum. Perform., 6, 649–744. (An extensive review and comparative analysis of the Bayesian and regression approaches to information utilization in judgment and choice.) 31. Slovic, P. and Tversky, A. (1974). Behav. Sci., 19, 368–373. 32. Slovic, P., Fischhoff, B., and Lichtenstein, S. C. (1977). Behavioral decision theory. Annu. Rev. Psychol., 28, 1–39. (A review of the literature, 1972–1977.) 33. Sta¨el von Holstein, C. A. S. (1970). Assessment and Evaluation of Subjective Probability Distributions. The Economic Research Institute at the Stockholm School of Economics, Stockholm. 34. Tversky, A. (1969). Psychol. Rev., 76, 31– 48. 35. Tversky, A. 281–299.

(1972).

Psychol.

Rev.,

79,

36. Tversky, A. and Kahnemann, D. (1974). Science, 185, 1124–1131. 37. Tversky, A. and Kahnemann, D. (1981). Science, 211, 453–458. (A summary of experimental demonstrations of the effects of the formulation of the decision problem on shifts of preference.) 38. von Neumann, J. and Morgenstern, O. (1947). Theory of Games and Economic Behavior, 2nd ed. Princeton University Press, Princeton, N.J. 39. Wallsten, T. S., ed. (1980). Cognitive Processes in Choice and Decision Behavior. Lawrence Erlbaum, Hillsdale, N.J. (A collection of papers on judgment and choice.) 40. Wendt, D. and Vlek, C. A. J., eds. (1975). Utility, Probability and Human Decision Making.

D. Reidel, Dordrecht, The Netherlands. (A collection of papers on decision behavior.) See also CHOICE AXIOM, LUCE’S; DECISION THEORY; JUDGMENTS UNDER UNCERTAINTY; MULTIPLE CRITERIA DECISION MAKING (MCDM); RISK THEORY; and UTILITY THEORY.

AMNON RAPOPORT

PSYCHOLOGICAL SCALING Psychological scaling refers to measuring peoples’ subjective perceptions of stimuli by analyzing their responses according to certain models. Empirical research has demonstrated that the appropriate scaling method depends both on the nature of the stimuli and the nature of the responses. The origins of psychological scaling reside in psychophysics∗ , and stem from a classic work by Gustav T. Fechner, the Elemente der Psychophysik, 1860 [3], often cited also as marking the formal beginning of experimental psychology. Fechner, trained in physics, discovered the earlier work of a physiologist, Ernst H. Weber, who had experimentally demonstrated that the amount of change in magnitude of a physical stimulus required to produce a ‘‘just noticeable difference’’ in sensation was in a fixed ratio to the initial stimulus magnitude. By assuming that all just noticeable differences of sensation are equal and proportional to the ratio of change in stimulus magnitude to initial stimulus magnitude, Fechner derived the Weber–Fechner equation—that sensation is proportional to the logarithm of stimulus magnitude. The classical Fechnerian psychophysical methods for gathering and analyzing data were presented and illustrated in detail by the Cornell psychologist, E. B. Titchener [9]. They typically require a ‘‘subject’’ to adjust physical magnitudes to satisfy some criterion of sensation (e.g., to adjust a variable stimulus to be detectably different from a standard stimulus) or to judge whether pairs of fixed stimuli are ‘‘the same’’ or ‘‘different.’’ From the results of many such judgments, it is possible to estimate values for the slope and intercept in the Weber–Fechner equation. S. S. Stevens, a psychologist at Harvard, contributed greatly to psychophysical research, from the 1930s through the

PSYCHOLOGICAL SCALING

1960s. He postulated an alternative to the Weber–Fechner equation for a broad class of stimuli, whereby the psychological magnitude is related to the physical stimulus magnitude by a power function [7]. Under this formulation, the logarithm of sensation, rather than sensation itself, is proportional to the logarithm of physical stimulus magnitude. In a series of papers in 1927, the Chicago psychologist L. L. Thurstone developed methods for scaling psychological variables that, unlike those studied in psychophysics, need not be directly coupled with physical stimulus magnitudes. Thurstone’s methods could be and have been used to ‘‘measure’’ or ‘‘scale’’ such psychological variables as attitudes, preferences, or aesthetic values. The fundamental assumption underlying Thurstone’s methods of psychological scaling is that of a normal distribution of ‘‘discriminant processes’’ associated with stimuli. The distribution is either over occasions, as with a single judge, or over judges. Data may be collected using the method of paired comparisons∗ , in which pairs of stimuli, Xi , Xj , are presented, and the judge must respond that Xi > Xj or Xj > Xi , where ‘‘>’’ represents a defined order relation on the subjective scale (e.g., ‘‘preferred’’ or ‘‘more favorable’’). Alternatively, data may be collected using the method of successive categories, in which each stimulus in a set is assigned by a judge to one of a group of categories that are ordered on the relevant

6607

subjective scale. In either case, a subjective continuum is estimated so as to most closely yield simultaneous normal distributions of transformed judgment values for all stimuli. The mean and the standard deviation of a resultant distribution for each stimulus are taken as estimates for the two parameters of the distribution of discriminal processes for that stimulus. To illustrate the method of successive categories, consider first the following experimental design. A set of stimuli, X1 , X2 , . . . , Xn is presented to N subjects, randomly selected from a specified subject population. Temporal positions of the Xj are independently randomized for each subject. Subject i is asked to judge each Xj in terms of an explicitly defined attribute, and to classify it into one and only one ordered attribute category k, k = 1, 2, . . . , m. Data are recorded in the form pjk , the cumulative proportion of judgments for the N subjects in which Xj is placed below the upper bound of category k. The model for relating parameters to data by the method of categorical judgment is detailed by Bock and Jones [1, pp. 214–237]. Estimated parameters include scale values for category boundaries, and means and variances of scale values for the stimuli. As an example of the use of the method of successive categories, consider data on food preferences collected from a sample of 255 Army enlisted men. Table 1 shows the frequency with which each of 12 food items was rated in each of nine categories on a rating

Table 1. Frequencies of Ratings over Nine Successive Categories for 12 Food Items Successive Category Food Item

1

2

3

4

5

6

7

8

9

N

No Response

1. Sweetbreads 2. Cauliflower 3. Fresh pineapple 4. Parsnips 5. Baked beans 6. Wieners 7. Chocolate cake 8. Salmon loaf 9. Blueberry pie 10. Turnips 11. Liver 12. Spaghetti

24 15 4 17 4 0 0 8 2 19 21 1

31 6 1 30 5 2 1 9 0 30 16 1

26 10 2 31 6 7 2 11 3 33 10 8

29 26 7 45 17 9 7 23 14 48 15 9

47 27 16 47 27 28 20 36 24 35 23 24

37 62 40 32 60 76 33 74 41 40 42 61

32 72 75 33 72 91 74 64 70 32 71 92

13 30 77 11 56 38 85 25 70 15 46 46

4 6 31 4 7 2 32 3 28 1 9 12

243 254 253 250 254 253 254 253 252 253 253 254

12 1 2 5 1 2 1 2 3 2 2 1

Source: Ref. 1.

6608

PSYCHOLOGICAL SCALING

form. (The categories were labeled from ‘‘dislike extremely’’ at the left to ‘‘like extremely’’ on the right.) These frequencies are converted into cumulative proportions (pjk ) in Table 2, from which are derived the unit normal deviates (yjk ) of Table 3. (An extrapolative method provides estimates of yjk when pjk is less than 0.01 or greater than 0.99.) The values

category (labeled ‘‘neither like nor dislike’’) a scale value of 0, we then estimate scale values of the category boundaries, τˆk , as shown in the final row entries of Table 3. Mean scale values and variances for the stimuli may be estimated by an algebraic solution, or graphically, by determining the intercept and slope for linear fits of the graphs of the pjk plotted against the τk on normal probability paper. Graphical estimates of mean scale values and standard deviations for the 12 food items are shown in Table 4.

dk = yk − yk−1 represent scaled category widths. By arbitrarily assigning the center of the middle

Table 2. Cumulative Proportions, pjk , of Ratings over Nine Successive Categories for 12 Food Items Successive Category Food Item 1 2 3 4 5 6 7 8 9 10 11 12

1

2

3

4

5

6

7

8

9

099 059 016 068 015 000 000 032 008 075 083 004

226 083 020 188 035 008 004 067 008 194 146 008

333 122 028 312 059 036 012 111 020 324 186 039

453 224 055 492 126 071 039 202 075 514 245 075

646 331 119 680 232 182 118 344 171 652 336 169

798 575 277 808 469 482 248 636 333 810 502 409

930 858 573 940 752 842 539 889 611 937 783 772

984 976 877 984 972 992 874 988 889 996 964 953

1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000 1.000

Source: Ref. 1.

Table 3. Normal Deviates, yjk , Associated with the pjk of Table 2 Successive Category Food Item 1 2 3 4 5 6 7 8 9 10 11 12  y  jk yjk /12 dk τˆk

1

2

3

4

5

6

7

8

−1.287 −0.752 −0.432 −0.118 0.374 0.834 1.476 2.144 −1.563 −1.385 −1.165 −0.759 −0.437 0.189 1.071 1.977 −2.144 −2.054 −1.911 −1.598 −1.180 −0.592 0.184 1.160 −1.491 −0.885 −0.490 −0.020 0.468 0.870 1.555 2.144 −2.170 −1.812 −1.563 −1.146 −0.732 −0.078 0.681 1.911 (−2.450) (−2.071) −1.799 −1.468 −0.908 −0.045 1.003 (1.937) (−2.908) (−2.529) −2.257 −1.762 −1.185 −0.681 0.098 1.146 −1.852 −1.498 −1.221 −0.834 −0.402 0.348 1.221 2.257 (−2.705) (−2.326) −2.054 −1.440 −0.950 −0.432 0.282 1.221 −1.440 −0.863 −0.456 0.035 0.391 0.878 1.530 (2.464) −1.385 −1.054 −0.893 −0.690 −0.423 0.005 0.782 1.799 (−2.413) (−2.034) −1.762 −1.440 −0.958 −0.230 0.745 1.675 −23.808 −19.263 −16.003 −11.240 −5.942 1.066 10.628 21.835 −1.984 −1.605 −1.333 −0.937 −0.495 0.089 0.886 1.820 0.379 0.272 0.396 0.442 0.584 0.797 0.934 −1.268 −0.889 −0.617 −0.221 0.221 0.805 1.602 2.536

Source: Ref. 1.

Sum 2.239 −2.072 −8.135 2.151 −4.909 −5.801 −10.078 −1.981 −8.404 2.539 −1.859 −6.417 −42.727

PSYCHOLOGICAL TESTING THEORY Table 4. Graphical Estimates for ˆιj and σˆ j , 12 Food Items Food Item 1. Sweetbreads 2. Cauliflower 3. Fresh pineapple 4. Parsnips 5. Baked beans 6. Wieners 7. Chocolate cake 8. Salmon loaf 9. Blueberry pie 10. Turnips 11. Liver 12. Spaghetti

µˆ j

σˆ j

−0.10 0.62 1.41 −0.18 0.88 0.82 1.46 0.52 1.25 −0.20 0.67 0.99

1.13 1.05 1.02 1.00 0.97 0.72 1.00 0.89 1.07 1.06 1.30 0.83

Source: Ref. 1

The original Thurstone papers on psychological scaling are reprinted in Thurstone [8]. Further exposition may be found in Edwards [2], Torgerson [10], Bock and Jones [1], and Luce [6]. A distinct concept of scaling, with primary applicability to attitude measurement, is that of Louis Guttman [5], a quantitative social scientist at Hebrew University of Jerusalem. Consider n statements, each representing a level of attitude or belief regarding a selected issue, where each statement might be ‘‘accepted’’ or ‘‘rejected’’ by a particular person. Let the statements be ordered by level of belief, lowest to highest, 1, 2, . . . , n. Let i represent the highest ordered statement that a person accepts, 1 < i < n. Then the set of statements form a Guttman scale if that person accepts all statements 1, 2, . . . , i and rejects all statements i + 1, i + 2, . . . , n. In practice, of course, this condition sometimes might be violated even when, intrinsically, there exists a Guttman scale. This has led to the development of statistical models of scalability (e.g., Goodman [4]). A form of psychological scaling that has received much attention in recent years is multidimensional scaling∗ . Related to psychological scaling are theoretical developments in the theory of psychological measurement. See MEASUREMENT THEORY. REFERENCES 1. Bock, R. D. and Jones, L. V. (1968). The Measurement and Prediction of Judgment and Choice. Holden-Day, San Francisco.

6609

2. Edwards, A. L. (1957). Techniques of Attitude Scale Construction. Appleton-Century-Crofts, New York. 3. Fechner, G. T. (1966). Elements of Psychophysics (English transl.). Holt, Rinehart and Winston, New York. 4. Goodman, L. A. (1975). J. Amer. Statist. Ass., 70, 755–768. [Also appears in Goodman, L. A. (1978). Analyzing Qualitative/Categorical Data, Abt Books, Cambridge, Mass., pp. 363–401.] 5. Guttman, L. (1950). In Measurement and Prediction, S. A. Stouffer et al. Princeton University Press, Princeton, N.J., pp. 60–90. 6. Luce, R. D. (1977). Psychometrika, 42, 461–489. 7. Stevens, S. S. (1961). Science, 133, 80–86. 8. Thurstone, L. L. (1959). The Measurement of Values. University of Chicago Press, Chicago. 9. Titchener, E. B. (1905). Experimental Psychology, Vol. II. Macmillan, New York. 10. Torgerson, W. S. (1958). Theory and Methods of Scaling. Wiley, New York. See also MEASUREMENT THEORY; MULTIDIMENSIONAL SCALING; PAIRED COMPARISONS; PSYCHOLOGY, STATISTICS IN; and PSYCHOPHYSICS, STATISTICAL METHODS IN.

LYLE V. JONES

PSYCHOLOGICAL TESTING THEORY Civil service testing began in China about 3000 years ago, according to DuBois [3]. The Jesuits used written tests in the sixteenth century. A psychological test is a device for collecting a sample of behavior. A basic problem is how to quantify the behavior. Problems include the optimal design of tests and scoring methods, their evaluation, and the proper interpretation of scores (see also FACTOR ANALYSIS). RELIABILITY Publishers try to prepare parallel forms of a test: forms yielding scores having identical statistical properties. The correlation ρXX  between scores X and X’ on parallel forms is the test reliability. By definition, µX = µX  , σX = σX  , and ρXV = ρX  V , where V is some external variable.

6610

PSYCHOLOGICAL TESTING THEORY

COMPOSITE SCORES AND TAUTOLOGIES Frequently, the testee’s score X is the sum of his orher scores Yi on n subtests or questions: X ≡ i Yi . If Yi = 1 (correct) or 0 (incorrect), then X is the number-right score. Two useful algebraic tautologies follow, expressing properties of X in terms of properties of the Yi : σX ≡

n 

σ (Yi )ρ(X, Yi ),

(1)

i=1

ρXV

n i=1 σ (Yi )ρ(V, Yi )   ≡ . { ni=1 nj=1 σ (Yi )σ (Yj )ρ(Yi , Yj )}1/2 (2)

If V is a criterion, ρXV is called the test validity. A useful lower bound for the test reliability is   n 1  2 n σ (Yi )  ρXX  . (3) 1− 2 a≡ n−1 σX i=1 This is Cronbach’s α, often cited as the ‘‘test reliability,’’ whereas it should be called a lower bound. When Yi = 0 or 1, a more easily computed lower bound, called KuderRichardson∗ formula 21, is KR(21) ≡

n {1 − µX (n − µX )/(nσX2 )} n−1

 α  ρXX  .

(4)

If the Yi are parallel, ρXX  ≡ nρYY  /{1 + (n − 1)ρYY  };

(5)

this is the Spearman-Brown formula, showing the effect of test length on ρXX  . For a test containing n = 50 questions or items scored 0 or 1, typical values might  be σ (Yi ) = 0.4583, ρ(Y, Y ) = 0.14, ρ(X, Yi ) = 0.3965, µX = 35. If all Yi are parallel, we find from (1) that σX = 9.085. From (3) to (5) in this case, ρXX  = α = KR(21) = 0.89. ITEM ANALYSIS AND TEST DESIGN An item analysis typically determines the proportion πi of all testees answering item i correctly, also some measure of item-test

correlation such as ρ(X, Yi ). If Yi = 0 or 1,  µX = i πi ; also σ 2 (Yi ) = πi (1 − πi ). As seen in (5), ρXX  is maximized by using items with high intercorrelations. In contrast, as seen in (2), the test validity is maximized by using items with high ρ(V, Yi ) and low intercorrelations. TRUE SCORE 



If we had scores X, X , X , . . ., on parallel test forms for a given testee, we would ordinarily use the mean of these scores in preference to any single score. This establishes an important conclusion: there is an (unobservable) latent variable T (called the true score) that is more important to us than the observed score X. X is here considered a random variable for a single testee and also across testees. T is defined as the expectation (E) of X for a fixed testee; T is a random variable across testees. ERROR OF MEASUREMENT E≡X −T

(6)

is the error of measurement in X. The errors of measurement impair the effectiveness of X as a substitute for T. The main purpose of classical test theory is to evaluate the effects of E and to try to determine what the data would be like if the errors were eliminated. From (6), E(E|T), the expectation of E for fixed T, is zero; the errors are unbiased. Thus in any group of people ρET = 0. Note that these are not assumptions, they follow from the definition of T. Also, from (6), for a group of people σX2 = σT2 + σE2 , σXT (the covariance of 2 X and T) = σT2 , and ρXT = 1 − σE2 /σX2 . CLASSICAL TEST THEORY The only latent-variable parameter that can be inferred from data up to this point is µT ≡ µX . Two assumptions are now made: ρ(EX , EY ) = 0,

ρ(EX , TY ) = 0,

(7)

where the subscripts denote two different tests. If we now administer a parallel test

PSYCHOLOGICAL TESTING THEORY 





form, denoted by a prime [T = T, σ (E |T ) = σ (E|T)], we can determine the important latent parameters from 2 (= ρ 2   ) = ρXX  ρXT X T

σT2 (=

σ 2 ) = σX2 ρXX 

σE2 (=

σ 2 ) E

T

=

σX2 (1

(8)

− ρXX  ).

For the numerical example in the section ‘‘Composite Scores and Tautologies,’’ we find from (8) that ρXT = 0.944, σT = 8.574, and σE = 3.005. For two tests X and Y, ρ(TX , TY ) = √ ρXY / ρXX  ρYY  . This correction for attenuation makes it possible to estimate the correlation between true scores on two tests. It follows  that ρXY  ρXT , in other words, validity  reliability. Classical test theory relies heavily on correlations. This is because the score units of a test are meaningless to most readers. When the score units are familiar, σE2 is a better measure of test effectiveness than ρXX  ; σE2 is more stable across groups of testees. No Gaussian distribution assumptions have been made. Higher-order moments of T, E, and E can be inferred [7, Chap. 10]. CRITERION-REFERENCED TESTS Tests are sometimes used only to provide a ranking of testees. When test scores have meaning on an externally fixed scale, however, their accuracy must be evaluated absolutely, not relatively. Correlation coefficients are no longer appropriate for such an evaluation; a part of classical test theory must be replaced (see ref. 1). ITEM SAMPLING AND MATRIX SAMPLING In a survey, group means and variances of scores may be important rather than individual scores. In this case, for fixed testing time, it is more efficient to administer a different sample of items to different groups of examinees than to give everyone the same test [11]. If the items are sampled at random from a pool of items and true score is defined as

6611

expected performance on the entire pool of items, then the key parameters µT , σT2 , σE2 , ρXT can be estimated from the observed scores without need for parallel forms and without any assumptions. For a given testee, number-right score X has a binomial distribution. An unbiased estimator of σ 2 (E|T) is X(n − X)/(n − 1). Across all testees taking the same test, an unbiased estimator of σE2 is [X(n − X) − s2X ]/(n − 1), where s2X is the sample variance of X [7, Chap. 11]. Stratified sampling is desirable in drawing items from a pool to build a test. This implements domain-referenced testing [5]. SCORING Given the polychotomous responses of many testees to a set of test items, how shall a score be assigned to each examinee? A solution to this problem, with no special assumptions made (see [12, Chap. 12]) was obtained by Guttman [4]. The solution is related to the method of reciprocal averages [10]. Scoring weights can be assigned empirically so as to maximize test reliability or test validity [7, Chap. 5]. Wilks [13] showed that an unweighted average of item scores will often be almost as good as a weighted average. When items are scored dichotomously, the number of correct answers (an unweighted average) is the usual test score. Across a group of testees, such scores often approximately follow the beta-binomial distribution [7, Chap. 23; 8]. ITEM RESPONSE THEORY (IRT) (See LATENT STRUCTURE ANALYSIS.) IRT (and test scores) are most useful when all items measure a single dimension of ‘‘ability’’ . IRT provides a statistical model for the probability of a correct response to an item [6]. This probability Pi (θ ) is an increasing function of the testee’s θ . If Pi (−∞) = 0 and Pi (∞) = 1, then Pi (θ ) necessarily has the form of a cumulative frequency distribution. If, because of guessing behavior, Pi (−∞) = γi > 0, then Pi (θ ) is commonly modeled by Pi (θ ) ≡ Pr(Yi = 1|θ ) = γi + (1 − γi )F[αi (θ − βi )],

(9)

6612

PSYCHOLOGICAL TESTING THEORY

where αi , βi , and γi are parameters describing item i, and where F is a cumulative distribution function, usually normal or logistic. (Note: F is not related to the frequency distribution of θ in any group of testees.) Since βi is a property of the items that offsets θ , βi represents the item difficulty. Similarly, αi represents item discriminating power. In a unidimensional test, by definition, all Yi are independent when θ is fixed (local independence). In IRT a test is evaluated by the Fisher information∗ function of the test score X:  I≡

d dθ E(X|θ )

2

var(X|θ )

(10)

[2, Chap. 20], a function of θ , inversely proportional to the square of the length of the asymptotic confidence interval for estimating θ from X = x. When the IRT model holds, the parameters of an item are invariant across groups of testees, except for an arbitrary origin and unit, and a testee’s θ is invariant across tests. IRT allows us to predict the properties of all types of test scores from the characteristics of the test items. IRT is useful for test design, for test evaluation, and for optimal scoring. The Rasch model∗ is a version of (9) in which items cannot be answered correctly by guessing (all γi = 0), all items are equally discriminating (all αi = 1), and F is logistic∗ . Under the Rasch model, there is a sufficient statistic (number-right score) for θ , independent of the βi . Rasch [9] urges that this unique property be required before we can speak of ‘‘measurement.’’

REFERENCES 1. Berk, R. A., ed. (1980). Criterion-Referenced Measurement: The State of the Art. Johns Hopkins University Press, Baltimore, Md. 2. Birnbaum, A. (1968). In Statistical Theories of Mental Test Scores, F. M. Lord and M. R. Novick, eds. Addison-Wesley, Reading, Mass., Part 5. (The basic ideas of IRT, as developed by a well-known statistical theorist.) 3. DuBois, P. H. (1970). A History of Psychological Testing. Allyn and Bacon, Boston. 4. Guttman, L. (1941). In The Prediction of Personal Adjustment, P. Horst et al., eds. Social Sciences Research Council, New York, pp. 321–344. (Modern statisticians keep reinventing the test scoring methods derived here.) 5. Hively, W. (1974). Domain-Referenced Testing. Educational Technology Publications, Englewood Cliffs, N.J. 6. Lord, F. M. (1980). Applications of Item Response Theory to Practical Testing Problems. Lawrence Erlbaum, Hillsdale, N.J. (Basic text on IRT.) 7. Lord, F. M. and Novick, M. R. (1968). Statistical Theories of Mental Test Scores. AddisonWesley, Reading, Mass. 8. Morrison, D. G. (1978). Amer. Statist., 32, 23–35. 9. Rasch, G. (1961). Proc. 4th Berkeley Symp. Math. Statist. Prob., Vol. 4. University of California Press, Berkeley, Calif., pp. 321–323. 10. Richardson, M. and Kuder, G. F. (1933). Pers. J., 12, 36–40. 11. Shoemaker, D. M. (1973). Principles and Procedures of Multiple Matrix Sampling. Ballinger, Cambridge, Mass. (The best available book on this subject.) 12. Torgerson, W. S. (1958). Theory and Methods of Scaling. Wiley, New York. (Basic test on assigning numbers to objects or to people.) 13. Wilks, S. S. (1938). Psychometrika, 3, 23–40.

TAILORED TESTING In tailored testing, the items administered to a testee are chosen from a pool, one at a time, so as to estimate his or her ability θ as accurately as possible. Since different testees take different sets of items, IRT is necessary in order to assign comparable scores. The testee’s score is his or her estimated θ , obtained by some standard statistical estimation procedure.

BIBLIOGRAPHY See the following works, as well as the references just given, for more information on the topic of psychological testing theory. Bock, R. D. and Aitkin, M. (1981). Psychometrika, 46, 443–459. (Two mathematical statisticians on the frontiers of IRT.) Cronbach, L. J., Gleser, G. C., Nanda, H., and Rajaratnam, N. (1972). The Dependability of

PSYCHOLOGY, STATISTICS IN Behavioral Measurements: Theory of Generalizability for Scores and Profiles. Wiley, New York. (The authoritative treatise on test reliability.) Gulliksen, H. (1950). Theory of Mental Tests. Wiley, New York. (Very readable basic reference on classical test theory.) Holland, P. W. and Rubin, D. B., eds. (1982). Test Equating. Academic Press, New York. (Mathematical statisticians tackle test score equating.) Novick, M. R., Jackson, P. H., and Thayer, D. T. (1971). Psychometrika, 36, 261–268. (A Bayesian treatment of classical test theory.) Popham, W. J., ed. (1971). Criterion-Referenced Measurement: An Introduction. Educational Technology Publications, Englewood Cliffs, N.J. Solomon, H., ed. (1961). Studies in Item Analysis and Prediction. Stanford University Press, Stanford, Calif. (An early collection of original work by mathematical statisticians.) Thorndike, R. L., ed. (1971). Educational Measurement, 2nd ed. American Council on Education, Washington, D.C. (Authoritative readable handbook.) Wright, B. D. and Stone, M. H. (1978). Best Test Design: A Handbook for Rasch Measurement. Scientific Press, Palo Alto, Calif. See also FACTOR ANALYSIS; GROUP TESTING; LATENT STRUCTURE ANALYSIS; MEASUREMENT STRUCTURES AND STATISTICS; MULTIPLE MATRIX SAMPLING; RASCH MODEL; and RELIABILITY COEFFICIENTS, KUDER –RICHARDSON.

FREDERIC M. LORD

PSYCHOLOGY, STATISTICS IN Psychology is a uniquely diverse discipline, ranging from biological aspects of behavior on the one hand to social psychiatry on the other, and from basic research to the applied professions of clinical, counseling, educational, industrial, organizational, and forensic psychology [4]. The relationship between psychology and statistics is a necessary one, although the claim by Cramer [5] that they are ‘‘sister sciences’’ is perhaps a little exaggerated. Early psychologists were familiar with Francis Galton’s claim that ‘‘until the phenomena of any branch of knowledge have been submitted to measurement and number, it cannot assume the dignity of a science’’ [12], and as a consequence submitted their observations to measurement and quantification. The resulting

6613

data were then often subjected to some form of statistical analysis. For example, in a paper on reaction times, Yerkes [28] discusses a range of statistical measures of spread, particularly those enabling researchers to equate stimulus intensity values across modalities in terms of reaction time variability. Yerkes also discusses the relative merits of the median and the mode as alternatives to the arithmetic mean (reported in ref. 19). And a century and a half previously, David Hartley in his major work, Observations on Man, discussed the relevance of probability theory to the collection of scientific evidence and argued for the use of mathematical and statistical ideas in the study of psychological processes. Statistics courses for psychologists first appeared in the latter part of the nineteenth century, Walker [29] crediting the psychologist Cattell as having taught such a course in 1887. In the early part of the twentieth century, books with the aim of acquainting psychologists with the developments in statistics began to be published. In 1911, for example, Brown published a text ‘‘primarily for the professed psychologist’’ that he hoped would ‘‘also prove of use to the educationalist.’’ Four of the chapters were concerned with correlation*, a concept introduced by Galton* in the 1880s and later developed at a more theoretical level by Karl Pearson*. It was applications of correlation and the associated technique of regression* that proved to be the staple statistical diet of psychologists in the early part of the twentieth century, particularly amongst educational psychologists, as is evidenced by the following quotation from Walker [29]: The influence of Pearson upon educational statistics in America has been so profound that it is all but impossible to overstate the extent to which he is responsible for modern statistical theory. (As we shall see later, the concentration on correlations at this stage also led to the introduction of the basic ideas of factor analysis*.)

A decade later, the work of ‘‘Student’’ (William Gosset) and, in particular, of R. A. Fisher* formalized John Stuart Mill’s experimental ‘‘method of difference’’ for comparing

6614

PSYCHOLOGY, STATISTICS IN

groups of subjects on some variable of interest and a second strand of statistical methodology was added to the psychologist’s bow. The significance test and analysis of variance (ANOVA) had arrived in psychology; the start, for many psychologists, of a life-long love affair. In some ways, this historical divide still has implications for the way psychologists address psychological questions, reflecting as it does what Cronbach [6] identified as two quite separate disciplines of scientific psychology. One sort of psychologist attempts to test general principles about human and animal behavior, and is concerned with documenting average performance; the other sort of psychologist is interested in describing and interpreting individual differences, in particular with respect to various dimensions of intellectual ability, personality, and psychopathology. The first sort of psychologist does experiments, typically on small samples obtained for reasons of convenience. The other sort of psychologist does larger-scale questionnaire surveys or interview studies, attempts to procure representative samples, and tends to use standardized, prevalidated measures. When analyzing results, the first sort of psychologist tends, for example, to compute t-tests and analyses of variance. The second sort tends to use correlation, regression, and factor-analytic techniques. Thus, two historically separate areas of substantive inquiry within psychology have been linked to different aspects of statistical theory. Even if this division is nowadays somewhat artificial it remains useful, at least here, since it allows a convenient separation of the following discussion, into significance tests, P-values, models and ANOVA on the one hand, and latent variables and factor analysis on the other. SIGNIFICANCE TESTS AND P-VALUES The first published use of reasoning that resembles a significance test is commonly attributed to John Arbuthnot* [1]; it involved a comparison of male and female births in London in each of 82 yrs. when birth records were kept. He calculated, on the assumption of an equal chance of a male or female being

born at any given birth, that the observed finding in all 82 yrs. of an excess of male births was exceedingly improbable. Another early example of something akin to a significance test was Mitchell’s argument in 1767 that the stars were not distributed at random. It was much later, however, that Edgeworth [8] introduced a procedure that most present day psychologists would see as a relatively accurate description of their approach to inference: In order to determine whether the difference between two proposed means is or is not accidental, form the probability curve under which the said difference, supposing it were accidental, would range. Consider whether the difference between the observed means exceeds two to three times the modules of that curve. If it does, the difference is not accidental.

Later, Edgeworth described such a result as significant. In the 1920s workers such as Fisher, Neyman, and Egon Pearson created various testing methodologies. Since then, the significance test, in all its manifestations, parametric and nonparametric, has been well and truly adopted by the psychologist. The psychological literature is littered with the results of the various sorts of tests; Student’s t-tests and Mann-Whitney Us, for example, abound. Indeed the latter, along with the host of other available nonparametric procedures that use only the ranked data, are particularly favored by many psychologists concerned about scales of measurement and assumptions of normality—overly concerned in many instances (see MEASUREMENT STRUCTURES AND STATISTICS). In addition, psychological journals are often liberally decorated with asterisks (*, **, or even ***) to denote varying degrees of statistical significance. (Sprent [27] comments that the ‘‘star’’ nomenclature is more suited to a hotel guide book than a serious scientific paper.) Over 20 yrs. ago, Oakes [22] showed that over 80% of articles in psychology journals used significance tests. We should note that the reliance on significance testing is not restricted to the experimental literature, where the setting of specific decision criteria a priori is part of the overall logic of the

PSYCHOLOGY, STATISTICS IN

research design. Significance tests are used as well in quasi–experimental designs, for example, in which a clinical group is being compared with matched control subjects, and often significance tests are all psychologists note from the results of regression analyses (see also SIGNIFICANCE TESTS, HISTORY AND LOGIC OF). Apparently, the significance test satisfies the need of a researcher who is searching for a way to distinguish between ‘‘weighty’’ and ‘‘less weighty’’ findings. But is such enthusiasm for the significance test and, in particular, for the associated P-value entirely justified? Although the P-value is probably the most ubiquitous statistical index found in the applied sciences literature and is particularly widely used in psychology, many statisticians, and quite a few psychologists, would answer ‘‘No.’’ Lets consider some reasons why. The P-value, as it is used and defined today, was first proposed as part of a quasiformal method of inference by Fisher in his influential 1925 book, Statistical Methods for Research Workers; it is defined as the probability of obtaining the observed data (or data that represent a more extreme departure from the null hypothesis), if the null hypothesis is true (see P-VALUES and P-VALUES AND POWER OF TESTS). For Fisher, the P-value represented an attempt to provide a measure of evidence against the null hypothesis; but Fisher intended it to be used informally (the smaller the P-value the greater the evidence against the null hypothesis), rather than providing a division of the results into ‘‘significant’’ or ‘‘nonsignificant.’’ Unfortunately it seems that despite the many caveats in the literature (for example, refs. 13 and 23), the ‘‘accept/reject’’ philosophy of hypothesis testing remains seductive to many researchers, including psychologists, who seem determined to continue to express joy on achieving a P-value of 0.049, and despair on finding one of ‘‘only’’ 0.051 (0.05 being the almost universally accepted threshold for labelling results, ‘‘significant’’ or ‘‘nonsignificant’’). Many psychologists seem to internalize the difference between a P-value of 0.05 and one of 0.06 as ‘‘right’’ versus ‘‘wrong,’’ ‘‘creditable’’ versus ‘‘embarrassing,’’ ‘‘success’’ versus ‘‘failure,’’ and, perhaps, the ‘‘renewal of grants’’ versus

6615

‘‘termination’’. Such practice was definitely not what Fisher had in mind as is evidenced by the following quotation [11]: A man who ‘rejects’ a hypothesis provisionally, as a matter of habitual practice, when the significance is 1% or higher, will certainly be mistaken in not more than 1% of such decision . . .. However, the calculation is absurdly academic, for in fact no scientific worker has a fixed level of significance at which from year to year, and in all circumstances, he rejects hypotheses; he rather gives his mind to each particular case in the light of his evidence and his ideas.

The most common alternative to presenting results from a clinical trial* in terms of Pvalues, in relation to a statistical null hypothesis, is to estimate the magnitude of the difference of a measured outcome between the groups under investigation, along with some interval that includes the population value of the difference with some specified probability, that is, to construct a confidence interval. This is an intuitively sensible approach since many psychological objectives translate into a need to estimate an important quantity. Such confidence intervals can be found relatively simply for many quantities of interest [13], and although the underlying logic of interval estimation is essentially similar to that of significance testing, they do not carry with them the pseudoscientific hypothesis testing language of such tests. Instead, they give a plausible range of values for the unknown difference. As Oakes [23] rightly comments, the significance test relates to what the population parameter is not; the confidence interval gives a plausible range for what the parameter is.

According to Gardner and Altman [13], Overemphasis on hypothesis testing - and the use of P-values to dichotomise significant or nonsignificant results - has distracted from more useful approaches to interpreting study results, such as estimation and confidence intervals. . .. The excessive use of hypothesis testing at the expense of other ways of assessing results has reached such a degree that levels of significance are often quoted alone in the main

6616

PSYCHOLOGY, STATISTICS IN

text and abstracts of papers, with no mention of actual concentration, proportions etc., or their differences. The implications of hypothesis testing - that there can always be a simple ‘yes’ or ‘no’ answer as the fundamental result from a medical study - is clearly false, and used in this way hypothesis testing is of limited value.

These comments are well illustrated by the following quotation taken from a report of a clinical trial comparing olanzapine and haloperidol for treating the symptoms of schizophrenia. Patients treated with olanzapine showed an average decrease of 10.9 points on the Brief Psychiatric Rating Scale; patients treated with haloperidol reported an average decrease of 7.9 points. This difference was statistically significant.

Perhaps partly as a result of Gardner and Altman’s paper, the use and reporting of confidence intervals have become more widespread in the medical literature in the past decade. Indeed, a considerable number of medical journals now demand such intervals rather than simply P-values. In many psychological journals however, there appears to be a continuing commitment to Pvalues; certainly there is no discernable move away from their use. There should be. ANALYSIS OF VARIANCE IN PSYCHOLOGY Fisher laid the mathematical foundations of ANOVA* in the early 1920s [10]. He described the technique thus: the separation of variance ascribable to one group of causes from the variance ascribable to the other groups.

Stated another way, the ANOVA is a partitioning of the total variance in a set of data into a number of component parts, so that the relative contributions of identifiable sources of variation to the total variation in measured responses can be determined. Analysis of variance was originally developed for use in agricultural research, and it was some time before psychologists realized its potential in the analysis of data from

psychological studies. According to Lovie [18], the earliest psychology paper that had ‘‘ANOVA’’ in its title was by Gaskill and Cox [15]. After 1940, there was a dramatic increase in the use of ANOVA reported in the psychological literature and, by 1943, the review paper of Garrett and Zubin [14] was able to cite 40 studies using ANOVA or covariance in psychology and education. Since then, ANOVA has become the central technique of analysis in experimental psychology. But for such a frequently used technique, there is considerable evidence that many psychologists have only a relatively tenuous appreciation of the form of statistical model on which it is based—indeed, according to Draper and Smith [7], the question ‘‘What model are you considering?’’ when posed to many experimental psychologists is often met with the response, ‘‘I am not considering one—I am using analysis of variance.’’ This lack of awareness, of what lies beneath the surface of an ANOVA, becomes even more apparent in the rigid distinction that most psychologists (and, unfortunately, many writers of statistics books for psychologists) draw between ANOVA and multiple regression*. Few psychologists seem aware that the techniques are essentially equivalent and based on the same model, or that this model can be extended to provide an elegant and unifying framework in which other than normally distributed variables can be dealt with, even though the resulting generalized linear model* has been around for nearly 30 yrs. (Nelder and Wedderburn [21]). For so-called repeated measure designs (see REPEATED MEASUREMENTS, DESIGN AND ANALYSIS FOR), in which a response variable of interest is recorded on more than one occasion or under more than one condition for each subject, the situation is, if anything, even worse. Many psychologists deal with such data with standard repeated measures ANOVA (see, for example, ref. 9), or perhaps multivariate analysis of variance (MANOVA), seemingly unaware of the developments of more appropriate models for such data developed over the last decade or so, particularly when the data contain missing values. Multilevel models [16], linear mixed models [25], and generalized

PSYCHOLOGY, STATISTICS IN

estimating equations* [24] have yet to make a major impact in psychology; but as, in particular, psychological treatments evolve and are subjected to the rigors of the clinical trial to gauge their effectiveness, psychological researchers will have to learn more about these relevant and powerful methodologies. LATENT VARIABLES AND FACTOR ANALYSIS The concept of a coefficient of correlation expressing a relationship between two variables was introduced by Galton in the late nineteenth century; a suitable statistical theory was later developed by Pearson. The introduction of correlation was closely followed by the basic ideas of what is now generally referred to as factor analysis*. In 1904, for example, Spearman [26] suggested that the correlation between two variables signified the existence of an underlying common factor, which became the basis of his theory of general intelligence. Spearman’s ideas were adopted and extended by other psychologists who saw the correlation coefficient as the key into the mysteries of personality and mental life. They equated the common factors derived from a matrix of correlations with psychological traits that could be given meaningful labels, for example, intelligence or personality. The idea of using correlations between observed variables, such as scores on standardized intelligence tests like those being developed by Binet in the early twentieth century, to identify an unobservable or latent variable as emotive as intelligence soon become an area of increasing controversy—then as now, ‘‘a considerable provoker of unseemly bouts of fisticuffs amidst the groves of academe,’’ according to Lovie [19]. Initially the major arguments occurred between psychologists such as Spearman, Thomson, and Thurstone. But statisticians could not resist becoming involved. Most statisticians were clearly sceptical, particularly because of the nonidentifiability (in mathematical terms) of the analyses used to identify common factors, and because the process of rotating the factors until they looked meaningful seemed to allow researchers a great deal of license to impose their own

6617

preconceived ideas or even their prejudices on the data. Other statisticians, however, took a more constructive view of the emerging work on factor analysis and perceived that the psychologists’ use of the technique presented a challenge to statistical theory that they should take up. However, it was not until the development of powerful computers that statistically more respectable methods of performing a factor analysis became feasible. Before this development, the computational problems for early workers applying the methods of factor analysis must have been considerable. The late Professor Albert Maxwell (personal communication) tells the story of Godfrey Thomson’s approach to the arithmetical problems faced when performing a factor analysis by hand. According to Maxwell, Godfrey Thomson and his wife would place themselves on either side of their sittingroom fire, Dr. Thomson equipped with several pencils and much paper, and Mrs. Thomson with a copy of Barlow’s multiplication tables. For several hours the conversation would consist of little more than ‘‘What’s 613.23 multiplied by 716.61?’’ ‘‘43,8226.42.’’ ‘‘What’s 904.72 divided by 986.31?’’ ‘‘0.91,728’’ etc., etc.! Those of us who still remember using mechanical computing machines, slide rules and tables of square roots, and who recall the chaos engendered by spilling decks of computer cards on the floor, are perhaps best able to appreciate the technological revolution that has transformed both statistical theory and empirical psychology in the last two decades. One of the most important methodological developments for psychology associated with the technological advances in computing has been an extension of factor analysis logic into more powerful statistical theory, particularly as developed by Joreskog [17], and Bentler [2]. These developments have led to the use of what are known as structural equations models* that permit quite complicated hypothesis testing in the context of nonexperimental data. The importance of these developments in statistical theory cannot be underestimated by the psychologist: According to one scholar, ‘‘The development of the rigorous and generalized methods for testing

6618

PSYCHOLOGY, STATISTICS IN

hypotheses concerning underlying structures in covariance matrices is perhaps the most important and influential statistical revolution to have occurred in the social science’’ [3]. The suggestion that one need not conduct experiments to make causal inferences (see NONEXPERIMENTAL INFERENCE), combined with the arrival of well-documented and forcefully marketed computer software that makes it all possible and relatively simple, has been hard for psychologists to resist. Unfortunately, many psychologists appear to have suspended their usual critical faculties in the initial wave of enthusiasm for structural equation models. The old, but still apposite aphorism, ‘‘Correlation does not imply causation,’’ once chanted by generations of experimental psychologists, appears to have been somewhat forgotten amongst the mass of path diagrams, parameter estimates, and models for everything from theories of intelligence to sexual behavior. Reification is rife: Latent variables are given names and tested as if they had an independent existence and might even be directly manipulable, (see LATENT-VARIABLE MODELING). In many cases, little attention is given to the purpose that the causal model finally adopted is intended to serve. The following caveat offered by Cliff [3] is a useful antidote to such thinking: Beautiful computer programs do not really change anything fundamental. Correlation data are still correlational and no computer program can take account of variables that are not in the analysis. Causal relations can only be established through patient, painstaking attention to all the relevant variables, and should involve active manipulation as a final confirmation.

Despite such implied criticisms, however, the possibilities opened up in psychology by continuing advances in the modeling of latent variables presents an exciting opportunity and challenge for many psychologists. (There are, of course, many other areas of statistics relevant to psychology that have been radically affected by the availability of inexpensive and increasing computer power. Two examples of computer-intensive methods of which psychologists need to be more aware are the bootstrap* and exact methods for categorical data—see ref. 9 for details.)

SUMMARY Most psychologists are well aware of the importance of statistical methodology for analyzing the data they produce in their experiments and other studies. Unfortunately however, many psychologists appear to have a relatively limited statistical horizon, one that encompasses primarily significance tests, ANOVA, and exploratory factor analysis. But more developments in statistics such as the generalized linear model*, multilevel models, the bootstrap and others, are of equal relevance to psychology and psychologists, and should, consequently, rapidly find their way into more statistics courses given for psychologists and, in particular, into textbooks about statistics aimed at psychologists. REFERENCES 1. Arbuthnot, J. (1710). An argument for divine providence, taken from the constant regularity observed in the births of both sexes. Philos. Trans., 27, 186–190. 2. Bentler, P. M. (1980). Multivariate analysis with latent variables. Causal models. Annu. Rev. Psychol., 31, 419–456. 3. Cliff, N. (1983). Some cautions concerning the application of causal modelling methods. Multivariate Behav. Res., 18, 115–126. 4. Colman, A. M. (1994). The Companion Encyclopedia of Psychology. Routledge, London. 5. Cramer, E. M. (1988). ‘‘Statistics in Psychology’’. In Encyclopedia of Statistical Sciences, Vol. 8, S. Kotz, N. L. Johnson, and C. B.Read, eds. Wiley, New York. 739–743. 6. Cronbach, L. (1957). The two disciplines of scientific psychology. Am. Psychol., 12, 671–684. 7. Draper, N. R. and Smith, H. (1981). Applied Regression Analysis, 2nd ed. Wiley, New York. 8. Edgeworth, F. Y. (1885). Methods of Statistics, Jubilee Volume of the Statistical Society, London. 9. Everitt, B. S. (2000). Statistics for Psychologists. LEA, Mahwah, N.J. 10. Fisher, R. A. (1922). The goodness of fit of regression formulae and the distribution of regression coefficients. J. R. Stat. Soc., 85, 597–612. 11. Fisher, R. A. Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh, Scotland.

PSYCHOMETRIKA 12. Galton, F. (1889). Natural Inheritance. Macmillan, London. 13. Gardner, M. J. and Altman, D. G. (1986). Confidence intervals rather than p-values: estimation rather than hypothesis testing. Br. Med. J., 292, 746–750. 14. Garrett, H. E. and Zubin, J. (1943). The analysis of variance in psychological research. Psychol. Rev., 40, 233–267. 15. Gaskill, H. V. and Cox, G. M. (1937). Patterns in emotional reactions: I. Respiration; the use of analysis of variance and covariance in psychological data. J. Gen. Psychol., 16, 21– 38. 16. Goldstein, H. (1995). Multilevel Statistical Models. Arnold, London. 17. Jøreskog, K. G. (1973). ‘‘Analysis of Covariance Structures’’. In Multivariate Analysis, Vol. III, P. R. Krishnaiah, ed. Academic Press, New York. 263–285. 18. Lovie, A. D. (1979). The analysis of variance in experimental psychology: 1934-1945. Br. J. Math. Stat. Psychol., 32, 151–178. 19. Lovie, A. D. (1991). ‘‘A Short History of Statistics in Twentieth Century Psychology’’. In New Developments in Statistics for Psychology and the Social sciences, P. Lovie and A. D. Lovie, eds. BPS Books and Routledge, London. 20. Nelder, J. A. and Wedderburn, R. W. M. (1972). Generalized linear models. J. R. Stat. Soc. A, 35, 370–384. 21. Oakes, M. L. (1979). The Statistical Evaluation of Psychological Evidence. Unpublished Doctoral Thesis. University of Hull, Hull. 22. Oakes, M. L. (1986). Statistical Inference: A Commentary for the Social and Behavioural Sciences. Wiley, Chichester. 23. Pickles, A. (1998). ‘‘Generalized Estimating Equations’’. In Encyclopedia of Biostatistics, Vol. 2, P. Armitage and T. Colton, eds. Wiley, Chichester. 1626–1637. 24. Pinheiro, J. C. and Bates, D. M. (2001). Mixed-Effects Models in S and S-PLUS. Springer, New York. 25. Spearman, C. (1904). General intelligence objectively determined and measured. Am. J. Psychol., 15, 201–293. 26. Sprent, P. (1970). Some problems of statistical consultancy. J. R. Stat. Soc. A, 133, 139– 165. 27. Yerkes, F. (1904). Variability of reaction time. Psychol. Bull., 1, 137–176. 28. Walker, H. M. (1929). Studies in the History of Statistical Methods. Williams & Wilkins, Baltimore, Md.

6619

See also CORRELATION; ESTIMATING EQUATIONS, THEORY OF; FACTOR ANALYSIS; LATENT-VARIABLE MODELING; MEASUREMENT STRUCTURES AND STATISTICS; PSYCHOLOGICAL DECISION MAKING; PSYCHOLOGICAL SCALING; PSYCHOLOGICAL TESTING THEORY; REPEATED MEASUREMENTS, DESIGN AND ANALYSIS FOR; REPEATED SIGNIFICANCE TESTS; and SIGNIFICANCE TESTS, HISTORY AND LOGIC OF.

BRIAN EVERITT

PSYCHOMETRICS. See COMPONENT ANALYSIS; FACTOR ANALYSIS; PSYCHOLOGICAL DECISION MAKING; PSYCHOLOGICAL SCALING; PSYCHOLOGICAL TESTING THEORY

PSYCHOMETRIKA [This entry has been updated by the Editors.] Psychometrika is a journal published by The Psychometric Society, a nonprofit professional organization devoted to the advancement of quantitative measurement practices in psychology, education and the social sciences. A link to the journal website is located at www.psychometrika.org. Psychometrika was founded in 1935, largely by workers associated with the late L. L. Thurstone at the University of Chicago. Thurstone was the leading mathematical psychologist of the time. The journal has been issued quarterly, March, June, September, and December, since starting in 1936. Psychometrika contains articles on the development of quantitative models for psychological phenomena, and on quantitative methodology in the social and behavioral sciences, including new mathematical and statistical techniques for the evaluation of psychological data, and the application of such techniques. Original research in quantitative methodology as well as insightful methodological reviews, appear in the main section of the journal which since July 2003 is called ‘‘Theory and Methodology’’. A new section, ‘‘Application Reviews and Case Studies’’, aims at highlighting the essential connection between psychometric methodology and its application to behavioral data analysis in psychology, educational sciences, and related areas in the social sciences and marketing. Psychometrika articles usually have a statistical flavor, recognizing

6620

PSYCHOPHYSICS, STATISTICAL METHODS IN

the stochastic nature of behavior, rather than proposing idealized mathematical models. Articles proposing or analyzing mathematical models for psychological data are a mainstay of the journal. For example, recent issues contain articles treating multilevel linear models, factor analysis and structural equations models, latent class and latent trait models for multiple choice test and survey questions, models for cognitive and developmental processes, and models for choice, preference and other other forms of human judgment. A number of articles are also about more traditional types of statistical data analysis, overlapping considerably with the types of papers that might be published in such journals as the Journal of the American Statistical Association*, Biometrika*, and the like. To a large extent these have to do with statistical methods or issues of particular concern to psychologists. Some examples are the topics of repeated measures* in analysis of variance, principal components, measures of correlation, canonical* correlation analysis, and ordinal statistics. The journal includes a book review section for books that are of special interest to its readers. Psychometrika has an Executive Editor, a Theory-and-Methods Editor, an ApplicationReviews-and-Case Studies Editor, and a Book-Review Editor. There are 18 or so Associate Editors. Since 2005 the journal is published by Springer-Verlag; however, the Psychometric Society retains editorial control and copyright for the journal, which is availabile electronically as well as in print to members of the Society. NORMAN CLIFF BRIAN JUNKER

PSYCHOPHYSICS, STATISTICAL METHODS IN Psychophysics is a subfield of experimental psychology which studies the effects of external stimuli on the various sensory systems (visual, auditory, tactile, etc.) of human or animal organisms. A basic concept is that the energy in a stimulus is coded into a sensory

signal interpretable by the organism for some decision making. Psychophysics uses behavioral data to investigate the form of that coding. DETECTION AND DISCRIMINATION In a classical experimental paradigm, data are collected to estimate the probability psn that a stimulus s is detected over a masking background of ‘‘noise’’ n. The symbol s denotes a real number measuring a physical intensity expressed in standard units; n may denote a physical intensity or a (possibly infinite dimensional) vector, such as a spectral density function. For any value of the noise∗ n, the function pn : s → psn is a psychometric function, and often assumed to be continuous and strictly increasing. The detection probabilities psn may be estimated from frequency data in the framework of some mathematical model. Such a model specifies any psychometric function up to the values of some parameters which, it is hoped, capture some essential features of the sensory mechanisms. An example is the Gaussian model  γ (s)−η(n) 1 2 e−z /2 dz (1) psn = √ 2π −∞ with γ and η, two real-valued functions depending on the stimulus and the background, respectively. Equation (1) arises as a special case of the model psn = Pr[Us
Vn ],

(2)

in which Us and Vn are two jointly distributed random variables, which may be regarded as symbolizing the excitations evoked in some neural location by the presentation of the stimulus and of masking noise. If Us and Vn are independent, normally distributed random variables, with expectations γ (s) and η(n), respectively, and a common variance equal to 12 , then (2) reduces to (1). With a different interpretation, this model is also encountered in choice theory (see RANDOM UTILITY MODELS). Variants of it are attributed to Thurstone [13,14]. Other distributional assumptions are also made concerning Us and Vn . Some

PSYCHOPHYSICS, STATISTICAL METHODS IN

researchers suppose that a given sensory mechanism behaves as a counting process. Under appropriate conditions, this leads to the conclusion that Us and Vn have a Poisson∗ distribution [2,3,5,9]. For another example, it is sometimes postulated that a sensory mechanism operates as if it were computing the maximum of the excitation registered over a large number of sensory ‘‘channels,’’ an assumption leading one to suppose that each of Us and Vn has an extreme value distribution∗ (e.g., double exponential) [12,16,17]. Typically, the available data do not permit a clear choice between these models. Accordingly, the distributional assumptions are often dictated by considerations of convenience. Of primary interest are the functions γ (s) and η(n) in (1), which provide a parametrization of the mechanisms responsible for sensory coding. These functions are sometimes regarded as sensory scales, measuring the sensations evoked by the stimulus and the noise. In this connection, (1) is a special case of the equation psn = F[γ (s) − η(n)],

(3)

with F, a continuous, strictly increasing, but otherwise unspecified function. The fact that γ and η, but not F, are of central interest suggests a critical change in the paradigm. Writing f −1 for the inverse of a one-toone function f , and defining the sensitivity

6621

function ξπ (n) = p−1 n (π ) (see Fig. 1), (3) can be rewritten ξπ (n) = γ −1 [F −1 (π ) + η(n)].

(4)

For a fixed value of the detection probabilities π = psn , the right member of (4) depends only on the critical functions ν and η. The form of these functions can thus be investigated by estimating ξπ (n) experimentally for various values of the masking background n. Many data are consistent with the equation ξπ (n) = nβ G(π ),

β > 0,

which is referred to as the near-miss-toWeber’s law. (See ref. 5; we assume here that n is a positive real number measuring the intensity of the background.) This is a special case of (4), in which γ (s) = log s1/β , η(n) = log n, and G(π ) = exp[βF −1 (π )]. When β = 1, one obtains Weber’s law. In modern laboratories, experimental estimates of the values ξπ (n) are secured by computerized applications of stochastic approximation∗ techniques, in particular, the staircase method∗ . The subject is presented on successive trials with backgrounds n1 , n2 , . . . , nk , . . . . The choice of each value nk depends on the probability π in (4), the previous value nk−1 , and the subject’s response. Under the assumptions of the stochastic approximation technique used, either nk or  (1/nk ) ki=1 nk tends to ξπ (n). To avoid the subject detecting a trend in the succession of the nk values, several staircase procedures may be ‘‘interleaved.’’ Psychophysicists also investigate the ability of subjects to discriminate between stimuli in the same sensory continuum. In a psychoacoustic context, for example, pab may denote the probability of the subject judging a as louder than b. Most of the concepts and methods used in the study of detection also apply to discrimination. SIGNAL DETECTION THEORY∗

Figure 1. Psychometric function pn : s → psn and sensitivity function ξπ (n) = p−1 n (π ).

A difficulty in psychophysics is that any experimental task has cognitive components, this term covering a variety of factors, such as response bias, guessing strategies, motivation, and so on. Consider a detection task

6622

PSYCHOPHYSICS, STATISTICAL METHODS IN

in which, in half of the trials, only the masking background is presented. There are thus two kinds of trials: the sn or stimulus trials, and the nn or noise-alone trials. Two types of errors can be made: (a) the subject may fail to report a stimulus presented on an sn trial; this is a miss; (b) the subject may report a detection on a nn trial; this is a false alarm. The two types of correct responses are called hit and correct rejection. Various kinds of strategies are available in this task. For example, when the subject is not quite sure that the stimulus was presented on a trial, he or she may nevertheless claim to have detected it. This strategy would be advantageous in a situation in which the misses are much more heavily penalized than the false alarms, for example, in the payoff matrix in Table 1, in which the numbers represent monetary values. If the subject reports a detection on every trial, whether or not the stimulus was presented and these two possibilities are equiprobable, the average gain per trial is 1.5. The opposite strategy of responding ‘‘no detection’’ on every trial results in an average loss of 0.5. This payoff strategy thus favors a positive guessing strategy over a conservative one. In fact, the data indicate a remarkable tendency of the subjects to alter their strategy according to nonsensory aspects of the task. A problem for the psychophysicist is thus that of disentangling the purely sensory aspects of the task from those resulting from cognitive factors. The solution of this problem was inspired by some application of statistical decision theory∗ to communication theory∗ , in the design of optimal decision procedures for the detection of signal in noise [6,10,11,15]. In psychophysics, however, considerations of optimality play a minor role. Let  be a large

Table 1. Example of a Payoff Matrix in Signal Detection Theory

Yes Stimulus No

Responses Yes No 4 −4 Hit Miss −1 3 False Correct alarm rejection

collection of payoff matrices. For any θ ∈ , let psn (θ ) and pnn (θ ) be the probability of a hit and of a false alarm, respectively. The set ρsn = {[pnn (θ ), psn (θ )]|θ ∈ } is referred to as a receiver-operating-characteristic (ROC) curve. Ideally, ρsn is assumed to be (the graph of) a continuous, increasing, typically concave function mapping the open interval (0, 1) into itself. In principle, the information contained in the ROC curve ρsn captures the purely sensory aspects of the detection of the stimulus s over the masking background n. This position is rationalized by the standard model of signal detection theory, the assumptions of which are as follows. In the spirit of the model of (2), the effect of the stimulation on an sn trial (respectively, nn trial) is represented by a random variable Us (respectively, Vn ). To each payoff matrix θ ∈  corresponds a real number λθ , the criterion. The separation of the sensory aspects of the task from the cognitive ones is achieved through the equations psn (θ ) = Pr[Us > λθ ],

(5)

pnn (θ ) = Pr[Vn > λθ ].

(6)

This means that a detection is reported on any trial if the sampled value of the induced random variable exceeds the criterion. If the random variables Us and Vn are independent, then 

1

ρsn (p)dp = Pr[Us > Vn ].

0

Thus the area under the ROC curve provides a measure of the detectability of the stimulus over the masking background, which is, in theory, unimpaired by cognitive factors (see Fig. 2). In practice, assumptions are made concerning the distributions of the random variables Us and Vn , which specify the ROC curve up to the values of some parameters. A few experimental points suffice to estimate these parameters and thus to locate the ROC curve. As in the case of (2), a frequent assumption is that Us and Vn are Gaussian.

PSYCHOPHYSICS, STATISTICAL METHODS IN

6623

The functional measurement methods of N. H. Anderson are similar to Stevens’s ‘‘direct’’ methods, in that the subject is asked to give a quantitative response (e.g., a number on a rating scale). The stimuli, however, vary according to several dimensions (such as the size and the weights of cubical blocks rated for ‘‘heaviness’’; see ref. 1). The aims of the analysis are to parse out the effects of the stimulus dimensions on the sensory impressions; this is typically, and often successfully, achieved by regression or ANOVA techniques.

REFERENCES Figure 2. ROC curve ρsn .

1. Anderson, N. H. (1970). Percept. Psychophys., 8, 153–170.

In a special case of this theory, the subject strategy, as embodied in (5) and (6), is consistent with an optimal statistical decision procedure based on the computation of likelihood ratios, and maximizing the expected gain.

2. Green, D. M. and Luce, R. D. (1974). In Contemporary Developments in Mathematical Psychology, Vol. 2: Measurement, Psychophysics, and Neural Information Processing, D. H. Krantz, R. C. Atkinson, R. D. Luce, and P. Suppes, eds. W. H. Freeman, San Francisco, pp. 372–415.

OTHER PARADIGMS AND CONCEPTS

3. Luce, R. D. and Green, D. M. (1974). J. Acoust. Soc. Amer., 56, 1554–1564.

Many other concepts and methods are used in psychophysics. One important current must be mentioned here. Some researchers favor experimental methods in which a human subject is required to provide a ‘‘direct’’ estimate of the sensory state evoked by stimulus. In the magnitude estimation paradigm advocated by S. S. Stevens and his followers [4,8], the subject gives a numerical response quantifying the subjective impression of the stimulus intensity. A variation of this method (crossmodality matching) has the subject compare the intensity of stimuli from different sensory continua (say, the brightness of a monochromatic light and the loudness of a pure tone), and adjust one of the two intensities so as to equalize the two sensory impressions. Generally, the data are consistent with the hypothesis that the various sensory continua, when the physical intensities are measured in standard ratio scale units, are related to each other and to the number continuum by power laws (see ref. 7).

4. Marks, L. E. (1974). Sensory Processes. Academic Press, New York. 5. McGill, W. J. and Goldberg, J. P. J. Acoust. Soc. Amer., 44, 576–581.

(1968).

6. Peterson, W. W., Birdsall, T. L., and Fox, W. C. (1954). Trans. IRE Prof. Group Inf. Theory, PGIT-4, 171–212. 7. Stevens, S. S. (1966). Percept. Psychophys., 1, 5–8. 8. Stevens, S. S. (1975). Psychophysics: Introduction to Its Perceptual Neural, and Social Prospects. Wiley, New York. 9. Strackee, G. and van der Gon, J. J. D. (1962). Statist. Neerlandica, 16, 17–23. 10. Tanner, W. P., Jr. and Swets, J. A. (1954). Psychol. Rev., 61, 401–409. 11. Tanner, W. P., Jr. and Swets, J. A. (1954). Trans. IRE Prof. Group Inf. Theory, PGIT-4, 213–221. 12. Thompson, W. A., Jr. and Singh, J. (1967). Psychometrika, 32, 255–264. 13. Thurstone, L. L. (1927). Psychol. Rev., 34, 273–286. 14. Thurstone, L. L. (1927). Amer. J. Psychol., 38, 368–389.

6624

PUBLIC OPINION POLLS

15. Van Meter, D. and Middleton, D. (1954). Trans. IRE Prof. Group Inf. Theory, PGIT-4, 119–141. 16. Wandell, B. and Luce, R. D. (1978). J. Math. Psychol., 17, 220–235. 17. Yellott, J. I., Jr. (1977). J. Math. Psychol., 15, 109–144.

BIBLIOGRAPHY See the following works, as well as the references just given, for more information on the topic of statistical methods in psycho-physics. Birnbaum, M. H. (1982). In Social Attitudes and Psychophysical Measurement, B. Wegener, ed. Lawrence Erlbaum, Hillsdale, N.J., pp. 410–485. (A discussion of the controversies in psychological measurement.) Egan, J. P. (1975). Signal Detection Theory and ROC Analysis. Academic Press, New York. (Contains a discussion of distributional assumptions in signal detection theory.) Falmagne, J. -C. (1985). Elements of Psychophysical Theory. Oxford University Press, New York. (An introductory text for the mathematically minded reader.) Fechner, G. T. (1966). Elements of Psychophysics, D. H. Howes and E. C. Boring, eds., H. E. Adler, trans. Holt, Rinehart and Winston, New York. (Originally published, 1860. The original monograph of the founder of psychophysics. At least of historical interest.) Gescheider, G. A. (1976). Psychophysics: Method and Theory. Lawrence Erlbaum, Hillsdale, N.J. (An elementary, nontechnical introduction.) Green, D. M. and Swets, J. A. (1974). Signal Detection Theory and Psychophysics. Krieger, Huntington, N.Y. (The basic text.) Levitt, H. (1970). J. Acoust. Soc. Amer., 49, 467–476. [A paper presenting the application of stochastic approximation methods in psychophysics (in particular, psychoacoustics).] See also PSYCHOLOGICAL SCALING.

JEAN-CLAUDE FALMAGNE

PUBLIC OPINION POLLS HISTORY The history of modern public opinion polling began in 1935 with the first publication of the Gallup polls. Researchers, however, have

pointed to references to data gathering in the Bible and during the Middle Ages. As early as 1824, newspapers were using informal straw polls to predict elections in the United States. In Europe at the end of the nineteenth century Booth was conducting his survey of the poor working people in London and Le Play did the first family budget surveys. In the United States marketing and advertising research procedures were being developed and used by Scott, Cherington, and Parlin after 1910. Commercial and advertising research continued to grow after World War I and provided the foundation of methods for studying public opinion. Gallup argued from the beginning that small, carefully selected quota samples∗ were superior to very large uncontrolled samples for measuring opinion. In 1936, the Literary Digest conducted a very large mail survey with more than 10,000,000 persons whose names had been obtained from automobile registrations and telephone directories. The same procedure had been used successfully in predicting the outcomes in several previous elections. Based on responses, the Literary Digest predicted a sweeping Landon victory. The much smaller Gallup, Roper, and Crossley polls correctly predicted a Roosevelt victory and established the credibility of public opinion polls among the media and politicians, including President Roosevelt, who used Gallup poll results as a source of information, especially on foreign policy issues [2]. Shortly thereafter the U.S. Department of Agriculture established the Division of Program Surveys to measure farmer attitudes. The first academic survey organization, the National Opinion Research Center, was started at the University of Denver in 1941 and then moved to the University of Chicago. Opinion polls continued to increase in popularity during World War II. There were many surveys of public morale both in the United States and in other countries. The U.S. Army conducted large and sophisticated studies of the attitudes of soldiers, under the leadership of Samuel Stouffer and his colleagues [11]. After the war, public opinion research organizations were organized in every country in Western Europe. The Program Surveys group left the

PUBLIC OPINION POLLS

U.S. Department of Agriculture and under the leadership of Rensis Likert established the Survey Research Center at the University of Michigan. State and regional polls have been established by many universities as well as commercial researchers. Major questions about the validity of poll results were raised in 1948 after all the polls incorrectly predicted a Dewey victory in the Presidential election. Statisticians pointed out the deficiencies in the quota samples then in use, and improved sampling methods were introduced [10]. The major reason for the error in 1948, however, was that the polls stopped interviewing too soon and missed late changes in opinion. More recently, polls interview through the last day prior to elections. The concern about poll accuracy soon died down as polls again resumed a series of successful predictions. The use of polls by politicians as an aid in election campaigns grew rapidly, as did their use in the social sciences, program evaluation, and as a source of news for magazines and television in addition to their traditional newspaper audience. As the fraction of households with telephones rose to the high 80% and then low 90% range and as field costs of face-to-face interviewing rose sharply, more and more polls were conducted on the telephone. The development of computer-assisted telephone interviewing (CATI) accelerated the use of telephone interviewing, which is today the most popular method of data collection∗ ; see TELEPHONE SURVEYS, COMPUTER ASSISTED. The legality of polls has been questioned by some communities that have attempted to regulate polling, both face-to-face and telephone. There has also been a continuing controversy about whether poll results change the outcome of elections by discouraging voters or changing the voter’s intentions. Most recently, this controversy has centered about exit polls that are conducted with voters as they leave the polling place. USES OF POLLS Of the many uses of polls, the most important is their use by policymakers in the private and public sectors. Since Franklin

6625

D. Roosevelt, U.S. presidents as well as leaders of many other countries have used polls as one source of information about public attitudes. For political leaders, the successful use of poll data is not to promote programs and policies that make the user more popular, but as a guide to public knowledge and attitudes that leaders will need to accept or modify in their policies. Local government executives and national and local legislative bodies are also potential users of poll data, but much less data is available on a local level, so that informal sources are more likely to be used. While some political figures mistrust polling procedures, almost all of them use polls as an aid to reelection. Polls are heavily used for business and program applications. It was estimated that of total expenditures of $1.2 billion for polls and surveys in the United States, less than 10% was for public opinion research with the rest for market and program applications [7]. The major uses include new product testing, customer satisfaction with products or services, use of media, testing advertising, and the polls as economic indicators (see also MARKETING, STATISTICS IN). The users are businesses, nonprofit organizations such as hospitals and schools, and local and federal administrators of a broad range of national and community programs. Extensive uses of polls are also made by professional and trade associations and other interest groups. These groups survey not only their own members, but also the general public, to determine the status of the group and public attitudes on issues of concern to it. The use of polls in legal cases and administrative hearings has increased rapidly since 1960. Earlier polls were excluded as hearsay evidence, but recently courts have accepted the idea that public opinion can be measured, and expert witnesses can report on how the results were obtained and what they mean. Polls have been used to determine whether media coverage in a local area has been so intense as to require a change of venue, whether advertising is deceptive and whether there is confusion about a product trademark. A growing but controversial use of polls is as an aid to attorneys in jury selection and in presenting evidence to juries.

6626

PUBLIC OPINION POLLS

Polls and surveys are the most widely used method of collecting data in the social sciences, especially in sociology and political science [12] (see DATA COLLECTION and SURVEY SAMPLING). Many social scientists conduct secondary analyses of poll data gathered by the federal government or from large survey organizations. Major social science archives of poll data exist and can readily be accessed by researchers [16]. The largest of these in the United States is the Roper Center at the University of Connecticut. The best known use of polls is as news. When the polls concentrate on issues they perform a service by helping to crystallize public opinion by providing people information on what others think. When the polls are used simply to predict in advance the outcome of elections, their main function is to entertain and to increase viewing and readership. METHODS OF DATA COLLECTION There are three major methods of collecting data: telephone, face-to-face, and mail. There are variations on each of these methods and they are also used in combination. The criteria for judging alternative procedures are the quality of the sample that will cooperate, the length and complexity of the questionnaire, and the costs. Telephone Surveys In the United States during the 1980s about 94% of all households have telephones. Very large percentages of households have telephones in some European countries and the percentage in other countries is growing rapidly. Thus the major bias caused by the omission of nontelephone households that caused a huge error in the 1936 Literary Digest poll is no longer found today. For this reason, telephone interviews are the most widely used method of data gathering for opinion polls. Costs are about half of face-to-face interviewing because no travel is involved, and there are no telephone charges and virtually no interviewer time spent if a respondent is not at home. Most telephone interviewing is conducted from central locations where a group of interviewers are under close supervision, so that interviewer quality is tightly controlled. It

has been found that cooperation rates can be as high and interviews can be as long on the telephone as face to face, and that telephone interviewing is substantially superior to mail questionnaires [5]. The major limitations are questions that require the respondent to look, touch, or taste something. Even here, combined methods may be used. Respondents can be mailed material and then telephoned for their opinions. A major technological innovation has been the general introduction of computer-assisted telephone interviewing (CATI), now used by most polling organizations. Instead of a paper questionnaire, the interviewer uses a computer terminal on which the proper question appears in sequence. It is possible to individualize questionnaires based on respondent answers to earlier questions as well as to introduce experimental variations. Also, editing of responses is conducted as the interview proceeds. An improper answer is immediately rejected by the computer. Results are available for analysis immediately after the interview is completed. Mail Surveys Mail methods are the cheapest to use, even when done carefully. This involves an initial mailing and two follow-up∗ mailings to those who do not respond, plus a small cash gift [3,4]. With general population samples only short questionnaires of fewer than four pages obtain reasonably high cooperation. Longer questionnaires that require much writing or following complex instructions are likely to result in very low cooperation, although these may sometimes be possible with special groups of highly educated respondents. Opinion polls by mail have been successful when conducted at the local level and dealing with important local issues. They are also successful when conducted by groups and associations with their members. In the United States there have been no successful national mail opinion polls, in part because there is no convenient complete mailing list of households or individuals as in some European countries. Combined methods using selfadministered questionnaires and telephone are sometimes used. When successful, mail or combined procedures can obtain cooperation rates as high as for personal methods.

PUBLIC OPINION POLLS

Face-to-Face Interviewing This once major method of polling has become much less popular because of the high cost of locating respondents at home. Nevertheless, it still is the most flexible method and allows the respondent to do tasks that are impossible by mail or telephone. Some very high quality surveys by federal agencies and universities are still conducted face-to-face, although even here follow-up interviews for panel∗ studies are conducted by telephone where possible. Some face-to-face polls are conducted in central locations such as shopping malls or recreation areas. When the population is intended to be users of such facilities, this can be a very efficient sampling method (but see QUOTA SAMPLING). Users of such facilities differ from the general population in many ways, however, and generalizing from location samples is difficult even if the facilities have been selected using probability procedures [14]. SAMPLING Almost all current public opinion polls use either strict or modified probability procedures in selecting their samples. The most commonly used procedure on the telephone is random digit dialing∗ . A list of all telephone area codes and working exchanges is available. Exchanges are selected at random and the last four digits are chosen using random numbers. This eliminates the need for directories, which are seriously incomplete in large cities because of unlisted numbers. Random digit dialing results in many selected numbers being ineligible because they belong to businesses or other institutions or simply are not assigned. Procedures have been developed for substantially reducing ineligible numbers by clustering the sample in banks of numbers known to contain working household numbers [17]. Although strict probability samples would require that the selection of a respondent within the household also use probability methods, most polls use procedures that balance for gender since otherwise a substantial overrepresentation of women would be found. The short time period allowed for most polls means that people who are hard

6627

to find at home are likely to be underrepresented. The face-to-face sampling of individuals is a far more complex and costly enterprise since local interviewers must be hired and trained. Area probability samples are always multistage. At the first stage, counties or metropolitan areas are selected. Since these geographic areas vary enormously in size, they are chosen by a procedure called sampling with probability proportional to size∗ . This gives the largest metropolitan areas a greater chance of selection than smaller counties and results in the largest metropolitan areas such as New York, Los Angeles, and Chicago always being selected. The sample is still self-weighting because at later stages a smaller fraction is taken from the larger locations. Within selected counties or metropolitan areas, additional selection is made of smaller geographic units in two or three stages, using census data. Selected blocks or segments are then either listed by the polling organization or directories (available for middle-sized cities) are used. Within a block or segment, all households have an equal probability of selection. To save interviewer travel costs, about five households will usually be chosen for each block. Once a household is selected, all members in the household are listed and one is chosen at random for interviewing. As a final step, it is necessary to weight the sample for household size both in phone and area samples since a respondent’s probability of selection within the household depends on household size. As with phone surveys, area samples are faced with serious problems of finding the selected respondent at home; usually, three attempts are made. Phone samples can do many more because there are no travel costs. Modified procedures that reduce the number of callbacks or that eliminate them entirely underrepresent people who are at home less often, introducing a potentially serious bias [6,9,13]. By far the simplest sampling procedure is used when there is a list available for a mail study. No clustering is required since the only purpose of clustering is to save interviewing costs. Using either hand or computer methods, the desired sample is

6628

PUBLIC OPINION POLLS

selected either by pure random selection (see SIMPLE RANDOM SAMPLING), for which a different random number is required for each selected unit, or by systematic sampling∗ , where a random start is made and then every nth unit is chosen. SAMPLE BIASES Since not all selected respondents can be located or are willing to be interviewed after being located, some sample biases are inevitable. The following are typically found: Rural and small-town respondents are overrepresented since cooperation rates decline as the size of the place increases. The largest metropolitan areas have cooperation rates about 20% lower than in rural areas. There is some tendency to underrepresent both the youngest and oldest respondents for different reasons. The youngest respondents (age 18–35) are difficult to find at home, while the oldest respondents (over 65) are easy to find but more likely to refuse to be interviewed. There are no major social class or racial or ethnic biases. Biases for telephone and faceto-face methods are roughly comparable, but for different reasons. Respondents who cannot be reached because they will not open their doors are also likely to refuse when telephoned. Since noncooperation occurs before the interview, there is seldom a high correlation∗ between the topic of the poll and sample biases for personal surveys. Much more serious sample biases are possible, however, for mail polls. Here the respondent can see all the questions before deciding whether to cooperate or not. People who feel strongly about the issues in the mail questionnaire, either pro or con, are more likely to respond than those who are in the middle. Also on mail surveys there is likely to be a clear education bias, with highest levels of cooperation coming from the best educated. RESPONSE EFFECTS From the earliest days of polling it was recognized that question wording∗ could affect responses on attitude questions. This is particularly true when issues are new or not very important so that people do not have well developed opinions.

Also there are often substantial errors in the reporting of behavior caused by memory error and social desirability of the responses. Well-known examples are the substantial overstatement of voting and charitable giving and the understatement of alcoholic consumption. In some cases, changes in the method of administration or wording of the questionnaire can improve results. Thus group or self-administered questionnaires are less likely to obtain overstatements of desirable behavior, and randomized response∗ methods may increase reporting of undesirable behavior. The use of open rather than closed questions increases the reported quantities of alcohol consumed [15,1]. However, it is sometimes impossible to obtain useful information from respondents even if they answer questions. As illustrations, although household informants can provide accurate information about the employment status of other family members, parents are very poor informants about the drug-use behavior of their teenaged children. Questions asking about annual household consumption of specific grocery products or even about purchases in a short time period have very large measurement errors∗ unless the data are collected by diaries. On attitude questions, respondents are unwilling to admit ignorance and may express views on issues they have never heard about or that do not exist. Although response effects are common, they are not ubiquitous. Where poll questions deal with important issues results are identical, even with substantial question wording changes. For nonthreatening questions, such as ownership or purchase of durable goods, responses are likely to be reasonably accurate, although there is always some memory error. INTERVIEWER EFFECTS If the poll is conducted by an interviewer, there is always the possibility that the interviewer will influence the results. In the most direct way, this occurs if the characteristics of the interviewer are related to the topic of the poll. Thus it has been found consistently that black and white respondents give different answers to black and white interviewers on

PURCHASING POWER PARITY

questions about racial issues [8]. No effects are found, however, if the interviewer characteristics are unrelated to the topic. Interviewer beliefs about the topic may have some effect in special cases. This may occur if the respondent gives an ambiguous answer to an open question. Deliberate interviewer falsification is very rare. It is possible for the interviewer to prompt the respondent with cues that suggest an answer, but the better trained the interviewer, the less likely this is to occur. Measuring interviewer effects has generally been difficult because it requires randomized assignments. With face-to-face interviewing, randomized assignments are usually impractical because of interviewer travel costs and availability. Computer-assisted telephone interviewing, however, makes randomized assignments and the computation of interviewer components of variance practical. REFERENCES 1. Bradburn, N. M. and Sudman, S. (1979). Improving Interview Method and Questionnaire Design: Response Effects to Threatening Questions in Survey Research. Jossey-Bass, San Francisco. (Reports results of experiments for improving reporting of threatening questions.) 2. Cantril, H. (1967). The Human Dimension: Experiences in Policy Research. Rutgers University Press, New Brunswick, N.J. (A memoir by an early academic pollster who was a consultant to President Franklin Roosevelt.) 3. Dillman, D. A. (1978). Mail and Telephone Surveys: The Total Design Method. Wiley, New York. (Gives detailed discussion of procedures for improving cooperation on mail surveys.) 4. Erd¨os, P. L. (1970). Professional Mail Surveys. McGraw-Hill, New York. (A very useful introduction to successful mail surveys, primarily for business uses.) 5. Groves, R. M. and Kahn, R. L. (1979). Surveys by Telephone: A National Comparison with Personal Interviews. Academic Press, New York. (The best source to date of experiences with phone surveys and comparisons to faceto-face interviewing.) 6. Hansen, M. H., Hurwitz, W. N., and Madow, W. G. (1953). Sample Survey Methods and Theory, 2 Vols. Wiley, New York. (The classic work on applied probability

6629

sampling. Still very useful for large-scale surveys.) 7. Honomichl, J. (1982). Advertising Age. June 21, pp. 48–52. 8. Hyman, H. (1954). Interviewing in Social Research. University of Chicago Press, Chicago. (The classic work on interviewer effects, but tends to exaggerate their importance.) 9. Kish, L. (1965). Survey Sampling. Wiley, New York. (Standard text for courses in applied sampling.) 10. Mosteller, F. et al. (1949). The Pre-election Polls of 1948. Social Science Research Council, New York. (The analysis of deficiencies in 1948 polling procedures that led to improved sampling methods.) 11. Stouffer, S. et al. (1949). The American Soldier. Princeton University Press, Princeton, N.J. (A classic work that shaped the direction of social science after World War II.) 12. Sudman, S. (1976). Annu. Rev. Sociol., 2, 107–120. 13. Sudman, S. (1976). Applied Sampling. Academic Press, New York. (An elementary text on sampling for users with limited statistical backgrounds.) 14. Sudman, S. (1980). J. Marketing Res., 17, 423–431. 15. Sudman, S. and Bradburn, N. M. (1974). Response Effects in Surveys: A Review and Synthesis. Aldine, Chicago. (A meta analysis of 500 surveys that generalizes types and magnitudes of response effects.) 16. Sudman, S. and Bradburn, N. M. (1982). Asking Questions. Jossey-Bass, San Francisco, pp. 15–16. 17. Waksberg, J. (1978). J. Amer. Statist. Ass., 73, 40–46. See also BIAS; DATA COLLECTION; ELECTION PROJECTIONS; PANEL DATA —I; QUESTION-WORDING EFFECTS IN SURVEYS; QUOTA SAMPLING; RANDOM-DIGIT DIALING, SAMPLING METHODS FOR; SURVEY SAMPLING; and TELEPHONE SURVEYS, COMPUTER ASSISTED.

SEYMOUR SUDMAN

PURCHASING POWER PARITY The purchasing power parity (PPP) is a price index which measures the price level in country s relative to that in the base country r, where the prices compared are expressed in

6630

PURCHASING POWER PARITY

national currency units. It is therefore a rate of exchange between two national currencies, except that while the official exchange rate is the number of units of the currency of country s required to buy one unit of the currency of country r, the PPP is the number of units of the currency of country s required to buy goods and services equivalent to what can be bought with one unit of the currency of country r. While the former provides a conversion factor to compare nominal income and output and for the settlement of international transactions, the PPP provides a conversion factor to compare real income and output between countries. BRIEF HISTORY The early history of PPP comparisons can be traced to comparisons of the real incomes of England, France, and Holland by Sir William Petty [24] in 1665 and 1676 and by Gregory King [17] in 1688. However, major international comparisons of real income began to appear only in this century. Two landmark studies based on consumer goods were made by the U.K. Board of Trade [29–33] between 1907 and 1911 and by the International Labour Office [15] in 1932. In the 1950s the Organization for European Economic Cooperation [11,12] sponsored pioneering comparisons for gross domestic product and its components. These were followed recently by extensive comparisons sponsored by the United Nations and the World Bank [19,21,22] and by the European Economic Community [9]. There are also scores of highly specialized studies by individual scholars and government agencies, most notably by Bergson [5] and by the Joint Economic Committee of the U.S. Congress [34]. A more detailed historical discussion can be found in refs. 18 and 25. TYPES AND PROPERTIES OF PPP COMPARISONS There are direct or ‘‘full-scale’’ and indirect or ‘‘shortcut’’ methods of PPP comparisons. In this section we define the direct comparisons and their desired properties, and in the following section we discuss some of

the methods used in these comparisons and present an example. In the final section we discuss briefly some of the shortcut methods. The direct PPP comparisons require the specification of a sample of commodities representative of total consumption or production of the countries compared, the collection of the prices and quantities of these commodities during a specified period, and the application of some index number∗ formulas. Thus suppose that one observes the same set of n commodities A = (a1 , a2 , . . . , an ) in m
2 countries at a given time. Let pij be the price of ai in country j expressed in national currency units and qij the corresponding quantity in physical units. Prs is a weighted average of the ratios (pis /pir ). The weights are functions of the quantities of either or both countries r and s or of all the m countries if m > 2. The corresponding quantity index, Qrs , which is a measure of real income in s relative to r, is obtained by dividing Prs into   the expenditure ratio Vrs (= ni=1 pis qis / ni=1 pir qir ). If m = 2, the comparisons are binary or bilateral, and if m > 2, they are multicountry or multilateral. The methods used in these comparisons are chosen on the basis of certain desired properties which include in particular the following. 1. Base-Country Invariance. For m = 2, this requires that Prs Psr = 1. 2. Transitivity. For m > 2, this requires that Prs Pst = Prt for any three countries r, s, and t among the m countries. 3. Consistency in Aggregation∗ . This requires that the value of the index is the same whether computed directly for the set A or as the weighted sum of the equivalent indices of disjoint subsets of A (see LOG-CHANGE INDEX NUMBERS). 4. Characteristicity. This requires that the weights used in computing Prs are representative of the patterns of consumption or production of the two countries r and s. METHODS OF DIRECT PPP COMPARISONS Laspeyres, Paasche∗ , and especially Fisher’s ideal index∗ are among the index number formulas widely used in bilateral PPP

PURCHASING POWER PARITY

comparisons. The PPP between two countries r and s calculated by Fisher’s ideal index, Frs , is given by   i Frs =  



pis qir pir qir

i ·

i

pis qis pir qis

1/2  

.

(1)

i

This index satisfies properties 1 and 4, but it fails property 3 and, in multilateral comparisons, it also fails 2. A multilateral version of ∗ , is given by (1) which satisfies property 2, Frs   2 ∗ = Frs Frs

1/m m " Frt  ,  Fst t=1

SHORTCUT METHODS

where the indices in the brackets are obtained from (1). This index is known as the EKS [9,18,22] after the initials of its three authors: Elteto, Koves, and Szulc. It has been argued in ref. 6 on economic grounds in favour of the T¨ornqvist bilateral index, Hrs , which is given by 1 2

m i

(wir + wis ) ln(pis /pir ),

(3)

 where wij = (pij qij / ni=1 pij qij ) for j = r and s. This index, like (1), satisfies properties 1 and 4, but it fails property 3 and, in multilateral comparisons, it also fails property 2. However, for m > 2, ref. 6 provides a multilateral ∗ , which satisfies property 2 version of (3), Hrs and is given by ∗ = ln H s − ln H r , ln Hrs

(4)

where ln H i =

m 1  ln Hij m

for i = r, s

fails property 3 and also property 4 unless m = 2. The Gerardi index, however, satisfies property 3 in addition to the other properties which the Geary–Khamis index satisfies [36, p. 83]. These two indices are studied in great detail in ref. 14. A serious problem common to all multilateral index number formulas is that for m > 2 the PPP between two countries r and s is not independent of the value of m. A typical example of the main use of PPP, based on [18, pp. 5–10], is shown in Table 1, which is self-explanatory. (The PPPs in this example were calculated by the Geary–Khamis method.)

(2)

t=r,s

ln Hrs =

6631

(5)

j=1

and the indices Hij in (5) are obtained from (3). Another method of multilateral comparisons is the Geary–Khamis index (see INDEX NUMBERS). This method has been used in refs. 19, 21, and 22, and a version of it called the Gerardi index [10] has been used in ref. 9. Voeller [36, p. 81] lists many index number properties which the Geary–Khamis index satisfies, including properties 1 and 2. But it

Part of the economic literature of real income comparisons analyzes the relation between estimated real incomes and living standards in the countries compared, as in refs. 1, 20, 23, 26, 27, 28 and 35, and the relation between nominal and real incomes, as in refs. 2, 3, 8, and 16. Based on economic arguments, the relation between nominal and real income can be estimated by regression methods. Thus David [8] estimated from cross-section data for industrial countries the regression equation ∗ ), Yr /Yrs = 0.559 + 0.441(Yr /Yrs

(6)

where Yr is the per capita income of the base country r (the United States) in its own ∗ are, respectively, currency and Yrs and Yrs nominal and real per capita income of country s expressed in the currency of country ∗ ) in (6) r. The estimated coefficient of (Yr /Yrs was interpreted [8, p. 985] to mean that ‘‘the ‘real’ percentage gap between the per capita income of a given country and the United States is only (0.441) four-ninths of the percentage gap indicated by a straight exchange rate conversion expressing all incomes in dollars.’’ Put differently, this also means that the official exchange rate of country s in terms of U.S. dollars is only four-ninths of the corresponding PPP. This method was further developed and applied in ref. 20 and other similar regression methods were applied in refs. 4 and 7. On comparative assessment of many shortcut methods, see ref. 13.

6632

a In

Dollar

Franc Deutsche Mark Rupee Lira Yen Pound

thousands of national currency units.

France West Germany India Italy Japan United Kingdom United States

Country

Currency Unit (1)

1.0000

3.140 2.160 483.000 244.000 0.308

3.6600 7.5000 625.0000 360.0000 0.4167

1.0000

4.480

Purchasing Power Parity to U.S. Dollars (3)

5.5540

Official Exchange Rate to U.S. Dollars (2)

4,801

11,272 736 1,062a 721a 893

16,118

In National Currency Units (4)

4,801

3,080 98 1,699 2,003 2,143

2,902

4,801

3,590 341 2,199 2,955 2,899

3,598

In U.S. Dollars Converted At Official At Purchasing Exchange Rate Power Parity (5) = (4) ÷ (2) (6) = (4) ÷ (3)

Per Capita GDP

Table 1. Per Capita Gross Domestic Product, Official Exchange Rates, and Purchasing Power Parities in 1970

100

86 29 77 68 74

81

Price Index (U.S. Prices = 100) (7) = (3) ÷ (2)

PURI’S EXPECTED NORMAL SCORES TEST

REFERENCES 1. Afriat, S. N. (1972). In International Comparisons of Prices and Output, D. J. Daly, ed. National Bureau of Economic Research, New York, pp. 1–69. (Economic theory, technical but foundational paper; references.) 2. Balassa, B. (1964). J. Pol. Econ., 72, 584–596. (Economic theory of a shortcut method.) 3. Balassa, B. (1973). Econ. J., 83, 1258–1267. 4. Beckermann, W. and Bacon, R. (1966). Econ. J., 76, 519–536. (Elementary.) 5. Bergson, A. (1978). Productivity and the Social System: The USSR and the West. Harvard University Press, Cambridge, Mass. 6. Caves, D. W., Christensen, L. R., and Diewert, W. E. (1982). Econ. J., 92, 73–86. (Intermediate; few references.) 7. Clague, C. and Tanzi, V. (1972). Econ. Int., 25, 3–18. 8. David, P. A. (1972). Econ. J., 82, 591–605. 9. European Community Statistical Office (1977). Comparisons in Real Values of the Aggregates ESA. European Community, Luxemburg. 10. Gerardi, D. (1982). Rev. Income Wealth, 28, 381–405. 11. Gilbert, M. and Kravis, I. (1954). An International Comparison of National Products. OEEC, Paris. 12. Gilbert, M. and Associates (1958). Comparative National Products and Price Levels. OEEC, Paris. 13. Heston, A. (1973). Rev. Income Wealth, 19, 79–104. (Elementary; few references.) 14. Hill, T. P. (1982). Multilateral Measurements of Purchasing Power and Real GDP. Publications of the European Community, Luxemburg. (Elementary; few references.) 15. International Labour Office (1932). A Contribution to the Study of International Comparisons of Costs of Living, Ser. N, No. 17. ILO, Geneva, Switzerland. 16. Jazairi, N. T. (1979). Econ. J., 89, 127–130. 17. King, G. (1688). Two Tracts, G. C. Barnett, ed. Johns Hopkins University Press, Baltimore, 1936. 18. Kravis, I. B. (1984). J. Econ. Lit., 22, 1–39. (Elementary survey; references.)

6633

21. Kravis, I. B., Heston, A., and Summers, R. (1982). World Product and Income. Johns Hopkins University Press, Baltimore, Md. 22. Kravis, I. B., Zoltan, K., Heston, A., and Summers, R. (1975). A System of International Comparisons. Johns Hopkins University Press, Baltimore, Md. 23. Marris, R. (1984). J. Econ. Lit., 22, 40–57. (Elementary, economic theory; few references.) 24. Petty, W. (1665/1676). In Economic Writings, C. H. Hull, ed. Published in 1899 by Cambridge University Press, Cambridge, England. 25. Ruggles, R. (1967). In Ten Economic Studies in the Tradition of Irving Fisher, Wiley, New York, pp. 171–205. 26. Samuelson, P. (1964). Rev. Econ. Stat., 46, 145–154. (Economic theory.) 27. Samuelson, P. (1974). Econ. J., 84, 595–608. (Economic theory.) 28. Sen, A. (1979). J. Econ. Lit., 17, 1–45. (Economic theory, elementary survey; references.) 29. U.K. Board of Trade (1908). Cd. 3864. H. M. Stationery Office, London. 30. U.K. Board of Trade (1908). Cd. 4032. H. M. Stationery Office, London. 31. U.K. Board of Trade (1909). Cd. 4512. H. M. Stationery Office, London. 32. U.K. Board of Trade (1910). Cd. 5065. H. M. Stationery Office, London. 33. U.K. Board of Trade (1911). Cd. 5609. H. M. Stationery Office, London. 34. U.S. Congress (1982). USSR: Measures of Economic Growth and Development. U.S. Government Printing Office, Washington, D.C. 35. Usher, D. (1968). The Price Mechanism and the Meaning of National Income Statistics. Oxford University Press, Oxford. (Theory and application.) 36. Voeller, J. (1981). Purchasing Power Parities for International Comparisons. Verlag, K¨onigstein, German Democratic Republic. See also FISHER’S IDEAL INDEX NUMBER; INDEX NUMBERS; and PAASCHE –LASPEYRES INDEX.

NURI T. JAZAIRI RASESH THAKKAR

19. Kravis, I. B., Heston, A., and Summers, R. (1978). International Comparisons, Johns Hopkins University Press, Baltimore, Md.

PURI’S EXPECTED NORMAL SCORES TEST

20. Kravis, I. B., Heston, A., and Summers, R. (1978). Econ. J., 88, 215–242.

When comparing k populations under completely randomized designs (i.e., one-way∗

6634

PURI’S EXPECTED NORMAL SCORES TEST

layouts), statisticians are often interested in testing for ordered treatment effects among the k groups. See also ISOTONIC INFERENCE and ORDER-RESTRICTED INFERENCES. To treat such problems, numerous statistical procedures have been devised. Under conditions appropriate for classical analysis, the statistician may select a (parametric) procedure from among those developed by Bartholomew [2] or by Abelson and Tukey [1]. On the other hand, under more general conditions one would select a (nonparametric) procedure from among those (chronologically) developed by Terpstra [25], Jonckheere [18], Chacko [11], Puri [21], Conover [13], Johnson and Mehrotra [17], Tryon and Hettmansperger [27], or Wolf and Naus [30]. DEVELOPMENT OF PURI’S TEST If θi represents some location parameter (e.g., the median) and if it is assumed that Fi (x) = F(x − θi ) for some unknown CDF F, the essential problem becomes one of ordered location shift, and the statistician would test H0 : θ1 = θ2 = · · · = θk

(1)

H1 : θ1  θ2  · · ·  θk

(2)

against

(with at least one inequality strict). Puri [21] proposed and developed a family of rank procedures for testing against such ordered alternatives. The test statistic devised in Section 2 of his paper possesses several interesting properties. Depending on the type of ‘‘constants’’ selected for his term (i,j) Ev , the Puri test becomes equivalent to the test developed by Jonckheere [18] for comparing k groups against ordered alternatives (see also ORDERED ALTERNATIVES, JONCKHEERE TESTS FOR) as well as to the tests developed by Mann and Whitney [19], Hoeffding [16] and Terry [26], and Savage [22] for comparing two groups against one-tailed alternatives. Since the Jonckheere test can be thought of as a k-sample generalization of the one-sided two-sample Mann–Whitney–Wilcoxon∗ procedure [19,29], the work of Puri can also be

viewed as basically a k-sample extension of a class of procedures devised for the twosample problem. DESCRIPTION Since Puri was more concerned with the mathematical and large-sample properties of the tests than with actual applications (see refs. 10, 20, and 21), Berenson [6] described a modified and simplified version of the powerful Puri expected normal scores procedure, which facilitates its usage when the sample sizes are equal. To perform the test, consider a one-way layout with k groups and (to avoid ties) n continuous observations per group. Let Xij represent the jth observation in the ith group (i = 1, 2, . . . , k and j = 1, 2, . . . , n). Replace all the nk observations by their corresponding ranks Rij , such that a rank of 1 is given to the smallest of the Xij and a rank of nk is given to the largest value. Now replace the ranks with ξij —their corresponding ‘‘expected normal order statistics.’’ First presented in 1938 by Fisher and Yates [14] in an attempt to transform a set of nonnormal data into one of normal form, the ξij represent the (expected) Z values for each of the nk order statistics when taken from a standardized normal distribution. These expected normal scores are easily read from tables—a most extensive set of which was published by Harter [15]. See FISHER-YATES TESTS and NORMAL SCORES TESTS. The test statistic for the Puri expected normal scores procedure can now be written [6] as V=n

k  (2i − k − 1)ξi· ,

(3)

i=1

 where ξi· = nj=1 ξij and i = 1, 2, . . . , k. The decision rule is to reject the null hypothesis of form (1) in favor of the ordered location alternative of form (2) if  1/2 k  n 3 k(k2 − 1)  n 2 ξij  , (4) V
Zα  3(nk − 1) i=1 j=1

where Zα is the upper α percent critical value under the standardized normal distribution.

PURI’S EXPECTED NORMAL SCORES TEST

The version of the test above has shown to be useful provided nk
15 (see ref. 7). A detailed application is demonstrated in ref. 6.

POWER COMPARISONS Asymptotic power comparisons of the test with some of its competitors under the restriction of equal sample sizes have been given by Bartholomew [3–5], Puri [21], Johnson and Mehrotra [17], and Berenson and Gross [9]. In these studies, two extreme types of ordered location alternatives are considered—the equally spaced linear trend alternative of the form H1 : θ1 < θ2 < · · · < θk

(5)

(where θi+1 − θi is constant for all i = 1, 2, . . . , k − 1) and the end gap alternative of the form H1 : θ1 = θ2 = · · · = θk−1 < θk .

(6)

The results of these studies indicate that Puri’s expected normal scores procedure is excellent for testing against equally spaced linear trend alternatives as in (5). These findings have been corroborated by Berenson [7] in an extensive Monte Carlo∗ study of the small-sample (empirical) powers of the various k sample tests under the restriction of equal n’s. Puri’s procedure may be highly recommended over its competitors as a test against linear trend alternatives as in (5), especially when the statistician is known to be sampling from (a) normal populations; (b) light-tailed populations of the beta family, regardless of symmetry; (c) heavy-tailed∗ and moderately skewed populations (such as Rayleigh and Weibull); and (d) heavy-tailed and very skewed populations (such as Erlang and exponential). A note of caution: Although the Puri test has been highly recommended for testing against equally spaced ordered location alternatives, one might prefer the procedure developed by Tryon and Hettmansperger [27] for such situations. While the Puri test enjoys (slight) power∗ advantages, the TryonHettmansperger procedure requires only the

6635

ranks (Rij ) and not a set of normal scores (ξij ) so that it possesses more ‘‘practical power’’ as defined by Tukey [28]. In addition, the statistician is typically unaware of the form of the parent population(s) being sampled as well as the form that the true ordered location alternative would take [between the extremes of (5) and (6)]. Thus, in general, from ref. 7 the most highly recommended procedure for testing against unspecified ordered location alternatives of form (2) is the Johnson–Mehrotra test [17].

EXTENSIONS Skillings and Wolfe [23,24] have extended the Puri test to the two-way layout by developing an ‘‘among-blocks’’ rank test for randomized complete block designs. While they demonstrate that their statistic has a limiting normal distribution, they do not provide a useful formula for practical applications. However, such an expression is presented in ref. 8.

REFERENCES 1. Abelson, R. P. and Tukey, J. W. (1963). Ann. Math. Statist., 34, 1347–1369. 2. Bartholomew, D. J. (1959). Biometrika, 46, 36–48. 3. Bartholomew, D. J. (1959). Biometrika, 46, 328–335. 4. Bartholomew, D. J. (1961). Biometrika, 48, 325–332. 5. Bartholomew, D. J. (1961). J. R. Statist. Soc. B, 23, 239–272. 6. Berenson, M. L. (1978). Educ. Psychol. Meas., 38, 905–912. 7. Berenson, M. L. (1982). Psychometrika, 47, 265–280, 535–539. 8. Berenson, M. L. (1982). Commun. Statist. Theor. Meth., 11, 1681–1693. 9. Berenson, M. L. and Gross, S. T. (1981). Commun. Statist. B, 10, 405–431. 10. Bhuchongkul, S. and Puri, M. L. (1965). Ann. Math. Statist., 36, 198–202. 11. Chacko, V. J. (1963). Ann. Math. Statist., 34, 945–956. 12. Chernoff, H. and Savage, I. R. (1958). Ann. Math. Statist., 29, 972–994.

6636

PURPOSIVE SAMPLING

13. Conover, W. J. (1967). Ann. Math. Statist., 38, 1726–1730. 14. Fisher, R. A. and Yates, F. (1963). Statistical Tables for Biological, Agricultural, and Medical Research, 6th ed. Hafner, New York. 15. Harter, H. L. (1961). Biometrika, 48, 151–165. 16. Hoeffding, W. (1951). Proc. 2nd Berkeley Symp. Math. Statist. Prob., University of California Press, Berkeley, pp. 83–92. 17. Johnson, R. A. and Mehrotra, K. G. (1971). J. Indian Statist. Ass., 9, 8–23. 18. Jonckheere, A. R. (1954). Biometrika, 41, 133–145. 19. Mann, H. B. and Whitney, D. R. (1947). Ann. Math. Statist., 18, 50–60. 20. Puri, M. L. (1964). Ann. Math. Statist., 35, 102–121. 21. Puri, M. L. (1965). Commun. Pure Appl. Math., 18, 51–63. 22. Savage, I. R. (1956). Ann. Math. Statist., 27, 590–615. 23. Skillings, J. H. and Wolfe, D. A. (1977). Commun. Statist. A, 6, 1453–1463. 24. Skillings, J. H. and Wolfe, D. A. (1978). J. Amer. Statist. Ass., 73, 427–431. 25. Terpstra, T. J. (1952). Nederl. Akad. Wet. Indag. Math., 55, 327–333. 26. Terry, M. E. (1952). Ann. Math. Statist., 23, 346–366. 27. Tryon, P. V. and Hettmansperger, T. P. (1973). Ann. Statist., 1, 1061–1070. 28. Tukey, J. W. (1959). Technometrics, 1, 31– 48. 29. Wilcoxon, F. (1945). Biometrics, 1, 80–83. 30. Wolf, E. H. and Naus, J. I. (1973). J. Amer. Statist. Ass., 68, 994–997. See also BLOCKS, RANDOMIZED COMPLETE; FISHER-YATES TESTS; ISOTONIC INFERENCE; NORMAL SCORES TESTS; ONE-WAY ANALYSIS OF VARIANCE; ORDERED ALTERNATIVES, JONCKHEERE TESTS FOR; ORDERED ALTERNATIVES, PAGE TEST FOR; ORDER-RESTRICTED INFERENCES; RANK TESTS; and WILCOXON-TYPE TESTS FOR ORDERED ALTERNATIVES IN RANDOMIZED BLOCKS.

M. L. BERENSON

PURPOSIVE SAMPLING. See REPRESENTATIVE

SAMPLING

PURPOSIVE SELECTION. See REPRESENTATIVE

SAMPLING

P -VALUES INTRODUCTION AND DEFINITIONS A P-value is frequently used to report the result of a statistical test of significance. A P value is defined as the probability, calculated under the null distribution, of obtaining a test result as extreme as that observed in the sample (in a particular direction). This definition implies that P values are useful when the alternative hypothesis is one-sided because then the relevant direction is determined by the alternative. As a simple illustration, suppose that we are testing H0 : µ = 50 vs. H1 : µ > 50 for a normal population with σ = 10, and a sample of size 36 gives x = 54. A value of X that exceeds the hypothesized µ value of 50 by a significant amount should make us tend to believe that the alternative µ > 50 is true. Therefore, we might find the probability of obtaining a sample X as large as 54, the observed value, when µ = 50. This is the P value, calculated as P = Pr[X
54|µ = 50] = Pr[Z
2.4] = 0.0082. If we need to make a decision and this probability is regarded as small, we should reject H0 ; if this probability is regarded as large, we should not reject H0 . Alternatively, we might simply report the P value. The more classical method of carrying out a test of significance is to preselect a significance level α, the probability of a Type I error, and use α to determine a critical region for the test statistic. In the example of the preceding paragraph, this classical procedure would lead us to find a number cα such that Pr[X
cα |µ = √50] = α. For α = 0.01, cα = 50 + 1.64(10/ 36) = 52.73. The 0.01-level test is to reject H0 if X
52.73 and accept H0 otherwise. With x = 54, we reject H0 for α = 0.01. Hence the relationship between P values and the classical method is that if P  α, we should reject H0 , and if P > α, we should accept H0 . This relationship shows that an important interpretation of a P value is that it is the smallest level at which the observations are significant in a particular direction. This relationship has prompted some other names for a P value, such as the critical level, borderline level, level attained, or observed

P -VALUES AND POWER OF TESTS

level of significance. A P value is also sometimes called a probvalue or an associated probability. ADVANTAGES AND DISADVANTAGES OF P VALUES In well-behaved problems the P value is well-defined, with a specific probability interpretation. It is not arbitrary, as a preselected significance level usually is in practice, and it is objective. An investigator who can report only a P value conveys the maximum amount of information contained in the sample and permits all readers to essentially choose their own level of significance and make their own individual decision about the null hypothesis. This is especially important when there is no justifiable reason for choosing one particular level of significance. P values are especially useful for test statistics that have a discrete null distribution because in such cases a preselected significance level frequently is not attainable (except by resorting to randomized tests∗ ). This phenomenon occurs primarily with distribution-free∗ or nonparametric test statistics, where P values are widely used. A disadvantage of the P value is that its meaning is sometimes misinterpreted. In a single experiment, the P value does indeed measure the degree of agreement or disagreement in a particular direction between the observations and the null hypothesis, but this interpretation cannot be extended to another experiment (or to another sample size). A lengthy discussion of possible mis-interpretations is given in Gibbons and Pratt [1]. Another disadvantage of P values is that they do not take power∗ (or the probability of a Type II error) into account. This criticism, of course, applies equally to classical significance test procedures where sample size is chosen arbitrarily as opposed to being chosen in a manner that would guarantee a preselected value for the probability of each type of error. When P values are used with distribution-free tests, where alternative distributions are not known in general, this criticism is inappropriate. Perhaps the most persuasive argument against an exclusive use of P values is the

6637

problem of defining a P value when the appropriate rejection region is in both tails of the null distribution. One approach to a solution here is to simply report a one-tailed P value that reflects the direction in which the sample statistic differs from its expected value under H0 . Another approach is to double the one-tailed P value, but this solution can only be recommended when the null distribution is symmetric (or nearly so), because then it corresponds to the classical procedure that divides the rejection region into two parts of equal probability. Many other possibilities for defining P values for a two-tailed rejection region have been advanced, but most of them can lead to absurdities or at least anomalies; see Gibbons and Pratt [1] for a complete exposition and references. SUMMARY P values provide a useful alternative approach to classical inference. They are especially appropriate for distribution-free tests (or any test statistic that has a discrete null distribution) with appropriate critical region in only one tail. They are also useful for one-tailed classical tests when the probability of a Type II error is not taken into account, either because there is to be no specific critical region or because of practical considerations. Most modern elementary statistical methods books discuss P values and use them to some extent. REFERENCE 1. Gibbons, J. D. and Pratt, J. W. (1975). Amer. Statist., 29, 20–25; ibid., 31, 99 (1977). See also DISTRIBUTION-FREE METHODS; HYPOTHESIS TESTING; INFERENCE, STATISTICAL —I; INFERENCE, STATISTICAL —II; RANDOMIZED TESTS; and SIGNIFICANCE TESTS, HISTORY AND LOGIC OF.

JEAN D. GIBBONS

P -VALUES AND POWER OF TESTS Although the use of P-values∗ has become most popular with the development of modern statistical packages∗ , their definition is

6638

P -VALUES AND POWER OF TESTS

nearly as old as the concept of hypothesis testing∗ . Indeed, it was in 1900 that Karl Pearson∗ [5] calculated the probability that the observed value of the chi-square statistic would be exceeded under the null hypothesis (see CHI-SQUARE TEST —I). The classical procedure of preassigning a significance level∗ and then determining critical values is due to Fisher in 1925 [3,7]. Gibbons [4] states that it is a disadvantage of P-values that they do not take power∗ (or the probability of Type II error) into account. There is, however, a simple relation between the distribution of the P-values and the power of a test, which we consider next. THEORETICAL BACKGROUND For simplicity consider the one-sided oneparameter hypothesis H0 : θ = θ0 versus the alternative H : θ > θ0 with some test statistic T having the cumulative continuous distribution function F(t; θ ). The classical procedure is to select a significance level∗ α and a critical level T1−α such that Pr[T
T1−α |θ = θ0 ] = α.

The power of the test statistic for a fixed level α is defined as the probability of rejecting H0 as a function of θ : β(θ |α) = Pr[T
T1−α |θ ] = 1 − F(T1−α ; θ ). The P-value of the test statistic T is a random variable defined as P(T) = 1 − F(T; θ0 ). The notions of power and P-value are related as follows: Pr[P(T)  α|θ ] = Pr[T
F −1 ((1 − α); θ0 )|θ ] = Pr[T
T1−α |θ ] = β(θ |α), i.e., the cumulative distribution function of P(T) when the parameter is equal to θ is precisely the power β(θ |α). Though power (for fixed sample size) is usually considered as a function of the parameter, it actually is a function of the parameter and of the level of significance, as shown in Fig. 1, where the hypothesis H0 : θ = 50 vs. H : θ > 50 is tested for a normal population with σ = 10, and a sample of size 36 with the test statistic Z = 6(X − 50)/10 (discussed

Figure 1. Power as function of the parameter θ and the level α.

P -VALUES AND POWER OF TESTS

in ref. [4]). For fixed levels of α we get the usual power functions (Fig. 2), whereas for fixed parameters θ we get the distribution functions of the P-values (Fig. 3). In Fig. 3 we see that the distribution of the P-values is uniform under H0 , a result which is always true if the distribution function of the test statistic is continuous and strictly monotone. The above relationship between the distribution function of the P-value and the

6639

power of the corresponding test holds in all cases (for example, two-sided tests, multiparameter hypotheses, discrete-valued test statistics), so long as the P-value has the property P(T)  α

if and only if

T ∈ R,

where R represents the critical region∗ of the test.

Figure 2. Power as function of the parameter θ.

Figure 3. Power as function of the level α.

6640

P -VALUES AND POWER OF TESTS

EMPIRICAL POWER For cases in which the distribution of the test statistic under H is not known, it is possible to estimate the power of the test by simulation via the empirical distribution of the generated P-values: ˆ ˆ , α) = Pr[P(T) β(θ  α|θ ] = #{P(T)  α|θ }/N, where N is the number of simulations. The advantage of using P-values is that, once these have been computed, they can be used to estimate β(θ , α) for any desired level of significance α. Secondly, the empirical distribution function (EDF) is not dependent on an arbitrarily chosen level of significance that can only be approximately met when the test statistic is discrete-valued. Note that #{P(T)  α|θ } is binomially distributed with parameters N and β(θ |α), a result that enables one to compute confidence intervals for β(θ |α). Graphical representations of the EDF [1,2,6] provide useful information about power, especially if one wants to compare alternative test statistics. For illustration we consider a situation where the distributions of the test statistics are known. Consider the sign test∗ statistic, B = #{observed values > θ0 }, for the above hypothesis.

Figure 4. Boxplots of Pvalues of Z and B for different values of the parameter θ (1,000 simulations each).

Figure 4 shows boxplots of simulated Pvalues of Z and B for θ = 50, 52, . . . , 58, based on 1000 normal samples of size 36 and with standard deviation σ = 10. For θ = 50 both distributions should be located at about 0.5. The distribution of B is not quite so, because it is discrete. That the EDFs of Z for θ > 50 are ‘‘lower’’ than those of B verifies the known fact that Z has a higher power than B for the hypothesis considered here. This superior power is also evident in Fig. 5, which shows a plot of the EDFs of the P-values of Z and B for θ = 52. The jumps of the EDF of B occur because B is discretevalued. These methods are also suitable for checking whether the test statistic is able to meet a desired level of significance in cases where only an asymptotic distribution of the test statistic is known under H0 . Consider, for example, the distribution of the P-values for the Pearson χ 2 statistic to test the hypothesis of independence in a 2 × 2 contingency table∗ . It is well known that the asymptotic distribution (χ12 ) is a poor approximation when some of the expected frequencies are small. This is illustrated in Fig. 6, which gives a plot of the empirical distribution of the P-values based on 1000 2 × 2 tables generated under H0 with sample size 100 and expected frequencies m11 = m22 = 9, m12 = 81, and m21 = 1. Also plotted is a 99% confidence band for the true (finite-sample) distribution.

P -VALUES AND POWER OF TESTS

6641

Figure 5. Empirical distribution function of the P-values for Z and B for fixed θ = 52.

Figure 6. Empirical distribution function of 1,000 P-values under H0 .

Acknowledgment I am indebted to Walter Zucchini for helpful comments on an earlier draft.

REFERENCES 1. B¨oker, F. and Dannenberg, O. (1995). ¨ ¨ Was k¨onnen P-Werte uber die Gute von ¨ oßen in vergleichenden SimulationPrufgr¨ sstudien aussagen? Allg. Statist. Arch., 79,

233–251. (The empirical distributions of the P-values of several test statistics are compared using methods of explorative data analysis.) 2. B¨oker, F. and Dannenberg, O. (1996). Exploratory data analysis for a comparison of statistical test procedures. In SoftStat ’95, Advances in Statistical Software 5, F. Faulbaum and W. Bandilla, eds. Lucius & Lucius, Stuttgart, pp. 97–104. 3. Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver & Boyd, Edinburgh.

6642

4.

5.

6.

7.

P -VALUES, COMBINATION OF

(See Section 20 for a discussion of the distinction between tables with P-values and tables with significance points.) Gibbons, J. D. (1986). P-values. In Encyclopedia of Statistical Sciences, Vol. 7, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley, New York, pp. 366–368. Pearson, K. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Phil. Mag., 5, 157–175. (Calculates P-values for the chisquare goodness-of-fit statistic.) Trenkler, D. (1996). Computergestutzte ¨ Tests in ¨ Statistik und Okonometrie: Grundlagen und Anwendungen. Habilitation thesis. Faculty of ¨ Economics, University of Osnabruck. (Gives boxplots of P-values.) Watson, G. S. (1983). Hypothesis testing. In Encyclopedia of Statistical Sciences, Vol. 3, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley, New York, pp. 713–722.

BIBLIOGRAPHY Bickel, P. J. and Doksum, K. J. (1977). Mathematical Statistics. Holden Day, San Francisco. (In this book you may find the definition and interpretation of a P-value.) Cox, D. R. (1977). The role of significance tests (with discussion). Scand. J. Statist., 4, 49–70. Singh, K. and Berk, R. H. (1994). A concept of type2 P-value. Statist. Sinica, 4, 493–504. Tsui, K.-W. and Weerahandi, S. (1989). Generalized P-values in significance testing of hypotheses in the presence of nuisance parameters. J. Amer. Statist. Ass., 84, 602–607; 86, 256. Westfall, P. H. and Young, S. S. (1993). Resampling-Based Multiple Testing, Examples and Methods for P-value Adjustment. Wiley, New York. Whitmore, G. A. and Xekalaki, E. (1990). P-values as measures of predictive validity. Biom. J., 32, 977–983. (Defines Predictive P-values and relates their distribution to predictive power.) See also P-VALUES. ¨ FRED BOKER

P -VALUES, COMBINATION OF Combinations of P-values from independent significance tests have long been used to

summarize data across studies and samples. Tests proposed by Fisher [9] and Tippett [31], and Westberg’s [34] adaptive combination of those tests, are described in earlier volumes (see TESTS AND P-VALUES, COMBINING; TESTS, WESTBERG ADAPTIVE COMBINATION OF). Combinations proposed by E. S. Pearson [24] and K. Pearson [26], as well as the tests of Fisher and Tippett, date from the 1930s. (Fisher’s method is described in UNIFORMITY, TESTS OF.) P-value combinations are easy to compute, and require only a series of observed P-values. They can therefore be applied to most sets of data, even ones that do not satisfy the stricter assumptions underlying analyses of effect magnitudes. Even when studies use different sampling designs and analyses, Pvalues from continuous tests are identically distributed under the null hypothesis. Consequently, P-value summaries are very flexible and widely applicable. Though flexible, the methods do not provide detailed information about outcomes. If negative outcomes can occur (but are not of interest), a significant summary does not even ensure that the average effect is positive. The combinations indicate whether any populations seem to show positive effects, but not how many populations (or which ones) show positive effects, or whether any populations show negative effects. Under any null hypothesis, the density of a P from a continuous test statistic is uniform [0, 1]. The densities of P’s under alternative models depend on the distribution of the test statistic used, the values of relevant parameters (under the alternative), and the sample size. Therefore, P-values∗ are not pure measures of treatment effects or relationships. Small P-values may arise from large effects or from large samples or experimental-design advantages. Thus, combinations based on effect-magnitude measures are usually preferable to P-value combinations. However, when study outcomes are not on commensurate scales, or when studies have used different analyses of a phenomenon, P-value summaries may be preferable. For instance, a series of medical studies may investigate possible deleterious effects of a drug. Different side effects may be examined

P -VALUES, COMBINATION OF

in each study, since the research (overall) aims to find any kind of deleterious effect. It may not be necessary to understand why one kind of side effect appears stronger than another. In such a case, P-value summaries may be the only summaries that can be applied to the full series of studies. HYPOTHESES The Null Hypothesis The null hypothesis tested by P-value combinations is that the substantive effects or phenomena of interest are not present in any of the populations studied. The statistical null hypothesis for a series of k studies is H0 : θi = 0 for i = 1, . . . , k; that is, all the individual null hypotheses must be true. The θi need not refer to the same parameters. For instance, in the medical example, θ1 could represent a difference in death rates due to the drug, and θ2 could represent a difference in mean blood pressures. Van Zwet and Oosterhoff [32] noted that in some cases H0 can be extended as H0 : θi  0, for i = 1, . . . , k, but they argued that the possibility of having both positive and negative θi values should be ‘‘simply ruled out in advance’’ [32, p. 660] in applications of one-sided combination procedures. Although two-sided tests (e.g., tests of H0 : θ = 0 vs. θ = 0) are common in primary research, most summaries of P-values are meant to be computed from one-sided P’s. Two-sided P-values are problematic because they do not convey information about the direction of deviations from H0 . Birnbaum [5] and George [10] have examined the problem of combining two-sided P-values. Alternative Hypotheses The classical one-sided combination problem has been the focus of almost all of the literature on combining P-values. Within each study, a directional test of θi = 0 vs. θi > 0 is conducted. The one-sided alternative for the series of k studies is H1 : θi
0 for i = 1, . . . , k, with θj > 0 for at least one j. At least one parameter must be positive. Without further analysis the reviewer cannot discern how many studies or which studies sample populations with nonnull effects.

6643

P-value combinations are much like the overall F-test for a multilevel factor in analysis of variance, indicating whether there is greater than random variability in outcomes, but not revealing the specific structure of that variability. The one-sided P-value combination can be used to detect individual contrary findings in a mass of support for a treatment or effect. The direction of the expected outcome would be treated as the null condition. For example, in our summary of evidence regarding the effects of a seemingly efficacious drug, θi  0 might represent positive outcomes (e.g., decreased blood pressure) which could be ignored if deleterious effects are sought. Health problems or detrimental outcomes would be represented by θi > 0. Sometimes prior information suggests that certain values of θi are implausible or impossible. For example, θi may represent the squared multiple correlation∗ coefficient, or a variance which can only be zero or positive. A directional test may then be used. Directional tests also apply when the reviewer is not interested in negative θi -values; negative estimates of θi are taken as evidence that θi = 0. This is common practice, for example, in estimating variance components∗ . Confusion can result if one-sided combinations are applied when there are both positive and negative θi -values. Some tests can support both one-sided alternatives: ‘‘at least one θi > 0’’ and ‘‘at least one θi < 0’’ (Adcock [1]; Rosenthal [28]). Other combinations are more likely to retain H0 when both positive and negative θi ’s exist, regardless of which directional hypothesis is examined. When all studies examine the same parameter, the classical P-value combinationsalso provide a test of H0M : θ = 0, where θ = θi /k = 0. However, if it is reasonable to expect both positive and negative values of θi , and if it is acceptable for negative and positive θi values to cancel each other, certain one-sided summaries should not be used to test H0M . Only tests that allow positive and negative θi -values (represented by large and small P’s) to cancel when one-sided P’s are combined (i.e., the sum of z’s [30], the logit [10], and weighted versions of these tests) can be used to examine the hypothesis H0M : θ = 0.

P -VALUES, COMBINATION OF

6644

METHODS FOR SUMMARIZING P -VALUES Combinations of P-values arise from a few general principles, primarily the assumption that the P-value is from a continuous test statistic. The combinations are based either directly on the uniform distribution, or on statistical theory for transformed uniform variables. Formulas and decision rules for these and other P-value combinations are given in Table 1. The uniform summaries encompass counting methods (including vote counting, discussed by Hedges and Olkin [13]) and linear combinations of P’s. In the probabilitytransformation summaries, the P-values are transformed through inverse probability distribution functions, F −1 (P). Any legitimate distributional form may be used for F −1 ; the

normal, t, and chi-square are used often. A summary of the transformed P-values is then calculated. Liptak [19] presented a comprehensive treatment of this idea. Discrete statistical tests, such as binomial tests for small samples, take on a limited set of values. Probability values based on discrete test statistics do not have the uniform distribution, even under H0 , and should be combined with P’s from continuous tests only with adjustments (Lancaster [17]; Pearson [25]; Wallis [33]). Weighting Weighting in P-value combinations has been proposed to allow for prior information about the sizes of effects, for subjective differences (e.g., in judged quality) between the studies being summarized, or for differences in

Table 1. Decision Rules for Significance-Level Summariesa Method

Wilkinson [35] Brozek and Tiede [6] Minimum P [31] Sign test [28] Chi-square [35] Sum of P’s [7]

Mean P [8] Sum of z’s [30] Weighted sum of z’s [20] Mean z [12] Inverse tb [36] Inverse chi-squareb [37] Sum of logs [9] Weighted sum of logs [21] Logit [10] Weighted logit [11]

Reject H0 if Uniform Summaries  k P∗ = ks=r α s (1 − α)k−s < α ∗ √ s (r − kα)/ kα(1 − α) > z(α ∗ ) Min (P1 , . . . , Pk ) = P[1] < α = 1 − (1 − α ∗ )1/k (r − r )2 /(r + r ) > χ12 (α ∗ ) J 2 2 ∗ j=1 (Oj − Ej ) /Ej > χJ−1 (α )  $ k  k   k #k Pr S  i=1 Pi = (k!)−1 Sr=0 i=1 Pi − r , s  where# ki=1 Pi
S ∈ Integers $ √ √  12k 0.5 − kj=1 Pi /k = 12k(0.5 − P) > z(α ∗ ) k

Probability-Transformation Summaries √ √ 

−1 (1 − Pi )/ k > z(α ∗ ) i=1 z(Pi )/ k%= k k w z(Pi )/ w2 > z(α ∗ )  i=1√ i i=1 i k ∗ i=1 z(Pi )/k /(sz / k) = z/sz > tk−1 (α )

% k k ∗ i=1 tfi (Pi )/ i=1 fi /(fi − 2) > z(α ) k 2 2 ∗ χfi (Pi ) > χfi (α ) i=1 & k 2 −2 log Pi = −2 log ki=1 Pi > χ2k (α ∗ ) i=1 & k k wi − log i=1 Pi = − i=1 wi log Pi > C(α ∗ )  −[kπ 2 (5k + 2)/3(5k + 4)]−1/2 ki=1 log[Pi /(1 − Pi )] > t5k+4 (α ∗ ) k − i=1 wi log[Pi /(1 − Pi )] > Cv tv (α ∗ )

a r is the number of P-values less than 0.5, r the number greater than 0.5, and O and E are the observed and expected j j  numbers of P-values in the jth of J classes, based on the assumption of uniformly distributed P’s. The mean P is P = ki=1 Pi /k. 2 Also, z(a) is the standard normal deviate associated with upper tail probability a, and χfi (a) is the 100(1 − a) percentile point

of the chi-square distribution with fi degrees of freedom. The computed variance for the kz(Pi )-values is s2z . The t-value with fi degrees of freedom associated with upper tail probability a is tfi (a), and for the weighted sum-of-logs and weighted logit methods, C(α ∗ ), Cv , and v depend on the values of w1 through wk . b Weighted inverse t and chi-square summaries are computed by selecting different degrees of freedom f than those used i in the unweighted summaries.

P -VALUES, COMBINATION OF

sample size or degrees of freedom (including the extreme situation in which data are missing). Weighted versions of all the tests are possible. Weighting according to prior knowledge about effect magnitude is equivalent to using Bayesian methods, and can greatly improve the power of the combination procedure (Koziol and Perlman [14,15]). However, Zelen and Joel [38] showed that improper weighting can reduce power. Weighting according to study quality or other subjective differences has been debated in the meta-analysis literature (e.g., Rosenthal [29, p. 55]), but little is known about the influence of such weighting on the statistical properties of P-value combinations. Weighting on the basis of sample size was suggested by Mosteller and Bush [20]. However, Perlman argued that P-values ‘‘are already weighted’’ [27, p. 4], because for any given effect size, larger studies will have smaller P’s. Giving larger studies even more weight would be inappropriate and might adversely affect the power of the methods. Weighting has also been discussed by Liptak [19], Oosterhoff [22], and Pape [23]. Power No P-value combination is most powerful. The alternative model greatly influences which test(s) have high power. Many authors have investigated the power of P-value combinations (e.g., Becker [2], George [10], Koziol and Perlman [15], Koziol et al. [16], Oosterhoff [22], Zelen and Joel [38]). Though the complete results of these power studies are very detailed, a few broad conclusions can be made: 1. Many of the tests do not vary greatly in their levels of power across a number of different situations. Becker [2] reported, though, that the mean P test was considerably less powerful than others for a number of common patterns of outcomes. (Presumably the sum of P’s would perform similarly.) 2. Fisher’s sum of logs [9] has been widely recommended for detecting single deviant outcomes. However, for summaries of P’s from chi-square tests, Koziol and Perlman [15] reported that

6645

Tippett’s [31] minimum P test was more powerful than Fisher’s summary for detecting single unusual populations. 3. The sum of z’s test and the logit test are more powerful than Fisher’s summary when sample sizes and population effect magnitudes are equal. The advantage in power of the sum of z’s test is not great, however, which has led a number of researchers to recommend Fisher’s test as a general purpose summary. 4. The mean z test statistic is large when the z(P) values are similar. Thus it can reject H0 when H0 is exactly true (e.g., if all P’s are close to 0.5), as well as when it is false. The power of this test has not been formally investigated, but it seems unlikely to be optimal. EXAMPLE This example shows the application of four Pvalue summaries to a set of studies on gender differences in conformity [3]. Large positive effect sizes and small P-values show higher average conformity scores for females. The data are listed in Table 2. The table also lists the components of three of the tests (the sum of z’s, sum of logs, and logit tests), which are summed to compute the tests. All four summaries reject the hypothesis of no gender difference in conformity and, given the direction of the alternative hypothesis, suggest that females conform more (on average) than males in at least one population. The minimum Pvalue is 0.0029, which is less than its critical value of 1 − (1 − α)0.10 = 0.0051 for an overall α = .05. The sum of z’s, sum of logs, and logit tests are all significant beyond the .05 level. Their sample values and observed P-values are given in Table 2. CONCLUSION When the data to be summarized meet the assumption of a parametric model, P-value combinations are relatively uninformative, are redundant with analyses of effect magnitudes, and can be misleading when misinterpreted [4]. Analyses of effect magnitudes

6646

P -VALUES, COMBINATION OF

Table 2. Significance-Level Summaries for Studies of Gender Differences in Conformity Study 1 2 3 4 5 6 7 8 9 10

Effect sizea

P-value

z(P)

0.35 0.37 −0.06 −0.30 0.70 0.40 0.48 0.85 −0.33 0.07

.0029 .0510 .6310 .9783 .0034 .0305 .0340 .0367 .9740 .3517

2.76 1.64 −0.33 −2.02 2.71 1.87 1.82 1.79 −1.94 0.38

Suma Test value Observed P-value for test a Sums

8.67 2.74 0.0031

−2 log P

− log[P/(1 − P)]

11.68 5.95 0.92 0.04 11.37 6.98 6.76 6.61 0.05 2.09

5.84 2.92 −0.54 −3.81 5.68 3.46 3.35 3.27 −3.62 0.61

52.45 52.45 0.0001

17.16 3.05 0.0018

were computed before individual entries were rounded to two decimal places.

are preferable to combinations of P-values when inferences about the size and strength of effects are sought. When studies have examined a common substantive issue but are represented by a variety of different effect-magnitude measures, combinations of P-values may be the best summaries available. REFERENCES 1. Adcock, C. J. (1960). A note on combining probabilities. Psychometrika, 25, 303–305. 2. Becker, B. J. (1985). Applying Tests of Combined Significance: Hypotheses and Power Considerations. Unpublished doctoral dissertation, University of Chicago. 3. Becker, B. J. (1986). Influence again: an examination of reviews and studies of gender differences in social influence. In The Psychology of Gender: Advances through Meta-analysis, J. S. Hyde and M. C. Linn, eds. Johns Hopkins University Press, Baltimore. 4. Becker, B. J. (1987). Applying tests of combined significance in meta-analysis. Psychol. Bull., 102, 164–171. 5. Birnbaum, A. (1954). Combining independent tests of significance. J. Amer. Statist. Ass., 49, 559–574. 6. Brozek, J. and Tiede, K. (1952). Reliable and questionable significance in a series of statistical tests. Psychol. Bull., 49, 339–341. 7. Edgington, E. S. (1972). An additive method for combining probability values from independent experiments. J. Psychol., 80, 351–363.

8. Edgington, E. S. (1972). A normal curve method for combining probability values from independent experiments. J. Psychol., 82, 85–89. 9. Fisher, R. A. (1932). Statistical Methods for Research Workers, 4th ed. Oliver & Boyd, London. 10. George, E. O. (1977). Combining Independent One-Sided and Two-Sided Statistical Tests: Some Theory and Applications. Unpublished doctoral dissertation, University of Rochester. 11. Good, I. J. (1955). On the weighted combination of significance tests. J. R. Statist. Soc. B, 17, 264–265. 12. Gordon, M. H., Loveland, E. H., and Cureton, E. E. (1952). An extended table of chisquare for two degrees of freedom, for use in combining probabilities from independent samples. Psychometrika, 17, 311–316. 13. Hedges, L. V. and Olkin, I. (1980). Votecounting methods in research synthesis. Psychol. Bull., 93, 563–573. 14. Koziol, J. A. and Perlman, M. (1976). Combining Independent Chi-Squared Tests. Tech. Rep. 19, Department of Statistics, University of Chicago. 15. Koziol, J. A. and Perlman, M. (1976). Combining Independent Chi-Squared Tests. J. Amer. Statist. Ass., 73, 753–763. 16. Koziol, J. A., Perlman, M. D., and Rasmussen, U. A. (1988). Combining independent F tests. Tech. Rep. 129, Department of Statistics, University of Washington. 17. Lancaster, H. O. (1949). The combination of probabilities arising from data in discrete distributions. Biometrika, 36, 370–382.

P -VALUES, LIMITING 18. Lancaster, H. O. (1961). The combination of probabilities: an application of orthonormal functions. Austral. J. Statist., 3, 20–33. 19. Liptak, T. (1958). On the combination of independent tests. Magyar Tudomanyos Akad. Mat. Kutato Intezetenek Kozlemenyei, 3, 171–197. 20. Mosteller, F., and Bush, R. R. (1954). Selected quantitative techniques. In Handbook of Social Psychology: Vol. 1. Theory and Method, G. Lindzey, ed. Addison-Wesley, Cambridge, Mass. 21. Mudholkar, G. S., and George, E. O. (1979). The logit statistic for combining probabilities: an overview. In Optimizing Methods in Statistics, J. S. Rustagi, ed. Academic Press, New York. 22. Oosterhoff, J. (1969). Combination of OneSided Statistical Tests. Mathematical Center Tracts 28. Mathematische Centrum, Amsterdam.

33. Wallis, W. A. (1942). Compounding probabilities from independent significance tests. Econometrica, 10, 229–248. 34. Westberg, M. (1985). An adaptive method of combining independent statistical tests. Statistician (London), 34, 287–296. 35. Wilkinson, B. A. (1951). A statistical consideration in psychological research. Psychol. Bull., 48, 156–158. 36. Winer, B. F. (1971). Statistical Principles in Experimental Design, 2nd ed. McGraw-Hill, New York. 37. Yates, F. (1955). A note on the application of the combination of probabilities test to a set of 2 × 2 tables. Biometrika, 42, 404–411. 38. Zelen, M., and Joel, L. (1959). The weighted compounding of two independent significance tests. Ann. Math. Statist., 30, 885–895. See also P-VALUES; TESTS AND P-VALUES, COMBINING; and TESTS, WESTBERG ADAPTIVE COMBINATION OF.

23. Pape, E. S. (1972). A combination of Fstatistics. Technometrics, 14, 89–99. 24. Pearson, E. S. (1938). The probability integral transformation for testing goodness of fit and combining independent tests of significance. Biometrika, 30, 134–148. 25. Pearson, E. S. (1950). On questions raised by the combination of tests based on discontinuous distributions. Biometrika, 37, 383–398. 26. Pearson, K. (1933). On a method of determining whether a sample of size n supposed to have been drawn from a parent population having a known probability integral has probably been drawn at random. Biometrika, 25, 379–410. 27. Perlman, M. D. (1978, October 27). Letter to Frederick Mosteller. 28. Rosenthal, R. (1978). Combining results of independent studies. Psychol. Bull., 85, 185–193. 29. Rosenthal, R. (1984). Meta-analytic Procedures for Social Research. Sage, Beverly Hills, Calif. 30. Stouffer, S. A., Suchman, E. A., DeVinney, L. C., Star, S. A., and Williams, R. M., Jr. (1949). The American Soldier: Adjustment during Army Life, Vol. 1. Princeton University Press, Princeton, N.J. 31. Tippett, L. H. C. (1931). The Methods of Statistics, 1st ed. Williams & Norgate, London. 32. van Zwet, W. R. and Oosterhoff, J. (1967). On the combination of independent test statistics. Ann. Math. Statist., 38, 659–680.

6647

BETSY JANE BECKER MUNIR AHMAD

P -VALUES, LIMITING This concept was introduced [4] to combine the notions of classical P-values* for test statistics and to allow for resampling from the empirical distribution while still satisfying null distribution constraints. Let X1 , . . . , Xn be a (possibly multivariate) random sample from a population with cumulative distribution function (cdf) F. We are testing H0 : F versus Ha : F ∗ . Let pn be a statistic defined by X1 , . . . , Xn . The sequence {pn } of statistics is called a limiting P-value (denoted by LP) sequence if pn [0, 1] and satisfies (a) lim supn→∞ PrF (pn  t)  t for all F and all t [0, 1], (b) pn → 0 in probability for all F ∗ . Replacing (a) by the requirement (a) lim supn→∞ sup Pr(pn  t)  t for all t [0, 1], we arrive at the notion of a limiting P-value in the strong sense. To motivate the definition, consider the size-α test φn derived from the statistic pn , namely,  1 iff pn  α φn = 0 otherwise

6648

PYRAMID SCHEME MODELS

(φn = 1 corresponds to the decision to reject H0 ). If {pn } is an LP sequence, then for any F ,

tests and for a broad class of problems involving finite or infinite-dimensional parameters.

PrF (Type I error using φn ) = PrF (pn  α)  α

REFERENCES

for all n sufficiently large (property (a)) and PrF (Type II error using φn ) = PrF (pn > α) tends to 0 as n → ∞ for F ∗ (property (b)). Several notions of approximate P-values are available in the literature (see, e.g., Refs. 1 and 2). These are all included in the LP concept. For example, let F be a standard normal cdf  and let X and Sn be the sample mean and sample standard deviation of the Xi ’s respectively. If X1∗ , . . . , Xn∗ is a bootstrap sample from the Xi ’s (based on resampling with replacement) and P∗ (·) is the bootstrap distribution conditional on the Xi ’s with the ∗ bootstrap counterparts X and S∗n , then the P-value ( ' X − µ0 √ pn = 1 −  Sn / n (for testing H0 : µ = µ0 vs Ha : µ > µ0 ) is based on the limiting considerations of the studentized statistics. We also have the socalled Type-2 P-value [5] ∗

pn = P∗ (X  µ0 ) and the two bootstrap-oriented P-values ∗

pn = P∗ (X > 2X − µ0 ) or [3] ' pn = P ∗

∗

X −X X − µ0 √ > √ S∗n / n Sn / n

( .

All of these pn ’s are LPs. Specifically, each one of them converges weakly to the uniform* [0,1] distribution for any fixed F and degenerates to zero for F ∗ . Liu and Singh [4] construct an LP on the basis of bootstrap and of the notion of data depth*, which is powerful for multivariate

1. Beran, R. (1986). Simulated power functions. Ann. Stat., 14, 151–173. 2. Hall, P. (1988). Theoretical comparison of bootstrap confidence intervals. Ann. Stat., 16, 927–953. 3. Lambert, D. (1981). Inference function for testing. J. Amer. Stat. Assoc., 76, 649–657. 4. Liu, R. Y. and Singh, K. (1997). Notions of limiting P values based on data depth and bootstrap. J. Am. Stat. Assoc., 92, 266–277. 5. Singh, K. and Berk, R. H. (1994). A concept of Type-2 P-value. Stat. Sin., 4, 493–504. See also BOOTSTRAP and P-VALUES.

PYRAMID SCHEME MODELS Promoters of chain letters and related pyramid business ventures who promise potential participants that they can earn a substantial return on their investment primarily by recruiting new participants are subject to prosecution under various laws [5] concerning fraud and ‘‘misrepresentation,’’ as the pool of potential participants is exhausted before most actual participants recover their initial fee much less make a profit. One approach to demonstrating this relies on the geometric progression. For example, assuming that each participant obtains two new recruits every month, after n months the total number of participants would be 3n . After a year and a half (n = 18), this number (about 387 × 106 ) would be larger than the U.S. population, so the process must stop. This approach has been used successfully in some cases, e.g., State of Connecticut v. Bull Investment Group [32 Conn., Sup. 279 (1975)]; however, it was rejected as being too far removed from the real world in Gerro-Mar, Inc. v. FTC [518 F. 2d, 33 (1975)]. Furthermore, promoters have modified their business ventures by limiting the total number of active participants or the number of persons a participant may recruit, unless he or she rejoins, in an effort to avoid prosecution.

PYRAMID SCHEME MODELS

Stochastic models of such pyramid processes were developed to provide more realistic models of their growth and ultimate extinction. Two models emphasizing how much a participant’s potential earnings depends on the time of entry will be described. A QUOTA PYRAMID SCHEME The ‘‘Golden Book of Values’’ case ([6] and State of Connecticut v. Bull Investment Group [32 Conn., Sup. 279 (1975)]) offered dealerships in the development of a book of discounts to be sold to the public. After paying $2,500 for a dealership, a participant could earn about $100 for obtaining a merchant to advertise a special discount in the book and could recruit other dealers, receiving $900 for each one. Thus the recruitment process provided the largest share of the potential earnings described in the brochure. The total number of dealerships sold in an area was limited to N (270 in this case). To model the number of recruits a participant may make, consider the kth entrant and his or her chance of recruiting the next (k + 1)st participant. Assuming that the current k participants have an equal chance, the kth entrant has probability 1/k of recruiting the (k + 1)st entrant. Similarly, the kth entrant has probability 1/i of recruiting the (i + 1)st entrant for i = k, . . . , N. Thus the number of recruits is expressible as Sk =

N−1 

Xi ,

(1)

i=1

where Xi is a Bernoulli random variable taking the values 1 and 0 with probabilities 1/i and 1 − 1/i, respectively. The misrepresentation in the promotional brochure can be demonstrated by the following consequences of (1), derived in refs. 3 and 6. 1. The expected fraction of all participants who recruit no one is one-half. 2. Participants who enter after the process reaches 37% of its ultimate size cannot expect to recruit more than one person.

6649

3. The probability of a participant recruiting sufficiently many new entrants to recoup his or her initial fee, P[Sk
3], decreases rapidly with k. Indeed, for k equal to one-third of the ultimate size, P[Sk
3] is slightly less than 0.1. Moreover, a Lorenz curve∗ of recruits by time of entry [7] shows that 33% of all participants are expected to be recruited by the first 10%, while participants entering during the last half of the process collectively are expected to recruit only 15% of the total. 4. For large N (i.e., implicitly assuming that the pyramid develops [6]) the expected fraction of all participants who obtain exactly r recruits tends to (1/2)r+1 . This last result was also obtained by Meir and Moon [9], who observed that the graph describing the process (i.e., which connects each participant to his or her recruiter), forms a recursive tree, and participants with exactly r recruits at the nth stage correspond to nodes of degree r + 1. Hence the result follows from Na and Rapoport [10]. For finite n, an exact expression has been obtained [4]. A PYRAMID LIMITING THE NUMBER OF RECRUITS A chain letter [2] was circulated with six names on it. A purchaser invests $2x, paying x to the person selling him the letter and $x to the person at the top of the letter. The purchaser then sells two letters, with his name at the bottom. The promoters note that the purchaser recoups the purchase price when he sells two letters and after his recruits sell their letters and the process continues, he will ultimately receive $64x. Moreover, they asserted that once participants earn money they will rejoin, so the pool of potential buyers is continually replenished. To describe the chain after the kth letter is sold, denote the number of participants who have sold 0, 1, or 2 letters by Uk , Vk , and Wk , respectively. Assuming that the Uk + Vk persons who can sell the (k + 1)st letter are equally likely to be successful, the process is a Markov chain.

6650

PYRAMID SCHEME MODELS

As Uk + Vk + Wk increase by one at each step and Uk+1 − Uk = Wk+1 − Wk , Wk = Uk − c, and Vk = k + 2c − 2Uk . Thus the process is described by the Markov chain {Uk } with U0 = c. Assuming that all participants who sell their two letters rejoin, the mathematical model implies that the process stops before 2n letters are sold in a population of n potential participants. Since the number of persons selling the letter must be less than n for the process to continue, Uk + Vk < n, while Vk > 0, that is, k + c − Uk < n, and 2Uk < k + c, or Uk < (k + c)/2. In order for k + c − Uk < n and Uk < (k + c)/2 both to hold, k < 2n − 2c. Thus the process cannot continue past 2n − 2c steps, and consequently only a very small fraction of the participants can possibly gain large amounts of money. The argument above already implies that most purchasers cannot earn the planned returns. Using the diffusion approximation [1,2,11] to the limiting Markov chain, it can be shown that: 1. At the time the process stops a total of approximately 1.62n letters will have been sold. 2. At this stopping point, all n persons are behind by either $x or $2x, depending on whether they have 1 or 2 letters to sell and approximately 61.8% of them are in the second category. FURTHER DEVELOPMENT Multilevel pyramid schemes [5] pay a commission to a participant when his or her descendants (i.e., recruits and their subsequent recruits) are successful. Using a P´olyaEggenberger model [8], one can show [3] that a person joining at the one-third point of the process has only a 30% chance of having three or more descendants (and even fewer direct recruits), so most participants should expect to lose money. Although the stochastic model described assumes that all current participants are equally likely to recruit the next one, the basic conclusion that most investors lose money remains valid when recruiting ability varies among participants. To further study the robustness of the results implied by these

models, more research is needed to model the growth of the process in real time and the fact that unsuccessful participants often drop out. The lifetime of the process is important, as the decision for Ger-ro-Mar, Inc. v. FTC [518 F.2d, 33 (1975)] rejected the ‘‘geometric progression’’ argument, in part because the company had existed for nine years and attracted only a few thousand participants. REFERENCES 1. Arnold, L. (1974). Stochastic Differential Equations: Theory and Applications. Wiley, New York. 2. Bhattacharya, P. K. and Gastwirth, J. L. (1983). In Recent Advances in Statistics; Papers in Honor of Herman Chernoff , M. H. Rizvi, J. S. Rustagi, and D. Siegmund, eds. Academic Press, New York. 3. Bhattacharya, P. K. and Gastwirth, J. L. (1983). (To appear in a special issue of Operations Research, N. Singpurwalla, ed.) 4. Dondajewski, M. and Szymanski, J. (1982). Bull. Acad. Pol. Sci., S´er. Sci. Math., 30(5, 6). 5. Feore, J. (1974). Georgetown Law J., 1257–1293. 6. Gastwirth, J. L. (1977). Amer. Statist., 31, 79–82. 7. Hammick, (1984). Statistical Properties of Pyramid Schemes and Related Stochastic Processes. (Ph.D. dissertation to be submitted to the George Washington University.) 8. Johnson, N. L. and Kotz, S. (1977). Urn Models and Their Application. Wiley, New York. 9. Meir, A. and Moon, J. W. (1978). Utilitas Math., 14, 313–333. 10. Na, H. S. and Rapoport, A. (1970). Math. Biosci., 6, 313–329. 11. Strook, D. W. and Varadhan, S. R. S. (1969). Commun. Pure Appl. Math., 22, 479–530.

BIBLIOGRAPHY See the following works, as well as the references just given, for more information on the topic of pyramid scheme models. Bailey, N. T. J. (1957). The Mathematical Theory of Epidemics. Charles Griffin, London. Feller, W. (1957). An Introduction to Probability Theory and Its Applications, Vol. 1, 2nd ed. Wiley, New York.

PYRRHO’S LEMMA Hodges, J. L., Jr. and LeCam, L. (1960). Ann. Math. Statist., 31, 737–740. Percus, O. E. (1983). Tech. Rep., Courant Institute of Mathematical Sciences, New York. See also MARKOV PROCESSES.

J. L. GASTWIRTH

PYRRHO’S LEMMA Pyrrho of Elis (361–270 B.C.) belonged to a group of ancient Greek philosophers known as Skeptics. They believed that even our senses tell us only how things appear, not what they really are. This lemma named after Pyrrho warns us against reliance on the extensive data mining* so popular at the end of the twentieth and beginning of the twentyfirst century. It also emphasizes that modeling is a matter of design and a test statistic is a design criterion. Conventionally acceptable values of a test statistic indicate solely that the model is not obviously ill-designed. The lemma echoes the skeptical attitude of J. M. Keynes [1]: ‘‘If you have a fair number of variables and can plan about at will with the coefficients and time lags, is it or is it not the case that more than one equally plausible result can be obtained? In short, how far

6651

are the results mechanically and uniquely obtainable from the data, and how far do they depend on the way the cook chooses to go to work?’’ The formal statement is ‘‘For every collection of vectors, consisting of observations on a regressand and regressors, it is possible to get any set of coefficients as well as any set of predictions with variances as small as one desires, just by adding one additional vector from a continuum of vectors.’’ A rigorous mathematical statement and a proof are provided in Reference 2, which also emphasizes the connection to Simpson’s paradox* in 2 × 2 contingency tables* (with nonempty cells). A provocative discussion is also given in Reference 3. REFERENCES 1. Carabelli, A. J. (1988). On Keynes’s Method. MacMillan Press, London. 2. Dijkstra, T. K. (1995). Pyrrho’s lemma, or have it your way. Metrika, 42, 119–124. 3. Hendry, D. (1997). Econometrics: Alchemy or Sciences. Cambridge University Press, Cambridge, U.K. See also REGRESSION COEFFICIENTS; REGRESSION, ITERATIVE; and REGRESSION VARIABLES, SELECTION OF.

Q Q-Q PLOTS. See GRAPHICAL REPRESENTATION OF

are given in the Handbook [17]. FORTRAN programs which are mostly translations of the procedures in ref. 17 are given in the EISPACK Guide [13]. FORTRAN programs which use the QR algorithm to calculate the singular value decomposition are given by Businger and Golub [1], Lawson and Hanson [9], and the LINPACK User’s Guide [4].

DATA

QR ALGORITHM The QR algorithm is the preferred method for calculating all eigenvalues∗ and (optionally) eigenvectors of a nonsparse matrix. A typical use of eigensystems in statistics is in principal components analysis∗ [2,3,6], in which some or all of the eigensystem of a correlation matrix XT X are computed in attempting to determine a few uncorrelated variables which have associated with them the bulk of the variance of the sample. A related application is the calculation of the singular value decomposition∗ of a matrix X (see LINEAR ALGEBRA, COMPUTATIONAL; ref. 9 or 14). The latter may be used to find the least-squares∗ solution of the overdetermined system Xb = y, and hence to solve linear regression∗ problems. It is especially useful when X is rank deficient or nearly so. The singular value problem is equivalent to an eigenvalue problem—the singular values of X are the square roots of the eigenvalues of XT X, and the right singular vectors of X are the eigenvectors of XT X. (The left singular vectors of X are the eigenvectors of XXT .) The most widely used algorithm for calculating the singular value decomposition is a variant of the QR algorithm. Because of the equivalence just noted, the principal components problem for XT X can be viewed as a problem of calculating singular values and singular vectors of X. This point of view has considerable merit: If the singular vectors of X are computed directly, the computed principal components will be more accurate than if XT X is formed and its eigenvectors are then computed [14].

HISTORY The QR algorithm was introduced independently by Francis [5] and Kublanovskaya [8] in 1961. It is an orthogonal variant of Rutishauser’s (1955) LR algorithm [12], itself an outgrowth of the quotient-difference algorithm [11]. Today’s implementations of the QR algorithm are not fundamentally different from the algorithms described by Francis [5]. Golub [7] was the first to suggest (in 1968) that the QR algorithm be used to calculate the singular value decomposition. High-quality software soon followed [1,17]. OTHER ALGORITHMS FOR THE EIGENVALUE PROBLEM The QR algorithm has supplanted the earlier Jacobi method [17, p. 202]. The slicing (Sturm sequence, bisection) method [17, p. 249] is superior to the QR algorithm when only a few eigenvalues are required. For large, sparse matrices other methods are preferred. In this rapidly evolving area Lanczos methods seem to be the best available at present. Subspace iteration is also widely used. See ref. 10. DESCRIPTION OF THE QR ALGORITHM This section describes the algorithm but does not attempt to explain why it works. For a detailed explanation, see ref. 15, in which it is shown that the QR algorithm is just a sophisticated extension of the venerable power method. Let A be an n by n real, symmetric, and positive semidefinite matrix. These

AVAILABILITY OF PROGRAMS High-quality software for the QR algorithm is widely available. Several ALGOL procedures 6652

QR ALGORITHM

assumptions simplify the analysis and are valid in typical statistical applications. (Every correlation matrix XT X has these properties.) However, most of the theory can be extended to the nonsymmetric case. The QR algorithm is based on the QR matrix decomposition: Every matrix can be expressed as a product, A = QR, where Q is orthogonal and R is upper triangular [14] (see LINEAR ALGEBRA, COMPUTATIONAL). If A is nonsingular, then Q and R can be specified uniquely if we require that the entries on the main diagonal of R be positive. The basic (unshifted) QR algorithm generates a sequence of matrices (Am ) as follows: Given Am−1 , let Qm and Rm be the orthogonal and upper triangular matrices, respectively, such that Am−1 = Qm Rm . Then let Am = Rm Qm . In summary, Am−1 = Qm Rm ,

Rm Qm = Am .

(1)

The algorithm starts with A0 = A. The matrices in the sequence (Am ) are orthogonally similar since Am = QTm Am−1 Am , so they all have the same eigenvalues as A. Under most conditions [10,15,16] the matrices will converge to diagonal form, exposing the eigenvalues on the main diagonal. The eigenvectors can be found by accumulating the transforming matrices Qm . Letting ˆ T AQ ˆ m. ˆ m = Q1 Q2 · · · Qm , we have Am = Q Q m For m sufficiently large, Am will be effectively diagonal, from which it follows that the ˆ m form a complete orthonormal columns of Q set of eigenvectors of A. The basic QR algorithm is too inefficient to be competitive. The two reasons are that (a) the cost of each QR step in (1) is high, and (b) the rate of convergence of (Am ) is usually slow. The arithmetic cost of the QR decomposition is about 23 n3 operations. (By an operation we mean one floating-point multiplication and one floating-point addition.) The cost of the second step in (1) is about 16 n3 operations. These costs can be decreased dramatically by first reducing A to tridiagonal form by an orthogonal similarity transformation. This can be done in about 23 n3 operations using elementary reflectors (Householder transformations), and it only has to be done once because the tridiagonal form is preserved by the QR algorithm [10]. The cost of

6653

one iteration of the QR algorithm for a tridiagonal matrix is of order n rather than n3 . The problem of slow convergence is solved by making shifts of origin. The shifted QR algorithm has the form Am−1 − σm−1 I = Qm Rm , Rm Qm + σm−1 I = Am ,

(2)

where the shift σm−1 is chosen to approximate an eigenvalue of A. Let a(k) ij denote the (i, j) entry of Ak . Since a(m) approaches an nn eigenvalue as m → ∞, a good choice is σm = a(m) nn , the Rayleigh quotient shift [10]. A better choice is the Wilkinson shift, which takes σm to be that eigenvalue of the lower right hand 2 by 2 submatrix of Am which is closer to a(m) nn . With this shift the algorithm is guaranteed to converge [10]. With either shift convergence to the nth eigenvalue is of third order (cubic), where it is only of first order (linear) in the unshifted case. Numerous other shifting strategies have been proposed [10]. The eigenvalue λn ≈ a(m) nn can be regarded as found as soon as |a(m) n,n−1 | is sufficiently small. At that point the problem is deflated by deletion of the last row and column of Am , and the search continues with the focus shifted to the next eigenvalue. With the Wilkinson shift the average number of QR steps required per eigenvalue is less than two. Example. We will calculate the eigenvalues of the 2 by 2 matrix
 8 2 A= , 2 5 which are easily seen to be λ1 = 9 and λ2 = 4. We consider the unshifted algorithm first; A = A0 = QR, where

 1 1 8 −2 68 26 , R= √ . Q= √ 68 2 8 68 0 36 Q is orthogonal and R is upper triangular. Then
 1 596 72 A1 = RQ = 68 72 288
 . 8.7647 1.0588 = . 1.0588 4.2353 Notice that A1 is closer to diagonal form than A0 is, in the sense that its off-diagonal entries are closer to zero, also that the main diagonal

6654

QR ALGORITHM

entries of A1 are closer to the eigen-values. On subsequent iterations the main diagonal entries of Ai give better and better estimates of the eigenvalues until after 10 iterations they agree with the true eigenvalues to seven decimal places. The rate of convergence is governed by the ratio of the eigenvalues, which is 49 in this case. If this ratio were closer to 1, then more iterations would be required. If it were closer to 0, fewer iterations would be needed. We now consider the shifted algorithm. The objective of shifting is to shift the eigenvalues in such a way that the ratio of the shifted eigenvalues is closer to zero. The Rayleigh quotient shift of A is σ0 = 5. Then the eigenvalues of A − σ0 I are λ1 − 5 = 4 and λ2 − 5 = −1. The new ratio is − 14 , which is closer to 0 than 49 is. A − σ0 I = QR, where

 1 1 13 6 3 2 . , R= √ Q= √ 13 0 4 13 2 −3 Then


1 51 13 8  0.6154 . 4.0769

A1 = RQ + σ0 I =
. 8.9231 = 0.6154


8 5 + −12 0

 0 5

Notice that the main diagonal entries of A1 are quite close to the eigenvalues. On the next step the Rayleigh quotient shift is σ1 = 4.0769. The eigenvalues of A1 − σ1 I are λ1 − σ1 = 4.9231 and λ2 − σ1 = −0.0769, whose ratio is very close to zero. Therefore, A2 is expected to be substantially better than A1 . In fact,
 . 8.999981 0.009766 . A2 = 0.009766 4.000019 On the next step the shift is σ2 = 4.000019, which gives an even better ratio, and 
. 9.000000 3.7 × 10−7 . A3 = −7 3.7 × 10 4.000000 It is evident from this example that shifting introduces a positive feedback mechanism which accelerates convergence dramatically. The Wilkinson shift is even better than the Rayleigh quotient shift; it chooses the shift based on a 2 by 2 submatrix, which would be the whole matrix in this case. The shift so chosen would be exactly an eigenvalue, and the algorithm would converge in one iteration.

NOTES ON IMPLEMENTATIONS Of the many ways of carrying out the computations there are two basic variants. The explicit QR algorithm performs each QR step in a straightforward manner, essentially as shown in (2). The implicit QR algorithm obtains the same result by a different sequence of operations [10,14]. In both cases the transforming matrix Qm is built up as a product of elementary reflectors (Householder transformations) or plane rotators (Givens transformations). Real nonsymmetric matrices may have complex eigenvalues, which always occur in conjugate pairs. It is therefore desirable to use complex shifts. A variant of the implicit QR algorithm called the double QR algorithm effectively takes two QR steps at once, using complex conjugate shifts χ and χ . Not only is the resulting matrix real, but the entire double step is carried out in real arithmetic. The double QR algorithm converges not to diagonal form, but to block upper triangular form, with a 2 by 2 block on the main diagonal for each complex conjugate pair of eigenvalues. Another variant of the implicit QR algorithm is used to find the singular values of X. First X is reduced by elementary reflectors to an upper bidiagonal matrix Y. Then YT Y is tridiagonal, and the square roots of the eigenvalues of YT Y are the singular values of Y (and of X). Y is further reduced to diagonal form (exposing the singular values on the main diagonal) by a sequence of transformations which is equivalent to the QR algorithm applied to YT Y. Thus the algorithm effectively calculates the eigenvalues of YT Y without ever forming YT Y. REFERENCES 1. Businger, P. A. and Golub, G. H. (1969). Numer. Math., 7, 269–276. (A FORTRAN program to calculate the singular value decomposition.) 2. Chambers, J. M. (1977). Computational Methods for Data Analysis. Wiley, New York. 3. Dempster, A. P. (1969). Elements of Continuous Multivariate Analysis. Addison-Wesley, Reading, Mass.

4. Dongarra, J. J. et al. (1979). LINPACK Users’ Guide. SIAM, Philadelphia. (Contains highquality FORTRAN programs for systems of linear equations and least-squares problems. Includes a singular value decomposition program.) 5. Francis, J. G. F. (1961/1962). Computer J., 4, 265–271, 332–345. 6. Gnanadesikan, R. (1977). Methods of Statistical Data Analysis of Multivariate Observations. Wiley, New York. 7. Golub, G. H. (1968). Apl. Mat., 13, 44–51. ˇ Vyˇcisl. Mat. 8. Kublanovskaya, V. N. (1961). Z. Mat. Fiz., 1, 555–570; USSR Com. Math. Math. Phys., 3, 637–657. 9. Lawson, C. L. and Hanson, R. J. (1974). Solving Least Squares Problems. Prentice-Hall, Englewood Cliffs, N.J. 10. Parlett, B. N. (1980). The Symmetric Eigenvalue Problem. Prentice-Hall, Englewood Cliffs, N.J. (Discusses many recent innovations.) 11. Rutishauser, H. (1954). Zeit. angew. Math. Phys., 5, 233–251. 12. Rutishauser, H. (1955). C. R. Acad. Sci. Paris, 240, 34–36. 13. Smith, B. T. et al. (1976). Matrix Eigensystem Routines—EISPACK Guide, 2nd ed. Springer-Verlag, New York. (Contains FORTRAN listings which are mostly translations of the ALGOL procedures given in ref. 17.) 14. Stewart, G. W. (1973). Introduction to Matrix Computations. Academic Press, New York. (The leading textbook on numerical linear algebra.) 15. Watkins, D. S. (1982). SIAM Rev., 24, 427–440. (Tells what the QR algorithm is and why it works.) 16. Wilkinson, J. H. (1965). The Algebraic Eigenvalue Problem. Clarendon Press, Oxford. (A rich mine of information. Difficult to penetrate.) 17. Wilkinson, J. H. and Reinsch, C. (1971). Handbook for Automatic Computation, Vol. 2: Linear Algebra. Springer-Verlag, New York. (Contains ALGOL procedures for the QR and other algorithms.)

See also EIGENVALUES AND EIGENVECTORS; LINEAR ALGEBRA, COMPUTATIONAL; SINGULAR-VALUE DECOMPOSITION; and STATISTICAL SOFTWARE.

D. S. WATKINS

QR

QUADRAT SAMPLING

6655

FACTORIZATION. See

LINEAR

ALGEBRA, COMPUTATIONAL

QUADE TEST. See WILCOXON-TYPE TESTS ORDERED ALTERNATIVES BLOCKS

FOR

IN

RANDOMIZED

QUADRANT DEPENDENCE. See DEPENDENCE, CONCEPTS OF

QUADRAT SAMPLING A quadrat is a small-sample plot used by ecologists investigating populations of sessile or sedentary organisms, for example plants in a meadow, or benthic organisms on shallow sea or lake floors. A quadrat is usually (but not necessarily) square and is often (but not always) 1 meter square in size. Quadrat sampling is usually done for one of two reasons: (1) to estimate the size of a biological population in a defined area, and (2) to examine the spatial pattern of a population of organisms so that biological conclusions may be drawn. The size of a population may be defined as the number of individuals comprising it, the total biomass of all the individuals comprising it, or (in the case of vegetatively spreading plants) as the percentage cover of the plants. Whichever of these measures of population size is used, sampling with quadrats as the sampling units is a convenient estimation method. Quadrats can be used for simple random sampling∗ , stratified sampling, systematic sampling∗ , or other methods. With organisms that are difficult or laborious to count, the best procedure may be double sampling∗ with eyeball estimates of the quantities in some of the quadrats as auxiliary variable. The spatial pattern of a species population is usually studied by recording the quantity (number of individuals or biomass) of the species in each quadrat of a grid, or block, of contiguous quadrats. Various ways of interpreting such data have been devised [2,3]. They aim to answer such questions as: Are the organisms arranged in clumps? If so, what are the means and variances of (1) the

6656

QUADRATIC EQUATION, MANDEL’S GENERALIZED

numbers of individuals (or the biomass) per clump, and (2) the areas of the clumps? Do the clumps have abrupt boundaries or do they merge into each other? Is there evidence for hierarchical clumping, or for long-distance trends in population density? Is the pattern isotropic? And the like. Ripley [3] gives examples of one- and twodimensional spectral analyses applied to data from rows, and grids, of quadrats respectively. The most direct method of studying the spatial pattern of a biological population is to construct an isopleth map showing how population density varies throughout an area [1]. Quadrat sampling is used to provide the data for such maps. The quadrats (usually circular in shape) are systematically arranged with their centers at the mesh points of a square grid. Quadrat size may be so chosen that adjacent quadrats are overlapping, or tangent to each other, or separated by gaps. REFERENCES 1. Pielou, E. C. (1974). Population and Community Ecology. Gordon and Breach, New York. 2. Pielou, E. C. (1977). Mathematical Ecology. Wiley, New York. 3. Ripley, B. D. (1981). Spatial Statistics. Wiley, New York. See also ECOLOGICAL STATISTICS; LINE INTERCEPT SAMPLING; LINE INTERSECT SAMPLING; LINE TRANSECT SAMPLING; and SPATIAL DATA ANALYSIS.

E. C. PIELOU

QUADRATIC EQUATION, MANDEL’S GENERALIZED A quadratic equation proposed by Mandel [1,2] for fitting ‘‘high precision two-way data’’ is of the form 

xα − 1 y =a+b α





xα − 1 +c α

2 .

(1)

(Recall that limα→0 [(xα − 1)/α] = ln x; see BOX –COX TRANSFORMATIONS —II.) This function can represent symmetrical as well as asymmetrical curves.

Mandel recommends normalizing x and y as follows: x−x , u = 2 +  (x − x

y−y . v =  (y − y)2

The range of u is between 1 and 3, which allows raising the independent variable x to the power α. The range of α for most practical purposes is between −2 and 2. For each α in the range −1.8 to 2.0 with the step 0.2, calculate z = (uα − 1)/α

(2)

and fit the equation vˆ = a + bz + cz2 .

(3)

 Calculating (ˆv − vˆ )2 for each α, select the value of α yielding the smallest sum of squares. Denoting this value by α0 , repeat the process for a new set of 20 values of α in a smaller2 range. The value of α0 minimizing (ˆv − vˆ ) is denoted by αˆ and is chosen as an ‘‘optimal’’ α. For each value of u, calculate z for αˆ using Equation 2 and fit vˆ using Equation 3, thus obtaining estimates for a, b, and c. Equation 1 becomes yˆ = (y + aSy ) + (bSy )z + (cSy )z2 , where A = y + aSy , B = bSy , C = cSy ,  Sy = (y − y)2 .

and

REFERENCES 1. Mandel, J. (1981). Fitting curves and surfaces with monotonic and non-monotonic four parameter equations. J. Res. Nat. Bur. Stand., 86, 1–25. 2. Mandel, J. (1991). Evaluations and Control of Measurements. Marcel Dekker, New York.

QUADRATIC FORMS Let y1 , . . . , yn be n real random variables and aij , i, j = 1, . . . , n be n2 given real numbers.

QUADRATIC FORMS

Put y = (y1 , . . . , yn ) and let A be the n × n matrix whose  (i, j)th  element is (aij + aji )/2. Then Q = n1 n1 aij yi yj = y Ay is a quadratic form in y1 , . . . , yn . If the yi ’s are independently distributed, then the rank of A is defined as the degrees of freedom (d.f.) of Q. If A is positive definite or semipositive definite, Q is a definite form; it is an indefinite form if some eigenvalues∗ of A are positive and some eigenvalues of A negative. Q is a central form if E(yj ) = 0 for all j and a noncentral form if E(yj ) = 0 for some j. If the yj ’s are p × 1 column vectors yj , one has then a p-dimensional multivariate quadratic form Q = YAY , where Y = (y1 , . . . , yn ) is the p × n matrix with jth column yj . Quadratic forms arise very often in many statistical procedures (see ref. 26), most important, in regression analysis and in analysis of variance∗ and covariance models (see GENERAL LINEAR MODEL). This follows from the fact that all sums of squares can be expressed as definite quadratic forms. The sums of squares divided by their respective degrees of freedom are the mean squares. As a simple illustration, consider a one-way fixed-effects∗ model: yij = µ + αi + eij , i = 1, . . . , k, j = 1, . . . , ni , where the parameters αi , i = 1, . . . , k, satisfy   the restriction k1 ni αi = 0. Put n = k1 ni ,  

k  ni n, yi =  yij  ni , y= ni yi j=1

i=1 

y =

(y1 , . . . , yk ),

with yi = (yi1 , . . . , yini ) . Denote by Jn the n × n matrix of 1’s and by k

⊕Ai = diag(A1 , . . . , Ak )

1

the direct sum of matrices A1 , . . . , Ak . Then the sums of squares Sµ , Sα , and Se for µ, (α1 , . . . , αk ) and (eij , i = 1, . . . , k, j = 1, . . . , ni ) are given, respectively, by Sµ = ny2 = y B1 y, Sα =

k 1

ni (yi − y)2 = y B2 y,

Se =

ni k 1

6657

(yij − yi )2 = y B3 y,

1

where  k



1 B2 = In − ⊕ Jn ni i 1  k  1 ⊕ Jn B3 = In − . ni i

1 B1 = Jn , n

 ,

1

The degrees of freedoms for Sµ , Sα , and Se are given, respectively, by rank B1 = 1, rank B2 = k − 1, and rank B3 = n − k. SAMPLING DISTRIBUTIONS OF QUADRATIC FORMS FROM NORMAL POPULATIONS Fisher [15] noted that the sampling distributions of sample variances from normal populations are scaled chi-square distributions∗ . Cochran [6] showed that when sampling from normal populations, the distribution and independence of quadratic forms are related to their degrees of freedom∗ , a result referred to later as Cochran’s theorem. After this prior work, at least 150 papers have been published concerning distribution theories of quadratic forms; most of these publications have been listed in Johnson and Kotz [26], Anderson et al. [2], Scarowsky [39], Khatri [30,31], Jensen [25], Rice [37], Ruben [38], Anderson and Styan [1], Lize van der Merwe and Crowther [33], and Tan [49]. For a brief summary, let x1 , . . . , xn be a random sample from a normal population with mean µ and variance σ 2 ; further, let ‘‘∼’’ denote ‘‘distributed as’’ and χf2 (δ 2 ) a noncentral chisquare distribution∗ with f d.f. and noncentrality δ 2 . Then the following give some of the important results concerning the distributions of quadratic forms Q = x Ax and Qi = x Ai x, i = 1, . . . , k, from normal populations. Extensions to singular normal distributions can be found in Good [16], Shanbhag [40], Styan [42], Rao and Mitra [36], Tan [45], and Khatri [27,30]. Multivariate extensions can be found in Hogg [22] and Khatri [27–31]. 1. Q ∼ σ 2 χf2 (δ 2 ), where f = rank A and δ 2 = µ2 (1 n A1n )/σ 2 , iff A2 = A.

6658

QUADRATIC FORMS

2. If A2 = A, then Q ∼ σ 2 χf2 (δ 2 ) iff the universe is normal. 3. Q1 , . . . , Qk (k
2) are independently distributed of one another iff Ai Aj = 0 for all i = j.  4. Suppose that A = k1 Ai and consider the following conditions (see also IDEMPOTENT MATRICES and COCHRAN’S THEOREM):  (a) f = k1 fi , fi = rank Ai , i = 1, . . . , k. (b) Q ∼ σ 2 χf2 (δ 2 ). (c) Qi ∼ σ 2 χf2 (δi2 ), δi2 = µ2 (1 n Ai 1n )/σ 2 , i i = 1, . . . , k. (d) Q1 , . . . , Qk (k
2) are independently distributed of one another. Then (a) and (b) imply (c) and (d); and any two conditions of (b), (c), and (d) imply all the conditions. 5. Let λ1 > λ2 > · · · > λl be all the distinct eigenvalues of A with multiplic ities r1 , r2 , . . . , rl , respectively, ( l1 ri = l 2 2 n). Then Q ∼ 1 λi χri (i ), a linear combination of independently distributed noncentral chi-square variables (χr2i (2i ), where 2i = µ2 (1 n Ei 1n )/σ 2 ,   1 Ei = (In − λj A) , (λi − λj ) j=i

i = 1, . . . , l. As a simple illustration for the applications of the results above, consider again the one-way fixed-effects model as given above. Assume now that the eij ’s are distributed independently as normal with expectation 0 and variance σ 2 . Then, with α = (α1 , . . . , αk ) , y ∼ Nn (u, σ 2 In ), where

k  ⊕1ni α, u = µ1n +

a noncentral F distribution∗ with (k − 1, n − k) d.f. and noncentrality parameter δ22 .

SERIES EXPANSIONS AND COMPUTATIONS OF DISTRIBUTION FUNCTIONS Various series expansions of quadratic forms have been discussed in Johnson and Kotz [26], including the power series expansion, the chi-square and the noncentral chisquare expansion, and the Laguerre polynomial expansion. The latter two are the most commonly used expansions for computing the distribution functions of definite quadratic forms. Depending on the moments of the first term, Laguerre series∗ expansions may further be classified as one moment series, two moments series, and three moments series, leading to different approximations of the distribution functions of definite quadratic forms [56]. Alternative competitive approximations of definite quadratic forms have been the Wilson-Hilferty approximation [34] and the beta-chi-square approximation [33]. In connection with analysis of variance models, the two moments Laguerre series have been used extensively for deriving approximations to the F-ratio from normal and nonnormal populations for two basic reasons (see refs. 47,48,52, and 54): (a) the first term of the series provides a basis for the Satterthwaite approximation (see ref. 52); and (b) it appears to be more convenient mathematically and computationally than other Laguerre polynomial expansion approximations. As a simple example, consider two positive-definite quadratic forms Q1 and Q2 with EQi > 0, and put ρi = var(Qi )/EQi and bi = (EQi )/ρi , i = 1, 2. Then the two moments series leads to the following approximation for Pr {Q1  zQ2 }:

1

1n being a n × 1 column of 1’s. Now, u B3 u =  0, u B2 u = k1 ni αi2 , and B2 B3 = 0. Further, B22 = B2 and B23 = B3 . Thus Sα and Se are distributed independently of each other; Se ∼ 2 2 2 (0) = σ 2 χn−k , and Sα ∼ σ 2 χ(k−1) (δ22 ), σ 2 χ(n−k)  k where δ22 = ( 1 ni αi2 )/σ 2 . It follows that (n − k)Sα /{(k − 1)Se } ∼ F(k − 1, n −

k; δ22 ),

Pr {Q1  zQ2 }

Iw (b1 , b2 )+

k i   i i=3u=0

+

k−1 k−i i=1 j=1

u

(−1)u Ciu (b1 , b2 ; w)

dij Dij (b1 , b2 ; w),

QUADRATIC FORMS

6659

Computation by Numerical Methods

where Ciu (b1 , b2 ; w) = di0 Iw (b1 + u, b2 ) +d0i Iw (b1 , b2 + u), j    i i j Diu (b1 , b2 ; w) = s t s=0 t=0

×(−1)s+t Iw (b1 + s, b2 + t),  where k
1 is a given integer ( ki=3 is defined as 0 if k < 3), w = ρ2 z/(ρ2 z + ρ1 ), and 

w

Iw (f1 , f2 ) =

xf1 −1 (1 − x)f2 −1 dx/B(f1 , f2 ),

0

the incomplete beta∗ integral; the coefficient dij is the expected value of the product of (b ) (b ) Laguerre polynomials Li 1 (Q1 /ρ1 ) and Lj 2 (Q2 /ρ2 ) and has been expressed as simple functions of cumulants and mixed cumulants of Q1 and Q2 in Tan and Cheng [51]. Formulas for mixed cumulants and cumulants of quadratic forms and bilinear forms up to order 4 have been given in Tan and Cheng [50]; using these formulas dij can readily be computed. As shown in refs. 47, 48, 51, and 54, the approximation above would appear to work well for definite forms; however, its application to indefinite forms in general is quite difficult (for distribution theories, see Johnson and Kotz [26] and Khatri [30,31]). Press [35] and Harville [18,19] have expressed the distributions of the difference of chi-square variables in terms of special functions. Realizing the difficulty in using Laguerre expansion approximations, Bayne and Tan [5] used the Pearson system∗ to fit distributions of differences of chi-square variables. Multivariate Extension of Series Expansions Series expansions for multivariate quadratic forms have been derived and discussed in Crowther [7], Hayakawa [20,21], Khatri [28,29], and Krishnaiah and Walker [32]. These expansions involve zonal polynomials∗ . and Laguerre polynomials in matrix argument; computations for these series are quite difficult and further complicated by the slow convergence of series involving zonal polynomials.

Computations of the distribution functions of quadratic forms using numerical methods have been developed by Davis [9] and Sheil and O’Muircheartaigh [41] for positivedefinite forms and by Davis [13] and Rice [37] for general forms. Earlier works along this line for definite and indefinite quadratic forms were given in Grad and Solomon [17] and Imhof [23]. QUADRATIC FORMS FROM NONNORMAL POPULATIONS Quadratic forms from nonnormal populations have been considered by Atiqullah [3,4], Subrahmanian [43,44], Tiku [55,57,58], Ito [24], Davis [12,14], Tan [46], and Tan and Wong [53]. David and Johnson [8] have derived mixed cumulants up to order 4 of between groups sums of squares and within groups sums of squares in one-way fixed-effects models. See Tan and Cheng [50] for mixed cumulants of bilinear and quadratic forms. Davis [11] and Tan [49] have shown that if the cumulants are finite, sums of squares in analysis-of-variance models can be expanded formally in terms of products of Laguerre polynomials and gamma densities. This suggests that Laguerre polynomial expansions can be used to approximate joint distributions of quadratic forms, especially definite forms, from nonnormal populations. Using Laguerre polynomial expansions with k = 4, Tiku [55,57] derived approximations to the null and nonnull distributions of the F-ratio in one-way fixed-effects models and evaluated the effects of departure from normality of the F-test. Similar approximations with m = 4 to the null and nonnull sampling distribution of the F-ratio in oneway random effects models were derived by Tan and Wong [54]. A summary of the robustness of these tests has been given in Tan [49]. Some multivariate extensions of these approximations have been given in Davis [12,14]. REFERENCES 1. Anderson, T. W. and Styan, G. H. P. (1982). In Statistics and Probability: Essays in Honor

6660

QUADRATIC FORMS

of C. R. Rao, G. Kallianpur, P. R. Krishnaiah, and J. K. Ghosh, eds. North-Holland, Amsterdam, pp. 1–23. 2. Anderson, T. W., Das Gupta, S. and Styan, G. P. H. (1972). A Bibliography of Multivariate Statistical Analysis. Oliver & Boyd, Edinburgh. 3. Atiqullah, M. (1962). J. R. Statist. Soc. B, 24, 140–147. 4. Atiqullah, M. (1962). Biometrika, 49, 83–91. 5. Bayne, C. K. and Tan, W. Y. (1981). Commun. Statist. A, 10, 2315–2326. 6. Cochran, W. G. (1934). Proc. Camb. Philos. Soc., 30, 178–191. 7. Crowther, C. G. (1975). S. Afr. Statist. J., 9, 27–36. 8. David, F. N. and Johnson, N. L. (1951). Biometrika, 38, 43–57. 9. Davis, R. B. (1973). Biometrika, 60, 415–417. 10. Davis, A. W. (1976). Biometrika, 63, 661–670. 11. Davis, A. W. (1977). J. Amer. Statist. Ass., 72, 212–214. 12. Davis, A. W. (1980). Biometrika, 67, 419–427. 13. Davis, R. B. (1980). Appl. Statist., 29, 323–333. 14. Davis, A. W. (1982). J. Amer. Statist. Ass., 77, 896–900. 15. Fisher, R. A. (1925). Metron, 5, 90–104. 16. Good, J. J. (1969). Biometrika, 56, 215–216; correction: Biometrika, 57, 225 (1970). 17. Grad, A. and Solomon, H. (1955). Ann. Math. Statist., 26, 464–477. 18. Harville, D. A. (1971). Ann. Math. Statist., 42, 809–811. 19. Harville, D. A. (1971). ARL71-0131 No. 7071. Air Force Systems Command, U.S. Air Force, Wright-Patterson Air Force Base, Ohio. 20. Hayakawa, T. (1966). Ann. Inst. Statist. Math., 18, 191–201. 21. Hayakawa, T. (1973). Ann. Inst. Statist. Math., 25, 205–230. 22. Hogg, R. V. (1963). Ann. Math. Statist., 34, 935–939. 23. Imhof, J. P. (1961). Biometrika, 48, 419–426. 24. Ito, K. (1980). In Handbook of Statistics, Vol. 1, P. R. Krishnaiah, ed. North-Holland, Amsterdam, pp. 199–236. 25. Jensen, D. R. (1982). SIAM J. Appl. Math., 32, 297–301. 26. Johnson, N. L. and Kotz, S. (1970). Distributions in Statistics, Vol. 2: Continuous Univariate Distributions. Wiley, New York. 27. Khatri, C. G. (1968). Sankhya¯ A, 30, 267–280.

28. Khatri, C. G. (1971). J. Multivariate Anal., 1, 199–214. 29. Khatri, C. G. (1977). S. Afr. Statist. J., 11, 167–179. 30. Khatri, C. G. (1980). In Handbook of Statistics, Vol. 1, P. R. Krishnaiah, ed. NorthHolland, Amsterdam, pp. 443–469. 31. Khatri, C. G. (1982). In Statistics and Probability: Essays in Honor of C. R. Rao, G. Kallianpur, P. R. Krishnaiah, and J. K. Ghosh, eds. North-Holland, Amsterdam, pp. 411–417. 32. Krishnaiah, P. R. and Walker, V. B. (1973). Commun. Statist., 1, 371–380. 33. Merwe, L. van der and Crowther, N. A. S. (1982). S. Afr. Statist. J., 16, 133–145. 34. Mudholkar, G. S. and Trevidi, M. C. (1981). J. Amer. Statist. Ass., 76, 479–485. 35. Press, S. J. (1966). Ann. Math. Statist., 37, 480–487. 36. Rao, C. R. and Mitra, S. K. (1971). Generalized Inverse of Matrices and its Applications. Wiley, New York. 37. Rice, S. O. (1980). SIAM J. Sci. Statist. Comput., 1, 438–448. 38. Ruben, H. (1978). Sankhya¯ A, 40, 156–173. 39. Scarowsky, I. (1973). M.S. thesis, McGill University, Montreal. 40. Shanbhag, D. N. 593–595.

(1968).

Biometrika,

55,

41. Sheil, J. and O’Muircheartaigh, I. (1977). Appl. Statist., 26, 92–98. 42. Styan, G. P. H. 567–572.

(1970).

Biometrika,

57,

43. Subrahmanian, K. (1966). Sankhya A, 28, 389–406. 44. Subrahmanian, K. (1968). Sankhya A, 30, 411–432. 45. Tan, W. Y. (1977). Canad. J. Statist., 5, 241–250. 46. Tan, W. Y. (1980). S. Afr. Statist. J., 14, 47–59. 47. Tan, W. Y. (1982). Commun. Statist. Theor. Meth., 11, 731–751. 48. Tan, W. Y. (1982). J. Statist. Comp. Simul., 16, 35–55. 49. Tan, W. Y. (1982). Commun. Statist. Theor. Meth., 11, 2482–2513. 50. Tan, W. Y. and Cheng, S. S. (1981). Commun. Statist. A, 10, 283–298. 51. Tan, W. Y. and Cheng, S. S. (1984). Sankhya B, 46, 188–200.

QUALITY ASSESSMENT FOR CLINICAL TRIALS 52. Tan, W. Y. and Wong, S. P. (1977). J. Amer. Statist. Ass., 72, 875–881. 53. Tan, W. Y. and Wong, S. P. (1977). Sankhya B, 30, 245–257. 54. Tan, W. Y. and Wong, S. P. (1980). J. Amer. Statist. Ass., 75, 655–662. 55. Tiku, M. L. (1964). Biometrika, 51, 83–95. 56. Tiku, M. L. (1965). Biometrika, 52, 415–427. 57. Tiku, M. L. (1972). J. Amer. Statist. Ass., 66, 913–916. 58. Tiku, M. L. (1975). In Applied Statistics, R. P. Gupta, ed. American Elsevier, New York. See also CHI-SQUARE DISTRIBUTION; COCHRAN’S THEOREM; GENERAL LINEAR MODEL; IDEMPOTENT MATRICES; MULTIVARIATE NORMAL DISTRIBUTIONS; and WISHART DISTRIBUTION.

W. Y. TAN

QUADRATIC VARIANCE FUNCTIONS. See NATURAL EXPONENTIAL FAMILIES QUADRATURE. See n-DIMENSIONAL QUADRATURE; NUMERICAL INTEGRATION

QUADRIGAMMA FUNCTION The second derivative of the digamma∗ or psi function [(x) = d log (x)/dx]. It is customarily denoted by   (x) =

d3 log (x) . dx3

QUADRINOMIAL DISTRIBUTION A multinomial distribution∗ with four cells. It is used in the theory of 2 × 2 tables∗ .

QUADRINORMAL

DISTRIBUTION.

See MODIFIED NORMAL DISTRIBUTIONS

QUALITATIVE DATA. See CATEGORICAL DATA; NOMINAL DATA; ORDINAL DATA

QUALITY ASSESSMENT FOR CLINICAL TRIALS Commercial airline pilots have enormous responsibilities. They are accountable for

6661

expensive equipment, undergo extensive training and life-long learning, and must abide by international standards regardless of where they fly to and from. Part of this standardization includes going through a checklist of tasks prior to any flight departure. For the flight to become airborne, the checklist must be completed. A completed checklist provides the flight crew with important information regarding the status of the aircraft and results in the safety and wellbeing of the passengers. Clinical trialists share several similarities with airline pilots. Many of them are investigators (and principal investigators) of large multicenter randomized controlled trials (RCTs). It is not uncommon for these studies to cost millions of dollars to complete, and this is typically the end stage of a long development process that can easily cost in excess of $100 million. Clinical trialists would have often completed advanced training in epidemiology and very definitely have the safety and welfare of trial participants uppermost in their minds. What seems to be missing from the trialist’s arsenal is standardization of the process of conducting such studies. There has been some consensus on how RCTs should be reported [26], although only a very small percent (2.5%) of the approximately 20,000 health care journals have brought in such reporting standards. Without an agreed-upon checklist of standard operating procedures, clinical trialists are left in the precarious position of operating in a partial and loose vacuum. Although eager to do the best possible job, it becomes increasingly difficult to know whether or not the investment of monies results in a quality product, namely, a report of an RCT whereby the estimates of the intervention’s effectiveness are free of bias (i.e., systematic error). Quality is a ubiquitous term, even in the context of RCTs. Importantly, it is a construct, and not something that we can actually put our hands on. It is a gauge for how well the RCT was conducted and/or reported. In the context of this chapter, the discussions around quality are related to internal validity, and it is defined as ‘‘the confidence that the trial design, conduct, analysis, and

6662

QUALITY ASSESSMENT FOR CLINICAL TRIALS

presentation has minimized or avoided biases in its intervention comparisons’’ [24]. It is possible to conduct an RCT with excellent standards and report it badly, thereby giving the reader an impression of a lowquality study. Alternatively, an RCT may be conducted rather poorly but it may be very well reported, providing readers with the impression of ‘‘high’’ quality. Typically, the only way we can ascertain the quality of an RCT is to examine its report. It is the only tangible evidence as to how such studies are conducted. In order to be able to reasonably rely on the quality of a trial’s report, it must be a close facsimile of how the trial was conducted. Early investigations pointed in this direction [13,23]. Liberati and colleagues assessed the quality of reports of 63 published RCTs using a 100-item quality assessment instrument developed by Thomas C. Chalmers [7]. To evaluate whether the quality of reports closely resembled their conduct, the investigators interviewed 62 (of 63) corresponding authors. The average quality ratings went from 50% (reports) to 57% (conduct), suggesting a similar relationship between the quality of reports and their conduct. However, latest studies suggest that this relationship may not be as simple as previously thought [14,36]. Soares and colleagues assessed the quality of 58 published reports of terminated phase III RCTs completed by the Radiation Therapy Oncology Group, comparing them to the quality of the protocols for which they had complete access. These investigators reported that in six of the seven quality items examined there was a substantially higher reporting in the protocol compared to what was published. For example, only 9% of published reports provided information on the sample size calculation, yet this information was readily available in 44% of the Group’s protocols. This study is important for several reasons, not the least of which is that the authors had access to all the protocols from which the published reports emanated. If we are to gain a more thorough understanding of the relationship between trialreporting and conduct, it is important that journals and their editors support the publication of RCT protocols ([12], Douglas G Altman, personnel

communication). Such information provides readers with a full and transparent account of what was planned and what actually happened. Although these results are impressive and might suggest more of a chasm between reporting and conduct, caution is advised. This is a recent publication and time is required to permit replication and extension by other groups. Historians may well view the first 50 years of reporting of RCTs with some amazement. They will encounter what might be described as a cognitive dissonance: a disconnection between the increased sophistication of the design of these studies and the apparent lack of care—disastrous in some cases—with which they were reported. Unfortunately, despite much evidence that trials are not all performed to high methodological standards, many surveys have found that trial reporting is often so inadequate that readers cannot know exactly how the trials were done [31–35]. There are three approaches to assess the quality of RCT reports: components, checklists, and scales. Component assessment focuses on how well a specific item is reported. Such an approach has the advantage of being able to complete the assessment quickly and is not encumbered with issues surrounding other methods of quality assessment. Allocation concealment is perhaps the best-known example of component quality assessment [29]. Checklists combine a series of components together, hopefully items that are conceptually linked. There is no attempt to combine the items together and come up with an overall summary score. The CONSORT group developed a 22-item checklist to help investigators improve the quality of reporting their trials [26]. Scales are similar to checklists with one major difference: scales are developed to sum their individual items together and come up with an overall summary score. Many scales have been developed [19,24]. Jaded and colleagues developed a three-item scale to assess the quality of reports of RCTs in pain [18]. Assessing the quality of trial reports using scales is considered contentious by some, presently [4,21]. ¨ Using 25 different scales, Juni and colleagues assessed the quality of 17 trial

QUALITY ASSESSMENT FOR CLINICAL TRIALS

reports included in a systematic review examining the thromoprophylaxis of low molecular weight heparin (LMWH) compared to standard heparin. They reported little consistency across the scales and found that different scales yielded different assessments of trial quality. When these quality scores were incorporated into the meta-analytic analysis, it resulted in different estimates of the intervention’s effectiveness (i.e., LMWH was ‘‘apparently’’ effective using some scales and ineffective using others). Psychometric theory would predict this and practice shows it. This observation is important: if different scales are applied to the same trial, inconsistent quality assessments and estimates of an intervention’s effectiveness, often in the opposite direction, can result. Given the unfortunate process by which quality scales have been developed, this finding is only altogether unexpected. Indeed, previous research pointed in this direction [24]. In the four years since this publication, there has been no published replication or extension of this work. Until this happens, the results should be cautiously interpreted. Scales can provide holistic information that may not be forthcoming with the use of a component approach. It is ironic that the use of scales in the context of systematic reviews is considered so problematic. The science of systematic reviews is predicated, somewhat, on the axiom that the sum is better than the individual parts. Yet, summary measures of quality assessment are considered inappropriate by some. Pocock and colleagues reported that a statement about sample size was mentioned in only 5 (11.1%) of 45 reports, and that only 6 (13.3%) made use of confidence intervals [28]. These investigators also noted that the statistical analysis tended to exaggerate intervention efficacy because authors reported a higher proportion of statistically significant results in their abstracts compared to the body of the papers. A review of 122 published RCTs that evaluated the effectiveness of selective serotonin reuptake inhibitors (SSRI) as first-line management strategy for depression found that only one (0.8%) paper described randomization adequately [15]. A review of 2,000 reports of trials in schizophrenia indicated that only 1%

6663

achieved a maximum possible quality score and there was little improvement over time [37]. Such results are the rule rather than the exception. And until quite lately, such results would have had little impact on clinical trialists and others. A landmark investigation from 1995 found empirical evidence that results may be biased when trials use inferior methods, or are reported without adequately describing the methods. Notably, the failure to conceal the allocation process is associated with an exaggeration of the effectiveness of an intervention by 30% or more [30]. Schulz and colleagues assessed the quality of randomization reporting for 250 controlled trials extracted from 33 meta-analyses of topics in pregnancy and childbirth, and then analyzed the associations between those assessments and estimated intervention effects. Trials in which the allocation sequence had been inadequately or unclearly concealed yielded larger estimates of treatment effects (odds ratios exaggerated, on average, by 30% to 40%) than those in which authors reported adequate allocation concealment. These results provide strong empirical evidence that inadequate allocation concealment contributes to bias in estimating treatment effects. Several studies subsequently examined the relationship between quality and estimated intervention effects [2,20,22,27]. In an important demonstration of the contributions methodologists can make in detecting and understanding the extent by which bias can influence the results of clinical trials, Egger and colleagues have set a standard: completing a series of systematic reviews around methodological questions, such as quality [10]. These authors report that, after pooling 838 RCT reports, the effect of low quality on the estimates of an intervention’s effectiveness may be large, in the order of 30%, although there may be some differences in some methodological evaluations. The cause for concern is obvious: treatments may be introduced that are perhaps less effective than was thought, or even ineffective (Fig. 1). In the mid-1990s, two independent initiatives to improve the quality of reports of RCTs led to the publication of the ‘‘original’’

6664

QUALITY ASSESSMENT FOR CLINICAL TRIALS

Ratio of effect p from test estimates (95% Cl) of heterogeneity

Studies

Reporting bias Unpublished vs published McAuley, 200063 This study Combined

1.15 (1.04 to 1.28) 1.07 (0.98 to 1.18) 1.11 (1.03 to 1.18)

0.31

Other languages vs English Moher, 200064 This study Combined

0.98 (0.81 to 1.17) 0.84 (0.74 to 0.97) 0.88 (0.80 to 0.99)

0.19

Not MEDLINE vs indexed This study

0.94 (0.82 to 1.07)

Trial quality Inadequate/unclear vs adequate concealment of allocation Schulz, 199549 Moher, 199850 Kjaergard, 200170 This study Combined

0.66 (0.59 to 0.73) 0.63 (0.45 to 0.88) 0.60 (0.31 to 1.15) 0.79 (0.70 to 0.89) 0.71 (0.66 to 0.77)

0.13

Not double-blind vs. double-blind Schulz, 199549 Moher, 199850 Kjaergard, 200170 This study Combined

0.83 (0.71 to 0.96) 1.11 (0.76 to 1.63) 0.56 (0.33 to 0.98) 0.88 (0.75 to 1.04) 0.86 (0.77 to 0.95)

0.22

0.30

0.50 0.75 1.00 Ratio of effect estimates (95% Cl)

2.00

Figure 1. Meta-analysis of empirical studies of reporting bias and trial quality. All studies compared estimates of treatment effects within a large number of meta-analyses and calculated ratios of effect estimates for this purpose. A ratio of estimates below 1.0 indicates that trials with the characteristic (e.g., published in a language other than English) showed a more beneficial treatment effect. Adapted from Egger et al.80 by permission of Oxford University Press.

CONSORT statement that was developed by an international group of clinical trialists, statisticians, epidemiologists, and biomedical editors [3]. The CONSORT statement consists of a checklist and flow diagram. The statement was revised in 2001, fine-tuning its focus on the ‘‘simple’’ two-group parallel design [26] along with an accompanying explanation and elaboration document [1]. The intent of the latter document, known as the E & E, is to help clarify the meaning and rationale of each checklist item (Table 1, Fig. 2). Two aspects separate CONSORT from previous efforts. First, authors are asked

to report particulars about the conduct of their studies because failure to do so clearly is associated with producing biased treatment results. This approach is in keeping with the emergence of evidence-based health care. Second, CONSORT was inclusionary, whereby a wide variety of experts, including clinical trialists, methodologists, epidemiologists, statisticians, and editors, participated in the whole process. Continual review and updating of CONSORT is essential. To maintain the activities of the CONSORT Group requires considerable effort, and a mechanism has been developed to monitor the evolving literature

Table 1. Checklist of Items to Include when Reporting a Randomized Trials Paper Section and Topic

Item Number

Descriptor

Title and abstract

1

How participants were allocated to interventions (e.g., ‘‘random allocation,’’ ‘‘randomized,’’ or ‘‘randomly assigned’’).

Introduction Background

2

Scientific background and explanation of rationale.

Methods Participants

3

Eligibility criteria for participants and the settings and locations where the data were collected. Precise details of the interventions intended for each group and how and when they were actually administered. Specific objectives and hypotheses. Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors). How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules.

Interventions

4

Objectives Outcomes

5 6

Sample size

7

Randomization Sequence generation

8

Allocation concealment

9

Implementation

10

Binding (masking)

11

Statistical methods

12

Results Participant flow

13

Method used to generate the random allocation sequence, including details of any restriction (e.g., blocking, stratification). Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned. Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups. Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment. If done, how the success of blinding was evaluated. Statistical methods used to compare groups for primary outcome(s); methods for additional analyses, such as subgroup analyses and adjusted analyses. Flow of participants through each stage (a diagram is strongly recommended). Specifically, for each group report the numbers of participants randomly assigned, receiving intended treatment, completing the study protocol, and anlayzed for the primary outcome. Describe protocol deviations from study as planned, together with reasons. 6665

Reported on Page Number

6666

QUALITY ASSESSMENT FOR CLINICAL TRIALS

Table 1. (continued) Recruitment

14

Baseline data

15

Numbers analyzed

16

Outcomes and estimation

17

Ancillary analyses

18

Adverse events

19

Discusssion Interpretation

20

Generalizability

21

Overall evidence

22

Dates defining the periods of recruitment and follow-up. Baseline demographic and clinical characteristics of each group. Number of participants (denominator) in each group included in each analysis and whether the analysis was by ‘‘intention to treat.’’ State the results in absolute numbers when feasible (e.g., 10 to 20, not 50%). For each primary and secondary outcome, a summary of results for each group and the estimated effect size and its precision (e.g., 95% confidence interval). Address multiplicity by reporting any other analyses performed, including subgroup analyses and adjusted analyses, indicating those prespecified and those exploratory. All important adverse events or side effects in each intervention group. Interpretation of the results, taking into account study hypotheses, sources of potential bias or imprecision, and the dangers associated with multiplicity of analyses and outcomes. Generalizability (external validity) of the trial findings. General interpretation of the results in the context of current evidence

and help keep the CONSORT Statement evidence-based. Some of the items on the CONSORT checklist are already justified by solid evidence that they affect the validity of the trial being reported. Methodological research validating other items is reported in a diverse set of journals, books, and proceedings. In order to bring this body of evidence together, several CONSORT members have formed an ‘‘ESCORT’’ working party. They are starting to track down, appraise, and annotate reports that provide Evidence Supporting (or refuting) the CONSORT standards On Reporting Trials (ESCORT). The ESCORT group would appreciate receiving citations of reports our readers consider relevant to any items on our checklist (via the CONSORT web site). CONSORT has been supported by a growing number of medical and health care journals (e.g., Canadian Medical Association Journal, Journal of the American Medical Association, and British Medical Journal)

and editorial groups, including the International Committee of Medical Journal Editors (ICMJE, The Vancouver Group), the Council of Science Editors (CSE), and the World Association of Medical Editors (WAME). CONSORT is also published in multiple languages. It can be accessed together with other information about the CONSORT group at www.consort-statement.org. There have been some initial indications that the use of CONSORT does improve the quality of reporting of RCTs. Moher and colleagues examined 71 published RCTs, in three journals in 1994 and found that allocation concealment was not clearly reported in 61% (n = 43) of the RCTs [25]. Four years later, after these three journals required authors reporting an RCT to use CONSORT, the percentage of papers in which allocation concealment was not clearly reported had dropped to 39% (30 of 77, mean difference = −22%; 95% confidence interval of the difference: −8%, −6%). Devereaux and colleagues

QUALITY ASSESSMENT FOR CLINICAL TRIALS

6667

Assessed for eligibility (n = ...) Excluded (n = ...) Enrollment

Did not meet inclusion criteria (n = ...) Refused to participate (n = ...) Other reasons (n = ...)

Analysis

Follow-up

Allocation

Randomized (n = ...)

Allocated to intervention (n = ...)

Allocated to intervention (n = ...)

Received allocated intervention (n = ...)

Received allocated intervention (n = ...)

Did not receive allocated intervention (give reasons) (n = ...)

Did not receive allocated intervention (give reasons) (n = ...)

Lost to follow-up (n = ...) (give reasons)

Lost to follow-up (n = ...) (give reasons)

Discontinued intervention (give reasons) (n = ...)

Discontinued intervention (give reasons) (n = ...)

Analyzed (n = ...) Excluded from analysis (give reasons) (n = ...)

Analyzed (n = ...) Excluded from analysis (give reasons) (n = ...)

reported similar encouraging results in an evaluation of 105 RCT reports from 29 journals [8]. CONSORT ‘‘promoter’’ journals reported a statistically higher number of factors (6.0 of 11) compared to nonpromoter journals (5.1). Egger and colleagues examined the usefulness of the flow diagram by reviewing 187 RCT reports published during 1998 in four CONSORT ‘adopting’ journals, comparing them to 83 reports from a nonadopting journal [11]. They observed that the use of flow diagrams led to better reporting, in general. Although the simple two-group design is perhaps the most common design reported, a quick examination of any journal issue would

Figure 2. Flow diagram of the progress through the phases of a randomized trial (enrollment, intervention allocation, follow-up, and data analysis).

indicate that other designs are used and reported. While most elements of the CONSORT statement apply equally to these other designs, certain elements need to be adapted, and in some cases, additional elements need to be added, to adequately report these other designs. The CONSORT group is now developing CONSORT ‘‘extension papers’’ to fill in the gaps. A CONSORT extension for reporting randomized cluster (group) designs was published recently [6]. Other trial extension papers in development consider equivalence, non-inferiority, multiarmed parallel, factorial (a special case of multiarm), and concurrent within individual trials. The six articles will

6668

QUALITY ASSESSMENT FOR CLINICAL TRIALS

have a standard structure, mirroring features of previous publications. The CONSORT Group will introduce and explain the key methodological features of that design, consider empirical evidence about how commonly these trials have been used (and misused), and review any published evidence relating to the quality of reporting of such trials. After these brief literature reviews, the Group will provide a design-specific CONSORT checklist (and flow diagram, if applicable), and provide examples of good reporting. Our goal is to publish these extensions in different journals, in the hope of increasing their dissemination throughout the disciplines of clinical medicine. The poor quality of reporting of harm (safety, side effects) in RCTs has recently received considerable attention. Among 60 RCTs on antiretroviral treatment with at least 100 patients, only a minority of reports provided reasons and numbers per arm of withdrawals resulting from toxicity, and of participants with severe or life-threatening, clinically adverse events [16]. These observations have been validated in a substantially larger study of 192 trials covering antiretroviral therapy and 6 other medical fields [17]. Overall, the space allocated to safety in the Results section of RCTs was slightly less than that allocated to the names of the authors and their affiliations. The median space was only 0.3 pages across the seven medical fields. To help address these problems, the Group developed a paper similar in format to the other extension papers. Ten recommendations that clarify harms-related issues are each accompanied by an explanation and examples to highlight specific aspects of proper reporting. For example, fever in vaccine trials may be defined with different cut-offs, measured at various body sites, and at different times after immunization [5]. The results of such assessments are obviously problematic. The fourth recommendation asks authors to report whether the measuring instruments used to assess adverse events were standardized and validated. This document is currently under peer review (personal communication, John Ioannidis). There is an increasing need to standardize many aspects of RCT conduct and reporting. Until such time as this happens, these studies

run the ever-increasing risk of misadventure and inappropriate interpretation. For example, Devereaux and colleagues reported on ‘‘attending’’ internal medicine physicians interpretation of various aspects of blinding in the context of trials [9]. The 91 respondents (92% response rate) provided 10, 17, and 15 unique interpretations of single, double, and triple blinding, respectively. More than 41,000 RCTs are now actively recruiting participants (CenterWatch (http:// www.centerwatch.com/) Accessed 25 February 2004). As their numbers suggest, such studies form an important and central role in the development and maintenance in the delivery of evidence-based health care. If RCTs are to be conducted and reported with the highest possible standards, considerable energies must be spent on improving their conduct and reporting. Only through a wellfunded and continuing program of research, evaluation, and dissemination will standardmaking groups be able to provide up-to-date knowledge and guidance as to the importance of specific conduct and reporting recommendations. REFERENCES 1. Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., and Lang, T., The CONSORT Group. (2001). The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med., 134, 663–694. 2. Balk, E. M., Bonis, P. A., Moskowitz, H., Schmid, C. H., Ioannidis, J. P., Wang, C., and Lau, J. (2002). Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA, 287, 2973–2982. 3. Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K., Simel, D., and Stroup, D. (1996). Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA, 276, 637–639. 4. Berlin, J. A. and Rennie, D. (1999). Measuring the quality of trials: the quality of quality scales. JAMA, 282, 1083–1085. 5. Bonhoeffer, J., Kohl, K., Chen, R., et al. (2001). The Brighton Collaboration: addressing the need for standardized case definitions of adverse events following immunization (AEFI). Vaccine, 21, 298–302.

QUALITY ASSESSMENT FOR CLINICAL TRIALS 6. Campbell, M. K., Elbourne, D. R., and Altman, D. G., The CONSORT Group. The CONSORT statement: extension to cluster randomised trials. Br. Med. J., In Press. 7. Chalmers, T. C., Smith, H., Blackburn, B., Silverman, B., Schroeder, B., Reitman, D., and Ambroz, A. (1981). A method for assessing the quality of a randomized control trial. Control. Clin. Trials, 2, 31–49. 8. Devereaux, P. J., Manns, B. J., Ghali, W. A., Quan, H., and Guyatt, G. H. (2002). The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist. Control. Clin. Trials, 23, 380–388. 9. Devereaux, P. J., Manns, B. J., Ghali, W. A., Quan, H., Lacchetti, C., Montori, V. M., Bhandari, M., and Guyatt, G. H. (2001). Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA, 285, 2000–2003. 10. Egger, M., Juni, P., Bartlett, C., Holenstein, F., and Sterne, J. (2003). How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol. Assess., 7, 1–76. 11. Egger, M., Juni, P., and Bartlett, C., The CONSORT Group. (2001). Value of flow diagrams in reports of randomized controlled trials. JAMA, 285, 1996–1999. 12. Godlee F. (2001). Publishing study protocols: making them visible will improve registration, reporting and recruitment. BMC News Views, 2(4), 1. 13. Hadhazy, V., Ezzo, J., and Berman, B. (1999). How valuable is effort to contact authors to obtain missing data in systematic reviews. Presented at The VII Cochrane Colloquium. Rome, Italy, October 5–9, 1999. 14. Hill, C. L., LaValley, M. P., and Felson, D. T. (2002). Discrepancy between published report and actual conduct of randomized clinical trials. J. Clin. Epidemiol., 55, 783–786. 15. Hotopf, M., Lewis, G., and Normand, C. (1997). Putting trials on trial—the costs and consequences of small trials in depression: a systematic review of methodology. J. Epidemiol. Community Health, 51, 354–358. 16. Ioannidis, J. P. A. and Contopoulos-Ioannidis, D. G. (1998). Reporting of safety data from randomized trials. Lancet, 352, 1752–1753. 17. Ioannidis, J. P. A. and Lau, J. (2001). Completeness of safety reporting in randomized

6669

trials: an evaluation of 7 medical areas. JAMA, 285, 437–443. 18. Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J., Gavaghan, D. J., and McQuay, H. J. (1996). Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control. Clin. Trials, 17, 1–12. ¨ 19. Juni, P., Altman, D. G., and Egger, M. (2001). Systematic reviews in health care: assessing the quality of controlled clinical trials. Br. Med. J., 323, 42–46. ¨ 20. Juni, P., Tallon, D., and Egger, M. (2000). ‘‘‘Garbage in—Garbage out’? Assessment of the Quality of Controlled Trials in MetaAnalyses Published in Leading Journals’’. Proceedings of the 3rd Symposium on Systematic Reviews: Beyond the Basics, St Catherine’s College, Oxford. Centre for Statistics in Medicine, Oxford, England, pp. 19. ¨ 21. Juni, P., Witschi, A., Bloch, R., and Egger, M. (1999). The hazards of scoring the quality of clinical trials for meta-analysis. JAMA, 282, 1054–1060. 22. Kjaergard, L. L., Villumsen, J., and Gluud, C. (2001). Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann. Intern. Med., 135, 982–989. 23. Liberati, A., Himel, H. N., and Chalmers, T. C. (1986). A quality assessment of randomized control trials of primary treatment of breast cancer. J. Clin. Oncol., 4, 942–951. 24. Moher, D., Jadad, A. R., and Tugwell, P. (1996). Assessing the quality of randomized controlled trials: current issues and future directions. Int. J. Technol. Assess. Health Care, 12, 195–208. 25. Moher, D., Jones, A., and Lepage, L., The CONSORT Group. (2001). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before and after evaluation? JAMA 285, 1992–1995. 26. Moher, D., Schulz, K. F., and Altman, D. G., The CONSORT Group. (2001). The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials. Ann. Intern. Med., 134, 657–662. 27. Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., and Klassen, T. P. (1998).Does quality of reports of randomised trials affect estimates of intervention efficacy reported in metaanalyses? Lancet, 352, 609–613.

6670

QUALITY CONCEPT FOR OFFICIAL STATISTICS

28. Pocock, S. J., Hughes, M. D., and Lee, R. J. (1987). Statistical problems in the reporting of clinical trials. N. Engl. J. Med., 317, 426– 432. 29. Schulz, K. F., Chalmers, I., Grimes, D. A., and Altman, D. G. (1994). Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA, 272, 125–128. 30. Schulz, K. F., Chalmers, I., Hayes, R. J., and Altman, D. G. (1995). Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA, 273, 408– 412. 31. Schulz, K. F and Grimes, D. A. (2002a). Sample size slippages in randomised trials: exclusions and the lost and wayward. Lancet, 359, 781–785. 32. Schulz, K. F. and Grimes, D. A. (2002b). Unequal group sizes in randomised trials: guarding against guessing. Lancet, 359, 966–970. 33. Schulz, K. F. and Grimes, D. A. (2002c). Blinding in randomised trials: hiding who got what. Lancet, 359, 696–700. 34. Schulz, K. F. and Grimes, D. A. (2002d). Allocation concealment in randomised trials: defending against deciphering. Lancet, 359, 614–618. 35. Schulz, K. F. and Grimes, D. A. (2002e). Generation of allocation sequences in randomised trials: chance, not choice. Lancet, 359, 515–519. 36. Soares, H. P., Daniels, S., Kumar, A., Clarke, M., Scott, C., Swann, S., and Djulbegovic, B. (2004). Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. Br. Med. J., 328, 22–24. 37. Thornley, B. and Adams, C. E. (1998). Content and quality of 2,000 controlled trials in schizophrenia over 50 years. Br. Med. J., 317, 1181–1184.

a certain product with regard to its intended use and in comparison with similar products. Sometimes the word ‘‘quality’’ is given a positive value, and is taken as a synonym for ‘‘good quality.’’ This makes the notion somewhat difficult to handle, and different definitions have been used. Even if the definition of quality has varied over time, quality improvement, control, assurance etc. have always concerned producers of goods and services. The current dominating approach to quality issues is based on the notion of total quality, which has the following main ingredients. 1. A product’s quality is determined by both the existing and potential opinions of users of the product and its fitness for their purposes in using it. 2. The quality concept should reflect all aspects of a product that affect users’ views on how well the product meets their needs and expectations. With this definition quality has a descriptive meaning for the producer. The producer’s quality concept should not take a stand on whether the product is of good or bad quality in any absolute sense. Quality assessment is left to the users, who are entitled to have subjective opinions on whether the quality is good or bad. Their assessments do not depend on the product alone, but on a combination of product and purpose. A certain product may be judged to be of good quality in one application and bad in another. For the producer it is, of course, essential to learn about users’ opinions, since they constitute the basis for work aimed at higher quality, in the sense of greater user satisfaction. QUALITY OF OFFICIAL STATISTICS

See also CLINICAL TRIALS and META-ANALYSIS.

DAVID MOHER

QUALITY CONCEPT FOR OFFICIAL STATISTICS In everyday language quality refers to where, on a scale bad–good–excellent, a user places

In the official statistics context the core part of a ‘‘product’’ consists of statistics, i.e. estimates of statistical characteristics. Such characteristics are numeric values that summarize individual variable values for the units (households, enterprises, farms, etc.) in a specific group via some statistical measure (total, mean, median, etc.). The total collection of

QUALITY CONCEPT FOR OFFICIAL STATISTICS

units of interest is the population. In most surveys the interest involves statistics not only for the entire population, but also for different subgroups, called study domains. We speak of ‘‘estimates’’ not only when the statistics emanate from sample surveys, but also when they come from total enumeration surveys. In the latter case one should ideally achieve exact figures, but reality is seldom ideal. Surveys are subject to various kinds of disturbances. Therefore, statistical characteristics are referred to as target characteristics. Quality considerations may relate to ‘‘statistics products’’ of different scope, from a single figure in a table cell to the entire outflow from a system of statistics sources, with survey repetitions over time as a vital ingredient. The quality concept to be formulated is meant to be wide enough to cover any type of such product. Nowadays many producers of official statistics have adopted the total quality approach, in which the notion of ‘‘quality of statistics’’ takes the following form. Quality of statistics refers to all aspects of how well statistics meet users’ needs and expectations of statistical information, once disseminated. In accordance with ingredient 2, the quality concept should list all aspects of statistics implicit in this definition. When making the concept concrete, it is natural to group the aspects by main quality components with subcomponents. This structure is used in the quality concept formulated in Table 1. However, even if there is wide agreement on what the subcomponents should be, there is no universal consensus on how to group them under main components. The grouping in Table 1 blends many views, notably those of Statistics Sweden∗ and Eurostat. The quality concept is used in the following areas: Quality Declarations. To be able to use statistics adequately, users require information about their properties. For this purpose the producer should provide neutral, descriptive information, commonly called a quality declaration.

6671

Table 1. Quality Concept for Official Statistics Contents of reports Statistical target characteristics Units and population Variables Statistical measures Study domains Reference time Comprehensiveness Accuracy Overall accuracy Sources of inaccuracy Sampling Coverage Measurement Nonresponse Data processing Model assumptions Presentation of accuracy measures Timeliness Frequency Production time Punctuality Coherence, especially comparability Comparability over time Comparability over space Coherence in general Availability and clarity Forms of dissemination Presentation Documentation Access to micro data Information services

Survey Planning. For a producer, as well as for a user with influence on the planning of a statistical survey (e.g. by financing it), the quality concept gives a checklist of quality aspects to take into consideration in the planning process. Productivity Evaluation and Quality Improvement. The processes that produce statistics need evaluation and revision with regard to costs and benefits of the resource allocation. The quality concept provides a basis for such analyses. The quality declaration context highlights the descriptive side of the quality concept. In the two other contexts it is important for the producer to know about users’ assessments

6672

QUALITY CONCEPT FOR OFFICIAL STATISTICS

of quality and their preferences. The vehicle for this task is dialogue between the user and producer of statistics. General Comments 1. Producers are well aware that users pay considerable regard to the cost of a product. Cost is not included as a quality component, however, in line with the general philosophy of quality. Often quality improvements can be achieved without increased cost. However, quality and cost have to be appropriately balanced in a final round. 2. False official statistics appear sometimes, statistics that are not objective. This is of course a serious quality defect, but objectivity is not included as an aspect of the quality concept for two reasons: (1) We believe that deliberately false official statistics are exceptional. (2) It is difficult to discuss, and assess, the objectivity aspect openly. 3. Some writers use the term relevance instead of contents, while others (including the present authors) think that the term leans too much to the assessing side. It should be the user’s privilege to judge if specific statistics are relevant. 4. Some writers advocate a broader quality concept, which takes into consideration not only the users but also the data suppliers. Then response burden, confidentiality, and integrity would enter the quality picture. Most producers have good knowledge of users’ quality preferences—at least, of whether users will regard a particular production change as a step in a positive or negative quality direction. Essentially all users agree on direction, but they often disagree on the weight they assign to a specific quality change. Moreover, a production change may have a positive effect on some quality components and a negative effect on others. Hence, conflicting interests often prevail between and within users. The following elaborates on the quality components, their descriptive side, and indications of conflicting interests.

CONTENTS OF REPORTS Users’ requirements for statistical information, i.e. information on values of statistical characteristics, emanate from their subjectmatter problems. These may concern issues of economics, demography∗ , environment, and many more. The preferable choices of units, population, variables, etc. in the target characteristics depend on the subject-matter problem. Hence, relevance is not an intrinsic property of statistics, but relates to the subjectmatter problem. A specific set of target characteristics can make the statistics highly relevant for some users, but less relevant for others. Conflicting interests often turn up, and compromises have to be made. Even if there is consensus about the most suitable target characteristics, considerations concerning cost, timeliness, measurement difficulties, etc. may lead to ‘‘second best’’ choices. The descriptive aspect of study domains concerns answers to the following questions: Which types of classifications are used to form study domains? How far-reaching are the subdivisions into study domains? Users commonly present very extensive requirements as regards statistics for small domains, but there are restraining factors. One is that data on a requested classification variable may not be available (e.g. for cost reasons); another, that statistics for many domains require overly extensive publication space. Additionally, when sample survey statistics are broken down to smaller and smaller domains, their accuracy deteriorates. Ultimately it becomes so low that the statistics no longer are meaningful, and the breakdown process has to be terminated at an appropriate level. Units and variable values relate to specific times, which may be narrowly delimited and are called reference time points (e.g. a specific day), or reference periods (e.g. a calendar year). Usually reference times agree for all variables and units, but they may differ. (Example: A survey target could concern salaries in 1985 and in 1995 for students

QUALITY CONCEPT FOR OFFICIAL STATISTICS

who graduated from a particular educational institution in 1975.) Comprehensiveness refers to a system of statistics for a specific subject matter field (example: the totality of economic statistics from the national statistics system). Many users want the statistics system to provide information on ‘‘all vital respects.’’ The better this request is met, the more comprehensive is the statistics system. In practice no national statistics system can satisfy all users according to their interpretation of ‘‘all vital respects.’’ ACCURACY Accuracy concerns the agreement between statistics and target characteristics. In a sample survey, the resulting statistics do not provide exact values of the target characteristics. Moreover total enumeration surveys are usually subject to so many disturbances that the resulting statistics should be regarded as estimates rather than exact values. Normally there is a discrepancy between the values of a statistic and its target characteristic, also referred to as an error. The (relatively) smaller the discrepancy is, the more accurate is the statistic. Discrepancies should be small, preferably negligible. Often, however, discrepancies are not negligible, in particular for sample survey statistics. Then statistically knowledgeable users want numerical bounds for the discrepancies, called accuracy measures or uncertainty measures. Exhibition of accuracy measures is somewhat intricate, since the discrepancies are defined in terms of target values that are unknown. (If they were known, it would be unnecessary to estimate them.) Statements concerning accuracy are therefore inevitably statements about states of uncertainty, a conceptually difficult topic. The usual structure for information about accuracy is as follows (in a nontechnical formulation): It is most likely that a specified interval of the type accuracy (or uncertainty) interval = value for the statistic ± margin of uncertainty (error)

6673

comprises the true value of the target characteristic. Sometimes such an interval can be interpreted as a confidence interval∗ with a specified confidence level, which in official statistics is often chosen to be 95%. Other accuracy measures are in essence equivalent to a confidence interval: the estimator’s standard deviation∗ , relative margin of error, and coefficient of variation∗ . Overall Accuracy Here interest is focused on the overall reliability of a statistic, in other words on the magnitude of the total error. In some cases the producer can provide precise overall accuracy intervals, but this is the exception rather than the rule. However, lacking precise bounds for total errors, the producer should do his/her best to provide information on, or at least judgments of, how certain source(s) of inaccuracy have affected the statistics. This is considered under the next quality component. Sources of Inaccuracy Classifications of error source usually employ the duality of sampling errors versus general survey errors (often called nonsampling errors). The former relate to sample surveys, and emanate from the fact that only a sample of population units, not all, are observed. The latter relate to the error sources to which all types of surveys are subject, total enumeration surveys as well as sample surveys. Another common classification duality is that of systematic errors, which lead to bias in the statistics, versus random errors. The former relate to errors which (for the majority of observations) go in the same direction, the latter to errors which spread randomly around 0. In this context the accuracy is commonly divided into the components bias (size of the systematic error) and precision (bound for the random error). The total error (i.e. the discrepancy between a statistic and its target value) is often viewed as a sum of partial errors, emanating from different error sources: Total error = sampling error+coverage error +measurement error +nonresponse error + · · · .

6674

QUALITY CONCEPT FOR OFFICIAL STATISTICS

Even if it is difficult to give quantitative bounds for the total error, it is often possible to provide accuracy information for at least some of the partial errors. In quality declarations the producer should, in addition to potential numerical error bounds, provide a verbal account of the data collection∗ , including obstacles encountered. We now turn to the main sources of inaccuracy. Sampling. The fact that only a sample of population units are observed in a sample survey contributes to the inaccuracy of the resulting statistics. One distinction is that of probability samples (yielding control of sample inclusion probabilities) versus nonprobability samples (‘‘expert samples’’ and ‘‘subjective samples’’ are synonyms). Probability sampling is a safeguard against bias; moreover, bounds for the sampling error can usually be given in terms of confidence intervals. General Survey Error Sources. Disagreement between survey frame and target population leads to coverage error. A measurement error∗ occurs if a respondent’s answer differs from the true variable value. Measurement errors may be systematic (e.g. underreporting of income) or random. Systematic measurement errors lead to biased statistics. The contribution to inaccuracy from random measurement errors is mostly covered by the sampling error confidence interval. Nonresponse occurs when values for a designated observation unit have not been collected at the time when the estimation process starts. Nonresponses may lead to bias if there is correlation between nonresponse and the value of the survey variable. Various procedures exist for adjustment, in the best possible manner, for nonresponse. Nonresponse rates are commonly reported. They indicate the quality of the data collection process, but do not give information about the crucial quantity, the order of magnitude of the nonresponse error. Collected data are processed in different steps, such as data entry, coding, editing, and estimation/aggregation. At each step of data processing mistakes/mishaps may occur, contributing to inaccuracy.

Some statistics rely on assumptions (e.g. stability of a consumption pattern), also referred to as models. A model assumption that is not perfectly fulfilled contributes to inaccuracy. Adjustment procedures (for nonresponse, coverage deficiencies, seasonal variations, etc.) also rely on assumptions/models. In such cases, the inaccuracy due to using models should be reported under the specific quality aspect. Presentation of Accuracy Measures Statistics with accuracy deficiencies may lead to fallacious conclusions if used uncritically. Knowledgeable users can avoid fallacies if appropriate accuracy measures are presented. Statistics with accompanying accuracy measures are more informative than ‘‘bare’’ statistics. TIMELINESS Many users want statistics from repeated surveys in order to monitor some specific development, prepared to take appropriate action if alarming levels are reached. In such situations a main requirement is that available statistics should be up to date. A vital aspect here is the time lag between now and the reference time for the last available statistics. This lag depends on how frequently the survey is repeated and its processing time. A user’s quality judgment in this respect does not, however, solely depend on the maximal time lag; his/her opinion of the pace of change for the development under consideration is also crucial. Statistics from repeated surveys are usually produced according to a regular scheme (monthly, quarterly, annually, etc.). In such situations it is natural to talk of frequency (or periodicity), including data collection frequency (the periodicity in the producer’s data collection), reference-time frequency (the periodicity of reference times for published statistics), and dissemination frequency (the periodicity with which statistics are made public). Normally the three frequencies agree, but they can differ. (Example: Swedish crime statistics are published quarterly, comprising statistics for each month in

QUALITY CONCEPT FOR OFFICIAL STATISTICS

the quarter.) Users normally are most interested in reference-time and dissemination frequencies. Production time is the lag between the reference time point (or end of the reference period) and the time for publication of a statistic. Normally, the shorter the production time the better. However, if the statistics carry unpleasant messages, some users/actors may wish delayed publication. The common policy is that official statistics that do not have, or are late relative to, a promised publication data should be published as soon as they are ready. Accuracy and production time may come into conflict with each other. Shortening of a production time often leads to increased nonresponse as well as more hasty editing, which in turn affect accuracy adversely. Punctuality refers to the agreement between promised and actual dissemination time. Interest in punctuality varies considerably among users. An extreme example: For economic statistics that affect stock-market prices, punctuality may involve fractions of a second. COHERENCE AND COMPARABILITY Coherence relates to sets of statistics, and takes into account how well the statistics can be used together. Two subaspects are of special importance. When the statistics set is a time series∗ , one speaks of comparability over time. When it comprises statistics for different domains with similar target characteristics, one speaks of comparability over space. In comparison contexts one ideally wants to compare true values of the same characteristic. This ideal situation may not be achievable. As a second best, one wants to compare statistics with similar target characteristics and good accuracy. When judging the similarity of target characteristics, their definitions (regarding units, population and domain delineation, variables, etc.) play a central role. The more stable a definition has been over time, the better comparability is over time. Analogously, for good comparisons over space, similarity in definitions of target characteristics is

6675

crucial. Statistical standard classifications (e.g., those of the Nomenclature G´en´erale des Activit´es Economiques dans les Communaut´es Europ´eenes for the classification of industries) are vital to achieve agreement, or at least good similarity, between target characteristics. The acuteness of comparisons also depends on the accuracy of the pertinent statistics, their bias and precision. If the statistics compared are severely inaccurate, observed differences may reflect ‘‘noise play’’ rather than true differences. Biases distrub comparisons, but the harm can be mitigated if the bias structures are similar. An important means for achieving good comparability is that statistics should be produced with a common, hopefully good, methodology, as regards questionnaire, data collection, and estimation; this will minimize bias and lead to similar bias structures. Common methodology is also important because the content/definition of a variable often depends upon the measurement and data collection procedures. Comparability over Time Surveys that are repeated over time yield statistical time series, which enable users to follow developments over time. Here one is concerned with the extent to which the statistics in a time series in fact estimate the ‘‘same thing’’ in the ‘‘same way.’’ Stability over time of target characteristic definition and survey methodology work in the direction of good comparability over time. Regarding stability of definitions, user interests may conflict. Users whose main interest is the present and future state of affairs want reality changes (e.g. changes in industry structure) to be met by appropriate changes in the statistics. But modifications of target characteristics to meet reality changes usually have adverse effects on comparability over time. Certain users, notably those of statistics indicating short-term changes in economic activity, are anxious to be able to separate changes ‘‘in substance’’ from effects due to fairly regular seasonal variations. These users require seasonal adjustments and calendar adjustments as complements to the basic time series.

6676

QUALITY CONCEPT FOR OFFICIAL STATISTICS

Comparability over Space A common usage of statistics is for the comparison of conditions in different geographical regions (e.g. average wages in different countries). The ‘‘space dimension’’ may also be of a nongeographical nature (example: comparison of average disposable incomes for families with 1, 2, 3, . . . children). Again, similarity of definitions of target characteristics and of survey methodology are crucial aspects. When the statistics (for different domains) emanate from the same survey (by the same producer), problems regarding comparability over space are usually reduced to questions about the precision of the statistics. However, the farther apart the producers are (different surveys at the same agency, different agencies in the same country, offices in different countries, etc.), the greater are the comparability problems. Coherence in General Coherence relates to the feasibility of making joint use of statistics from different sources, not only for comparison purposes. (Example: In order to judge the consequences of a potential change in taxation and benefits rules, it might be of interest to combine statistics from an Income survey, an Expenditure survey, and a Rent survey. Then it is important that the stastistics be coherent, for instance that the same definition of ‘‘household’’ be used in the different surveys.) There should be agreement in definitions of basic target characteristic quantities (units, population, domains, variables, and reference times).

Documentation refers to users’ ability to acquire documentation relating to published statistics. Most users want an easily readable quality declaration. More advanced users are often interested in precise documentation of the production process, which is particularly important when the user has access to micro data for personal use. For that purpose, users may be interested in statistics that are not provided by the producer, but which can be derived from already collected micro data. There are two main options in this context. The producer makes special derivations from available data, in accordance with requests formulated by the user. The user obtains access to micro data for his/her own ‘‘statistics production.’’ Users with well-specified problems tend to prefer the first alternative. Important points are then how fast the derivations can be carried out, and at what cost. Researchers are commonly interested in obtaining a micro data for their own processing. Thereby they can make analyses more flexibly, faster, and cheaper than via special derivations by the producer. Release of micro data is, however, associated with problems of secrecy, and special precautions have to be taken by the producer. Removal of the means of identification is a minimum requirement. One main aspect of information services is what assistance a user can get to find his/her way in the ‘‘statistics storage.’’ Another is the possibility of getting answers to questions about published statistics: their interpretation, specifics of definitions, details about data collection, etc.

AVAILABILITY AND CLARITY Forms of dissemination refer to what dissemination media (print on paper, diskette, CD-ROM, etc.) and what distribution channels are used. Presentation refers to how statistics are presented in printed publications, databases, etc. Specifically it concerns the presence, layout, and clarity of texts, tables, charts, and other figures; referencing; etc. It also covers how well particularly interesting features of the statistics are emphasized.

SELECTED REFERENCES ON QUALITY WORK AT SOME STATISTICAL AGENCIES Official statistics has a long tradition. It has developed considerably during this century due to new demands (e.g. as regards subjectmatter areas), new methodology (e.g. survey sampling), new technology (e.g. for data collection and processing), etc. The numbers of uses and users have increased greatly. To give a comprehensive review of the notion of quality of official statistics over time and

QUALITY CONCEPT FOR OFFICIAL STATISTICS

space is too big a task to be covered here. We restrict ourselves to some recent milestones and a brief review of current views and activities. Milestones First, much survey development work has its origin in statistical agencies of the U.S. Federal Government, notably the Bureau of the Census∗ . The U.S. role in this development is described by Bailar [1] and by Fienberg and Tanur [7]. Second, works on quality issues by Statistics Canada are often cited by other agencies. An important example is Quality Guidelines [11], which is a manual ‘‘providing advice for the production, maintenance and promotion of quality for statistical processes.’’ Related works [12,13] focus on how to inform users. Third, and not least, instrumental work has been carried out by international statistical organizations. The task of informing users was discussed in the 1980s by U.N. statisticians [16], who were influenced by work by Statistics Canada, Statistics Sweden, and U.S. Federal statistical agencies. The latter work is presented in Gonzales et al. [8]. The UN guidelines emphasize two main types of quality presentations: (1) extensive presentations with technical orientation, written for professional statisticians, and (2) presentations for statistics users in general, to assist them in interpretation of the statistics and in deciding whether, and how, to use them. Some Current Views and Activities Only a few papers discuss the quality concept in such structural detail as here; Statistics Sweden and Eurostat are two exceptions. However, quality concepts emerge implicitly from papers on quality endeavors. We try to emphasize these aspects in the review below. Statistics Sweden∗ [15] presents a definition of quality and recommendations for statements on quality. This document updates 1979 guidelines for presentations on the quality of statistics and a 1983 policy for a user-oriented presentation. Eurostat has an internal quality policy document, drafted in 1996. Moreover, there

6677

are documents on the quality of business statistics, tied to regulations on business statistics [5]. Harmonization and coordination of statistical systems are important in work that is aimed at good comparability and coherence of international statistics. These quality components are emphasized in the UN guidelines and in the Eurostat quality concept for business statistics. The U.N. System of National Accounts is an important example of a world-wide harmonized system, which also influences other branches of economic statistics. Beekman and Struijs [2] discuss economic concepts and the quality of the statistical output. Statistisches Bundesamt [14] provides a compendium of discussions on the quality of statistics from a user’s point of view, for political decision makers, scientists, in econometric uses, etc. Quality components that recur in several discussions are timeliness, accuracy, and comparability. Dippo [4] considers survey measurement and process improvement. The paper links early work on nonsampling errors and different components of the overall error with recent work on process improvement. It includes the quality measurement model—which has the user at the center—of the U.S. Bureau of Labor Statistics∗ . McLennan [10] describes the history of British official statistics and developments in the 1990s, and lists some operational principles for the U.K. Central Statistical Office (CSO) under three headings: ‘‘definitions and methodology,’’ ‘‘integrity and validity of CSO output,’’ and ‘‘timing and coverage of publications.’’ Linacre [9], when describing the methodology followed in a statistical agency, refers to the objectives of the Australian Bureau of Statistics as ‘‘informed and satisfied clients through an objective, relevant, and responsive statistical system.’’ A statistical product should comprise ‘‘reliable, timely, and coherent statistics.’’ Characteristics of an effective statistical system are discussed by Fellegi [6], who states that the ‘‘objective of national statistical systems is to provide relevant, comprehensive, accurate, and objective (politically untainted) statistical information.’’

6678

QUALITY CONTROL, STATISTICAL

Colledge and March [3] report on a study, comprising 16 national statistical agencies around the world, on the existence of ‘‘quality practices’’ (classified as policies, standards, guidelines, and recommended practices) as well as the degree of compliance with prescribed practices. REFERENCES 1. Bailar, B. A. (1990). Contributions to statistical methodology from the U.S. federal government. Survey Methodol., 16, 51–61. 2. Beekman, M. M. and Struijs, P. (1993). The quality of economic concepts and definitions. Statist. J. United Nations Economic Commission for Europe, 10, 1–15. 3. Colledge, M. and March, M. (1997). Quality policies, standards, guidelines, and recommended practices at natinal statistical agencies. In Survey Measurement and Process Quality, L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwartz, and D. Trewin, eds. Wiley, New York. 4. Dippo, C. S. (1997). Survey measurement and process improvement: concepts and integration. In Survey Measurement and Process Quality, L. Lyberg, P. Biemer, M. Collins, E. de Leeuw, C. Dippo, N. Schwartz, and D. Trewin, eds. Wiley, New York. 5. Eurostat (1996). Quality in Business Statistics. (Eurostat/D3/Quality/96/02-final for structural business statistics.) 6. Fellegi, I. P. (1996). Characteristics of an effective statistical system. Int. Statist. Rev., 64, 165–197. 7. Fienberg, S. E. and Tanur, J. M. (1990). A historical perspective on the institutional bases for survey research in the United States. Survey Methodol., 16, 31–50. 8. Gonzales, M. E., Ogus, J. L., Shapiro, G., and Tepping, B. J. (1975). Standards for discussion and presentation of errors in survey and census data. J. Amer. Statist. Ass., 70, 5–23. 9. Linacre, S. (1995). Planning the methodology work program in a statistical agency. J. Official Statist., 11, 41–53. 10. McLennan, B. (1995). You can count on us—with confidence. J. R. Statist. Soc. A, 158, 467–489. 11. Statistics Canada (1987). Quality Guidelines, 2nd ed. Statistics Canada, Ottawa. 12. Statistics Canada (1987). Statistics Canada’s policy on informing users of data quality and methodology. J. Official Statist., 3, 83–91.

13. Statistics Canada (1992). Policy on Informing Users of Data Quality and Methodology. Policy Manual. Statistics Canada, Ottawa. 14. Statistisches Bundesamt (1993). Qualitat ¨ statistischer Daten (Quality of Statistics; in ¨ German.) Beitrage zum wissenschaftlichen Kolloquium am 12./13. November 1992 in Wiesbaden. Schriftenreihe Forum der Bundesstatistik herausgegeben vom Statistischen Bundesamt 25. 15. Statistics Sweden (1994). Kvalitetsbegrepp och riktlinjer f¨or kvalitetsdeklaration av officiell statistik (Quality Definition and Recommendations for Quality Declarations of Official Statistics; in Swedish), Meddelanden: samord˚ ningsfragor 1994:3. Statistics Sweden, Stockholm. 16. United Nations (1983). Guidelines for Quality Presentations That are Prepared for Users of Statistics. Statistical Commission and Economic Commission for Europe, Conference of European Statisticians, Meeting on Statistical Methodology, 21–24 November 1983. See also TOTAL QUALITY MANAGEMENT.

EVA ELVERS ´ BENGT ROSEN

QUALITY CONTROL, STATISTICAL Statistical quality control refers to the statistical procedures used to control the quality of the output of some production process, including the output of services. Quality of output is characterized in various ways, frequently by the percentage or fraction of items that do not individually conform to product specifications. (Actually, comparison may be made with ‘‘test limits’’ that allow for measurement errors∗ and lie outside the specification limits; see ref. 22.) For some products (e.g., sheets of material) quality is characterized by the number of nonconformities per ‘‘unit area of opportunity,’’ say, per square foot, or per 100 unit areas. Sometimes the quality of a lot or process is characterized by its mean quality or occasionally by a percentile. In early writings and in some current ones, the terms defect and defective are used in the general sense of nonconforming. In the more restricted sense now approved, these terms relate to a product that does ‘‘not satisfy

QUALITY CONTROL, STATISTICAL

intended normal, or reasonably foreseeable usage requirements’’ [2]. Statistical quality control is principally concerned with control charts∗ and sampling inspection plans (see SAMPLING PLANS). Control charts are used in deciding whether variations in the quality of the output of a production process can be viewed as coming from a statistically stable population. They are used to study past operations and to monitor current operations. If a chart suggests the existence of special causes of variation, it is expected that the process can be adjusted to attain a state of statistical control at an acceptable level. Sampling inspection plans provide an economic inspection procedure with known risks of passing non-conforming output. Control charts and sampling inspection plans will be discussed in turn. A concluding section presents some pertinent history and indicates current trends. The statistical design of experiments∗ may be used in the study of control procedures and in the establishment of an inspection system as well as in the design of the product itself, but this aspect will not be discussed. CONTROL CHARTS The principles and considerations involved in process quality control were presented in a memorandum prepared in May 1924 by Walter E. Shewhart of the Bell Telephone Laboratories. His thoughts were later developed in several articles in the Bell System Technical Journal (see particularly refs. 54 and 55) and subsequently published in 1931 in his book The Economic Control of Quality of Manufactured Product [56]. Lectures given at the Graduate School of the U.S. Department of Agriculture, Statistical Method from the Viewpoint of Quality Control, were published in 1939 [57]. The charts he developed for the study of a process are today known as Shewhart control charts. Shewhart Control Charts Such a chart consists of a plot of some sample statistic—a mean∗ , range∗ , or sample proportion—for segments of output known as rational subgroups. Time is a rational basis

6679

Figure 1. Shewhart R-chart (redrawn from Johnson and Leone [31]).

for subgrouping of output, so rational subgroups might be small portions of output occurring at intervals of, say, 2 hours, a day, or a week; they might also consist of portions of the output of different operators or different machines. Figure 1 shows a Shewhart range or R-chart. The assumption underlying control chart analysis is that variation in the quality of output is due in part to chance causes described in ASQC Standard A1 [2] as ‘‘generally numerous and individually of relatively small importance . . . which are not feasible to detect or identify’’ and in part to special or ‘‘assignable’’ causes that are feasible to detect and identify [2]. If in the analysis of a given segment of output, subgroups are appropriately chosen, the variation within subgroups will be representative of that due to chance causes, while variation due to assignable causes will tend to show up in differences between subgroups. Every Shewhart control chart has an upper or lower control limit or both; these represent the limits of variation in the plotted statistic that may reasonably be expected if chance forces are alone at work. If a point exceeds these limits, it is taken as a signal of the presence of an assignable cause. The R-chart of Fig. 1 has only an upper limit, but there may be cases in which it is desirable for an R-chart to have a lower limit as well. Determination of a State of Statistical Control If output for a past period consists of units of product, the quality characteristics of which have been individually measured, and if samples of equal size are available for a set of rational subgroups, these data may be effectively studied to determine the existence of a state of statistical control by plotting sample

6680

QUALITY CONTROL, STATISTICAL

averages on an X-chart and sample ranges on an R-chart. (With modern computer facilities available, the more efficient sample standard deviation chart or s-chart may be preferred to the R-chart. Our discussion will be in terms of the R-chart.) If there are no points outside estimated 3σR control limits on the R-chart, and no long runs above or below the central line or other evidence of nonrandom variation (see Fig. 1), the average range (R) can be used to set up estimated 3σX control limits on the √ X-chart falling at X ± 3R/(d2 n). See ref. 17 (Chap. 21). (The variation between the sample averages on the X-chart cannot be used to determine the control limits for this chart since that variability may be contaminated by the presence of assignable causes that it is the function of the X-chart to detect.) If the points on the X-chart also all fall within the control limits and there are no long runs above or below the central line on the chart or other evidence of nonrandom variation, the analysis will suggest that assignable causes have not been active in the period studied. To make a positive statement that control exists, the data studied should be reasonably extensive. Shewart stated [57, p. 46] that it has ‘‘been observed that a person would seldom if ever be justified in concluding that a state of statistical control of a given repetitive operation or production process has been reached until he has obtained, under presumably the same essential conditions, a sequence of not less than twenty-five samples of four that satisfied Criterion I (i.e., all points fall within 3σ control limits).’’ This criterion continues to be commonly accepted. For a theoretical study of the operating characteristics of X-charts used to study past data, see ref. 34. The use of a pair of X- and R-charts to determine a state of statistical control is similar to running a one-way analysis of variance∗ . The control charts have the advantage, however, of retaining the order of the subgroups and offering a graphical presentation. With the availability of modern computer facilities, both analyses could be incorporated in a single computer program. For attributes data, lack of control in past output may be detected by a numberof-nonconforming-units chart (an np-chart), a sample-proportion-nonconforming chart (a

p-chart∗ ), or a chart for the sample number of nonconformities (a c-chart∗ ). See refs. 3 and 17. An np-chart for k subgroups is similar to the analysis of a 2 × k contingency table∗ , showing the numbers of conforming and nonconforming items for each subgroup. See ref. 17 (Chap. 28, Sec. 3). Analysis of Data from a Controlled Process When a process is in a state of statistical control, certain meaningful statistical analyses can be carried out. Without control, these analyses may have little meaning and may even be misleading. For example, with a controlled process, a histogram∗ of the variation in product quality will show process capabilities with respect to given specifications and may indicate the gain from possible recentering of the process. Also, the hypothesis of normal variability in output quality can be tested. In addition, the attainment of a state of statistical control in the manufacture of piece parts will permit random assembly and the computation of statistical limits for the assembled unit. Attaining and Maintaining a State of Statistical Control If study of past production data reveals that a process is in a state of statistical control, the central lines and control limits of the charts that were used may be extended for use in monitoring current production. If study of past data does not show control, investigation of the process may reveal assignable causes that can be eliminated, and control charts for current use can be set up with tentative central lines and control limits that can be revised as experience with the process develops. Sometimes standard values for the central lines and control limits are used. After a process has been currently brought into a state of statistical control, the effectiveness of a control chart in detecting departures from the state of control becomes a major interest. This has led to proposals for modifying the usual Shewhart control chart when employed for purposes of current control by using other than 3σ control limits and sample sizes other than 4 or 5 (see e.g., ref. 15). Interest in effectiveness has also led to the introduction of ‘‘warning limits’’ on

QUALITY CONTROL, STATISTICAL

a Shewhart chart, say at ±2σ (see refs. 46 and 68) and to development of geometric moving-average∗ control charts (see ref. 50), cumulative sum∗ (CUSUM) control charts (see refs. 45 and 48), combinations of Shewhart and CUSUM charts (see ref. 38), and moving sum∗ (MOSUM) and moving sum of squares (MOSUM-SQ) controls (see refs. 6 and 7). The use of sums and averages can be very effective. A CUSUM chart, for example, will detect a sudden small but persistent change in a process mean more quickly [i.e., the average run length (ARL) following the change will be shorter] than in the case of an ordinary Shewhart chart and will facilitate locating the time of occurrence of the change. If deviations from a designated central value are cumulated, a V-mask can be used not only to detect changes but also to estimate the new level of operations. Furthermore, cusum charts readily lend themselves to the use of computers. [For textbook discussions, see refs. 17 (Chap. 22) and 31 (Sec. 10.4).] Geometric moving averages weight past observations in accordance with their distance in time from the most recent observation and MOSUMs limit the number of past observations included in a sum, making both procedures more sensitive to a current shift in the process mean. Broader studies that pertain to the overall cost-effectiveness of control charts used for current control have led to consideration of prior probabilities∗ of occurrence of assignable causes and to the inclusion of the frequency of sampling as a design parameter. (See refs. 15, 11, and 18 and the 15 other references cited in ref. 18.) Adaptive control charts have been developed that anticipate changes in the process and permit early adjustments (see ref. 9). For a list of a variety of special control charts, see ref. 2. Acceptance control charts are described below, following the discussion of sampling inspection. For a review of recent developments in control charts used to monitor a process, see ref. 21, and for a review and literature survey of the economic design of control charts, see ref. 42. Two-sided cusum charts that sum deviations from a target or reference value and use a V-mask based on a known value

6681

of the process standard deviation, derived, say, from a previously constructed R-chart or s-chart, have been likened to sequential sampling∗ plans in reverse [30]. The discussion of MOSUMs in ref. 6 runs entirely in terms of sequential testing of the hypothesis of a constant process mean. Other Uses of Control Charts Shewhart charts can be used for purposes other than control of product quality. In general, when data arise out of repetitious operations, say, for example, in making a series of measurements, a Shewhart chart can be used in deciding whether statistically stable conditions exist. See, for example, ERROR ANALYSIS. SAMPLING INSPECTION PLANS In the years when Shewhart was developing his ideas on control charts, Harold F. Dodge and Harry G. Romig were working at the Bell Telephone Laboratories on sampling inspection plans. Their historic paper on ‘‘A method of sampling inspection’’ appeared in the Bell System Technical Journal of October 1929. Work continued in subsequent years, culminating in their Sampling Inspection Tables— Single and Double Sampling [14], published in book form in 1944. The discussion that follows seeks to develop an understanding of the principles of sampling inspection as applied to the acceptance and rejection of lots. For a discussion of sampling inspection applied to the output of a process not normally involving the formation of lots, see ref. 17 (Chap. 17). The initial discussion here pertains to attributes inspection, in which interest centers on the percent of the items in a lot or process that do not conform to product specifications. With minor modifications it can be related to sampling inspection, in which interest centers on the number of nonconformities per unit area of opportunity. A brief section is presented on life testing∗ and a concluding section discusses the sampling of bulk∗ material. In the two latter cases interest centers on the mean quality of a lot or process. Reference 62 is an example of the case in which interest centers on a specified percentile. See also ACCEPTANCE SAMPLING; INSPECTION SAMPLING; SAMPLING PLANS.

6682

QUALITY CONTROL, STATISTICAL

Single-Sampling Plans and Schemes for Attributes Inspection of a Series of Lots A single-sampling plan for attributes inspection calls for taking a random sample of specified size n from each lot submitted for inspection, testing the sample items in accordance with a prescribed test method, and noting the number of items that do not conform to product specifications. If for a given lot the number of nonconforming items in a sample is less than or equal to a specified acceptance number c, the lot is accepted; otherwise, it is rejected. Except for the rectifying inspection plans discussed below, the disposition of a rejected lot is not part of the sampling inspection plan. OC Curve. The operating characteristic or OC curve of a sampling inspection plan gives the probability of lot acceptance as a function of submitted product quality. As explained in ACCEPTANCE SAMPLING, two types of OC curves are distinguished. A type A OC curve gives the probability of accepting a particular lot as a function of the quality of that lot. A type B OC curve relates to the process producing the product being inspected and gives the proportion of lots that can be expected to be accepted in a continuing series as a function of the process average. Figure 2 depicts a type B OC curve. OC Features of Special Interest: The AQL and Lq. A feature of the OC curve of a sampling inspection plan that is of special interest is the acceptable quality level∗ (AQL). For attributes inspection the AQL is defined as the maximum percent nonconforming or

Figure 2. Type B operating characteristic (OC) curve for a sampling inspection plan (redrawn from Johnson and Leone [31]).

maximum number of nonconformities per unit that for purposes of acceptance sampling can be considered acceptable as a process average (see ref. 2). The naming of an AQL carries with it the implication that the sampling plan will have a high probability of lot acceptance, say 0.95 or higher, for submitted product of AQL quality or better. A given sampling plan will be acceptable to a producer if he or she believes the process capabilities are such that the process average will be at least as good as the AQL. If a producer agrees to a plan with a stated AQL, the consumer will expect that most of the product that he or she will receive will be of AQL quality or better. In addition to the AQL, a feature of a sampling plan that the consumer will be particularly interested in is the probability of accepting an especially poor lot that might be occasionally turned out by the producer. He may thus wish to name a limiting quality (LQ) and seek a plan for which the probability of lot acceptance is low, say less than 0.10, for lots of LQ quality or worse. For relatively large lots an LQ with probability of acceptance β will be approximately the same as a process level with a β probability of lot acceptance. This is the rejectable quality level (RQL; see Fig. 2). For a given lot size a sampling inspection plan could be tailor-made to accord with the producer’s and consumer’s interests regarding the AQL and LQ [see ref. 17 (Chap. 7, Sec. 2.3)]. Some modification may be necessary if an unsatisfactorily large sample size is needed to meet the desired goals. An alternative procedure would be to abstract a mutually satisfactory individual sampling plan from some table of sampling schemes (see below) or other systematized table. It will be noted that ‘‘producer’’ and ‘‘consumer’’ could be different departments of the same company. Effect of a Lot Sampling Inspection Plan. If a single-sampling plan is applied to the inspection of the product turned out by a process in a state of statistical control and if the quality of the lots from the process follows a binomial distribution∗ , the number of nonconforming items in the uninspected portion of a lot is independent of the number of nonconforming items in the sample (see ref. 43). For this

QUALITY CONTROL, STATISTICAL

case, the distribution of the number of nonconforming items in the uninspected portion of a lot will be the same whether a lot has been accepted or rejected. Thus in this case the sampling plan has no direct effect on the quality of the lots submitted for inspection. If the sampling plan is applied, however, to lots from processes of different quality levels, it will accept more lots from a process of good quality and reject more lots from a process of poor quality. Hence, if a producer’s process is of poor quality, the percentage of lots rejected will be high and the producer will be under pressure to improve it. Thus with every sampling plan for inspecting a continuing series of lots, the producer will be under pressure to turn out product quality in the neighborhood of the AQL, where the probability of lot acceptance is high. Sampling Inspection Schemes. Awareness of the pressure of lot rejection noted above has led to the development of sampling schemes in which a switch to a tightened inspection plan (say, a plan with a smaller acceptance number) is required when there is evidence of deterioration in process quality, indicated, say, by the rejection of two of the last five lots submitted for inspection. This switch to tightened inspection significantly increases the percentage of poor lots rejected and thus augments the pressure on the producer to turn out product of AQL quality or better. [It is suggested by Hamaker in ref. 26 (p. 275) that this extra pressure may possibly be excessive and thus undesirable. He has also personally indicated to the writer that in practice, when rejections tend to run high on normal inspection, some feedback to the supplier is desirable to help them locate the source of the deterioration of the process. If this does not lead to improvement, tightened inspection can be imposed.] Most schemes also provide for discontinuance of inspection if a producer has been on tightened inspection for a specified period of time. Further incentive to produce good quality is usually provided in these sampling schemes by providing for a shift to a smaller sample size and hence less costly inspection when there is evidence that the process is running at AQL quality or better. Sampling inspection schemes can only be applied, of course, to inspection of a continuing series of lots and

6683

are usually referred to as AQL schemes, since this is a key element in their application. Rectifying Inspection Plans. Whereas sampling inspection schemes seek long-range AQL quality assurance through application of economic pressures and incentives, there are attributes sampling inspection procedures, referred to as rectifying inspection plans, that seek assurance on average outgoing quality∗ (AOQ) by formally incorporating a provision for 100% inspection of rejected lots as part of the sampling plan. The worst AOQ for various levels of incoming quality is the average outgoing quality limit∗ (AOQL), a key figure in an attributes rectifying inspection plan. Such plans are suitable only for a continuing series of lots and, of course, cannot be applied if tests are destructive. Procedures for Cutting Average Sample Size Considerable effort has gone into developing sampling inspection procedures that will allow smaller samples to be used on the average without changing sampling risks. These procedures include double sampling∗ , multiple sampling, and item-by-item sequential sampling∗ . Thus a double-sampling plan that on the first sample permits decisions to accept, reject, or take a second sample may have an average sample size or average sample number∗ (ASN) that for levels of incoming quality in the neighborhood of the AQL is less than the sample size of a single-sampling plan with approximately the same OC curve. Multiple-sampling plans that allow the taking of several samples before a decision is required may have even smaller ASNs. Unit sequential sampling that allows a decision to be made as each sample item is tested and has an OC curve that matches a comparable single-sampling plan at two points, say at the AQL and LQ, will have the smallest ASN at these points (see ref. 65). If the product quality characteristic has a specific distributional form (e.g., normal), a variables sampling inspection plan (one, for example, based on the sample mean and sample standard deviation) can be used to attain assurance regarding the percent nonconforming at a considerable saving in sample size. However, the reduction in cost resulting from a reduction in average sample size obtained

6684

QUALITY CONTROL, STATISTICAL

by the foregoing procedures may be partly offset by greater administrative costs or in the case of variables sampling, by higher measurement costs. There is also the problem of matching various procedures so that they will have approximately the same OC curve. Furthermore, a given variables plan can be applied only to a single quality characteristic. In conclusion, an ASQC standard has been written for ‘‘skiplot’’ sampling∗ that will reduce average inspection costs. Minimum-Cost Designs There may be cases, say in application to within-company operations, in which it is feasible to derive minimum-cost sampling inspection plans and schemes. Reference 24 presents a mathematical discussion of Bayesian∗ sampling plans that consider the expected distribution of lot quality, the cost of inspection, and the cost of accepting a nonconforming item without inspection. Interesting comments on process curves, ‘‘discovery sampling,’’ the minimax∗ principle, and other economic theories of sampling inspection will be found in ref. 25. In some cases the study of costs may suggest that the only economic alternatives are 100% inspection or zero inspection (see refs. 12 and 24). In accepting such alternatives it must be remembered that routine 100% inspection can be boring and hence far from perfect. For zero lot inspection there will also be a need for some checks on the process, to assure that it is maintained at a satisfactory level. Special References References 24 and 53 and the bibliographies they contain provide a comprehensive survey of sampling inspection by attributes. The reader will also find ref. 26 interesting. Reference 13 is a review paper by H. F. Dodge. Tables Systems of sampling schemes for attributes inspection, indexed by AQLs, lot-size ranges, and inspection levels, and allowing for single, double, and multiple sampling, are provided by U.S. MIL-STD-105D, its U.S. civilian counterpart ANSI Z1.4, and by the international standard ISO 2859. Similar systems

for sampling inspection by variables for percent nonconforming, allowing for single sampling only, are provided by U.S. MIL-STD 414 and its civilian counterparts ANSI Z1.9 and ISO 3951 (see INTERNATIONAL STANDARDIZATION, APPLICATION OF STATISTICS IN). The Dodge–Romig sampling inspection tables contain sampling plans indexed by LTPDs (= LQs) and AOQLs and by process levels for which the plans yield minimum average total inspection. A 1983 supplement to ISO 2859 offers single, double, and multiple sampling plans indexed by LQs and by inspection levels and lot size ranges that are compatible with those of MIL-STD-105D. All these tables give OC curves for individual plans and ANSI Z1.4 gives scheme OC curves. Reference 27 gives a small set of sampling plans indexed by indifference quality (= p0.50 or point of control) and lot sizes, and refs. 60 and 61 give the slope of the tangent of the OC curve at the point of inflection for a number of sampling plans. Reference 24 contains a large set of tabular data useful for deriving sampling plans when the probability of lot acceptance can be assumed to be based on a Poisson distribution∗ with parameter λ and λ can be assumed to have a prior gamma distribution∗ , and ref. 53 contains a large collection of tables pertaining to acceptance sampling. Acceptance Control Charts An acceptance control chart (see ref. 20) is a cross between a control chart and a sampling inspection plan. It is intended for use in situations in which process capabilities are well within specification limits and there is little interest in variations in process levels that do not lead to product falling outside these limits. The limits on an acceptance control chart are based on the product specification limits and are intended to detect assignable causes that will lead to the production of an unacceptable fraction of nonconforming product. A succession of sample points within these limits does not necessarily mean that the process is in a state of statistical control as defined above. Acceptance control charts may be viewed as a form of repetitive application of a variables sampling inspection plan with

QUALITY CONTROL, STATISTICAL

6685

known, constant σ . They should be accompanied by an R-chart or an s-chart for continued validation of the assumption of constant σ .

HISTORY

Life Testing∗

Work on statistical quality control was pioneered in the Bell Telephone Laboratories in the late 1920s and early 1930s under the leadership of Walter E. Shewhart, Harold F. Dodge, and Harry G. Romig. Shewhart and Dodge also fostered interest in statistical quality control in the American Society for Testing and Materials. This led to the publication in the 1930s of material on statistical methods that in revised form is now issued as the ASTM Manual on Presentation of Data and Control Chart Analysis (STP 15D). Under Shewhart’s guidance, Leslie Simon started work on statistical quality control at the Picatinny Arsenal in 1934, and Simon’s book An Engineer’s Manual of Statistical Methods appeared in 1941. The big boom in statistical quality control in the United States came during and immediately following World War II (see also MILITARY STATISTICS). Bell Telephone personnel were sent to Washington to develop a sampling inspection program for Army Ordnance, and the Statistical Research Group, Columbia University, prepared a manual on sampling inspection for the U.S. Navy. The principles of the latter were subsequently expounded in Sampling Inspection, edited by H. A. Freeman, M. Friedman, F. Mosteller, and W. A. Wallis, and published in 1947 by McGraw-Hill. (These Army and Navy sampling tables were later amalgamated and issued in 1950 as the Department of Defense MIL-STD-105A.) A significant contribution of the Columbia research group was the work by A. Wald∗ on sequential sampling. The Statistical Research Group Report No. 255, Sequential Analysis of Statistical Data: Applications, was published by Columbia University Press in loose-leaf form in 1945 and Wald published his book on sequential analysis∗ in 1947 [64]. Almost a year before America’s entry into the war, the U.S. War Department induced the American Standards Association to initiate the preparation of instructional material that subsequently became the American War Standards Z1.1-1941, Guide for Quality Control; Z1.2-1941, Control Chart Method of Analyzing Data; and Z1.3-1942, Control

When the quality characteristic of an item that is of interest is its life, a sampling inspection plan will probably seek to give assurance regarding mean life. If items are used in sequence, mean life will become the mean time between failures (MTFB). Reference 63 is a sampling standard suitable for controlling the MTBF when the distribution of life is exponential∗ . A more general type of life distribution which includes the exponential is the Weibull distribution∗ , which has been studied in depth. See, for example refs. 28, 31, 40, and 66. Weibull probability paper is described in ref. 44. Life testing is a very important part of reliability∗ engineering. Sampling Inspection of Bulk Material Sampling inspection of bulk material takes a special form since physical averaging can be substituted for arithmetic averaging of separate test results. It usually consists of random or systematic selection of increments of the material (generally when it is moving on a conveyor belt or being loaded or unloaded), compositing these increments to form a single gross sample, and then reducing this gross sample to a laboratory sample from which test units are drawn for analysis. Pilot studies of how interincrement variances relate to increment size and how the subsampling variances (called reduction variances) relate to the size of the composite sample are found useful in designing a sampling plan and in setting up rough confidence limits for the mean of a lot or process. In inspection of a series of lots, the use of two composites instead of one or the taking of parallel subsamples will provide routine checks on the basic variances obtained in a preliminary pilot study. See ref. 16 and for composite sampling in particular, see refs. 10 and 51. JIS M8100, General Rules for Methods of Sampling Bulk Materials, is a well-received Japanese standard and ref. 23 is a comprehensive study of sampling particulate material.

American Developments

6686

QUALITY CONTROL, STATISTICAL

Chart Method of Controlling Quality during Production. This material was used in training courses given in numerous cities during the war. In 1944, publication of the journal Industrial Quality Control was begun under the auspices of the University of Buffalo (see JOURNAL OF QUALITY TECHNOLOGY ). The interest in statistical quality control aroused during World War II led to the formation in 1946 of the American Society for Quality Control. The society took over publication of Industrial Quality Control and subsequently replaced it by the two journals Quality Progress and the Journal of Quality Technology∗ . In 1959, it initiated with the American Statistical Association∗ the joint publication of Technometrics∗ . ASQC technical conferences are held annually. It has many local chapters and its membership as of March 31, 1982, was 37,254. The year 1946 also saw the establishment of ASTM Committee E-11 on Quality Control of Materials under the chairmanship of Harold Dodge. This subsequently became the current ASTM Committee E-11 on Statistical Methods. Besides giving advice to ASTM technical committees regarding statistical matters, Committee E-11 has issued standards on indication of significant figures in specified limiting values (E29), on probability sampling of materials (E105, E122, and E141), on the use of the terms precision and accuracy as applied to measurement of a property of a material (E177), on dealing with outlying observations (E178), and on conducting an interlaboratory study to determine the precision of a test method (E691). The committee also has responsibility for maintaining ASTM Special Technical Publication 15D, noted above. The idea of using variables sampling involving the sample mean and sample standard deviation or sample range to give assurance regarding the percent nonconforming had been considered by the Columbia Research Group, but the Group was disbanded in 1945 before it was possible to publish tables of variables plans. The Office of Naval Research, however, undertook in 1948 to support this work at Stanford University. The book by Bowker and Goode on Sampling Inspection by Variables was published in 1952 and the basic paper by Lieberman and

Resnikoff on ‘‘Sampling plans for inspection by variables,’’ which subsequently became the Department of Defense Standard MILSTD-414, was published in the Journal of the American Statistical Association∗ in 1955. The paper by A. J. Duncan on the economic design of X-charts, published in 1956 (see ref. 15), was also supported under Stanford University’s contract with the Office of Naval Research. In 1950, the American Statistical Association sponsored a symposium on acceptance sampling [1] in which Paul Peach of the University of North Carolina presented a paper on ‘‘Statistical developments in acceptance sampling by attributes prior to 1941,’’ Edwin G. Olds of Carnegie Institute of Technology presented a paper on ‘‘War-time developments in acceptance sampling by attributes,’’ J. H. Curtis of the National Bureau of Standards∗ presented a paper on ‘‘Lot quality measured by average or variability,’’ and W. Allen Wallis of Stanford University presented a paper on ‘‘Lot quality measurement by proportion defective.’’ The proceedings include considerable discussion by others in attendance at the symposium, as well as closing remarks by the chairman, John W. Tukey, of Princeton University. A more recent American development has been the organization in 1974 of the American National Standards Institute (ANSI) Z-1 Committee on Quality Assurance, with a Subcommittee on Statistical Methods to serve in an advisory capacity regarding quality control standards. ASQC serves as secretariat for the committee. British Developments In England, statistical analyses relating to the variation in quality of output were undertaken as early as the 1920s by Bernard Dudding of the General Electric Co.’s research laboratories at Wembley. The need to study small-sample variations in means against a probability background was emphasized in his 1929 company paper. Since reference is made in this paper to articles by Shewhart and others in Vols. 1–6 of the Bell System Technical Journal (see, e.g., refs. 54 and 55), it is not clear whether the application of sampling theory as a control for

QUALITY CONTROL, STATISTICAL

mean values occurred to Dudding before he came across these earlier papers. In 1925, L. H. C. Tippett, a statistician for the British cotton industry, published a paper [58] on the means of sample ranges from a normal distribution which contained results that subsequently had significant use in statistical quality control. In 1931, Egon S. Pearson∗ of University College, London, visited the United States and spent some days with Shewhart. As a result, Shewhart was invited to come to England in May 1932 to give three lectures in University College on The Role of Statistical Method in Industrial Standardization. Following this, things moved rapidly in England. Pearson gave a paper entitled ‘‘A survey of the uses of statistical method in the control and standardization of the quality of manufactured products’’ at the December 1932 meeting of the Royal Statistical Society∗ , and the next year the society established an Industrial and Agricultural Research Section. Following Shewhart’s lectures, the British Standards Institute set up a small committee on Statistical Methods in Standardization and Specification, including Egon Pearson, Dudding, and representatives from various British Industries. An outcome was BS 600-1935, The Application of Statistical Methods to Industrial Standardization and Quality Control, which bore Egon Pearson’s name. Another ‘‘guide’’ subsequently issued was BS 2564 (1955), Control Chart Technique when Manufacturing to a Specification, by B. P. Dudding and W. J. Jennett (originally prepared for the General Electric Co. in 1944). An excellent review of these early developments in Great Britain is contained in ref. 49. In the 1950s and 1960s interest developed in Great Britain in improved procedures for detecting process changes. A particularly significant contribution was E. S. Page’s article on cusum charts [45] that appeared in 1954. (See also his 1963 paper [48].) In 1959, G. A. Barnard [5] proposed the use of a Vmask for a two-sided cusum scheme, and Kenneth W. Kemp in a 1961 paper [33] discussed the ‘‘Average run length∗ of the cumulative sum chart when a V-mask is used.’’ A review paper by Page [47] was published in

6687

1961. In 1955, Page published a paper on the use of warning limits (see ref. 46). Also in the 1950s and 1960s, interest was manifested in applying Bayesian procedures to sampling inspection. In 1954, G. A. Barnard presented a paper at a meeting of the Royal Statistical Society entitled ‘‘Sampling inspection and statistical decisions’’ [4]. In February 1960 at a meeting at Imperial College in London, further papers on Bayesian aspects of sampling inspection were presented by A. Hald of the University of Copenhagen and by G. B. Wetherill and D. R. Cox of the University of London. These papers were published in the August 1960 issue of Technometrics together with the discussion (pp. 275–372) that followed. Other Developments From the United States and Great Britain interest in statistical quality control spread to other countries. W. Edwards Deming’s lectures on statistical quality control in Japan following World War II led to the development there of one of the finest quality control systems in the world. Prominent in this development was the Japanese Union of Scientists and Engineers. For an account of the Japanese development, see ref. 32. American and British demand for European goods was influential in developing an interest in statistical quality control in Europe. H. C. Hamaker, who played a prominent role in the development of statistical quality control procedures in the Philips Electric Co. at Eindhoven (see ref. 25), has suggested that it was the success that these methods had in Philips’s English factories during World War II that provided the main stimulus for their introduction at Eindhoven. A European Organization for Quality Control (EOQC) was established in 1956 and became active in promoting statistical quality control on the continent. In 1965, it published A Glossary of Terms Used in Quality Control with their equivalents (in the 1972 edition) in 14 European languages (see ref. 19). Further international activity has taken the form of standards sponsored by the International Organization for Standardization∗ (ISO) Technical Committee 69 on Applications of Statistical Methods. The United

6688

QUALITY CONTROL, STATISTICAL

States participates in this through the American National Standards Institute (ANSI), and the United Kingdom through the British Standards Institute. Special international co-operation has taken the form of quality control agreements sponsored by the ABCA Group, made up of representatives of the American, British, Canadian, and Australian armies. (New Zealand participates through Australia.) American, British, and Canadian military representatives constituted the ABC committee that was responsible for the 1963 revision of MIL-STD-105 known as MILSTD-105D. Interest in statistical quality control has thus become worldwide, a great quantity of literature on the subject exists, and the application of statistical quality control techniques is widespread. Recent Trends Prominent in recent years has been the interest in the dynamic aspects of sampling inspection schemes. Koyama’s 1979 paper [35] studies the switching characteristics of MIL-STD105D. Hald in ref. 24 devotes a whole chapter to switching rules. ANSI/ASQC Z1.4 (the civilian version of MIL-STD-105D) gives data on the operating characteristics of its sampling schemes for the case in which the producer makes no change in his or her process and there is no discontinuance of inspection. These hypothetical operating characteristics are useful in comparing various schemes. However, they do not generally give a realistic representation of what would happen in practice, since when the process is running at levels significantly worse than the AQL, the resulting high rate of rejections will usually lead to attempts to improve it. A very recent development has been the mounting interest in the consideration of costs in the determination of inspection procedures. Prominent in this has been W. E. Deming’s book Quality, Productivity and Competitive Position [12]. The book was published in 1983, but a preliminary version was available for use in his courses and seminars for some time before that. The considerable discussion given to Bayesian sampling plans in A. Hald’s 1981 book on sampling inspection plans [24] has also heightened the interest in relative costs.

There has been an increased interest in lot LQs of sampling inspection plans. Whereas ISO Std 2859 (the international counterpart of MIL-STD-105D) is intended for inspection of a series of lots and is indexed by AQLs, a 1983 supplement is indexed by LQs and is intended for inspection of lots in isolation. In 1978 [52] Schilling discussed ‘‘lot-sensitive’’ sampling plans. Interest in CUSUM procedures has continued strong. In 1976, Lucas [37] discussed the design and use of V-mask control schemes, and in 1982 [38], a combination of Shewhart and CUSUM schemes. In 1982, Lucas and Crosier [39] discussed the use of a ‘‘fast initial response’’ for a CUSUM scheme. At the 1982 annual conference of the American Statistical Association, Donald Marquardt noted the loss of information in going from continuous to attribute data and recommended the retention of continuous data in that form with the use of cusum techniques for process control; these techniques can easily be computerized and the computer results can readily be understood by operating personnel. ISO Technical Committee 69 is preparing a standard for cusum charts. Improved methods for reporting quality assurance audit results have been presented by Hoadley in his paper ‘‘The quality measurement plan (QMP)∗ ’’ [29]. In the QMP, confidence intervals are computed from both current and past data and are derived from a new Bayesian approach to the empirical Bayes∗ problem for Poisson observations. Computer graphics are being used increasingly. For a prominent example of the use of J. Tukey’s ‘‘box and whisker plots’’ (see ref. 59 and EXPLORATORY DATA ANALYSIS) in the reporting of process quality, see Hoadley [29]. Videotapes of lectures on statistical quality control are currently being sold or briefly rented by the publishers of Deming’s book [12] for training courses using his book as a text. Such presentations will probably become an important form of instruction in this field. REFERENCES 1. Acceptance Sampling—A Symposium (1950). American Statistical Association, Washington, D.C.

QUALITY CONTROL, STATISTICAL 2. American Society for Quality Control (1978). Definitions, Symbols, Formulas and Tables for Quality Control. ANSI/ASQC Standard A1–1978.

21. Gibra, I. N. 183–192.

3.

American Society for Testing and Materials (1976). ASTM Manual on Presentation of Data and Control Chart Analysis. STP 15D.

23. Gy, P. M. (1979). Sampling of Particulate Materials: Theory and Practice. Elsevier, New York.

4. Barnard, G. A. (1954). J. R. Statist. Soc. B, 16, 151–165.

24. Hald, A. (1981). Statistical Theory of Sampling Inspection by Attributes. Academic Press, New York.

5. Barnard, G. A. (1959). J. R. Statist. Soc. B, 21, 239–257; discussion, 257–271. 6. Bauer, P. and Hackl, P. (1978). Technometrics, 20, 431–436. 7. Bauer, P. and Hackl, P. (1980). Technometrics, 22, 1–7. 8. Bowker, A. H. and Goode, H. P. (1952). Sampling Inspection by Variables. McGraw-Hill, New York. 9. Box, G. E. P. and Jenkins, G. M. (1970). Time Series Analysis, Forecasting and Control. Holden-Day, San Francisco.

(1975).

J. Quality

6689 Tech.,

7,

22. Grubbs, F. E. and Coon, H. F. (1954). Ind. Quality Control, 10(5), 15–20.

25. Hamaker, H. C. (1958). Appl. Statist., 7, 149–159. 26. Hamaker, H. C. (1960). Bull. Int. Statist. Inst., 37, 265–281. 27. Hamaker, H. C., Taudin Chanot, J. J. M., and Willemze, F. G. (1950). Philips Tech. Rev., 11, 362–370. 28. Harter, H. L. and Moore, A. H. (1976). IEEE Trans. Rel., R-25, 100–104. 29. Hoadley, B. (1981). Bell Syst. Tech. J., 60, 215–273.

10. Brown, G. H. and Fisher, J. I. (1972). Technometrics, 14, 663–668.

30. Johnson, N. L. (1961). J. Amer. Statist. Ass., 56, 835–840.

11. Chiu, W. K. and Wetherill, G. N. J. Quality Tech., 6, 63–69.

31. Johnson, N. L. and Leone, F. C. (1977). Statistics and Experimental Design in Engineering and the Physical Sciences, 2nd ed. Wiley, New York. (This discusses a variety of uses of cusum charts.)

(1974).

12. Deming, W. E. (1983). Quality, Productivity and Competitive Position. MIT Center for Advanced Engineering Study, Cambridge, Mass. 13. Dodge, H. F. (1969). J. Quality Tech., 1. Part I, Apr., 77–88; Part II, July, 155–162; Part III, Oct., 225–232; 2. Part IV, Jan. 1970, 1–8. 14. Dodge, H. F. and Romig, H. G. (1959). Sampling Inspection Tables—Single and Double Sampling. Wiley, New York. (First edition in 1944.) 15. Duncan, A. J. (1956). J. Amer. Statist. Ass., 51, 228–242. 16. Duncan, A. J. 319–344.

(1962).

Technometrics,

4,

17. Duncan, A. J. (1974). Quality Control and Industrial Statistics, 4th ed. Richard D. Irwin, Homewood, Ill. 18. Duncan, A. J. 235–243.

(1978).

Technometrics,

20,

19. European Organization for Quality Control—EOQC (1972). Glossary of Terms Used in Quality Control with Their Equivalents in Arabic, Bulgarian, Czech, Dutch, French, German, Italian; Norwegian, Polish, Portuguese, Romanian, Russian, Spanish and Swedish. EOQC, Rotterdam, The Netherlands. 20. Freund, R. A. (1957). Ind. Quality Control, 14(4), 13–23.

32. Juran, J. M. (1975). Rep. Statist. Appl. Res., 22, 66–72. Union of Japanese Scientists and Engineers, Tokyo. 33. Kemp, K. W. (1961). J. R. Statist. Soc. B, 23, 149–153. 34. King, E. P. (1952). Ann. Math. Statist., 23, 384–394. 35. Koyama, T. (1979). Technometrics, 21, 9–19. 36. Lieberman, G. J. and Resnikoff, G. J. (1955). J. Amer. Statist. Ass., 50, 457–516. 37. Lucas, J. M. (1976). J. Quality Tech., 8, 1–8. 38. Lucas, J. M. (1982). J. Quality Tech., 14, 51–59. 39. Lucas, J. M. and Crosier, R. B. (1982). Technometrics, 24, 199–205. 40. Mann, N. R., Schafer, R. E., and Singpurwalla, N. D. (1974). Methods for Statistical Analysis of Reliability and Life Data. Wiley, New York. 41. Marquardt, D. W. (1982). Amer. Statist. Ass., Proc. Sec. Statist. Educ., pp. 21–27. 42. Montgomery, D. C. (1980). J. Quality Tech., 12, 75–89. 43. Mood, A. M. (1943). Ann. Math. Statist., 13, 415–425.

6690

QUALITY MEASUREMENT PLAN (QMP)

44. Nelson, L. S. (1967). Ind. Quality Control, 24, 452. 45. Page, E. S. (1954). Biometrika, 41, 100–114.

66. Weibull, W. 293–297.

46. Page, E. S. (1955). Biometrika, 42, 243–257. 47. Page, E. S. (1961). Technometrics, 3, 1–9. 48. Page, E. S. (1963). Technometrics, 5, 307–315.

BIBLIOGRAPHY

49. Pearson, E. S. (1973). Statistician, 22, 165–179. 50. Roberts, S. W. (1959). Technometrics, 1, 239–250. 51. Rohde, C. A. (1976). Biometrics, 32, 273–282. 52. Schilling, E. G. (1978). J. Quality Tech., 10, 47–51. 53. Schilling, E. G. (1982). Acceptance Sampling in Quality Control. Marcel Dekker, New York. 54. Shewhart, W. A. (1926). Bell Syst. Tech. J., 5, 593–606. 55. Shewhart, W. A. (1927). Bell Syst. Tech. J., 6, 722–735. 56. Shewhart, W. A. (1931). Economic Control of Quality of Manufactured Product. D. Van Nostrand, New York. 57. Shewhart, W. A. with the editorial assistance of W. E. Deming. (1939). Statistical Method from the Viewpoint of Quality Control. Graduate School, U.S. Dept. of Agriculture, Washington, D.C. 58. Tippett, L. H. C. (1925). Biometrika, 17, 364–387. 59. Tukey, J. W. (1977). Exploratory Data Analysis. Addison-Wesley, Reading, Mass. 60. U.S. Dept. of the Army, Chemical Corps Engineering Agency (1953). A Method of Discrimination for Single and Double Sampling OC Curves Utilizing the Tangent at the Point of Inflection. ENASR No. PR-7. 61. U.S. Dept. of the Army, Chemical Corps Engineering Agency (1954). A Method of Fitting and Comparing Variables and Attributes Operating Characteristic Curves Using the Inflection Tangent with Tables of Inflection Tangents for Variables Sampling OC Curves. ENASR No. PR-12. 62. U.S. Dept. of Defense (1973). Maintainability Verification/Demonstration/Evaluation. MIL-STD-471A. 63. U.S. Dept. of Defense (1977). Reliability Design Qualification and Production Acceptance Tests: Exponential Distribution. MILSTD-781C. 64. Wald, A. (1947). Sequential Analysis. Wiley, New York. 65. Wald, A. and Wolfowitz, J. (1948). Ann. Math. Statist., 29, 326–329.

(1951).

J. Appl.

Mech.,

18,

See the following works, as well as the references just given, for more information on the topic of statistical quality control. American Society for Quality Control. Publications Catalogue, 230 West Wells Street, Milwaukee, WI 53202. Ishikawa, K. (1982). Guide to Quality Control, 2nd rev. ed. Asian Productivity Organization, Tokyo. [May be ordered through the American Society for Quality Control (see address above). The book, a translation from the Japanese, is a summary of articles and exercises originally published in the magazine Quality Control for the Foreman.] Juran, J. M., Gryna, F. M., and Bingham, R. S., eds. (1974). Quality Control Handbook, 3rd ed. McGraw-Hill, New York. (A wide coverage.) Ott, E. R. (1975). Process Quality Control. McGraw-Hill, New York. (Contains many examples.) See also ACCEPTANCE SAMPLING; AVERAGE OUTGOING QUALITY (AOQ); AVERAGE OUTGOING QUALITY LIMIT (AOQL); AVERAGE RUN LENGTH (ARL); BULK SAMPLING; CONTROL CHARTS; CUMULATIVE SUM CONTROL CHARTS; LOT TOLERANCE TABLES, DODGE –ROMIG; EXPLORATORY DATA ANALYSIS; INSPECTION SAMPLING; INTERNATIONAL STANDARDIZATION, APPLICATION OF STATISTICS IN; MILITARY STATISTICS; MOVING SUMS (MOSUM); and SAMPLING PLANS.

ACHESON J. DUNCAN

QUALITY MEASUREMENT PLAN (QMP) An important function of quality assurance is to measure the outgoing quality of a product or service. In the Western Electric Company (1983) this is done with the Quality Measurement Plan (QMP), a system for analyzing and reporting attribute quality audit results. A quality audit is a highly structured system of inspections done continually on a sampling basis. Sampled product is inspected and defects are assessed for failure to meet the engineering requirements. The results during a rating period (eight per year) are

QUALITY MEASUREMENT PLAN (QMP)

aggregated and compared to a quality standard. The quality standard is a target value for defects per unit, which reflects a tradeoff between manufacturing cost, operating costs, and customer need. Specific examples of audits are (a) a functional test audit for digital hybrid integrated circuits and (b) a workmanship audit for spring-type modular cords. QMP replaces Shewhart-type control charts∗ such as the p-chart, the c-chart, and the T-rate, all described in ref. 4. Box and whisker plots are used to graphically display the posterior distribution∗ of true current quality (see EXPLORATORY DATA ANALYSIS). The posterior distribution is computed from both current and past data and is derived from a Bayesian approach to the empirical Bayes∗ problem for Poisson observations. The idea for QMP evolved from the work of Efron and Morris on empirical Bayes∗ estimation [1], in which they illustrate the method with baseball batting averages. There is a clear analogy between traditional quality assurance attribute measures and batting averages. QMP advances the empirical Bayes method in two directions. The first is to consider a given product type as a random selection from a large population of product types. This leads to a Bayes estimate of the empirical Bayes shrinkage weight rather than the usual maximum likelihood estimate∗ . The second is to provide an empirical Bayes interval estimate rather than just a point estimate. The complete documentation of QMP is given in ref. 2, where the rationale, mathematical derivations, dynamics, operating characteristics, and many examples are discussed. This article is a summary of material in both refs. 2 and 3. QMP ANALYSIS As an example, consider Fig. 1. This is a comparison of the QMP reporting format (a) with the old T-rate reporting format (b), which is based on the Shewhart control chart [4]. Each year is divided into eight periods. In Fig. 1b, the T-rate is plotted each period and measures the difference between the observed and standard defect rates in units of sampling

6691

Figure 1. (From ref. 2. Reprinted with permission of the Bell System Technical Journal, Copyright AT & T, 1981.)

standard deviation (given standard quality). The idea is that if the T-rate is, for example, less than −2 or −3, the hypothesis of standard quality is rejected. The T-rate is simple but has problems; for example, it does not measure quality. A T-rate of −6 does not mean that quality is twice as bad as when the T-rate is −3. The T-rate is only a measure of statistical evidence with respect to the hypothesis of standard quality. QMP provides a complete statistical inference of the true quality process. Under QMP, a box and whisker plot (Fig. 1a) is plotted each period. The box plot is a graphical representation∗ of the posterior distribution of current true quality on a quality index scale. ‘‘One’’ is the standard value on the quality index scale. ‘‘Two’’ means twice as

6692

QUALITY MEASUREMENT PLAN (QMP)

many defects as expected under the standard. The posterior probability that the true quality index is larger than the top whisker is 0.99. The top of the box, the bottom of the box, and the bottom whisker correspond to probabilities of 0.95, 0.05, and 0.01, respectively. The heavy dot is a Bayes estimate of the long-run process average, the X is the observed value in the current sample, and the dash is the Bayes estimate of the current true quality index and is called the best measure of current quality. This is an empirical Bayes estimate—a shrinkage toward the process average. The process averages (dots) are joined to show trends. Although the T-rate chart and the QMP chart sometimes convey similar messages, there are differences. The QMP chart provides a measure of quality; the T-rate chart does not. For example, in period 6, 1978, both charts imply that the quality is substandard, but the QMP chart also implies that the true quality index is somewhere between 1 and 3. Comparing period 6, 1977, with period 4, 1978, reveals similar T-rates but QMP box plots with different messages. The QMP chart is a modern control chart for defect rates. However, it can be used for management reporting rather than as part of a shop process control system. This is the way it is used at present in the Western Electric Company.

presented herein via a transformation [2, p. 229]. We measure the defect rate with the quality index It = xt /et , which is the defect rate expressed as a multiple of the standard defect rate. So It = 2 means that we observed twice as many defects as expected. The formulas used for computing the QMP box plots shown in Fig. 1a were derived from a Bayesian analysis of a particular statistical model. The assumptions of the model are: 1. xt is the observed value of a random variable Xt , whose sampling distribution is Poisson∗ with mean = nt λt , where λt is the true defect rate per unit. For convenience, we reparameterize λt on a quality index scale as θt = true quality index = λt /s. So the standard value of θt is 1. 2. θt , t = 1, . . . , T, is a random process (or random sample) from a gamma distribution∗ with θ = process average γ 2 = process variance,

STATISTICAL FOUNDATIONS OF QMP For rating period t [t = 1, . . . , T(current period)], the data from the audit are of the form nt = audit sample size xt = defects observed in the audit sample et = expected defects in the audit sample when the quality standard is met = snt , where s represents the standard defects per unit. In practice, defectives or weighted defects are sometimes used as the quality measure. These cases can be treated by the theory

which are unknown. This assumption makes this a parametric empirical Bayes model. 3. θ and γ 2 have a joint prior distribution∗ . The physical interpretation of this prior is that each product type has its own value of θ and γ 2 , and these vary at random across product types according to the foregoing prior distribution. This assumption makes this a Bayes empirical Bayes model. It is now a full Bayesian model, specifying the joint distribution of all variables. The quality rating in QMP is based on the conditional or posterior distribution of current quality (θT ) given all the data, x = (x1 , . . . , xT ).

QUALITY MEASUREMENT PLAN (QMP)

POSTERIOR DISTRIBUTION OF CURRENT QUALITY The formulas for the posterior mean and variance of current quality are given in ref. 2. Here we provide the essence of them. The posterior mean of current quality (θT ) is θˆT = ωˆ T θˆ + (1 + ωˆ T )IT , where θˆ = Bayes estimate of the process average(θ ), ωˆ T = Bayes estimate of the shrinkage, weight   ωT = [θ/eT ] (θ/eT ) + γ 2 . This is a weighted average of the estimated process average θˆ and the sample index IT . It is the dynamics of the shrinkage weight (ωˆ T ) that causes the Bayes estimate to work so well. For any t, the sampling variance of It (under the Poisson assumption) is θt /et . The expected value of this is E[θt /et ] = θ/et . So the shrinkage weight (ωT ) is of the form sampling variance . (sampling variance) + (process variance) If the process is stable, the process variance is relatively small and the weight is mostly on the process average; but if the process is unstable, the process variance is relatively large and the weight is mostly on the current sample index. The reverse is true of the sampling variance. If is it large (e.g., small expectancy), the current data are weak and the weight is mostly on the process average; but if the sampling variance is small (e.g., large expectancy), the weight is mostly on the current sample index. In other words, ωT is monotonically increasing with the ratio of sampling variance to process variance. The posterior variance of current quality (θT ) is VT = (1 − ωˆ T )θˆT /eT +ωˆ T2 (posterior variance of the process average) +(θˆ − IT )2 (posterior variance of the shrinkage weight).

6693

If the process average and variance are known, the posterior variance of θT can easily be shown to be (1 − ωT )θˆT /eT , which is estimated by the first term in VT . But since the process average and variance are unknown, the posterior variance has two additional terms. One contains the posterior variance of the process average and the other the posterior variance of the shrinkage weight. The first term dominates. A large ωˆ T (relatively stable process), a small ωˆ T (good current quality), and a large eT (large audit) all tend to make the posterior variance of θT small. If ωˆ T is small, the second term is negligible. This is because the past data are not used much, so the uncertainty about the process average is irrelevant. If the current sample index is far from the process average, the third term can be important. This is because outlying observations add to our uncertainty. If the process average and variance were known, the posterior distribution would be gamma, so we approximate the posterior distribution with a gamma fitted by the method of moments∗ . The parameters of the fitted gamma are α = shape parameter = θˆT2 /VT , τ = scale parameter = VT /θˆT . QMP BOX AND WHISKER PLOT For the box and whisker plots shown in Fig. 1a, the top whisker, top of box, bottom of box, and bottom whisker are the 1st, 5th, 95th, and 99th percentiles of the approximate posterior distribution of current quality, respectively. So a posteriori, there is a 95% chance that the current quality is worse than the top of the box. EXCEPTION REPORTING For QMP, there are two kinds of exceptions: Below normal means that the top whisker is worse than the standard (1); that is, there is more than a 0.99 posterior probability that current true quality is worse than standard. Alert means that the top of the box is worse than standard but the product is not below normal; that is, the posterior probability that current quality is worse than standard is between 0.95 and 0.99. Products that meet

6694

QUALITY MEASUREMENT PLAN (QMP) Table 1. 1977 period

7703

7704

7705

7706

7707

t Number of defects (xt ) Quality index (It )

1 17 2.4

2 20 2.9

3 19 2.7

4 12 1.7

5 7 1.0

these conditions are highlighted in an exception report. NUMERICAL EXAMPLE To illustrate the QMP calculations, consider the eighth period of 1977 in Fig. 1a. Six periods of data (T = 6) are used—period 3 through period 8. The sample size (nt ) in each period is 500 and the standard defects per unit (s) is 0.014, so the expected number of defects in each sample is et = snt = (0.014)(500) = 7. The number of defects (xt ) observed in each period is given in Table 1 together with the quality index, It = xt /et = xt /7. The posterior distribution analysis described in the section ‘‘Posterior Distribution of Current Quality’’ is simplified because of the equal sample sizes. The Bayes estimate (θˆ ) of the process average is approximately the average of the It ’s (2.4 + 2.9 + 2.7 + 1.7 + 1.0 + 1.6)/6 = 2.0. The estimated sampling variance for the current period is θˆ /e6 = 2.0/7 = 0.28. The process variance is approximated by the total variance of the It ’s minus the average sampling The total variance is thus 6 variance. ˆ2 t=1 (It − θ) /5 = 0.54 and the average sampling variance is 0.28, because the sample sizes do not vary. So the process variance is 0.54 − 0.28 = 0.26. The Bayes estimate of the shrinkage weight is ωˆ 6 = 0.28/(0.28 + 0.26) = 0.52. The posterior mean of current quality is ωˆ T θˆ + (1 − ωˆ T )TT = (0.52)(2.0) + (0.48)(1.6) = 1.8. The dominant term in the posterior variance of current quality is (1 − ωˆ T )θˆT /eT = (0.48)(1.8)/7 = 0.123. The second term is ωˆ T2 (posterior variance of the process average) ≈ ωˆ T2 (total variance)/T = (0.52)2 (0.54)/6 = 0.024. The posterior variance of the shrinkage weight can be shown to be 0.019; hence the third term is (2.0 − 1.6)2 (0.019) = 0.003. So the posterior

7708 6 11 1.6

variance (VT ) of current quality is 0.123 + 0.024 + 0.003 = 0.15. The approximate posterior distribution of current quality is a gamma distribution with shape parameter α = θˆT2 /VT = (1.8)2 /0.15 = 21.6 and scale parameter τ = VT /θˆT = 0.15/1.8 = 0.083. The box and whisker plot is constructed from the 1st, 5th, 95th, and 99th percentiles of this gamma distribution, which are 1.01, 1.2, 2.5, and 2.8, respectively. The quality in period 8, 1977, is below normal, because the top whisker is below the standard (1). The corresponding T-rate shown in Fig. 1b is given by T-rate =

7 − 11 et − xt = √ = −1.5. √ et 7

This is well within the control limit of −2.0. So with QMP, we reach a different conclusion, because of the poor quality history. REFERENCES 1. Efron, B. and Morris, C. (1977). Sci. Amer., 236(5), 119–127. 2. Hoadley, B. (1981). Bell Syst. Tech. J., 60, 215–271. 3. Hoadley, B. (1981). Proc. 31st Electron. Components Conf., pp. 499–505. 4. Small, B. B. (Chairman of Writing Committee) (1958). Western Electric Statistical Quality Control Handbook. Western Electric, Indianapolis, Ind. (Sixth printing by Mack Printing Co., Easton, Pa., 1982.) See also CONTROL CHARTS; EMPIRICAL BAYES THEORY; EXPLORATORY DATA ANALYSIS; NOTCHED BOX-AND-WHISKER PLOT; POSTERIOR DISTRIBUTIONS; QUALITY CONTROL, STATISTICAL; and SHRINKAGE ESTIMATORS.

BRUCE HOADLEY

QUANGLE

QUANGLE A quangle (quality control angle chart) [1,3,5] is a diagram for representing a series of numbers by directions (rather than lengths, as on a conventional graph). It consists of a chain of equal straight-line segments, one for each term of the series, and each pointing in the direction corresponding to that term. Figure 1 shows a quangle of the maximum temperature in London for each day of the first three months of 1983. It consists of 90 segments, starting at A and finishing at B. The orientation has been chosen so that the general direction is across the paper. The ‘‘horizontal’’ direction represents 9◦ C. (The average for the three months was 9.28◦ C.) The value chosen for the horizontal is called the neutral value. The angular scale is 10◦ of angle for 1◦ C. In accordance with the usual convention, a change in direction anticlockwise corresponds to an increase in temperature. The quangle shows that on the whole, February was a cooler month than January or March, with a particularly cold spell from February 9 to 15, when the maximum temperature reached only 2 or 3◦ C. There was a warm fortnight from March 5 to 19, with maxima around 12◦ C, followed by a rather cooler week. It is important to choose an appropriate angular scale, so that relevant changes can be seen clearly but minor fluctuations are not exaggerated. Any series of numbers can be plotted as a quangle. The general shape of the plot will show at a glance whether the data are on the whole uniform or whether there is a sudden (or gradual) change in the mean. If the data form a stationary time series∗ , the quangle will tend to go in a constant direction, with fluctuations about this direction corresponding to the standard deviation. A sudden

Figure 1. Quangle of the maximum daily temperature in London for the first three months of 1983.

6695

change in the underlying mean corresponds to a bend in the quangle, whereas a gradual change gives rise to a steady curve. In industrial quality control∗ a CUSUM (cumulative sum control chart∗ ) [4] is often used to indicate when the mean value of a process has changed. If the system is then brought ‘‘under control’’ again by altering the mean, a new CUSUM is started. However, in certain situations where it is not possible to correct the process immediately, it is preferable to have a chart that will monitor successive changes, responding equally sensitively to each. A CUSUM changes direction less for a given change in the mean as it gets steeper, whereas in a quangle the change in direction is always proportional to the change in mean. As long as the process is in control, the two charts are virtually indistinguishable, and so can be used interchangeably. Figures 2 and 3 show two CUSUMS of the temperature data used in Fig. 1. In Fig. 2 the target value is the mean temperature for January (10.46◦ C), whereas Fig. 3 uses the mean for February (5.75◦ C). The scale has been chosen to make them directly comparable to the quangle: that is, the ratio of the

Figure 2. CUSUM of the same data, taking the mean temperature for January as target value.

Figure 3. Another CUSUM, taking the mean temperature for February as target value.

6696

QUANTAL DATA

units in the y and x directions is equal to the angular scale in radians. The January section of Fig. 2 is very similar to that in Fig. 1. The February section is longer (because the CUSUM takes equal intervals in the x-direction). In Fig. 3, the February section is similar to the quangle (but rotated through about 35◦ ), while the January and March sections are distorted. In summary, a CUSUM indicates well when a process departs from a preassigned target value, but a quangle gives a representative picture of the whole series. It is therefore more appropriate for use in examining a series retrospectively for change points. The target value affects the shape of the CUSUM. The neutral value affects the orientation, but not the shape, of a quangle. For small amounts of data it is easy to draw a quangle on squared paper, using a 360◦ protractor. For routine use, it is better to write a computer program for use with a graph plotter. Finding the appropriate angular scale may require some experimentation. A reasonable rule is to take about 30◦ to correspond to the minimum change that needs to be detected. For retrospective use, a scale of 30◦ to 1 standard deviation will probably show up salient features. The theoretical properties of the quangle involve the statistics of directional data∗ [2]. If the data follow a von Mises distribution (see DIRECTIONAL DISTRIBUTIONS), the direction of the quangle is a consistent estimator of the mean. Acknowledgment I would like to thank the London Weather Centre for permission to use their temperature data, and my colleague S. G. Thompson for the computer program that drew the quangle.

REFERENCES 1. Lear, E. (1877). ‘‘The Quangle Wangle’s Hat.’’ [In Laughable Lyrics, a fourth book of nonsense poems, songs, botany, music, etc. (Edward Lear is responsible only for the name.)] 2. Mardia, K. V. (1972). Statistics of Directional Data. Academic Press, London. 3. North, W. R. S. (1982). Appl. Statist., 31, 155–158. (A fuller account of the quangle and its properties.)

4. Page, E. S. (1961). Technometrics, 3, 1–9. (A discussion of the CUSUM and other control schemes.) 5. Thompson, S. G. (1983). Statist. Med. 2, 147–153. See also CUMULATIVE SUM CONTROL CHARTS; DIRECTIONAL DATA ANALYSIS; and QUALITY CONTROL, STATISTICAL.

W. R. S. NORTH

QUANTAL DATA. See BIOASSAY, STATISMETHODS IN; CATEGORICAL DATA; NOMINAL DATA; PROBIT ANALYSIS

TICAL

QUANTAL RESPONSE ANALYSIS. See PROBIT ANALYSIS

QUANTILE Quantiles play a fundamental role in statistics, although their use is often disguised by notational and other artifices. They are the critical values we use in hypothesis testing∗ and interval estimation and often are the characteristics of a distribution we wish most to estimate. Sample quantiles are utilized in numerous inferential settings and, recently, have received increased attention as a useful tool in data modeling. Historically, the use of sample quantiles in statistics dates back, at least, to Quetelet∗ [40], who considered the use of the semiinterquartile range as an estimator of the probable error∗ for a distribution. Subsequent papers by Galton∗ and Edgeworth (see, e.g., Galton [17] and Edgeworth [11,12] and references therein) discussed the use of other quantiles, such as the median, in various estimation settings. Sheppard [45] and then Pearson [38] studied the problem of optimal quantile selection for the estimation of the mean and standard deviation of the normal distribution by linear functions of subsets of the sample quantiles. Pearson’s paper also contained most of the details involved in the derivation of the asymptotic distribution of a sample quantile. The large-sample behavior of a sample quantile was later investigated

QUANTILE

by Smirnoff [47], who gave a rigorous derivation of its limiting distribution. Smirnoff’s results were generalized in a landmark paper by Mosteller [33], which, along with work by Ogawa [34], generated considerable interest in quantiles as estimation tools in location and scale parameter models. In more recent years quantiles have been utilized in a variety of problems of both classical and robust statistical inference, and have played an important part in the work of Tukey [50] and Parzen [36] on exploratory data analysis∗ and nonparametric data modeling. In this article the focus will be on the role of quantiles in various areas of statistics both as parameters of interest as well as means to other ends. We begin by defining the notion of population and sample quantiles. Let F be a distribution function (DF) for a random variable X and define the associated quantile function (QF) by Q(u) = F −1 (u) = inf {x : F(x)
u}, 0 < u < 1. (1) Thus, for a fixed p in (0, 1), the pth population quantile for X is Q(p). It follows from definition (1) that knowledge of Q is equivalent to knowledge of F. Further relationships between F and Q are (a) FQ(u)
u with equality when F is continuous. (b) QF(x)  x with equality when F is continuous and strictly increasing. (c) F(x)
u if and only if Q(u)  x. Another important property of the QF which follows easily from relationship (c) is that if U has a uniform distribution on [0, 1], then Q(U) and X have identical distributions. This fact provides one of the basic tools in many areas of statistical analysis. For example, in statistical simulation∗ it has the consequences that a random sample from the uniform distribution may be used in conjunction with Q, to obtain a random sample from X. The sample analog of Q is obtained by use of the empirical distribution function (EDF∗ ). Let X1:n , X2:n , . . . , Xn:n denote the order statistics∗ for a random sample of size n from

6697

a distribution F; then the usual empirical estimator of F is  0, x < X1:n ,     j/n, Xj:n  x < Xj+1:n , ˜ (2) F(x) =  j = 1, . . . , n − 1,    1, x
Xn:n . Replacing F with F˜ in (1) gives the sample or empirical quantile function (EQF) ˜ Q(u) = Xj:n ,

j−1 j 0

x 0

1 x 2e ,

1−

(log x) 1 − e−x , x > 0 1 − exp(−xc ) c, x > 0 exp(−e−x ) {1 + e−x }−1 1 − (1 + x)−ν , ν, x > 0 0.5 + π −1 arctan x x, 0  x  1

(x) =

x

Distribution Function

(1 − u)−1 − 1  log 2u, u < 0.5 − log 2(1 − u), u > 0.5

e − log(1 − u) c{− log(1 − u)}1−1/c − log log u1 u log 1−u (1 − u)−1/ν − 1 tan[π (u − 0.5)] u

−1 (u)

−1 (u)

Quantile Function

1 −|x| 2e

(x + 1)−2

e c cxc−1 e−x −x −x −e e e e−x (1 + e−x )−2 ν(1 + x)−(ν+1) [π (1 + x2 )]−1 1

1 x φ(log x) −x

2 /2

φ(x) = (2π )−1/2 e−x

Density Function

Table 1. Distribution, Quantile, Density, and Density-Quantile Functions for Selected Probability Laws

(1 − u)2  u, u < 0.5 1 − u, u > 0.5

φ−1 (u)e−−1 (u) 1−u c(1 − u)[− log(1 − u)]1−1/c −u log(u) u(1 − u) ν(1 − u)1+1/ν π −1 sin2 π u 1

φ−1 (u) = (2π )−1/2 e−|−1 (u)|2 /2

Density-Quantile Function

QUANTILE

and Xj:n or, more generally, several order statistics in the neighborhood of Xj:n to obtain an estimator of Q(p). Such considerations have led to the usual definition for the sample ˜ (0.5) only when median, which agrees with Q n is odd, and have prompted several authors ˜ (see, e.g., to propose linearized versions of Q Parzen [36]). Estimators of Q(p) that utilize local smoothing of the order statistics near Xj:n and appear to have good small-sample properties have been suggested by Kaigh and Lachenbruch [26], Kaigh [25], and Harrell and Davis [20]. Reiss [41] has considered the use of quasi-quantiles and shown them to be superior to sample quantiles when compared on the basis of deficiency rather than efficiency (see EFFICIENCY, SECOND-ORDER). For 0 < p < 1 it is well known (see Serfling [44]) that for f Q positive and continuous ˜ near p, Q(p) is asymptotically normally distributed with mean Q(p) and variance p(1 − p)/{nf Q(p)2 }. An extension of this result to k quantiles, for fixed k
1, can be found in Mosteller [33] and Walker [51], with the case of k growing with n treated by Ikeda and Matsunawa [23]. Necessary and sufficient conditions for the existence of moments for sample quantiles and for the convergence of these moments to those of the limiting distribution are provided by Bickel [4]. For a discussion of the asymptotic properties of ˜ Q(p) for certain types of dependent samples, see Sen [43] and Babu and Singh [1]. ˜ From the asymptotic distribution of Q(p) an asymptotic 100(1 − α)% confidence interval for Q(p) is given by    ˜ Q(p) ± −1 (α/2) p(1 − p)/ nf Q(p)2 , which, unfortunately, requires knowledge of f Q(p). This difficulty can be resolved by using ˜ 1 /n), Q(k ˜ 2 /n)), where instead the interval (Q(k k1 and k2 are integers chosen so that  k1 np − −1 (α/2) np(1 − p) and  k2 np + −1 (α/2) np(1 − p). The latter interval is asymptotically equivalent to the former but utilizes the asymptotic

6699

˜ ˜ i /n) and Q(p) to estirelationship between Q(k mate f Q(p) (see Serfling [44, p. 103]). An alternative but similar approach is given in Walker [51]. For an exact confidence interval based on order statistics, see Wilks [55, p. 329]. Interval estimates obtained by bootstrapping and jackknifing have been proposed by Harrel and Davis [20] and Kaigh [25]. In testing hypotheses about Q(p) the most widely known procedure is probably the quantile test, based on the fact that if H0 : Q(p) = Q0 (p) is true, then the number of sample quantiles below or equal to Q0 (p) will be binomial with parameters n and p. As a result, the binomial distribution may be utilized to obtain an exact test, or the normal approximation to the binomial for an approximate test, of H0 . The quantile test, as well as several other tests concerning the median, can be found in standard texts such as Conover [8]. From a data modeling perspective, what is of interest is not Q(p) for some particular p, but rather the entire function Q(·), as its knowledge is equivalent to knowing the data’s underlying probability law. Thus we now consider the construction of nonparaˆ that are random funcmetric estimators Q(·) tions or stochastic processes∗ on (0, 1) (this is the quantile domain analog of nonparametric probability distribution and density estimation∗ ). The natural estimator of Q(·) ˜ is Q(·), whose asymptotic distribution theory, when considered as a stochastic process, has been studied by Shorack [46], Cs¨org¨o and R´ev´esz [10], Cs¨org¨o [9], Mason [32], and others. From their work it follows that when f Q is positive and differentiable on [0, 1] and satisfies certain other regularity conditions near √ ˜ − Q(u)} converges in 0 and 1, nf Q(u){Q(u) distribution to a Brownian bridge∗ process on (0, 1), that is, a zero-mean normal process with covariance kernel K(u, υ) = u − uυ, u  υ (analogous results for a linearized version ˜ and for the case of randomly censored of Q data can be found in Bickel [4], Sander [42], and Cs¨org¨o [9]). Tests in this setting are of the ∗ goodness-of-fit variety. The asymptotic distribution of many classical statistics, such as ˜ − Q0 (u)|, are available sup0 0, the pth quantile is −θ ln(1 − p) = υp θ (say). Robertson [19] has shown that the estimator with minimum mean square error in predicted distribution functions is given by KX, where

DISTRIBUTION-SPECIFIC QUANTILE ESTIMATORS

For the two-parameter exponential distribution with PDF θ −1 e−(x−λ)/θ ; x > λ; θ , λ > 0, the pth quantile is λ + υp θ . An estimator of the pth quantile is

General Estimators Kubat and Epstein [11] propose linear asymptotically unbiased estimators of Tp based on two or three order statistics selected in a neighborhood of the sample quantile. The method requires knowledge of the form of the distribution F, but the location and scale parameters are unknown.

A minimum variance unbiased estimator∗ for the pth quantile of the normal distribution∗ is given by Q5 (p) = X + ksZp , where 

n−1 2

1/2



  1 (X − X1 : n ). Q5 (p) = X1 : n + Vp + n Rukhin and Strawderman [20] show that Q5 (p) is inadmissible for most practical situations and develop an improved estimator Q6 (p) = Q5 (p) − 2Un /(n + 1),

The Normal Distribution

k=

K = n[eυp /(n+1) − 1]/[2 − eυp /(n+1) ].

 " # n n−1 . 2 2

Un = (b − 1 − n−1 )(X − X1 : n ) −(bn − 1)X1 : n . Other Distributions For the Weibull distribution∗ Lawless [12], Mann and Fertig [14,15], and Schafer and Angus [22] have proposed relevant estimators. Angus and Schafer [2] and Eubank [6] discuss estimation of the

QUANTILE FUNCTION, MEDIAN-ORIENTED

quantiles from the logistic distribution∗ , Ali et al. [1] from a double-exponential distribution, Umbach et al. [28] from the Pareto distribution∗ , and Lawless [12] and Mann and Fertig [15] from the extreme-value distribution∗ . Wheeler [29] estimates parameters of Johnson∗ curves.

REFERENCES 1. Ali, M. M., Umbach, D., and Hassanein, K. M. (1981). Commun. Statist. A, 10, 1921–1932. 2. Angus, J. E. and Schafer, R. E. (1979). Commun. Statist. A, 8, 1271–1284. 3. Breth, M. 207–211.

(1980).

Aust.

J. Statist.,

22,

4. David, H. A. (1981). Order Statistics, 2nd ed. Wiley, New York. 5. Efron, B. (1981). J. Amer. Statist. Ass., 76, 312–319. 6. Eubank, R. L. 229–236.

(1981).

Scand.

Actu.

J.,

7. Harrell, F. E., Jr. and Davis, C. E. (1982). Biometrika, 69, 635–640. 8. Kaigh, W. D. (1983). Commun. Statist. Theor. Meth, 12, 2427–2443. 9. Kaigh, W. D. and Lachenbruch, P. A. (1982). Commun. Statist. Theor. Meth., 11, 2217–2238. 10. Krewski, D. (1976). J. Amer. Statist. Ass., 71, 420–422. 11. Kubat, P. and Epstein, B. (1980). Technometrics, 22, 575–581. 12. Lawless, J. F. 255–261.

(1975).

Technometrics,

17,

13. Loynes, R. M. (1966). J. R. Statist. Soc. B, 28, 497–512. 14. Mann, N. R. and Fertig, K. W. (1975). Technometrics, 17, 361–368. 15. Mann, N. R. and Fertig, K. W. (1977). Technometrics, 19, 87–93. 16. McCarthy, P. J. (1965). J. Amer. Statist. Ass., 60, 772–783. 17. Owen, D. B. (1968). Technometrics, 10, 445– 478. 18. Reiss, R. D. and Ruschendorf, L. J. Amer. Statist. Ass., 71, 940–944.

(1976).

19. Robertson, C. A. (1977). J. Amer. Statist. Ass., 72, 162–164. 20. Rukhin, A. L. and Strawderman, W. E. (1982). J. Amer. Statist. Ass., 77, 159–162.

6707

21. Sarhan, A. E. and Greenberg, B. G., eds. (1962). Contributions to Order Statistics. Wiley, New York, pp. 383–390. 22. Schafer, R. E. and Angus, J. E. (1979). Technometrics, 21, 367–370. 23. Scheff´e, H. and Tukey, J. W. (1945). Ann. Math. Statist., 16, 187–192. 24. Schmeiser, B. W. (1977). Commun. Statist. B, 6, 221–234. 25. Sedransk, J. and Meyer, J. (1978). J. R. Statist. Soc. B, 40, 239–252. 26. Steinberg, S. M. (1983). Confidence Intervals for Functions of Quantiles Using Linear Combinations of Order Statistics. Unpublished Ph.D. dissertation, University of North Carolina, Chapel Hill, N.C. 27. Thompson, W. R. (1936). Ann. Math. Statist., 7, 122–128. 28. Umbach, D., Ali, M. M., and Hassanein, K. M. (1981). Commun. Statist. A, 10, 1933–1941. 29. Wheeler, R. E. (1980). Biometrika, 67, 725–728. 30. Wilks, S. S. (1962). Mathematical Statistics. Wiley, New York. See also L-STATISTICS; MEDIAN ESTIMATION —I; MEDIAN ESTIMATION —II; ORDER STATISTICS; and QUANTILE.

C. E. DAVIS S. M. STEINBERG

QUANTILE FUNCTION, MEDIAN-ORIENTED This concept was introduced [1] as a tool for generalizing univariate medians and testing numerous multivariate extensions of medians and quantiles. For a univariate distribution with cdf F(·) and median M, a median-oriented quantile function Q(u, p) is defined, with u = ±1 denoting direction from M, by Q(u, 0) ≡ M, and satisfies the following relations: Q(−1, p) = F −1 ((1 − p)/2); Q(+1, p) = F −1 (1 − (1 − p)/2). Here F −1 (p) denotes the usual pth quantile of the cdf F. Other properties are: 1. For each fixed p[0, 1), the set {Q(u, t) : 0  t  p, all u} comprises a pth quantile inner region with boundary points

6708

QUANTILE PROCESSES

Q(u, p) and with Q(u, 0) ≡ M (the median). 2. For each fixed direction u from M, the distance ||Q(u, p) − M|| increases with p, 0  p  1.

Let Q be the quantile function of the RV X, defined to be the left-continuous inverse of F, Q(y) : = inf x{x ∈ R : F(x)
y}, Q(0) : = Q(0+),

To extend Q(u, p) to the multivariate context, one should choose a particular multivariate median* M (see ref. 2 for a wide menu of possible choices). Given a family A = {Aγ , 0  γ < ∞} of regions nested about M, including {M} and such that Aγ ⊂ Aγ  for 0  γ  γ  , define

0 < y < 1,

Q(1) := Q(1−).

(2)

Q has the fundamental property that for x ∈ R and 0 < y < 1, F(x)
y if and only if Q(y)  x,

(3)

and that

γp = inf {γ : Pr(Aγ )
p}

F(Q(y)−)  y  F(Q(y)).

(i.e., indexing each point on the boundary of Aγ ). Then Q(u, p) is given by the boundary point of Aγp in the direction u from M, and Aγp represents a pth quantile inner region. The boundaries ∂Aγ (contours) provide interpretations of median-oriented quantile functions. The utility and suitability of the pth quantile inner regions {Q(u, t) : 0  t  p, ∀u} depend crucially on the choice of family A and are to some extent a matter of judgment in a particular application.

Consequently, if F is a continuous distribution function, then

(4)

F(Q(y)) = y and Q(y) = inf {x ∈ R : F(x) = y},

0 < y < 1. (5)

Hence the so-called probability integral transformation∗ F : X → F(X) yields If F is continuous, then F(X) is a uniform (0, 1) RV.

REFERENCES

Also, if U is a uniform (0, 1) RV, then by (3) we have

1. Serfling, R. (2002). Quantile functions for multivariate analysis; approaches and applications. Statistica Neerlandica, 56, 214–232. 2. Small, C. G. (1990). A survey of multidimensional medians. Int. Stat. Rev., 58, 263–277. See also MULTIVARIATE QUANTILES, NORM-MINIMIZING.

QUANTILE PROCESSES

where := denotes ‘‘is defined as.’’

P{Q(U)  x} = P{U  F(x)} = F(x) = P{X  x},

x ∈ R,

(7)

for an arbitrary distribution function F of a RV X; that is, the so-called inverse probability integral transformation or quantile transformation Q : U → Q(U) yields D

Let X be a real-valued random variable (RV) with distribution function F, defined to be right-continuous on the real line R; that is, if the underlying probability space is (, A, P), then F(x) := P{ω ∈  : X(ω)  x},

(6)

x ∈ R,

(1)

If U is a uniform (0, 1) RV, then Q(U)=X (8) for any RV X having an arbitrary distribuD tion function F, where ‘‘=’’ means equality in distribution. In this particular example it means that P{Q(U)  x} = P{X  x}, x ∈ R [see (7)]. Given a random sample X1 , X2 , . . . , Xn , n
1, on a RV X with distribution function F and quantile function Q, one of the basic problems

QUANTILE PROCESSES

of statistics is to estimate the latter functions on the basis of the said random sample. Let Fn (x), x ∈ R, be the proportion of those observations of a given random sample of size n
1 which are less than or equal to x. Fn is called the empirical distribution function of X1 , . . . , Xn (see EMPIRICAL DISTRIBUTION FUNCTION (EDF) STATISTICS). An equivalent way of defining Fn is in terms of the order statistics∗ X1 : n  X2 : n  · · ·  Xn : n of a random sample X1 , . . . , Xn , as follows:  0    k/n Fn (x) :=     1

if F is arbitrary, by (5) applied to (11); {Fn (Q(y)); 0 < y < 1, n
1} D

={Gn (y); 0 < y < 1, n
1}

(12)

if F is continuous, by (8); D

{Qn (y); 0 < y < 1, n
1}={Q(Un (y)); 0 < y < 1, n
1}

(13)

if F is arbitrary, and by (5) applied to (13); if X1 : n > x, if Xk : n  x < Xk : n (k = 1, . . . , n − 1), if Xn : n  x, x ∈ R.

D

(9)

Qn (y) := inf {x ∈ R : Fn (x)
y} (k = 1, . . . , n). (10) From a probabilistic point of view, the distribution function F view of the distribution of a RV X is equivalent to its view via the quantile function Q. From a statistical point of view, F and Q represent two natural, complementary views of a distribution. Fn is a picture of (statistic for) F, while Qn is a picture of (statistic for) Q. They are two different pictures (empirics) of the distribution of a RV X, which complement each other, featuring the shape of a distribution from two different angles. Both have their much intertwined but nevertheless distinct distribution theories. To facilitate the description, let U1 , U2 , . . . , Un , n
1, be independent, identically distributed (i.i.d.) uniform (0, 1) RVs with distribution function G(y) = y on [0, 1], empirical distribution function Gn (y), and quantile function Un (y). Then by (3), {Fn (x); x ∈ R, n
1} ={Gn (F(x)); x ∈ R, n
1}

(14)

if F is continuous. The empirical process βn is defined by {βn (x) := n1/2 (Fn (x) − F(x)); x ∈ R, n
1},

(15)

while the uniform empirical process αn is defined by

= Xk : n if (k − 1)/n < y  k/n

{F(Qn (y)); 0 < y < 1, n
1}={Un (y); 0 < y < 1, n
1}

The empirical quantile function Qn of X1 , . . . , Xn is defined to be the left-continuous inverse of the right-continuously defined Fn , that is,

D

6709

{αn (y) := n1/2 (Gn (y) − y); 0  y  1, n
1}.

The uniform quantile process un is defined by {un (y) := n1/2 (Un (y) − y); 0  y  1, n
1},

(17)

and the general quantile process qn by {qn (y) := n1/2 (Qn (y) − Q(y)); 0 < y < 1, n
1}.

(18)

By (11), (15), and (16) we have {βn (x); x ∈ R, n
1} D

={αn (F(x)); x ∈ R, n
1}

(19)

if F is arbitrary, and by (12), (15), and (16), D

{βn (Q(y)); 0 < y < 1, n
1}={αn (y); 0 < y < 1, n
1}

(11)

(16)

if F is continuous.

(20)

6710

QUANTILE PROCESSES

By virtue of (19) and (20), when studying the empirical process βn , we may restrict attention to the uniform empirical process αn . According to (20), the distribution of the empirical process βn does not depend on the form of F as long as the latter is continuous. This is the so-called distribution-free property of βn for continuous F (see Doob [27]). According to (19), the distribution of the empirical process βn does depend on the form of F if the latter is discrete, but it can be still expressed in terms of the distribution of αn (F). See also EMPIRICAL PROCESSES. Unfortunately, there are no simple relationships between un and qn like those of (19) and (20) for αn and βn . By (13) and (18) we have

Tukey [69] calls Q the representing function, and Q the sparsity function. Since un (k/n) = −αn (Uk:n ), where U1:n  U2:n  · · ·  Un:n are the order statistics of a uniform (0, 1) random sample U1 , . . . , Un , it is reasonable to expect that the asymptotic distribution theory of αn and un should be the same. Due to (23), we have 

 f (Q(y))qn (y); 0 < y < 1, n
1  f (Q(y)) D = un (y) ; 0 < y < 1, f (Q(θy,n )) Un (y) ∧ y < θy,n < Un (Y) ∨ y,  n
1 . (24)

{qn (y); 0 < y < 1, n
1} D

={n1/2 (Q(Un (y)) − Q(y)); 0 < y < 1, n
1}

(21)

if F is arbitrary, and by (5), (14), and (17), {un (y); 0 < y < 1, n
1} D

={n

1/2

{ρn (y) := f (Q(y))qn (y); 0 < y < 1, n
1}, (25)

(F(Qn (y)) − F(Q(y)));

0 < y < 1, n
1}

(22)

if F is continuous. Let F now be an absolutely continuous distribution function on R, let f := F  be its density function (with respect to Lebesgue measure), and assume that f (·) > 0 on the support of F. Then any one of (21) and (22) implies, if a ∧ b and a ∨ b denote the minimum and maximum of a and b, respectively, {qn (y); 0 < y < 1, n
1} D

={un (y)/f (Q(θy,n )); 0 < y < 1, Un (y) ∧ y < θy,n < Un (y) ∨ y, n
1}.

This, in turn, suggests that f (Q)qn should have the same kind of asymptotic theory as un if f is ‘‘nice.’’ Hence we introduce ρn , the renormed general quantile process, by defining it to be

(23)

Clearly then, even if F is continuous and also differentiable, the distribution of qn depends on F through f (Q), the so-called density-quantile function of F, or, equivalently, through Q = 1/f (Q), the so-called quantile-density function of F. The latter two terminologies are due to Parzen [49].

and attempt to summarize here the intertwined distribution theories of αn , un , and ρn . First we define several Gaussian processes∗ which play a basic role in approximating the said empirical processes. Wiener Process (Brownian Motion∗ ). A realvalued separable Gaussian process {W(t); t
0} with continuous sample paths is called a Wiener process if EW(t) = 0 and EW(s)W(t) = s ∧ t, s, t
0. Brownian Bridge. {B(s); 0  s  1} := {W(s) − sW(1); 0  s  1}; hence EB(s) = 0 and EB(s)B(y) = s ∧ y − sy, 0  s, y  1.

QUANTILE PROCESSES

Kiefer Process∗ . {K(y, t); 0  y  1, t
0} := {W(y, t) − y(W(1, t); 0  y  1, t
0}, where {W(y, t); y
0, t
0} is a real valued two-time parameter separable Gaussian process with EW(y, t) = 0 and EW(y, t)W(s, l) = (y ∧ s)(t ∧ l)(y, s, t, l
0). Hence EK(y, t) = 0 and EK(y, t)K(s, l) = (t ∧ l)(y ∧ s − ys) (0  y, s  1; t, l
0).

6711

ρn , we must say a few words on strong approximation of αn . The first strong approximation result of αn by a sequence of Brownian bridges Bn was proved by Brillinger [7], using the Skorohod embedding∗ [66] based results of Strassen [67] for partial sums of i.i.d. RVs. A fundamental breakthrough for αn is Theorem 1. (Koml´os et al. [41]). For αn there exists a probability space with a sequence of Brownian bridges Bn and a Kiefer process K(y, t) such that a.s. sup |αn (y) − Bn (y)| = O(n−1/2 log n)

(26)

0y1

and A Kiefer process at integer-valued arguments t = n can be viewed as the partial sum process of a sequence of independent Brownian bridges {Bi (y); 0  y  1}∞ i=1 : {K(y, n); 0  y  1, n = 1, 2, . . .}   n Bi (y); 0  y  1, n = 1, 2, . . . , = i=1

n−1/2 sup sup |k1/2 αk (y) − K(y, k)| 1kn 0y1

a.s. = O(n−1/2 log2 n),

(27)

that is, there exists a positive constant M < ∞ such that on the said probability space lim sup n−1/2 sup |αn (y) − Bn (y)|/ log n  M n→∞

0y1

and

(28)

{Bn (y); 0  y  1}

and

= {K(y, n) − K(y, n − 1); 0  y  1} (n = 1, 2, . . .)

lim sup sup sup |k1/2 αk (y) − K(y, k)|/ log2 n n→∞

1kn 0y1

M

(29)

with probability 1. is a sequence of independent Brownian bridges. For proof of existence and further properties of these Gaussian processes, we refer to Cs˝org˝o and R´ev´esz [18, Chap. 1]. The theory of weak convergence of αn to a Brownian bridge B was initiated by Doob [27], formalized by Donsker [26], Prohorov [54], Skorohod [65], and summarized and further developed in Billingsley [5] and Breiman [6]. See also Dudley [28], Wichura [71], Durbin [29,30], Burke et al. [8], and for a review and further references, see EMPIRICAL PROCESSES and GLIVENKO –CANTELLI THEOREMS, Serfling [57], and Gaenssler and Stute [33]. To facilitate our approach to describing un and

It follows by (20) that Theorem 1 holds true for βn if F is continuous, while (19), sup |αn (F(x)) − Bn (F(x))| x∈R

 sup |αn (y) − Bn (y)|, 0y1

and sup |k1/2 αk (F(x)) − Bn (F(x))| x∈R

 sup |k1/2 αk (y) − K(y, k)| 0y1

together imply that Theorem 1 also holds true for βn if F is arbitrary.

6712

QUANTILE PROCESSES

This fundamental theorem has proved to be of basic importance in a wide and growing range of practical and theoretical problems alike. The monographs by Cs˝org˝o and R´ev´esz [18], and Cs˝org˝o [13] contain an exposition, further developments, and a variety of applications. Recent developments and applications are also reviewed in a paper by Cs˝org˝o and Hall [14]. See also EMPIRICAL PROCESSES and references therein. As to quantile processes, Theorem 1 has inspired

(b) F  (x) = f (x) > 0 on (a, b), (c) for some γ > 0 we have 
|f  (Q(y))|  γ. y(1 − y) 2 f (Q(y)) 0 1, where  > 0 is arbitrary and γ is as in (c). This theorem is a strong invariance∗ principle for ρn in terms of un . For further developments, see Cs˝org˝o et al. [20]. The latter results are reviewed in Cs˝org˝o [13, Sec. 5.4]. When combined with (22) and Theorem 2, it results in a strong invariance principle for

QUANTILE PROCESSES

ρn in terms of the Brownian bridges Bn and the Kiefer process K(y, t) of Theorem 2 (for a discussion of these results, we refer the reader to Cs˝org˝o and R´ev´esz [18, Chaps. 4, 5] and Cs˝org˝o [13, Chap. 3]). It can be used to construct confidence bands for the quantile function Q (see Cs˝org˝o and R´ev´esz [19] and Cs˝org˝o [13, Chap. 4]), for example: under conditions (a) to (c) of Theorem 3 we have

6713

(for a preview of the latter results, see Cs˝org˝o [13, Chap. 8] and references therein). The modern theory of sample quantiles was initiated by Bahadur [3], who, in terms of our notation, studied the following representation of the yth sample quantile Qn (y): Qn (y) = Q(y) +

$ lim P Qn (y − n−1/2 c)

1 − Fn (Q(y)) − (1 − y) f (Q(y)) + Rn (y),

(33)

n→∞

as a stochastic process∗ in n for fixed y ∈ (0, 1). For an excellent review of sample quantiles Qn (y) vs. Q(y) for y ∈ (0, 1) fixed, see Serfling [57, Secs. 2.3.1 to 2.6.6]. On assuming (a) and (b) of Theorem 3, and continuing along the lines of (24) and (25) we get

 Q(y)  Qn (y + n−1/2 c); % n  y  1 − n   =P

sup |B(y)|  c 0y1

=1−



2 c2

(−1)k+1 e−2k

,

c > 0,

(32)

k=0

ρn (y)/n1/2

where {n } is such that n1/2 n → ∞(n → ∞), and B(·) is a Brownian bridge. Notice that the confidence band of (32) for Q(·) on the interval [n , 1 − n ] does not require the estimation of the unknown density quantile function f (Q(·)). A word about condition (c) of Theorem 3. In the literature of nonparametric statistics it is customary to define the so-called score ˘ ak ´ [35, p. function (see, e.g., Hajek and Sid 19]): J(y) = −f  (Q(y))/f (Q(y)) d = − f (Q(y)), dy

=


 un (y) un (y) f (Q(y)) + − 1 n1/2 n1/2 f (Q(θy,n ))

=

un (y) un (y) y − θy,n f  (Q(δy,n )) + 1/2 n1/2 n f (Q(θy,n )) f (Q(δy,n ))

:= n−1/2 un (y) + n−1/2 n (y),

(34)

where θy,n ∧ y < δy,n < θy,n ∨ y with θy,n ∈ (Un (y) ∧ y, Un (y) ∨ y). Consequently, by (33) and (34), we get f (Q(y))Rn (y) = n−1/2 un (y) + n−1/2 αn (y) + n−1/2 n (y)

(35)

0 < y < 1.

Hence the said condition (c) can be viewed as a condition on the score function J and the quantile-density function Q = 1/f (Q). For a review of estimates of Q (respectively, J), see Cs˝org˝o [13, Sec. 4.1 and Chap. 10]. For examples and an excellent discussion of tail monotonicity assumptions of extreme-value theory as related to condition (c), see Parzen [49, Sec. 9] and Parzen [53]. Theorem 3 as well as the statement of (32) were recently extended to quantile-quantile plots by Aly and Bleuer [1], and to the quantile process of the product-limit estimator (see KAPLAN –MEIER ESTIMATOR) by Aly et al. [1,2]

with n−1/2 n (y) = n−1/2 (ρn (y) − un (y)).

(36)

Let R∗n (y) := n−1/2 (un (y) + αn (y)). Then by (35) and (36), we get f (Q(y))Rn (y) − R∗n (y) = n−1/2 n (y) = n−1/2 (ρn (y) − un (y)), y ∈ (0, 1).

(37)

Hence, given the conditions of Theorem 3, we have (see Cs˝org˝o and R´ev´esz [16,18], and

6714

QUANTILE PROCESSES

Cs˝org˝o [13]) sup |f (Q(y))Rn (y) − R∗n (y)|

0y1

 O(n−1 log log n)     if (e, α) obtains, or γ < 1, = −1 (log log n)2 )  if γ = 1, O(n    O(n−1 (log log n)(log n(1+)(γ −1) ) if γ > 1,

a Glivenko–Cantelli theorem for ρn under the conditions of Theorem 3. Mason [44] proved: Let ν > 0 be fixed, and assume that Q is continuous. Then we have lim sup sup (y(1 − y))ν |Qn (y) − Q(y)| n→∞

a.s. =

(38) where  > 0 is arbitrary and γ is as in (c). The latter result is a strong invariance statement for the two-parameter stochastic process {f (Q(y))Rn (y); 0 < y < 1, n = 1, 2, . . .} in terms of {R∗n (y); 0 < y < 1, n = 1, 2, , . . . , }, provided, of course, that one can prove an almost sure rate of convergence to zero of the latter remainder term so that, via (39), it should be inherited by the former. The final answer concerning R∗n was given by Kiefer [40]:  lim sup sup |R∗n (y)|n3/4 (log n)−1/2 n→∞

0y1

×(log log n)−1/4



a.s. −1/4 =2 .

(40)

This, in turn, when combined with (38), gives [16]  lim sup sup |f (Q(y))Rn (y)|n3/4 n→∞

0y1

×(log n)−1/2 (log log n)−1/4 a.s. −1/4 =2

 (41)

under the conditions of Theorem 3, and thus extends Kiefer’s [40] theory of deviations between the sample quantile and empirical processes (for a detailed discussion, see Cs˝org˝o [13, Chap. 6]). Shorack [62] showed that (40) itself can be also deduced from strong invariance considerations. a.s. From sup0y1 n−1/2 |un (y)| → 0(n → ∞), an obvious Glivenko–Cantelli theorem∗ for the uniform quantile process un , by Theorem 3 we can also conclude that sup n−1/2 |ρn (y)|

0y1

a.s. = sup f (Q(y))|Qn (y) − Q(y)| → 0(n → ∞), 0y1

0y1

   0

 if

  ∞ if

1

0 1

|Q(y)|1/ν dy < ∞, |Q(y)|1/ν dy = ∞.

0

The latter is a very nice Glivenko–Cantelli theorem for the quantile process, replacing the tail conditions of Theorem 3 by the moment conditions 

1 0

 |Q(y)|1/ν dy =

∞

−∞

|x|1/ν dF(x) < ∞.

Mason [44] also discusses the relationship between his moment condition and those of Theorem 3. For applications of Theorem 3, see Cs˝org˝o [13, Chaps. 7 to 10] and references therein to topics on quadratic forms∗ of the quantile process of the product-limit estimator, nearestneighbor empirical density, and score functions. For related results on nearest-neighbor density function estimators, we refer the reader to Moore and Yackel [47] and references therein. With the sequence of Brownian bridges Bn of Theorem 2 and under the conditions of Theorem 3, we have sup |ρn (y) − Bn (y)| = oP (1)

0y1

(n → ∞), (42)

and hence the weak convergence of ρn (·) to a Brownian bridge B(·) in Skorohod’s D [0, 1] space also follows. For a description of the latter space, we refer the reader to Billingsley [5, Chap. 3]. On the other hand, Theorems 2 and 3 were proved having mainly strong approximations in mind. It is of independent interest to see if (42) could be true under milder conditions of f = F  than those of Theorem 3. Toward this end we first note that an improved construction for approximating the uniform quantile process un (see Cs˝org˝o et al. [23, Paper 1, Sec. 1]) leads to the following common

QUANTILE PROCESSES

generalization of (26) and (30) of Theorems 1 and 2, respectively: Theorem 4. (Cs˝org˝o et al. [22]). For un and αn there exists a probability space with a sequence of Brownian bridges Bn such that for every 0 < λ < ∞ as n → ∞, sup λ/ny1−λ/n

nν |(−1)un (y) − Bn (y)| (y(1 − y))(1/2)−ν

and nν |αn (y) − Bn (y)| (1/2)−ν λ/ny1−λ/n (y(1 − y))  Op (log n) when ν = 14 , = (44) when 0  ν < 14 . Op (1) sup

For the far-reaching implications of this theorem, see the two papers in Cs˝org˝o et al. [23] and the first four papers in Cs˝org˝o et al. [24]. Here we illustrate only its immediate usefulness in studying ρn . Let p be any such positive function on (0, 1) for which lim p(s) = lim p(s) = ∞, s↓0

s↑1

and sup 0

xp∼h/(4π ).

(3)

(The additional factor of 4π results from considering pulse shapes which yield a minimum lower bound on the uncertainty product.) 4. Because Planck’s constant h is very small, the de Broglie waves characterizing macroscopic objects are extremely short—far smaller even than nuclear dimensions. Our senses are so coarse that they average over many wave crests, thus destroying all interference phenomena we might otherwise observe. This is why quantum mechanics plays no role in our normal experience.

Figure 1. Pulse of finite duration.

QUANTUM MECHANICS AND PROBABILITY

The central innovation in quantum mechanics was the introduction of a wavefunction, that is, a complex-valued function of coordinates and time, whose evolution in time describes the dynamics of the system to which it corresponds. The time evolution of the wavefunction is determined by a first-order partial differential equation   2π ∂ψ = −i Hψ, (4) ∂t h where i is the square root of −1 and H is a differential operator which can depend on coordinates, gradient operators, and even time (the time dependence of H occurs with systems perturbed by external forces that vary with time). The operator H is a representation of the classical Hamiltonian of the system, which is constructed according to certain rules: The chief of these is that canonical momenta p are replaced by derivatives with respect to the corresponding canonical coordinates q via the identification h ∂ . p → −i 2π ∂q

(5)

(This rule must be applied with discretion, however, because in curvilinear coordinates, or when describing rigid-body rotation, its naive application leads to ambiguities [15].) For the sake of completeness, note that it is possible to regard the wavefunctions as abstract vectors in a Hilbert space, and the Hamiltonian as an abstract operator on that space. The conversion from an abstract vector to a wavefunction which depends on coordinates is then a form of unitary transformation from one representation to another. This is the procedure followed in modern quantum mechanics texts. One of the most useful signposts in the unfamiliar terrain of quantum physics was the correspondence principle, which required that in the limit of macroscopic systems, quantum dynamics must lead to classical mechanics (Newton’s laws). For this to happen, it was necessary to reinterpret the idea of a particle’s position x(t) in terms of an expectation value; that is, only the expectation x(t), defined in terms of the wavefunction ψ(x, t) by  (6) x(t) = dxψ ∗ (x, t)xψ(x, t),

6729

has physical significance. If we take the Hamiltonian of a particle moving along one spatial axis under the influence of the potential V(x) to be  H=−

h 2π

2

∂2 + V(x), ∂x2

(7)

then it is easy to see, by differentiating (6) with respect to time and then applying (4) and its complex conjugate, that d x(t) = i dt



dxψ ∗ (x, t)(Hx − xH)ψ(x, t) (8)

and hence that d2 x(t) = − dt2



dxψ ∗ (x, t)[H(Hx − xH)

− (Hx − xH)] × ψ(x, t)

(9)

or d2 −1 x(t) = dt2 m



dxψ ∗ (x, t)

∂V(x) ψ(x, t). ∂x (10)

Now, the force acting on a particle is just the negative gradient of its potential energy. In general, the right side of (10) involves the expectation of all moments of x. However, for macroscopic objects the wavelength is typically so small that the quantum fluctuations of position and momentum are also small in comparison with their expected values. Hence it is permissible to replace −V  (x) by −V  (x), thereby obtaining Newton’s law of motion. (This result is known as Ehrenfest’s theorem.) To summarize, when we interpret the position of a particle in terms of (6), we obtain the classical equations of motion in the limit that fluctuations are ignored. (This is, of course, the sense in which any stochastic process yields a deterministic equation of motion.) The physical content of this mathematical formalism is simply that our classical picture of the trajectory of a particle acted on by an external force requires simultaneously measuring both the position and the momentum of the particle at successive (short) time intervals. For macroscopic objects the measurement process introduces only negligibly small perturbations of the values of these

6730

QUANTUM MECHANICS AND PROBABILITY

quantities relative to their mean values, so Newtonian physics adequately approximates the dynamics of such objects. Like all successful physical theories, wave mechanics immediately generated more problems than it solved. The chief of these was how to interpret the ‘‘wavefunction’’ associated with the particles, which we obtain by solving the Schr¨odinger wave equation. It would be an overstatement to suggest that the interpretation proposed by Bohr, Born, and others (the so-called Copenhagen school of quantum mechanics) [2,19] and which today is the predominant way of looking at things, was immediately accepted by acclamation. Einstein steadfastly refused to accept the probabilistic interpretation of quantum mechanics all his life, a refusal based on his intuition that ‘‘God does not play dice!’’ Various authors have attempted to find logical paradoxes in the Copenhagen interpretation [10], or to find mechanistic theories of a ‘‘guiding wave’’ that would retain the wave equation for particles while reintroducing determinism at the microscopic level [5]. All such attempts have ended in failure. PROBABILISTIC INTERPRETATION The idea that certain physical phenomena must be treated in a probabilistic fashion was certainly not new in the 1920s and 1930s, and certainly not unique to quantum mechanics. For example, by about 1900 all of thermodynamics had been placed on a statistical basis through the efforts of Maxwell∗ , Boltzmann∗ , and Gibbs. There are two crucial differences between the use of probability in quantum mechanics and in classical physics, and the exploration of these differences will comprise the remainder of this section. To appreciate the role of probability in quantum physics, we first examine how it is employed in classical physics. Here probability is applicable to systems whose final states are highly unstable with regard to small variations in the initial conditions. Thus a coin toss or the fall of a die is generally regarded as a random event because minute variations in the initial conditions (e.g., at the level of environmental noise) will considerably alter the outcome. As long as the variability of

the outcomes with respect to small variations in the initial conditions is sufficiently chaotic (even though entirely deterministic) in the sense that standard statistical tests for the absence of correlation∗ are satisfied (thereby rendering prediction of the outcome effectively impossible), we are entitled to regard such events as random. The same justification can be invoked for statistical methods in classical thermodynamics or fluid mechanics. Even though the microscopic constituents are supposed to obey deterministic and time-reversal-invariant laws of motion, large aggregates of such constituents become enormously sensitive to small variations in initial state; hence the ‘‘equal likelihood’’ hypothesis for the statistical ensembles of thermal physics makes good sense. The probabilistic interpretation of quantum mechanics has an entirely different conceptual foundation. The mathematical objects that we interpret as probability distributions are all that can ever be known, in principle, about a system. That is, the entire goal of quantum dynamics is to calculate the probability distribution of the outcomes of a particular experiment [1]. The fact that we deal with probabilities is not the reflection of lack of detailed information about the initial state; there simply is no other information to be had. (The Copenhagen interpretation contains certain subtleties and logical difficulties; the very notion of ‘‘experiment’’ presupposes an external ‘‘observer’’ who performs measurements using macroscopic instruments. This dichotomy between the quantum system being observed and the classical external observer raises profound questions about epistemology, the nature of ‘‘consciousness,’’ and similar philosophical concerns [11]. We shall return to these problems in the next section.) The second way that quantum and classical ideas of probability differ is in the composition of probabilities. Here we encounter the problems that early workers in the field found paradoxical and confusing, because it is at this point in the theory that the question of ‘‘wave–particle duality’’ enters. As Feynman has emphasized [13], the central axiom of quantum mechanics is that to each physical process, representing the transition of a system from an initial state A to a final state

QUANTUM MECHANICS AND PROBABILITY

Z, there corresponds a complex number, the probability amplitude. The probability that the system makes the transition in question is the squared modulus of the probability amplitude. The law of composition of probabilities is then as follows: If there is more than one way that a system can make a transition from initial state A to final state Z, say by passing through intermediate states B, C, D, and so on, then as long as we make no measurement to determine which of the intermediate states the system goes through, the total amplitude for the transition A to Z is the sum of the individual amplitudes involving the various intermediate states. The corresponding probability is the squared modulus of the total amplitude. Conversely, if we determine by some measurement that the system has gone through, say, intermediate state D, then the total amplitude is simply the partial amplitude corresponding only to that intermediate state, and the corresponding probability of the squared modulus of that partial amplitude.

To see what this law means in a specific example, consider the apparatus shown in Fig. 2, where electrons from a point source fall on a fluorescent screen after passing through two narrow parallel slits in an

6731

opaque screen (the slits are of width a and are separated by a distance d). Clearly, an electron emitted by filament A can pass through either slit B or slit C and end up somewhere on the detecting screen Z. The amplitude ψ(x, y, z, t) for an electron to propagate from slit B or C to screen Z is obtained by solving the free-particle Schr¨odinger equation h ∂ψ =i ∇ 2ψ ∂t 4mπ

(11)

and is given for θ = arctan(x/z) by φ(θ ) = const. exp[−ih2 k2 t/(8mπ 2 )]| cos θ |1/2     kd ka × cos sin θ sin sin θ sin θ , 2 2 (12) where x is the vertical distance along the screen. If we make no attempt to determine whether the electron has gone through slit B or slit C, we must add the amplitudes and square them, thereby obtaining a classical two-slit interference pattern (Fig. 3). By direct experiment, namely, by adjusting the electron current to be so low that only one electron at a time is in the apparatus, we see that the observed interference pattern

Figure 2. Electron beam incident on two narrow parallel slits.

6732

QUANTUM MECHANICS AND PROBABILITY

Figure 3. Two-slit interference pattern obtained with both slits open.

is a probability distribution—the individual electrons fall randomly, but after enough have impinged on screen Z, the interference pattern emerges. We also see from this experiment that the interference of the amplitudes corresponding to the two slits is entirely characteristic of electrons taken one at a time. That is, it is not some sort of collective effect depending on the mutual interaction of swarms of electrons. This buildup of the interference pattern from individual events in a probabilistic manner was actually first proposed for photons from a weak light source as early as 1909 [20], and represented one of the cornerstones of the probabilistic interpretation. (The experiment actually failed to see the effect because the photographic emulsions available in 1909 were too ‘‘slow.’’ Rather, it was used to place an upper limit on Planck’s constant h.) What happens if we try to determine which of the two slits the electron passed through? Suppose that we cover first slit B, then C: we obtain patterns like those in Fig. 4a, given by the squares of each of the corresponding amplitudes. Their sum, Fig. 4b, exhibits no interference pattern. But wait a moment: What if instead of blocking off the electrons (so that we guaranteed which slit they had gone through) we had merely placed a light behind one of the slits, so that if an electron

went through it would scatter some light and we would see a flash? This situation can be analyzed precisely, but to avoid excessive detail we simply state the result in qualitative terms: If the light is sufficiently intense that there is a high probability that an electron passing through that slit will scatter a photon, and if the wavelength of the photon is sufficiently short that we can resolve which of the two slits the electron passed through (this means that the wavelength must be somewhat smaller than the spacing of the slits), then a random amount of momentum will be transferred to that electron [7]. This will effectively average the two-slit interference pattern over a distance large compared with its oscillations, thereby causing it to look precisely like the sum of the two individual slit patterns and exhibiting no interference. In other words, quantum mechanics predicts that if we try to tell, by any means whatsoever, which slit the electron goes through, we destroy the two-slit interference pattern. Moreover, because the interference between the two slits averages to zero when integrated along the plane of the fluorescent screen, the total probability remains unity. That is, the electron goes somewhere. There is one further situation that we must consider when speaking of the quantum law for composition of probabilities, because

QUANTUM MECHANICS AND PROBABILITY

6733

Figure 4. (a) Results from one slit at a time; (b) the sum of the intensity patterns from slits B and C taken separately. This is also what is observed if any attempt is made to determine which slit the electrons pass through.

we shall need the idea it embodies in the next section. Suppose that it were possible to determine which intermediate state a system passed through, even long after the experiment is finished. For example, electrons possess intrinsic spin and therefore (for reasons that go beyond the scope of this article) can be polarized either ‘‘up’’ or ‘‘down’’ with respect to some definite direction. What if slit B contained a polarizing window, so that it permitted only ‘‘up’’ electrons to pass through it, and what if slit C were similarly equipped and passed only ‘‘down’’ electrons? By looking at the individual electrons reaching Z to see whether they are polarized ‘‘up’’ or ‘‘down,’’ we can tell immediately whether they went through slit B or C. The answer is

that the resulting pattern exhibits no interference, because this is essentially the same experiment as having closed off one slit and then the other. Even if we do not actually measure the polarization at Z, there is no interference pattern. To summarize, an essential difference between classical ideas of probability and the interpretation of the experimental results of quantum physics is the law of composition of probabilities of disjoint trials. Classically, an electron can go through slit B or slit C, and the resulting probability is the sum of the probabilities of the experiments conducted independently (since the slits do not interact with each other). We find that the data compel us to add the amplitudes of disjoint trials

6734

QUANTUM MECHANICS AND PROBABILITY

and then to compute the total probability as the squared modulus of the sum. For macroscopic systems, as emphasized above, the wavelength associated with the quantum interference effects becomes small even compared with nuclear dimensions. Thus any random disturbance of the system will average the quantum interference pattern over many oscillations, washing it out. This is how quantum mechanical laws of probability approach classical ones in macroscopic systems. THE ROLE OF THE OBSERVER The chief concern of physics is the analysis of dynamical processes, that is, processes that evolve in time. The mathematical description of quantum phenomena implied by (4) includes dynamics in the following way. We are given an initial amplitude ψ0 whose squared modulus is the probability distribution of the system variables at time t = 0 (the beginning of the experiment). Time evolution then proceeds according to the Schr¨odinger equation, which is of first order in the time variable, so that the initial condition specifies completely the subsequent probability distribution of the system. As long as we are content to regard the wavefunction as a probability amplitude and to ascribe no further physical significance to it, we encounter no problems of interpretation. If we wish to ascribe an independent physical reality to the wavefunction of a particle or of a complex system, all sorts of philosophical difficulties arise. To make this discussion precise, it is important to define clearly what we mean by ‘‘observation.’’ Implicit in the definition is the idea of ‘‘observables,’’ by which we mean numerical quantities associated with quantum systems, and to which we give the names ‘‘energy,’’ ‘‘position,’’ ‘‘charge,’’ ‘‘mass,’’ ‘‘momentum,’’ or ‘‘intrinsic angular momentum’’ (spin), or the vector component of spin along a particular direction. There is by no means universal agreement as to why nature singles out these entities as the appropriate observables. For the most part observables are the extension to the microworld of concepts which apply to macroscopic physics,

although some (such as ‘‘isospin,’’ ‘‘hypercharge,’’ ‘‘flavor,’’ or ‘‘color’’) are picturesquely named abstractions with no macroscopic counterparts. With each of these observables we associate a Hermitian linear operator defined on the linear vector space of wavefunctions, whose expectation value [in the sense of (6)] is what we actually measure. Some measurements always yield a definite value of the observable (i.e., with no statistical fluctuation). For this to happen, it is necessary that the wavefunction representing the state of the system at that time be an eigenfunction of the operator representing the observable. To state this more mathematically, if (O − O)2  ≡ O2  − O2 = 0

(13)

then by virtue of the Cauchy–Bunyakovsky– Schwartz inequality [i.e., equality holds in (13) if and only if the Hilbert-space vectors ψ and Oψ are proportional to each other], ψ is an eigenvector of O. Now quantum mechanics as described above gives a perfectly unequivocal description of the evolution of a wavefunction as it interacts with an experimental apparatus. Consider the following experiment. Electrons from an unpolarized source are incident on a polarization ‘‘filter’’ whose function is to produce a spatial or temporal separation between those electrons with spin ‘‘up’’ and those with spin ‘‘down.’’ [Such a filter might be a Stern–Gerlach apparatus, for example; or more schematically, a spindependent potential barrier of the form V(x) = (1 + 2sz )V0 θ ((a/2)2 − x2 ), where V0 can be as large as we wish.] Clearly, with such an apparatus we can be sure that all the electrons that emerge are in an eigenstate of the z-component of their spin operator, and thereby possess a definite value for this observable. Now, to calculate the results of an experiment where we send a beam of particles of unknown (random) spin orientations into a polarization filter, we first construct a wave packet which is a superposition of plane waves of all wavelengths (representing solutions of the Schr¨odinger equation in the absence of interaction). This packet is to be fairly sharply peaked about some definite momentum, so that it represents a fairly

QUANTUM MECHANICS AND PROBABILITY

monochromatic beam. It will propagate at the (group) velocity υg =

dω(k) dk

(14)

and, because the phase velocity depends on wavelength, it will eventually disperse (we shall return to this point) and broaden. The packet will also have to represent particles with a definite spin orientation with respect to the axis singled out by the filter. The condition that the beam be unpolarized (or that it have some definite proportion with spins up and the remainder with spins down) is put in at the end when we calculate probabilities. We calculate what happens to the particles with spins up and multiply by the fraction with that orientation, and add the result to the corresponding quantity for the spin-down particles, since as mentioned in the section ‘‘Probabilistic Interpretation,’’ such probabilities are disjoint. The evolution of the wave packet now proceeds as follows. A packet representing an ‘‘up’’ particle experiences no interaction and so propagates through the apparatus undisturbed. A ‘‘down’’ packet, conversely, gives rise to two packets which, a long time after the incident packet entered the apparatus, look like a transmitted packet (perhaps with some time delay built in) with a very small amplitude (the transmission coefficient is assumed to be small) and a reflected packet moving in the opposite direction from the original, with an amplitude nearly the same as in the incident packet. We can easily see what happens to a packet representing a superposition of up and down particles (this is not the same as a probability distribution of orientations in the beam, but rather represents particles with a definite orientation along some direction which does not coincide with the axis of the filter): the up component propagates undisturbed while the down component is almost completely reflected. Hence after a long time a packet which is a superposition of up and down is separated into two disjoint packets which have no spatial overlap, each of which is essentially pure in spin orientation with respect to the preferred axis of the filter. Thus when light is incident on a polaroid, we can be sure that all the photons that pass through have a definite linear polarization.

6735

When we observe a photon hitting a photographic emulsion, we do not observe either its wavefunction or its probability of arriving in a particular spot. We see rather that it has either arrived, or not arrived, at that spot. Thus our act of observing the photon has apparently caused its wavefunction to ‘‘collapse’’ to a state with a definite value of its position (or at least, to a superposition of states whose position observables are confined to a small interval), in the sense that subsequently that photon’s state will evolve from the initial condition that at the time of observation it definitely appeared where it exposed the silver grain in the photograph. The mysterious aspect of the observation process is how the probability gets converted into a definite outcome. For example, should the observer be included in the total wavefunction of the system? It is sometimes said that the wavefunction is in some sense a measure of our knowledge of the system, and that it is not therefore surprising that a packet should disperse, since this simply represents the expected decay of knowledge with time. Alternatively, one might consider the wavefunction to have an independent physical existence, perhaps something like a field strength. In this view there is a pronounced asymmetry between the observer and that which is observed. The interaction between them must be highly nonlinear and essentially discontinuous in order to produce the ‘‘collapse’’ of the wavefunction. But because quantum mechanics is an inherently linear theory, the process of observation must be somehow ‘‘special.’’ Some authors [16,23] have gone so far as to suggest that the special role of the observer is the result of (or the cause of—they are not specific) consciousness and/or self-awareness. Others [6,12,22] have proposed that each observation that converts a probability into an outcome is accompanied by a bifurcation in the physical universe, so that alternate worlds are generated in one of which, for example, a photon went through the apparatus, whereas in the other it was reflected. Given the rate of events on the microscopic scale, the rate of universe generation must dwarf the term ‘‘astronomical.’’ All evidence presently available supports the idea that observations could as well be conducted by simple counting apparatus as

6736

QUANTUM MECHANICS AND PROBABILITY

by sentient beings. If we want to suppose that the image on a photograph has no existence (i.e., that the wavefunction of photons cum reduced silver grains has not collapsed) until a self-aware human being has looked at it, we venture onto the very thin ice of solipsism. One or another of the mystical viewpoints of the Berkelian variety described above may, after all, be correct. However, it is as reasonable to suppose that they arise out of semantic confusion between the words ‘‘intelligence,’’ ‘‘observation,’’ and ‘‘knowledge’’ and therefore represent logic as circular as St. Anselm’s ‘‘proof’’ of the existence of God. Most physicists today support the view that the wavefunction is no more than a probability measure with no independent physical significance, whose sole predictive power is restricted to the distribution of outcomes of large ensembles of identical experiments. The deeper question of how probabilities become actualities is one that we ignore mainly because we possess no objective way to investigate the question at present, and there is at least some suspicion that the question itself may be as meaningless as the paradox of the Spanish barber. That nature should behave in the odd way implicit in the probabilistic interpretation of quantum mechanics defies our intuition. But as we have seen in other instances, our everyday experiences generate intuitions that have little relevance to the unfamiliar realms now accessible to our exploration.

8.

9. 10. 11. 12. 13.

14. 15.

16.

17. 18.

19.

REFERENCES 1. Bohm, D. (1951). Quantum Theory. PrenticeHall, Englewood Cliffs, N.J., p. 81ff. 2. Cropper, W. H. (1970). The Quantum Physicists. Oxford University Press, London, p. 120ff. 3. Davisson, C. J. and Germer, L. H. (1927). Phys. Rev., 30, 705. 4. de Broglie, L. (1924). Philos. Mag., 6th Ser., 47, 446. 5. de Broglie, L. (1960). Nonlinear Wave Mechanics—A Causal Interpretation. Elsevier, Amsterdam. 6. DeWitt, B. (1970). Phys. Today, 23(Sept.), 30. 7. Dicke, R. H. (1981). Amer. J. Phys., 49, 925. (In this article Dicke analyzes the consequences of using photons to determine a

20.

21. 22. 23.

particle’s position in considerable detail and resolves several of the apparently paradoxical consequences of Heisenberg’s original analysis.) Drude, P. (1900). The Theory of Optics. (Reprint of the English translation by C. R. Mann and R. A. Millikan of the German edition, Dover, New York, p. 124ff.) Einstein, A. (1905). Ann. Phys., 17, 132. Einstein, A., Podolsky, B., and Rosen, N. (1935). Phys. Rev., 47, 777. Epstein, Paul S. (1945). Amer. J. Phys., 13, 127. Everett, H., III. (1957). Rev. Mod. Phys., 29, 454. Feynman, R. P. (1961). The Theory of Fundamental Processes. W. A. Benjamin, Menlo Park, Calif. Heisenberg, W. (1927). Zeit, Phys., 43, 1972. Kemble, E. C. (1937). The Fundamental Principles of Quantum Mechanics. McGraw-Hill, New York, p. 293ff. Peierls, Sir R. (1979). Surprises in Theoretical Physics. Princeton University Press, Princeton, N. J., pp. 23–34. Planck, M. (1900). Verh. Deut. Phys. Ges., 2, 237; Ann. Phys., 4, 553 (1901). Schr¨odinger, E. (1926). Ann. Phys., 79, 361, 489, 784; ibid., 80, 437 (1926); ibid., 81, 109 (1926). Stapp, H. P. (1972). Amer. J. Phys., 40, 1098. (Stapp’s article is one of the most literate and complete expositions of the Copenhagen interpretation to be found in the general literature. It is intended for readers who already have considerable familiarity with the formalism of quantum mechanics and who wish to explore its philosophical basis.) Taylor, G. I. (1909). Proc. Camb. Philos. Soc., 15, 114. [Modern variants of this experiment can be found in Merli, P. G., Missiroli, G. F., and Pozzi, G. (1976); Amer. J. Phys., 44, 306.] Thomson, G. P. and Reid, A. (1927). Nature, 119, 890. Wheeler, J. A. (1957). Rev. Mod. Phys., 29, 463. Wigner, E. P. (1962). In The Scientist Speculates, I. J. Good, ed. Basic Books, New York, p. 284.

See also PHYSICS, STATISTICS IN (EARLY HISTORY); QUANTUM MECHANICS, STATISTICAL INTERPRETATION OF; and QUANTUM PHYSICS AND FUNCTIONAL INTEGRATION.

J. V. NOBLE

QUANTUM MECHANICS, STATISTICAL INTERPRETATION OF

QUANTUM MECHANICS, STATISTICAL INTERPRETATION OF The statistical interpretation of quantum mechanics arose from the discussion of scattering experiments. In a typical experimental setup of electron scattering we can distinguish the following parts: 1. An electron source, emitting electrons under specified, reproducible conditions (including perhaps focusing, shielding, etc.) 2. A target (scattering center) 3. A screen (detector) where the electrons are registered after the scattering process One may run the experiment at sufficiently low intensity for a long time so that one can be sure that the behavior of a single electron is studied (i.e., the interaction between different electrons in the beam is negligible). The source may then be regarded as a preparation procedure for a statistical ensemble of electrons. Under optimal conditions this ensemble may be mathematically described by a Schr¨odinger wavefunction. In the language of quantum mechanics, one says that the source prepares a state of an electron. Operationally, the statement that an electron is in a specific state means that the electron is a member of a specific statistical ensemble, the ensemble prepared by a specified source. The screen provides the opportunity for each electron in the ensemble to produce one observable event, such as the blackening of a grain in a photographic emulsion covering the screen or a scintillation flash originating from a small area on the screen. In our example the different possible events correspond to different distinguishable small surface cells on the screen. The general characteristic feature of such an observation procedure (here the screen) is that it supplies a set of mutually exclusive possible events which may only be caused by the system under observation (here the electron) and, in the ideal case, each individual in the ensemble will have to produce one such event. If we call an event a measuring result and parametrize the event pi by a number

6737

(n) ai (or several numbers a(1) i · · · ai ) called the ‘‘measured value(s),’’ we are lead to the quantum mechanical concept of an observable. It is the same as the ‘‘observation procedure’’ mentioned above, except that equivalent procedures are not distinguished; for example, in our example it is irrelevant whether the screen is a photographic plate or a scintillation screen, the elementary events caused by an electron being essentially in one-to-one correspondence. On the other hand, the concept of observable involves a specific assignment of measured values to events. For instance, we could assign to each surface cell of the screen the Cartesian x-coordinate of its center as the measured value and thereby obtain an observable called X; or we might choose the y-coordinate and obtain the observable Y; or we might use other parametrizations of the events. The screen then obviously provides a ‘‘simultaneous measuring instrument’’ for the observables X and Y (and all other possible labellings of the possible events). The target may—for conceptual simplicity—be taken together with the source as a part of the ‘‘state preparing procedure’’ or alternatively together with the screen as part of the ‘‘observation procedure.’’ Or we may give it an independent significance as an ‘‘operation’’ that changes the state (i.e., we consider it as a transformation in the set of states). The fundamental experience in experiments of the sort described is that one is not able to construct a source so that all individual systems in the ensemble prepared behave in the same way (i.e., produce the same event, ‘‘hit the screen at the same position’’). Given the source (plus target), only the probability for the different events is determined. The question to be answered by the theory is: Given a state s and a potential event p, what is the probability w(p; s) for this event to happen? Experimentally, it is, of course, given by the ratio between the total number of cases in which the specified event has been produced and the total number of systems prepared. One important property of the set of all states of a system can be immediately abstracted from the operational definition. If it is possible to prepare the states s1 and s2 , then one can also prepare any mixture

6738

QUANTUM MECHANICS, STATISTICAL INTERPRETATION OF

between the two with weights λ and 1 − λ, respectively (0 < λ < 1). This mixture we denote by s = λs1 + (1 − λ)s2 , and since there is no physical interaction between any individuals in the ensemble, the probabilities for any event in the states s, s1 , and s2 must be related by w(p; s) = λw(p; s1 ) + (1 − λ)w(p; s2 ). Mathematically, this means that the set S of all states is a convex set that may be canonically embedded in a real linear space. While the mixing of states is always possible, the purification has its limitations. There are states that cannot be obtained as the mixtures of any other states; these are the pure states or the extremal points of the convex body S . All other states may be ultimately expressed as convex combinations (mixtures) of pure states. In the mathematical formalism of quantum mechanics a pure state s is represented by a vector  of length 1 in a Hilbert space H , a potential event (experimentally corresponding to a detector which gives a signal when the event is produced) by an orthogonal projection operator P acting on H . The probability w is given by w(p; s) =
P
2 ,

(1)

the square of the length of the projected vector. There have been many investigations of whether this specific mathematical structure can be understood as a consequence of simple operational principles [1]. If one focuses on the structure of the set of events, one has to show that they form an ‘‘orthocomplemented, orthomodular, and semimodular lattice.’’ This is the approach of quantum logic, where the events (respectively, the detectors that signal them) are regarded as ‘‘questions’’ or logical ‘‘propositions’’ [2]. If one focuses on the set of states, one has to show that they form the base of a ‘‘self-dual, facially homogeneous cone’’ in a real linear space [3]. If one focuses on the (pure) operations, one has to show that they form a ‘‘Baer semigroup’’ [4]. Any one of these three structural statements is essentially equivalent to the mathematical formalism of quantum mechanics.

Concerning the underlying principles, we have to be content here with a few remarks. Obviously, the events (propositions) may be partially ordered. We may call a proposition p1 finer than p2 if w(p1 ; s)
w(p2 ; s) for all states s. It is seen from (1) that in quantum mechanics the finest propositions (‘‘elementary events’’) correspond to projectors on one-dimensional subspaces of H ; that is, they can, like the pure states, be characterized by unit vectors in H and (1) becomes in that case w(p ; s ) = |(, )|2 . Thus in quantum mechanics there is a oneto-one mapping between pure states and elementary events and the probability function w is symmetric with respect to the interchange: w(p ; s ) = w(p ; s ). This property, together with some maximality assumptions about the sets of states and events, implies that state space is the base of a self-dual cone. The existence of sufficiently many pure operations brings the homogeneity, and after this there remain only a few possibilities: 1. The situation of classical mechanics, where state space is a simplex (i.e., every mixed state has a unique decomposition into pure states) 2. The quantum mechanical structure with a Hilbert space over either the real, the complex, or the quaternionic field 3. A few exceptional cases when there are only two or three mutually exclusive events The quantum theoretical description of nature has raised two (not entirely unconnected) epistemological questions: Are the fundamental laws really indeterministic, or is the statistical description only due to our

QUANTUM PHYSICS AND FUNCTIONAL INTEGRATION

ignorance? This is the question of the possible existence of ‘‘hidden variables’’ which may have so far escaped our observation. The other, more fundamental, question concerns the meaning of physical reality. The notions of ‘‘state’’ and ‘‘event’’ as used in quantum theory are not properties of the ‘‘system’’ (the electron in our example). Rather, the state is an attribute of the system plus the source and the event an attribute of the system plus the detector. If one wanted to assume, for example, that a single particle always has some position in space and the role of the screen is merely to reveal this position to us, we would immediately get into contradictions with experimental facts. If one wanted to assume that the Schr¨odinger wavefunction is a full description of the state of each individual particle, in the sense that each particle, if far away from any other matter, has to have a definite wavefunction (which may be unknown to us), one would get a contradiction to the generalized Einstein–Podolsky–Rosen phenomenon which has recently been tested experimentally [5]. One may, of course, consider the particle together with the source and the screen as one physical system. But then, if one wishes to discuss microscopic states of this big system and events caused by it, one meets the same problem again. One has to bring preparation and observation procedures for the big system into the picture without which the notions of microscopic state and event are not defined. Thus one is always forced to make some cut, mentally dividing the physical world into an observed system and the rest of the universe, while the interplay between these two parts cannot be ignored in the very definition of microscopic state and event of the ‘‘system.’’ If, on the other hand, one is content with a coarser description of the big system, the indeterminacy of events already results from our ignorance of the precise conditions. One can, then, also maintain that an event—such as a flash from the scintillations screen—has either happened or not happened, irrespective of observation. The finality of the event is then tied to the irreversibility as it is known in thermodynamics. This irreversibility is usually interpreted, however, not as a fundamental one but as a practical one, due to the high complexity of the system.

6739

REFERENCES 1. The following books give an account of such efforts up to 1970: G. Mackey, The Mathematical Foundations of Quantum Mechanics, Benjamin, New York, 1963; J. M. Jauch: Foundations of Quantum Mechanics, Addison-Wesley, Reading, Mass., 1968; G. Ludwig, Deutung des Begriffs physikalische Theorie und axiomatische Grundlegung der Hilbert-Raum-Struktur der Quantenmechanik, Lect. Notes Phys., 4, Springer-Verlag, Heldelberg, 1970. 2. This approach was started by G. Birkoff and J. von Neumann, ‘‘The logic of quantum mechanics,’’ Ann. Math., 37, 823–843 (1936). Concerning the role of semimodularity, see J. M. Jauch and C. Piron, ‘‘On the structure of quantal proposition systems,’’ Helv. Phys. Acta, 42, 842–848 (1969). 3. E. M. Alfsen and F. W. Schultz: ‘‘State space of Jordan algebras,’’ Acta Math., 140, 155–190 (1978); H. Araki, On the Characterization of the State Space of Quantum Mechanics, RIMS Publications, Kyoto, 1979; E. B. Vinberg, ‘‘The structure of the group of automorphisms of a homogeneous convex cone,’’ Trans. Mosc. Math. Soc., 63–93 (1965); J. Bellisard and B. Jochum: ‘‘Homogeneous self dual cones versus Jordan algebras,’’ Ann. Inst. Fourier, Vol. XXVIII, Fascicule 1, 1978. 4. J. C. T. Pool, ‘‘Baer∗ -semigroups and the logic of quantum mechanics,’’ Commun. Math. Phys., 9, 118–141 (1968). 5. J. S. Bell, Physics, 1, 195 (1964); J. F. Clauser, M. A. Horne, A. Shimony, and R. A. Holt, Phys. Rev. Lett., 23, 880 (1969). See also QUANTUM MECHANICS AND PROBABILITY.

RUDOLPH HAAG

QUANTUM PHYSICS AND FUNCTIONAL INTEGRATION Methods based on integration in function space are a powerful tool in the understanding of quantum dynamics, especially in quantum field theory, where they not only give a useful formulation of perturbation theory but also provide one of the few tools that go beyond perturbation theory. Since the time-dependent Schr¨odinger equation (in units with
= 1) iψ˙ t (x) = −(2m)−1 ψt (x) + V(x)ψt (x)

(1)

6740

QUANTUM PHYSICS AND FUNCTIONAL INTEGRATION

is similar to the diffusion equation −φ˙ t = −φt (x),

(2)

for which Wiener invented functional integration [11], it is not surprising that functional integration can be used to study (1). Of course, (1) is solved by ψt = e−itH ψ0 ; H = −(2m)−1  + V, while (2) is solved by φt = e−2tH0 φ0 ; H0 = − 12 . Feynman’s initial formal framework of ‘‘path integrals in quantum mechanics [3]’’ involved the study of e−itH . Although there has been some partially successful rigorous study of Feynman ‘‘integrals’’ (one must extend the notion of measure in all these approaches) [1,4,5], most of the rigorous work and an increasing fraction of the heuristic literature has studied the semigroup e−tH rather than e−itH . At first sight it seems surprising that the study of e−tH would be a suitable substitute for the study of e−itH , but the semigroup even has some real advantages. (1) If one wants to understand the lowest eigenvalue∗ , E (ground state energy), and corresponding eigenvector (ground state) of H, the operator e−tH is often more useful than e−itH ; for example, E = −t−1 ln
e−tH
. (2) If one views the parameter t as 1/(kT), with k equal to Boltzmann’s constant and T the temperature, then the semigroup is the basic object of quantum statistical mechanics. (3) In quantum field theory, the formal continuation from it to t should replace Lorentz invariance by invariance under a Euclidean group of rotations and various hyperbolic equations by better-behaved elliptic equations. This passage to Euclidean invariance is often called analytic continuation to the Euclidean region in the theoretical physics literature. In units with m = 1, the basic formula for e−tH is the Feynman–Kac formula, named after fundamental contributions by their authors [3,6]: (e−tH f )(x) = Ex (e−


t

0 V(b(s))ds f (b(t))).

(3)

In this formula, b is Brownian motion∗ and Ex denotes expectation with respect to paths

starting at x, that is, b(s) is an Rν -valued Gaussian process∗ with mean x and covariance: Ex ((bi (s) − xi )(bj (t) − xj )) = δij min(t, s). For detailed conditions on f , V for (3) to hold and several proofs, see ref. 8. An especially useful way of looking at (3) is in terms of the Lie–Trotter formula: exp(−tH) = lim [exp(−tH0 /n) exp(−tV/n)]n n→∞

and Riemann sum approximations to
t 0 V(b(s)) ds. While (3) is very useful to study H, it can also be turned around and used to study Brownian motion; for example [see ref. 8 (pp. 58–60)], one can compute the distribution of the Lebesgue measure of {s
t|b(s) > 0} by using (3) with V the indicator function of {x|x > 0}. The differential operator H(a) ≡ 12 (−i∇ − a)2 + V

(4)

is the Hamiltonian operator for particles (with
= 1, m = 1, e/(mc) = 1) moving in a magnetic field B = curl (a) and potential V. There is an analog of (3) for H(a), sometimes called the Feynman–Kac–Ito formula: (e−tH f )(x) = Ex (e−iF(b) e−


t

0 V(b(s))ds f (b(t)))

(5)

with 

t

F(b) =

a(b(s)) · db

0

−

1 2



t

(div a)(b(s))ds.

(6)

0


In (6), a · db is an Ito stochastic integral∗ (see also BROWNIAN MOTION); if the Stratonovich integral is used instead, the second term in (6) will not be present. Since F is real-valued, (5) immediately implies the diamagnetic inequality of Nelson and Simon: |(e−tH(a) f )(x)|
(e−tH(a=0) |f |)(x), which has been very useful in the study of operators of the form (4).

QUANTUM PHYSICS AND FUNCTIONAL INTEGRATION

As an intermediate situation to describing quantum field theory, one can consider P(φ)1 -processes. Let h = − 12 d2 /dx2 + υ(x) on L2 (R, dx). Suppose that e is the groundstate energy and (x) the ground state. Let Pt (x, y) be the integral kernel of the operator

exp[−t(h − e)] −1 . Pt (x, y) has the semigroup
property; it is positive since (x)  0, and Pt (x, y)dx = 1 since e−th is symmetric and e−th = . Thus Pt (x, y) defines a Markov process∗ with invariant measure 2 dx. The Markov process q(t) with this invariant measure as initial distribution and transition kernel Pt (x, y) is called the P(φ)1 -process [since V is often a polynomial P(x)]; we use E to denote the corresponding expectation. The expectations E(υ) and E(υ+w) associated to h and h + w are related by E(υ+w) (F)

   (υ) F exp − E = lim Z−1 t t→∞

t

 w(q(s))ds

with   Zt = E(υ) exp −

E(φ(x)φ(y)) = G0 (x − y) is called the free Euclidean field or occasionally the free Markov field (since it has a kind of multitime Markov property). If (− + 1)−1 is replaced by (− + m2 )−1 , the phrase ‘‘of mass m’’ is added to ‘‘free Euclidean field.’’ This process is the analytic continuation to the Euclidean region of quantum field theory describing noninteracting spinless particles. The process above is only ‘‘generalized’’ [i.e., φ(x) must be smeared in (x), since G0 is singular at x = y, if ν  2]. For ν = 2, the singularity is only logarithmic, but it is a power singularity if ν  3. The natural analog of (7) is to try to construct an expectation by     limν Z()−1 F(φ) exp − w(φ(x))dν x , →R



(9)

t

 w(q(s)) ds.

exp(−|t − s|).



   exp − w(φ(x))dν x . 

This sets up an analogy to statistical mechanics which is often useful. The P(φ)1 -process with υ(x) = 12 x2 is of particular interest since it is a Gaussian process. Its covariance is 1 2

where Z() =

−t

E(q(s)q(t)) =

L2 (Rν , dν x). The generalized Gaussian process φ(x) with mean zero and covariance

−t

(7)

6741

(8)

Up to changes of scale of q and/or t, it is the unique stationary, Markov-Gaussian process, called the Ornstein–Uhlenbeck∗ velocity process, or occasionally the oscillator process, since − 12 d2 /dx2 + 12 x2 is the Hamiltonian of a harmonic oscillator. Quantum field theories analytically continued to the Euclidean region are analogs of P(φ)1 -processes with the time, s, now a multidimensional variable. In generalizing (8), it is important to realize that the right side of (8) is the integral kernel of the operator (−d2 /dt2 + 1)−1 . Let G0 (x − y) be the integral kernel of the operator (− + 1)−1 on

Such a construction would yield models of quantum fields. In ν = 2 and 3 (ν = 4 is the physical case), this program, called constructive quantum field theory, has been successful, due to the efforts of many mathematical physicists, most notably J. Glimm and A. Jaffe. In (9), there are two general problems. The limit  → Rν is not trivial to control; it has been controlled by making an analogy to the corresponding limit in statistical mechanics and extending various ideas from that discipline. The other problem is special to quantum field theory. When ν = 1, the oscillator process is supported on continuous functions, but for ν  2, the corresponding free field is supported on distributions that are signed measures with probability zero! Thus the typical choice w(φ(x)) = φ(x)4 is meaningless since φ(x) does not have powers. This is the celebrated problem of ultraviolet divergences, which occurred in all the earliest attempts to study quantum field theory. It is solved by renormalization theory. For example, let

6742

QUANTUM PHYSICS AND FUNCTIONAL INTEGRATION


φf (x) = f (x − y)φ(y)dν y, which yields a nice process if f ∈ C∞ 0 . We want to let f → δ. One lets w depend on f and in (9) lets w(φ) be replaced by wf (φf ) and then also takes a limit as f → δ. For example, in ν = 2 dimensions, one can take wf (φ(x)) = φf (x)4 − 6E(φ 2 (x))φf2 (x)

(10)

and control the limit in (9) if first f → δ and then  → ∞. In (10), E(−) denotes expectation with respect to the free Euclidean field, so  2 E(φf (x)) = f (x)G0 (x − y)f (y)dxdy → ∞ as f → δ. The resulting theory is often denoted as the φ24 theory. There have been developed φ34 theories and P(φ)2 for a large class of polynomials P. Fermion theories and certain gauge models have been constructed in two and three dimensions, but this goes beyond the scope of this article. An especially useful device in the theory is the lattice approximation, where x is replaced by a discrete variable and − by a finite difference∗ operator. One realizes that these models are lattice systems∗ with unbounded spins and one uses intuition from that subject. As the lattice spacing goes to zero, one can hope to recover the continuum theory. Recently, steepest descent ideas have come into use in functional integration pictures of quantum theory. A major role is played by functions minimizing an exponent. The quantity in the exponent is usually called Euclidean action and the minimizing functions are instantons. Glossary Fermion Theories: Field theories describing particles that obey the Pauli exclusion principle (also called Fermi–Dirac∗ statistics, hence ‘‘fermions’’). Gauge Models (also called ‘‘Yang–Mills’’ or ‘‘non-Abelian gauge’’ theories): These are the current popular quantum field models which describe both weak interactions (Weinberg–Salam theories) and strong interactions (quantum chromodynamics). Perturbation Theory: Expansion physical coupling constants of the basic

quantities of a quantum field theory which is especially useful in quantum electrodynamics, where the natural coupling constant is small (about 1/137). Quantum Field Theory: The only successful way of synthesizing quantum mechanics and special relativity. REFERENCES 1. Albeverio, S. and Hoegh-Krohn, R. (1976). Mathematical Theory of Feynman Path Integrals. Lect. Notes Math., 523, Springer-Verlag, New York. (This reference and refs. 2 and 5 constitute three attempts at mathematical formulations of a path integral view of e−itH .) 2. DeWitt-Morette, C., Maheshwari, A., and Nelson, B. (1979). Phys. Rep., 50, 255–372. 3. Feynman, R. P. (1948). Rev. Mod. Phys., 20, 367–387. (This reference and ref. 6 are two classics with a wealth of insight.) 4. Feynman, R. P. and Hibbs, A. (1965). Quantum Mechanics and Path Integrals. McGrawHill, New York. (The main reference on the formal path integral for studying e−itH .) 5. Fujiwara, D. (1974). Proc. Japan Acad., 50, 566–569; ibid., 50, 699–701; ibid., 54, 62–66 (1978). 6. Kac, M. (1950). Proc. 2nd Berkeley Symp. Math. Statist. Prob. University of California Press, Berkeley, Calif., pp. 189–215. 7. Nelson, E. (1960). Phys. Rev., 150, 1079–1085. (Another interface of stochastic processes and quantum theory.) 8. Simon, B. (1974). The P(φ)2 Euclidean (Quantum) Field Theory. Princeton University Press, Princeton, N. J. (This reference and ref. 10 cover the details of Euclidean constructive quantum field theory.) 9. Simon, B. (1979). Functional Integration and Quantum Physics. Academic Press, New York. (A general discussion of Brownian motion and e−tH .) 10. Velo, G. and Wightman, A. S., eds. (1973). Constructive Quantum Field Theory. Springer-Verlag, Berlin. 11. Wiener, N. (1923). J. Math. Phys., 2, 131– 174. See also LATTICE SYSTEMS; QUANTUM MECHANICS AND PROBABILITY; QUANTUM MECHANICS, STATISTICAL INTERPRETATION OF; and STOCHASTIC INTEGRALS.

BARRY SIMON

QUARTIMAX

QUARTIC EXPONENTIAL DISTRIBUTION

used as the measure of central tendency. It is defined as

The quartic exponential distribution has a probability density function of the form f (x) ∝ exp[−(α1 x + α2 x + α3 x + α4 x )], 2

α4 > 0,

6743

3

4

−∞ < x < ∞.

Q = 12 (Q3 − Q1 ) where Q1 and Q3 are the first and third quartiles, respectively. The relative variation is measured by expressing the quartile deviation as the percentage of the midpoint between Q1 and Q3 . This percentage is denoted by

It belongs to the general exponential family∗ . Fisher [2] discussed a general ‘‘polynomial’’ exponential family, but the fourth-degree case seems to be first explicitly considered by O’Toole [4] and subsequently by Aroian [1]. Matz [3] studied the quartic exponential family in detail, with special emphasis on maximum likelihood∗ estimation of parameters. Application to bimodal data seems the principal use of this distribution.

and is called the coefficient of quartile deviation. It is an analog of the coefficient of variation∗ .

REFERENCES

QUARTILE MATCHING

1. Aroian, L. A. (1948). Ann. Math. Statist., 19, 589–592. 2. Fisher, R. A. (1921). Phil. Trans. R. Soc. Lond. A, 222, 309–368. 3. Matz, A. W. (1978). Technometrics, 20, 475–484. 4. O’Toole, A. L. (1933). Ann. Math. Statist., 4, 1–29, 79–93. See also EXPONENTIAL DISTRIBUTION and EXPONENTIAL FAMILIES.

QUARTILE. See QUANTILE

VQ = =

(Q3 − Q1 )/2 × 100 (Q3 + Q1 )/2 Q3 − Q1 × 100 Q3 + Q1

Quartile matching is a device to explore the tail length of a symmetric distribution. For ‘‘basic’’ continuous symmetric distributions, this is achieved by choosing the scale parameter for each distribution so that the quartiles (25th and 75th percentiles) match those of the standard normal distribution*. Thus, the matched distributions all have their 75th percentile equal to 0.6745. However, the 95th percentile, which is 1.64 for the standard normal becomes 1.81 for the logistic*, 2.24 for the Laplace*, and 4.26 for the Cauchy distribution*. Analogous values for the 99th percentile are 2.33, 2.39, 3.81, and 21.5, respectively. See also QUANTILE.

QUARTILE DEVIATION This is one-half of the interquartile∗ distance. It is also called the semi-interquartile range. See also PROBABLE ERROR and QUARTILE DEVIATION, COEFFICIENT OF.

QUARTILE DEVIATION, COEFFICIENT OF The quartile deviation∗ is an appropriate measure of variation when the median is

QUARTIMAX An algorithm for orthogonal rotation occasionally used in factor analysis∗ . The quartimax method rotates the factors in such a manner as to accomplish for a given variable one and only one major loading on a given factor. An apparently undesirable property of this method is a tendency toward producing a general factor with all or most of the variables having high loadings. Compare with Varimax∗ . See, e.g., Rummel [1] for details of the computations.

6744

QUASI-BAYESIAN INFERENCE

to test H0 : pi = 1/K, i = 1, . . . , K, the Bayes factor would be given by

REFERENCE 1. Rummel, R. J. (1970). Applied Factor Analysis. Northwestern University Press, Evanston, Ill.

n

K (Kα) See also FACTOR ANALYSIS and VARIMAX METHOD.

QUASI-BAYESIAN INFERENCE Quasi-Bayesian (or pseudo- or semi-Bayesian) inference refers to a combination of, or compromise between Bayesian∗ and classical procedures sometimes used in the analysis of hierarchical models∗ . In the hierarchical model, multiple levels or stages of uncertainty are generated in the following way. First, a likelihood function, which is conditional on values of unknown parameters, represents the data generated by some random process. A Bayesian statistician then puts a prior distribution∗ on these unknown parameters, and this distribution itself is conditional on the values of other parameters called hyperparameters. If these hyperparameters are unknown, a distribution (sometimes called a hyperprior) may be given for their values, and so on. In quasi-Bayesian inference, the hyperparameters at some stage are estimated by non-Bayesian methods, such as maximum likelihood estimation∗ , thus eliminating a hyperprior at that stage. Although this practice is not viewed as Bayesian, it has been termed part of a ‘‘Bayes/non-Bayes compromise’’ in ref. 1. For example, suppose that (n1 , . . . , nK ) represent a sample from a K-category multiK nomial distribution∗ , with i=1 ni = n. Let pi represent the probability attached to cell  i, with K i=1 pi = 1. A Bayesian statistician would represent uncertainty about (p1 , . . . , pK ) in a prior distribution; for example, consider a (symmetrical) Dirichlet∗ prior distribution having density f (pi , . . . , pk |α) =

K

(Kα)  α−1 Pi , { (α)}K i=1

where α > 0 is the hyperparameter. A Bayes estimate of pi , under squared-error loss, would be E(pi |α) = (ni + α)/(n + Kα). In order

F(α) =

K



(ni + α)

i=1

[ (α)]K (n + Kα)

.

The hyperparameter α is seen to play an important role in inference. If α is uncertain, a full Bayesian analysis would require that a hyperprior, f (α), be specified. A quasi-Bayesian approach might use maximum likelihood estimation to circumvent the need for f (α). In other words, define αmax as the value of α that maximizes F(α), and let Fmax = F(αmax ). Then a quasiBayesian estimate of pi would be (ni + αmax )/(n + Kαmax ). Since Fmax tends to overestimate the Bayes factor against H0 , alternative interpretations of Fmax may be employed; see ref. 1.

REFERENCE 1. Good, I. J. and Crook, J. F. (1974). J. Amer. Statist. Ass., 69, 711–720. See also BAYESIAN INFERENCE; EMPIRICAL BAYES THEORY; and ESTIMATION, CLASSICAL.

R. L. TRADER

QUASIBINOMIAL DISTRIBUTIONS The simplest form for the probability function of a quasibinomial variate r is Pr(r = k) =

n p(p + kφ)k−1 (1 − p − kφ)n−k k (k = 0, 1, . . . , n; 0 < p < 1), (1)

where n is a positive integer and p + nφ
1. It was introduced by Consul [3] in connection with a two-urn model. A modified urn model

QUASIBINOMIAL DISTRIBUTIONS

due to Consul and Mittal [4] led to the more general four-parameter quasibinomial form 1 ab n Pr(r = k) = k (a + b)(a + kθ )

k a + kθ × a + b + nθ

b + nθ − kθ n−k−1 × a + b + nθ (k = 0, 1, . . . , n),

(2)

which reduces to the binomial form when θ = 0. An identity due to Jensen [7] is needed to derive expressions for the moments. For the mean,

LINKS WITH LAGRANGE’S EXPANSION In 1770, Lagrange [10] gave the formula f (t) = f (a) +

(n, a, b) =

s=0

(n − 2)! (n − 2 − s)!



θ a + b + nθ

(uk /k!) × Dk−1 [f (t){φ(t)}k ]t=a

(3)

for a root of the equation t = a + uφ(t) under certain restrictions. In the early part of the present century, Jensen [7] exploited this formula, together with a second basic formulation [see his expression (5)], to produce numerous identities, for the most part involving positive components and hence often translatable into distributional forms. A wellknown example is exp(at) =

∞  a(a + kβ)k−1 k u k! k=0

[t = u exp(βt)].

(4)

Similarly, from a convolution∗ point of view, Jensen used {exp(at)} ∗ {exp(bt)} to derive

where n−2 

∞  k=1

E(r) = na/(a + b), and for the second factorial moment,   b(n, a, b) n(n − 1)a 1− , E{r(r − 1)} = a+b a + b + nθ

6745

s .

SOME PROPERTIES

(a + b)(a + b + nβ)n−1 n  n a(a + kβ)k−1 = k k=0

1. There are interesting limiting cases. For example, if n is large, a and θ small with na = c, nθ = d, then the quasibinomial in the limit becomes the generalized Poisson distribution∗ . Under certain restrictions, another limiting form produces the normal distribution. 2. The sum of quasibinomial variates in general is not a quasibinomial variate. 3. Properties of the binomial distribution, such as the location of the largest probability, expression of the partial sum of probabilities in integral form, and so on, are not readily available for the quasi-binomial distribution. 4. An optimal asymptotic test for a quasibinomial distribution has been given by Fazal [5]. See also Janardan [6] and Korwar [9].

× b(b + nβ − kβ)n−k−1 ,

(5)

which relates to a quasibinomial [8]. Surprisingly enough, some of these formulas were given nearly a century earlier by Abel [1] and Cauchy [2], the latter considering expansions such as the binomial form [f (x, z) + f (z, x)]n F(x, z) and using the calculus of residues. For example, [(x + a + n)n − (x + a)n ]/a n−1  n (a + n − k)n−k−1 (x + k)k .(6) = k k=0

For generalizations, see Riordan [11].

6746

QUASI-COPULAS

LINKS WITH AN OPERATIONAL FORM OF LAGRANGE

Note that the validity of the expansions set up from the operational approach has to be scrutinized.

The function t exp(−βt) has a powerful interpretation in terms of finite differences∗ . For if t = ln(1 + ), where f (x) = f (x + 1) − f (x), E{f (x)} = f (x + 1), then [t exp(−βt)] ≡ D E k

k

−kβ

,

(7)

the operators being commutative and distributive. Also recall that f (x + h) ≡ Eh f (x) ≡ ehD {f (x)}. Jensen [7] uses Lagrange expansions as operators; for example, from (4) he produces (x + a) =

∞  a(a + kβ)k−1 k=0

k! × (k) (x − kβ),

(8)

leading to a form similar to Consul’s expression (2) for a quasibinomial distribution when (x) = xn , x = 1 − p, a = p. Again using the Lagrange expansion for t = υ(1 + t)β and the function (1 + t)a , where t = , quasidistributions may be constructed; for example,



a + β + nt β + nt = n n

n  1 a + kt +a k a + kt k=1

β + nt − kt × . n−k To derive the quasihypergeometric probability function (see Johnson and Kotz [8, p. 84, eq. (70)], filter out the coefficients of υ n in the convolution

[t = υ(1 + t)β ], using ∞  k=0

a a + kβ



a + kβ υ k. k

1. Abel, N. H. (1826). J. Math., 1, 159–160. 2. Cauchy, A. L. (1826). Exer. Math., first year, 44–55. 3. Consul, P. C. (1974). Sankhya¯ B, 36, 391– 399. 4. Consul, P. C. and Mittal, S. P. (1975). Biom. Z., 17, 67–75. 5. Fazal, S. S. (1976). Biom. Z., 18, 619– 622. 6. Janardan, K. G. (1974). SDSW, 3, 359– 364. 7. Jensen, J. L. W. V. (1902). Acta Math., 26, 307–318. [Systematic uses of operational calculus are attributed to Oliver Heaviside (1850–1925). It has been used to derive Stirling’s formula for ln (·).] 8. Johnson, N. L. and Kotz, S. (1982). Int. Statist. Rev., 50(1), 71–101. (Gives a comprehensive view of recent developments in discrete distributions.) 9. Korwar, R. M. (1977). Commun. Statist. A, 6, 1409–1415. 10. Lagrange, J. L. (1770). Oeuvres, 2, 25. 11. Riordan, J. (1968). Combinatorial Identities. Wiley, New York. (Chapter 6 treats the difference and differential operators  and D.) See also BINOMIAL DISTRIBUTION; CLASSICAL DISCRETE DISTRIBUTIONS, GENERALIZATIONS OF; GENERALIZED HYPERGEOMETRIC DISTRIBUTIONS; HYPERGEOMETRIC DISTRIBUTIONS; LAGRANGE AND RELATED PROBABILITY DISTRIBUTIONS; LAGRANGE EXPANSIONS; and POWER SERIES DISTRIBUTIONS.

L. R. SHENTON

QUASI-COPULAS

(1 + t)a ∗ (1 + t)b = (1 + t)a+b

(1 + t)a =

REFERENCES

(9)

Varieties of this kind are legion—the inverse problem of discovering the correct Lagrange form is another matter.

A copula* is essentially a bivariate cumulative distribution function (cdf) with uniform marginals. Evidently, not every bivariate distribution that is nondecreasing in each of its arguments can serve as a copula; additional constraints may be required. To relax these constraints, the concept of quasi-copula has been introduced and investigated [1,2,3]. An additional motivation was

QUASI-FACTORIAL DESIGNS

to show that certain operations on univariate distribution functions are not derivable from the corresponding operations on random variables defined on the same probability space. The somewhat cumbersome original definition is based on the notion of a track, which is a subset B of a unit square written in the form B = {(F(t), G(t)) :

0
t
1}

for continuous cdfs F and G satisfying F(0) = G(0) = 0, F(1) = G(1) = 1.

where f : [0, 1] → [0,

1 24 ]

6747

is given by

  if 0
y
1/4,  0 f (y) = (4y − 1)/24 if 1/4
y
1/2,   (1 − y)/12 if 1/2
y
1. This function Q is an absolutely continuous quasi-copula satisfying conditions (1), (2), and (3). This can be verified using the inequalities −6x
sin(2π x)
12x, 0
x
1. However, the second mixed derivative of Q is strictly negative for 1/4 < y < 1/2, which prevents this function from being a copula. REFERENCES

Now, a quasi-copula is a function Q : [0, 1]2 → [0, 1], such that for every track B there exists a copula CB that coincides with Q on B:

1. Alsina, C., Nelsen, R. B., and Schweizer, B. (1993). On the characterization of a class of binary operations on distribution functions. Stat. Probab. Lett., 17, 85–89.

Q(x, y) ≡ CB (xy).

2. Genest, C., Quesada Molina, J. J., Rodriguez Lallena, J. A., and Sempi, C. (1999). A characterization of quasi-copulas. J. Multivariate Anal., 69, 193–205. 3. Nelsen, R. B., Quesada Molina, J. J., Schweizer, B., and Sempi, C. (1996). ‘‘Derivability of Some Operations on Distribution Functions’’. In Distributions with Fixed Marginals and ¨ Related Topics, L. Ruschendorf, B. Schweizer, and M. D. Taylor, eds. IMS Lecture NotesMonograph Series, 28. Inst. of Math. Statistics, Hayward, Calif., pp. 233–243.

A quasi-copula can be characterized by three properties [2]. A function Q : [0, 1]2 → [0, 1] is a quasi-copula if it satisfies 1. Q(0, x) = Q(x, 0) = 0 and Q(x, 1) = Q(1, x) = x for all 0
x
1; 2. Q(x, y) is nondecreasing in each of its arguments; 3. |Q(x , y ) − Q(x, y)|
|x − x| + |y − y| for all 0
x, x , y, y
1 (Lipschitz condition). Moreover, every quasi-copula (like every copula) is contained between the Fr´echet bounds*

See also COPULAS.

QUASI-EXPERIMENT. See max(0, x + y − 1)
Q(x, y)
min(x, y). An alternative characterization of a quasicopula is that of a function Q : [0, 1]2 → [0, 1] that satisfies condition (1) above and such that Q(x , y ) + Q(x, y)  Q(x , y) + Q(x, y ) whenever at least one x, x , y, or y is either 0 or 1. Consider the function Q : [0, 1]2 → [0, 1] defined by Q(x, y) = xy + f (y) sin(2π x),

NAL

OBSERVATIO-

STUDIES

QUASI-FACTORIAL DESIGNS The name quasi-factorial design was given by Yates [4] to a class of designs that bear some resemblance to factorial∗ designs. These designs are used for comparing a large number of treatments in blocks, where there are more treatments than plots in a block, so that they are incomplete block designs∗ . The word quasi comes from a Latin word meaning ‘‘as if,’’ but a common present use, and dictionary definition, is ‘‘in appearance only.’’

6748

QUASI-INDEPENDENCE

To make matters more confusing, the sets into which the treatments of the design are divided are known not as quasi-factors but as pseudo-factors. The word pseudo comes from a Greek word meaning ‘‘false,’’ and its modern use is, in practice and in the dictionary, ‘‘deceptively resembling.’’ Perhaps it is just as well that for most practical purposes, the name ‘‘quasi-factorial design’’ has almost disappeared from the modern statistical literature; instead, this type of design is known as a lattice design∗ . The idea underlying the name ‘‘quasifactorial design’’ is that it shall be possible to establish a correspondence between the treatments of the design and the treatment combinations of a factorial set, and to make use of this correspondence to get a simpler form of statistical analysis for the design [2]. In the present computer age, ease of statistical analysis is no longer the problem it once was. Thus although quasi-factorial designs remain valuable in their own right, the original quasi-factorial concept has outlived its usefulness. The simplest, and commonest, quasifactorial or lattice designs are those for n2 treatments in blocks of n plots each; such designs are resolvable∗ , that is, a number of blocks, here n, form a complete replicate. If a fully orthogonal set of n × n Latin squares∗ exists, it is possible to have (n + 1) replicates of the treatments in the design, which then forms a balanced lattice. This is the same as a balanced incomplete block∗ design, and standard methods of analysis of such designs are well known. With fewer replicates the design is not balanced, and indeed may not come into the category of partially balanced∗ incomplete block designs. However, the use of pseudo-factors renders a quasifactorial design analyzable, and a general recursive algorithm for analysis of variance∗ , which includes these designs, was described by Wilkinson [3] and implemented in the general statistical program GENSTAT [1]. With n2 treatments and only two replicates it is sensible to regard the treatments as an n × n pseudo-factorial arrangement; however, with more than two replicates it is still possible to implement the algorithm, by having further pseudo-factors whose levels correspond to the treatments of a Latin square.

REFERENCES 1. Alvey, N. G. et al. (1977). Genstat, a General Statistical Program. Rothamsted Experimental Station, Harpenden, England. 2. Kempthorne, O. (1952). Design and Analysis of Experiments. Wiley, New York. 3. Wilkinson, G. N. (1970). Biometrika, 57, 19– 46. 4. Yates, F. (1936). J. Agric. Sci., 26, 424–455. (The paper that introduced these designs, and the leading reference.) See also LATTICE DESIGNS.

G. H. FREEMAN

QUASI-INDEPENDENCE HISTORICAL REMARKS The concept of quasi-independence (QI) arose in response to a dispute between Harris (e.g., ref. 16) and Pearson (e.g., ref. 19) on how to test for independence in incomplete contingency tables∗ . A classic set of data used by both Harris and Pearson is reproduced in Table 1. A valid model for these data must recognize that 18 of the 36 cell counts are zero a priori (e.g., a ‘‘locular composition’’ value of ‘‘3 even 0 odd’’ cannot occur when the ‘‘coefficient of radial symmetry’’ is 0.47, 0.82, 1.25, 1.41, or 1.70). Contingency tables with ‘‘structural zeros’’ are found in genetics∗ (e.g., certain chromosome combinations are lethal), the social sciences (e.g., a cross-classification of birth order and sibship size produces a ‘‘triangular’’ table), epidemiology (e.g., some diseases are sex specific), psychology (e.g., data on dyad formation, since subjects do not form dyads with themselves), and other areas. See refs. 2, 6, and 10 for examples. With the partial exception of Watson [20], correct methods for dealing with such data did not appear until the 1960s in the work of Caussinus [3] and Goodman [8–10]. Another context where the QI concept arose was in the analysis of mobility tables [8,10]. Table 2 is a classic 3 × 3 mobility table used by Goodman. Although this table is complete in the sense that all cell frequencies are positive, it can be worthwhile to superimpose an incomplete table on it in order to simplify

QUASI-INDEPENDENCE

6749

Table 1. Relationship between Radial Asymmetry and Locular Composition in Staphylea (Series A) Coefficient of Radial Symmetry Locular Composition

j=1 0.00

2 0.47

3 0.82

4 0.94

5 1.25

6 1.41

7 1.63

8 1.70

9 1.89

i=1 2 3 4

462 — — 103

— 614 443 —

— 138 95 —

130 — — 35

— 21 22 —

— 14 8 —

2 — — 1

— 1 5 —

1 — — 0

3 even 0 odd 2 even 1 odd 1 even 2 odd 0 even 3 odd

Table 2. Cross-Classification of a Sample of British Males According to Subject’s Status and Father’s Status

by pij = αi βj

[for cells (i, j) in S]

(2)

Subject’s Status Father’s Status i = 1 Upper 2 Middle 3 Lower

j=1 Upper

2 Middle

3 Lower

588 349 114

395 714 320

159 447 411

the analysis of association. For example, the (1, 1) cell in Table 2 is relatively large, and it is natural to ask what association remains after the row–column affinity in this cell is eliminated.

THE QI MODEL Let pij denote the population proportion in cell (i, j) of an I × J contingency table. Rows and columns are independent when pij = αi βj

(for i = 1, . . . , I; j = 1, . . . , J) (1)

for sets of positive constants αi and βj . When  these  constants are normalized so that i αi = j βj = 1, αi = pi· and βj = p·j are the row and column marginal proportions, respectively. The definition of independence implies that pij > 0 for all i and j, so this model cannot be directly relevant when some of the pij = 0 (as is the case in incomplete tables). Let U denote the set of deleted cells and S the set of nondeleted cells. In analyzing incomplete tables, U is the set of cells having structural zeros∗ . As in Caussinus [3] or Goodman [10], quasi-independence is defined

for sets of positive constants αi and βj . It is no longer generally true that these constants correspond to marginal proportions, even after normalization. Because of this, maximum likelihood estimates∗ often do not exist in closed form. The QI model says nothing about the pij for cells (i, j) in U; these can be zero (as for incomplete tables) or positive (as for complete tables where some entries are blanked out). Two other forms are often used when the QI model is applied to complete tables. The first consists of (2) plus the relationship pij = αi βj γk

[for cells (i, j) in U],

(3)

where k indexes cells in the set U(k = 1, . . . , K, for K deleted cells) and γk is a parameter specific to the kth deleted call. The QI model applied to complete tables is a special model for interaction∗ , since the γk depend on both i and j values. When γk > 1 for all k, the QI model is equivalent to a latent class model, which is a special kind of finite mixture model for categorical data∗ . The latent class model has K + 1 ‘‘latent classes.’’ The proportions πk in these classes (or mixing weights) are given by the relationships πk = pij − αi βj [for k = 1, . . . , K, and (i, j) in U], πK+1 = 1 −

K  k=1

πk .

(4a) (4b)

6750

QUASI-INDEPENDENCE

Much early work on the QI model was prompted by its affinity to the latent class model (see refs. 4 and 8 for discussion and references). The alternative versions of the model are useful both for computation and for summarization of results. For example, either the γk or the πk values may be used to assess the importance of the kth deleted cell in the overall relationship (see ref. 11). ESTIMATION Maximum likelihood estimation is discussed in refs. 1 to 3, 10, and 14, among others. Other estimation methods have not been emphasized in the literature dealing with QI or related models, although weighted leastsquares∗ methods can be used. When the observed frequencies arise from either independent Poisson or multinomial sampling schemes, the likelihood equations are 

δij (xij − m ˆ ij ) = 0

(5a)

δij (xij − m ˆ ij ) = 0,

(5b)

j

 i

where δij = 1 if cell (i, j) is in S and δij = 0 otherwise, xij is the observed frequency in cell (i, j), and m ˆ ij is the maximum likelihood estimate of the corresponding expected frequency. The observed values of the sufficient statistics∗ are x∗i· =

 j

δij xij ,

x∗·j =



δij xij .

i

Various numerical methods have been used to solve these equations. Early treatments (e.g., refs. 1, 3, 8, and 10) emphasized variants of iterative proportional fitting∗ . Newton–Raphson methods∗ , extensions of Newton’s elementary (onedimensional) method, and the EM algorithm (see MISSING INFORMATION PRINCIPLE) have also been used. A comparison of algorithms applied to two-way tables is provided in ref. 18. Most of the computer programs cited in the entry LOG-LINEAR MODELS IN CONTINGENCY TABLES can deal with QI models or related models for incomplete tables.

DEGREES OF FREEDOM Much of the technical literature on QI models deals with determining the degrees of freedom for chi-square∗ statistics. Correct rules can be found in ref. 10 or 14. To illustrate the source of the difficulty, consider Table 2, where the (1, 1) cell is deleted. A chi-square statistic for the independence model applied to a 3 × 3 table has (3 − 1)(3 − 1) = 4 degrees of freedom (d.f.). (The value of Pearson’s chisquared statistic is 505.5, so independence is decisively rejected.) Since one cell is deleted, the QI model has 4 − 1 = 3 d.f. [Pearson’s chisquared squared is 137.6 for this QI model; removal of the (1, 1) cell provides a dramatic reduction in chi-squared.] Now consider Table 1, where 18 cells are zero a priori. The logic above would indicate that d.f. = (4 − 1)(9 − 1) − 18 = 6, but the correct number is 7. Rearrange the entries in Table 1 so that row levels are 1, 4, 2, 3 and column levels are 1, 4, 7, 9, 2, 3, 5, 6, 8. When this is done, there is a 2 × 4 complete subtable in the upper left and a 2 × 5 complete subtable in the lower right. No rows or columns are shared by the two subtables. Table 1 can thus be broken up into two separable subtables, as recognized by Pearson [19]. The QI model here means that independence holds in each subtable; d.f. = (2 − 1)(4 − 1) + (2 − 1)(5 − 1) = 7. (We obtain Pearson chi-squared statistics 1.4 and 6.1 for these subtables, and the sum 7.5 is a chi-square statistic on 7 d.f. The QI model cannot be rejected for Table 1.) Rules for d.f. calculations applied to separable tables, triangular tables, and other special tables commonly encountered in practice are provided in refs. 2, 6, and 10. If Newton–Raphson estimation methods are used, many difficulties associated with d.f. calculations can be avoided. If a QI model is overparameterized (giving incorrect d.f.), this fact will appear as a rank problem in the iterative matrix inversions associated with this procedure. REMARKS Recent research has emphasized the relationship between QI and many other models. Fienberg and Larntz [7] show how Bradley–

QUASI-INDEPENDENCE

Terry models∗ for paired-comparisons∗ experiments are equivalent to QI models when the data are arranged appropriately in a contingency table format. Larntz and Weisberg [17] show how QI and related models can be used for psychological data on dyad formation. Goodman [12] uses QI models to test the scalability of a set of three or more dichotomous items. Clogg [4] links QI to latent structure∗ models. QI models are especially attractive for square contingency tables where there is a one-to-one correspondence between row and column categories. Let S = {(i, j) : i = j} denote the set of deleted cells; S refers to diagonal entries (consistent responses). For I = J = 3, the QI model in (2) is equivalent to the following condition on the expected frequencies: (m12 m23 m31 /(m21 m32 m13 )) = 1. For this case QI is equivalent to quasi-symmetry∗ (QS). For I = J > 3, QI implies QS. Because QI is nested within QS it often forms a natural baseline model that can be used to partition chi-squared statistics whenever QS models are relevant. Many special models for two-way tables based on both QI and QS can be found in ref. 11. Extensions of QI to multidimensional contingency tables∗ are taken up in refs. 1, 2, and 5, among others. Any log-linear model for a complete multiway contingency table can be considered as a quasi-log-linear model for a subset S of the cells in the complete table. Let the log-linear model be denoted as ν = Xλ,

(6)

where ν is a t-dimensional vector of logarithms of expected frequencies (t is the number of cells), X (of order t × s) is the design matrix∗ (assumed to be of full column rank), and λ is an s-dimensional vector of parameters. A quasi-log-linear model results when (6) is posited for only a subset S of the t cells. This means that some elements in ν, the corresponding rows of X, and possibly some elements of λ are deleted. Problems in d.f. calculations (or in estimability∗ ) arise because deleting rows of X will often create rank problems if some columns of X (corresponding to elements of λ) are not also deleted. Quasi-log-linear models have many uses in the social sciences [2–5, 11, 15]. Such

6751

models are also used occasionally to detect cell ‘‘outliers’’ [2]. Nested sequences of such models can be used to partition association between (among) discrete variables, as in Goodman [10,11,13]. Often a major problem is determining whether a zero cell frequency is, in fact, zero a priori or merely the result of sparse sampling relative to small (but nonzero) cell probabilities. Acknowledgment The preparation of this entry was supported in part by Grants SES-7823759 and SES-8303838 from the Division of Social and Economic Sciences, National Science Foundation.

REFERENCES 1. Bishop, Y. M. M. and Fienberg, S. E. (1969). Biometrics, 22, 119–128. (Extends quasiindependence to multiway tables.) 2. Bishop, Y. M. M., Fienberg, S. E., and Holland, P. W. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, Mass. (Chapter 5 presents intermediate-level survey of models for incomplete tables, including QI model.) 3. Caussinus, H. (1965). Ann. Fac. Sci. Univ. Toulouse, 29, 77–182. [Proposes an iterative proportional fitting algorithm for QI model. Also develops many related models (e.g., model of quasi-symmetry).] 4. Clogg, C. C. (1981). Amer. J. Sociol., 86, 836– 868. 5. Clogg, C. C. (1982). J. Amer. Statist. Ass., 77, 803–815. (Uses QI and related models for analysis of ordinal data.) 6. Fienberg, S. E. (1981). The Analysis of CrossClassified Categorical Data, 2nd ed. MIT Press, Cambridge, Mass. (Chapter 9 gives a nontechnical survey of models for incomplete tables.) 7. Fienberg, S. E. and Larntz, K. Biometrika, 63, 245–254.

(1976).

8. Goodman, L. A. (1961). J. Amer. Statist. Ass., 56, 841–868. (Links QI model to classical ‘‘moverstayer’’ model.) 9. Goodman, L. A. (1965). Amer. J. Sociol., 70, 564–585. (Develops QI model for mobility tables.) 10. Goodman, L. A. (1968). J. Amer. Statist. Ass., 63, 1091–1131. (Best survey article. Compares various algorithms, shows use of model

6752

QUASI LIFE TABLES

for partitioning association, and deals with d.f. calculations for many types of tables.) 11. Goodman, L. A. (1972). Proc. 6th Berkeley Symp. Math. Statist. Prob., Vol. 1, University of California Press, Berkeley, Calif., pp. 646–696. (Develops many special models for 2-way tables, most built on quasiindependence model.) 12. Goodman, L. A. (1975). J. Amer. Statist. Ass., 70, 755–768. (Applied quasi-independence to analyze item scalability.) 13. Goodman, L. A. (1979). Biometrics, 35, 651– 656. 14. Haberman, S. J. (1974). Analysis of Frequency Data. University of Chicago Press, Chicago. (Chapter 7 provides technical survey of quasiindependence and other models for incomplete tables: a key theoretical work.) 15. Haberman, S. J. (1979). The Analysis of Qualitative Data, Vol. 2: New Developments. Academic Press, New York. (Chapter 7 gives a nontechnical survey; the Newton–Raphson algorithm is emphasized and methods for ordinal data are discussed.) 16. Harris, J. A. (1927). J. Amer. Statist. Ass., 22, 460–472. (One of several papers by author criticizing Pearson’s chi-square tests applied to incomplete tables.) 17. Larntz, K. and Weisberg, S. (1976). J. Amer. Statist. Ass., 71, 455–461. 18. Morgan, B. J. T. and Titterington, D. M. (1977). Biometrika, 64, 265–269. 19. Pearson, K. (1930). J. Amer. Statist. Ass., 25, 320–323. (One of several papers defending Pearson’s methods for chi-square tests.) 20. Watson, G. S. (1956). Biometrics, 12, 47–50. See also CATEGORICAL DATA; CHI-SQUARE TEST —I; ITERATIVE PROPORTIONAL FITTING; LATENT CLASS ANALYSIS; LATENT STRUCTURE ANALYSIS; LOG-LINEAR MODELS IN CONTINGENCY TABLES; and QUASI-SYMMETRY.

CLIFFORD C. CLOGG

QUASI LIFE TABLES A common problem in survival analysis∗ is as follows: A manufacturer produces devices of a certain type and sells them to customers who install them, use them, and return them to the manufacturer on failure, replacing a failed device with a new device of the same type. The manufacturer wishes to estimate the parameters of the distribution of the component lifelengths. Given the observed

lifelengths of a number of devices and an assumption concerning the functional form of the lifelength distribution—that it is, for example, Weibull, lognormal, or gamma— this is a straightforward matter. This procedure is predicated on the supposition that the devices are distinguishable, e.g., by a serial number. In some instances, this is not the case, and the only facts available are the numbers of new devices shipped to customers and of failed devices returned each month (or whatever the accounting period may be). Since the data essentially comprise empirical failure rates, Baxter [2] coined the phrase quasi life table to describe them. A quasi life table differs from a conventional life table in two crucial respects: (i) the failure rates pertain to a mixture of devices of varying ages rather than a single cohort; (ii) each failed device is replaced, so the table is of indefinite duration. In estimating the parameters θ of the lifelength distribution from a quasi life table there are two stages [2]: (1) construct a nonparametric estimate of a discretized form of the lifelength distribution, and (2) find a vector θˆ such that the discretized distribution function with parameters θˆ is as close as possible, in some sense, to the nonparametric estimate of stage (1). These two stages are now briefly reviewed. Let F(·; θ ) be the distribution function of the lifelength of a new device, and let Fnθ = F(n; θ)for n = 1, 2, 3, . . . , so that {Fnθ } is a discretization of F(·; θ). In order to construct an estimate of {Fnθ }, it is necessary to make some assumptions concerning the rate at which devices are manufactured. We suppose that, during the nth month, N + xn devices are sold to customers, where N is fixed and xn is the number of failed devices that were returned by customers during the preceding month. So each month, N new customers are added to the manufacturer’s clientele and, in addition, any devices that failed during the previous month are replaced. Suppose further that, once a customer purchases a device, he or she makes indefinite use of it (and its replacements). Write xn = y1 + y2 + . . . + yn , where yj denotes the number of failures of devices owned by customers who first became members of the manufacturer’s clientele n − j months previously.

QUASI LIFE TABLES

Under these conditions, it is reasonable to suppose that the sequence of replacements of a device constitutes a discrete renewal process for which the distribution function of the number of trials between successive events is {Fnθ }. Let Un denote the corresponding discrete renewal function, i.e., Un is the expected number of events (viz. failures) in trials {1, 2, . . . , n}, given that an event occurred on trial 0. Then Fnθ and Un are related by means of the identity Un =

Fnθ

+

n−1 

Fjθ un−j ,

j=1

n  1, where uj = Uj − Uj−1 is the probability of an event on the jth trial. Let Xn and Yj be random variables, realizations of which are the variables xn and yj previously introduced. Since Yj is the number of failures of a cohort of N devices (or their replacements) first put into service j months previously, and since the devices operate independently of each other, we have Yj ∼ Bin (N, uj ) for all j, and E(Yj ) = Nuj . The yj are unobservable, but we can estimate Un as follows. Since  xn = y1 + y2+ . . . + yn , we have E(Xn ) = nj=1 E(Yj ) = N nj=1 uj = NUn . Thus the method-of-moments∗ estimate of ˆ n = xn /N. Un is U ˆ n }, we can construct an estimate Given {U {Fˆ n } of {Fnθ } by means of the recursion ˆn− Fˆ n = U

n−1 

ˆ n−j − U ˆ n−j−1 ), Fˆ j (U

j=1

which follows from the above relationship between Fnθ and Un . The sequence {Fˆ n } may not be monotone nondecreasing, i.e., there may be values of n for which Fˆ n > Fˆ n+1 . In such cases, we replace both Fˆ n and Fˆ n+1 by 12 (Fˆ n + Fˆ n+1 ) to ensure monotonicity, if necessary averaging over three or more consecutive values [1, p. 75]. Next, we find an estimate θˆ of θ by determining a vector that minimizes ρ({Fˆ n }, {Fnθ }), where ρ is a probability metric on the space of distribution functions whose support is the natural numbers. It is assumed that ρ({Fˆ n }, {Fnθ }) =

∞  n=1

an ρ( ˜ Fˆ n , Fnθ ),

6753

where ρ˜ is a metric on the real numbers and {an } is a sequence of weights which is assumed to comprise a probability mass function. Two convenient choices for ρ˜ are: (i) ρ(x, ˜ y) = |x − y|, (ii) ρ(x, ˜ y) = (x − y)2 , i.e., we fit {Fnθ } to {Fˆ n } by weighted least squares∗ . There are several possible choices for {an } including: (iii) an ∝ Fˆ n − Fˆ n−1 , i.e., weights proportional to the estimated probability that a device fails during the nth month, (iv) an ∝ 1/ var{log[− log(1 − Fˆ n+1 )]}, i.e., weights inversely proportional to the estimate of the standard deviation of the error term of a regression of {Fnθ } on {Fˆ n }. As an example, Baxter [2] analyzed five quasi life tables, data from a manufacturer of electronic components. In each case, it was known that the lifelength distribution is Weibull with survival function F(x) = 1 − F(x) = exp(−αxβ ). Since we may write log[− log F(x)] = log α + β log x, it is reasonable to use weighted least squares to estimate α and β, i.e., to use the metric (ii). A number of different weights were compared by means of a Monte Carlo simulation study. It was found that weights (iii) and (iv) outperformed all others and hence these were used to calculate estimates of α and β. Baxter [2] generalized this model to the case where a number of prototypes of the device were in use before month 0. Tortorella [3] investigated a model in which N is permitted to vary from month to month. In this case, there is not enough information available to construct an estimate θˆ of the parameters of F, so Tortorella was only able to prove the existence of an estimator {uˆ n } of {un }. Lastly, although the methodology of quasi life tables was devised to solve a problem in manufacturing reliability, there is scope for application in a variety of other disciplines. At the time of writing, however, no other applications have been published.

6754

QUASI-LIKELIHOOD FUNCTIONS

REFERENCES 1. Barlow, R. E., Bartholomew, D. J., Bremmer, J. M., and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York. 2. Baxter, L. A. (1994). Estimation from quasi life tables. Biometrika, 81, 567–577. 3. Tortorella, M. (1995). Life estimation from pooled discrete renewal counts. In Lifetime Data: Models in Reliability and Survival Analysis, N. P. Jewell, A. C. Kimber, M. -L. T. Lee, and G. A. Whitmore, eds. Kluwer, Dordrecht, pp. 331–338. See also LIFE TABLES; RENEWAL THEORY; and SURVIVAL ANALYSIS.

L. A. BAXTER

QUASI-LIKELIHOOD FUNCTIONS The term quasi-likelihood is used in a number of related senses. First, it describes a method of estimation that is applicable under second-moment assumptions rather than full distributional assumptions. Second, it describes a function which, when maximized, produces the estimator described above and a consistent estimate of its variance. Third, ratios of this maximized function may be used for significance testing∗ or for model selection in much the same way that likelihood-ratio∗ statistics are used. The unifying theme is that second-moment assumptions rather than full distributional assumptions are used throughout. Reliance on low-order moment assumptions is often appealing in applications where more detailed assumptions would often be suspect and difficult to check. The following example, which is analyzed in greater detail toward the end of this article, is introduced here to motivate the general discussion in the sections that follow. Example. Regression with Constant Coefficient of Variation. Suppose that the random variables Y1 , . . . , Yn are uncorrelated, that E(Yi ) = µi < ∞, and that var(Yi ) = σ 2 µ2i , so that the coefficient of variation∗ , σ , rather than the variance, is constant over all observations. Suppose further that inference is required for (β0 , β1 ), where log(µi ) = β0 + β1 (xi − x), i = 1, . . . , n and x1 , . . . , xn are known constants.

(1)

The principal features of this example that we wish to emphasize are: 1. The relation (1) between µ = E(Y) and β = (β0 , β1 ) is not linear in β. 2. The covariance matrix of Y is given by cov(Y) = σ 2 V(µ),

(2)

where V(·) is a matrix of known functions and σ 2 is known as the dispersion∗ parameter. 3. The model is specified entirely in terms of first and second moments of Y so that it is not possible to write down the likelihood function. We require a method of estimation that is reliable under the second moment assumptions just described. LEAST SQUARES To simplify the notation we consider arbitrary nonlinear regression∗ models µ = µ(β), where β is a p-dimensional unknown parameter vector. It is assumed that the model is identifiable in the sense that if β = β , then µ(β) = µ(β ). Special cases include (1) above and generalized linear models∗ [3,4]. In the special case where V(·) is a matrix of constants, the method of nonlinear weighted least squares minimizes the weighted sum of squares X 2 (β) = [Y − µ(β)]T W[Y − µ(β)]

(3)

where W = V−1 . The estimating equations may be written ˆ = 0, DT W[Y − µ(β)]

(4)

where D = {∂µi /∂βj } is order n × p and is a function of β. To extend the method of weighted least squares∗ to the case where V(·) is not constant, we may choose to generalize either (3) or (4) in the obvious way, simply taking W to be a function of µ(β). Unfortunately, (3) and (4) are no longer equivalent in the sense that the solution βˆ to (4) no longer minimizes (3). To emphasize this distinction, we refer to (4)

QUASI-LIKELIHOOD FUNCTIONS

6755

as the quasi-likelihood equations: they have the appealing geometrical interpretation that ˆ is orthogonal to the residual vector Y − µ(β) the columns of D with respect to the inner ˆ We product matrix W, both evaluated at β. take the view that the appropriate generalization of least squares is based on (4) and not on (3) [2,5]. On the other hand, the method of minimum chi-square∗ [1] is based on (3). Thus it seems appropriate to refer to the minimizing value β˜ as the ‘‘minimum chi-square estimate’’ of β. The two methods coincide only if V is constant.

In general, for arbitrary covariance matrices V(µ, U (β) is not symmetric and no solution to (5) exists. This has no effect on the ˆ but there may be statistical properties of β, other statistical implications that have not been investigated. It does seem paradoxical, however, that the least-squares equations (4) cannot be formulated as a minimization problem. Note that (5) has a solution if V(µ) = AC(µ)AT , where C(·) is diagonal and A is constant, or if β is scalar, but these conditions do not appear to be necessary.

QUASI-LIKELIHOOD FUNCTIONS

The discussion here refers to the solution βˆ to (4) in the limit as n → ∞. The principal results are

The estimation equations (4) may be written ˆ = 0, where in the form U(β) U(β) = D W[Y − µ(β)].

ASYMPTOTIC PROPERTIES

D

n1/2 (βˆ − β)→Np {0, nσ 2 (DT WD)−1 }

(6)

T

Both D and W are treated as functions of β. It may be verified that E{U(β)} = 0, E{U (β)} = −DT WD, cov{U(β)} = σ 2 DT WD, so that, apart from the factor σ 2 above, U(β) behaves like the derivative of a log-likelihood. Consistency∗ and asymptotic normality∗ of βˆ follow from the three properties listed above. In the majority of statistical problems it is possible to construct a function l(β) such that ∂l = U(β). ∂β

(5)

The function l(β) is then known as a logquasi-likelihood and can be used in much the same way as an ordinary log-likelihood, for example, in the construction of likelihoodratio tests∗ . Furthermore, βˆ maximizes l(β) and ˆ σ 2 (DT WD)−1 , cov(β) which is proportional to the inverse of the expected second derivative matrix of l(β).

and plim {X 2 (β)/(n − p)} = σ 2 , where Np refers to the p-variate normal distribution. Apart from the dispersion factor σ 2 , this is exactly the kind of result that one would expect from maximizing a log-likelihood. The results above are independent of whether or not (5) has a solution. In contrast, the minimum chi-square estimator β˜ based on (3) does not have similar desirable properties in the same limit. In fact, β˜ is generally inconsistent for β. However, other limits such as µi → ∞ may be considered in which β˜ is consistent [1]. In applications it is frequently the case that we need to consider a sequence of nested hypotheses, say H0 : β ∈ ω0 , H1 : β ∈ ω1 , . . ., where ω0 has dimension q, ω1 has dimension p, and ω0 ⊂ ω1 ⊂ · · ·. It is then very convenient, if a solution to (5) exists, to use the quasi-likelihood ratio 01 = 2l(βˆ 1 ) − 2l(βˆ 0 ) as a test statistic, where βˆ j is the estimate of β under Hj . The approximate null distribution 2 . If l(β) does not exist or if of 01 is σ 2 χp−q it cannot be computed explicitly, score tests based on U(βˆ 0 ) may be used instead of 01 . The principal advantage of  is that test statistics for a sequence of nested models are additive (see PARTITION OF CHI-SQUARE).

6756

QUASILINEAR MEAN

Example (Continued). Continuing with model (1), we suppose for definiteness that n = 10, Y = (0.53, 0.38, 1.97, 1.65, 1.27, 1.34, 0.05, 0.45, 1.88, 0.19) and xi = i/2. The quasi-likelihood function is l = − {yi /µi + log(µi )}, where µi satisfies (1). The estimating equations (4) may be written  (yi − µˆ i )/µˆ i = 0 and  (xi − x)(yi − µˆ i )/µˆ i = 0 and give values βˆ = (−0.034, −0.078), with  covariance matrix diag{σ 2 /n, σ 2 / (xi − x)2 } estimated as diag{0.066, 0.032} with s2 = ˆ = 0.66. The quasi-likelihood-ratio X 2 (β)/8 ˆ − statistic for testing H0 : β1 = 0 is {2l(β) 2l(βˆ 0 )}/s2 = 0.15, so that there is no evidence of any relationship between X and Y. The minimum chi-square estimates based ˜ = on (3) are β˜ = (0.384, 0.031) and X 2 (β) ˆ = 5.30. It is evident 3.36, as opposed to X 2 (β) that β˜ and βˆ are not estimating the same quantity, in agreement with earlier claims ˜ concerning the consistency of β. An alternative procedure, and one that seems appealing if the data are symmetrically distributed or nearly so, is to assume normality and independence and to use maximum likelihood. The resulting estimators are consistent and in this case the values are β ∗ = (−0.030, −0.001), σˆ 2 = X 2 (β ∗ )/10 = 0.51. The asymptoticvariance of β1∗ is given by σ 2 (1 + 2σ 2 )−1 / (xi − x)2 , which is less than the asymptotic variance of βˆ1 by a factor of (1 + 2σ 2 )−1 . However, this estimate of variance, based on the second derivative matrix of the log-likelihood, is heavily dependent on the assumptions of normality and independence. If the normality assumption is dropped, it can be shown that, asymptotically in n,   var(β1∗ ) = σ 2 1 + 2σ γ1 + σ 2 (γ2 + 2)  (xi − x)2 , (7) × (1 + 2σ 2 )−2 where γ1 and γ2 are the skewness and kurtosis of Yi , assumed constant for each i. In fact, these data were generated from a distribution with σ 2 = 1, β0 = β1 = 0, γ1 = 2,

 γ2 = 6, so that var(β1∗ ) = 13/[9 (xi − x)2 ] or more than four times the apparent variance. Furthermore, although β1∗ appears to be three times as efficient∗ as βˆ1 , it is, in fact, 44% less efficient by (7). Efficiency is, of course, only a minor consideration. A more serious objection to β1∗ is that the true variance (7) is difficult to estimate and naive application of maximum likelihood theory is likely to lead to grossly misleading inferences. REFERENCES 1. Berkson, J. (1980). Ann. Statist., 8, 457–487. [Recommends the method of minimum chisquare, of which (3) is a special case, in the context of discrete data and appears to consider the limit µi → ∞ rather than n → ∞.] 2. McCullagh, P. (1983). Ann. Statist., 11, 59–67. [Establishes asymptotic properties of quasilikelihood statistics assuming a solution to (5) exists.] 3. McCullagh, P. and Nelder, J. A. (1983). Generalized Linear Models. Chapman & Hall, London. (Emphasizes applications.) 4. Nelder, J. A. and Wedderburn, R. W. M. (1972). J. R. Statist. Soc. A, 135, 370–384. (Demonstrates the connection between least squares and maximum likelihood and discusses implications for computing.) 5. Wedderburn, R. W. M. (1974). Biometrika, 61, 439–477. (Introduces the term ‘‘quasilikelihood’’ and considers independent observations satisfying a model of the generalized linear type.) See also ESTIMATION, CLASSICAL; GENERALIZED LINEAR MODELS; LEAST SQUARES; MAXIMUM LIKELIHOOD ESTIMATION; and MINIMUM CHI-SQUARE.

P. MCCULLAGH

QUASILINEAR MEAN. See WEIGHTED QUASILINEAR MEAN

QUASI-LINEARIZATION. See NEWTON –RAPHSON METHODS

QUASI-MEDIANS These concepts were introduced by Hodges and Lehmann [1]. Let x(1)
· · ·
x(n) be the

QUASI-RANDOM SEQUENCES

sample observations ordered from smallest to largest. Then for i = 0, 1, 2, . . ., x (k+1−i) + x(k+1+i)  if n = 2k + 1,  2 θi =   x(k−i) + x(k+1+i) if n = 2k. 2 form a sequence of quasi-medians. (Each θ i is the average of two symmetrically ordered x observations; the sample median is equal to θ 0 ). Hodges and Lehmann computed approximations to the variance θ i . Quasi-medians were used by Hollander and Wolfe [2] as estimators associated with sign statistics. REFERENCES 1. Hodges, J. L. and Lehmann, E. H. (1967). J. Amer. Statist. Ass., 62, 926–931. 2. Hollander, M. and Wolfe, D. A. (1973). Nonparametric Statistical Methods. Wiley, New York. See also PSEUDO-MEDIAN.

QUASI-MONTE CARLO METHODS. See NUMBER-THEORETIC METHODS

QUASI-NEWTON METHODS. See NEWTON –RAPHSON METHODS

QUASI-RANDOM NUMBERS. See QUASI-RANDOM SEQUENCES

QUASI-RANDOM SAMPLING Quasi-random sampling is a special case of systematic sampling∗ . From a list whose elements are in random order, a sample is obtained by choosing every kth element. The first element in the sample is selected randomly from the first k elements in the list. The term ‘‘quasi-random sampling’’ developed in the fields of sociology∗ and economics. Currently it is seldom used. A review of the development of this topic can be found in ref. 1. An early example of the use of quasirandom sampling is ref. 2.

6757

REFERENCES 1. Buckland, W. R. (1951). J. R. Statist. Soc. B, 13, 208–215. 2. Hilton, J. (1924). J. R. Statist. Soc., 87, 544– 570. See also SYSTEMATIC SAMPLING.

STEPHEN G. NASH

QUASI-RANDOM SEQUENCES The concept of a quasi-random sequence arose in the application of number-theoretic* techniques
to Monte Carlo methods∗ . To estimate f over the s-dimensional unit box Is = [0, 1]s , a sequence of random points {xi } ⊂ Is is generated, and the approximation  f

N 1  f (xn ) N n=1

is used. By the strong law of large numbers∗ and the central limit theorem∗ , this converges almost surely, with expected integration error O(N −1/2 ). This error does not depend explicitly on the dimension s (although the implied constant may), making this technique attractive for large-dimensional problems. Although the sequence {xn } can be generated using a pseudo-random number generator∗ , deterministic sequences designed for Monte Carlo integration can be more efficient. Such sequences are called quasirandom. Unlike a pseudo-random sequence, which imitates the behavior of a truly random sample, a quasi-random sequence will have only a particular random characteristic. For this application, the sequence should be more evenly distributed over Is . More precisely, for a set E ⊂ [0, 1], let A (E; N) count the number of xn ∈ E. Define the discrepancy D∗N of the N numbers x1 , . . . , xN in [0, 1] by    A([0, t); N)  − t . D∗N = sup  N 0
t
1 If f is of bounded variation V (f ) on [0, 1], then [5]    1 N 1     f (xn ) − f (x)dx
V(f )D∗N .  N  0 n=1

6758

QUASI-RANDOM SEQUENCES

This formula separates the influences of the function f and the sequence {xn } on the effectiveness of the Monte Carlo scheme. In higher dimensions, a formal analog of this result also holds [4]. In this case, the function f should be of bounded variation in the sense of Hardy and Krause (the variations of f and its restrictions to coordinate subspaces must all be finite). Related bounds can be obtained for some integrals over more general regions [7]. These results suggest using low-discrepancy or quasi-random sequences for Monte Carlo integration. With some such sequences, the error can be reduced to O(N −1 (log N)s−1 ). In one dimension, it can be shown that D∗N  1/(2N) for any N numbers in I, and that D∗N = 1/(2N) for the sequence 1/(2N), 3/(2N), . . . , (2N − 1)/(2N). However, to use this sequence, N must be prespecified. Instead, in practice it is preferable to use portions of a single infinite sequence whose subsequences have low discrepancy for any N. For example, if α is an irrational number that has a continued-fraction expansion with uniformly bounded partial quotients (such as the square root of a prime number), then the sequence of fractional parts {α}, {2α}, . . . satisfies D∗N = O(N −1 log N). Sequences with even lower discrepancies can also be constructed [1,8]. For quasi-random sequences in higher dimensions, see refs. 2, 3, and 9. The techniques above do not exploit regularity in the function f , beyond its bounded variation. If the function is periodic of period 1 in each variable, and if its Fourier coefficients decay sufficiently rapidly, then an approximation of the form  f (x)dx

N 1  n  f g N N n=1

can be used, where N  2 is a fixed integer and g ∈ Zs is an s-dimensional lattice point. The lattice point g is chosen to reduce the error in the approximation above [12,13]. In dimension s = 2, g can be optimally selected [11]; tables exist for 3
s
10 [6].

A similar technique is based on using multiples of a point α = (α1 , . . . , αs ) for which 1, α1 , . . . , αs are linearly independent over the rationals. The function f must again be periodic. An example is α = (a, a2 , . . . , as ), where a = p1/(s+1) and p is prime [7]. If the function f is not periodic, it may be replaced by a related periodic function which has the same integral. Alternatives are to change variables or to add polynomials to the integrand (see refs. 10 and 11). These techniques can increase the cost of the method, however. REFERENCES 1. Faure, H. (1981). Bull. Soc. Math. Fr., 109, 143–182 (in French). 2. Halton, J. H. (1960). Numer. Math., 2, 84–90. 3. Hammersley, J. M. (1960). Ann. New York Acad. Sci., 86, 844–874. 4. Hlawka, E. (1961). Ann. Mat. Pura Appl., 54, 325–333 (in German). 5. Koksma, J. F. (1942–43). Mathematica B (Zutphen), 11, 7–11 (in German). 6. Maisonneuve, D. (1972). In ref. 13, pp. 121–201 (in French). 7. Niederreiter, H. (1973). In Diophantine Approximation and Its Applications, C. F. Osgood, ed. Academic Press, New York, pp. 129–199. 8. van der Corput, J. G. (1935). Nederl. Akad. Wet. Proc., 38, 813–821, 1058–1066 (in Dutch). 9. Warnock, T. T. (1972). In ref. 13, pp. 319–343. 10. Zaremba, S. K. (1970). Aequationes Math., 4, 11–22. 11. Zaremba, S. K. (1970). Fibonacci Quart., 8, 185–198. 12. Zaremba, S. K. (1972). In ref. 13, pp. 39–119 (in French). 13. Zaremba, S. K., ed. (1972). Applications of Number Theory to Numerical Analysis. Academic Press, New York.

BIBLIOGRAPHY See the following works, as well as the references just given, for more information on the topic of quasi-random sequences. Kuipers, L. and Niederreiter, H. (1974). Uniform Distribution of Sequences. Wiley, New York. (Discusses the concepts of discrepancy and equidistribution of sequences.)

QUASI RESIDUALS Niederreiter, H. (1978). Bull. Amer. Math. Soc., 84, 957–1041. (Extensive survey of the theory of quasi- and pseudo-Monte Carlo methods.) See also RANDOM NUMBER GENERATION; MONTE CARLO METHODS; NUMBER-THEORETIC METHODS; and PSEUDO-RANDOM NUMBER GENERATORS.

STEPHEN G. NASH

QUASI-RANGES. See RANGES

6759

equal sample sizes. All these tests are based on quasi residuals from the fitted curves. The testing of equality of the regression curves typically requires linear estimation of the underlying regression curves [2,3,7]. One such linear estimator is a version of the kernel estimator of a regression function. For example, in estimating f and g using the samples (Y1i , x1i ), i = 1, . . . , n, and (Y2j , x2j ), j = 1, . . . , m, respectively, we define a kernel estimator [2] as fˆ (x) =

QUASI RESIDUALS

n 

Y1i w1i (x),

(3)

i=1

Consider two regression models in which the data are observed in the form {(Y1i , x1i ), i = 1, . . . , n} and {(Y2j , x2i ), j = 1, . . . , m}, where Y1i = f (x1i ) + i ,

i = 1, . . . , n,

Y2j = g(x2j ) + ηj ,

j = 1, . . . , m,

(1)

in which i , i = 1, . . . , n, and ηj , j = 1, . . . , m, are independent, random errors for the two groups. One is interested in testing the hypothesis H0 : f = g vs. H1 : f = g

(2)

over the domain of the covariate (x) measurements, which can be taken as [0, 1] for convenience. The covariate values (design points) for the two samples can be different, and the sample sizes can be unequal. It is usually further assumed that within each group the errors are identically distributed with mean zero and that the distributions of the ’s and η’s may be different with var(1 ) = σ 2 and Var(η1 ) = τ 2 . Testing hypotheses of the above type in a nonparametric setting has been discussed [4,5,6], as have tests assuming that m = n and that the covariate values x1i and x2i are the same for the two samples [1,4,5]. The equality of designs and/or sample sizes for the two populations may not be easy to achieve in practical studies. Equality of designs also fails to hold when the design variable is random. Kulasekera [6] gives three tests for testing hypotheses (2) using independent samples from the two populations, where the tests do not require common design points or

where  w1i (x) =

s1,i s1,i−1

K((x − u)/h1 ) du, h1

s1,0 = 0, s1,i = (x1i + x1i+1 )/2, and ˆ g(x) =

m 

and

s1,n = 1,

Y2j w2j (x),

(4)

i=1

where  w2j (x) =

s2,j s2,j−1

K((x − u)/h2 ) du, h2

s2,0 = 0, s2,j = (x2j + x2j+1 )/2, and s2,m = 1. Here, h1 and h2 are the bandwidths of the kernel estimator, and they are typically functions of the sample sizes n and m respectively. It is assumed without loss of generality that covariate values are ordered for each sample. Once the estimators of f and g are defined, quasi residuals are defined as ˆ 1i ), e1i = Y1i − g(x

i = 1, . . . , n,

(5)

e2j = Y2j − fˆ (x2j ),

j = 1, . . . , m.

(6)

From the properties of kernel estimators of regression functions [2], we have E(e1i ) = f (x1i ) − g(x1i ) + O(h22 ), E(e2j ) = g(x2j ) − f (x2j ) + O(h21 ), Var(e1i ) = σ 2 + O((mh2 )−1 ), and Var (e2j ) = τ 2 + O((nh1 )−1 ). Also, Cov(e1i , e1l ) = O((mh2 )−1 ) for |x1i − x1l |
2h2 and Cov(e1i , e1l ) = 0 otherwise. Similar assertions hold for the e2j ’s. Thus, the quasi residuals act almost as residuals from each sample under the assumption f = g.

6760

QUASISCORE

REFERENCES 1. Delgado, M. A. (1993). Testing the equality of the nonparametric regression curves. Statist. Probab. Lett., 17, 199–204. 2. Eubank, R. L. (1988). Spline Smoothing and Nonparametric Regression. Marcel Dekker, New York. 3. Fan, J. (1992). Design adaptive nonparametric regression. J. Amer. Statist. Ass., 87, 998–1004. 4. Hall, P. and Hart, J. D. (1990). Bootstrap test for difference between means in nonparametric regression. J. Amer. Statist. Ass., 85, 1039–1049. 5. King, E. C., Hart, J. D., and Wehrly, T. E. (1991). Testing the equality of two regression curves using linear smoothers. Statist. Probab. Lett., 12, 239–247. 6. Kulasekera, K. B. (1995). Testing the equality of regression curves using quasi residuals. J. Amer. Statist. Ass., 90, 1085–1093. ¨ 7. Muller, H. -G. (1987). Nonparametric Regression Analysis of Longitudinal Data. SpringerVerlag, Berlin.

The conditions (1), unbiasedness, and (2), the information identity, are familiar from the theory of maximum likelihood estimation∗ , where they apply to the score function u(Y, β) = ∇l(y, β) obtained by taking partial derivatives of the log likelihood l. In what situations, short of a full likelihood specification, does there exist a u satisfying (1) and (2)? The answer, which is amplified in the entry on QUASI-LIKELIHOOD FUNCTIONS, is that if the means, variances, and covariances of Y are specified: E(Y) = µ(β), cov(Y) = σ 2 V(β), where µ(·) is a vector and V(·) a matrix of known functions, then u = DT V−1 (Y − µ)/σ 2

(3)

In applications, β T = (β1 , . . . , βp ) is typically a vector of regression coefficients, e.g., coefficients in a linear or generalized linear model∗ . Quasiscore estimates βˆ are defined as solutions of the quasiscore equations (also known as quasi-likelihood equations),

is a quasiscore function, where D = ∂µ/∂β. Subject to certain technical and limiting conditions (see [1,9]), estimates of β obtained by solving u = 0 are consistent and asymptotically normal, with large-sample covariance matrix [i(β)]−1 , where i(β) is the quasiinformation matrix cov(u) = −E(∂u/∂β) = DT V−1 D/σ 2 . Moreover, the quasiscore estimator is optimal among estimators obtained as solutions to unbiased estimating equations linear in Y, a result which generalizes the Gauss–Markov theorem∗ on optimality of least squares in the linear model [3,9]. Estimation of σ 2 , if σ 2 is unknown, must be done by another method; most often the ˆ − p), where n = estimate is σ˜ 2 = X 2 (β)/(n dim(Y) and X 2 (β) = [Y − µ(β)]T V−1 (β)[Y − µ(β)]. If u is a quasiscore function, and if there exists a function l(β) such that ∇l = u, then l is said to be a quasi (log) likelihood. To first order, the asymptotic properties of a quasi likelihood are the same as those of a true likelihood, so that large-sample confidence intervals and tests of hypotheses about β may be based on quasi-likelihood ratio statistics. A quasi likelihood exists if u is a conservative vector function, i.e., if ∂ur /∂βs = ∂us /∂βr for all r, s [11, §9.3]. If V is of the particular form

ˆ = 0. u(y, β)

V(β) = diag{V1 [µ1 (β)], . . . , Vn [µn (β)]},

K. B. KULASEKERA

QUASISCORE The terms quasiscore and quasi likelihood∗ are often used interchangeably, but also sometimes to distinguish situations where a quasiscore function exists but there is no corresponding quasi-likelihood. If the distribution of a random vector Y depends on a p-dimensional vector parameter β, a quasiscore is a p-dimensional vector function u(Y, β) which satisfies E[u(Y, β)] = 0, cov[u(Y, β)] = −E

(1) ∂u . ∂β

(2)

QUASISCORE

then u is conservative, and indeed it was for this case that the term ‘‘quasi likelihood’’ was first used by Wedderburn [14]. More generally, ∂u/∂β typically is asymmetric; a notable exception is when V, although possibly nondiagonal, does not depend on β, in which case l is simply the log likelihood for the multivariate Gaussian model Y ∼ N[µ(β), σ 2 V]. A simple example of the more typical behavior is the quasi-bivariate-Poisson specification for a pair of counts Y = (Y1 , Y2 ): µT = (µ1 , µ2 ) = [exp(β1 ), exp(β2 )],

√ α µ1 µ2 µ √ 1 , cov(Y) = V(β) = α µ1 µ2 µ2 where α is a correlation parameter; here ∂u1 /∂β2 = ∂u2 /∂β1 . Where ∂u/∂β is asymmetric and so no quasi likelihood exists, various constructions have been proposed in its place [6,7,4]; alternatively, inference may be based on quasiscore tests and associated confidence regions constructed using the asymptotic χp2 distribution [10] of the quasiscore statistic Wu (β) = uT [i(β)]−1 u. In many applications, the specified covariance structure V(·) is tentative. For example, in a regression model for counts where there is overdispersion∗ relative to the Poisson distributions, a working assumption may be that var(Yi ) = σ 2 µi , where σ 2 > 1 is a constant intended to accommodate the extraPoisson variation; in the case of overdispersed binomial proportions, it might be reasonable to suppose, again as a working assumption, that var(Yi ) = σ 2 πi (1 − πi )/mi , where {mi , πi (β)} are the binomial index and probability for the ith case. In such situations, the assumed mean–variance relationship is often an empirical guess, rather than a form derived from theoretical arguments, say. If V is incorrectly specified, consistency of βˆ is usually unaffected, but standard errors derived from [i(β)]−1 are systematically biased. In general, the asymptotic variance matrix of βˆ is the information sandwich i−1 DT V−1 cov(Y)V−1 Di−1 /σ 4 ,

(4)

which reduces to i−1 if V is correct. An obvious estimate of (4) is obtained by substiˆ ˆ T for cov(Y), and tuting [Y − µ(β)][Y − µ(β)]

6761

use of this in place of i−1 to derive standard errors provides some protection against misspecification of V [12,13]. The use of quasiscore estimates in conjunction with this type of robust covariance estimator forms the basis of the so-called generalized estimating equations∗ approach [8] to the analysis of clustered and longitudinal∗ data. Major applications of quasiscore methods include accommodation of overdispersion when modeling counts or proportions, and generalized linear models for clustered and longitudinal data, as mentioned. Quasiscore methods have been found useful also in connection with inference on stochastic processes∗ when likelihood methods are intractable. These and other applications are described, along with more detailed theoretical development, in [11, Chap. 9] and in recent review articles [2,5,10]. REFERENCES 1. Crowder, M. J. (1987). On linear and quadratic estimating functions. Biometrika, 74, 591–597. (Exemplifies situations in which quasiscore estimators are inconsistent.) 2. Firth, D. (1993). Recent developments in quasi-likelihood methods. Bull. Int. Statist. Inst., 55 (Book 2), 341–358. (Reviews quasiscore methods and their applications.) 3. Godambe, V. P. and Heyde, C. C. (1987). Quasi-likelihood and optimal estimation. Int. Statist. Rev., 55, 231–244. (Theoretical development of quasiscore methods with particular reference to stochastic processes.) 4. Hanfelt, J. J. and Liang, K. -Y. (1995). Approximate likelihood ratios for general estimating functions. Biometrika, 82, 461–477. 5. Heyde, C. C. (1989). Quasi-likelihood and optimality for estimating functions: some current unifying themes. Bull. Int. Statist. Inst., 53 (Book 1), 19–29. (A brief review of some foundational and theoretical aspects.) 6. Li, B. (1993). A deviance function for the quasi-likelihood method. Biometrika, 80, 741–753. 7. Li, B. and McCullagh, P. (1994). Potential functions and conservative estimating functions. Ann. Statist., 22, 340–356. 8. Liang, K. -Y. and Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models. Biometrika, 73, 13–22. (Gives quasiscore methods, in conjunction with the

6762

QUASI-SYMMETRY

sandwich covariance estimator, for longitudinal and clustered data: the generalizedestimating-equations approach.) 9. McCullagh, P. (1983). Quasi-likelihood functions. Ann. Statist., 11, 59–67. (Extends the notion of a quasiscore to correlated data, and gives the main asymptotic results.) 10. McCullagh, P. (1991). Quasi-likelihood and estimating functions. In Statistical Theory and Modelling, D. V., Hinkley, N. Reid, and E. J. Snell, eds. Chapman and Hall, London, pp. 265–286. (An accessible introduction and review of theoretical aspects.) 11. McCullagh, P. and Nelder, J. A. (1989). Generalized Linear Models, 2nd ed. Chapman and Hall, London. (Especially Chapters 9, 10; emphasis on applications.) 12. Pregibon, D. (1983). An alternative covariance estimate for generalized linear models. GLAIM Newsletter, 6, 51–55. (An early paper suggesting the sandwich covariance estimator in generalized linear models.) 13. Royall, R. M. (1986). Model robust confidence intervals using maximum likelihood estimators. Int. Statist. Rev., 54, 221–226. (General treatment of the sandwich covariance estimator for maximum likelihood estimates.) 14. Wedderburn, R. W. M. (1974). Quasilikelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika, 61, 439–447. (Introduces and develops the idea of a quasiscore in the context of a generalized linear model with uncorrelated responses.)

of a t-dimensional Response (R1 , . . . , Rt ) measured on a randomly selected Individual, each Ri taking one of the values 1, . . . , r. The probability that R1 = j1 , . . . , Rt = jt is denoted by ρ(j1 , . . . , jt ) = ρ(j). The marginal probability that Ri = j is denoted by ρji and the hypothesis H2 of marginal symmetry is H2 : ρji is constant with respect to i. A fuller description of the following examples, together with sample data, is given in MS. Example 1. Individual = voter in a mayoral election being contested by three candidates, t = 2, r = 3; R1 and R2 are the voting preferences one month before and one week before the election. Example 2. Individual = boy, t = 3, r = 2; R1 , R2 , and R3 are each binary responses with levels 1 = considered inactive, 2 = considered active; R1 is a self-assessment and R2 and R3 are assessments by the boy’s parent and teacher. Quasi-symmetry is easier to appreciate if we first consider the hypothesis H0 of complete symmetry. In Example 1, and in general when t = 2, H0 simply means that the matrix (ρ(j1 , j2 )) is symmetric, that is, H0 : ρ(j1 , j2 ) = ρ(j2 , j1 ).

See also ESTIMATING EQUATIONS, THEORY OF; ESTIMATING FUNCTIONS; ESTIMATION, CLASSICAL; GENERALIZED LINEAR MODELS; MAXIMUM LIKELIHOOD ESTIMATION; and QUASI-LIKELIHOOD FUNCTIONS.

DAVID FIRTH

QUASI-SYMMETRY

In Example 2 H0 is defined by H0 : ρ(1, 1, 2) = ρ(1, 2, 1) = ρ(2, 1, 1) and ρ(1, 2, 2) = ρ(2, 1, 2) = ρ(2, 2, 1). In general, H0 is defined by

QUASI-SYMMETRY, MARGINAL SYMMETRY, AND COMPLETE SYMMETRY

H0 : ρ(j1 , . . . , jt ) is constant with respect to

This entry should be read in conjunction with the entry MARGINAL SYMMETRY, henceforth referred to as MS; much of the practical interest in quasi-symmetry lies in its role as a background structure against which to test the null hypothesis of marginal symmetry. Both marginal symmetry and quasisymmetry are properties of the distribution

It is obvious that H0 ⇒ H2 ; that is, marginal symmetry is more general than complete symmetry. Quasi-symmetry, denoted by H1 , is also more general than H0 . It can be defined by the symmetry of the log-linear interactions, but the best working definition of H1 says that ρ(j1 , . . . , jt ) is the product of a function

permutations of (j1 , . . . , jt ).

QUASI-SYMMETRY

of j1 , . . ., a function of jt , and a completely symmetric function of j1 , . . . , jt ; that is, H1 : ρ(j1 , . . . , jt ) = λ1j1 . . . λtjt ν(j1 , . . . , jt ), where ν is completely symmetric. In the case t = 2, H1 says that ρ(j1 , j2 ) = λ1j λ2j ν(j1 , j2 ). 1 2 Further, H1 is expressible through the crossproduct ratio conditions ρ(j, j )ρ(j , j )ρ(j , j) = 1. ρ(j, j )ρ(j , j )ρ(j , j)

(1)

Since H0 ⇒ H1 and H0 ⇒ H2 it is immediate that H0 ⇒ H1 ∧ H2 , the latter hypothesis denoting the conjunction of H1 and H2 . Not immediate, but more important, is that H1 ∧ H2 ⇒ H0 . In other words, complete symmetry is the conjunction of quasi-symmetry and marginal symmetry. A proof is given by Darroch and Speed [6]. None of the symmetry properties involve the ‘‘diagonal’’ probabilities ρ(j, . . . , j). It follows from H1 ∧ H2 ⇔ H0 that, assuming that quasi-symmetry holds, testing marginal symmetry is equivalent to testing complete symmetry. In this way the problem of testing a linear hypothesis, namely marginal symmetry, can be handled using log-linear methodology, because quasi-symmetry is loglinear and complete symmetry is log-linear (and linear). Proofs of results stated here, and other details, are given in Darroch [5].

6763

GLIM∗ ). The hypothesis H1 should first be tested against the alternative hypothesis H of no restriction, and this test can also be performed as a likelihood-ratio test. A good discussion of this test and of other aspects of quasi-symmetry is given by Bishop et al. [3, pp. 286–293]. Darroch [5] constructed the ‘‘conditional likelihood score’’ statistic for testing H0 against H1 . It has the theoretical advantage over other tests of being expressible in closed form. It looks at the magnitude of the differences dij = yij − y.j , where yij is the marginal frequency of Ri = j and yj is the average over i. Let d denote the vector of the (t − 1)(r − 1)dij s, i = 1, . . . , t − 1, j = 1, . . . , r − 1. The conditional likelihood score statistic takes the form d Vd−1 d. It closely resembles the adjusted Wald statistic −1

˜ d d, d  for testing H2 against H, discussed in MS. Both statistics involve the univariate marginal frequencies yij and the bivariate mar

ginal frequencies yiijj , i = i . The typical ele˜ d are ments of Vd and 

TESTS OF MARGINAL SYMMETRY ASSUMING QUASI-SYMMETRY Denote by y (j) the frequency of response j in a sample of s individuals. The probability distribution of the y (j) is multinomial and underlies the various test statistics that have been used for testing H2 : marginal symmetry assuming H1 : quasi-symmetry, that is, for testing H0 : complete symmetry against H1 . All test statistics are approximately chisquare on H0 and have (t − 1)(r − 1) degrees of freedom. The likelihood-ratio test∗ is possibly the easiest to perform given the wide availability of computing systems for performing likelihood-ratio tests of log-linear hypotheses (e.g.,

Vd : cov(dij , dij ) = (δii − t−1 )(δjj y.j − y..jj ), (2) ˜ d : cov(di , di )  j j



= (1 − δii )yiijj − (1 − t−1 )(yi.jj + y.ijj − y..jj )

+ δjj {δii yij − t−1 (yij + yij − y.j )}.

(3)

The averages yi.jj and y..jj are calculated over i : i = i and over (i, i ) : i = i. When t = 2, (2) and (3) are identical. In other words, we get the same answer in testing H2 , whether or not we test and accept H1 first. Example 1 (See MS). The likelihood ratio test statistic for testing H1 : quasi-symmetry against H is 31.65, which, on one degree of freedom, is very highly significant. Thus

6764

QUASI-SYMMETRY

quasi-symmetry is not acceptable. This can be seen directly [recall (1)] from the crossproduct ratio 17 × 22 × 15 = 0.07, 72 × 35 × 32 which is very different from 1. The likelihood ratio test statistic for testing H0 against H1 , that is, for testing H2 assuming H1 , is 16.63. The conditional likelihood score statistic∗ is 16.29, identical to the adjusted Wald statistic for testing H2 against H, which was reported in MS. All these tests are on 2 degrees of freedom. Example 2 (See MS). The likelihood ratio test statistic for testing H1 against H is 2.73, which, on two degrees of freedom, shows that H1 , is very acceptable. The likelihood ratio test statistic for testing H2 assuming H1 is 18.84, which, on two degrees of freedom, makes H2 : marginal symmetry unacceptable. The conditional likelihood score statistic calculated using (2) is 19.19. The adjusted Wald statistic for testing H2 against H is 16.29, the same value as in Example 1. (See MS for a discussion of this coincidence.) MATCHED PAIRS AND t-PLETS The individual may comprise a matched pair∗ (t = 2) or, in general, a matched t-plet of individuals, on the ith of which is measured the response Ri . Example 3 (See MS). t = 2, r = 3, Individual = matched (case, control) pair, the case and control being people under 30 with and without heart disease. The response is level of smoking. The likelihood ratio test statistic for testing H1 against H is 0.34, which, on one degree of freedom, shows that H1 is acceptable. The likelihood ratio test statistic for testing H2 assuming H1 is 16.51, while the conditional likelihood score statistic is 15.52. The latter value is the same as the value of the adjusted maximum likelihood statistic, reported in MS.

of individuals is divided into S homogeneous strata. In Example 3, for instance, each stratum is homogeneous with respect to sex, age, occupation, and place of residence. In the hth stratum let πh1j and πh2j be the probabilities of response j for the case and control. The probability of a matched pair from a randomly selected stratum exhibiting response (j1 , j2 ) is then ρ(j1 , j2 ) = S−1

S 

πh1j1 πh2j2 .

h=1

In MS it was shown that H2 is equivalent to G2 where G2 : π.1j = π.2j , with ‘‘.’’ denoting average. G2 may thus be described as average (over strata) disease, exposure independence. The hypotheses H0 and H1 are related by implication to two hypotheses which feature prominently in the analysis of stratified frequencies. These are, in the language of casecontrol studies, G0 : within stratum (disease, exposure) independence and G1 : no (stratum, disease, exposure) interaction. They are defined by G0 : πhij = φhj , G1 : πhij = θhi φhj ψij . Therefore, under G0 , ρ(j1 , j2 ) = S−1

S 

φhj1 φhj2 .

(4)

h=1

Property (4) implies the complete symmetry of (ρ(j1 , j2 )). Thus G0 implies H0 . Similarly, G1 implies H1 , because, under G1 , ρ(j1 , j2 ) = ψ1j1 ψ2j2 ν(j1 , j2 ), where ν(j1 , j2 ) = S−1

S 

θh1 θh2 φhj1 φhj2

h=1

The matching of the individuals in a (case, control) pair, or more generally in a t-plet, can be modeled as follows. The population

and is completely symmetric. Thus there is a close relationship between no interaction and

QUASI-SYMMETRY

quasi-symmetry which helps to give credence to the latter. McCullagh [10] discusses other aspects of the relationship; see the section ‘‘Other Aspects of Quasi-Symmetry.’’ Under G0 the matrix (ρ(j1 , j2 )) given by (4) is seen to be nonnegative definite. This implies, for instance, that ρ(j, j)ρ(j , j ) 1 ρ(j, j )ρ(j , j) and this inequality also holds under G1 . The amount by which the left side is greater than 1 can be shown to reflect the effectiveness of the matching. Example 3 (Continued). The sample versions of these inequalities are not all satisfied since 13 × 5 < 1. 16 × 6 However, this discrepancy is not significant. CONNECTIONS WITH THE MANTEL– HAENSZEL TEST AND COCHRAN’S Q-TEST statistic∗

is designed The Mantel–Haenszel for the following problem. There are s fixed strata, rather than a random sample of strata. There are t treatments to be compared. Some individuals from each stratum are given treatment i, i = 1, . . . , t. A response, taking one of the values 1, . . . , r, is measured on each individual. The problem is to test for treatment–response independence after allowing for strata. A very common application is to the retrospective study in which t = 2, the two ‘‘treatments’’ being the possession or nonpossession of a disease and the ‘‘response’’ being the exposure variable. Birch [1,2] proved that the Mantel– Haenszel statistic is the conditional likelihood score statistic CLS (G0 |G1 ) for testing G0 against G1 . It turns out that when there is only one individual per (stratum, treatment), the Mantel–Haenszel statistic, which is expressed as a function of an s × t × r table (xhij ), is also expressible as a function of the rt table (y (j)). Darroch [5] found that when this is done, CLS(G0 |G1 ) =

6765

˜ −1 CLS(H0 |H1 ), where CLS(H0 |H1 ) = d  d d, the conditional likelihood score statistic that we had earlier. In other words, when there is only one individual per (stratum, treatment), the Mantel–Haenszel test can be interpreted as a test of marginal symmetry assuming quasi-symmetry. Earlier, Mantel and Haenszel [9] and Mantel and Byar [8] had shown, mainly through numerical examples, that there must be strong connections between the Mantel–Haenszel test and tests of marginal symmetry. Cochran’s [4] Q-test is a special case of the Mantel–Haenszel test, specialized to the case of matched t-plets and a binary response (r = 2). Thus it too may be interpreted as a test of marginal symmetry assuming quasisymmetry (see PROPORTIONS, COCHRAN’S QSTATISTIC FOR). OTHER ASPECTS OF QUASI-SYMMETRY Referring to the section ‘‘Matched Pairs and t-Plets,’’ let ρh (j1 , j2 ) = πh1j1 πh2j2 , the probability that a matched pair from stratum h exhibits response (j1 , j2 ). Then, under G1 , ρh (j1 , j2 )/ρh (j2 , j1 ) is independent of h and has the same value as ρ(j1 , j2 )/ρ(j2 , j1 ) under H1 , namely the ψ cross-product ratio ψ1j1 ψ2j2 . ψ1j2 ψ2j1 McCullagh [10] notes this fact, its implications, and its generalization to t > 2. He also discusses the fact that the transition matrix of a reversible Markov chain is quasisymmetric. Fienberg and Larntz [7] discuss the relationship between the Bradley–Terry model∗ for paired-comparison∗ probabilities and quasi-symmetry. REFERENCES 1. Birch, M. W. (1964). J. R. Statist. Soc. B, 26, 313–324. 2. Birch, M. W. (1965). J. R. Statist. Soc. B, 27, 111–124.

6766

QUENOUILLE’S ESTIMATOR

3. Bishop, Y. M. M., Fienberg, S. E., and Holland, P. W. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Cambridge, Mass. 4. Cochran, W. G. (1950). Biometrika, 37, 256– 266. 5. Darroch, J. N. (1981). Int. Statist. Rev., 49, 285–307. 6. Darroch, J. N. and Speed, T. P. (1979). Multiplicative and Additive Models and Interactions. Res. Rep. 49, Department of Theoretical Statistics, Aarhus University, Aarhus, Denmark. 7. Fienberg, S. E. and Larntz, K. (1976). Biometrika, 63, 245–254. 8. Mantel, N. and Byar, D. P. (1978). Commun. Statist. A, 7, 953–976. 9. Mantel, N. and Haenszel, W. (1959). J. Natl. Canad. Inst., 22, 719–748. 10. McCullagh, P. (1982). Biometrika, 69, 303– 308.

BIBLIOGRAPHY

where {at } is a series of independent, identically distributed variates and the constants φ1 , . . . , φp are the autoregressive coefficients. Quenouille [4] showed that, defining rk to be the kth sample correlation rk =

n 

(yt − y)(yt−k − y)

t=1

t=k+1

and writing φ0 = −1, the quantities Rk =

p p  

Haberman, S. (1979). Analysis of Qualitative Data, Vol. 2. Academic Press, New York. Plackett, R. L. (1981). Analysis of Categorical Data, 2nd ed. Charles Griffin, London. See also LOG-LINEAR MODELS IN CONTINGENCY TABLES; MANTEL –HAENSZEL STATISTIC; and MARGINAL SYMMETRY.

J. N. DARROCH

QUENOUILLE’S ESTIMATOR. See BIAS REDUCTION, QUENOUILLE’S METHOD FOR

QUENOUILLE’S TEST In its simplest form Quenouille’s test is a significance test∗ for autoregressive timeseries∗ models. An observed time series {yt : t = 1, . . . , n} is an autoregression of order p if yt − φ1 yt−1 − · · · − φp yt−p = at ,

φi φj rk−i−j ,

k > p,

i=0 j=0

are normally distributed with mean zero and variance n−1 [var(at )/ var(yt )]2 in large samples. This result is still true when the φj are replaced by estimated values obtained from least-squares∗ fitting of the autoregressive model, so the Rk may be used to test the fit of the model, either individually or via the statistic 

See the following works, as well as the references just given, for more information on the topic of quasi-symmetry. Breslow, N. E. and Day, N. E. (1980). Statistical Methods in Cancer Research. IARC Sci. Publ. No. 32. IARC, Lyon, France.

 n (yt − y)2 ,

Qm = n

var(yt ) var(at )

2  m

R2p+k ,

k=1

2 distribution which has an asymptotic χm when the fitted model is true. This is Quenouille’s test. As an example, Quenouille’s test will be applied to series F of Box and Jenkins [1]. The series consists of 70 consecutive yields from a batch chemical process, and inspection of the partial correlations∗ of the series suggests that a first- or second-order autoregression may be an appropriate model. For a first-order model calculation of R2 leads to a test statistic Q1 = 2.42, which corresponds to a significance level of 12% for a χ12 distribution. Although not highly significant, this value suggests that it may be worthwhile to consider a model of higher order. When a second-order model is fitted there is no doubt about the fit; none of the first dozen Rk statistics is significant and the overall goodness-of-fit∗ statistic Q15 , based on R3 through R17 , has the value 8.55, which is not at all significant and indicates no inadequacy of the model. The quantities Rk are seen to be linear combinations of the sample correlations, weighted so as to make them independent.

¨ ´ QUESTII O

Another interpretation has been given by Quenouille [5]; Rk is asymptotically equivalent to the partial correlation between yt and yt−k with the intervening variables yt−i , i = 1, . . . , k − 1, held constant. This interpretation means that Quenouille’s test is particularly apt for autoregressive models, since these are commonly identified and their order estimated by examination of the partial correlation function of the observed series. In view of this point and the fact that its betterknown rival, the portmanteau test∗ for residual correlation, does not share its advantage of consisting of independent components, Quenouille’s test deserves to be more widely known and used than it is at present. Quenouille’s test can be extended to cover a wider range of time-series∗ models, although with some loss of convenience. For autoregressive-integrated moving-average (ARIMA)∗ processes, a test can be based on the Rk [6], although these are no longer independent. Alternatively, one may generalize the partial correlations by defining statistics dˆ i , which for an ARIMA process are independent of each other and of the estimated parameters and are such that the test statis tic n dˆ 2i has an asymptotic χ 2 distribution if the fitted model is true [2]. Hosking [3] has also derived an extension of Quenouille’s test to multivariate ARIMA processes.

6767

¨ ´ QUESTII O This journal was founded by a number of Departments of the Universidad Polit´ecnica de Catalunya in 1977. Currently it appeares twice a year, the two issues constituting a volume. The goal of the journal was to publish research work in any of the languages spoken in Spain, in French, or in English. Early editors of the journal have been Xavier Berenguer (1977–1979), Narcis Nabona (1980–1983), Albert Prat (1983– 1985), and Josep Ma . Oller (1986—). In 2003, the journal was renamed SORT. Its website is www.idescat.es/sort/ The editorial policy of the journal has changed little since its foundation. It is aimed to achieve a balance between theoretical and applied articles in the following areas:

STATISTICS E-0 Methods. E-1 Statistical inference. E-2 Stochastic processes. E-3 Statistical methods for estimate and decision making. E-4 Data analysis. E-5 Planning of experiments. E6 Sampling of finite populations. E-7 Nonparametric statistics. E-15 Applications.

SYSTEMS REFERENCES 1. Box, G. E. P. and Jenkins, G. M. (1970). Time Series Analysis Forecasting and Control. Holden-Day, San Francisco. 2. Hosking, J. R. M. (1980). J. R. Statist. Soc. B, 42, 170–181. 3. Hosking, J. R. M. (1981). J. R. Statist. Soc. B, 43, 219–230. 4. Quenouille, M. H. (1947). J. R. Statist. Soc. A, 110, 123–129.

S-0 Methods. S-1 Modeling, estimation, and simulation. S-2 Adaptive and learning systems. S-3 Form recognition. S-4 Biosystems. S-5 Control theory (linear, nonlinear, and stochastic systems, optimum control . . .). S15 Specific applications of control (generative systems and transmission of energy flows—industrial applications, communications, agriculture, and other natural systems—socio-economic systems).

5. Quenouille, M. H. (1949). J. R. Statist. Soc. B, 11, 68–84. 6. Walker, A. M. (1950). J. R. Statist. Soc. B, 12, 102–107. See also AUTOREGRESSIVE –INTEGRATED MOVING AVERAGE (ARIMA) MODELS; PORTMANTEAU TEST; and TIME SERIES.

J. R. M. HOSKING

COMPUTER SCIENCE I-0 Methods. I-1 Supervisory systems (monitors, operative systems, data bases . . .). I2 Processors (compilers, generators, interpreters . . .). I-3 Programming languages. I4 Software evaluation. I-5 Logic design of

6768

QUESTIONNAIRES

hardware. I-6 Components and circuits. I7 Logic and formal systems. I-8 Automata. I-9 Program analysis and effectiveness. I15 Applications (engineering, scientific calculation, artificial intelligence, research, real time . . .). OPERATIONS RESEARCH IO-0 Methods. IO-1 Linear programming (continuous, complete and mixed, stochastic). IO-2 Nonlinear programming. IO-3 Directed exploration. IO-4 Network analysis. IO-5 Dynamic programming (deterministic, probabilistic). IO-6 Decisions and games. IO-7 Queueing. IO-9 Research. IO-10 Simulation. IO-15 Applications. No ranking is implied in this list. All submissions are refereed by at least two referees, some of them from outside Spain. The journal also accepts case studies that are of potential interest to people working in industry. The contents of issues in the 1980s include: E. J. Hannan, Multivariable ARMA systems and practicable calculations. Tunnicliffe Wilson, Problems in scientific time series analysis. Arthur B. Treadway, On the properties typical of economic time series. William Kruskal, Concepts of relative importance. G. Arnaiz Tovar and C. Ruiz Rivas, Outliers in circular data: A Bayesian approach. M. Pilar Garcıa-Carrasco, Distribuciones mınimo informativas, caso de espacio param´etrico finito. M. P. Galindo Villard´on, Una alternativa de ´ representaci´on simultanea: HJBIPLOT. ´ and A. Moya, GenJ. Dopazo, A. Barbera, eraci´on de selecci´on dependiente de frecuencias en sistemas de dos competidores: 1 modelo del espacio biol´ogico unidad aplicado a competidores m´oviles. Andr´es Moya, Problemas de metrizaci´on y medici´on en biologıa de poblaciones. Finally, the journal invites well-known specialists to publish surveys on their subject matter. ALBERT PRAT

QUESTIONNAIRES. See QUESTIONWORDING EFFECTS POLLS

IN

SURVEYS; PUBLIC OPINION

QUESTION-WORDING EFFECTS IN SURVEYS The data collection∗ instrument in survey research (see SURVEY SAMPLING) is the questionnaire, a document that contains the set of questions for which responses are required. The questionnaire may be administered by interviewers, either face to face with respondents or by telephone, or it may be a selfadministered form on which respondents record their answers, as in mail surveys. In interview surveys, interviewers are usually instructed to adhere rigidly to the questionnaire, keeping to the designated order of questions and asking them exactly as specified. If a respondent fails to provide a satisfactory answer, the interviewer is permitted only to follow up with standard probes or to repeat the question. Whichever mode of data collection is used, the questionnaire is a carefully constructed standardized instrument that aims to collect data that can be meaningfully aggregated for statistical analyses. Standardization is needed to yield comparable data from different respondents, for it has been well documented that survey responses can be sensitive to variations in the precise wording, format, and context of the questions asked. Much of the research on question wording, format, and context deals with opinion questions. This research has mainly employed a split-ballot technique in which alternative forms of a question are administered to comparable samples of respondents. Numerous split-ballot experiments were conducted during the 1940s and 1950s, with the results being reported in the Public Opinion Quarterly and similar journals, in the collection of papers by Cantril [5], and in the classic The Art of Asking Questions by Payne [12]. After this spate of activity, interest in such experiments waned for a time but has now revived. The results of a recent wideranging program of research on opinion questions are reported by Schuman and Presser [16]. See also PUBLIC OPINION POLLS.

QUESTION-WORDING EFFECTS IN SURVEYS

Rugg [13] conducted a well-known early split-ballot experiment in which he compared responses to the questions ‘‘Do you think the United States should allow public speeches against democracy?’’ and ‘‘Do you think the United States should forbid public speeches against democracy?’’ While only 25% of respondents to the first question would allow such speeches, 46% of respondents to the second question would not forbid them (excluding the ‘‘don’t knows’’). The approximately 20% difference between these percentages, which has been replicated by Schuman and Presser [16], illustrates how responses to opinion questions can sometimes be affected by what might appear to be slight wording changes. In addition to wording changes per se, research on opinion questions has examined the effects of various aspects of question format. One issue concerns the balance built into the question between the alternative opinions offered to the respondent. Thus the question ‘‘Do you favor X?’’ is unbalanced, ‘‘Do you favor or oppose X?’’ has a token balance, while ‘‘Do you favor X or Y?’’ is balanced. Experimental comparisons between the token and unbalanced forms have generally found only small differences in response distributions, but the balanced form has sometimes given rise to large differences. The inclusion of the Y alternative in the balanced form may, however, effectively change the question so that these differences cannot necessarily be attributed to format variation [8]. A related issue is that of an acquiescence response set, which denotes a tendency for respondents to give ‘‘agree’’ rather than ‘‘disagree’’ responses regardless of content. There is some evidence to support the existence of this set [16]. Another group of question format issues concerns the alternative response options offered to respondents. An initial consideration is whether respondents should be supplied with a list of alternative responses (a closed question) or whether they should be left to compose their own answers (an open question). Experiments have demonstrated that open and closed questions can produce markedly different response distributions. Moreover, although constructing the alternatives for the closed question to correspond to the open question responses reduces

6769

the differences, it does not remove them entirely [14,16]. When, as is often the case, a closed question is used, decisions then need to be made on whether to offer a ‘‘don’t know’’ alternative explicitly or to accept such an answer only when it is given spontaneously, whether to offer a middle neutral alternative or not, and in which order to present the alternatives. As some experimental studies have shown, many respondents who are ignorant about the issue involved in a closed question will nevertheless choose one of the alternatives offered [3,15]. Thus, in the experiment conducted by Bishop et al. [3], for instance, respondents were asked: ‘‘Some people say that the 1975 Public Affairs Act should be repealed. Do you agree or disagree with this idea?’’ Although the act was entirely fictitious, 16% of the respondents agreed and 18% disagreed with its repeal. One way that can be used to screen out some uninformed respondents from expressing an opinion is to make an explicit offer of a ‘‘don’t know’’ answer in the alternatives presented. A more forceful way is to use a preliminary filter question of the form ‘‘Do you have an opinion on . . . ?,’’ with the opinion question then being asked only of those answering ‘‘yes’’ to the filter. Another factor that influences the extent to which respondents without opinions are screened out is the instruction given to interviewers on how readily they are to accept ‘‘don’t know’’ answers; as a study by Smith [18] indicates, differences in interviewer instructions can lead to an appreciable difference in the proportions of ‘‘don’t know’’ responses. The explicit offer of a middle alternative may be justified on the grounds that some respondents may hold a considered neutral view on the topic under study, and they need the offer of that alternative in order to be able to indicate their position. The argument against offering a neutral—or ‘‘don’t know’’—alternative is that there is a risk that many respondents may choose it as a way to avoid deciding on their preference. As several experiments have shown, when the neutral alternative is offered, the proportion of respondents choosing it may increase substantially [11,16]. The presentation order of the alternatives with closed questions may affect the

6770

QUESTION-WORDING EFFECTS IN SURVEYS

responses, and the nature of this order effect may depend on whether the alternatives are presented in written form or orally. Some experiments have shown a primacy effect, favoring alternatives at the top of the list, when the alternatives are presented in written form [2], while some have shown a recency effect, favoring later alternatives, when they are presented orally [16]. Yet another issue examined with opinion questions is the effect of reversing the order of two adjacent questions. In many cases the change of order has had no discernible effect for either question, but there are a few examples where the response distributions have been affected. A situation where an order effect has been found on several occasions is when the two questions comprise a general one and a specific one on the same topic. In this case the response distribution for the specific question is as a rule unaffected by the question order, but that for the general question may differ according to whether the question comes before or after the specific question [10,17,20]. In the Kalton et al. [10] experiment, for example, respondents were asked a general question, ‘‘Do you think that driving standards generally are lower than they used to be, or higher than they used to be, or about the same?’’ and an equivalent specific question about ‘‘the driving standards among younger drivers.’’ The order of these questions was varied for two random halves of the sample. The responses to the specific question were unchanged by the question order: both when it was asked first and when it was asked after the general question, 35% of respondents said that the driving standards of younger drivers were lower than they used to be. The question order did, however, affect the responses to the general question: 34% of respondents said that general standards were lower when the general question was asked first, but only 27% gave this answer when the general question followed the specific question. Further analysis revealed that this question order effect applied only with respondents aged 45 and over: among these respondents, 38% said general driving standards were lower when the general question was asked first, compared with only 26% when the question was asked second.

The demonstrated sensitivity of the distributions of opinion responses to the precise wording, format, and context of the questions has led experienced survey researchers to interpret these distributions with considerable caution. When the overall distribution of opinion is of major interest, the researcher can seek protection against question artifacts by using several questions that differ in wording and format. However, partly because of the instability of overall distributions, researchers usually place little emphasis on them. Rather, they concentrate their analyses mainly on examining the associations between the opinions and other responses, for example, examining the variation in the distributions of opinions for respondents in different age groups or with different levels of education. This type of analysis is justified under the assumption that although overall distributions may be affected by questioning effects, associations between opinions and other responses will be unaffected. Although this assumption of ‘‘form-resistant correlation’’ [16] may often hold as a reasonable approximation, there are nevertheless examples to demonstrate that it does not always do so. While questioning effects with factual questions differ from those with opinion questions, there are nevertheless some similarities. Responses to factual questions can be equally affected by wording changes, but such effects are usually more easily accounted for. Responses to factual questions may also be affected by format, as for instance by the presentation order of response alternatives. There is also evidence that factual responses can be affected by other items on the questionnaire. One illustration of this effect comes from a survey in which respondents were asked to report their readership of a sizable number of periodicals. The presentation order of the different types of periodical was varied for different parts of the sample, with a resulting variation in reported readership levels [1]. The largest variation occurred with the weekly publications: when they appeared last in the presentation order, their reported level of readership was only three-fourths of what it was when they appeared first. Another illustration comes from an examination by Gibson et al. [7] of the effects of

QUESTION-WORDING EFFECTS IN SURVEYS

including supplements on the responses to the core items in three major surveys conducted by the U.S. Bureau of the Census∗ . In the National Crime Survey Cities Sample, a random half of the sample was asked a lengthy series of attitude questions about crime and the police before being asked the core crime victimization questions. This half sample reported victimization rates for personal crimes and for property crimes that were greater than the rates reported by the half sample asked only the victimization questions by around 20% and 13%, respectively (see also Cowan et al. [6]). A particularly disturbing feature of this effect of question context on responses is that it may invalidate comparisons between surveys: it is well recognized that comparison of survey results is hazardous unless the identical question is used, but these findings indicate that even repetition of the identical question may not yield comparable responses if the question is asked in a different context. Two issues of especial concern with factual questions are obtaining information about sensitive matters and minimizing memory errors. Several approaches are available for attempting to elicit accurate reports on sensitive matters, of which the randomized response∗ technique is a recent addition. The use of records, diaries, and aided recall methods can sometimes be useful for reducing memory errors. Cannell and his colleagues have experimented with several procedures for dealing with problems of both memory errors and of sensitive questions, including lengthening the questions, including instructions on the questionnaire to inform respondents of what is expected of them in answering the questions, the use of feedback to tell them how they are performing, and securing their commitment to respond conscientiously. The evidence from the various experiments using these techniques suggests that each of them leads to an improvement in reporting [4]. While further research is needed, these techniques hold promise for improving the quality of survey responses. There is a considerable, widely scattered literature on questioning effects. Introductions to this literature are provided by Kalton et al. [10], Kalton and Schuman [9], Schuman

6771

and Presser [16], Sudman [19], and Sudman and Bradburn [20]. REFERENCES 1. Belson, W. A. (1962). Studies in Readership. Business Publications, London. 2. Belson, W. A. (1966). J. Advert. Res., 6(4), 30–37. 3. Bishop, G. F., Oldendick, R. W., Tuchfarber, A. J., and Bennett, S. E. (1980). Public Opinion Quart., 44, 198–209. 4. Cannell, C. F., Miller, P. V., and Oksenberg, L. (1981). In Sociological Methodology, 1981, S. Leinhardt, ed. Jossey-Bass, San Francisco. 5. Cantril, H., ed. (1944). Gauging Public Opinion. Princeton University Press, Princeton, N.J. 6. Cowan, C. D., Murphy, L. R., and Wiener, J. (1978). Proc. Sec. Surv. Res. Meth., Amer. Statist. Ass., pp. 277–282. 7. Gibson, C. O., Shapiro, G. M., Murphy, L. R., and Stanko, G. J. (1978). Proc. Sec. Surv. Res. Meth., Amer. Statist. Ass., pp. 251–256. 8. Hedges, B. M. (1979). Statistician, 28, 83–99. 9. Kalton, G. and Schuman, H. (1982). J. R. Statist. Soc. A, 145, 42–73. 10. Kalton, G., Collins, M., and Brooks, L. (1978). Appl. Statist., 27, 149–161. 11. Kalton, G., Roberts, J., and Holt, D. (1980). Statistician, 29, 65–78. 12. Payne, S. L. (1951). The Art of Asking Questions. Princeton University Press, Princeton, N.J. 13. Rugg, D. (1941). Public Opinion Quart., 5, 91–92. 14. Schuman, H. and Presser, S. (1979). Amer. Sociol. Rev., 44, 692–712. 15. Schuman, H. and Presser, S. (1980). Amer. J. Sociol., 85, 1214–1225. 16. Schuman, H. and Presser, S. (1981). Questions and Answers in Attitude Surveys. Academic Press, New York. 17. Schuman, H., Presser, S., and Ludwig, J. (1981). Public Opinion Quart., 45, 216–223. 18. Smith, T. W. (1982). Public Opinion Quart., 46, 54–68. 19. Sudman, S. (1980). Statistician, 29, 237–273. 20. Sudman, S. and Bradburn, N. M. (1982). Asking Questions. Jossey-Bass, San Francisco. See also PUBLIC OPINION POLLS and SURVEY SAMPLING.

GRAHAM KALTON

6772

QUETELET, ADOLPHE

QUETELET, ADOLPHE Born: February 22, 1796, in Ghent, Belgium. Died: February 17, 1874, in Brussels, Belgium. Contributed to: descriptive statistics, demography, vital statistics, statistics in the social sciences. Adolphe Quetelet was one of the nineteenth century’s most influential social statisticians. He was born Lambert Adolphe Jacques Quetelet in Ghent, Belgium on February 22, 1796. He received a doctorate of science in 1819 from the University of Ghent, with a dissertation on conic sections. From 1819 on he taught mathematics in Brussels, founded and directed the Royal Observatory, and he dominated Belgian science for a half century, from the mid-1820s to his death in 1874. Early in 1824 Quetelet spent three months in Paris, where he studied astronomy and probability and learned what he could about the running of an observatory. Upon his return to Brussels he convinced the government to found an observatory, and for most of his life he operated from this base, giving particular attention to its meteorological functions. But one science was not enough to contain his energy and interests, and from about 1830 he became heavily involved in statistics and sociology∗ . Quetelet’s international reputation was made in 1835 with the publication of a treatise where he coined the term ‘‘social physics’’ [7,8,12]. This work is today best known for the introduction of that now-famous character, the ‘‘average man’’ (‘‘l’homme moyen’’). The average man began as a simple way of summarizing some characteristic of a population (usually a national population), but he took on a life of his own, and in some of Quetelet’s later work he is presented as an ideal type, as if nature were shooting at the average man as a target and deviations from this target were errors. The concept was criticized by Cournot∗ and others, for example on the grounds that an individual average in all dimensions might not even be biologically feasible (the average of a set of right triangles may not be a right triangle).

In 1846 he published a book [9,10] on probability and social science in the form of a series of letters to two German princes he had tutored (one of them, Albert, had married Queen Victoria of England in 1840). That book exerted a significant influence on social statistics by demonstrating that as diverse a collection of human measurements as the heights of French conscripts and the chest circumferences of Scottish soldiers could be taken as approximately normally distributed. This gave a further dimension to the idea of an average man—deviations from the average were normal, just as errors of observation deviated normally from their mean. Quetelet devised a scheme for fitting normal distributions (actually, symmetric binomial distributions with n = 999) to grouped data∗ that was essentially equivalent to the use of normal probability paper [14]. The appearance of the normal curve in such areas so far from astronomy and geodesy had a powerful influence on Francis Galton’s∗ thinking, and reading of this work of Quetelet may have inspired James Clark Maxwell∗ in formulating his kinetic theory of gases. Quetelet made few technical contributions to statistics, although in 1852 he did anticipate some later work on the use of runs∗ for testing independence. Quetelet derived the expected numbers of runs of different lengths both for independent and for simple Markov sequences, and compared them with rainfall records, concluding that there was strong evidence of persistence in rainy or dry weather [11,14]. In his earlier 1835 treatise, Quetelet was led to a forerunner of a measure of association in 2 × 2 tables, although the measure was neither developed nor expressed algebraically [2]. In other work (e.g., ref. 9) Quetelet gave much attention to classifying sources of variation as due to accidental, periodic, or constant causes, in a sort of informal precursor to the analysis of variance∗ or the decomposition of time series∗ . Quetelet was a prolific writer and editor. He wrote a dozen books, founded and wrote much of the material for several journals, and still found time to fill the pages of the official publications of the Belgian Acad´emie Royale des Sciences. In addition, he carried on an immense correspondence with scientists and

QUEUEING THEORY

others all over Europe [1,15]. He was a highly successful entrepreneur of science who was instrumental in the founding of the Statistical Society of London, the International Statistical Congresses, and the Statistical Section of the British Association for the Advancement of Science, not to mention several Belgian bureaus and commissions, and similar activities in meteorology. He was the first foreign member of the American Statistical Association∗ . The historian of science George Sarton has called him the ‘‘patriarch of statistics’’ [13].

6773

11. Quetelet, A. (1852). Bulletins de l’Acad´emie Royale de Belgique, 19, 303–317. 12. Quetelet, A. (1869). Physique Sociale, ou Essai sur le D´eveloppement des Facult´es de l’Homme, 2 vols. Muquardt, Brussels. (An expanded second edition of ref. 7.) 13. Sarton, G. (1935). Isis, 23, 6–24. 14. Stigler, S. M. (1975). Bull. Int. Statist. Inst., 46, 332–340. 15. Wellens-De Donder, L. (1966). M´emories de l’Acad´emie royale de Belgique, 37(2), 1–299. (A catalog of an extensive archive of Quetelet’s correspondence.)

STEPHEN M. STIGLER REFERENCES 1. Diamond, M. and Stone, M. (1981). J. R. Statist. Soc. A, 144, 66–79, 176–214, 332– 351. (Presents manuscript material on the relationship between Quetelet and Florence Nightingale∗ .) 2. Goodman, L. and Kruskal, W. H. (1959). J. Amer. Statist. Ass., 54, 123–163. 3. Hankins, F. H. (1908). Adolphe Quetelet as Statistician. Longman, New York. 4. Landau, D. and Lazarsfeld, P. F. (1978). International Encyclopedia of Statistics, Vol. 2, pp. 824–834. (Reprinted from The International Encyclopedia of the Social Sciences, with minor additions.) 5. Lazarsfeld, P. F. (1961). Isis, 52, 277–333. (An excellent discussion of Quetelet’s role in the quantification of sociology.) 6. Lottin, J. (1912). Quetelet, Statisticien et Sociologue. Alcan, Paris/Institut Sup´erieur de Philosophie, Louvain. 7. Quetelet, A. (1835). Sur l’homme et l`e d´eveloppement de ses facult´es, ou Essai de physique sociale, 2 vols. Bachelier, Paris. 8. Quetelet, A. (1842). A Treatise on Man and the Development of his Faculties. W. & R. Chambers, Edinburgh. (Reprinted by Burt Franklin, 1968.) 9. Quetelet, A. (1846). Lettres a` S.A.R. le Duc R´egnant de Saxe-Cobourg et Gotha, sur la Th´eorie des Probabilit´es, appliqu´ee aux Sciences Morales et Politiques. Hayez, Brussels. 10. Quetelet, A. (1849). Letters Addressed to H.R.H. the Grand Duke of Saxe Coburg and Gotha, on the Theory of Probabilities as Applied to the Moral and Political Sciences. Layton, London. (A mediocre translation of ref. 9.)

QUETELET INDEX The Quetelet index is the ratio 1, 000 × weight/(height)2 with weight measured in kilograms and height in centimeters. It is used as an index of build; the greater the index, the sturdier the build.

QUEUEING THEORY Queueing theory is concerned with the modeling of situations where customers wait for service and congestion occurs through randomness in service times and patterns of arrival. Examples include customer queues at checkout counters in a supermarket and queues of jobs in a network of time-sharing computers. The simplest queueing model comprises three elements, the first of which is a point process describing arrival instants of customers at a given place and the most common assumption is that customers arrive singly at event times of a renewal∗ process. Second, customers require a period of service from one of k servers; it is easiest to assume that service times are independent, identically distributed and independent of the arrival process. Finally, there is a queue discipline specifying how customers queue in order of arrival within an infinite waiting room and move from the head of the queue to the first available server. Servers are idle only if no

6774

QUEUEING THEORY

customer awaits service. All other situations are best regarded as modifications of this simple model. Analysis of the queueing model may comprise determination of the distributions of the queue length at any time, the waiting time before commencement of service of a customer, the duration of periods of continual work by the server—the busy periods—and the idle periods during which the server waits for further customers. Determination of finite time distributions is difficult but often it is possible to determine their large time limits, the so-called equilibrium theory. Further discussion is facilitated by Kendall’s [38] notation A/B/k for the simple queueing model; A denotes the type of interarrival time distribution, B the service-time distribution, and k the number of servers. For special purposes this notation is sometimes augmented. For example, an A/B/k queue having a finite waiting room of size N is denoted A/B/k/N. In particular, M denotes the negative exponential∗ or Markov distribution, Ej is the j-fold convolution∗ of M, or Erlang distribution∗ , and D denotes the deterministic distribution. The general kserver queueing model is denoted by GI/G/k. Here, however, we shall concentrate mainly on the more tractable singleserver case and refer the reader to the companion entry MULTISERVER QUEUES for an account of the complications engendered by several servers. Also, see NETWORKS OF QUEUES for an account of the properties of interconnected queues. The earliest papers on queueing theory were written by A. K. Erlang [7] during the period 1909–1929, to solve congestion problems arising in teletraffic contexts. He discussed the equilibrium theory of the M/M/k, M/D/k, and M/Ej /1 systems with finite waiting rooms. Other workers extended and consolidated Erlang’s work, and this is summarized by Fry [27]. F. Pollaczek [59] and A. Khinchin [40] independently worked on the equilibrium theory of M/G/1, obtaining expressions (called Pollaczek–Khinchin formulae∗ ) for the Laplace–Stieltjes transform and expectation of the equilibrium waiting-time distribution in terms of the arrival rate and service-time distribution [41; 44, p. 177]. Subsequently, Pollaczek (1934) considered M/G/k, and E. Volberg

(1939) obtained equilibrium and some timedependent properties of Ej /G/1. In 1942, E. Borel obtained the distribution of the number of customers served during a typical busy period of M/D/1. See refs. 37, 38, and 65 (p. 20), for more details. The modern phase of queueing theory was marked by the publication by D. G. Kendall [37] in 1951 of a paper which acquainted a wide circle of mathematicians with these early contributions. More important, by using the concept of embedded processes∗ , Kendall showed how the equilibrium theory of M/G/1 could be derived from Markov chain theory (see MARKOV PROCESSES), then recently expounded in the first edition of Feller’s famous text [25]. The queue-length process {Q(t) : t  0} is a birth-and-death process∗ for M/M/1 but is not even Markovian otherwise. Roughly, this is because both the current queue length and elapsed service time of the customer receiving service are required to decouple the past and future evolution of {Q(t)}. This difficulty vanishes if queue lengths are inspected at successive departure epochs—if these are denoted by {Dn } then {Q(Dn +)} is a Markov chain. Kendall also generalized Borel’s work by showing that customers served during a busy period of M/G/1 can be grouped into generations of a Galton–Watson process∗ . Consequently, the class of distributions of the numbers of customers served during a busy period essentially coincides with the class of Lagrange distributions∗ [58]. SUBSEQUENT DEVELOPMENT OF THE SIMPLE QUEUE MODEL The theory of the GI/G/1 system developed rapidly during the five years following Kendall’s paper [37]. Lindley [53] obtained the fundamental relation Wn+1 = (Wn + Un )+ , where Wn is the time spent waiting for service by the nth customer, who arrives at Tn and demands service lasting Sn , and Un = Sn+1 − (Tn+1 − Tn ). Thus {Wn } is a random walk∗ on the positive real line with a reflecting barrier at the origin. Lindley exploited this observation to show that an equilibrium waiting-time distribution exists if EUn < 0 but not if EUn  0; then waiting times can

QUEUEING THEORY

be arbitrarily long. These results were later extended to the GI/G/k system [42,78]. Lindley derived an integral equation∗ for the equilibrium waiting-time distribution and solved it for M/G/1 and D/Ej /1. This equation, of Wiener–Hopf type, was treated by Smith [67]. The deep connection between {Wn } and the random walk {Vn }, where V0 = 0 and  Vn = nj=1 Uj (n  1), was exposed by Spitzer [68], who showed that if W0 = 0, then Wn and max0
j
n Vj have the same distribution. He also obtained the identity ∞  n=0

n

−θ Wn

t E(e

) = exp

∞ 

n

−θ Vn+

(t /n)E(e

 ) ,

n=1

independently discovered using more analytical methods by Pollaczek [60], and now called the Pollaczek–Spitzer identity. See ref. 43 for an elegant algebraic derivation and illuminating discussion of related matters. The virtual waiting time W(t)(t  0) is the total work load facing the server at time t. This concept was introduced by ´ [72] and analyzed for the M/G/1 Takacs system, where {W(t)} is a Markov process. It is not Markovian for other GI/G/1 systems, but the waiting times constitute an embedded Markov chain, since Wn = W(Tn− ). Non-Markovian processes can be rendered Markovian by adding supplementary variables. For example, if E(t) is the expended service time of the customer, if any, receiving service at time t, then the stochastic process∗ having state (Q(t), E(t)) if Q(t) > 0 and (0) otherwise is Markovian for the M/G/1 system. The idea was first introduced by Cox [17] and later exploited by others [34,35]. Properties of GI/G/1 can be obtained by numerous methods. The use of transform and generating function∗ techniques is described ´ in ref. 73 and combinatorial methby Takacs ods in ref. 74. Prabhu [62] describes the use of Wiener–Hopf and fluctuation theory and Cohen [13] the use of complex variable methods, originally due to Pollaczek, and regenerative processes in ref. 12. Frequently, these analytical techniques yield derived distributions in forms having limited practical utility, for example, as an integral transform∗ . Although a transform can be used to obtain moments and tail behavior,

6775

explicit inversion is not often possible (but see ref. 28 on numerical inversion). Recognizing these problems, some researchers have sought explicit expressions for derived distributions, but often they occur as infinite series containing high-order convolutions. Several approximation procedures have been devised to circumvent these problems. For example, it is often possible to obtain bounds for the distributions of interest [45, 48,71]. Again, transient distributions may be adequately approximated by the equilibrium distribution if some estimate of the rate of convergence of the former is available. In most practical cases convergence is geometrically fast [76]. Finally, if the arrival- and service-time distributions of a system can be closely approximated by those of a second, computationally tractable system, then we can expect that derived distributions of the first system will be closely approximated by the corresponding and easily computed quantities of the second system. Neuts [56] has shown that this approach can be powerfully implemented with phase-type distributions∗ . Standard statistical methods can be used to estimate parameters of queueing models. For example, if the continuous-time queuelength process is Markovian, then likelihood methods provide point estimates and confidence intervals from continuously observed records of queue lengths [1,18,30]. Frequently, ad hoc methods must be devised to handle incomplete observations and/or the nonMarkovian nature of many derived queueing processes. Inference procedures for point processes [19] will yield the characteristics of the arrival process from observations of successive arrival times. Again, the method of moments∗ can be used to construct approximate estimates of the service-time distribution of an M/G/1 system from observations on waiting times [18,30]. The regenerative structure of many queueing systems can be exploited to obtain estimates by simulation∗ . For example, the lengths of successive busy periods of a GI/G/1 system, and the numbers of customers served therein, are serially independent. Estimates of the limiting expected waiting time can be obtained by observing successive busy periods [21,64]. The stochastic processes of interest in the GI/G/1 system are functionals of the random

6776

QUEUEING THEORY

walk {Vn }. When EUn 0, the central limit theorem∗ shows that {Vn } can be approximated by a Brownian motion∗ process. It is plausible that the distribution of the queueing functional of interest is approximated by that of the same functional of the Brownian motion, which in most cases can be computed. The resolution of this question is encompassed by the theory of functional limit and heavy traffic theorems. They provide useful approximations for much more general systems than GI/G/1 and with only weak dependence assumptions [5,32,77]. Related to this are continuity theorems [39] which show that distributions of functionals, like waiting times, are insensitive to small perturbations of service-time and interarrival-time distributions. These theorems show that Neuts’ work [56] effectively solves a wide range of queueing problems. VARIANTS OF THE SIMPLE QUEUEING MODEL The GI/G/k system can be generalized in a myriad ways, and we mention only a few. Thus independence assumptions can be weakened while retaining some qualitative features and approximation procedures of the simple system [5,49,54]. General arrival processes can be modeled with point processes∗ theory [26,63], which also facilitates understanding of the process of departures [22]. Arrival and service rates may vary with the state of the system [49,55], particularly if control policies are implemented [20,75]. Variations of the queue discipline include finite waiting rooms and bounds on waiting times [13] and arrivals and servicing of batches of customers [11,13,65]. Again, arriving customers may not join the queue with a probability depending on queue length (balking) or may leave after waiting for some unpredictable period (reneging) [70]. Many of these variants can usefully be modeled by discrete-state Markov processes [15, Chaps. 3, 4; 24, 69]. Arriving customers may be allocated to priority classes giving high-priority customers earlier access to the server than previously arrived lower-priority customers, even to the extent of interrupting the service of such customers [33; 45, Chap. 3].

These modifications mainly concern individual service facilities, but frequently individual queues are part of a larger network where customers leaving one service facility may queue for service at another. Although such networks were first studied in the 1950s, the desire to understand computer and communication networks, for example, has recently sparked intense research [16,46,51]. The equilibrium theory of networks in which service and interarrival distributions are negative-exponential can be elucidated by concepts of reversibility of Markov processes [36]. Many results so obtained extend to certain classes of networks allowing more general distributions [9]. A recent dynamical approach portrays a network as a system of interacting point processes and makes extensive use of martingale∗ theory [4,6]. Diffusion approximations and other weak convergence techniques are also efficacious [8,52]. See also NETWORKS OF QUEUES.

CONCLUDING REMARKS Queueing theory has been a very popular area of research, with some 50 published books and a journal literature numbering about 4,000 papers. Yet critics have contended that much of this literature deals with unimportant variations of the basic theme, has little theoretical interest or practical value, and that the literature which is useful is only the elementary part of the theory [10; 31, p. 280; 50, pp. vii, 27; 61, p. 243; 66]. For example, Lee [50] shows how complicated problems of congestion at airline terminals can be resolved with quite simple queueing models. There are many mathematical investigations of systems where arrival- and service-time distributions vary in response to queue lengths, waiting times, and so on [49]. However, little of this work is tailored to real situations and the state dependencies analyzed arise more from the analyst’s imagination than through empirical investigation. Most researchers stand aside from such criticism, but on occasion rebuttals have been published. Thus refs. 3 and 47 are the first of several rejoinders to ref. 10, the most

QUEUEING THEORY

vigorous attack on the subject. Points made include: 1. The assertion of little applicability is simply false; many successful applications are either not reported in the literature or are reported in journals not normally read by the critics [47]. 2. The literature contains a much higher proportion of practically motivated papers than critics are willing to admit. Bhat [3] asserts that 65% of the 700odd references in ref. 65 are application oriented. 3. Kolesar [47] asserts that much of the theoretical literature is relevant in practice and he cites several papers, including ref. 51, used by himself. 4. Prabhu [61] observes that the criticisms above can be leveled at other theoretical disciplines and that it is not realistic to suspend research until all extant theory is utilized. The structures of queueing theory also arise in dam theory∗ and risk theory∗ [62]. These structures are largely amenable to analysis based on Markov chains and sums of random variables, and hence queueing theory has been a very attractive subject for exploration by probabilists [31, p. 281]. The development of queueing theory has also been closely related to other topics, for example, point processes∗ , semi-Markov processes∗ , and functional limit theorems. Problems arising in queueing theory have inspired progress in these areas, and reciprocally, it has been enriched by general developments in these subjects. There is no sign that this symbiotic relationship is weakening, and hence queueing research is likely to remain fertile for some time to come. Queueing theory first arose in response to problems of telephony, and the need to measure the performance of modern communication systems has inspired a resurgence of research activity [45,46]. Indeed, Kleinrock [44] states that queueing theory is ‘‘one of the few tools we have for analyzing the performance of computer systems’’ and he further asserts that the successful application of queueing theory to this problem is a significant reason for its continuing popularity among researchers.

6777

REFERENCES Together refs. 23, 65, and 66 give a bibliography complete up to about 1966, but there is no complete listing of the thousands of articles written thereafter. Papers of more recent years listed below have been chosen largely for their bibliographic value. References 15 and 57 contain (almost) complete lists of English language books on queues and are updated by the following list. See also the reference list for MULTISERVER QUEUES. 1. Basawa, I. V. and Prakasa Rao, B. L. S. (1980). Statistical Inference for Stochastic Processes. Academic Press, New York. 2. Bhat, U. N. (1969). Manag. Sci., 15, B280– B294. (Useful historical survey.) 3. Bhat, U. N. (1978). Interfaces, 8, 27–28. (Rebuts ref. 10.) 4. Boel, R. (1981). In Stochastic Systems: The Mathematics of Filtering and Identification and Applications, M. Hazewinkel and J. C. Willems, eds. D. Reidel, Dordrecht, The Netherlands, pp. 141–167. (Surveys networks.) 5. Borovkov, A. A. (1976). Stochastic Processes in Queueing Theory. Springer-Verlag, Berlin. (Highly mathematical, stressing generality and weak convergence.) 6. Bremaud, P. (1981). Point Processes and Queues: Martingale Dynamics. SpringerVerlag, Berlin. 7. Brockmeyer, E., Halstrom, H. L., and Jensen, A. (1948). The Life and Works of A. K. Erlang. Transactions 2. Danish Academy of Technical Sciences, pp. 1–277. (Reprinted in 1960 in Acta Polytech. Scand. Appl. Maths. Comp. Machinery No. 6.) 8. Bruell, S. C. and Balbo, G. (1980). Computational Algorithms for Closed Queueing Networks. North-Holland, New York. 9. Burman, D. Y. (1981). Adv. Appl. Prob., 13, 846–859. (Surveys and extends insensitivity results.) 10. Byrd, J. (1978). Interfaces, 8, 22–26. (Castigates queueing theory.) 11. Chaudhry, M. L. and Templeton, J. G. C. (1983). A First Course in Bulk Queues. Wiley, New York. 12. Cohen, J. W. (1976). On Regenerative Processes in Queueing Theory. Lect. Notes Econ. Math. Syst., 121. Springer-Verlag, Berlin. 13. Cohen, J. W. (1982). The Single Server Queue, 2nd ed. North-Holland, Amsterdam. (Encyclopediac in scope.)

6778

QUEUEING THEORY

14. Cohen, J. W. and Boxma, O. J. (1983). Boundary Value Problems in Queueing System Analysis. North-Holland, Amsterdam. (Analytical treatment of two-dimensional random walk with applications to some two-server queues.) 15. Cooper, R. B. (1981). Introduction to Queueing Theory, 2nd ed. North-Holland, New York. (Good introductory text containing a useful guide to journals that publish queueing theory papers.) 16. Courtois, P. J. (1977). Decomposability: Queueing and Computer System Applications. Academic Press, New York. 17. Cox, D. R. (1955). Prob. Camb. Philos. Soc., 51, 433–441. 18. Cox, D. R. (1965). In Proc. Symp. Congestion Theory, W. L. Smith and W. E. Wilkinson, eds. The University of North Carolina Press, Chapel Hill, N.C., pp. 289–316. 19. Cox, D. R. and Lewis, P. A. W. (1966). The Statistical Analysis of Series of Events. Methuen, London. 20. Crabill, T. B. (1977). Operat. Res., 25, 219– 232. (Bibliography on optimal control of queues.) 21. Crane, M. A. and Lemoine, A. J. (1977). An Introduction to the Regenerative Method for Simulation Analysis. Lect. Notes Control Inf. Sci., 4. Springer-Verlag, Berlin. 22. Daley, D. J. (1976). Adv. Appl. Prob., 8, 395–415. (Surveys the output process.) 23. Doig, A. (1957). Biometrika, 44, 490–514. (An extensive bibliography.) 24. Doorn, E. van (1980). Stochastic Monotonicity and Queueing Applications of Birth–Death Processes. Lect. Notes Statist., 4. SpringerVerlag, Berlin. 25. Feller, W. (1968). An Introduction to Probability Theory and Its Applications, Vol. I, 3rd ed. Wiley, New York. 26. Franken, P., K¨onig, D., Arndt, U., and Schmidt, V. (1982). Queues and Point Processes. Wiley, Chichester, England. (Mathematically advanced.) 27. Fry, T. C. (1928). Probability and Its Engineering Uses. Van Nostrand, New York. (Surveys the earliest work.) 28. Gaver, D. P., Jr. (1966). Operat. Res., 14, 444–459. 29. Gelenbe, E., Labetoulle, J., Marie, R., Metivier, M., Pujolle, G., and Stewart, W. (1980). R´eseaux de files d’attente. Editions Homme et Techniques, Suresnes, France.

30. Harris, C. (1974). In Mathematical Methods in Queueing Theory, A. B. Clarke, ed. Lect. Notes Econ. Math. Syst., 98. Springer-Verlag, Berlin, pp. 157–184. (Surveys statistical inference for queues.) 31. Hedye, C. C. (1981). Aust. J. Statist., 23, 273–286. 32. Iglehart, D. L. (1973). Adv. Appl. Prob., 5, 570–594. (Surveys weak convergence theory.) 33. Jaiswal, N. K. (1968). Priority Queues. Academic Press, New York. 34. Keilson, J. and Kooharian, A. (1960). Ann. Math. Statist., 31, 104–112. 35. Keilson, J. and Kooharian, A. (1962). Ann. Math. Statist., 33, 767–791. 36. Kelly, F. P. (1979). Reversibility and Stochastic Networks. Wiley, Chichester, England. 37. Kendall, D. G. (1951). J. R. Statist. Soc. B, 13, 151–185. (A seminal paper.) 38. Kendall, D. G. (1953). Ann. Math. Statist., 24, 338–354. 39. Kennedy, D. P. (1977). Bull. ISI, 47, Book 2, 353–365. (Survey of continuity theorems.) 40. Khinchin, A. (1932). Mat. Sb., 39, 73–84. 41. Khintchine, A. Y. (1969). Mathematical Methods in the Theory of Queueing, 2nd ed. Charles Griffin, London. (Translation of the 1955 Russian original. This booklet collects the contributions of one of the pioneers and was also important to the development of point process theory.) 42. Kiefer, J. and Wolfowitz, J. (1955). Trans. Amer. Math. Soc., 78, 1–18. 43. Kingman, J. F. C. (1966). J. Appl. Prob., 3, 285–326. (Beautiful exposition of GI/G/1 waiting time theory.) 44. Kleinrock, L. (1975). Queueing Systems, Vol. 1: Theory. Wiley, New York. (Excellent introduction to Markov queues, M/G/1, GI/M/1 and their variants.) 45. Kleinrock, L. (1976). Queueing Systems, Vol. 2: Computer Applications. Wiley, New York. (Continues with GI/G/1 theory and networks.) 46. Kobayashi, H. and Konheim, A. (1977). IEEE. Trans. Commun., COM-25, 2–29. (Survey of queueing models for computer networks.) 47. Kolesar, P. (1979). Interfaces, 9, 77–82. (Rebuts ref. 10.) 48. K¨ollerstr¨om, J. (1976). Math. Proc. Camb. Philos. Soc., 80, 521–525. 49. K¨onig, D., Rykov, V. V., and Schmidt, V. (1983). J. Sov. Math., 21, 938–994. (Surveys queues with various kinds of dependencies.)

QUICKSORT 50. Lee, A. M. (1966). Applied Queueing Theory. St. Martin’s Press, New York. (Interesting case studies using elementary theory.) 51. Lemoine, A. J. (1977). Manag. Sci., 24, 464–484. (Surveys network equilibrium theory.) 52. Lemoine, A. J. (1978). Manag. Sci., 24, 1175–1193. (Surveys application of weak convergence theory to networks.) 53. Lindley, D. V. (1952). Proc. Camb. Philos. Soc., 48, 277–289. 54. Loynes, R. M. (1962). Proc. Camb. Philos. Soc., 58, 497–520. 55. Minh, Do Le. (1980). Math. Operat. Res., 5, 147–159. (State-dependent queues.) 56. Neuts, M. F. (1981). Matrix-Geometric Solutions in Stochastic Models—An Algorithmic Approach. Johns Hopkins University Press, Baltimore. 57. Newell, G. F. (1982). Applications of Queueing Theory, 2nd ed. Chapman & Hall, London. (Concerned with approximate methods and contains a useful list of applications to road traffic situations.) 58. Pakes, A. G. and Speed, T. P. (1977). SIAM J. Appl. Math., 32, 745–754. 59. Pollaczek, F. (1930). Math. Z., 32, 64–100, 729–750. 60. Pollaczek, F. (1952). C. R. Acad. Sci. Paris, 234, 2334–2336. 61. Prabhu, N. U. (1975). Stoch. Processes Appl., 3, 223–258. 62. Prabhu, N. U. (1980). Stochastic Storage Processes—Queues, Insurance Risk, and Dams. Springer-Verlag, Berlin.

6779

70. Stanford, B. E. (1979). Math. Operat. Res., 4, 162–178. 71. Stoyan, D. and Daley, D. J. (1982). Comparison Methods for Queues and Other Stochastic Models. Wiley, New York. ´ 72. Takacs, L. (1955). Acta Math. Acad. Sci. Hung., 6, 101–129. ´ L. (1962). Introduction to the Theory of 73. Takacs, Queues. Oxford University Press, New York. ´ 74. Takacs, L. (1968). Combinatorial Methods in the Theory of Stochastic Processes. Wiley, New York. 75. Teghem, J. (1982). In Operations Research in Progress, G. Feichtinger and P. Kall, eds. D. Reidel, Dordrecht, The Netherlands, pp. 333–345. (Surveys optimal control for queues.) 76. Tweedie, R. L. (1983). In Probability, Statistics and Analysis, J. F. C. Kingman and G. E. H. Reuter, eds. Cambridge University Press, Cambridge, England, pp. 260–276. (Surveys rates of convergence to equilibrium distributions.) 77. Whitt, W. (1974). In Mathematical Methods in Queueing Theory, A. B. Clarke, ed. Lect. Notes Econ. Math. Syst., 98. Springer-Verlag, New York, pp. 307–350. (Surveys heavy traffic results.) 78. Whitt, W. (1982). Math. Operat. Res., 7, 88– 94. See also BIRTH-AND-DEATH PROCESSES; DAM THEORY; EMBEDDED PROCESSES; MARKOV PROCESSES; MULTISERVER QUEUES; NETWORKS OF QUEUES; PHASE TYPE DISTRIBUTIONS —I; RANDOM WALKS; RENEWAL THEORY; RISK THEORY; and STOCHASTIC PROCESSES, POINT.

63. Rolski, T. (1981). Stationary Random Processes Associated with Point Processes. Lect. Notes Statist., 5, Springer-Verlag, Berlin.

ANTHONY G. PAKES

64. Rubinstein, R. Y. (1981). Simulation and the Monte Carlo Method. Wiley, New York.

QUICKSORT

65. Saaty, T. L. (1961). Elements of Queueing Theory. McGraw-Hill, New York. (Useful general text.)

Quicksort is an algorithm invented by Hoare [4] for efficiently sorting an array of data values selected at random from a continuous distribution. We begin by selecting one data value (usually the first one) to be used as a pivot. Comparing the pivot with each of the other values, it is placed in its correct sorted position. In the course of the comparison process, quicksort assigns all values less than the pivot to positions that are left of the pivot and all values greater than the pivot to positions to the right. The original array is thus partitioned into two smaller unsorted arrays

66. Saaty, T. L. (1966). Naval Res. Logist. Quart., 13, 447–476. (Extensive bibliography.) 67. Smith, W. L. (1953). Proc. Camb. Philos. Soc., 49, 449–461. 68. Spitzer, F. (1956). Trans. Amer. Math. Soc., 82, 323–339. 69. Srivastava, H. M. and Kashyap, B. R. K. (1982). Special Functions in Queueing Theory: And Related Stochastic Processes. Academic Press, New York.

6780

QUICKSORT

on opposite sides of the pivot. Recursively, quicksort next continues the partitioning process with each one of the smaller arrays. The distribution of the number of comparisons required by quicksort has received substantial attention in the literature. Let Xn be the random number of comparisons required to sort a list of length n by quicksort. Using martingale theory, Regnier [8] has shown that the number of comparisons, when suitably standardized via W=

Xn − E(Xn ) , n

converges in distribution to some random variable W with zero mean and finite variance. R¨ossler [9] characterizes this distribution as the unique fixed point of a contraction operation in the space of distribution functions with zero mean and finite second moment. He showed that

Figure 1. General form of the density function fW (w) of the limiting distribution.

L(W) = L(UY + (1 − U)Z + G(U)), where L(W) is the distribution function of W; U is uniformly distributed on [0, 1]; Y and Z have the same distribution as W; U, Y, and Z are independent; and G(u) = 2u ln u + 2(1 − u) ln(1 − u) + 1. Hennequin [2,3] obtained an explicit formula for the cumulants of the limiting random variable W; W has a density with respect to Lebesgue measure, and the distribution is supported on the whole real line [10]. Eddy and Schervish [1] utilized R¨ossler’s result [9] to obtain iterative numerical approximations for the characteristic function∗ φW (t) and the density of W; see Fig. 1 for a general form of the density. Tan and Hadjicostas [10] have shown that the iterations of approximations of the characteristic function converge pointwise to φW (t). McDiarmid and Hayward [7] obtained figures of the frequency distribution of Xn using simulation. Under the model of randomness∗ the ranks of the data elements form a random permutation of the integers from 1 to n. Eddy and Schervish [1] examined the performance of quicksort under other models (including the case when the ranks of the data follow a triangular distribution). Mahmoud et al. [6] studied the distribution of the number of comparisons required

in quickselect (a variant of quicksort) for identifying a particular order statistic in a set of random values from a continuous distribution. Lent and Mahmoud [5] analyzed multiple quickselect, which is another modified version of quicksort used to identify several order statistics from a given data set. This algorithm is useful for computing L-estimates.∗ REFERENCES 1. Eddy, W. and Schervish, M. (1995). How many comparisons does quicksort use? J. Algorithms, 19, 402–431. 2. Hennequin, P. (1987). Combinatorial analysis of quicksort algorithm. RAIRO Theor. Inf. Appl., 23, 317–333. 3. Hennequin, P. (1991). Analyse en moyenne d’algorithmes, tri rapide et arbres de ´ recherche. Ph.D. thesis, Ecole Polytechnique Palaiseau. 4. Hoare, C. (1962). Quicksort. Comput. J., 5, 10–15. 5. Lent, J. and Mahmoud, H. (1996). Averagecase analysis of multiple quickselect: an algorithm for finding order statistics. Statist. Probab. Lett., 28, 299–309. 6. Mahmoud, H., Modarres, R., and Smythe, R. (1995). Analysis of quickselect: an algorithm

QUINCUNX for order statistics. RAIRO Theor. Inf. Appl., 29, 255–276. 7. McDiarmid, C. and Hayward, R. (1992). Strong concentration for quicksort. Proc. 3rd Annual ACM–SIAM Symp. on Discrete Algorithms. 8. R´egnier, M. (1989). A limiting distribution for quicksort. RAIRO Theor. Inf. Appl., 23, 335–343. 9. R¨ossler, U. (1991). A limit theorem for quicksort. RAIRO Theor. Inf. Appl., 25, 85–100. 10. Tan, K. and Hadjicostas, P. (1995). Some properties of a limiting distribution in quicksort. Statist. Probab. Lett., 25, 87–94.

QUICK TEST STATISTICS, LORD’S

6781

0.02, and 0.01 are presented in Lord [1] (for the equal-sample-size case). An abridged version of Lord’s tables appears in Snedecor and Cochran [4]. REFERENCES 1. Lord, E. (1947). Biometrika, 34, 41–67. 2. Moore, P. G. (1957). Biometrika, 34, 482–489. 3. Pillai, K. C. S. (1951). Ann. Math. Statist., 22, 469–472. 4. Snedecor, G. W. and Cochran, W. G. (1980). Statistical Methods, 7th ed. Iowa State University Press, Ames, Iowa. See also STUDENTIZED RANGE and STUDENT’S t-TESTS.

These are ‘‘quick’’ test statistics proposed by Lord [1]. 1. A one-sample analog of the t-test with test statistic of the form tW =

X −µ , W

where W is the sample range∗ . This test is often used in industrial applications, although it is not resistant to outliers∗ . The efficiency of this test was studied by Pillai [3]. Tables of critical values for various significance levels are presented in Lord [1], Pillai [3], and Snedecor and Cochran [4].

QUINCUNX An arrangement of five objects (e.g., spots on a die) in a square or rectangle with one at each corner and one in the middle is called a quincunx. In statistics the term is used to refer to a probability device (see Fig. 1) designed to demonstrate mechanically the generation of normal or binomial curves. This device was originated by Francis Galton∗ [1,2] and consists of a glass-enclosed board with rows of equally spaced nails or pins. These rows are arranged so that each successive row is shifted with each nail directly

2. A two-sample analog of the t-test for testing the equality of two means based on two independent samples from normal populations. The test statistic is of the form tw =

|X 2 − X 1 | W

,

where W = (W1 + W2 )/2 and Wi (i = 1, 2) are the sample ranges. The test is no more robust under nonnormality than the t-test and it is more vulnerable to extreme sample values. Moore [2] has extended this test for the case of independent samples of unequal size. Tables of critical values for significance levels α = 0.10, 0.05,

Figure 1. From Pearson [6].

m

Figure 2. From Pearson [4]. 6782

QUOTA SAMPLING

below the midpoint of two adjacent nails in the row above. Except for those on the boundary of the formation, each nail is the center of a quincunx of five nails. Metal balls (shot) are poured through a funnel or hopper and directed at the middle nail in the top row so that each ball has a 50:50 chance of darting to the right or left. The same condition holds for all successive lower nails that a ball may approach. In effect, each ball is a unit in an independent random walk∗ with a probability of 0.50 of moving one step to the right or left for each lower nail it approaches. The final row, instead of nails, consists of a set of vertical columns into which all balls drop forming a histogram∗ of columns of balls. The theoretical resulting histogram is binomial∗ with p = 12 and n equal to the number of rows (of pins plus collector column row). If the number of balls is large, the resulting observed histogram will fairly accurately represent the binomial histogram. If the number of rows is reasonably large (20 or 30 or so), this empirical histogram will serve as a good representation of the normal∗ curve since a normal probability distribution closely approximates a symmetric binomial probability distribution. Further, the resulting observed ‘‘normal curve’’ can be considered as a supportive demonstration of the physical validity of Hagen’s derivation of the normal probability distribution, based on the idea that a normal random variable is a sum of many small independent elementary ‘‘errors’’; see LAWS OF ERROR —II: THE GAUSSIAN DISTRIBUTION. Galton also used a quincunx-type device to illustrate the phenomenon of revision (i.e., regression toward the mean) in biological populations. [A number of authors use Galton’s Natural Inheritance as the original source of the quincunx. This appears to be the proper source for this name; however, Galton used a similar device 12 years earlier (1877) and referred to its use some three years before this. See also ref. 5.] Modifications of Galton’s original device include replacing the pins with wedges and the balls with other items, such as seed or sand. Each wedge is placed to divide the stream of sand or seed into two streams. Karl Pearson∗ [4] constructed such a device with the position of the wedges adjustable so that each wedge will split a stream into two parts

6783

in the proportion p : q, with p having arbitrary values according to the placement of the wedges (see Fig. 2). Varying choices for p produce varying skewed binomial distributions and, consequently, a wide variety of distribution shapes can be physically generated by such a device. Recently, a computer program has been developed for a microcomputer with video display facility for simulating a quincunx [3]. The program also provides a slow-mode option to enable the viewer to follow the drop of individual balls and, thereby, to study the pattern of runs to the left or right. REFERENCES 1. Galton, F. (1877). Nature (Lond.), 15, 492– 495, 512–514, 532–533. 2. Galton, F. (1889). Natural Inheritance. Macmillan, London, p. 63. 3. Hilsenrath, J. and Field, B. (1983). The Mathematics Teacher, 76, 571–573. 4. Pearson, K. (1895). Philos. Trans. R. Soc. Lond. (A), 186, 343–414 +9 plates. 5. Pearson, K. (1930). The Life, Letters and Labours of Francis Galton, Vol. IIIA. Cambridge University Press, London, England. 6. Pearson, E. S. (1969). Some Historical Reflections Traced through the Development of the Use of Frequency Curves. Tech. Rep. No. 38 (THEMIS). Department of Statistics, Southern Methodist University, Dallas, Tex. See also BINOMIAL DISTRIBUTION; GALTON, FRANCIS; LAWS OF ERROR —II: THE GAUSSIAN DISTRIBUTION; and NORMAL DISTRIBUTION.

HARRY O. POSTEN

QUOTA SAMPLING Quota sampling is a method of judgmental selection in which specified numbers (quotas) of particular types of population units are required in the final or realized sample. For example, a quota-sampling design for a survey of shoppers in a particular shopping center might specify that the sample include 50 white males, 150 white females, 25 nonwhite males, and 75 nonwhite females, but give no further instructions on how these quotas were to be filled. Quota sampling is

6784

QUOTA SAMPLING

therefore fundamentally different from probability sampling, in which each population unit has a known nonzero probability of inclusion in the sample. Quota sampling is used in situations where probability sampling is impracticable, unduly costly, or considered to be unnecessary. Estimates based on quota sampling lack the usual criterion of acceptability which applies to most estimates based on probability samples, namely that they are actually or nearly unbiased over repeated sampling. A sample estimator is said to be unbiased in this sense if its design-based expectation is equal to the value which would have been obtained if the entire population had been enumerated. This means that if the same selection and estimation procedures are repeated many times, the average of the resulting estimates is close to that population value. The extent of the validity that can be ascribed to an estimate based on a quota sample in fact depends on the extent to which an implicit population model underlying the quota-sampling design accurately describes the population being sampled. To illustrate, suppose that the objective of sampling is to estimate the mean over the entire population of the item Yi (i = 1, 2, . . . , N), where N is the number of units in the population. This population mean will be denoted by Y and defined by Y=N

−1

N 

Yi .

(1)

Suppose further that a sample of n distinct units is selected from this population (the method of selection not as yet specified) and that the values yi (i = 1, 2, . . . , n) are observed on these n sample units. (Note that the yi values are simply a subset of the Yi values. If the first unit selected in the sample happened to be the 47th population unit, then y1 would be numerically the same as Y47 .) Finally, suppose that the sample estimator y of the population mean Y is n 

N−n  (Yi − Y)2 . (3) Nn(N − 1) N

vard (y) =

i=1

2. Regardless of the manner in which the sample was selected, if the Yi values were generated by the stochastic process∗ Yi = µ + i ,

yi .

(2)

i=1

The validity of the estimator y can be established in one of two ways.

(4)

where µ is a constant and the i are independent random variables with mean zero and variance σ 2 , then the expectation of both Y and y under that stochastic process is µ and the (model based) variance of y considered as an estimator (not of µ but) of Y is varm (y) = [(N − n)/(Nn)]σ 2 .

(5)

The best unbiased estimator of σ 2 obtainable from the whole population is σˆ 2 = (N − 1)−1

i=1

y = n−1

1. If the n sample units were selected using simple random sampling∗ without replacement∗ , a form of probability sampling in which each population unit has the same probability of inclusion in the sample, the estimator y is unbiased over repeated sampling and its (design based) variance is

N 

(Yi − Y)2 .

(6)

i=1

Comparing (5) and (6) with (3), the model under which the population is generated by the stochastic process (4) can be used to validate both the estimator (2) and its variance formula (3) even though the sample is not a probability sample. Estimates of this kind based on quota samples are therefore validated to the extent that, for each type of unit for which a separate quote is specified, the population can be regarded as generated by a separate stochastic process of the type (4). An example of a situation where quota sampling has been used successfully is the following. The Australian Bureau of Statistics collects its international migration∗ statistics using Incoming and Outgoing Passenger Cards which are completed by the

QUOTA SAMPLING

passengers themselves. Some years ago the cards were redesigned in a fashion that was intended to be easier to follow, and it was considered desirable to test out the new design in a realistic situation. Most international passengers arrive and depart by air, and it was necessary to conduct the test at airports in a situation where passengers were continually arriving and departing. It was not possible to select probability samples in these circumstances. Instead, quotas were set for persons arriving from and departing to various overseas airports in accordance with statistics of passenger movements. Quota sampling was especially suitable for this test because: 1. Probability sampling was impracticable. 2. The statistics were required for internal decision making rather than for public use. 3. The effect observed was very clear cut (in favor of the new format). Usually, however, statistical offices tend to avoid the use of quota sampling on the grounds that the assumptions on which it is based may not be reliable [i.e., the model (4) may not describe reality sufficiently well]. Dangers in the use of quota sampling arise chiefly from the element of subjective choice allowed in the selection of the sample in the field. Interviewers tend to choose, if they can, potential respondents whom it would be pleasant and not threatening to interview. In the shopping center situation mentioned above, the following types of shopper tend to be avoided by most interviewers: Aggressive-looking shoppers Down-at-heel shoppers Harrassed-looking shoppers Impressive-looking shoppers Mothers with several small children Shoppers laden down with parcels The reader can no doubt add to this list. The element of subjective choice in quota sampling tends to invalidate the model (4) in

6785

that it breaks any population, or any subpopulation for which a quota is set, into recognizable and relevant subsets with different values of µ and σ 2 . An important special case of quota sampling is probability sampling with quotas, a term used frequently in household-based surveys. Probability sampling is used down to the level of street block, and quotas (e.g., on the numbers of respondents in various age–sex cells) are set either within the individual blocks or over a number of blocks. The chief problem here is that the respondents tend to be chosen from those who spend most time at home. As a result, characteristics which are strongly correlated with time spent at home, such as extent of television viewing, can be quite poorly measured. For many practical purposes, however, probability sampling with quotas provides an acceptable compromise between full probability sampling, which can be extremely expensive, and pure quota sampling, which is unduly hazardous. BIBLIOGRAPHY Barnett, V. D. (1974). Elements of Sampling Theory. English Universities Press, London, pp. 102–103. Conway, F. (1967). Sampling: An Introduction for Social Scientists. Minerva Ser. No. 18. Allen & Unwin, London, pp. 133–134. Deming, W. E. (1950). Some Theory of Sampling. Wiley, New York, pp. 11–14. Deming, W. E. (1960). Sample Design in Business Research. Wiley, New York, pp. 31–33. Huff, D. (1954). How to Lie With Statistics. Gollancz, London, pp. 11–26. (An amusing and instructive account of ‘‘samples with a built-in bias,’’ not limited to quota sampling.) Stephan, F. F. and McCarthy, P. J. (1958). Sampling Opinions. Wiley, New York. (Several relevant passages can be found using the index.) Stuart, A. (1962). Basic Ideas of Scientific Sampling. Griffin’s Statist. Monogr. Courses, No. 4, London, pp. 10–11. (Explains some of the dangers of nonrandom sampling.) Sudman, S. (1966). J. Amer. Statist. Ass., 61, 749– 771. Sukhatme, P. V. (1954). Sampling Theory of Surveys with Applications. Indian Society of Agricultural Statistics, New Delhi/Iowa State University Press, Ames, Iowa, p. 10.

6786

QUOTIENT METHOD

Tull, D. S. and Albaum, G. S. (1973). Survey Research, A Decisional Approach. Intext Educational Publications, New York, pp. 38–39. See also PROPORTIONAL ALLOCATION; SIMPLE RANDOM SAMPLING; and SURVEY SAMPLING.

The variances of the difference series are computed as follows. For the original observations, V0 =

N 

(log xt − log x)2 /(N − 1),

t=1

log x =

QUOTIENT METHOD

xt = mt + mt ut = mt (1 + ut ). Taking logarithms, we have log xt = log mt + log(1 + ut ) log mt + ut . The approximation holds only for small values of ut . We define differences recursively:  log xt = log xt+1 − log xt ,  log xt = (

k−1

log xt / N,

t=1

Let m be a smooth function of time, that is, a function which can be eliminated or greatly reduced by forming finite differences∗ . If an observation xt = mt + ut (ut a random function), we use the variate difference method; if xt = mt + mt ut , the quotient method is appropriate. The quotient method is a variant of the variate difference method∗ and has been developed by Strecker [6,7,9]. The model of the variate difference method is additive; here the model is multiplicative. Let xt > 0 (t = 1, 2, 3, . . . , N) be the observations of a time series∗ ; mt is a smooth systematic part, ut a random variable, and

k

N 

log xt ).

The taking of differences is a linear operation. We have k log xt = k log mt + k log(1 + ut ) k log mt + k ut . Now log mt may be approximated by a polynomial of order K = k0 − 1, say (recall that mt is a smooth function). The properly reduced variances of the observations of order k0 , k0 + 1, k0 + 2, . . ., must be equal apart from random fluctuations.

and for the difference series, Vk =

N−k 

(k log xt )2 /[(N − k)2k Ck ],

t=1

where 2k Ck denotes the binomial coefficient 2k . Using the formulae given in Tintner k [8, p. 51ff] we get tests for the differences Vk+1 − Vk . These yield a large-sample test for the estimation of k0 . A more general test is due to T. W. Anderson [1]. We test (Vr − Vq )(N − q)/Vq . The quantity   N−q          4r 4q 2q + 2r          r  2    +  2q − 2  q + r      2 2 2q2r   2r 2q     q r r q ×(Vr − Vq )/Vq has for large samples a normal distribution with mean zero and variance 1. For a small-sample test based on a circular distribution, see Rao and Tintner [4]. For a consideration of the multiple-choice problem, see Rao and Tintner [5]. See also Tintner et al. [9]. By testing successively V0 − V1 , V1 − V2 , . . ., we estimate k0 . We may then use Vk0 as an approximation to σ 2 = Eu2t . K = k0 − 1 is the degree of a polynomial which approximately represents log mt . Now let k0 = 2n for k0 even and k0 = 2n + 1 for k0 odd. A smoothing formula that minimizes E{log(1 + ut ) + log ft }2

QUOTIENT METHOD

results as follows: log ft = b1 2n (ut−n ) + b2 2n+2 (ut−n−1 ) + · · · , where the bi are constants. Hence log mt ≈ log m t = log xt + log ft , log m t =

m 

gmn (s)(log xt+s + log xt−s )

s=1

+gmn (0) log xt . gmn are optimum weights; these are tabulated [8, pp. 100 ff.]. n depends on the degree of the polynomial and m represents the desired accuracy. Instead of fitting a polynomial to the whole series of observations, we can use moving averages∗ , which are more flexible; see Fuller [9]. Strecker [7,9] suggests a nonparametric test for the choice between the additive variate difference method and the multiplicative quotient method. A more general model is the Box and Cox [2] transformation∗ ; see Zarembka [10]. Transform the variable x as follows: x(λ) = (xλ − 1)/λ, so that limλ→0 x(λ) = loge x. Then λ = 0 corresponds to the quotient method and λ = 1 to the variate difference method.

6787

REFERENCES 1. Anderson, T. W. (1971). Time Series Analysis. Wiley, New York. 2. Box, G. E. P. and Cox, D. R. (1964). J. R. Statist. Soc. B, 26, 211–252. 3. Fuller, W. A. (1976). Introduction to Statistical Time Series. Wiley, New York. 4. Rao, J. N. K. and Tintner, G. (1962). Sankhya¯ A, 24, 385–394. 5. Rao, J. N. K. and Tintner, G. (1963). Aust. J. Statist., 5, 106–116. 6. Strecker, H. (1949). Mitteilungsbl. Math. Statist., 1, 115–130. 7. Strecker, H. (1970). Metrika, 16, 130–187; 17, 257–259 (1971). 8. Tintner, G. (1940). Variate Difference Method. Principia Press, Bloomington, Ind. 9. Tintner, G., Rao, J. N. K., Strecker, H. (1978). New Results in the Variate Difference Method. Vandenhoeck and Ruprecht, G¨ottingen, Federal Republic of Germany. 10. Zarembka, P. (1974). In Frontiers of Econometrics, P. Zarembka, ed. Academic Press, New York, p. 81. See also MOVING AVERAGES; TIME SERIES; and VARIATE DIFFERENCE METHOD.

GERHARD TINTNER HEINRICH STRECKER

R RADEMACHER FUNCTIONS

trigonometric functions, the Lebesgue functions Ln (x) are independent of x, so that they reduce to Lebesgue constants Ln . Parenthetically, besides the mentioned systems of trigonometric and of Rademacher functions, it seems that only one other system of orthogonal functions is known for which the Lebesgue functions reduce to constants, namely that of the Haar functions. In the case of the {rm (x)}, Rademacher showed that for n → ∞, the Lebesgue constants satisfy the asymptotic equality Ln  (2n/π )1/2 ; this proves, in particular, that for arbitrary orthogonal systems, Ln = O(n1/2 ) is the best possible. For specific systems one may, of course, do better; indeed, for the aforementioned system of Haar functions, one has Ln (x) = 1, and Ln (x) is independent not only of x, but also of n. The original reason for the consideration of the system {rm (x)} had been the construction of a counterexample, and since then the system has proven useful for the construction of many other counterexamples, but the interest and general relevance of the Rademacher functions far outweigh that particular use. As pointed out in ref. 5, the system {rm (x)} is not complete; indeed, the functions fmk (x) = rm (x)rk (x), m = k, are not identically zero but are orthogonal over (0, 1) to all rj (x). The system is, however, easily completed; the resulting complete system is known as the system of Walsh functions {wm (x)}. The system {rm (x)} has many remarkable properties, some of which are shared by other lacunary systems [see ref. 4 (Chap. VII)]; these are orthonormal systems {φm (x)}
∞ 2 such that if m=1 cm converges and if the


m Let x = ∞ m=1 em /2 be the dyadic expansion of the real number x, 0
x
1, where em = em (x) is either 0 or 1. If we set rm (x) = 1 − 2em (x) (m = 1, 2, 3, . . .), then the {rm (x)} form a system of functions, orthonormal over the interval (0, 1); these are the Rademacher functions, as usually defined at present. It is easy to visualize these functions (see Fig. 1). The only values taken by the functions rm (x) are +1 and −1; occasionally, one also sets rm (x) = 0, if x is a dyadic rational, where the rm (x) are discontinuous. One observes that if one ‘‘rounds off’’ the corners of the graph, then r1 (x) resembles sin 2π x, r2 (x) behaves like sin 4π x, and generally, rm (x) mimics the behavior of sin(2π · 2m−1 x); the precise relation between these functions is rm (x) = sgn sin 2m π x. These functions were defined by Hans Rademacher in Part VI of his paper [5] on series of general orthogonal functions. For the sake of comparison, one should observe that Rademacher’s ψm (x) are actually our −rm (x). The change of sign made here conforms to contemporary usage and, although it is only a trivial modification, it makes the system more convenient. Rademacher introduced these functions to prove that his two estimates for the Lebesgue functions Ln (x) of general orthonormal systems, namely Ln (x) = O(n1/2 (log n)3/2+ ) unconditionally and Ln (x) = Ln = O(n1/2 ) if the Ln (x) are independent of x, are essentially the best possible. Indeed, Rademacher showed that for the {rm (x)}, just as for the orthonormal system {cos 2π mx, sin 2π mx} of

Figure 1. 6788

RADEMACHER FUNCTIONS


series ∞ m=1 cm φm (x) converges (strongly) in 2 L , then it represents a function f (x) ∈ Lp for some p > 2, the order of lacunarity. The Rademacher system {rm (x)} is lacunary to all orders p, 1
p < ∞. Among its properties are the following: 1. Let cm (m = 1, 2, . . .) be real numbers
and denote the formal series ∞ m=1 cm rm (x) by S(x), regardless of convergence.
∞ 2 If m=1 cm converges, then S(x) converges a.e. (almost everywhere) on (0, 1) and represents there a function f (x). The series S(x) is called the R-F (Rademacher–Fourier) series of f (x) and the cm are the R-F coefficients of f (x). One recalls that the same condition suffices to ensure the convergence a.e. of Haar function series, while for arbitrary orthonormal systems one needs the stronger [best see ref. 4 (p.
possible; 2 2 162)] condition ∞ m=1 cm log m < ∞ for convergence a.e. If, on the other hand,
∞ 2 m=1 cm diverges, then S(x) diverges a.e.
2. The condition ∞ m=1 |cm | < ∞ is necessary and sufficient for the convergence
of ∞ m=1 cm rm (x) everywhere on (0, 1).
3. If ∞ m=1 |cm | diverges, while limm→∞ cm = 0, then, given reals a
b (not necessarily finite), each subinterval of (0, 1) contains a set {ξ } of the power of the continuum such that the partial sums sn (ξ ) of S(ξ ) satisfy lim inf n→∞ sn (ξ ) = a, lim supn→∞ sn (x) = b. 4. If f (x) ∈ L(0, 1) and also f (x) ∈ Lp (0, 1) and if we denote the p-norm of f (x) 1 by f p = { 0 |f (x)|p dx}1/p , then, if {cm } are the R-F coefficients of f (x), the two inequalities ∞ 2p − 2  2 cm
f (x)2p 3p − 2 m=1

∞  p  2
1+ cm 2 m=1

hold. From the first follows
inequality 2 p the convergence of ∞ m=1 cm if f (x) ∈ L for some p > 1, and then the R-F series of f (x) converges at least a.e. The second inequality shows that the convergence

6789


2 p of ∞ m=1 cm implies that f ∈ L for 1
p < ∞. In analogy with the Riesz–Fischer theorem∗ we also have:
∞ 2 5. (a) If m=1 cm converges, then there exists a continuous function f (x) that has the R-F coefficients cm . (b) If limm→∞ cm = 0, then there exists a Lebesgue integrable function f (x), with the R-F coefficients cm . 1 (c) In either case, cm = 0 f (x)rm (x) dx. To be able to state some converse results, because of the incompleteness of the system {rm (x)} we also need the following concept. We say that f (x) belongs to the (generally incomplete) orthogonal set {φm (x)} if f (x) is orthogonal to every function g(x) that is itself orthogonal to all functions φm (x). 6. (a) If
f (x) belongs to {rm (x)} and f (x) ∞ =
∞ m=12 cm rm (x) formally, then m=1 cm converges and f (x) is represented by its R-F series at least a.e. (b) If, furthermore, f (x)
is bounded and ∞ measurable, then m=1 |cm | converges and [see (2)] f (x) is represented by its R-F series everywhere on (0, 1). These results are due to many authors, among them, in addition to Rademacher, Steinhaus, Kaczmarz, Paley, Zygmund, Kolmogorov, and Menchov; for an attribution of discoveries, see refs. 3 and 4. The completion {wm (x)} of the system {rm (x)} was discovered independently by Rademacher [6], by Kaczmarz [3], and by Walsh [7] and earlier was sometimes referred to as the Rademacher–Kaczmarz or Walsh–Kaczmarz system. For details on the history of the contributions to the theory of the Rademacher and Walsh functions, see ref. 1. The importance of the system {rm (x)} of Rademacher functions for statistics comes from the fact that it may be considered as a model for the tossing of a fair coin (i.e., for a random variable∗ X(x) with Pr[X(x) = +1] =

6790

RADEMACHER FUNCTIONS

Pr[X(x) = −1] = 12 ). To see that, let us consider the set Ej ⊂ [0, 1] on which rj (x) = δj , where δj = either + 1 or −1. From the definition of the rj (x) (see also Fig. 1), it follows that its measure µ(Ej ) = 12 . More generally, if for j = 1, 2, 3, . . . we denote by Ej the set on which rj (x) = δj , where the sequence δ1 , δ2 , . . . consists only of +1 or −1, then for j = k, µ(Ej ∩ Ek ) = µ(rj (x) = δj and rk (x) = δk ) = 2−2 and   m  µ  Ej  = µ(r1 (x) = δ1 , r2 (x) j=1

= δ2 , . . . , rm (x) = δm ) = 2−m =

m 

µ(Ej )

Similarly, the normal∗ (Gauss) law may be formulated as follows:   n  lim µ a < n−1/2 rm (x) < b n→∞

= (2π )−1

=

µ(rj (x) = δj ).

=

(1)

j=1

Comparison of (1) and (2) justifies our previous claim that the system {rm (x)} may be interpreted as a sample space for the tossing of a fair coin, that is, for a random variable X(x), that may take only one of the two values +1 or −1, each with equal probability 12 . Using the {rm (x)}, one can express with ease many probabilistic concepts. So, for example, Pr[X = +1 exactly k times in a sample of n values]   n  rm (x) = 2k − n . =µ m=1

Next, the weak law of large numbers∗ for X is equivalent to the statement that for every  > 0,    n     rm (x) > n = 0. lim µ  n→∞   m=1

dz.

m=1

n 



1

rm (x) dx = 0,

m=1 0

j=1

j=1

2 /2

e−z

As a last example, let us consider the random variable Zn = X1 + · · · + Xn , where for 1
m
n, each Xm = Xm (x) is a random variable with Pr[Xm = +1] = Pr[Xm = −1] = 1 2 ; also set Z = limn→∞ Zn . Then the expected value∗   1  n E(Zn ) = rm (x) dx 0

If, on the other hand, we interpret the event Ej as the outcome of the jth toss of a fair coin, then the probability of m consecutive tosses with preassigned outcomes is 2−m and, from the statistical independence∗ of the tosses it follows that   m m   Pr[Ej ]. (2) Pr  Ej  = 2−m =

b a

j=1 m 

m=1



whence E(Z) = limn→∞ E(Zn ) = 0. It is convenient to remark that one cannot write directly
E(Z) = ∫10 ( ∞ m=1 rm (x))dx, because the series diverges. There exist many other important and fascinating applications of the Rademacher functions in analysis (e.g., the proof of Vieta’s trigonometric identity), in number theory (e.g., the Borel normality∗ of almost all real numbers), in probability theory∗ , and so on. Many of these topics are beautifully discussed in the book [2] by M. Kac. REFERENCES 1. Grosswald, E. (1981). Analytic Number Theory, Lect. Notes Math., 899, Springer-Verlag, New York, pp. 1–9. 2. Kac, M. (1959). Statistical Independence in Probability, Analysis and Number Theory. Carus Math. Monogr. 12, Math. Ass. Amer.; distributed by Wiley, New York. (A beautifully written, most pleasant book to read.) 3. Kaczmarz, S. (1929). C. R. I-er Congr. Math. Pays Slaves, Warsaw, pp. 189–192. 4. Kaczmarz, S. and Steinhaus, H. (1935). Theorie der Orthogonalreihen. Monogr. Mat. VI. Warsaw. (Reprinted, Chelsea, New York, 1951.) (A very clearly written, comprehensive presentation.) 5. Rademacher, H. 112–138.

(1922).

Math.

Ann.,

87,

RADIAL ERROR 6. Rademacher, H. (1922). Unpublished. (Longlost manuscript, found posthumously and dated by its author January 1922.) 7. Walsh, J. (1923). Amer. J. Math., 55, 5–24. See also ORTHOGONAL EXPANSIONS and RIESZ —FISCHER THEOREM.

EMIL GROSSWALD

6791

c = 0.0(0.1)1.0. Tables of F(r) by Harter [9] have been extended by Lowe [11] and DiDonato and Jarnagin [4] to the case where the means of X and Y are not zero. Without explicitly using its PDF, Scheuer [13] gives the nth moment of R in the case where X and Y have zero means but non-zero covariance σXY . E(Rn ) = (2σ12 )(1/2)n ( 12 n + 1)

RADIAL ERROR

× F(− 21 n, 12 ; 1; k2 )

If X and Y are two random variables (RVs) measured along Cartesian coordinate axes, the distribution of the radial distance R = (X 2 + Y 2 )1/2 has numerous applications. For example, if (X, Y) is the impact point of a missile aimed at a target at the origin, the miss distance is called the radial error. If the lethal radius of a missile is r, the probability of destroying a target is P[R
r] = F(r), the cumulative distribution function (CDF) of R. The median of R, obtained by solving F(r) = 0.5, is called the circular probable error∗ . The RV R is clearly the radial distance of a point undergoing a random walk∗ [12]. Other applications of R are in the fields of signal detection [10], meteorology∗ [2], and astronomy [14], among others. Assuming X and Y to be normally and independently distributed with zero means and standard deviations σX and σY such that c = (σX /σY )
1, Chew and Boyce [3] give the following probability density function (PDF) and CDF of R. f (r) = {r/(σX σY )} exp(−ar2 )I0 (br2 ), 0
r r]. More realistically, d(R) is a continuously decreasing function of R. The estimation problem is considered in Eckler [6] and, in the case of truncated observations, in Dyer [5].

REFERENCES 1. Abramowitz, M. and Stegun, I. A., eds. (1972). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Table. Wiley, New York. 2. Brooks, C. E. P. and Carruthers, N. (1953). Handbook of Statistical Methods in Meteorology. H. M. Stationery Office, London. 3. Chew, V. and Boyce, R. (1962). Technometrics, 4, 138–140. 4. DiDonato, A. R. and Jarnagin, M. P. (1962). Math. Comp., 16, 347–355. 5. Dyer, D. D. (1974). Operat. Res., 22, 197– 201. 6. Eckler, A. R. (1969). Technometrics, 561–589. (Contains 60 references.)

11,

7. Edmundson, H. P. (1961). Operat. Res., 9, 8–21. 8. Guenther, W. C. and Terragno, P. J. (1964). Ann. Math. Statist., 35, 232–260. (Contains 58 references.) 9. Harter, H. L. (1960). J. Amer. Statist. Ass., 55, 723–731. 10. Helstrom, C. W. (1968). Statistical Theory of Signal Detection, 2nd ed. Pergamon Press, Elmsford, N.Y. 11. Lowe, J. R. (1960). J. R. Statist. Soc. B, 22, 177–187. 12. Papoulis, A. (1965). Probability, Random Variables, and Stochastic Processes. McGrawHill, New York. 13. Scheuer, E. M. (1962). J. Amer. Statist. Ass., 57, 187–190; corrigenda: 60, 1251 (1965). 14. Trumpler, R. J. and Weaver, H. F. (1962). Statistical Astronomy. Dover, New York. See also BIVARIATE NORMAL DISTRIBUTION; CIRCULAR NORMAL DISTRIBUTION; and TARGET COVERAGE.

RADON–NIKODYM THEOREM THE THEOREM A precise statement of the theorem is as follows: Let (X, S, µ) be a σ -finite measure space and ν a σ -finite, possibly signed measure on (X, S). If ν is absolutely continuous with respect to µ, then there is a finite-valued, measurable function f uniquely determined almost everywhere (µ) such that   g dν = gf dµ X

whenever either integral is well defined, and in particular for every E in S we have ν(E) = ∫E fdµ. The function f is called the Radon– Nikodym derivative of ν with respect to µ and is denoted by f = dν/dµ. Thus the Radon–Nikodym theorem enables us to express integration with respect to a measure ν which is absolutely continuous to µ, in terms of integration with respect to µ via their Radon–Nikodym derivative. Its proof can be found in any book on measure theory or real analysis. Like most ‘‘derivatives,’’ the Radon–Nikodym derivative can be expressed at almost every point as the limit of the ratios of the two measures over sets that contain the point and belong to a refining sequence of partitions (see Hewitt and Stromberg [2, p. 373]). The theorem is not necessarily valid for measures that are not σ -finite (see Zaanen [3, Sec. 33]). ν is absolutely continuous with respect to µ if all sets of µ-measure zero have also ν-measure zero. In contrast to absolute continuity∗ is the notion of singularity: µ and ν are singular if they live on disjoint sets [i.e., for some E in S, µ(E) = 0 and ν(Ec ) = 0, or |ν|(Ec ) = 0 in the signed case]. The Lebesgue decomposition theorem says that ν can always be decomposed into two terms, νa and νs (ν = νa + νs ), of which one is absolutely continuous and the other singular to µ, so that    g dν = gf dν + g dν, X

V. CHEW

RADICO-NORMAL

DISTRIBUTION.

See MODIFIED NORMAL DISTRIBUTIONS

X

X

E0

where f = dνa /dµ and µ(E0 ) = 0, and in particular ν(E) = ∫E fdµ + ν(E ∩ E0 ). The Radon–Nikodym theorem has applications to analysis, probability, and statistics.

RADON–NIKODYM THEOREM

APPLICATIONS Probability Density Functions Absolutely continuous distribution functions F(x) have densities f (x) : F(x) = ∫x−∞ f (t)dt and F  (x) = f (x) for almost every x. Similarly, each absolutely continuous function is the indefinite integral of its derivative. Change of Variable of Integration The real variable of integration can be changed by an absolutely continuous function h as follows: 

h(b) h(a)



b

g(x)dx =



g(h(t))h (t)dt, a

whenever either integral is well defined. Local Times The local time∗ of a function or of a random process or field plays an important role in the study of its smoothness properties (for a comprehensive survey, see Geman and Horowitz [1]). It is well defined as the Radon– Nikodym derivative of its occupation measure as follows. Let f be a (Borel measurable) function defined on the interval I, and ν the occupation measure of f defined for every Borel subset B of the real line as the ‘‘time’’ the function f spends at values in B [i.e., ν(B) = µ{x ∈ I : f (x) ∈ B}, where µ is Lebesgue measure]. If the occupation measure ν is absolutely continuous with respect to Lebesgue measure, its Radon–Nikodym derivative α(x) = (dν/dµ)(x) is called the local time or occupation density of the function f and carries the interpretation that α(x) is the time spent by the function f at the level x. Conditional Probability and Expectation∗ Conditional probabilities and expectations are defined in the general (nondiscrete) case as Radon–Nikodym derivatives. To be specific, let us fix two random variables X and Y with E|X| < ∞ and define E(X|Y = y). Notice that when Y is continuous, P(Y = y) = 0 for all y, and the conditional expectation cannot be defined in the elementary fashion. It can be defined though, at least for almost every

6793

value y of Y, by means of the Radon–Nikodym theorem via the relationship  E[X|Y = y] dFY (y) E[X1B (Y)] = B

for all Borel sets B. Indeed, the left-hand side defines a finite signed measure on the Borel sets, which is absolutely continuous with respect to the distribution FY of Y, and thus by the Radon–Nikodym theorem there is a function of y, defined uniquely for almost every y (FY ) and denoted by E[X|Y = y], which satisfies the foregoing relationship. Likelihood Ratios∗ The Neyman–Pearson∗ test of two hypotheses is based on the likelihood ratio, which is closely related to Radon–Nikodym derivatives. Let P0 and P1 be the probability measures associated with the two hypotheses. The likelihood ratio L = (dP1 /dµ)/(dP0 /dµ) is the ratio of the Radon–Nikodym derivatives, where µ is a probability measure with respect to which both P0 and P1 are absolutely continuous [such as (P0 + P1 )/2]. An alternative expression is via the Lebesgue decomposition of P1 with respect to P0 : L = ∞ on the (P0 -null) event where the singular part of P1 lives, and outside this event L = dP1a /dP0 , the Radon–Nikodym derivative of the absolutely continuous part of P1 . If a finite sample is available to the statistician, the probabilities P0 and P1 are defined on a finite-dimensional Euclidean space and typically µ is Lebesgue or counting measure and L the ratio of two probability densities. If a continuous sample (such as the sample of a time series over an interval), is available to the statistician, the probabilities P0 and P1 are defined on infinite-dimensional function spaces, typically no natural dominating measure µ exists, and one works with the alternative expression via the Lebesgue decomposition; in such cases L is quite difficult to compute and has been calculated only in few cases, such as when under both hypotheses the observed continuous random process is Gaussian, or Poisson, or more generally infinitely divisible. REFERENCES 1. Geman, D. and Horowitz, J. (1980). Ann. Prob., 8, 1–67.

6794

RADON TRANSFORMATION

2. Hewitt, E. and Stromberg, K. (1965). Real and Abstract Analysis. Springer-Verlag, New York. 3. Zaanen, A. C. (1967). Integration. NorthHolland, Amsterdam. See also ABSOLUTE CONTINUITY and MEASURE THEORY IN PROBABILITY AND STATISTICS.

parental dose group by using litter size as a covariate, structured on the assumption that larger litter sizes represent better maternal health and hence lower fetal risk. The model has the form P(d, s) = λ(d)P[d, s; λ(d)],

S. CAMBANIS

RADON TRANSFORMATION The Radon transform of a two-variable function f (x, y) is defined as R(f ) = p(µ|θ )  ∞ f (u cos θ − υ sin θ , = −∞

u sin θ + υ cos θ ) dυ. If f (x, y) is the probability density function (PDF) of random variables X, Y then p(µ|θ ) is the PDF of U = X cos θ + Y sin θ , that is, the projection of the point (X, Y) on the line Y/X = tan θ . The transform is useful in the analysis of spatial data∗ and signal processing. Reference 1 contains a discussion of estimation problems, and a bibliography. REFERENCE 1. Pattey, P. A. and Lindgren, A. G. (1981). IEEE Trans. Acoust. Speech Signal Process., 29, 994–1002. See also SPATIAL DATA ANALYSIS and SPATIAL PROCESSES.

RAI AND VAN RYZIN DOSE–RESPONSE MODEL Rai and Van Ryzin [1] proposed a dose—response model specifically for teratological experiments involving quantal responses which they characterize as a mechanistic model because it can represent two stages in the development of a defect in a fetus whose parent has been exposed to a toxicant just before or during gestation. This model deals with the relatedness within litters (intralitter correlation) by accounting for the variability among litters in the same

where P(d, s) is the probability of an adverse response in a fetus from a litter of size s with parent exposed to dose level d, λ(d) is the probability of a predisposing disturbance in the litter environment, and P[d, s; λ(d)] is the conditional probability of an adverse response in a fetus with parental exposure dose d, in a litter of size s, given a predisposing disturbance in its litter environment. In their article the first stage, λ(d), is represented by a one-hit model, but other models could be used. The second stage is expressed as an exponential function in dose containing litter size s. P(d, s) is thus formulated as P(d, s) = {1− exp[−(α + βd)]} × exp[−s(θ1 + θ2 d)]. The marginal probability of fetal response, P(d), is obtained by assuming a distribution for litter size, such as the Poisson∗ , and taking the expectation of P(d, s) over all litter sizes. The likelihood function∗ for parameter estimation and hypothesis testing uses the binomial distribution∗ with parameter P(d, s). Since the log-likelihood for P(d) divides into two discrete sections, the dose—response parameters and the litter-size distribution parameters are estimated separately. Standard errors estimation and a goodness-of-fit∗ test are based on the information matrix and the assumption of joint asymptotic normality of the parameter estimates. Two methods are proposed for estimating doses that produce very low specified additional risks and their confidence intervals. Method 1 uses the assumption of normality of the dose estimate and a first-order Taylor expansion to obtain an approximation of its standard error. Method 2 provides a conservative method of low-dose interpolation by using the upper bound on P(d, s), i.e., P(d, 0), for estimation of the ‘‘virtually safe dose’’ associated with a specific risk.

RAINFALL, LANDFORMS, AND STREAMFLOW

REFERENCES 1. Rai, K. and Van Ryzin, J. (1985). Biometrics, 41, 1–10. 2. Van Ryzin, J. (1985). Toxicology and Environmental Health, 1, 299–310. (This is a version of ref. 1 for toxicologists.) See also BIOASSAY, STATISTICAL METHODS IN; DOSAGE –RESPONSE CURVE; and PROBIT ANALYSIS.

D. G. WELLINGTON

6795

(e.g., by rarefactions) of Poisson processes∗ , which in turn play important roles in reliability theory∗ , telecommunications, nuclear physics, and other fields. Raikov’s original work appeared in [3]. ´ Lukacs [2] discusses in detail the problem of factorization of characteristic functions. Some basic results on renewal processes, in which Raikov’s theorem plays an implicit role, can be found in Galambos and Kotz [1, Chap. 4].

RAIKOV’S THEOREM

REFERENCES

Raikov’s theorem is concerned with the following factorization problem of the Poisson∗ characteristic function∗

1. Galambos, J. and Kotz, S. (1978). Characterizations of Probability Distributions. SpringerVerlag, Berlin. ´ 2. Lukacs, E. (1970). Characteristic Functions. Charles Griffin, London. 3. Raikov, D. A. (1938). Izv. Akad. Nauk SSSR, Ser. Mat., 2, 91–124.

φ(t) = exp[λ(eit − 1)]. Assume that φ1 (t) and φ2 (t) are two characteristic functions satisfying φ(t) = φ1 (t)φ2 (t).

(1)

See also CHARACTERISTIC FUNCTIONS and POISSON PROCESSES.

JANOS GALAMBOS What can be said about the two factors φ1 (t) and φ2 (t)? One immediately finds the following two solutions of (1): (a) φ1 (t) = eict ,

φ2 (t) = φ(t)e−ict ,

c real ;

and (b) φ1 (t) = exp[λ1 (eit − 1) + ict], λ1 > 0, c real, φ2 (t) = exp[λ2 (eit − 1) − ict], λ2 > 0, with λ1 + λ2 = λ. Now, Raikov’s theorem states that (a) and (b) represent all solutions of (1). It thus implies that if a Poisson random variable X can be decomposed as X = Y + Z, where Y  0 and Z  0 are independent random variables (neither of which is identically zero), then both Y and Z are Poisson [note that the assumption Y  0, Z  0 implies that c = 0 in both (a) and (b)]. Raikov’s theorem is of interest in its own right. It also has implications in applied fields, such as superpositions of independent renewal∗ processes, and decompositions

RAINFALL, LANDFORMS, AND STREAMFLOW RAINFALL Rain gages and land-based radar are the two most widely used sources of rainfall data. Rainfall measurements made with gages are generally accurate, giving rainfall depth as a function of period of accumulation at points where gages are located. Rain-gauge networks, however, are generally sparse, making it difficult to assess spatial variability of rainfall. On the other hand, radar measures volume-averaged returned power, which must be converted to rainfall by a calibration* relationship. The estimates of rainfall obtained using radar are therefore generally continuous in space but are subject to a high degree of uncertainty due to errors in the calibration. Rainfall typically exhibits a high degree of variability in both space and time on all scales. Temporal variability is illustrated in Fig. 1, which shows a daily rainfall time series for a 1-yr. period in northern Mississippi. The rainfall spans the period from

6796

RAINFALL, LANDFORMS, AND STREAMFLOW

180 160

Rainfall (mm)

140 120 100 80 60 40 20

October 1, 2001 until September 30, 2002 and is the average of data collected at 30 rain gages located in or near the Goodwin Creek Experimental Watershed, covering an area of about 20 sq km. This data may be downloaded from the web site http://www.sedlab.olemiss. edu/cwp unit/Goodwin.html. The most noticeable feature of rainfall, which makes it difficult to model stochastically, is its intermittency, or the subsets of space-time for which there is little or no accumulation of rainfall. We shall consider first methods that have been used for modeling temporal variation of rainfall at a fixed location in space, and then methods for modeling spatial rainfall variability. Temporal Rainfall Consider first the temporal variability of rainfall at a single point in space, or more precisely, volumes accumulated over a very small region. Modeling the temporal structure of rainfall at the monthly, seasonal, or annual scale has been done in many studies with autoregressive-moving average (ARMA)* or periodic ARMA models [62,44]. With shorter time scales, however, it is generally necessary to account for the intermittency, or existence of discrete ‘‘events,’’ where an event is taken to be a wet period between two long periods of no rainfall. This is usually done by considering two processes, one (a counting

30-Sep-02

1-Aug-02

1-Jun-02

1-Apr-02

30-Jan-02

30-Nov-01

Figure 1. Daily rainfall time series for Goodwin Creek, MS, for a 1-yr period.

1-Oct-01

0

Time

process*) to describe the timing of the occurrence of events, and the other to describe event magnitude, given that an event has occurred. Modeling of the occurrence process may be done either in discrete time intervals (e.g., hourly or daily) or in continuous time. A common approach in modeling daily rainfall, for example, is first to define a two-state Markov chain that determines ‘‘rain’’ or ‘‘no rain,’’ and then, given a rainy day, to sample rainfall volume from a continuous distribution. An extension of this idea is to employ a multistate Markov chain model and then to sample rainfall volumes from a distribution, conditional on state. In this model, the Markov chain is often interpreted as describing an underlying unobservable ‘‘climatic state’’ variable, and the model is a so-called hidden Markov model∗ [69]. Gibbs sampling* is used in Reference 55 for estimating parameters of such a model, placed in a Bayesian framework. The simplest continuous time event occurrence model is the Poisson process*, which has been used extensively in rainfall modeling. In the rectangular pulses model [38], for example, a Poisson process defines timing of event beginnings, and a random duration and intensity is associated with each event. It is assumed that rainfall intensity associated with each event is constant over its duration, and that total intensity at a given time

RAINFALL, LANDFORMS, AND STREAMFLOW

is the sum of contributions from all events active at that time. Typically, durations and intensities are mutually independent random variables, independent of the Poisson arrival process. This model points to one difficulty in continuous time models; because there may be superposition of rainfall amounts from different events at any given time, it is not possible in general to uniquely extract all arrival time, duration, and intensity random variables from an observed record. The independent increments property of a Poisson occurrence model renders it unrealistic for modeling rainfall in most cases [21,56]. Various more general approaches have been taken. For example, the counting process in an early work [56] was taken to be a nonhomogenous pure birth Markov process. More recent approaches tend to focus on explicitly preserving the hierarchical structure of rainfall intensity, that is, the fact that within a storm event there are identifiable and distinct structures (‘‘cells’’) that are separated by periods of zero or lower intensity. Clustered point processes have proven to be very useful in modeling this phenomenon [66]. An example [38] of a model based on the Neyman–Scott cluster process is an extension of the rectangular pulses model for which the following assumptions are made: (i) storm event origins arrive according to a Poisson process; (ii) each storm origin generates a random number (Poisson distributed) of rain cells, for which the waiting time to beginning of the pulse after the storm origin is exponentially distributed; (iii) the duration of each cell is exponentially distributed; (iv) the intensity of each cell is an exponentially distributed random variable; (v) the total intensity at any instant in time is the sum of intensities due to all active cells at that time. Random variables in this model are assumed to be mutually independent. This approach has been generalized in a number of ways, for example, by allowing more levels of clustering (e.g., there may be ‘‘subcells’’ within cells) or by modeling dependence between cells within a storm [39]. Also, it may be assumed that a Bartlett–Lewis (rather than Neyman–Scott) mechanism, for which cell pulse timing within a storm is governed by a Poisson process, applies.

6797

Latest efforts at analyzing and modeling rainfall have focused increasingly on the hypothesis that rainfall may exhibit certain forms of scaling invariance. In such an approach, dependence of rainfall properties on scale of observation, as specified by an index λ, where 0 < λ
1, is analyzed and modeled explicitly. This approach to modeling rainfall has been seen as a significant advance by many. The clustered point-process models of rainfall may have a relatively large number of parameters to estimate, and fitting with data at one scale does not guarantee preservation of properties at other scales. Application of models exhibiting scaling invariance provides a means of representing in a parsimonious manner the relationship among different scales. For a two-dimensional spatial region B and an interval of time T, let V(B, T) stand for volume of rain falling in region B during time interval T. Data analysis to detect or quantify temporal scaling properties often involves looking at two quantities Mλ (h) =



V(B, Tλ,j )h

(1)

j

and Nλ (α, ) =



I[λα+ ,λα ) (V(B, Tλ,j )),

(2)

j

where Tλ,j is the time interval [λj, λ(j + 1)). Power-law* dependence on scale, λ, is the sign that some sort of scale invariance holds; in particular, it is of interest to estimate scaling exponents τ (h) and f (α) such that log Mλ (h) ≈ τ (h), − log λ

(3)

log Nλ (α, ) ≈ f (α), − log λ

(4)

for small λ and . A number of studies based on a so-called multifractal formalism (see ref. 8, Chap. 6) have indicated that the scaling exponents τ (h) and f (α) are Legendre transform pairs under various assumptions on the underlying generating process. Rigorous results for analyses along these lines (see the Appendix) usually proceed from the construction of a stochastic process {X(s); s ∈ S}

6798

RAINFALL, LANDFORMS, AND STREAMFLOW

that is simple-scaling or multiscaling in that it exhibits a distributional invariance when elements s of the parameter space are scaled by λ. For temporal rainfall, S represents an interval of time, and for a fixed region B, X(s) = V(B, [0, s)). Analysis of behavior of Equation 1 for h = 2 can be used as an indicator of long-range dependence*. Also, τ (h) is a function of moment order h, linear for simplescaling processes and nonlinear for multiscaling processes (see the Appendix). Some rigorous inferential procedures are available for estimation of τ (h) (e.g., Refs. 31 and 61). Another commonly applied data analysis tool to detect scaling is spectral analysis*, where power-law dependence on frequency is sought. Case studies that have appeared in the literature range from various data analyses that provide evidence of scaling (i.e., powerlaw behavior as a function of scale) to fitting of parametric models such as the Mandelbrot cascade with a family of distributions assumed for the generator (Appendix). An example of an analysis demonstrating evidence for scaling is the study in reference 40, in which long-range temporal dependence is indicated based on an analysis of second moments using composite radar and raingauge data with temporal resolution of 1 d and spatial resolution of 4 km. In reference 49, evidence of nonlinearity of τ (h) (multiscaling) is found in analyzing daily rainfall time series* data from a temperate climate in Sweden and a monsoon climate in China, and a difference in scaling exponents between the two regions is also found. Spatial Rainfall Historically, spatial prediction has been a question of primary interest in rainfall analysis; that is, it is desired to predict V(B, T) for a region B and a time interval T based on data from a (probably sparse) rain gage network. The rain gage sampling error problem is exemplified by the Global Precipitation Climatology Project, one goal of which is to provide mean rainfall estimates on a 2.5 degree by 2.5 degree grid; only about 15% of these grid boxes over the land part of Earth have more than three gages [27]. Optimal (unbiased and minimum variance) linear estimation, or kriging*, has been widely used

to extrapolate rainfall spatially and to predict spatial averages (block kriging) on the basis of data from a network. Use of radar data in conjunction with gage data is widely used to improve spatial prediction. Given that radar does not measure rainfall intensity directly, an important question is how best to combine information from the two sources to obtain predictions. Optimal linear estimation with an auxiliary variable (cokriging) has been used in this case. Generally, however, because of the sparsity of rain-gage data the second-order statistics required for either kriging or cokriging can only be estimated with large uncertainty [45]. Spatial rainfall fields, like rainfall time series at a point, exhibit a hierarchical, intermittent structure that is challenging to model. Clusters of high intensity rain cells are embedded in clusters of lower intensity regions, called small mesoscale areas, which are in turn embedded in larger regions of even lower intensity, called large mesoscale areas, which are embedded in even larger regions known as synoptic areas. Each decrease in scale results in an approximate increase of rain rate by a factor of 2 and an approximate decrease of structure size by an order of magnitude [13]. The clustered point-process models used in temporal rainfall analysis may be generalized to a spatial or a spatial-temporal setting. In Reference 4, storm origins are modeled as a Poisson process in space and time, cells are taken to be circular in shape with a random radius and random duration, and temporal clustering of cells within storms is taken to have a Bartlett–Lewis structure with cells originating at the same spatial location. All cells are assumed to move with a common velocity. This model is generalized in reference 29 by allowing a Neyman–Scott mechanism for spatial clustering. See reference 67 for a more complete review of these approaches and a discussion of parameter estimation using the method of moments* and spectral methods. Latest research in modeling spatial rainfall has, as in the modeling of temporal rainfall, focused on explicit preservation of scaling properties. A scaling approach to modeling spatial rain structure is more parsimonious than hierarchically clustered point-process models,

RAINFALL, LANDFORMS, AND STREAMFLOW

and hence parameter estimation is more straightforward. As in the case of temporal rainfall, rigorous analysis is based on postulating an underlying simple-scaling or multiscaling stochastic process {X(s); s ∈ S} (see the Appendix), where S now represents a set of spatial scales (usually areas of two-dimensional regions) rather than time scales. Spatial analogs to Equations 1 and 2 may be defined,  V(Bλ,j , T)h , (5) Mλ∗ (h) = j

Nλ∗ (α, )

=



I[λα+ ,λα ) (V(Bλ,j , T)),

(6)

j

where the sets Bλ,j each have area λ, and again scaling exponents τ (h) and f (α) analogous to those in Equations 3 and 4 are sought. Among the first efforts to apply a simplescaling spatial model to rainfall is the study in reference 24. It is demonstrated in Reference 20, however, that a simple-scaling structure is not consistent with spatial intermittency. A clear deviation from simple-scaling behavior is indicated in reference 12, in which an analysis of radar rain rate obtained during the Global Atlantic Tropical Experiment (GATE) shows that moments exhibit log linearity in scale, with a slope function τ (h) that is nonlinear and concave in h. A random cascade (Appendix) having a one-parameter generator with an atom at zero, which explicitly models the fraction of nonrainy areas, is applied in reference 32; it is found that the parameter varies monotonically with spatial average rain rate, which is a measure of large-scale forcing. Likewise, in reference 30, wide-sense multiscaling behavior of a 30yr. record of daily spatial rainfall obtained from a dense gage network (230 gages covering 10,000 sq km) in southern Sweden is analyzed. This is one of the few studies that uses gage data rather than radar data to study multifractal spatial properties. Separate analyses were done for different rainfall-generating mechanisms, including warm fronts, cold fronts, and convection. The notion of spectral multiscaling is introduced in reference 35, and it is demonstrated in a case study that spatially averaged rain rate exhibits this property. The random cascade spatial model is extended to a space-time

6799

setting in references 33 and 26; in reference 33, this extension is accomplished by explicitly representing the generator as a temporal stochastic process. LANDFORMS The study of surface landforms and their geometric properties has of late been revolutionized by the widespread availability of digital elevation data. Such data is typically given in the form of land-surface elevation values on a regular grid, usually with grid spacing of approximately 30 or 90 m; a single grid square is known as a pixel. Given such elevation data for a region, there are algorithms that may be applied to extract the location of the river channels constituting the channel network draining the region; these algorithms are usually based on an assumption that water will flow from a point in the direction of steepest descent. It may be desired to examine statistical properties of the elevation field, or the channel network may be projected onto a horizontal surface and analyzed as a two-dimensional object. An example of a network that has been extracted using 30-m elevation data for Goodwin Creek, MS, is illustrated in Fig. 2; the software used to extract the network is described in Reference 64. The basin shown has an area of 21.3 sq km. In the next section, we shall first consider two-dimensional geometry and discuss some empirical studies. Then a review of some approaches to stochastic modeling of networks will be presented, followed by a brief review of three-dimensional geometry. Two-Dimensional Geometry For purposes of looking at two-dimensional geometry, a drainage basin may be viewed as a finite region in R2 , which is drained by a channel network for which the outlet is located at some point on the boundary of the region. The basin may be characterized by various quantities that describe its spatial magnitude and shape, the most common of which are its area, denoted by A, and basin diameter, denoted by R, which is the maximal distance from the outlet to any point on the boundary. The channel network draining the watershed is conveniently analyzed as a finite

6800

RAINFALL, LANDFORMS, AND STREAMFLOW

Figure 2. Channel network for Goodwin Creek, MS.

planted plane tree (see, e.g., Ref. 58), with the root (degree one) corresponding to the outlet of the network. Nodes of degree one other than the root are known as sources, and typically, interest is focused on trees for which all nonsource nodes are constrained to have in-degree two, that is, binary trees, as confluences of higher degree are rarely seen in nature. The edges of the tree are links, those having a source at the upstream end defined to be exterior links and all other links taken to be interior links. The magnitude M of the network is the number of sources; there are 2M − 1 links in a binary network. We now define several variables (functions of the drainage basin and the network) that are of interest to geomorphologists and hydrologists. Consider a network of magnitude M. For 1
i
2M − 1, we may associate with the ith link two variables, length Li and area Ai , with the area corresponding to the local drainage area of the ith link, excluding areas upstream. Let DL (i) (DU (i)) be the distance of the downstream (upstream) end of link i from the outlet, where ‘‘distance’’ is computed by summing the lengths in the unique path to the outlet. Thus, DU (i) = DL (i) + Li . We then define A=

2M−1  i=1

Ai ,

(7)

L= W(x) =

DU (i),

(8)

I[DL (i), DU (i)) (x),

(9)

max

1
i
2M−1 2M−1  i=1

where A is drainage area, L is mainstream length, and W(x) is the width function. W(x) is simply the number of links some point of which lies at distance x from the outlet; the width function for the network in Fig. 2 is shown in Fig. 3. There has been much interest in the frequency distribution of these variables and in relationships between pairs of variables when a number of different basins in a region are sampled. In many cases, the frequency distributions exhibit power-law behavior in the upper tail, and a log-linear relationship exists between pairs. Denote by γX the power-law exponent defining tail decay of the density of a variable X and by γY|X the slope of the relationship of log Y against log X; these may be regarded as parameters characterizing a population of drainage basins, although in most studies, the population of interest has not been clearly specified. The range of values typically seen in empirical studies [41] is 1.40
γA
1.45, 1.80
γL
1.90, 0.57
γL|A
0.60, 1.0
γL|R
1.2, and 0.50
γR|A
0.60. These parameters are often estimated in empirical studies with the slopes of ordinary least-squares lines.

RAINFALL, LANDFORMS, AND STREAMFLOW

6801

16 Number of links, W(x )

14 12 10 8 6 4 2 0 0

2

4 6 8 Distance from outlet, x (km)

10

There are a number of geomorphologically interesting issues, also relevant to parameter estimation, that continue to be the subject of research in this area. One such issue is that of defining the range of values over which power-law behavior holds. The lower end of this range depends on the extent of the channel network, which is usually determined by setting a threshold drainage area, and taking pixels with associated area less than this value to be ‘‘hillslope’’ regions and pixels with area greater than this value to be ‘‘channelized.’’ Also, estimates of parameters will of course generally be better for data over a large range of drainage basin sizes, but another issue is that regional data may involve nested basins for which there will exist a high degree of dependence in morphometric measurements. Figures 4 and 5 illustrate the powerlaw behavior for Goodwin Creek (Fig. 2),

12

Figure 3. Width function for Goodwin Creek, MS, and (smooth curve) the expected width function given magnitude under the random model for channel networks.

although this is a small basin, which increases uncertainty in parameter estimates obtained. Figure 4 shows the (complementary) cumulative frequency distribution of area A for all pixels in the basin (i.e., each pixel may be taken to be the outlet of a subbasin with area A), and Fig. 5 shows the relationship between L and A for channel pixels; the threshold in Fig. 2 for defining the network was taken to be 0.028 sq km. Leastsquares lines are shown (in Fig. 5, an upper cutoff was chosen in addition to the lower cutoff based on the threshold), and estimates of γA and γL|A based on the slopes are 1.44 and 0.54 respectively. The slope of 0.54 in Fig. 5 is a bit lower than typically seen, and in fact if the lower threshold is increased the slope also increases somewhat, indicating that the use of 0.028 sq km is probably overestimating the extent of the channel network in Goodwin Creek.

100000

Number of pixels

10000 Slope = − 0.44

1000 100 10 1 0.001

0.01

0.1 1 Area (km2)

10

100

Figure 4. Area frequency distribution for Goodwin Creek, MS.

6802

RAINFALL, LANDFORMS, AND STREAMFLOW

Mainstream length (km)

100

10

Slope = 0.54

1

0.1 Figure 5. Mainstream length versus area for Goodwin Creek, MS.

0.01

The observed power-law behavior has been seen as evidence that there is an underlying scaling invariance, or self-similarity, in individual river channels, drainage basins and the networks that drain them, which has led to their analysis as self-similar* (or self-affine) fractal* objects. For example, the tendency of γL|R to be greater than unity may be interpreted as a sign of fractal ‘‘wiggliness’’ in river channels. Such scaling ideas constitute the latest approach to explaining many of the observed geometric features of drainage basins. A feature that has received much attention in previous years by geomorphologists is the tendency for γL|A to be consistently greater than 1/2; this is known as Hack’s law [15]. The fact that γR|A also tends to be greater than 1/2 is an indication that larger drainage basins tend to be more elongated than smaller basins. If the variability around the relationships is small, one will have γL|A ≈ γL|R γR|A . Thus, Hack’s law may be explained in terms of the combined effect of channel sinuosity and basin elongation. Other relationships may be argued along similar lines. For example, the exponents γA and γL are related to the Hack exponent γL|A [41] by γA ≈ 2 − γL|A ,

γL ≈ 1/γL|A .

(10)

Another measure of network branching complexity, known as Strahler order [48], has proven to be useful to geomorphologists in analyzing the branching structure of channel networks, and has proven to be important in quantifying scaling behavior. All exterior

0.1

1 Area (km2)

10

100

links in the network are assigned order 1, and the order assigned to an interior link is determined by the orders of the two links immediately upstream. If the upstream links have orders k1 and k2 with k1 = k2 , the downstream link has order equal to the common value plus 1; if k1 = k2 , the order of the downstream link is max(k1 , k2 ). A maximal contiguous path of links of the same order is known as a stream of that order. The Strahler ordering of the network draining Goodwin Creek watershed is shown in Fig. 2. ˜ k denote the number of order k Let N streams in a network. Approximately linear ˜ k on k has been observed dependence of log N for many networks, leading to definition of the bifurcation ratio RB such that ˜ k ≈ RB . ˜ k−1 /N N

(11)

RB is typically computed by exponentiating ˜ k on k. Addithe slope of a regression of log N tionally, each stream has a length obtained by summing the link lengths L , which constitute the stream and an associated area equal to the area A of the most downstream ˜ k be averlink in the stream. Letting L˜ k and A age length and area respectively of streams of ˜k order k, linear dependence of log L˜ k and log A on k has been observed, leading to a length ratio RL and area ratio RA , again obtained by exponentiating the slope of least-squares lines. The three relationships involving RB , RL , and RA have collectively been referred to as Horton’s laws; for natural networks, many studies have shown that RB ≈ 4, RL ≈ 2, and RA ≈ 5.

RAINFALL, LANDFORMS, AND STREAMFLOW

The joint behavior of the number of streams of different orders has also been examined. For example, for k2 < k1 , define ˜ k ,k to be the number of order k2 streams N 1 2 entering order k1 streams as side tributaries in a network. This quantity tends in real networks to depend only on the difference between k1 and k2 and often it varies geometrically with this difference: ˜ k ,k ≈ cRk1 −k2 , N T 1 2

(12)

where RT ≈ 2 [36,57]. Finally, there is a fairly extensive body of work relating various fractal dimensions to Horton’s ratios [13]. For example, one definition of the fractal dimension of a geometric object in R2 is the power-law scaling exponent in the relation between number of boxes of a given size needed to cover the object and size of the boxes [8], so that from Horton’s laws it may then be demonstrated in a straightforward manner that the fractal dimension D of a network is D = log RB / log RL .

(13)

Stochastic Models of Channel Networks The so-called random model for channel networks has been the most widely studied framework for looking at branching properties probabilistically in the earth science community since its introduction [46] in 1967. Denote the sample space of possible network configurations with magnitude M —that is, the set of all binary planted plane trees with M sources—by M . The two assumptions of the random model are 1. All network configurations in M are equally likely (uniform distribution on M ). 2. Link length and area pairs, {Li , Ai }, are mutually independent random vectors, with the vectors for interior links identically distributed and the vectors for exterior links identically distributed. This model seems to preserve many properties of networks observed in nature, at least approximately [1]. Closer examination, however, has revealed consistent deviations,

6803

particularly as better and more detailed data (i.e., digital elevation) on landforms has become available. For example, the bifurcation ratio RB is equal to 4 for the random model in an asymptotic sense to be discussed below, but analysis of a number of large data sets in Reference 36 has produced empirical values consistently greater than 4. Many theoretical results have been obtained under the assumptions of the random model. Consider first the case for which only graph-theoretic properties, determined by assumption (1), are of interest; in this situation, the Li and Ai are in essence taken to be nonrandom and equal to 1, yielding A ≡ 2M − 1, and L is the number of links in the maximal flow path. Let Nm (M) be the number of links of magnitude m in a magnitude M network obeying assumption (1). Then [9,43], 2−(2m−1) Nm (M) → 2M − 1 2m − 1



2m − 1 m



1 , ∼ √ 2 πm3/2 (14)

where convergence on the left is in probability as M → ∞ and the asymptotic expansion in m on the right holds as m → ∞; this gives γA = 3/2, which lies outside the range of empirical values given in the previous section. The Hack exponent γL|A = 1/2, which again tends to be smaller than empirical val˜ k (M) to be ues [9]. Further [65], defining N number of streams of order k in a magnitude M network, ˜ 1 (M) → 1/4 ˜ 2 (M)/N N

(15)

in probability as M → ∞. Thus, the theoretical bifurcation ratio RB is 4 for order 1 and 2 streams. In the more general case for which link lengths and areas are allowed to be random, theoretical results usually involve restrictions on the distributions. For example, if the moment generating function* of the Li exists in a neighborhood of zero, for both interior and exterior links, and mean interior link length is denoted by µ, then [58]  √Mµx ∞

0

0

EWM (y)dy

EWM (y)dy

→ F(x),

(16)

6804

RAINFALL, LANDFORMS, AND STREAMFLOW

where the width function is as defined in (9) but is given subscript M to show the magnitude of the underlying network, and where F is a Rayleigh distribution* with density (x/2) exp(−x2 /4), x  0. Figure 3 shows the asymptotic Rayleigh expected width function for the Goodwin Creek network; the magnitude of the network is M = 206 and the sample estimate for µ is 0.185 km. Assumption (1) of the random model is combinatorial, and it may be asked what branching or growth process would produce trees obeying this assumption. In fact, a critical Galton–Watson* branching process* with p0 = 1/2 and p2 = 1/2, where pk is the probability that a parent gives birth to k offspring, yields tree configurations in M that are equally likely. In this case, Equation (16) may be regarded as a result on the conditional expectation given the total progeny number is 2M − 1. Some results on width function and mainstream length convergence for weighted Galton–Watson trees (where the weights correspond to link lengths) under general assumptions on the offspring distribution and the distribution of weights are discussed in reference 13. The fact that the width function is asymptotically close to the search-depth process associated with the tree is used to show weak convergence to a Brownian excursion process, which in turn leads to a Rayleigh distribution for conditional mean width function shape. Recursive link replacement processes, both deterministic and random, have been widely used in the geomorphology literature as a means of modeling channel network growth. In such a process, the network begins as a single link and evolves in a sequence of time steps, with one or more links being replaced by a more complicated structure at each step. In general, it is necessary in such a scheme to begin by defining two labeled trees, an interior generator and an exterior generator, which are used to replace interior and exterior links in a recursive manner. In the first time step, the initial link, which is by definition an exterior link, is replaced by the exterior generator. The geometric properties of the generated tree as it grows in subsequent time steps then depend on the replacement rules that are adopted.

In an early model of this type [5], for example, the generators both have the simplest possible bifurcating structure (‘‘’’), and only one randomly chosen link, which may be either interior or exterior, is replaced at each time step. If an exterior link is replaced, it is interpreted physically as headward growth, and if an interior link is replaced, it is sideward growth (or tributary development). Three variations of such a model were explored with the intention of drawing conclusions about the actual temporal sequence of branching events taking place in channel network evolution and about the relative geologic importance of headward and sideward growth processes. An example, developed in the spirit of modeling fractal self-similarity, is the recursive replacement tree [13] in which all links are replaced at each time step. Figure 6 shows the generator (interior and exterior generators are identical) and first few replacement steps for the so-called Peano network, one widely studied recursive replacement tree (see ref. 41). In the random self-similar network model [63], the interior and exterior generators are randomized, so that all links are replaced at each time step with independently selected generators. It is known that both the recursive replacement tree and random self-similar models can yield a range of Horton ratios depending on the generators, so there is flexibility in modeling real network geometry. Both the random model and the recursive replacement models have a serious weakness when used as a tool to study drainage basin hydrology*: the spatial embedding of the network is not specified. For example, the first assumption of the random model describes the graph-theoretic structure of the network, without specifying how the network fits into two-dimensional space. The second assumption begins to address this issue by modeling the behavior of link lengths and areas, but the dependence that may be induced among lengths and areas by spatial constraints is ignored in assuming independence. This is problematic, because many basin geometric properties, such as basin shape, channel sinuosity, and network branching angles, can be understood only in the context of spatial embedding. Also, spatial variability of rainfall is known to be a major factor in

RAINFALL, LANDFORMS, AND STREAMFLOW

6805

Generator

t0

t1

t2

t3

streamflow prediction, and spatial embedding of the network is clearly of importance when looking at the effects of this variability. One way to model spatially embedded networks is to define them as subgraph of a lattice; that is, the networks are known as lattice trees. One of the earliest models to take such an approach [22] begins with a finite set of lattice points, all of which are initially taken to be ‘‘unoccupied.’’ The following steps (1) and (2) are repeated sequentially until all sites are occupied: 1. an unoccupied site is selected at random and an unbiased (in the sense that movement probabilities in all directions are equal) self-avoiding random walk* is generated using this site as the origin; 2. the walk terminates when it reaches the edge of the lattice or enters a site that has been entered by a previous walk. The main difficulty with this model is that a walk may become ‘‘trapped,’’ that is, the self-avoiding constraint may result in no movement at all being possible. There are various ways of dealing with traps, and scaling exponents very much depend on how this is done. For example [28], if a trapped walk is simply started again at a new randomly chosen site, simulations yield γL|A ≈ 0.64, γR|A ≈

Figure 6. Steps for generating the Peano recursive replacement tree. Tree ti+1 is obtained by replacing each link in tree ti by the generator.

0.50, and γA ≈ 1.38, but use of ‘‘winding numbers’’ to avoid trapping configurations yields γL|A ≈ 0.86, γR|A ≈ 0.50, and γA ≈ 1.50. An obvious extension of the random walk model is to introduce ‘‘bias’’ by assigning higher probabilities to movement in certain preferred directions (e.g., by assigning a ‘‘downhill’’ direction). In the Scheidegger model, movement from each lattice point is allowed in only two orthogonal directions, with independent Bernoulli trials determining which direction is chosen at each point. Scaling exponents [6] for this model are γL|A = 2/3, γR|A = 2/3, and γA = 4/3. The deviation of γL|A from 1/2 (as seen in Hack’s law) is entirely a basin elongation effect. Finally, we consider a general class of spatial network models known as Gibbsian models. Consider the nearest-neighbor* graph on the two-dimensional integer lattice. Take Vn to be the set of points {(i, j) : 0
i, j
n}, designate the origin (0, 0) as the root, and let the set Sn be the finite set of all (rooted) spanning trees of Vn (see NETWORK ANALYSIS). For the Gibbsian model, the probability of a tree t is Pβ [t] = Z−1 exp[−β0 H(t, β1 )], t ∈ Sn , (17) where Z is the normalizing constant (partition function), β = (β0 , β1 ) is a two-dimensional parameter, and H is given by  ˜ t)β1 ; H(t, β1 ) = (18) A(v, v∈Vn

6806

RAINFALL, LANDFORMS, AND STREAMFLOW

˜ t) is the number of here, for the tree t, A(v, points upstream (with respect to the root) of the point v in Vn , not including the point v ˜ t) is proportional to the drainage itself. A(v, area associated with point v, so H is essentially a spatial summation over grid points in the basin of area raised to power β1 . Maximum likelihood estimation of the parameters and Markov chain Monte Carlo* procedures for simulating networks are discussed in Reference 59. When β0 = 0, the dependence of the probability of a tree on H vanishes and all trees become equally likely (uniform spanning tree); illustrated in Fig. 7 is a randomly generated uniform spanning tree (using the algorithm described in Ref. 3) on a 16 × 16 lattice. At the other extreme is the class of so-called optimal channel networks [41], obtained by letting β0 → ∞, which leads to a uniform distribution on the set of networks that minimize H(t, β1 ). The form of the Gibbsian model is analogous to the canonical distribution in statistical mechanics, where H is the Hamiltonian, so optimal channel networks have been interpreted physically as those that in some sense minimize a measure of energy expenditure. Some results are available for scaling exponents for the Gibbsian model. For the uniform model (β0 = 0), γA = 11/8 and γL|A = 5/8; for optimal channel networks, γA = 3/2 for 12
β1 < 1 and γA = 4/3 for β1 = 1 (see ref. 59 for references and other results).

Figure 7. A random channel network sampled from a uniform distribution on spanning trees of a 16 × 16 grid. Arrows show flow directions, and outlet is at the bottom right corner.

Three-Dimensional Geometry Many studies have been done on statistical properties of the elevation field and its gradient (slope). Surface roughness has been investigated in a number of investigations using kriging and a variety of fractal analysis techniques. Scaling properties of higherorder moments (‘‘generalized variogram* analysis’’) of increments have been used to examine the possibility of multiscaling, as opposed to simple-scaling, structure [41]. Other studies have focused on elevation in relation to the superimposed drainage system. Most prominent among these are studies on longitudinal profiles of individual stream courses and on the relationship of slope at a point on a stream to properties of the basin upstream from that point. For example, one study [50] identifies power-law dependence of the first two moments of slope on drainage area for areas larger than a threshold value, but multiscaling rather than simple scaling is suggested by the comparative value of the exponents. In a related study [11], a link concentration function is defined exactly as the network width function in Equation 9, except that elevation above the outlet is used in place of distance from the outlet; it is shown that an i.i.d. assumption for link elevation drops, analogous to the corresponding assumption on link lengths in the random model, is untenable. A number of models have been developed with the intention of providing a physical

RAINFALL, LANDFORMS, AND STREAMFLOW

framework, in terms of the geologic processes at work, for explaining the observed properties of landscapes. Although it is clearly impossible to capture all the details of the processes, many of these models are tied to some physically based description (a differential or difference equation) of how the land surface evolves through time via erosion [68]. The physical model may incorporate a random noise term added to account for smallscale fluctuations, and randomness may also come about through the initial conditions or random perturbations employed in the solution algorithm. STREAMFLOW Knowledge of the statistical properties of streamflow is essential for many aspects of water resources planning and management, including design of reservoirs and other engineering structures, evaluation of alternative reservoir operation policies and reliability, and assessment of flood and drought risk. The problem that plagues many studies is inadequate sample size; that is, streamflow records in time are often short, and flow measurement stations are often sparsely located in space. This is especially critical, for example, when making inferences about extreme values, and it has spurred the need for various modeling strategies that can be used

6807

for prediction, forecasting, and simulation of flow sequences, which can augment historical sequences. Modeling has for the most part been concerned either with flow volumes at discrete time scales, such as daily, monthly, or annual, or extremes such as maximum or minimum discharge (instantaneous flux, in volume per unit time) within an annual time period. Figure 8 shows average daily discharge for the Goodwin Creek basin (Fig. 2) over the same 1-yr. period (Oct. 1, 2001 until Sept. 30, 2002) as the rainfall plotted in Fig. 1. We shall first consider time series modeling of flows at a point, then spatial flow properties, and finally we shall examine how relationships among flows, landforms, and rainfall can be exploited to improve knowledge of statistical streamflow characteristics. Temporal Streamflow Among the first applications of stochastic modeling of flow volumes were reference 2, where annual flow volumes were modeled as an uncorrelated Gaussian process*, and reference 54, where serial correlation* was incorporated into simulated monthly flows via a first-order AR model. ARMA and periodic ARMA models have been widely used since this time to model flows. During the mid-1960s, however, the possibility of longrange dependence* in annual flow series also

100

10

1

Time

30-Sep-02

1-Aug-02

1-Jun-02

1-Apr-02

30-Jan-02

30-Nov-01

0.1 1-Oct-01

Discharge (m3/s)

1000

Figure 8. Daily streamflow time series for Goodwin Creek, MS, for a 1-yr. period.

6808

RAINFALL, LANDFORMS, AND STREAMFLOW

came to the forefront, stimulated by unexpected results in a study of design capacity of reservoirs by Hurst [17] (see HURST COEFFICIENT). Hurst analyzed a number of geophysical time series, including annual flows for the Nile River, and found that the range of cumulative departures from the mean value over a period of n years tends to increase at a rate nH , where the exponent H has a value greater than 1/2. (H = 1/2 for short-memory processes.) This tendency for H to be larger than 1/2 subsequently became known as the Hurst phenomenon. It was during this time that fractional Brownian motion* [25] and other models (e.g., brokenline models) were proposed as more appropriate for preserving the long-range dependence that seemed to be present in streamflow time series. A major concern in modeling streamflow time series at a given scale is whether such models preserve properties, for example, dependence structure, at scales other than that for which fitting is performed. One set of procedures used to address this issue explicitly is the set of so-called streamflow disaggregation methods. In the simplest of these schemes, for example, monthly flows are represented via a linear model in annual flows; a constraint that the sum of monthly flows must equal the annual flow is incorporated and parameters are estimated so as to preserve second-moment properties of the disaggregated flows. One difficulty in these methods is that the number of parameters may be large relative to the amount of data available for estimation. More recent approaches to modeling temporal streamflow have, as in rainfall and river network modeling, addressed the scale problem by explicitly assuming different forms of scale invariance; such an approach automatically incorporates preservation of flow properties under aggregation*. For example, in Reference 34, spectral and multifractal analysis of daily streamflow from a number of rivers is performed to confirm the existence of definite scaling regimes; heavytailed extremes with long-range dependence are predicted by this framework, in contrast with the assumption of independent annual maxima typically made in flood studies. Much of the work on analysis of streamflow extremes at a single site has focused

on estimation of the distribution of the maximum discharge during a year, or annual maximum. The so-called T-year flood is defined to be the 1 − 1/T quantile of this distribution; in engineering applications, it is often of interest to obtain reliable estimates of the 50-yr or 100-yr flood. Usually, the assumption has been made that observed annual maxima constitute an i.i.d. random sample from this underlying probability distribution. A complete review of the various probability distributions and estimation procedures used in flood-frequency analysis at a single site may be found in reference 37. Studies of climate change and the response of hydrologic extremes to large-scale climatic forcing, however, have questioned the i.i.d. assumption [18]. The major problem in specifying the form of the underlying distribution, in estimating parameters, and in testing assumptions such as stationarity is that most hydrologic streamflow records are relatively short. This problem is usually addressed by attempting to pool flow data from different sites, as discussed in the next section. Spatial Streamflow and Regionalization Because samples of annual flow maxima at a single site are often small, much effort has been devoted to finding ways to combine information from different sites within a region to improve estimates of flood quantiles. This approach has come to be known in hydrologic applications as regional frequency analysis, or simply regionalization. When data from different sites are combined, the problem is complicated by the possibility of unequal sample sizes, nonconcurrent records, and the existence of spatial correlation. One of the earliest approaches to regionalization, which is still widely used, is known as the index-flood procedure. Let Qij , 1
j
ni , 1
i
N, be a sample of annual maxima at N sites in a region. For the index-flood method, it is assumed that there exist N scale parameters µi , 1
i
N, such that the Qij /µi are identically distributed, and the common distribution is known as the regional frequency distribution. Statistical procedures are used to identify homogeneous regions within which such an assumption is reasonable. Often, parameter µi is taken to be the

RAINFALL, LANDFORMS, AND STREAMFLOW

mean EQij and is estimated by the sample mean of the data at site i, and parameters of the regional distribution are commonly estimated by a weighted mean (with weights proportional to sample size) of at-site estimates. The method of L-moments, based on linear combinations of order statistics (see LSTATISTICS), has been found to be an efficient and computationally fast estimation procedure and is widely used for flood-frequency problems [16]. Regional quantile regression is another common approach to regionalization. In this method, data from gaged sites are used to develop regression relations of the form log[qi (p)] = β0 +

K 

βk Zki + ui ,

(19)

k=1

where qi (p) is the pth percentile of the annual maximum distribution at site i, the Zki , 1
k
K, are K (possibly transformed) observable characteristics at site i, and ui is known as ‘‘model error’’ for the ith site; the ui are usually taken to be independent with E(ui ) = 0, Var(ui ) = σ 2 , but some sort of spatial correlation structure may also be assumed. The logarithm of drainage area is a variable that usually shows up as the most significant regressor in regional regressions. The quantity qi (p) is not known and must be estimated using flow data from site i. Denote such an estimate by qˆ i (p), and denote by
the covariance matrix of the sampling errors log[qˆ i (p)] − log[qi (p)];  will of course depend on the sample sizes ni . The error in regressing quantile estimates against basin characteristics then has covariance matrix σ 2 I +
if the model errors are independent of each other and of the sampling error; generalized least squares has been used for parameter estimation with this model [51]. Regional regression relations are important not only as a means of combining information from a number of sites to improve flood-frequency estimates, but also because they may be used for prediction of flow characteristics at ungaged sites, that is, points on a river system where no direct streamflow measurements are available. Studies of streamflow have focused on understanding streamflow variability and

6809

spatial correlation at continental and global scales, in contrast to smaller scales at which regional flood-frequency studies are usually performed. It is of interest to identify large geographical zones that exhibit homogeneity in streamflow response and subsequently to relate these zones to climatic forcing and atmospheric circulation patterns. Rotated principal components analysis* on 48 years of streamflow data for 559 stations across the United States was applied in reference 23 to identify spatial patterns and dominant regimes. A similar analysis was applied to wavelet spectra in Reference 43 to account for disparate temporal fluctuation scales. Relationship with Rainfall and Landforms Streamflow records may be short or nonexistent at a point for which statistical properties of flow are of interest. In this situation, a logical approach is to attempt to utilize knowledge about the processes generating flow. In fact, much is known about the physical processes whereby rainfall falling over a drainage basin is converted to streamflow, with landforms acting as one subset of a number of intermediary agents. The conversion is quite complex, however, and many of the important processes are controlled by variables such as soil properties, vegetation cover, channel geometry, and precipitation itself, which exhibit a high degree of spatial and temporal variability. There is therefore ample opportunity for statistical and probabilistic methods to be employed. Figure 9 shows 5-min concurrent rainfall and streamflow data for a rainy 3-d period (Dec. 12 through Dec. 14, 2001) for Goodwin Creek. Streamflow volumes have been scaled by drainage area to give them the same units as rainfall intensity. Note the time lag between the two series, the smoothness of the streamflow time series compared to the rainfall, and the fact that losses due to infiltration into the soil are greater during the first part of the storm. Approaches to looking at the processes determining streamflow range from the very complex to the very simple. Complex approaches are exemplified by the rainfallrunoff model, which is typically a collection of computer algorithms attempting to

6810

RAINFALL, LANDFORMS, AND STREAMFLOW

Rainfall or streamflow (mm/h)

25

Figure 9. Rainfall and streamflow for a 3-d storm, Goodwin Creek, MS (tic marks on time axis are for 12:01 AM).

20

15 Rainfall Streamflow

10

5

0 12-Dec-01

track the movement of water through different components of a watershed system. These may incorporate numerical solutions to physically based dynamical flow equations [47]. Input to such models typically includes rainfall (although often data from only a single rain gage is available) and landform properties, such as the branching structure of the channel network, that are derivable from digital elevation data [64]. These models are used in many contexts, including simulation of streamflow, using actual and randomly generated rainfall, and operational forecasting (flood warnings, operation of waterways, releases from reservoirs, etc.). One important application of simulations is prediction of the magnitude of extreme flood events. Typical approaches are design flood estimation, in which rainfall for socalled individual design storms is used as input to a model, and continuous simulation, in which long-term rainfall is used as input and statistical analysis is then performed on the resulting streamflow output. Although rainfall-runoff models are considered to be deterministic, they have been analyzed in a (usually Bayesian) nonlinear regression* framework for purposes of parameter estimation and quantification of the various sources of uncertainty [53]. One simplification that has been widely used in practice and that has led to many interesting theoretical results is the assumption that streamflow at the outlet

13-Dec-01 Time

14-Dec-01

15-Dec-01

of a drainage basin may be expressed as a linear convolution of temporal rainfall and an instantaneous unit hydrograph (IUH); the IUH thus represents response of the basin to rainfall applied instantaneously. If such instantaneous rainfall is deposited immediately into the channel system and then flows at a uniform velocity to the outlet, the shape of the IUH will coincide with the width function of the channel network. This illustrates one approach that has been exploited for studying the relationship between streamflow and landforms. In fact, through the 1980s, an entire body of research grew around the idea of relating the IUH to network branching properties, including the width function, Horton ratios, and other morphological features, under various simplifying assumptions on flow, resulting in the so-called geomorphological IUH [41]. In the original formulation, the tributary num˜ k ,k were used to determine transition bers N 1 2 probabilities in a Markov chain model of movement of individual raindrops through the channel system [42], an approach that was given more general expression in Reference 14. It is sometimes possible to derive analytically, or at least to express in relatively simple form for numerical solution, the probability distribution of various streamflow random variables given assumptions on rainfall and on rainfall-runoff mechanisms, provided the assumptions are simple enough to render the

RAINFALL, LANDFORMS, AND STREAMFLOW

problem tractable. In an early example of this [7], the distribution of the annual maximum flow is obtained under assumptions that duration and intensity of rainfall for individual storms are independent and exponentially distributed, a fixed number of storm events occur each year, and flow is governed by a physically based kinematic wave model. In reference 10, similar assumptions on rainfall are made, but the geomorphological IUH is used rather than a kinematic wave flow model. Of late, dependence on scale has been seen as a unifying theme in relating streamflows, landforms, and rainfall, and assumptions of scale invariance have led to simplified analyses. For example, in References 10 and 60, the power-law dependence of peak storm flow on drainage area is investigated, given a recursive replacement tree network model and random cascade spatial rainfall. Such investigations shed light on regional regression approaches to flood frequency, with the possibility that variations in the coefficient of area in regression models such as (19) may be interpreted physically. Similarly, in reference 52 it is reported on the basis of multifractal analysis over a wide range of temporal scales that low-frequency covariability of rainfall and streamflow may be modeled by a linear transfer function model*. Such approaches are relevant to the scientific interest in understanding hydrologic processes and land-surface/atmospheric interactions at the continental and global scale, particularly in light of global climate change investigations. To address such questions, atmospheric general circulation models (GCMs), which model climate on a global scale, have come into widespread use. GCMs produce grid-based output of meteorologic variables with a grid spacing typically much greater than areas of drainage basins of interest in streamflow studies. Thus, an important issue that has received much attention by hydrologists has been how to couple GCMs and the smaller scale hydrological models. APPENDIX We present here a few definitions related to scaling properties of stochastic processes;

6811

see reference 35 for more elaboration. Let {X(s); s ∈ S} be a real-valued stochastic process for which the index set S obeys λs ∈ S for every s ∈ S and 0 < λ
1. The process {X(s)} is said to be simple-scaling (or self-similar) if for each 0 < λ
1, there exists a positive constant Cλ such that the finite-dimensional distributions of {X(λs)} and {Cλ X(s)} coincide (see SELF-SIMILAR PROCESSES). It may be shown that Cλ must be of the form Cλ = λθ for some real-valued constant θ known as the scaling exponent. This class of processes includes fractional Brownian motion and certain Levy stable processes. A multiscaling process {X(s); s ∈ S} is one for which again the finite-dimensional distributions of {X(λs)} and {Cλ X(s)} coincide but for which Cλ is allowed to be random. One-dimensional distributions are seen to obey the multiplicative relation d

X(λ1 λ2 . . . λn s) = Cλ1 Cλ2 . . . Cλn X(s).

(20)

Under the additional assumptions that {X(s)} is positive and that moments of order h  1 exist, {X(s)} is said to be wide-sense multiscaling if for every 0 < λ
1, there is a nonlinear convex function a(h), not depending on λ, such that log(EX(λs)h ) = −a(h) log λ + log(EX(s)h ). (21) Thus, moments are log-linear in scale λ. Processes that are simple-scaling obey Equation 21 with a(h) = −hθ , which is linear in h, so that nonlinearity in h is an indicator of multiscaling behavior. Discrete analogs of these definitions may also be given. One such model which has been widely studied and which illustrates the multiplicative structure in Equation 20 is the Mandelbrot cascade [19]. For this model, S = [0, 1] and X(s) represents amount of mass in interval [0, s); the process may be generated at a sequence of discrete scales (λ = 1, b−1 , b−2 , . . .), where b > 1 is an integer known as the branching number) using i.i.d. generator random variables. , define the For γi∈{0,1,...,b−1}

subinterval (γ1 , γ2 , . . . , γn = [ ni=1 γi b−i , ni=1 γi b−i + b−n ] and let Z be total mass in [0, 1]. The mass in a given subinterval (γ1 , γ2 , . . . , γn ) is

6812

RAINFALL, LANDFORMS, AND STREAMFLOW

equal in distribution to b−n W(γ1 )W(γ1 , γ2 ) · · · W(γ1 , γ2 , . . . , γn ) Zn (γ1 , γ2 , . . . , γn ), where the W’s (generators) are i.i.d. non-negative, mean 1 random variables independent of Zn (γ1 , γ2 , . . . , γn ), and Zn (γ1 , γ2 , . . . , γn ) is distributed as Z. This distributional equality also holds jointly for all subintervals, with the Zn ’s i.i.d. Let Mb−n (h) :=

n −1 b

[X((j + 1)b−n ) − X(jb−n )]h .

j=0

(22) It may be shown that under certain conditions [31], the limit τ (h) := lim

n→∞

log Mb−n (h) log bn

(23)

exists with probability one and is equal to logb EW h − (h − 1). Also, Equation (21) holds for λ = 1, b−1 , b−2 , . . . with a(h) = logb EW h − h. REFERENCES

10. Gupta, V. K., Castro, S. L., and Over, T. M. (1996). Scaling exponents of spatial peak flows from rainfall and river network geometry. J. Hydrol., 187, 81–104. 11. Gupta, V. K. and Waymire, E. C. (1989). Statistical self-similarity in river networks parameterized by elevation. Water Resour. Res., 25, 463–476. 12. Gupta, V. K. and Waymire, E. C. (1990). Multiscaling properties of spatial rainfall and river flow distributions. J. Geophys. Res., 95, 1999–2009. 13. Gupta, V. K. and Waymire, E. C. (1998). ‘‘Some Mathematical Aspects of Rainfall, Landforms, and Floods’’. In Stochastic Methods in Hydrology: Rainfall, Landforms, and Floods, O. E. Barndorff-Nielsen, V. K. Gupta, V Perez-Abreu, and E. Waymire, eds. World Scientific, Singapore, pp. 129–172. 14. Gupta, V. K., Waymire, E. C., and Wang, C. T. (1980). A representation of an instantaneous unit hydrograph from geomorphology. Water Resour. Res., 16, 855–862. 15. Hack, J. T. (1957). Studies of Longitudinal Stream Profiles in Virginia and Maryland. U.S. Geol. Survey Prof. Paper 294-B. 16. Hosking, J. R. M. and Wallis, J. R. (1997). Regional Frequency Analysis. Cambridge University Press, Cambridge, U.K.

1. Abrahams, A. D. (1984). Channel networks: a geomorphological perspective. Water Resour. Res., 20, 161–168.

17. Hurst, H. E. (1951). Long-term storage capacity of reservoirs. Trans. Am. Soc. Civil Engrs., 116, 770–799.

2. Barnes, F. B. (1954). Storage required for a city water supply. J. Inst. Eng., Austral., 26, 198–203.

18. Jain, S. and Lall, U. (2001). Floods in a changing climate: Does the past represent the future? Water Resour. Res., 37, 3193–3205.

3. Broder, A. Z. (1989). ‘‘Generating Random Spanning Trees’’. Proceedings of the 30th Annual IEEE Symposium on Foundations of Computer Science. IEEE, New York, 442–447.

19. Kahane, J. P. and Peyri`ere, J. (1976). Sur certaines martingales de Benoit Mandelbrot. Adv. Math., 22, 131–145.

4. Cox, D. R. and Isham, V. (1988). A simple spatial-temporal model of rainfall. Proc. R. Soc. London, A, 415, 317–328. 5. Dacey, M. F. and Krumbein, W. C. (1976). Three models for stream network growth. J. Geol., 84, 153–163. 6. Dodds, P. S. and Rothman, D. H. (2000). Geometry of river networks. I. Scaling, fluctuations, and deviations. Phys. Rev. E, 63, 016115-1-13.

20. Kedem, B. and Chiu, L. S. (1987). Are rain rate processes self-similar? Water Resour. Res., 23, 1816–1818. 21. Le Cam, L. (1961). ‘‘A Stochastic Description of Precipitation’’. In Proc. Fourth Berkeley Symp. Math. Statist. Probab., Vol. 3, J. Neyman, ed. University of California Press, Berkeley, Calif., pp. 165–186. 22. Leopold, L. B. and Langbein, W. B. (1962). The Concept of Entropy in Landscape Evolution. U.S. Geol. Survey Prof. Paper 500-A.

7. Eagleson, P. S. (1972). Dynamics of flood frequency. Water Resour. Res., 8, 878–898. 8. Feder, J. (1988). Fractals. Plenum, New York.

23. Lins, H. F. (1997). Regional streamflow regimes and hydroclimatology of the United States. Water Resour. Res., 33, 1655–1667.

9. Flajolet, P. and Odlyzko, A. (1982). The average height of binary trees and other simple trees. J. Comp. Syst. Sci., 25, 171–213.

24. Lovejoy, S. and Mandelbrot, B. B. (1985). Fractal properties of rain and a fractal model. Tellus, 37A, 209–232.

RAINFALL, LANDFORMS, AND STREAMFLOW 25. Mandelbrot, B. B. and Van Ness, J. W. (1968). Fractional Brownian motions fractional noises and applications. SIAM Rev., 10, 422–437. 26. Marsan, D. and Schertzer, D. (1996). Causal space-time multifractal processes: predictability and forecasting of rain fields. J. Geophys. Res., 101(D21), 26333–26346. 27. McCollum, J. R. and Krajewski, W. F. (1998). Uncertainty of monthly rainfall estimates from rain gauges in the global precipitation climatology project. Water Resour. Res., 34, 2647–2654. 28. Meakin, P., Feder, J., and Jossang, T. (1991). Simple statistical models for river network. Physica A, 176, 409–429. 29. Northrop, P. (1997). A clustered spatialtemporal model of rainfall. Proc. R. Soc. London, A, 454, 1875–1888. 30. Olsson, J. and Niemczynowicz, J. (1996). Multifractal analysis of daily spatial rainfall distributions. J. Hydrol., 187, 29–43. 31. Ossiander, M. and Waymire, E. (2000). Statistical estimation for multiplicative cascades. Ann. Stat., 28, 1533–1560. 32. Over, T. M. and Gupta, V. K. (1994). Statistical analysis of mesoscale rainfall: dependence of a random cascade generator on large-scale forcing. J. Appl. Meteorol., 33, 1526–1542. 33. Over, T. M. and Gupta, V. K. (1996). A spacetime theory of mesoscale rainfall using random cascades. J. Geophys. Res., 101(D21), 26319–26331.

6813

further developments. Proc. R. Soc. London, A, 417, 283–298. 40. Rodriguez-Iturbe, I., Marani, M., D’Odorico, P., and Rinaldo, A. (1998). On space-time scaling of cumulated rainfall fields. Water Resour. Res., 34, 3461–3469. 41. Rodriguez-Iturbe, I. and Rinaldo, A. (1997). Fractal River Basins: Chance and SelfOrganization. Cambridge University Press, Cambridge, U.K. 42. Rodriguez-Iturbe, I. and Valdes, J. B. (1979). The geomorphologic structure of hydrologic response. Water Resour. Res., 15, 1409–1420. 43. Saco, P. and Kumar, P. (2000). Coherent modes in multiscale variability of streamflow over the United States. Water Resour. Res., 36, 1049–1067. 44. Salas, J. D. (1993) ‘‘Analysis and Modeling of Hydrologic Time Series’’. In Handbook of Hydrology, D. Maidment, ed. McGraw-Hill, New York, Chapter 19, pp. 19.1–19.72. 45. Seo, D.-J., Krajewski, W. F., and Bowles, D. S. (1990). Stochastic interpolation of rainfall data from rain gages and radar using cokriging, 1. Design of experiments. Water Resour. Res., 26, 469–477. 46. Shreve, R. L. (1967). Infinite topologically random channel networks. J. Geol., 75, 178–186. 47. Singh, V. P. and Woolhiser, D. A. (2002). Mathematical modeling of watershed hydrology. J. Hydrol. Eng., 7, 270–292.

34. Pandey, G., Lovejoy, S., and Schertzer, D. (1998). Multifractal analysis of daily river flows including extremes for basins of five to two million square kilometres, one day to 75 years. J. Hydrol., 208, 62–81.

48. Strahler, A. N. (1964). ‘‘Quantitative Geomorphology of Drainage Basins and Channel Networks’’. In Handbook of Hydrology, V. T. Chow, ed. McGraw-Hill, New York, pp. 39–76.

35. Pavlopoulos, H. and Makatis, G. (1998). Spectral multiscaling of spatially averaged rain rate: hints for spatio-temporal modeling. Environmetrics, 9, 689–713.

49. Svensson, C., Olsson, J., and Berndtsson, R. (1996). Multifractal properties of daily rainfall in two different climates. Water Resour. Res., 32, 2463–2472.

36. Peckham, S. (1995). Self-Similarity in the Three-Dimensional Geometry and Dynamics of Large River Basins. Ph.D. Dissertation, University of Colorado, Boulder, Colo.

50. Tarboton, D. G., Bras, R. L., and RodriguezIturbe, I. (1989). Scaling and elevation in river networks. Water Resour. Res., 25, 2037–2051.

37. Rao, A. R. and Hamed, K. H. (2000). Flood Frequency Analysis. CRC Press, Boca Raton, Fla.

51. Tasker, G. D. and Stedinger, J. R. (1989). An operational GLS model for hydrologic regression. J. Hydrol., 111, 361–375.

38. Rodriguez-Iturbe, I., Cox, D. R., and Isham, V. (1987). Some models for rainfall based on stochastic point processes. Proc. R. Soc. London, A, 410, 269–288.

52. Tessier, Y., Lovejoy, S., Hubert, P., Schertzer, D., and Pecknold, S. (1996). Multifractal analysis and modeling of rainfall and river flows and scaling, causal transfer functions. J. Geophys. Res., 101(D21), 26427–26440.

39. Rodriguez-Iturbe, I., Cox, D. R., and Isham, V. (1988). A point process model for rainfall:

53. Thiemann, M., Trosset, M., Gupta, H., and Sorooshian, S. (2001). Bayesian recursive

6814

RAIN-RATE DISTRIBUTIONS

parameter estimation for hydrologic models. Water Resour. Res., 37, 2521–2535.

April 19-23, 1998, Chapter 7, pp. 73–80, http://wwwbrr.cr.usgs.gov/weasel/doc/ffiamc/.

54. Thomas, H. A. and Fiering, M. B. (1962). ‘‘Mathematical Synthesis of Streamflow Sequences for the Analysis of River Basins by Simulations’’. In Design of Water Resource Systems, A. Mass, ed. Harvard University Press, Cambridge, pp. 459–493.

65. Wang, S. X. and Waymire, E. C. (1991). A large deviation rate and central limit theorem for Horton ratios. SIAM J. Disc. Math., 4, 575–588.

55. Thyer, M. and Kuczera, G. (2000). Modeling long-term persistence in hydroclimatic time series using a hidden state Markov model. Water Resour. Res., 36, 3301–3310. 56. Todorovic, P. and Woolhiser, D. A. (1976). ‘‘Stochastic Structure of the Local Pattern of Precipitation’’. In Stochastic Approaches to Water Resources, Vol. 2, H. W. Shen, ed. Colorado State University, Fort Collins, Colo, Chapter 15, pp. 15.1–15.37. 57. Tokunaga, E. (1978). Consideration on the Composition of Drainage Networks and their Evolution. Geog. Rep. 13, Tokyo Metrop. Univ., Tokyo, Japan. 58. Troutman, B. M. and Karlinger, M. R. (1984). On the expected width function for topologically random channel networks. J. Appl. Probab., 21, 836–849. 59. Troutman, B. M. and Karlinger, M. R. (1998). ‘‘Spatial Channel Network Models in Hydrology’’. In Stochastic Methods in Hydrology: Rainfall, Landforms, and Floods, O. E. Barndorff-Nielsen, V. K. Gupta, V. Perez-Abreu, and E. Waymire, eds. World Scientific, Singapore, pp. 85–128. 60. Troutman, B. M. and Over, T. M. (2001). River flow mass exponents with fractal channel networks and rainfall. Adv. Water Resour., 24, 967–989. 61. Troutman, B. M. and Vecchia, A. V. (1999). Estimation of Renyi exponents in random cascades. Bernoulli, 5(2), 191–207. 62. Vecchia, A. V. and Ballerini, R. (1991). Testing for periodic autocorrelations in seasonal time series data. Biometrika, 78, 53–64. 63. Veitzer, S. A. and Gupta, V. K. (2000). Random self-similar river networks and derivations of generalized Horton laws in terms of statistical simple scaling. Water Resour. Res., 36, 1033–1048. 64. Viger, R. J., Markstrom, S. L., and Leavesley, G. H. (1998). The GIS Weasel - An interface for the treatment of spatial information used in watershed modeling and water resource management, Proceedings of the 1st Federal Interagency Hydrologic Modeling Conference, Vol. II, Las Vegas, Nevada,

66. Waymire, E. and Gupta, V. K. (1981). The mathematical structure of rainfall representations 3. Some applications of the point process theory to rainfall processes. Water Resour. Res., 17, 1287–1294. 67. Wheater, H. S., Isham, V. S., Cox, D. R., Chandler, R. E., Kakou, A., Northrop, P. J., Oh, L., Onof, C., and Rodriguez-Iturbe, I. (2000). Spatial-temporal rainfall fields: modelling and statistical aspects. Hydrol. Earth Syst. Sci., 4, 581–601. 68. Willgoose, G., Bras, R. L., and RodriguezIturbe, I. (1991). A coupled channel network growth and hillslope evolution model 1. Theory. Water Resour. Res., 27, 1671–1684. 69. Zucchini, W. and Guttorp, P. (1991). A hidden Markov model for space-time precipitation. Water Resour. Res., 27, 1917–1923.

See also FRACTIONAL BROWNIAN MOTIONS AND FRACTIONAL GAUSSIAN NOISES; HURST COEFFICIENT; KRIGING; LONG-RANGE DEPENDENCE; RAIN-RATE DISTRIBUTIONS; and SELF-SIMILAR PROCESSES.

BRENT M. TROUTMAN

RAIN-RATE DISTRIBUTIONS This concept was introduced in References 4 and 5 and refined in References 2 and 7; see also References 8 and 9. Given a collection of nonnegative random variables {Xt ; t[0, ∞)} forming a stochastic process where t denotes time. For each t, Xt is interpreted as either instantaneous rain rate at a fixed geographic location or instantaneous area-average rain rate over a fixed region. It is assumed that Xt follows a mixed-type distribution. More specifically, the probability that Xt takes on value 0 (indicated absence of rain) is 1 − pt (0 < pt < 1). For given Xt > 0 (conditional on the rain), the distribution of Xt is continuous, represented by a continuous cdf Ft (x, θ t ) with a vector parameter θt . Thus, the unconditional

RAIN-RATE DISTRIBUTIONS

distribution Gt (x) of Xt is

A distance function approach [4] defines

Gt (x) = P(Xt
x) = (1 − pt )H(x) + pt Ft (x, θ t );

d(τ ) = 13 [|β(τ )L − β(τ )G | + |β(τ )L − β(τ )I | (1)

H(x) = 1, x  0, =0

otherwise.

Short et al. [8] observed the linear relationship E(Xtk ) = βk (k, τ , θt )P(X > τ ),

(2)

where βt (k, τ , θt ) =

E(Xtk |Xt > 0| . P[Xt > τ |Xt > 0)

6815

(3)

Note that the slope (3) is independent of the probability of rain, and Equation 2 indicates the linear relationship between the kth moment of the rain rate and the probability that the rain rate exceeds a given threshold level τ ( 0). Kedem and others [3,4,5] have presented this relation for k = 1, providing a probabilistic explanation of the empirical linear relationship originally discovered by Chiu [1]. Assume that there exists on [0, ∞) the weak limit of Gt as t → ∞, denoted by G. In this case [6,7], the slope can be written as β(k, τ , θ ) independently of t. But [2] this assumption is violated when the rain system is dominated by a frontal rain. Various distributions were proposed for the continuous component Ft (x, θ). These include [8] lognormal, gamma, and inverseGaussian distributions, in addition [7] to two-parameter Weibull distributions and a mixture of lognormal distributions as the continuous part. A substantial amount of research in the references cited is devoted to determination of theoretical optimal thresholds—namely, determination of a maximum likelihood estimator of β(k, τ , θ ). This estimator has an asymptotic normal distribution with mean β(k, τ , θ ) and variance v(k, τ , θ )/n; a theoretical optimal threshold is that which minimizes the normalized asymptotic variance σ (k, τ , θ)/[β(k, τ , θ )]2 . The asymptotic variance v(k, τ , θ ) is calculated using the Fisher information* matrix [2].

+ |β(τ )G − β(τ )I |] for τ  0, where β(τ )L , β(τ )G , and β(τ )I respectively denote the slopes for the first moment (k = 1) in the case of lognormal, gamma, and inverse Gaussian distributions as the continuous part. It is supposed that these three distributions are candidate distributions of rain rate, conditional on rain. These authors [4] choose an optimal threshold τ > 0 that minimizes d(τ ). The basic source of data is the Global Atmospheric Research Program (GARP), Atlantic Tropical Experiment (GATE), and Phase I data set (see ref. 1). There seems to be no definitive answer as to which one of the continuous distributions is most appropriate, but the basic conclusions are that special care is needed for fitting theoretical distributions to histograms* and that the optimal thresholds are fairly stable with respect to distributional changes when k = 1 is used and are quite unstable when k = 2. REFERENCES 1. Chiu, L. S. (1988). Rain estimates from satellites: a real rainfall-rain area relations. 3rd Conference on Satellite Meteorology and Oceanography. Anaheim, Calif., Amer. Meteor. Soc., pp. 363–368. 2. Kayano, K. and Shimizu, K. (1994). Optimal threshold for a mixture of lognormal distributions as a continuous part of the mixed distribution. J. Appl. Meteorol., 33, 1543– 1550. 3. Kedem, B., Chiu, L. S., and Karni, Z. (1990). An analysis of the threshold method for measuring area-average rainfall. J. Appl. Meteorol., 29, 3–20. 4. Kedem, B., Chiu, L. S., and North, G. R. (1990). Estimation of mean rain rate: application to satellite observations. J. Geophys. Res., 95, 1965–1972. 5. Kedem, B. and Pavlopoulos, H. (1991). On the threshold method for rainfall estimation: choosing the optimal threshold level. J. Am. Stat. Assoc., 86, 626–633. 6. Pavlopoulos, H. and Kedem, B. (1992). Stochastic modelling of rain rate processes:

6816

RAKED (ADJUSTED) KAPPA

a diffusion model. Commun. Stat.-Stochastic Models, 8, 397–420. 7. Shimizu, K. and Kayano, K. (1994). A further study of the estimation of area rain-rate moments by the threshold method. J. Meteorol. Soc. Jpn., 72, 833–839. 8. Short, D. A., Shimizu, K., and Kedem, B. (1993). Optimal thresholds for the estimation of area rain-rate moments by the threshold method. J. Appl. Meteorol., 32, 182–192. 9. Short, D. A., Wolff, D. B., Rosenfeld, D., and Atlas, D. (1993). A study of the threshold method utilizing rain gage data. J. Appl. Meteorol., 32, 1379–1387. See also METEOROLOGY, STATISTICS IN and RAINFALL, LANDFORMS, AND STREAMFLOW.

RAKED (ADJUSTED) KAPPA The kappa coefficient introduced by Cohen [2] is one of the most widely used measures of association∗ for describing agreement between one or more pairs of observers (see AGREEMENT, MEASURES OF). Its definition implies that the value of the statistic depends on the marginal distributions of the corresponding contingency tables∗ . In fact, its construction includes the sum of the product of the marginal values (see KAPPA COEFFICIENT in Encyclopedia of Statistical Sciences Vol. 6, p. 3816). Hence, for instance, two values of kappa can be computed from two tables having the same intrinsic agreement but different prevalences of cases of the various types. The 2 × 2 tables in Table 1 (with hypothetical data), illustrate that in both tables there can be the same agreement (160) and disagreement (40), but different prevalences and kappa coefficients. Specifically in the left table the kappa coefficient is 0.6, while in the right table its

Table 1.

Yes No Tot.

Yes

(a) No

Tot.

80 20 100

20 80 100

100 100 200

Yes No Tot.

(b) Yes No

Tot.

140 20 160

160 40 200

20 20 40

value is 0.375. Hence in certain applications where comparisons are made between two (or among several) studies having substantially different marginal distributions, it may be misleading to compare the kappa coefficients. For such a situation a new version of this statistic, called raked kappa, has been introduced by Agresti et al. [1]. The raked kappa is computed for a sample table which is adjusted to prespecified marginal distributions. The adjustment process—applied to a sample table—standardizes the table in order to achieve targeted marginal values without altering its original association structure, in terms of the odds radios∗ pii pjj / (pij pji ), 1
i
m and 1
j
m (see Example below). Let {pij } denote the proportions of the N items classified by two observers A and B into categories i and j, respectively, of two identical categorical classifications with m levels. After defining specific marginal values, the raked
table, having new
proportions {rij }with { j rij = ri .} and { i rij = r.j }, can be obtained from the original sample ∗ table by the iterative proportional fitting procedure and rescaling the cell entries while still satisfying the row and the column margins [4]. In the raked table the {rij } values preserve the same odds ratios as the {pij } for all 2 × 2 subtables formed by using pairs of rows and columns; in other words, the equality rii rjj /(rij rji ) = pii pjj /(pij pji ), with 1
i
m and 1
j
m, is satisfied. However, preserving the original association is not the only requirement for adjusting tables (see, e.g., ref. [5]). Using the new proportions and taking into account the expression for Cohen’s kappa, the raked kappa is defined as follows: kˆ ∗ =


rii − ai bi ,
, 1 − ai bi




where {ai = ri .} and {bi = r.i } denote the prespecified marginal values. This definition was originally introduced by Agresti et al. [1]. The target margins may assume uniform values {ai = bi = 1/m}, or may correspond to the observed row margins or to the observed column margins or, alternatively, to an average of the observed row and column margins.

RAKED (ADJUSTED) KAPPA Table 2a. 1/m2 1/m2 1/m2

1/m2 1/m2 1/m2

1/m2 1/m2 1/m2

Table 2b. 1/m 0 0

0 1/m 0

0 0 1/m

The value of the raked kappa always differs from that of Cohen’s kappa except when the classifications are independent, or when there is perfect agreement in the original table. These two situations can clearly be seen by uniformly raking as indicated in Table 2a and Table 2b. In the first instance the raked table displays the proportions {rij = 1/m2 } as in Table 2a, and both kappa coefficients (adjusting and original) are equal to zero; in the second case the values are {rii = 1/m, rij = 0 for i = j}, which results in Table 2b, where both kappa coefficients are equal to 1. As with Cohen’s kappa, the raked kappa was originally designed for nominal classifications. The corresponding formulas were later generalized by introducing an appropriate system of weights 0
wij
1, with wii = 1, in order to include the case of ordered categories [1,3]. This was initially described by Spitzer et al. [6] for the original kappa coefficient. To carry out the inference, asymptotic standard errors can also be computed in both nominal and ordinal cases of classifications (see, e.g., [11] for details). The raked kappa was primarily aimed at highlighting the dependence of the kappa measure on the marginal distributions of contingency tables∗ . It is also an appropriate tool for analyzing the effect of ‘‘calibration’’ in the cell proportions to achieve marginal homogeneity. In addition, its use can be extended to a particular case of the evaluation and comparison among degrees of agreement in several studies that possess quite different marginal

6817

values, namely, when each pair of classifications of different tables includes the same ‘‘true’’ classification. In this case, the standardization process of the tables followed by the computation of the kappa coefficient seem to be the most appropriate techniques for analyzing the degree of agreement. Example. Computation of raked kappa is illustrated in the following example. We show, with hypothetical data, two different situations of a sample of 150 items classified by two observers, A and B, using a nominal scale with three levels. In the left table (Table 3a) the row and column margins are very similar, while in the other one (Table 3b) the margins are quite different. When both observers calibrate their ratings in order to arrive at a raked table having, for instance, uniform marginal values, the cells {pij } change as shown in Tables 4a and 4b. The raked tables preserve the same structure of association; in fact, if one computes the odds ratios for each pair of 2 × 2 subtables corresponding to the pair of tables (sample and raked), one will attain the same values. As shown from the computation of the kappa coefficients, the value of the raked version is different from that obtained from the original sample table, but in the other case the raked kappa (raked uniformly) is close to the sample kappa. Thus an improvement in the agreement can be achieved when the table is raked; however, in each case the level of improvement depends on the discrepancies between the marginal distributions of the contingency tables. One may compare the numerical values in the preceding example with those in Agresti et al. [1], as to the amount and the direction of the modifications implied. REFERENCES 1. Agresti, A., Ghosh, A., and Bini, M. (1995). Raking kappa: describing potential impact of marginal distributions on measures of agreement. Biom. J., 37, 811–820. (The basic paper in which the concept of raked kappa was introduced.) 2. Cohen, J., (1960). A coefficient of agreement for nominal scales. Educ. Psychd. Meas., 20,

6818

RAMP FUNCTION

Table 3. (a) Observer B

(b) Observer B

Observer A

1

2

3

Total

Observer A

1

2

3

Total

1 2 3

35 9 1

9 78 6

1 2 9

45 89 16

1 2 3

25 4 1

1 18 79

1 1 20

27 23 100

Total

49

93

12

150

Total

30

98

22

150

Sample kappa = 0.650 with asympt. stand. err. = 0.059

Sample kappa = 0.243 with asympt. stand. err. = 0.043

Table 4. (a) Observer B

(b) Observer B

Observer A

1

2

3

Total

Observer A

1

2

3

Total

1 2 3

0.272 0.041 0.021

0.050 0.253 0.030

0.011 0.040 0.282

0.333 0.333 0.333

1 2 3

0.272 0.008 0.053

0.060 0.201 0.073

0.002 0.124 0.207

0.333 0.333 0.333

0.333

0.333

0.333

1

Total

0.333

0.333

0.333

1

Total

Raked kappa = 0.709 stand. err. = 0.057

3.

4.

5.

6.

with

asympt.

220–321. (The basic paper on kappa coefficient. Compare with Encyclopedia of Statistical Science Vol. 6, p. 3816.) Cohen, J., (1968). Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull., 70, 213–220. Deming, W. E. and Stephan, F. F. (1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Ann. Math. Statist., 11, 427–444. Mosteller, F., (1968). Association and estimation in contingency tables J. Amer. Statist. Ass., 63, 1–28. (A lucid exposition of standardization methods for contingency tables.) Spitzer, R. L., Cohen, J., Fleiss, J. L., and Endicott, J. (1967). Quantification of agreement in psychiatric diagnosis. Arch. Gen. Psychiatry, 17, 83–87.

See also ASSOCIATION, MEASURES OF; KAPPA COEFFICIENT; KAPPA COEFFICIENT; and AGREEMENT, MEASURES OF.

MATILDE BINI

RAMP FUNCTION This is an alternative name for the cumulative distribution function of the uniform

Raked kappa = 0.518 stand. err. = 0.100

with

asympt.

distribution on [0,1], used in neural network* models. See, for example, reference 1. REFERENCE 1. Hornik, K. and Leisch, F. (2001). ‘‘Neural Network Models’’. In A Course in Time Series ˜ G. C. Tiao, and Analysis, Chapter 13, D. Pena, R. S. Tsay, eds. Wiley, New York, pp. 348– 362.

RAMSEY, FRANK PLUMPTON Born: February 22, 1903, in Cambridge, England. Died: January 19, 1930, in Cambridge, England. Contributed to: foundations of probability, foundations of mathematics. The following notes draw largely on a much fuller account by Newman [4]. All otherwise unascribed quotations are from that article. Frank Plumpton Ramsey came from an academic background. ‘‘His father was a

RAMSEY, FRANK PLUMPTON

mathematician, Fellow and later President of Magdalene College . . . and his brother Michael became Archbishop of Canterbury. He was educated at Winchester and at Trinity College Cambridge, and was a Scholar of both those ancient foundations.’’ A major influence in Ramsey’s life was his friendship with the Austrian philosopher Ludwig Wittgenstein. While still an undergraduate, Ramsey was the main translator of the German text of Wittgenstein’s Tractatus [6] into English. The author was pleased with the translation, and the two became close friends, following a visit by Ramsey to Austria, in September 1923, to discuss the meanings of certain passages in the book. ‘‘The eccentric philosopher and the brilliant undergraduate hit it off immediately.’’ However, a letter from Ramsey to Wittgenstein, late in 1923, contains the following passage, indicating that there were some difficulties in Ramsey’s life: ‘‘But I am awfully idle; and most of my energy has been absorbed since January by an unhappy passion for a married woman, which produced such psychological disorder, that I nearly resorted to psychoanalysis, and should probably have gone at Christmas to live in Vienna for nine months and be analyzed, had not I suddenly got better a fortnight ago, since when I have been happy and done a fair amount of work.’’ Nevertheless, in 1924 Ramsey did ‘‘spend six months in Vienna in psychoanalysis (rarer then than now).’’ This seemed to have some effect. In the fall of 1924, ‘‘he became a Fellow of King’s College and University Lecturer in Mathematics and soon afterwards married Lettice Baker, who had been a student in the Moral Sciences Tripos.’’ Ramsey worked in both the foundations of mathematics and the foundations of probability. Here we are primarily concerned with the latter, although this work was essentially couched in terms of abstract mathematics, despite the use of such apparently ‘‘applied’’ terms as ‘‘risk,’’ ‘‘utility,’’ ‘‘valuations,’’ etc. Ramsey’s work on the foundations of probability is set out in Chap. VII (‘‘Truth and Probability’’) of ref. [5], published posthumously. Zabell [7] provides a penetrating analysis of this chapter, examining in particular its

6819

relation to earlier work by Keynes [3]. Following a summary of Keynes’ theory of probability, Ramsey developed an approach based on concepts of ‘‘mathematical expectations, probabilities and valuations.’’ ‘‘Essentially, given any two’’ of these three concepts, ‘‘the remaining one follows more or less naturally.’’ In contrast to earlier writers, Ramsey effectively bootstrapped both the valuations and the probabilities from mathematical expectation, at the small cost of (a) a very general assumption about preferences; (b) an assumed existence of a certain kind of event; and (c) a further principle, original with him, that no agent’s subjective probabilities should be inconsistent. To be inconsistent means that: ‘‘He could have a book made against him by a cunning better and would then stand to lose in any event’’: this no-win situation is now usually called a Dutch book.

Ramsey did not prove the existence of valuations and probabilities constructed from his system of axioms. Newman [4] outlined a proof based on Davidson and Suppes [2]. Zabell [7] gives a detailed account of the concepts introduced by Ramsey—noting, in particular, an ‘‘ethically neutral proposition: the philosophical equivalent of tossing a coin.’’ Zabell also notes that Ramsey avoids use of the ‘‘principle of indifference’’—the choice of ‘‘equally likely’’ outcomes—which was regarded as a desirable accomplishment. Ultimately, Ramsey’s ideas did not have a notable influence on ideas about ways of formalizing the intuitive concept of probability. This was not a rapid process. Broadbent [1], in the course of an article mainly devoted to Ramsey’s work in mathematical logic, dismissed his studies on the foundations of probability as ‘‘interesting, if not altogether convincing.’’ This may well be a not unreasonable, though somewhat perfunctory, assessment of Ramsey’s contributions, insofar as he did not always write as lucidly as one might wish. Perhaps because of this, his ideas did not receive a great deal of attention, even among the limited circle of cognoscenti, until some years after his death. Ramsey died in 1930, at the early age of 26, from the effects of jaundice. The philosopher Wittgenstein, whose association with Ramsey we have already noted, ‘‘was at Ramsey’s

6820

RAMSEY’S PRIOR

bedside in the hospital until a few hours before he died.’’ REFERENCES 1. Broadbent, T. A. A. (1975). Ramsey, Frank Plumpton. In Dictionary of Scientific Biography. Scribner’s, New York, vol. 11, pp. 285–286. 2. Davidson, D. and Suppes, P. (1956). A finitistic axiomatization of subjective probability and utility. Econometrica, 24, 264–275. 3. Keynes, J. M. (1921). A Treatise on Probability. Macmillan, London. 4. Newman, P. (1987). Ramsey, Frank Plumpton (1903–1930). In The New Palgrave: A Dictionary of Economics in 4 volumes, (J. Eatwell, M. Milgate, and P. Newman, eds.) Macmillan, Stockton Press, pp. 41–45. 5. Ramsey, F. P. (1931). The Foundations of Mathematics and Other Essays, R. B. Braithwaite, ed. (London, Preface by G. E. Moore., Routledge and Kegan Paul, A later version was published in 1978, edited by D. H. Melkov, L. Mirsky, T. J. Smiley, and J. R. N. Stone.) 6. Wittgenstein, L. (1922). Tractatus LogicoPhilosophicus. Kegan Paul, Trench and Trubner, London. (Introduction by Bertrand Russell.) 7. Zabell, S. A. (1991). Ramsey, truth and probability. Theoria, 57, 211–238.

RAMSEY’S PRIOR An experimenter attempting to estimate a potency curve by running a quantal-response experiment may be unwilling to specify the functional form for the curve or be unable to run more than one experimental unit per dose level. In this situation Ramsey [6] proposed looking at the potency curve as a cumulative distribution function (CDF) and then employing a Bayesian non-parametric prior. The prior∗ he chose is essentially the Dirichlet process prior of Ferguson [4]. The posterior distribution on the potency curve is a mixture∗ of Dirichlet processes which Antoniak [1] showed to be computationally intractable. However, Ramsey obtained the mode of the posterior∗ , which he then used as his estimate of the potency curve. His estimator is a smoothed version of the isotonic regression∗ estimator.

Using his estimate of the potency curve, Ramsey showed that one may estimate any effective dose. Disch [3] derived the prior and posterior distributions of any effective dose assuming Ramsey’s prior on the potency curve. The posterior is difficult to calculate exactly, so two approximations were developed. Ramsey also addressed the question of optimal experimental design if his prior is to be used in the analysis of the data. He showed by example that an optimal design seems to assign one experimental unit per dose level. Kuo [5] has done a theoretical study tending to corroborate this choice of design, although for a different optimality criterion than Ramsey used. She also showed that for her special case (the Dirichlet process∗ prior had parameter uniform over [0, 1]), the optimal design did not pick the design doses uniformly spaced but shifted somewhat toward the prior estimate of the ED50. Ramsey’s model and prior will now be described. A dose, x, is administered to an experimental unit which then either responds with probability P(x) or fails to respond. The function P is the potency curve and is assumed to be an increasing function. Observational doses x1 , x2 , . . . , xM are chosen, and ni experimental units are assigned to dose xi with the result that si units independently respond, i = 1, 2, . . . , M. The joint likelihood for P(x1 ), P(x2 ), . . . , P(xm ) is proportional to M  [P(xi )]si [1 − P(xi )]ni −si .

(1)

i=1

Ramsey viewed P as a CDF and essentially assigned a Dirichlet process prior to P where the parameter of the process is βP∗ . The function P∗ is the experimenter’s prior guess at the potency curve, and the constant β determines the degree of smoothing in the posterior estimate where for β = 0 the posterior estimate is the isotonic regressor and for β → ∞ the posterior estimate is P∗ . Alternatively, β may be thought of as the experimenter’s strength of belief in the accuracy of P∗ measured in number of pieces of data at a given dose level. The Dirichlet process prior implies in particular that the joint density of P(x1 ), P(x2 ), . . . , P(xM ) is

RANDOM BALANCE DESIGNS

is the incomplete beta function. Hence the prior CDF of ED(p) is

proportional to M+1 

∗ (x )−P∗ (x i i−1 ))

[P(xi ) − P(xi−1 )]β(P

(2)

i=1

Pr{ ED (p)
x} = 1 − Pr{P(x) < p} = 1 − Ip (a, b)

when integrating with respect to M  i=1

6821

 M+1  dP(xi ) [P(xi ) − P(xi−1 )] i=1

as Ramsey does. Note that P(x0 ) = P∗ (x0 ) = 0 and P(xM+1 ) = P∗ (xM+1 ) = 1. A consequence of (2) is that P(xi ) has a beta distribution. In fact, as Ferguson [4] showed, P(x) has a beta distribution regardless of the value of x. Taking the product of (1) and (2) and setting the partial derivatives with respect to P(xi ), i = 1, 2, . . . , M, to zero yields the following system of equations:   si ni − P(xi ) P(xi )[1 − P(xi )] ni   ∗ P (xi+1 ) − P∗ (xi ) P∗ (xi ) − P∗ (xi−1 ) − . =β P(xi+1 ) − P(xi ) P(xi ) − P(xi−1 ) This system may be solved to get the posterior mode using a constrained Newton–Raphson method∗ and a special matrix inversion method applicable to tridiagonal matrices∗ . If xi
x
xi+1 , then the modal estimate is P(x), where P∗ (x) − P∗ (xi ) P(x) − P(xi ) = ∗ P (xi+1 ) − P∗ (xi ) × [P(xi+1 ) − P(xi )], so that the posterior mode has the same shape as the prior guess, P∗ , between xi and xi+1 . Disch [3] observed that Ramsey’s prior implies a prior on the ED(p), where, for example, ED (0.5) = ED50. Let FP(x) (p) be the prior CDF of P(x). Since Ramsey’s prior implies that P(x) has a priori a beta distribution∗ , FP(x) (p) = Ip (a, b), where a = βP∗ (x), b = β(1 − P∗ (x)), and Ip (a, b) = { (a + b)/[ (a) (b)]}  1 × xa−1 (1 − x)b−1 dx 0

= I1−p [β{1 − P∗ (x)}, βP∗ (x)]. The posterior CDF may be approximated by replacing P∗ with the modal estimate of Ramsey with β and β + mini (Di ), where Di = ni + (number of successes at dose levels less than xi ) + (number of failures at dose levels greater than xi ) [3]. Investigating the implied priors for several effective doses can be a useful tool in properly assessing Ramsey’s prior. However, Ramsey showed that his modal estimator can still provide an improvement to isotonic regression even when P∗ is badly specified, provided that β is moderate.

REFERENCES 1. Antoniak, C. (1974). Ann. Statist., 2, 1152–1174. 2. Ayer, M., Brunk, H. D., Ewing, G. M., Reid, W. T., and Silverman, E. (1955). Ann. Math. Statist., 26, 641–647. 3. Disch, D. (1981). Biometrics, 37, 713–722. (Extends Ramsey’s results to the distribution of effective doses.) 4. Ferguson, T. S. (1973). Ann. Statist., 1, 209–230. (The basic introduction to Dirichlet processes.) 5. Kuo, L. (1983). Ann. Statist., 11, 886–895. (Discusses the experimental design question.) 6. Ramsey, F. L. (1972). Biometrics, 28, 841–858. (Introduces Ramsey’s prior and has several examples.) See also ISOTONIC INFERENCE and PROBIT ANALYSIS.

D. DISCH

RAMSEY THEORY. See GRAPH THEORY RANDOM BALANCE DESIGNS In the statistical literature, the word ‘‘balanced’’ is used in several contexts

6822

RANDOM DETERMINANT

(see BALANCING IN EXPERIMENTAL DESIGN; GENERAL BALANCE); Preece [2] wrote an interesting article about the muddle in terminology surrounding it. Random balanced designs is another such class of balanced designs. These, initially introduced by Satterthwaite [3,4] for industrial experiments where the effects of several input variables are to be studied, have not gained much popularity and are not even known to some statisticians. Satterthwaite [4] distinguishes between exact balance and random balance in connection with a factorial experiment∗ treatment structure. Consider a s1 × s2 × · · · × sn factorial experiment in n factors A1 , A2 , . . . , An , where Ai is at si levels. The factor Aj is exactly balanced with respect to the factor Ai if the distribution of the levels of Aj is the same for each level of the factor Ai . This is clearly the case for a complete factorial with all treatment combinations. There are several known fractional factorials meeting the exact balance criterion. The factor Aj is randomly balanced with respect to the factor Ai if the random sampling process used to select the sj levels of factor Aj is the same for each level of factor Ai . The factor Aj is randomly unbalanced with respect to the factor Ai if the random sampling process used to select the sj levels of factor Aj is not the same for each level of factor Ai . Consider a 32 experiment with factors A and B, where the levels of a are represented by a0 , a1 , a2 and the levels of B by b0 , b1 , b2 . Then the nine treatment combinations a0 b0 , a0 b1 , a0 b2 , a1 b0 , a1 b1 , a1 b2 , a2 b0 , a2 b1 , and a2 b2 provide exact balance for factor B with respect to factor A. Alternatively, suppose that we choose two of the three levels of B by simple random sampling at each level of A and obtain the six treatment combinations a0 b1 , a0 b2 , a1 b0 , a1 b2 , a2 b0 , and a2 b1 ; then factor B is randomly balanced with respect to factor A. Instead, if one chooses levels b0 , b1 , with each of levels a0 , a2 of A and levels b1 , b2 with level a1 of A, getting the treatment combinations a0 b0 , a0 b1 , a1 b1 , a1 b2 , a2 b0 , and a2 b1 , then factor B is randomly unbalanced with respect to factor A. No restrictions are put on the random process of selecting the levels of a factor Aj that

is randomly balanced with respect to the factor Ai . However, if the correlation coefficient between two independent variables exceeds a prechosen value, then at least one of these two variables is eliminated from the experiment, to avoid singularities in the information matrix. The situation is analogous to the problem of multicollinearities∗ in regression analysis. Note that the terms ‘‘balance’’ and ‘‘unbalance’’ are closely connected with the underlying model used in the analysis. A set of treatment combinations providing a balanced design under one model need not be a balanced design with another model. Any type of desired analysis using data analytic tools, multiple regression∗ , or ANOVA methods can be carried on all or on a subset of the factors for a random balance design. A numerical illustration of a random balanced design with a synthesized example was given by Budne [1]. REFERENCES 1. Budne, T. A. (1959). Technometrics, 1, 139– 155. (Analysis of random balanced designs is illustrated.) 2. Preece, D. A. (1982). Utilitas Math., 21C, 85–186. (Review article on the concept of balance in experimental designs.) 3. Satterthwaite, F. E. (1957). J. Am. Statist. Ass., 52, 379–380. (Random balance designs introduced.) 4. Satterthwaite, F. E. (1959). Technometrics, 1, 111–137. (Detailed treatment of random balance designs given.) See also BALANCING IN EXPERIMENTAL DESIGN; GENERAL BALANCE; and UNBALANCEDNESS OF DESIGNS, MEASURES OF.

DAMARAJU RAGHAVARAO

RANDOM

DETERMINANT. See

RANDOM MATRICES

RANDOM-DIGIT DIALING, SAMPLING METHODS FOR Random-digit dialing is a sampling methodology that uses as a frame all possible

RANDOM-DIGIT DIALING, SAMPLING METHODS FOR

telephone numbers. In the United States, a 10-digit telephone number is comprised of a three-digit area code, a three-digit central office code, and a four-digit suffix. There are presently 34,000 active area code/central office code combinations in use in the United States (based on information updated monthly by AT & T Long Lines Division). 340,000,000 possible 10-digit telephone numbers exist when a four-digit suffix in the range 0000 to 9999 is added to all 34,000 distinct six-digit area code/central office code combinations. Simple random-digit dialing involves selecting area code/central office code combinations at random from a list of combinations in use within the geographic area of interest, and then selecting four-digit random numbers as suffixes. Simple random-digit dialing provides complete coverage of all residential telephone numbers but at the cost of a sample largely comprised of nonworking and nonhousehold numbers. Seventy-five percent of sample numbers will be business, government, pay phone, and unassigned numbers. The total survey costs are quite high with simple random-digit dialing, since threefourths of the initial call attempts will be made to nonworking or nonhousehold numbers. A variation of simple random-digit dialing which reduces unproductive calls has been described by Waksberg [4]. This form of random-digit dialing involves two-stage sampling As a first step, all possible choices for the seventh and eighth digits are added to all six-digit area code/central office code combinations obtained from AT & T. These eight-digit numbers are treated as primary sampling units∗ (PSUs). A random two-digit suffix is selected and appended to a randomly selected eight-digit number. This 10digit number is then dialed. If the dialed number is a residential number, an interview is attempted and the PSU is retained in sample. Additional random two-digit suffixes are selected and dialed with the same eight-digit PSU until a fixed number of residential phone numbers are contacted within that PSU. If the original number dialed was not a working number or was not residential, the PSU is eliminated from the sample and no further calls are made to that PSU.

6823

The foregoing steps are repeated until a predesignated number of PSUs are selected for inclusion in the sample. The total sample size for such a survey is the product of the predesignated number of PSUs and the number of residential interviews desired per PSU. With this design, all residential telephone numbers have the same final probability of selection, despite the fact that the first-stage and second-stage probabilities are unknown. In addition, the PSUs are selected with probabilities proportionate to size∗ (PPS), based on the actual number of residential phone numbers within the PSU. Since a fairly high proportion of 100phone-number blocks contain no residential phone numbers (generally unassigned blocks), this two-stage design reduces the number of call attempts to these blocks since they would be rejected after the first stage. Overall, the two-stage Waksberg approach increases the contact rate for residential numbers from about 25% to close to 60%. Thus substantially reduced costs are associated with this method when compared to simple random-digit dialing. Both simple random-digit dialing and Waksberg random-digit dialing are alternatives to methods collectively known as directory sampling. One form of directory sampling, simple random directory dialing, uses as a frame all listed telephone numbers in current telephone directories for the geographic area of interest. Frame errors with such a methodology are significant. Numbers unlisted by customer request and numbers unlisted due to the time lag of directory publishing together account for about 22% of total U.S. household telephone numbers. This problem of unlisted numbers is most severe in large cities, where over 40% of household numbers are unlisted. Households with unlisted numbers move more frequently, are more urban, are younger, and have less education than do listed households. A second form of directory sampling, directory ‘‘N + 1’’ dialing, involves the addition or subtraction of a constant from the fourdigit suffix of each number in a directory sample. Although this methodology reduces the unlisted number frame error, it does not fully overcome the disadvantage associated

6824

RANDOM EFFECTS MODEL

with directory samples. Most significantly, new central office codes are missed due to the time lag involved between telephone company assignment of a central office code and publication of the telephone directory. A more complete description of the two forms of random-digit dialing and of alternative forms of directory-based telephone samples can be found in Frey [2]. A detailed comparison of random-digit-dialing telephone surveys and personal interview surveys can be found in Groves and Kahns [3]. Blankenship [1] provides an introduction to telephone survey methodology.

providing a directory of introductions to these areas. The article is divided into four sections. We begin with a brief historical survey, describe some applications, and then discuss, in turn, various choices for the parameter space of random fields, different types of probabilistic structure, and finally, some sample path properties. We shall often refer to MARKOV RANDOM FIELDS (MRFs) to avoid unnecessary overlap of material. Indeed, the two articles should be read in conjunction to obtain a more complete picture of the area. SOME HISTORY AND APPLICATIONS

REFERENCES 1. Blankenship, A. B. (1977). Professional Telephone Surveys. McGraw-Hill, New York. 2. Frey, J. H. (1983). Survey Research by Telephone. Sage, Beverly Hills, Calif. 3. Groves, R. M. and Kahn, R. L. (1979). Surveys by Telephone: A National Comparison with Personal Interviews. Academic Press, New York. 4. Waksberg, J. (1978). J. Amer. Statist. Ass., 73, 40–46. See also SURVEY SAMPLING and TELEPHONE SURVEYS, COMPUTER ASSISTED.

KENNETH H. GROSS

RANDOM

EFFECTS

MODEL.

See FIXED-, RANDOM-, AND MIXED-EFFECTS MODELS; and VARIANCE COMPONENTS

RANDOM FIELDS Random fields are simply stochastic processes∗ whose ‘‘time’’ parameter ranges over some space more complicated than the usual real line. Usually, the parameter space is either all or part of Euclidean N-space (yielding an ‘‘N dimensional random field’’), but it may, on occasion, be something far more complex. As is well known, the theory of stochastic processes is vast, and so, a fortiori, must be the theory of random fields. Consequently, it will be impossible in this brief article to give anything other than the most condensed of outlines of this theory. Indeed, we shall do little more than indicate the various areas of current interest in random fields, while

The history of continuous-parameter random fields dates back at least to the beginning of the twentieth century, at which time there was considerable interest in the mathematical theory of turbulence. It was not, however, until the middle of the century that serious advances were made. In 1952, Longuet-Higgins [19], a Cambridge applied mathematician, wrote the first of a series of path-breaking papers in which he studied a number of sample path properties of Gaussian random fields. His motivation came from the study of ocean surfaces, which he modeled as either two-dimensional (when time was fixed) or three-dimensional (when time was allowed to vary) random fields. Consequently, the sample path properties to which he devoted most attention were related to level-crossing behavior and the structure of local maxima (see below for more on this), these subjects being of obvious oceanographic interest. Independently of Longuet-Higgins but at about the same time, interest in random fields started developing in the Soviet Union, and in 1957 and 1961 Yaglom published two important papers developing the mathematical basis of continuous parameter random fields, considering, for example, their spectral∗ theory. By the late 1960s considerable effort was being devoted to studying the sample path properties of random fields, particularly by Belyaev and members of his group, culminating in a collection of papers [7,8]. More recently, this work was consolidated and expanded in a series of papers by Adler

RANDOM FIELDS

and co-workers. These developments, as well as an overview of previous results and the general theory of continuous parameter random fields, are described in detail in Adler [1]. At the same time that the theoretical advances described above were occurring, random fields were being used as mathematical models in a number of different areas. One of the most fruitful applications has been to the modeling of the microscopic roughness of macroscopically smooth surfaces, such as steel, plastic, and so on. A recent review of the rather extensive literature of this area is given in Archard et al. [5], with more sophisticated models given in Adler and Firman [3]. Continuous parameter random fields have also been applied to such diverse areas as forestry [23], geomorphology [20], geology∗ [16], turbulence [6,21,25], and seismology [29]. The book by Mandelbrot [22] discusses a wide variety of further applications of random fields exhibiting fractal∗ properties, while a number of more pedestrian applications are listed in Adler [1]. CHOICES OF THE PARAMETER SET There are essentially three different choices of parameter set over which a random field can be defined. The choice that one would initially expect to yield the most mathematically tractable models is that of a finite or countable subset of Euclidean N-space, such as a simple lattice or graph. These are discussed in some detail in the companion entry on MRFs, so we shall say no more on them here, other than to note that despite the apparent simplicity of these parameter spaces, the difficulties inherent in the analysis of the resulting fields are formidable. The second class of random field, which we discuss in some detail below, involves processes X(t) for which t = (t1 , t2 , . . . , tN ) is allowed to vary in a continuous fashion over some subset of N-space. Here, when N = 1, we have a simple continuous-time stochastic process∗ ; when N = 2, a random surface; and when N = 3, a spatial process such as local pressure or speed in fluid turbulence. These random fields, defined on parameter sets of intermediate difficulty, are those which have been studied with the greatest degree of success.

6825

The widest class of fields is the so-called generalized fields, which are defined not on points in N-space but on functions, themselves defined on N-space. Since the class of index functions may be a family of delta functions, all ordinary fields, whether they be indexed either by a lattice or continuously on N-space, are clearly special cases of generalized fields. The level of generality afforded by this distinctly more complex parameter space is generally not of great significance for simple applications of random fields, so we shall not discuss this case further. However, it is important to note that these fields arise in a natural fashion in physics and are central to any discussion of Markov-type properties for random fields (see MARKOV RANDOM FIELDS). STRUCTURAL PROPERTIES OF RANDOM FIELDS Once the parameter set of a random field has been decided on, further study, be it theoretical or applied, will fall into one of a number of essentially disjoint areas depending on the probabilistic structure of the field. For example, as discussed in detail in the entry MARKOV RANDOM FIELDS, the assumption of any type of Markov structure immediately imposes a very distinct probabilistic structure on the distributions associated with the field that dictates specific directions of investigation and exploitation. A generally milder assumption than Markovianess is that the field satisfies some type of martingale∗ property, which is, as usual, characterized by considering expectations of the field conditioned on the ‘‘past.’’ However, since the parameter set of a random field is not totally ordered, there is no unique way to define the ‘‘past,’’ and thus no unique way in which to define the martingale property. The result of this has been a proliferation of different types of multiparameter martingales, originating with the work of Wong and Zakai [31]. There has been substantial interest in this area over the past decade, with the French school of probabilists having made substantial contributions, many of which have been published in the Springer series Lecture Notes in Mathematics either as proceedings of the Strasbourg seminars (e.g.,

6826

RANDOM FIELDS

Vols, 784, 850, 920, 921) or as special topics Volumes (e.g., Vols. 851, 863). A class of stochastic processes on the line about which a great deal is known is the class of processes with independent increments. Random fields possessing an analogous property are reasonably easy to define but considerably more difficult to study. For example, only recently have the most basic sample path properties of right continuity and left limits (suitably defined) been established. It seems that it is going to be extremely difficult to obtain as broad a picture of these fields as has been possible for their one-dimensional counterparts, primarily because one of the most important tools used in the study of the latter, namely stopping times, is not really available in the multiparameter case. Indeed, the fact that stopping times cannot be defined easily in the multiparameter setting (consider trying to generalize to N-space the simplest of all stopping times, the first hitting time) has been a serious impediment to creating a theory in this setting as rich as the one-dimensional theory. Some progress in this direction has been made via the notion, in two dimensions, of stopping lines, but this tool is not as powerful as its one-dimensional counterpart. Once again, the French school of probabilists has been active in this area, although the original idea is due to Wong and Zakai [31] with important contributions by Merzbach (e.g., ref. 24). Moving away from structural properties based on some form of dependence on the past, we come to the properties of stationarity, isotropy, and self-similarity∗ . Each of these properties describes some form of statistical homogeneity of the process. Stationarity requires that the field behave (statistically) in the same fashion in every region of its domain of definition. Isotropy requires that the joint distribution of the field at two points be a function only of the distance between them. Both of these requirements can be expressed in terms of the spectral representation∗ of the field, details of which can be found in Adler [1]. It is worth noting that properties such as stationarity are essentially no more difficult to study in the N-parameter case than in the one-parameter case, since their definition and study are in no way connected with ordering properties

of the parameter space. The property of selfsimilarity is somewhat more complex than the other two just described. Basically, it describes the fact that after the parameter set has been rescaled in some fashion, the original field can be recovered (statistically) by a simple linear transformation of the field values. This property (which is related to the famous ‘‘renormalization group’’ of statistical physics) and its implications for the possible distributions of a random field are discussed in some detail in Dobrushin and Sinai [14] and the references listed therein. The last structural property that we shall consider is that of Gaussianity. It is in the case of Gaussian random fields that there has been the most success in developing, for fields, an analog of many of the results known about one-dimensional processes. There are two reasons for this. The first is the comparatively simple and tractable form of the multivariate normal distribution, which allows many problems to be solved for Gaussian processes that are completely intractable in general. The other reason is that, even on the real line, many of the aspects of Gaussian processes that have been of central interest are not connected to the total ordering of the line, and so are far easier to extend to multiparameter situations than are those of, for example, Markov processes∗ or martingales. (For relationships between two-parameter martingale theory and Gaussian fields, see Bromley and Kallianpur [10].) Since the theory is so rich in the Gaussian case, we shall now devote some time to considering it in detail. GAUSSIAN RANDOM FIELDS Let us assume now that a Gaussian random field X(t) is defined continuously on N-space and has mean E{X(t)} = 0. Then all the statistical properties of X depend only on the covariance function R(s, t) = E{X(s) · X(t)} of X. For example, whether or not the field will have continuous sample paths can be read off in a reasonably straightforward fashion from the behavior of the covariance function at the origin. A little reflection shows why this must be the case. If R(t) decays slowly in a neighborhood of the origin, close values of the field will be highly correlated, and thus

RANDOM FIELDS

highly dependent, leading to continuous sample paths. Rapid decay will lead to the opposite phenomenon. Indeed, the behavior of the covariance near the origin also determines such things as Hausdorff dimension∗ properties of the sample paths and the properties of local time∗ . Many of these phenomena are not, in principle, more difficult to study in the random field situation than they are for simple stochastic processes on the line, although they occasionally involve more sophisticated calculations. One problem that is significantly more difficult in the multiparameter case is that of level crossings. The level crossings of a continuous process in univariate time are simply those times t at which the process takes the value u, where u is some predetermined value [i.e., {t : X(t) = u}]. Knowledge of the number of level crossings of various levels is a useful tool in many applications of stochastic processes, and one of the most useful results available is the so-called Rice formula, which states that for stationary Gaussian processes∗ the mean number of level crossings in time [0, T] is given by λ1/2 (σ π )−1/2 exp(− 21 u2 /σ 2 ) where λ and σ are parameters of the process. (For details, see Cram´er and Leadbetter [13].) It has long been a problem of some considerable interest to obtain a generalization of the Rice formula for random fields. However, the generalization of the notion of level crossings to the multiparameter situation involves uncountable sets of the form {t : X(t) = u} which, when N = 2 form families of contour lines in the plane, when N = 3 form surfaces in three-dimensional space, and in general define (N − 1)-dimensional manifolds∗ . Clearly, then, not only is it not trivial to generalize Rice’s formula, but it is not even clear how best to generalize the notion of level crossings itself. Let it suffice to say here that this can be done, by employing a number of concepts from integral geometry and differential topology, and an interesting and useful theory of multidimensional level crossings can be developed. For details see Adler [1]. Another problem of both theoretical interest and immediate applicability is the behavior of Gaussian fields in the neighborhood

6827

of high maxima. Early work in this area was done by Longuet-Higgins, who was interested in the shape of wave crests, and whose results, together with more recent advances, have been put to substantial use in studying the contact between two surfaces modeled as random fields. Both Lindgren [18] and Nosko [26–28] made significant contributions to this problem, with some more recent results due to Wilson and Adler [30]. Essentially all these authors have studied various aspects of the fact that in the vicinity of a high maximum (or low minimum) a Gaussian field exhibits almost deterministic behavior, so that its sample paths, appropriately normalized, look just like its covariance function in a neighborhood of the origin. Problems related to the distribution of the global maximum of Gaussian fields over large sets have also been studied in considerable detail, and a number of asymptotic results giving values for Pr[sup(X(t) : t ∈ S) > x], for large x, have been obtained (e.g., Hasofer [17], Bickel and Rosenblatt [9]). Nonasymptotic results are much harder to find, although good bounds to the foregoing excursion probability are known in some spe˜ and cial cases (e.g., Goodman [15], Cabana Wschebor [11,12], Adler [2]). Finally, note that some of the results on level crossings and sample path behavior have been extended from the Gaussian case to classes of processes that exhibit, in some form, a near-Gaussian behavior. An example of such a field is given by the socalled ‘‘χ 2 field,’’ which can be defined as the sum of a number of squared, zero-mean, independent, Gaussian fields. Although this field often exhibits highly non-Gaussian behavior, the fact that it is defined via Gaussian fields makes it analytically tractable. Further Reading The reader interested in pursuing this subject further would be best advised to begin with the three books listed in the references. Adler [1] deals with continuous parameter fields, Dobrushin and Sinai [14] with lattice indexed and generalized fields, while Mandelbrot [22] has something to say about nearly everything.

6828

RANDOM GRAPHS

REFERENCES 1. Adler, R. J. (1981). The Geometry of Random Fields. Wiley, Chichester, England. 2. Adler, R. J. (1984). Ann. Prob., 12, 436–444. 3. Adler, R. J. and Firman, D. (1981). Philos. Trans. R. Soc., 303, 433–462. 4. Adler, R. J., Monrad, D., Scissors, R. H., and Wilson, R. (1983). Stoch. Proc. Appl., 15, 3–30. 5. Archard, J. F. et al. (1975). In The Mechanics of Contact between Deformable Bodies, A. D. de Pater and J. J. Kalker, eds. Delft University Press, Delft, The Netherlands, pp. 282–303. 6. Batchelor, G. K. (1953). The Theory of Homogeneous Turbulence. Cambridge Monogr. Mech. Appl. Math. Cambridge University Press, New York. 7. Belyaev, Yu. K., ed. (1972). Bursts of Random Fields. Moscow University Press, Moscow (in Russian). 8. Belyaev, Yu. K. (1972). Proc. 6th Berkeley Symp. Math. Statist. Prob., Vol. 2. University of California Press, Berkeley, pp. 1–17. 9. Bickel, P. and Rosenblatt, M. (1972). In Multivariate Analysis III, P. R. Krishnaiah, ed. Academic Press, New York, pp. 3–15. 10. Bromley, C. and Kallianpur, G. (1980). Appl. Math. Optim., 6, 361–376. ˜ 11. Cabana, E. M. and Wschebor, M. (1981). J. Appl. Prob., 18, 536–541. ˜ E. M. and Wschebor, M. (1982). Ann. 12. Cabana, Prob., 10, 289–302. 13. Cram´er, H. and Leadbetter, M. R. (1967). Stationary and Related Stochastic Processes. Wiley, New York. 14. Dobrushin, R. L. and Sinai, Ya. G. (1980). Multi-component Random Systems. Marcel Dekker, New York. 15. Goodman, V. (1976). Ann. Prob., 4, 977–982. 16. Harbaugh, J. W. and Preston, F. W. (1968). In Spatial Analysis: A Reader in Statistical Geography, B. J. L. Berry and S. F. Marble, eds. Prentice-Hall, Englewood Cliffs, N.J., pp. 218–238. 17. Hasofer, A. M. (1978). Adv. Appl. Prob. Suppl., 10, 14–21. 18. Lindgren, G. (1972). Ark. Math., 10, 195–218. 19. Longuet-Higgins, M. S. (1952). J. Mar. Res., 11, 245–266. 20. Mandelbrot, B. B. (1975a). Proc. Natl. Acad. Sci. USA, 72, 3825–3828. 21. Mandelbrot, B. B. (1975b). J. Fluid Mech., 72, 401–416.

22. Mandelbrot, B. B. (1982). The Fractal Geometry of Nature. W. H. Freeman, San Francisco. 23. Mat´ern, B. (1960). Commun. Swed. For. Res. Inst., 49, 1–144. 24. Merzbach, E. (1980). Stoch. Proc. Appl., 10, 49–63. 25. North, G. R. and Cahalan, R. F. J. Atmos. Sci., 38, 504–513.

(1981).

26. Nosko, V. P. (1969). Sov. Math. Dokl., 10, 1481–1484. 27. Nosko, V. P. (1969). Proc. USSR—Japan Symp. Prob. (Harbarovsk, 1969), Novosibirsk, pp. 216–222 (in Russian). 28. Nosko, V. P. (1970). Vestnik Mosk. Univ. Ser. I Mat. Meh., pp. 18–22 (in Russian). 29. Robinson, E. A. (1967). Statistical Communication and Detection. Charles Griffin, London. 30. Wilson, R. J. and Adler, R. J. (1982). Adv. Appl. Prob., 14, 543–565. 31. Wong, E. and Zakai, M. (1974). Wahrscheinl. verw. Geb., 29, 109–122.

Z.

See also FRACTALS; MARKOV RANDOM FIELDS; and STOCHASTIC PROCESSES.

ROBERT J. ADLER

RANDOM GRAPHS A random graph is a pair (G, P) where G is a family of graphs and P is a probability distribution over G (see GRAPH THEORY). Two extensively studied cases are: 1. The Edge Probability Model. G is the set of all graphs on n-vertices where any particular m-edge graph occurs   n  −m 2 . Such with probability pm (1 − p) a random graph is often denoted by Kn,p to indicate selection of edges from the complete graph Kn , with each edge chosen independently with probability p. 2. The Equiprobable Model. G is a particular class of graphs on n vertices and m edges (e.g., trees, regular graphs) where each graph occurs with equal probability. Intuitively, the edge probability model is readily identified with applications, say in

RANDOM GRAPHS

chemistry or biology, where edges are introduced (‘‘evolve’’) independently with probability p. Equivalently, for engineering and communications applications, the edges in the edge probability model may be considered to be independently deleted (through ‘‘failure’’) with probability (1 − p). The alternative equiprobable model has a more obvious tie to enumeration problems for particular classes of graphs in combinatorial graph theory. Applications of probabilistic methods in combinatorics∗ were started by Szekeres, ´ Turan, Szele, and Erd¨os in the 1930s and 1940s (see Erd¨os [5]), while the theory of random graphs was initiated by Erd¨os and R´enyi in the 1960s in their seminal paper on the evolution of random graphs [7] followed by a sequence of fundamental contributions (for the entire collection, see ref. 6 or 22). Since then several hundred papers have been published in various areas of ran´ dom graph theory. See the books by Bollobas [1,3], Palmer [21], Erd¨os and Spencer [8], and Sachkov [23] as well as review papers [2,10, 12,24]. ALMOST CERTAIN PROPERTIES OF RANDOM GRAPHS The principal asymptotic results for the edge probability model of random graphs are essentially equivalent to those of the equiprobable model over the class of alln-vertex m-edge  graphs if we take p and m/ n2 to be asymptotically of the same order. For convenience we state results pertaining to the class of all n-vertex graphs as properties of the random graph Kn,p . Erd¨os and R´enyi’s fundamental result for random graphs is that for many so-called monotone structural properties A (such as connectivity) of Kn,p , there exists a threshold function tA (n) for property A where, with p = p(n), tA = t(n), lim Pr{Kn,p has property A }  0 if p/tA → 0, = as n → ∞. 1 if p/tA → ∞,

n→∞

The notion of random graph evolution relates to the changes in the typical structure of Kn,p as the edge occurrence probability p increases. By ‘‘typical structure’’ we mean

6829

properties possessed almost surely (a.s.), that is, with probability 1 as n → ∞. Consider how the typical structure of Kn,p changes during its evolution. For very sparse graphs, characterized by the property that pn3/2 → 0 as n → ∞, Kn,p a.s. consists only of disjoint edges and isolated vertices. When pn3/2 → c, where c is a positive constant, trees on at most three vertices appear with positive probability. For pn → 0, Kn,p is a.s. a union of disjoint (isolated) trees. For pn → c, where 0 < c < 1, cycles are present in Kn,p with a positive probability and Kn,p is a.s. a union of disjoint trees and components containing exactly one cycle. The structure of the random graph changes dramatically when pn = c as c passes through the value 1. For c > 1, Kn,p a.s. consists of one gigantic component of order comparable to n, along with a number of isolated trees and relatively small components with exactly one cycle. This epoch of the evolution is often called phase transition (e.g., the gel point of polymer chemistry). When p increases further, the gigantic component subsumes the smaller components. As p reaches the order of magnitude log n/n, only isolated vertices remain outside the large component. Finally, as pn − log n → ∞, the random graph becomes a.s. connected. At this same point the graph is a.s. asymmetric, and for even n a.s. has a perfect matching. More specifically, for pn − log n − (r − 1) log log n → ∞, the graph a.s. has minimum degree at least r and is r-connected, and is Hamiltonian (see [16]) for r = 2. Proceeding to somewhat denser graphs where pn2/(r−1) → ∞, the graph a.s. has a complete subgraph on r vertices. Note that all of these important transitions occur in the region of relatively sparse edge density, in that all occur with the edge probability asymptotically smaller than any fixed constant c. See the fundamental paper of Erd¨os and R´enyi [7], where the notion of evolution was introduced and the majority of basic facts about the evolutionary process were proved. NUMERICAL CHARACTERISTICS OF RANDOM GRAPHS The theory of random graphs is not only concerned with dichotomous graph properties. The exact and / or approximate distributions

6830

RANDOM GRAPHS

of numerical graph characteristics constituting random variables defined over random graphs have been investigated. The number of vertices of a given degree, the numbers of independent and dominating sets of vertices, and the number of subgraphs of a given type are examples of such random variables which have been investigated. For typical results ´ dealing with subgraphs, see Karonski [13]. The asymptotic distribution of extremal and / or rank statistics associated with random graphs, such as the maximal and minimal degree, have been investigated extensively. Often, the extremal characteristic asymptotically has its probability distribution concentrated at only one or two discrete values. In particular, for the ‘‘relatively dense’’ random graphs characterized by a constant edge probability p = c for any 0 < c < 1, we trivially obtain that the random graph Kn,p a.s. has radius 2, diameter 2, longest path length n − 1, smallest cycle size 3, and largest cycle size n as n → ∞. More interesting is the result that the largest clique size and largest independent set size a.s. assume one of at most two values in the neighborhood of 2 log n/ log(1/p) for largest clique size and 2 log n/ log(1/(1 − p)) for largest independent ´ and set size (see Matula [17] and Bollobas Erd¨os [4]). The chromatic number of Kn,p with b = 1/(1 − p) is known to a.s. fall in the interval   ( 12 − )n (1 + )n , . logb n logb n for any 0 <  < 12 , with an open conjecture that the result is a.s. in a narrow interval about the lower limit n/(2 logb n) (see Grimmett and McDiarmid [11]). RANDOM TREES A random tree is understood to be a tree picked at random from a given family of trees T, such as the family of all nn−2 labeled trees on n vertices, as well as families of plane and binary trees. Characteristics studied include the number of vertices of a given degree, the distance of a given vertex from the root (altitude), the number of vertices at altitude k (width), the height of a rooted

tree (the maximal altitude of its vertices), and the diameter of the random tree. Metric characteristics related to the length of paths between specified vertices in a random tree have also been investigated. Details on these topics and more general structures, such as random forests (collections of trees), random hypertrees (connected hypergraphs without cycles), and k-dimensional trees, are contained in Moon’s monograph [19] and a ´ section devoted to trees in Karonski’s review paper [12]. RANDOM REGULAR GRAPHS AND DIGRAPHS Besides trees, the equiprobable model has been applied to random regular graphs (all vertex degrees equal) and out-regular digraphs (all vertex out-degrees equal). The asymptotic enumeration of the number of graphs in the family of all d-regular graphs (all vertices of degree d) on n vertices has provided a procedure to analyze the average properties of this important family of graphs. ´ See Bollobas’s review paper [2]. A mapping f : V → V of an n-element set into itself may be interpreted as a digraph with out-degree 1 at each vertex. Mappings without fixed points correspond to digraphs without loops. One-to-one mappings (permutations) correspond to regular digraphs with both in-degree and out-degree 1. Mappings correspond to digraphs where each component contains at most one cycle, and permutations yield components each of which is a directed cycle. A random mapping and a random permutation then may each be defined by the process of uniform selection from the appropriate set of corresponding digraphs as per the equiprobable model. Issues such as connectedness and the cycle-length distribution of random mappings have been investigated. See the monograph by Kolchin [15], or the review paper by Mutafchiev [20]. Generalizing this approach, random digraphs of out-degree d corresponding to random dmultiple-valued mappings have also been investigated using the equiprobable model. RANDOM TOURNAMENTS A tournament between n players can be represented by a directed graph where an arc

RANDOM GRAPHS

(i, j) denotes that player i defeated player   n j. If the outcomes of the tournament’s 2 matches are chosen at random, the resulting digraph is called a random tournament. Random tournaments can serve as a model for statistical multiple-comparison∗ methods. A wide range of problems and results on random tournaments is presented in the monograph by Moon [18]. PERCOLATION ON LATTICES In a regular lattice suppose that we color edges (or vertices) white or black independently with probability p and q = 1 − p, respectively. Assume that a fluid spreads through this lattice, with the white edges (or vertices) interpreted as ‘‘open’’ and black as ‘‘closed.’’ With this interpretation, percolation theory∗ is concerned with the structural properties of such random lattices. See the monograph by Kesten [14]. STATISTICAL GRAPHS Statistical graph models deal with the situation when the structure of the initial graph (population graph) is unknown or partially unknown, but we are able to randomly observe its subgraphs (sample graphs). Generally speaking, statistical graph theory is concerned with the estimation of various characteristics of the population graph based on knowledge of its sample graphs. A comprehensive survey is provided by Frank [9]. REFERENCES ´ B. (1979). Graph Theory: An Intro1. Bollobas, ductory Course. Springer-Verlag, New York. ´ B. (1981). Combinatorics (Swansea, 2. Bollobas, 1981) Lond. Math. Soc. Lect. Notes Ser. 52, Cambridge University Press, Cambridge, England, pp. 80–102. ´ B. (1985). Random Graphs. Aca3. Bollobas, demic Press, London. ´ B. and Erd¨os, P. (1976). Math. Proc. 4. Bollobas, Camb. Philos. Soc., 80, 419–427. 5. Erd¨os, P. (1985). Random Graphs ’83, Ann. ´ Discrete Math., 28, M. Karonski and A. ´ Rucinski, eds. North Holland, Amsterdam.

6831

6. Erd¨os, P. (1973). The Art of Counting. MIT Press, Cambridge, Mass. (Selected writings, J. Spencer, ed.). 7. Erd¨os, P. and R´enyi, A. (1960). Publ. Math. Inst. Hung. Acad. Sci., 5, 17–61. 8. Erd¨os, P. and Spencer, J. (1974). Probabilistic Methods in Combinatorics. Academic Press, New York. 9. Frank, O. (1980). In Sociological Methodology, 1981, S. Leinhardt, ed. Jossey-Bass, San Francisco. 10. Grimmett, G. R. (1983). Selected Topics in Graph Theory 2, L. Beineke and R. Wilson, eds. Academic Press, London, pp. 201–235. 11. Grimmett, G. R. and McDiarmid, C. J. H. (1975). Math. Proc. Camb. Philos. Soc., 77, 313–324. ´ 12. Karonski, M. (1982). J. Graph Theory, 6, 349–389. ´ 13. Karonski, M. (1984). Balanced Subgraphs of Large Random Graphs. Adam Mickiewicz ´ Poland. University Press, Poznan, 14. Kesten, H. (1982). Percolation Theory for Mathematicians. Birkhauser Boston, Cambridge, Mass. 15. Kolchin, V. F. (1984). Random Mappings. Izd. Nauka, Moscow (in Russian). 16. Koml´os, J. and Szemeredi, E. (1972). Discrete Math., 43, 55–64. 17. Matula, D. W. (1972). Notices Amer. Math. Soc., 19, A–382. [See also ref. 21 (pp. 75–80).] 18. Moon, J. W. (1968). Topics on Tournaments. Holt, Rinehart, and Winston, New York. 19. Moon, J. W. (1970). Counting Labelled Trees. Canad. Math. Congress, Montreal, Quebec, Canada. 20. Mutafchiev, L. R. (1984). Proc. 13th Spring Conf. Union Bulg. Math., pp. 57–80. 21. Palmer, E. M. (1985). Graphical Evolution: An Introduction to the Theory of Random Graphs. Wiley, New York. ´ 22. R´enyi, A. (1976). Selected Papers, P. Turan, ed. Akad´emiai Kiad´o, Budapest. 23. Sachkov, V. N. (1978). Probabilistic Methods in Combinatorial Analysis. Izd. Nauka, Moscow, (in Russian). 24. Stepanov, V. N. (1973). Voprosy Kibernetiki. Izd. Nauka, Moscow, pp. 164–185. (In Russian.) See also CLUSTER ANALYSIS, GRAPH-THEORETIC; GRAPH THEORY; and PERCOLATION THEORY. ´ MICHAL KARONSKI DAVID W. MATULA

6832

RANDOMIZATION—I

RANDOMIZATION—I We discuss issues involved in arguments for and against randomization in the design of surveys∗ and comparative experiments and the use of randomization tests∗ in the latter. For brevity we focus on full-response surveys of fixed size obtained by random sampling without replacement from a finite population and comparing two treatments by a design related to the survey framework. In both cases the observations are supposed free of measurement error. Let U = {uf : f ∈ F} be an indexed set of N distinct units. The indexing set F is the sampling frame and a subset J of it is a sample. When x is a real-valued function on U, we write xf for x(uf ), x(J) for the mean of the J-sample x-values, and x for x(F), the population mean. The
J-sample variance of x is s2x (J) = (n − 1)−1 J [xf − x(J)]2 , where n > 1 is the size of J. As J runs equiprobably through Fn , the set of all samples of size n, E{s2x (J)} is the population variance σx2 = [(N − 1)/N]s2x (F), E{x(J)} is x, and var{x(J)} is [(N − n)/(Nn)]σx2 . If J is chosen at random from Fn , x(J) is an unbiased estimate of x and [(N − n)/(Nn)]s2x (J) is an unbiased estimate of its variance. The aim of a survey is usually seen as the estimation of the population quantities x, Nx from sample x-values. The preceding random sampling procedure seems to achieve this because the x(J) obtained then locates x with an estimated standard error [(N − n)/(Nn)]1/2 sx (J). This view is questioned by Basu [2,3,4], on the grounds that the likelihood principle∗ entails that the way the sample x-values are obtained plays no role in inference from them to the population quantity x. Before considering this objection, we discuss a comparative experiment open to the same criticism. Suppose that we wish to compare two treatments, A and B, when only one of them can be applied to a given unit and do so by applying A to the uj with j in the sample J and B to the uk with k in the sample K = F − J. Let the individual potential effects of A and B on uf be xf and yf . When J is chosen at random from Fn , the mean effects x, y are located by x(J), y(K) and their estimated standard errors. The mean treatment difference

 = x − y is located by (J) = x(J) − y(K), with a standard error that may be conservatively approximated by the sum of those of x(J) and y(K), because  var{(J)}


N−n Nn



1/2

n + N(N − n)

σx 2

1/2 σy

,

equality holding when x(J), y(K) are perfectly negatively correlated, for instance when xf − yf does not depend on f . Thus randomized design, with the A-treatment group determined by the random choice of J from Fn , gives an estimate of the error in taking the mean treatment difference between the samples J and K to be the differential treatment effect in the population U. The provision of such estimates of error was seen by Fisher∗ [14] as one of the principal aims of experimental randomization, and he argued they could not be obtained without it. Against this, Harville [17] asserts that randomization is no basis for inference because we know what sample J was realized when the experiment is carried out and just how it was obtained is irrelevant from the standpoint of inference conditional on that realization. This position is similar to that reached about surveys by Basu, whose argument we now consider. If J in Fn is the sample chosen in a survey, the data comprise the function xJ = {(j, xj ) : j ∈ J} with domain J. Let D(x) = {xJ : J ∈ Fn } be the set of possible data sets for samples from Fn when x is the population function. Let X be a finite set of feasible population functions x, and let D(X ) = ∪{D(x) : x ∈ X }. Consider a sampling procedure  over Fn yielding J in Fn with probability π (J) > 0. If ξ in D(X ) has domain J, then conditional on x in X , p(ξ |x), the probability that the observed data set is ξ , is π (J) for ξ = xJ and 0 otherwise. For a given ξ with domain J, the likelihood function on X [i.e., L(x|ξ ) = p(ξ |x)] is π (J) when xJ = ξ and 0 otherwise. If Xξ is the number of x in X with xJ = ξ , the normalized likelihood L(x|ξ ) is Xξ−1 when xJ = ξ , 0 otherwise. It does not depend on the sampling procedure . If one accepts the likelihood

RANDOMIZATION—I

principle, this means that  is irrelevant to inference about x from ξ . All that is relevant is the observed data set xJ . Basu [2] concludes that there is little, if any, use for randomization in survey design. Others, like Kempthorne [22], see this conclusion as so opposed to intuition that it casts doubt on the likelihood principle. A similar argument applies to the comparative experiment. Conditional on potential A and B treatment effects x and y, the probability p{(ξ , η)|(x, y)} that the observed A and B data sets are ξ and η, with domains J and K = F − J, is π (J) when ξ = xJ , η = yK , and 0 otherwise. The likelihood principle again suggests that randomization in design has no role in the analysis of such an experiment. This view is shared by Bayesians, who, accepting the likelihood principle, take the posterior density to be the normalized restriction of the prior density to the (x, y) with xJ = ξ , yK = η. Fisher’s arguments for randomization in refs. 14 and 15 have been defended and clarified by Greenberg [16], White [39], Easterling [9], and Kempthorne in the works cited below, but the issues remain controversial. Kruskal [29] sees Fisher’s own exposition as confusing. But it is arguable that Fisher’s critics, not understanding the practical motivation behind randomization, have been misled by what he himself saw as their mistaken emphasis on the deductive reasoning of pure mathematics instead of the inductive procedures of the practical scientist, [6, p. 256], a point also made by Kempthorne [26]. From a practical viewpoint, the effect one is talking about is (J), not , and the principal question of the comparative experiment is not inference about , regarded as an unknown parameter, but whether (J) is more appropriately said to be a treatment effect than a J effect or a general combined treatment—J effect, which might, as a special case, include a ‘‘treatment ×J’’ interaction∗ . If it is, the observed (J) is the treatment effect of the experiment and it is its magnitude and direction that have contextual meaning. If the A-treatment group consisted of the n oldest units, it might be said the treatments were confounded with age and that (J) is then a treatment–age effect. If the units are plots, the treatments manures

6833

and x, y crop yields, the practical question is whether the observed difference (J) can be attributed to the manures rather than to something else (e.g., greater soil fertility within A-plots). In practice, randomization in design is not directed to inference from (J) to  but to such possibilities of misnomenclature [13]. It has to do with obtaining samples J, K which hopefully will not exhibit features that might invalidate talking about the treatment-J effect (J) as if it were only a treatment effect. But it is often difficult to specify these features beyond the fact that even after blocking∗ , there is a possibility of confounding∗ with a few variables whose practical importance was not foreseen. Thus there is usually a relatively small unknown collection C of samples in Fn that one  wishes to avoid. If their number m  Nn and J is chosen at random from Fn , the probability that one of them is obtained is correspondingly small  [i.e., m/ Nn ] whatever C may be. This is the practical purpose of randomization in design. Whether it can be achieved in other ways is a moot point. When analyzing the experiment one will still need to enquire if randomization did avoid unwanted samples. Cornfield [8] is a good example of such an enquiry. Similar remarks apply to surveys. It is true that once J is known just how it was obtained does not alter the fact that all one knows about x is xJ , but it may have a bearing on whether xJ meets the aims of practical enquiry. An example of a potential design flaw that was quickly remedied because the practical aims would not have been met is noted in Brewer [7]. In the comparative experiment, x is known when y = x and (J) is then N[x(J) − x]/(N − n). Following Fisher [14] the distribution of this (J) as J runs through Fn has been used in the so-called randomization test∗ , whereby the hypothesis of no treatment effect, y = x, is rejected when |(J)| is improbably large. The rationale seems to be that it is preferable to hold y = x than concede that design randomization has led to an improbably extreme outcome, but the grounds for this preference remain obscure. Easterling [9] suggests that a descriptive interpretation might be more appropriate, and Finch [10] argues that such tests are rudimentary descriptive procedures

6834

RANDOMIZATION—I

that may be applied even in the absence of these design randomizations. Interest in randomization procedures was stimulated by the recovery from them of ANOVA∗ normal theory tests in general settings (Welch [38], Pitman [31], Kempthorne [18,19]). These ANOVA tests and a descriptive interpretation of standard error have also been recovered in a descriptive setting by Finch [11,12]. This calls into question the claim of Fisher [14] and Barbacki and Fisher [1] that systematic designs such as those suggested by Student [35,36] are ruled out because randomization is necessary to obtain valid estimates of error. Not only is such a claim difficult to sustain with inference conditional on the sample used but, more important from a practical viewpoint, these error estimates have a descriptive interpretation that does not depend on the way the sample was obtained. In summary, randomization is an easily reportable, reproducible, and unambiguous way of collecting data to meet specific aims of practical enquiry. But the analysis of data so collected is primarily descriptive, not inferential. For example, not randomizing in the comparative experiment might result in xJ , yK not meaning what we want them to mean in the performed experiment. But once it is performed, the practical question, in the absence of confounding, is whether the data set (xJ , yK ) is better described as one noisy constant on F or two noisy constants, one on J and one on K, and how sensitive this description is to data perturbation. This question may be investigated by the two-sample descriptive procedure of Finch [11,12]. REFERENCES 1. Barbacki, S. and Fisher, R. A. (1936). Ann. Eugen. (Lond.), 7, 189–193. 2. Basu, D. (1969). Sankhya A, 31, 441–454. 3. Basu, D. (1971). In Waterloo Symposium on Foundations of Statistical Inference. Holt, Rinehart and Winston, Toronto, pp. 203–234. 4. Basu, D. (1978). In Survey Sampling and Measurement, N. K. Namboodiri, ed. Academic Press, New York, pp. 267–339. 5. Basu, D. (1980). J. Amer. Statist. Ass., 75, 575–595.

6. Box, J. F. (1978). R. A. Fisher: The Life of a Scientist. Wiley, New York. 7. Brewer, K. R. W. (1981). Aust. J. Statist., 23, 139–148. 8. Cornfield, J. (1971). J. Amer. Medical Ass., 217, 1676–1687. (A detailed post hoc examination of the extent to which randomization had been successful in a large clinical trial.) 9. Easterling, R. G. (1975). Commun. Statist., 4, 723–735. 10. Finch, P. D. (1979). Biometrika, 66, 195–208. (Introduces procedures for assessing descriptions and argues that randomization tests are a rudimentary version of them.) 11. Finch, P. D. (1980). Biometrika, 67, 539–550. (Gives a descriptive interpretation of the usual ANOVA procedures that does not depend on the assumption of normality.) 12. Finch, P. D. (1981). Aust. J. Statist., 23, 296–299. (Gives a descriptive interpretation of standard error.) 13. Finch, P. D. (1982). Aust. J. Statist., 24, 146–147. 14. Fisher, R. A. 503–513.

(1926).

J. Min.

Agric.,

33,

15. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh. 16. Greenberg, B. G. 309–322.

(1951).

Biometrics,

7,

17. Harville, D. A. (1975). Amer. Statist., 29, 27–31. 18. Kempthorne, O. (1952). The Design and Analysis of Experiments. Wiley, New York. (One of the few texts to include a detailed discussion of randomization in design.) 19. Kempthorne, O. (1955). J. Amer. Statist. Ass., 50, 946–967. 20. Kempthorne, O. (1966). J. Amer. Statist. Ass., 61, 11–34. 21. Kempthorne, O. (1969). In New Developments in Survey Sampling, N. L. Johnson and H. Smith, Jr., eds. 22. Kempthorne, O. (1971). Contribution to Rao [32]. 23. Kempthorne, O. (1975). In A Survey of Statistical Design and Linear Models, J. N. Srivastava, ed. North-Holland, Amsterdam, pp. 303–331. 24. Kempthorne, O. (1977). J. Statist. Plann. Inf., 1, 1–25. 25. Kempthorne, O. (1978). In Contributions to Survey Sampling and Applied Statistics: Papers in Honor of H. O. Hartley, H. A. David, ed. Academic Press, New York, pp. 11–28.

RANDOMIZATION—II 26. Kempthorne, O. (1980). Contribution to Basu [5]. 27. Kempthorne, O. and Doerfler, T. E. (1969). Biometrika, 56, 231–248. 28. Kempthorne, O. and Folks, J. L. (1971). Probability, Statistics, and Data Analysis. Iowa State University Press, Ames, Iowa. 29. Kruskal, W. (1980). J. Amer. Statist. Ass., 75, 1019–1029. 30. Pearson, E. S. (1939). Biometrika, 31, 159– 179. 31. Pitman, E. J. G. (1937). Biometrika, 29, 322– 335. 32. Rao, C. R. (1971). In Waterloo Symposium on Foundations of Statistical Inference. Holt, Rinehart and Winston, Toronto, pp. 177–190. 33. Smith, T. M. F. (1976). J. R. Statist. Soc. A, 139, 183–204. (A useful review of the theory of surveys.) 34. Student (1923). Biometrika, 15, 271–293. 35. Student (1936). J. R. Statist. Soc. Suppl., 3, 115–122. 36. Student (1936). Nature (Lond.), 88, 971. 37. Thornett, M. L. (1982). Aust. J. Statist., 24, 137–145. 38. Welch, B. L. (1937). Biometrika, 29, 21–52. 39. White, R. F. (1975). Biometrics, 31, 552–572. See also ANALYSIS OF VARIANCE; BAYESIAN INFERENCE; CONFOUNDING; DESIGN OF EXPERIMENTS; LIKELIHOOD PRINCIPLE; and RANDOMIZATION TESTS.

PETER D. FINCH

RANDOMIZATION—II Consider a given population of experimental units (e.g., plots of land, children of age 6, trees, mice). These units—children, trees, and mice—alter over time, and the dynamics over time is influenced by various factors (e.g., nutrition and environment). Suppose that we wish to modify the dynamics as, for instance, that we wish children of age 6 to grow more rapidly than they have in the past. Suppose also that we have acts of intervention, such as defined supplementations of diet or instantaneous or semicontinuous or continuous treatment with drugs of various sorts. In attempting to assess the effects of modes of intervention, we have to call on all the scientific opinion that is available. We shall find that the general system we are

6835

looking at is so complex and the background of partial knowledge is so weak that we are forced to do a comparative experiment. Name the various interventions we wish to compare with the word ‘‘treatments.’’ Then we have to apply each treatment to a subset of our population of units. Necessarily, each unit can receive only one of the treatments. Suppose that we have 30 units and wish to compare six treatments, with equal attention to them. Then we have to select five units to receive any one of the treatments. This is obvious, and the only design problem in performing the comparative experiment is to have a process of assignment of treatments to units. The principle of randomization in its simplest form is merely that we partition the 30 units into six subsets, each of five units, at random, and then assign each one of the treatments to one of the subsets. The implementation of this prescription can be done in various ways which will not be discussed. Having decided on the assignment of treatments to units, we then perform the study with the resulting treatment–unit combinations, and observe the chosen outcome variables at the end of the experimental period. The design so induced is called a completely randomized design. If we surmise that subsets of the original 30 units would behave differently, we would consider a different scheme of treatment allocation. The randomized block design for t treatments is based on the surmise that in the totality of available units, there are disjoint subsets of t units, such that variability within subsets in the outcome variables under the same treatment would be small. If we surmise that there are subsets of size k (e.g., if we are experimenting with mice that come to us in litters of, say, 4 = k), we will adopt a scheme of random allocation that utilizes the surmise. We shall then have, if k < t, some sort of randomized incomplete block design, and if k > t, some sort of randomized super-complete randomized block design. Suppose that we have t2 units on which we wish to compare t treatments, and the t2 units may be classified into a t × t two-way array with one unit in each cell of the array. Then we impose on the twoway array a random t × t Latin square∗ , and associate each treatment with one letter of the Latin square. This yields a (randomized)

6836

RANDOMIZATION—II

Latin square design. If we classify the units by two factors, one nested by the other, we obtain a split-plot design∗ . The ideas in randomized experiments correspond closely to those used in finite sampling of a finite population∗ . So, for example, the completely randomized design is analogous to simple random sampling, the randomized block design to stratified sampling, and the randomized Latin square design to deep sampling. There is, however, one very critical difference between randomization in pure sampling and randomization in comparative experiments. In the latter, we wish to compare treatments, whereas in the former, there is, in a certain sense, only one treatment. Also, in basic sampling, there is no experimental period and no dynamics over time. In both sampling∗ and experimental design∗ , use is made of surmises about the variability among units in the population of units, that is, of a process of informal empirical Bayesian∗ thinking. In the case of the randomized block design, the Latin square design, and the split-plot design, for instance, the surmises are categorical, and the randomization imposes a classificatory structure that is used to restrict the randomization. The places of the units in this classificatory structure is ‘‘concomitant information.’’ We may extend the ideation to a more general situation in which the concomitant information is not solely classificatory. A classical procedure, on which there has much controversy over the past five decades, is to use an idea of balance∗ . One attempts to choose a partition of the units into treatment subsets that are closely alike with respect to the concomitant variables. In the Latin square case controversy arose on the use of what are called systematic squares, the idea being that the subsets of units which receive the different treatments should appear to be balanced relative to each other with respect to position in the square. The classical example of this is the Knut-Vik square∗ . A simpler example occurs with experimentation with units that occur on the time line, as with successive experimental periods of a chemical reactor. Consider experimentation ‘‘on a line.’’ Then concomitant information on the unit is the position of the unit on the line. Let X1j , j = 1, 2, . . . , r, be the positions

of the units that receive treatment 1, with X2j , . . . , Xtj having corresponding meanings. Then a natural way to make the assignment is to require ave(j) {Xij } to be as nearly the same as possible for the various values of i. This could result in the units of some treatments being closer together than the units with other treatments. So one might then also require avej (Xij2 ) to be as constant as possible. This would give a partition of the units that is somewhat balanced with respect to position on the line. One would then assign the subsets of the partition at random to the treatments. This process is thought to be desirable because it would lead to treatment differences with smaller error than would occur without the balancing. It had very great appeal before Fisher [6] and Yates [14] wrote about it. Their criticisms arise from a simple question: Given that one has used a systematic design of the type outlined, how is one to assess the statistical significance of whatever differences in treatment means one observes, and how is one to obtain standard errors for treatment differences? One would, according to the Fisher [6] discussion, use a conventional linear model with treatment effects entering additively and with a conventional linear form in the concomitants or covariates: that is, one would use the ordinary analysis-of-covariance∗ process. As Fisher [6] describes, such a procedure will cause an overestimation of the variance under randomization of differences of treatment means. This would have the consequence that even in the presence of actual treatment effects, one would conclude that there is no evidence for treatment effects. Fisher [6] and Yates [14] wished to achieve (a) unbiased estimation of treatment effects and unbiased estimation of the error of estimated treatment effects; and (b) validity of the test of significance of the null hypothesis of no treatment effects or of any hypothesized set of treatment effects. To examine these, we have to envisage a population of repetitions. Fisher∗ showed in the case of the Latin square design that if one’s population of repetitions is the totality of possible Latin squares, then with fixed (but unknown) unit values and no treatment effects, the expectation over the randomization set of the mean square for treatments is equal to the

RANDOMIZATION—II

expectation of the residual mean square if one uses the conventional analysis of variance. This property was called ‘‘unbiasedness of design’’ by Yates and holds for the common designs. If additivity holds, in that the observation obtained is equal to a unit effect plus a treatment effect plus purely random measurement error, then one obtains unbiased estimation of error of estimated treatment effects for the standard designs by conventional least squares (for proofs, see Kempthorne [7]. To address tests, we have to decide what inferential process we shall follow when we have obtained the results of the experiment. The elementary and natural process that is almost uniformly presented in teaching texts is to apply the model yi = xi β + τ t(i) + ei , with the error vector e being multivariate normal∗ with null mean vector and variance matrix equal to σ 2 I, with the associated tests. This is altered, say, for a split-plot design with respect to error by assuming that the whole of the error consists of two parts, one part being common to several observations and the other unique to each observation. This error assumption is patently unreasonable in most experimental contexts. It amounts to assuming that the units we use are a random sample from some population of units that we could have used. Because we are interested in comparing the several treatments, we shall and should attempt to obtain a set of units that are as alike as possible. Then, clearly, the set of units we use will be a correlated subset of any ‘‘whole’’ population that is available. The way out of the difficulty was given by Fisher, namely, the use of randomization tests. Suppose that we have the results of the experiment. Then if there are no treatment effects, our actual data set is the same as we would get with any of the different plans we might have used. We may then superimpose each of the different plans on the observed data set, compute for each such imposition the value of treatment-determined criterion, and obtain the randomization distribution of the criterion. We place the observed value of the criterion in this randomization distribution and calculate the probability under randomization that we would have obtained a criterion value equal to or greater than that actually obtained. This probability is then

6837

called the significance level (SL) of the test of the null hypotheses of no treatment differences. The property of this process that is thought, by some, to be very valuable is: Pr(SL
α|null hypotheses) = α, if α is an achievable level. This ideation was exposited by Kempthorne [7–10]. If we wish to examine the nonnull hypotheses that treatment j gives a result greater than treatment 1 by δj (j = 2, 3, . . . , t), then we merely substract δj from the observations under treatment j and make a randomization test of the null hypothesis on the adjusted data. The idea of sampling the randomization distribution was given first by Eden and Yates [3], was examined mathematically by Dwass [2], and exemplified for complex multivariate responses by Cox and Kempthorne [1]. It is now a routine way of testing hypotheses on treatment effects. Edgington [4] describes randomization tests for simple experiments. The next step in ideation counters the problems that are associated with the fact that almost any randomization scheme will produce plans that are unreasonable from an intuitive viewpoint; for example, with six units ‘‘on a line’’ and two treatments A and B, a possible plan with the completely randomized design is AAABBB, which is ‘‘unreasonable.’’ The new idea is that one considers, say, the completely randomized design, which gives, say, M possible plans before treatment randomization. One looks at each of the M plans, perhaps intuitively or perhaps with computations, using ideas of distance between subsets, and one rejects some of the plans as ‘‘unreasonable.’’ Suppose that we delete these plans from the test and are left with M1 plans as ‘‘reasonable.’’ Then we pick at random one these M1 plans and assign treatments at random to the treatment symbols in the chosen plan. After obtaining the response data, we use the subset of M1 plans for randomization tests on treatment differences. Ordinarily, one would like M1 to be at least 100 (if this is possible) in order to enable the experiment to give significance at a level no greater than 0.01, if there are in fact large treatment effects. The generality of the prescription is clear. The undoubted obscurity in the whole randomization choice of plan and then test evaluation is the choice of criterion that we are

6838

RANDOMIZATION—II

to use to measure deviation from the null hypothesis of no treatment differences. The commonly used process is to apply ordinary linear model theory with independent and homoscedastic error to obtain a criterion. Such a process can be applied regardless of the nature of the concomitant variation. Another aspect of experiment analysis is estimation of treatment effects. This requires assumption of a model for treatment effects, and the natural one is to assume additivity, as explained earlier. Let δji = 1 if treatment j falls on unit i and 0 otherwise. Then the observed mean of r observations on treatment j is 1 i 1 i δj (ui + tj ) = tj + δj ui . r r i

i

If Pr(δji = 1) = 1/t, as occurs with randomization of any equi-replicate plan with respect to treatments, the expectation of this is tj + u, u being the average of unit values. So we then have unbiasedness of treatment means in the sense that the average for treatment j under repetitions, of which we have only one, is what we would observe if we placed it on all the units. Under the additivity assumption the difference of treatment means is    1  i δj ui − δji ui . tj − tj + r i

i

In the case of the simple balanced designs, we find that variances of estimated treatment differences are given by the use of the simple linear model under special circumstances, as exposited by Kempthorne [7]. There are, however, unresolved obscurities in that to obtain intervals of uncertainty for a parameter θ , given an estimate θˆ and a so-called standard error of θˆ (i.e., estimated ˆ SE(θˆ ), we need to standard deviation of θ), ˆ know as a fact that (θ − θ )/SE(θˆ ) is a pivotal quantity with a known distribution (e.g., a tdistribution). From this we should infer that interval estimation is just a particular form of a significance test∗ —an inversion thereof. We accomplish our aim by the use of the randomization test∗ procedure. The process that is often followed is to use randomization, then to verify that the randomization

distribution of a criterion is like the distribution of that criterion under a normal law (or infinite model) model assumption in what are thought to be critical respects, and finally to use the normal law theory of error as an approximation to the actual randomization distribution of criterion (see Pitman [11] and Welch [12]). The idea of incorporating design random variables, incorporating the randomization that is used, to lead to derived models and expected values of mean squares in ordinary analysis of variance∗ has been described by Kempthorne [7] and by Wilk and Kempthorne [13] for basic designs. The modern view held by supporters of randomization ideas is that the randomization of design and the use of randomization tests on the preselected class of possible designs should be viewed as an integral whole. Any relation to conventional analysis of variance associated with linear models is, in general, fortuitous. Other modes of inference (e.g., Bayesian∗ ) will not be discussed. There appears to be no justification for randomization ideas in such inferences. If one follows the ideas of randomization and randomization test inference, it is clear that the inferences obtained may be associated unambiguously with the population of experimental units that is used in the experiment. If we have a large population of experimental units and select (e.g., at random) a subset of that population, then extension of the statistical test from the set of units that is used to the totality of available units in the whole population can be justified by the randomization test argument, using a simple conditional argument. So one can use randomization arguments to justify a (partial) inference to the set of experimental units that is actually used or sampled. Extension of such an inference to some larger population [e.g., from an experiment on 6-year-old children in 1950 to all 6-year-old children of the United States in 1950, or to the (nonexistent in 1950) population of 6-year-old-children in the United States in 1990], is a matter of substantive judgment, which can be aided by a statistical formulation but cannot be justified by substantive reasoning. It follows that randomization with randomization inference is not a panacea.

RANDOMIZATION—II

The ideas of randomization tests, based on the experiment randomization used, were difficult to apply in most real experimental situations, because of the necessity of considering each of the possible realizations. Analysis of variance was easy computationally, with easy significance testing if one used the assumption that errors are independent, homoscedastic and Gaussian, that is, ‘‘normal law theory’’ or ‘‘infinite model theory’’ (Kempthorne, [7]). On the basis of the work of Pitman [11] and Welch [12], it seemed that randomization distributions of test statistics can be well approximated by such normal law theory. This was the justification of normal law theory used by Kempthorne [7]. With the advent of high-speed computation, the need to call on such approximation decreased and exact randomization testing of null hypotheses could be done. However, even today, a more complete inference to give exact confidence intervals requires massive computation, as do considerations of power of tests. Curiously, it is informal experience that the approximation is good in many circumstances, but it can be very bad, in that the set of achievable levels by the randomization test can be very small and bounded away from zero. In the case of the 3 × 3 Latin square design, the only achievable levels are 1 and 12 . The ideas of experiment randomization and associated randomization tests must be clearly distinguished from testing techniques based merely on random relabeling of observations, as exposited by Finch [5]. Such techniques question whether an observed labeling is significant within a chosen class of relabelings. Such procedures may give a significance level that is the same as would result from a randomized experiment with a set of possible plans the same as the set allowed for in the population of relabelings. The ideas of randomized design and analysis are aimed, clearly, at the proving of causation∗ by intervention, in contrast to random relabeling, which may be termed purely data analytic. The distinction between randomization tests and permutation tests∗ is important. The latter are based on the assumption of random sampling, an assumption that is often patently false or unverifiable, even though

6839

necessary to make an attack on the substantive problem being addressed. Acknowledgment This article is Journal Paper No J-11257 of the Iowa Agriculture and Home Economics Experiment Station, Ames, Iowa. Project 890.

REFERENCES 1. Cox, D. F. and Kempthorne, O. (1963). Biometrics, 19, 307–317. 2. Dwass, M. (1957). Ann. Math. Statist., 28, 183–187. 3. Eden, T. and Yates, F. (1933). J. Agric. Sci., 23, 6–16. (The first Monte Carlo examination of a randomization distribution.) 4. Edgington, E. S. (1980). Randomization Tests. Marcel Dekker, New York. (Exposition of computational procedures.) 5. Finch, P. D. Biometrika, 66, 195–208. 6. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh. (The basic ideas of randomization, but incomplete because of reversion to infinite model theory.) 7. Kempthorne, O. (1952). The Design and Analysis of Experiments. Wiley, New York. (Reprinted by Krieger, Huntington, N.Y.) (Theory of randomization and infinite model analysis of experiments.) 8. Kempthorne, O. (1955). J. Amer. Statist. Ass., 50, 946–967. (Exposition of inference based solely on randomization.) 9. Kempthorne, O. (1975). In A Survey of Statistical Design and Linear Models. NorthHolland, Amsterdam. (Randomization ideas and obscurities in Fisher’s writings.) 10. Kempthorne, O. (1977). J. Statist. Plann. Inf., 1, 1–25. (General basic discussion.) 11. Pitman, E. J. G. (1937). Biometrika, 29, 322–335. (Approximation of randomization distributions.) 12. Welch, B. L. (1937). Biometrika, 29, 21–52. (Approximation of randomization distributions.) 13. Wilk, M. B. and Kempthorne, O. (1956). Ann. Math. Statist., 27, 950–985. (Linear models based on randomization ideas.) 14. Yates, F. (1933). Empire J. Experimental Agriculture, 1, 235–244. See also ANALYSIS OF VARIANCE; DESIGN OF EXPERIMENTS; INFERENCE, STATISTICAL; and RANDOMIZATION TESTS.

OSCAR KEMPTHORNE

6840

RANDOMIZATION, CONSTRAINED

RANDOMIZATION, CONSTRAINED When, as in agricultural experiments, the experimental units have a spatial, or even temporal, layout, there is always the possibility that ordinary randomization∗ may produce a plan where the pattern of the treatments accidentally corresponds closely to the underlying spatial pattern of the plots. For example, if there were 12 plots side by side in a long strip and four treatments to be replicated three times each, ordinary randomization might give a plan such as A A

A B B

B C C

C D D D. (1)

The chance of obtaining such a plan is 1/ 15,400, which is not high, but neither is it negligibly small. The sheer number of experiments that are conducted each year make it certain that plans like (1) will be used from time to time if ordinary randomization is properly used. However, many experimenters will be unhappy about using plans like (1), for several reasons. The first is the fear that, even in the absence of treatment effects, the yields might exhibit a pattern corresponding to the linear layout. This pattern may consist of either or both of two parts: (a) a fixed trend, such as a fertility trend along the length of the experimental area; and (b) inequality of correlations, so that, for example, yields on neighboring plots are more highly correlated than those on nonadjacent plots. For simplicity, we shall refer to both of these as plot pattern. Although part of the purpose of randomization is to allow for our ignorance about point (b), by ensuring equality of correlations on the average, many people will find plan (1) too extreme, too far from average, for randomization-based inference to be applicable. The second reason is a similar worry about competition or interference between neighboring treatments. If one variety of wheat germinates early, it may steal nutrients from neighboring plots, or grow tall and shade its neighbors. In fungicide and insecticide experiments, a treatment giving poor control can severely affect its neighbors, as pests or disease may multiply in the poorly controlled

plot and then spread to neighboring plots. If plan (1) is used and treatment C provides poor control, then treatment A has an advantage over B and D. Such treatment interference is a quite different problem from plot pattern, although the two are often confused. A third reason is more pragmatic, but nonetheless important. Randomization must be seen to be random. No matter how objectively the randomization is carried out, a plan like (1) will always cause some people to think it systematic and unfair. Suppose that plan (1) is used in a variety trial, and variety D fails to perform well enough to gain inclusion in the national list of recommended varieties. The breeder of variety D may then complain that his variety was unfairly allotted to the worst part of the field; and it may be hard to refute this. Even a statistician’s colleague—a scientist or a field worker—faced with a plan like (1) may question the statistician’s judgment, and so may decide to alter the plan, or other aspects of the experiment, without consulting or informing the statistician, possibly destroying months of careful planning. A plan like (1) is also a poor guard against crop damage by disease or wildlife, which tends to occur in patches. If two or three of the D plots in plan (1) have their crop destroyed by disease, it is impossible to tell whether this is an unfortunate accident or whether variety D is particularly susceptible. If there are reasons to believe the former, then it is legitimate to use missing value techniques (see INCOMPLETE DATA) or to fit an extra model term for disease presence, but in either case the comparison of D with the remaining varieties is either impossible (if all three plots are lost), or has very low precision. SOLUTIONS The most obvious solution to the problem of undesirably patterned plans like (1) is simply to reject them and rerandomize. Since this destroys the exact justification for the form of the analysis of variance, this is not satisfactory unless the proportion of plans to be rejected is very small. Moreover, these must be specified in advance, for it is easy

RANDOMIZATION, CONSTRAINED

to find undesirable patterns in almost any plan if it is scrutinized long enough; and post hoc rejection raises doubts about objectivity and fairness. Until they are required to specify them in advance, many experimenters do not realize just how many plans they would reject. For example, in a trial in which each of eight people measured each of four sows twice, the experimenter initially requested that all plans be rejected which entailed any person measuring the same sow twice in succession. With ordinary randomization, this would have entailed rejecting over 99.98% of plans. In many cases blocking∗ provides a better solution. For example, plan (1) could be avoided by dividing the strip into four blocks of three plots each, and insisting that each block contain every treatment. If blocks are chosen carefully, and if the subsequent analysis is modified to take account of the blocks, many of the problems of bad patterns simply disappear. However, blocking is not always sufficient. Table 1 shows a plan for a 24 × 4 factorial experiment∗ . The square layout was blocked into eight rows and eight columns, and ordinary randomization was used on a design confounding∗ high-order interactions∗ with rows and with columns. Nevertheless, the main effect of potash (levels indicated by presence or absence of the symbol k) was completely confounded with the contrast between pairs of diagonally opposite corners of the square. Moreover, blocking may not be desirable if there are few plots. In a trial comparing two fungicides on eight large plots in a 2 × 4 layout, the only way to avoid plans such as (2) to (4) by blocking is to consider both the two rows and the four columns as

blocking systems, which reduces the number of degrees of freedom for error from 6 to 2; this is quite inadequate for meaningful conclusions. A A A A B B B B

(2)

A A B B A A B B

(3)

A B B A A B B A

(4)

A third suggestion is to fit one or more covariates to account for positional effects. This is a reasonable solution if suspected plot pattern is the basic objection to certain plans, but is not without difficulty. Some treatment effects may be more confounded with the covariate than others, so that effects are estimated with different precision. There are problems of randomization theory in the presence of covariates (see Cox [8,9]). Modern computers∗ make short work of the arithmetical problems of the analysis of such nonorthogonal designs, but their programming needs care, and there may be difficulties in interpreting such an analysis to a nonstatistical client. However, a major problem in using a covariate is that it may give a poor representation of the plot pattern: for example, a linear covariate for plot number would not be helpful if the plot pattern had a curved smooth trend or a clumpy arrangement. Recent work [18] on nearest-neighbor models attempts to take account of general, but unknown, underlying smooth trends, but the theory is not yet sufficiently worked out for any general methods to be recommended. A more serious case

Table 1. Plan for 24 × 4 Factorial Experiment on Beansa 2 np

0 dp

1

3 dn

0 nk

2 dk

1 dnpk

3 pk

0 dnp

2p

3d

1n

2 dnk

0k

3 npk

1 dpk

1d

3n

2 dnp

0p

3 dpk

1 npk

2k

0 dnk

3

1 dn

0 np

2 dp

1 pk

3 dnpk

0 dk

2 nk

0 pk

2 dnpk

3 nk

1 dk

2

0 dn

3 dp

1 np

3 dnk

1k

0 dpk

2 npk

1 dnp

3p

0n

2d

1 nk

3 dk

2 pk

0 dnpk

3 np

1 dp

2 dn

0

2 dpk

0 npk

1 dnk

3k

0d

2n

1p

3 dnp

a 0,

6841

1, 2, 3, borax at four levels; d, farmyard manure; n, nitrochalk; p, superphosphate; k, muriate of potash.

6842

RANDOMIZATION, CONSTRAINED

against covariates is that they are inadequate to deal with the other three objections to bad patterns: they are (a) insufficient to ensure that the experiment is seen to be fair; (b) incapable of preventing patches of disease coinciding with patches of a single treatment; and (c) quite inappropriate for the problem of treatment interference. What is required is a method of randomization which ensures that bad plans cannot occur, while guaranteeing that the usual analysis of variance is still valid. Constrained randomization is the term used for such methods. Constrained randomization is often used in conjunction with blocking; the experiment is analyzed in the normal way appropriate to the given block structure, which usually permits straightforward interpretation of results. THEORY AND METHODS The main justification for constrained randomization is that randomization dictates, or at least validates, the analysis (see ref. 1). This is true whether randomization is done by randomly choosing a plan from a set of plans, by randomly labeling the treatments, by randomly labeling the plots, or a combination of these methods (see ref. 16). For simplicity, we concentrate on the first method. Fisher∗ [12] argued that the justification for the usual analysis of a completely randomized design is that, if a plan is picked at random from among all those with the correct replications of the given treatments, the probability that a specified pair of plots are both allotted the same treatment is independent of the pair of plots. But the same is true of several incomplete randomization schemes, such as the set (5) of plans for three treatments in a linear strip of six plots. Therefore, this incomplete randomization scheme validates the same analysis of variance as complete randomization. Moreover, the highly undesirable plan AABBCC can never occur. For strong validity [14], that is, to ensure that all normalized treatment contrasts are estimated with the same variance, one must also randomly allocate the actual treatments to the letters A to C. ABBACC ABACCB ABCBAC ABCCBA AABCBC

(5)

Similarly, Fisher [12] showed that the analysis of variance of a complete block design∗ depends only on the fact that the probability (over the randomization) of a pair of plots in different blocks having the same treatment is independent of the pair of plots. Likewise, for a Latin square design, all that is necessary is a constant probability of two given plots in different rows and columns receiving the same treatment: this may be achieved by taking as the randomization set one or more complete sets of mutually orthogonal Latin squares∗ [11]. Fisher’s justification of these observations was brief, and they have been rediscovered by many subsequent authors. A randomization set is said to be valid if it has the required property of constant probability on suitable pairs of plots. Methods that have been found for generating valid randomization sets include the use of combinatorial objects, such as balanced incomplete block designs∗ and Latin squares∗ , the theory of D-optimal designs, and 2-transitive permutation groups. These methods need specialist knowledge, so we shall not go into details here, but present some results (see also ORTHOGONAL DESIGNS). Unfortunately, there is no single recipe for constrained randomization. The undesirable plans vary from experiment to experiment; they depend on the shape and size of the experimental area and on the nature of the treatment structure, whether factorial or not, whether quantitative or qualitative, and, if quantitative, on the precise levels. The experimenter must specify which plans would be undesirable; the statistician can then test the known methods for generating valid randomization sets to see if there is a set which omits all the undesirable plans. If there is not, the experimenter may have to weaken or modify the criteria of desirability. EXAMPLES No Blocking Youden [22] showed that a valid randomization set for t treatments equireplicated in tr plots with no blocking is equivalent to a resolvable balanced incomplete-block design (BIBD) for tr treatments in blocks of size r. The treatments of the BIBD correspond to

RANDOMIZATION, CONSTRAINED Table 2. I

II

Replicate III

IV

V

14 23 56

13 26 45

15 24 36

16 25 34

12 35 46

randomization scheme is needed, all of whose plans are quasi-complete Latin squares. Bailey [4] showed how this could be done when the number of treatments is a power of an odd prime number. For example, for seven treatments the sequence (6) may be used. 0 6

the original plots, and each replicate of the BIBD gives a plan in the randomization set: each block of the replicate gives the positions for a single treatment in the original design. Table 2 shows a resolvable BIBD for six treatments in blocks of size two: this yields the randomization set (5), with the plans in the same order as the replicates. Complete Blocks Suppose that a 3 × 3 factorial experiment is to be conducted in linear blocks of size 9. Within any block, it is undesirable for any three consecutive plots to receive the same level of either treatment factor (called A and B, say) or of either component of their interaction, AB or AB2 . It is also undesirable for the four plots at either end of the block to contain all three occurrences of any level of A, B, AB, or AB2 . A valid randomization scheme that avoids these patterns consists of randomly allocating A and B to two different rows of Table 3, and then randomizing the levels of each factor independently. Latin Squares Freeman [13] introduced quasi-complete Latin squares to balance the effects of treatment interference in two dimensions, assuming an east neighbor to have the same effect as a west neighbor, and similarly for north and south. Each treatment must have each other treatment as a row neighbor exactly twice, and similarly for columns. Ordinary randomization of Latin squares would destroy this neighbor balance, so a constrained Table 3. 0 0 0 0

1 0 1 1

0 1 1 2

2 1 0 1

0 2 2 1

2 0 2 2

2 2 1 0

1 1 2 0

1 2 0 2

6843

1 5

2 4

3

(6)

Choose a random permutation of the form x → ax + b

modulo 7;

apply this to the sequence (6) and write the result along the top border of the square. Repeat the process, this time writing the result down the left-hand border of the square. Now the entry in any plot is obtained by adding the two corresponding border entries modulo 7. For example, if we choose the permutations x → 2x + 1 and x → 5x + 4, respectively, we obtain the layout in Table 4. Other Shapes If there are three treatments replicated three times in a square layout, it is unfortunate if one treatment occurs entirely in one row, in one column, or in one corner, as in plans (7) (a) to (c), respectively. AAA ABA BAA BCB CBC CBB CCB ABC CCA (a) (b) (c)

(7)

Two-way blocking would avoid these but would seriously reduce the number of error degrees of freedom. A valid constrained randomization set that avoids both these problems is given in Table 5: one plan should be chosen at random, and then the treatments should be randomly allocated to the letters A to C. The 2 × 4 layout is discussed in ref. 5. Table 4.

4 6 2 1 0 3 5

1

6

3

4

5

2

0

5 0 3 2 1 4 6

3 5 1 0 6 2 4

0 2 5 4 3 6 1

1 3 6 5 4 0 2

2 4 0 6 5 1 3

6 1 4 3 2 5 0

4 6 2 1 0 3 5

6844

RANDOMIZATION, CONSTRAINED

Table 5. AAB CAB CBC

ABA CCB ACB

ABC ACA BBC

ACC CBB BAA

HISTORY Yates [20] first drew attention to the problem of bad patterns, citing an experiment done at Rothamsted Experimental Station [19], whose plan is given in Table 1. The solution he suggested was constrained randomization, which he called restricted randomization; this name is still used in the United Kingdom. Grundy and Healy [14] found a valid restricted randomization scheme for Yates’s particular example; they also gave the scheme in Table 3 for a 3 × 3 experiment in linear blocks of size 9. They showed the importance of 2-transitive permutation groups, and how such groups could be combined for a simple crossed layout. Dyke [10] extended their ideas to deal with linear blocks of 16 plots. Independently of the British work, Youden [21,22] considered the problem of bad patterns in linear layouts, and advocated constrained randomization. He showed how to obtain valid randomization sets from resolvable balanced incomplete block designs. He and Kempthorne [15] used this method to find good restricted randomization schemes for three and four treatments in linear arrays of six and eight plots, respectively. More recently, White and Welch [17] have introduced D-optimal design theory into constrained randomization, while Bailey [2] has extended the use of permutation groups, giving a catalog of valid restricted randomization schemes for several common situations. The theory in ref. 7 shows how 2-transitive permutation groups may be combined to give valid restricted randomization schemes for more complicated block structures, such as those described in NESTING AND CROSSING IN DESIGN. REFERENCES 1. Bailey, R. A. (1981). J. R. Statist. Soc. A, 144, 214–223.

2. Bailey, R. A. (1983). Biometrika, 70, 183–198. (Includes a catalog of restricted randomization schemes for linear blocks of up to 32 plots.) 3. Bailey, R. A. (1983). In Recent Trends in Statistics, S. Heiler, ed. Vandenhoeck und Ruprecht, G¨ottingen, pp. 9–26. (Gives more detail of the technical arguments.) 4. Bailey, R. A. (1984). J. R. Statist. Soc. B, 46, 323–334. (Randomization of quasi-complete Latin squares.) 5. Bailey, R. A. (1985). Int. Statist. Rev., 53, 171–182. (Restricted randomization in place of losing degrees of freedom by blocking.) 6. Bailey, R. A. (1985). Statistics and Decisions, Suppl., 2, 237–248. (Problems when randomization is constrained to preserve neighborbalance.) 7. Bailey, R. A., Praeger, C. E., Rowley, C. A., and Speed, T. P. (1983). Proc. Lond. Math. Soc., 47, 69–82. 8. Cox, D. R. (1956). Ann. Math. Statist., 27, 1144–1151. (Randomization in the presence of covariates.) 9. Cox, D. R. (1982). In Statistics and Probability: Essays in Honor of C. R. Rao, G. Kallianpur, P. R. Krishnaiah, and J. K. Ghosh, eds. North-Holland, Amsterdam, pp. 197–202. 10. Dyke, G. V. (1964). J. Agric. Sci., 62, 215–217. (Restricted randomization for linear blocks of 16 plots.) 11. Fisher, R. A. (1935). J. R. Statist. Soc. Suppl., 2, 154–157. 12. Fisher, R. A. (1966). The Design of Experiments. Oliver & Boyd, Edinburgh (1st ed., 1935). 13. Freeman, G. H. (1979). J. R. Statist. Soc. B, 41, 253–262. 14. Grundy, P. M. and Healy, M. J. R. (1950). J. R. Statist. Soc. B, 12, 286–291. (The first real progress in restricted randomization, including the introduction of permutation groups.) 15. Kempthorne, O. (1961). In Analysis of Variance Procedures, ARL 149, by O. Kempthorne, G. Zyskind, S. Addelman, T. N. Throckmorton and R. F. White. Aeronautical Res. Lab., Office of Aerospace Res., U.S. Air Force, Wright-Patterson Air Force Base, Ohio, pp. 190–202. 16. Preece, D. A., Bailey, R. A., and Patterson, H. D. (1978). Aust. J. Statist., 20, 111–125. 17. White, L. V. and Welch, W. J. (1981). J. R. Statist. Soc. B, 43, 167–172. (Restricted randomization using D-optimal design theory.)

RANDOMIZATION TESTS 18. Wilkinson, G. N., Eckert, S. R., Hancock, T. W., and Mayo, O. (1983). J. R. Statist. Soc. B, 45, 151–178. 19. Yates, F. (1937). The Design and Analysis of Factorial Experiments, Imp. Bur. Soil Sci. Tech. Commun., 35. 20. Yates, F. (1948). J. R. Statist. Soc. A, 111, 204–205. (Introduction of the problem.) 21. Youden, W. J. (1964). Technometrics, 6, 103–104. 22. Youden, W. J. (1972). Technometrics, 14, 13–22. (Independent introduction of the problem, first published in 1956.) See also ANALYSIS OF VARIANCE; BLOCKS, BALANCED INCOMPLETE; CONCOMITANT VARIABLES; CONFOUNDING; DESIGN OF EXPERIMENTS; FACTORIAL EXPERIMENTS; INCOMPLETE DATA; INTERACTION; LATIN SQUARES, LATIN CUBES, LATIN RECTANGLES; NESTING AND CROSSING IN DESIGN; OPTIMAL DESIGN OF EXPERIMENTS; and RANDOMIZATION.

R. A. BAILEY

RANDOMIZATION TESTS A randomization test is a permutation test∗ which is based on randomization∗ (random assignment), where the test is carried out in the following manner. A test statistic is computed for the experimental data (measurements or observations), then the data are permuted (divided or rearranged) repeatedly in a manner consistent with the random assignment procedure, and the test statistic is computed for each of the resulting data permutations. These data permutations, including the one representing the obtained results, constitute the reference set for determining significance. The proportion of data permutations in the reference set with test statistic values greater than or equal to (or, for certain test statistics, less than or equal to) the value for the experimentally obtained results is the P-value∗ (significance or probability value). Determining significance on the basis of a distribution of test statistics generated by permuting the data is characteristic of all permutation tests; it is when the basis for permuting the data is random assignment that a permutation test is called a randomization test. The preceding definition is broad enough to include procedures called randomization

6845

tests that depend on random sampling as well as randomization. The modern conception of a randomization test is a permutation test that is based on randomization alone, where it does not matter how the sample is selected. It is this narrower conception of randomization tests that will be the concern of this article. The null hypothesis for a randomization test is that the measurement for each experimental unit (e.g., a subject or a plot of land) is the same under one assignment to treatments as under any alternative assignment. Thus, under the null hypothesis, assignment of experimental units to treatments randomly divides the measurements among the treatments. Each data permutation in the reference set represents the results that, if the null hypothesis is true, would have been obtained for a particular assignment. (The validity of the foregoing procedure depends on all possible assignments being equally probable; situations with unequally probable assignments require modification of the foregoing procedure and will not be dealt with here.) Example. Suppose that five subjects have been assigned randomly to treatments A and B, with two subjects for A and three for B, and that these results were obtained: A: 1, 2; B: 3, 5, 9. A two-tailed (double-tailed) t test gives a t of 1.81, and the randomization test procedure, rather than a t table, is used to determine the significance of t. The data are divided into two parts with two measure  ments for A and three for B, in all 52 = 10 ways. The reference set of 10 data permutations is shown in Table 1, with the absolute value of t for each data permutation. Only Table 1. A 1, 2 1, 3 1, 5 1, 9 2, 3 2, 5 2, 9 3, 5 3, 9 5, 9

B

|t|

3, 5, 9 2, 5, 9 2, 3, 9 2, 3, 5 1, 5, 9 1, 3, 9 1, 3, 5 1, 2, 9 1, 2, 5 1, 2, 3

1.81 1.22 0.52 0.52 0.83 0.25 0.83 0.00 1.22 3.00

6846

RANDOMIZATION TESTS

the first (obtained) and last data permutations have |t|’s as large as 1.81, the obtained value, so the P-value is 2/10, or 0.20. If the null hypothesis was true, random assignment of the subjects to the treatment conditions was, in effect, random assignment of the measurements to the treatment conditions. There were 10 ways the measurements could have been ‘‘assigned,’’ and those are the 10 data permutations above, which provide the distribution of t for determining the significance of the obtained t. Each data permutation represents the results associated with a particular assignment, so the same reference set would be used even if a different test statistic was computed. For example, if σA2 /σB2 was the test statistic, that variance ratio would be computed for each of the data permutations above to provide a distribution of values for determining the significance of the variance ratio obtained. If a repeated-measurement∗ design was employed, with each of five subjects taking treatments A and B in an order determined randomly and independently for each subject, the 25 = 32 possible assignments to treatments would be associated with 32 data permutations which differed only in the order of the measurements within pairs. Just as the reference set of data permutations is independent of the test statistics, so is the null hypothesis. A difference between means may be used as a test statistic, but the null hypothesis does not refer to a difference between means. The null hypothesis, no matter what test statistic is used, is that there is no differential effect of the treatments for any of the subjects. (Exceptions, such as a directional null hypothesis [6, pp. 137–138], are possible but seldom used.) Thus the alternative hypothesis∗ is that the measurement of at least one subject would have been different under one of the other treatment conditions. Inferences about means must be based on nonstatistical considerations; the randomization test does not justify them. CONTRIBUTIONS OF FISHER AND PITMAN The test usually regarded as being the first randomization test was published by Fisher∗ in 1935 [13, pp. 43–47]. The hypothetical

experiment and test were described as follows. Random samples of 15 seeds were taken from each of two populations to test the null hypothesis of identity of the populations with respect to the size of plants the seeds would produce under the same conditions. As the seeds could not be grown under identical conditions, to prevent population differences from being confounded with soil and environmental differences, the seeds in the samples were assigned randomly to the pots in which they were to be planted. Pots with similar soil and locations were paired, and for each of the 15 pairs of pots it was randomly determined which of a pair of seeds would be planted in each pot. After the plants reached a certain age, the heights were measured and paired off as they would be for a paired t test. Fisher, however, carried out a randomization test instead. He generated a reference set of 215 data permutations by switching the signs of the differences for all 15 pairs and computing the difference between totals for the two types of seeds as the test statistic. (Switching the sign of a difference has the same effect on the test statistic as switching the two measurements on which the difference depends.) The P-value was the proportion of those data permutations with as large a difference between totals as the difference obtained. The procedure for determining significance was definitely novel, and this test seems to be the first permutation test of any kind, not just the first randomization test. As this test is based on random sampling as well as random assignment, however, it is not a test of the modern kind, where the nature of the sampling is irrelevant. Fisher described another test in 1935 [13, pp. 11–25] that is at times cited as a randomization test but which, although based on randomization, did not explicitly involve permuting of data. In a hypothetical experiment, a lady was told that she would be presented with eight cups of tea in a random order and that there would be four cups with milk added to tea and four cups with tea added to milk before being mixed. The lady’s task was to distinguish between the two types of preparation on the basis of taste. (If eight presentation times were designated in advance, as they should be, the eight times would be the experimental

RANDOMIZATION TESTS

units and the responses ‘‘milk first’’ and ‘‘tea first’’ would be the ‘‘measurements’’ for the eight experimental units.) She was hypothesized to identify correctly all eight cups, and Fisher determined the P-value to  be 1/70, there being only one way out of 84 = 70 divisions of the cups into the two categories that would correctly match the way they were presented. Unlike the other test, this test did not involve a new procedure for significance determination; there was no explicit permuting of data. Nevertheless, the demonstration of how randomization could provide the basis for a valid test for a single subject influenced the development of single-subject randomization tests. In a series of articles in 1937 and 1938 [25–27], Pitman∗ provided a theoretical basis for experimental and nonexperimental applications of permutation tests. Some investigators believe Pitman’s contributions to randomization tests to be more important than Fisher’s. It is certainly true that whereas Fisher dealt with randomization at length, he barely touched on randomization tests. Pitman was thorough and clear in his treatment of randomization tests; Fisher was neither. The principal contribution of Pitman was demonstrating that randomization tests could be based on randomization alone, for nonrandomly selected experimental units. As random selection from populations is uncommon in experimentation, a statistical test that is valid in the absence of random sampling is very useful. RANDOMIZATION AND RANDOMIZATION TESTS Randomization and randomization tests tend to be similarly evaluated, and what is nominally a criticism of randomization tests frequently is a criticism of the use of randomization in an experiment. In a series of papers, headed by a paper by Basu [1], debating the value of randomization tests, the principal disagreement concerned the relative advantages of systematic and random assignment of experimental units to treatments. Similarly, criticisms of the application of randomization tests to single-subject experimental data [17; 18, pp. 328–329] stem primarily from the belief that decisions to introduce

6847

or withdraw experimental treatments should be made during the course of the experiment on the basis of a subject’s responses. It has been pointed out [9,10] that although such criticisms seem to be directed toward randomization tests, they are in fact criticisms of a widespread philosophy of experimentation which requires randomization in order for any statistical test to be acceptable. ROBUSTNESS∗ OF PARAMETRIC TESTS Many studies have been performed to assess the robustness∗ (insensitivity to violations of assumptions) of parametric tests when populations are nonnormal and variances are heterogeneous. (See Bradley [2, pp. 24–43] for a critical evaluation of those studies.) For the typical experiment, in which there is randomization but not random sampling, these studies have little relevance. The important consideration is the robustness of the tests in the absence of random sampling. Building on work begun by Pitman, several investigators have investigated null and nonnull distributions of parametric test statistics (primarily F) under randomization. Kempthorne [19] has been the principal contributor to this field of study, which is called randomization analysis. (Despite the similarity of names, randomization analysis and randomization tests have different objectives and employ different approaches to statistical testing.) It is frequently asserted that randomization analysis has demonstrated that parametric tests give essentially the same P-values as randomization tests and consequently are valid for nonrandom samples, given experimental randomization. That is a misconception; parametric tests have not been demonstrated to be robust in the absence of random sampling. Parametric tests are based on infinite, continuous distributions and randomization tests on finite, discrete distributions of test statistics, so a parametric distribution can never exactly correspond to a randomization test distribution. Parametric and randomization test P-values are very similar for some data sets and very different for others, and there is no way of knowing the degree of correspondence without determining both Pvalues for a given set of data.

6848

RANDOMIZATION TESTS

The experimenter who is concerned about the validity of a particular application of a parametric test can perform a randomization test instead of approximating the randomization test P-value by use of the parametric test. For many years, the only serious objection raised against using randomization tests instead of parametric tests for analysis of experimental data has been the impracticality of performing the large amount of computation required for a randomization test, but the availability of high-speed computers makes the objection obsolete. RANK TESTS Rank tests∗ were produced in great numbers immediately after the writings of Fisher and Pitman on experimental and nonexperimental permutation tests. Rank tests are permutation tests applied to ranks and are usually represented as requiring the assumption of random sampling. When rank tests are applied to ranks of experimental data from a randomized experiment, however, the tests become randomization tests and do not require the assumption of random sampling. In his book on rank tests, Lehmann [22] discusses the tests in terms of both the random sampling and the randomization models. RESTRICTED NULL HYPOTHESES The reference set for a randomization test represents outcomes that, under the null hypothesis, would be associated with alternative assignments. A reference set consisting of data permutations associated with all possible assignments can be used to test the general null hypothesis of no effect of any treatment manipulation. To test restricted null hypotheses, however, that refer to some but not all treatment manipulations, a reference subset with data permutations for only some of the possible assignments (randomizations) must be used. The following examples of the use of reference subsets to extend the applicability of randomization tests to complex experimental designs employ procedures used by Edgington [6,7]. Consider the random assignment of six subjects to treatments A, B, and C with two

subjects per treatment, where the following results are obtained: A: 3, 5; B: 7, 9; C: 6, 4. A planned comparison of A and B cannot use the reference set of data permutations associated with all 90 possible assignments. The restricted null hypothesis of no difference between A and B not refer to C; consequently, under the null hypothesis, depending on the effect of C, the subjects providing 6 and 4 may or may not have provided those values if they had been assigned to other treatments. There is thus no basis for constructing data permutations for those random assignments. A procedure that generates only data permutations representing results dictated by the restricted null hypothesis is to divide the four A and   B measurements between A and B in all 42 = 6 ways to generate a reference subset of six data permutations. This procedure provides data permutations for that subset of randomizations where the same two subjects are always assigned to C. Another example of a restricted null hypothesis is the hypothesis of no effect of factor A in a completely randomized factorial experiment∗ with factors A and B. Factors A and B each have two levels, and both factors are experimentally manipulated. Eight subjects are randomly assigned to the four cells, with two subjects per cell. For a test of factor A alone, data permutations for only 36 of the 2520 possible assignments can be determined under the restricted null hypothesis, namely, those generated by permuting the obtained data for the four subjects at one level of B over both levels of A in conjunction with permuting the data for the four subjects at the other level of B over both levels of A. Those 36 data permutations constitute the reference subset for determining the significance of the obtained F (see F-TESTS). The preceding two examples illustrate an approach that can be used in generating a reference subset for any restricted null hypothesis, which is to hold fixed the results of all treatment manipulations not specified in the null hypothesis and permute the data only with respect to the specified treatment manipulations. For none but the simplest of experiments is there only a general null hypothesis to test. The applicability of randomization tests is thus greatly extended through the use

RANDOMIZATION TESTS

of reference subsets to test restricted null hypotheses. PERMUTATION GROUPS After noting that reference sets based on all assignments frequently would be too large to be practical, Chung and Fraser [3] proposed the use of reference subsets to reduce the number of data permutations to a practical level. An example will illustrate their approach. Three subjects are assigned at random to 5-, 10-, and 15-gram drug dosages, with one subject per dosage level. The test statistic is the product-moment coefficient of correlation∗ (r) between dosage and response magnitude. The experimental results are these: 5-g dosage, 62; 10-g dosage, 78; 15g dosage, 83. The reference set based on all assignments would consist of the six possible pairings of the drug dosages and responses. A reference subset of three pairings, however, will be used here to determine significance. The reference subset consists of the data permutation obtained and the two data permutations shown below, with their correlation coefficients: Dosage (g) 5

10

15

r

62 78 83

78 83 62

83 62 78

+0.96 −0.73 −0.23

The procedure for deriving the last two data permutations from the first (obtained) data permutation is this: Move the first measurement in the sequence of three measurements to the end to produce a new data permutation, apply the same procedure to that data permutation to produce another one, and continue until no new data permutations are produced. This procedure produces only three data permutations because applying the procedure to the third data permutation produces the first. Starting the permuting operations with any of the three data permutations will result in the same subset of data permutations. Thus the operations form what Chung and Fraser call a permutation group, which is defined as

6849

a collection of operations that will result in the same set of data permutations, no matter which member of the reference set (or subset) is the one that is initially permuted. The significance is based on the foregoing reference subset of three data permutations. As the correlation +0.96 obtained is the largest correlation, the P-value is 13 . DATA-PERMUTING AND RANDOMIZATION-REFERRAL TESTS From the earliest times, randomization tests have been based on permutation groups, but until the example of Chung and Fraser the permutation groups were ones that generated a reference set of data permutations for all randomizations. The demonstration that randomization tests can be based on permutation groups that provide a reference subset stimulated a study of the role of permutation groups [11], which will be outlined in this section. The study proceeded from a division of randomization tests into two classes: (1) datapermuting tests, which do not require knowledge of which experimental unit provided a given measurement, and (2) randomizationreferral tests, which do require such knowledge. All of the examples considered up to this point, including the drug dosage example, employed data-permuting tests, and, in fact, virtually all randomization tests are of this kind. An additional example of a datapermuting test, consequently, is unnecessary. It is necessary, however, to explain the randomization-referral test. The randomization-referral test involves these steps to produce a reference set (or subset) of data permutations: (1) Before the experiment, partition the entire set of randomizations (possible assignments) into subsets such that each randomization belongs to one and only one subset. (The performance of a ‘‘null’’ partition that leaves the set un-partitioned, is an acceptable first step.) (2) After the experiment, on the basis of the obtained data permutation and the obtained randomization (the actual assignment), transform the randomizations in the subset containing the obtained randomization into data permutations to form the reference subset. For instance, before performing

6850

RANDOMIZATION TESTS

the drug-dosage experiment, the set of six possible randomizations, showing the order in which subjects a, b, and c could be assigned to the drug dosage levels, is divided into two subsets: {abc, acb} and {bac, bca, cab, cba}. Suppose that the measurements for the data permutation obtained, (62, 78, 83), were obtained from subjects a, c, and b, respectively. Then abc, the other randomization in the subset containing acb, is transformed into 62, 83, 78, by substituting 62 for a, 78 for c, and 83 for b. As the data permutation obtained provides the larger correlation coefficient of the two, the P-value is 12 . If any of the four randomizations in the other subset had been the obtained randomization, there would have been four data permutations constituting the reference subset for determining significance. The rationale is simple: when a distribution of all randomizations is divided in advance into subsets (or left undivided as a ‘‘subset’’ of itself) the probability of the obtained randomization providing a test statistic value that is one of the k largest of the n values for its subset is no greater than k/n, so the determination of the P-value as k/n is valid. The conceptual simplicity of a randomization-referral test makes it easy to assess its validity. This is convenient because the validity of data-permuting tests can then be demonstrated by showing that there is a valid randomization-referral test that is equivalent in the sense that it necessarily gives the same reference set and, consequently, the same P-value. The use of permutation groups, as defined by Chung and Fraser, for data-permuting tests ensures that there is an equivalent randomization-referral test. An example will show what is meant. Let us consider a randomization-referral test that will necessarily give the same reference subset as the data-permuting procedure in the drug-dosage example. The randomization subsets are generated by applying to each of the six randomizations the same permutation group that was applied to the data permutations in the data-permuting test. That is, for each randomization, the first letter is moved repeatedly to the end until no new randomizations are produced. When that is done, the randomizations fall into two subsets, within each of which the

randomizations are related to each other in the same way as the data permutations are interrelated within the reference set: any of the randomizations would generate the others in the subset by applying the permutation group to it. The two subsets are {abc, bca, cab} and {acb, cba, bac}. Now, it can be seen that no matter which randomization provided 62, 78, 83, the other randomizations in its subset would be transformed into 78, 83, 62, and 83, 62, 78, resulting in the same reference set as was provided by the data-permuting procedure. In general, if a randomizationreferral test with the randomizations partitioned by applying the same permutation group to the entire set of randomizations as is applied to the data permutations in a datapermuting test is valid, the data-permuting test is valid. In the study, application of the foregoing approach to the specific data-permuting procedures given in the section on restricted null hypotheses provided a rigorous demonstration of their validity. Also, it was shown that for any test of a restricted null hypothesis, the procedure of holding fixed the results of all treatment manipulations not specified in a restricted null hypothesis and permuting the data only with respect to the specified treatment manipulations is a prescription for an appropriate permutation group. To sum up, the study of the role of permutation groups resulted in a comprehensive rationale for randomization tests, based on three key concepts: the permutation group, the data-permuting test, and the randomization-referral test. The practical and theoretical consequences are far-reaching. RANDOMIZATION TESTS IN THE COMPUTER AGE Access to computers makes it possible to carry out computations on thousands of data permutations quickly and inexpensively. Even with high-speed computers, however, it is frequently necessary to use a reference subset. For example, for a completely randomized design, with 10 subjects per treatment for three treatments, there are over 5 trillion possible assignments. The use of permutation groups to generate systematic (nonrandom) reference subsets is

RANDOMIZATION TESTS

less practical than forming random reference subsets. A random procedure that is completely valid was discovered by several investigators, the first being Dwass [4]. The data are permuted randomly n times, and the reference set consists of the n data permutations plus the obtained data permutation. Green [15, pp. 172–173; 16] has provided efficient algorithms for random permuting. The validity of a random permuting procedure is harder to assess than the validity of a data-permuting test because we must be sure that (1) the random procedure provides a random sample of data permutations from the relevant data-permuting reference set (or subset), and (2) the data-permuting test for the sampled reference set would be valid. Until randomization tests were developed for restricted null hypotheses, the uses of randomization tests were limited, but now parametric tests, no matter how complex, can be transformed into distribution-free∗ tests by determining significance by the randomization test procedure. Randomization tests also can be custom-made to meet special experimental requirements. Conventional experimental designs can be combined to provide more complex and efficient experiments. For example, 10 subjects may be paired and assigned randomly within the five pairs to two treatments, while 10 other subjects, for whom there is no good basis for pairing, are randomly and equally divided between the two treatments, without pairing. The test statistic is the difference between means, computed over all 20 subjects, and the reference set is derived by permuting the data for the five pairs of subjects in all 25 = 32 ways, in conjunction with permuting   the data for the other 10 subjects in all 10 5 = 252 ways. Two areas in which randomization tests have been strongly recommended in recent years are behavior therapy single-subject research [5; 7, Chap. 10; 8–10, 12, 23, 29] and weather modification∗ research [14,24,28]. These areas have several problems in common that militate against the use of parametric tests: nonrandom sampling, constraints on randomization, and lack of independence of observations. Each area can profit from

6851

studying tests used in the other area. Tests that were developed for other purposes also have relevance to single-subject research and weather modification. Zerbe and Walker [30] devised randomization tests of differences between growth curves∗ which have potential for single-subject experimentation, and Klauber’s [20,21] space-time clustering randomization tests for determining whether certain events that occur close together in space are also close together in time would appear to have uses in weather modification experimentation. The most complete and up-to-date guide to the application of randomization tests is a recent book by Edgington [7]. It discusses the logic of randomization tests, presents computer programs for commonly used tests, and describes ways to develop new tests and programs. Acknowledgment This work was supported by a Killam Resident Fellowship from the University of Calgary.

REFERENCES 1. Basu, D. (1980). J. Amer. Statist. Ass., 75, 575–582. (Followed by comments by five other contributors, plus a rejoinder.) 2. Bradley, J. V. (1968). Distribution-Free Statistical Tests. Prentice-Hall, Englewood Cliffs, N.J. 3. Chung, J. H. and Fraser, D. A. S. J. Amer. Statist. Ass., 53, 729–735.

(1958).

4. Dwass, M. (1957). Ann. Math. Statist., 28, 181–187. 5. Edgington, E. S. (1967). J. Psychol., 65, 195–199. 6. Edgington, E. S. (1969). Statistical Inference: The Distribution-Free Approach. McGrawHill, New York. (Chapter on randomization tests comprises one-third of book; substantial portions of Chapters 6 and 7 also deal with randomization tests.) 7. Edgington, E. S. (1980). Randomization Tests. Marcel Dekker, New York. (Practical guide for development and application of randomization tests; 11 FORTRAN computer programs; many numerical examples.) 8. Edgington, E. S. (1980). J. Educ. Statist., 5, 235–251. (Validity of single-subject randomization tests.)

6852

RANDOMIZED BLOCKS

9. Edgington, E. S. (1980). J. Educ. Statist., 5, 261–267. 10. Edgington, E. S. (1983). Contemp. Psychol., 28, 64–65. 11. Edgington, E. S. (1983). J. Educ. Statist., 8, 121–135. 12. Edgington, E. S. (1984). In Progress in Behavior Modification, Vol. XVI, M. Hersen, R. M. Eisler, and P. M. Miller, eds. Academic Press, New York. (Single-subject randomization tests, including rank tests; randomization; statistical and experimental independence.) 13. Fisher, R. A. (1951). The Design of Experiments, 6th ed. Hafner, New York. (Similar to the first edition, which is harder to find; a classic in experimental design; first randomization test; discusses randomization in depth.) 14. Gabriel, K. R. (1979). Commun. Statist. A, 8, 975–1015. 15. Green, B. F. (1963). Digital Computers in Research. McGraw-Hill, New York.

in Weather Resources Management. Department of Commerce, U.S. Government Printing Office, Washington, D.C. (Discusses many aspects of randomization and randomization tests in weather modification research.) 29. Wampold, B. E. and Furlong, M. J. (1981). J. Behav. Assess., 3, 329–341. 30. Zerbe, Z. O. and Walker, S. H. (1977). Biometrics, 33, 653–657. See also DESIGN OF EXPERIMENTS; FISHER, RONALD AYLMER; PERMUTATION TESTS; RANDOMIZATION; and RANDOMIZATION, CONSTRAINED.

EUGENE S. EDGINGTON

RANDOMIZED BLOCKS. See BLOCKS, RANDOMIZED COMPLETE

RANDOMIZED CONSENT DESIGN. See COMPLIANCE, ALL-OR-NONE

16. Green, B. F. (1977). Amer. Statist., 31, 37–39. 17. Kazdin, A. E. (1980). J. Educ. Statist., 5, 253–260. 18. Kazdin, A. E. (1982). Single-Case Research Designs: Methods for Clinical and Applied Settings. Oxford University Press, New York. 19. Kempthorne, O. (1952). Design and Analysis of Experiments. Wiley, New York. (A widely known book employing randomization analysis of many complex statistical procedures.) 20. Klauber, M. R. (1971). Biometrics, 27, 129– 142. 21. Klauber, M. R. 719–726.

(1975).

Biometrics,

31,

22. Lehmann, E. L. (1975). Statistical Methods Based on Ranks. Holden-Day, San Francisco. 23. Levin, J. R., Marascuilo, L. A., and Hubert, L. J. (1978). In Single Subject Research: Strategies for Evaluating Change, T. R. Kratochwill, ed. Academic Press, New York. 24. Miller, A. J., Shaw, D. E., and Veitch, L. G. (1979). Commun. Statist. A, 8, 1017–1047. 25. Pitman, E. J. G. (1937). J. R. Statist. Soc. B, 4, 119–130. 26. Pitman, E. J. G. (1937). J. R. Statist. Soc. B, 4, 225–232. 27. Pitman, E. J. G. 322–335.

(1938).

Biometrika,

29,

28. Tukey, J. W., Brillinger, D. R., and Jones, L. V. (1978). The Management of Weather Resources, Vol. II: The Role of Statistics

RANDOMIZED GAMMA DISTRIBUTION The randomized gamma distribution is a kind of compound gamma distribution∗ , and is also closely connected with the noncentral chisquare distribution∗ . Denote by g(x; θ , s) the probability density function (PDF) [1/{θ (s)}] (x/θ )s−1 exp(−x/θ ) of the gamma distribution∗ . Let X1 , X2 , . . . be independent random variables from the gamma distribution with PDF g(x, 1, 1) (i.e., exponential distribution∗
with mean 1). Then ki=1 Xi has a gamma distribution with PDF g(x; 1, k) for k = 1, 2, . . .. If K is a Poisson random variable with mean
ρ+K+1 Xi is µ, then for fixed ρ(= 0, 1, . . .), i=1 distributed as a randomized gamma distribution. On the other hand, if X1 , X2 , . . . are independent random variables from the gamma distribution with PDF g(x; 2, 12 ) (i.e., chisquare distribution with 1 degree of freedom), and if K is a Poisson random variable with
n+2K mean δ/2, then i=1 Xi is distributed as the noncentral chi-square distribution with n degrees of freedom and noncentrality parameter δ. The definition of randomized gamma distribution can be generalized to the case where ρ(> −1) is a real number [1]. For fixed real

RANDOMIZED GAMMA DISTRIBUTION

ρ > −1 consider the gamma distribution with PDF g(x; 1, ρ + K + 1). Taking the parameter K as an integer-valued random variable subject to the Poisson distribution∗ with mean µ we get the randomized gamma distribution with PDF: w(x; µ, ρ) = (x/µ)

ρ/2

Iv (z) =

∞  m=0

Mean: µ + ρ + 1 Variance: 2µ + ρ + 1 Moments about the mean: µ3 = 2(3µ + ρ + 1) µ4 = 3(4µ2 + 4µρ + 12µ + ρ 2 + 4ρ + 3)

√ {exp(−µ − x)}Iρ (2 µx), (1)

x > 0, µ > 0, ρ > −1, where Iv (z) is the modified Bessel function∗ of the first kind given by 1 (z/2)2m+v . m! (m + v + 1)

6853

µ5 = 4(30µ2 + 25µρ + 55µ +5ρ 2 + 16ρ + 11) vth moment about zero: µv +

Figure 1.

  v (ρ + v − i + 1) · · · (ρ + v)µv−i i=1 i

v

6854

RANDOMIZED MODEL

Moment generating function: (1 − t)−ρ−1 exp{µt/(1 − t)}, t < 1 Convolution formula: If X1 and X2 are independent random variables with PDFs w(x; µ, ρ) and g(x; 1, s), respectively, the sum X = X1 + X2 has a randomized gamma distribution with PDF w(x; µ, ρ + s)—the convolution formula w(x; µ, ρ) ∗ g(x; 1, s) = w(x; µ, ρ + s) given by Feller [1] holds. Denote by f (x; β, θ , λ) the PDF √ f (x; β, θ , λ) = Cx(λ−1)/2 {exp(−θ x)}Iλ−1 (β x), x  0, C = (2/β)λ−1 θ λ exp{−β 2 /(4θ )}, θ > 0, λ > 0, β > 0, for type I Bessel function distributions. Then the randomized gamma distribution with PDF w(x; µ, ρ) and the noncentral chi-square distribution with n degrees of freedom and noncentrality parameter δ are the special cases of the Type I Bessel function distributions obtained with the √ parameter β = 2 µ, θ = 1, λ = ρ + 1 √ values and β = δ, θ = 12 , λ = n/2, respectively. For additional information, see refs. 2 to 5. Figure 1 shows the graphs of the PDFs w(x; µ, ρ). In drawing the figure we used a program in ref. 2. REFERENCES 1. Feller, W. (1966). An Introduction to Probability Theory and Its Applications, Vol. 2. Wiley, New York, Chap. II, Sec. 7. 2. Hirano, K., Kuboki, H., Aki, S., and Kuribayashi, A. (1983). Comput. Sci. Monogr., No. 19, Inst. Statist. Math., Tokyo, Appendix. 3. Kotz, S. and Srinivasan, R. (1969). Ann. Inst. Statist. Math., Tokyo, 21, 201–210. 4. McNolty, F. (1967). Sankhya B, 29, 235– 248. 5. Springer, M. D. (1979). The Algebra of Random Variables. Wiley, New York, Chap. 9.

KATUOMI HIRANO

RANDOMIZED MODEL. See RANDOMIZATION —I; RANDOMIZATION —II

RANDOMIZED RESPONSE Description Randomized response is a method used in statistical surveys whose goal is to reduce or eliminate response errors which occur when respondents are queried about sensitive or highly personal matters. First introduced by Warner [40], a randomized response design requests information in an indirect manner. By using a randomizing device, the respondent selects a question on a probability basis from two or more questions at least one of which is sensitive. The respondent answers the question selected without revealing to the interviewer which question was chosen. Since the type and ranges of the responses are the same for each question, no respondent can be classified with certainty a posteriori with respect to the sensitive characteristic. The privacy of the respondent is thereby protected but the information obtained from a sample of respondents is sufficient, with knowledge of the probability distribution(s) used in the design, to compute unbiased estimates of parameters associated with the sensitive characteristic, such as the proportion with that attribute. In most applications it is assumed that each respondent truthfully answers the question selected by the randomizing device. In view of the protection provided the respondents, randomized response offers unusual opportunities for eliciting accurate information on sensitive subjects, including even illegal and socially deviant behavior. It has no applicability to the individual respondent except in a probability sense and is used to describe only population groups. CATEGORICAL DATA DESIGNS Theoretical development of randomized response designs was relatively rapid, especially during the first several years after the initial Warner design. The Warner design and three alternate classes of designs highlight this early period. The original Warner design developed a method for estimating the proportion πA of persons with a sensitive attribute, A, without requiring the individual respondent to report

RANDOMIZED RESPONSE

his actual classification, whether it be A or not-A, to the interviewer. The respondent is provided with a randomizing device in order to choose one of two statements of the form: I belong to group A (selected with probability p). I do not belong to group A (selected with probability 1 − p). Without revealing to the interviewer which statement has been chosen, the respondent answers ‘‘yes’’ or ‘‘no’’ according to the statement selected. The responses to either question divide the sample into two mutually exclusive and complementary classes. With a random sample of n respondents, πA is estimated by πˆ A =

λˆ − (1 − p) , 2p − 1

p = 12 ,

where λˆ is the observed proportion of ‘‘yes’’ answers. This is an unbiased estimate if all respondents answer the selected statement truthfully, with variance given by var(πˆ A ) =

πA (1 − πA ) p(1 − p) . + n n(2p − 1)2

The second term on the right-hand side represents the addition to the variance attributable to the uncertainty of the question selected. Had a direct question been asked, its response bias need not be too great for the resulting mean square error to exceed var(πˆ A ). Abul-Ela et al. [3] extended Warner’s design to the trichotomous case to estimate the proportions of three related, mutually exclusive groups, one or two of which are sensitive, and showed that the extension is easily made to the multichotomous case. The basis for this design was to use an extra independent sample for each additional parameter in the multinomial case. Eriksson [12] and Bourke [7] have shown, however, how multinomial proportions can be estimated with only one sample using a different kind of randomizing device (see below). The unrelated question design is attributed to Walt R. Simmons [23], who suggested that confidence of respondents in the anonymity

6855

provided by the randomized response might be increased, and hence their cooperation and the truthfulness of their responses, if one of the two questions referred to a nonsensitive, innocuous attribute, say Y, unrelated to the sensitive attribute A. Two such questions (in statement form) might be: (A) I had an induced abortion last year. (Y) I was born in the month of April. The theoretical framework for this unrelated question randomized response design was developed extensively by Greenberg et al. [18], including recommendations on the choice of parameters. Abul-Ela [2] and Horvitz et al. [23] provided some earlier, but limited results for this design. If πY , the proportion in the population with the nonsensitive attribute Y, is known in advance, only one sample is required to estimate the proportion with the sensitive attribute πA . The estimator of πA given πY , and its variance are πˆ A |πY =

λˆ − (1 − p)πY p

var(πˆ A |πY ) =

λ(1 − λ) , np2

where λˆ is the proportion of ‘‘yes’’ answers as before, and λ is the probability of a ‘‘yes’’ response. If πY is not known, the design can be altered to permit estimation not only of the proportion with the sensitive characteristic, but πY as well. This requires two independent samples of sizes n1 and n2 to be selected with different probabilities, p1 and p2 , of choosing the sensitive question. Moors [36] showed that for optimally allocated n1 and n2 , it is best to choose p2 = 0 in order to reduce the sampling variance of πˆ A . That is, the randomizing device is used in the first sample only, while the second sample is asked the unrelated question directly to estimate πY . Moors further showed that with optimal choice of n1 and n2 , and p2 = 0, the unrelated question design would be more efficient than the Warner design for p1 > 12 , regardless of the choice of πY . Dowling and Shachtman [11] proved that πˆ A in the unrelated question design has less variance than πˆ A in the original Warner design, for all πA

6856

RANDOMIZED RESPONSE

and πY , provided that p [or the max(p1 , p2 ) in the two-sample case] is greater than approximately one-third. Folsom et al. [14] evaluated an alternative two-sample design which consists of using two nonsensitive alternate questions Y1 and Y2 , in conjunction with the sensitive question A. One sample is used to estimate πY1 in a direct question with the randomizing device choosing between A and Y2 . In the second sample the roles of Y1 and Y2 are reversed. Both samples are used to estimate πA . In practice, two samples are never required inasmuch as Y can be selected such that πY is known or incorporated into the randomizing device as described below. Contamination (permutation) designs are discussed by Boruch [6], who suggested that error be introduced into classificatory data during the interview for inquiries of sensitive attributes. He proposed to accomplish this by presenting the respondent with a single question (or statement) and instructing him or her to lie or tell the truth depending on the outcome of a randomizing device. A different version of Boruch’s contamination design had been proposed by Warner [41] to reduce the loss in efficiency with respondents who do not belong to the sensitive class. This loss may be reduced by keeping the proportion of false positives low. Multiple trials designs seek to reduce additional variance introduced by the random selection of questions (or response sets) by repeated trials with each respondent. The use of two trials per respondent has been discussed by Horvitz et al. [23], Gould et al. [17], Liu and Chow [30], Chow and Liu [9] and Greenberg et al. [21]. Ordinarily, doubling the number of observations will reduce the variance in random samples by one-half. The gain in efficiency is somewhat less with two trials per respondent (pooled data), however, because of the correlation between the two responses. QUANTITATIVE DATA DESIGNS Randomized response designs need not be restricted to qualitative data. Greenberg et al. [20] extended the theory appropriate to the unrelated question randomized response

design to estimating the mean and variance of the distribution of a quantitative measure, whether discrete or continuous. Assuming two independent samples of size n1 and n2 , unbiased estimators for the means of the sensitive and nonsensitive distributions, µA and µY , respectively, are µˆ A = [(1 − p2 )Z1 − (1 − p1 )Z2 ]/(p1 − p2 ), µˆ Y = (p2 Z1 − p1 Z2 )/(p2 − p1 ), where p1 and p2 are the probabilities with which the sensitive question is selected in the two samples, p1 = p2 , and Z1 and Z2 are the sample means computed from the responses in the two samples. The variance of µˆ A is given by V(µˆ A ) =

1 [(1 − p2 )2 V(Z1 ) (p2 − p1 )2 +(1 − p1 )2 V(Z2 )],

where the variance of Zi for i = 1, 2 is V(Zi ) = [σY2 + pi (σA2 − σY2 ) +pi (1 − pi )(µA − µY )2 ]/ni . From the standpoint of both efficiency and privacy of the respondent the nonsensitive variable Y should be chosen with mean and variance as close as possible to the corresponding moments for the sensitive variable. Poole [37] used a contamination design to estimate the distribution function of a continuous variable. The respondent is asked to multiply the true response to the variable of interest by a random number and to tell only the result to the interviewer. This randomized information, together with known properties of the distribution of the random multiplier, is sufficient to estimate the distribution of the sensitive variable. Eriksson [12] discussed in some detail a design for collecting sensitive data on a discrete quantitative variable. It is essentially the quantitative counterpart to his design, discussed below, for estimating a multinomial distribution∗ . He suggests a deck with two types of cards for the randomizing device; the first type of card asks the sensitive question and the second type of card asks the

RANDOMIZED RESPONSE

respondent to state a designated value for Y as shown on the card. For example, Card-Type 1: How many abortions have you ever had? Card-Type 2: State the numerical value shown on this card. (Cards will have values 0, 1, 2, . . . , m). With a random sample of n respondents, an unbiased estimator of µA is µˆ A = [Z − (1 − p)µY ]/p, where Z is the mean response for all n respondents, µY the mean of numerical values shown on card-type 2, and p the proportion of cards of the first type in the deck. The variance of µˆ A reduces to σA2 /(np2 ) when µA = µY and σA2 = σY2 . THE LINEAR RANDOMIZED RESPONSE MODEL The various randomized response designs discussed above have been developed without any apparent unifying framework. Warner [41] has supplied that framework with his formulation of a general linear randomized response model. Let X be a random vector of q elements, some of which are sensitive measures, with E(X) = π or some linear function of π without observing sample values of X. The observations reported by the ith respondent in a random sample of size n are represented by the r-element column vector Yi defined by the matrix product Yi = Ti Xi , where Ti is an observation from an r × q random matrix of known distribution. The actual values of Ti and Xi (observed value of X) remain unknown to the interviewer and the researcher. Since E(Yi ) = τi π , the model may be written as Y = τ π + U, where E(U) = 0 and E(UU  ) is the covariance matrix. Thus the linear randomized response model may be interpreted as an application of the generalized linear regression∗ model with weighted least-squares∗ used to estimate the unknown π . The randomized response designs discussed above are all special cases of this general model.

6857

COMPARISON OF ALTERNATIVE RANDOMIZED RESPONSE DESIGNS Under the best of design conditions, with p of the order of 0.7, the estimated proportion with the sensitive characteristic will have about twice the variability of the estimate obtained with direct questioning. This ratio can be reduced by schemes offering less privacy protection but possibly at the expense of introducing bias. Greenberg et al. [21] have compared six different randomized response designs for estimating the proportion belonging to a sensitive class by computing design effects for πA ranging from 0.05 to 0.40, p from 0.50 to 0.90, and sample sizes 100, 400, and 800 for each design. The design effect was computed as the ratio of the variance for the particular randomized response design to the mean square error of the estimated proportion obtained by a direct question with the same number of respondents. It was assumed that a proportion TA of respondents with the sensitive attribute A would answer the direct question truthfully. It was also assumed that respondents not in the sensitive group would answer the direct question truthfully, as would all respondents for each of the randomized response designs. The designs compared included the Warner design and unrelated question design, an extended contamination design, and the Chow and Liu [9] multiple trials designs. The comparisons indicate rather conclusively that randomized response designs should be given consideration over direct questioning whenever TA < 1.00, and, depending on πA , p and TA , with samples as small as 100. Except for the Warner design, all the designs compared were more efficient than direct questioning for n  400 and πA  0.10, p  0.70 and TA < 0.90. None of the designs was found to be uniformly superior, although the unrelated question designs were generally efficient, and the multiple trials design slightly better when equivalent to 10 trials per respondent and πA  0.20. RANDOMIZING DEVICES Randomized response requires a mechanical device or rule of procedure that selects at

6858

RANDOMIZED RESPONSE

random with known probability the question to be answered by the respondent. This process must be unbiased, completely adaptable to use in the field, and easily understood and acceptable to the respondent with assurance of the protection of privacy in its use. The device suggested by Warner [40] in the original paper was a spinner with an arrow pointer similar to the type found in games that involve a chance element. This device is not too satisfactory, because the spinner must be perfectly horizontal to be unbiased and confusion results when the arrow rests on the line separating two segments. In early field trials of randomized response reported by Horvitz et al. [23], use was made instead of decks of specially printed cards. An identical pack of 50 cards with each card containing one of two alternate questions was used by each interviewer. The respondent was asked to mix the cards and select one for reply. This is not a completely satisfactory randomizing device since interviewers can lose cards from the deck. Further, respondents may not shuffle the cards adequately to generate the expected probabilities for selecting the alternate questions. Cards are also used by Eriksson [12] and Bourke [7] in an ingenious manner with the unrelated question to estimate k > 2 mutually exclusive classes with (k − 1) sensitive groups and requiring only one sample. For example, Bourke suggests cards on which each has three (or more) statements listed in a numbered, permuted sequence. After selecting a card at random, the respondent replies by stating the number associated with his or her category on that card. Another variation using special cards for quantitative discrete variables by Eriksson was described in the foregoing text. All card designs, however, have the problem of adequate shuffling to reproduce the design probabilities. To overcome the sampling difficulties with cards, Abernathy et al. [1] developed a randomizing device that consisted of a sealed, transparent, plastic box approximately the size of a pack of cigarettes. Inside are 50 colored beads, tested to be electrostatically free. When the box is shaken, beads can roll down a small track to a window at the end of the track which displays one and only one bead. On the front surface of the box, there is a

printed card with each color bead in front of the question to be answered. The respondent is advised to inspect the box on the obverse side to see that there are at least two colors of beads inside. The respondent is also encouraged to shake the box several times to assure that each color can appear in the window. This device fulfills all the criteria mentioned except that sets of boxes are not always easily accessible to investigators. The plastic box can also be used as a randomizing device with Y built into the design. Using three colors of beads, say red, white, and blue, the respondent answers the sensitive question if the red bead appears, and is instructed to reply ‘‘yes’’ when the white one shows, and ‘‘no’’ when the bead is blue. The value of Y is the proportion of white beads to the combined total number of white and blue beads. A variation of the sealed box is the sampler developed by Chow and Liu [9] and discussed by Liu et al. [32] and Liu and Chow [31]. They constructed a volumetrical flask with a thin narrow neck and inside the spherical bottle are six red and six white balls. The respondent is told that the red balls refer to the sensitive category and the white ones to the nonsensitive category. The respondent is then asked to shake the device thoroughly before turning it upside down, permitting exactly five balls to move into the neck of the bottle, which is frosted on one side so that the interviewer cannot observe the result of the trial. Without mentioning color, respondents who belong to the sensitive class report the number of red balls in the neck of the bottle and respondents who do not belong to the sensitive class report the number of white balls. This yields an estimate with variance roughly equivalent to the variance of the Warner estimate based upon five trials per respondent. As with the plastic box, surveyors may not find this special bottle to be easily accessible. After the unrelated question technique was developed, the Warner restriction that p = 12 could be dropped. This opened up the possibility of using a coin as the randomizing device. The coin apparently works well except for the disadvantage that since the probability of selecting the sensitive question is only 50%, sample size must be rather large (i.e.,

RANDOMIZED RESPONSE

four times greater than for p = 1) if the sampling variance is to be kept within reasonable bounds. A simple variation of the coin toss is to use two coins. The respondent is told to answer the unrelated question if both coins are heads, but to answer the sensitive question otherwise. This procedure reduces variance by more than one-half over a single coin but may also decrease confidence in the technique and ability to follow instructions. A randomizing procedure that can be used in telephone∗ interviews as well as mail questionnaires is to instruct the respondent to answer a specific question according to his or her birth month. For example, if the respondent was born in any one of the months from January through August, the first question is answered; if the month of birth was September through December, the second question is answered. Respondent’s month of birth should not be used where the date might be known to the survey designer or interviewer, and the unrelated question must also avoid the question of birth. In the latter case, a good substitute for the unrelated question is the month of birth of the respondent’s mother. RESPONDENT HAZARD An honest, affirmative response to a direct question which is truly sensitive results in classification of the respondent as a member of stigmatizing group A with certainty. The purpose of the randomized response technique is to reduce the hazard to the respondent who belongs to group A as being perceived as such a member and redistributing some of the jeopardy to the usually larger number of not-A (i.e., A) group members who also reply in the affirmative. A ‘‘yes’’ response by a randomly selected respondent to the question chosen by a randomizing device in a randomized response design increases the prior probability∗ A (that the respondent belongs to the sensitive group) to a posterior probability∗ greater than A but less than 1. The exact posterior probability or hazard for a given design can be computed using Bayes’ theorem∗ and specific parameter values for p, A , and Y . The challenge to the statistician is to choose

6859

parameter values p and Y such that the hazard to the respondent, whether from group A or A, is at a level low enough to induce an accurate response, yet at the same time minimize the added variance associated with the use of a randomizing device to select the question to be answered. Lanke [28] measured the ‘‘risk of suspicion’’ for those respondents who reply ‘‘yes’’ in an unrelated question design, where Y is known, as P(A|Yes) =

A p + A (1 − p)Y . A p + (1 − p)Y

He recommended that this risk should be bounded and Y chosen so that Y = 1, resulting in the maximum number of respondents replying ‘‘yes’’ for a given p. Loynes [34] came to the same conclusion. Anderson [4] pointed out that there is also a risk of suspicion if the response is ‘‘no,’’ and he added to P(A|Yes) the value of P(A|No) = 1 − P(A|No). Leysieffer [29] had previously generalized this by saying that any respondent is in jeopardy if P(A|Response) > A , and comparisons of efficiency in design must be for a given level of respondent jeopardy. Leysieffer also concluded that among all unrelated question designs with the same level of jeopardy, Y = 1 has the smallest variance. Greenberg et al. [22] have addressed this problem by pointing out that too much reduction in hazard for A respondents endangers the cooperation of A respondents of which there are (1 − A )/A times more of them. They defined the benefit BA to the A respondents as (1 − HA ), where HA = P(A is perceived as A|Yes), and the corresponding benefit BA to the A respondents as (−HA ), where HA = P(A is perceived as A|Yes). Inasmuch as BA is a true reduction in hazard or gain and BA is really a loss, they maximized the ratio BA /BA . This ratio is a monotonically decreasing function of (1 − p)Y . This means that to maximize the real benefit for both A and A respondents as well as to ˆ A , it is better reduce the sampling error of 

6860

RANDOMIZED RESPONSE

to select an unrelated question with a low frequency, usually in the order of magnitude of A , and to choose as high a value for p as will be acceptable to the respondents. (A value of p in the range 0.50 to 0.75 has proved to be acceptable in many field trials.) These criteria for selecting p and Y had previously been recommended by Greenberg et al. [18] and Moors [36] as the preferred strategy for randomized response unrelated question designs without formal consideration of respondent hazard.

child abuse (see, e.g., Brown and Harding [8], Goodstadt and Gruson [16], Zdep et al. [43], Krotki and Fox [26], I-Cheng et al. [24], Gerstel et al. [15], Barth and Sandler [5], Zdep and Rhodes [42], and Fidler and Kleinknect [13]). Overall, although conflicting results have been realized in some of the validation and comparison tests, the randomized response technique has generally yielded a greater frequency of reporting of sensitive behavior, and its estimates are closer to actual values than those obtained by direct questioning.

VALIDATION AND COMPARISON STUDIES APPLICATIONS Optimal validation tests of randomized response require knowledge of the actual attribute or behavioral category for each person selected for the test. As a minimum, validation can also be tested by having knowledge of the actual value of the population parameter to be estimated by the randomized response sample. When validation data are not available, an indication of the efficacy of the technique may be obtained by comparison of the randomized response-derived estimates with estimates derived from other methods. An indication of efficacy may also be obtained by using the two-sample unrelated question design to estimate πY when that population parameter was actually known beforehand, such as proportion born in a given month. Finally, confidence that the procedure was properly administered and accepted by respondents may also be gained through analysis of the consistency and plausibility of randomized response estimates across demographic and socioeconomic subgroups of the sample. The validity of the technique has been tested on a variety of characteristics considered to be subject to inaccurate responses in sample surveys, including illegitimate births, persons charged with drunken driving, persons registered to vote, number of college courses failed, and number of arrests in a specified time period (see, e.g., Horvitz et al. [23], Locander et al. [33], Lamb and Stern [27] and Tracy and Fox [39]). Comparison tests have been reported for variables such as illicit drug usage, abortion incidence and prevalence, alcoholic beverage drinking and driving behavior, and

Reports in the literature on experience with randomized response designs have increased considerably as awareness of the technique has become widespread. One of the earliest studies, reported by Abernathy et al. [1], provided estimates of the incidence and prevalence of induced abortion in urban North Carolina. Both single-sample and two-sample unrelated-question designs were used. Women in the same sample were also queried about use of the contraceptive pill; with the same type of plastic box randomizing device, Greenberg et al. [19] reported the findings of this component of the study as well as the results of using the randomizing device with a value for Y built in to obtain information on emotional problems. Experience with randomized response collection of quantitative data on number of abortions and also on annual income was reported by Greenberg et al. [20]. Other applications reported in the literature include a study of organized crime in Illinois which used the Warner design with p = 0.20 to estimate the proportions of the adult population in metropolitan areas of the state that engaged in illegal betting and the proportion which have had contact with organized crime reported by IIT Research Institute [25]; a study that used the twoalternate-questions design with a coin toss to estimate the annual incidence of at-fault automobile accidents among licensed drivers who drink alcoholic beverages reported by Folsom et al. [14]; a study of the extent of deliberate concealment of deaths in household surveys of vital events in the Philippines

RANDOMIZED RESPONSE

reported by Madigan et al. [35]; and a study of abortion in a national survey reported by Shimizu and Bonham [38]. An excellent bibliography of randomized response, including additional applications of the technique, may be found in Daniel [10]. REFERENCES 1. Abernathy, J. R., Greenberg, B. G., and Horvitz, D. G. (1970). Demography, 7, 19–29. (Early application of randomized response. Excellent interview procedures.) 2. Abul-Ela, A. A. (1966). Unpublished Ph.D. thesis, University of North Carolina, Chapel Hill, N. C. 3. Abul-Ela, A. A., Greenberg, B. G., and Horvitz, D. G. (1967). J. Amer. Statist. Ass., 62, 990–1008. 4. Anderson, H. (1976). Int. Statist. Rev., 44, 213–217. 5. Barth, J. T. and Sandler, H. M. (1976). J. Stud. Alcohol, 37, 690–693. 6. Boruch, R. F. (1972). Social Sci. Res., 1, 403–414. 7. Bourke, P. D. (1974). Errors in Surveys Research Project. Rep. No. 74, Institute of Statistics, University of Stockholm, Sweden. 8. Brown, G. H. and Harding, F. D. (1973). Tech. Rep. 73–9, Human Resources Research Organization, Alexandria, Va. 9. Chow, L. P. and Liu, P. T. (1973). A New Randomized Response Technique: The Multiple Answer Model. Department of Population Dynamics, School of Hygiene and Public Health, Johns Hopkins University, Baltimore, Md. (mimeo). 10. Daniel, W. W. (1979). Collecting Sensitive Data by Randomized Response: An Annotated Bibliography. Res. Monogr. No. 85. College of Business Administration, Georgia State University, Atlanta, Ga. (Thorough coverage, well annotated.) 11. Dowling, T. A. and Shachtman, R. (1975). J. Amer. Statist. Ass., 70, 84–87. 12. Eriksson, S. A. (1973). Int. Statist. Rev., 41, 101–113. 13. Fidler, D. S. and Kleinknect, R. E. (1977). Psychol. Bull., 84, 1045–1049. 14. Folsom, R. E., Greenberg, B. G., Horvitz, D. G., and Abernathy, J. R. (1973). J. Amer. Statist. Ass., 68, 525–530. 15. Gerstel, E. K., Moore, P., Folsom, R. E., and King, D. A. (1970). Mecklenburg County

6861

Drinking—Driving Attitude Survey. Report Prepared for U.S. Department of Transportation, Research Triangle Institute, Research Triangle Park, N.C. 16. Goodstadt, M. W. and Gruson, V. J. Amer. Statist. Ass., 70, 814–818.

(1975).

17. Gould, A. L., Shah, B. V., and Abernathy, J. R. (1969). Amer. Statist. Ass., Proc. Social Statist. Sec., pp. 351–359. (Suggests respondent behavior model to explain response distribution.) 18. Greenberg, B. G., Abul-Ela, A. A., Simmons, W. R., and Horvitz, D. G. (1969). J. Amer. Statist. Ass., 64, 520–539. 19. Greenberg, B. G., Abernathy, J. R., and Horvitz, D. G. (1970). Milbank Mem. Fund Quart., 48, 39–55. 20. Greenberg, B. G., Kuebler, R. R., Jr., Abernathy, J. R., and Horvitz, D. G. (1971). J. Amer. Statist. Ass., 66, 243–250. 21. Greenberg, B. G., Horvitz, D. G., and Abernathy, J. R. (1974). Reliability and Biometry, Statistical Analysis of Lifelength, F. Prochan and R. J. Serfling, eds. SIAM, Philadelphia, pp. 787–815. 22. Greenberg, B. G., Kuebler, R. R., Abernathy, J. R., and Horvitz, D. G. (1977). J. Statist. Plann. Inf., 1, 53–60. 23. Horvitz, D. G., Shah, B. V., and Simmons, W. R. (1967). Amer. Statist. Ass., Proc. Social Statist. Sec., pp. 65–72. 24. I-Cheng, C., Chow, L. P., and Rider, R. V. (1972). Stud. Family Plann., 3, 265–269. 25. IIT Research Institute and the Chicago Crime Commission (1971). A Study of Organized Crime in Illinois. Report prepared for the Illinois Law Enforcement Commission, Chicago. 26. Krotki, K. J. and Fox, B. (1974). Amer. Statist. Ass., Proc. Social Statist. Sec., pp. 367–371. 27. Lamb, C. W., Jr., and Stern, D. E., Jr. (1978). J. Marketing Res., 15, 616–621. 28. Lanke, J. (1976). Int. Statist. Rev., 44, 197–203. 29. Leysieffer, F. W. (1975). Respondent Jeopardy in Randomized Response Procedures. FSU Statist. Rep. M338, ONR Tech. Rep. No. 93, Department of Statistics, Florida State University, Tallahassee, Fla. 30. Liu, P. T. and Chow, L. P. (1976). Biometrics, 32, 607–618. 31. Liu, P. T. and Chow, L. P. (1976). J. Amer. Statist. Ass., 71, 72–73. 32. Liu, P. T., Chow, L. P., and Mosley, W. H. (1975). J. Amer. Statist. Ass., 70, 329–332.

6862

RANDOMIZED STOPPED-SUM DISTRIBUTIONS

33. Locander, W., Sudman, S., and Bradburn, N. (1974). Amer. Statist. Ass., Proc. Social Statist. Sec., pp. 21–27. (See also J. Amer. Statist. Ass., 71, 269–275, 1976.) 34. Loynes, R. M. (1976). J. Amer. Statist. Ass., 71, 924–928. 35. Madigan, F. C., Abernathy, J. R., Herrin, A. N., and Tan, C. (1976). Popul. Stud., 30, 295–303. 36. Moors, J. J. A. (1971). J. Amer. Statist. Ass., 66, 627–629. 37. Poole, W. K. (1974). J. Amer. Statist. Ass., 69, 1002–1005. 38. Shimizu, I. M. and Bonham, G. S. (1978). J. Amer. Statist. Ass., 73, 35–39. 39. Tracy, P. E. and Fox, J. A. (1981). Amer. Sociol. Rev., 46, 187–200. 40. Warner, S. L. (1965). J. Amer. Statist. Ass., 60, 63–69. (Original paper introducing randomized response.) 41. Warner, S. L. (1971). J. Amer. Statist. Ass., 66, 884–888. 42. Zdep, S. M. and Rhodes, I. N. (1976). Public Opinion Quart., 40, 531–537. 43. Zdep, S. M., Rhodes, I. N., Schwarz, R. M., and Kilkenny, M. J. (1979). Public Opinion Quart., 43, 544–549. See also SURVEY SAMPLING.

B. G. GREENBERG J. R. ABERNATHY D. G. HORVITZ

RANDOMIZED STOPPED-SUM DISTRIBUTIONS. See CLASSICAL DISCRETE DISTRIBUTIONS, GENERALIZATIONS OF

RANDOMIZED TESTS Statistical inference∗ procedures are generally based on the outcome of a random experiment or an observable random variable. In a randomized procedure, the inference is also based on the outcome of some additional random experiment which is unrelated to the original random experiment. A randomized test is one that rejects with a specified probability for each possible outcome X. An ordinary (nonrandomized) test is a special case of a randomized test where the specified rejection probability is always equal to 1 or

zero. In symbols, a randomized test is frequently denoted by φ(x), the critical function, 0
φ(x)
1, which represents the probability of rejection of the null hypothesis for various outcomes x. In a nonrandomized test, φ(x) takes on only the values 0 and 1. As a numerical example, consider a binomial test at level 0.10 with null hypothesis H0 : p  0.4 and alternative H1 : p < 0.4 for p the probability of success in the population. The random variable on which the test should be based is the number of successes X in a sample of n and the appropriate rejection region is small values of X. For n = 6, say, we should reject for X
c, where c is chosen such that Pr[X
c] =

c    6 k=0

k

(0.4)k (0.6)6−k = 0.10.

(1) The left tail cumulative binomial probabilities for n = 6, p = 0.4 are Pr[X
0] = 0.0467, Pr[X
1] = 0.2333. It is clear that there is no integer c which satisfies (1); hence a nonrandomized test at level 0.10 would reject for X = 0 only and the actual level is 0.0467. A randomized test at level 0.10 is to reject always when X
0 and reject with probability φ(1) when X = 1, where φ(1) is chosen such that Pr[X
0] + φ(1) Pr[X = 1] = 0.10 0.0467 + 0.1866φ(1) = 0.10 or φ(1) = 0.2856. If we observe X = 1, a separate randomized procedure is performed to see whether or not rejection of H0 is appropriate. The probability of rejection of H0 by this randomized test for any p is equal to Pr[p] = (1 − p)6 + 0.2856[6p(1 − p)5 ];

(2)

this value is always greater than (1 − p)6 , the probability of rejection by the nonrandomized test, for any p = 0, 1. Therefore, the power of this randomized test for any 0 < p < 0.4 is always larger than that of the nonrandomized test, and the probability of a Type I error never exceeds the given level 0.10 for any p  0.4.

RANDOM MATRICES

A right-tail randomized test should be used for H0 : p
0.4 and H1 : p > 0.4. If the alternative is two-sided, as in H0 : p = 0.4, H1 : p = 0.4, a two-tailed randomized test should be used. This would be of the form: reject always if X < c1 or X > c2 , reject with probability φ1 if X = c1 , reject with probability φ2 if X = c2 , and do not reject otherwise. For a test at exact level α, c1 , c2 , φ1 , and φ2 must be chosen such that Pr[X < c1 ] + φ1 Pr[X = c1 ] + Pr[X > c2 ] + φ2 Pr[X = c2 ] = α,

(3)

where c1 < c2 , 0
φ1 , φ2
1 or φ1 + φ2
1 if c1 = c2 . Randomized tests are probably seldom used in applications, even in the one-sided case where they are uniquely defined. In practice, most researchers take the conservative approach and use a nonrandomized test with a critical value that makes the exact level as close to α as possible without exceeding α. Randomized tests are important in theory though, especially for test statistics with a discrete null sampling distribution when the choice of exact levels is extremely limited. In comparing the power of two tests, for example, the exact level of the two tests must be the same for the comparison to be unbiased. Further, a randomized test is usually the optimum test for a statistic with a discrete null sampling distribution. Few textbooks discuss randomized tests. Notable exceptions are Lehmann [1] and Pratt and Gibbons [2].

REFERENCES 1. Lehmann, E. L. (1959). Testing Statistical Hypotheses. Wiley, New York. (Graduate-level text and reference book on inference.) 2. Pratt, J. W. and Gibbons, J. D. (1981). Concepts of Nonparametric Theory. Springer-Verlag, New York. (Graduate-level text and reference book on nonparametric statistical inference.) See also HYPOTHESIS TESTING and NEYMAN –PEARSON LEMMA.

JEAN DICKINSON GIBBONS

6863

RANDOM MATRICES Random matrices occur in many contexts, ranging from dynamical systems in physics and biology through to multivariate statistics and the Wishart∗ matrix. For example, in quantum mechanics∗ the energy levels of a system are described by eigenvalues of an operator called the Hamiltonian, which is represented as a random matrix (e.g., Mehta [9]). In mathematical biology, model ecosystems are often discussed in terms of a square (random) community matrix A = B − I, with I the identity and B having elements which are independent and identically distributed (i.i.d.) with zero mean and variance σ 2 (e.g., May [8, pp. 62–65] and Cohen and Newman [1, Sec. 4]). The eigenvalues∗ of A determine the stability consequences of various biological assumptions as the size of A increases. Another standard application comes from demography∗ and concerns the evolution of an age-structured population which is modeled by the equation Yt+1 = Xt+1 Yt ,

t  0,

where Yk is the (column) vector of the number of individuals in each age class at time k and {Xk } is a sequence of random matrices of vital rates. Here the behavior of the system is determined by that of the backwards matrix products Xt Xt−1 · · · X1 (e.g., Tuljapurkar and Orzack [11], Heyde and Cohen [6]). Quite a substantial theory has been developed which elucidates the examples above and a range of similar ones. Most of this theory is concerned with investigating either 1. The spectrum of an n × n random matrix as n increases, or 2. The asymptotic behavior of a product of t random matrices (of fixed size) as t increases. We shall illustrate this theory with some particular results, first relating to problem 1. Theorem 1. (Geman [4]). Let {υij , i  1, j  1} be i.i.d. random variables with Eυ11 = 0 and E|υ11 |n
nαn for all n  2 and some α.

6864

RANDOM MATRICES

Then, if Vn = (υij )1
i,j
n , the maximum eigenvalue of n−1 Vn Vn , λmax (n), satisfies 2 lim λmax (n) = 22 Eυ11 a.s. (almost surely).

n→∞

Next, let A(t), t  1, be random n × n matrices with elements (A(t))ij , 1
i, j
n, and let x(0) be a nonzero vector in Rn . Define x(t) = A(t)A(t − 1) · · · A(1)x(0),

t  1,


n

and let ||x|| denote ( j=1 x2j )1/2 if x = (x1 , . . . , xn ). Also, write log+ u for max(0, log u). Theorem 2. (Furstenberg and Kesten [3]). If {A(t), t  1} is a stationary sequence of random matrices and   n  + E log max |A(1)|ij  < ∞, i

j=1

then limt→∞ log ||x(t)|| exists a.s. and is finite. In general the limit in Theorem 2 may be random but under more specific assumptions much more explicit results can sometimes be obtained, as in the following theorem. Theorem 3. (Cohen and Newman [1]). Suppose that {A(1)ij , 1
j, j
n} are i.i.d. N(0, s2 ) variables. Then lim t−1 log ||x(t)||

t→∞

= 12 (log s2 + log 2 + ψ(n/2)) = log λ a.s., say, independent of x(0), where ψ is the digamma function∗ . Moreover, for any x(0) = 0, t−1/2 (log ||x(t)|| − t log λ) converges in distribution to N(0, σ 2 ) with σ 2 = 14 ψ  (n/2). Now suppose that the A(t) are independent while the elements of β t = (β1(t) (x(t)), . . . , βn(t) (x(t))) are measurable functions of the elements of x(t). Girko [5] has given sufficient conditions for the convergence of β t − Eβ t to infinitely divisible∗ laws.

Rather surprisingly, it can happen that the product A(t)A(t − 1) · · · A(1) of i.i.d. nonnegative matrices of size n can converge in distribution, without normalization, as t → ∞, to a distribution not concentrated on the zero matrix. Kesten and Spitzer [7] provide various equivalent sufficient conditions for this behavior. It should be noted that this behavior is impossible for n = 1 since log{A(t) . . . A(1)} is then a sum of i.i.d. random variables and it is well known that such a sum cannot converge in distribution unless all the summands log A(i) are zero with probability 1. Finally, mention needs to be made of random matrices occurring in multivariate analysis∗ . Here the emphasis is usually on distribution theory rather than asymptotic behavior. However, for situations where multivariate normality cannot realistically be assumed, asymptotic expansions∗ can provide useful information. For example, Edgeworth-type expansions have been used to investigate the asymptotic joint distributions of certain functions of the eigenvalues of the sample covariance matrix, correlation matrix, and canonical correlation matrix by Fang and Krishnaiah [2]. Now let ui have the Np (µi ,
) distribution, i = 1, 2, . . . , k, namely p-dimensional normal with mean vector µi and covariance matrix
. Then the most ubiquitous of the random matrices studied in multivariate analysis is
S = kj=1 uj uj . This has a Wishart distribution on k degrees of freedom, Wp (k,
, M), where M is a p × k matrix with µ1 , µ2 , . . . , µk as its columns (e.g., Rao [10, Chap. 8]). For more details, see MULTIVARIATE ANALYSIS and WISHART DISTRIBUTION. REFERENCES 1. Cohen, J. E. and Newman, C. M. (1984). Ann. Prob., 12, 283–310. 2. Fang, C. and Krishnaiah, P. R. (1982). J. Multivariate Anal., 12, 39–63. 3. Furstenberg, H. and Kesten, H. (1960). Ann. Math. Statist., 31, 457–469. 4. Geman, S. (1980). Ann. Prob., 8, 252–261. 5. Girko, V. L. (1982). Theor. Prob. Appl., 27, 837–844. 6. Heyde, C. C. and Cohen, J. E. (1985). Theor. Popul. Biol., 27, 120–153.

RANDOMNESS, BDS TEST FOR 7. Kesten, H. and Spitzer, F. (1984). Wahrscheinl. verw. Geb., 67, 363–386.

Z.

8. May, R. M. (1974). Stability and Complexity in Model Ecosystems. Princeton University Press, Princeton, N.J. 9. Mehta, M. L. (1967). Random Matrices and the Statistical Theory of Energy Levels. Academic Press, New York. 10. Rao, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd ed. Wiley, New York. 11. Tuljapurkar, S. D. and Orzack, S. H. (1980). Theor. Popul. Biol., 18, 314–342. See also MATRIX-VALUED DISTRIBUTIONS; MULTIVARIATE ANALYSIS; and WISHART DISTRIBUTION.

C. C. HEYDE

RANDOMNESS, BDS TEST FOR [This entry is an addendum to RANDOMNESS, TESTS OF.] The BDS statistics introduced by Brock et al. [2] provide a formal statistical test for randomness of the elements of a univariate time series∗ . The test is constructed as follows. Let xt : t = 1, 2, . . . , T denote a sequence of (scalar) observations. Now form a sequence m of T − m + 1m-dimensional vectors: {xm 1 , x2 , m = [x , x }, where x , . . . , x . . . , xm r r+1 r+m−1 ] r T−m+1 and m is a positive integer such that 1
m
T. These vectors have been described [5] as m-histories. Let Cm,T (ε) denote the fraction of all m-histories in the sequence that are within a (Euclidean) distance ε of each other for some positive real number ε, and let Tm = T − m + 1. Then Cm,T (ε) = 2[Tm (Tm − 1)]−1 #{(s, t)|1
m s, t
Tm ,  xm s − xt < ε}, where # denotes the number of elements in the set. The quantity Cm,T (ε) is the correlation integral [5]. The distribution of the statistic √ T[Cm,T (ε) − C1,T (ε)m ] Wm,T (ε) = 2 σm,T (ε) is asymptotically N(0, 1) under the null hypothesis that the elements of the sequence xt are independently and identically distributed (i.i.d.) [2]. Wm,T (ε) is the BDS statistic, and 2 (ε) is an estimate of the variance of σm,T

6865

Cm,T (ε) − C1,T (ε)m under the i.i.d. null hypothesis. The variance can be consistently estimated by  m−1  Km,T (ε)m−i C1,T (ε)2i 4 Km,T (ε)m + 2 i=1

+ (m − 1) C1,T (ε)2m 2



− m2 Km,T (ε)C1,T (ε)2m−2 , where Km,T (ε) = N#{(r, s, t)|1
r, s, t
Tm , m m m  xm r − xs < ε,  xs − xt < ε},

N = 6[Tm (Tm − 1)(Tm − 2)]−1 [1]. BDS statistics have good power against quite a wide range of alternatives to the i.i.d. case. Brock et al. [3] contains a detailed discussion of a number of popular alternatives: these include first-order autoregression, firstorder moving average∗ , threshold autoregression, nonlinear moving average, autoregressive conditional heteroscedasticity (ARCH), and generalized ARCH (GARCH) models (see ARCH AND GARCH MODELS). BDS tests can detect all of these alternatives. Software packages [4,6] are available to calculate BDS statistics for a real scalar time series. It is usual to produce a range of BDS statistics corresponding to several alternative values for each of the two parameters m and ε. Brock et al. [3] recommend values of ε between 12 and 32 times the standard deviation of the data and values of m between 2 and 10. In many applications, we may wish to assess the adequacy of a particular linear or nonlinear model as a model for the scalar time series under investigation. We would proceed by fitting the chosen model to the data and then testing the estimated residuals under the null of i.i.d. using BDS statistics. Acceptance of the null hypothesis supports the choice of fitted model. There is a wealth of useful material on practical applications in ref. [3]. Of particular value are the tables giving size and power of the BDS statistics for a range of values of m and ε, for several alternative distributions for the residuals and for a range of sample sizes T.

6866

RANDOMNESS, COX’S TEST OF

BDS statistics can also provide a general test of nonlinearity of a time series against an unspecified alternative hypothesis. The method involves fitting a ‘‘best’’ linear model to the data and then testing the estimated residuals under i.i.d. using BDS statistics. The existence of some hidden nonlinear structure in the data is indicated if we cannot accept the i.i.d. null; see the discussion in ref. [3]. Alternatives to BDS tests include those proposed by McLeod and Li [7] and Tsay [8]. The BDS test is at least as powerful as both of these for the six alternatives to i.i.d. discussed here [3].

REFERENCES 1. Brock, W. A. and Dechert, W. D. (1988). A general class of specification tests: the scalar case. 1988 Proc. Business and Economic Statistics Section of the Amer. Statist. Ass., pp. 70– 79. 2. Brock, W. A., Dechert, W. D., and Scheinkman, J. (1987). A test for independence based on the correlation dimension. University of Wisconsin–Madison SSRI Workshop, Paper 8702. 3. Brock, W. A., Hsieh, D. A., and LeBaron, B. (1991). Nonlinear Dynamics, Chaos, and Instability: Statistical Theory and Economic Evidence. MIT Press, Cambridge, Mass.

RANDOMNESS, COX’S TEST OF It is often necessary to test whether a sequence of events is random in time, the alternative being that there exists a trend in the rate of occurrence. Let n events occur at times t1 , . . . , tn during the time interval (0, T) and let P[event occurs in(t, t + t)] = λ(t)t + O(t). Assuming that the rate λ(t) = CeBt , Cox’s test of randomness [2] is testing H0 : B = 0 against the alternative H1 : B = 0. The test statistic is m=

n 

ti /nT

i=1

and the distribution of this statistic under H0 is the Irwin–Hall distribution∗ with mean 1 2 and variance 1 / (12 n). As n increases the distribution approaches normality very rapidly and the normal approximation can be used safely for n  20. The power function∗ of the test for values of n = 5 was computed by Bartholomew [1] and for n = 20 (10), 80, 100, 200 by Mansfield [3] for values of M = BT = ±0.10, ±0.20, ±0.40, ±0.80, ±1.2, ±1.6, ±3.0 at the two significance levels∗ α = 0.05 and α = 0.010 for both one-tailed and two-tailed tests. REFERENCES

4. Dechert, W. D. (1987). A Program to Calculate BDS Statistics for the IBM PC. Department of Economics, University of Houston, Houston, Tex.

1. Bartholomew, D. J. (1956). J. R. Statist. Soc. B, 18, 234–239. 2. Cox, D. R. (1955). J. R. Statist. Soc. B, 17, 129–157. [Followed by discussion.]

5. Grassberger, P. and Procaccia, I. (1983). Measuring the strangeness of strange attractors. Physica, 9D, 189–208.

3. Mansfield, E. (1962). Technometrics, 4, 430– 432.

6. LeBaron, B. (1987). A Program to Calculate BDS Statistics for the Macintosh PC. Department of Economics, University of WisconsinMadison, WI.

RANDOMNESS, TESTS OF

7. McLeod, A. and Li, W. (1983). Diagnostic checking ARMA time series models using squaredresidual autocorrelation. J. Time Ser. Anal., 4, 269–273. 8. Tsay, R. (1986). Nonlinearity tests for time series. Biometrika, 73, 461–466. See also RANDOMNESS, TESTS OF and TIME SERIES.

DAVID CHAPPELL

Randomness tests are tests of the null hypothesis that a set of observations or measurements can be considered random. They are an important part of statistical methodology because it is frequently important to examine for nonrandomness data that occur in an ordered sequence or series. Randomness tests are also useful to check on the validity of the assumption of randomness which is inherent in most statistical procedures.

RANDOMNESS, TESTS OF

A test for the null hypothesis of randomness is difficult because there are many different kinds of nonrandomness (e.g., a tendency to cluster or bunch, a tendency to mix or alternate direction, a serial correlation∗ or autocorrelation, a trend∗ , a cyclical or seasonal pattern of movement, etc.). Because of this, the alternative hypothesis is usually quite general, sometimes stating simply non-randomness. The tests covered here are used primarily in this general situation; they include tests based on runs∗ in a dichotomous sequence, on runs up and down in an ordinal measured sequence, on successive differences∗ , on times of occurrence, and ∗ chi-square goodness of fit tests. References for other tests are also given. The conclusion from one of these randomness tests is usually simply a rejection or nonrejection of the null hypothesis. However, when using a test that is known to be particularly sensitive to a specific kind or indication of nonrandomness, the investigator may wish to conclude that the sequence does or does not exhibit that specific kind of nonrandomness. When the alternative is more specific (e.g., states an upward or downward trend), other tests may be more appropriate (see TREND TESTS), although the general tests covered here can also be used for such specific alternatives. Tests for the randomness of a set of numbers generated by a computer are also important. Most of the procedures covered in this entry can also be used for this situation, although many others have been developed specifically for such an application. Knuth [37] discusses many tests of this kind. TESTS BASED ON RUNS OF DICHOTOMOUS VARIABLES For an ordered sequence of dichotomous observations, a run is defined as a succession of one or more identical symbols which are preceded and followed by the other symbol or no symbol at all. For example, the ordered sequence of nine symbols, S S F S S F F S F, has three runs of S’s, the first and second of length 2, the third of length 1, and three runs of F’s, of lengths 1, 2, and 1, respectively. Both the total number of runs and their lengths provide clues to lack of

6867

randomness because too few runs, too many runs, a run of excessive length, and so on, rarely occur in a truly random sequence. Of course, the number of runs and their lengths are highly interrelated, so that only one of these characteristics needs to be considered. The total number of runs is the characteristic commonly used to test the null hypothesis of randomness. This is actually a test for nonrandomness of the appearance of S and F in the sequence. Suppose that there are m S’s and n F’s in an ordered sequence of m + n = N dichotoN = N!/m!n! dismous symbols. There are m tinguishable arrangements, and each is equally likely under the null hypothesis of randomness. The null distribution of the total number of runs, R, is easily derived by combinatorial arguments [31,39,58,63] as P(R = r)      n−1 2 m − 1    k−1 k−1      for r = 2k     m+n     m         = m−1 n−1 m−1 n−1  +   k−1 k k k−1          m+n      m    for r = 2k + 1 (1) for k = 1, 2, . . . , N/2. If m and n approach infinity while m/n approaches a constant, Wald and Wolfowitz [63] show that the null distribution of R − 1 − 2mn/N Z= % 2mn(2mn − N) N 2 (N − 1)

(2)

is approximately standard normal. A continuity correction∗ of ±0.5 can be incorporated in the numerator of (2). The alternative A+ : tendency to mix, is supported if there are too many runs and hence a right-tail rejection region (or P-value) is appropriate; the alternative A− : tendency to cluster, calls for a left-tail rejection region. A two-tailed rejection region should be used for the alternative A: nonrandomness. Tables

6868

RANDOMNESS, TESTS OF

of the exact distribution of R for m
n
20 are given in Swed and Eisenhart [61] and reproduced in some textbooks on applied nonparametric or distribution-free statistics (e.g., Bradley [8], Daniel [14], and Gibbons [22]). For an ordered sequence of N observations X1 , X2 , . . . , XN that are measured on at least an ordinal scale, this same runs test can be applied by transforming the observations into two kinds of symbols. David [15] suggests this test for detecting serial correlation in time-series∗ data by comparing each observation with the magnitude of the median M of the sequence (i.e., Xi is replaced by an S if Xi − M  0 and by an F if Xi − M < 0). The distribution of R is given by (1); this test is frequently called runs above and below the median. Goodman [23] proposes another number of runs test, based on the statistic S = NR/(2mn) − 1,

(3)

where R, m, and n are defined as before and Xi is replaced by an S if Xi − k  0 and by an F if Xi − k < 0 for k any appropriate constant. The asymptotic distribution of √ Z = NS is approximately standard normal. Granger [25] shows that this test is quite powerful for detecting serial correlation∗ in nonstationary time series with N  100, m  N/3 and n  N/3, for any k. O’Brien [45] suggests a different test of randomness for a dichotomous sequence; it is a slight modification of the criterion given by Dixon [17] for the hypothesis that two mutually independent random samples are drawn from the same population. O’Brien’s test criterion for m S’s and n F’s is C2 =

n+1 

(Wi − W)2 /m2 ,

(4)

i=1

where

Wi = number of S’s between the (i − 1)th F and the ith F, i = 2, 3, . . . , n, W1 = number of S’s prior to the first F, Wn+1 = number of S’s subsequent to the nth F, n+1  W= Wi /(n + 1) = m/(n + 1). i=1

Small values of C2 suggest a systematic arrangement and large values suggest clustering. Dixon [17, p. 286] gives a table of critical values of C2 for n
m
10 and shows that mn2 C2 /(2N) has approximately the chisquare distribution with n/2 degrees of freedom. O’Brien finds this test more sensitive to multiple clustering than the number of runs test or the rank tests∗ to detect trend. It is, of course, basically a test on the lengths of the runs of the S’s. Another test of randomness for a dichotomous sequence, given in Moore [40], is called the group test, according to David [15]. This test is sequential and therefore uses fewer data in general than do the nonsequential tests mentioned earlier. The alternative H1 is dependence of the kind found in a simple Markov chain. Here we specify both α and β, the probabilities of a Type I and Type II error, respectively, and then take observations one at a time. At each stage we calculate a quantity L and follow the following stopping rule: If L > (1 − β)/α, accept H1 . If L < α/(1 − β), accept H0 . If β(1 − α)
L
(1 − β)/α, take more observations. The L is a function of α, β, the number of runs at that stage, and the conditional probability of an S given that it is preceded by an S. For observations measured on at least an ordinal scale, the dichotomy may be effected by comparing each observation with the median of all observations collected at that stage. A different use of the runs test based on (1) or (2) arises when the data consist of two mutually independent random samples of sizes m and n, say X1 , X2 , . . . , Xm and Y1 , Y2 , . . . , Yn , and the null hypothesis is FX = FY . If the N = m + n observations are pooled and arranged from smallest to largest while keeping track of which observation comes from which sample, the resulting pattern of X’s and Y’s should be well mixed if the samples come from identical populations; hence R is defined here as the number of runs of X’s and Y’s. For the alternative that the populations are not identical, a lefttail critical region for R should be used. Raoul and Sathe [52] give a computer program for this technique, frequently called

RANDOMNESS, TESTS OF

the Wald–Wolfowitz runs test [63]. This test is consistent against a variety of differences between populations, but is not particularly sensitive to any specific kind of difference between the populations (like location) and is not very powerful for specific alternatives (see Gibbons [20] and RUNS).

6869

An equivalent procedure is called the turning points test for randomness based on the number of reversals of direction of movement in the series. The turning points statistic is always one less than R because a reversal cannot occur at XN . TESTS BASED ON SUCCESSIVE DIFFERENCES∗

TESTS BASED ON RUNS UP AND DOWN For an ordered sequence of data measured on at least an ordinal scale, a sequence of two types of symbols can be generated by noting the magnitude of each observation relative to that of the observation immediately preceding it in the sequence. If the preceding value is smaller, a ‘‘run up’’ is started; if it is larger, a ‘‘run down’’ is started. For example, the ordered sequence X1 , X2 , . . . , XN can be transformed into a sequence of (N − 1) plus and minus signs by looking at the signs of the successive differences Xi+1 − Xi for i = 1, 2, . . . , N − 1. The null hypothesis of randomness can be tested using the total number of runs (whether up or down) statistic, say V. The null distribution of V is derived in Kermack and McKendrick [33] and Moore and Wallis [43], and studied further in Olmstead [46]. The most convenient table for N
25 is given in Edgington [18]; this table is reproduced in Bradley [8] and Gibbons [22]. For large samples the standard normal distribution can be used with the random variable Z=

V − (2N − 1)/3 16N−29 90

;

(5)

a continuity correction of ±0.5 can be incorporated in the numerator of (5). If the alternative to randomness is trend or gradual oscillation, the appropriate rejection region is left tail; for the alternative rapid oscillation, the appropriate rejection region is right tail. Methods for computing the asymptotic power of this test are given in Levene [38] and tables are given for alternatives of a constant location shift from one observation to the next and for normal and uniform populations.

In an ordered sequence of N observations measured on at least an ordinal scale, X1 , X2 , . . . , XN , a test of randomness can be based on the squares of the differences of successive observations with the ratio statistic
N−1 2 i=1 (Xi+1 − Xi ) (6) η=
N 2 i=1 (Xi − X) investigated by von Neumann [62]. Young [67] gives critical values of |1 − η/2| for 8
N
25, and Hart [27] gives critical values of η for 4
N
60, while Hart and von Neumann [28] give tail probabilities, in each case for normal samples. For large N (say N  20) and a normal population the statistic  η& (N − 2)/(N 2 − 1) Z= 1− 2 is approximately standard normal [67]. A trend alternative calls for a right-tail rejection region and a short oscillation alternative calls for left-tail. Kleijnen et al. [34] investigate the power of this procedure in testing for independence in simulation subruns and recommend that the number of subruns be at least 100. Bartels [2] gives a distribution-free rank version of the statistic (6) for ordinal scale data as
N−1 2 i=1 (Ri+1 − Ri ) RVN =
N 2 i=1 (Ri − R) =
N

S

i=1 (Ri

−

R)2

=

12S (7) N(N 2 − 1)

where Ri is the rank of Xi in the ordered sequence. This reference gives tables of critical values of S for N
10 and of RVN for 10
N
100 and shows that Z=

RVN − 2 ' 4/N

6870

RANDOMNESS, TESTS OF

is asymptotically standard normal. Monte Carlo∗ comparisons of power against the alternative of first-order autoregression for several different distributions and N = 10, 25, 50 show that this test is always far superior to the runs up and down tests, and frequently superior to the normal theory von Neumann test. Hence this procedure may be used more specifically as a test for serial correlation (see SERIAL CORRELATION). A simpler procedure that is also distribution free is the difference sign test proposed by Wallis and Moore [65], which is simply the number c of positive signs among the Xi+1 − Xi . They give the null distribution; it approaches the normal distribution with mean (N − 1)/2 and variance (N + 1)/12. This procedure is appropriate only with a trend alternative. TESTS BASED ON TIMES OF OCCURRENCE If we are observing the occurrence or nonoccurrence of some phenomenon over a fixed period of time, the time of each occurrence can be noted and then transformed to points on the interval (0, 1). A test for the randomness of these points may be of interest. Bartholomew [3] discusses the test statistic  N+1  1  1   − X  i N + 1, 2 i=1

where X1 , X2 , . . . , XN are successive intervals between the observed points, X1 = 0, and XN+1 = 1 − largest observation. This statistic is then a comparison between each observed interval and the interval expected with uniform spacing, 1/(N + 1), and hence gives a test for randomness of spacing with large values indicating nonrandomness. The moments and asymptotic distribution of this test are developed in Sherman [55] and it is sometimes called the Sherman statistic. Bartholomew [4] examines the properties of a test proposed in Cox [11] that is based on the sum of the times of occurrence. Bartholomew shows a simple relation of this test statistic to the one-sided Kolmogorov– Smirnov∗ goodness-of-fit test and gives some power calculations.

CHI-SQUARE TESTS FOR RANDOMNESS ∗

Chi-square goodness-of-fit tests can sometimes be used to test for randomness in interval scale data by testing the null hypothesis that subsequences of a certain size n have a constant probability θ of a certain dichotomous characteristic, say, Success (S) and Failure (F), and hence the number of S’s in each subsequence follows the binomial distribution with parameters n and θ . For example, a sequence of N observations could be divided into k subsequences, each of size n, where kn = N. Suppose that a success is defined as the event that an observation is an even number. Under the null hypothesis that the observations are independent, identically distributed random variables, the expected number of S’s in each subsequence is   n (0.5)n ei = N i for i = 1, 2, . . . , k. These expected frequencies can be compared with the respective observed frequencies of success fi using the ordinary chi-square goodness-of-fit test. A success could be defined in other ways, as, for example, a final digit greater than or equal to 5, and so on. Wallis and Moore [65] suggest using a chi-square goodness-of-fit test with classes defined as runs of length 1, runs of length 2, and runs of length 3 or more, as a test for randomness. The respective expected frequencies for these classes are 5(N − 3)/12, 11(N − 4)/60, and (4N − 21)/60. ADDITIONAL NOTES Other general tests for randomness have been proposed, some based on the lengths of runs. These are discussed in Kermack and McKendrick [32,33], Mood [39], Mosteller [44], Olmstead [46], David [15], Bateman [7], Goodman [23], Burr and Cane [10], David and Barton [16], and Barton [5]. Mood [39] gives a history of runs tests and extends the theory of runs to sequences consisting of more than two kinds of symbols, as do Wallis and Roberts [66]. Some discussion of power functions appears in David [15], Bateman [7], Barton and David [6], and Levene [38], but power∗ cannot be calculated without giving a specific alternative.

RANDOMNESS, TESTS OF

APPLICATIONS A major area of application or randomness tests is in quality control. If nonrandomness is detected and can then be explained, the cause can perhaps be corrected promptly to restore the process to randomness. Applications in quality control are given in Mosteller [44], Olmstead [47–50], and Prairie et al. [51], and in many quality control∗ textbooks. If we have data that give measurements on at least an ordinal scale as well as times of occurrence, we have time-series data in the traditional sense. This is another important use of tests of randomness, especially when the alternative is trend (see TREND TESTS) or serial correlation (see SERIAL CORRELATION). Alam [1] gives a class of unbiased tests for this situation. In addition to those mentioned specifically here, the best known are those based on the Spearman rank correlation∗ coefficient (Daniels test), the Kendall tau∗ coefficient (Mann test), runs up and down (turning points test), and the rank serial correlation test. Specific procedures with timeseries applications are given in Wallis and Moore [65], Moore and Wallis [43], Wald and Wolfowitz [64], Olmstead [47], Foster and Stuart [19], Cox and Stuart [13], Goodman and Grunfield [24], Sen [54], Knoke [35,36], Gupta and Govindarajulu [26], and in many books. Stuart [59,60] compares the asymptotic relative efficiencies of several of these tests relative to normal regression alternatives. Knoke [36] gives a table of power comparisons for both normal and nonnormal distributions. Table 1.

EXAMPLE An illustration of these procedures is provided for the data given in Bartels [2, p. 42] (see Table 1) on changes in stock levels from 1968–1969 to 1977–1978 after deflating by the Australian gross domestic product (GDP) price index (base period 1966–1967).

Change in Stocks GDP Index Deflated (millions of dollars) (1966-1967 = 100) Stock Change

1968–1969 1969–1970 1970–1971 1971–1972 1972–1973 1973–1974 1974–1975 1975–1976 1976–1977 1977–1978 a Source:

Trend tests might be called tests of randomness of occurrence over time. Tests of randomness of occurrence over space are also sometimes of interest. One group of such tests is based on distance from nearest neighbor; these have applications in statistical ecology for studying the pattern of plant distributions. Some tests based on distance and proposed by others are discussed in Holgate [30]. Other tests are proposed and discussed in Holgate [29]. Pearson [50] compares four tests based on Kolmogorov– Smirnov and von Mises statistics. Brown and Rothery [9] give tests based on squared distances. In a recent survey, Ripley [53] compares seven randomness tests for spatial∗ point patterns that are all based on some function of this distance. Stephens [57] describes tests for randomness of directions in the form of points on the circumference of a unit circle, gives tables, and some results for power. These tests have application in the study of wind directions and migratory habits of birds. A similar study for the three-dimensional problem is given in Stephens [56]. Cox and Lewis [12] give several applications of randomness in series of events.

a

Year

Ref. [2].

561 386 309 −25 −227 893 757 −8 1009 −309

6871

106.2 110.9 117.2 125.5 136.3 155.4 184.5 211.8 234.9 254.1

528 348 264 −20 −167 575 410 −4 430 −122

6872

RANDOMNESS, TESTS OF

We will test the null hypothesis of randomness against the alternative of a trend using three of the procedures given here. The signs of successive differences of the deflated stock changes are −, −, −, −, +, −, −, +, −, so that the runs up and down test statistic is V = 5. The left-tail P-value is 0.2427, so we cannot reject the null hypothesis of randomness by this procedure. The ranks of the deflated stock changes are 9, 6, 5, 3, 1, 10, 7, 4, 8, 2, and the numerator of the rank von Neumann test statistic from (7) is S = 169, which is again not significant. The median of the 10 deflated stock changes is between 264 and 348. If we let B denote an observation below the median and A above, the sequence of changes is A, A, B, B, B, A, A, B, A, B, which has U = 6 runs for m = 5, n = 5. The left-tail P-value then is P > 0.5 so we cannot reject the null hypothesis of randomness. In this example, all three procedures illustrated lead to the same conclusion. This will not always be the case because each procedure is sensitive to randomness in a different way. On the other hand, the Wallis and Moore [65] test designed specifically for trend gives c = 2 with a one-tailed P-value of 0.013, and hence we would conclude a downward trend with this test. REFERENCES 1. Alam, K. (1974). J. Amer. Statist. Ass., 69, 738–739. 2. Bartels, R. (1982). J. Amer. Statist. Ass., 77, 40–46. 3. Bartholomew, D. J. (1954). Biometrika, 41, 556–558. 4. Bartholomew, D. J. (1956). J. R. Statist. Soc. B, 18, 234–239. 5. Barton, D. E. (1967). Technometrics, 9, 682–694. 6. Barton, D. E. and David, F. N. (1958). Biometrika, 45, 253–256.

7. Bateman, G. (1948). Biometrika, 35, 97–112. 8. Bradley, J. V. (1968). Distribution-Free Statistical Tests. Prentice-Hall, Englewood Cliffs, N.J. (Elementary; Chapters 11 and 12 cover runs tests and runs up and down tests, respectively.) 9. Brown, D. and Rothery, P. (1978). Biometrika, 65, 115–122. 10. Burr, E. J. and Cane, G. (1961). Biometrika, 48, 461–465. 11. Cox, D. R. (1955). J. R. Statist. Soc. B, 17, 129–157. 12. Cox, D. R. and Lewis, P. A. W. (1966). The Statistical Analysis of Series of Events. Methuen, London. 13. Cox, D. R. and Stuart, A. (1955). Biometrika, 42, 80–95. 14. Daniel, W. W. (1978). Applied Nonparametric Statistics. Houghton Mifflin, Boston. (Elementary; Section 2.3 covers number of runs tests.) 15. David, F. N. (1947). Biometrika, 34, 335–339. 16. David, F. N. and Barton, D. E. (1962). Combinatorial Chance. Charles Griffin, London. 17. Dixon, W. J. (1940). Ann. Math. Statist., 11, 199–204. 18. Edgington, E. S. (1961). J. Amer. Statist. Ass., 56, 156–159. 19. Foster, F. G. and Stuart, A. (1954). J. R. Statist. Soc. B, 16, 1–22. 20. Gibbons, J. D. (1964). J. R. Statist. Soc. B, 26, 293–304. 21. Gibbons, J. D. (1985). Nonparametric Statistical Inference, 2nd ed. Marcel Dekker, New York. (Intermediate level; mostly theory; runs tests are covered in Chapters 3 and 7.) 22. Gibbons, J. D. (1985). Nonparametric Methods for Quantitative Analysis, 2nd ed. American Sciences Press, Columbus, Ohio. (Elementary; ‘applied approach; many numerical examples; runs tests are covered in Chapter 8.) 23. Goodman, L. A. (1958). Biometrika, 45, 181–197. 24. Goodman, L. A. and Grunfield, Y. (1961). J. Amer. Statist. Ass., 56, 11–26. 25. Granger, G. W. J. (1963). J. Amer. Statist. Ass., 58, 728–736. 26. Gupta, S. S. and Govindarajulu, Z. (1980). Biometrika, 67, 375–380. 27. Hart, B. I. (1942). Ann. Math. Statist., 13, 445–447. 28. Hart, B. I. and von Neumann, J. (1942). Ann. Math. Statist., 13, 207–214. 29. Holgate, P. (1965). J. Ecol., 53, 261–266.

RANDOM NUMBER GENERATION

6873

31. Ising, E. (1925). Zeit. Phys., 31, 253–258.

56. Stephens, M. A. (1964). J. Amer. Statist. Ass., 59, 160–167.

32. Kermack, W. O. and McKendrick, A. G. (1937). Proc. R. Soc. Edinb., 57, 332–376.

57. Stephens, M. A. (1969). J. Amer. Statist. Ass., 64, 280–289.

33. Kermack, W. O. and McKendrick, A. G. (1937). Proc. R. Soc. Edinb., 57, 228–240.

58. Stevens, W. L. (1939). Ann. Eugen. (Lond.), 9, 10–17. 59. Stuart, A. (1954). J. Amer. Statist. Ass., 49, 147–157. 60. Stuart, A. (1956). J. Amer. Statist. Ass., 51, 285–287.

30. Holgate, P. (1965). Biometrika, 52, 345–353.

34. Kleijnen, J. P. C., van der Ven, R. and Sanders, B. (1982). Eur. J. Operat. Res., 9, 92–93. 35. Knoke, J. D. (1975). Biometrika, 62, 571–576. 36. Knoke, J. D. (1977). Biometrika, 64, 523–529. 37. Knuth, D. E. (1968). The Art of Computer Programming, Vol. II. Addison-Wesley, Reading, Mass. 38. Levene, H. (1952). Ann. Math. Statist., 22, 34–56. 39. Mood, A. M. (1940). Ann. Math. Statist., 11, 367–392.

61. Swed, F. S. and Eisenhart, C. (1943). Ann. Math. Statist., 14, 66–87. 62. von Neumann, J. (1941). Ann. Math. Statist., 12, 367–395. 63. Wald, A. and Wolfowitz, J. (1940). Ann. Math. Statist., 11, 147–162. 64. Wald, A. and Wolfowitz, J. (1943). Ann. Math. Statist., 14, 378–388.

42. Moore, P. G. (1958). Biometrika, 45, 89–95; corrigenda: ibid., 46, 279 (1959).

65. Wallis, W. A. and Moore, G. H. (1941). J. Amer. Statist. Ass., 36, 401–409. 66. Wallis, W. A. and Roberts, H. V. (1956). Statistics: A New Approach. Free Press, Glencoe, Ill. (Elementary; runs tests are discussed in Chapter 18.)

43. Moore, G. H. and Wallis, W. A. J. Amer. Statist. Ass., 38, 153–164.

67. Young, L. C. (1941). Ann. Math. Statist., 12, 293–300.

40. Moore, P. G. (1953). Biometrika, 40, 111–114. 41. Moore, P. G. (1955). J. Amer. Statist. Ass., 50, 434–465; corrigenda: ibid., 51, 651 (1956).

(1943).

44. Mosteller, F. (1941). Ann. Math. Statist., 12, 228–232. 45. O’Brien, P. C. 391–401.

(1976).

Biometrics,

32,

46. Olmstead, P. S. (1946). Ann. Math. Statist., 17, 24–33. 47. Olmstead, P. S. (1952). Ind. Quality Control, 9(3), 32–38.

See also GOODNESS OF FIT; RUNS; RUNS TEST, MULTIDIMENSIONAL SERIAL CORRELATION; SUCCESSIVE DIFFERENCES; and TREND TESTS.

JEAN DICKINSON GIBBONS

RANDOM NOISE. See NOISE

48. Olmstead, P. S. (1958). Bell Syst. Tech. J., 37, 55–82. 49. Olmstead, P. S. (1958). Statistical quality control in research and development. Trans. 13th Midwest Quality Control Conf., Kansas City, Mo., Nov. 6–7, 1958. 50. Pearson, E. S. 315–326.

(1963).

Biometrika,

50,

51. Prairie, R. R., Zimmer, W. J., and Brookhouse, J. K. (1962). Technometrics, 4, 177–185. 52. Raoul, A. and Sathe, P. T. (1975). J. Quality Tech., 7, 196–199. 53. Ripley, B. D. (1979). J. R. Statist. Soc. B, 41, 368–374. 54. Sen, P. K. (1965). J. Amer. Statist. Ass., 60, 134–147. 55. Sherman, B. (1950). Ann. Math. Statist., 21, 339–361.

RANDOM NUMBER GENERATION The ability to synthesize observations from a wide variety of probability distributions is an essential part of statistical practice. Applications range from Monte Carlo∗ studies of analytically intractable problems in statistical theory (see EDITING STATISTICAL DATA) to simulation∗ studies of complex stochastic systems in operations research∗ . Empirical synthesis of random variables (RVs) has a long history, dating back at least as far as ‘‘Student’’ [16], who discovered the sampling distributions of the t-statistic and the correlation coefficient∗ from simulations based on height and finger measurements of criminals. More recently, digital computers∗

6874

RANDOM NUMBER GENERATION

have become invaluable aids in simulation and Monte Carlo work, so that now almost all synthetic observations or random numbers are generated by a computer when required, instead of being taken from a table. The process begins by generating observations from the uniform distribution∗ on the interval (0, 1), which can be done quickly with the computer’s arithmetic operations. If the application requires observations from some nonuniform distribution, either continuous or discrete, a variety of techniques are available. Some of these are standard methods, applicable to many distributions; others are clever special-purpose devices, invented for a single distribution. This article describes the basic techniques for generating uniform random numbers, several common techniques for continuous distributions, and some techniques for discrete distributions. It also examines the journal literature, where new methods continue to advance the state of the art. UNIFORM RANDOM NUMBERS Almost all algorithms for generating nonuniform random numbers rely on a sequence {Un } of independent observations from the uniform distribution on the interval (0, 1). In practice, the uniform ‘‘random numbers’’ that play the role of {Un ·} are not precisely random. Instead, pseudo-random numbers are obtained by some mechanism that, it is hoped, produces an adequate approximation to randomness. Some early computers used electronic circuits or electromechanical devices; they could deliver close approximations to randomness, but required continuing surveillance to ensure that they were functioning properly. Also, because their output was not reproducible, debugging a program that used them was rather uncertain. To avoid these practical difficulties, John von Neumann and others suggested using the computer’s arithmetic operations to produce sequences of numbers that, while being entirely deterministic, had the appearance of randomness. Von Neumann’s ‘‘middlesquare’’ method, however, turned out to be quite unsatisfactory (see Knuth [7]). For reasons of reproducibility and convenience, most sources of uniform pseudorandom numbers now use a technique suggested

by D. H. Lehmer in 1948. This is the linear congruential generator, which constructs a sequence of integers through simple arithmetic, multiplying the previous integer by a constant, adding another constant, and taking the remainder from division by m: Xn+1 = aXn + c(mod m).

(1)

The corresponding uniform deviates are Un = Xn /m. The modulus, m, is usually a large integer, related to the largest integer that the computer can hold in one word of storage. The multiplier, a, is an integer between 1 and m, chosen so that when a term Xn in the sequence is multiplied by a, increased by the increment c, and then reduced to its remainder modulo m, the result, Xn+1 , appears to be unrelated to Xn . The starting value or seed, X0 , provides a simple way of reproducing any portion of the sequence. Many generators in common use follow a special case (c = 0) of (1) known as the multiplicative congruential generator: Xn+1 = aXn (mod m).

(2)

Both in this form and in the linear congruential form of (1), the relationship between a and m determines how well the generated sequence approximates a truly random one. The modulus m controls how coarse the discrete sequence {Un } is, compared with a continuous uniform sequence, because the Un are all integer multiples of 1/m. It is customary to use a large value of m and to neglect the discreteness of the generated numbers, especially when consecutive numbers from the interval (0, 1) that can be exactly represented in the computer differ by 1/m. Another reason for using a fairly large modulus m is that the longest sequence generated by (1) or (2) is cyclic. That is, it eventually starts over at X0 . Because the remainders modulo m are 0, 1, . . . , m − 1, the length of the cycle cannot exceed m. In the linear congruential generator one can always choose a and c so that this maximum is achieved; but the best possible result in the multiplicative congruential generator depends on m, as well as on a suitable choice of a. For example, when m is a power of 2, the maximum cycle length is m/4, a must leave a remainder of either 3

RANDOM NUMBER GENERATION

or 5 modulo 8, and X0 must be odd. Or, when m is a prime number, the maximum cycle length is m − 1, and a must, in the language of number theory, be a ‘‘primitive root’’ of m. These conditions (and analogous ones for other cases) are not particularly restrictive; when m is large, a great many potential values of a satisfy them. Not surprisingly, such a value of a ensures that the sequence will not run out prematurely, but it need not provide an adequate approximation to randomness∗ . That involves more complicated criteria and generally requires a search among potential a-values satisfying the basic conditions. QUALITY OF CONGRUENTIAL GENERATORS Unavoidably, congruential generators, multiplicative and linear alike, are far from perfect. As Marsaglia demonstrated in 1968 [8], all the n-tuples of consecutive numbers which they are capable of generating must, when viewed as points in n-dimensional space, lie on at most a certain number of parallel hyperplanes. The number of such hyperplanes is relatively small, and a poor choice of a can make it quite small indeed. Figure 1 illustrates this crystalline structure in the two-dimensional output of the simple generator Xn+1 = 12Xn (mod17). Given the limitations of the extremely small modulus, 17,

Figure 1. Structure of the pairs generated by the congruential sequence Xn+1 = 12Xn (mod17).

6875

this lattice of pairs is actually quite regular. A paradigm of poor behavior, the generator Xn+1 = 65539Xn (mod231 ), often known as RANDU [4], produces triples that lie on only 15 parallel planes in 3-space, defined by the relation Xn+2 − 6Xn+1 + 9Xn ≡ 0(mod 231 ). Two theoretical tests have been devised to measure the n-dimensional regularity of congruential generators for n  2. The lattice test [9] finds the sides of a fundamental cell in the lattice of n-tuples and uses the length ratio of the longest side to the shortest side as a measure of irregularity. A perfectly cubic lattice would yield a ratio of 1; values below 2 are desirable. Among the families of parallel hyperplanes in the n-dimensional lattice produced by a congruential generator, the spectral test∗ [3,7] finds the most widely separated family and calculates the distance between adjacent hyperplanes. (The name ‘‘spectral test’’ comes from its derivation by Coveyou and MacPherson [3] in terms of the frequencies of n-dimensional waves.) In the 0–1 scale (to facilitate comparisons among generators with different values of m) denote the maximum interhyperplane distance by dn . Knuth [7] works with the n-dimensional ‘‘accuracy,’’ νn = 1/dn , and rates generators according to µn =

π n/2 νnn , (n/2)!m

√ where (n/2)! = (n/2)[(n/2) − 1] . . . ( 12 ) π when n is odd. A good generator should have µn  1 for 2
n
6. The spectral and lattice tests are theoretical in the sense that they work with the full period of the congruential sequence. Over the years, a variety of empirical tests for randomness∗ have been applied to actual generated sequences of random numbers. Unfortunately, many of those tests did not expose generators that later were found to be poor. The spectral test, however, has proved to be much more effective. Knuth points out [7, p. 89] that ‘‘not only do all good generators pass this test, all generators now known to be bad actually fail it.’’ Atkinson [2] discusses the use of these theoretical tests and their comparison with certain empirical tests. For technical reasons the spectral test and lattice test apple directly to only three classes

6876

RANDOM NUMBER GENERATION

of congruential generators: the linear congruential ones of (1), with c = 0 and suitable a; the multiplicative congruential ones of (2) with m = 2e , e  4, and a mod 8 = 5; and (2) with m prime and a a primitive root of m. This limitation causes no practical difficulty, however, because these three classes contain many generators that are adequate for most applications. The discussions of methods for generating nonuniform continuous and discrete random variables (RVs) below assume a perfect source of uniform random numbers. NONUNIFORM CONTINUOUS DISTRIBUTIONS A few general techniques are available for transforming a sequence of uniform random numbers {Un } into a sequence of random observations from a nonuniform continuous distribution. The chief methods use the inverse cumulative distribution function, rejection, mixtures, and the ratio of uniform deviates. Often, however, an algorithm for a particular distribution employs some special property of that distribution (perhaps in conjunction with one or more of the basic general techniques). The literature abounds with imaginative approaches of this kind. For the most common distributions, research has produced several competing algorithms. The dominant goal is speed, but some algorithms introduce a trade-off between speed and storage by using bulky tables. Generally, these algorithms are exact; i.e., they generate the desired distribution as accurately as the finite-precision arithmetic of the computer permits. Exact methods are available for all the most commonly used distributions, including normal, exponential, chi-squared, gamma, and Student’s t. For example, one need never resort to approximating the normal distribution∗ by adding uniform deviates and appealing to the central limit theorem∗ . The algorithm based on (U1 + · · · + U12 ) − 6 is both inexact and far slower than the best exact methods. Inverse-CDF Method Given a RV X with cumulative distribution function (CDF) F, a straightforward probability argument establishes that transforming X by F yields a uniform RV on (0, 1)

[i.e., F(X) = U]. Thus, when it is available in closed form, the inverse CDF, F −1 , provides a convenient link between a source of uniform random numbers and random observations from the desired nonuniform distribution: X = F −1 (U). Two examples are the exponential distribution∗ , where F(x) = 1 − e−x and F −1 (u) = − loge (1 − u), applied as X = − loge U more conveniently, and the logistic distribution∗ , which has F(x) = ex /(1 + ex ) and F −1 (u) = loge (u) − loge (1 − u) = loge [u/(1 − u)]. Use of the inverse CDF is generally slower than other, seemingly more complicated methods; the exponential distribution provides a notable example [1]. When F −1 can only be approximated, other methods are preferable. For example, in addition to being inexact, an approximate inverse CDF for the normal distribution is quite slow. Still, the inverse CDF is an important basic technique, sometimes the only approach available. Rejection Methods One approach to generating random observations from a desired distribution begins with a simpler or more convenient distribution and, in a sense, cuts it to fit. Most commonly, the procedure works with the probability density functions, f for the desired distribution and g for the more convenient one. Both densities are assumed to be nonzero throughout the same finite or infinite interval I. The basic idea is to use g as an envelope for f , choosing a constant, k, such that kg(x)  f (x) for all x ∈ I. Ordinarily, then, k = max{f (x)/g(x)|x ∈ I}. Figure 2 illustrates the relationship between f and kg in a simple situation: f is the density of the beta distribution∗ with parameters 5 and 3, f (x) = 105x4 (1 − x)2 , x ∈ (0, 1), and g is uniform on (0, 1); k = f (2/3) = 2.305. The actual sampling process begins by generating a random observation y from g. It next generates a uniform random number u. If u
f (y)/kg(y), it accepts y as a random observation from f ; otherwise, it rejects y and starts over. In this form the rejection method initially generates points uniformly over the area beneath kg and then rejects those not beneath f . Because g is a density,

RANDOM NUMBER GENERATION

6877

the normal distribution N(µ, σ 2 ) with probability 1 − γ and otherwise (p2 = γ ) yields an observation from N(µ, 9σ 2 ). To generate random numbers from a desired distribution, one can begin with its density as f in (3) and construct components, fi , and mixing probabilities, pi , that yield a fast, exact algorithm [11]. Imaginative application of this approach can involve 30 or more components, few of which bear any resemblance to f . Ratio of Uniform Deviates

Figure 2. Rejection method with a uniform envelope for a beta density, f (x) = 105x4 (1 − x)2 . The maximum value of f is f ( 23 ) = 2.305.

1/k measures the efficiency of the rejection scheme; it is the fraction of observations generated from g that are accepted as observations from f . Thus the example in Fig. 2 is rather inefficient: 1/k = 1/2.305 = 0.434. A number of refinements in the rejection method can improve efficiency. For example, an envelope based on two triangular densities would come much closer to f in Fig. 2. And, when f /g is difficult to evaluate in the rejection step, it is often possible to find a fairly simple function h such that h(x)
f (x)/g(x) throughout I (or a subinterval). Only when u > h(y) is it necessary to calculate f (y)/g(y) in determining acceptance or rejection. Mixture Methods∗ The random process for a mixture first selects a component distribution and then samples an observation from that distribution. If the m components have mixing probabilities p1 , . . . , pm and CDFs F1 , . . . , Fm , then the CDF for the mixture distribution is F = p1 F1 + · · · + pm Fm . Similarly, the density function is f = p1 f1 + · · · + pm fm .

(3)

As an example, the occurrence of outliers∗ in data may follow a two-component mixture distribution which yields an observation from

Kinderman and Monahan [5] showed that it is possible to construct efficient algorithms for generating many distributions by sampling a point uniformly from a certain region in the plane and then taking the ratio of the coordinate values of the point. For the density f , the region is Cf = {(u, υ)|0
u
[f (υ/u)]1/2 }.

(4)

If (U, V) is distributed uniformly over Cf , then X = V/U has the desired density, f . Many densities permit an explicit solution of the equation u = [f (υ/u)]1/2 , which serves as one boundary of Cf in (4). For example, the exponential density, f (x) = e−x , x  0, leads to Cf defined by 0
u
1 and 0
υ
−2u loge u, as shown in Fig. 3. Example: Student’s t -Distributions∗ A selection of the algorithms that have been developed to generate random observations from Student’s t with n degrees of freedom serves to emphasize the diversity of approaches. The direct method generates Z from N (0, 1) and, independently, V from chi-squared∗ with'n degrees of freedom and returns X = Z/ V/n. Kinderman et al. [6] discuss several algorithms based on rejection and on mixtures. Kinderman and Monahan use the ratio of uniform deviates. For any n > 2, Marsaglia [10] begins with RNOR and REXP, normal and exponential random variables generated by very fast procedures, and proceeds as follows: Generate A = RNOR, B = A2 /(n − 2), and C = REXP/( 12 n − 1) until e−B−C
1 − B; then exit with T = A[(1 − 2/n)(1 − B)]−1/2 . Marsaglia’s

6878

RANDOM NUMBER GENERATION

Two-Point Distributions

Figure 3. Acceptance region, Cf , for generating exponential random variables by the ratio-ofuniforms method. The curved boundary has the equation υ = −2u loge u.

algorithm exploits a particular property of Student’s t-distributions and has efficiency greater than 1 − 1/n, even if n changes from one use to the next. NONUNIFORM DISCRETE DISTRIBUTIONS Algorithms for generating random variables from common discrete distributions, such as the binomial∗ and the Poisson∗ , generally rely on particular properties of the distribution. Among the few general techniques for discrete distributions, the main ones are table lookup and the method of two-point distributions. Both apply to random variables X that have a finite number of outcomes, x1 , . . . , xk , with corresponding probabilities p1 , . . . , pk such that p1 + · · · + pk = 1. Table Lookup The most straightforward approach generates a uniform random number U on the interval (0, 1); successively checks whether it is less than p1 , p1 + p2 , · · · , p1 + · · · + pk−1 until the comparison succeeds; and returns the corresponding xi as the value of X. Thus, if U < p1 , the result is X = x1 ; and if U  p1 + · · · + pk−1 , the method yields X = xk .

At the heart of this method, devised by Walker [17], is the fact that any discrete distribution with k possible outcomes can be represented as an equiprobable mixture of k two-point distributions. Thus a preparatory step determines the alternative outcomes, a1 , . . . , ak , and the probabilities, f1 , . . . , fk , for a set of two-point distributions. The generation algorithm uses two random numbers: V discrete uniform on 1, . . . , k and U uniform on (0, 1). If U < fV , it will return X = xV ; otherwise, it returns X = aV . The representation in terms of two-point distributions thus eliminates all but one of the comparisons required in table lookup. As an example, for the binomial distribution with n = 3 and p = 0.3, the method of two-point distributions uses the ai and fi shown below. i

xi

pi

ai

fi

1 2 3 4

0 1 2 3

0.343 0.441 0.189 0.027

0 0 0 1

1.0 0.872 0.756 0.108

LITERATURE Generation of random numbers has a long history and a far-flung literature. Bibliographies by Sowey [14,15] and by Nance and Overstreet [12] provide comprehensive coverage up through 1976. Starting with 1975, most articles can be found through Current Index to Statistics∗ . The core journals are ACM Transactions on Mathematical Software; Applied Statistics; Computing; Journal of the American Statistical Association∗ ; Journal of the Royal Statistical Society∗ , Series A; and Mathematics of Computation. The best single account of theory and techniques in the generation of random numbers appears in the book by Knuth [7]. The survey article by Niederreiter [13] includes a mathematical discussion of tests on pseudo-random numbers and the theory of linear congruential and related generators. REFERENCES 1. Ahrens, J. H. and Dieter, U. (1972). Commun. ACM, 15, 873–882.

RANDOM SETS OF POINTS 2. Atkinson, A. C. (1980). Appl. Statist., 29, 164–171. 3. Coveyou, R. R. and MacPherson, R. D. (1967). J. ACM, 14, 100–119. 4. IBM Corporation (1968. System/360 Scientific Subroutine Package (360A-CM-03X) Version III, Programmer’s Manual (H20-0205-3). 5. Kinderman, A. J. and Monahan, J. F. (1977). ACM Trans. Math. Software, 3, 257–260. 6. Kinderman, A. J., Monahan, J. F., and Ramage, J. G. (1977). Math. Comp., 31, 1009–1018. 7. Knuth, D. E. (1981). The Art of Computer Programming, Vol. 2: Seminumerical Algorithms, 2nd ed. Addison-Wesley, Reading, Mass. 8. Marsaglia, G. (1968). Proc. Natl. Acad. Sci. USA, 61, 25–28. 9. Marsaglia, G. (1972). In Applications of Number Theory to Numerical Analysis, S. K. Zaremba, ed. Academic Press, New York, pp. 249–285. 10. Marsaglia, G. (1980). Math. Comp., 34, 235–236. 11. Marsaglia, G., MacLaren, M. D., and Bray, T. A. (1964). Commun. ACM, 7, 4–10. 12. Nance, R. E. and Overstreet, C., Jr. (1972). Computing Rev., 13, 495–508. 13. Niederreiter, H. (1978). Bull. Amer. Math. Soc., 84, 957–1041. 14. Sowey, E. R. (1972). Int. Statist. Rev., 40, 355–371. 15. Sowey, E. R. (1978). Int. Statist. Rev., 46, 89–102. 16. ‘‘Student’’ (W. S. Gosset) (1908). Biometrika, 6, 1–25, 302–310. 17. Walker, A. J. (1977). ACM Trans. Math. Software, 3, 253–256. See also COMPUTERS AND STATISTICS; GENERATION OF RANDOM VARIABLES, COMPUTER; SIMULATION; SPECTRAL ANALYSIS; and UNIFORM DISTRIBUTIONS.

DAVID C. HOAGLIN

RANDOM SAMPLING, SIMPLE. See

6879

random (real) functions, and so on, in a natural sequence of generalizations. The concept of a random set is more subtle and has to be approached obliquely via suitable random functions. However, there is a natural link with a familiar object; a point process (supposed ‘‘unmarked’’) is obviously a random set of a sort, and in the familiar simple cases we can describe it adequately by means of the ‘‘counting random variables’’ N(T) = number of random points in T, where T is a ‘‘test set’’: we write T for the class of test sets in use. This approach fits neatly into the classical theory of (real) stochastic processes∗ , although it should be noted that the customary ‘‘time parameter’’ t has now been replaced by the test set T. But in many important cases it is not very helpful. For example, it is no good if we wish to describe the set of zeros of Brownian motion∗ ; on R it is natural to take the test sets to be open intervals, and then N(T) will almost surely be zero or infinite, for each T. The fruitful idea lying at the root of the whole theory is due to Choquet [3] and is connected with capacity theory. We continue to employ test sets, but now, instead of thinking of them as sampling frames, we think of them as targets which may or may not be hit by the random set. The idea was elaborated about 10 years later, independently by Davidson [4] and Kendall [7], and by Matheron [8]. We now give a necessarily brief sketch. A test set T will be called a trap, and to start with it is sufficient to suppose that the collection of traps consists of enough nonempty sets T to cover the ambient space X in which the random set is to live. We now write f for any 0-1-valued function of T, and we associate with an arbitrary subset E of X a function fE defined as follows: fE (T) = 1(0) when T does (does not) hit E.

SIMPLE RANDOM SAMPLING

RANDOM SETS OF POINTS Probability theory starts with (real) random variables taking their values in R, and then proceeds to consider complex random variables, random vectors, random matrices∗ ,

Now it may happen that the union of E and all the traps disjoint with E is the whole of X. When this is so, we shall call E, T -closed (this is not a topological closure!). We next look at some special 0–1 functions of T, called strong incidence functions (SIFs). An f is to be a SIF when and only when it satisfies the following condition:

6880

RANDOM SETS OF POINTS

SIF: no trap T such that f (T) = 1 can be covered by a (finite or infinite) collection of traps for which f is zero. We shall also need the weaker definition WIF for a weak incidence function; this differs from SIF only in that the collection of traps referred to there must now be finite collections. Obviously, a SIF is a WIF, but not conversely. It is then a basic theorem [7] that f can be expressed in the form fE if and only if it is a SIF, and that while there can be many solutions E to the equation f = fE , there is one and only one that is T -closed. We have thus succeeded in identifying the concept of a SIF with that of a T -closed set, so that to build a theory of random T -closed sets we have only to consider the class of 01-valued T-parametered stochastic processes whose realizations are almost surely SIFs. In practice the choice of the trapping system T will be made so that the random sets we wish to discuss are all T -closed (and there will usually be a variety of acceptable trapping systems, from which we select one that is convenient for our purposes.). There is a hidden obstacle here because of the arbitrary unions permitted in the SIF definition. In practice, we have to start with the concept of a random WIF, for the measurability problems are minimal there, and then pass from that to the concept of a random SIF. It is at this point that the theories in refs. 7 and 8 begin to diverge. For the most part the theory of Matheron is directed to the very important special case when the ambient space is Rk for some finite k. The simplest version of the theory in ref. 7 covers the case when X is second-countable locally compact Hausdorff (which of course allows X to be Rk , or indeed any reasonable manifold). This situation is also dealt with by Matheron. What makes these cases specially simple is that we have some countability present; we can choose the trapping system to be a countable base of open sets or (if X is Rn ) the set of open balls with rational radii and centers. In either case the T -closed sets will be exactly the closed sets in the ordinary topological sense. If for some reason we wish to consider a random open set, we apply the theory to its complement. A more general solution, given in ref. 7, discards all overt reference to a

topology for X and instead imposes two extra set-theoretic conditions on the trapping system; these distil from the simpler topological case described above what is essential for the construction of a (measurable!) random SIF to go through. We omit the details. When working with ordinary (real) random variables the absolutely basic tool is the use of distribution functions and characteristic functions∗ to characterize laws of distribution. This is more than a manipulative tool; it enables us to identify two apparently distinct theoretical constructions as operationally equivalent. The extreme case of this remark is that for practical purposes two real stochastic processes (Yt ) and (Zt ) are the same when, for each m and each (t1 , t2 , . . . , tm ), the vectors (Yt1 , Yt2 , . . . , Ytm ) and (Zt1 , Zt2 , . . . , Ztm ) have the same m-dimensional distribution function. Thus we can speak of the collection of such ‘‘finite-dimensional distributions’’ as the ‘‘name’’ of the process. Two processes with the same name specify the same real situation, although one may have much better analytical properties than the other. So we need to know how we are to specify the name in the case of a random set. The solution in ref. 7 to this problem again exploits a fundamental invention of Choquet: the avoidance function. To simplify the notation, let us write U for any finite union of traps. Then, given a model for a random T closed set, E say, we define the avoidance function A by A(U) = Pr(E does not hit U). Because the random set will ultimately have been defined in terms of a random WIF g, we observe that one can also write A(U) = Pr[g(Tj ) = 0 for each Tj which is a member of the finite collection whose union is U]. Notice that, precisely because g is a WIF, the condition within the brackets is unchanged if we respecify U by representing it in a different way as a finite union of traps. Now such functions A have some beautiful analytical properties. In particular, we always have A(Ø) = 1 (where Ø is the empty set), and furthermore for any traps T1 , T2 , . . . ,

RANDOM SETS OF POINTS

Tn , and any finite union V of (perhaps other) traps, we have   A(V ∪ Ti ) + A(V ∪ Ti ∪ Tj ) A(V) − i

i<j

+ · · · + (−) A(V ∪ T1 ∪ · · · ∪ Tn )  0. n

(This follows from a classical inclusion-exclusion∗ argument.) In addition, the function A(·) will be ‘‘continuous from below’’ in the sense that A(V) converges (downward) to A(U) when V swells up to U as limit. These three conditions characterize an avoidance function associated with a random T -closed set, and there is a good sense in which two constructions of a random T -closed set can be regarded as equivalent for practical purposes when and only when they have the same A. So A is the sought-for ‘‘name,’’ and we now have the basic prerequisites for the random-set calculus. Turning now to the applications, these arise in many biological, geological, and technological contexts wherever the morphological structure (e.g., of composite materials) is of importance. For these, reference should be made to refs. 1, 5, 6, 8, 9, and 11. There are also other quite different applications to control theory∗ , but here further sophistication is needed to allow the introduction of the essential ‘‘filtering’’ concepts. It seems appropriate to give a little more detail in the classical paradigm for stochastic geometry: the theory of random sets of lines. A sensed line L in R2 can be identified relative to a given origin and frame of axes by quoting the pair (p, φ), where φ specifies the direction of the line in [0, 2π )(= S1 ), and P measures its (signed!) lateral displacement from the parallel and similarly sensed line through the origin. In this way we have identified the set of all sensed lines with the cylinder C = R × S1 . But now suppose that the lines are not sensed. We can continue to use the coordinates above, but now we must remember that (p, φ + π ) is to be identified with (−p, φ). A diagram does not much help here, but readers will be able to convince themselves that we can represent all the lines, each of them once only, if we agree to restrict φ to the half-open interval [0, π ). Actually, it is a little more convenient to use the closed interval [0, π ]

6881

and to remember that we are now left with the identification of edges, (p, π ) = (−p, 0). J. F. Adams pointed out that the need to make the final ‘‘twisted’’ identification tells us that topologically the natural home for the family of all unsensed lines is an (open) M¨obius band M. So random sets of unsensed lines can be identified with borel sets in M. In particular, a Poisson field of unsensed lines in R2 is just a Poisson point process on M, and so on. We close with two last references. An alternative approach to random sets will be found in ref. 2; it is very different from that described here. On the historical side the earliest reference to random sets known to me is a paper by Herbert Robbins [10], which, however, does not go into the questions we have been examining. REFERENCES 1. Bartlett, M. S. (1964). The spectral analysis of two-dimensional point processes. Biometrika, 51, 299–311. 2. Carter, D. S. and Prenter, P. M. (1972). Exponential spaces and counting processes. Z. Wahrscheinl. verw. Geb., 21, 1–19. 3. Choquet, G. (1955). Theory of capacities. Ann. Inst. Fourier, 5, 131–295. 4. Davidson, R. (1967). Some Geometry and Analysis in Probability Theory. Smith’s Prize Essay, Cambridge, England. (See ref. 5.) 5. Harding, E. F. and Kendall, D. G., eds. (1974). Stochastic Geometry. Wiley, London. 6. Kendall, D. G. (1974). An introduction to stochastic geometry, pp. 3–9 in ref. 5. 7. Kendall, D. G. (1974). Foundations of a theory of random sets, pp. 322–376 in ref. 5. 8. Matheron, G. (1975). Random Sets and Integral Geometry. Wiley, New York. 9. Ripley, B. D. (1981). Spatial Statistics. Wiley, New York. 10. Robbins, H. E. (1944). On the measure of a random set. Ann. Math. Statist., 15, 70–74. 11. Tweedie, R. L., ed. (1978). Spatial patterns and processes. Suppl. Adv. Appl. Prob., pp. 1–143. See also GEOMETRY IN STATISTICS; RANDOM TESSELLATIONS; and STOCHASTIC PROCESSES.

D. G. KENDALL

6882

RANDOM SUM DISTRIBUTIONS

RANDOM SUM DISTRIBUTIONS Let Xi , i = 1, 2, . . . be independent and identically distributed random variables (i.i.d. RVs) with
characteristic function φ(u) and write Sn = ni=1 Xi . Let N be a nonnegative integervalued random variable which is independent of the sequence {Xi }. Then SN has characteristic function∗ EeiuSN =

∞ 

E(eiuSn |N = n)P(N = n)

n=0

=

∞ 

φ n (u)P(N = n)

n=0

and ESN = ENEX1 , var SN = (EN) var X1 + (var N)(EX1 )2 when the various moments exist. This random sum model has found quite widespread use. For example, an application in entomology interprets N as the number of female insects in a region and Xi as the number of eggs laid by female i, so that SN is the total number of eggs laid in the region. In the case where the distribution of N depends on a parameter λ, a discussion of the asymptotic properties of SN as λ → ∞ has been given by Robbins [6]. When N depends on the Xi , it is not possible in general to obtain the distribution of SN explicitly. However, substantial research relating to random sums has been conducted in two important contexts. These are in the theory of optimal stopping∗ and in the study of the preservation of classical limit theory (laws of large numbers∗ and central limit theorem∗ ) under random time changes. The general theory of optimal stopping is concerned with a finite-mean stochastic process∗ {xn } and stopping rules t. That is, t is a positive integer-valued random variable such that the event {t = n} depends only on the values x1 , . . . , xn . The theory deals with the attainability of V = sup Ext , where the supremum is taken over the class of all possible stopping rules t for which Ext exists. A particular rule is said to be optimal if Ext = V. Many problems within this framework are concerned with random sums. For example,

if the Xi , i = 1, 2, . . ., are i.i.d.
with zero mean and unit variance and Sn = ni=1 Xi , then it is known that there is an optimal stopping rule for the process {n−1 Sn } [e.g., ref. 2, (Theorem 4.11)]. However, an exact description of such an optimal stopping rule is not available, nor is a precise value for V in this context. A comprehensive discussion of optimal stopping∗ is provided in Chow et al. [2]. Randomly stopped sums have some important applications in statistics, for example in sequential analysis∗ . For Wald’s sequential probability ratio test we have Y1 , Y2 , . . ., a sequence of i.i.d. RVs with PDF p(y|θ ), and we define Xi = log[p(Yi |θ1 )/p(Yi |θ0 )]
k and Sk = i=1 Xi . Then, to test H0 : θ = θ0 vs. H1 : θ = θ1 , we use the rule to stop at t = n if Sn
a or Sn  b for appropriately assigned levels a and b. In the context of the preservation of the classical limit theorem under random time changes we first consider the laws of large numbers. Here we take Sn as a sum of n independent random variables, while Nn is any nonnegative integer-valued random variable defined on the same probability space. Then, p a.s. using −→ and −→ to denote almost sure convergence and convergence in probability, respectively, a.s. a.s. n−1 Sn −→ 0 and Nn −→ ∞ a.s. imply Nn−1 SNn −→ 0, and p a.s. n−1 Sn −→ 0 and Nn −→∞ p imply Nn−1 SNn −→0, but p a.s. n−1 Sn −→0 and Nn −→ ∞ p do not imply Nn−1 SNn −→0 as n → ∞ [5, Theorem 10.1]. More detailed assumptions on the behavior of Nn as n → ∞ can be imposed to repair this last result. p Indeed, if n−1 Sn −→0 and for any  > 0 there exists 0 < a = a() < b = b() < ∞

RANDOM TESSELLATIONS

and a sequence {f (n), n  1} with f (n) ↑ ∞ as n → ∞ such that P(af (n) < Nn < bf (n))  1 − 

(1)

p for n > n0 (), then Nn−1 SNn −→0 as n → ∞ [5, Theorem 10.2]. It should be noted that the p condition (1) is weaker than (f (n))−1 Nn −→λ (> 0 a.s.). These results can be extended to more general sums, and in particular to martingales, without difficulty. As an illustration of the foregoing result based on (1), let Z0 = 1, Z1 , Z2 , . . . denote a supercritical Galton–Watson∗ branching process∗ with nondegenerate offspring distribution and let 1 < m = EZ1 < ∞. This branching process evolves in such a way that (2) (Zn ) Zn+1 = Z(1) n + Zn + · · · + Zn ,

where the Z(i) n are i.i.d., each with the offspring distribution. Then, identifying Zn with Nn and Zn+1 − mZn with SNn , we obtain consistency of Z−1 n Zn+1 as an estimator of m, it being known that (1) always holds in this case. For the central limit theorem, comprehensive results are available for the case of i.i.d. RVs. These have found important application in contexts such as renewal theory∗ and queueing theory∗ . As usual, write Sn for the sum of n i.i.d. RVs, here with zero mean p and unit variance. If n−1 Nn −→λ, where λ is −1/2 a positive random variable, then Nn SNn converges in distribution to the unit normal law (e.g., Billingsley [1, Theorem 17.2], where a stronger functional central limit result is established). Sharp results on the rate of this convergence in the case where λ is a constant have been provided by Landers and Rogge [3], and for nonuniform convergence, [4]. For some generalizations to dependent variables, see Billingsley [1, Theorem 17.1]. REFERENCES 1. Billingsley, P. (1968). Convergence of Probability Measures. Wiley, New York. 2. Chow, Y. S., Robbins, H., and Siegmund, D. (1971). Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, Boston. 3. Landers, D. and Rogge, L. (1976). Z. Wahrscheinl. verw. Geb., 36, 269–283.

6883

4. Landers, D. and Rogge, L. (1978). Metrika, 25, 95–114. 5. R´ev´esz, P. (1968). The Laws of Large Numbers. Academic Press, New York. 6. Robbins, H. (1948). Bull. Amer. Math. Soc., 54, 1151–1161. See also OPTIMAL STOPPING RULES; SEQUENTIAL ANALYSIS; and SEQUENTIAL ESTIMATION.

C. C. HEYDE

RANDOM TESSELLATIONS For the purposes of this entry, a tessellation is an arrangement of (convex) polygons fitting together without overlapping so as to cover the plane R2 (or sometimes X ⊂ R2 ); or (convex) polyhedra similarly covering R3 ; or, indeed, (convex) polytopes similarly covering Rd (d = 1, 2, . . .). [Geometry: d-dimensional Euclidean space Rd contains flats of varying dimensions: 0-flat = point, 1-flat = line, 2-flat = plane, 3-flat = solid space, . . . , (d − 1)-flat = hyperplane, d-flat = Rd itself. (s, d) below always relates to ‘‘s-flats in Rd .’’] Everyone is familiar with deterministic tessellations, such as the hexagonal planar or the cubical spatial. However, we do not live in a deterministic world, so that many real-life tessellations are ‘‘random’’ (e.g., the granular structure of metals) both in space and in plane section. Hence the need for a theory of random tessellations, quite apart from their intrinsic interest. Random tessellations naturally lie in the province of geometric probability theory∗ . In particular, the following specific examples are based on the invariant geometric measures and the Poisson fields of geometric objects discussed in that entry. We first discuss certain natural specific random planar tessellations, which comprise random polygons. LINE-GENERATED RANDOM TESSELLATION Suppose that L is an arbitrary (directed) line in R2 , and that P(0, 1) is a Poisson point process of constant intensity 2τ/π on L, with ‘‘particles’’ {ai }. Independently, suppose

6884

RANDOM TESSELLATIONS

that {θi } are independent, identically distributed (i.i.d.) random variables with density 12 sin θ (0
θ < π ). Write Li for the line through ai making an angle θi with L. Then {Li } is P(1, 2), the standard Poisson line process in R2 of intensity τ , with the following basic properties: 1. It is homogeneous and isotropic in R2 [i.e., stochastically invariant under Euclidean motions (of the coordinate frame)]. 2. For any bounded convex domain K of R2 , the number of lines of P(1, 2) intersecting it has a Poisson distribution∗ with mean value τ S(K)/π (S = perimeter). In an obvious way (Fig. 1), P(1, 2) determines a random tessellation L of R2 . For a polygon, write N, S, A for the number of its vertices (or sides), its perimeter, and its area, respectively. P(1, 2) being homogeneous in R2 , it may be anticipated that the values of (N, S, A) for the almost surely countable number of members of L conform to some trivariate probability distribution, and similarly for more general translation-invariant polygon characteristics Z. In fact, if FZ,r is the empiric distribution function (df) of Z for those polygons of L ⊂ the disk center 0 radius r, then there exists a df FZ such that, for any given Z-value Z , almost surely FZ,r (Z ) → FZ (Z ) as r → ∞. A key tool in the proof of such properties is Wiener’s multi-parameter ergodic theorem [36]. Distributions, moments, and so on, corresponding to such df’s FZ are naturally called ‘‘ergodic,’’ a term often omitted in

Figure 1. Random tessellation L.

practice. Loosely speaking, the ergodic distribution of Z for L is the distribution of Z for a ‘‘uniform random’’ member of L. Ergodic Properties of L For further details, see Miles [16,21] and Solomon [30, Chap. 3]. 1. The in-radius distribution is exponential (τ ). 2. The distribution of width in a given direction has density (8τ 2 x/π 3 )K−1 (2τ x/π ) (x  0) [14, p. 183]. 3. The conditional distribution of S given N is (N − 2, τ/π )[∼ sum of N − 2 independent exponential (τ/π ) random variables]. 4. Pr[N = 3] = 2 − (π 2 /6) = 0.355066 Pr[N = 4] = − 31 −

7 2 36 π

+4

 π/2 0

x2 cot xdx

= 0.381466. (Pr[N = 4]: from refs. 34 and 35.) 5. E[N] = 4,

E[N 2 ] = (π 2 + 24)/2;

E[S] = 2π/τ ,

E[S2 ] = π 2 (π 2 + 4)/(2τ 2 );

E[A] = π/τ 2 ,

E[A2 ] = π 4 /(2τ 4 ),

E[A3 ] = 4π 7 /(7τ 6 ); E[AN] = π 3 /(2τ 2 ),

E[SA] = π 4 /(2τ 3 ), E[NS] = π (π 2 + 8)/(2τ ).

Other ergodic distributional properties of L have been estimated by Monte Carlo methods∗ [4]. Many properties extend to the anisotropic case, in which the {θi } have probability element ∝ sin θ d(θ ); the (arbitrary) df  then governs the orientation distribution of L [16]. This model has immediate relevance to the location-time trajectories of a collection of vehicles, each moving with its own constant velocity, along a highway [30, Chap. 3; 32]. If the lines are randomly and independently ‘‘thickened,’’ the thickness w of a line with orientation θ having an arbitrary conditional distribution depending on θ , then the polygonal interstices between the thick lines have

RANDOM TESSELLATIONS

the same ergodic distributions as before [14, p. 167; 16]! Although Kallenberg [10] disproved Davidson’s conjecture that a homogeneous line process with almost surely no parallel lines is necessarily a mixture of anisotropic P(1, 2), for all practical purposes the conjecture applies, severely restricting the range of line-generated homogeneous tessellations. VORONOI TESSELLATION Suppose that P(0, 2) is a Poisson point process of point ‘‘particles’’ with constant intensity ρ in R2 . Those points of R2 closer to a given particle of P(0, 2) than to any other particle constitute a convex polygon, and the aggregate of such polygons determines the Voronoi (sometimes Dirichlet, or Thiessen) random tessellation V relative to P(0, 2) as ‘‘base’’ (Fig. 2). The multiplicity of description reflects the repeated discoveries and manifold applications of this model. Each common side of adjoining polygons is part of the perpendicular bisector of the corresponding particles. Each vertex is a common vertex of three adjoining polygons, and is the circumcenter of the three corresponding particles. No ergodic distributions of V are known, but mean values are E[N] = 6,

E[S] = 4ρ −1/2 , E[A] = ρ −1 .

Gilbert [5] numerically evaluated an integral to find E[A2 ] = 1.280ρ −2 , and various other

6885

ergodic distributional quantities have been estimated in a number of Monte Carlo studies, the most extensive being by Hinde and Miles [8]. Green and Sibson [7] have devised an efficient computer algorithm to generate V relative to given (finite) particle aggregates. Again by computer, Tanemura and Hasegawa [33] iterated the Voronoi operation by replacing each particle of the ‘‘old’’ point process by a ‘‘new’’ particle at the old vertex centroid Their ‘‘limit’’ random tessellation is similar to a regular hexagonal lattice, but without its preferred 120◦ directions; its difference from V is strikingly illustrated on p. 482 of their article. Solomon and Stephens [31] have fitted Pearson∗ and generalized gamma∗ distributions to the limited moment information available for both L and V. DELAUNAY TESSELLATION Constructing the line segments joining neighboring particles of V yields the triangular Delaunay random tessellation D of R2 (Fig. 2), in a certain sense ‘‘dual’’ to V. Its full ergodic distribution is known [17]: representing a triangle by its circumradius r and the directions θ1 , θ2 , θ3 relative to the circumcenter of the particle vertices on its circumference (0
θ1 , θ2 , θ3 < 2π ), r is ergodically independent of (θ1 , θ2 , θ3 ) with densities 2(πρ)2 r3 exp(−πρr2 ) (r  0), (24π 2 )−1 | sin(θ2 − θ3 ) + sin(θ3 − θ1 ) + sin(θ1 − θ2 )|. For further details, see Kendall [11], and for a statistical application, see Mardia et al. [12]. GENERALIZED VORONOI TESSELLATIONS

Figure 2. Random tessellations V (continuous thick line) and D (triangles, dashed thin line).

A point x ∈ R2 has (almost surely) not only a nearest particle of P(0, 2), but also a set of n nearest particles (n = 2, 3, . . .). The set of x having these n particles as nearest n (in some order) is a convex polygon, and the aggregate of such polygons constitutes the generalized Voronoi random tessellation Vn of R2 (n = 2, 3, . . .) [17,24]. Note that V1 ≡ V. R. J. Maillardet has developed

6886

RANDOM TESSELLATIONS

a computer algorithm for generating them up to n ∼ 500—for illustrations, see Miles and Maillardet [24, p. 110]. Unlike any of the tessellations already considered, including V, almost surely they contain no triangles. Two ergodic mean values of {Vn }(n = 1, 2, . . .) are essentially geometrical, demanding simply that (say) the underlying point process be homogeneous and ergodic, with particles almost surely in sufficiently general position; they are En [N] = 6,

En [A] = 1/[(2n − 1)ρ]

[the latter was derived by Miles [17] for P(0, 2) base, its full generality subsequently being demonstrated by Maillardet]. The following geometric identity due to Sibson [28] demands only that the particles be in general position (i.e., there is no need even for randomness). Suppose that y is a particle and that its associated V1 polygon T has N sides. Writing yi for a neighboring particle and Ti for the polygon of V2 corresponding to y and yi (i = 1, . . . , N), we have N Ti and T ⊂ Ui=1 |T|y =

N 

|T ∩ Ti |yi

(| . . . | = area).

i=1

above). Ambartzumian [1] considered homogeneous and isotropic random tessellations T —he calls them ‘‘random mosaics’’—as a special case of random fields∗ of line segments. His approach is through consideration of the intersection of an arbitrary transect line L with T . As above, write {ai , θi } for the successive intersection points of L with the segment sides of T and the accompanying angles. He proved that if {ai } is independent of {θi }, and the θi mutually independent, then the vertices of T can only be of T and X types. Moreover, if there are no T vertices, then T is determined by a τ -mixture of P(1, 2). Later [2] he showed how the ergodic distribution of (ai+1 , −ai , θi , θi+1 ), which does not depend on i, permits the determination of the tessellation side-length ergodic distribution, together with E[A], E[A2 ], and E[AS]; these are stereological∗ results. Miles [22, Sec. 3.4.6] sketched an ‘‘ergodic program’’ for general homogeneous random tessellations, based on the property that the ergodic df’s F are related to the df’s G for the single random polygon containing an arbitrary point a of R2 (e.g., the origin) by  F(dA, dZ) = G(dZ, dZ)/A A−1 G(dA),

This result, which generalizes to all dimensions, has application in spatial interpolation and smoothing. Another interesting P(0, 2)-based random tessellation is ‘‘Gilbert’s radiating segments model’’ [3], for which E[N] = 4 but little else is known.

which has the heuristic meaning that ‘‘the chance that a falls in a given polygon ∝ the area of that polygon.’’ Cowan [3] partly filled in this picture, demonstrating in particular that certain edge effects on the boundary of the increasing polygon sampling circle are asymptotically negligible when homogeneity prevails. He also generalized Matschinski’s result [15].

GENERAL HOMOGENEOUS PLANAR RANDOM TESSELLATIONS

RANDOM TESSELLATIONS IN HIGHER DIMENSIONS AND OTHER SPACES

Knowledge of the foregoing specific random tessellations is relatively complete. When it comes to general random tessellations, present knowledge is rather limited and fragmentary—therein lies a prime area for further research. The first result in this direction [15] was the relation E[N] = 2χ /(χ − 2), where χ is the (ergodic) mean number of sides of the tessellation meeting of each vertex (χ = 4, 3

Poisson planes P(2, 3) tessellate R3 into random convex polyhedra, while Poisson points P(0, 3) form the basis of the Voronoi polyhedral and dual Delaunay tetrahedral tessellations. These polyhedra are ‘‘simple’’ in the sense that almost surely each vertex of each polyhedron lies in three polyhedron faces. Basic properties are given in Miles [20]. Results for the corresponding polytopal tessellations in Rd generated by P(d − 1, d) and

RANDOM TESSELLATIONS

P(0, d), and properties of Poisson flats P(s, d), are to be found in Miles [18,22]. An extensive account of the fundamental role of Poisson flats in the theory of random sets∗ , together with the tessellation determined by P(d − 1, d), is given by Matheron [14]. General flat processes are considered by Kallenberg [9]; in particular, he demonstrated that homogeneous Poisson flat processes are ergodic (= metrically transitive). In an obvious notation, s-flat sections of L(d) simply yield another L(s), but sflat sections of V(d) yield new tessellations V(s, d)(0 < s < d).{V(3, d)}(d = 3, 4, . . .) is a class of polyhedral tessellations of R3 , with the mean number of plane faces decreasing from 15.54 to 13.99 as d increases from 3 to ∝ [20,23]; they are ‘‘normal’’ inasmuch as each edge, vertex belongs to 3, 4 polyhedra, respectively. Extending beyond our original definition, random tessellations on the surface of a sphere [19] and the hyperbolic plane [27] have also been considered. CONCLUDING REMARKS Over the years there have been many empiric studies of naturally occurring irregular tessellations, such as those contained in several papers by Marvin and Matzke [13]; see especially their references. However, the stochastic theory of random tessellations has a relatively brief history, initiated by the pathbreaking approach to L of Santal´o [26] and Goudsmit [6]. Here are some instances of the practical occurrence of random tessellations: L-like ones in the fibrous structure of thin paper sheets, in geological faulting and fragmentation, and as a representation of vehicular traffic along a road (see above); planar V-like ones in mudcrack formations, crystal growth, and all manner of territorial situations (a particle is the ‘‘nucleus’’ of a polygonal ‘‘cell’’); spatial V-like ones in foam and froth, crystal growth, and cellular and granular structure. The tessellation V has recently found useful application in the statistical analysis of point process data [25, Chap. 8]. Many possible additional applications are discussed by Sibson [29]. Future statistical work may demand the development of more diverse classes of specific random tessellations.

6887

REFERENCES 1. Ambartzumian, R. V. (1972). Random fields of segments and random mosaics on the plane. Proc. 6th Berkeley Symp. Math. Statist. Prob., Vol. 3, University of California Press, Berkeley, Calif., pp. 369–381. 2. Ambartzumian, R. V. (1974). In Stochastic Geometry, E. F. Harding and D. G. Kendall, eds. Wiley, London, pp. 176–191. 3. Cowan, R. (1978). The use of the ergodic theorems in random geometry. Suppl. Adv. Appl. Prob., 10, 47–57. 4. Crain, I. K. and Miles, R. E. (1976). Monte Carlo estimates of the distributions of the random polygons determined by random lines in the plane. J. Statist. Comput. Simul., 4, 293–325. 5. Gilbert, E. N. (1962). Random subdivisions of space into crystals. Ann. Math. Statist., 33, 958–972. 6. Goudsmit, S. (1945). Random distribution of lines in a plane. Rev. Mod. Phys., 17, 321–322. 7. Green, P. J. and Sibson, R. (1978). Computing Dirichlet tessellations in the plane. Comput. J., 21, 168–173. 8. Hinde, A. L. and Miles, R. E. (1980). Monte Carlo estimates of the distributions of the random polygons of the Voronoi tessellation with respect to a Poisson process. J. Statist. Comput. Simul., 10, 205–223. 9. Kallenberg, O. (1976). On the structure of stationary flat processes. Z. Wahrscheinl. verw. Geb., 37, 157–174. 10. Kallenberg, O. (1977). A counterexample to R. Davidson’s conjecture on line processes. Math. Proc. Camb. Philos. Soc., 82, 301–307. 11. Kendall, D. G. (1983). The shape of Poisson–Delaunay triangles. In Studies in Probability and Related Topics in Honour of Octav Onicescu, M. C. Demetrescu and M. Iosifescu, eds. Nagard, Montreal, pp. 321–330. 12. Mardia, K. V., Edwards, R., and Puri, M. L. (1977). Analysis of central place theory. Bull. Int. Statist. Inst., 47(2), 93–110. 13. Marvin, J. W. and Matzke, E. B. (1939). Amer. J. Bot., 26, 100–103, 280–288, 288–295, 487–504. 14. Matheron, G. (1975). Random Sets and Integral Geometry. Wiley, New York. 15. Matschinski, M. (1954). Consid´erations statistiques sur les polygones et les poly`edres. Publ. Inst. Statist. Univ. Paris, 3, 179–201. 16. Miles, R. E. (1964). Random polygons determined by random lines in a plane. Proc. Natl. Acad. Sci. (USA), 52, 901–907; II, 1157–1160.

6888

RANDOM UTILITY MODELS

17. Miles, R. E. (1970). On the homogeneous planar Poisson point process. Math. Biosci., 6, 85–127. 18. Miles, R. E. (1971). Poisson flats in Euclidean spaces: Part II: Homogeneous Poisson flats and the complementary theorem. Adv. Appl. Prob., 3, 1–43.

33. Tanemura, M. and Hasegawa, M. (1980). Geometrical models of territory: I. Models for synchronous and asynchronous settlement of territories. J. Theor. Biol., 82, 477–496. 34. Tanner, J. C. (1983). The proportion of quadrilaterals formed by random lines in a plane. J. Appl. Prob., 20, 400–404.

19. Miles, R. E. (1971). Random points, sets and tessellations on the surface of a sphere. Sankhya¯ A, 33, 145–174.

35. Tanner, J. C. (1983). Polygons formed by random lines in a plane: some further results. J. Appl. Prob., 20, 778–787.

20. Miles, R. E. (1972). The random division of space. Suppl. Adv. Appl. Prob., 4, 243–266.

36. Wiener, N. (1939). The ergodic theorem. Duke Math. J., 5, 1–18.

21. Miles, R. E. (1973). The various aggregates of random polygons determined by random lines in a plane. Adv. Math., 10, 256–290.

See also GEOMETRIC PROBABILITY THEORY and RANDOM SETS OF POINTS.

22. Miles, R. E. (1974). In Stochastic Geometry, E. F. Harding and D. G. Kendall, eds. Wiley, London, pp. 202–227. 23. Miles, R. E. (1984). Sectional Voronoi tessellations. Rev. Uni´on Mat. Argentina, 29, 310–327. 24. Miles, R. E. and Maillardet, R. J. (1982). The basic structures of Voronoi and generalized Voronoi polygons. In Essays in Statistical Science, J. Gani and E. J. Hannan, eds., Appl. Prob. Trust. J. Appl. Prob., 19A, 97–111. 25. Ripley, B. D. (1981). Spatial Statistics. Wiley, New York. 26. Santal´o, L. A. (1941). Valor medio del numero de partes en que una figura convexa es dividida por n rectas arbitrarias. Rev. Union Mat. Arg., 7, 33–37. ˜ 27. Santal´o, L. A. and Yanez, I. (1972). Averages for polygons formed by random lines in Euclidean and hyperbolic planes. J. Appl. Prob., 9, 140–157. 28. Sibson, R. (1980). A vector identity for the Dirichlet tessellation. Math. Proc. Camb. Philos. Soc., 87, 151–155. 29. Sibson, R. (1980). The Dirichlet tessellation as an aid in data analysis. Scand. J. Statist., 7, 14–20. 30. Solomon, H. (1978). Geometric Probability. CBMS-NSF Reg. Conf. Ser. Appl. Math. 28. SIAM, Philadelphia. 31. Solomon, H. and Stephens, M. A. (1980). Approximations to densities in geometric probability. J. Appl. Prob., 17, 145–153. 32. Solomon, H. and Wang, P. C. C. (1972). Nonhomogeneous Poisson fields of random lines, with applications to traffic flow. Proc. 6th Berkeley Symp. Math. Statist. Prob., Vol. 3. University of California Press, Berkeley, Calif., pp. 383–400.

R. E. MILES

RANDOM UTILITY MODELS Utility theory∗ is concerned with structures of preferences or indifferences on a set of choice alternatives. Its origins go back at least to the eighteenth century, when Jeremy Bentham and others began the study of what has been called algebraic utility theory. Much of the development has been carried out in economics. In this work a subject’s preferences or indifferences are nonprobabilistic. A landmark in this tradition is the celebrated theorem of von Neumann and Morgenstern [7], giving a set of sufficient conditions for the existence of a (real-valued) utility function on the set of choice alternatives. See UTILITY THEORY. In a probabilistic utility model, on the other hand, it is supposed that when presented with a subset of alternatives A from a universe , the subject will choose item x ∈ A with some probability px (A). Probabilistic models have been developed mainly in psychology, partly in response to the observed probabilistic choice behavior of subjects in psychophysical experiments. The subclass of random utility models has played a prominent role in the theory. A set of choice probabilities {Px (A) : x ∈ A, A ⊆ } is a random utility model if there are random variables {Ux : x ∈ } such that Px (A) = Pr(Ux = max{Uy }), y∈A

x ∈ A ⊆ .

RANDOM UTILITY MODELS

It is usually assumed that the maximizing variable is unique with probability 1. The underlying idea is that a subject’s ‘‘utility’’ for stimulus x is a random quantity Ux , and that he or she makes a choice according to which utility variable is, on that realization of {Ux }, the largest. The classical embodiment of this idea is L. L. Thurstone’s law of comparative judgment [6], according to which the Ux ’s are identically distributed apart from location shifts. Thurstone described five cases for the distributional form of the Ux ; the best known is his case V model, wherein the Ux are independent normally distributed variables with a common variance. This form of the Thurstone model has been widely applied in the scaling of psychometric stimuli and in educational∗ testing. In the latter the ‘‘alternatives’’ might be tasks Ti and respondents Rj , with ‘‘Rj preferred to Ti ’’ taken to mean that Rj is successful at task Ti . The model is also related to probit analysis. Note that the location parameters in Thurstone’s models may be viewed loosely as the ‘‘utilities’’ of the choice alternatives. Thus the important notion of a unidimensional utility scale can be retained in a natural way. Conversely, the Thurstone model will not be appropriate in multidimensional situations, such as commonly arise in market research (preferences among cars, soft drinks, etc.). See also THURSTONE’S THEORY OF COMPARATIVE JUDGMENT. The properties of the general random utility model have for the most part been studied by mathematical psychologists. As in the economic literature for algebraic utility theory, the emphasis has been on the development of axiomatic theory; except in connection with the Thurstone model, there seems to have been little empirical work or application. We conclude by summarizing some of the results; see, for example, Luce and Suppes (1965). The class of random utility models is a very broad one but does not cover all choice situations. Consider, for instance, the guest at a restaurant who, out of deference to his host, will not choose item x, the most expensive on the menu. The introduction of an even more expensive item will then increase the chance of his choosing x. This example

6889

violates the regularity condition Px (A)  px (B)

whenever A ⊆ B.

It is easy to see that any random utility model satisfies regularity. On the other hand, there are some random utility models which violate the rather mild weak stochastic transitivity condition p(x; y)  12 ,

p(y; z) 

1 2

⇒ p(x, z)  12 ,

where p(u; υ) denotes pu ({u, υ}). For example, we can choose three random variables Ux , Uy , Uz (which may even be independent) such that P(Ux > Uy > Uz ) = 0.3 P(Uy > Uz > Ux ) = 0.3 P(Uz > Ux > Uy ) = 0.4. Then p(x; y) = 0.7 and p(y; z) = 0.6 but p(z; x) = 0.7. See Blyth [1] for some interesting examples of this kind. There has been considerable interest in the relationship between the Thurstone form of the independent random utility model and Luce’s choice axiom∗ ; see Yellott [8]. Yellott also characterizes the condition that a vector of independent variables {Ux : x ∈ } is equivalent to another independent vector {Vx }, in the sense that the two collections of utility models obtained by varying the location parameters are the same and thus {Ux } and {Vx } are experimentally indistinguishable. For ||  3, the equivalence condition is essentially that for each x ∈  there are constants ax > 0, bx such that (ax Ux + bx ) has the same distribution as Vx . Strauss [4] gives the corresponding result for arbitrary dependent random vectors U,V. Note that the random utility model with dependencies among the variables is a natural way to model similarities between the choice alternatives [5]. Falmagne [2] derives the necessary and sufficient conditions on the choice probabilities {px (A)} for them to be a random utility model. REFERENCES 1. Blyth, C. (1972). J. Amer. Statist. Ass., 67, 366–373.

6890

RANDOM VARIABLES

2. Falmagne, J. C. (1978). J. Math. Psychol., 18, 52–72. 3. Luce, R. D. and Suppes, P. (1965). In Handbook of Mathematical Psychology, Vol. 3, R. D. Luce, R. R. Bush, and E. Galanter, eds. Wiley, New York. 4. Strauss, D. J. (1979). J. Math. Psychol., 20, 35–52. 5. Strauss, D. J. (1981). Br. J. Math. Statist. Psychol., 34, 50–61. 6. Thurstone, L. L. (1927). Psychol. Rev., 34, 273–286. 7. von Neumann, J. and Morgenstern, O. (1947). Theory of Games and Economic Behavior, 2nd ed. Princeton University Press, Princeton, N.J. 8. Yellott, J. I. (1977). J. Math. Psychol., 15, 109–144.

(a) Empirical Random Variables Examples of empirical r.v.’s include collecting a sample of people and tabulating their heights and weights, taking a random image and measuring the intensity of its pixels, and tabulating the prices of a sample of stocks.

See also CHOICE AXIOM, LUCE’S; DECISION THEORY; THURSTONE’S THEORY OF COMPARATIVE JUDGMENT; and UTILITY THEORY.

(c) Complex Random Variables Examples of complex r.v.’s include the solution to a stochastic differential equation* with a term driven by white noise*, and a random manifold* created by a construction that involves certain random elements.

DAVID J. STRAUSS

RANDOM VARIABLES The somewhat ambiguous concept of a random variable (r.v.) lies at the core of probability theory* and statistics, appearing in many guises. Random variables are also known as stochastic variables and variates. Explicitly, the concept of a r.v. and of its expectation appears in the works of Chebyshev* [1] around 1867. The realization that a random variable fits the general concept of a function followed somewhat later and culminated in A. N. Kolmogorov’s treatise on probability theory in 1933 [4], which asserts that a r.v. is nothing else but a measurable function on a probability space* (see MEASURE THEORY IN PROBABILITY AND STATISTICS). W. Feller emphasized this idea in the influential texts he wrote in the 1950s and 1960s [2]. Rigorous mathematical definitions of r.v.’s are given in all upper level books on probability theory. We shall present a more intuitive and statistical-oriented approach in describing this notion. In a conversation with I. J. Good taped for the American Statistical Association* in 1993, P. Diaconis made a meaningful distinction between various types of random variables that ignores the traditional classification into discrete and continuous classes (see ref. 6):

(b) Elementary Random Variables Examples of elementary r.v.’s include taking a random sample from a finite set having equal likelihood* (such as throwing a fair die), a random sample of normally distributed real numbers or a sample generated via Brownian motion*. This class consists in general of r.v.’s generated by probability distributions.

(d) Bayesian Approach Lauritzen and Spiegelhalter [5] point out while advocating a Bayesian approach that a physician’s diagnosis can be viewed as a random sample from his or her posterior distribution* on the state of a patient’s body, given, 1. his or her personal experience, 2. the degree of his or her knowledge gleaned from the medical literature, and through interaction with colleagues, 3. the patient’s individual medical history, and 4. the patient’s test results. (e) Measurable Functions The measure-theoretic concept of a random variable is that of a measurable function X from a probability space*  into a set X of real numbers. In practice,  plays a secondary role, and one can view X as behaving like a ‘‘floating’’ member of X (similar to a generic point in algebraic geometry). For empirical random variables, the space  is unknowable, for elementary random variables it coincides with X , for complex

RANDOM VARIABLES, COMPUTER-GENERATED

random variables the space  is a huge product of several probability spaces from which the random elements involved in the construction are drawn, and in the Bayesian diagnosis,  is the whole probability model constructed by the physician to describe how ‘‘the world works.’’ One approach in the development of probability theory is to express it, when possible, via measure theory, eliminating specific probabilistic language. The space  is omitted and a measure p(x) or p(x)dx is assigned to the set X. The other approach puts the random variable at the center and uses manipulations of this variable. Typically, the infinite divisibility* (ID) of a real-valued random variable X is defined as a property such that its measure p can be represented for every n via

6891

This second representation seems to be more informative. Some researchers [3,6] attempt to put random variables into foundations of both logic and mathematics. Jaynes [3], among many others, is concerned with the foundations of Bayesian probability in statistics. He views probabilities as developments of a normative theory of any ‘‘home-spun’’ idea of plausibility. Mumford [6] proposes a variant of stochastic predicate calculus to define random variables and asserts that if we make random variables one of the basic elements of mathematics, we will get rid of meaningless conundrums of set theory, such as disproving the continuum hypothesis. For modeling of thought as a computation process, random variables play a pivotal role.

p = qn × . . . × qn (n factors of qn for some probability density qn (x)). The other approach asserts that X is infinitely divisible if for every n x ∼ y1 + . . . + yn , where y1 , . . . , yn are i.i.d. random variables and ∼ means ‘‘distributed as.’’ Mumford [6] provides another example of the dual statement via the Levy–Khinchine* theorem. (a) Non-random Variable Approach X is ID if and only if the Fourier transform (the characteristic function) p(ξ ˆ ) of the measure p(x) can be written p(ξ ˆ ) = exp[iaξ − bξ 2 − c  × (eiξ y − 1 − iξ y/(1 + y2 ))dµ(y)] for some constants a, b, and c and measure µ(y). (b) The Random Variable Approach X is an infinitely divisible random variable if and only if X ∼ a + bXnorm + c

n 

REFERENCES 1. Chebyshev (Tchebychev), P. L. (1867). ‘‘On Mean Quantities (in Russian)’’. In Complete Collected Works, Vol. 2. Moscow-Leningrad (1947). 2. Feller, W. (1957, 1966). An Introduction to Probability Theory and its Applications, Vol. 1, 2. Wiley, New York. 3. Jaynes, E. T. (1996-2000). Probability Theory: The Logic of Science. Cambridge University Press, Cambridge, UK. Available at http://bayes.wustl.edu/ etj/prob.html. 4. Kolmogorov, A. N. (1933). Foundations of Probability Theory. Springer, Berlin. Various editions in several languages. English version translated by N. Morrison (1956); Chelsea, New York. Original version (1933) in German. 5. Lauritzen, S. and Spiegelhalter, D. (1988). Local computations with probabilities on graphical structures. J. R. Stat. Soc. B, 50, 157–224. 6. Mumford, D. (2000). The dawning of the age of stochasticity. Rend. Math. Acc. Lincei, 9, 107–125. See also PROBABILITY, FOUNDATIONS OF.

(Xi − ci ),

i=1

where Xnorm denotes a standard normal random variable, Xi represents a Poisson process*, and the ci are convergence factors.

RANDOM VARIABLES, COMPUTERGENERATED. See RANDOM NUMBER GENERATION

6892

RANDOM VECTOR

RANDOM

VECTOR. See PROBABILITY THEORY: AN OUTLINE and RANDOM VARIABLES RANDOM WALKS By a random walk we will mean a sum of random variables. Some authors restrict the term to mean discrete sums, so that the time variable is an integer, but in this article our terminology will include limiting diffusion processes∗ which are continuous in time. Problems that are naturally phrased in random walk terminology were discussed by Pascal and Fermat (see Maistrov [15]) in their correspondence related to the theory of gambling∗ . The first use of the terminology ‘‘random walk’’ appeared in a note to Nature in 1905 by Karl Pearson [22], in the form of a query: ‘‘A man starts from a point 0 and walks l yards in a straight line: he then turns through any angle whatever and walks another l yards in a straight line. He then repeats this process n times. I require the probability that after these n stretches he is at a distance between r and r + dr from his starting point 0.’’ Even the random walk in this specific formulation had a precedent in earlier literature, as Lord Rayleigh (J. W. R. Strutt) was quick to point out in the same volume of Nature [24–25]. Since this early statement, the random walk has suggested important research directions in probability theory∗ and has played a significant role as a mathematical model for many scientific phenomena. Some of the former are summarized in the monograph by Spitzer [27], and some of the latter are discussed in a recent introductory article [28]. Pearson’s random walk is one in which the increment of change in any step along an axis can take on any value between a minimum and maximum. Problems related to gambling∗ in which discrete units of money change hands suggest the study of random walks on lattice structures and, most recently, random walks defined on more abstract mathematical structures such as groups have been analyzed [13]. Additionally, one can introduce time as a variable in different ways for modeling processes that are continuous in time. For example, consider a random walk on an infinite onedimensional lattice with sites separated by

x, and let the time between successive steps be t. Provided that the mean step size is zero and the variance is finite, it follows that when x, t → 0 in such a way that (x)2 /t = D = constant, the probability density function (PDF) of displacement satisfies ∂ 2P ∂P =D 2. ∂t ∂x

(1)

Aside from its importance in biological and physical applications, the continuum limit is useful as an approximation in many statistical problems because properties of partial differential equations are frequently easier to analyze than those of partial difference equations. An interesting illustration of this notion is provided in a monograph of Khintchin [14], in which many useful central limit∗ results are obtained by analyzing continuous rather than discrete processes. The solution to (1) for a diffusing particle initially at x = 0 is p(x, t) = (4π Dt)−1/2 exp[−x2 /(4Dt)],

(2)

which is seen to be a normal distribution with mean equal to zero and variance equal to 2Dt. The time t therefore plays the same role as n, the number of steps of a discrete random walk or the number of terms in a discrete sum. Equation (2) can also be obtained by passing to the continuum limit of the solution to the discrete analog of (1), which is a heuristic way of establishing the relation between the sum of independent random variables and the diffusion limit. We note that the problems mentioned so far involve sums of independent random variables. There are important and particularly difficult problems posed by sums of dependent random variables which have so far eluded complete solution. These will be mentioned later. In the study of random walks with independent steps a central role is played by the characteristic function∗ , or the appropriate analog for functions in continuous time. For example, for a random walk on a Ddimensional Euclidean lattice, let p(j1 , j2 , . . . jD ) be the probability that the transition in a single step is equal to j (note here the assumption of a homogeneous lattice). A convenient

RANDOM WALKS

definition of the generating function is  p(j) exp(ij ·
), λ(
) =

(3)

{j}

where j ·
= j1 1 + j2 2 + · · · + jD D is a dot product. The probability that a random walker starting from r = 0 is at site r at step n is  π  π 1 ··· λn (
) exp(−ir ·
) Pn (r) = (2π )D −π −π d1 d2 · · · dD .

(4)

The central limit theorem together with correction terms can be obtained from this representation when n is large and the appropriate moments of single-step transition probabilities are finite. A second function important in many analyses is the probability that a random walker reachers r for the first time at step n. This function is sometimes called the first-passage-time probability. It will be denoted by Fn (r) and is related to Pn (r) by Pn (r) =

n 

r = 0

Fj (r)Pn−j (0),

j=0

= δn,0 δr,0 +

n 

Fj (0)Pn−j (0),

r = 0,

j=1

(5) where δn,0 is a Kronecker delta∗ . While the Fj (r) are not easily related to the Pj (r) directly, the generating functions∗ with respect to the time variable, Fn (r; z) =

∞ 

Fn (r)zn ,

n=0

Pn (r; z) =

∞ 

Pn (r)zn

(6)

n=0

are related by F(r; z) = P(r; z)/P(0; z), = (P(0; z) − 1)/P(0; z),

r = 0 r = 0 (7)

Some interesting properties follow from these identities. A famous problem posed by P´olya in 1921 asks for the probability that a random walker initially at the origin will return to that point at some future step [23].

6893

The probability of such return is, from (6), equal to F(0; 1). Thus, by (7), when P(0; 1) is infinite the random walker is certain to return to the origin (i.e., it does so with probability equal to 1). An explicit expression for P(0; 1) can be derived from (4) and (6):  π  π d1 · · · dD 1 · · · , (8) P(0; 1) = 1 − λ(
) (2π )D −π −π so that the question can be answered by studying properties of λ(
) in different dimensions. When the single-step variances are finite (i.e., ∂ 2 λ/∂2i |
=0 < ∞, i = 1, 2, . . . , D), the return probabilities are equal to 1 in D = 1 and two dimensions, and < 1 in higher dimensions. This is proved by noting that the integrand is singular when
= 0, so that P(0; 1) is finite when the singularity is integrable. The argument proceeds by observing that when the variances of the single-step transition probabilities are finite, 1 − λ(
) is a quadratic form of
in a neighborhood of
= 0. The numerator, transformed to local spherical coordinates, gives a factor D−1 which cancels the singularity when D  3. Random walks that return to the origin with probability equal to 1 are known as recurrent; otherwise, they are termed transient. This terminology also appears in the theory of Markov chains, with a similar meaning. Even in one and two dimensions the average time to return to the origin is infinite. When transition probabilities are in the domain of attraction∗ of stable laws, random walks can be transient even in one dimension. Asymptotic properties of the Fn (r) can be deduced from analytic properties of P(r; z) regarded as a function of z, which, in turn, in most cases of interest can be related to the behavior of λ(
) in a neighborhood of
= 0. More elaborate versions of these classic results have been investigated in recent years [27]. These, for example, replace the notion of recurrence to a point by recurrence to a given set. For this more complicated problem not only the time to reach the set is of interest, but also the site at which it is reached. The time to reach a given set is sometimes termed the hitting probability and the time at which the set is reached is known as the hitting time. All of these problems can be subsumed under the heading of

6894

RANDOM WALKS

first-passage-time problems. Let the random walker initially be at a point belonging to a set S. Let Rn be the random walker’s position at step n. The first passage time for leaving S is defined by the relation / S}. T = min{n|Rn ∈ n

(9)

Much is known about properties of the first passage time, T, particularly in one dimension [12], where the geometry of S is usually a connected interval. In higher dimensions the many possible configurations of S render any general analysis of first passage times quite difficult, although particular symmetric configurations sometimes allow a solution. In one dimension, when S is a connected interval whose boundary consists of its two end points the use of Wald’s identity [31] allows one to generate identities and approximations for the moments of first passage time and the place at which the random walk leaves S. When the second moment of step size is finite, an elegant analysis by Erd¨os and Kac [8] allows one to approximate the PDF of first passage time. The use of these statistics is crucial to sequential analysis∗ and it was in the context of that subject that Wald devised his famous identity [31,32]; see also FUNDAMENTAL IDENTITY OF SEQUENTIAL ANALYSIS. The notion of a random walk with boundaries is suggested by many applications. For example, if the size of a queue is modeled as a random walk, and queue size is limited, say, by a finite waiting room, the maximum queue size can be regarded either as an absorbing or a reflecting barrier. In the first case the random walk is terminated when the barrier is reached, and in the second, the random walk continues after the barrier is reached, but the first event thereafter must lead to a decrease in queue size. Different rules are possible for the behavior of the random walk after contact with a reflecting barrier. The effects of barriers can be included in the mathematical statement of the random walk problem as boundary conditions to the equations that describe the evolution of the system. The boundary condition corresponding to an absorbing barrier sets the PDF of the random walk equal to zero at the barrier. Thus, if a diffusion process whose evolution

is governed by (1) is to include an absorbing barrier at x = 0, one would need to solve (1) subject to the boundary condition P(0, t) = 0. Although there are different possibilities for the definition of a reflecting barrier, the most common usage is to set the gradient of the PDF in a direction normal to the barrier equal to 0. In the case of simple diffusion [exemplified by Equation (1)] a reflecting barrier at x = 0 would be incorporated by the boundary condition  ∂P  = 0. (10) ∂x x=0 The inclusion of any barrier changes the nature of the PDF. As an example, consider (1) with an absorbing barrier at x = 0, and the random walker initially at x = x0 . The resulting PDF is found to be P(x, t) =

   (x − x0 )2 1 exp − 4π Dt 4Dt   (x + x0 )2 , (11) − exp − 4Dt

which has the property that  ∞ lim P(x, t)dx = 0, t→∞ −∞

(12)

in contrast to the barrier-free case, in which the integral = 1 for all time. Sequential analysis can be regarded as the study of random walks in the presence of certain absorbing barriers, which is the point of view adopted in the pioneering studies by Wald [31]. A property of lattice random walks that has attracted attention recently is that of the number of distinct sites visited during the course of an n-step walk. We denote this quantity by Sn . It is not too difficult to find asymptotic expressions for E(Sn ) in different dimensions, but the calculation of higher moments presents a much more challenging problem. One can find E(Sn ) by observing that E(Sn ) =

n   j=0

Fj (r),

(13)

r

which allows us to use (7) together with a Tauberian theorem∗ to infer E(Sn ) for large

RANDOM WALKS

n [17]. No such representation can be given for higher moments. One finds, specifically, that for zero-mean and finite-variance random walks,  1/2  one-dimensional n E(Sn ) ∼ n/ ln n two-dimensional   n three-dimensional

(14)

asymptotically, where ‘‘∼’’ means ‘‘asymptotic to’’ and where each term on the right side of this equation is to be multiplied by a constant calculated from the actual transition probabilities. The basic difficulty in the analysis of Sn is that it is not a Markovian random variable. Jain and Pruitt, by a sophisticated analysis, have determined the asymptotic distribution of Sn in three dimensions [11]. Typical further properties that have attracted research effort include the occupancy of a set by an n-step walk, that is, the fraction of time spent by the random walker in the set [5], and the spans, of a random walk. These are the dimensions of the smallest box with sides parallel to the coordinate axes entirely enclosing the random walk [4]. Another generalization useful in problems that arise in chemical physics is to keep the discrete lattice structure but allow the time between steps to be continuously variable [18]. The literature, both pure and applied, abounds with calculations of different random walk functionals that are too numerous to summarize with any completeness here. APPLICATIONS From the generality of our definition of random walks one might guess that many applications of the basic model have been made. This is indeed the case, and random walk methodology is central to a variety of subject areas in statistics and the natural sciences. Pearson’s original query related to a theory of biological migration that he had developed. At about the same time Bachelier developed a diffusion model for speculation in the stock market [1]. Because even the enumeration of these applications would lead to too large an article, we mention only several representative ones.

6895

A major application of random walks in statistics is to sequential analysis∗ , in which results of an experiment conducted sequentially in time are summarized in terms of a probability ratio, which in turn can be represented as a random walk in a bounded region of the plane. The random walk continues as long as it remains within the region. Termination with acceptance of the appropriate hypothesis occurs when any part of the boundary is reached [32]. A related application is that of CUSUM charts. The statistical questions of interest are related to the determination of the boundaries, the probability of termination at one or another section of the boundary, and the determination of properties of the first passage times for leaving the region of interest. All of these questions lead to difficult analytical problems. The one general result of Wald mentioned earlier provides approximations for the relevant parameters but is valid only in one dimension (although some limited generalizations to higher dimensions have been made). There is not much known about random walks in regions of a general shape because of the mathematical complexity of the related calculations, although some results are available for specific curvilinear boundaries. Any Markov process∗ can be regarded as a random walk, and the many applications of Markov methods lead to corresponding problems about random walks. For example, queueing theory∗ has led to some elegant investigations on properties of onedimensional random walks. A large class of problems relates to the probability of reaching a half-line and first-passage-time problems for that event [20]. These would arise, for example, if one were interested in the maximum queue size during a given interval of time. An interesting early application of these ideas is to the theory of insurance, in which one is interested in the initial amount of capital necessary to insure avoidance of bankruptcy with predetermined probability [26]. This is a sophisticated generalization of the classical gambler’s ruin problem. Similar questions arise in inventory theory∗ and dam theory∗ [19]. Many applications of random walks in the physical sciences have been made which in

6896

RANDOM WALKS

turn have generated some profoundly difficult problems in the theory of random walks. An excellent example of this is provided by the self-avoiding walk. Random walks are often used to model configurational properties of long-chain molecules [8]. Since no two atoms can occupy the same site, one is led to study random walks forbidden to intersect themselves. The mathematical difficulty inherent in these models is that the walks are not Markovian. Few results about self-avoiding walks are known rigorously, although an enormous amount is known by simulation∗ and from approximate theories. As an example of results that differ from standard random walk results, the mean square end-to-end distance of a zero-mean two-dimensional self-avoiding walk is asymptotically proportional to n3/2 in contrast to n for the unrestricted walk. The corresponding result in three dimensions is n6/5 . In four and higher dimensions E(R2n ) ∼ n for both unrestricted and self-avoiding walks. The limiting probability density function for the end-toend distance is distinctly nonnormal in two dimensions, and is probably normal in four or more dimensions, but these results have not been proved rigorously [6]. Another class of problems suggested by applications in both the biological and physical sciences is that of random walks in random environments. The simplest example of this class is a random walk on a lattice in which the transition probabilities associated with each lattice site are themselves random variables. For example, random walks on lattices in which each point is absorbing with probability c < 1 are of current interest in chemical applications [34]. The techniques applicable to homogeneous random walks are not useful for random environment problems (e.g., generating or characteristic functions). An example of a random environment problem is the diffusion process representing population growth∗ in a time-dependent random environment [30]. If the population size at time t is X(t), then a typical model has the form dX = f (X)dt + g(X)dW,

(15)

where dW is W(t + h) − W(t), where W(t) is a representation of Brownian motion∗ and h

tends to 0. Many problems of this kind can be analyzed by the Ito calculus and other methods used for solving stochastic differential equations. There is a considerable literature on random walk models of stock price behavior dating back to the work of Bachelier [1,3,9,21]. One line of research in this area is the determination of the distribution of such quantities as the logarithm of the ratio of stock price on two successive days. It is known that the distribution of this parameter is highly peaked and heavy-tailed compared with a normal distribution. Mandelbrot was the first to suggest that these variables have a stable law∗ distribution [3,9,16]. However, it is not certain at this time whether the distribution is even stationary, and over intervals where stationarity exists, it is not known whether there are more appropriate distributions than stable laws. Whether characterization of stock market behavior as a random walk will lead to useful predictive information is also unclear. Investigators have found that the locomotion of many microorganisms can be described in terms of random walks or diffusion processes. This finding has been used to try to distinguish mechanisms by which microorganisms respond to chemically favorable substances (‘‘chemotaxis’’). The observations show that bacterial motion takes place along approximately straight line segments randomly interrupted by periods during which the organism rotates in place, then chooses a new direction of motion. Thus the organism performs a type of Pearson random walk. Two different phenomenological responses to a chemical attractant have been identified. In one the organism tends to turn in the general direction of the attractant, the lengths of the straight-line segments being uncorrelated with turn angles. In the second the random times spent in the straight-line segments tend to be longer on average when motion is toward the attracting source. Other responses are undoubtedly possible. These experimental investigations suggest further research into developing statistical tests to distinguish between different random walk models. A good reference to this work is ref. 20.

RANDOM WALKS

Other areas in which random walks play an important role include the theory of the Kolmogorov–Smirnov statistics∗ [7] and crystallography [29]. In the first of these, problems related to first passage times to curved boundaries are a focus for current research, while in the second, approximations to the PDF of various forms of the Pearson random walk are of interest. Recently, many models have been suggested for the study of multiparticle random walks in which there is a nontrivial interaction between the different walkers [28]. This class of problems is a simplified form of questions that arise in statistical mechanics, where one is interested in determining the equilibrium states of interacting particles. Progress in this area is difficult and results are mainly available for one-dimensional systems, using techniques that cannot be generalized to the more realistic three-dimensional case. REFERENCES 1. Bachelier, L. (1900). Th´eorie de la speculation. Gauthier-Villars, Paris. 2. Baxter, G. (1961). J. Anal. Math., 9, 31–70. 3. Cootner, P. H. (1964). The Random Character of Stock Market Prices. MIT Press, Cambridge, Mass. (Contains reprints of significant studies prior to 1964. An English translation of Bachelier’s thesis is included.) 4. Daniels, H. E. (1941). Proc. Camb. Philos. Soc., 37, 244–251. 5. Darling, D. A. and Kac, M. (1957). Trans. Amer. Math. Soc., 84, 444–458. 6. Domb, C., Gillis, J. and Wilmers, G. (1965). Proc. Phys. Soc., 85, 625–645. 7. Durbin, J. (1973). Distribution Theory for Tests Based on the Sample Distribution Function. SIAM, Philadelphia. (A compact account of distribution related to Kolmogorov–Smirnov tests and their generalizations.) 8. Erd¨os, P. and Kac, M. (1946). Bull. Amer. Soc., 52, 292–302. (The genesis of the idea of an invariance principle, developed so beautifully later by Donsker.) 9. Fama, E. F. (1965). J. Bus., 38, 34–105. 10. Hermans, J. J., ed. (1978). Polymer Solution Properties: Part II. Hydrodynamics and Light Scattering. Dowden, Hutchinson & Ross, Stroudsburg, Pa. (A reprinting of classic

6897

papers in polymer physics, many of which relate to the application of random walk theory to the characterization of polymer configurations.) 11. Jain, N. C. and Pruitt, W. E. (1971). J. Anal. Math., 24, 369–393. 12. Kemperman, J. H. B. (1961). The Passage Problem for a Stationary Markov Chain. University of Chicago Press, Chicago. (A tightly written and highly theoretical account of one-dimensional first-passage-time problems. Exact rather than approximate results are emphasized.) 13. Kesten, H. (1959). Trans. Amer. Math. Soc., 92, 336–354. 14. Khintchin, A. (1948). Asymptotische Gesetze der Wahrscheinlichkeitsrechnung. (Reprinted by Chelsea, New York.) (Although 50 years old, a monograph still worth reading for derivations of central limit theorems that emphasize the relation to diffusion processes rather than the use of characterize functions.) 15. Maistrov, L. E. (1974). Probability Theory. A Historical Sketch, S. Kotz, trans. Academic Press, New York. (An interesting account of some of the early problems in probability considered by Cardano, Tartaglia, Fermat, Pascal, and others.) 16. Mandelbrot, B. (1963). J. Bus., 36, 394–419. (Also reprinted in ref. 3.) 17. Montroll, E. W. (1964). Proc. Symp. Appl. Math. Amer. Math. Soc., 16, 193–208. 18. Montroll, E. W. and Weiss, G. H. (1965). J. Math. Phys., 6, 167–181. (Written for physicists, but contains a number of applications of Tauberian methods to random walk problems.) 19. Moran, P. A. P. (1959). The Theory of Storage. Wiley, New York. 20. Nossal, R. (1980). In Biological Growth and Spread. Mathematical Theories and Applications, W. Jager and H. R¨ost, eds. SpringerVerlag, New York, pp. 410–440. 21. Osborne, M. F. M. (1959). Amer., 7, 145–173.

J. Operat.

Res.

22. Pearson, K. (1950). Nature (Lond.), 72, 294. 23. Polya, G. (1921). Math. Ann., 84, 149–160. 24. Rayleigh, Lord (J. W. R. Strutt) Nature (Lond.), 72, 318.

(1905).

25. Rayleigh, Lord (J. W. R. Strutt) (1880). Philos. Mag., 5th Ser., 10, 73–78. 26. Seal, H. (1978). Survival Probabilities: The Goal of Risk Theory. Wiley-Interscience, New York.

6898

RANGE-PRESERVING ESTIMATORS

27. Spitzer, F. (1975). Principles of Random Walk, 2nd ed. Springer-Verlag, New York. (A classic monograph on properties of lattice random walks. Diffusion processes are not covered and few applications are discussed.) 28. Spitzer, F. (1977). Bull. Amer. Math. Soc., 83, 880–890. 29. Srinivasan, R. and Parthasarathy, S. (1976). Some Statistical Applications in X-Ray Crystallography. Pergamon Press, London. (The Pearson random walk and its generalizations are applied as an aid to determining crystal structure from x-ray data.) 30. Turelli, M. (1977). Theor. Popul. Biol., 12, 140–178. (Discussion of the use of stochastic differential equation techniques to ecological models.) 31. Wald, A. (1944). Ann. Math. Statist., 15, 283–294. 32. Wald, A. (1947). Sequential Analysis. Wiley, New York. 33. Weiss, G. H. (1983). Amer. Sci., 71, 65–71. 34. Weiss, G. H. and Rubin, R. J. (1983). Adv. Chem. Phys., 52, 363–504. (A review of analytical techniques for random walks with an account of many applications, mainly in chemical physics. Over 400 references are included.) See also CUMULATIVE SUM CONTROL CHARTS; DIFFUSION PROCESSES; KOLMOGOROV –SMIRNOV STATISTICS; MARKOV PROCESSES; QUEUEING THEORY; RISK THEORY; SEQUENTIAL ANALYSIS; and STABLE DISTRIBUTIONS.

GEORGE H. WEISS

RANGE-PRESERVING ESTIMATORS An estimator is said to be range preserving if its values are confined to the range of what it is to estimate. The property of being range preserving is an essential property of an estimator, a sine qua non. Other properties, such as unbiasedness, may be desirable in some situations, but an unbiased estimator that is not range preserving should be ruled out as an estimator. [We are not speaking of uses of estimators for purposes other than estimation (e.g., as test statistics).] Suppose that the observation vector X takes values in a space X and that the probability distribution P of X is known to belong to a family P of distributions. It is desired to estimate a function θ (P) defined for P ∈ P. To

fix ideas, suppose that θ (P) takes values in a Euclidean space Rk , k  1. Following Hoeffding [3], the set  = {θ (P) : P ∈ P} will be called the prior range of θ (P). The posterior range, x , of θ (P) is, informally, the least set in which θ (P) is known to lie after an observation x from X has been made. For example, if P is the family of the uniform distributions Pη on the interval (η, η + 1), η ∈ R1 , then  = R1 , x = [x − 1, x]. In many common estimation problems the posterior range coincides with the prior range. The estimator t(x) of θ (P) is said to be range preserving if t(x) ∈ x with P probability 1, for all P ∈ P. Some types of estimators are range preserving by definition. Suppose that for each θ  ∈  there is a unique pθ ∈ P such that θ (Pθ  ) = θ  . If a maximum likelihood∗ estimator (defined as a value in  that maximizes the likelihood) exists, it is range preserving. Similarly, Bayes estimators are range-preserving. On the other hand, roots of the maximum likelihood equations may fail to preserve the range (e.g., a sample of n = 2 observations from a mixture of two known probability densities with unknown mixture parameter). The same is true of moment estimators; thus Rider [6] has shown that in the case of a mixture of two exponential distributions the moment estimator is not range preserving. Few general results on range-preserving estimators are available. The fact that standard unbiased estimators of (positive) variance components often can take negative values has attracted considerable attention. LaMotte [4] characterized linear combinations of variance components for which there exist unbiased, nonnegative quadratic estimators. He showed that the ‘‘error’’ component in ANOVA models is the only single component that can be so estimated. Pukelsheim [5] proved analogous results on the estimation of linear combinations of variance–covariance components. Hartung [1] derived nonnegative minimum-biased invariant estimators in variance component∗ models. Hoeffding [3] gave some necessary conditions for the existence of unbiased

RANGES

range-preserving estimators, including the following. The prior range  of θ (P), P ∈ P, is assumed to be a subset of a Euclidean space Rk . The convex hull of  is denoted C. A supporting hyperplane H = Hc of C at the point θ (P0 ) is given by a c = 0 in Rk such that (c, θ (P))  (c, θ (P0 ))

for all P ∈ P.

Propostion 1. Suppose that there is a P0 ∈ P such that C has a supporting hyperplane H at θ (P0 ). Then if the estimator t(x) is both range preserving and unbiased, we have t(x) ∈  ∩ H with P0 probability 1.

6899

REFERENCES 1. Hartung, J. (1981). Ann. Statist., 9, 278– 292. 2. Hoeffding, W. (1984). J. Amer. Statist. Ass., 79, 712–714. 3. Hoeffding, W. (1983). Unbiased RangePreserving Estimators. A Festschrift for Erich L. Lehmann. Wadsworth, Belmont, Calif., pp. 249–260. 4. LaMotte, L. R. (1973). J. Amer. Statist. Ass., 68, 728–730. 5. Pukelsheim, F. 293–299.

(1981).

Ann.

Statist.,

9,

6. Rider, P. R. (1961). Ann. Math. Statist., 32, 143–147.

Example 1. Let

W. HOEFFDING

X = {0, 1, . . . , n}, P = {Pu : 0
u
1},   n x u (1 − u)n−x , Pu (x) = x x ∈ X ; θ (Pu ) = u. The posterior range of u given x is (0, 1) if 1
x
n − 1; [0, 1) if x = 0; (0, 1] if x = n. The unbiased estimator t(x) = x/n is range preserving. In accordance with Proposition 1, t(x) = 0 with P0 probability 1 and t(x) = 1 with P1 probability 1.

Proposition 2. Let P0 and H satisfy the conditions of Proposition 1. If there is a distribution P ∈ P dominated by P0 and satisfying / H, θ (P ) ∈ then no range-preserving estimator of θ (P) is unbiased. Example 2. Let X1 and X2 be independent normal with common unknown mean µ and variance 1. Denote by Pµ the corresponding distribution of (X1 , X2 ), and consider estimating θ (Pµ ) = µ2 . By Proposition 2 with P0 = P0 , P = Pµ (µ = 0), H the point 0, no rangepreserving estimator of µ2 is unbiased. (This can be seen directly from the fact that the only unbiased estimate of µ2 is X1 X2 , and the probability of X1 X2 < 0 is positive.) For the case of a sample from a multinomial distribution∗ , Hoeffding [2] gave a necessary and sufficient condition for the (unique) unbiased estimator to be range preserving.

RANGES The range of a statistical distribution or random variable X is b − a, where [a, b] is the support of X. The range is infinite if a or b is infinite. The range of an ordered set of data x(1)
x(2)
· · ·
x(n) is w = x(n) − x(1) . The data set is of most interest when it comprises realizations of order statistics∗ X(1) , . . . , X(n) from a sample of size n based on a parent distribution having cumulative distribution function∗ (CDF) F(·) and probability density function (PDF) f (·), with W = X(n) − X(1) . Other statistics related to the range are: 1. Quasi-ranges W(r) = X(n−r+1) − X(r) , r
[ 12 n]. 2. Studentized ranges∗ , discussed in the entry of that name. 3. The mean range Wn,k of k sample ranges, each based on samples of size n. 4. Range ratios (see David [7, Secs. 5.4, 7.7]). 5. The bivariate range R. If (Xi , Yi ) for i = 1, . . . , n are n target impact points, say, then R is the maximum distance between any two of them, that is, R = max[(Xi − Xj )2 + (Yi − Yj )2 ]1/2 . i=j

6900

RANGES

Cacoullos and DeCicco [3] give approximations to the distribution and percentage points of R when the data come from a bivariate normal distribution.

the PDF of the range being the special case r = 1. Further, E(W(r) k ) =

(n − 2r + k)!n! , (n − 2r)!(n + k)! k = 1, 2, 3, . . . ,

BASIC PROPERTIES In most applications F is continuous. The CDF of W is then given by  G(w) = n

∞ −∞

f (x)[F(x + w) − F(x)]n−1 dx, w > 0,

(1)

with PDF  g(w) = n(n − 1)

∞

−∞

[F(x + w) − F(x)]n−2 × f (x)f (x + w)dx,

w > 0. (2)

The moments can be derived from those of X(1) and X(n) ; for example,

and in particular, n−1 , n+1 2(n − 1) . var(W) = (n + 2)(n + 1)2 E(W) =

Example 2. X is normally distributed with variance σ 2 . A summary of results appears in Patel and Read [2], Secs. 8.4, 8.5], including expressions for the PDFs of W and W(r) ; explicit expressions for the distribution of W are given for the cases n = 2, 3, and 4, and for moments of W for n = 2, 3, 4, and 5. Some approximations to the distributions of W and of W(r) are given; see also David [7, Sec. 7.3]. Approximations to the distribution of W when the parent distribution is normal are based on chi∗ or chi-square∗ . The most accurate and adequate is given [4,7,24] by

E(W) = E(X(n) ) − E(X(1) ), var(W) = var(X(n) ) − 2 cov(X(1) , X(n) ) + var(X(1) ).

W/σ = (χν2 /c)α ,

The PDF of the quasi-range W(r) is given by n! gr (w) = [(r − 1)!]2 (n − 2r)!  ∞ × f (x)f (x + w) −∞

×[F(x)(1 − F(x + w))]r−1 ×[F(x + w) − F(x)]n−2r dx.

(3)

In most applications F is rectangular or normal. Studies of W and of W(r) have been made, however, for cases where F has a gamma∗ [13] or a logistic distribution∗ [14]. Example 1. X is rectangular∗ on the interval 0
x
1. Then W(r) has a beta distribution∗ with PDF gr (w) =

1 wn−2r B(n − 2r + 1, 2r) ×(1 − w)2r−1 ,

0
w
1,

where ν, c, and α are derived by equating the first three moments of W/σ with those of the right-hand side. These approximations were also developed for the distribution of the mean range Wn,k under normality assumptions; see Bland et al. [2] for exact distributions of W2,2 , W2,3 , W3,2 , and W2,4 . A listing of available tables of percent points, cumulative probabilities and moment values of W, W(r) , the ratio of independent ranges W1 /W2 , and of studentized ranges appears in Patel and Read [21, Table 8.1]. Thus Owen [20, pp. 139–140] gives percent points and moments of W for n
100 to three decimal places; Harter [10] gives these, and CDF and PDF values to at least six places, as well as percent points and CDF and PDF values for the ratio W1 /W2 with samples up to size 15. Harter [11] tabulates percent points, CDF, and moment values of W(r) to at least five places. See also Pearson and Hartley [23].

RANGES

Barnard [1] gives a FORTRAN computer program for the probability integral of W under normality assumptions; see also el Lozy [9]. RANGES IN INFERENCE The range W, quasi-range W(r) , and mean range W are used primarily as measures of dispersion. The use of the range in place of the standard deviation σ in examining stability of variation in industrial processes was suggested in 1927 by Student [27]. In 1925, Tippett [28] studied the estimation of σ by W and fitted Pearson-type curves to the CDF of W under normality by the method of moments∗ . The exact derivation of the CDF of W was given by Hartley [12], but in the meantime the use of the range to measure variation in control charts∗ had been proposed by Pearson and Haines [22] as an alternative to the standard deviation. This occurred in 1933, only two years after Shewhart [26] had introduced control charts as a practical tool in quality control∗ . The impetus to measure variation by the range comes, of course, from its greater ease of computation. Since the efficiency∗ of W as an estimator of σ in normal populations is at least 0.91 when n
7 and at least 0.85 when n
10 (see David [7, Sec. 7.3]), the range is still used in computing control limits for x-charts, and is still used together with the mean range to control for variability in R-charts (see CONTROL CHARTS) because of its ease of calculation, but also because it compares favorably with the sample standard deviation s in small samples in nonnormal populations [6]. Table 1 shows for n
30 the values d2 and d3 such that in normal samples E(W) = d2 σ and s.d. (W) = d3 σ ,

Table 1. n

2

3

4

5

6

7

d2 d3 Eff.

1.128 0.853 1

1.693 0.888 0.992

2.059 0.880 0.975

2.326 0.864 0.955

2.534 0.848 0.933

2.704 0.833 0.911

n d2 d3 Eff.

8 2.847 0.820 0.890

10 3.078 0.797 0.850

12 3.258 0.778 0.814

15 3.472 0.756 0.766

20 3.735 0.729 0.700

30 4.086 0.693 –

6901

as well as the efficiency (eff.) of W relative to that of the minimum variance unbiased estimator∗ √ {[ 12 (n − 1)]/[ 2( 12 n)]} 1/2  × (Xi − X)2 of σ. Among quasi-ranges in providing estimators of σ in normal samples (Cadwell [5]), W = W(1) is most efficient for n
17 and W(2) for 18
n
30; W(3) becomes preferable for n  31. Dixon [8] determined subsets of , not necessarily consecuquasi-ranges W(i)
tive, such that k W(i) is unbiased and most efficient in estimating σ among such subsets. For a discussion of a quick test using the mean range in a one-way analysis of variance, see David [7, Sec. 7.7] and Sheesley [25]. Nelson [18,19] summarizes the use of W and of Wn,k in normal samples to estimate σ and test for heterogeneity of variance. McDonald [15–17] has constructed subset selection procedures (choosing a subset of k populations so that, based on available samples, the ‘‘best’’ population is included in the subset with a given probability) for uniform and normal populations, in which the selection is based on range or quasi-range statistics. REFERENCES 1. Barnard, J. (1978). Appl. Statist., 27, 197–198. 2. Bland, R. P., Gilbert, R. D., Kapadia, C. H., and Owen, D. B. (1966). Biometrika, 53, 245–248. 3. Cacoullos, T. and DeCicco, H. (1967). Technometrics, 9, 476–480. 4. Cadwell, J. H. (1953). Biometrika, 40, 336–346. 5. Cadwell, J. H. (1953). Ann. Math. Statist., 24, 603–613. 6. David, H. A. (1962). In Contributions to Order Statistics, A. E. Sarhan and B. G. Greenberg, eds. Wiley, New York, pp. 94–128. 7. David, H. A. (1981). Order Statistics, 2nd ed. Wiley, New York. (A standard work, well written and presented. See Chapter 7 in particular.) 8. Dixon, W. J. (1957). Ann. Math. Statist., 28, 806–809. 9. el Lozy, M. (1982). Appl. Statist., 31, 99.

6902

RANK ANALYSIS OF COVARIANCE

10. Harter, H. L. (1969). Order Statistics and Their Use in Testing and Estimation, Vol. 1. Aerospace Research Laboratories, U.S. Air Force, Dayton, Ohio. 11. Harter, H. L. (1969). Order Statistics and Their Use in Testing and Estimation, Vol. 2. Aerospace Research Laboratories, U.S. Air Force, Dayton, Ohio. 12. Hartley, H. O. (1942). Biometrika, 32, 334–348. 13. Lee, K. R. and Kapadia, C. H. (1982). Commun. Statist. Simul. Comp., 11, 175–195. 14. Malik, H. J. (1980). Commun. Statist. A, 14, 1527–1534. 15. McDonald, G. C. (1976). Technometrics, 18, 343–349. 16. McDonald, G. C. (1977). Commun. Statist. A, 6, 1055–1079. 17. McDonald, G. C. (1978). Sankhya¯ B, 40, 163–191. 18. Nelson, L. S. (1975). J. Quality Tech., 7, 46–48. 19. Nelson, L. S. (1975). J. Quality Tech., 7, 99–100. 20. Owen, D. B. (1962). Handbook of Statistical Tables. Addison-Wesley, Reading, Mass. 21. Patel, J. K. and Read, C. B. (1982). Handbook of the Normal Distribution. Marcel Dekker, New York. (See Chapter 8 for results on the range, quasi-ranges, mean range, studentized ranges, and range ratios.) 22. Pearson, E. S. and Haines, J. (1935). J. R. Statist., Soc. Suppl., 2, 83–98. 23. Pearson, E. S. and Hartley, H. O. (1966). Biometrika Tables for Statisticians, Vol. 1. Cambridge University Press, London. 24. Pillai, K. C. S. (1950). Ann. Math. Statist., 21, 100–105. 25. Sheesley, J. H. (1981). J. Quality Tech., 13, 184–185. 26. Shewhart, W. A. (1931). Economic Control of Quality of Manufactured Product. D. Van Nostrand, New York. 27. Student (1927). Biometrika, 19, 151–164. 28. Tippett, L. H. C. (1925). Biometrika, 17, 364–387. See also CONTROL CHARTS; L-STATISTICS; MIDRANGES; ORDER STATISTICS; and SHORTCUT METHODS.

CAMPBELL B. READ

RANK ANALYSIS OF COVARIANCE. See RANK STATISTICS

RANK CORRELATION, COEFFICIENT OF. See KENDALL’S TAU —I; KENDALL’S TAU —II

RANKED SET SAMPLING Ranked set sampling was first suggested by McIntyre [10], without the supporting mathematical theory, as a method for estimating mean pasture yields. He found that the yield of several small plots could be ranked fairly accurately with negligible cost, while exact measurement of the yield of each was time consuming. To take advantage of this ability, he proposed ranked set sampling, a procedure that improves the efficiency of the sample mean as an estimator of the population mean µ in situations where the characteristic of interest is difficult or expensive to measure, but may be readily ‘‘judgment ordered.’’ This term was introduced later by Dell and Clutter [5,6] to mean ranking of the observations by eye or some other relatively cheap method not requiring actual measurement. Meanwhile, Takahasi and Wakimoto [25] supplied the theory missing from McIntyre’s intuitively appealing suggestion and explored the extent to which the efficiency is improved when the judgment ordering can be performed perfectly. Dell and Clutter go on to show that even if the sample cannot be ranked perfectly by judgment, the procedure may still produce improved precision in the estimator. This is important for their applications in forestry, since accurate ranking is usually not possible. ESTIMATION OF MEAN Mtw Procedure The ranked set sampling procedure, as originally described [10,25], consists of drawing n random samples from the population, each of size n, and (judgment) ranking each of them. Then the smallest observation from the first sample is chosen for measurement, as is the second smallest observation from the second sample. The process continues in this way until the largest observation from the nth sample is measured, for a total of n measured observations, one from each order class. The

RANKED SET SAMPLING

entire cycle is repeated m times until a total of mn2 elements have been drawn from the population but only mn have been measured. These mn observations are the ranked set sample and the selection process described will be referred to as the MTW procedure. For example, suppose that 3 is the maximum number of observations which can be ordered by eye, but enough time (or money) is available for six exact measurements to be made. Then choose n = 3 and m = 2. The sampling scheme can be diagrammed as follows:  | | |  | | | 

 | | |  | | | 

Each row represents a (judgment) ordered sample, and the circled observations indicate those to be chosen for measurement. Let X[j]i denote the jth judgment order statistic* from the ith cycle. Then

µˆ MTW

m n 1  = X[j]i mn i=1 j=1

[6,25] is an unbiased estimator of the population mean µ. Furthermore, its variance is never greater than that of a sample mean from a random sample of size mn. The relative efficiency* (RE) is var X var µˆ MTW −1  n  1 2 = 1 − τj   1, nσ 2

RE(X, µˆ MTW ) =

6903

Table 1. RE(X, µ ˆ MTW ) for Given n Distribution

Uniform Normal Exponential

2

3

4

5

1.50 1.47 1.33

2.00 1.91 1.64

2.50 2.35 1.92

3.00 2.77 2.19

with equality achieved if and only if X is uniform [25]. However, other unimodal distributions show RE close to this maximum value [6], as shown in Table 1. As an example of the computation process, suppose that the values of the circled observations from the sampling scheme diagrammed above are chosen from a normal population and are as given in Table 2. Then µˆ MTW = 19.3 and, if the ranking was done accurately, var(µˆ MTW ) = (1/1.91)(σ 2 /6); that is, the variance of the estimator is nearly halved over that of the sample mean from a sample random sample. It may be possible to judgment order X by ranking on a concomitant∗ Y; that is, in the first sample of size n, the X associated with the smallest Y is measured, in the second sample, the X associated with the second smallest Y is measured, and so on. As in stratified sampling, the strength of the correlation∗ between X and Y determines the amount gained by this procedure. If the regression of X and Y is linear (e.g., bivariate normal), where ρ(X, Y) = ρ, then −1 n ρ2  2 τj  . RE(X, µˆ MTW ) = 1 − nσ 2 

j=1

j=1

where σ 2 is the population variance and τi = E[X[j]i − µ]. These properties remain true when the ranking is imperfect. If judgment ranking is no better than random, RE(X, µˆ MTW ) = 1. The amount of improvement in precision depends on the underlying distribution. When ranking is perfect, so that X[j]i is the jth order statistic, RE(X, µˆ MTW )


1 2 (n

+ 1),

The variance of µˆ MTW can be decreased by an increase in either m, the number of cycles, Table 2. Values of X [j]i Where n = 3, m = 2 j i

1

2

3

1 2

15.6 15.8

15.1 19.9

26.2 23.4

6904

RANKED SET SAMPLING

or n, the size of each sample. However, accurate judgment ordering of a large number of observations would be difficult in most experimental situations, so an increase in sample size is typically implemented by increasing m rather than n. Takahasi [23] has modeled the judgment ordering process for n = 2 by assuming that the elements can be accurately ordered when and only when |X[1]i − X[2]i |  δ. Here δ is a parameter in the judgment error model that represents the smallest possible difference an observer can discern without measurement. (This assumption is conservative, since correct ordering can occur by chance when |X[1]i − X[2]i |
δ.) He found by looking at examples that for distributions that are highly concentrated, judgment ordering and thus ranked set sampling will not be very effective. Other Procedures A generalization of the MTW procedure [27] provides a method for estimating the population mean when N samples of size n are chosen from the population and N (one from each sample) are measured. If N = mn, the best scheme is not necessarily to measure one from each order class. This method, called the YS procedure, differs by the manner in which an observation is chosen for measurement after ranking of each sample. The jth largest observation from the ith sample is chosen for
measurement with probability pij , where nj=1 pij = 1 for each i. Then P = {pij ; i = 1, . . . , N; j = 1, . . . , n} is the selective probability matrix. Then [27] (a) The arithmetic mean of these N measured observations, µˆ YS , is an unbiased
n estimator of µ if and only if i=1 pij = N/n for j = 1, . . . , n. Denote those P satisfying this condition by P. (b) var(µˆ YS )
var(X N ) for P ∈ P. Furthermore, there is a procedure for finding P that minimizes var(µˆ YS ) over P. When N = mn, we have µˆ YS = µˆ MTW by choosing pij = 1 for i = j, j + n, . . . , j + (m − 1)n. The MTW estimator is optimal for this case.

The YS procedure was improved still further [26], by modifying the selective probability matrix to an l × n matrix, where N = 2l [or (l + 1) × n if N = 2l + 1]. After judgment ordering the n observations in each sample, the jth smallest element from the ith sample is paired with the (n + 1 − j)th element from the (N + 1 − i)th sample and they are called ‘‘pair j”. Then pair j is selected for measurement from the ith [and therefore (N +
1 − i)th] sample with probability pij , with nj=1 pij = 1 for i = 1, . . . , l. This is referred to as the MG procedure. The arithmetic mean µˆ MG of these N = 2l measured observations is unbiased if l 

(pij + pi,m+1−j ) = N/m

i=1

for j = 1, . . . , n. The optimal unbiased estimator µˆ MG has variance smaller than the optimal estimator from the YS procedure, unless N = m, in which case they are identical. ESTIMATION OF OTHER PARAMETERS Variance The estimator σˆ 2 =

m  n 

(X[j]i − µˆ MTW )2 /(mn − 1)

i=1 j=1

is an asymptotically (as either m or n increases) unbiased estimator of σ 2 [9]. Further, if mn is large enough, it has greater precision than the usual variance estimator s2 based on a sample of size mn. Examples show that ‘‘large enough’’ is quite small (between 3 and 8) for all distributions considered. The gain from ranked set sampling for estimation of variance is modest compared with that for the mean, however, and it is not likely that the procedure would be worthwhile for estimation of variance alone. Correlation Coefficient Let (X, Y) denote a bivariate normal random vector. Suppose that X is difficult to either measure or rank, but a sample from the Y population can at least be easily ordered. For

RANKED SET SAMPLING

6905

example, consider the problem of a psychologist who wishes to estimate the correlation∗ between an individual (X) and a group (Y) intelligence test. The scores from group tests are readily available from students’ records or can easily be administered. Individual tests are extremely costly since they require a large amount of time and professional expertise to administer and score. (See also SCREENING BY CORRELATED VARIATES). Suppose that a ranked set sample of Y’s and their concomitant X’s are selected and measured in order to estimate their correlation ρ. If the sampling of the Y’s is done according to the MTW procedure, there is no improvement in the precision of any estimator of ρ over Pearson’s correlation coefficient r from a random sample of the same size. If, however, the extreme order statistics∗ from the X sample and their concomitant Y’s are measured, giving a selective probability matrix of the form   p1 0 · · · 0 q1  p2 0 · · · 0 q2    p= . ,   .. pN 0 · · · 0 qN

measured, or at least ranked. In this method, optimally spaced ordered concomitants indicate which subset of the sample should be measured to produce the best estimate of the mean of a symmetrical population. Barnett et al. [1] discuss a procedure that makes use of concomitant variables to estimate ρ in a bivariate normal∗ population. Their asymptotically efficient estimator is computed entirely from the concomitants of the ordered observations of one of the variables, say X, and does not require measurement of the Y’s. Thus their estimator would be valuable when one of the variables is impossible to measure, but can be ranked accurately. Nomination sampling∗ is a procedure for estimating population quantities when exact measurements of only one order statistic from each of several samples are known. This method is not necessarily used because measurement is expensive, but perhaps because only a subset of the values is available (e.g., censoring of the sample may have occurred).

(where pi + qi = 1 for i = 1, . . . , N), then a considerable improvement in estimation of ρ can be achieved.

Later applications of ranked set sampling are prominent in environmental statistics [15,16,17]. It has been used in parametric settings mainly for location- and scaleparametric families [3,4,8,9,22].

[EDITORS’ ADDENDUM]

RELATED METHODS Ranked set sampling has some similarities to stratified sampling, where the population of each of the n order statistics is a stratum. Although the ‘‘stratification’’ is done after the samples are selected, there is an advantage over post-stratification, since the number chosen from each stratum can be controlled. Mosteller [11] investigated methods for choosing the best subset of order statistics from a sample for estimating the mean, variance, or correlation coefficient from normal populations. For this procedure, exact measurements must be made on only the smaller subset, although one must be able to rank the larger sample accurately. A similar scheme [13] is useful when a difficult-to-measure variable has a concomitant that is more easily

Unequal Samples Bhoj [2] proposes a ranked set sampling procedure with unequal samples (RSSU). The steps in RSSU are the same as in RSS. In both procedures only n observations out of n2 ranked units are measured. However in the RSSU approach we draw n samples, where the size of the ith sample is ni , i = 1, 2, . . . , n. Let x(ii)ni denote the ith ordered observation from the ith sample, of size ni = 2i − 1. These observations constitute RSSU
–they are independently distributed and i ni = n2 . Compare this set up with a more restrictive scheme [7] involving unequal allocation, generated via different numbers rj of measurements made corresponding for the jth rank, for j = 1, 2, . . . , n. (In RSSU the actual sample sizes are unequal.)

6906

RANKED SET SAMPLING

Median RSS

REFERENCES

For the family of distributions with c.d.f. of the form F((x − µ))/σ ), where µ and σ are the location and scale parameters, Bhoj discusses estimation of µ and σ using the RSSU scheme and median ranked set sampling (MRSS) [3,12,14]. In MRSS, the same n2 ranked observations as in the original RSS are used; however, the observations with rank (n + 1)/2 (for n odd) are measured and for n = 2m the mth order statistic from the first m samples and the (m + 1)-th order statistic from the last m samples are measured. Then the estimator of µ based on MSRR is

1. Barnett, V., Green, P. J., and Robinson, A. (1976). Concomitants and correlation estimates, Biometrika, 63, 323–328. 2. Bhoj, D. S. (2001). Ranked set sampling with unequal samples. Biometrics, 57, 957–962. 3. Bhoj, D. S. (1997). Estimation of parameters using modified ranked set sampling. In Applied Statistical Science, Vol. 2 (M. Ahsanullah, ed.). Nova Science, New York, pp. 145–163. 4. Bhoj, D. S. and Ahsanullah, M. (1996). Estimation of parameters of the generalized geometric distribution using ranked set sampling. Biometrics, 52, 685–694. 5. David, H. A. and Levine, D. N. (1972). Biometrics, 28, 553–555. (This is an appendix to Dell and Clutter’s paper, which provides some insight into how much damage is done to the estimator by inaccurate ordering.) 6. Dell, T. R. and Clutter, J. L. (1972). Ranked set sampling theory with order statistics background. Biometrics, 28, 545–553. 7. Kaur, A., Patil, G. P., and Taillie, C. (1997). Unequal allocation models for ranked set sampling with skew distributions. Biometrics, 53, 123–130. 8. Lam, K., Sinha, B. K., and Zhong, W. (1994). Estimation of parameters in the twoparameter exponential distribution using ranked set sample. Ann. Inst. Statist. Math., 46, 723–736. 9. Lam, K., Sinha, B. K., and Zhong, W. (1995). Estimation of location and scale parameters of a logistic distribution using a ranked set sample. In Collected Essays in Honor of Professor Herbert A. David (H. N. Nagaraja, P. K. Sen, and D. F. Morrison, eds.), 189–197. Springer, New York. 10. McIntyre, G. A. (1952). Aust. J. Agric. Res., 3, 385–390. 11. Mosteller, F. (1946). Ann. Math. Statist., 17, 377–407. 12. Muttlak, H. A. (1997). Median ranked set sampling. J. Appl. Statist. Sci., 6, 245–255. 13. O’Connell, M. J. and David, H. A. (1976). Essays in Probability and Statistics in Honor of J. Ogawa. Shinko, Tsusho Company, Tokyo, pp. 451–466. (Discusses use of concomitants in double sampling.) ¨ urk, ¨ and Wolfe, D. A. (2000). Alterna¨ 14. Ozt O. tive ranked set sampling protocols for the sign test. Statist. Probab. Letters, 47, 15–23. 15. Patil, G. P., Sinha, A. K., and Taillie, C. (1993a). Ranked set sampling from a finite

∗

µ =

  

1 n 1 n


m

i=1 x(im)

+


n

i=m+1 x(im+1)



,

n even, n odd.


n

i=1 x(ik) ,

This estimator can also be used in nonparametric settings. For RSSU Bhoj proposes six estimators, µ˜ k =

n 

wk xii(ni ) ,

k = 1, . . . , 6,

i=1

with the weights w1 = ni /n2 , w2 = (4ni + 1)/[n(4n + 1)], w3 = (2ni + 1)/[n(2n + 1)], w4 = (4ni + 3)/[n(4n + 3)], 1 [(ni /n) + (2ni + 1)/(2n + 1)], 2n 1 w6 = [(ni /n) + (ni + 1)/(n + 1)]. 2n

w5 =

The first four weights wk are proportional to ni + j/4, j = 0, 1, 2, 3. Note that w5 is the average of w1 and w3 , and w6 is obtained by taking the average of the weights proportional to ni and nj . The weights wi (i = 1, 2, 3) are ‘‘nearly optimal” for Laplace, logistic and normal distributions, respectively, and w5 performs adequately for all three of them. Bhoj [1] also discusses errors in ranking.

RANKING AND SELECTION PROCEDURES

16.

17.

18. 19.

population in the presence of a trend on a site. J. Appl. Statist. Sci., 1, 51–65. Patil, G. P., Sinha, A. K., and Taillie, C. (1993b). Relative precision of ranked set sampling: A comparison with the regression estimator. Environmetrics, 4, 399–412. Patil, G. P., Gore, S. D., and Sinha, A. K. (1994a). Environmental chemistry, statistical modelling, and observational economy. In Environmental Statistics, Assessment and Forecasting, (C. R. Cothern and N. P. Ross, eds.), 57–97. Lewis/CRC, Boca Raton, Florida. Shirahata, S. (1977). Bull. Math. Statist., 17, 33–47. Stokes, S. L. (1977). Commun. Statist. A, 6, 1207–1211.

20. Stokes, S. L. (1980). Estimation of variance using judgment ordered ranked set samples. Biometrics, 36, 35–42. 21. Stokes, S. L. (1980). Inferences on the correlation coefficient in bivariate normal populations from ranked set samples. J. Amer. Statist. Ass., 75, 989–995. 22. Stokes, S. L. (1995). Parametric ranked set sampling, Ann. Inst. Statist. Math., 47, 465–482. 23. Takahasi, K. (1969). Ann. Inst. Statist. Math., 21, 249–255. 24. Takahasi, K. (1970). Ann. Inst. Statist. Math., 22, 421–428. 25. Takahasi, K. and Wakimoto, K. (1968). Ann. Inst. Statist. Math., 20, 1–31. 26. Yanagawa, T. and Chen, S. (1980). J. Statist. Plann. Inf., 4, 33–44. 27. Yanagawa, T. and Shirahata, S. (1976). Aust. J. Statist., 18, 45–52. See also CONCOMITANT VARIABLES and ORDER STATISTICS.

S. LYNNE STOKES

RANK ESTIMATORS, SEQUENTIAL. See SEQUENTIAL RANK ESTIMATORS

RANKING AND SELECTION PROCEDURES A statistical ranking or a statistical selection procedure typically comes into play when the experimenter (the decision-maker) is faced with the problem of comparing a certain number k of populations in order to make a decision about preferences among them.

6907

Although ranking procedures and selection procedures were handled separately in the first edition of this Encyclopedia, they can be treated together by defining a generalized ranking goal (to be defined later) that will include the usual selection goal of selecting a fixed number of populations. In a given situation in which the distinction between describing the goal as one of ranking or of selection is rather academic, we will follow the traditional use in the literature. Ranking and selection theory is a part of multiple decision theory, a field greatly influenced by Abraham Wald. ORIGINS OF THE PROBLEM Consider k populations, each being characterized by the value of a parameter θ . In an agricultural experiment, the different populations may represent different varieties of wheat and the parameter θ may be the average yield of a variety. In an experiment concerning the effectiveness of several competing drugs for a specific ailment, the parameter θ may represent the probability of a success for each trial. The classical approach in the preceding situations has been to test the so-called homogeneity hypothesis H0 that θ1 = · · · = θk , where the θi are the unknown values of the parameter for the k populations. In the case of the familiar one-way classification model, the populations are assumed to be normal with unknown means θ1 , . . . , θk , and a common unknown variance σ 2 , which is a nuisance parameter. The homogeneity hypothesis H0 is tested using Fisher’s analysis of variance (ANOVA) technique. However, this usually does not serve the real purpose of the experimenter, which is not simply to accept or reject the homogeneity hypothesis. The real goal is often to choose the best population (the variety with the largest average yield, the drug having the largest singletrial probability of a success, and so on). The inadequacy of the ANOVA lies in the types of decisions that are made on the basis of the data and not in the design aspects of the procedure. The method of estimating the sizes of the differences between treatments was often employed as an indirect way of deciding which treatments are best. The

6908

RANKING AND SELECTION PROCEDURES

attempts to formulate the decision problem in order to achieve this realistic goal of selecting the best treatment set the stage for the development of ranking and selection theory. The first step in the direction of a somewhat more realistic approach came in the form of k-sample slippage tests where the alternative hypothesis states that one of the populations has ‘‘slipped’’ to the right or to the left. Some of the early contributors to the theory of slippage tests are Karlin and Truax [38], Mosteller [39], Paulson [42], and Truax [48]. A survey of early developments is given by Doornbos [13]. A related development is that of multiple comparison tests and simultaneous confidence intervals. Some important References are 6, 7, 33, 35, and 49. The formulation of a k-sample problem as a multiple decision problem enables one to answer the natural questions regarding the best populations. Early investigators of multiple decision procedures are Bahadur [1], Bahadur and Robbins [2], and Paulson [41]. The formulation of multiple decision procedures in the framework of what has now come to be known as ranking and selection procedures started with the now-classic 1954 paper by Bechhofer [3]. BASIC FORMULATIONS OF THE RANKING AND SELECTION PROBLEM In the basic situation, we have k populations, 1 , . . . , k , each indexed by a parameter θ , where the cumulative distribution function (cdf) of i is G(x; θi ) for i = 1, 2, . . . , k. We assume that the family {G(x; θ )} is stochastically increasing in θ , that is, G(x; θ1 )  G(x; θ2 ) for θ1
θ2 for all x, and that the parameters can be ordered from the smallest to the largest. Denote the true ordered θ values by θ[1]
θ[2]
· · ·
θ[k] . To fix ideas, we assume that larger the value of θ , more preferable is the population. Hence, the population associated with θ[k] is called the best population. We assume that there is no prior information as to the correspondence between the ordered and the unordered θi . Ranking and selection problems have generally been formulated adopting one of two main approaches now familiarly known as the indifference-zone formulation and the subset selection formulation.

THE INDIFFERENCE-ZONE FORMULATION In the indifference-zone (IZ) formulation due to Bechhofer [3], the goal is to select a fixed number of populations. Consider the goal of selecting the one best population; this might be called the basic goal. On the basis of samples of size n taken from each population, we seek a procedure to select one of the populations as the best. The natural procedure would be to compute estimates θˆ1 , θˆ2 , . . . , θˆk from each sample and claim that the population that yielded the largest θˆi is the best population. Obviously, one is concerned with the probability that this claim is correct, called the probability of a correct selection (PCS). We require a guaranteed minimum PCS, denoted by P∗ , whenever the best population is sufficiently better than all others, or equivalently, whenever θ[k] is sufficiently larger than θ[k−1] . Let δ = δ(θ[k] , θ[k−1] ) denote a suitably defined measure of the distance between the populations associated with θ[k] and θ[k−1] . Let  = {θ = (θ1 , θ2 , . . . , θk )}. Define (δ ∗ ) = {θ|δ(θ[k] , θ[k−1] )  δ ∗ > 0}. For specified δ ∗ and P∗ (1/k < P∗ < 1), it is required that PCS  P∗ whenever θ ∈ (δ ∗ ).

(1)

It is not meaningful to chose P∗
1/k, as in this case the requirement (1) can be met by randomly choosing one of the populations as the best. The region (δ ∗ ) of the parameter space  is called the preference-zone (PZ) as this is where we have strong preference for a correct selection. The complement of the PZ is known as the indifference zone (IZ), a region in which we do not require a guaranteed PCS. The PCS in the PZ, in general, depends on the configuration of θ. In many cases, there is a least favorable configuration (LFC) of θ for which the PCS attains a minimum over the PZ for any sample size. If we can make the PCS at the LFC equal to P∗ , then the probability requirement (1) will be satisfied. The usual choices for δ = δ(θ[k] , θ[k−1] ) are δ = θ[k] − θ[k−1] in the case of a location parameter and δ = θ[k] /θ[k−1] in the case of a scale parameter. In the case of nonnegative θ , which is not a scale parameter, one

RANKING AND SELECTION PROCEDURES

may choose either of these two special forms depending on other aspects of the problem. Bechhofer [3] introduced the IZ formulation by considering k normal populations with means θ1 , . . . , θk , and a common known variance σ 2 . Here, δ = θ[k] − θ[k−1] . On the basis of samples of size n from these normal populations, he investigated the natural selection procedure, say R1 , which selects the population that yielded the largest sample mean. The LFC for R1 is θ[1] = · · · = θ[k−1] = θ[k] − δ ∗ . For a specified (δ ∗ , P∗ ), the minimum sample size needed to meet the probability requirement (1) is given by   n = 2(σ H/δ ∗ )2 ,

(2)

where b stands for the smallest integer greater than or equal to b, H satisfies ∗

Pr{Z1
H, . . . , Zk−1
H} = P ,

(3)

and the Zi are standard normal variates with equal correlation ρ = 0.5. Values of H can be obtained for several selected values of k and P∗ from the tables of Bechhofer [3], Gibbons et al. [17], Gupta [20], and Gupta et al. [28]. Hall [32] has shown that Bechhofer’s Rule R1 , with the sample size n given by Equation 2, is the most economical in the sense of requiring fewest observations per population among all single-stage location invariant procedures satisfying the minimum PCS guarantee. Eaton [15] has shown some optimum properties of the general ranking problem of Bechhofer [3]. Some generalized goals that have been considered are as follows: 1. Selecting the t best populations for t  2, (i) in an ordered manner or (ii) in an unordered manner (Bechhofer [3]). 2. Selecting a fixed subset of size m that will contain at least s of the t best populations (Desu and Sobel [12]). Goal 1 with t = 1, and goal 2 with m = s = t = 1, become the basic goal of selecting the best. Goal 1(i) with t = k reduces to the goal of complete ranking of the populations. Goal 1 itself is a special case of the general ranking goal of Bechhofer [3], which

6909

is to partition the set of k populations into s nonempty subsets I1 , I2 , . . . , Is consisting of k1 , k2 , . . . , ks (k1 + k2 + · · · + ks = k) populations, respectively, such that for i ∈ Iα , j ∈ Iβ , 1
α < β
s, we have θi < θj . Any partition {I1 , I2 , . . . , Is } corresponds to a ranking decision and a partition consistent with the goal is a correct ranking. It should be noted that any ranking decision can be regarded as k classification statements of the form ‘‘i belongs to the group Iα .’’ Let δi,j denote a measure of the distance between the populations associated with θi and θj for i > j. Let kˆ i =

i 

kα ,

(kˆ 0 ≡ 0),

α=1

δ∗ = (δk∗ˆ , δk∗ˆ , . . . , δk∗ˆ ), 1

2

S

δk∗ˆ > 0, i

i = 1, 2, . . . , s − 1, (δ∗ ) = {θ|δkˆ +1,kˆ  δk∗ˆ }, i

i

i

i = 1, 2, . . . , s − 1. It is required that for any ranking procedure  PCS  P∗ whenever θ ∈ (δ). For a meaningful problem, we need k1 !k2 ! · · · ks !/k! < P∗ < 1. In the above general ranking problem, Fabian [16] introduced the idea of –correct ranking; for a formal definition of this (which is omitted here), one can also refer to Gupta and Panchapakesan [29, p. 40]. Roughly speaking, a ranking decision is –correct if wrongly classified populations are not too much apart. The special case of s = 2 and k1 = k − 1 for a location parameter family is of interest. In this case, a –correct ranking is equivalent to selecting one population i for which θi > θ[k] − ,  > 0; such a population is called a good population. In the case of normal means problem (assuming a common known variance), Bechhofer’s rule R1 of selecting the population that yielded the largest sample mean, based on samples of same size from all populations, guarantees a probability P∗ that a good population is selected (i.e., a –correct selection occurs) no matter what the unknown population means

6910

RANKING AND SELECTION PROCEDURES

are. In other words, with the same guaranteed probability, we can make a stronger claim than that made by Bechhofer [3] with the PZ () = {θ|θ[k] − θ[k−1]  }. Note that, in the PZ, the best population is the only good population. For further discussion of this, see reference 16. Different definitions of –correct selection for the IZ formulation and their mutual implications are discussed by Guiard [18]. In the normal means selection problem of Bechhofer [3] mentioned previously, if the common variance σ 2 is unknown, a singlesample procedure does not exist. It can be seen from Equation 2 that the minimum sample size n needed in order to satisfy the probability requirement (1) cannot be determined without the knowledge of the variance. In this case, a two-stage selection procedure is necessary to control the PCS. The first two-stage procedure for this problem was proposed and studied by Bechhofer, Dunnett, and Sobel [4]. This procedure is to take a first stage sample of an arbitrary size n0 from each population and use the data to calculate the pooled sample variance s2 based on v = k(n0 − 1) degrees of freedom. For specified (δ ∗ , P∗ ), with the same interpretation as before, we then calculate 2

N = max(n0 , 2s2 h2k−1,v /δ ∗ ),

(4)

where b denotes the smallest integer greater than or equal to b and hk−1,v is the one-sided upper (1 − P∗ ) equicoordinate point of the equicorrelated (k − 1)–variate central t distribution with equal correlation ρ = 0.5 and the associated degrees of freedom v. If N > n0 , a second stage sample of N − n0 additional observations must be taken. If N = n0 , no additional observations are required. We select the population that yielded the largest sample mean based on all N observations. THE SUBSET SELECTION FORMULATION Consider again the basic goal of selecting the best (i.e., the population associated with θ[k] ). We now seek a rule that will select a nonempty subset of random size that includes the best population. Here no assertion is made about which population in the selected

subset is the best. The size S of the selected subset ranges between 1 and k and is determined by the sample data. In contrast with the IZ formulation, there is no specification of a PZ (or an IZ). The experimenter specifies P∗ , the minimum PCS to be guaranteed no matter what the unknown values of the θi are. The selection rule is based on the estimates θˆ1 , θˆ2 , . . . , θˆk . In the case of normal means problem, assuming a common known variance σ 2 , Gupta [19,21] proposed a procedure based on a sample of size n from each population. This rule, say R2 , selects population i if the sample mean X i from it satisfies √ X i  X [k] − dσ/ n,

(5)

where d is a positive constant to be chosen so that the minimum PCS is guaranteed. The LFC in this case is given by θ1 = θ2 = · · · = θk . By equating the PCS at the LFC to P∗ , we get d=

√ 2H,

where H is given by Equation 3. When σ 2 is unknown, Gupta [19] proposed the rule R3 , which is R2 with σ 2 replaced by the pooled sample variance s2 , based on v = k(n − 1) degrees of freedom and a different constant d = hk−1,v as defined in Equation 4. Seal [46] proposed a class of procedures that included Gupta’s maximum-type procedure and an alternative (average-type) procedure that Seal advocated using. The superiority of Gupta’s procedure under certain slippage configurations and with regard to certain optimality properties (see refs. 11, 22, and 25), and its comparative ease in handling theoretical details accelerated the growth of the literature concerning subset selection. Subset selection can be thought of as a screening procedure towards selecting one population as the best (see ref. 47). The IZ approach does not have any requirements regarding correct selection when the true parametric configuration lies in the IZ. On the other hand, the (random-sized) subset selection formulation does not control the size of the selected subset. Attempts have been made to combine the features of the two approaches by Chen and Sobel [9] by defining correct selection differently for the

RANKING AND SELECTION PROCEDURES

IZ and the PZ, and controlling the PCS in each region. Another modified formulation, called the restricted subset selection, puts an upper bound for the size of the selected subset (see refs. 31, 36, and 44). This formulation involves specification of a PZ. IDENTIFICATION PROBLEMS Ranking and selection problems are closely related to simpler problems called identification problems. Suppose we have k populations 1 , . . . , k , each indexed by a parameter θ , where the cdf of i is G(x; θi ) for i = 1, . . . , k. Unlike in the setup of a ranking and selection problem, we now assume that all the θi are known up to a permutation. In other words, the ranked values of the θi 0 0 0 are θ[1]
θ[2]
· · ·
θ[k] and these values are assumed to be known a priori. It is further assumed that the true pairing of the i with 0 (i, j = 1, . . . , k) is unknown. Suppose the θ[j] 0 0 that θ[k−1] < θ[k] . Then, a basic identification goal would be to identify the population i 0 associated with θ[k] . We require a decision procedure that guarantees that Pr{Correct Decision}  P∗ for a specified P∗ such that 1/k < P∗ < 1 0 with whatever the true pairing of the θ[j] the i . Of course, there are several other possible goals, which may involve additional requirements. Although the study of identification problems is of considerable theoretical interest per se, one of the main practical reasons for considering these problems is that these studies provide us insight into the solution of corresponding ranking problems. Identification problems provide the theoretical structure upon which the solutions to the corresponding ranking problems are built. For detailed discussions on identification problems and their relation to ranking problems, the best reference is the book by Bechhofer, Kiefer, and Sobel [5]. A BRIEF SUMMARY OF DEVELOPMENTS Over the last 50 years, several aspects of selection and ranking have been investigated.

6911

Substantial research has been accomplished concerning procedures for specific univariate and multivariate parametric families, conditional procedures, nonparametric procedures, sequential and multistage procedures, procedures for restricted families such as the increasing failure rate (IFR) and increasing failure rate on the average (IFRA) distributions, decision-theoretic developments, and Bayes and empirical Bayes procedures. Some related goals have also been studied from the early years. These are selecting the best population in comparison with a standard or a control, and partitioning a set of populations with respect to two controls. Discussions of multiple comparisons with the best population can be found in references 5, 32, and 34. A related inference problem is that of estimating (point as well as interval) the ordered parameters θ[1] , . . . , θ[k] . Some attempts have been made to combine selecting the population associated with θ[k] and estimating θ[k] with simultaneous probability control (see, e.g., ref. 43). Another related inference problem is the estimation of the PCS for a selection procedure (see, e.g., ref. 24). Another interesting related inference problem is to estimate the parameter of the selected population in the case of a procedure for selecting one population, or to estimate a known function of the parameters of the selected populations in the case of subset selection. Here the object of inference is chosen on the basis of the same sample data that are to be used in the procedure. Such a statistical procedure has been called a selective inference procedure. This is different from a nonselective inference procedure in which the object of inference may be fixed or random but its identity is taken to be fixed and to have been determined before the data were obtained. Early contributions regarding estimation after selection were due to Dahiya [10], Hsieh [34], and Sarkadi [45]. Jeyaratnam and Panchapakesan [37] initiated estimation after selection for subset selection procedures. More references to estimation after selection can be had from references 26, 27, and 30. There are several books on ranking and selection problems dealing with various aspects of the problem. The first book to

6912

RANKING AND SELECTION PROCEDURES

appear was by Bechhofer, Kiefer, and Sobel [5], and deals with sequential identification and ranking procedures with special reference to Koopman-Darmois populations. They use a generalization of the sequential probability ratio test to derive a class of selection ¨ procedures. Two other books by Buringer, Martin and Schriever [8] and by Mukhopadhyay and Solanky [40] also concentrate on sequential procedures. The former book provides a comprehensive treatment of a large number of Bernoulli sequential selection procedures and also discusses subset selection procedures based on linear rank-order statistics for continuous responses. The latter book deals with an asymptotic analysis of the average sample size performance of open multistage procedures. Gupta and Panchapakesan [29] provide a comprehensive survey of the literature as of 1979 with a bibliography of some 600 main references. Gupta and Huang [23] deal with decision-theoretic developments. All the above books aim at researchers who are interested in and equipped for theoretical studies. Gibbons, Olkin, and Sobel [17] give an elementary treatment of the subject, dealing with many basic aspects with emphasis on methods aiming at readers who have only a knowledge of elementary statistics. They provide a large number of useful tables needed for implementing the procedures discussed in the text. They show how these tables may be applied to many examples and problems from a wide variety of specific fields of application. They have given a separate list of references to applications. The section on notes, references, and remarks at the end of each chapter provide useful information regarding general developments in the field. Bechhofer, Santner, and Goldsman [6] also provide a methods oriented treatment but assume that the reader is familiar with standard experimental design. The Notes sections at the end of the chapters describe developments beyond the discussions in the text. In addition to providing tables to implement the procedures, they provide a number of FORTRAN programs to supplement their tables. The categorized guide by Dudewicz and Koo [14] lists published books, journal articles, technical reports, and theses on ranking and selection procedures and related

topics under several categories. The book also includes reproduced reviews of several cited works that appeared in Mathematical ¨ Mathematik. Reviews and Zentralblatt fur There are a few special journal issues devoted to ranking and selection, namely, Communications in Statistics–Theory and Methods (Vol A6, No. 11, 1977), American Journal of Mathematical and Management Sciences (Vol. 5, Nos. 3 & 4, 1985 and Vol 6, Nos. 1 & 2, 1986), Journal of Statistical Planning and Inference (Vol. 54, Nos. 2 & 3, 1996), and Statistics & Decisions (Supplement Issue No. 4, 1999). In addition to these, there are also proceedings of conferences and edited volumes devoted entirely or substantially to the topic of ranking and selection. These and a few survey articles highlighting developments in the field of ranking and selection are listed under Further Reading List, which is not meant to be exhaustive.

REFERENCES 1. Bahadur, R. R. (1950). On the problem in the theory of k populations. Ann. Math. Stat., 21, 362–365. 2. Bahadur, R. R. and Robbins, H. (1950). The problem of the greater mean. Ann. Math. Stat., 21, 469–487. Correction: 22, 1951, 310. 3. Bechhofer, R. E. (1954). A single-sample multiple decision procedure for ranking means of normal populations with known variances. Ann. Math. Stat., 25, 16–39. 4. Bechhofer, R. E., Dunnett, C. W., and Sobel, M. (1954). A two-sample multiple decision procedure for ranking means of normal populations with a common unknown variance. Biometrika, 41, 170–176. 5. Bechhofer, R. E., Kiefer, J., and Sobel, M. (1968). Sequential Identification and Ranking Procedures (with Special Reference to Koopman-Darmois Populations). The University of Chicago Press, Chicago. 6. Bechhofer, R. E., Santner, T. J., and Goldsman, D. M. (1995). Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons. Wiley, New York. 7. Braun, H., ed. (1994). The Collected Works of John W. Tukey: Multiple Comparisons, Vol. VIII. Chapman & Hall/CRC Press, Boca Raton, Fla.

RANKING AND SELECTION PROCEDURES ¨ 8. Buringer, H., Martin, H., and Schriever, K.H. (1980). Nonparametric Sequential Selection ¨ Procedures. Birkhauser, Boston, Mass.

24.

9. Chen, P. and Sobel, M. (1987). An integrated formulation for selecting the t best of k normal populations. Commun. Stat.–Theory Methods, 16, 121–146.

25.

10. Dahiya, R. C. (1974). Estimation of the mean of selected population. J. Am. Stat. Assoc., 69, 226–230. 11. Deely, J. J. and Gupta, S. S. (1968). On the properties of subset selection procedures. Sankhya¯ Ser. A, 30, 37–50.

26.

12. Desu, M. M. and Sobel, M. (1968). A fixed subset-size approach to a selection problem. Biometrika, 55, 401–410. Corrections and amendments: 63, 685, 1976.

27.

13. Doornbos, R. (1966). Slippage Tests. Mathematical Research Center, Amsterdam.

28.

14. Dudewicz, E. J. and Koo, J. O. (1982). The Complete Categorized Guide to Statistical Selection and Ranking Procedures. Series in Mathematical and Management Sciences, Vol. 6, American Sciences Press, Columbus, Ohio.

29.

15. Eaton, M. L. (1967). Some optimum properties of ranking procedures. Ann. Math. Stat., 38, 124–137. 16. Fabian, V. (1962). On multiple decision methods for ranking population means. Ann. Math. Stat., 33, 248–254.

30.

17. Gibbons, J. D., Olkin, I., and Sobel, M. (1977). Selecting and Ordering Populations: A New Statistical Methodology. Wiley, New York. (Reprinted by Society for Industrial and Applied Mathematics, Philadelphia, 1999) 18. Guiard, V. (1996). Different definitions of correct selection for the indifference zone formulation. J. Stat. Plann. Inference, 54, 175–199.

31.

19. Gupta, S. S. (1956). On a Decision Rule for a Problem in Ranking Means. Ph. D. Thesis (Mimeo. Ser. 150), University of North Carolina, Chapel Hill, N.C.

32.

20. Gupta, S. S. (1963). Probability integrals of the multivariate normal and multivariate t. Ann. Math. Stat., 34, 792–828.

33.

21. Gupta, S. S. (1965). On some multiple decision (selection and ranking) rules. Technometrics, 7, 225–245.

34.

22. Gupta, S. S. and Hsu, J. C. (1978). On the performance of some subset selection procedures. Commun. Stat.–Simul. Comput., B7, 561–591.

35.

23. Gupta, S. S. and Huang, D.-Y. (1981). Multiple Decision Theory: Recent Developments,

6913

Lecture Notes in Statistics, Vol. 6, Springer, New York. Gupta, S. S., Leu, L.-Y., and Liang, T. (1990). On lower confidence bounds for PCS in truncated location parameter models. Commun. Stat.–Theory Methods, 19, 527–546. Gupta, S. S. and Miescke, K. J. (1981). Optimality of subset selection procedures for ranking means of three normal population. Sankhya Ser. B, 43, 1–17. Gupta, S. S. and Miescke, K. J. (1990). On finding the largest mean and estimating the selected mean. Sankhya Ser. B, 52, 144–157. Gupta, S. S. and Miescke, K. J. (1993). On combining selection and estimation in the search for the largest binomial parameters. J. Stat. Plann. Inference, 36, 129–140. Gupta, S. S., Nagel, K., and Panchapakesan, S. (1973). On the order statistics from equally correlated normal random variables. Biometrika, 60, 403–413. Gupta, S. S. and Panchapakesan, S. (1979). Multiple Decision Procedures: Theory and Methodology of Selecting and Ranking Populations. Wiley, New York. (Reprinted by Society for Industrial and Applied Mathematics, Philadelphia, 2002) Gupta, S. S. and Panchapakesan, S. (1996). ‘‘Design of Experiments with Selection and Ranking Goals’’. In Design and Analysis of Experiments, S. Ghosh and C. R. Rao, eds. Elsevier, Amsterdam, pp. 555–585. (A survey paper dealing mostly with selection procedures and simultaneous confidence intervals under normal theory involving not only singlefactor experiments but also blocking and factorial designs.) Gupta, S. S. and Santner, T. J. (1973). ‘‘On Selection and Ranking Procedures—a Restricted Subset Selection Rule’’. In Proceedings of the 39th Session of the International Statistical Institute, Volume 45, Book I, Vienna, Austria, 1973 pp. 478–486. Hall, W. J. (1959). The most economical character of Bechhofer and Sobel decision rules. Ann. Math. Stat., 30, 964–969. Hochberg, Y. and Tamhane, A. C. (1987). Multiple Comparison Procedures. Wiley, New York. Hsieh, H.-K. (1981). On estimating the mean of the selected population with unknown variance. Commun. Stat.–Theory Methods, 10, 1869–1878. Hsu, J. C. (1996). Multiple Comparisons: Theory and Methods. Chapman & Hall/CRC Press, Boca Raton, Fla.

6914

RANKING AND SELECTION PROCEDURES

36. Hsu, L. and Panchapakesan, S. (2002). ‘‘Selecting the Normal Population with the Largest Mean: A Restricted Subset Selection Rule’’. In Statistical Methods and Practice: Recent Advances, N. Balakrishnan, N. Kannan, and M. R. Srinivasan, eds. Narosa Publishing House, New Delhi, India, pp. 145–161. 37. Jeyaratnam, S. and Panchapakesan, S. (1984). ‘‘An Estimation Problem Relating to Subset Selection for Normal Populations’’. In Design of Experiments: Ranking and Selection, T. J. Santner and A. C. Tamhane, eds. Marcel Dekker, New York, pp. 287– 302. 38. Karlin, S. and Truax, D. R. (1960). Slippage problems. Ann. Math. Stat., 31, 296–324. 39. Mosteller, F. (1948). A k-sample slippage test for an extreme population. Ann. Math. Stat., 19, 58–65. 40. Mukhopadhyay, N. and Solanky, T. K. S. (1994). Multistage Selection and Ranking Procedures: Second-Order Asymptotics. Marcel Dekker, New York. 41. Paulson, E. (1949). A multiple decision procedure for certain problems in analysis of variance. Ann. Math. Stat., 20, 95–98. 42. Paulson, E. (1952). An optimum solution to the k-sample slippage problem for the normal distribution. Ann. Math. Stat., 23, 610– 616. 43. Rizvi, M. H. and Lal Saxena, K. M. (1974). On interval estimation and simultaneous selection of ordered location or scale parameters. Ann. Stat., 2, 1340–1345. 44. Santner, T. J. (1975). A restricted subset selection approach to ranking and selection problems. Ann. Stat., 3, 334–349. 45. Sarkadi, K. (1967). Estimation after selection. Stud. Sci. Math. Hung., 2, 341–350. 46. Seal, K. C. (1955). On a class of decision procedures for ranking means of normal populations. Ann. Math. Stat., 26, 387–398. 47. Tamhane, A. C. and Bechhofer, R. E. (1977). A two-stage minimax procedure with screening for selecting the largest normal mean. Commun. Stat.–Theory Methods, A6, 1003–1033. 48. Truax, D. R. (1953). An optimum slippage test for the variances of normal distributions. Ann. Math. Stat., 24, 669–673. 49. Westfall, P. H., Tobias, R. L., Rom, D., Wolfinger, R. D., and Hochberg, Y. (1999). Multiple Comparisons and Multiple Tests using the SAS System. SAS Institute, Cary, N.C.

FURTHER READING Bechhofer, R. E. (1985). Selection and ranking procedures—some personal reminiscences, and thoughts about its past, present, and future. Am. J. Math. Manage. Sci., 5(3), 201–234. (This article, by a pioneer in the field, offers insight into his thinking and motivation. It recounts the historical origins of the field and discusses modern developments with an up-to-date bibliography.) Bofinger, E., Dudewicz, E. J., Lewis, G. J., and Mengersen, K., eds. (1992). The Frontiers of Modern Statistical Inference Procedures, II. Proceedings and Discussions of the IPASRAS-II Conference on Inference Procedures Associated with Statistical Ranking and Selection held in Sydney, Australia, June 1987, Series in Mathematical and Management Sciences, Vol. 28, American Sciences Press, Columbus, Ohio. Dudewicz, E. J., ed. (1985). The Frontiers of Modern Statistical Inference Procedures. Proceedings and Discussions of the IPASRAS Conference on Inference Procedures Associated with Statistical Ranking and Selection held in Honolulu, Hawaii, June 1982, Series in Mathematical and Management Sciences, Vol. 10, American Sciences Press, Columbus, Ohio. Gupta, S. S. and Panchapakesan, S. (1985). Subset selection procedures: review and assessment. Am. J. Math. Manage. Sci., 5(3), 235–311. (An extensive chronological account of subset selection theory and practice to date, with comprehensive references.) Herrend¨orfer, G. and Miescke, K. J., eds. (1993). Selection Procedures I. Proceedings of the 3rd Schwerin Conference in Mathematical Statistics held in Bad Doberan, Germany, September ¨ die Biologie 5–10, 1993, Forshunginstitut fur landwirtschaftlicher Nutztiere, Dummerstorf, Germany. Herrend¨orfer, G. and Miescke, K. J., eds. (1994). Selection Procedures II. Proceedings of the 3rd Schwerin Conference in Mathematical Statistics held in Bad Doberan, Germany, September ¨ die Biologie 5–10, 1993, Forshunginstitut fur landwirtschaftlicher Nutztiere, Dummerstorf, Germany. Hoppe, F. M., ed. (1993). Multiple Comparisons, Selection, and Applications in Biometry: A Festschrift in Honor of Charles W. Dunnett. Marcel Dekker, New York. (This volume contains 32 research articles of which ten are devoted to ranking and selection and another ten to multiple comparisons.) Panchapakesan, S. and Balakrishnan, N., eds. (1997). Advances in Statistical Decision Theory

RANKING PROCEDURES ¨ and Applications. Birkhauser, Boston, Mass. (A volume of 28 research articles in honor of Shanti S. Gupta, a pioneer in the field of ranking and selection. Besides ten articles on ranking and selection procedures, the volume includes a list of publications of Gupta.) Santner, T. J. and Tamhane, A. C., eds. (1984). Design of Experiments: Ranking and Selection. Marcel Dekker, New York. (A volume of 21 research articles in honor of Robert E. Bechhofer, a pioneer in the field of ranking and selection.) See also MULTIPLE DECISION PROCEDURES; RANKING PROCEDURES; and SELECTION PROCEDURES.

S. PANCHAPAKESAN

RANKING PROCEDURES A statistical ranking procedure uses sample data to order or rank members of a family of k populations with respect to the relative magnitudes of some parameter in such a way that we maintain control over the probability that the ranking asserted to be correct is indeed correct. Ranking procedures are one approach in multiple decision theory∗ (see Gupta and Huang [13]), a field greatly influenced by Abraham Wald∗ , where we use a simple (zero–one) loss function and our risk is an incorrect decision. A related approach in multiple decision theory is selection procedures (see SELECTION PROCEDURES). The two kinds of problems are frequently linked together under the heading of ranking and selection procedures∗ . Selection procedures differ from ranking procedures in that they seek to select (identify) certain ones of the k populations as having certain parameter characteristics and say nothing specific (except by elimination) about the remaining populations. For example, a selection procedure might have the goal of selecting only the one population with the largest parameter value, or selecting a subset of populations that contains the one with the largest parameter. In such cases we maintain control over the probability that the selection is correct. It is also possible to combine the approaches of ranking and selection

6915

procedures in the sense that we may want to select the best t populations for t < k and rank these t with respect to their parameter values. The complete ranking problem is analogous to the above, where t = k. Procedures that combine the subset selection approach pioneered by Gupta [12] with ranking are also available; these are covered extensively in Chapter 19 of Gupta and Panchapakesan [15], which is a comprehensive discussion of various aspects of selection and ranking. Other important references here are Huang and Panchapakesan [17] and Gupta and Miescke [14]. Dudewicz and Koo [8] give a comprehensive categorized bibliography of selection and ranking procedures. The problem of a complete ranking was first mentioned in Bechhofer [1] and tables for applying the procedure to rank k = 3 normal populations according to their variances were given in Bechhofer and Sobel [3]. The procedures were slow to develop for k > 3 because of the mathematical complexity of computations involved. To date, procedures have been developed for ranking normal means (with variances known or unknown), and for ranking normal variances (with means known or unknown), using the indifference zone approach. Researchers are currently working on solutions for other distributions, other parameters, and with other approaches. Related confidence procedures for estimating certain parameter values are also being developed. Confidence procedures directly analogous to the complete ranking problem are only those that simultaneously estimate all of the parameter values. THE COMPLETE RANKING PROBLEM Suppose that we have k independent populations π1 , π2 , . . . , πk , where the cumulative distribution function (CDF) of πi is G(x; θi ) for i = 1, 2, . . . , k, where G(x; θ ) is a stochastically increasing function of θ [i.e., G(x; θ  )  G(x; θ  ) for θ  < θ  for all x]. The problem is to use parameter estimates computed from sample data to order or rank the populations with respect to their true θ values. Denote the true ordered θ values by θ[1]
θ[2]
· · ·
θ[k] . The ranking procedure used should guarantee that the probability is at

6916

RANKING PROCEDURES

least some specified value P∗ that the ranking asserted from the estimates is correct whenever the true parameter configuration θ = (θ1 , θ2 , . . . , θk ) lies in a certain specified subset of the parameter space, called the preference zone (PZ). We require P∗ > 1/k! since the probability that a random ordering of the k populations (not based on sample data) is correct is equal to 1/k!. The remaining portion of the parameter space is called the indifference zone (IZ), and hence this formulation is called the indifference zone approach to ranking procedures. If the parameter of interest is a location parameter, the preference zone is where the differences of successive θ values are at least some specified value δ ∗ , that is, the set PZ = {θ : θ[i] − θ[i−1]  δ ∗ , i = 2, 3, . . . , k}. The preference zone for the scale parameter problem, where θ denotes a population variance, is where the ratios of successive θ values are at least some specified value δ ∗ , that is, the set PZ = {θ : θ[i] /θ[i−1]  δ ∗ , i = 2, 3, . . . , k}. The configuration of θ values in the preference zone for which the probability of a correct ranking (PCR) is a minimum is called the least favorable configuration∗ (LFC). If we can determine a common sample size n required from each population such that the PCR is equal to the specified value P∗ when the parameters are in the least favor∗ able configuration , we know that the PCR ∗ is at least P for any configuration in the preference zone for that n value and hence we have a conservative solution for n. The least favorable configuration for the location problem is LFC = {θ : θ[i] − θ[i−1] = δ ∗ , i = 2, 3, . . . , k}, and for the scale problem is LFC = {θ : θ[i] /θ[i−1] = δ ∗ , i = 2, 3, . . . , k}. To solve the complete ranking problem for any distribution and any parameter, the requirements are a procedure for using the

sample data to order the populations, and an expression (or set of tables) that relates the values of n, δ ∗ , and P∗ . Such a solution will enable a person designing an experiment to determine the minimum common sample size n required to assert that the ranking resulting from sample data is correct with probability at least P∗ for all parameter configurations in the preference zone specified by δ ∗ . Alternatively, the sample size n may be fixed, either because the data have already been obtained or because n is restricted by cost or other practical limitations. In this case, the same solution can be used to construct a graph of the pairs (δ ∗ , P∗ ) that are satisfied for the fixed value of n. ONE-STAGE PROCEDURES FOR THE NORMAL MEANS PROBLEM A special case of the location problem arises when the sample observations from πi are normally distributed with unknown mean µi and known variance σ 2 which is common for each of the k populations. The parameter is µi . For a one-stage procedure, a common sample size n is taken from each population and the ordering asserted is the same as that of the X i , the mean of the sample from πi , for i = 1, 2, . . . , k. The investigator specifies P∗ and δ ∗ and determines the required sample size as a function of τ from n = (τ σ/δ)2 .

(1)

This procedure was given in Bechhofer [1]; tables for τ when k = 3 can be computed from tables of the bivariate normal distribution with ρ = −0.5, but the computation when k  4 is much more difficult. Carroll and Gupta [7] give an approximate solution for τ based on numerical integration∗ , or Gibbons et al. [11, Table P.1] can be used. Alternatively, if n is outside the control of the investigator, the same τ value for any chosen P∗ can be used to compute the (δ ∗ , P∗ ) pairs that are satisfied by the selection procedure with √ δ ∗ = τ σ/ n.

(2)

Another special case arises for normal populations with known but unequal variances.

RANKING PROCEDURES

The same tables can be used if the sample sizes for the k populations are chosen so that σ2 σ2 σ12 = 2 = ··· = k . n1 n2 nk

(3)

The value n determined from (1) is used to set the total of all sample sizes as N = kn and then find the respective individual sample sizes from Nσ 2 ni =
i 2 . σi

(4)

TWO-STAGE PROCEDURES FOR THE NORMAL MEANS PROBLEM The more typical location problem in practice is normal distributions with unknown variances. Then a single-stage procedure will not satisfy the requirements of the problem. A two-stage procedure is required, where an initial sample is taken from each population in order to obtain some useful information about the variances before asserting a ranking. If the unknown variances are assumed common, the procedure is to take an initial sample of n0 observations from each population and compute a pooled sample variance as s2 =

s21 + s22 + · · · + s2k , k

(5)

where each sample variance is the usual unbiased estimator of the population variance. In the second stage, the investigator specifies δ ∗ and P∗ and takes an additional n − n0 observations from each of the k populations, where n is given as a function of h by   (6) n = max n0 , {2(sh/δ ∗ )2 }+ and {c}+ means the smallest integer equal to or greater than c. Then the mean of all n observations from both stages is calculated for each sample, and the populations are ranked according to these sample values. This procedure was given by Bechhofer et al. [2]. Dunnett and Sobel [9] give tables of h for k = 3. Freeman et al. [10] give additional

6917

tables, or Table P.2 of Gibbons et al. [11] can be used for p∗ = 0.95 and k = 3, 4, 5, 6, n0 = 10(10)100, 200, 500. If the unknown variances are not assumed equal, a similar but more complex two-stage procedure can be used. An initial sample of n0 observations is taken from each population and used to compute s21 , s22 , . . . , s2k . Then ni − n0 additional observations are taken from each population where   (7) ni = max n0 + 1, {(si h/δ ∗ )2 }+ . This solution and tables of h were given by Beirlant et al. [4]. The final step is to compute a weighted mean of all the observations in the ith sample, Xi1 , Xi2 , . . . , Xini , as Xi =

n0 

ai Xij +

ni 

bi Xij ,

(8)

j=n0 +1

j=1

where ai and bi are the solutions to the system n0 a2i + (ni − n0 )b2i = (δ ∗ /si h)2 , n0 ai + (ni − n0 )bi = 1,

(9)

for i = 1, 2, . . . , k. Note that the si in (9) are computed from only the initial n0 observations. The populations are then ranked according to the values of these weighted means. ONE-STAGE PROCEDURES FOR THE NORMAL VARIANCES PROBLEM A special case of the scale parameter problem arises when the sample observations from πi are normally distributed with mean µi (known or unknown) and unknown variance σi2 . The scale parameter is σi2 and the appropriate estimator is the sample variance s2i =

n  (Xij − X i )2 /(n − 1) j=1

if µi is unknown, and Vi =

n  (Xij − µi )2 /n j=1

6918

RANKING PROCEDURES

if µi is known, where Xi1 , Xi2 , . . . , Xin is the sample of n observations from πi . The populations are ranked according to the values of these estimators. Bechhofer and Sobel [3] considered this problem for µ1 , µ2 , . . . , µk known and k = 2 and 3, and gave some tables. Schafer [18] used Monte Carlo methods∗ to evaluate the probability of a correct complete ranking for this problem, and these tables are readily available in Schafer and Rutemiller [19], as well as in Gibbons et al. [11, Table P.3] for k = 3, 4, . . . , 8. Bishop and Dudewicz [6] and Bishop [5] give a general solution that can be used for any distribution.

An example where a complete ranking with respect to means is needed is provided by Holt [16]. Four helmet types, No Helmet, PASGT A, PASGT B, and Standard M-1, are to be ranked with respect to integrated sum mean values of induced muscle stress based on a sample of n = 144 observations on each type. Low muscle stress, as measured by electromyography, is an important characteristic of a good infantry helmet. The data, assumed to be normally distributed with a common variance, are given as follows: Mean (n = 144)

None PASGT A PASGT B Standard

REFERENCES 1. Bechhofer, R. E. (1954). Ann. Math. Statist., 25, 16–39. 2. Bechhofer, R. E., Dunnett, C. W., and Sobel, M. (1954). Biometrika, 41, 170–176. 3. Bechhofer, R. E. and Sobel, M. (1954). Ann. Math. Statist., 25, 273–289.

NUMERICAL EXAMPLE

Helmet

the n is determined in advance or by other considerations. Note that, in general, large sample sizes are required for any reasonable p∗ value even for moderate k and typical δ ∗ values. This is because the goal of a complete ranking is asking a great deal from the data.

9,949 10,745 10,539 10,785 s = 1, 196.91

Holt set p∗ = 0.90 and δ ∗ = 170 for these k = 4 populations and assumed a common σ = 1196.91 to compute n as n = 335 using τ = 2.599 and (1). He then concluded that since only n = 144 observations were taken and 335 are required, the order of the sample means does not reflect the order of the population means with any confidence. In fact, the order observed for n = 144 could be asserted to be the correct ranking with δ ∗ = 170 for a p∗ of only about 0.68. This example shows how sample size determination can be used effectively in a statistical analysis even when

4. Beirlant, J., Dudewicz, E. J., and van der Meulen, E. C. (1981). Complete statistical ranking, with tables and applications. Mededelingen uit het Wiskundig Instituut Katholieke Universiteit, Leuven, Belgium, No. 145, December. 5. Bishop, T. A. (1978). Technometrics, 20, 495–496. 6. Bishop, T. A. and Dudewicz, E. J. (1977). IEEE Trans. Reliab., R-26, 362–365. 7. Carroll, R. J. and Gupta, S. S. (1977). J. Statist. Comput. Simul., 4, 145–157. 8. Dudewicz, E. J. and Koo, J. O. (1982). The Complete Categorized Guide to Statistical Selection and Ranking Procedures. American Sciences Press, Columbus, Ohio. 9. Dunnett, C. W. and Sobel, M. (1954). Biometrika, 41, 153–169. 10. Freeman, H., Kuzmack, A., and Maurice, R. J. (1967). Biometrika, 54, 305–308. 11. Gibbons, J. D., Olkin, I., and Sobel, M. (1977). Selecting and Ordering Populations: A New Statistical Methodology. Wiley, New York. (Elementary and applied; extensive tables and references; Chapter 12 covers ranking procedures.) 12. Gupta, S. S. (1956). On a Decision Rule for a Problem in Ranking Means. Ph.D. dissertation, Inst. Statist. Mimeo Ser. No. 150, University of North Carolina, Chapel Hill. 13. Gupta, S. S. and Huang, D. Y. (1981). Multiple Statistical Decision Theory. SpringerVerlag, New York. 14. Gupta, S. S. and Miescke, K.-J. (1981). Sankhya¯ B, 43, 1–17. 15. Gupta, S. S. and Panchapakesan, S. (1979). Multiple Decision Procedures: Theory and

RANK LIKELIHOOD

16. 17. 18.

19.

Methodology of Selecting and Ranking Populations. Wiley, New York. (A comprehensive theoretical treatment of ranking and selection; extensive references.) Holt, W. R. (1980). Personal communication. Huang, D. Y. and Panchapakesan, S. (1978). J. Chinese Statist. Ass., 16, 5801–5810. Schafer, R. E. (1974). In Reliability and Biometry: Statistical Analysis of Lifelength, F. Proschan and R. J. Serfling, eds., SIAM, Philadelphia, pp. 597–617. Schafer, R. E. and Rutemiller, H. C. (1975). Technometrics, 17, 327–331.

See also LEAST FAVORABLE CONFIGURATION; MULTIPLE COMPARISONS; and SELECTION PROCEDURES.

JEAN DICKINSON GIBBONS

RANKIT

6919

The rank likelihood for the Yj ’s is then given by Pr(Yj has rank rj , j = 1, . . . , n) = Pr(Wj has rank rj , j = 1, . . . , n), and the latter probability can be found for observed ranks r1 , . . . , rn and given values of θ . Now Pr(Wj has rank rj , j = 1, . . . , n)  n {f(j) (uj ; θ )duj }, =

(1)

A j=1

where A = {(u1 , . . . , un ) : u1 < · · · < un } and (j) is the index of the Y1 , . . . , Yn , which is the jth smallest in the ordered sample of Y’s (i.e., r(j) = j).

The expected value of a unit normal order statistic. See NORMAL SCORES TESTS; PROBIT ANALYSIS.

AN IMPORTANT EXACT RESULT

RANK LIKELIHOOD

In one particularly important case, (1) can be found analytically. Savage [13] shows that if Lehmann alternatives∗ are considered for the CDFs of the Wj ’s, that is,

Rank likelihoods or, more fully, marginal likelihoods of ranks, have been used in models for the statistical analysis of various problems, ranging from the analysis of paired comparisons∗ to randomly censored∗ survival data. Ranks, rather than the original values of observations, if available, prove useful in statistical analysis if there are no strong prior beliefs about the distributional properties of the observations or the distributional properties of the observations are difficult to assess, as can be the case with censored data. Let W1 , . . . , Wn be independent random variables, with probability density functions (PDFs) fj (w; θ )(j = 1, . . . , n) known except for the vector parameter θ. Suppose that G(·) is an unknown monotone-increasing differentiable function and we can observe Yj ’s (j = 1, . . . , n) with G(Yj ) = Wj (j = 1, . . . , n). In such a case the ranks of the Yj ’s are marginally sufficient for θ (see Kalbfleisch [5]), or the ranks provide the basis for invariant tests for θ (see Cox and Hinkley [4, Sec. 5.3]). The model implies that for observations y1 , . . . , yn we are willing to assume a parametric model after some arbitrary transformation.

Fj (w; θ ) = wδj

(δj > 0, 0
w
1),

with θ = (δ1 , . . . , δn ), then Pr(W1 < · · · < Wn )    =

n

j=1

n

δj  



j=1

j 

−1 δi 

.

(2)

i=1

Putting n = 2, in (2) essentially gives the Bradley and Terry [1] model for paired comparisons∗ , and n = 3 gives a model for triple comparisons. The value of (1) can also be found if the PDFs of the Wj ’s are given by fj (w; θ ) = exp{(w) − θj − exp(w − θj )}, where exp[w − exp(w)] is an extreme value∗ density and θ = (θ1 , . . . , θn ). We then have Pr(W1 < · · · < Wn )   n   θ j = exp −  j=1

n

j=1

 

n  i=j

−1  exp(−θi ) .  (3)

6920

RANK LIKELIHOOD

If δj = exp(−θj ), then, with these Wj ’s, Pr(Wn < · · · < W1 ) is given by the righthand side of (2). If the θj ’s are related by a linear model, θj = xTj · β, where xj is a vector of known explanatory variables, then (3) gives Cox’s [3] proportional hazards∗ regression model, for survival times with no censoring, except that β has been replaced by −β. Kalbfleisch and Prentice [6] define the rank vector r for a sample that contains randomly right censored observed survival times and obtain the rank likelihood   k  T x · β f (r; β) = exp − (j)

j=1

×

  

k

j=1

 

 i∈R(t(j) )

exp(−xTi

−1  · β) ,  (4)

where for the k(
n) distinct uncensored survival times t(1) < · · · < t(k) , R(t(j) ) is the set of all indices i so that ti  t(j) , j = 1, . . . , k. Here x(j) refers to the explanatory variable of the uncensored observation with survival time ranked j among t(1) < · · · < t(k) . The rank likelihood (4) is identical to the partial likelihood∗ of Cox [3], except for the changed sign of β. This identity occurs provided that there are no ties in the data and there are no timedependent covariates in Cox’s [3] regression model. The hazard function for Yj , where Yj is observed and G(Yj ) = Wj , is given by   −xT ·β λ(y; x) = G (y) exp G(y) e j , and since G(y) is unspecified, so also is G (y) exp[G(y)], giving the proportional hazards condition. LINEAR REGRESSION∗ MODELS AND TEST STATISTICS The choice fj (w; θ) = exp[w − θj − exp(w − θj )] of fj (w; θ ) belongs to the location shift family of choices of fj (w; θ) specified by fj (w; θ ) = fe (w − θj ), where fe (w) is a known density, such as standard normal, double exponential, logistic, or an extreme value density as given above. When θj = xTj · β the model for the Wj ’s is the linear model Wj = xTj · β + ej , where the e1 , . . . , en are independent with

the completely known PDF fe (w). Therefore, the model for the responses Y1 , . . . , Yn , is that after some unknown monotone transformation, the g(Yj ) = Wj follow the linear model Wj = xTj · β + ej . In general, the rank likelihood (1) cannot be calculated for arbitrary choices of fe (w), an exception being (4), of course. If for this case (1) is denoted by f (r; β) and l(r; β) = log f (r; β), then tests of β = 0 are based on the score statistic∗ derived from l(r; β), and an estimate of the variance is given by the observed information at β = 0; note that Cox and Hinkley [4, Sec. 6.3] discuss a similar approach but consider variances under randomization∗ distributions. The resulting statistic has an approximate χ 2 distribution, with degrees of freedom equal to the dimension of β, if the hypothesis β = 0 is true (see Prentice [12]). The vector-valued score statistic l (r; 0) = T X a, where XT = (x1 , . . . , xn ) and (a)j = E{g(Wrj :n )}, where W1:n < · · · < Wn:n are the order statistics of a sample of size n from the population with density fe (w) and g(w) = −fe (w)/fe (w). For the two-sample problem, where xj is a scalar taking the value 0 or 1 according to which sample the observation belongs, the statistic l (r; 0) is a two-sample rank statistic, being (a) the Wilcoxon∗ statistic if fe (w) is the logistic density, (b) the normal scores∗ statistic if fe (w) is the normal density, or (c) the log-rank∗ or Savage’s rank statistic if fe (w) = exp[w − exp(w)], an extreme value density. Applying these ideas to case (a), we find the score statistic to be  n   ri 2 −1 , N+1 i=1

where r1 , . . . , rn are the ranks of one of the two samples of combined size N. We note this statistic is just 2(N + 1)−1 W − n, where W is the Mann–Whitney–Wilcoxon∗ statistic. The estimate of variance is the reciprocal of the observed information at β = 0, that is, the reciprocal of n  2 ri (N + 1 − ri ) N2 − 1 i=1

−

 8 ri (N + 1 − rj ). 2 (N + 1) (N − 1) r 0, and in practice t = 4 to 6 iterations might ensure convergence of Ni.(t) and

N.j(t) to the marginals Ni. and N.j , respec
(t) (t)
(t) tively, where Ni.(t) = b Nib , N.j = a Nia , and Nij(t) = Wij(t) nij is the raking ratio estimator of Nij . The corresponding raking ratio estimators of Yij and Y are given by Yij(t) = Wij(t) yij ,

Y (t) =



Wij(t) Yij ,

where yij denotes the sample total in the (i, j)th cell. For t = 1, Y (t) reduces to the
(1) = (N /n post-stratified estimator Y i. i. )yi. ,
where yi. = j yij . Rao [24] and Konijn [11] have given the large-sample variance formula for Yij(t) , t = 1, 2, including conditional variance formulae

for given ni. = b nij or given n.j = a naj . Konijn also derived
the corresponding
formu(t) lae for Yi.(t) = b Yit(t) and Y.j(t) = a Yaj ,t = 1, 2. Brackstone and Rao [2] have obtained the following general recurrence formulae for the approximate variance of Y (t) :    . Wij(t−1) (yij − R.j nij ) , V(Y (t) ) = V t even, where R.j = Y.j /N.j . Thus V(Y (t) ) for even t can be obtained from V(Y (t−1) ) by substituting yij − R.j nij for yij in the latter. For t odd, a similar formula is given by    . Wij(t−1) (yij − Ri. nij ) , V(Y (t) ) = V t odd, where Ri. = Yi. /Ni. . Therefore, given the starting expression V(Y (0) ), the formula for V(Y (t) ) can be obtained iteratively for any (0) = (N/n)y in the case of SRS t, where
Y
and y = yij is the sample total. Brackstone and Rao [2] extended these results to single-stage cluster sampling∗ . A bibliography on raking ratio estimators to 1978 is provided by Oh and Scheuren [18]. They have also described a multivariate extension of raking ratio estimation. Multivariate Ratio Estimators In some surveys two or more concomitant variables with known population totals might

RATIO ESTIMATORS—I

be available. Then a multivariate ratio estimator of Y, given by Yˆ Mr =

p 

ˆ Xˆ j )Xj = wj (Y/

j=1

p 

wj Yˆ rj ,

i=1

can be used [19]. Here Xˆ j is the unbiased estimator of the population total, Xj , of the jth concomitant variable (j = 1, . . . , p  2) and the weights wj are determined to minimize the approximate variance of Yˆ mr subject to
wj = 1. The optimal vector of weights w∗ is obtained as w∗ =  −1 1/(1’ −1 1), where 1 is the p-vector of 1’s and  is the approximate covariance matrix of the individual ratio estimators Yˆ ri with (i, j)th element σij = cov(Yˆ − Ri Xˆ i , Yˆ − Rj Xˆ j ) and Ri = Y/Xi . In practice, the optimal weights w∗ are obtained from the estimated ˆ As in the case of the covariance matrix, . regression estimator, Yˆ mr is computationally cumbersome for general sampling designs due to difficulty in estimating . The estimator Yˆ mr is more efficient, in large samples, than Yˆ ri for any i or the corresponding multivariate ratio estimator based on q (< p) concomitant variables∗ . Olkin [19] generalized the Hartley—Ross unbiased ratio estimator for simple random sampling as follows: Yˆ HR (m) =

p 

wj Yˆ HR,j ,

j=1

where Yˆ HR,j is the Hartley—Ross unbiased estimator based on the individual ratios rtj = yt /xtj (t = 1, . . . , n) corresponding to jth concomitant variable. Similarly, the other unbiased ratio estimators or approximately unbiased ratio estimators can be generalized to the multivariate case. REFERENCES 1. Beale, E. M. L. (1962). Ind. Org., 31, 51–52. 2. Brackstone, G. J. and Rao, J. N. K. (1979). Sankhya¯ C, 41, 97–114.

6977

3. Chakrabarty, R. P. (1979). J. Indian Soc. Agric. Statist., 31, 49–62. 4. Cochran, W. G. (1977). Sampling Techniques, 3rd ed. Wiley, New York, Chap. 6. 5. Cochran, W. G. (1978). In Contributions to Survey Sampling and Applied Statistics, H. A. David, ed. Academic Press, New York, pp. 3–10. ´ 6. Hajek, J. (1949). Statist. Obzor., 29, 384–394 (in Czech). 7. Hartley, H. O. and Ross, A. (1954). Nature (Lond.), 174, 270–271. 8. Kish, L. (1962). Amer. Statist. Ass., Proc. Soc. Statist. Sec., pp. 190–199. 9. Kish, L. and Frankel, M. R. J. R. Statist. Soc. B, 36, 1–37.

(1974).

10. Kish, L., Namboodiri, N. K., and Pillai, R. K. (1962). J. Amer. Statist. Ass., 57, 863–876. 11. Konijn, H. S. (1981). Metrika, 28, 109–121. 12. Krewski, D. and Rao, J. N. K. (1981). Ann. Statist., 9, 1010–1019. 13. Lahiri, D. B. (1951). Bull. Int. Statist. Inst., No. 2, 33, 133–140. 14. McCarthy, P. J. (1969). Rev. Int. Statist. Inst., 37, 239–264. 15. Mickey, M. R. (1959). J. Amer. Statist. Ass., 54, 594–612. 16. Midzuno, H. (1952). Ann. Inst. Statist. Math., 3, 99–107. 17. Murthy, M. N. and Nanjamma, N. S. (1959). Sankhya, ¯ 21, 381–392. 18. Oh, H. L. and Scheuren, F. J. (1978). Amer. Statist. Ass., Proc. Sect. Surv. Res. Meth., pp. 723–728. 19. Olkin, I. (1958). Biometrika, 45, 154–165. 20. Quenouille, M. H. (1956). Biometrika, 43, 353–360. 21. Rao, J. N. K. (1957). J. Indian Soc. Agric. Statist., 9, 191–204. 22. Rao, J. N. K. (1964). J. Indian Soc. Agric. Statist., 16, 175–188. 23. Rao, J. N. K. (1968). J. Indian Statist. Assoc., 6, 160–168. 24. Rao, J. N. K. (1969). In New Developments in Survey Sampling, N. L. Johnson and H. Smith, Jr., eds. Wiley, New York, pp. 213–234. 25. Rao, J. N. K. (1974). Tech. Rep., Statistics Canada. 26. Rao, J. N. K. and Kuzik, R. A. Sankhya¯ C, 36, 43–58.

(1974).

27. Rao, J. N. K. and Pereira, N. P. Sankhya¯ A, 30, 83–90.

(1968).

6978

RATIO ESTIMATORS—II

28. Rao, J. N. K. and Vijayan, K. (1977). J. Amer. Statist. Ass., 72, 579–584. 29. Rao, P. S. R. S. (1981). In Current Topics in Survey Sampling, D. Krewski, R. Platek and J. N. K. Rao, eds. Academic Press, New York, pp. 305–316. 30. Rao, P. S. R. S. and Rao, J. N. K. (1971). Biometrika, 58, 625–630. 31. Rao, T. J. (1966). Ann. Inst. Statist. Math., 18, 117–121. 32. Robson, D. S. (1957). J. Amer. Statist. Ass., 52, 511–522.

total sales of all the items of interest, are only two illustrations. To estimate this ratio, consider a sample of n units selected without replacement from the above population.
n The means
n of this sample are x = 1 xi /n and y = 1 yi /n. The
sample variances and covariance are s2x = n1 (xi − x)2 /(n − 1), s2y =
n
n 2 1 (yi − y) /(n − 1), and sxy = 1 (xi − x)(yi − ˆ = y/x is biased y)/(n − 1). The sample ratio R for R, and its approximate variance is given by

33. Royall, R. M. and Cumberland, W. G. (1981). J. Amer. Statist. Ass., 76, 66–77.

ˆ = (1 − f )S2 /(nX ), V(R) d

34. Royall, R. M. and Eberhardt, K. R. (1975). Sankhya¯ C, 37, 43–52. 35. Scott, A. J. and Wu, C. F. (1981). J. Amer. Statist. Ass., 76, 98–102. 36. Srivastava, S. K. (1980). Canad. J. Statist., 8, 253–254. 37. Tin, M. (1965). J. Amer. Statist. Ass., 60, 294–307.


2 where f = n/N and S2d = N 1 (yi − Rxi ) /(N − 1). An estimate of this variance is given ˆ = (1 − f )s2 /(nx2 ), where s2 =
n (yi − by v(R) 1 d d ˆ i )2 /(n − 1). Rx ˆ The ratio estimator for Y is Yˆ R = X R, and its large-sample variance is given by 2 ˆ This variance is smaller V(Yˆ R ) = X V(R). than the variance of the sample mean V(y) = (1 − f )S2y /n if ρ > Cx /(2Cy ), where Cx = Sx /X and Cy = Sy /Y are the coefficients of variation of x and y, respectively. An estimator of 2 ˆ the above variance is v(Yˆ R ) = X v(R). Cochran [5] presents the developments on this topic until 1976. A description of the ratio method of estimation up to 1985 was provided by J. N. K. Rao [24]. The following sections update this with results that appeared after that date and supplement it with relevant earlier material. The author [38] has also reviewed this topic along with the regression method of estimation.

38. Wu, C. F. (1982). Biometrika, 69, 183–189. 39. Wu, C. F. and Deng, L. Y. (1983). In Scientific Inference, Data Analysis and Robustness, G. E. P. Box et al., Academic Press, New York, pp. 245–277. 40. Yates, F. (1960). Sampling Methods for Censuses and Surveys, 4th ed. Macmillan, New York, p. 271. See also BIAS REDUCTION, QUENOUILLE’S METHOD FOR; JACKKNIFE METHODS; MULTIVARIATE RATIO ESTIMATORS; RATIO AND REGRESSION ESTIMATORS, MICKEY’S; RATIO ESTIMATOR, PASCUAL’S; and RATIO ESTIMATORS, TIN’S.

J. N. K. RAO

RATIO ESTIMATORS—II Consider a finite population of N units having characteristics (xi , yi ) for i = (1, 2, . . . , N) with

N totals X = N X= 1 xi and Y = 1 yi , means
X/N and Y = Y/N, variances S2x = N (xi − 1
2 X)2 /(N − 1) and S2y = N 1 (yi − Y) /(N − 1),
N covariance Sxy = 1 (xi − X)(yi − Y)/(N − 1), and correlation ρ = Sxy /Sx Sy . Estimation of the ratio R = Y/X = Y/X is of much practical importance. The ratio of the total yield of a specified variety of food grains to the total yield of all the varieties, and of the total sales of an item to the

2

OPTIMALITY OF THE RATIO METHOD Consider the linear model yi = α + βxi + εi for i = 1, 2, . . . , n, where the random error εi in general has zero mean and variance σ 2 xti , 0
t
2. When the regression of y on x goes through the origin, that is, α = 0, then ˆ is the best linear unbiased estimator of β R provided t = 1. For the superpopulation approach, the finite population is considered to be a random sample from an infinite population.

RATIO ESTIMATORS—II

Rao [27] considered the above model to hold for i = 1, 2, . . . , N and compared Yˆ R with two estimators obtained from selecting the sample with unequal probabilities. Royall [41] established the optimality of the ratio method through the superpopulation model and the model-based approach.

6979

There have been further extensions and modifications of the jackknife method. For instance, reapplication of the jackknife procedure was examined by Schucany et al. [43] for g = 2 and by Rao [31] for g = n. Rao [34] also considered the modified jackknife estimator ∗ ˆ Q (X − x). Yˆ Q = y + R

BIAS REDUCTION ˆ and Yˆ R are of O(n−1 ), and The bias of R more than one approach has been suggested for completely eliminating the bias or reducing it. For instance, from the Taylor expansion of Yˆ R , Tin [47] reduces its bias to O(n−2 ). A review of this procedure appears in Schreuder [42].

The motivation for this approach arises from noting that Yˆ R can be expressed as the ˆ − x), and it is regression estimator y + R(X biased due to the estimation of R. Ghosh and Ruberto [8] suggest ratio estimators with interpenetrating subsamples and jackknifing. VARIANCE ESTIMATION

THE JACKKNIFE For reducing the bias of an estimator and estimating its variance, Quenouille [23] suggested the jackknife∗ procedure. For this procedure, the sample is divided into g groups of size m = n/g with means (xj , yj ) for j = 1, 2, . . . , g. Let xj = (nx − mxj )/(n − ˆ  = y /x , and m), yj = (ny − myj )/(n − m), R j j j
g ˆ   ˆ R = ( 1 Rj )/g. The estimator for R now is obtained from the relation ˆ Q = gR ˆ − (g − 1)R ˆ . R The corresponding estimator for Y is Yˆ Q = ˆ Q . If N is not large, the coefficients XR g and g − 1 should be replaced by w and w − 1 respectively, where w = g(N − n + m)/N [12]. With g = n, the bias of this estimator will be O(n−2 ). ˆ and R ˆ Q with g = Durbin [7] compared R 2 through the preceding regression model when t = 0 and x follows a gamma distribution. Rao [28] compared Yˆ R , Yˆ Q with g = 2, an unbiased estimator Yˆ , suggested HR

by Hartley and Ross [10], and an estimator constructed by Pascual [21] from the Taylor expansion of Yˆ R when 0
t
2 and x has a gamma distribution. Rao and Rao [29] compared Yˆ Q when g = n with Yˆ and other ratio-type estimators for R

the mean through the regression model, and Huchison [11] through a Monte Carlo study.

As mentioned, an estimator of the variance 2 ˆ The jackknife estiof Yˆ R is v(Yˆ R ) = X v(R). mator is (1 − f )(g − 1) 2  ˆ  ˆ  )2 . X (Rj − R vQ (Yˆ R ) = g g

1

Rao and Rao [29] compared the biases and stabilities of these two estimators through the regression model. An alternative estimator for the variance of Yˆ R is given by v (Yˆ R ) = (1 − f )s2d /n. For large populations, the weighted least-squares estimator is vl (Yˆ R ) =

2 n ˆ i )2  X (yi − Rx . n(n − 1)x xi 1

Rao [36] compared the relative merits of these and the above two estimators. Chaudhry [4] also examines some variance estimators for the variance of Yˆ R . STRATIFICATION When the population is divided into L strata of sizes Nh , h = (1, 2, . . . , L), the sample means (xh , yh ) are obtained by selecting nh units randomly without replacement from the hth stratum. The separate ratio estimator for Y is Yˆ RS =

L ˆ Nh , Wh = Nh /N, 1 Wh Y Rh , where N =

6980

RATIO ESTIMATORS—II

and Yˆ Rh = X h (yh /xh ). Procedures for reducing the bias of this estimator and for estimating its variance are easily obtained by extending the results of the preceding sections to more than one stratum. The combined ratio estimator for Y is ˆ C, Yˆ RC = X R ˆ C = (
Wh y )/
Wh xh . Let dhi = where R h ˆ C (xhi − xh ) for i = (1, 2, . . . , nh ) (yhi − yh ) −
R n and S2dh = 1 h d2hi /(nh − 1). The classical or linearized estimator for the variance or mean ˆ C is squared error (MSE) of Yˆ C = X R v(Yˆ C ) =

h 

Wh2 (1 − fh )S2dh /nh .

1

Denoting the means obtained by deleting ˆ (hj) = the jth observation as (xhj , yhj ), let R
L
L   (Nh yhj + i=h Ni yi )/(Nh xhj + i=h Ni xi ) and ˆ (h) =
nh R ˆ (hj) /nh . The jackknife estimator R 1 ˆ ˆ J , where R ˆ J = nR ˆ −
h (nh − for Y is Y J = X R 1 ˆ (h) . The estimator for the MSE obtained 1)R from this approach is 2  Nh − nh nh − 1 vJ (Yˆ C ) = X Nh nh L 1

nk

×



ˆ (hj) − R ˆ (h) )2 . (R

ˆ in this expresFor an alternative estimator, R
B ˆ ˆ sion is replaced by Rb = 1 Rb /B and the denominator by B − 1. The corresponding estimator for the variance of Yˆ R is given 2 ˆ b ). This approach can be extended by X y(R to find the variance of the combined ratio estimator for Y described earlier. McCarthy and Snowden [18] examined the bootstrap method for estimating the variance of Yˆ R . For large stratum sizes, the bootstrap method for the combined ratio estimator was investigated by J. N. K. Rao and Wu [26] and a modification of this approach by Kovar et al. [14]. For finite populations, a two-step boot-strap method was considered by Sitter [45,46], and a method for selecting the bootstrap samples by Booth et al. [2]. Rao and Katzoff [39] examined the relative merits of the different procedures for selecting the bootstrap samples with or without replacement from the initial sample to estimate the variances of the ratio estimator and the combined ratio estimator. All these investigations showed that the bootstrap variance estimator can be less stable than the simple estimators obtained through Taylor’s series. However, for some of the bootstrap procedures, the coverage probability of the resulting confidence limits can be close to the specified value, provided the bootstrap sample sizes are chosen suitably.

1

Jones [13] and Krewski and Rao [15] present alternative jackknife-type variance estimators. THE BOOTSTRAP This procedure can be employed for estiˆ or Yˆ R and for mating the variance of R finding confidence limits for the population ratio, mean or total. In this approach a large number B of independent samples each of size m
n are selected from the initial sample. Denoting the means obtained from these samples by (xb , yb ), b = 1, 2, . . . , B, estimates ˆ b = y /xb . The bootstrap∗ for R are given by R b ˆ now is estimator for the variance of R ˆ b) = v(R

B  ˆ b − R) ˆ 2 /B. (R 1

DOUBLE SAMPLING For the ratio estimator Yˆ R , the population mean of the auxiliary variable X should be known. If not, it is first estimated from the mean x1 of a sample of n1 units selected randomly without replacement from the N units. In the second phase, the sample means (x, y) are obtained from a subsample of n units selected from the n1 units. The ratio estimator for Y now is Yˆ s = (y/x)x1 . Rao [35] compared this and eight estimators with reduced biases; for the linear model described earlier, Yˆ s has smaller bias and MSE than these estimators, provided α = 0 and t = 1. For the sake of convenience or otherwise, in some situations the second sample is drawn independently of the first. When (x1 , x) are available from such a procedure, the minimum variance unbiased estimator∗ for X is

RATIO ESTIMATORS—II

given by x∗ = ax1 + (1 − a)x, where a = (N − n)n1 /(Nn1 + Nn − 2nn1 ). An alternative esti∗ mator for Y now takes the form Yˆ s = (y/x)x∗ . The mean xν of the ν distinct units, which is unbiased, has smaller variance than either x1 or x∗ . Now the estimator for Y becomes Yˆ ν = (y/x)xν . When the second sample is drawn ∗ independently of the first, both Yˆ s and Yˆ ν have smaller MSEs than (y/x)x1 [33]. These two methods of combining the information from independent samples were also examined by Rao [32] for estimating Y by the procedure of Hartley and Ross [10]. NONRESPONSE When only n1 of the n units selected from the population respond, their means (x , y ) are biased for the population means (X, Y). In such a case, Hansen and Hurwitz [9] suggest selecting a subsample of m units from the n2 = n − n1 nonrespondents and eliciting responses from all of them. Denoting the means of this subsample by (x2m , y2m ), unbiased estimators for the population mean are given by n1 x + n2 x2m Xˆ H = n 

n1 Y + n2 y2m Yˆ H = n NONRESPONDENTS, (see SUBSAMPLING HANSEN-HURWITZ METHOD FOR). Rao [37] investigated the properties of the ratio estimators t1 =

Yˆ H Yˆ H X, t2 = X. x Xˆ H

The first estimator can be considered when there is nonresponse∗ on both x and y, and the second one only on y. He also examined two estimators obtained by poststratifying the n2 nonrespondents. RELATED TOPICS Raking ratio estimation is an iterative method for adjusting the estimates of the cell counts in a two-way table. Binder and

6981

Th´eberge [1] describe procedures for estimating the variance of the raking ratio estimators. The approximate MSE of the product estimator Yˆ p = X(y/x) is smaller than the variance of y if ρ is negatively large. Chaubey et al. [3] compared this estimator with Yˆ R through the linear model described earlier. Some alternative estimators of the form ˆ ˆ for the population total Y have been Y(X/ X) suggested in the literature. An empirical investigation [40] did not find them to be beneficial. The model for regression through the origin described above and Pitman’s measure of closeness∗ were used by Lee and Peddada [17] for comparing some of the ratiotype estimators. The absolute bias of the ratio estimator was examined by Meng [19]. Ratio and regression-type estimators have been compared [22] when the current population total of the supplementary variable X is updated from information on previous occasions. Singh et al. [44] construct a linear combination of the supplementary variables which is highly correlated with the main variable. Kumar [16] examines the ratio estimators with size stratification. Mullenix [20] examines the ratio method for estimating count totals in quality control. Deville [6] suggests a two-stage sample design for quality control of census data. REFERENCES 1. Binder, D. A. and Th´eberge, A. (1988). Estimating the variance of raking-ratio estimators. Can. J. Statist., 16 (Supp.), 47–55. 2. Booth, J. G., Butler, R. W., and Hall, P. (1994). Bootstrap method for finite populations. J. Amer. Statist. Ass., 89, 1282–1289. 3. Chaubey, Y. P., Dwivedi, T. D., and Singh, M. (1984). An efficiency comparison of product and ratio estimators. Commun. Statist. Theory and Methods, 13(16), 699–709; 14(5), 1249–1250. 4. Chaudhry, N. A. (1990). The weighted jackknife for ratio estimation. Commun. Statist. Theory and Methods, 19(9), 3283–3313. 5. Cochran, W. G. (1977). Sampling Techniques, 3rd ed. Wiley, New York. 6. Deville, J. C. (1993). Optimum two-stage sample design for ratio estimators. Application to

6982

RATIO ESTIMATORS—II

quality control—1990 French Census. Survey Methodol., 19, 173–182. 7. Durbin, J. (1959). A note on the application of Quenouille’s method of bias reduction to the estimation of ratios. Biometrika, 46, 477–480. 8. Ghosh, S. and Roberto, G. (1986). Comparison of ratio estimators based on interpenetrating subsamples with or without jackknifing. J. Indian Soc. Agric. Statist., 38(2), 200– 210. 9. Hansen, M. H. and Hurwitz, W. N. (1946). The problem of nonresponse in sample surveys. J. Amer. Statist. Ass., 41, 517–529. 10. Hartley, H. O. and Ross, A. (1954). Unbiased ratio estimates. Nature, 174, 270–271. 11. Huchison, M. C. (1971). A Monte Carlo comparison of some ratio estimators. Biometrika, 58, 313–321. 12. Jones, H. L. (1963). The jack-knife method. Proc. IBM Sci. Comput. Symp. Statist. 13. Jones, H. L. (1974). Jackknife estimation of functions of stratum means. Biometrika, 61, 343–348. 14. Kovar, J. G., Rao, J. N. K., and Wu, C. F. J. (1988). Measuring errors in survey estimates. Can. J. Statist., 16 (Supp2.), 25–46. 15. Krewski, D. and Rao, J. N. K. (1981). Inference from stratified samples: properties of the linearization, jackknife and balanced repeated replication methods. Ann. Statist., 9, 1010–1019. 16. Kumar, P. (1985). On the efficiency of the ratio estimators under size stratification. J. Indian Soc. Agric. Statist., 37, 121–130. 17. Lee, C. M. S. and Peddada, S. D. (1987). Comparison of some common ratio estimators using Pitman measure of nearness. Commun. Statist. Theory and Methods, 16, 2017–2027. 18. McCarthy, P. J. and Snowden, C. B. (1985). The bootstrap and finite population sampling. Vital and Health Statistics, Ser. 2, No. 95, Public Health Service Publication 85–1369, U. S. Government Printing Office, Washington. 19. Meng, X.-L. (1993). On the absolute bias ratio of ratio estimators. Statist. and Probab. Lett., 18, 345–348. 20. Mullenix, P. (1990). Accurate estimation of count totals using ratio estimators. J. Quality Technol., 22, 277–282. 21. Pascual, J. N. (1961). Unbiased ratio estimators in stratified sampling. J. Amer. Statist. Ass., 56, 70–87; 58, 1298–1299. 22. P´erez-Salvador, B. R. and M´endez-Ramırez, I. (1995). Estimating the population mean using

23. 24.

25.

26.

27.

28.

29.

30. 31. 32.

33.

34.

35.

36.

37.

38.

39.

supplementary dephased information. Commun. Statist. Simul. and Comput., 24(31), 733–743. Quenouille, M. H. (1956). Notes on bias in estimation. Biometrika, 43, 353–360. Rao, J. N. K. (1986). Ratio estimators. In Encyclopedia of Statistical Sciences, S. Kotz, N. L. Johnson, and C. B. Read, eds., Vol. 7. Wiley, New York, pp. 639–646. Rao, J. N. K. (1988). Variance estimation in sample surveys. In Handbook of Statistics, Vol. 6, P. R. Krishnaiah and C. R. Rao, eds. Elsevier Science. Rao, J. N. K. and Wu, C. F. J. (1988). Resampling inference with complex survey data. J. Amer. Statist. Ass., 83, 231–241. Rao, P. S. R. S. (1968). On three procedures of sampling from finite populations. Biometrika, 55, 438–440. Rao, P. S. R. S. (1969). Comparison of four ratio-type estimates. J. Amer. Statist. Ass., 64, 574–580. Rao, P. S. R. S. and Rao, J. N. K. (1971). Small sample results for ratio estimators. Biometrika, 58, 625–630. Rao, P. S. R. S. (1972). On two phase regression estimator. Sankhya A, 33, 473–476. Rao, P. S. R. S. (1974). Jack-knifing the ratio estimator. Sankhya C, 36, 84–97. Rao, P. S. R. S. (1975). Hartley-Ross type estimators with two phase sampling. Sankhya C, 37, 140–146. Rao, P. S. R. S. (1975). On the two-phase ratio estimator in finite populations. J. Amer. Statist. Ass., 70, 839–845. Rao, P. S. R. S. (1979). On applying the jackknife procedure to the ratio estimator. Sankhya C, 41, 115–126. Rao, P. S. R. S. (1981). Efficiencies of nine two-phase ratio estimators for the mean. J. Amer. Statist. Ass., 76, 434–442. Rao, P. S. R. S. (1981). Estimation of the mean square error of the ratio estimator. In Current Topics in Survey Sampling, D. Krewski, R. Platek, and J. N. K. Rao, eds. Academic Press, New York, pp. 305–316. Rao, P. S. R. S. (1986). Ratio estimation with subsampling the nonrespondents. Survey Methodol., 12(2), 217–230. Rao, P. S. R. S. (1988). Ratio and regression estimators. In Handbook of Statistics, Vol. 6, P. R. Krishnaiah and C. R. Rao, eds. Elsevier Science, pp. 449–468. Rao, P. S. R. S. and Katzoff, M. H. (1996). Bootstrap for finite populations. Commun.

RATIO ESTIMATORS, TIN’S

40.

41.

42.

43.

44.

45.

46.

47.

Statist. Simul. and Comput., 25, 979– 994. Rao, T. J. (1991). On certain methods of improving ratio and regression estimators. Commun. Statist. Theory and Methods, 20(10), 3325–3340. Royall, R. M. (1970). On finite population sampling theory under certain linear regression models. Biometrika, 57, 377–387. Schreuder, H. T. (1988). Tin’s ratio estimators. In Encyclopedia of Statistical Sciences, Vol. 9, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley, New York, pp. 270–271. Schucany, W. R., Gray, H. L., and Owen, D. B. (1971). On bias reduction in estimation. J. Amer. Statist. Ass., 66, 524–533. Singh, M., Kumar, P., and Chandak, R. R. (1983). Use of multiauxiliary variable as a condensed auxiliary variable in selecting a sample. Commun. Statist. Theory and Methods, 12(14), 1685–1697. Sitter, R. R. (1992). Resampling procedure for complex survey data. J. Amer. Statist. Ass., 87, 755–765. Sitter, R. R. (1992). Comparing three bootstrap methods for survey data. Can. J. Statist., 20(2), 135–154. Tin, M. (1965). Comparison of some ratio estimators. J. Amer. Statist. Ass., 60, 294–307.

See also JACKKNIFE METHODS; MULTIVARIATE RATIO ESTIMATORS; RATIO AND REGRESSION ESTIMATORS, MICKEY’S; RATIO ESTIMATORS —I; RATIO ESTIMATORS, TIN’S; and SUBSAMPLING NONRESPONDENTS, HANSEN-HURWITZ METHOD FOR.

PODURI S.R.S. RAO

RATIO ESTIMATORS, TIN’S These are estimators of the ratio R = E[Y]/E[X] of expected values of random variables Y and X, based on a simple random sample of n values (xi , yi ) from a population of size N(i = 1, . . . , n). Tin [11] proposed a modified ratio estimator∗ , as follows. Let $ n %7  n x= xi i=1

and

$ y=

n  i=1

%7 yi

n,

6983

so that the simple ratio estimator is defined by r = y/x. Tin’s estimator of R is given by    Sxy S2 − 2x tT1 = r 1 + θ1 , xy x where θ1 = n−1 − N −1 , S2x = (n − 1)−1

n 

(xi − x)2 ,

i=1

Sxy = (n − 1)−1

n 

(xi − x)(yi − y).

i=1

He also proposed a modification, obtained by subtracting an estimate of the bias of r from tT1 , which yields     Sxy S2 S2 1 − 3θ1 2x . − 2x tT2 = r 1 + θ1 xy x x He further suggested two ratio estimators based on extensions of Quenouille’s [5] bias reduction method (see JACKKNIFE METHODS). The first one was tT3 =

g  g 1 ri , r− g−1 g(g − 1) i=1

where g(2
g
n) is the number of equal parts into which the sample of n is divided and ri is the ratio of sample means of y and x from the ith part of the sample where i = 1, 2, 3, . . . , g. According to Pascual [4] and Sastry [10], H. O. Hartley had proposed this estimator earlier, with g = n. The second one was tT4 = 83 r − (r1 + r2 ) +

1 12 (r11

+ r12 + r21 + r22 ),

where r1 and r2 are the ratio estimators from the first and second halves (g = 2) and r11 , r12 , r21 , and r22 are analogous ratio estimators obtained from dividing the first and second halves of the sample into two equal parts (g = 4). These estimators all have biases of order n−2 .

6984

RATIO OF BIVARIATE NORMAL VARIABLES

Tin compared tT1 with r, with a special case of Quenouille’s estimator∗ TQ2 = 2r − r2 , 2 rj /2, and with Beale’s ratio where r2 = i=1 estimator  7  Sxy S2 t B = r 1 + θ1 1 + θ1 2x . xy x It is assumed that |(x − E[x])/E[x]|  1 for n large enough. Analytical comparisons were made for finite populations, infinite populations in which X and Y have a bivariate normal distribution∗ , and for infinite populations in which Y = α + βX + e, where the e’s are uncorrelated with zero mean and constant variance δ for fixed X, and X has a gamma distribution∗ with parameter h. He concluded that generally tT1 is the most efficient, followed by tB and tQ1 in that order, and that tT2 and tT4 are less attractive than the alternatives considered. See RATIO AND REGRESSION ESTIMATORS, MICKEY’S and BIAS REDUCTION, QUENOUILLE’S METHOD FOR for discussions of comparisons of Tin’s with other ratio estimators by Hutchinson [3], Rao and Beegle [6], Rao and Webster [7], deGraft-Johnson and Sedransk [2], and Rao [8]. Sahoo [9] studied the estimator  −1   Sxy S2 . − 2x rS = r 1 − θ xy x In summary, it appears that the Tin estimator tT1 is one of the more efficient ratio estimators available, but it is somewhat more difficult to compute than alternatives and has no readily computable variance estimate.

9. Sahoo, L. N. (1985). J. Statist. Res., 17, 1–6. 10. Sastry, K. V. R. (1965). Indian Soc. Agric. Statist., 17, 19–29. 11. Tin, M. (1965). J. Amer. Statist. Ass., 60, 294–307. See also BIAS REDUCTION, QUENOUILLE’S METHOD FOR; RATIO AND REGRESSION ESTIMATORS, MICKEY’S; RATIO ESTIMATOR, PASCUAL’S; and RATIO ESTIMATORS.

HANS T. SCHREUDER

RATIO OF BIVARIATE NORMAL VARIABLES The distribution of the ratio Y = X1 /X2 when the joint distribution of X1 and X2 is bivariate normal∗ arises in statistical theory in a number of ways, among which the following may be mentioned: 1. Estimation of intercept of a regression∗ line on the axis of the independent (predicting) variable (or more generally, of the value of the predicting variable giving a specified value for the regression) 2. Computation of relative closeness∗ of estimators (see Johnson [2]) Given that the joint distribution of X1 , X2 has probability density function (PDF) fX1 ,X2 (x1 , x2 ) =

REFERENCES 1. Beale, E. M. L. (1962). Ind. Organ. 31, 27–28. 2. deGraft-Johnson, K. T. and Sedransk, J. (1974). Ann. Inst. Statist. Math., 26, 339– 350. 3. Hutchinson, M. C. (1971). Biometrika, 58, 2,313–2,321. 4. Pascual, J. N. (1961). J. Amer. Statist. Ass., 56, 70–87. 5. Quenouille, M. H. (1956). Biometrika, 43, 353–360. 6. Rao, J. N. K. and Beegle, L. D. Sankhya B, 29, 47–56.

7. Rao, J. N. K. and Webster, J. T. (1966). Biometrika, 53, 571–577. 8. Rao, P. S. R. S. (1981). J. Amer. Statist. Ass., 76, 434–442.

(1967).

1 6

2π σ1 σ2 1 − ρ 2   1 2 2 × exp − − 2ρu u + u ) , (u 1 2 2 2(1 − ρ 2 ) 1 ui = (xi − ξi )/σi ,

i = 1, 2

[so that Xj has a normal N(ξj , σj2 ) distribution (j = 1, 2) and the correlation∗ between X1 and X2 is ρ], we have Y= =

ξ 1 + σ1 U1 σ1 U1 + ξ1 /σ1 X1 = = X2 ξ 2 + σ1 U2 σ2 U2 + ξ2 /σ2 σ1 U1 + δ1 , σ2 U2 + δ2

(1)

RATIONALITY CRITERION

where U1 , U2 has a standardized bivariate normal distribution with correlation ρ, and δj−1 = (ξj /σj )−1 is the coefficient of variation∗ of Xj (j = 1, 2). Apart from the multiplying factor σ1 /σ2 , the distribution of Y depends only on δ1 and δ2 , and the correlation coefficient, ρ. If δ2 is large so that Pr[U2 + δ2 < 0] is negligible, the formula   U1 + δ Pr 125. Although we do not know X2 , we do know that the action taken by R would be to reject. If, instead, Y1 = 20, Y2 = 100 were observed, then we would not know whether or not X1 + X2  125. In this case rationality implies nothing about the procedure I.

6986

RATIONAL SUBGROUP

Thus, when no censoring occurs, the same action as with R will always be taken. In other cases, where, despite some (or all) censoring of the data, it is still possible to determine the action of R, one will again adopt this action. The ‘‘optimality’’ or ‘‘reasonability’’ criterion of R may vary according to the problem. In Sackrowitz and Samuel-Cahn [5], the criterion of unbiasedness∗ was used in a model with censored data. In that problem no unbiased test based on the censored test was rational with respect to the test which is UMPU when there is no censoring. The rationality principle is closely related to the censoring principle, formally introduced on p. 193 of Berger and Wolpert [2], but attributed to Pratt [3,4]. A very clear and coherent discussion of these and other related concepts can be found in Berger’s book [1], especially 1.6, where additional relevant references are given.

in a problem in acoustics. Let Y1 , Y2 , . . . , Yn be a sample of size n from a normal distribution N(0, σ 2 ). The 8
probability density n 2 function (PDF) of X = i=1 Yi —that is, the distance from the origin to a point (Y1 , . . . , Yn ) in n-dimensional Euclidean space—is f (x; n, σ ) =

2xn−1 exp[−x2 /(2σ 2 )] (2σ 2 )n/2 (n/2) x > 0,

This is the Rayleigh distribution [5]. The distribution with PDF f (x; n, 1) is the chi distribution∗ with n degrees of freedom [3]. Some important properties of the Rayleigh distribution are as follows: 1. The cumulative distribution function is P(n/2, x2 /(2σ 2 )) where P(a, x) =

REFERENCES 1. Berger, J. O. (1985). Statistical Decision Theory, 2nd ed. Springer-Verlag, New York. 2. Berger, J. O. and Wolpert, R. L. (1988). The Likelihood Principle, Institute of Mathematical Statistics Lecture Notes–Monograph Series, Vol. 6, 2nd ed. IMS, Hayward, Calif. 3. Pratt, J. W. (1961). Review of Lehmann’s Testing Statistical Hypotheses, J. Amer. Statist. Assoc. 56, 163–166. 4. Pratt, J. W. (1962). Discussion to ‘‘On the foundations of statistical inference’’. J. Amer. Statist. Assoc. 57, 314–316. 5. Sackrowitz, H. and Samuel-Cahn, E. (1994). Rationality and unbiasedness in hypothesis testing. J. Amer. Statist. Assoc. 89, 967–971. See also CENSORED DATA and INCOMPLETE DATA.

RATIONAL

SUBGROUP. See ACCEPTSAMPLING; CONTROL CHARTS; QUALITY CONTROL, STATISTICAL RAYLEIGH DISTRIBUTION The Rayleigh distribution was originally derived by Lord Rayleigh (J. W. Strutt) [6]

1 (a)



x

ta−1 e−t dt.

0

2. The νth moment about zero is  2ν/2 σ ν  3. The mode is

 ν+n 5 (n/2). 2

√ n − 1σ .

4. For fixed n, the distribution belongs to the one-parameter exponential family of distributions∗ . Suppose that X1 , . . . , XN is a sample of size N from the distribution. A sufficient statistic∗
N 2 for σ is i=1 Xi , and its mean and variance are 2 E

HAROLD SACKROWITZ

ANCE

σ > 0.

2 var

N  i=1 N 

3 = nNσ 2 ,

Xi2 3

Xi2 = 2nNσ 4 .

i=1

The 8 maximum likelihood∗ estimator of
N 2 σ is i=1 Xi /nN. 5. The distribution is a special case of type II Bessel function distributions∗ . For further information, see refs. 1, 4, and 7.

RAYLEIGH DISTRIBUTION

The definition of a Rayleigh random vector, based on Miller [5], is as follows. Let Xn(k) = (x1k , x2k , . . . , xnk ), 1
k
p, be ndimensional normal random vectors. Let E[Xn(k) ] = An(k) , 1
k
p, be the mean vec(j) tors. Let Vp , 1
j
n, be the p-dimensional vector composed of the jth components (j) of Xn(k) . Let Vp , 1
j
n, be independent with positive-definite covariance matrix Mp independent of j. Then we shall say (p) that Znp = {Xn(1) , Xn(2) , . . . , Xn } is of class Nn [Mp : Anp ], where Anp is the n × p matrix whose columns are An(k) , 1
k
p, vectors. If Znp is of class Nn [Mp : Anp ], then we say 8Rp = {R1 , R2 , . . . , Rp }, where Rk =
n 2 |Xn(k) | = i=1 xik , 1
k
p, is a Rayleigh random vector. Miller [5] discusses the following: (a) the PDF of R = |Xn | when {Xn } belongs to class Nn [ψ0 In : An ], where ψ0 is a positive constant and In is the n × n identity matrix, and the νth moment of R; (b) the joint PDF of R = |Xn |, S = |Yn | when {Xn , Yn } is of class Nn [M2 : An : An ], and the second moment E[Rµ Sν ] when An is zero vector; (c) the PDF (p) of Rp when Znp = {Xn(1) , . . . , Xn } is of class Nn [Mp : 0np ] where 0np is a n × p zero matrix; and (d) the PDFs of the products RS and the quotient S/R, where R = |Xn |, S = |Ym |, and Xn and Ym are mutually independent and of class Nn [ψ0 In : An ] and Nm [ψ0 Im : Bm ], respectively, and the νth moments of the product RS and the quotient S/R. The distributions with f (x; 1, σ ), f (x; 2, σ ) and f (x; 3, σ ) are sometimes called the folded∗ Gaussian (the half normal distribution), the Rayleigh distribution, and the Maxwell (–Boltzmann) distribution, respectively. Some important properties of the Rayleigh distribution with PDF f (x; 2, θ ) are as follows:

6987

√ . Skewness: 2(π − 3) π /(4 − π )3/2 = 0.631110 . Kurtosis: (32 − 3π 2 )/(4 − π )2 = 3.24509 Mode: θ √ . Median: ln 4θ = 1.17741θ pth Percentile of the Distribution: x(p) = [−2θ 2 ln(1 − p)]1/2 ,

1>p>0

Hazard Function: x/θ 2 (The Rayleigh distribution has been frequently used as a model with increasing hazard function of the form ax, a > 0.) The distribution with PDF f (x; 2, θ ) is the special case of the Weibull distribution∗ . There are several generalizations of this distribution [5,9]. For estimation of the parameters of a generalized Rayleigh distribution, see ref. 9. When a random variable X is distributed as the Rayleigh distribution with PDF f (x; 2, θ ), the variate X −1 is distributed as the inverse Rayleigh distribution with PDF g(x; θ ) = [1/(θ 2 x3 )] exp[−1/(2θ 2 x2 )], x > 0,

θ > 0.

Probability Density Function: f (x; 2, θ ) = (x/θ 2 ) exp[−x2 /(2θ 2 )], x > 0, θ > 0 Cumulative Distribution Function: 1 − exp[−x2 /(2θ 2 )] √ vth Moment about Zero: ( 2θ )ν (ν/2) (ν/2) 6 . Mean: π/2θ = 1.25331θ . Variance: [(4 − π )/2]θ 2 = 0.42920θ 2

Figure 1. Above: θ = 0.5, 1, 1.5, 2, 2.5; below: θ = 3, 4, 5, 6, 7.

6988

RAYLEIGH QUOTIENT

For estimation and tests of hypotheses of the parameter in the inverse Rayleigh distribution, see refs. 2 and 8. Figure 1 shows the graphs of the PDF f (x; 2, θ ). In drawing the figure we used a program in ref. 1. REFERENCES 1. Hirano, K., Kuboki, H., Aki, S., and Kuribayashi, A. (1983). Comput. Sci. Monogr., No. 19, Inst. Statist. Math., Tokyo, Appendix. 2. Iliescu, D. V. and Voda, V. Gh. (1973). Studii si Cerc. Mat. Bucuresti, 25, 1507–1521 (in Rumanian). 3. Johnson, N. L. and Kotz, S. (1976). Distributions in Statistics, Vol. 1: Continuous Univariate Distributions. Wiley, New York, Chap. 17. 4. Kotz, S. and Srinivasan, R. (1969). Ann. Inst. Statist. Math., Tokyo, 21, 201–210. 5. Miller, K. S. (1964). Multidimensional Gaussian Distributions. Wiley, New York, Chaps. 2 and 3. 6. Rayleigh, J. W. S. (1919). Philos. Mag., 6th Ser., 37, 321–347. 7. Springer, M. D. (1979). The Algebra of Random Variables. Wiley, New York, Chap. 9. 8. Voda, V. Gh. (1972). Rep. Statist. Appl. Res. JUSE, 19(4), 13–21. 9. Voda, V. Gh. (1976). Apl. Mat., 21, 395–419. See also CHI DISTRIBUTION and CHI-SQUARE DISTRIBUTION.

KATUOMI HIRANO

RAYLEIGH’S TEST This is one of the most popular circular single sample tests∗ , which seek to test the null hypothesis that an angular direction φ has a uniform distribution—that is, probability density function fφ (t) = (2π )−1 ,

0
t
2π ,

—based on a random sample of n observed values t1 , t2 , . . . , tn . The test statistic is r2 = n−1

$ n   

$

%2 cos ti

i=1

+

n 

%2   sin ti

i=1



.

Sufficiently large values of r2 indicate ‘‘onesidedness’’ of the parent population. Significance limits are available in refs. 1–4. The test is uniformly most powerful∗ against circular normal∗ (von Mises distributions) alternatives. For unimodal alternatives the Rayleigh test is sensitive not only to ‘‘one-sidedness’’ but also to concentration around a mean direction—that is, there is a preferred direction. The V test is a competitor to Rayleigh’s test. REFERENCES

RAYLEIGH QUOTIENT

1. Batschelet, E. (1981). Circular Statistics in Biology. Academic, New York.

The Rayleigh quotient of the symmetric n × n matrix A with respect to the n × 1 vector X is X’AX / X’X. This function arises in the conditional maximization of a quadratic form y’Ay subject to the constraint y y = 1 (or some other positive constant). See ref. 1 for more details.

2. Greenwood, J. A.. and Durand, D. (1955). Ann. Math. Statist., 26, 233–246. 3. Stephens, M. A. (1969). J. Amer. Statist. Ass., 64, 280–289. 4. Zar, J. H. (1974). Biostatistical Analysis. Prentice-Hall, Englewood Cliffs, NJ. See also CIRCULAR NORMAL DISTRIBUTION; DIRECTIONAL DATA ANALYSIS; and PERIODOGRAM ANALYSIS.

REFERENCE 1. Basilevsky, A. (1983). Applied Matrix Algebra in the Statistical Sciences. North-Holland, Amsterdam, Netherlands.

R -CHART. See CONTROL CHARTS RECIPROCAL AVERAGING. See

See also QUADRATIC FORMS

CORRESPONDENCE ANALYSIS

RECIPROCAL INVERSE GAUSSIAN DISTRIBUTION

6989

RECIPROCAL DIFFERENCES

RECIPROCAL DISTRIBUTION

These are formed by a process that can be regarded as a sort of inversion of the construction of divided differences∗ . The reciprocal difference of f (x) for x = x0 , x1 is

This is a Pareto distribution∗ with index 1. Its probability density function (PDF) is of the form

ρ(x0 , x1 ) =

ρ2 (x0 , x1 , x2 ) x0 − x2 + f (x1 ); ρ(x0 , x1 ) − ρ(x1 , x2 )

and generally, the nth-order reciprocal difference of x0 , x1 , . . . , xn is ρn (x0 , x1 , . . . , xn ) =

(0 < a
x
b).

x0 − x1 ; f (x0 ) − f (x1 )

for x0 , x1 , x2 (a second-order reciprocal difference) it is

=

fx (x) = x−1 /(log b − log a)

x0 − xn ρn−1 (x0 , . . . , xn−1 ) − ρn−1 (x1 , . . . , xn ) +ρn−2 (x1 , . . . , xn−1 ).

RECIPROCAL INVERSE GAUSSIAN DISTRIBUTION A reciprocal inverse Gaussian RIG(µ, λ) random variable Y = 1/X is the reciprocal of the random variable X, where X has an inverse Gaussian∗ IG(µ, λ) distribution. Both RIG(µ, λ) and IG(µ, λ) consist of unimodal right-skewed distributions with non-negative support, and are subfamilies of the generalized inverse Gaussian∗ family. The probability density function (p.d.f.) of Y is   λ 1/2 −λ(1 − µy)2 , exp − fY (y|µ, λ) = 2π y 2µ2 y

Thiele’s interpolation∗ formula expresses f (x) as a continued fraction x − x1 f (x)  f (x1 ) + ρ(x1 , x2 )+ x − x2 × ρ2 (x1 , x2 , x3 ) − f (x1 )+ x − x3 × ρ3 (x1 , x2 , x3 , x4 ) − ρ(x1 , x2 )+ ···, the sequence continuing until the numerator (x − xn−1 ) is reached. This gives correct values for f (x) at x = x1 , . . . , xn and is an approximation in the form of a ratio of polynomials, as compared with the more conventional polynomial interpolation formulas (Bessel∗ , Lagrange, Newton∗ ). Further details are available in ref. 1 (Chap. 5). REFERENCE 1. Milne-Thompson, L. M. (1933). Calculus of Finite Differences. Macmillan, London. See also FINITE DIFFERENCES, CALCULUS OF and INTERPOLATION.

y > 0, µ > 0, λ > 0, (1) with mean and variance E(Y) = Var(Y) =

1 1 + , µ λ

(2)

1 2 + 2, λµ λ

(3)

and coefficients of skewness and kurtosis (moment ratios) 6

3θ + 8 , (θ + 2)3/2

(4)

3(5θ + 16) , (θ + 2)2

(5)

β1 =

and β2 =

where θ = λ/µ. The variable Y is equivalent in law to the sum U + V of independently distributed random variables, where U ∼ IG(µ−1 , λµ−2 ) and V ∼ χ12 /λ. In other words, the corresponding p.d.f. satisfies the convolution relation: fY (y) = fU (u) ∗ fV (v).

(6)

6990

RECIPROCAL INVERSE GAUSSIAN DISTRIBUTION

Thus, the expectation of sample mean Y based on n observations, E(Y) =

1 1 + , µ λ

(7)

results in a bias of 1/λ as an estimate of 1/µ. Note that µ = E(1/Y), so that 1/µ is the harmonic mean∗ of Y. Occasionally, the harmonic mean is the natural parameter of interest; for example, when measuring fuel efficiency, the number of miles per gallon is recorded and the ratio of interest is the reciprocal ratio of gallons per mile. Similarly, in the context of Brownian motion∗ with positive drift, when the barrier is at a fixed distance and the time taken to hit the barrier on the first occurence is measured, the harmonic mean is also a natural parameter. Additionally, the genesis of the IG distribution is associated with Brownian motion and is referred to as the first passage time distribution [4,5]. Interest in the RIG distribution stems from the fact that it plays a key role in the inference methods associated with the IG model as well as in the development of IG linear models [2,6]. Tweedie [7] introduced the analogue of normal-theory analysis of variance for the inverse Gaussian (IG) model which he termed the analysis of reciprocals (ANORE) because of the decomposition n n   −1 (Xi − µ)2 /(µ2 Xi ) = (Xi−1 − X ) i=1

be interpreted as A − H for the RIG variable and, based on the property that A  H for positive random variables, is guaranteed to be positive. Thus, the RIG random variable is associated with estimation of both µ and λ. Similarly, in the Bayesian analysis of the parameters of the IG distribution, conjugate priors for µ−1 and λ are truncated normal and gamma distributions (See CONJUGATE FAMILIES OF DISTRIBUTIONS. These priors match those for µ and σ −2 associated with the normal distribution. Thus, the reparametrization of the harmonic mean of RIG leads to its natural conjugate prior; for details see [1]. It is known that there are remarkable similarities between the normal and inverse Gaussian distributions; see INVERSE GAUSSIAN AND GAUSSIAN ANALOGIES∗ . The normal distribution has support over the real line, while the IG and RIG distributions both have support over the positive real line. In a sense, two distributions over the positive real line are needed to balance the support of the normal distribution over the entire real line for the analogies between the normal and both IG and RIG to hold. Acknowledgments The author wishes to thank Bill Schucany, Govind Mudholkar and Campbell Read for their useful comments on this article.

REFERENCES

i=1

+ n(X − µ)2 /(µ2 X). (8) The analysis of variance is invariant under both location and scale changes whereas the analysis of reciprocals is invariant only under scale changes. This resulted in parallel developments for the one-way classification in linear models. The maximum likelihood estimate of λ−1 based on n observations is given by  n  1 1 1 , (9) − λˆ −1 = n Xi X i=1 which in terms of the IG variable is (1/H − 1/A), H referring to harmonic mean and A to the arithmetic mean. This estimate can also

1. Banerjee, A. K. and Bhattacharyya, G. K. (1979). Bayesian results for the inverse Gaussian distribution with an application. Technometrics, 21(2), 247–251. 2. Chhikara, R. S. and Folks, J. L. (1989). The Inverse Gaussian distribution. Dekker, New York. 3. Kirmani, S. N. U. A. and Ahsanullah, M. (1987). A note on weighted distributions. Commun. Statist. Theor. Meth., 16, 275–280. ¨ 4. Schmoluchowski, M. V. (1915). Notiz uber die Berechning der Brownshen Molkularbewegung bei des Ehrenhaft-millikanchen Versuchsanordnung. Physikalische Zeitschr., 16, 318–321. 5. Schr¨odinger, E. (1915). Zur theorie der fallund steigversuche an teilchenn mit Brownsche bewegung. Physikalische Zeitschr., 16, 289–295.

RECORD LINKAGE AND MATCHING SYSTEMS 6. Seshadri, V. (1999). The Inverse Gaussian Distribution: Statistical Theory and Applications. Lecture Notes in Statistics 137, SpringerVerlag, New York. 7. Tweedie, M. C. K. (1956). Some statistical properties of inverse Gaussian distributions. Virginia J. Sci., 7, 160–165. See also BROWNIAN MOTION; GENERALIZED GAUSSIAN DISTRIBUTIONS; HARMONIC MEAN; INVERSE GAUSSIAN AND GAUSSIAN ANALOGIES; and INVERSE GAUSSIAN DISTRIBUTION.

RAJESHWARI NATARAJAN

RECIPROCAL MODEL This is the regression model E[Y|x] = (α + βx)−1 . BIBLIOGRAPHY Mead, R. (1979). Biometrics, 35, 41–54.

6991

from a sample survey∗ of some specific population of individuals, households, or businesses; decennial census data; medical or educational records; and birth or death certificates. For example, data on the age, ethnic background, and sex of a large group of individuals might be matched with income tax data in order to learn about the relationship between taxes and those variables. Also, income tax data for a given year might be matched with the corresponding data for a later year in order to study the changes that have occurred. Of course, record linkage and matching can be carried out for three or more files. However, throughout this article we shall restrict ourselves to problems of matching just two files. Roughly speaking, there are two different broad categories of matching problems: (1) problems of exact matching in which it is desired to identify pairs of records in the two files that pertain to the same individual, and (2) problems of statistical matching in which it is desired to identify pairs of records in the two files, or groups of records in the two files, that pertain to similar individuals.

Shinozaki, K. and Kira, T. (1956). J. Inst. Poly., Osaka City Univ., Ser. D, 7, 35–72.

EXACT MATCHING

RECORD LINKAGE AND MATCHING SYSTEMS Problems of record linkage and matching arise when it is desired to pair the records in two different files or data sets with each other. For example, if the two files contain different types of records for a sample of units from some specified population, it may be desired to match each record in the first file with that of the same or a similar unit in the second file in order to obtain a fuller or more complete record for each unit. In many of the applications of record linkage and matching, the units in each file are individual persons, and in this article it will often be convenient to refer to these units as individuals or persons. Some of the types of records for a particular group of individuals that have been used in matching studies are federal income tax returns; employment histories maintained by the Social Security Administration; data

The central purpose of carrying out the exact matching of two different files is often to combine the two different records of the individuals that are present in both files into a single enlarged record. The combining of different records for an individual into one large record in this way raises important issues regarding the individual’s rights to privacy and confidentiality. For this reason, matching studies based on government files must often be carried out under rigid restrictions regarding the sharing of data across different agencies. A second common purpose for the exact matching of two files is the construction of a single master file with duplications eliminated. A related but different purpose in some problems is simply to determine the presence or absence of individuals in the two files. For example, one method of estimating the magnitude of the population undercount in the decennial U.S. census∗ is to determine whether persons listed in birth records,

6992

RECORD LINKAGE AND MATCHING SYSTEMS

listings of drivers’ licenses, or alien registrations are included in the census files. When the records in each file contain accurate identifying information such as each individual’s name and social security number, the process of exact matching can be carried out essentially without any errors. In general, however, some of the identifying information in a record may be missing, erroneous, or not unique. Matching procedures must then be developed which control both the probability of pairing two records that actually pertain to different individuals and the probability of not pairing two records that actually do pertain to the same individual. Optimal methods must take into account the relative seriousness of each of these two types of errors. Statistical models and methods for exact matching procedures have been presented by Nathan [13], Tepping [21], DuBois [8], and Fellegi and Sunter [9]. For example, DuBois [8] regards the identifying information in each record as an 11-dimensional vector of which the components are the following variables: the individual’s last name, first name, middle initial, place of birth, month of birth, day of birth, and year of birth; the first initial of his or her spouse; the first three digits of his or her social security number, the next two digits of that number, and the final four digits of the number. He develops a model for deciding whether or not two records are linked (i.e., matched), based on which of these components are present in both vectors and which are identical in both vectors. He also presents an example in which approximately 73,000 questionnaires for males in a 1957 American Legion lung cancer study are matched with more than 77,000 death certificates for males in the 1960 death index of the California State Department of Public Health. Fellegi and Sunter [9] consider similar models that make use of somewhat more information. If the last names in the two records are identical, it might be noted whether it is a common or an unusual name. If these names are not identical, it might be noted whether they sound alike or whether the first several letters in the names agree. Once it has been decided that two records are linked, in the sense that they pertain

to the same individual, there often remains the problem of reconciling the discrepancies that were present in the two records. Thus, if the individual’s date of birth or reported income is not identical in the two records, it must often be decided which value, if either, it is appropriate to retain in the merged record. STATISTICAL MATCHING Problems of statistical matching∗ arise when there are expected to be relatively few or no individuals whose records are present in both files. For example, the two files might have been obtained from two distinct or independent samples drawn from some large fixed population. The two files might contain a common set of variables X that were observed for all individuals in either of the files, as well as a set of variables Y that were observed only for individuals in the first file and a set of variables Z that were observed only for individuals in the second file. One central purpose of a statistical matching study in this context might be to study relationships between Y and Z in the overall population from which the samples were drawn. Many of the commonly used procedures have the following general form. One of the two files is regarded as the base file or primary file. For each individual A in the base file, a measure mA (B) of similarity to A is specified for each individual B in the other file. This measure is based on the similarity between the observed values of the common variables X for A and B, and possibly also on other information contained in the observed values of Y for A or of Z for B. An individual B in the second file for which mA (B) is a maximum might then be matched with A or, among all individuals B in the second file for which mA (B) exceeds some fixed level of similarity, one individual might be selected at random to be matched with A, where the probability of selection of any individual B is an increasing function of mA (B). In some problems, a synthetic ‘‘average’’ individual might be created from those individuals B in the second file for which mA (B) is sufficiently large, and this synthetic individual matched with A. The matching procedure

RECORD LINKAGE AND MATCHING SYSTEMS

could either be constrained, in the sense that no individual in the second file can be matched with more than one individual in the base file, or it can be unconstrained, in which case the records in the second file can be used repeatedly in the construction of matches. A basic assumption that underlies almost all matching procedures either explicitly or implicitly, including procedures having the general form just described, is that Y and Z are conditionally independent given X. One way of thinking about statistical matching is that it is a problem of trying to make inferences about the joint distribution of X, Y, and Z based on information about just the joint marginal distribution of X and Y and the joint marginal distribution of X and Z. Such inferences must be based on some assumption linking these marginal distributions, such as the assumption of conditional independence just mentioned. Unfortunately, the two data files typically contain no information regarding the appropriateness of this assumption, and prior information from other sources must be considered. The methodology of statistical matching can be used to reduce the response burden on the individuals participating in a survey∗ . Suppose that the total time that would be required for a person to respond carefully to all the items in the survey is so great that many individuals in the population of interest would be expected to refuse to participate. One way to proceed in order to increase the response rate is to have different samples from the population respond to different sets of items, with a common core of items for all the samples. The different samples could then be statistically matched based on the common core of items in order to study relationships among the other items. One purpose in carrying out a statistical matching procedure for the individuals in a base file is to fill in gaps and missing information in the records of those individuals. From this point of view, statistical matching is a problem of missing data and many of the same techniques and imputation∗ methods that have been developed for filling in missing data in surveys are relevant here (see also INCOMPLETE DATA).

6993

EXAMPLES AND DISCUSSIONS OF EXACT AND STATISTICAL MATCHING Ruggles and Ruggles [18] discuss the shortcomings of trying to use elementary regression methods for imputation purposes, and the relative advantages of using a statistical matching procedure of the general form just described. They present in detail the matching of the Bureau of the Census∗ 1970 Public Use Sample (the PUS) with the Social Security Administration Longitudinal Employer-Employee Data File (the LEED). Each of these data files contained approximately 2 million individuals. These files had 15 X-variables in common after adjustments, realignments, and the creation and imputation of some values of X that could be derived from other observed values. To determine an appropriate similarity measure between values of X in the two files, weights had to be assigned to the 15 X-variables that appropriately reflected their relative importance in the matching process. These weights were constructed by studying the relationships between the X-variables and the Y-variables in the PUS and between the X-variables and the Z-variables in the LEED, and considering the goals of the study in terms of which relationships between Y and Z it was most important to explore. The subcommittee on Statistical Uses of Administrative Records, Federal Committee on Statistical Methodology [20] describes in detail four specific studies of the matching of administrative records by government agencies. Other examples of statistical matching studies are given by Okner [14] and by Alter [1], who describes the creation of a synthetic data set by matching family units from two Canadian surveys that were carried out for the year 1970: the Survey of Consumer Finances (the SCF) and the Family Expenditure Survey (the FEX). The SCF, containing about 10,000 records, was regarded as the primary file; the FEX, containing about 14,000 records, was the secondary file; and an unconstrained matching procedure was carried out. Kadane [11] discusses both exact and statistical matching methods when data are assumed to have a multivariate normal distribution, and proposes a Bayesian∗

6994

RECORD LINKAGE AND MATCHING SYSTEMS

approach. Radner [16] also proposes a formal framework for statistical matching. The Subcommittee on Matching Techniques, Federal Committee on Statistical Methodology [19], presents an excellent overview of exact and statistical matching for research and statistical purposes (rather than for administrative purposes). They describe several examples and give a useful bibliography. Okner [15] also presents a brief but insightful summary of problems of record linkage. Rodgers [17] gives a useful review of statistical matching that includes an evaluation of matching procedures based on simulation∗ studies. Cassel [2] also carries out an evaluation of statistical matching by means of a simulation study based on a file containing actual data from the Survey on Living Conditions carried out by Statistical Sweden∗ . Klevmarken [12] proposes a method to estimate a linear relation between Y and Z based on statistical matching.

BROKEN SAMPLES We shall now describe a class of problems of exact matching for which there is an explicit statistical model and a well-defined optimal matching procedure that in some circumstances can be explicitly described. Suppose that U is an r-dimensional random vector, V is an s-dimensional random vector, and U and V have some specified (r + s)-dimensional joint distribution. Suppose also that a random sample (U1 , V1 ), . . . , (Un , Vn ) has been drawn from this distribution but that before the sample values (u1 , υ1 ), . . . , (un , υn ) can be observed, each vector (ui , υi ) gets broken into the two separate components ui and υi . As a result, all that can be observed are the r-dimensional vectors y1 , . . . , yn and the s-dimensional vectors z1 , . . . , zn , where y1 , . . . , yn are a random permutation of the vectors u1 , . . . , un , and z1 , . . . , zn are a random permutation of the vectors υ1 , . . . , υn . The observations y1 , . . . , yn and z1 , . . . , zn are said to form a broken random sample from the joint distribution of U and V. In this context it is known that y1 , . . . , yn and z1 , . . . , zn are exactly matched with each other in some unknown rearrangement. The

basic problem, in general terms, is to determine a matching procedure that will reproduce as many pairs in the original unbroken sample as possible. This problem of repairing a broken sample was introduced by DeGroot et al. [7], who discuss the following three different criteria: (1) maximizing the probability of obtaining a completely correct set of n matches, (2) maximizing the probability of correctly matching one particular yi , and (3) maximizing the expected number of correct matches in the n pairings. Some of the results that they obtain are as follows: Suppose that r = s = 1 (i.e., both U and V are one-dimensional random variables), and that the joint distribution of U and V is a bivariate normal distribution with correlation ρ. If ρ > 0, the procedure that maximizes the probability of obtaining a completely correct set of n matches is always to order the observations y1 , . . . , yn and z1 , . . . , zn so that y1 < · · · < yn and z1 < · · · < zn , and then to pair yi with zi for i = 1, . . . , n. If ρ < 0, the optimal procedure is always to pair y1 with zn , y2 with zn−1 , and in general, yi with zn+1−i for i = 1, . . . , n. This procedure is called the maximum likelihood∗ procedure. When ρ > 0, in order to maximize the probability of correctly matching y1 , it should always be paired with z1 ; and in order to maximize the probability of matching yn , it should always be paired with zn . However, for n  3, there are some broken samples for which the probability of correctly matching y2 is not maximized by pairing it with z2 . Similarly, when ρ > 0, in order to maximize the expected number of correct matches, it is always optimal to pair y1 with z1 and yn with zn . However, for n  4, there are some broken samples for which it is not optimal to pair yi with zi for i = 2, . . . , n − 1. Nevertheless, Monte Carlo∗ studies show that the maximum likelihood procedure does maximize the expected number of correct matches for the vast majority of broken samples [5]. Furthermore, Zolotuchina and Latishev [23] have derived an asymptotic value for the expected number of correct matches when the maximum likelihood procedure is used, where the expectation is taken over all possible broken samples. They show that this expectation converges to 1/(1 − |ρ|) as n → ∞.

RECORD VALUES AND RECORD STATISTICS

Now suppose that the parameters of the bivariate normal distribution are unknown. The problem of making inferences about these parameters based on a broken random sample is considered in DeGroot and Goel [6]. In particular, it is emphasized that a broken sample does contain information about the value of ρ, and procedures for deciding whether ρ is positive or negative are developed. Extensions of some of these ideas to arbitrary joint distributions of U and V with monotone likelihood ratio∗ have been given by Chew [3] and Goel [10], and to problems of approximate matching by Yahav [22]. A survey of this topic is presented in DeGroot [4]. Acknowledgment This research was supported in part by the National Science Foundation under Grant SES8207295. I am indebted to Miron L. Straf for helpful comments.

REFERENCES 1. Alter, H. E. (1974). Ann. Econ. Social Meas., 3, 373–394. 2. Cassel, C. M. (1983). Statistical Review 1983:5 Essays in Honour of Tore E. Dalenius, L. Lyberg, ed. Statistics Sweden, Stockholm, pp. 55–66. 3. Chew, M. C., Jr. (1973). Ann. Statist., 1, 433–445. 4. DeGroot, M. H. (1980). Symposia Mathematica, Vol. XXV (Istituto Nazionale di Alta Matematica, Francesco Severi, Roma). Academic Press, New York, pp. 123–135. 5. DeGroot, M. H. and SankhyaB, 38, 14–29.

Goel, P. K.

7. DeGroot, M. H., Feder, P. I., and Goel, P. K. (1971). Ann. Math. Statist., 42, 578–593. 8. DuBois, N. S. D’A., Jr. (1969). Statist. Ass., 64, 163–174.

J. Amer.

9. Fellegi, I. T. and Sunter, A. B. (1969). J. Amer. Statist. Ass., 64, 1183–1210. 10. Goel, P. K. 1364–1369.

(1976).

Ann.

12. Klevmarken, N. A. (1983). Statistical Review 1983:5 Essays in Honour of Tore E. Dalenius, L. Lyberg, ed. Statistics Sweden, Stockholm, pp. 67–79. 13. Nathan, G. (1967). J. Amer. Statist. Ass., 62, 454–469. 14. Okner, B. (1972). Ann. Econ. Social Meas., 1, 325–342. 15. Okner, B. (1974). Ann. Econ. Social Meas., 3, 347–352. 16. Radner, D. B. (1979). Amer. Statist. Ass., 1978 Proc. Social Statist. Sec., pp. 503–508. 17. Rodgers, W. L. (1984). J. Bus. Econ. Statist., 2, 91–102. 18. Ruggles, N. and Ruggles, R. (1974). Ann. Econ. Social Meas., 3, 353–371. 19. Subcommittee on Matching Techniques, Federal Committee on Statistical Methodology (1980). Statistical Policy Working Paper 5, Office of Federal Statistical Policy and Standards, U.S. Dept. of Commerce, Washington, D.C. 20. Subcommittee on Statistical Uses of Administrative Records, Federal Committee on Statistical Methodology (1980). Statistical Policy Working Paper 6, Office of Federal Statistical Policy and Standards, U.S. Dept. of Commerce, Washington, D.C. 21. Tepping, B. J. (1968). J. Amer. Statist. Ass., 63, 1321–1332. 22. Yahav, J. A. (1982). Statistical Decision Theory and Related Topics III, Vol. 2, S. S. Gupta and J. O. Berger, eds. Academic Press, New York, pp. 497–504. 23. Zolotuchina, L. A. and Latishev, K. P. (1978). Leningrad Otd. Mat. Inst., Akad. Nauk USSR, 29, 4–10 (in Russian). See also STATISTICAL MATCHING.

MORRIS H. DEGROOT

(1976).

6. DeGroot, M. H. and Goel, P. K. (1980). Ann. Statist., 8, 264–278.

Statist.,

3,

11. Kadane, J. B. (1978). Compendium of Tax Research. Office of Tax Analysis, Dept. of the Treasury, Washington, D.C., pp. 159–171.

6995

RECORD VALUES AND RECORD STATISTICS INTRODUCTION In his pioneering paper in 1952, Chandler [12] defined record times and record values and gave groundwork for a mathematical theory of records. In many respects, this theory is very closely connected with that of order statistics∗ and, in particular, with extreme order statistics. The period 1952–1979 can

6996

RECORD VALUES AND RECORD STATISTICS

be considered as the time when the classical theory of records, based on sequences of independent and identically distributed random variables, was initiated and consolidated considerably. The next two decades gave a lot of papers and results for nonclassical record models and provided different generalizations of records. Now, the theory of records has become a very well-developed area of statistics with more than 400 papers and some monographs, which discuss its different aspects; see, for example, references 2, 4, 29, and 34.

in all their details. In fact, we can obtain the lower records from the upper records if we change the sequence X1 , X2 , . . . by the sequence −X1 , −X2 , . . .. Note that for discrete distributions, we can introduce the so-called weak records using the sign  in (3) instead of >. In this case, any repetition of a record value is also a record. We will use N(n, k) (N(n) = N(n, 1)) and (n, k) = L(n, k) − L(n − 1, k) ((n) = L(n, 1) − L(n − 1, 1)), n = 1, 2, . . ., to denote the number of kth records in a sequence X1 , X2 , . . . and kth interrecord times, respectively.

DEFINITIONS Let X1 , X2 , . . . be a sequence of random variables, X1,n
· · ·
Xn,n (n = 1, 2, . . .) be the corresponding order statistics, M(n) = Xn,n = and m(n) = X1,n = max{X1 , X2 , . . . , Xn }, min{X1 , X2 , . . . , Xn }. For any k = 1, 2, . . ., we define kth record times L(n, k) and kth record values X(n, k) as follows: L(0, k) = 0, L(1, k) = k, L(n + 1, k) = min{j > L(n, k) : Xj > Xj−k,j−1 } (1) and X(n, k) = XL(n,k)−k+1,L(n,k)

(2)

The more simpler notation L(n) = L(n, 1) and X(n) = X(n, 1) will be used for the most important case k = 1. We can define the upper record times L(n) and the upper record values X(n) in the following alternative way: L(0) = 0, L(1) = 1, L(n + 1) = min{j : Xj > M(L(n))}

(3)

and

RECORD INDICATORS AND USEFUL REPRESENTATIONS OF RECORD STATISTICS VIA SUM OF INDEPENDENT TERMS In many respects, the intensive development of the theory of records was due to the early results of Dwass [18], R´enyi [42], Tata [50], and Shorrock [44,45], which have allowed to impress record statistics in terms of sums of independent random variables. Theorem 1. Let X1 , X2 , . . . be a sequence of independent random variables with a common continuous distribution function, and let record indicators ξ1 , ξ2 , . . . be defined as follows: ξn = 1 if Xn is a record value, and ξn = 0 otherwise. Then, the random variables ξ1 , ξ2 , . . . are independent and P{ξn = 1} = 1 − P{ξn = 0} = 1/n, n = 1, 2, . . .. Theorem 1 presents very important representations for the random variables N(n) and L(n). If X1 , X2 , . . . are independent random variables with a continuous distribution function, then N(n) = ξ1 + ξ2 + · · · + ξn ,

X(n) = M(L(n)),

n = 1, 2, . . .

(4)

If we replace < and m(L(n)) by > and M(L(n)), respectively, in (3), we obtain the definitions of the lower record times and the lower record values. In an analogous way, we can present the kth lower record times and the kth lower record values. We will consider here only the upper records, as the theories of lower and upper records practically coincide

n = 1, 2, . . . ,

and P{L(m)  n} = P{N(n)
m} = P{ξ1 + ξ2 + · · · + ξn
m}. Note that if X1 , X2 , . . . are independent and have a common continuous distribution function F, then the distributions of N(n) and L(n) do not depend on F.

RECORD VALUES AND RECORD STATISTICS

The analogous results are valid for the kth records as well. Let X1 , X2 , . . . be a sequence of independent random variables with a common continuous distribution function, and let the indicators ξn(k) , n  k, be defined as follows: ξn(k) = 1 if Xn is a kth record value, and ξn(k) = 0 otherwise. Then, for any fixed k = 1, 2, . . ., the (k) random variables ξk(k) , ξk+1 , . . . are independent and P{ξn(k) = 1} = k/n, n  k. This result then implies the following for any k = 1, 2, . . ., (k) + · · · + ξn(k) , N(n, k) = ξk(k) + ξk+1

n = k, k + 1, . . . , and P{L(m, k)  n} = P{N(n, k)
m} =

P{ξk(k)

(k) + ξk+1 + · · · + ξn(k)
m}.

A number of such representations were obtained for record values X(n) [50] and the kth record values X(n, k) [20]. See also reference 5 for results in this topic. Theorem 2. Let X1 , X2 , . . . be a sequence of independent random variables having the standard exponential E(0, 1) distribution. Then, the random variables X(1), X(2) − X(1), . . . , X(n) − X(n − 1), . . . are independent and also have the same E(0, 1) distribution. Under the conditions of Theorem 2, we have d

X(n) = X1 + X2 + · · · + Xn ,

n = 1, 2, . . . .

The following generalization of the previous equality is valid for the kth record values, under the conditions of Theorem 2, for any k = 1, 2, . . . ,: d

∞ {kX(n, k)}∞ n=1 = {X1 + X2 + · · · + Xn }n=1 .

A very important property of records for discrete sequences was discovered by Shorrock [44]. Without loss of generality, we consider nonnegative integer values X’s and suppose that P{X = n} > 0 for any n = 0, 1, 2, . . ..

6997

Theorem 3. Let X, X1 , X2 , . . . be a sequence of independent random variables taking values 0, 1, 2, . . ., and let indicators η0 , η1 , . . . be defined as follows: ηn = 1 if n is a record value in the sequence X1 , X2 , . . ., that is, X(m) = n for some m, and ηn = 0 otherwise. Then, the random variables η0 , η1 , . . . are independent and P{ηn = 1} = 1 − P{ηn = 0} = P{X = n}/P{X  n}, n = 0, 1, 2, . . .. This theorem allows us to obtain the distributions of record values X(n) in the discrete case. In fact, it is easy to see that under the conditions of Theorem 3, P{X(n) > m} = P{η0 + η1 + · · · + ηm < n}, n  1,

m  0.

Nevzorov [28] generalized Theorem 3 for the case of kth records. He introduced indicators ηn(k) in the following way: ηn(k) = 1 if n is a kth record value in the sequence X1 , X2 , . . . and ηn(k) = 0 otherwise. It turns out that under the conditions of Theorem 3, for any k = 1, 2, . . . ,, the indicators η0(k) , η1(k) , . . . are independent and P{ηn(k) = 1} = (P{X = n}/P{X  n})k , n = 0, 1, 2, . . . . It follows from this equality that for any k = 1, 2, . . . , (k) < n}, P{X(n, k) > m} = P{η0(k) + · · · + ηm

m  0,

n  1.

DISTRIBUTIONS OF RECORD TIMES AND NUMBERS OF RECORDS FOR CONTINUOUS DISTRIBUTIONS We list below some of the most important results connected with distributions of record times in the case when a parent distribution function F is continuous. The joint and marginal distributions of record times are given as follows: (a) if 1 < 2 < · · · < n , then P{L(2) = 2 , . . . , L(n) = n } = 1/n (n − 1)(n−1 − 1) · · · (2 − 1);

6998

RECORD VALUES AND RECORD STATISTICS

(b) for any n  2 and   n,  P{L(n) = } = 1 k|L(n − 1) = m} = m/k,

Var(L(n)) = n − π 2 /6 + n/2(n+1) + O(2−n ),

k  m.

r = −1, −2, . . . ,

RECORD VALUES AND RECORD STATISTICS

and, in particular, E(1/(L(n) + 1)) = 2−n , E(1/(L(n) + 1)(L(n)+2)) = 3−n /2, n = 1,2, . . .. LIMIT THEOREMS FOR RECORD TIMES Most of the classical limit theorems for records were obtained by R´enyi [42]. It has 1/2 and been shown that √ (N(n) − log n)/(log n) (log L(n) − n)/ n both asymptotically have a standard normal distribution. The laws of large numbers and the law of the iterated logarithm were also obtained for the random variables N(n) and log L(n).

6999

The central limit theorem, the laws of large numbers, and the law of the iterative logarithm were proved for interrecord times as well. The joint asymptotic distributions of two successive interrecord times were obtained by Tata [50] as √ lim P{log (n) − n > x n, log (n − 1) √ − n < y n} = max{0, φ(y) − φ(x)}, as n → ∞, where φ(x) is a distribution function of the standard normal law.

INTERRECORD TIMES We will present some useful facts for interrecord times (n) = L(n) − L(n − 1), n = 1, 2, . . . . Distributions of interrecord times for sequences of independent random variables X1 , X2 , . . ., having a common continuous distribution function F, do not depend on F. These distributions are as follows [25]: 

∞

P{(n) = r} =

(1 − e−x )r−1

0

r = 1, 2, . . . ;



∞

P{(n) > r} = 

xn−2 e−2x dx, (n − 2)!

0

xn−2 e−x (1 − e−x )r dx (n − 2)!

(− log(1 − ν)) dν (n − 2)! 0 r    r (−1)m (1 + m)1−n , = m =

1

νr

n−2

m=0

r = 0, 1, . . . , n = 2, 3, . . . , from which it follows immediately that E(n) = ∞ for any n = 2, 3, . . .. The logarithmic moments of interrecord times were given by Nevzorov and Stepanov [35] as E log (n) = n − 1 − C + (n − 1 + δ1 )/2

n+1

+ (1/2)(n − 1 + δ2 )/3n ,

DISTRIBUTIONS OF RECORD VALUES IN SEQUENCES OF CONTINUOUS RANDOM VARIABLES Distribution functions for record values in the case, when i.i.d. X’s have any continuous distribution function F, are given as follows: P{X(n) < x} =

1 (n − 1)! ×ν



n−1 −ν

e

− log(1−F(x)) 0

dν.

It is of interest to compare the latter equality with P{X(n) < x} = E((F(x))L(n) ). If X’s have a common density function f , then one can obtain the joint density function fn (x1 , x2 , . . . , xn ) = r(x1 )r(x2 ) · · · r(xn ) × (1 − F(xn )), for x1 < x2 < · · · < xn , where r(x) = f (x)/(1 − F(x)), and the density function of X(n) takes on the form fn (x) = {− log(1 − F(x)}n−1 f (x)/(n − 1)!.

n = 2, 3, . . . , where C is the Euler’s constant, 0
θ1
2/3 and 0
θ2
6.

The following relation for conditional distributions, which implies the Markov structure of record values are valid not only for

7000

RECORD VALUES AND RECORD STATISTICS

continuous distribution functions F. P{X(n + 1)  x|X(1), X(2), . . . , X(n) = v} = P{X(n + 1)  x|X(n) = v} = P{X  x}/P{X > v},

x > v,

The only restriction to prove this result is the existence with probability one of any record value X(n). For continuous distributions, the following relationship between order statistics and record values is of interest (see refs. 14 and 23): for any n > 1 and m > 1,

STRONG AND WEAK RECORDS FOR DISCRETE DISTRIBUTIONS Without loss of generality, we confine ourselves now to i.i.d. discrete random variables X, X1 , X2 , . . . taking values 0, 1, 2, . . . such that P{X < n} < 1 for any integer n. The simplest way to investigate record values X(n) in this situation is to utilize record indicators. The joint distribution of record values is given as follows: P{X(1) = j1 , X(2) = j2 , . . . , X(n) = jn } n−1

= P{X = jn }

(P{X = jr }/P{X > jr }), r=1

P{X(n) > y|X(n − 1) = x)

0
j1 < · · · < jn .

= P{Xm,m > y|Xm−1,m = x} a.s.

ASYMPTOTIC DISTRIBUTIONS OF RECORD VALUES It is of interest to compare relationships between the asymptotic behavior of maximal order statistics M(n) = max{X1 , X2 , . . . , Xn } and record values X(n) = M(L(n)). There are three types of nondegenerate asymptotic distributions for centered and normalized maxima M(n). The corresponding limiting distribution functions may be written as exp{− exp(−g(x))}, where g(x) is taken from the following set of functions: (i) g(x) = x, −∞ < x < ∞; (ii) g(x) = α log x, α > 0, x > 0, and g(x) = −∞, x < 0; (iii) g(x) = ∞, x > 0 and g(x) = −α log(−x), x < 0, α > 0. Tata [50] and Resnick [43] solved the problem of describing the set of all possible asymptotic distributions for record values X(n) under suitable normalization. It turns out that these limiting distribution functions have the form (g(x)), where g(x) is any of the functions given above and (x) is the distribution function of the standard normal law.

It follows from the latter formula that the sequence X(1), X(2), . . . forms a Markov chain and P{X(n + 1) = j|X(n) = i} = P{X = j}/P{X > i}, j > i  n − 1. For geometric distributions, results for record values can be simplified considerably. If P{X = n} = (1 − p)pn−1 , 0 < p < 1, n = 1, 2, . . ., then the random variables X(1), X(2) − X(1), X(3) − X(2), . . . are independent and X(n) has the same distribution as the sum X1 + · · · + Xn . In the discrete case, we can consider weak record models in which a repetition of a record value can be counted as a new record (see, Refs. 49 and 51). Definitions of weak record times Lw (n) and weak record values X w (n) are given as follows: Lw (1) = 1,

Lw (n + 1) = min{j > Lw (n) : Xj  max(X1 , X2 , . . . , Xj−1 )},

X (n) = XLw (n) , w

n  1.

Note that all weak records, unlike classical (‘‘strong’’) records, exist with probability one. The joint distribution of weak records is given

RECORD VALUES AND RECORD STATISTICS

by the equality P{X w (1) = j1 , X w (2) = j2 , . . . , X w (n) = jn } n−1

= P{X = jn }

(P{X = jr }/P{X  jr }), r=1

0
j1
· · ·
jh . The sequence X w (1), X w (2), . . . forms a Markov chain with probabilities P{X w (n + 1) = j|X w (n) = i} = P{X = j}/P{X  i},

Theorem 4. Let X1 , X2 , . . . , Xn be independent random variables with a common continuous distribution function. Then, for any n, indicators of records ξ1 , ξ2 , . . . , ξn , which were introduced in Theorem 1, do not depend on M(n) = max{X1 , X2 , . . . , Xn }. The joint distribution of record times and record values has been known from R´enyi [42]. Let the i.i.d. random variables X1 , X2 , . . . have a common continuous distribution function F. Then, for any 1 < k(2) < · · · < k(n), P{L(1) = 1, L(2) = k(2), . . . , L(n) = k(n),

j  i.

If X has a geometric distribution with probabilities P{X = n} = (1 − p)pn , n = 0, 1, 2, . . . , then the random variables X w (1), X w (2) − X w (1), X w (3) − X w (2), . . . are independent and have the same geometric distribution as X. It is convenient to study weak records by introducing new random variables µ0 , µ1 , µ2 , . . . , where µn is the number of records in the sequence X w (1), X w (2), . . . taking value n, that is, µn coincides with a number of repetitions of a record value n. It turns out that µ0 , µ1 , µ2 , . . . are independent and P{µ0 = m} = (1 − pn )pm n, n= 1, 2, . . . , m = 0, 1, 2, . . ., where pn = P{X = n}/P{X  n}. This result yields the following useful representations: for any n = 1, 2, . . . and m = 0, 1, . . ., P{X w (n) > m} = P{µ0 + µ1 + · · · + µm < n} and P{X w (n)} = P{µ0 + µ1 + · · · + µm−1 < n, µ0 + µ1 + · · · + µm  n}.

X(1) < x1 , X(2) < x2 , . . . , X(n) < xn }   = · · · uk(2)−1 uk(3)−k(2)−1 ... 1 2 du1 du2 . . . dun , × uk(n)−k(n−1)−1 n−1 where the integrals are taken over the set A of all points (u1 , u2 , . . . , un ), such that −∞ < u1 < · · · < un , and F(uj ) < xj , j = 1, . . . , n}. It is more convenient to consider the joint behavior of record values X(n) and interrecord times (n). It turns out that the sequence of two-dimensional vectors (X(n), (n)), n = 1, 2, . . . , is a Markov chain with probabilities P{X(n) > x, (n) = m|(1), X(1), . . . , (n − 1), X(n − 1) = y} = (F(y))m−1 (1 − F(x)),

It was mentioned above that P{X(n) < x} = E((F(x))L(n) ). The following useful theorem (see ref. 10) allows us to obtain this result and gives some other relationships between record values and the corresponding record times.

x > y.

It was also proved that the interrecord times (n), n = 1, 2, . . ., are conditionally independent under fixed values of random variables X(1), X(2), . . ., and P{(n) = m|X(1), X(2), . . .} = {1 − F(X(n))} × {F(X(n))}m−1 ,

JOINT DISTRIBUTIONS OF RECORD TIMES AND RECORD VALUES

7001

m = 1, 2, . . . .

There are some results on the rates of closeness of random variables (n), L(n) and τn = − log(1 − F(X(n))) . Shorrock [45] proved that almost surely as n → ∞, lim sup |(log (n + 1) − τn )|/ log n = 1 and lim sup |(log L(n) − τn )|/ log n = 1.

7002

RECORD VALUES AND RECORD STATISTICS

Mention should be made to the following estimate, given in reference 32. Theorem 5. Let H(x) = exp{− exp(−x)}. Then, for any q, 0 < q < 1, the following inequality is valid. |P{τn − log L(n) < x} − H(x)|
rn (x),

It was also proved that for any k = 1, 2, . . . , the random variables L(1, k), L(2, k), . . . form a Markov chain and P{L(n, k) > j|L(n − 1, k) = i} = (j − k)!i!/(i − k)!j!. The following expression for generating functions was given in reference 29: For any k = 1, 2, . . . , EsL(n,k) = H(− log(1 − s)),

where rn (x) = H(x)(e−2x 22−n + 2e−4x 31−n /(1 − q)), if x > −(log(3nq/2))/2, and rn (x) = exp{−(3qn/2)1/2 }(1 + 3qn21−n +q n 3

2 2 3−n

/2(1 − q)), otherwise.

DISTRIBUTIONS AND MOMENTS OF K TH RECORD TIMES AND K TH INTER-RECORD TIMES The theory of kth records, which present a generalization of the classical (k=1) records, relate to the theory of classical records in much the same way as the theory of the kth extremes to the theory of maxima. Distributions of the kth record times L(n, k) and the kth inter-record times (n, k) = L(n, k) − L(n − 1, k) are given as follows:

n = 1, 2, . . . ,

 x n−1 −ku kn where H(x) = (n−1)! e (1 − e−(x−u) )k−1 0 u du. Listed below are some useful expressions for the moments of L(n, k) and (n, k): (i) for any k = 2, 3, . . . and n = 1, 2, . . . , EL(n, k) = kn /(k − 1)n−1 ; (ii) E(n, k) =

kn−1 , (k−1)n−1

k  2;

(iii) for any k = 1, 2, . . . and α > 0, as n → ∞,  n k (k − 1)! E(L(n, k))k−α = (α) α  n k (α − k)(α − k + 1) + 2α α+1 n   k . +O α+2

k TH RECORD VALUES

(i) for any k = 1, 2, . . . and k < l2 < · · · < ln ,

It is of interest to note that distributions of the kth record values X(n, k) can be expressed via distributions of the classical record val= k!(ln − k)!kn−1 /ln !(l2 − k) ues. Let the independent random variables X1 , X2 , . . . have a continuous distribution . . . (ln − k); function F, and Y1 = min{X1 , . . . , Xk }, Y2 = min{Xk+1 , . . . , X2k }, . . .. Further, let X(n, k) be (ii)  ∞ 1 the kth record value based on X’s and Y(n) be n−1 n−2 P{(n, k) = m} = k x (n − 2)! the classical record values based on Y’s. Then 0 ∞ for any k = 1, 2, . . ., {X(n, k)}∞ n=1 = {Y(n)}n=1 × e−x(k+1) (1 − e−x )m−1 dx, k  1, n  2; in distribution. Note that this relation is valid for discrete distributions as well. and The latter result, together with theorems (iii) for the classical record values, enables us to  ∞ 1 state several results for the kth record values. kn−1 xn−2 e−xk P{(n, k) > r} = (n − 2)! 0 For example, it can be immediately obtained (see ref. 19) that × (1 − e−x )r dx P{L(1, k) = k, L(2, k) = l2 , . . . , L(n, k) = ln }

=

r 

(−1)m

m=0

 + m ,1−n m 1+ , r k r  0, n  2;

P{X(n, k) < x} = P{X(n, k) < x}  −k log(1−F(x)) 1 un−1 e−u du. = (n − 1)! 0

RECORD VALUES AND RECORD STATISTICS

Note also that X(1, k), X(2, k), . . . forms a Markov chain and P{X(n + 1, k) > x|X(n, k) = y} = {(1 − F(x))/(1 − F(y))}k ,

x > y.

proved that for any k = 1, 2, . . . the random variables µ0 (k), µ1 (k), . . . are independent and have negative binomial distributions with probabilities  P{µi (k) = m} =

It implies that for any k = 1, 2, . . ., n = 2, 3, . . . and m = k + 1, k + 2, . . ., almost surely, P{X(n, k) > x|X(n − 1, k) = y} = P{Xm−k+1,m > x|Xm−k,m = y},

x > y.

Note also that if the initial distribution function F is continuous, then the set of possible limit distribution functions for suitably normalized random variables X(n, k) is the same as that for X(n). Coming back to i.i.d. random variables X, X1 , X2 , . . . taking values 0, 1, 2, . . . such that P{X < n} < 1 for any n, we can express the distribution functions of random variables X(n, k) via distributions of independent indicators as follows:

7003

 k−1 (1 − qi )k qm i , k+m−1 i = 0, 1, . . . , m = 0, 1, . . . ,

where qi = P{X = i}/P{X  i}. Then, the distributions of weak kth records can be obtained from the following relation: P{X w (n, k) > m} = P{µ0 (k) + µ1 (k) + · · · + µm (k) < n}.

KN -RECORDS A further generalization of the kth records, the so-called kn -records, were introduced by Nevzerov [28]. The reader is referred to reference 16 for definitions and properties of kn -records.

(k)  n}, P{X(n, k)
m} = P{η0(k) + · · · + ηm

RECORDS BASED ON DEPENDENT RANDOM VARIABLES

where P{ηn(k) = 1} = 1 − P{ηn(k) = 0} = (P{X = n}/P{X  n})k ,

n = 0, 1, 2, . . . .

In reference 49 weak kth record times Lw (n, k) and weak kth record values X w (n, k) were introduced in the following manner: Lw (1, k) = k,

Lw (n + 1, k)

= min{j > Lw (n, k) : Xj  Xj−k,j−1 }, X w (n, k) = XLw (n,k)−k+1,Lw (n,k) ,

n  1.

Let us consider again a sequence of i.i.d. random variables X, X1 , X2 , . . . taking values 0, 1, 2 . . ., with P{X < n} < 1 for any n. Define the random variables µi (k), i = 0, 1, . . ., as follows: µi (k) = m, m = 0, 1, . . . if in the sequence X1 , X2 , . . . we have exactly m, weak kth records taking a value i. Stepanov [49]

There are some record models in which the initial random variables are supposed to be dependent. It turns out that the statement of Theorem 1 remains valid for sequences of exchangeable random variables. It implies that all results for record times in the i.i.d. sequences, which were based on the independence of record indicators, remain true if we consider sequences of exchangeable variables. Then, de Finettis representation of exchangeable random variables as a mixture of i.i.d. random variables enable us to investigate record values in this situation. Haiman [24] investigated properties of record times and record values in the stationary m-dependent sequences, while Andel [3] proved some results for autoregressive processes with an exponentially distributed white noise. Records in Markov sequences were the subject of investigations of Biondini and Siddiqui [11,46] and Adke [1].

7004

RECORD VALUES AND RECORD STATISTICS

NONSTATIONARY RECORD MODELS We will briefly mention below some of the most interesting nonstationary record schemes. Pfeifer [38,39] considered records in scheme of series of independent random variables Xn1 , Xn2 , . . ., n = 0, 1, 2, . . ., which can have in different series different (but common for all random variables of the given series) distribution functions. This model reflects situations when conditions of experiment change after the occurrence of a new record value. Balabekyan--Nevzorov’s [6] record model can be described as follows: distribution functions of initial random variables form a sequence F1 , , F2 . . . such that Fnm+k = Fk , 1
k
m, n = 1, 2, . . ., that is, we have repeated groups of m different distribution functions. This model corresponds, for example, to the situation when m athletes have in succession n starts each. It turns out that the number of records N(nm) among the random variables X1 , . . . , Xnm has the same asymptotic distribution as in the classical scheme. Sometimes, a statistician deals with a set of data covering a long time period. For example, record results for mile races are available beginning from 1860. Of course, it is impossible to apply the classical record model to analyze such data, while models with different trends can be suitable for these situations. Numerous results for record models with trends are given in references 8, 9, 13, 21, 22, 47, and 48. Yang [53] supposed that the breaking of sport records must correspond in some degree to the exponential increase of the population of the world. In his record scheme, he considered sequences of random variables having distribution functions Fn = F m(n) , n = 1, 2, . . ., where F is some continuous distribution functions and coefficients m(1), m(2), . . . form a geometric proportion. It has been found in References 26 and 27 that indicators of records ξ1 , ξ2 , . . . remain independent in Yang’s model. Moreover, these indicators are independent for any sequence of positive coefficients m(1), m(2), . . .. This property has led to extensive studies of the new model (see refs. 15, 17, 30, 31, 32, 40, and 41) . A detailed study of this model, the so-called F α -scheme, is present in the monograph of Nevzerov [33].

RECORDS IN ARCHIMEDEAN COPULA PROCESSES Currently, the most general record model which deals with dependent nonstationary random variables is connected with the socalled Archimedean copula processes. This model includes as partial cases the classical record model and the F α -schemes. Distributions, limit theorems, properties of record times, and record values in Archimedean copula processes can be found in references 7, 36, and 37. Interestingly, it turns out that the set of all possible limit distributions for suitably normalizing record values in Archimedean sequences coincides with the corresponding set of limit distributions for the classical record values. Acknowledgments The author’s work was partially supported by RFBR grants 01-01-00031 and 00-15-96019.

REFERENCES 1. Adke, S. R. (1993). Records generated by Markov sequences. Stat. Prob. Lett., 18, 257–263. 2. Ahsanullah, M. and Nevzorov V. B. (2001). Ordered Random Variables. Nova Science Publishers. 3. Andel, J. (1990). Records in an AR(1) process. Ricerche Mat., 39, 327–332. 4. Arnold, B. C., Nagaraja, H. N. New York.

Balakrishnan, N. (1997). Records.

and Wiley,

5. Bairamov, I. G. (2000). On the characteristic properties of exponential distribution. Ann. Inst. Stat. Math., 52, 448–458. 6. Balabekyan, V. A. and Nevzorov, V. B. (1986). ‘‘On the Number of Records in a Sequence of Series of Non-identically Distributed Random Variables’’. in Rings and Modules -- Limit Theorems of Probability Theory 1, Z. I. Borevich and V. V. Petrov eds., Leningrad State University, Leningrad, pp. 147–153. 7. Ballerini, R. (1994). A dependent F α -scheme. Stat. Probab. Lett, 21, 21–25. 8. Ballerini, R. and Resnick, S. (1985). Records from improving populations. J. Appl. Probab., 22, 487–502.

RECORD VALUES AND RECORD STATISTICS 9. Ballerini, R. and Resnick, S. (1987a). Records in the presence of a linear trend. Adv. Appl. Probab., 19, 801–828. 10. Ballerini, R. and Resnick, S. (1987b). Embedding sequences of successive maxima in extremal processes, with applications. J. Appl. Probab., 24, 827–837. 11. Biondini, R. and Siddiqui, M. M. (1975). Record Values in Markov Sequences. In Statistical Inference and Related Topics--2, M. L. Puri, ed., Academic Press, New York, pp. 291–352. 12. Chandler, K. N. (1952). The distribution and frequency of record values. J. R. Stat. Soc., Ser. B, 14, 220–228. 13. de Haan, L. and Verkade, E. (1987). On extreme-value theory in the presence of a trend. J. Appl. Probab., 24, 62–76. 14. Deheuvels, P. (1984). The characterization of distributions by order statistics and record values--a unified approach. J. Appl. Probab., 21, 326–334. Correction, 22, 997. 15. Deheuvels, P. and Nevzorov, V. B. (1993). Records in F α -scheme. I. Martingale properties. Zap. Naucn. Semin. Leningr. Otdel. Mat. Inst. Steklova, 207, 19–36 (in Russian). 16. Deheuvels, P. and Nevzorov, V. B. (1994a). Limit laws for K-record times. J. Stat. Plann. Infer., 38, 279–308. 17. Deheuvels, P. and Nevzorov, V. B. (1994b). Records in F α -scheme. II. Limit theorems. Zap. Naucn. Semin. Leningr. Otdel. Mat. Inst. Steklova, 216, 42–51 (in Russian). 18. Dwass, M. (1960) Some k-Sample Rank-order Tests. in Contributions to Probability and Statistics---Essays in honor H. Hotelling. Stanford University Press, pp. 198–202. 19. Dziubdziela, W. (1977). Rozklady graniczne ekstremalnych statystyk pozycyjnych (Limit distributions of the extreme order statistics). Rocz. Polsk. Tow. Mat, Ser. 3, 9, 45–71 (in Polish). 20. Dziubdziela, W. and Kopocinsky, B. (1976). Limiting properties of the k-th record values. Zastos. Mat., 15, 187–190. 21. Foster, F. G. and Stuart, A. (1954). Distribution free tests in time-series band on the breaking of records. J. R. Stat. Soc., B16, 1–22. 22. Foster, F. G. and Teichroew, D. (1955). A sampling experiment on the powers of the records tests for trend in a time series. J. R. Stat. Soc., B17, 115–121. 23. Gupta, R. C. (1984). Relationships between order statistics and record values and some

7005

characterization results. J. Appl. Probab., 21, 425–430. 24. Haiman, G. (1987). Almost Sure Asymptotic Behavior of the Record and Recored Time Sequences of a Stationary Gaussian Process. in Mathematical Statistics and Probability Theory, Vol. A, M. L. Puri, P. Revesz, and W. Wertz eds., D. Reidel, Dordrecht, The Netherlands, pp. 105–120. 25. Neuts M. F. (1967). Waiting times between record observations. J. Appl. Probab., 4, 206–208. 26. Nevzorov, V. B. (1981). Limit Theorems for Order Statistics and Record Values. in Abstracts of Third International Vilnius Conference on Probability Theory and Mathematical Statistics. Vol. 2, Vilnius (in Russian), pp. 86–87. 27. Nevzorov, V. B. (1984). Record times in the case of nonidentically distributed random variables. Teoriya veroyatnostey i ee Primenenija, 29, 808–809 (in Russian). Translated version in Theory of Probability and its Applications. 28. Nevzorov, V. B. (1986). K-th record times and their generalizations. Zap. Nauchn. Semin. LOMI, 153, 115–121 (in Russian). Translated version in J. Soviet Math., 44, (1989), 510–515. 29. Nevzorov, V. B. (1990a). Generating functions for k-th record values -- a martingale approach. Zap. Nauchn. Semin. LOMI, 184, 208–214 (in Russian). Translated version in J. Math. Sci., 68, (1994), 545–550. 30. Nevzorov, V. B. (1990b). Records for Nonidentically Distributed Random Variables. in Proceedings of the Fifth Vilnius Conference on Probability and Statistics, Vol. 2, VSP Publishers, Mokslas, pp. 227–233. 31. Nevzorov, V. B. (1993). Characterizations of Certain Nonstationary Sequences by Properties of Maxima and Records. in Rings and Modules. Limit Theorems of Probability Theory, Z. I. Borevich and V. V. Petrov), eds., Vol. 3, St.-Petersburg State University, St.-Petersburg (in Russian), pp. 188– 197. 32. Nevzorov, V. B. (1995). Asymptotic distributions of records in nonstationary schemes. J. Stat. Plann. and Infer., 44, 261–273. 33. Nevzorov, V. B. (2000). Records: Mathematical Theory. American Mathematical Society, Providence. 34. Nevzorov, V. B. and Balakrishnan N. (1998). Record of Records. in Handbook of

7006

35.

36.

37.

38.

39.

40.

41.

42.

43. 44. 45.

46.

RECOVERY OF INTERBLOCK INFORMATION

Statistics-16, N. Balakrishnan and C. R. Rao), eds. Elsevier, Amsterdam. Nevzorov, V. B. and Stepanov, A. V. (1988). Records: Martingale Approach to Finding of Moments. in Rings and Modules. Limit Theorems of Probability Theory, Z. I. Borevich and V. V. Petrov), eds., Vol. 2, St.-Petersburg State University, St.-Petersburg (in Russian). pp. 171–181. Nevzorova, L. N., Nevzorov, V. B. and Balakrishnan, N. (1997). Characterizations of Distributions by Extremes and Records in Archimedean Copula Process. in Advances in the Theory and Practice of Statistics: A Volume in Honor of Samuel Kotz, N. L. Johnson and N. Balakrishnan), eds., Wiley, New York, pp. 469–478. Nevzorova, L. N., Nevzorov, V. B. and Balakrishnan, N. (2001). in Asymptotic Methods in Probability and Statistics, with Applications, N. Balakrishnan, I. A. Ibragimov, and V. B. Nevzorov), eds., Birkhauser, Boston, Mass. Pfeifer, D. (1982). Characterizations of exponential distributions by independent nonstationary record increments. J. Appl. Probab., 19, 127–135. Correction, 19, 906. Pfeifer, D. (1984c). Limit laws for inter-record times from non-homogeneous record values. J. Organ. Behav. Stat., 1, 69–74. Pfeifer, D. (1989). Extremal processes, secretary problems and the 1/e law. J. Appl. Probab., 26, 722–733. Pfeifer, D. (1991). Some remarks on Nevzorov’s record model. Adv. Appl. Probab., 23, 823–834. R´enyi, A. (1962). Theorie Des Elements Saillants D’une Suite D’observations. in Colloquium on Combinatorial Methods in Probability Theory. (August 1-10, 1962), Mathematical Institute, Aarhus University, Aarhus, Denmark (in French), pp. 104–117. See also Selected Papers of Alfred Renyi. Vol. 3 (1976), Akademiai Kiado, Budapest, Hungary, pp. 50–65. Resnick, S. I. (1973a). Limit laws for record values. Stochastic Processes Appl., 1, 67–82. Shorrock, R. W. (1972a). On record values and record times. J. Appl. Probab., 9, 316–326. Shorrock, R. W. (1972b). A limit theorem for inter-record times. J. Appl. Probab., 9, 219–223. Siddiqui, M. M. and Biondini, R. W. (1975). The joint distribution of record values and inter-record times. Ann. Probab., 3, 1012–1013.

47. Smith, R. L. (1988). Forecasting records by maximum likelihood. J. Am. Stat. Assoc., 83, 331–338. 48. Smith, R. L. and Miller, J. E. (1986). A nonGaussian state space model and application to prediction of records. J. R. Stat. Soc., Ser. B, 48, 79–88. 49. Stepanov, A. V. (1992). Limit theorems for weak records. Theory of Probab. Appl., 37, 586–590. 50. Tata, M. N. (1969). On outstanding values in a sequence of random variables. Z. Wahr. Verw. Geb., 12, 9–20. 51. Vervaat, W. (1973). Limit theorems for records from discrete distributions. Stochastic Processes Appl., 1, 317–334. 52. Williams, D. (1973). On R´enyi’s record problem and Engel’s series. Bullet. Lond. Math. Soc., 5, 235–237. 53. Yang, M. C. K. (1975). On the distribution of the inter-record times in an increasing population. J Appl. Probab., 12, 148–154.

FURTHER READING Galambos, J. (1978, 1985). The Asymptotic Theory of Extreme Order Statistics. First edition. John Wiley & Sons, New York; Second edition, Krieger, Malabar, Florida. Kamps, U. (1995). A Concept of Generalized Order Statistics. Teubner, Stuttgart, Germany. See also ORDER STATISTICS.

VALERY NEVZOROV N. BALAKRISHNAN

RECOVERY OF INTERBLOCK INFORMATION Recovery of interblock information refers to an estimation technique, first suggested by Yates [13,14], for extracting treatment information from block totals in incomplete block designs∗ . In randomized complete block designs (RCBD), treatments and blocks are orthogonal and all information about treatment effects is contained in the treatment totals—which comprise the set of sufficient statistics∗ for the treatment effects. Incomplete block designs (IBDs), characterized by a subset of the treatment levels occurring in each block, result

RECOVERY OF INTERBLOCK INFORMATION

in partial confounding∗ between the treatment and block effects. Thus the block totals contain information about treatments called interblock information∗ , which is usually ignored. The treatment information usually obtained from an IBD is the intrablock information∗ , derived from treatment comparisons within blocks, and referred to as treatment effects adjusted for blocks. By recovering the interblock, in addition to the intrablock information, two independent sets of treatment effect estimates are available. Combined estimates of the treatment effects can then be constructed by weighting the inter- and intrablock estimates by the inverse of their estimated variances. In most cases these combined estimates are preferred over the interblock estimates as described below. BALANCED INCOMPLETE BLOCK DESIGNS The simplest setting for recovery of interblock information is with balanced incomplete block designs (BIBDs). Let t be the number of treatments, b the number of blocks, and k < t the block size. The design is specified by the numbers nij , which determine the number of times treatment i occurs in block j. This t × b matrix N = (nij ) is called the incidence matrix of the design. For IBDs, the nij are 0 or 1, which characterizes a binary design. An equireplicate design is characterized by the additional restriction that treatments are replicated an equal number of times in the entire design (i.e., ri = r). The number of times treatment pairs occur together in blocks (say λil ) is equal for BIBDs, where λil =

b 

where i = 1, . . . , t, j = 1, . . . , b, µ + τi are the treatment means, bj ∼ N(0, σb2 ) and eij ∼ N(0, σe2 ) and bj and eij are all independent. The E(yij ) = µ + τi and var(yij ) = σb2 + σe2 (see John [6, p. 233]). The usual treatment estimates for BIBDs are found by solving the normal equations obtained by differentiating t  b 

nij (yij − µ − τi − bj )2

i=1 j=1

with respect to µ, τi , and bj , setting to zero, and simplifying using the equalities t 

nij = k,

i=1

b 

τˆi =

kQi , λt

where Qi = yi· −

b 1 nij y·j , k j=1

i = 1, . . . , t,

and yi· = bj=1 nij yij and y·j = ti=1 nij yij· The variances of the intrablock estimates are given by V(τˆi ) =

k(t − 1) 2 σe . λt2

The interblock estimates are derived from the model equation for the block totals:

y·j = kµ +

t 

nij τi + fj ,

i=1


and fj ∼ where fj = kbj + ti=1 nij eij N(0, k2 σb2 + kσe2 ). Since fj is an error term with constant variance, ordinary least squares∗ produces estimates of the parameters which are called the interblock estimates of the treatment effects,

The model for IBDs is given by yij = µ + τi + bj + eij ,

nij nlj = λ,

j=1

which characterize BIBDs. This yields the intrablock estimates of the treatment effects

j=1

INTRABLOCK AND INTERBLOCK ESTIMATES

b 

nij = r,

j=1

nij nlj ·

However, BIBDs exist only for values of the parameters (b, k, t, r) satisfying b  t, bk = rt and λ = r(k − 1)/(t − 1) an integer.

7007


b τ˜i =

j=1

nij y·j − kry.. r−λ

for i = 1, . . . , t.

7008

RECOVERY OF INTERBLOCK INFORMATION

These estimates are independent of the intrablock estimates and have variances V(τ˜i ) =

k(t − 1) 2 (σ + kσb2 ). t(r − λ) e

The combined treatment effect estimates are obtained as a weighted combination τi∗ = (α1 τˆi + α2 τ˜i )/(α1 + α2 ), where α1 and α2 are the inverses of the variances or estimated variances of the intra- and interblock estimates, respectively. HISTORY Yates [13] first described the recovery of interblock information for three-dimensional lattice designs∗ and the next year [14] for balanced incomplete block designs (BIBDs). The potential gain in efficiency from using small incomplete blocks is compared with the loss due to the partial confounding of treatment effects with blocks. However, with the recovery of interblock information, there is no appreciable loss in efficiency, and the choice of smaller blocks can result in a gain in efficiency. Yates [15] extended the recovery principle to recovery of interrow and intercolumn information in lattice square designs with both row and column blocking factors. Graybill and Weeks [4] showed that the combined estimator presented by Yates is unbiased and based on a set of minimal sufficient statistics. Since the combined estimator τi∗ depends on the variance ratio σb2 /σe2 which usually must be estimated, τi∗ can have higher variance than τˆi , the intrablock estimator. Graybill and Weeks (1959) found conditions (BIBD with b − t > 10) for which the variance of τi∗ is uniformly less than that of τˆi . Shan [11] provides another criterion for any IBD which ensures that τi∗ is uniformly better than τˆi , and an example not meeting the criterion with τi∗ having higher variance. Nair [8], Rao [9,10], Shan [11], and Stein [12] extended the recovery idea to obtain improved combined estimators under various conditions. Brown and Cohen [3], Khatri and Shan [7], and more recently, Bhattacharya [2] have discussed a family of

improved combined estimators for any incomplete block design with more than three blocks. REFERENCES 1. Bement, T. R. and Milliken, G. A. (1977). J. Amer. Statist. Ass., 72, 157–159. (Describes recovery of interblock information where the blocks can be divided into independent groups, such that within groups the blocks are correlated.) 2. Bhattacharya, C. G. (1980). Ann. Statist., 8, 205–211. 3. Brown, L. D. and Cohen, A. (1974). Ann. Statist., 2, 963–976. 4. Graybill, F. A. and Weeks, D. L. (1959). Ann. Math. Statist., 30, 799–805. 5. Harville, D. A. (1975). J. Amer. Statist. Ass., 70, 200–206. (Clear presentation of the combined intra- and interblock analysis in the context of the general mixed linear model. Computational formulas are given for all incomplete block designs.) 6. John, P. W. M. (1971). Statistical Design and Analysis of Experiments. Macmillan, New York, pp. 233–245. (General text for advanced treatment of the analysis and construction of incomplete block designs.) 7. Khatri, C. G. and Shah, K. R. (1975). J. Amer. Statist. Ass., 70, 402–406. (Calculates exact variances of combined inter- and intrablock estimates for several procedures. Results are given for several incomplete block designs for a range of ratios σb2 /σe2 .) 8. Nair, K. R. (1944). Sankhya, ¯ 6, 383–390. 9. Rao, C. R. (1947). J. Amer. Statist. Ass., 42, 541–561. 10. Rao, C. R. (1956). Sankhya, ¯ 17, 105–114. 11. Shah, K. R. (1964). Ann. Math. Statist., 35, 1064–1078. 12. Stein, C. (1966). In Research Papers in Statistics, H. A. David, ed. Wiley, New York, pp. 351–366. 13. Yates, F. (1939). Ann. Eugen. (Lond.), 9, 136–156. 14. Yates, F. (1940). Ann. Eugen. (Lind.), 10, 317–325. 15. Yates, F. (1940). J. Agric. Sci., 30, 672– 687. See also BLOCKS, BALANCED INCOMPLETE; BLOCKS, RANDOMIZED COMPLETE; INCOMPLETE BLOCK DESIGNS; and POOLING.

JAMES L. ROSENBERGER