Testing Statistical Hypotheses of Equivalence and Noninferiority, Second Edition

TESTING STATISTICAL HYPOTHESES OF EQUIVALENCE AND NONINFERIORITY SECOND EDITION

STEFAN WELLEK

K10408_FM.indd 1

5/11/10 12:25:01 PM

MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software.

Chapman & Hall/CRC Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2010 by Taylor and Francis Group, LLC Chapman & Hall/CRC is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Printed in the United States of America on acid-free paper 10 9 8 7 6 5 4 3 2 1 International Standard Book Number: 978-1-4398-0818-4 (Hardback) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright. com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Wellek, Stefan. Testing statistical hypothesis of equivalence and noninferiority / Stefan Wellek. -- 2nd ed. p. cm. Includes bibliographical references and index. ISBN 978-1-4398-0818-4 (hardcover : alk. paper) 1. Statistical hypothesis testing. I. Title. QA277.W46 2010 519.5’6--dc22

2010017666

Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com

K10408_FM.indd 2

5/11/10 12:25:01 PM

To Brigitte

Contents

Preface 1 Introduction 1.1 Statistical meaning of the concepts of equivalence and noninferiority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Demonstration of equivalence as a basic problem of applied statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Major fields of application of equivalence tests . . . . . . . . 1.3.1 Comparative bioequivalence trials . . . . . . . . . . . 1.3.2 Clinical trials involving an active control . . . . . . . 1.3.3 Preliminary tests for checking assumptions underlying other methods of statistical inference . . . . . . . . . 1.4 Role of equivalence/noninferiority studies in current medical research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Formulation of hypotheses . . . . . . . . . . . . . . . . . . . 1.6 Choosing the main distributional parameter . . . . . . . . . 1.7 Numerical specification of the limits of equivalence . . . . .

xi 1 .

1

. . . .

2 5 5 7

.

8

. 8 . 10 . 13 . 15

2 General techniques for dealing with noninferiority problems 2.1 Standard solution in the case of location parameter families . 2.1.1 Paired observations . . . . . . . . . . . . . . . . . . . . 2.1.2 Two independent samples . . . . . . . . . . . . . . . . 2.1.3 Power and sample size calculation based on tests for noninferiority under location-shift models . . . . . . . 2.2 Methods of constructing exact optimal tests for settings beyond the location-shift model . . . . . . . . . . . . . . . . . 2.3 Large-sample solutions for problems inaccessible for exact constructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Objective Bayesian methods . . . . . . . . . . . . . . . . . . . 2.5 Improved nonrandomized tests for discrete distributions . . . 2.6 Relationship between tests for noninferiority and two-sided equivalence tests . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Halving alpha? . . . . . . . . . . . . . . . . . . . . . . . . . .

19 19 19 22 23 24 26 27 28 30 31

v

vi

CONTENTS

3 General approaches to the construction of tests for equivalence in the strict sense 3.1 The principle of confidence interval inclusion . . . . . . . . . 3.2 Bayesian tests for two-sided equivalence . . . . . . . . . . . . 3.3 The classical approach to deriving optimal parametric tests for equivalence hypotheses . . . . . . . . . . . . . . . . . . . . 3.4 Construction of asymptotic tests for equivalence . . . . . . . . 4 Equivalence tests for selected one-parameter problems 4.1 The one-sample problem with normally distributed observations of known variance . . . . . . . . . . . . . . . . . . . . . 4.2 Test for equivalence of a hazard rate to some given reference value with exponentially distributed survival times . . . . . . 4.3 Testing for equivalence of a single binomial proportion to a fixed reference success probability . . . . . . . . . . . . . . . . 4.4 Confidence-interval inclusion rules as asymptotically UMP tests for equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Noninferiority analogues of the tests derived in this chapter .

33 33 36 40 45 49 49 55 59 64 68

5 Equivalence tests for designs with paired observations 5.1 Sign test for equivalence . . . . . . . . . . . . . . . . . . . . 5.2 Equivalence tests for the McNemar setting . . . . . . . . . . 5.2.1 Large-sample solution . . . . . . . . . . . . . . . . . 5.2.2 Corrected finite-sample version of the large-sample test . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.3 Modifications for the noninferiority case . . . . . . . 5.3 Paired t-test for equivalence . . . . . . . . . . . . . . . . . . 5.4 Signed rank test for equivalence . . . . . . . . . . . . . . . . 5.5 A generalization of the signed rank test for equivalence for noncontinuous data . . . . . . . . . . . . . . . . . . . . . . .

71 . 71 . 76 . 78

6 Equivalence tests for two unrelated samples 6.1 Two-sample t-test for equivalence . . . . . . . . . . . . . . . 6.2 Mann-Whitney test for equivalence . . . . . . . . . . . . . . 6.3 Two-sample equivalence tests based on linear rank statistics 6.4 A distribution-free two-sample equivalence test allowing for arbitrary patterns of ties . . . . . . . . . . . . . . . . . . . . 6.5 Testing for dispersion equivalence of two Gaussian distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Equivalence tests for two binomial samples . . . . . . . . . . 6.6.1 Exact Fisher type test for noninferiority with respect to the odds ratio . . . . . . . . . . . . . . . . . . . . 6.6.2 Improved nonrandomized tests for noninferiority with respect to the odds ratio . . . . . . . . . . . . . . . .

119 . 119 . 126 . 136

. . . .

81 85 92 99

. 108

. 150 . 164 . 172 . 172 . 177

CONTENTS 6.6.3

6.7

Tests for noninferiority using alternative parametrizations . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.4 Exact test for two-sided equivalence with respect to the odds ratio . . . . . . . . . . . . . . . . . . . . . . 6.6.5 An improved nonrandomized version of the UMPU test for two-sided equivalence . . . . . . . . . . . . . 6.6.6 Tests for two-sided equivalence with respect to the difference of success probabilities . . . . . . . . . . . . Log-rank test for equivalence of two survivor functions . . . 6.7.1 Rationale of the log-rank test for equivalence in the two-sided sense . . . . . . . . . . . . . . . . . . . . . 6.7.2 Power approximation and sample size calculation for the log-rank test for equivalence . . . . . . . . . . . . 6.7.3 Log-rank test for noninferiority . . . . . . . . . . . .

vii

7 Multisample tests for equivalence 7.1 The intersection-union principle as a general solution to multisample equivalence problems . . . . . . . . . . . . . . . . . 7.2 F -test for equivalence of k normal distributions . . . . . . . 7.3 Modified studentized range test for equivalence . . . . . . . 7.4 Testing for dispersion equivalence of more than two Gaussian distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 A nonparametric k-sample test for equivalence . . . . . . . 8 Equivalence tests for multivariate data 8.1 Equivalence tests for several dependent samples from normal distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1.1 Generalizing the paired t-test for equivalence by means of the T 2 -statistic . . . . . . . . . . . . . . . . . . . . 8.1.2 A many-one equivalence test for dependent samples based on Euclidean distances . . . . . . . . . . . . . 8.1.3 Testing for equivalence with dependent samples and an indifference zone of rectangular shape . . . . . . . 8.1.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Multivariate two-sample tests for equivalence . . . . . . . . 8.2.1 A two-sample test for equivalence based on Hotelling’s T 2 . . . . . . . . . . . . . . . . . . . . . . 8.2.2 Behavior of the two-sample T 2 -test for equivalence under heteroskedasticity . . . . . . . . . . . . . . . . . 8.2.3 Multivariate two-sample tests for equivalence regions of rectangular shape . . . . . . . . . . . . . . . . . .

. 181 . 186 . 193 . 194 . 202 . 203 . 210 . 214 219 . 219 . 221 . 225 . 227 . 231 235 . 235 . 235 . 241 . 248 . 252 . 253 . 253 . 259 . 262

9 Tests for establishing goodness of fit 265 9.1 Testing for equivalence of a single multinomial distribution with a fully specified reference distribution . . . . . . . . . . . 265

viii

CONTENTS 9.2 9.3

9.4

Testing for approximate collapsibility of multiway contingency tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Establishing goodness of fit of linear models for normally distributed data . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.1 An exact optimal test for negligibility of interactions in a two-way ANOVA layout . . . . . . . . . . . . . . 9.3.2 Establishing negligibility of carryover effects in the analysis of two-period crossover trials . . . . . . . . . . Testing for approximate compatibility of a genotype distribution with the Hardy-Weinberg condition . . . . . . . . . . . . 9.4.1 Genetical background, measures of HW disequilibrium 9.4.2 Exact conditional tests for absence of substantial disequilibrium . . . . . . . . . . . . . . . . . . . . . . . . 9.4.3 Confidence-interval–based procedures for assessing HWE . . . . . . . . . . . . . . . . . . . . . . . . . . .

270 278 278 284 290 290 293 303

10 The assessment of bioequivalence 311 10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 10.2 Methods of testing for average bioequivalence . . . . . . . . . 315 10.2.1 Equivalence with respect to nonstandardized mean bioavailabilities . . . . . . . . . . . . . . . . . . . . . . . . 315 10.2.2 Testing for scaled average bioequivalence . . . . . . . . 322 10.3 Individual bioequivalence: Criteria and testing procedures . . 325 10.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 325 10.3.2 Distribution-free approach to testing for probabilitybased individual bioequivalence . . . . . . . . . . . . . 327 10.3.3 An improved parametric test for probability-based individual bioequivalence . . . . . . . . . . . . . . . . . . 329 10.4 Approaches to defining and establishing population bioequivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340 10.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . 340 10.4.2 A testing procedure for establishing one-sided bioequivalence with respect to total variability . . . . . . . . 342 10.4.3 Complete disaggregate testing procedure and illustrating example . . . . . . . . . . . . . . . . . . . . . . . . 344 10.4.4 Some results on power and sample sizes . . . . . . . . 345 10.4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 347 10.5 Bioequivalence assessment as a problem of comparing bivariate distributions . . . . . . . . . . . . . . . . . . . . . . . . . 348 11 Tests for relevant differences between treatments 355 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 11.2 Exploiting the duality between testing for two-sided equivalence and existence of relevant differences . . . . . . . . . . . 356

CONTENTS

ix

11.3 Solutions to some special problems of testing for relevant differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 One-sample problem with normally distributed data of known variance . . . . . . . . . . . . . . . . . . . . 11.3.2 Two-sample t-test for relevant differences . . . . . . . 11.3.3 Exact Fisher type test for relevant differences between two binomial distributions . . . . . . . . . . . . . . . . Appendix A Basic theoretical results A.1 UMP tests for equivalence problems in STP3 families . . . . A.2 UMPU equivalence tests in multiparameter exponential families . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 A sufficient condition for the asymptotic validity of tests for equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . .

359 359 361 364

369 . 369 . 375 . 376

Appendix B List of special computer programs

379

Appendix C Frequently used special symbols and abbreviations

383

References

387

Author index

403

Subject index

407

Preface

During the time since finalization of the manuscript of the first edition of this book, research in the field of equivalence testing methods expanded at an unexpectedly fast rate so that there seems to be a considerable need for updating its coverage. Furthermore, in clinical research, there developed an increasing preference for replacing trials following the classical placebo-controlled design with active-control trials requiring methods of testing for noninferiority rather than equivalence in the strict, i.e., two-sided sense. On the one hand, noninferiority problems are nothing else but generalized one-sided testing problems in the usual sense arising from a shift of the upper bound set under the null hypothesis to the parameter of interest away from zero or unity. Furthermore, from a technical point of view, the modifications required for transforming a test for two-sided equivalence into a test for noninferiority for the same setting are largely straightforward. On the other hand, it cannot be ignored that a book on the topic is likely to better serve the needs of readers mainly interested in applications when for each specific scenario the noninferiority version of the testing procedure is also described in full detail. Another extension of considerable interest for research workers in a multitude of empirical areas refers to testing for “relevant differences” between treatments or experimental conditions. Testing problems of this latter kind are dual to two-sided equivalence problems in that the assumption of nonexistence of differences of clinical or otherwise practical relevance plays the role of the null hypothesis to be as-

xi

xii

PREFACE

sessed. The new edition discusses solutions to such problems in an additional chapter. Roughly speaking, tests for equivalence in the strict sense provide the adequate answer to the most natural question of how to proceed in a traditional two-sided testing problem if it turns out that primary interest is in verifying rather than rejecting the null hypothesis. Put in more technical terms, equivalence assessment deals with a particular category of testing problems characterized by the fact that the alternative hypothesis specifies a sufficiently small neighborhood of the point in the space of the target parameter which indicates perfect coincidence of the distributions to be compared. The relevance of inferential procedures which, in the sense of this notion, enable one to “prove the null hypothesis” for many areas of applied statistical data analysis, is obvious enough. A particularly striking phenomenon which demonstrates the real need for such methods, is the adherence of generations of authors to using the term “goodness-of-fit tests” for methods which are actually tailored for solving the reverse problem of establishing absence or lack of fit. From a “historical” perspective (the first journal article on an equivalence test appeared as late as in the sixties of the twentieth century), the interest of statistical researchers in equivalence assessment was almost exclusively triggered by the introduction of special approval regulations for so-called generic drugs by the Food and Drug Administration (FDA) of the U.S. as well as the drug regulation authorities of many other industrial countries. Essentially, these regulations provide that the positive result of a test, which enables one to demonstrate with the data obtained from a so-called comparative bioavailability trial the equivalence of the new generic version of a drug to the primary manufacturer’s formulation, shall be accepted as a sufficient condition for approval of the generic formulation to the market. The overwhelming practical importance of the entailed problems of bioequivalence assessment (drugs whose equivalence with respect to the measured bioavailabilities can be taken for granted, are termed “bioequivalent” in clinical pharmacology literature), arises mainly out of quantity: Nowadays, at least half of the prescription drug units sold in the leading industrial countries are generic drugs that have been approved to be marketed on the basis of some bioequivalence trial. Considerations of one-sided equivalence (noninferiority) play an increasingly important role in the design and analysis of genuine clinical trials of therapeutic methods. The subjects recruited for such trials are patients suffering from some disease rather than healthy volunteers. Whenever well-established therapeutic strategies of proven efficacy and tolerability are already available for the disease under consideration, it would be unethical to launch a new trial involving a negative control (in particular, placebo). From the statistical perspective, using a positive or active control instead frequently implies that a classical procedure tailored for establishing superiority of the experimental treatment over the control condition has to be replaced with the corresponding test for noninferiority. Although noninferiority testing is given much more attention in the new

PREFACE

xiii

book as compared to the first edition, the core of this monograph still deals with methods of testing for equivalence in the strict, i.e., two-sided sense. The spectrum of specific equivalence testing problems of both types it covers range from the one-sample problem with normally distributed observations of fixed known variance (which will serve as the basis for the derivation of asymptotic equivalence tests for rather complex multiparameter and even semi- and nonparametric models), to problems involving several dependent or independent samples and multivariate data. A substantial part of the testing procedures presented here satisfy rather strong optimality criteria, which is to say that they maximize the power of detecting equivalence uniformly over a large class of valid tests for the same (or an asymptotically equivalent) problem. In equivalence testing, the availability of such optimal procedures seems still more important than in testing conventional one- or two-sided hypotheses. The reason is that even those equivalence tests which can be shown to be uniformly most powerful among all valid tests of the same hypotheses, turn out to require much higher sample sizes in order to maintain some given bounds on both types of error risks than do ordinary one- or two-sided tests for the same statistical models, unless one starts from an extremely liberal specification of the equivalence limits. The theoretical basis of the construction of optimal tests for interval hypotheses was laid within the mathematical statistics literature of the nineteen fifties. However, up to now the pertinent results have only rarely been exploited in the applied, in particular the biostatistical, literature on equivalence testing. In a mathematical appendix to this book, they will be presented in a coherent way and supplemented with a corollary which allows great simplification of the computation of the critical constants of optimal equivalence tests under suitable symmetry restrictions. An additional appendix contains a listing of all computer programs supplied at the URL http://www.crcpress.com/product/isbn/9781439808184 for facilitating as much as possible the routine application of all testing procedures discussed in the book. The collection of all program files contained in that directory is referenced as the WKTSHEQ2 Source Code Package throughout the text. In contrast to the Web material which accompanied the first edition, the majority of the programs have now been made available also as R scripts or shared objects which can be called within the R system. Most of the concrete numerical examples given in the text for purposes of illustrating the individual methods, are taken from the author’s own field of application, i.e., from medical research. The book can be used in several ways depending on the reader’s interests and level of statistical background. Chapter 1 gives a general introduction to the topic and should at least be skimmed by readers of any category. Chapters 2 and 3 deal with general approaches to problems of testing for noninferiority and two-sided equivalence and are mainly intended for readers with interests in a systematic account of equivalence testing procedures and their mathematical basis. Readers seeking information about specific procedures and their

xiv

PREFACE

practical implementation are advised to skip these chapters except for the sections on noninferiority testing in location-shift models (§ 2.1) and confidence interval inclusion rules (§ 3.1). Apart from occasional cross-references, all remaining chapters (and even individual sections of them) can be read independently of each other. The material they contain is to provide the working statistician of any background and level of sophistication with a sufficiently rich repertoire of efficient solutions to specific equivalence and noninferiority testing problems frequently encountered in the analysis of real data sets. Except for Section 4.4 which introduces no additional testing procedure, Chapters 4–10 should even be suited for serving as a procedure reference book in the field of equivalence and noninferiority testing. The last chapter summarizes both some basic theoretical results about tests for relevant differences (arising from switching the roles of both hypotheses in a two-sided equivalence problem) and describes solutions for some specific settings frequently arising in practice. In order to keep the page count within reasonable limits, the coverage in this book is confined to methods for samples of fixed size. Fully and group sequential methods for equivalence testing problems are left out of account. All in all, one of the following alternative guides to the book should be followed:

A) [for readers primarily interested in practical applications]

Ch. 1

§§ 2.1, 3.1

Ch. 4

Ch. 5

Ch. 6

Ch. 7

Ch. 8

App. B

App. C

Ch. 9

Ch. 10

Ch. 11

PREFACE

xv

B) [for the reader particularly interested in theory and mathematical background]

Ch.1 Ch.2

Ch. 3, App. A, § 11.2

Ch. 4

Ch. 5

Ch. 6

Ch. 7

Ch. 8

Ch. 9

Ch. 10

§ 11.3

App. B/C

I have many people to thank for helping me in the endeavor of preparing this book, without being able to mention here more than a few of them. Niels Keiding played an initiating role, not only by encouraging me to make the material contained in a book on the same topic I had published in 1994 in German accessible to an international readership, but also by bringing me in contact with Chapman & Hall/CRC. Cooperation with the staff of this publisher proved remarkably smooth and constructive, and I would like to acknowledge in particular the role of Rob Calver as the present statistics editor at Taylor & Francis during the whole phase until finalizing the manuscript of the new edition. The former vice-president of Gustav Fischer Verlag Stuttgart, Dr. Wolf D. von Lucius, is gratefully acknowledged for having given his permission to utilize in part the content of my book of 1994. As was already the case with the first edition, there are two people from the staff in my de-

xvi

PREFACE

partment at the Central Institute of Mental Health at Mannheim to whom I owe a debt of gratitude: Mireille Lukas spent her expert skills in handling the LATEX system on typesetting the new parts of the book and reorganizing the whole document. The editorial component of my job as the author was greatly facilitated by the fact that I could delegate to her a considerable part of the work entailed in compiling the bibliography and both indices by means of special TEX-based tools. Peter Ziegler took over the task of making available within R more than 30 computer programs originally written in Fortran or SAS. Moreover, he generated the better part of the figures contained in the book writing suitable source scripts in SAS/GRAPH. Last but not least, special thanks are due to the following two colleagues from external departments in statistics and related fields: Arnold Janssen (University of D¨ usseldorf) for agreeing to present results from an unpublished joint paper by him and myself, and Andreas Ziegler (University at L¨ ubeck) for the fruitful cooperation on the topic of Chapter 9.4 Mannheim January 2010

Stefan Wellek

Disclaimers 1. All computer programs included with this book are provided in good faith and after a reasonable amount of validation. However, the author, publishers and distributors do not guarantee their accuracy and take no responsibility for the consequences of their use. R 2. MATLAB is a registered trademark of The MathWorks, Inc. For product information, please contact:

The MathWorks, Inc. 3 Apple Hill Drive Natick, MA 01760-2098 USA Tel: 508 647 7000 Fax: 508-647-7001 E-mail: [email protected] Web: www.mathworks.com

1 Introduction

1.1

Statistical meaning of the concepts of equivalence and noninferiority

Although the notions of equivalence and noninferiority have nowadays become part of standard terminology of applied statistics, the precise meaning of these terms is not self-explanatory. The first of them is used in statistics to denote a weak or, more adequately speaking, fuzzy form of an identity relation referring to the distribution(s) which underly the data under analysis. The fuzziness of equivalence hypotheses as considered in this book is induced by enlarging the null hypothesis of the traditional two-sided testing problem referring to the same statistical setting, through adding an “indifference zone” around the corresponding region (or point) in the parameter space. In other words, equivalence means here equality except for practically irrelevant deviations. Such an indifference zone is a basic and necessary ingredient of any kind of testing problem to be addressed in the planning and confirmatory analysis of a study, trial or experiment run with the objective of demonstrating equivalence. Admittedly, finding a consensus on how to specify that indifference zone concretely is far from easy in the majority of applications. However, it is an indispensable step without which the testing problem the experimenter proposes would make no statistical sense at all. The reason behind this fact whose proper understanding is an elementary prerequisite for a sensible use of the methods discussed in this book, will be made precise in § 1.5. Recalling the way the word noninferiority is used in everyday language provides little if any insight into the real meaning of the second of the concepts to be introduced here. The term has been originally coined in the clinical trials literature in order to denote a study which aims at demonstrating that some new, experimental therapy falls short in efficacy by a clinically acceptable amount at most as compared to a well-established reference treatment for the same disease. Thus, noninferiority means absence of a relevant difference in favor of the comparator against which the experimental treatment has to be assessed. Formalizing a noninferiority problem by translating it into a pair of statistical hypotheses leads to a generalized one-sided testing problem. The only difference to a standard testing problem of the one-sided type is that the common boundary of the two hypotheses is now shifted to the left, away from

1

2

INTRODUCTION

the null which often, yet not always, coincides with the origin of the real line. At the same time, noninferiority problems exhibit a clear-cut relationship to equivalence problems: Obviously, every equivalence hypothesis can be modified into a hypothesis of noninferiority simply by letting the right-hand limit (which in some cases will be a curve rather than a single point) increase to infinity. This justifies considering noninferiority as a one-sided form of equivalence. In the earlier literature in the field, some authors (for notable examples, see Dunnett and Gent, 1977; Mehta et al., 1984) did little care to distinguish between equivalence and noninferiority. In order to avoid potential confusion resulting from such usage, we will adhere to the following terminological rule: When referring to specific problems and procedures, equivalence per se will always used in the strict, two-sided sense of the term. Noninferiority problems will be either called that way or, alternatively, addressed as one-sided equivalence problems.

1.2

Demonstration of equivalence as a basic problem of applied statistics

It is a basic fact well known to every statistician that in any hypotheses testing problem there is an inherent logical asymmetry concerning the roles played by the two statements (traditionally termed null and alternative hypotheses) between which a decision shall be taken on the basis of the data collected in a suitable trial or experiment: Any valid testing procedure guarantees that the risk of deciding erroneously in favor of the alternative does not exceed some prespecified bound whereas the risk of taking a wrong decision in favor of the null hypothesis can typically be as high as 1 minus the significance level (i.e., 95% in the majority of practical applications). On the other hand, from an experimental or clinical researcher’s perspective, there seems little reason why he should not be allowed to switch his views towards the problem under consideration and define what he had treated as the null hypothesis in a previous study, as the hypothesis of primary interest in a subsequent trial. If, as is so often the case in practice, the “traditional” formulation of the testing problem has been a two-sided one specifying equality of the effects of, say, two treatments under the null hypothesis, then such a switch of research interest leads to designing a study which aims at proving absence of a (relevant) difference between both treatment effects, i.e., equivalence. The term treatment is used here in the generic sense covering also experimental conditions etc. being compared in a fully non-medical context. Typically, the rigorous construction of a testing procedure for the confirmatory analysis of an equivalence trial requires rather heavy mathematical machinery. Nevertheless, the basic idea leading to a logically sound formulation of an equivalence

Demonstration of Equivalence as a Basic Statistical Problem

3

testing problem and the major steps making up an appropriate statistical decision procedure can be illustrated by an example as simple as the following.

Example 1.1 Of an antihypertensive drug which has been in successful use for many years, a new generic version has recently been approved to the market and started to be sold in the pharmacies at a price undercutting that of the reference formulation (R) by about 40%. A group of experts in hypertensiology doubt the clinical relevance of existing data showing the bioequivalence of the generic to the reference formulation, notwithstanding the fact that these data had been accepted by the drug regulation authorities as sufficient for approving the new formulation. Accordingly, the hypertensiologists agree to launch into a clinical trial aiming at establishing the therapeutic equivalence of the new formulation of the drug. Instead of recruiting a control group of patients to be treated with R, one decides to base the assessment of the therapeutic equivalence of the new formulation on comparison to a fixed responder rate of 60% obtained from long-term experience with formulation R. Out of n = 125 patients eventually recruited for the current trial, 56% showed a positive response in the sense of reaching a target diastolic blood pressure below 90 mmHg. Statistical assessment of this result was done by means of a conventional binomial test of the null hypothesis that the probability p, say, of obtaining a positive response in a patient given the generic formulation, equals the reference value p◦ = .60, versus the two-sided alternative p = p◦ . Since the significance probability (pvalue) computed in this way turned out to be as high as .41, the researchers came to the conclusion that the therapeutic equivalence of the generic to the reference formulation could be taken for granted, implying that the basic requirement for switching to the new formulation whenever confining the costs of treatment is an issue, was satisfied. Unfortunately, it follows from the logical asymmetry between null and alternative hypothesis mentioned at the beginning that such kind of reasoning misses the following point of fundamental importance: “Converting” a traditional two-sided test of significance by inferring equivalence of the treatments under comparison from a nonsignificant result of the former, generally fails to yield a valid testing procedure. In a word: A nonsignificant difference must not be confused with significant homogeneity, or, as Altman and Bland (1995) did put it, “absence of evidence is not evidence of absence.” Even in the extremely simple setting of the present example, i.e., of a one-arm trial conducted for the purpose of establishing (therapeutic) equivalence of a single treatment with regard to a binary success criterion, correct inference requires the application of a testing procedure exhibiting genuinely new features. (i) First of all, it is essential to notice that the problem of establishing the alternative hypothesis of exact equality of the responder rate p associated

4

INTRODUCTION with the generic formulation of the drug, to its reference value p◦ = .60 by means of a statistical test admits no sensible solution (the reader interested in the logical basis of this statement is referred to § 1.5). The natural way around this difficulty consists of introducing a region of values of p close enough to the target value p◦ for considering the deviations practically irrelevant. For the moment, let us specify this region as the interval (p◦ − .10 , p◦ + .10) = (.50 , .70) . Hence, by equivalence of p to p◦ we eventually mean a weakened form of identity specifying equality except for ignorable differences.

(ii) The closer the observed responder rate X/n comes up to the target rate p◦ = .60, the stronger the evidence in favor of equivalence provided by the available data. Thus, a reasonable test for equivalence of p to p◦ will use a decision rule of the following form: The null hypothesis of inequivalence is rejected if and only if the difference X/n − p◦ between the observed and the target responder rate falls between suitable critical bounds, say c1 and c2 , such that c1 is some negative and c2 some positive real number, respectively. (iii) Optimally, the rejection region of the desired test, i.e., the set of possible outcomes of the trial allowing a decision in favor of equivalence, should be defined in such a way that the associated requirement on the degree of closeness of the observed responder rate X/n to the reference rate p◦ is as weak as possible without increasing the risk of an erroneous equivalence decision over α, the prespecified level of significance (chosen to be .05 in the majority of practical applications). (iv) As follows from applying the results to be presented in § 4.3 with p◦ ∓ .10 as the theoretical range of equivalence and at level α = 5%, the optimal critical bounds to X/n−p◦ to be used in a test for equivalence of p to p◦ = .60 based on a random sample of size n = 125 are given by c1 = −2.4% and c2 = 3.2%, respectively. Despite the considerable size of the sample recruited to the trial, the rejection interval for X/n − p◦ corresponding to these values of c1 and c2 is pretty narrow, and the observed rate 56% of responders falls relatively far outside giving X/n − p◦ = −4.0%. Consequently, at significance level 5%, the data collected during the trial do not contain sufficient evidence in favor of equivalence of the generic to the reference formulation in the sense of |p − p◦ | < .10. The confirmatory analysis of experimental studies, clinical trials etc. which are performed in order to establish equivalence of treatments is only one out of many inferential tasks of principal importance which can adequately be dealt with only by means of methods allowing to establish the (suitably enlarged) null hypothesis of a conventional two-sided testing problem. Another category of problems for which exactly the same holds true, refers to the verification of statistical model assumptions of any kind. Notwithstanding the traditional

Major fields of application of equivalence tests

5

usage of the term “goodness-of-fit test” obscuring the fact that the testing procedures subsumed in the associated category are tailored for solving the reverse problem of establishing lack of fit, in the majority of cases it is much more important to positively demonstrate the compatibility of the model with observed data. Thus, if a goodness-of-fit test is actually to achieve what is implied by its name, then it has to be constructed as an equivalence test in the sense that a positive result supports the conclusion that the true distribution from which the data have been taken, except for minor discrepancies, coincides with the distribution specified by the model. The primary objective of this book is a systematic and fairly comprehensive account of testing procedures for problems such that the alternative hypothesis specifies a sufficiently small neighborhood of the point in the space of the target parameter (or functional) which indicates perfect coincidence of the probability distributions under comparison. As will become evident from the numerical material presented in the subsequent chapters, the sample sizes required in an equivalence test in order to achieve a reasonable power typically tend to be considerably larger than in an ordinary one- or two-sided testing procedure for the same setting unless the range of tolerable deviations of the distributions from each other is chosen so wide that even distributions exhibiting pronounced dissimilarities would be declared “equivalent”. This is the reason why in equivalence testing optimization of the procedures with respect to power is by no means an issue of purely academic interest but a necessary condition for keeping sample size requirements within the limits of practicality. The theory of hypotheses testing as developed in the fundamental work of E.L. Lehmann having appeared a few years ago in a third edition (Lehmann and Romano, 2005) provides in full mathematical generality methods for the construction of optimal procedures for four basic types of testing problems covering equivalence problems in the sense of the present monograph as well. Converting these general results into explicit decision rules suitable for routine applications will be a major objective in the chapters to follow.

1.3 1.3.1

Major fields of application of equivalence tests Comparative bioequivalence trials

It was not until the late nineteen sixties that statistical researchers started to direct some attention to methods of testing for equivalence of distributions in the sense made precise in the previous section. In this initial phase, work on equivalence assessment was almost exclusively triggered by the introduction of special approval regulations for so-called generic drugs by the Food and Drug Administration (FDA) of the U.S. as well as the drug regulation authorities of many other industrialized countries. Loosely speaking, a generic drug is an

6

INTRODUCTION

imitation of a specific drug product of some primary manufacturer that has already been approved to the market and prescribed for therapeutic purposes for many years but is no longer protected by patent. As a matter of course, with regard to the biologically active ingredients, every such generic drug is chemically identical to the original product. However, the actual biological effect of a drug depends on a multitude of additional factors referring to the whole process of the pharmaceutical preparation of the drug. Examples of these are – chemical properties and concentrations of excipients – kind of milling procedure – choice of tablet coatings – time and strength of compression applied during manufacture. For the approval of a generic drug, the regulatory authorities do not require evidence of therapeutic efficacy and tolerability based on comparative clinical trials. Instead, it is considered sufficient that in a trial on healthy volunteers comparing the generic to the original formulation of the drug, the hypothesis of absence of relevant differences in basic pharmacokinetic characteristics (called “measures of bioavailability”) can be established. If this is the case, then the generic drug is declared equivalent with respect to biovailability or, for short, bioequivalent to the original formulation. Assessment of bioequivalence between a new and a reference formulation of some drug is still by far the largest field of application for statistical tests of the type this book focusses upon, and can be expected to keep holding this position for many years to come. The overwhelming importance of the problem of bioequivalence assessment has to do much more with economic facts and public health policies than with truly scientific interest: During the last two decades, the market share of generic drugs has been rising in the major industrial countries to levels ranging between 43% (U.S., 1996) and 67.5% (Germany, 1993) ! From the statistical perspective, the field of bioequivalence assessment is comparatively narrow. In fact, under standard model assumptions [to be made explicit in Ch. 10], the confirmatory analysis of a prototypical bioequivalence study reduces to a comparison of two Gaussian distributions. In view of this, it is quite misleading that equivalence testing is still more or less identified with bioequivalence assessment by many (maybe even the majority) of statisticians. As will hopefully become clear enough from further reading of the present monograph, problems of equivalence assessment are encountered in virtually every context where the application of the methodology of testing statistical hypotheses makes any sense at all. Accordingly, it is almost harder to identify a field of application of statistics where equivalence problems play no or at most a minor role, than to give reasons why they merit particular attention in some specific field.

Major fields of application of equivalence tests

1.3.2

7

Clinical trials involving an active control

In medical research, clinical trials which involve an active (also called positive) control make up the second largest category of studies commonly analyzed by means of equivalence testing methods. An active rather than negative control (typically placebo) is used in an increasing number of clinical trials referring to the treatment of diseases for which well-established therapeutic strategies of proven efficacy and tolerability already exist. Under such circumstances it would be clearly unethical to leave dozens or hundreds of patients suffering from the respective disease without any real treatment until the end of the study. What has to be and is frequently done instead, is replacing the traditional negative control by a group to which the best therapy having been in use up to now, is administered. Usually, it is not realistic to expect then that the group which is given the new treatment will do still better than the control group with respect to efficacy endpoints. In return, the experimental therapy is typically known in advance to have much better tolerability so that its use can and should be recommended as soon as there is convincing evidence of equivalent efficacy. A particularly important example are trials of modifying adjuvant chemotherapy regimes well established in oncology, by reducing dosages and/or omitting the most toxic of the substances used. For such a reduced regime, superiority with respect to tolerability can be taken for granted without conducting any additional trial at all, and it is likewise obvious that demonstrating merely noninferiority with regard to efficacy would entail a valuable success. In the clinical trials methodology literature, it has sometimes been argued (cf. Windeler and Trampisch, 1996) that tests for equivalence in the strict, i.e., two-sided sense are generally inappropriate for an active-control study and should always be replaced by one-sided equivalence tests or tests for noninferiority. In contrast, we believe that there are several convincing points (not to be discussed in detail here) for the view that the question whether a one- or a two-sided formulation of the equivalence hypothesis eventually to be tested is the appropriate one, should be carefully discussed with the clinicians planning a specific active-control trial rather than decided by biostatistical decree once and for all. An undisputed major difference between clinical trials involving an active control, and comparative bioavailability studies (the consensus about the adequacy of two-sided equivalence tests for the confirmatory analysis of the latter has never been seriously challenged) refers to the structure of the distributions which the variables of primary interest typically follow: Quite often, the analysis of an active-control trial has to deal with binomial proportions [→ § 6.6] or even empirical survivor functions computed from partially censored observations [→ § 6.7] rather than with means and variances determined from samples of normally distributed observations.

8

INTRODUCTION

1.3.3

Preliminary tests for checking assumptions underlying other methods of statistical inference

Looking through statistical textbooks of virtually all kinds and levels of sophistication, it is hard to find any which does not give at least some brief account of methods for checking the assumptions that the most frequently used inferential procedures have to rely upon. All of them approach this basic problem from the same side: The testing procedures provided are tests of the null hypothesis that the assumptions to be checked hold true, versus the alternative hypothesis that they are violated in one way or the other. Since the aim a user of such a preliminary test commonly has in mind is to give evidence of the correctness of the required assumptions, one cannot but state that the usual approach is based on an inadequate formulation of the hypotheses. It is clear that equivalence tests in the sense of § 1.2 are exactly the methods needed for finding a way around this logical difficulty so that another potentially huge field of applications of equivalence testing methods comes within view. One group of methods needed in this context are of course, tests for goodness rather than lack of fit since they allow in particular the verification of parametric distributional assumptions of any kind. Other important special cases covered by the methods presented in subsequent chapters refer to restrictions on nuisance parameters in standard linear models such as – homoskedasticity [→ §§ 6.5, 7.4] – additivity of main effects [→ § 9.3.1] – identity of carryover effects in crossover trials [→ § 9.3.2]. Establishing goodness of fit by means of equivalence testing procedures is even an important issue in genetic epidemiology. This will will explained in detail in § 9.4 which is devoted to methods for assessing the validity of the Hardy-Weinberg assumption upon which some of the most basic and widely used techniques for the analysis of genetic association studies have to rely.

1.4

Role of equivalence/noninferiority studies in current medical research

The increasing relevance of both types of an equivalence study for current medical research is reflected in a number of facts comparatively easy to grasp from widely accessible sources and databases. A well-accepted way of getting an objective basis for statements about the development of some specific area of scientific research is through a systematic search over the pertinent part of published literature. A rough summary

Role of equivalence/noninferiority studies in current medical research

9

of results to be obtained following that line is presented in Figure 1.1 where the count of entries in the PubMed database containing a keyword indicating the use of equivalence/nonferiority testing methods in the respective paper, is plotted by calendar year ranging from 1990 through 2008. The keywords which were selected for that purpose were (i) bioequivalence, (ii) non(-)inferiority study, and (iii) equivalence study, with the parentheses around the hyphenation sign indicating that both spellings in use were covered.

Figure 1.1 Count of hits obtained by entering the keywords (i) bioequivalence, (ii) non(-)inferiority study, and (iii) equivalence study in PubMed, by year of publication. Admittedly, these figures must be interpreted with considerable caution, due to the unknown extent of the overlap between the scopes of the keywords of interest. Nevertheless, the following conclusions seem to be justified: • The majority of published studies aiming to establish equivalence are comparative bioavailability trials as described in § 1.3.1. Since 1990, the number of bioequivalence-related publications has increased by about 200%. • The time-lag in the quantitative development of published literature on studies devoted to establishing noninferiority as compared to equivalence, is likely to reflect mainly a change in terminological usage: Adoption of the term noninferiority by the medical community is a process which started only quite recently; before the beginning of the new millenium, systematic distinction between one- and two-sided equivalence

10

INTRODUCTION was lacking, and even nowadays, outside the area of bioequivalence, noninferiority and equivalence are often used more or less as synonyms. • Since 2002, the number of publications dealing with methodological issues and results of equivalence/noninferiority studies involving patients rather than healthy volunteers has reached the same order of magnitude as the number of bioequivalence-related publications per year.

Although the proportion of clinical trials leading to publications in the medical sciences literature is hard to estimate, it is likely that the vast majority remain unpublished. Thus, even a better basis for assessing the relevance of the equivalence paradigm for the medical progress than searching for pertinent literature might be provided by looking at the proportion of prescription drugs which have been approved to the market due to positive results of equivalence trials. Unfortunately, the question of how large this proportion is, admits of a well-grounded answer only for generic drugs, namely 100%, simply by definition. However, even if the corresponding proportion of innovator drugs were as low as 10%, calculation of the overall rate for the U.S. would yield the following result: In the Orange Book, Version 12/2008 of the FDA (2008) 12,751 prescription drug products were listed of which only 3,154 are innovator products; assuming that for 10% of the latter, the positive approval decision was based on equivalence trials, the rate among all authorized prescription drugs is obtained to be 100 × (9, 597 + 315)/12, 751 ≈ 78 % . This figure, which is likely to apply also to a number of other industrialized countries, gives sufficient evidence of the extent to which in our era, the medical sciences have to rely on a well-developed statistical methodology for the planning and analysis of studies conducted with the objective of establishing equivalence or noninferiority.

1.5

Formulation of hypotheses

As explained in nontechnical terms in Section 1.1, equivalence problems are distinguished from conventional testing problems by the form of the hypotheses to be established by means of the data obtained from the experiment or study under analysis. Typically [for an exception see § 10.3], the hypothesis formulation refers to some real-valued parameter θ which provides a sensible measure of the degree of dissimilarity of the probability distributions involved. For example, in the specific case of a standard parallel group design used for the purpose of testing for equivalence of two treatments A and B, an obvious choice is θ = μ1 − μ2 with μ1 and μ2 denoting a measure of location for the distribution of the endpoint variable under A and B, respectively. The equivalence hypothesis whose compatibility with the data one wants to

Formulation of hypotheses

11

assess, specifies that θ is contained in a suitable neighborhood around some reference value θ◦ taken on by θ if and only if the distributions under comparison are exactly equal. This neighborhood comprises those values of θ whose distance from θ◦ is considered compatible with the notion of equivalence for the respective setting. It will be specified as an open interval throughout with endpoints denoted by θ◦ − ε1 and θ◦ + ε2 , respectively. Of course, both ε1 and ε2 are positive constants whose numerical values must be assigned a priori, i.e., without knowledge of the data under analysis. Specifically, in the case of the simple parallel group design with θ = μ1 − μ2 , the usual choice of θ◦ is θ◦ = 0, and the equivalence interval is frequently chosen symmetrical about θ◦ , i.e., in the form (−ε, ε). Accordingly, in this book, our main objects of study are statistical decision procedures which define a valid statistical test at some prespecified level α ∈ (0, 1) of the null hypothesis H : θ ≤ θ◦ − ε1 or θ ≥ θ◦ + ε2

(1.1a)

of nonequivalence, versus the equivalence assumption K : θ◦ − ε1 < θ < θ◦ + ε2

(1.1b)

as the alternative hypothesis. Such a decision rule has not necessarily to exhibit the form of a significance test in the usual sense. For example, it can and will [see § 3.2] also be given by a Bayes rule for which there is additional evidence that the “objective probability” of a false decision in favor of equivalence will never exceed the desired significance level α. Bayesian methods for which we cannot be sure enough about this property taken for crucial from the frequentist point of view, are of limited use in the present context as long as the regulatory authorities to which drug approval applications based on equivalence studies have to be submitted, keep insisting on the maintenance of a prespecified significance level in the classical sense. It is worth noticing that an equivalence hypothesis of the general form (1.1b) will never be the same as the null hypothesis H0 : θ = θ◦ of the corresponding two-sided testing problem, irrespective of what particular positive values are assigned to the constants ε1 and ε2 . In other words, switching attention from an ordinary two-sided to an equivalence testing problem entails not simply an exchange of both hypotheses involved but in addition a more or less farreaching modification of them. Replacing the nondegenerate interval K of (1.1b) by the singleton {θ◦ } would give rise to a testing problem admitting of no worthwhile solution at all. In fact, in all families of distributions being of interest for concrete applications, the rejection probability of any statistical test is a continuous function of the target parameter θ. But continuity of the power function θ → β(θ), say, clearly implies, that the test can maintain level α on {θ = θ◦ } only if its power against θ = θ◦ exceeds α neither. Consequently, if we tried to test the null hypothesis θ = θ◦ against the alternative θ = θ◦ , we would not be able to replace the trivial “test” rejecting the null hypothesis independently of the data with probability α, by a useful decision rule.

12

INTRODUCTION

The inferential problems to be treated in the subsequent chapters under the heading of noninferiority assessment share two basic properties with equivalence testing problems in the strict sense. In the first place, they likewise arise from modifying the hypotheses making up some customary type of testing problem arising very frequently in routine data analysis. In the second place, modification of hypotheses again entails the introduction of a region in the space of the target distributional parameter θ within which the difference between the actual value of θ and its reference value θ◦ is considered practically irrelevant. However, there remains one crucial difference of considerable importance for the correct interpretation of the results eventually established by means of the corresponding testing procedures, as well as the mathematical treatment of the testing problems: The region of tolerable discrepancies between θ and θ◦ is now bounded to below only whereas excesses in value of θ over θ◦ of arbitrary magnitude are considered acceptable or even desirable. In other words, the testing procedures required in this other context have to enable the experimenter to make it sufficiently sure that the experimental treatment A, say, is not substantially inferior to some standard treatment B, without ruling out the possibility that A may even do considerably better than B. In contrast, for an equivalence trial in the strict sense made precise before, the idea is constitutive that one may encounter hypo- as well as hyperefficacy of the new as compared to the standard treatment and that protecting oneself against both forms of a substantial dissimilarity between A and B is a definite requirement. Formally speaking, the crucial difference between equivalence testing and testing for absence of substantial inferiority is that in the latter type of problem the right-hand boundary θ◦ + ε2 of the equivalence interval is replaced with +∞ or, in cases where the parameter space Θ of θ is bounded to the right itself, by θ ∗ = sup Θ. The corresponding hypothesis testing problem reads (1.2) H1 : θ ≤ θ◦ − ε versus K1 : θ > θ◦ − ε with sufficiently small ε > 0. From a mathematical point of view, the direction of the shift of the common boundary of an ordinary one-sided testing problem does not matter. In fact, approaches well suited for the construction of tests for one-sided equivalence in the sense of (1.2) can also be used for the derivation of tests for one-sided problems with a boundary of hypotheses shifted to the right, and vice versa. If θ keeps denoting a meaningful measure for the extent of superiority of a new treatment A over some standard treatment B and θ = θ◦ indicates identity in effectiveness of both treatments, testing θ ≤ θ◦ + ε versus θ > θ◦ + ε rather than θ ≤ θ◦ − ε versus θ > θ◦ − ε makes sense whenever one wants to ensure that a significant result of the corresponding test indicates that replacing A by B entails a relevant improvement. As pointed out by Victor (1987) this holds true for the majority of clinical trials aiming at giving evidence of treatment differences rather than equivalence.

Choosing the main distributional parameter

1.6

13

Choosing the main distributional parameter

Except for single-parameter problems, the scientific relevance of the result of an equivalence testing procedure highly depends on a careful and sensible choice of the target parameter θ [recall (1.1a), (1.1b) and (1.2)] in terms of which the hypotheses have been formulated. The reason is that, in contrast to the corresponding conventional testing problems with the common boundary of null and alternative hypothesis being given by zero, equivalence problems remain generally not invariant under redefinitions of the main distributional parameter. A simple, yet practically quite important example which illustrates this fact, is the two-sample setting with binomial data. If we denote the two unknown parameters in the usual way, i.e., by p1 and p2 , and define  δ and ρ as the difference p1 − p2 and the odds ratio p1 (1 − p2 )/ (1 − p1 )p2 , respectively, then the null hypotheses δ = 0 and ρ = 1 correspond of course to exactly the same subset in the space [0, 1] × [0, 1] of the primary parameter (p1 , p2 ). On the other hand, the set {(p1 , p2 )| − δ1 < δ < δ2 } will be different from {(p1 , p2 )|1 − ε1 < ρ < 1 + ε2 } for any choice of the constants 0 < δ1 , δ2 < 1 and 0 < ε1 < 1, ε2 > 0 determining the equivalence limits under both specifications of the target parameter. On the one hand, there are no mathematical or otherwise formal criteria leading to a unique answer to the question about the appropriate choice of the parameter of main interest for purposes of formulating equivalence hypotheses for a given model or setting. On the other, this is by no means a matter of purely subjective taste but in numerous cases there are convincing arguments for preferring a specific parametrization to an alternative one, with the discrimination between the difference of the responder rates and the odds ratio in the binomial two-sample setting giving an interesting case in point. Although simplicity and ease of interpretability even for the mathematically less educated user clearly speak in favor of the difference δ, plotting the regions corresponding to the two equivalence hypotheses −δ1 < δ < δ2 and 1 − ε1 < ρ < 1 + ε2 as done in Figure 1.2, shows to the contrary that defining equivalent binomial distributions in terms of p1 − p2 entails a serious logical flaw: Whereas the hypothesis of equivalence with respect to the odds ratio corresponds to a proper subset of the parameter space for (p2 , δ), the range of δ-coordinates of points equivalent to 0 in the sense of the first hypothesis formulation, is distinctly beyond the limits imposed by the side conditions −p2 ≤ δ ≤ 1 − p2 , for all sufficiently small and large values of the baseline responder rate p2 . This fact suggests that the choice θ = δ, notwithstanding its popularity in the existing literature on equivalence testing with binomially distributed data [for a selection of pertinent references see § 6.6.3] leads to an ill-considered testing problem.

14

INTRODUCTION

1.0

1.0

0.5

0.5

0.1 0 -0.1

0.1 0 -0.1

-0.5

-0.5

-1.0 0.0

0.25

0.5

0.75

1.0

p2

-1.0 0.0

0.25

0.5

0.75

1.0

p2

Figure 1.2 Equivalence hypotheses in terms of the difference between both responder rates [left] and the odds ratio [right] as regions in the p2 × δ-plane  set of possible values of ( p2 , δ).] with δ = p1 − p2 . [Rhomboid = Another special case whose importance for practical work can hardly be overestimated since, under standard parametric modeling, bioequivalence assessment with data from two-period crossover studies reduces to it, concerns the comparison of two Gaussian distributions with common but unknown variance on the basis of independent samples. Denoting, as usual, the two expected values and the common standard deviation by μ1 , μ2 and σ, respectively, the predominant approach starts from the choice θ = μ1 − μ2 although there are clear reasons why setting θ = (μ1 − μ2 )/σ yields a much more sensible measure of distance between the distributions to compare in the setting of the two-sample t-test: It is an elementary fact (whose implications are given due attention in Lehmann and Romano, 2005, § 5.3) that, given any whatever large value of |μ1 − μ2 |, both distributions become practically indistinguishable if σ is large enough, whereas the areas under the corresponding densities are next to disjoint if σ approaches zero. At the same time, focusing on the standardized rather than the raw difference between the means, facilitates

Numerical specification of the limits of equivalence

15

the step to be discussed in some more detail in the subsequent section: Even in discussions with clinical researchers caring little for statistical subtleties, presentation of a handful of graphs usually suffices to reach a consensus that |μ1 −μ2 |/σ ≥ 1 is incompatible with the notion of equivalence of two Gaussian distributions, and so on. Interestingly enough, under some circumstances, a thoughtful discussion of the question how the target parameter for defining the equivalence region should most appropriately be chosen, will even lead to the conclusion that the equivalence testing problem originally in mind should better be replaced by an ordinary one-sided testing problem. An example of this kind arises in bioequivalence studies of which one knows (or feels justified to assume) that no period effects have to be taken into account (Anderson and Hauck, 1990; Wellek, 1990, 1993a) [see also Ch. 10.3 of the present book].

1.7

Numerical specification of the limits of equivalence

The first question which arises when we want to reach a decision on what numerical values shall be assigned to the equivalence limits θ◦ − ε1 , θ◦ + ε2 defining the hypotheses in a testing problem of the form (1.1), is whether or not the equivalence interval has to be symmetric about the reference value θ◦ . More often than not it seems reasonable to answer this in the affirmative, although virtually all procedures presented in the chapters following the next allow full flexibility in that respect. Perhaps the still best known example of a whole area of application for methods of establishing equivalence in a nonsymmetric sense is bioequivalence assessment along the former FDA guidelines. Before the 1992 revision of its guidance for bioequivalence studies, the FDA strongly recommended to use the specifications θ◦ − ε1 = 2 log(.80) ≈ −.446, θ◦ + ε2 = 2 log(1.20) ≈ .365 for the maximally tolerable shift between the Gaussian distributions eventually to compare. Essentially, the corresponding interval is the log-transform of what is customarily called the 80 to 120% range for the ratio of the true drug formulation effects. In the revised version of the guidelines, the latter has been replaced with the range 80–125%. With regard to the question whether it is advisable for the statistician to give general recommendations concerning the form of the equivalence interval, we take the same position as on the one- versus two-sidedness controversy in the context of active-control trials [recall § 1.3.2]: This is a point for careful discussion with the researcher planning an individual study and should not made subject to fixed general rules. Instead, full generality should be aimed at in developing the pertinent statistical methods so that we can provide the experimental or clinical researcher with a range of options sufficiently large for allowing him to cover the question he really wants to answer by means of

16

INTRODUCTION

his data. Even if the problem is one of testing for noninferiority or has been symmetricized by introducing the restriction ε1 = ε2 = ε in (1.1a) and (1.1b), coming to an agreement with the experimenter about a specific numerical value to be assigned to the only remaining constant determining the equivalence interval, is not always easy. The following table is intended to give some guidance for some of the most frequently encountered settings: Table 1.1 Proposals for choosing the limits of a symmetrical equivalence interval or the noninferiority margin in some standard settings.

(Serial No.) (i) (ii) (iii) (iv) (v) (vi)

(vii) †) ‡)

Setting

Target Parameter or Functional p+ = P [D > 0]†)

Sign test MannWhitney π+ = P [X > Y ]‡) Two binolog ρ =  1 (1−p2 )  mial samples log p(1−p 1 )p2 Paired t-Test δ/σ Two-Sample t-Test (μ1 − μ2 )/σ Two Gaussian distr., comparison of var. log(σ1 /σ2 ) Two exponential distr. log(σ1 /σ2 )

ReferTolerance ε : ence Strict Liberal Value Choice 1/2 .10 .20 1/2

.10

.20

0

.41

.85

0

.25

.50

0

.36

.74

0

.41

.69

0

.405

.847

D ≡ intraindividual difference for a randomly chosen observational unit X, Y ≡ independent observations from different distributions

Here are some reasons motivating the above suggestions: → (i),(ii): Everyday experience shows that most people will rate probabilities of medium size differing by no more than 10%, as rather similar; 20% or more is usually considered indicating a different order of magnitude in the same context. → (iii):

Assuming that the reference responder rate is given by p2 = 1/2, straightforward steps of converting inequalities show the condition 1+2ε ) ≡ ερ˜. −ε < p1 − p2 < ε to be equivalent to | log ρ| < log( 1−2ε According to this relationship, the choices ε = .10 and ε = .20 [recall (i)] correspond to ερ˜ = log(12/8) = .4055 ≈ .41 and ερ˜ = log(14/6) = .8473 ≈ .85, respectively.

Numerical specification of the limits of equivalence → (iv):

17

Under the Gaussian model D ∼ N (δ, σ 2 ), we can write: ⇔ ⇔ ⇔

1/2 − ε < p+ ≡ P [D > 0] < 1/2 + ε Φ−1 (1/2 − ε) < δ/σ < Φ−1 (1/2 + ε) −Φ−1 (1 − (1/2 − ε)) < δ/σ < Φ−1 (1/2 + ε) −Φ−1 (1/2 + ε) < δ/σ < Φ−1 (1/2 + ε)

where Φ−1 denotes the quantile function for the standard normal distribution. Hence, the choice ε = .10 and ε = .20 in case (i) corresponds here to ε = Φ−1 (.60) = .2529 and ε = Φ−1 (.70) = .5240, respectively. → (v):

Analogously, relating (ii) to the special case X ∼ N (μ1 , σ 2 ), Y ∼ N (μ2 , σ 2 ), yields the first in the following chain of inequalities:

⇔ ⇔

√ 1/2 − ε < Φ((μ1 − μ2 )/ 2σ) < 1/2 + ε √ Φ−1 (1/2 − ε) < (μ1 − μ2 )/ 2σ < Φ−1 (1/2 + ε) √ √ − 2 Φ−1 (1/2 + ε) < (μ1 − μ2 )/σ < 2 Φ−1 (1/2 + ε) .

Thus, the√choices suggested for (ii) are this √ time equivalent to setting ε = 2 Φ−1 (.60) = .3577 and ε = 2 Φ−1 (.70) = .7411, respectively. → (vi):

Unlike (ii) – (v), this setting cannot be related in a natural way to case (i). The suggested values of ε, except for rounding to two significant decimals, are equivalent to the requirements 2/3 < σ1 /σ2 < 3/2 and 1/2 < σ1 /σ2 < 2, respectively. The latter seem again plausible for common statistical sense.

→ (vii):

The exponential scale model is a particularly important special case of a proportional hazards model so that the general considerations of § 6.3 about the latter apply.

2 General techniques for dealing with noninferiority problems

2.1

Standard solution in the case of location parameter families

Whenever the target parameter θ is a measure of the shift in location of the distributions of interest, shifting the common boundary of a pair of one-sided hypotheses produces a testing problem which is new only when we look at the concrete meaning of a positive decision in favor of the alternative. From a purely statistical point of view, no more than a trivial modification of the usual test for the corresponding conventional problem θ ≤ θ◦ versus θ > θ◦ is required. Subsequently we describe the rationale behind this modification in some detail for the case of comparing two treatments A and B on the basis of paired and of independent samples of univariate observations, respectively.

2.1.1

Paired observations

The data to be analyzed in any trial following the basic scheme of paired comparisons consists of random pairs (X, Y ), say, such that X and Y gives the result of applying treatment A and B to the same arbitrarily selected observational unit. Except for the treatment, the conditions under which X and Y are taken are supposed to be strictly balanced allowing the experimenter to interpret the intra-subject difference D = X − Y as quantifying in that individual case the superiority in effectiveness of treatment A as compared to B. In this setting, speaking of a location problem means to make the additional assumption that in the underlying population of subjects, any potential difference between both treatments is reflected by a shift θ in the location of the distribution of D away from θ◦ = 0 leaving the distributional shape per se totally unchanged. In absence of any treatment difference at all, let this distribution be given by some continuous cumulative distribution function (cdf) F◦ : IR → [0, 1] symmetric about zero. For the time being, we do not specify whether the baseline cdf has some known form (e.g., F◦ = Φ(·/σ) with Φ denoting the standard normal cdf), or is allowed to vary over the whole class of all continuous cdf’s on the real line being symmetric about zero [→

19

20

GENERAL TECHNIQUES FOR NONINFERIORITY PROBLEMS

nonparametric one-sample location problem, cf. Randles and Wolfe (1979), Ch. 10)]. Regarding the full data set, i.e., the sample (X1 , Y1 ), . . . , (Xn , Yn ) of all n pairs of measurements obtained under both treatments, statistical inference is based on the following assumptions about the corresponding intraindividual differences Di = Xi − Yi : (a) The vector (D1 , . . . , Dn ) is independent and identically distributed (iid) ; (b) P [Di ≤ d] = F◦ (d − θ) for arbitrary z ∈ IR and all i = 1, . . . , n, with F◦ (−d) = 1 − F◦ (d) ∀d and θ denoting the parameter of interest. (By convention, positive values of θ are assumed to indicate a tendency towards “better” results under treatment A as compared to B). Now, the general form of the rejection region of a test at level α for the traditional one-sided testing problem H1◦ : θ ≤ 0 versus K1◦ : θ > 0 [→ (1.2), specialized to the case that θ◦ = 0, ε = 0] is well known to be given by   T (D1 , . . . , Dn ) > cα where T (·) denotes a suitable real-valued function of n arguments (usually called the test statistic), and cα the upper 100α percentage point of the distribution of the random variable (D1 , . . . , Dn ) under θ = 0. If we change the ˜ i = Di + ε, then the moddirectly observed intra-subject differences Di to D ˜ ˜ ified sample (D1 , . . . , Dn ) obviously satisfies again (a) and (b), provided the parameter θ is likewise shifted introducing the transform θ˜ = θ + ε. Furthermore, the testing problem H1 : θ ≤ −ε versus K1 : θ > −ε we are primarily interested in, is clearly the same as the ordinary one-sided problem θ˜ ≤ 0 ˜ i . Hence, versus θ˜ > 0 relating to the transformed intra-subject differences D in the present setting we obtain the desired test for the one-sided equivalence problem H1 vs. K1 simply by using the rejection region of the test for the associated nonshifted null hypothesis in terms of the observations shifted the same distance as the common boundary of the hypotheses but in the opposite direction. The test obtained in this way rejects the null hypothesis H1 : θ ≤ −ε [→ relevant inferiority] if and only if we find that T (D1 + ε, . . . , Dn + ε) > cα where T (·) and cα are computed in exactly the same way as before.

Example 2.1 We illustrate the approach described above by reanalyzing the data from a study (Miller et al., 1990) of possible effects of the fat substitute olestra on the absorption of highly lipophilic oral contraceptives. The sample recruited for this trial consisted of 28 healthy premenopausal women. During the verum phase, each subject consumed 18 gm/day olestra for 28 days while taking a combination of norgestrel (300μg) and ethinyl estradiol (30μg) as an oral con-

Standard solutions in the case of location parameter families

21

Table 2.1 Maximal concentrations of norgestrel (ng/ml) in the sera of 28 women while consuming olestra (Xi ) and meals containing or˜ i = Di + ε ; R ˜ + = rank of dinary triglycerides (Yi ), respectively [ D i ˜ the ith subject with respect to |Di |; ε = 1.5 ]. i

Xi

Yi

˜i D

1 6.03 6.62 0.91 2 5.62 6.78 0.34 3 6.93 6.85 1.58 4 5.86 8.09 –0.73 5 8.91 9.18 1.23 6 5.86 7.47 –0.11 7 9.43 9.90 1.03 8 5.30 4.29 2.51 9 4.99 3.80 2.69 10 6.12 7.01 0.61 11 12.45 9.53 4.42 12 5.48 6.39 0.59 13 6.04 4.63 2.91 14 8.32 5.54 4.28

˜+ R i 12 5 18 11 15 1 13 22 24 8 28 7 25 27

i

Xi

Yi

˜i D

15 11.81 11.19 2.12 16 8.72 9.55 0.67 17 7.01 5.53 2.98 18 7.13 6.71 1.92 19 6.56 6.53 1.53 20 4.22 5.39 0.33 21 4.13 4.92 0.71 22 6.57 9.92 –1.85 23 8.83 10.51 –0.18 24 9.05 10.15 0.40 25 9.31 9.55 1.26 26 7.67 8.95 0.22 27 7.66 6.63 2.53 28 5.45 8.01 –1.06

˜+ R i 21 9 26 20 17 4 10 19 2 6 16 3 23 14

traceptive. Blood samples were taken on days 12 to 14 of the cycle and analyzed for ethinyl and estradiol concentrations. For the placebo phase, the experimental and measurement procedure was exactly the same as for verum except for replacing olestra with conventional triglycerides at each meal. Table 2.1 gives the individual results for norgestrel and the maximum concentration Cmax as the pharmacokinetic parameter of interest. According to the general objective of the trial, let us aim at establishing that the consumption of olestra does not reduce the bioavailability of norgestrel (as measured by Cmax ) to a relevant extent. Further, let us define θ as denoting the population median of the distribution of the intra-subject differences Di = Xi − Yi with ε = 1.5 as the limit of relevance, and base the confirmatory analysis of the data on the Wilcoxon signed rank statistic. Then, the computational steps which have to be carried out in order to test for one-sided equivalence are as follows. ˜ i [→ Table (i) For each i = 1, . . . , 28, the shifted intra-subject difference D + ˜ ˜ 2.1, 4th column] and the rank Ri with respect to |Di | [→ Table 2.1, 5th column] have to be determined. ˜ i by (ii) Denoting the sum of ranks of subjects with a positive sign of D + ˜ Vs , the value of this modified signed rank statistic is computed to be V˜s+ = 359.

22

GENERAL TECHNIQUES FOR NONINFERIORITY PROBLEMS

(iii) Under θ = −ε = −1.5, V˜s+ has an asymptotic normal distribution with expected value E0 (V˜s+ ) = n(n + 1)/4 = 28 · 29/4 = 203 and variance V ar0 (V˜s+ ) = n(n + 1)(2n + 1)/24 = 1928.5. Hence, the usual approximation with continuity correction gives the √ p-value (observed significance probability) pobs = Φ[(203 − 359 + .5)/ 1928.5 = Φ[−3.5410] = .0002. In view of the order of magnitude of the significance probability obtained in this way, the decision of the modified signed rank test for one-sided equivalence is positive even at the 1% level in the present case. In other words, the results of the study performed by Miller et al. (1990) contain sufficient evidence in favor of the hypothesis that the consumption of olestra does not lead to a relevant decrease of the bioavailability of norgestrel.

2.1.2

Two independent samples

If a comparative study of two treatments A and B follows the parallel group design, the data set to be analyzed consists of values of m + n mutually independent random variables X1 , . . . , Xm , Y1 , . . . , Yn . By convention, it is assumed that the Xi are observed in subjects who are given treatment A whereas the Yj relate to the other treatment, i.e., to B. In this setting, the shift model implies that any possible treatment difference can be represented d by means of the relationship Xi = Yj + θ where, as usual (cf. Randles and d

Wolfe, 1979, §1.3), the symbol “=” indicates identity of the distributions of the two random variables appearing on its left- and right-hand side, respectively. In other words, in the case of the parallel group design the shift model assumes that the distributions associated with the two treatments have exactly the same shape, implying that distribution A can be generated by shifting all individual values making up distribution B the same distance |θ| to the right (for θ > 0) or left (for θ < 0). Making this idea mathematically precise leads to specifying the following assumptions about the two distributions under comparison: (a∗ ) The complete data vector X1 , . . . , Xm , Y1 , . . . , Yn is independent, and all Xi and Yj have the same continuous distribution function F and G, respectively. (b∗ ) There exists a real constant θ such that F (x) = G(x − θ) for all x ∈ IR. Reduction of the one-sided equivalence problem H1 : θ ≤ −ε vs. K1 : θ > −ε to the corresponding ordinary one-sided testing problem θ˜ ≤ 0 vs. θ˜ > 0 proceeds here along analogous lines as in the case of paired observations discussed in the previous subsection. To start with, one has to select a suitable test for the nonshifted hypothesis which rejects if and only if one has T (X1 , . . . , Xm , Y1 , . . . , Yn ) > cα , where the test statistic T (·) is a real-valued function of m + n arguments and cα stands for the upper 100α percentage point of the distribution of T (X1 , . . . , Xm , Y1 , . . . , Yn ) under θ = 0. As before,

Standard solutions in the case of location parameter families

23

˜ i and Y˜j , say, of the this test has to be carried out with suitable transforms X ˜ i = Xi + ε (for i = 1, . . . , m) and primary observations Xi and Yj , given by X Y˜j = Yj (for j = 1, . . . , n), respectively. In view of the close analogy to the paired observations case, illustration of the approach by another numerical example is dispensable. Its implementation gets particularly simple in the parametric case with F (x) = Φ((x − θ)/σ), G(y) = Φ(y/σ) and T chosen to be the ordinary two-sample t-statistic: One has just to replace x ¯ − y¯ by x ¯ − y¯ + ε in the numerator of T then and proceed exactly as usual in all remaining steps.

2.1.3

Power and sample size calculation based on tests for noninferiority under location-shift models

The changes required for adapting the usual formula for power and sample size calculation for tests for one-sided location-shift hypotheses to the noninferiority case are likewise straightforward. Suppose the specific alternative to be detected in a test for noninferiority of the form discussed in this section is given by some fixed value θa of the shift parameter to which the proposed hypothesis is referring. Then, the power of the test for noninferiority with margin ε is the same as that of the ordinary one-sided test for the respective setting against the alternative θ˜a = ε + θa , and the sample size required for ensuring that the test for noninferiority rejects with given probability β under this alternative, is likewise obtained by replacing θa with θ˜a in the formula or algorithm for the one-sided case. In the majority of practical applications, the alternative of primary interest is given by θa = 0 specifying that the effects of both treatments are identical. The power of the noninferiority test against this “null alternative” is obtained by calculating the rejection probability of the corresponding test for H1◦ : θ ≤ 0 versus K1◦ : θ > 0 under θ = ε . Specifically, for the shifted t-tests for noninferiority, the power against θ = 0 is given by POW0 = 1 − G√nε/σD (tn−1;1−α ) and

POW0 = 1 − G∗√

mn/N ε/σ

(tN −2;1−α )

(2.1a) (2.1b)

in the paired-sample and independent-sample case, respectively. In the first of these formula, Gλnc (·) stands for the cdf of the noncentral t-distribution with noncentrality parameter λnc ∈ IR and n − 1 degrees of freedom. Furthermore, σD denotes the population standard deviation of the intraindividual differences, tn−1;1−α the (1 − α)-quantile of the central t-distribution with df = n − 1 . G∗λnc (·) differs from Gλnc (·) by changing the number of degrees of freedom from n − 1 to N − 2 ≡ m + n − 2, and the analogous change has to be made concerning the central t-quantile when proceeding from the oneto the two-sample case. Evaluation of the above expression for the power is

24

GENERAL TECHNIQUES FOR NONINFERIORITY PROBLEMS

particularly easy in a programming environment like R and SAS providing a predefined function for computing the noncentral t-distribution function.

2.2

Methods of constructing exact optimal tests for settings beyond the location-shift model

Clearly, the simple trick behind the approach of § 2.1 works only with problems relating to location-shift models, and the repertoire of noninferiority tests which can be constructed in that way is much too narrow for covering even the standard settings occurring in real applications. However, from a theoretical point of view, noninferiority problems are nothing but one-sided testing problems with a nonzero cutoff specified as the boundary point of the hypotheses between which one wants to decide. Fortunately, the mathematical principles leading to exact optimum solutions of one-sided hypothesis testing problems apply for arbitrary specifications of that boundary point, and the only modification required when proceeding from the classical to the noninferiority formulation concerns the way of determining the critical bounds and constants. In the noninferiority case, a suitable noncentral version of the sampling distribution of the test statistic involved has to be used. In order to make these general statements more concrete, let us denote by X the collection of all observations obtained in terms of the experiment or study under current analysis, i.e., a random vector of dimension at least as large as the sum of all sample sizes involved [ e.g., in an ordinary parallel group design for a trial of two treatments one has X = (X1 , . . . , Xm , Y1 , . . . , Yn ) ]. Let us further assume that this primary data vector can be reduced to some real valued statistic T (X) such that the possible distributions of T (X) constitute a family with monotone likelihood ratios in the parameter θ of interest (also called a strictly totally positive family of order 2 or STP2 family — see Definition A.1.1 in the Appendix). Then, it is a well-known fact of the mathematical theory of hypotheses testing (see, e.g., Lehmann and Romano, 2005, § 3.4) that for any choice of the noninferiority margin ε, there is a test for H1 : θ ≤ θ◦ − ε versus K1 : θ > θ◦ − ε which is uniformly most powerful among all tests at the same significance level α depending on the data only through T . The rejection region of this test is given by   x | T (x) > k . (2.2) The way in which the critical constant k has to be determined depends on the type of the distribution which the test statistic follows under θ = θ◦ − ε. In the continuous case, the optimal k is obtained by solving the equation Pθ◦ −ε [T (X) > k] = α , ∞ < k < ∞ .

(2.3)

Exact optimal tests for settings beyond the location-shift model

25

For noncontinuous T (X), a solution to (2.3) will typically not exist. However, it will always be possible to uniquely determine a pair (k, γ) of real numbers the second of which belongs to the half-open unit interval [0 , 1) such that there holds Pθ◦ −ε [T (X) > k] + γPθ◦ −ε [T (X) = k] = α , ∞ < k < ∞ .

(2.4)

For γ > 0, the exact level-α test has to be carried out entailing a randomized decision between H1 and K1 when T (X) falls on the critical point k. In this case which can almost surely ruled out for continuously distributed T (X), the null hypothesis H1 of (relevant) inferiority has to be rejected [accepted] with probability γ [1 − γ]. Since randomized decision rules are rarely applicable in the confirmatory statistical analysis of real research data, the point k is usually incorporated in the acceptance region even if the event {T (X) = k} has positive probability, giving a test which is more or less conservative. Promising techniques for reducing this conservatism will be discussed in § 2.5 . Even if the STP2 −property of the family of distributions of T (X) can be taken for granted, the precise meaning of the adjective “optimal” we used above with regard of a test of the form (2.2) (or, in the noncontinuous case, its randomized counterpart) depends on the relationship between T (X) and the distributions from which the primary data X have been taken. The most important cases to be distinguished from this point of view are the following: (i) T (X) is sufficient for the family of the possible distributions of X; then, optimal means uniformly most powerful (UMP). (ii) The model underlying the data corresponds to a multiparameter exponential family of distributions (see Definition A.2.1 in the Appendix); then, in (2.3) and (2.4), the symbol Pθ◦ −ε [ · ] must be interpreted as denoting the conditional distribution of T (X) given some fixed value of another (maybe multidimensional statistic) being sufficient for the remaining parameters of the model, and the test rejecting for sufficiently large values of T (X) is uniformly most powerful among all unbiased tests (UMPU). (iii) The proposed testing problem remains invariant under some group of transformations, and the statistic T (X) is maximal invariant with respect to that group; then, the test with rejection region (2.2) is uniformly most powerful among all invariant level-α tests (UMPI) for the same problem. [For proofs of these statements see Sections 1.9, 4.4 and 6.3 of the book by Lehmann and Romano (2005).] Settings admitting the construction of UMP tests for (one-sided) equivalence are dealt with in Chapter 4. Important special cases of equivalence testing problems which can be solved through reduction by sufficiency are the comparison of two binomial distributions from which paired [→ § 5.2] or

26

GENERAL TECHNIQUES FOR NONINFERIORITY PROBLEMS

independent samples [→ § 6.6] are available. Somewhat surprisingly, the same approach works in constructing a UMPU test for goodness of fit (rather than lack of fit) of the distribution of a biallelic genetic marker (SNP) with the Hardy-Weinberg model, as will be shown in § 9.4 . Among the practically most useful specific tests of category (iii) are the equivalence versions of the paired [→ § 5.3] and the two-sample t-tests [→ § 6.1] .

2.3

Large-sample solutions for problems inaccessible for exact constructions

As is true for virtually all areas of statistical inference, the range of noninferiority testing problems accessible to exact methods is too limited for covering the needs arising in practical data analysis. Problems for which no satisfactory exact solution is available, are by no means predominantly of a kind involving nonparametric or other rather complex models like the proportional hazards model for possibly censored survival times. Even such comparatively simple problems as those of testing for one-sided equivalence of binomial proportions with respect to the difference δ = p1 − p2 in the analysis of two independent or paired samples of binary data are cases in point. Fortunately, there is an asymptotic approach to the construction of tests for noninferiority which will enable us to fill a considerable part of the gaps left by exact constructional approaches. It applies to virtually any situation where the hypothesis to be tested refers to a parameter or (in non- or semiparametric models) functional for which an asymptotically normal estimator can be found. The notation which was introduced in § 1.5 is interpreted here in its broadest sense covering in particular cases where the target parameter θ depends on more complex characteristics of the distribution functions under comparison than merely their first two moments or selected quantiles. In other words, θ is allowed to have the meaning of an arbitrary functional of the vector of all underlying distribution functions. The dimension of this vector can be any positive integer k which must not vary with the total sample size N . For illustration, let us anticipate a small bit of the material presented in § 6.2 referring to the nonparametric two-sample problem with continuous data X1 , . . . , Xm and Y1 . . . , Yn . In this setting, one wishes to compare k = 2 distribution functions usually denoted F and G, and an intuitively appealing choice of a functional which a reasonable measure of distance between them can be based upon, is θ = GdF = P [Xi > Yj ]. Returning to the problem of testing for equivalence in its most general formulation, suppose further that for each given value of the total sample size N , there is some real-valued statistic TN , say, such that the associated sequence (TN )N ∈ IR is an asymptotically

Objective Bayesian methods normal estimator for θ in the sense that we have √ L N (TN − θ)/σ → N (0, 1) as

27

N →∞

(2.5)

for a suitable positive functional σ of the vector (F1 , . . . , Fk ) of distribution functions under consideration. Now, the assumed weak-convergence property of TN clearly implies that an asymptotically valid test for the noninferiority problem H1 : θ ≤ θ◦ − ε versus K1 : θ > θ◦ − ε can be based on the rejection region

   TN − (θ◦ − ε) /τN > u1−α (2.6) with u1−α denoting the (1 − α)-quantile of the standard normal distribution. Of course, the asymptotic standard error τN will not be known √ in practice. ˆN / N based on a However, it can be replaced with any estimator τˆN = σ weakly consistent estimator of σ.

2.4

Objective Bayesian methods

From the Bayesian viewpoint, problems of testing for noninferiority exhibit no peculiarities at all, neither conceptually nor technically. All that is needed as soon as the target parameter θ and the common boundary θ◦ − ε of the hypotheses has been fixed, is a joint prior distribution π(·), say, of all unknown parameters contained in the underlying statistical model. The Bayesian test for one-sided equivalence of θ with θ◦ rejects if the posterior probability of the interval (θ◦ − ε, ∞) specified by the alternative hypothesis with respect to this prior is computed to be larger than a suitably lower bound which is customarily set equal to the complement of the nominal significance level. In other words, given the prior distribution π(·), the Bayesian test for noninferiority uses the decision rule∗ : Reject (relevant) inferiority if and only if π θ|X=x (θ◦ − ε , ∞) ≥ 1 − α .

(2.7)

For the sake of eliminating subjectivism entailed by selecting the prior in an arbitrary way, relying on so-called noninformative priors has much to recommend it. By definition, a noninformative or “objective” prior is a (maybe improper) uniform distribution of the model parameters involved or of suitable parametric functions. Although there is still considerable controversy ∗ Introducing

extra symbols for parameters treated as random variables rather than constants seems dispensable for the moment.

28

GENERAL TECHNIQUES FOR NONINFERIORITY PROBLEMS

about the theoretical foundation of the concept (cf. Berger, 1985, § 3.3 ; Cox and Hinkley, 1974, § 10.4), there is a useful pragmatic rule for determining a noninformative prior which has been extensively exploited in many applications of Bayesian inference (see in particular the landmark monograph of Box and Tiao, 1973) . The rule was first proposed by Jeffreys (1961) and states that π(·) is (approximately) noninformative if it is defined by a density (with respect to Lebesgue measure) taken proportional to the square root of the determinant of the information matrix. Experience shows that for many problems of hypotheses testing, the Bayesian approach yields procedures with fairly satisfactory frequentist properties as long as a neutral prior is used. However, even then it can and will happen [see, e.g., § 6.6.2] that the resulting test requires adjustment for level or size through decreasing the nominal α used in in the basic decision rule (2.7). One of the major advantages of this approach is its flexibility with respect to the choice of the target parameter in terms of which the equivalence region shall be defined.

2.5

Improved nonrandomized tests for discrete distributions

The loss of power entailed by dispensing with randomized decisions in performing tests obtained by applying the principles introduced in § 2.2 to discrete distributions can be dismissed as irrelevant for practical applications only as long as the sample sizes are fairly large. In small samples this conservatism can take on extreme proportions, and that is the reason why even in the case of a traditional one-sided null hypothesis, the exact Fisher type test for the two-sample problem with binomial data which is probably still the most frequently used non-asymptotic test for discrete data, has been criticized in the literature almost from the beginning (cf. Pearson, 1947) . In settings relating to single-parameter problems, the only recommendable approach to reducing the conservatism of a nonrandomized test based on a statistic T (X) following a distribution of the discrete type, is the so-called mid–p-value method. In order to understand how this does work, it is necessary to realize that, in a noninferiority setting, the nonrandomized version of the test based on (2.2) can equivalently be defined by the p-value–based decision rule Reject H1 : θ ≤ θ◦ − ε if and only if p(x) ≡ Pθ◦ −ε [T (X) ≥ T (x)] ≤ α . (2.8) Replacing p(x) with p+ (x) ≡ Pθ◦ −ε [T (X) > T (x)] in (2.8) would clearly lead to an anti-conservative testing procedure, and the simple idea behind the

Improved nonrandomized tests for discrete distributions

29

mid–p-value technique (cf. Agresti, 2002, § 1.4.5) is to compromise between over- and anti-conservatism by using the average of p(x) and p+ (x) . The resulting modification to (2.8) can be written Reject H1 : θ ≤ θ◦ − ε if and only if

1 pmid (x) ≡ Pθ◦ −ε [T (X) > T (x)] + Pθ◦ −ε [T (X) = T (x)] ≤ α . (2.9) 2 It is important to note that there is no mathematical result due to which validity of the test given by (2.9) with respect to the significance level can be generally guaranteed. However, numerical results from simulation studies of a variety of discrete families of distributions show that mid-p–value based tests are typically still conservative but to a markedly lesser extent than their exact counterparts. Of course, subject to the reservation made above, the mid-p–value approach works also in situations where the critical lower bound k [recall (2.2)] has to be determined from the conditional distribution of T (X) given some other statistics S, say, as has been outlined in § 2.2 under (ii). However, in conditional settings there are more promising ways of avoiding unnecessarily large losses in power through relying on nonrandomized testing procedures. Such methods have been most extensively discussed in the literature for the binomial two-sample setting, mostly in connection with the keyword exact Fisher type test (for a still reasonably up-to-date review see Mart´ın, 1998). For all problems which, like the exact Fisher test, relate to multiparameter exponential families, it seems desirable to preserve the basic conditional structure of the test, by reasons which are carefully explained by Little (1989) for that special case. Essentially, two different approaches to constructing improved nonrandomized versions of conditional tests will be used in later chapters of this book: (i) In order to describe the first of these approaches let us assume that the statistic S upon which conditioning has to be performed, has finite sample space {s1 , . . . , sq }. The construction of an improved nonrandomized conditional test starts with arranging the set {k(sν ) | ν = 1, . . . , q} of all critical bounds to T (X) in ascending order with respect to their unconditional rejection probabilities maximized over H1 . Afterwards, the union of all q conditional rejection regions in the sample space of T (X) is successively enlarged by adding in that order as many boundary points k(sν ) as possible without raising the size of the test (i.e., the maximum of the rejection probability taken over the null hypothesis) over the significance level α . (ii) The other approach follows a strikingly simple idea: The usual nonrandomized version of the conditional test is carried out at an increased nominal significance level α∗ > α determined iteratively by maximization over the set of all nominal levels which are admissible in the sense of keeping the size of the resulting test below α .

30

GENERAL TECHNIQUES FOR NONINFERIORITY PROBLEMS

Sort of a hybrid between both approaches was used by Starks (1979) in order to construct critical regions of an improved nonrandomized sign test for noncontinuous distributions. The second approach goes back to Boschloo (1970) who exploited it for constructing a more powerful version of the one-sided exact Fisher type test for two independent binomially distributed samples. The algorithm described by McDonald et al. (1977, 1981) leads to exactly the same critical regions as Boschloo’s increased nominal levels. The most important difference between (i) and (ii) is that the first approach may produce rejection regions containing points of the sample space whose conditional p-value is larger than that of some or even several points retained in the acceptance region. Our preference for (ii) is mainly motivated by the fact that it lacks this rather counterintuitive property. Moreover, for the practical implementation of Boschloo’s technique one can rely on tables of an extremely simple and compact form since, given the sample size(s), the boundary of the hypotheses and the target significance level, the nonrandomized test modified according to (ii) is uniquely determined by just one additional constant, viz. α∗ .

2.6

Relationship between tests for noninferiority and twosided equivalence tests

From the mathematical point of view, the construction of procedures for noninferiory problems entails no specific ideas and techniques beyond those which will be presented in the next chapter as basic tools for the derivation of twosided equivalence tests. On the contrary, solving a noninferiority problem is generally a good bit easier than constructing its two-sided counterpart, due to the fact, that for the latter, the number of critical constants to be determined is double (2 versus 1 in the case of a continuously distributed test statistic, and 4 versus 2 in constructing exact nonrandomized tests for discrete distributions). This difference in technical complexity will be directly reflected in the subsequent chapters where for the majority of specific settings considered, the exposition of the test for noninferiority is kept much shorter than that of the equivalence testing procedure. In practice, there is even the possibility of converting any given test for equivalence in the strict sense into a test for noninferiority: all that is necessary for exploiting the basic logical relationship between the hypotheses to be established, is simply to specify an extremely large value for the right-hand equivalence margin ε2 set in the equivalence case. As an obvious implication of this remark, it can be stated that the repertoire of noninferiority testing procedures ready for application to real problems and data sets is at least as large as that of procedures for the assessment of two-sided equivalence.

Halving alpha?

2.7

31

Halving alpha?

In the statistical parts of the guidelines issued by the drug regulation authorities, there is an increasing tendency to extend the verdict against the use of one-sided testing procedures in the confirmatory analysis of clinical trials also to noninferiority studies (see, e.g., EMEA, 1998, p. 27 ; EMEA, 2005, p. 3 ). In this book, we will not follow the recommendation to carry out by default one-sided equivalence tests at level 2.5% instead of 5%. Actually, the only rational argument in favor of this modified convention concerning the choice of the significance level is that one wants to prevent inflating the effective type-I error risk through reporting significance decisions as belonging to a one-sided hypothesis formulation although, in truth, the direction of the difference to detect was not fixed a priori. Even in a conventional placebo-controlled trial, this option seems to exist only in a theoretical sense since it is hard to imagine that a clinical investigator could really try to declare a significant benefit of placebo as compared to the active treatment of interest, as a positive result. In a noninferiority trial involving an active control, the suspicion that such post-hoc switching of the roles of both treatments involved might really be intended, seems plainly absurd: every expert in the clinical discipline to which the trial relates, will be perfectly sure about which of the two treatments was used as a reference so that a “positive” result of a test for the problem arising from an interchange of hypotheses in (1.2) would mean that the experimental treatment was shown to be relevantly inferior to the control. Of course, unwillingness to adopt the α/2 doctrine by no means implies that the tests for noninferiority discussed in the subsequent chapters are useless for a researcher feeling like performing them at level 2.5% nevertheless. The only concrete consequence is that all examples will be worked out under the specification α = .05 .

3 General approaches to the construction of tests for equivalence in the strict sense

3.1

The principle of confidence interval inclusion

The most popular and frequently used approach to problems of equivalence testing in the strict (i.e., two-sided) sense is still that which starts from interval estimation. The principle of confidence interval inclusion was originally introduced in a rather inconspicuous paper (written in the form of a Letter to the Editor of a pharmaceutical journal) by Westlake (1972). Since that time, it has been discussed in at least three different versions by numerous authors (among many others, see Metzler, 1974; Westlake, 1976, 1979, 1981; Steinijans and Diletti, 1983, 1985; Mau, 1988; Hauschke et al., 1990) dealing almost exclusively with bioequivalence assessment (→ Ch. 10) as the only worthwhile field of applying such methods. Our exposition of the approach concentrates on that version which is most efficient with regard to the power of the corresponding testing procedures. ¯ α) denote a lower and upper confidence bound for θ Let θ(X; α) and θ(X; at the same one-sided confidence level 1 − α. In the symbols used for these confidence bounds (and similarly for test statistics), X represents the collection of all observations obtained from the study under current analysis. Then, according to the confidence interval inclusion principle, we get a valid test for equivalence in the sense of (1.1b) [→ p. 11] by way of rejecting the null hypothesis H : θ ≤ θ◦ − ε1 or θ ≥ θ◦ + ε2 if and only if the random  ¯ α) is completely covered by the equivalence interinterval θ(X; α) , θ(X; val (θ◦ − ε1 , θ◦ + ε2 ) specified by our alternative hypothesis K. Of course, this means that the interval inclusion test decides in favor of equivalence pro¯ α) < θ◦ + ε2 are satisfied vided both inequalities θ(X; α) > θ◦ − ε1 and θ(X; simultaneously. To be sure, under the assumptions made explicit in the previous paragraph,  ¯ α) is just 1 − the two-sided confidence level of the interval θ(X; α) , θ(X; 2α. Nevertheless, the equivalence test based upon it has always significance level α rather than 2α. The proof of this important fact is almost trivial and requires just a few lines: Let θ be any point in the parameter space belonging to the left-hand part of H such that we have θ ≤ θ◦ − ε1 . By

33

34

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

definition, the event that the interval inclusion test rejects can be written   ¯ θ◦ − ε1 < θ(X; α) , θ(X; α) < θ◦ + ε2 . Hence, for θ ≤ θ◦ − ε1 , an error of the first kind can only occur if we find θ(X;  means that the  α) > θ which corresponding one-sided confidence region θ(X; α) , ∞ fails to cover the true value of θ. By construction of θ(X; α), this happens in at most 100α% of the applications of the corresponding confidence procedure so that we have in fact Pθ [type-I error] ≤ α for all θ ≤ θ◦ −ε1 . A completely analogous argument ¯ α) ≤ shows that for any θ ≥ θ◦ + ε2 , we have Pθ [type-I error] ≤ Pθ [ θ(X; θ ] ≤ α. For all remaining points in the parameter space, i.e., for θ◦ − ε1 < θ < θ◦ + ε2 , there is nothing to show because rejecting H will be a correct decision then. Although the mathematical reasoning required for establishing the validity of interval inclusion tests based on confidence intervals at level (1 − 2α) is so simple, most of the earlier work on the approach (see Westlake, 1981) used intervals of two-sided confidence level 1 − α yielding unnecessarily conservative testing procedures. (Nonequal tails confidence intervals at level 1 − α for which the associated equivalence tests do not suffer from such a marked conservatism, are constructed in the paper of Hsu et al., 1994.) Another fact of considerable importance for a proper understanding of the confidence interval inclusion principle was first pointed out by Schuirmann (1987) for the special case of the setting of the two-sample t-test. Generally, any interval inclusion test for equivalence as defined above is logically equivalent to a combination of two one-sided tests. In fact, it is obvious from its definition that an interval inclusion test decides in favor of equivalence if and only if both the test of θ ≤ θ◦ −ε1 vs. θ > θ◦ −ε1 based on θ(X; α), and that ¯ α) as the test statistic, can reject its of θ ≥ θ◦ + ε2 vs. θ < θ◦ + ε2 using θ(X; null hypothesis [→ “double one-sided testing procedure”]. As will be explained in detail in § 7.1, this means that the principle of confidence interval inclusion is just a special case of the so-called intersection-union principle introduced by Berger (1982) in the context of a problem of statistical quality control (see also Berger and Hsu, 1996). For the setting of the two-sample t-test with θ = μ1 − μ2 and θ◦ = 0, the  ¯ (1 − 2α)−confidence interval θ(X; α) , θ(X; α) introduced above is symˆ ¯ ¯ metric about the pivot θ = X − Y rather than 0. In Westlake (1976) it was argued that applying the interval inclusion rule with confidence intervals nonsymmetric with respect to 0 is unsuitable whenever we deal with a symmetric equivalence hypothesis specifying that |θ| = |μ1 −μ2 | < ε. Adopting this point of view would imply that the test be based on a (1 − α)-confidence interval of ¯ − Y¯ + Cα ) with Cα denoting a suitable real-valued ¯ − Cα , X the form (Y¯ − X ¯ − Y¯ + Cα > 0. However, on closer examination random variable such that X it becomes obvious that this suggestion relates the symmetry requirement to the wrong object. In fact, assuming two samples from homoskedastic Gaussian distributions, the testing problem |μ1 − μ2 | ≥ ε vs. |μ1 − μ2 | < ε as such turns out to exhibit a clear-cut symmetry structure in that it remains invariant against treating the Y ’s as making up the first rather than the second of the

The principle of confidence interval inclusion

35

two samples [formally: against the transformation (x1 , . . . , xm , y1 , . . . , yn ) → (y1 , . . . , yn , x1 , . . . , xm )]. In such a case, the well-accepted principle of invariance (cf. Cox and Hinkley, 1974, § 2.3 (vi) ; Lehmann and Romano, 2005, Ch. 6) requires that we use a decision rule leading to the same conclusion irrespective of whether or not the samples are labeled in the original manner. It is easy to check that the test based on the symmetric confidence ¯ − Y¯ + Cα ) lacks this invariance property. In fact, ¯ − Cα , X interval (Y¯ − X ¯ ¯ ¯ ¯ (Y − X − Cα , X − Y + Cα ) is included in the equivalence interval (−ε, ε) if and ¯ − Y¯ + Cα < ε, and the latter inequality is obviously not only if there holds X ¯ ¯ equivalent to Y − X +Cα < ε. On the other hand, the test based on the short¯ − Y¯ rejects if and only if est (1 − 2α)-confidence interval symmetric about X ¯ ¯ it happens that |X − Y | < ε −S(X1 , . . . , Xm , Y1 , . . . , Yn ) · tm+n−2;1−α where  m ¯ 2 n (Yj − S(X1 , . . . , Xm , Y1 , . . . , Yn ) = 1/m + 1/n · i=1 (Xi − X) + j=1  1/2 Y¯ )2 (m + n − 2) and tm+n−2 ; 1−α denotes the upper 100α percentage point of a central t-distribution with m + n − 2 degrees of freedom. In view of S(X1 , . . . , Xm , Y1 , . . . , Yn ) = S(Y1 , . . . , Yn , X1 , . . . , Xm ), the condition for re¯ < ε − S(Y1 , . . . , Yn , X1 , . . . , Xm )· jecting nonequivalence is the same as |Y¯ −X| tm+n−2;1−α . In other words, the nonsymmetric confidence interval leads to a symmetric test, and vice versa. If we assume the common variance of the two normal distributions under comparison to be a known constant with respect to which all observations in both samples have been standardized, then the test of H : |μ1 − μ2 | ≥ ε vs. K : |μ1 − μ2 | < ε based on shortest (1 − 2α)-confidence intervals rejects if and only if we have  ¯ − Y¯ | < ε − u1−α · m−1 + n−1 , (3.1) |X defines a nonempty region where u1−α = Φ−1 (1−α). Trivially, condition (3.1) √ in the sample space only if the bound ε − u1−α · m−1 + n−1 represents a positive real number which is true only for sufficiently large sample sizes m, n. For example, in the balanced case m = n with ε = .25 and α = .05, (3.1) corresponds to a test which will never be able to reject nonequivalence unless the number n of observations available in both groups is at least 87. At first sight, this example seems to be fatal for the interval inclusion principle as a general approach to constructing equivalence tests. In fact, it refers to an ¯ Y¯ ) extremely regular setting since the distribution of the sufficient statistic (X, belongs to a two-parameter exponential family and the distribution of the ¯ Y¯ ) is absolutely continuous for any parameter constellation test statistic (X, (μ1 , μ2 ). Nevertheless, even if the common sample size is as large as 86, the testing procedure given by (3.1) is still poorer than the trivial level-α “test” deciding independently of the data in favor of equivalence whenever in some external random experiment an event of probability α happens to get realized. On the other hand, as will be shown in § 4.4, this is well compatible with the fact that for sample sizes exceeding the minimum value of 87 sufficiently far, decision rule (3.1) yields a test which is practically indistinguishable from the optimal solution to exactly the same testing problem.

36

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

3.2

Bayesian tests for two-sided equivalence

The basic ingredients required for a Bayesian treatment of two-sided equivalence testing problems are the same as introduced in § 2.4 with regard to testing for noninferiority. Again, any Bayesian construction of a test starts from a fully specified joint prior distribution π(·) of all unknown parameters appearing in the model behind the data under analysis, and the test rejects if the posterior probability of the region corresponding to the alternative hypothesis turns out to be larger than a suitably lower bound specified 1 − α as default. The only difference relates to the form of the region in the parameter space whose posterior probability has to be determined. In the case of twosided equivalence testing, this is given by the bounded interval (θ◦ −ε1 , θ◦ +ε2 ) so that the decision rule (2.7) for a Bayesian test for noninferiority has to be replaced with Reject nonequivalence if and only if π θ|X=x (θ◦ − ε1 , θ◦ + ε2 ) ≥ 1 − α .

(3.2)

As in the noninferiority case, the Bayesian approach will exclusively be used in its “objective” version, i.e., with a noninformative prior chosen according to the rule proposed by Jeffreys (1961). For illustration, let us consider the setting of the paired t-test with the intra-subject differences Di = Xi − Yi as the data eventually to be analyzed and the expected value δ of the Di as the parameter of interest. This is a case in which the posterior distribution of the target parameter with respect to the noninformative “reference prior” admits a simple explicit representation. As shown, e.g., in Lindley (1970, pp. 36-7) and Box and Tiao (1973, § 2.4), ¯ and standard deviation SD given the observed values d¯ and sD of the mean D ¯ of the Di , n1/2 (δ − d)/s has a posterior distribution then which is central D t with the usual number n − 1 of degrees of freedom. Hence, the posterior probability of the equivalence interval specified by the alternative hypothesis whose limits are this time denoted by δ1 , δ2 can be written T T 1/2 ¯ ¯ (n1/2 (δ2 − d)/s (δ1 − d)/s π δ|D=d (δ1 , δ2 ) = Fn−1 D ) − Fn−1 (n D)

(3.3)

T (·) stands for the cdf of a central t-distribution with n − 1 degrees where Fn−1 of freedom. Suppose specifically that we obtained d¯ = .16, sD = 3.99 in a sample of size n = 23† and that δ1 = −1.75, δ2 = 1.75 have been chosen as equivalence limits for δ. Then, evaluating the expression on the righthand side of (3.3) by means of the SAS function probt gives for the posterior probability of (δ1 , δ2 ): † The

source of these data is a pilot study (described in some more detail in Example 5.3, p. 96) of the temporal stability of the capillary flow in the brain of rabbits.

Bayesian tests for two-sided equivalence

37

π δ|D=d (−1.75 , 1.75) = T T F22 (231/2 (1.75 − .16)/3.99) − F22 (231/2 (−1.75 − .16)/3.99) = T T F22 (1.911121) − F22 (−2.295749) = .965447 − .015796 = .949651 .

Hence, at the nominal level α = 0.05, for this specific data set the Bayesian test (3.2) with respect to the reference prior distribution of (δ, σD ) would not be able to reject nonequivalence in the sense of |δ| ≥ 1.75. It is worth noticing that in situations where Bayesian credible intervals coincide with classical confidence intervals, the test obtained by applying decision rule (3.2) is the same as the interval inclusion test with confidence limits at one-sided confidence level 1 − α/2 rather than 1 − α. Recalling the correspondence between interval inclusion tests and double one-sided tests for equivalence, this suggests that for the sake of avoiding overconservatism, the double one-sided Bayesian test given by the decision rule Reject nonequivalence if and only if π θ|X=x (θ◦ − ε1 , ∞) ≥ 1 − α and

π θ|X=x (−∞, θ◦ + ε2 ) ≥ 1 − α

(3.4)

should generally preferred to the “direct” Bayesian equivalence test (3.2). In the numerical example introduced before, the posterior probabilities of the two infinite intervals to be intersected in order to get the equivalence interval itself, T (−2.295749) = 1 − .015796 = are computed to be π δ|D=d (−1.75, ∞) = 1 − F22 δ|D=d T (−∞, 1.75) = F22 (1.911121) = .965447. Since with α = .05, .984204, π both of these values exceed the critical lower bound 1 − α, the double onesided Bayesian test (3.4), in contrast to the direct Bayesian decision rule (3.2), leads to a positive result which reflects its improved power. As any technique of Bayesian inference, the approach to equivalence testing given by (3.2) or (3.4) is attractive not only for its conceptual simplicity, but likewise for its high flexibility with respect to reparametrizations of the problem. Like the prior, the posterior distribution is determined jointly for all unknown parameters of the model under analysis. Hence, it can easily be calculated for any sufficiently regular function of the primary parameters by means of the well-known transformation theorem for probability densities (cf. Bickel and Doksum, 2001, § B.2). In order to illustrate this technique, we keep considering the paired t-test setting and demonstrate how to compute the posterior density of the standardized expected difference θ = δ/σD assuming the same prior distribution of (δ, σD ) which underlies formula (3.3). For that purpose, the following results (to be found again in the book of Box and Tiao, 1973, pp. 95–6) are needed: ¯ σ 2 /n) [i.e., 1) The conditional posterior distribution of δ given σD is N (d, D 2 ¯ Gaussian with mean d and variance σD /n] . 2) The marginal posterior distribution of σD is that of sD (n − 1)1/2 /Xn−1 2 with Xn−1 ∼ χ2n−1 [= central χ2 -distribution with n − 1 degrees of freedom].

38

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

From 1) and 2), it follows that the joint posteriori density of (δ, σD ) can be written:  −1 ¯ f (δ, σD | D = d) = (n1/2 /σD ) ϕ(n1/2 (δ − d)/σ ) Γ((n − 1)/2) × D     (n−1)/2 (3−n)/2 −n (n − 1)sD2 2 σD exp − ((n − 1)/2) (sD /σD )2 (3.5) with Γ(·) and ϕ(·) denoting the gamma function and the standard Gaussian density, respectively. (3.5) leads to the following expression for the posterior probability of the hypothesis θ1 < θ < θ2 with θ = δ/σD and arbitrarily fixed equivalence limits −∞ < θ1 < θ2 < ∞ : π

θ|D=d

∞    ¯ ¯ Φ(n1/2 (θ2 σD − d)/σ (θ1 , θ2 ) = ) − Φ(n1/2 (θ1 σD − d)/σ ) × D D 0

(n−1)/2 (3−n)/2 −n  −1  (n − 1)sD2 Γ((n − 1)/2) 2 σ ×  D   (3.6) exp − ((n − 1)/2) (sD /σD )2 dσD .

The integral on the right-hand side of equation (3.6) can be evaluated by means of a suitable quadrature rule to any desired degree of numerical accuracy. A numerical method which serves remarkably well with integrals of that type is Gaussian quadrature (cf. Davis and Rabinowitz, 1975, § 2.7). The error entailed by truncating the interval of integration from above at some point c◦ −6 can be made  smaller than any prespecified ε◦ > 0 [e.g., ε◦ = 10 ] by choosing c◦ >

(n − 1)sD2 /χ2n−1 ; ε◦ with χ2n−1 ; ε◦ denoting the 100ε◦ -percentage

point of a central χ2 -distribution with n − 1 degrees of freedom. The program postmys to be found in the WKTSHEQ2 Source Code Package both as a SAS macro and a R function implements this computational approach using a grid of 96 abscissas (for a table of all constants involved in 96-point Gaussian quadrature see Abramowitz and Stegun, 1965, p. 919). In the specific case n = 23, d¯ = .16, sD = 3.99 used for illustration once more, it gives for the posterior probability of the equivalence region {(δ, σD ) | − 0.5 < δ/σD < 0.5} the value π θ|D=d (−0.5, 0.5) = .981496. Hence, with these data the Bayesian test for equivalence with respect to δ/σD leads to the same decision as the optimal frequentist testing procedure to be discussed in § 5.3 (see in particular p. 96). Clearly, the most crucial issue to be raised in discussing the suitability of the decision rules (3.2) and (3.4) concerns the point that in the corresponding test maintenance of the prespecified significance level in the classical sense cannot generally be taken for granted. Since the latter is still considered an indispensable property of any procedure to be relied upon for confirmatory purposes by the majority of the users of statistical methodology [in particular, the regulatory authorities deciding upon the eventual success of drug development projects], it seems advisable to check Bayesian decision rules from a

Bayesian tests for two-sided equivalence

39

frequentist perspective as carefully as possible before adopting them for real applications. Unfortunately, deriving sufficient conditions for the validity of Bayesian tests with regard to the risk of a type-I error is still more difficult a task in the case of equivalence than that of conventional one-sided hypotheses. In fact, the results obtained by Casella and Berger (1987) for the latter cannot be exploited for establishing the validity of Bayesian equivalence tests even if attention is restricted to simple location-parameter problems decomposed into two one-sided problems by replacing (3.2) with (3.4). The reason is that the results proved by these authors require symmetry of the density of the data about the common boundary of null and alternative hypothesis, and it can easily be shown that there exists no proper probability density which is symmetric about two different points [viz., θ◦ − ε1 and θ◦ + ε2 ]. An abstract concept which proves worthwhile as a starting-point for investigations into possible relationships between Bayesian and frequentist tests for equivalence is that of a confidence distribution as introduced by Cox (1958) and exploited for a systematic discussion of the interval inclusion approach to equivalence testing by Mau (1987, 1988). Roughly speaking, a confidence distribution is defined to be a random probability measure [depending in some mathematically clear-cut way on the data under analysis] such that its quantiles give confidence bounds to the parameter θ of interest. Whenever the posterior distribution of θ admits a representation as a realization of a random measure of that type, both versions (3.2) and (3.4) of a Bayesian rule for deciding between equivalence and existence of relevant differences coincide with interval inclusion rules as introduced in the previous section and hence share with them the property of being valid with regard to the type-I error risk. Unfortunately, representability of Bayesian posterior as frequentist confidence distributions turns out to be a highly restrictive condition. In fact, from a practical point of view, it provides hardly more than a characterization in abstract terms of situations where the distributions of the data (typically after suitable reductions by sufficiency) belong either to a location family (Pθ )θ∈IR , a scale family (Pσ )σ>0 , or a location-scale family (Pθ,σ )(θ,σ)∈IR×IR+ , and the prior is the usual reference prior defined by an improper density proportional to 1 or σ −1 , respectively. However, for most models involving only a location or a scale parameter or both, there exist well-established methods of interval estimation and even optimal tests for equivalence are comparatively easy to implement. In view of this, the possibility of establishing the validity of a Bayesian test for equivalence by way of verifying that the posterior distribution involved satisfies the definition of an observed confidence distribution is mainly of theoretical and conceptual interest. Moreover, for purposes of proving the validity of equivalence tests using decision rule (3.4) with a suitable noninformative reference prior, an alternative method exists which avoids detailed comparative inspections of the underlying posterior distributions. It is obtained by exploiting the theory of right-invariant Haar densities and provides, at least potentially, a more versatile tool than the approach via paralleling posterior and confidence distributions, since it covers transforma-

40

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

tion families of any kind (for a detailed exposition of this theory see Berger, 1985, § 6.6). Notwithstanding the considerable difficulties encountered in proving analytically that a Bayesian testing procedure satisfies at the same time basic frequentist criteria, the proportion of Bayesian contributions to the literature on statistical methods for equivalence assessment has been substantial all along. This has especially been true in the field of the analysis of comparative bioavailability trials. A representative selection of influential papers on Bayesian methods of bioequivalence assessment is constituted of Selwyn et al. (1981); Rodda and Davis (1980); Fl¨ uhler et al. (1983); Mandallaz and Mau (1981); Selwyn and Hall (1984) and Racine-Poon et al. (1987) . Furthermore, in later chapters, the reader will become acquainted with a number of instances of Bayesian constructions of tests for equivalence and noninferiority for which ensuring validity is done via direct numerical determination of the maximum rejection probability taken over the respective null hypothesis. In § 10.3, we will even see that the Bayesian approach has the potential of filling gaps in the repertoire of useful frequentist (including asymptotic) solutions to equivalence testing problems of considerable practical relevance.

3.3

The classical approach to deriving optimal parametric tests for equivalence hypotheses

In the classical theory of hypotheses testing, statistical tests are derived as solutions to optimization problems arising from the endeavor to maximize the power over a class of alternative decision procedures chosen as large as possible. Ideally, this class contains the totality of all tests of the null hypothesis under consideration which maintain the prespecified significance level, and maximization of power can be achieved uniformly over the whole subset of the parameter space specified by the alternative hypothesis. In the majority of situations occurring in practice, the class of testing procedures to be taken into consideration must be restricted by means of supplementary criteria to be imposed in addition to that of validity with respect to the type-I error risk. The most useful of them are based on the principle of sufficiency and invariance, respectively. The basic mathematical results which the construction of such optimal testing procedures for two-sided equivalence hypotheses relies upon, are presented in due rigor and generality in § A.1 of the Appendix. Hence, it suffices to give in the present section a brief outline of the basic techniques available for purposes of carrying out optimal constructions of tests for equivalence. As is the case in the treatment of one-sided problems [including problems of testing for noninferiority — recall § 2.2] and conventional two-sided testing

Optimal parametric tests for equivalence hypotheses

41

problems, the natural starting-point of the classical approach to the construction of tests for equivalence is the consideration of settings where the possible distributions of the data form a one-parameter family of sufficiently regular structure. When dealing with interval hypotheses, the suitable way of making this regularity precise is based on the concept of total positivity whose fundamental importance for the theory of statistical decision procedures has been demonstrated by S. Karlin in a series of frequently cited papers and an encyclopedic monograph (Karlin, 1956, 1957a, b, 1968). For purposes of deriving optimal one-sided tests, the assumption of monotone likelihood ratios (or, equivalently, of strict total positivity of order 2) proves mathematically natural. In contrast, in order to ensure the existence of optimal tests for equivalence, families of distributions are required whose densities are STP3 (strictly totally positive of order 3 — see Definition A.1.1 in Appendix A) and depend in a continuous manner both on the realized value of the observed random variable and the parameter θ. The basic fact is then that in any STP3 family exhibiting these additional continuity properties, an optimal test for equivalence in the sense made precise in (1.1. a,b) [→ p. 11] can be constructed by means of the generalized Fundamental Lemma of Neyman and Pearson (1936). Interestingly, this construction whose mathematical basis is given by Theorem A.1.5 of Appendix A, leads to tests which are uniformly most powerful (UMP) among all tests at level α of the same hypothesis whereas for the dual testing problem θ◦ − ε1 ≤ θ ≤ θ◦ + ε2 versus θ < θ◦ − ε1 or θ > θ◦ + ε2 (usually termed the problem of testing for relevant differences by biostatisticians — see Ch. 11) only a uniformly most powerful unbiased (UMPU) solution exists (Lehmann and Romano, 2005, p. 83). As suggested by intuition, every reasonably structured test for equivalence has a form which is complementary to that of a traditional two-sided test. Thus, optimal solutions to some given problem of equivalence testing have to be sought for in the class of tests whose rejection region can be written as   (3.7) x | C1 < T (x) < C2 , which means that the null hypothesis H of nonequivalence has to be rejected as long as the observed value of a suitable real-valued statistic remains within some sufficiently short interval. The limits C1 , C2 of that critical interval may depend, in addition to the significance level α, on the value observed for some other statistic S, say, which will typically be chosen sufficient for a (maybe multidimensional) nuisance parameter. In order to obtain an optimal test, C1 and C2 must be determined in such a way that its (conditional) probability takes on exact value α at both boundaries of the null hypothesis, i.e., under both θ = θ◦ − ε1 and θ = θ◦ + ε2 . In cases where the distributions of the test statistic T (X) are continuous, this implies that C1 , C2 must simultaneously satisfy the equations Pθ1 [C1 < T (X) < C2 ] = α = Pθ2 [C1 < T (X) < C2 ], , C1 , C2 ∈ T , C1 < C2

(3.8)

42

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

where T denotes the range of T (·) and θ1 = θ◦ − ε1 , θ2 = θ◦ + ε2 . If the distribution functions of T (X) exhibit jump discontinuities, a solution to (3.8) generally does not exist. Then, one has to deal with the more complicated system Pθ1 [ C1 < T (X) < C2 ] +

2 

γν Pθ1 [ T (X) = Cν ] = α =

ν=1

Pθ2 [ C1 < T (X) < C2 ] +

2 

γν Pθ2 [ T (X) = Cν ] ,

ν=1

C1 , C2 ∈ T , C1 ≤ C2 , 0 ≤ γ1 , γ2 < 1

(3.9)

instead. The existence of a unique solution to (3.9) is guaranteed whenever the family of distributions of T (X) is STP3 , and the optimal test entails a randomized decision in favor of equivalence with probability γ1 or γ2 if T (X) takes on value C1 or C2 , respectively. As has been pointed out in the previous chapter in connection with exact tests for noninferiority problems relating to discrete families of distributions, in the overwhelming majority of applications of statistical testing procedures to real data randomized decisions between the hypotheses are clearly undesirable. Accordingly, in the noncontinuous case the optimal test for equivalence is usually modified in the following way: The critical interval (C1 , C2 ) has still to be determined by solving (3.9) but at its boundaries, i.e., both for T (X) = C1 and for T (X) = C2 , the null hypothesis is always accepted (corresponding to the specification γ1 = 0 = γ2 ). In situations where the distributions of T (X) are discrete and at the same time conditional on some other statistic sufficient for the nuisance parameter(s) involved in the underlying model [→ §§5.1, 6.6.5, 9.4.2], adopting the technique of raised nominal significance levels introduced by Boschloo (1970) [recall §2.5] will prove a simple and effective device for reducing the conservatism entailed by accepting nonequivalence whenever we observe that T (X) ∈ {C1 , C2 }. In the continuous case, numerical determination of the optimal critical constants C1 , C2 is greatly facilitated by the availability of a general result (proved by Lehmann and Romano, 2005, as Lemma 3.4.2(iv)) on the sign of the difference of the power functions of two tests of the form (3.7) in families with monotone likelihood ratios, which implies the following fact: Let the family of distributions of T (X) be STP3 and (C1 , C2 ) denote any interval on the real line such that Pθ1 [C1 < T (X) < C2 ] = α but Pθ2 [C1 < T (X) < C2 ] < or > α. Then it follows that the optimal critical interval C1 , C2 whose endpoints satisfy (3.8) lies to the right or left of (C1 , C2 ), respectively. Clearly, this shows that a simple iteration scheme for approximating the solution of (3.8) to any desired degree of numerical accuracy, consists of the following steps: (i) Choose some estimate C10 , say, of the left-hand limit C1 of the optimal critical interval.

Optimal parametric tests for equivalence hypotheses

43

 α+ (ii) Compute the corresponding upper critical bound C20 as C20 = Fθ−1 1  Fθ1 (C10 ) , with Fθ and Fθ−1 denoting the distribution and quantile function of T (X), respectively, at an arbitrary point θ in the parameter space.   0 0 0 0 = F (C )−F (C ) of the critical region x  C1 < (iii) Compute the size α θ θ 2 2 1 2 2  0 T (x) < C2 at θ = θ2 . (iv) Update C10 by replacing it with (C˜1 + C10 )/2 or (C10 + C˜1 )/2 depending on whether there holds α20 > α or α20 < α and assuming that (C˜1 , C˜1 ) has been previously determined as an interval containing both C10 and the exact solution C1 . (v) Repeat steps (i)–(iv) as many times as required in order to ensure that |α20 − α| < TOL with, e.g., TOL = 10−4 . In the discrete case with integer-valued test statistic T (X), extensive experience has shown that the following search algorithm (forming the basis of the programs bi1st, powsign, bi2st, bi2aeq1 – bi2aeq3 and gofhwex) works very efficiently on finding the solution of (3.9): (i∗ ) Choose an initial value C10 for the lower critical bound known to be greater or equal to the correct value C1 . (ii∗ ) Keeping C10 fixed, find the largest integer C20 > C10 such that the probability of observing T (X) to take on its value in the closed interval [C10 + 1, C20 − 1] does not exceed α, neither for θ = θ1 nor for θ = θ2 . (iii∗ ) Treat (3.9) as a system of linear equations in the two unknowns γ1 , γ2 and compute its solution γ10 , γ20 , say (which of course depends on C10 and C20 ). (iv*) Verify the condition 0 ≤ γ10 , γ20 < 1. If it is satisfied, the solution of the full system (3.9) is found and given as (C1 , C2 , γ1 , γ2 ) = (C10 , C20 , γ10 , γ20 ); if not, diminish C10 by 1 and repeat steps (ii*) and (iii*).

Although both of the algorithms outlined above are logically straightforward, concrete numerical determination of the optimal critical constants is a much more demanding task for equivalence problems than for one- or ordinary two-sided testing problems. This explains why in the existing literature contributions presenting classical tests for equivalence as fully explicit decision rules ready for being used by the working statistician, have been sparse for several decades. By the time of appearance of the first edition of this book, the following list has been largely exhaustive: Bondy (1969); Wellek

44

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

and Michaelis (1991); Wellek (1993a); Mehring (1993); Wellek (1994); Wang (1997). Perhaps, the main fact making optimal tests for equivalence (as well as those for noninferiority) computationally more complicated than their traditional one- and two-sided analogues, is that the sampling distributions of customary test statistics are needed in their noncentral versions in order to determine sizes of rejection regions etc. An additional complication refers to situations entailing elimination of nuisance parameters by means of conditioning: The possibility of reducing the conditional tests primarily obtained to equivalent nonconditional procedures, which is well known (cf. Lehmann and Romano, 2005, § 5.1) to lead not infrequently to considerable simplifications with one- and two-sided problems, generally does not exist in the equivalence case. Nevertheless, in subsequent chapters the reader will become acquainted with numerous specific equivalence testing problems for which an optimal procedure can be made available in a form suitable for routine use by anybody having access to standard computational tools. The computational effort entailed by performing an optimal test for equivalence in practice, reduces to a minimum in all cases where the following bipartite symmetry condition is satisfied: θ1 = −ε, θ2 = ε for some ε > 0 ; the distribution of T (X) under θ = ε coincides with that of − T (X) under θ = −ε .

(3.10a) (3.10b)

In fact, by the result made precise and proved in the Appendix as Lemma A.1.6, (3.10) implies that the critical interval for T (X) defining an optimal test for equivalence is symmetric about zero, i.e., of the form (−C, C). In the continuous case, C remains as the only critical constant uniquely determined by the single equation Pε [|T (X)| < C] = α,

C ∈ T ∩ (0, ∞).

(3.11)

An alternative formulation of the same fact is as follows: If, in a situation being symmetric in the sense of (3.10), there exists an optimal test for equivalence, then this can be carried out by means of a p-value defined as the lower tail probability of the absolute value of the test statistic under θ = ε or, equivalently, under θ = −ε. Dropping the continuity assumption, one has γ1 = γ2 = γ in addition to −C1 = C2 = C where C and γ are defined by 

  C = max t ∈ T ∩ (0, ∞)  Pε [ |T (X)| < t ] ≤ α , (3.12a)

 γ = α − Pε [ |T (X)| < C ] Pε [ |T (X)| = C ] .

(3.12b)

In particular, an optimal test for equivalence allows this kind of reduction to a one-sided test in terms of the absolute value of the respective test statistic,

Construction of asymptotic tests for equivalence

45

whenever θ plays the role of a location parameter for the distribution of T (X), d provided θ = 0 implies −T (X) = T (X) and we are willing to choose the equivalence range for θ as a symmetric interval. The most important special case of such a location problem occurs if T (X) is normally distributed with unknown expected value θ ∈ IR and fixed known variance σ◦2 > 0 which will be treated in full detail in § 4.1. The greater part of the equivalence assessment procedures described in the subsequent chapters are tests which maximize the power function uniformly over the class of all valid tests for the same problem, of invariant tests maintaining the prespecified level α, of all unbiased tests at level α, or of all valid tests for an associated asymptotic problem. Although optimal testing procedures generally deserve preference over less powerful tests of the same hypotheses, in providing a sufficiently rich repertoire of equivalence testing procedures unification of the constructional approach seems neither desirable nor promising. In fact, there are many rather elementary equivalence testing problems encountered very frequently in practice which admit no optimal solution. Perhaps, the best known and most important of them refers to location equivalence of two Gaussian distributions N (μ1 , σ 2 ), N (μ2 , σ 2 ) with common unknown variance. Here, even the construction of an unbiased test for |μ1 − μ2 | ≥ ε vs. |μ1 − μ2 | < ε as carried out by Brown et al. (1997) proves mathematically quite complicated and leads to rejection regions which in view of several counterintuitive properties seem hardly suitable for practice (for critical discussions of this procedure see Hauck and Anderson, 1996; Meredith and Heise, 1996; Schuirmann, 1996; Perlman and Wu, 1999). Although the interval inclusion rule with standard equal-tails confidence intervals at twosided level (1 − 2α) gives a solution to this specific problem which still can be strongly recommended, it would be clearly misleading to suppose that the principle of confidence interval inclusion is generally easier to implement than the classical approach to constructing equivalence tests. One out of many other important examples showing this not to hold true occurs if, in the setting of the ordinary two-sample t-test, equivalence of N (μ1 , σ 2 ) and N (μ2 , σ 2 ) is redefined by requiring that we should have |μ1 − μ2 |/σ < ε : Computing exact confidence limits for the difference between the standardized means of these distributions is certainly at least as complicated as determining the critical constant of the uniformly most powerful invariant (UMPI) test for the same problem as discussed in § 6.1.

3.4

Construction of asymptotic tests for equivalence

In § 2.3 it has been pointed out with regard to noninferiority testing that the range of problems accessible to exact methods is much too limited for cover-

46

APPROACHES TO TESTING FOR TWO-SIDED EQUIVALENCE

ing the needs arising in practice. This statement applies at least as much to the case of two-sided equivalence testing. In this section, we will show that an asymptotic approach which enables one to derive solutions for settings in which exact methods are not in sight, is also available for problems of testing for equivalence in the strict sense. However, as compared to noninferiority problems, equivalence problems turn out to be considerably more complicated to solve also in an asymptotic framework. The theoretical basis of the asymptotic construction of tests for equivalence is given by a result which has been established for the two-sample case by Wellek (1996). In the Appendix [→ § A.3], it is rigorously shown to generalize in a way allowing us to cover virtually any situation where the equivalence hypothesis to be tested refers to a parameter or (in non- or semiparametric models) to a functional for which an asymptotically normal estimator can be found. In what follows, we give an informal yet exhaustive description of the general form of an asymptotic test for equivalence. As in § 2.3, we start from the basic assumption that for the target parameter (or functional) θ in terms of which the equivalence hypothesis under √ consideration is formulated, there exists an estimator (TN )N ∈ IR such that N (TN −θ) is asymptotically normal with zero mean and variance σ 2 . Again, N denotes the total number of oberservations making up the data set under analysis, and σ 2 is another functional of the vector (F1 , . . . , Fk ) of underlying distribution functions taking on only positive values. Now, the basic steps leading to an asymptotically valid test of H : θ ≤ θ1 or θ ≥ θ2 versus K : θ1 < θ < θ2 are as follows: (i) Take (2.5) as if allowing to be strengthened to a statement on the exact sampling distribution of TN . √ (ii) Treat the standard error σ/ N of TN as a known constant τN , say, and carry out in terms of TN the test which is optimal for H vs. K when θ stands for the expected value of a normal distribution with known 2 from which a single observation has been taken variance τN (iii) Derive an √ estimator σ ˆN which is weakly consistent for σ and plug in ˆN / N in the expressions obtained in step (ii) at every place τˆN = σ where τN appears. The results derived in § 4.1 strongly suggest to use in step (ii) of such a construction the rejection region

    TN − (θ1 + θ2 )/2/τN < Cα (θ2 − θ1 )/2τN (3.13) where for any ψ > 0 and 0 < α < 1, Cα (ψ) has to be taken equal to the square root of the α-quantile of a χ2 -distribution with a single degree of freedom and noncentrality parameter ψ 2 . Since the computation of quantiles of noncentral χ2 -distributions is a process perfectly supported by the most widely used statistical software (see the intrinsic function cinv of SAS or the qchisq function

Construction of asymptotic tests for equivalence

47

of the R Stats Package), the practical implementation of an asymptotic test for equivalence on the basis of (3.13) is an easy exercise as soon as the test statistic TN as well as its estimated asymptotic standard error τˆN has been calculated. What remains to be done is to determine the ordinary distance in units of τˆN between TN and the middle of the equivalence limits  specified by (θ2 − θ1 )/2ˆ the hypothesis, and compare the result with the number C α   τN τN as a critical upper bound. If the observed value of TN − (θ1 + θ2 )/2/ˆ is found to be smaller than this bound, nonequivalence can be rejected, and has to be accepted otherwise. Of course, replacing the theoretical standard error τN with its consistent estimator τˆN in determining thenoncentrality pa τN a rameter of the χ2 -distribution to be relied upon, makes Cα (θ2 − θ1 )/2ˆ random critical bound rather than a critical constant. Nevertheless, very mild conditions suffice to guarantee that the approach yields a test for equivalence which is asymptotically valid with respect to the significance level. Usually, the most laborious step to be taken in deriving an asymptotic solution to a concrete equivalence testing problem on the basis of the theory outlined in this section, is to find an explicit expression for the asymptotic standard error of the test statistic as well as for a suitable estimator of it. Except for cases in which the complications entailed in variance calculation prove prohibitive, the approach is very useful. One of its nicest and most important properties stems from the finite-sample behavior of the corresponding testing procedures which according to the results of extensive simulation studies often turns out to be remarkably good. As one out of not a few noteworthy instances of this, the reader is referred to the log-rank test for equivalence of two survivor functions described in detail in § 6.7 . Even if the expected proportion of censored observations contained in the data is about 50% and both sample sizes are as small as 25, the risk of committing a type-I error does not exceed the nominal significance level to a practically relevant extent.

4 Equivalence tests for selected one-parameter problems

4.1

The one-sample problem with normally distributed observations of known variance

In this section it is assumed throughout that the hypotheses (1.1.a) and (1.1.b) [→ p. 11] refer to the expected value of a single Gaussian distribution whose variance has some fixed known value. In other words, we suppose that the data under analysis can be described by a vector (X1 , . . . , Xn ) of mutually independent random variables having all distribution N (θ, σ◦2 ) where σ◦2 > 0 denotes a fixed positive constant. It is easy to verify that it entails no loss of generality if we set σ◦2 = 1, θ◦ − ε1 = −ε and θ◦ + ε2 = ε with arbitrarily fixed ε > 0. In fact, if the variance of the primary observations differs from unity and/or the equivalence interval specified by the alternative hypothesis (1.1.b) fails to exhibit symmetry about zero, both of these properties can be ensured by applying the following simple transformation of the observed variables: Xi = (Xi −θ◦ −(ε2 −ε1 )/2)/σ◦ . In view of the relations Eθ◦ −ε1 (Xi ) = (θ◦ − ε1 − θ◦ − (ε2 − ε1 )/2)/σ◦ = −(ε1 + ε2 )/2σ◦ , Eθ◦ +ε2 (Xi ) = (ε1 + ε2 )/2σ◦ , it makes no difference whether we use the original sample to test θ ≤ θ◦ − ε1 or θ ≥ θ◦ + ε2 versus θ◦ − ε1 < θ < θ◦ + ε2 , or base a test of the null hypothesis θ  ≤ −ε or θ  ≥ ε versus the alternative −ε < θ  < ε on the transformed sample (X1 , . . . , Xn ), provided we define θ  = E(Xi ) and ε = (ε1 + ε2 )/2σ◦ . To simplify notation, we drop the distinction between primed and unprimed symbols and assume that we have Xi ∼ N (θ, 1), i = 1, . . . , n

(4.1a)

and that the equivalence problem referring to these observations reads H : θ ≤ −ε or θ ≥ ε

K : −ε < θ < ε . (4.1b) n ¯ Since (4.1a) implies √ ¯ that the sample mean X = (1/n) i=1 Xi and hence also the statistic nX is sufficient for the family of the joint distributions of (X1 , . . . , Xn ), we may argue as follows (using a well-known result √ ¯ to be found, differs from e.g., in Witting, 1985, Theorem 3.30): The distribution of nX versus

49

50

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

that of √ an individual Xi only by its expected value which is of course given ˜ the testing problem put forward above reads by θ˜ = nθ. In terms of θ, ˜ : θ˜ ≤ −˜ ˜ : −˜ H ε or θ˜ ≥ ε˜ versus K ε < θ˜ < ε˜ , (4.2) √ on the understanding that we choose ε˜ = nε. Now, if there is an UMP test, say ψ˜ at level α for (4.2) based on a single random variable Z with ˜ 1) , Z ∼ N (θ,

(4.3)

then a UMP level-α test for the original problem (4.1) is given by φ(x1 , . . . , ˜ −1/2 n xi ), (x1 , . . . , xn ) ∈ IRn . xn ) = ψ(n i=1 By (4.3), the possible distributions of Z form a one-parameter exponential family in θ˜ and Z (cf. Bickel and Doksum, 2001, § 1.6.1), and by Lemma A.1.2 [→ Appendix, p. 369] this is a specific example of a STP3 family. Moreover, the density functions involved obviously exhibit the continuity properties required in the assumptions of Theorem A.1.5 [→ Appendix, p. 371], and in view of the symmetry of the standard Gaussian distribution N (0, 1), the distribution of Z under θ˜ = ε˜ is clearly the same as that of −Z under θ˜ = −˜ ε. Thus, applying Lemma A.1.6 [→ p. 372] we can conclude that a UMP level-α test for (4.2) is given by  1 for |z| < Cα; ε˜ ˜ , (4.4) ψ(z) = 0 for |z| ≥ Cα; ε˜ where Cα; ε˜ the unique solution of Φ(C − ε˜) − Φ(−C − ε˜) = α ,

C > 0.

(4.5)

The expression on the left-hand side of this equation equals the probability of  the event |Zε˜| ≤ C with Zε˜ ∼ N (˜ ε, 1). But Zε˜ ∼ N (˜ ε, 1) implies that Zε˜2 satisfies the definition of a variable whose distribution is noncentral χ2 with a single degree of freedom (df ) and noncentrality parameter ε˜2 (cf. Johnson et al., 1995, § 29.1). Hence, the solution to equation (4.5) determining the critical constant of a UMP level-α test, admits the explicit representation  Cα; ε˜ = χ21; α (˜ ε2 ) , (4.6) where χ21; α (˜ ε2 ) denotes the α-quantile of a χ2 -distribution with df = 1 and noncentrality parameter ε˜2 . Finally, the desired optimal test for the problem (4.1) we started from, is obtained by applying the decision rule Reject H : θ ≤ −ε or θ ≥ ε in favor of K : −ε < θ < ε ¯ < n−1/2 Cα; √nε . if and only if it can be verified that |X|

(4.7)

In√(4.7), the critical constant has to be determined simply by plugging in ε˜ = nε into (4.5) or (4.6), and constructing a program which computes Cα; ε˜

The one-sample problem with data from N (θ, σ◦2 )

51

for any ε˜ > 0 is extremely easy whenever a programming language is used in terms of which the standard normal cdf Φ(·) is predefined either directly √ z 2 or, as in Fortran, via the so-called error integral erf(z) ≡ (2/ π) 0 e−y dy. In fact, as a function of C, Φ(C − ε˜) − Φ(−C − ε˜) is continuous and strictly increasing. Hence, as soon as Φ(·) can be treated as explicitly given, no more than an elementary interval halving algorithm is needed for computing Cα; ε˜ by means of (4.5) to any desired degree of numerical accuracy. Of course, the most convenient way of implementing the optimal test for equivalence of N (θ, 1) to N (0, 1) on the basis of a single sample (X1 , . . . , Xn ) from N (θ, 1) is to use a software package which provides the inverse noncentral χ2 cdf with df = 1 as a predefined function (in SAS, the name of this function is cinv ; in R, one has to call the function qchisq). For the conventional level α = .05 of significance, Table 4.1 gives a tabulation of Cα; ε˜ on the grid ε˜ = .1(.1)4.9. Table 4.1 Optimal critical constant Cα; ε˜ [→ (4.5), (4.6)] for equivalence testing at the 5% level in the one-sample case with normally distributed data of known variance. ε˜ .0 .1 .2 .3 .4 .5 .6 .7 .8 .9

0. – 0.06302 0.06397 0.06559 0.06793 0.07105 0.07507 0.08011 0.08634 0.09398

1. 0.10332 0.11470 0.12859 0.14553 0.16624 0.19157 0.22255 0.26032 0.30608 0.36085

2. 0.42519 0.49889 0.58088 0.66944 0.76268 0.85893 0.95696 1.05598 1.15552 1.25530

3. 1.35521 1.45517 1.55516 1.65515 1.75515 1.85515 1.95515 2.05515 2.15515 2.25515

4. 2.35515 2.45515 2.55515 2.65515 2.75515 2.85515 2.95515 3.05515 3.15515 3.25515

√ ¯ Figure 4.1 shows the density functions of the test statistic nX at both boundaries of the hypotheses and the critical interval of the optimal test at level α = .05 for the case that the sample consists of n = 100 data points and the constant ε defining the width of the equivalence range for the expectation θ of the underlying distribution has been chosen to be ε = .25. As becomes obvious from the graph, the area under both curves over the critical interval (−Cα; √nε , Cα; √nε ) = (−.85893, .85893) is exactly equal to .05 = α. According to Theorem A.1.5, this property characterizes the optimal test.

52

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

√ √ Figure 4.1 Density of the distributions N (− nε, 1) [left] and N ( √ nε, 1) [right] ¯ in the and critical interval [horizontal bar centered about 0] for Z = nX UMP test at level α = .05, with ε = .25 and n = 100. Figure 4.2 shows, for the same values of n, ε and α, the power of the UMP test for (4.1) as a function of the expectation θ of the individual observations X1 , . . . , Xn [recall (4.1a)]. In accordance with the characterization of the form of the power functions of optimal tests for equivalence in symmetric cases given in general terms in Corollary A.1.7 [→ pp. 372–3], the curve is strictly unimodal and symmetric about zero. From a practical point of view, perhaps the most striking conclusion to be drawn from this picture is the following: In contrast to tests for traditional one- or two-sided problems, equivalence tests do not admit the possibility of increasing the power to values arbitrarily close to 100% simply by selecting sufficiently extreme points in the parameter subspace corresponding to the alternative hypothesis under consideration. The implications of this fact for sample size requirements to be satisfied in equivalence trials are obvious and far-reaching enough. ∗ If we denote for arbitrary values of n, ε and α by βn,ε; α the ordinate of the peak of the power curve associated to the test (4.7), there holds the simple explicit formula ∗ √ (4.8) βn,ε; α = 2 Φ(Cα; nε ) − 1 .

The one-sample problem with data from N (θ, σ◦2 )

53

Figure 4.2 Power function of the UMP test at level α = 5% for |θ| ≥ .25 vs. |θ| < .25 based on n = 100 observations from N (θ, 1). [Bold-drawn bar on horizontal coordinate axis ↔ equivalence interval (−ε, ε = (−.25, .25) specified by the alternative hypothesis.] This implies in particular that the maximum power attainable in any test√for the equivalence problem (4.1) likewise depends on n and ε only through nε = ε˜. Table 4.2 shows, for the same selection of values of ε˜ covered by the preceding √ table and again to 5 decimal places, the result of evaluating (4.8). For ε˜ = 100·.25 = 2.50, we read the number .60962 which is the precise value of the maximum ordinate of the curve plotted in Figure 4.2. Conversely, Table 4.2 can be used for obtaining a rough estimate of the sample size required for an equivalence trial to be analyzed in a confirmatory manner by means of (4.7). E.g., the smallest ε˜ guaranteeing a maximum power of 80% is seen to lie between 2.9 and 3.0. Thus, for a trial in which we aim at establishing equivalence in the sense of −.25 < θ < .25, we need a sample of size (2.9/.25)2 = 134.56 < n < 144.00 = (3.0/.25)2 in order to detect coincidence of θ with zero with a power of 80%, and so on.

54

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS Table 4.2 Maximum power of the UMP test at level α = .05 for the one-sample equivalence problem with Gaussian data of unit √ variance, for the same grid of values of ε˜ = nε as covered by Table 4.1. ε˜ .0 .1 .2 .3 .4 .5 .6 .7 .8 .9

0. – .05025 .05101 .05230 .05416 .05665 .05984 .06385 .06880 .07488

1. .08229 .09132 .10232 .11571 .13203 .15192 .17611 .20538 .24046 .28179

2. .32930 .38214 .43868 .49679 .55434 .60962 .66141 .70902 .75212 .79063

3. .82465 .85438 .88009 .90211 .92077 .93642 .94943 .96014 .96885 .97588

4. .98148 .98592 .98939 .99207 .99413 .99570 .99687 .99775 .99840 .99887

To be sure, as a testing problem per se, the one-sample problem with normally distributed observations of known variance is of limited practical importance since in real applications precise knowledge of the true value of σ 2 will rarely be available. However, the testing procedure (4.7) which has been shown above to give a UMP solution to this problem, merits due consideration by the fact that it forms the basis of the construction of asymptotically valid tests for equivalence described in general terms in § 3.4 . Recalling the form of the rejection region of such an asymptotic test [→ (3.13)], it is readily checked that the latter is obtained from the results of the present section by formally treating the test statistic TN as a single observation from a normal 2 =σ ˆ 2 /N [→ p. 46 (iii)] and expected value θ = distribution with variance τˆN θ(F1 , . . . , Fk ) . With respect to the existing literature, it is worth noticing that in a paper going back as far as 40 years, Bondy (1969) suggested an appropriately modified version of (4.7) as a solution to the problem of testing for equivalence of two Gaussian distributions with known common variance. Bondy’s contribution has been almost completely ignored by later authors in the field of equivalence testing (for one of the very few exceptions see Chester, 1986). In particular, although having appeared 3 years earlier than the seminal paper on the confidence interval inclusion approach (Westlake, 1972), Bondy’s idea of having recourse to the classical theory of hypotheses testing has never been referenced by any one of the pioneers of the statistical methodology of bioequivalence assessment. One of the reasons why his paper has been left largely unnoticed in the pertinent literature of the years to come might have been that Bondy gave an unnecessarily complicated representation of the critical constant of his test making it appear hardly suited for applications in routine

The case of exponentially distributed survival times

55

work. Especially, any hint is missing that the optimal critical constant Cα; ε˜ [→ (4.6)] can be computed by means of the quantile function of a noncentral χ2 -distribution with df = 1.

4.2

Test for equivalence of a hazard rate to some given reference value with exponentially distributed survival times

Let X denote a random variable whose density function (with respect to Lebesgue measure on IR) is given by fX (x; σ) = (1/σ) exp{−x/σ} ,

x > 0.

(4.9)

As is well known (cf. Johnson et al., 1994, § 19.4), under this model the scale parameter σ > 0 equals the expected value of X. In the sequel, we mostly use the short-hand notation X ∼ E(σ) in order to indicate that X is a random variable having density (4.9). In the present section, it is our purpose to construct an optimal test of H: σ ≤ σ1 or σ ≥ σ2 versus K: σ1 < σ < σ2 (0 < σ1 < σ1 < ∞) based on a sample (X1 , . . . , Xn ) of mutually independent observations from E(σ). The interval (σ1 , σ2 ) specified by that alternative hypothesis K is supposed to contain a point σ◦ serving as the target or reference value of the only parameter appearing in the model under consideration. As is the case in the one-sample problem with normally distributed data of known variance [recall the introductory paragraph of §4.1], it entails no loss of generality to assume that the interval corresponding to K has a normalized form which this time means that we have σ1 = 1/(1 + ε), σ2 = 1 + ε for some ε > 0. In order to induce such a normalization with respect to the form of the equivalence hypothesis, we have simply to apply the following transformation of the individual observations and the parameter, respectively: √ Xi = Xi / σ1 σ2 , i = 1, . . . , n ;

√ σ  = σ/ σ1 σ2 .

(4.10)

This follows from the obvious fact that for Xi ∼ E(σ), the distribution of Xi is  likewise  exponential but with scale parameter σ instead of σ. Moreover, with ε = σ2 /σ1 − 1, the condition σ1 < σ < σ2 is of course logically equivalent to 1/(1 + ε) < σ  < (1 + ε). In view of these relationships, we suppose in analogy to (4.1) that the hypotheses making up our testing problem have been proposed in the form H : σ ≤ 1/(1 + ε) or σ ≥ 1 + ε versus K : 1/(1 + ε) < σ < 1 + ε

(4.11a)

56

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

from the start (with fixed ε > 0), and the data set to be analyzed consists of n mutually independent observations X1 , . . . , Xn such that Xi ∼ E(σ) , i = 1, . . . , n .

(4.11b)

Now, the distributional assumption (4.11b) is well known to imply that a statistic sufficient for  the class of joint distributions of the sample (X1 , . . . , n Xn ) is given by T = i=1 Xi . The sufficient statistic T can be shown (see, e.g., Feller, 1971, p. 11) to follow a gamma distribution with shape parameter n and scale parameter σ, given by the density function Γ (t) = (1/Γ(n))σ −n tn−1 exp{−t/σ} , t > 0 . fσ,n

(4.12)

Since n denotes a known constant whereas σ is allowed to vary over the whole positive half-axis, (4.12) is an element of a one-parameter exponential family in t and θ = −1/σ. According to Lemma A.1.2 [→ p. 369], each such family is STP∞ and thus a fortiori STP3 . Furthermore, it is readily verified that the Γ (t) is continuous in both of its arguments. In view of function (t, σ) → fσ,n Theorem A.1.5 [→ p. 371] and the sufficiency of T , we are justified to infer from these facts that the decision rule Reject H

if and only if

1 Cα; n,ε
0 and n ∈ IN) the cdf corresponding to the density function (4.12). Unfortunately, notwithstanding the possibility of symmetrizing the hypotheses in the way having lead to (4.11a), the one-sample setting with exponentially distributed data does not satisfy condition (3.10) for symmetrization of the optimal critical interval. In other words, the system (4.14) does not admit reduction to a single equation involving the right-hand critical bound only. Consequently, implementation of (4.13) requires use of the iteration algorithm described in general terms on pp. 42-3. In order to adapt this computational scheme to the specific setting under discussion, using C1◦ = n as a starting value of the left-hand critical bound C1 to T is a sensible choice because T /n is a consistent estimator of σ and the interval of values of σ specified by K contains unity. Generally, if a UMP test for equivalence is based on a statistic which consistently estimates the parameter to be assessed, the critical region will be a proper subset of the interval corresponding to the alternative hypothesis sharing basic formal properties with the latter. Hence, we can take

The case of exponentially distributed survival times

57

it for granted that the solution (C1 , C2 ) to (4.14) gives, after rescaling by the factor 1/n, an interval covering the point σ = 1. Thus, we have C1 < n < C2 so that the initial value C1◦ = n is known to lie to the right of the left-hand boundary C1 of the critical interval to be determined. In accordance with the relationship C1 < n, the program supplied in the WKTSHEQ2 Source Code Package under the name exp1st for computing the solution to (4.14), runs through a sequence of preliminary steps diminishing C1◦ by τ = .05 n each Γ Γ (C2◦ ) − F1+ε,n (C1◦ ) ≤ α, with C2◦ computed until it occurs that we have F1+ε,n 

−1  Γ Γ (C1◦ ) . The point where this condition as C2◦ = F1/(1+ε),n α + F1/(1+ε),n is met, is chosen as the value of C˜1 in the sense of part (iv) of the general description given on pp. 42-3, and C˜1 + τ as the corresponding upper bound C˜1 to C1 . Finally, successive interval halving is applied to (C˜1 , C˜1 ) in order to approximate the exact value of C1 to the desired degree of accuracy, again as explained on p. 43, (iv). 1 2 In addition to the optimal critical constants Cα; n,ε , Cα; n,ε [to resume the explicit notation introduced in (4.13)], the program named exp1st also computes the power of the test established above against the specific alternative σ = 1. To be sure, this is not the point in the parameter space where the power function β(·), say, of the UMP level-α test for the equivalence problem made precise in (4.11) takes on its exact maximum. But by the results to be found in § 4.4 we know that the difference between β(1) and maxσ>0 β(σ) is asymptotically negligible (as n → ∞). Table 4.3 gives the critical constants 1 2 Cα; n,ε , Cα; n,ε for the level α = .05, ε ∈ {.30, .50}, and sample sizes ≤ 120 being a multiple of 10. The corresponding values of the power attained at σ = 1 are displayed in Table 4.4. Example 4.1 In a retrospective study involving n = 80 patients, it was one of the aims to rule out the possibility that a particular drug used in the long-term treatment of progressive chronic polyarthritis (PCP) changes the mean life span of erythrocytes to a relevant degree. For a reference population, the expected length of the average erythrocytal life cycle was known to be 3.84 ≈ 4 months. Let the distribution of the intraindividual mean erythrocytal life span be of the form (4.9) and choose the equivalence interval for σ as centered about 4, with length  being equal to 2. In the sample recruited for the study a total time of ni=1 Xi = 329.7873 was obtained. Rescaling the originally measured time values limits according to (4.10) yields T =  √ as well as the equivalence 329.7873/ 15 = 85.1507 and ε = 5/3−1 = 0.2910. Rounding ε up to .30 we obtain from Table 4.3 (73.22150 , 85.80868) as the critical interval to be used in (4.13). Since the observed value of T happens to be an inner point of this interval, we can decide in favor of equivalence between study and reference population with respect to mean erythrocytal life span.

58

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS 1 2 Table 4.3 Critical interval ( C.05; n,ε , C.05; n,ε ) of the UMP equivalence test at the 5% level for the one-sample setting with exponentially distributed data, for ε = .30, .50 and n = 10(10)120.

n

ε = .30

ε = .50

10 (9.610923 , 10.167879) 20 (19.227987 , 20.331723) 30 (28.728536 , 30.622505) 40 (38.059577 , 41.104285) 50 (47.173320 , 51.843905) 60 (56.048597 , 62.887533) 70 (64.711420 , 74.228599) 80 (73.221500 , 85.808680) 90 (81.635963 , 97.555833) 100 (89.991186 , 109.414666) 110 (98.306216 , 121.350753) 120 (106.590707 , 133.344176)

(9.302916 , 10.179687) (18.175346 , 20.878033) (26.300336 , 32.650207) (33.955275 , 45.339863) (41.447117 , 58.454257) (48.855774 , 71.779509) (56.204318 , 85.244644) (63.505485 , 98.817061) (70.767971 , 112.476582) (77.998090 , 126.208916) (85.200605 , 140.003340) (92.379217 , 153.851535)

Table 4.4 Power attained at σ = 1 when using the critical intervals shown in Table 4.3. n

ε = .30

ε = .50

10 20 30 40 50 60 70 80 90 100 110 120

.070340 .098785 .138156 .191426 .260318 .342885 .432381 .520229 .600305 .670069 .729339 .778984

.111904 .240904 .442380 .635809 .773372 .862271 .917672 .951433 .971657 .983607 .990588 .994629

The binomial one-sample problem

59

Remark on testing for dispersion equivalence of a single sample of normally distributed data. Suppose that the distribution which the random sample (X1 , . . . , Xn ) has been drawn from is Gaussian with expectation μ and variance σ 2 (with both parameters being unknown) and that our main interest is in establishing dispersion equivalence of this distribution to some reference distribution of the same form whose variance is known to equal σ◦2 . For convenience, let us formulate the equivalence hypothesis in this case as K : σ12 < σ 2 < σ22 with (σ12 , σ22 ) as a suitably chosen fixed interval covering the reference value σ◦2 . Applying the same transformation as before [recall (4.10)] and redefining ε as an abbreviation for σ2 /σ1 − 1 rather than  σ2 /σ1 − 1 leads to the problem of testing H : σ 2 ≤ 1/(1 + ε) or σ 2 ≥ 1 + ε versus K : 1/(1 + ε) < σ 2 < 1 + ε (4.15a) with a sample (X1 , . . . , Xn ) of mutually independent observations satisfying Xi ∼ N (μ, σ 2 ) , i = 1, . . . , n .

(4.15b)

Now, it is easy to verify that the problem (4.15) remains invariant under arbitrary common translations Xi → Xi + c of all individual observations contained in the sample. Furthermore, it is well known (see Lehmann and Romano, 2005, p. 220) that any test remaining invariant under all transformations of that depend on the data only through the sum of squares n type, must ¯ 2 or, equivalently, T = (1/2) · SQX . But under the SQX = i=1 (Xi − X) assumption of (4.15b) the latter statistic has a distribution whose density is precisely as defined in (4.12) with σ and n replaced by σ 2 and n = (n − 1)/2, respectively (cf. Stuart and Ord, 1994, p. 385). Hence, in the one-sample setting with normally distributed data, a UMP invariant test for dispersion equivalence in the sense of (4.15a) is obtained simply by comparing the ob1 2 served value of T = (1/2) · SQX with the critical bounds Cα; n ,ε , Cα; n ,ε to be computed in exactly the same way as in the case of exponentially distributed survival times. In particular, the critical interval (73.22150 , 85.80868) computed in the above example for the sum of n = 80 observations from E(σ) can also be used for testing fordispersion equivalence of N (μ, σ 2 ) to N (μ, 1) in 3/5 < σ 2 < 5/3 with a sample of size n = 2n + 1 = 161 and the sense of n ¯ 2 as the test statistic. (Note that the program exp1st (1/2) · i=1 (Xi − X) works also for arbitrary noninteger values of the constant n).

4.3

Testing for equivalence of a single binomial proportion to a fixed reference success probability

Both specific equivalence testing problems treated in the first two sections of this chapter refer to distributions of the absolutely continuous type. In contrast, the present section deals with the most extreme case of a one-sample

60

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

setting with data following a discrete distribution. More precisely speaking we now assume that each of the n mutually independent observations X1 , . . . , Xn takes on only two different values, say 0 (↔ “no response”) and 1 (↔ “response”) with probability p and 1 − p, respectively. Once more, the (unweighted) sum of all individual observations turns out to be sufficient for the class of the possible  distributions of the vector (X1 , . . . , Xn ). Of course, the distribution of T = ni=1 Xi is now given by the probability mass function   n t b(t; n, p) = p (1 − p)n−t , t ∈ {0, 1, . . . , n} . (4.16) t   Likewise, it is easily seen that the corresponding family b(· ; n, p) 0 −ε and √ x n¯ x be satisfied. Clearly, this holds true if and only if n is a point in the open √ ). In other√words, the interval inclusion interval with limits ∓( nε − u1−α√ xn | < nε − u1−α . Accordingly, its test rejects if and only if we have n|¯ rejection region is exactly of the same form as that of the optimal level-α test [ → (4.7)], with the sole difference that the critical constant Cα; √nε as defined

Asymptotic optimality of interval inclusion tests

65

√ by (4.6) is replaced by nε − u1−α . Trivially, the equation determining Cα; √nε according to (4.5) can be rewritten as √ √ (4.23) Φ(c − nε) + Φ(c + nε) = 1 + α , 0 < c < ∞ . √ Since ε stands for a positive number, one has of√course nε → +∞ (as √ n → ∞) and consequently c − nε → −∞, √ c + nε → +∞. √In view of Φ(−∞) = 0, Φ(+∞) = 1 this implies Φ(c − nε) → 0, Φ(c + nε) → 1. Furthermore, like every reasonable testing procedure, the UMP level-α test for (4.1b) is consistent (Wellek, 1994, Lemma A.3.3). But convergence of the rejection probability to 1 under any θ ∈ (−ε, ε) implies [recall (4.5)] that the area under the density curve of a Gaussian variable with unit variance over the critical interval (−Cα; √nε , Cα; √nε ) tends to 1, which can only be true if the length of that interval and hence Cα; √nε itself increases to ∞ as n → ∞. In view of this, the values of c to be taken into consideration when solving equation (4.23) √ are very large positive numbers if n is large, and for i.e., to 1, already fixed c  0, Φ(c + nε) will come very close to its limit, √ for values of n which are still too small for carrying Φ(c − nε) into a small neighborhood of zero. According to this intuitive notion the solution to (4.23), i.e., the optimal critical constant Cα; √nε satisfies the approximate equation Φ(Cα; √nε −

√ nε) ≈ α .

(4.24)

On the other hand, in view of the definition of uγ and the symmetry of the standard normal distribution we clearly have α = Φ(−u1−α ) . Hence, (4.24) holds if and only if √ Φ(Cα; √nε − nε) ≈ Φ(−u1−α ) . (4.25) Since the standard normal cdf has a continuous inverse we are justified to drop Φ(·) on both sides of (4.25) and write for “sufficiently large” sample sizes n √ Cα; √nε ≈ nε − u1−α . (4.26) Finally, each of the two tests under √ comparison is uniquely determined by one of the critical bounds Cα; √nε and nε − u1−α being approximately equal according to (4.26). In view of this, we have given a heuristic proof of the assertion that approximate identity holds between both tests with respect to all probabilistic properties, in particular the power against any specific alternative θ ∈ (−ε, ε). In fact, according to the result proved as Theorem A.3.5 in Wellek (1994), still more can be said: Asymptotically, the interval inclusion test is uniformly most powerful for the equivalence problem (4.1) even if tests are admitted which maintain the nominal level only in the limit and/or the arbitrarily selected alternative θ ∈ K is allowed to converge to one of the boundary points of the equivalence interval rather than kept fixed for all values of n.

66

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

To be sure, demonstration of asymptotic optimality of a test per se is mainly of theoretical interest. The actual relevance of such a result can only be assessed if detailed numerical investigations have been performed providing in particular precise information on the order of magnitude of the sample sizes required to ensure that the loss of power entailed by replacing the optimal finite-sample test with the asymptotic procedure under consideration be practically negligible. Since both the UMP level-α test and the test based on the principle of confidence interval inclusion depends on n and ε only through √ ε˜ = nε, it suffices to compare the procedures for various values of this latter quantity. Tables 4.7 and 4.8 are the direct counterparts of Tables 4.1 and 4.2 for the interval inclusion test at the same level α = .05. Inspecting the entries in Table 4.8 as compared to the corresponding values shown in Table 4.2 leads to the following major conclusions: √ (i) As long as ε˜ = nε fails to exceed the value 1.7 the interval inclusion test is very unsatisfactory since its maximum power falls below the significance level. In contrast, for ε˜ = 1.7 the power of the UMP test against θ = 0 amounts to 21% and is thus more than four times larger than α. (ii) The cases where the interval inclusion test does better than the trivial procedure rejecting H with constant probability α (independently of the data), but is still markedly inferior to the UMP test at the same level are characterized by the condition 1.7 < ε˜ ≤ 2.3. (iii) If ε˜ is increased beyond the right-hand limit of this interval (by weakening the equivalence condition under the hypothesis and/or increasing the sample size), the difference between both tests disappears very quickly (except of course for some remainder being negligible for all practical purposes).

Numerical power comparisons for various other one-sample problems whose results are not reported in detail here, fully confirm these findings. Summarily speaking, the gain from applying the UMP instead of the corresponding interval inclusion test is very modest whenever the power attained in the former has the order of magnitude usually required in a study to be analyzed by means of a conventional one- or two-sided test. However, in equivalence testing power values exceeding 50% can only be obtained if either the equivalence range specified by the alternative hypothesis is chosen extremely wide or the sample size requirements are beyond the scope of feasibility for most, if not all, applications.

Asymptotic optimality of interval inclusion tests

67

Table 4.7 Critical constant ε˜−u.95 for the interval inclusion test at level α = .05 in the one-sample setting with data from N (θ, 1). ε˜

0.

1.

2.

3.

4.

.0 .1 .2 .3 .4 .5 .6 .7 .8 .9

– −1.5449 −1.4449 −1.3449 −1.2449 −1.1449 −1.0449 −.94485 −.84485 −.74485

−.64485 −.54485 −.44485 −.34485 −.24485 −.14485 −.04485 0.05515 0.15515 0.25515

0.35515 0.45515 0.55515 0.65515 0.75515 0.85515 0.95515 1.05515 1.15515 1.25515

1.35515 1.45515 1.55515 1.65515 1.75515 1.85515 1.95515 2.05515 2.15515 2.25515

2.35515 2.45515 2.55515 2.65515 2.75515 2.85515 2.95515 3.05515 3.15515 3.25515

Table 4.8 Power of the test with critical region √ ¯ n | < ε˜ − u.95 against θ = 0 for the same grid of n |X √ values of ε˜ = nε as covered by Table 4.7. ε˜

0.

1.

2.

3.

4.

.0 .1 .2 .3 .4 .5 .6 .7 .8 .9

– .00000 .00000 .00000 .00000 .00000 .00000 .00000 .00000 .00000

.00000 .00000 .00000 .00000 .00000 .00000 .00000 .04398 .12329 .20139

.27752 .35100 .42121 .48763 .54984 .60753 .66050 .70864 .75197 .79057

.82463 .85437 .88009 .90211 .92077 .93642 .94943 .96014 .96885 .97588

.98148 .98592 .98939 .99207 .99413 .99570 .99687 .99775 .99840 .99887

Clearly, the intuitive notion of equivalence suggests to tolerate only small deviations of the parameter θ from its target value θ◦ . If we adopt this point of view there is no choice but to either refrain from launching a suitable study at all, or to be content with power values falling distinctly below 50%. The latter case is the one where replacing an interval inclusion procedure with an optimal test typically yields the most marked gains in power, which can amount to up to 30% in these instances. In addition to such considerations concerning power and efficiency, there is another major reason (which has been alluded to already in § 3.3) why there is a real need for carrying out classical optimal

68

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

constructions of tests for equivalence in spite of the option of solving testing problems of that kind by means of the interval inclusion principle: The range of settings for which well-established interval estimation procedures exist, is by far too narrow to provide a clinical or experimental investigator with a sufficiently rich arsenal of specific equivalence testing methods.

4.5

Noninferiority analogues of the tests derived in this chapter

For all three concrete one-sample settings considered in this chapter, derivation of a noninferiority test is fairly straightforward. In the case of a single sample from N (θ, σ◦2 ) with known variance σ◦2 > 0, : θ ≤ −ε versus K1 : θ > −ε rejects if it the UMP level-α test of H1 √ ¯ > σ◦ u1−α / n − ε . The power of this test against an turns out that X arbitrarily chosen alternative  be computed by evaluating the √ θa > −ε can expression 1 − Φ u1−α − n(ε + θa )/σ◦ . Thus, for fixed significance level α, the power of the test for noninferiority of a normal distribution with unknown expected value θ to a Gaussian distribution of the same known variance√σ◦2 being centered at zero, depends on n, ε, θa and σ◦ only through ε = n(ε + θa )/σ◦ , and it is tempting to look at the noninferiority analogue of Table 4.2 which is shown below as Table 4.9. Comparing the entries in these two tables we can concretely assess the price which has to be paid for establishing with the same degree of certainty in terms of the type-I error risk a hypothesis which is considerably more precise than that stated in the noninferiority problem as the alternative: The power attainable with the same Table 4.9 Analogue of Table 4.2 for the noninferiority case. ε˜

0.

1.

2.

3.

4.

.0 .1 .2 .3 .4 .5 .6 .7 .8 .9

– .06119 .07425 .08934 .10659 .12613 .14805 .17237 .19910 .22818

.25951 .29293 .32821 .36510 .40328 .44241 .48211 .52199 .56165 .60069

.63876 .67550 .71060 .74381 .77492 .80376 .83025 .85432 .87598 .89529

.91231 .92719 .94004 .95105 .96038 .96821 .97472 .98007 .98442 .98794

.99074 .99296 .99469 .99604 .99707 .99785 .99844 .99888 .99920 .99943

Noninferiority analogues of the tests derived in this chapter

69

sample size in the two-sided equivalence testing scenario is substantially lower unless the power of the test for noninferiority is close to unity. Let us suppose next that the individual observations Xi making up the sample under analysis follow a standard exponential distribution E(σ) with unknown scale parameter σ > 0 and the problem of testing H1 : σ ≤ 1/(1 + ε) versus H1 : σ > 1/(1 + ε) has been proposed. Then, an UMP test at level  α is carried out through deciding in favor of noninferiority for values of ni=1 Xi found to be larger than the upper 100α percentage point γα (n, ε), say, of the gamma distribution with parameters n and 1/(1 + ε). In SAS, the critical constant γα (n, ε) is readily obtained by calling the intrinsic function gaminv with arguments 1 − α and n (in that order) and dividing the result by 1 + ε. In the R system, the function qgamma is available for the same purpose. Alternatively, the test can be carried  out by computing the p-value associated with the realized value t of T = ni=1 Xi . In SAS, this just requires to write down the assignment statement p=1-probgam((1+eps)*t,n). Revisiting Example 4.1 where we had n = 80, ε = .3 and t = 85.1507, this yields p = .00094 so that noninferiority can be rejected a fortiori. In the binomial case, the decision rule of the UMP level-α test of H1 : p ≤ p◦ − ε versus K1 : p > p◦ − ε is obtained by simplifying (4.18) to ⎧ ⎨ Rejection of H1 for T > Cα (n; p◦ , ε) Rejection with prob. γα (n; p◦ , ε) for T = Cα (n; p◦ , ε) , (4.27) ⎩ Acceptance for T < Cα (n; p◦ , ε) where n 

  b(j; n, p◦ − ε) > α , Cα (n; p◦ , ε) = max k ∈ IN0 

(4.28)

j=k

γα (n; p◦ , ε) =  α−

    b(j; n, p◦ − ε) b Cα (n; p◦ , ε); n, p◦ − ε . (4.29)

n 

j=Cα (n; p◦ ,ε)+1

The power of the exact randomized UMP test (4.27) against any alternative p > p◦ − ε is given by βα,n (p◦ , ε; p) = n 

  b(j; n, p) + γα (n; p◦ , ε) b Cα (n; p◦ , ε); n, p ,

(4.30)

j=Cα (n; p◦ ,ε)+1

which reduces to (0) βα,n (p◦ , ε; p) =

n  j=Cα (n; p◦ ,ε)+1

b(j; n, p)

(4.31)

70

EQUIVALENCE TESTS FOR ONE-PARAMETER PROBLEMS

when randomization is omitted. The corresponding functions of p are plotted in Figure 4.4 for the same specifications as underly Figure 4.3, except of course for setting the right-hand boundary of the hypothetical equivalence range equal to unity. For α = .05, n = 82, p◦ = .80 and ε = .10 the critical constants are computed to be Cα (n; p◦ , ε) = 64 and γα (n; p◦ , ε) = .35671 using either the above formulae (4.28) and (4.29), or running the program bi1st with p2 = 1 − 10−15 .

Figure 4.4 Power function of the UMP level-α test of p ≤ .70 vs. p > .70 [solid line] and its nonrandomized counterpart [broken line] for n = 82 and α = .05.

5 Equivalence tests for designs with paired observations

5.1

Sign test for equivalence

The assumptions which must be satisfied in order to ensure that the sign test for equivalence be an exactly valid procedure exhibiting some basic optimality property are as weak as in the case of its traditional one- and two-sided analogue. This means that the data have to be given by n mutually independent pairs (X1 , Y1 ), . . . , (Xn , Yn ) of random variables following all the same bivariate distribution which may be of arbitrary form. In particular, the distribution of the (Xi , Yi ) is allowed to be discrete. Within each such random pair, the first and second component gives the result of applying the two treatments, say A and B, under comparison to the respective observational unit. Except for treatment, the observations Xi and Yi are taken under completely homogeneous conditions so that each intraindividual difference Di ≡ Xi − Yi can be interpreted as an observed treatment effect. For definiteness, we assume further that large values of the X ’s and Y ’s are favorable, which implies that A has been more efficient than B in the ith individual if and only if the value taken on by Di turns out to be positive. Now, let us agree on defining equivalence of both treatments by means of the condition that, except for practically irrelevant differences, the probability p+ ≡ P [Di > 0 ] of observing A to turn out superior to B in a randomly selected individual, coincides with the probability p− ≡ P [Di < 0 ] of observing A to yield a weaker effect than B. In view of our aim to construct an optimal testing procedure, it proves advisable that we relate this condition to the logarithmic scale which leads to considering the statement −ε1 < log p+ − log p− < ε2 (with sufficiently small positive real numbers ε1 , ε2 ) as the equivalence hypothesis of interest. In fact, the mathematically natural choice of a target parameter for the inferential problem in mind is given by θ = log(p+ /p− ) .

(5.1)

This is readily seen by examining the joint distribution of the usual counting statistics N+ = #{i | Di > 0}, N0 = #{i | Di = 0}, N− = #{i | Di < 0}

71

72

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

whose probability mass function admits the representation f (n+ , n0 , n− ; n, θ, ζ) =

n! exp{n+ θ + n0 ζ} (1 + eθ + eζ )−n , (5.2) n+ !n0 !n− !

where θ = log(p+ /p− ) ,

ζ = log(p0 /p− ) .

(5.3)

Furthermore, the argument made explicit in Lehmann and Romano (2005, pp. 137-8) shows that a test which is uniformly most powerful unbiased (UMPU) in the class of those level-α tests of H : θ ≤ −ε1 ∨ θ ≥ ε2 versus K : −ε1 < θ < ε2

(5.4)

which depend on the data only through (N+ , N0 , N− ), uniformly maximizes the power also within the class of all unbiased tests at the same level for (5.4). By (5.2), the class of all possible joint distributions of (N+ , N0 , N− ), i.e., of all trinomial distributions based on a sample of size n, is a two-parameter exponential family in (θ, ζ) and (N+ , N0 ). Hence, we can apply Theorem A.2.2 [→ Appendix, p. 375] with k = 1, T = N+ , S = N0 which implies that a UMPU level-α test for the problem (5.4) exists and is of the following form: Given the observed number of ties between A and B, i.e., of zeroes contained in the sample (D1 , . . . , Dn ), is n0 ∈ {0, 1, . . . , n}, the null hypothesis H of nonequivalence has to be rejected if the number N+ of pos1 2 itive signs satisfies the double inequality Cα|n + 1 ≤ N+ ≤ Cα|n − 1. On 0 0 ν ,a the boundaries of the corresponding critical interval, i.e., for N+ = Cα|n 0 ν randomized decision in favor of K has to be taken with probability γα|n 0 ν ν (ν = 1, 2). The complete set Cα|n , γα|n , ν = 1, 2, of (conditional) critical 0 0 constants has to be determined by solving the equations 2 Cα|n −1 0 

P−ε1 [N+ = n+ |N0 = n0 ] +

1 n+ =Cα|n +1 0

2 

  ν ν N+ = Cα|n γα|n P | N0 = n0 = α , 0 −ε1 0

(5.5a)

ν=1 2 Cα|n −1 0 

Pε2 [N+ = n+ |N0 = n0 ]

+

1 n+ =Cα|n +1 0

2 

  ν ν γα|n P N+ = Cα|n | N0 = n0 = α . 0 ε2 0

(5.5b)

ν=1

Since the conditional distribution of N+ given N0 = n0 is well known (cf. Johnson et al., 1997, p. 35) to be binomial with parameters n−n0 and p+ /(p+ +p− )

Sign test for equivalence

73

= eθ /(1 + eθ ) for each n0 ∈ {0, 1, . . . , n} and θ ∈ IR, the system (5.5) differs from (4.19) only by changes in notation. More precisely speaking, (5.5) turns into the system of equations to be solved in testing for equivalence of a single binomial proportion to 1/2 simply by making the substitution n ↔ n − n0 , p1 ↔ e−ε1 /(1 + e−ε1 ), p2 ↔ eε2 /(1 + eε2 ) . Thus, it eventually follows that an UMPU level-α test for the nonparametric equivalence problem proposed at the beginning of this section is obtained by treating the observed number n0 of pairs with tied components as a fixed quantity and carrying out the test derived in  § 4.3 with n − n0 as nominal  sample size and e−ε1 /(1+e−ε1 ) , eε2 /(1+eε2 ) as the equivalence range for the binomial parameter. Accordingly, the practical implementation of the decision rule to be used in the sign test for equivalence entails no new computational steps. In particular, in the symmetric case ε1 = ε2 = ε, the nonrandomized version of the test can be carried out by means of the p-value as defined in (4.21) (with ε and n replaced with eε /(1 + eε ) − 1/2 and n − n0 , respectively), and even the table of critical bounds to the test statistic presented before for the binomial one-sample problem can be used for some values of n − n0 and some specifications of ε. The only true extension brought into play when using the binomial one-sample test of § 4.3 as a conditional procedure for testing the hypotheses (5.4) concerns the computation of power values and minimum sample sizes required for guaranteeing specified lower bounds to the power. As is generally the case with conditional testing procedures, the power function of the sign test for equivalence is of practical interest only in its nonconditional form, which implies that the respective computations involve averaging with respect to the distribution of the conditioning statistic N0 . Obviously, irrespective of the form of the hypotheses under consideration, the nonconditional power function of any test based on the statistics N+ and N0 is a function of two arguments, viz. p+ and p0 . A program which enables one to compute the rejection probability of both the exact and the nonrandomized version of the UMPU level-α test for (5.4) against the specific alternative θ = 0 (or, equivalently, p+ = p− which in turn is equivalent to p+ = (1−p0 )/2 for arbitrary rates p0 of zeroes among the intra-subject differences Di ), can be found in the WKTSHEQ2 Source Code Package under the program name powsign (again, both SAS, and R code is provided). Clearly, the problem of inducing considerable conservatism by not allowing randomized decisions in favor of the alternative hypothesis which has been discussed from a general perspective in § 2.5 (for the noninferiority case) and § 3.3, affects the sign test for (two-sided) equivalence as well. The sign test is among the settings where mitigating this conservatism by increasing the nominal significance level as far as possible without rising the nonconditional rejection probability under any null parameter constellation above the target level, provides a very satisfactory compromise between maximization of power and suitability of the resulting testing procedure for real applications. For sample sizes up to 150 being multiples of 25 and cases where the equivalence limits to the target parameter θ are chosen in accordance with Table 1.1

74

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

(note that in presence of ties, p+ /(1 − p0 ) = p+ /(p+ + p− ) is the analogue of p+ as to be considered in the continuous case), Table 5.1 shows maximally increased nominal levels α∗ for use in the nonrandomized sign test for equivalence at target level α = .05. Table 5.1 Nominal significance levels and sizes of improved nonrandomized sign tests for equivalence at target level 5% for ε1 = ε2 = ε = log(6/4), log(7/3) and n = 25(25)150. [Number in ( ): size of the nonrandomized test at nominal level α = .05; p∗0 = value of the nuisance parameter maximizing the rejection probability of H.] ε n α∗ p∗0 .405465 25 .11714 .0675 " 50 .08429 .0000 " 75 .06699 .0113 " 100 .05802 .0671 " 125 .05750 .0296 " 150 .05709 .0408

Size .04746 .04046 .04832 .04413 .04743 .04899

(.00000) (.04046) (.04014) (.03935) (.04018) (.04366)

.847298 25 .07060 .0000 " 50 .06580 .2214 " 75 .06000 .0206 " 100 .05305 .0494 " 125 .06000 .3580 " 150 .05963 .4851

.03825 .04831 .04928 .04545 .04983 .04888

(.03825) (.04771) (.04133) (.04101) (.04157) (.04704)

Example 5.1 In a study based on a sample of size n = 50, the number of patients doing better under treatment A as compared to B was observed to be n+ = 17. In n0 = 13 patients, no difference between the responses to A and B could be detected. The equivalence interval for θ = log(p+ /p− ) was chosen symmetric about 0 specifying ε1 = ε2 = ε = log(7/3) = .847298 which is the same as requiring of the parameter of the conditional binomial distribution of N+ given N0 = n0 to fall in the interval (3/10 , 7/10). With these data and specifications of constants, the sign test for equivalence has to be carried out as a binomial test for equivalence of p = eθ /(1 + eθ ) to 1/2 in a sample of size n = n − n0 = 37 and equivalence limits .5 ∓ .2 to p. At the 5% level, the rejection region of this test is computed (by means of the program bi1st referred to in § 4.3) to be the set { 16 < N+ < 21 }. Since the observed value of the test statistic N+ is an element of this set, we are justified to reject nonequivalence in the sense of | log(p+ /p− )| ≥ .847298 .

Sign test for equivalence

75

For assessing the power of the test let us assume that the true proportion p0 of patients responding equally well to both treatments coincides with the observed relative frequency 13/50 = .26 of ties and that in the subpopulation of patients providing nonindifferent responses, superiority of A to B is exactly as frequent as inferiority. Running the enclosed program powsign we find the power against this specific alternative to be 61.53% (UMPU test) and 54.19% (nonrandomized version), respectively. Replacing the target significance level .05 with the raised nominal level α∗ = .0658 as displayed in the pertinent line of the above table, the power of the nonrandomized test increases to 60.91%. Thus, under the present circumstances, the loss of efficiency entailed by discarding randomization can be compensated almost completely without changing the basic structure of the test. Noninferiority version of the sign test The modifications to be performed in order to construct a UMPU test for noninferiority rather than two-sided equivalence based on the sign statistic, are straightforward. Conditional on the value n0 observed in the sample for the number of tied observations, the decision between H1 : θ ≤ −ε and K1 : θ > −ε has to be taken according to the rule (4.27) substituting p◦ −ε and n with e−ε /(1 + e−ε ) and n − n0 , respectively. Conditional on {N0 = n0 }, the power against an arbitrarily selected alternative (θ, p0 ) is obtained by making the analogous substitutions 1/2 → p◦ , 1/2 − e−ε /(1 + e−ε ) → ε, n − n0 → n, eθ /(1 + eθ ) → p on the right-hand side of Equation (4.30). Again, averaging the result with respect to the distribution of N0 which is given by B(n, p0 ) yields the nonconditional power against that alternative. The problem of constructing an improved nonrandomized version of the test can likewise be addressed in the same manner as in the case of twosided equivalence, i.e., by determining largest admissible nominal levels α∗ . Keeping everything else fixed, except for replacing (−ε, ε) by (−ε, ∞) as the θ-range specified by the hypothesis to be established, produces Table 5.2 as a noninferiority analogue of Table 5.1. In terms of α∗ , the results differ between both hypotheses formulations the more the smaller the sample size has been chosen. Given n, the more liberal of the two specifications of the equivalence margin is associated with smaller changes in the resulting value of the admissible nominal significance level when interest is in establishing noninferiority instead of equivalence in the two-sided sense.

76

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS Table 5.2 Nominal significance levels and sizes of improved nonrandomized sign tests for noninferiority at target level 5% for ε = log(6/4), log(7/3) and n = 25(25)150. [Number in ( ): size of the nonrandomized test at nominal level α = .05; p∗0 = value of the nuisance parameter maximizing the rejection probability on H.]

5.2

ε n α∗ p∗0 .405465 25 .07780 .2940 " 50 .05734 .1710 " 75 .06000 .0480 " 100 .05875 .0210 " 125 .05500 .0170 " 150 .05688 .0960

Size .04606 .04391 .04825 .04879 .04819 .04847

(.03423) (.03931) (.03981) (.04195) (.04205) (.04278)

.847298 25 .07424 .1610 " 50 .06584 .2255 " 75 .06000 .0205 " 100 .05305 .0495 " 125 .06000 .3580 " 150 .05963 .4850

.04874 .04857 .04928 .04546 .04983 .04888

(.04409) (.04753) (.04114) (.04101) (.04157) (.04672)

Equivalence tests for the McNemar setting

The test commonly named after McNemar (1947) deals with a setting which is just a special case of that underlying the previous section. Specialization results from the fact that all primary observations Xi and Yi are now assumed to be binary response-status indicators taking on values 1 ( ↔ success) and 0 ( ↔ failure), respectively. (Of course, any other way of coding the two alternative elementary outcomes leads to the same inferences.) This obviously implies that the set (X1 , Y1 ), . . . , (Xn , Yn ) of bivariate data points make up a sample of size n from a quadrinomial distribution with parameters p00 , . . . , p11 , say, given by p00 = P [Xi = 0, Yi = 0 ], p01 = P [Xi = 0, Yi = 1 ], p10 = P [Xi = 1, Yi = 0 ], p11 = P [Xi = 1, Yi = 1 ]. In view of the sufficiency of the respective frequencies for the corresponding family of joint distributions of all n random pairs, there is no loss of efficiency in reducing the full data set as directly observed, to the entries in a 2 × 2 contingency table of the form displayed in Table 5.3.

Equivalence tests for the McNemar setting

77

Table 5.3 Outline of a 2 × 2 contingency table for use in the analysis of a trial yielding paired binary data. Treat- Treatment B ment A 0 1

Σ

0

N00 (p00 )

N01 (p01 )

N0 . (p0 . )

1

N10 (p10 )

N11 (p11 )

N1 . (p1 . )

Σ

N. 0 (p. 0 )

N. 1 (p. 1 )

n (1.00)

As is the case in the “classical” McNemar test concerning an ordinary oneor two-sided problem, in constructing a test for equivalence of the two treatments A and B based on a contingency table of that structure, the difference rather than the ratio of the probabilities p10 , p01 of the two possible kinds of discordant pairs of primary observations will be regarded as the target parameter. Evidently, establishing equivalence or noninferiority with respect to p10 /p01 leads not to a new testing problem but can be done simply by applying the appropriate version of the test obtained in § 5.1 with N+ = N10 , N− = N01 and the analogous replacements among the p’s. Although the parametrization through p10 /p01 is not only mathematically the most convenient one but also provides an intuitively reasonable measure of dissimilarity of the effects of the treatments to be compared in a matchedpair design using a binary outcome criterion, there is considerable practical interest in alternative hypotheses formulations. Conceptually, a closely related formulation is that used in the papers by Lachenbruch and Lynch (1998) and Nam and Blackwelder (2002). These authors propose to define equivalence of the treatments under study by the condition that the true value of the ratio p1 . /p. 1 ≡ (p10 + p11 )/(p01 + p11 ) of both marginal binomial parameters must be sufficiently close to unity. Replacing p10 /p01 with p1 . /p. 1 makes the construction of exact tests technically very complicated, and the procedures which can be found in the literature referenced above are both asymptotic in nature. Of course, exactly the same objection raised in § 1.6 against basing equivalence assessment of binomial distributions from which independent samples have been taken, on the difference of the two population parameters, likewise applies to the case of correlated proportions. Notwithstanding this fact, in McNemar’s setting, the still by far most popular choice of the target parameter for measuring the dissimilarity between the underlying theoretical distribution is δ = p10 − p01 . Accordingly, the testing problems to be dealt

78

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

with in the better part of the present section are given by H : δ ≤ −δ1 or δ ≥ δ2 versus K : −δ1 < δ < δ2

(5.6a)

H1 : δ ≤ −δ0 versus K1 : δ > −δ0

(5.6b)

and where δ = p10 − p01 ,

δk ∈ (0, 1)

for k ∈ {0, 1, 2} .

(5.7)

In the journal literature which has become available about tests for these problems (see Lu and Bean, 1995; Morikawa et al., 1996; Tango, 1998; Hsueh et al., 2001; Sidik, 2003; Nam, 2006), there is a preference in favor of using test statistics which are based on maximum likelihood estimators restricted by the assumption that δ falls on the common boundary of the hypotheses. The first to propose the use of restricted maximum likelihood estimation in a comparable context (testing for one-sided equivalence of binomial distributions in a parallel-group design) were Farrington and Manning (1990). In contrast, the tests which will be presented in Subsections 5.2.1 and 5.2.2 are based on Wald type statistics computed from the likelihood maximized over the whole parameter space. In the derivation of these procedures, it will be convenient to introdue an extra symbol, say η, for denoting the probability of observing in a randomly selected experimental unit a constellation belonging to the off-diagonal cells of Table 5.3. In other words, we define η = p10 + p01 .

5.2.1

(5.8)

Large-sample solution

A large-sample solution to the testing problem (5.6a) is obtained by applying the theory outlined in § 3.4. In fact, standard results on the limiting distribution of arbitrary linear functions of the absolute frequencies appearing as entries in a contingency table with random marginal sums  (see, e.g., Rao, √ 1973, p. 383, 6a.1(i)) imply that the statistic n(δˆn − δ)/ η − δ 2 , with δˆn =  (N10 − N01 )/n, is asymptotically a standard normal variate. Furthermore, η − δ 2 is obviously a continuous function of the basic multinomial parameters p00 , . . . , p11 so that plugging in the natural estimators ηˆn = (N10 +N01 )/n, δˆn = (N10 − N01 )/n yields a consistent estimator of the standard error of the numerator of that statistic (as follows from another well-known result — see Rao, 1973, p. 345 — on the asymptotic properties of the multinomial family). Hence, on the understanding that both extreme constellations η = 0 √ and |δ| = 1 under which the variance of n(δˆn − δ) vanishes can be ruled out, the testing problem (5.6a) satisfies all conditions of using the approach of § 3.4 with the following specifications: N = n, TN = δˆn , θ = δ, and ηn − δˆn2 )/n]1/2 . In terms of the off-diagonal frequencies N10 and N01 , τˆN = [(ˆ

Equivalence tests for the McNemar setting

79

the decision rule of the corresponding asymptotically valid test for the problem (5.6a) admits the following representation: Reject nonequivalence iff

n1/2 |(N10 − N01 ) − n(δ2 − δ1 )/2|

< [n (N10 + N01 ) − (N10 − N01 )2 ]1/2   n3/2 (δ1 + δ2 )/2 Cα . [n (N10 + N01 ) − (N10 − N01 )2 ]1/2

(5.9)

For definiteness, let us recall that the critical upper bound appearing in the above rule has to be determined as the square root of the lower 100 α percentage point of a χ2 -distribution with a single degree of freedom and a (random) noncentrality parameter which equals the square of the expression appearing in the argument to the functional symbol Cα , i.e., n3 (δ1 + δ2 )2 /4[n (N10 + N01 ) − (N10 − N01 )2 ] . Example 5.2 We illustrate the asymptotic testing procedure given by (5.9), by analyzing a contingency table compiled from data contained in the reference database of the CSE (Common Standards for Quantitative Electrocardiography) project launched in the early 1980s. One of the major objectives of this project was the establishment of tools for a comparative assessment of the diagnostic accuracy of various computer programs for the automatic classification of electrocardiograms (ECGs). For that purpose, a diagnostic “gold standard” was set up by classifying many hundred subjects with well-documented ECG recordings into clear-cut diagnostic categories by means of carefully validated clinical criteria irrespective of the ECG. In the version underlying the report of Willems et al. (1990) the database contained n = 72 ECGs recorded from patients suffering from left ventricular hypertrophy (LVH), a pathological condition of the heart notoriously difficult to recognize by means of noninvasive methods. Table 5.4 summarizes the results of the assessment of these ECGs by a panel of human experts (CR [“Combined Referee”]) on the one hand, and one specific computer program denoted by F on the other. In this table, 0 stands for classification into a wrong category (different from LVH) whereas 1 indicates that the LVH was recognized from the patient’s ECG. For the general purposes of the CSE project, it was in particular of interest to assess the equivalence of the computer program to the human experts with respect to the capability of diagnosing LVH. Since the difference between the respective probabilities of detecting an existing hypertrophy of the left ventricle equals to δ as defined in (5.7), this amounts to raising a testing problem of the form (5.6a). Finally, let us choose the equivalence interval once more in a symmetric manner setting δ1 = δ2 = .20, and use the conventional value α = .05 for the nominal significance level.

80

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS Table 5.4 Comparison between the computer program F and the “Combined Referee” (CR) in the diagnostic classification of the ECGs of 72 patients with clinically proven hypertrophy of the left ventricle of the heart. [Data from Willems et al. (1990, p. 113).] C R 0 1 Σ

F 0

1

Σ

47 (.6528) 4 (.0556)

5 52 (.0694) (.7222) 16 20 (.2222) (.2778)

51 (.7083)

21 72 (.2917) (1.000)

With δ1 = δ2 and the observed frequencies displayed in the above table, the expression on the left-hand side of the critical inequality (5.9) becomes n1/2 |(N10 − N01 ) − n(δ2 − δ1 )/2| [n (N10 + N01 ) − (N10 − N01 )2 ]1/2 √  72|4 − 5| 72 = √.111283 = .3336 . =  = 647 2 72 (4 + 5) − (4 − 5) On the other hand, we get for the noncentrality parameter of the χ2 -squared distribution providing an upper critical bound to our test statistic: !2 n3/2 (δ1 + δ1 )/2 2 ˆ ψ = [n (N10 + N01 ) − (N10 − N01 )2 ]1/2 =

723 · .202 = 23.075611 . 72 · 9 − (−1)2

But the .05-quantile of a χ2 -distribution with a single degree of freedom and that noncentrality parameter is computed (e.g., by means of the SAS intrinsic function cinv) to be 9.978361. Consequently, as the critical upper bound we obtain   √ n3/2 (δ1 + δ2 )/2 Cα = 9.978361 = 3.1589 . 2 1/2 [n (N10 + N01 ) − (N10 − N01 ) ] Since the latter is much larger than the observed value of the test statistic n1/2 |(N10 − N01 ) − n(δ2 − δ1 )/2|/[n (N10 + N01 ) − (N10 − N01 )2 ]1/2 , it follows that at the 5% level, the data of Table 5.4 allow a decision in favor of equivalence of F and CR in the sense of −.20 < δ < .20 .

Equivalence tests for the McNemar setting

5.2.2

81

Corrected finite-sample version of the large-sample test

In view of the purely asymptotic nature of the approach leading to (5.9), the validity of the procedure in finite samples can obviously not be taken for granted without further study. The clearest and most reliable answer to the question of possible discrepancies between actual and nominal level of an asymptotic testing procedure is obtained by determining the rejection probability under arbitrary null parameter constellations by means of exact computational methods. For that purpose, it is convenient to introduce the following transformation of variables and parameters, respectively: U = N10 , V = N10 + N01 ;

π = p10 /η ,

(5.10)

with η defined as in (5.8). In terms of (U, V ), the critical region of the test (5.9) can be rewritten 

  (u, v) ∈ IN20  u ≤ v ≤ n, Tnδ1 ,δ2 (u, v) < cδα;1 ,δn2 (u, v) (5.11) where Tnδ1 ,δ2 (u, v)

√ n|(2u − v) − n(δ2 − δ1 )/2| = , [nv − (2u − v)2 ]1/2 

cδα;1 ,δn2 (u, v) = Cα

n3/2 (δ1 + δ2 )/2 [nv − (2u − v)2 ]1/2

 .

The advantage of representing the critical region in this form stems from the fact that the conditional distribution of U given {V = v} is binomial with parameters (v, π) for any v = 0, 1, . . . n, and the marginal distribution of V is likewise binomial, with parameters (n, η). Hence, the rejection probability of the test (5.9) under any (π, η) admits the representation # " n   δ1 ,δ2 b(u; v, π) b(v; n, η) , (5.12) Qα,n (π, η) = v=0

δ ,δ

1 2 (v) u∈Uα,n

δ1 ,δ2 (v) denotes the v-section of the set (5.11) given by where Uα,n 

  δ1 ,δ2 Uα,n (v) = u ∈ IN0 u ≤ v, Tnδ1 ,δ2 (u, v) < cδα;1 ,δn2 (u, v) ,

(5.13)

which is typically∗ an (maybe empty) interval of nonnegative integer numbers ≤ v. Furthermore, in (5.12), both symbols of the type b(k ; m, p) stand for the respective binomial point masses. ∗ By

“typically,” we mean in the present context that in constructing the respective sets explicitly by direct computation, we never found an exception but no mathematical proof of the corresponding general assertion is available. In view of this, both computer programs implementing evaluations of (5.12) (mcnemasc, mcnempow), although working on the understanding that it is a generally valid fact, check upon it during each individual run. In case of finding an exception, either program would stop and produce an error message.

82

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

Now, in order to determine the actual size of the asymptotic McNemar test 1 ,δ2 (π, η) over the set of all (π, η) ∈ for equivalence, we have to maximize Qδα,n 2 (0, 1) satisfying the above [ → (5.6a) ] null hypothesis H of nonequivalence. As an immediate consequence of the definition of the parametric functions π, δ and η, one has π = (δ + η)/2η so that for any fixed η > 0 the η-section of H equals the complement of the interval (1/2 − δ1 /2η , 1/2 + δ2 /2η) . Hence, for the size of the test we obtain the formula 

 1 ,δ2 SIZEα (δ1 , δ2 ; n) = sup Qδα,n (π, η)  0 < η < 1,  0 < π ≤ 1/2 − δ1 /2η or 1/2 + δ2 /2η ≤ π < 1 . (5.14) Qualitatively speaking, the results to be obtained by evaluating the righthand side of (5.14) numerically admit the conclusion that the finite-sample behavior of the asymptotic test given by (5.9) is surprisingly poor (in the subsequent sections and chapters, the reader will find numerous applications of the theory of § 3.4 to much more complex models yielding very satisfactory solutions to the respective equivalence testing problem). In fact, the convergence of the exact size of the test to its limiting value, i.e., the target significance level α, turns out to be extremely slow, with a general tendency towards anticonservatism. For instance, in the specific case of Example 5.2, the exact size of the critical region is seen to be as large as .0721 although the size of the sample distinctly exceeds that available for many equivalence studies conducted in practice. Even if the number of observations in the sample is raised to 150 and thus more than doubled, the smallest significance level maintained by the asymptotic test is still .0638 rather than .05 . Clearly, these results, together with many other examples pointing in the same direction, strongly suggest to replace the asymptotic solution to the problem of testing for equivalence in McNemar’s setting by a finite-sample version corrected with respect to the significance level. Provided repeated evaluation of (5.14) with sufficiently high numerical precision causes no serious problem with respect to computing time, such a correction is easily carried out: All that is left to do is to determine by means of an iteration process the largest nominal significance level α∗ such that SIZEα∗ (δ1 , δ2 ; n) ≤ α. Of course, it is tempting to speed up this procedure by restricting at each step the search for the maximum probability of an incorrect decision in favor of equivalence to the boundary of the null hypothesis H of (5.6a). Since, in terms of (π, η), the latter is given as the set { (1/2 − δ1 /2η, η) | δ1 < η < 1 } ∪ { (1/2 + δ2 /2η, η) | δ2 < η < 1 }, this leads to determining for varied values of α

1 ,δ2 SIZEαB (δ1 , δ2 ; n) = max sup Qδα,n (1/2 − δ1 /2η, η) , η>δ1

1 ,δ2 sup Qδα,n (1/2 + δ2 /2η, η)

η>δ2

 (5.15)

Equivalence tests for the McNemar setting

83

instead of SIZEα (δ1 , δ2 ; n) as obtained by searching through the whole set of parameter values compatible with H. Although, in general we can only state that there holds the inequality SIZEαB (δ1 , δ2 ; n) ≤ SIZEα (δ1 , δ2 ; n), taking δ1 ,δ2 (v) [ → (5.13) ] are intervals allows to restrict for granted that the sets Uα,n the search for the maximum probability of a type-I error to the common boundary of the hypotheses (5.6a) whenever the equivalence range for δ is chosen as a symmetric interval. In fact, δ1 = δ2 clearly implies that each such interval in the sample space of U |{V = v} must be symmetric about v/2 from which it can be concluded by means of elementary arguments (starting from the well-known relationship between the binomial cdf and the incomplete beta integral — see, e.g., Feller, 1968, p. 173) that, as a function of π, the vth inner sum of (5.12) is nondecreasing on (0 , 1/2] and nonincreasing on [1/2 , 1). The program mcnemasc [→ Appendix B] which for cases with δ1 = δ2 = δ0 serves the purpose of determining a nominal significance level α∗ such that the exact size of the respective test for (5.6a) approaches the target level from below as closely as possible, exploits this possibility of simplifying the search for the maximum of the rejection probability (5.12) over the null hypothesis. For two symmetric choices of the equivalence range (−δ1 , δ2 ) for δ = p10 − p01 and selected sample sizes ranging from 50 to 200, corrected nominal levels computed in this way are shown in Table 5.5, together with the exact size of the corrected test and that of the original large-sample procedure. All results Table 5.5 Nominal significance level and exact size for the corrected McNemar test for equivalence maintaining the 5% level in finite samples of size n = 50(25)200, for δ1 = δ2 = δ0 = .2, .4. [Number in ( ): size of the noncorrected test at nominal level α = .05.] δ0 .20 " " " " " "

n 50 75 100 125 150 175 200

α∗ .024929 .016502 .036807 .037268 .032418 .027880 .039334

SIZEα∗ (δ0 , δ0 ; n) .04801 .02671 .04884 .04766 .04137 .03581 .04919

(.10338) (.09275) (.08444) (.07047) (.06376) (.07501) (.06396)

.40 " " " " " "

50 75 100 125 150 175 200

.029388 .039807 .043173 .043206 .041632 .039291 .046252

.03532 .04201 .04737 .04601 .04358 .04076 .04920

(.05651) (.06257) (.06150) (.05981) (.05608) (.05363) (.05327)

84

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

presented in this table have been obtained by means of mcnemasc setting the fineness of the grid of η-values searched through for the maximum rejection probability under nonequivalence [recall (5.14)] equal to .0005 throughout. Strikingly, neither the difference between the target significance level α and the nominal level to be used in the corrected test, nor between α and the size attainable under the correction decreases monotonically with n, and even with samples comprising several hundred observations it happens that the nominal level has to be halved. Once the corrected nominal level has been determined, computing the exact power of the corresponding finite-sample test for the equivalence problem (5.6a) against arbitrary specific alternatives δ ∈ (−δ1 , δ2 ) is comparatively easy since it requires only a single evaluation of the double sum on the righthand side of (5.12). However, given the alternative of interest in terms of δ, the power depends in a rather complicated way on the nuisance parameter η. As becomes obvious from the numerical example shown in Table 5.6, the power is in particular neither monotonic nor concave in η. Generally, the program mcnempow allows the computation of the exact rejection probability of the test (5.9) at any nominal significance level α ˜ under arbitrary parameter constellations (p10 , p01 ) . Table 5.6 Exact power of the corrected version of (5.9) against the alternative δ = 0 [ ⇔ π = 1/2] for n = 50, δ1 = δ2 = .20, α∗ = .024902 and various values of the nuisance parameter η. η

.0002

.002

Q.20,.20 α∗ ,50 (1/2, η)

.005

.01

.00995 .20 .73720

.09525 .22169 .39499 .40 .60 .80 .29702 .11515 .06281

.02 .63581 1.00 .11228

Example 5.2 (continued) For n = 72 and δ1 = δ2 = .20, the smallest nominal level α∗ to be used in (5.9) in order to maintain the target significance level α = .05, is obtained (by running the program mcnemasc) to be α∗ = .027148 . Since the noncentrality parameter of the χ2 -distribution through which the critical upper bound to the test statistic n1/2 |(N10 − N01 )|/[n (N10 + N01 ) − (N10 − N01 )2 ]1/2 has to be determined, has been estimated by means of the data of the present exrecomputing the critical bound with α∗ instead ample to be ψˆ2 = 23.075611,  √ ˆ = cinv(.027148, 1, 23.075611) = 8.290026 = 2.8792, of α gives: Cα∗ (ψ) after another application of the SAS intrinsic function cinv. This bound is

Equivalence tests for the McNemar setting

85

still much larger than the value which has been observed for the test statistic so that the decision keeps the same as if the nominal level was set equal to the target significance level. Finally, let us assume that the true rates of discordant pairs of both kinds in the underlying population coincide with the corresponding relative frequencies of the observed 2 × 2 table so that we have p10 = 4/9, p01 = 5/9. Running program mcnempow, the power of the corrected test against this “observed alternative” is found to be only slightly larger than the target significance level, namely .0599, as compared to .1794 which turns out to be the power of the noncorrected asymptotic test (5.9) against the same alternative.

5.2.3

Modifications for the noninferiority case

The noninferiority analogue of the asymptotic test given by (5.9) rejects the null hypothesis H1 : δ ≡ p10 − p01 ≤ −δ0 if and only if the observed value of (N10 , N10 + N01 ) ≡ (U, V ) turns out to fall in the critical region 

  (u, v) ∈ IN20  u ≤ v ≤ n, Tnδ0 (u, v) > u1−α , (5.16) where Tnδ0 (u, v)

 √  n (2u − v) + nδ0 = [nv − (2u − v)2 ]1/2

and u1−α denotes the (1 − α)-quantile of the standard normal distribution [recall (2.6)]. In order to exactly control the significance level in finite samples, the test has likewise to be carried out at an corrected nominal level α which can be determined by similar numerical methods as were described above for the case of two-sided equivalence. The starting point of the computational procedure is the following expression for the exact rejection probability of the an arbitrary parameter constellation (π, η) ≡   test given by (5.16) under p10 /(p10 + p01 ), p10 + p01 : # " n   δ0 Qα,n (π, η) = b(u; v, π) b(v; n, η) . (5.17) v=0

δ

0 (v) u∈Uα,n

The only difference between (5.12) and (5.17) refers to the inner summation region which is now given by  √  

 n (2u − v) + nδ0  δ0 > u1−α . Uα,n (v) = u ∈ IN0 u ≤ v, (5.18) 2 1/2 [nv − (2u − v) ] δ1 ,δ2 In contrast to the v-sections Uα,n (v) of the critical region (5.11) of the asymptotic test for the two-sided version of the equivalence problem under

86

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

δ0 (v) can be shown analytically to exhibit the approdiscussion, the sets Uα,n priate form. A sufficient condition which ensures that for each v = 1, . . . , n, there exists some integer kv ∈ {0, 1, . . . , v} such that δ0 Uα,n (v) = {u ∈ IN0 kv ≤ u ≤ v}

is that we have n>

u21−α . 4δ0 (1 + δ0 )

(5.19)

(5.20)

Since in a binomial distribution with parameter π ∈ (0, 1), the probability of any set of the form (5.19) is well known to be an increasing function of π (cf. Johnson et al., 1992, p. 117), it follows that as long as the very mild condition (5.20) is satisfied, the size of the test with critical region (5.16) can be computed from 0 SIZEα (δ0 ; n) = sup Qδα,n (1/2 − δ0 /2η, η) .

(5.21)

δ0 −δ0 based on (5.22) is fairly easy to compute and admits exact calculation of its size and power against any specific alternative (p10 , p01 ) with δ ≡ p10 − p01 > ˜0 ) ∼ M(n; p10 , p01 , p˜0 ), the joint posterior −δ0 . Under the model (N10 , N01 , N distribution associated with this prior is well known (see, e.g., Berger, 1985, pp. 287,561) to be again Dirichlet, but with data-dependent parameters (n10 + ˜ 0 + 1/2). The corresponding density function is 1/2, n01 + 1/2, n ˜0) = g(p10 , p01 | n10 , n01 , n

Γ(n + 3/2) Γ(n10 + 1/2)Γ(n01 + 1/2)Γ(˜ n0 + 1/2) n

· p1010

−1/2

n

p0101

−1/2

(1 − p10 − p01 )n˜ 0 −1/2 . (5.23)

Making use of some of the basic properties of Dirichlet distributions (see Kotz et al., 2000, § 49.1), the posterior probability of the subset K1 ≡ {(p10 , p01 ) | 0 < p10 + p01 < 1 , p10 − p01 > −δ0 } of the parameter space specified by the noninferiority hypothesis K1 to be established can be written P [K1 | n10 , n01 , n ˜ 0 ] = Iδ0 (n01 + 1/2, n − n01 + 1) +  (1+δ0 )/2    n −1/2 B n01 + 1/2, n − n01 + 1/2 p0101 (1 − p01 )n−n01 −1/2 δ0

 · 1 − I(p01 −δ0 )/(1−p01 ) (n10 + 1/2, n ˜ 0 + 1/2) dp01 , (5.24) where B(a, b) Iu (a, b)

Γ(a + b) , a,b > 0, Γ(a) Γ(b) ≡ cdf at u ∈ (0, 1) of a beta distribution with parameters a, b > 0 .

≡

Equivalence tests for the McNemar setting

89

The integral on the right-hand side of this equation is just one-dimensional and can thus be evaluated by means of a standard quadrature procedure with comparably low computational effort. Again, all numerical results to be presented about the test based on (5.24) were obtained by means of 96-point Gauss-Legendre integration. In order to ensure a high degree of numerical accuracy, the range of integration was partitioned into 10 subintervals over each of which integration was done separately using this rule. Even in this high-accuracy version, the algorithm is fast enough for allowing us to search ˜0 ) checking each of its points through the whole sample space of (N10 , N01 , N for inclusion in the critical region

  (n10 , n01 , n ˜ 0 ) ∈ IN30 n10 +n01 +˜ n0 = n, P [K1 | n10 , n01 , n ˜ 0 ] ≥ 1−α . (5.25) As before, it is more convenient to represent this region as a subset of the sample space of the statistic (U, V ) introduced in (5.10) yielding

  (u, v) ∈ IN20  u ≤ v, P [K1 | u, v − u, n − v] ≥ 1 − α . (5.26) The remaining steps to be carried out in order to determine a largest nominal significance level α∗ which is admissible in the sense of generating a rejection region whose probability maximized over the null hypothesis H1 remain below the target level α, are the same as before for the Wald type test: Again, irrespective of the choice of the nominal significance level in (5.26) and for any v = 0, . . . , n, the set of all u satisfying the critical inequality P [K1 | u, v −u, n−v] ≥ 1−α turns out to be of the form (5.19). Consequently, the size of the critical region (5.26) can be computed in the same way as before [recall (5.21)], and the corrected nominal level α∗ is readily found by bisection. A tool for implementing the complete computational procedure which underlies the objective Bayesian solution of the problem of constructing an exactly valid test for noninferiority in terms of δ for the matched-pair design with binary data, has been made available under the program name mcnby_ni. The program is designed to output exact values of the power against specific null alternatives, i.e., parameter constellations with δ = 0 in addition. The selection of numerical results shown in Tables 5.9 and 5.10 have been computed by means of this tool for the same constellations covered by the preceding pair of tables. For a brief illustration of the level-corrected asymptotic and the objective Bayesian test for noninferiority, let us reanalyze Table 5.4 using a noninferiority margin of δ0 = .05 . With this specification, the Wald type statistic Tnδ0 (u, v) is found to have value 0.86734 which is clearly insignificant. Evaluating the integral on the right-hand side of (5.24) for n = 72, δ0 = .05, n10 = 4, n01 = 5 through running the program mcnby_ni_pp which serves the purpose of computing the posterior probability of K1 for individual points in the sample space, yields P [K1 | n10 , n01 , n ˜ 0 ] = .812488 < .95 so that the null hypotheses H1 : δ < .05 of substantial inferiority of the “combined referee” to

90

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

the computer-aided diagnostic procedure has to be accepted by both tests. Table 5.9 Nominal significance level and exact size for the objective Bayes solution to the problem of testing for noninferiority in terms of δ with a sample of paired binary variables of size n ∈ {50, 60, 80, 100, 150, 200} for δ0 = .05 and δ0 = .10. [Number in ( ): size of the noncorrected test when the nominal level is chosen equal to the target significance level α = .05.] δ0 n α∗ .05 50 .0365381 " 60 .0395752 " 80 .0299854 " 100 .0441748 " 150 .0244629 " 200 .0320557

SIZEα∗ (δ0 ; n) .0486168 (.0578281) .0432548 (.0696555) .0330561 (.0688144) .0497734 (.0573983) .0347813 (.0627488) .0393598 (.0672558)

.10 50 .0421094 " 60 .0204492 " 80 .0387744 " 100 .0285938 " 150 .0342871 " 200 .0346631

.0481380 .0258535 .0463279 .0364573 .0396826 .0435619

(.1056926) (.0630695) (.0866751) (.0626736) (.0628134) (.0637456)

Table 5.10 Power of the objective Bayesian test for one-sided δequivalence of binomial proportions with paired data against different alternatives with δ = 0 and the same choices of δ0 and n as in Table 5.9. n .0002 .002 δ0 .05 50 .004963 .046513 " 60 .005947 .055016 " 80 .007968 .076695 " 100 .019755 .177215 " 150 .029557 .258976 " 200 .039214 .329947

.02 .267703 .289644 .454858 .724317 .879858 .968756

η= .20 .170682 .191543 .189916 .287333 .274944 .391696

.30 .133745 .145117 .143495 .213761 .199468 .291472

.50 .101893 .115914 .108030 .162102 .136064 .199715

.80 .084922 .097100 .084760 .125306 .099494 .144822

.10 50 .009939 .094122 " 60 .011912 .111577 " 80 .015874 .147991 " 100 .019803 .181433 " 150 .029557 .259404 " 200 .039214 .329948

.575710 .625577 .796701 .865856 .951697 .982412

.436718 .386520 .604435 .627323 .818218 .906756

.347309 .279759 .459459 .474800 .666396 .784520

.240782 .183838 .320297 .323541 .471277 .576421

.178784 .120505 .223178 .217856 .330827 .411560

Equivalence tests for the McNemar setting

91

Discussion Comparing the objective Bayesian with the level-corrected asymptotic test in terms of the power achieved under the specific alternatives appearing in Tables 5.8 and 5.10 does not lead to a clear-cut preference: As holds typically true for any exactly valid nonrandomized test of a hypothesis relating to a family of discrete distributions, both tests fail to be unbiased in that their power converges to zero as the probability η of observing a discordant pair [i.e., finding that (Xi , Yi ) equals either (1, 0) or (0, 1)] becomes arbitrarily small. Furthermore, for both tests the power against fixed alternatives considered as a function of the sample size n, fails to be monotonically increasing although the exceptions do not seem to be too serious an issue from a practical perspective. Looking at the rowwise maxima of the values shown in the tables, the Bayesian test comes out to be clearly superior. However, these maxima are attained against the alternative p10 = p01 = .01, and settings where the true proportion of discordant pairs is as low as 2% will rarely if ever occur in real applications. For the other, much more realistic alternatives, the powers of both tests do not differ substantially from each other. The journal literature contains several proposals to address the problem of constructing exact nonconditional tests for one-sided equivalence of correlated binomial proportions from a conceptually different perspective: Instead of adjusting the nominal level as far as necessary for ensuring that the supremum of the probability of the resulting rejecting region taken over the nuisance parameter η does not exceed the target value of α, maximized p-values are compared to the latter. In the work of Sidik (2003) (for the special case δ0 = 0 of the traditional one-sided testing problem see also Berger and Sidik, 2003), the technique introduced by Berger and Boos (1994) is adopted, which is to say that p-value maximization is done over a confidence interval rather than the whole range of η, adding the risk of noncoverage to the result as a correction term. The test proposed by Lloyd and Moldovan (2008) is based on so-called E+M p-values (for a general discussion and other applications of this concept see Lloyd, 2008a, b) which are obtained in two steps: Suppose that some “starting p-value” has been selected whose computation  from the data (U, V ) involves no unknown parameters, e.g., p(u, v) = 1 − Φ Tnδ0 (u, v) the beginning of this with Tnδ0 (u, v) defined as at   subsection. In a first step, the probability of the event p(U, V ) ≤ p(u, v) that this p-value falls below its observed value, is evaluated replacing the nuisance parameter η with a suitable point estimator η˜ (which might depend on the noninferiority margin δ0 ). Denoting the result by P˜ (u, v), the second step  consists of finding  the maximum, say P ∗ (u, v), of the exact probability of P˜ (U, V ) ≤ P˜ (u, v) taken over the whole boundary of the null hypothesis H1 , i.e., all values of η being compatible with δ = −δ0 . Finally, the rejection region consists of all values (u, v) in the sample space of (U, V ) satisfying P ∗ (u, v) ≤ α . Obviously, all approaches involving maximized p-values are computationally much more demanding than those based on corrected nominal significance

92

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

levels since the maximization process applies to the individual points in the sample space rather than the rejection region as a whole. The return in terms of power which the user of these tests can expect for this extra computational effort, is only moderate: For δ0 = .05, n = 50 and η ∈ {.20, .30, .50}, Sidik (2003) obtained the power values .175, .154 and .119 which are not much different from the respective entries in Tables 5.8 and 5.10. Sensible comparisons with the E+M test proposed by Lloyd and Moldovan (2008) are difficult since these authors provided data about power only in graphical, not in numerical form and for the choice α = .10 which is very unusual in real applications. Roughly speaking, the most substantial differences will occur under small values of the probability η of obtaining a discordant pair, reflecting the fact that the power of the E+M test converges to unity rather than zero as η approaches 0. This is made possible by including the point (u, v) = (0, 0) in the rejection region implying that the power approaches unity even for n = 1. Since attaining maximum power with minimum sample size is clearly counter to common statistical sense, Lloyd and Moldovan’s test cannot be recommended for real applications even if computational burden is not an issue.

5.3

Paired t-test for equivalence

In this section, we assume throughout that the setting described in § 5.1 has to be considered in the special version that the distribution of the intraindividual differences Di between the measurements taken under both treatments is Gaussian with (unknown) parameters δ = E(Di ) and σD2 = Var(Di ) . In the special case (5.27) Di ∼ N (δ, σD2 ) ∀ i = 1, . . . , n , the hypothesis of equivalence with respect to the probability of a positive sign [cf. (5.1), (5.4)] reduces to a statement about the standardized expected value δ/σD . In fact, under the parametric model (5.27), we have p0 = 0 and hence p− = 1 − p+ . Consequently, we may write: −ε1 < log(p+ /p− ) < ε2 ⇔ −ε1 < log(p+ /(1 − p+ )) < ε2 ⇔ e−ε1 < p+ /(1 − p+ ) < eε2 ⇔ e−ε1 /(1 + e−ε1 ) < p+ < eε2 /(1 + eε2 ). On the other hand, under (5.27), p+ admits the representation p+ = P [Di > 0] = 1 − P [(Di − δ)/σD < −δ/σD ] = 1 − Φ(−δ/σD ) = Φ(δ/σD ). Since the standard normal distribution function Φ is strictly increasing and continuous, we eventually see that under (5.27), the hypothesis −ε1 < log(p+ /p− ) < ε2 to be established by means of the sign test for equivalence is logically equivalent to Φ−1 (e−ε1 /(1 + e−ε1 )) < δ/σD < Φ−1 (eε2 /(1 + eε2 )). Hence, setting for brevity Φ−1 (e−ε1 /(1 + e−ε1 )) = θ1 , Φ−1 (eε2 /(1 + eε2 )) = θ2 , the testing problem which in its nonparametric form was dealt with in § 5.1, can now be written H : δ/σD ≤ θ1 ∨ δ/σD ≥ θ2

vs.

K : θ1 < δ/σD < θ2 .

(5.28)

Paired t-test for equivalence

93

Evidently, this problem does not change if the observations Di are rescaled by multiplying each of them by the same positive constant c. Put in more technical terms, this means that the parametric equivalence testing problem (5.28) remains invariant under all transformations of the sample space IRn of (D1 , . . . , Dn ) taking the form of (d1 , . . . , dn ) → (cd1 , . . . , cdn ), with arbitrary c > 0. As is shown in Lehmann and Romano (2005, § 6.4), the construction of a test of the ordinary one-sided null hypothesis about δ/σD which is uniformly most powerful among all tests being invariant against all transformations of that type, conveniently starts from reducing the data (D1 , . . . , Dn ) to the suf¯ SD ) before applying the principle of invariance. The same ficient statistics (D, two-step reduction can be carried out in the present context which implies that a uniformly most powerful invariant (UMPI) level α test for (5.28) is obtained in the following way: After reducing the primary data set (D1 , . . . , Dn ) to the usual one-sample t-statistic √ ¯ D, (5.29) T = nD/S a UMP level-α test based on T is carried out for √ √ √ √ ˜ : θ˜ ≤ nθ1 ∨ θ˜ ≥ nθ2 vs. K ˜ : nθ1 < θ˜ < nθ2 , H

$ (5.28)

where θ˜ denotes the parameter of a noncentral t-distribution with n−1 degrees of freedom. Denoting the density function of that distribution by gθ˜(·), we know from a result derived in Karlin (1968, §4(iii)) [cf. also Lemma A.1.3 of Appendix A to this book] that the family (gθ˜)θ∈ ˜ IR is strictly positive of any order and ˜ t), g ˜(t) admits the hence a fortiori STP3 . Furthermore, for arbitrary (θ, θ representation ˜2 /2

gθ˜(t) = e−θ

(n − 1 + t2 )−n/2

∞ 

˜ j , cj [h(t, θ)]

(5.30)

j=0

where

√  ˜ = tθ˜ 2/ n − 1 + t2 h(t, θ)

∞

(5.31)

and the power series j=0 cj x converges for any x ∈ IR (see Johnson et al., 1995, p. 516). This ensures continuity of gθ˜(t) in both of its arguments so that the family (gθ˜)θ∈ ˜ IR of noncentral t-densities with an arbitrary number (n − 1) of degrees of freedom satisfies all the conditions of Theorem A.1.5 [ → Appendix, p. 371]. Hence, we may conclude that an UMP level-α test for $ exists and is given by the critical region (5.28) 

1 2 ˜ C˜α; (θ , θ ) < T < C (θ , θ ) , (5.32) n−1 1 2 α; n−1 1 2 j

ν where the bounds C˜α; n−1 (θ1 , θ2 ) are uniquely determined by the equations

Gθ˜1 (C2 ) − Gθ˜1 (C1 ) = α = Gθ˜2 (C2 ) − Gθ˜2 (C1 ) ,

−∞ < C1 < C2 < ∞ . (5.33)

94

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

In (5.33), Gθ˜(·) denotes for any θ˜ ∈ IR the cumulative distribution function √ associated to gθ˜(·) , and θ˜ν has to be set equal to nθν both for ν = 1 and ν = 2 . Both in view of the form of the statistic T and the distribution which the computation of the critical constants has to be based upon, the term paired t-test seems fully appropriate for the testing procedure given by (5.32). ˜ t) ∈ A glance at (5.30) and (5.31) shows that we have g−θ˜(−t) = gθ˜(t) ∀(θ, 2 ˜ ˜ IR . Hence, for any θ > 0, the distribution of −T under −θ coincides with that of T under θ˜ so that for a symmetric choice of the equivalence interval, ε, θ˜2 = ε˜ with ε > 0, i.e., √ for θ1 = −ε, θ2 = ε and correspondingly for θ˜1 = −˜ ε˜ = nε, the conditions of Lemma A.1.6 are satisfied as well. Applying this result we see that in the symmetric case, (5.32) and (5.33) can be simplified to

 |T | < C˜α; n−1 (ε) (5.34) and Gε˜(C) − Gε˜(−C) = α, 0 < C < ∞ ,

(5.35)

respectively, where C˜α; n−1 (ε) stands for the solution of the latter equation. Furthermore, it is seen at once that the expression on the left-hand side of (5.35) is equal to the probability of the event {|T | < C} = {T 2 < C 2 } under θ˜ = ε˜. Since it is a well known fact (cf. Johnson et al., 1995, loc. cit.) that squaring a random variable following a t-distribution with n − 1 degrees of freedom and noncentrality parameter ε˜ yields a variable which is noncentral F with 1, n − 1 degrees of freedom and ε˜2 as the noncentrality parameter, it follows that the solution to (5.35) admits the explicit representation  1/2 C˜α; n−1 (ε) = F1,n−1; α (˜ ε2 ) ,

(5.35 )

ε2 ) denotes the quantile function of that F -distribution. where F1,n−1; · (˜ Due to the relationship (5.35 ), when the equivalence hypothesis is formulated symmetrically setting −θ1 = θ2 = ε > 0, the practical implementation of the paired t-test for equivalence providing an UMPI solution to (5.28), involves only slightly more effort than that of the ordinary one- or two-sided t-test for paired observations. In fact, from the viewpoint of any user having access to up-to-date statistical software, the noncentral versions of the basic sampling distribution functions as well as their inverses, can be considered as explicitly given. In particular, the SAS system provides an intrinsic function named finv returning specific quantiles of F -distributions with arbitrary numbers of degrees of freedom and values of the noncentrality parameter with very high numerical accuracy (for details about the syntax to be used in calling this routine and the underlying algorithm see the SAS language reference dictionary). In the Stats Package of R, a program serving the same purpose can be run calling the function named qf. Table 5.11 enables the reader to perform the test in the symmetric case without any computational tools at

Paired t-test for equivalence

95

the 5% level, for sample sizes up to 100 being multiples of 10 and equivalence margin ε = .25(.25)1.00. The corresponding maximum rejection probabilities β˜α; n−1 (ε) attained at δ = 0 (⇔ P [Di > 0] = 1/2) are shown in Table 5.12. The latter are readily computed by means of the central t-distribution function G0 (·), say, using the formula β˜α; n−1 (ε) = 2 G0 (C˜α; n−1 (ε)) − 1 .

Table 5.11 Critical constant C˜.05; n−1 (ε) of the one-sample t-test for equivalence at level α = 5% in the symmetric case (θ1 , θ2 ) = (−ε, ε), for ε = .25(.25)1.00 and n = 10(10)100. ε = n

.25

.50

.75

1.00

10 20 30 40 50 60 70 80 90 100

0.08811 0.11855 0.16079 0.21752 0.29164 0.38432 0.49319 0.61250 0.73573 0.85817

0.22188 0.61357 1.08722 1.50339 1.87104 2.20387 2.51030 2.79581 3.06421 3.31826

0.73424 1.67161 2.40334 3.02359 3.57213 4.06944 4.52773 4.95499 5.35682 5.73729

1.46265 2.70944 3.68005 4.50470 5.23475 5.89697 6.50742 7.07666 7.61210 8.11913

Table 5.12 Power attained at θ = δ/σD = 0 when using the critical constants shown in Table 5.11 for the one-sample t-statistic. ε = n

.25

.50

.75

1.00

10 20 30 40 50 60 70 80 90 100

0.06828 0.09313 0.12662 0.17106 0.22821 0.29787 0.37656 0.45803 0.53617 0.60713

0.17064 0.45323 0.71411 0.85921 0.93268 0.96855 0.98559 0.99350 0.99711 0.99873

0.51851 0.88901 0.97713 0.99560 0.99919 0.99986 0.99998 1.00000 1.00000 1.00000

0.82241 0.98610 0.99905 0.99994 1.00000 1.00000 1.00000 1.00000 1.00000 1.00000

(5.36)

96

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

Example 5.3 In an experimental study of the effects of increased intracranial pressure on the cortical microflow of rabbits, a preliminary test had to be done to ensure that the measurements would exhibit sufficient stability during a pre-treatment period of 15 minutes’ duration. In a sample of n = 23 animals, at the beginning and the end of that period the mean flow [ml/min/100g body mass] ¯ = 52.65 and Y¯ = 52.49, respectively, corresponding was observed to be X ¯ = 0.16; the standard deviation of the intraindividual to a mean change of D ock and Kempski, changes was computed to be SD = 3.99 (data from Ungersb¨ 1992). The equivalence limits for the standardized expected change of the microflow were chosen to be ∓ε with ε =.50. For the values of n and ε applying to the present example, the noncentrality parameter of the F -distribution to be referred to for computing the critical upper bound to the absolute value of the t-statistic, is ψ 2 = 23 · 0.502 = 5.75. In order to compute the critical bound using SAS, one has to write in a data step just the single statement csquare=finv(.05,1,22,5.75) which yields the output:√ csquare=.576779 . From this, the critical constant itself is C˜.05;22 (.50) = .576779 = .7595 . On the other hand, for a sample of size n = 23, the t-value corresponding to the observed point (0.16, 3.99) in the space of the sufficient statistic ¯ SD ), is .1923. Thus, we have |T | < C˜.05;22 (.50) indeed which implies that (D, ¯ SD ) belongs to the rejection region of our test. Acthe observed value of (D, cordingly, we can conclude that at the 5% level, the experimental data of the present example contain sufficient evidence in support of the hypothesis that the cortical microflow of rabbits does not change to a relevant extent over a time interval of 15 minutes during which no active treatment is administered. The power of the UMPI test at level α = .05 for (5.28) with θ1 = −.50, θ2 = .50 and sample size n = 23 is computed by means of formula (5.36) at 54.44%. If the equivalence hypothesis K of (5.28) is formulated in a nonsymmetric ν way, computation of the critical constants C˜α; n−1 (θ1 , θ2 ), ν = 1, 2, of the one-sample t-test for equivalence becomes considerably more complicated. It requires an iterative algorithm, then, for solving the system (5.33) of equations numerically. An iteration procedure for accomplishing this task can be obtained by suitable modifications to the algorithm described in § 4.2 for the one-sample equivalence testing problem with exponentially distributed data. Γ (·) being the baIt suffices to replace the gamma distribution function Fσ,n sic building block there, by the noncentral t-distribution function Gθ˜ν (·) (for ν = 1, 2) and redefine the initial value C1◦ of C1 . Extensive experience has √ ◦ shown that C1 = (θ˜1 + θ˜2 )/2 = n(θ1 + θ2 )/2 is a sensible choice. The program tt1st which is again made available both as a SAS macro and a R function in the WKTSHEQ2 Source Code Package implements this computational procedure for determining the critical constants of the one-sample t-test for equivalence in cases of an arbitrary choice of the equivalence range for δ/σD .

Paired t-test for equivalence

97

Before moving on to a description of a the noninferiority analogue of the paired t-test for equivalence, a remark on the possible role of the test discussed on the preceding pages in bioequivalence assessment seems in place: Under a set of conditions satisfied by the majority of comparative bioavailability trials conducted in practice (lognormality of the intraindividual bioavailability ratios; negligibility of period effects), the procedure has much to recommend it as an alternative to the conventional interval inclusion approach to so-called average bioequivalence [a more elaborate argument for this view will be given in Ch. 10] .

Paired t-test for noninferiority In the noninferiority version of the testing problem considered in this section, the hypotheses read H1 : δ/σD ≤ −ε

versus

K1 : δ/σD ≥ −ε ,

(5.37)

where, as before, ε stands for a prespecified positive number. The same kind of argument which lead us to use the critical region (5.32) for testing for twosided equivalence with respect to δ/σD shows that in order to perform a UMPI level-α test for (5.37), we have to reject the null hypothesis H1 if and only if it turns out that  √  (5.38) T > tn−1;1−α − nε . According to this critical inequality, the standard paired-sample t-statistic has to be compared with the (1 − α)-quantile of a noncentral √ t-distribution with density obtained by making the substitution θ˜ = − nε in (5.30–1). Computation of these quantiles is as easy as of the noncentral F -quantiles to be used in the equivalence version of the test with symmetric choice of the margins. In SAS, the intrinsic function named tinv serves this purpose, and again, this has a direct counterpart in R, namely the function qt. The rejection probability under√an arbitrary parameter constellation (δ, σD ) is given by   1 − G√nθ tn−1;1−α (− nε) with θ = δ/σD and Gθ˜(·) denoting the noncentral t-distribution function with generic nc-parameter θ˜ and df = n−1. In particular, the power against the null alternative θ = 0 ⇔ δ = 0 is simply obtained √ by evaluating the central t-distribution function G0 (·) at t = −tn−1;1−α (− nε) . Making use of these tools and relations, it is easy to generate tables of critical constants and power values. For the same selection of sample sizes as considered in the two-sided case and equivalence margins ε ∈ {k/10 | k = 1, . . . , 5}, Table 5.13 shows the critical bound to which the t-statistic has to be compared in the UMPI test of (5.37) at level α = .05. The power against θ = 0 for the same sample sizes and specifications of the equivalence margin can be read from Table 5.14. The major differences between the one- and the two-sided versions of the paired t-test for equivalence become apparent when the entries in the last

98

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS  √  Table 5.13 Critical constant tn−1;.95 − nε of the one-sample t-test for noninferiority at level α = 5% for ε = .10(.10).50 and n = 10(10)100.

n 10 20 30 40 50 60 70 80 90 100

.10

.20

1.45767 1.24639 1.12433 1.03000 0.95008 0.87932 0.81507 0.75577 0.70039 0.64823

1.09339 0.77315 0.55857 0.38394 0.23227 0.09613 −0.02856 −0.14435 −0.25293 −0.35552

ε = .30 0.74018 0.30929 0.00170 −0.25346 −0.47699 −0.67861 −0.86382 −1.03613 −1.19794 −1.35099

.40 0.39776 −0.14545 −0.54652 −0.88244 −1.17798 −1.44515 −1.69094 −1.91983 −2.13492 −2.33845

.50 0.06570 −0.59145 −1.08647 −1.50337 −1.87104 −2.20387 −2.51030 −2.79581 −3.06421 −3.31826

Table 5.14 Power against the null alternative θ = 0 attained in the paired t-test for noninferiority for the sample sizes and equivalence-margin specifications covered by Table 5.13.

n

.10

.20

ε = .30

.40

.50

10 20 30 40 50 60 70 80 90 100

0.08946 0.11388 0.13505 0.15468 0.17337 0.19140 0.20892 0.22602 0.24276 0.25917

0.15131 0.22447 0.29037 0.35156 0.40865 0.46187 0.51135 0.55720 0.59955 0.63852

0.23903 0.38023 0.49933 0.59938 0.68226 0.74998 0.80466 0.84835 0.88294 0.91011

0.35004 0.55706 0.70556 0.80853 0.87775 0.92315 0.95232 0.97076 0.98224 0.98931

0.47453 0.71940 0.85689 0.92960 0.96634 0.98428 0.99279 0.99675 0.99856 0.99937

column of Table 5.14 are compared to those in the middle column of Table 5.12. Under the same choice of the left-hand endpoint of the equivalence range specified by the alternative hypothesis, the noninferiority test is, uniformly in n, more powerful than the test for equivalence in the two-sided sense. Of course, this statement is by no means at variance with the fact, that for n ≥ 50, the lower critical bound to the t-statistic is the same for both tests

Signed rank test for equivalence

99

(except for negligible differences less than .5 × 10−6 in absolute value): Due to the existence of a finite right-hand critical bound, the rejection region of the equivalence test is necessarily a proper subset of that of the test for noninferiority.

5.4

Signed rank test for equivalence

The testing procedure to be derived in the present section is a natural nonparametric competitor to the paired t-test for equivalence as discussed in § 5.3. It will be obtained by modifying the most frequently applied nonparametric test for paired data in such a way that the resulting test for equivalence is both interpretable in terms of a sensible measure of distance between distribution functions on IR and fairly easy to implement in practice. The assumptions which the construction to be described on the pages to follow starts from, are as weak as those underlying the sign test for equivalence with continuously distributed intraindividual differences Di . This means that, in contrast to the “classical” signed rank test of the hypothesis of symmetry of the distribution of the Di , the distribution function F of the Di is not assumed symmetric, neither under the null nor the alternative hypothesis. Basically, this possibility of enlarging the class of admissible distributions is a consequence of the fact that the procedure we propose to term signed rank test for equivalence, is asymptotic in nature. However, we will be able to show by means of the results of suitable simulation experiments that under surprisingly mild restrictions on the order of magnitude of the sample size n, the discrepancies between the nominal level of significance and the actual size of the test are practically negligible even if the distribution of the Di exhibits gross asymmetry. As is well known from textbooks on nonparametric statistics (see, e.g., Lehmann, 1975, p. 129, (3.16)) for continuously distributed Di , the test statistic Vs , say, of the ordinary signed rank test admits the representation Vs =

n  i=1

I(0,∞) (Di ) +

n−1 

n 

I(0,∞) (Di + Dj ) ,

(5.39)

i=1 j=i+1

with I(0,∞) (·) as the indicator of a positive sign of any real number defined formally by  1 for x > 0 I(0,∞) (x) = . (5.40) 0 for x ≤ 0 From (5.39) it is obvious that the finite-sample expectation of the signed rank statistic is a weighted sum of the two parameters (functionals) p+ = P [Di > 0] and (5.41) q+ = P [Di + Dj > 0] .

100

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

On the other hand, it seems reasonable to formulate the equivalence hypothesis to be established by means of the test to be constructed in the present section, in terms of the mean of the asymptotic distribution of the customary signed rank statistic. In view of this, it is preferable to drop the first term on the right-hand side of (5.39) and base the construction on the statistic U+ =

n   n−1 n−1 2

I(0,∞) (Di + Dj )

(5.42)

i=1 j=i+1

instead of Vs . In fact, U+ is an unbiased estimator of the probability q+ of getting a positive sign of the so-called Walsh-average within any pair of D’s and is asymptotically equivalent to Vs , in the sense that the difference between the standardized versions of both statistics vanishes in probability for n → ∞ under any specification of the true distribution of Di (cf. Randles and Wolfe, 1979, pp. 84-5). Therefore, as an asymptotic procedure, the conventional signed rank test is a test of a hypothesis on q+ and accordingly, we refer throughout this section to the following equivalence testing problem:     H : 0 < q+ ≤ q+ ∨ q+ ≤ q+ < 1 versus K : q+ < q+ < q+ ,

(5.43)

  where the limits q+ , q+ of the equivalence range can be chosen as arbitrary   < q+ < 1. The exact finite-sample variance fixed numbers satisfying 0 < q+ of U+ admits the representation

Var[U+ ] =

n−1 2

+ 2 {2(n − 2)[q1(2,3) − q+ ] + q+ (1 − q+ )} ,

(5.44)

with + q1(2,3) = P [Di + Dj > 0, Di + Dk > 0]

(5.45)

+ (cf. Randles and Wolfe, 1979, (3.1.21)). For q1(2,3) , there exists a natural estimator as well which exhibits the form of a U -statistic and is given by

+ = qˆ1(2,3)

 n−1  n−1 n−2 3

n 

 (1/3) I(0,∞) (Di + Dj ) ·

i=1 j=i+1 k=j+1

I(0,∞) (Di + Dk ) + I(0,∞) (Di + Dj )I(0,∞) (Dj + Dk ) +

 I(0,∞) (Di + Dk )I(0,∞) (Dj + Dk ) . (5.46)

From the general theory of U -statistic estimators, it is in particular known + + as defined by (5.46) is consistent for q1(2,3) (see once more Randles that qˆ1(2,3) and Wolfe, 1979, Corollary 3.2.5) which implies that the variance of U+ can in turn be estimated consistently by σ ˆ 2 [U+ ] =

n−1 2

+ 2 {2(n − 2)[ˆ q1(2,3) − U+ ] + U+ (1 − U+ )} .

(5.47)

Signed rank test for equivalence

101

 Since (U+ − q+ )/ Var[U+ ] is asymptotically standard normal for any distribution of the Di under which Var[U+ ] does not vanish, we may apply the result presented in § 3.4 yielding the decision rule Reject nonequivalence ⇔         + q+ )/ˆ σ [U+ ] < C±R (α; q+ , q+ ), U+ − (1/2)(q+

(5.48)

with

  , q+ ) = 100α−percentage point of the χ2 − distriC±R (α; q+

1/2   2 bution with df = 1 and λ2nc = (q+ − q+ ) /4ˆ σ 2 [U+ ] . (5.49)

Although the quantile function of any noncentral χ2 -distribution is predefined in well-known software packages for use in statistics, the practical implementation of the signed rank test for equivalence based on (5.48–9) entails considerably more effort than that of the conventional one- and two-sided Wilcoxon test for paired observations. The reason is that in addition to U+ + or Vs , the quantity qˆ1(2,3) not appearing in the “ordinary” signed rank test, is needed. Hence, at the URL associated with this book, the reader may find another SAS macro [program name: sgnrk] designed for carrying out all computational steps to be taken in an application of (5.48). As input data, it requires in addition to the size n of the sample, the nominal level α of signifi  , q+ of the equivalence range for the target parameter cance and the limits q+ q+ , only the name of a raw-data file containing just the observed values of the Di . Example 5.4 In a pilot study preceding a large-scale multicenter trial of the antihypertensive efficacy of various drugs, the aim was to establish comparability of the measurements of blood pressure (BP) taken with two different automatic devices (denoted by A and B in the sequel). In each of a total of 20 volunteers participating in the pilot study, 12 repeated BP measurements were taken during the same session, 6 using each of the two devices. The temporal spacing of all individual measurements was strictly uniform, with an interval of 2 minutes between consecutive readings and altering between A and B. The 6 values produced by each device within one such series were collapsed to a single one by simple averaging. For the analysis of the data of this pilot study, the comparison of both measurement devices with respect to the diastolic BP (DBP) was of primary interest. Correspondingly, Table 5.15 shows for each subject the averages of 6 DBP values obtained by the first device (A) as compared to B. The lim  = .2398, q+ = .7602, its of the equivalence range for q+ were set equal to q+

102

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

Table 5.15 Results of a pilot study on the comparability of two different devices A (↔ Xi ) and B (↔ Yi ) for measuring diastolic blood pressure. i

Xi

Yi

1 62.167 62.667 2 85.667 85.333 3 80.667 80.000 4 55.167 53.833 5 92.000 93.500 6 91.000 93.000 7 107.833 108.833 8 93.667 93.833 9 101.167 100.000 10 80.500 79.667

Di −0.500 0.333 0.667 1.333 1.500 −2.000 −1.000 −0.167 1.667 0.833

i

Xi

11 74.500 12 91.667 13 73.667 14 63.833 15 80.333 16 61.167 17 63.167 18 73.167 19 103.333 20 87.333

Yi 76.667 93.500 69.167 71.333 77.667 57.833 67.333 67.500 101.000 89.833

Di −2.167 −1.833 4.500 −7.500 2.667 3.333 −4.167 5.667 2.333 −2.500

which can be motivated as follows: In the specific case that the Di are nor2 mal with expectation δ and √ variance σD , it is easily verified that there holds the relationship q+ = Φ( 2 δ/σD ), and this suggests to transform the interval (−.5, .5) making up a reasonable equivalence range in terms of δ/σ √ √    D [recall Tab. 1.1 (iv)], to the set q+  Φ(− 2 δ/σD ) < q+ < Φ( 2 δ/σD ) = (.2398, .7602) . Now, in order to test for equivalence of the measuring devices A and B,   and q+ ,α= we apply the above decision rule (5.48) with these value of q+ .05, and the n = 20 intra-subject differences shown in the above table as the observations eventually to be analyzed. Running the program sgnrk we ˆ [U+ ] = .12071, C±R (.05; .2398, .7602) = 0.54351 , and find U+ = .55263, σ   + q+ )|/ˆ σ [U+ ] = |.55263 − .50000|/.12071 = it follows that |U+ − (1/2)(q+ .43600 < C±R (.05; .2398, .7602). Hence, we can reject the null hypothesis of nonequivalence of the two measuring devices under comparison. For the remaining part of this section, we restrict consideration to the wellknown semiparametric shift model assuming that the density and cumulative distribution function of the intraindividual differences Di is given by z → f◦ (z − ϑ) and z → F◦ (z − ϑ) with fixed (though unspecified) baseline f◦ : IR → [0, ∞) and F◦ : IR → [0, 1], respectively. Even under this comparatively simple submodel, exact computation of rejection probabilities of the test (5.48) is unfeasible so that for purposes of investigating the finite-sample power function of the signed rank test for equivalence we have recourse to simulation methods. A question of particular interest to be answered by this way concerns the possible dependence of the power against some specific alternative in terms of the target functional q+ , on the form of the baseline distribution function F◦ . Let us denote by β(F◦ ; q+ ) the probability that the test rejects its null hypothesis if it is performed with data D1 , . . . , Dn from a distribution which

Signed rank test for equivalence

103

belongs to the location parameter family generated by baseline cdf F◦ such that the true value of P [Di +Dj > 0] is q+ . Clearly, in a simulation experiment carried out in order to determine β(F◦ ; q+ ), we have to generate sets of values of random variables of the form Di = Di◦ + ϑ(q+ ) where Di◦ has distribution function F◦ for all i = 1, . . . , n, and ϑ(q+ ) is the unique solution of the equation  ∞ [1 − F◦ (−2ϑ − z)]dF◦ (z) = q+ , ϑ ∈ IR . (5.50) −∞

Given some specific baseline cdf F◦ , the expression on the left-hand side of (5.50) is a function, say w+ (ϑ), of the location parameter which admits an explicit representation only for some of the most common location families. Table 5.16 gives a summary of the pertinent formulae. Making use of these relationships, for the models covered by the table, the solution ϑ(q+ ) to (5.50) can either be written down explicitly (Gaussian, Cauchy, uniform, exponential distribution) or computed numerically by means of a simple interval halving algorithm (Laplace, logistic distribution). Furthermore, it follows immediately from a general result of advanced mathematical analysis (see Pratt, 1960, Corollary 3) that w+ (ϑ) is a continuous function of ϑ in each location parameter family generated by a continuously differentiable baseline cdf F◦ (·).   Hence, to each specification of the equivalence range (q+ , q+ ) for the target functional q+ , there corresponds a unique interval (ϑ1 , ϑ2 ), say, of values of Table 5.16 Explicit representation of w+ (ϑ) = P [ (Di◦ +ϑ)+(Dj◦ +ϑ) > 0 ] for independent random variables Di◦ , Dj◦ with common density f◦ (·). Location

f◦ (z)

w+ (ϑ) = ∞

Family

−∞

2

√ Φ( 2ϑ)

[1 − F◦ (−2ϑ − z)] dF◦ (z)

Gaussian

(2π)−1/2 e−z

Cauchy

π −1 (1 + z 2 )−1

1/2 + π −1 arctan(ϑ)

Uniform∗

I(−1/2 , 1/2) (z)

1/2 + 2ϑ(1 − ϑ) for 0 ≤ ϑ < 1/2; 1 for ϑ ≥ 1/2

Exponential†

e−z I(0,∞) (z)

1 − G4; 0 ((−4ϑ) ∨ 0)

Laplace

∗

Logistic

/2

−|z−ϑ|

e

1 − (1/2)e−2ϑ (1 + ϑ) for ϑ ≥ 0

e−z (1 + e−z )−2

e2ϑ (e2ϑ − 1 − 2ϑ)(e2ϑ − 1)−2

∗

For ϑ < 0, the relationship w+ (−ϑ) = 1 − w+ (ϑ) can be made use of being valid for arbitrary location families with symmetric baseline density f◦ (·).

†

In the formula given for w+ (ϑ), G4; 0 (·) is to denote the cdf of a central χ2 -distribution with df = 4. Furthermore, (−4ϑ) ∨ 0 stands for max{−4ϑ, 0}.

104

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

the parameter indexing the specific location model in mind. In Table 5.17, one finds the thus associated equivalence interval for ϑ for both specifications   , q+ ) used in the simulation study and all location families appearing in of (q+ Table 5.16. The simulation study whose results are presented below in Table 5.18, was performed to provide sufficiently complete data both on the level properties and the maximum power of the signed rank test for equivalence. Therefore, for each specification of F◦ and (ϑ1 , ϑ2 ) appearing in Table 5.17 , 100,000 samples of the respective size n ∈ {20, 30, 50} were drawn from the distributions given by F◦ (· − ϑ1 ), F◦ (· − ϑ2 ) and F◦ (· − ϑ◦ ) where ϑ◦ denotes that point in the parameter space which satisfies w+ (ϑ◦ ) = 1/2. The case that ϑ coincides with   , q+ ) = Table 5.17 Equivalence range for ϑ corresponding to (q+   (.3618, .6382) and (q+ , q+ ) = (.2398, .7602), respectively, in the location parameter families covered by Table 5.16.

Family of distribution

Gaussian Cauchy Uniform Exponential Laplace Logistic

Equivalence Range (ϑ1 , ϑ2 )   , q+ )= corresponding to (q+ (0.3618 , 0.6382)

(0.2398 , 0.7602)

(−0.2500 , 0.2500) (−0.4637 , 0.4637) (−0.0747 , 0.0747) (−1.0853 , −0.6340) (−0.2885 , 0.2885) (−0.4245 , 0.4245)

(−0.5000 , 0.5000) (−1.0662 , 1.0662) (−0.1537 , 0.1537) (−1.3748 , −0.4668) (−0.6035 , 0.6035) (−0.8563 , 0.8563)

ϑ◦ or, equivalently, q+ with 1/2, occurs in particular if for each i = 1, . . . , n, the random pair (Xi , Yi ) giving rise to the intraindividual difference Di is continuous and exchangeable in the sense of having the same distribution as (Yi , Xi ). Of course, exchangeability is a natural assumption whenever both treatments under comparison are actually identical and assessed by means of an endpoint criterion admitting numerically precise measurement. To be sure, exchangeability of a continuously distributed random pair (Xi , Yi ) implies still more than the validity of P [Di + Dj > 0] = 1/2, namely symmetry about zero of the distribution of the associated intrapair difference Di = Xi − Yi . Going through the rejection probabilities shown in Table 5.18, one notices in particular (see the entries in line 4) that in the setting of the above Example 5.4 with normally distributed intra-subject differences, the nominal level of significance is strictly maintained. Furthermore, it can be seen that the power attained against the alternative of no difference between both measuring devices comes out as low as 37.5%. On the one hand, this value is exceeded by that of the optimal parametric test for the same setting by about 8%.

Signed rank test for equivalence

105

Table 5.18 Simulated rejection probability of the signed rank test for equivalence at level α = .05 at both boundaries of the null hypothesis (columns 4 and 5) and at q+ = 1/2 (rightmost column) for the distributional shapes and equivalence ranges covered by Table 5.17. [Italicized values: power of the paired t-test for equivalence against the alternative q+ = 1/2.] Location

  q+ , q+

n

 β(F◦ ; q+ )

 β(F◦ ; q+ )

β(F◦ ; 1/2)

20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50

.04398 .04869 .04982 .04287 .04133 .03785 .04441 .05150 .04952 .04578 .04215 .04126 .04058 .04713 .04761 .03996 .03822 .03871 .04587 .05315 .05135 .04589 .04129 .03919 .04543 .05004 .04990 .04302 .04070 .03959 .04497 .05021 .05037 .04322 .04010 .03900

.04299 .04912 .04895 .04145 .03977 .03858 .04460 .05041 .04970 .04647 .04195 .03885 .04293 .04750 .04822 .03980 .03901 .03804 .04048 .04661 .04580 .03932 .03656 .03602 .04338 .04990 .04850 .04435 .04035 .03941 .04314 .05028 .04954 .04284 .03995 .03829

.07982 .09313 .11893 .12662 .20635 .22821 .37516 .45323 .63069 .71411 .89987 .93268 .07785 .11613 .20805 .37366 .63091 .89866 .07828 .11698 .21146 .37255 .63042 .89656 .07311 .11383 .19046 .34001 .59280 .87640 .07807 .12144 .20773 .37238 .63089 .89908 .07831 .11682 .20822 .37418 .63520 .89718

Family Gaussian

Cauchy

Uniform

Exponential

Laplace

Logistic

(.3618, .6382) " " (.2398, .7602) " " (.3618, .6382) " " (.2398, .7602) " " (.3618, .6382) " " (.2398, .7602) " " (.3618, .6382) " " (.2398, .7602) " " (.3618, .6382) " " (.2398, .7602) " " (.3618, .6382) " " (.2398, .7602) " "

106

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

On the other, the power of the sign test for equivalence with limits Φ(∓.5) = .5 ∓ .1915 to p+ , against p+ = 1/2 falls short of that of the signed rank test by almost 15% even if the former is performed in its randomized version. From a more general perspective, the simulation results shown in Table 5.18 admit the following conclusions: (i) Even for sample sizes as small as 20 and highly skewed distributions (→ exponential location parameter family), there is no reason to suspect that the nominal significance level could be exceeded to a practically relevant extent. (ii) Both the effective size and the power of the signed rank test for equivalence seems rather robust even against gross changes in the form of the underlying distribution. (iii) If the distribution of the intra-subject differences Di is Gaussian, the loss in efficiency of the signed rank as compared to the t-statistic seems to be considerably more marked in the equivalence case than in testing problems with conventional form of the hypotheses (cf. Randles and Wolfe, 1979, Tab. 4.1.7). Remark. Except for the case that the underlying distribution is exponential, the corresponding entries in column 4 and 5 of the above Table 5.18 are strikingly similar suggesting that each such pair actually gives two repeated estimates of the same quantity. This impression is far from misleading since it is not hard to derive analytically the following general result: Whenever   , q+ are chosen symmetrically about 1/2 and the the equivalence limits q+ location family under consideration is generated by a baseline distribution function symmetric about 0, the power function q+ → β(F◦ ; q+ ) of the signed rank test for equivalence is symmetric for its own part. Signed rank test for noninferiority In the noninferiority testing problem associated with (5.43), the hypotheses read H1 : q+ ≤ 1/2 − ε versus K1 : q+ > 1/2 − ε . (5.51) Due to the facts stated before in the derivation of the signed rank test for two-sided equivalence, the test statistic U+ also satisfies the conditions for carrying out a construction of the kind described in § 2.3 . Setting TN = U+ , τˆN = σ ˆ 2 [U+ ], and θ◦ = 1/2 yields   (5.52) (U+ − 1/2 + ε)/ˆ σ [U+ ] > u1−α as the rejection region of an asymptotically valid test for (5.51). As regards the choice of the equivalence margin ε, we propose to adopt the values ap-

Signed rank test for equivalence

107

pearing in Table 5.17 also for the noninferiority case. In particular, the simulation results shown below in Table 5.19 likewise refer to the specifications Table 5.19 Simulated rejection probability of the signed rank test for noninferiority at level α = .05 at the boundary of the null hypothesis (column 4) and q+ = 1/2 (rightmost column) for the distributional shapes and equivalence margins specified in Table 5.17. [Italicized values: power of the paired t-test for noninferiority against the alternative q+ = 1/2.] Location Family

1/2 − ε

n

β(F◦ ; 1/2 − ε)

β(F◦ ; 1/2)

Gaussian

.3618 " " .2398 " " .3618 " " .2398 " " .3618 " " .2398 " " .3618 " " .2398 " " .3618 " " .2398 " " .3618 " " .2398 " "

20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50 20 30 50

.06171 .05459 .04960 .04298 .03971 .03991 .06195 .05484 .05126 .04515 .04050 .03966 .05978 .05271 .04893 .04047 .03924 .03756 .06413 .05653 .05242 .04461 .04207 .03921 .05962 .05471 .05013 .04390 .04105 .03812 .06068 .05468 .04885 .04342 .04092 .03872

.32401 .29771 .39560 .39110 .53011 .54886 .67383 .71940 .81432 .85689 .95033 .96634 .32191 .39325 .53232 .67292 .81476 .95052 .32320 .39639 .53045 .67455 .81512 .94969 .32651 .39117 .51927 .65234 .78600 .92933 .32186 .39417 .53298 .67318 .81568 .94834 .32253 .39497 .53138 .67443 .81620 .94965

Cauchy

Uniform

Exponential

Laplace

Logistic

108

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

1/2 − ε = .3618 and 1/2 − ε = .2398 which under the location-shift models studied here transform to lower bounds for ϑ in the way made precise in Table 5.17. As to the conclusions to be drawn from the data shown in the above table, the only difference compared with the test for two-sided equivalence worth mentioning refers to a tendency toward a less strict control of the target significance level due to limitations in accuracy of the large-sample approximation involved. In all parametric submodels investigated, with samples of size n = 20 and noninferiority margin ε = .1382, the size of the critical region of the signed-rank test for noninferiority is about 6 rather than 5 percent. On the one hand, this deviation from the target level seems still tolerable for most practical purposes. On the other, under the Gaussian model, it increases the power against the alternative q+ = 1/2 ⇔ δ = 0 slightly above that of the paired t-test for noninferiority although the latter is UMP among all tests being invariant against transformations under which the test based on the signed rank statistic remains invariant as well. Through reducing the size of the latter to its target value, the sign of this difference in power would be reversed, leading to similar patterns as were obtained in the two-sided case.

5.5

A generalization of the signed rank test for equivalence for noncontinuous data

In the previous section, the construction of an equivalence version of the signed rank test was carried out under the basic assumption that the distribution of the intraindividual differences D1 , . . . , Dn is of the continuous type. For a considerable proportion of possible applications of the procedure to concrete data sets, this restriction is prohibitive since in practice the occurrence of ties between the Di is quite common. Lack of continuity of the distribution of the Di implies in particular that the probability q0 = P [Di + Dj = 0]

(5.53)

of observing a pair of intra-subject differences of equal absolute value but opposite sign, does not necessarily vanish but might take on any value in the unit interval. Obviously, for q0 > 0, it no longer makes sense to compare the true value of the functional q+ to 1/2 and consider |q+ − 1/2| as a reasonable measure of distance between the actual distribution of the Di and a distribution exhibiting the same form but being centered about 0. Hence, what we primarily need for purposes of adapting the signed rank test for equivalence to settings with noncontinuous data, is a meaningful generalization of the basic distance measure |q+ − 1/2|.

Signed rank test for equivalence for noncontinuous data

109

In the continuous case, we obviously have q+ = 1 − q− with q− = P [Di + Dj < 0]

(5.54)

so that |q+ −1/2| = 0 is satisfied if and only if there holds q+ = q− . Irrespective of the type of the distributions involved, marked discrepancies between q+ and q− are always at variance with equivalence of the treatments behind the primary observations Xi and Yi from which the ith intra-subject difference is computed. In fact, even if the joint distribution of (Xi , Yi ) is discrete, it must be exchangeable whenever there exists no treatment difference at d all, and exchangeability of (Xi , Yi ) clearly implies Di = Xi − Yi = Yi − Xi = −Di and hence q+ = P [(−Di ) + (−Dj ) > 0] = q− . Now, in view of the basic restriction q+ + q0 + q− = 1, q+ and q− coincide if and only if there holds the equation 2q+ + q0 = 1, or equivalently, q+ /(1 − q0 ) = 1/2 . The latter fact suggests to replace q+ by its conditional counterpart q+ /(1 − q0 ) throughout, where conditioning is on absence of any pair (i, j) of sampling units such that the Walsh average (Di + Dj )/2 of the associated intra-subject differences vanishes. This leads to use |q+ /(1 − q0 ) − 1/2| as a basic measure of distance suitable for any type of distribution of the Di . Defining, as usual, equivalence by the requirement that the value taken on by the selected measure of distance between the distributions under comparison be sufficiently small, the nonparametric testing problem to be dealt with in this section reads H:

q+ q+ ≤ 1/2 − ε1 or ≥ 1/2 + ε2 1 − q0 1 − q0 versus

K : 1/2 − ε1
0, Di + Dk = 0 ] ,

(5.61)

and

respectively. Now, we can proceed by applying the so-called δ-method (cf. √ ∗ −q+ /(1−q0 )) is Bishop et al., 1975, § 14.6) which allows us to infer that n(U+ 2 asymptotically univariate normal with mean 0 and variance σ∗n = ∇g(q+ , q0 )  Σn (∇g(q+ , q0 )) where ∇g(q+ , q0 ) stands for the gradient (row) vector of the transformation (q+ , q0 ) → q+ /(1−q0 ) . Writing down ∇g(q+ , q0 ) explicitly and expanding the quadratic form yields, after some straightforward algebraic simplifications, the expression: 2 = σ∗n

2 2 2 σ0n σ+n q+ 2q+ σ+0;n + . 2 4 + (1 − q0 ) (1 − q0 ) (1 − q0 )3

(5.62)

√ ∗ 2 The natural way of estimating the asymptotic variance σ∗n of nU+ consists in plugging in U -statistic estimators of all functionals of the distribution of the Di appearing on the right-hand side of equations (5.59a–c). Since the + and q0 have already been defined [recall (5.42), U -statistics for q+ , q1(2,3) (5.46) and (5.57), respectively], all what remains to be done is to provide ex+0 0 and q1(2,3) : plicit formulae for the analogous estimators of q1(2,3) 0 qˆ1(2,3)

=

 n−1  n−1 n−2 3

n 

 (1/3) I{0} (Di + Dj ) I{0} (Di + Dk ) +

i=1 j=i+1 k=j+1

 I{0} (Di + Dj ) I{0} (Dj + Dk ) + I{0} (Di + Dk ) I{0} (Dj + Dk ) ,

(5.63)

Signed rank test for equivalence for noncontinuous data +0 qˆ1(2,3) =

 n−1  n−1 n−2 3

n 

111

 (1/6) I(0,∞) (Di + Dj ) I{0} (Di + Dk ) +

i=1 j=i+1 k=j+1

I(0,∞) (Di + Dj ) I{0} (Dj + Dk ) + I(0,∞) (Di + Dk ) I{0} (Dj + Dk ) + I(0,∞) (Di + Dk ) I{0} (Di + Dj ) + I(0,∞) (Dj + Dk ) I{0} (Di + Dj )  + I(0,∞) (Dj + Dk )I{0} (Di + Dk ) . (5.64) Additional results from the asymptotic theory of U -statistics (see, e.g., Lee, 1990, § 3.4.2) ensure the (strong) consistency of all these estimators for their 2 expectations. This implies that the plug-in estimator σ ˆ+n obtained from + + (5.59a) by replacing q+ and q1(2,3) with U+ and qˆ1(2,3) , respectively, is con2 2 sistent for σ+n , and so on for σ ˆ0n and σ ˆ+0;n . Finally, consistency of√U+ , U0 , 2 2 2 ∗ , σ ˆ and σ ˆ ensures that the asymptotic variance σ∗n of nU+ = σ ˆ +0;n √+n 0n nU+ /(1 − U0 ) can be consistently estimated by 2 = σ ˆ∗n

2 2 2 σ ˆ0n σ ˆ+n U+ ˆ+0;n 2U+ σ + . 2 + (1 − U0 ) (1 − U0 )4 (1 − U0 )3

(5.65)

Now, in order to complete the derivation of a signed rank test for equivalence being asymptotically valid for arbitrary noncontinuous distributions of the Di , it suffices to invoke for another time the general result stated in § 3.4, with the following specifications: N = n, TN√= U+ /(1 − U0 ), θ = q+ /(1 − q0 ), σ = limn→∞ σ∗n , k = 1, and τN = σ∗n / n . This verifies that the decision rule q+ q+ ≤ 1/2 − ε1 or ≥ 1/2 + ε2 iff 1 − q0 1 − q0   √   ∗ n U+ /(1 − U0 ) − (1 − ε1 + ε2 )/2)/ˆ σ∗n < C±R (α; ε1 , ε2 ) ,

Reject H :

(5.66)

with

∗ C±R (α; ε1 , ε2 ) = 100α−percentage point of the χ2 − distri2 bution with df = 1 and λ2nc = n(ε1 + ε2 )2 /4ˆ σ∗n

1/2

(5.67)

defines an asymptotically valid test for the nonparametric equivalence problem (5.55), provided the limit σ∗2 , say, of (5.62) is a positive number. For the implementation of the generalized signed rank test for equivalence allowing for arbitrary patterns of ties in the set {D1 , . . . , Dn } of observed intra-subject differences, another special program is available for download from WKTSHEQ2 Source Code Package. The program name is srktie_d, and again, it exists both in a SAS and a R version.

112

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

Simplifying computations by reducing raw data to counts Often computation of the U -statistics required for carrying out the generalized signed rank test for equivalence can largely be simplified by determining in a preliminary step the configuration of ties which occurred in the observed vector (D1 , . . . , Dn ) of intra-subject differences. This is the case whenever the Di take on their values in a finite lattice of known span w > 0, say. Without loss of generality, this lattice can be assumed symmetric about 0, i.e., as a set of the form { −rw, −(r − 1)w, . . . , −w, 0, w, . . . , (r − 1)w, rw } where r ∈ IN denotes the number of points lying on both sides of the origin. Of course, a particularly frequent specific case of that kind occurs when the Di are integervalued random variables with finite support. Given a lattice containing the set of possible values of each Di in the sample of intra-subject differences, let us define   (5.68) Mk = # i | Di = kw , k = −r, . . . r so that for any k ∈ {−r, . . . , r}, Mk counts the number of sample points tied at the respective point of the lattice. (Note that some of the frequencies Mk may and typically will be equal to zero.) In terms of the frequencies Mk , the formulae for the U -statistic estimators to be considered in connection with the generalized signed rank test for equivalence can be rewritten as follows: U+ =

! k−1 r r r     1 Mk Ml + Mk2 − Mk , 2 n(n − 1) k=1 l=−k+1

+ qˆ1(2,3) =

1 n(n − 1)(n − 2) −2

r k−1  

k=1

r 

Mk Ml + 2

U0 = 0 qˆ1(2,3)

1 2 n(n − 1)

k=1

r 

k=−r+1

k=1 l=−k+1 r 

Mk

Ml

2

−

l=−k+1 r 

Mk − 2

k=1

r  k=1

r r  

Mk2 !

Mk Ml ,

(5.46∗ )

k=1 l=−k+1

!

(5.57∗ )

Mk M−k + M0 (M0 − 1) ,

k=1

1 = n(n − 1)(n − 2)

r 

2 Mk M−k

−2

k=−r

r 

Mk M−k

k=1

! + 2M0 ,

(5.63∗ )

! Mk (M−k + M0 ) .

(5.64∗ )

−3M02 +0 qˆ1(2,3) =

(5.42∗ )

1 n(n − 1)(n − 2)

r  Mk M−k k=−r+1

r 

Ml

l=−k+1

−

r  k=1

Signed rank test for equivalence for noncontinuous data

113

These identities (formally proved in Firle, 1998, pp. 86–88) underlie the program supplied under the name srktie_m [→ Appendix B] which allows its user to compute both the test statistic and its critical upper bound at any desired significance level α. Of course, in every setting such that the Di are lattice variables, the function srktie_d can be used instead, and the results will necessarily coincide with those obtained by means of srktie_m. The only difference is that the algorithm underlying the M-version of the program enables to dispense with triple loops and typically needs much shorter execution time. Example 5.5 In a comparative trial of the efficacy of two mydriatic agents (compounds administered for the purpose of dilating the eye pupils) A and B, each of n = 24 patients got dropped substance A to the one eye and B to the other. The outcome was measured as the increase of the pupil diameter [mm] attained after 30 minutes since administration of the tinctures. The individual results obtained in this way are shown in Table 5.20 . Due to limited accuracy of measurements, all Xi and Yi observed in this trial are multiples of w = 0.1, and so are the associated intra-subject differences Di = Xi − Yi . In order to test for equivalence in efficacy of both mydriatic drugs, decision rule (5.66) is clearly appropriate. Using the same equivalence limits as have been proposed in Example 5.4 for q+ , i.e., setting ε1 = ε2 = .2602 and α = .05, the procedure srktie_m yields with the entries in column 4 of the table as raw data ∗ ∗ = .57769, σ ˆ∗n = .59819, C±R (α; ε1 , ε2 ) = .52345 . the following results: U+ Table 5.20 Results of a comparative trial of two mydriatic agents, with Xi and Yi as the increase of the pupil diameter [mm] attained in the ith patient 30 minutes after administration of drug A and B, respectively. i 1 2 3 4 5 6 7 8 9 10 11 12

Xi 2.4 2.6 3.3 3.9 3.6 2.7 4.4 3.7 3.6 3.4 2.6 3.9

Yi 1.6 2.4 3.3 4.0 3.9 2.4 4.5 3.3 3.0 3.2 2.6 4.1

Di 0.8 0.2 0.0 0.1 −0.3 0.3 −0.1 0.4 0.6 0.2 0.0 −0.2

i 13 14 15 16 17 18 19 20 21 22 23 24

Xi 3.8 4.2 3.5 3.6 4.0 3.6 3.8 3.3 2.7 4.0 3.7 3.3

Yi 4.1 4.2 3.4 3.3 4.3 3.5 4.0 3.8 2.5 4.1 3.5 3.4

Di −0.3 0.0 0.1 0.3 −0.3 0.1 −0.2 −0.5 0.2 −0.1 0.2 −0.1

114

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

But with n = 24 and ε1 = ε2 = .2602, these values do not lead to a rejection of the null hypothesis H : q+ /(1 − q0 ) ≤ 1/2 − ε1 or q+ /(1 − q0 ) ≥ 1/2 + ε2 √ ∗ since we have 24 |.57769 − 1/2|/.59819 = .63626 > C±R (α; ε1 , ε2 ) . Table 5.21 is to give an impression of the multiformity of discrete distributions concentrated on the specific lattice {−2, −1, 0, 1, 2} leading to the same selected value of the target functional q+ /(1 − q0 ). From the entries in lines (b), (e) and (f), it becomes in particular obvious that symmetry of the distribution of the Di is sufficient but by no means necessary for minimizing the distance measure |q+ /(1−q0 )−1/2| underlying hypotheses formulation (5.55). Table 5.21 Examples of distributions on a five-point lattice belonging to the left [(a),(d)], the right [(c),(g)] boundary and the center [(b), (e), (f )] of the equivalence hypothesis K of (5.55) with ε1 = ε2 = .2602.

(a) (b) (c) (d) (e) (f) (g)

P [Di = −2] P [Di = −1] P [Di = 0] P [Di = 1] .313300 .249480 .229700 .086820 .091300 .346251 .229700 .116049 .120700 .086820 .229700 .249480 .300000 .366789 .000000 .233211 .200000 .400000 .000000 .100000 .369855 .130145 .000000 .130145 .299300 .052384 .000000 .048016

P [Di = 2] .120700 .216700 .313300 .100000 .300000 .369855 .600300

The rejection probabilities of the generalized signed rank test for equivalence under all specific distributions covered by Table 5.21 have been determined by means of Monte Carlo simulation. The results of this simulation study which are shown in Table 5.22 allow the conclusion that correcting the equivalence version of the signed rank test for ties in the way established in this section, is another instance of an asymptotic construction along the lines of § 3.4 yielding an equivalence testing procedure which tends to mild conservatism in finite samples. Furthermore, by comparison with Table 5.18, it can be seen that the maximum power attainable with a given sample size n, is markedly lower in the discrete than in the continuous case, as to be expected according to intuition.

Signed rank test for equivalence for noncontinuous data

115

Table 5.22 Simulated rejection probabilities† of the tie-corrected version of the signed rank test for equivalence under the distributions shown in Table 5.21, for sample sizes varying over {20, 30, 50, 100}. [The values obtained under (a), (d) and (c), (g) allow to estimate the actual type-I error risk in finite samples; the entries into lines (b), (e) and (f ) are values of the power against q+ /(1 − q0 ) = 1/2.] n = 20 n = 30 n = 50 n = 100 (a) (d)

.03910 .03695

.03657 .03738

.03606 .03422

.03619 .03622

(c) (g)

.03834 .03348

.03485 .03790

.03263 .03445

.03537 .03564

(b) (e) (f)

.19518 .17185 .12228

.36714 .33191 .24260

.70671 .67104 .53774

.97368 .96635 .92583

†

100,000 replications of each Monte Carlo experiment

Discussion In order to gain better insight into the meaning of the distance measure |q+ /(1 − q0 ) − 1/2| we chose as a basis for formulating a nonparametric equivalence hypothesis for settings with noncontinuous paired observations, it is helpful to analyze the extreme case that the Di can take on only three different values, namely −1, 0 and +1. Denoting the probabilities of these values by p− , p0 and p+ , respectively, it is easily verified that we can write q+ = 2p0 p+ +p2+ , q0 = 2(1−p0 −p+ )p+ +p20 . Hence, in the trinomial case, the alternative hypothesis to be established by means of the tie-corrected signed rank test for equivalence corresponds to the following subset of the parameter space:    ±R 2 % Kε1 ,ε2 = (p0 , p+ ) ∈ (0, 1)  p0 + p+ ≤ 1, 1/2 − ε1 <  2p0 p+ + p2+ < 1/2 + ε2 . (5.69) 1 − 2(1 − p0 − p+ )p+ − p20

In contrast, in the sign test for equivalence which, after the appropriate sufficiency reduction, refers to exactly the same family of distributions, the alternative hypothesis [cf. p. 72, (5.4)] is easily seen to admit the representation

116 % S  K ε1 ,ε2

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS    = (p0 , p+ ) ∈ (0, 1)2  p0 + p+ ≤ 1 , 1/2 − ε1 <  p+ < 1/2 + ε2 (5.70) 1 − p0

where the εν , like the εν (ν = 1,2) of (5.69), can be arbitrarily chosen in the interval (0 , 1/2) . The graphs presented in Figure 5.1 and 5.2 visualize the differences in geo% S  : Whereas the boundaries % ε±R and K metrical shape of the subspaces K ε1 ,ε2 1 ,ε2 % ε±R of the latter are simply straight lines, those of K are slightly curved. If, 1 ,ε2 as done in the first couple of graphs, the margins εν and εν are set equal to each other (both for ν = 1 and ν = 2), the alternative hypothesis of the sign test for equivalence is much larger than that of the generalized signed rank test. On the other hand, if the εν and εν are chosen in a way ensuring that the vertical section at p0 = 0 of both parameter subspaces coin% S  , as illustrated by Figure 5.2. All in all, it is % ε±R covers K cide, then K ε1 ,ε2 1 ,ε2 clear that we have no possibility to make the two versions of an alternative hypothesis about (p0 , p+ ) perfectly coincident by a suitable adjustment of the respective tolerances εν and εν . Admittedly, it would be desirable to base a nonparametric equivalence test for paired noncontinuous observations on a functional which reduces in the case of trinomially distributed Di to the parametric function underlying the sign test for equivalence. However, there seems to exist no possibility to implement this idea without discarding all other information provided by the observations except that contained in the counting statistic (N0 , N+ ) to which the data are reduced in the sign test.

% ε±R Figure 5.1 Boundaries of the equivalence hypotheses K ( 1 ,ε2 ( - - - - ) for ε1 = ε1 = ε2 = ε2 = .2602.

% S  ) and K ε ,ε 1

2

Signed rank test for equivalence for noncontinuous data

% ε±R Figure 5.2 Boundaries of the equivalence hypotheses K ( 1 ,ε2   ( - - - - ) for ε1 = ε2 = .2602, ε1 = ε2 = .4095.

117

% S  ) and K ε ,ε 1

2

Tied-data version of the signed rank test for noninferiority A noninferiority analogue of the generalized signed rank test for equivalence derived in this section is obtained by replacing the critical region to be used in the two-sided case with

   U+ /(1 − U0 ) − 1/2 + ε /ˆ (5.71) σ∗n > u1−α where all symbols appearing in the expression for the test statistic have exactly the same meaning as before. In particular, the simplified formula for the U -statistics involved can be applied in the noninferiority case as well leading to considerable savings of computation time whenever the observed intraindividual differences take values in a sufficiently coarse lattice of points. Of course, the basic property of the test given by (5.71) is that, asymptotically, its significance level does not exceed α provided the testing problem is rewritten H1 :

q+ q+ ≤ 1/2 − ε versus K1 : > 1/2 − ε . 1 − q0 1 − q0

(5.72)

The extent to which the power of the test will change when the statement made about the target functional under the alternative hypothesis to be established is much less precise, can be expected to be of the same order of magnitude as in the corresponding tests for continuous data. In order to get a more precise idea of the finite-sample behavior of the test based on (5.71),

118

EQUIVALENCE TESTS FOR PAIRED OBSERVATIONS

we studied by simulation its rejection probabilities under all discrete models considered in the two-sided case except those for which the true value of the target functional q+ /(1 − q0 ) falls on the right-hand boundary of the equivalence hypothesis specified in (5.55). Regarding the type-I error risk, the differences between both versions of the test are negligible, so that in the noninferiority case, the size is likewise less than 4% for all sample sizes and both discrete models studied. Furthermore, except for n = 100, leaving the theoretical equivalence range unbounded to the right (except, of course, for the bound limiting the parameter space as a whole) leads to a marked increase in power indeed. For n = 20 and n = 30, this difference exceeds 30% for all specific alternatives studied. Table 5.23 Simulated rejection probabilities† of the tie-corrected version of the signed rank test for noninferiority under the null hypothesis [(a), (d)] and three alternatives with q+ /(1 − q0 ) = 1/2 [(b), (e) and (f )] in samples of size n ∈ {20, 30, 50, 100}. [For a fully precise description of the distributions from which the data were generated see Table 5.21.] n = 20 n = 30 n = 50 n = 100 (a) .04000 .03702 .03571 .03694 (d) .03618 .03461 .03527 .03638 (b) (e) (f) †

.53048 .50557 .43928

.66813 .64704 .56918

.84671 .83111 .76780

.98430 .98098 .96391

100,000 replications of each Monte Carlo experiment

6 Equivalence tests for two unrelated samples

6.1

Two-sample t-test for equivalence

If the trial run in order to compare the two treatments A and B follows an ordinary two-arm design, the data set to be analyzed consists of m+n mutually independent observations X1 , . . . , Xm , Y1 , . . . , Yn . For definiteness, we keep assigning the X’s to treatment group A, whereas the Y ’s are assumed to belong to the group of subjects or patients given treatment B. The statistical model which is assumed to hold for these variables is the same as in the ordinary two-sample t-test for conventional one- and two-sided hypotheses. Accordingly, we suppose throughout this section that both the Xi and the Yj follow a normal distribution with some common variance and possibly different expected values where all three parameters are unknown constants allowed to vary unrestrictedly over the respective part of the parameter space. Formally speaking, this means that the observations are assumed to satisfy the basic parametric model Xi ∼ N (ξ, σ 2 ) ∀ i = 1, . . . , m ,

Yj ∼ N (η, σ 2 ) ∀ j = 1, . . . , n ,

(6.1)

with ξ, η ∈ IR, σ 2 ∈ IR+ . In § 1.6, we have argued that the most natural measure of dissimilarity of two homoskedastic Gaussian distributions is the standardized difference of their means. Accordingly, under the present parametric model, we define equivalence of treatments A and B through the condition that the true value of this measure falls into a sufficiently narrow interval (−ε1 , ε2 ) around zero. In other words, we formulate the testing problem as H : (ξ − η)/σ ≤ −ε1 or (ξ − η)/σ ≥ ε2 versus K : −ε1 < (ξ − η)/σ < ε2

(ε1 , ε2 > 0) ,

(6.2)

in direct analogy to the setting of the paired t-test for equivalence [recall (5.27), (5.28)]. The construction of an optimal solution to this problem exploits essentially the same ideas as underlie the derivation of the paired t-test for equivalence. In the unpaired case, both hypotheses remain invariant under any transformation of the form (x1 , . . . , xm , y1 , . . . , yn ) → (a+bx1 , . . . , a+bxm , a+by1 , . . . , a+byn )

119

120

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

with (a, b) ∈ IR×IR+ , applying to each coordinate of a point in the joint sample space IRN (with N = m + n) the same scale change and the same translation. Denoting the group of all transformations of this form by G, it is easy to show that the construction of a test which is uniformly most powerful among all level-α tests for (6.2) remaining invariant under G has been accomplished as soon as a test has been found which is UMP at the same level for the reduced problem ˜ : θ˜ ≤ −˜ H ε1 or θ˜ ≥ ε˜2

versus

˜ : −˜ K ε1 < θ˜ < ε˜2 .

& (6.2)

In the two-sample case, θ˜ has to be interpreted as the parameter of a noncentral t-distribution with m + n − 2 = N − 2 degrees of freedom, and ε˜2 ) denotes the equivalence interval of (6.2) rescaled by multiplication (−˜ ε1 , with mn/N . Except for differences in notation for the number of degrees of freedom and & is the same as the testing problem the limits of the equivalence range, (6.2) $ which has been considered in § 5.3 in connection with the paired t-test (5.28) for equivalence. Hence, it follows [cf. (5.32), (5.33)] that in order to obtain & we have to use a critical region of the form an UMP level-α test for (6.2),

 1 ˜2 C˜α; , (6.3) m,n (−ε1 , ε2 ) < T < Cα; m,n (−ε1 , ε2 ) k where the critical constants C˜α; m,n (−ε1 , ε2 ), k = 1, 2, have to be determined by solving the equation system

G∗−˜ε1 (C2 ) − G∗−˜ε1 (C1 ) = α = Gε∗˜2 (C2 ) − Gε∗˜2 (C1 ) , −∞ < C1 < C2 < ∞ .

(6.4)

In (6.4), the superscript ∗ is added to the symbol Gε˜(·) which has been used in § 5.3 for noncentral t-distribution functions, in order to make conspicuous the change in the way the number of degrees of freedom has to be determined. In the two-sample case, the latter must be set equal to N − 2 instead of n − 1. The desired UMPI (uniformly most powerful invariant) test for the primary problem (6.2) is obtained by applying (6.3) with T as the ordinary two-sample t-statistic to be computed from the individual observations by means of the well known formula T =

m n 1/2

   ¯ − Y¯ )/ ¯ 2+ mn(N − 2)/N (X (Xi − X) (Yj − Y¯ )2 . (6.5) i=1

j=1

The simplified computational scheme for determining the critical bounds to the t-statistic derived in § 5.3 for symmetrically chosen equivalence limits to the standardized population mean, has likewise its direct counterpart in the two-sample case. More precisely speaking, it follows by Lemma A.1.6

Two-sample t-test for equivalence

121

that whenever both tolerances ε1 and ε2 are set equal to some common value ε > 0, say, the critical region (6.3) admits the representation

 |T | < C˜α; m,n (ε) , (6.6) where C˜α; m,n (ε) is explicitly given as the square root of the lower 100 α percentage point of an F -distribution with 1, N − 2 degrees of freedom and noncentrality parameter λ2nc = mnε2 /N . In view of this, the practical implementation of the two-sample t-test for equivalence in a symmetric sense is extremely easy in any programming environment providing an intrinsic function for the computation of quantiles of noncentral F -distributions (like finv in SAS and qf in R). Moreover, making use of Table 6.1, no extra computational tools are needed at all to carry out the test in cases of balanced designs with common sample sizes being multiples of 5 between 10 and 75, pro-

Table 6.1 Critical constant C˜.05; n,n (ε) of the two-sample t-test for equivalence at level α = 5% with common size n of both samples in the symmetric case (ε1 , ε2 ) = (−ε, ε), for ε = .25(.25)1.00 and n = 10(5)75. ε = n

.25

.50

.75

1.00

10 15 20 25 30 35 40 45 50 55 60 65 70 75

0.07434 0.07997 0.08626 0.09313 0.10058 0.10866 0.11738 0.12681 0.13698 0.14795 0.15977 0.17248 0.18615 0.20082

0.11864 0.16083 0.21755 0.29167 0.38434 0.49320 0.61250 0.73572 0.85816 0.97741 1.09257 1.20355 1.31055 1.41389

0.25412 0.46595 0.73546 1.00477 1.25435 1.48481 1.69952 1.90128 2.09219 2.27384 2.44747 2.61406 2.77441 2.92917

0.61365 1.08697 1.50303 1.87065 2.20346 2.50990 2.79542 3.06383 3.31790 3.55971 3.79090 4.01275 4.22631 4.43246

vided one chooses ε1 = ε2 = ε ∈ {.25, .50, .75, 1.00}. The maxima of the power functions of the tests determined by these critical constants are shown in Table 6.2. Denoting, for ε1 = ε2 = ε and m = n, the rejection probability of the test with critical region (6.3) against the specific alternative θ = (ξ − η)/σ = 0 [↔

122

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

ξ = η] by β˜α;n,n (ε), this maximal power is readily computed by means of the simple formula   (6.7) β˜α;n,n (ε) = 2 G∗0 C˜α; n,n (ε) − 1 , with G∗0 (·) as the central version of the t-distribution function with N − 2 degrees of freedom.

Table 6.2 Power attained at θ = (ξ − η)/σ = 0 when using the critical constants shown in Table 6.1 for the two-sample t-statistic. ε = n

.25

.50

.75

1.00

10 15 20 25 30 35 40 45 50 55 60 65 70 75

.05844 .06317 .06829 .07381 .07977 .08620 .09314 .10062 .10867 .11734 .12666 .13667 .14740 .15888

.09312 .12662 .17106 .22820 .29786 .37654 .45801 .53614 .60710 .66945 .72319 .76901 .80781 .84051

.19772 .35514 .53343 .67995 .78525 .85779 .90679 .93946 .96099 .97505 .98414 .98998 .99370 .99606

.45288 .71368 .85890 .93250 .96845 .98554 .99348 .99710 .99873 .99945 .99976 .99990 .99996 .99998

Even if the restriction of a symmetric choice of the equivalence limits to the target parameter θ = (ξ − η)/σ has to be abandoned, the practical implementation of the two-sample t-test for equivalence continues to be a quick and easy matter, provided one makes use of an appropriate computer program. A SAS macro or, alternatively, a R function which does all necessary computations determining in particular both critical bounds to the test statistic, is to be found in the WKTSHEQ2 Source Code Package under the program name tt2st. The basic algorithm is the same as that underlying its analogue for the one-sample case [→ tt1st]. In addition to solving the pair of equations (6.4) for arbitrary choices of ε1 and ε2 > 0, it outputs the power of the test against the alternative of coincident expected values.

Two-sample t-test for equivalence

123

Example 6.1 Table 6.3 shows the raw data obtained from a comparative trial of moxonodin [an alpha receptor blocking agent] and captopril [an inhibitor of the angiotensin converting enzyme] in the antihypertensive treatment of patients suffering from major depression. Since moxonodin was known to have not only

Table 6.3 Reduction of diastolic blood pressure [mm Hg] observed in patients suffering from major depression after 4 weeks of daily treatment with 0.2 – 0.4 mg moxonodin (a) and 25 – 50 mg captopril (b), respectively. (a) moxonidin

(b) captopril

i

Xi

j

Yj

1 2 3 4 5 6 7 8 9 10 11 12

10.3 11.3 2.0 −6.1 6.2 6.8 3.7 −3.3 −3.6 −3.5 13.7 12.6

1 2 3 4 5 6 7 8 9 10 11 12

3.3 17.7 6.7 11.1 −5.8 6.9 5.8 3.0 6.0 3.5 18.7 9.6

¯ = 4.175, SX = 7.050 X

Y¯ = 7.208, SY = 6.625

antihypertensive but also neuroregulative effects alleviating some of the key symptoms making up a major depression, the aim of the trial was to establish equivalence of this drug to the ACE inhibitor with respect to antihypertensive efficacy. Furthermore, only rather extreme increases in the reduction of blood pressure induced by captopril were considered undesirable, on the ground that the latter drug was known to produce comparatively weak effects. On the other hand, only comparatively small losses in antihypertensive effectiveness were considered compatible with equivalence. Making allowance for both of these aspects, a nonsymmetric choice of the equivalence range for the standardized difference of means was considered appropriate specifying (−ε1 , ε2 ) = (−0.50 , 1.00). With thedata given in the above table,√the test statistic (6.5) is computed to be T = 122 · 22/24 · (4.175 − 7.208)/ 11 · 7.0502 + 11 · 6.6252 = −1.086 .

124

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

On the other hand, with α = .05, m = n = 12, ε1 = 0.50 and ε2 = 1.00, tt2st yields (0.27977 , 0.93088) as the critical interval for T . Since the observed value of T falls clearly outside this interval, the data do not allow to reject the null hypothesis of nonequivalence of moxonodin to captopril with respect to antihypertensive efficacy. Furthermore, the power against the alternative that the underlying Gaussian distributions coincide, is computed to be .21013. Thus, even under the most extreme alternative, the chance of establishing equivalence was rather low in the present study. In view of the small size of both samples, this is hardly surprising. Two-sample t-test for noninferiority The modifications required in order to obtain an optimum testing procedure for the noninferiority problem associated with (6.2) are largely analogous to those described in § 5.3 for the case of paired observations. The first step consists of replacing the hypotheses of (6.2) with H1 : (ξ − η)/σ ≤ −ε

vs. K1 : (ξ − η)/σ > −ε (ε > 0) .

(6.8)

Essentially the same arguments allowing one to establish the UMPI property of the t-test for two-sided equivalence show that the class of all invariant levelα tests for (6.8) contains a uniformly most powerful element, namely the test given by the rejection region

   T > tN −2;1−α − mn/N ε , (6.9)    where tN −2;1−α − mn/Nε denotes the (1 − α)-quantile of a t-distribution  with df = N − 2 and noncentrality parameter − mn/N ε . As in the paired t-test for noninferiority, these critical values of the t-statistic are readily computed by calling the SAS intrinsic function tinv or the R function qt. A compilation of critical values for balanced designs with n = 10 through 75 observations per group and equivalence margins ε ∈ {k/10 | k = 1, . . . , 5} is shown in Table 6.4. The power of the test against any alternative (ξ, η, σ 2 ) with (ξ − η)/σ = θ > −ε is as easy to compute as the critical constant, (t) of the noncennamely by evaluating the survivor function 1 − G∗√ mn/N θ  tral t-distributionwith df = N − 2 and noncentrality parameter mn/N θ at   t = tN −2;1−α − mn/N ε . If the alternative of interest is a null alternative specifying ξ = η, power computation is particularly simple since we then can    ∗ write POW0 = G0 − tN −2;1−α − mn/N ε where G∗0 ( · ) is as in (6.7). The results of computing POW0 for the same sample sizes and choices of the equivalence margin as in Table 6.4 are the entries in Table 6.5.

Two-sample t-test for equivalence

125 



 Table 6.4 Critical constant t2n−2;.95 − n/2 ε of the two-sample t-test for noninferiority at level α = 5% in the case of a balanced design with n = 10(5)75 and equivalence margin ε = .10(.10).50. ε = n 10 15 20 25 30 35 40 45 50 55 60 65 70 75

.10 1.49038 1.41206 1.35708 1.31267 1.27442 1.24029 1.20915 1.18032 1.15334 1.12787 1.10369 1.08061 1.05849 1.03721

.20 1.24922 1.12533 1.03048 0.95034 0.87948 0.81518 0.75584 0.70044 0.64826 0.59879 0.55163 0.50649 0.46311 0.42131

.30 1.01058 0.84095 0.70613 0.59022 0.48672 0.39223 0.30468 0.22269 0.14531 0.07183 0.00169 −0.06552 −0.13016 −0.19249

.40 0.77445 0.55890 0.38404 0.23231 0.09614 −0.02857 −0.14435 −0.25293 −0.35552 −0.45303 −0.54615 −0.63544 −0.72134 −0.80420

.50 0.54081 0.27916 0.06418 −0.12342 −0.29229 −0.44723 −0.59127 −0.72645 −0.85427 −0.97581 −1.09192 −1.20329 −1.31045 −1.41385

Table 6.5 Power against the null alternative θ = 0 attained in the two-sample t-test for noninferiority for the sample sizes and equivalence-margin specifications underlying Table 6.4.

n 10 15 20 25 30 35 40 45 50 55 60 65 70 75

.10 .07672 .08448 .09138 .09777 .10380 .10957 .11513 .12053 .12579 .13094 .13599 .14095 .14584 .15066

ε .20 .11379 .13500 .15465 .17335 .19139 .20891 .22601 .24275 .25917 .27528 .29112 .30669 .32201 .33707

= .30 .16280 .20375 .24221 .27891 .31415 .34806 .38071 .41215 .44238 .47144 .49933 .52607 .55168 .57619

.40 .22436 .29034 .35154 .40864 .46187 .51135 .55720 .59954 .63852 .67428 .70700 .73686 .76404 .78871

.50 .29763 .39109 .47458 .54886 .61445 .67194 .72197 .76526 .80248 .83433 .86145 .88446 .90389 .92025

126

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

A glance over Tables 6.2 and 6.5 shows that the differences in power between the two- and the one-sided version of the two-sample t-test for equivalence are roughly comparable to those found in § 5.3 between both versions of the paired t-test for equivalence: Given some common value of the equivalence margin, the much higher precision of the statement about the target parameter which can be made in the two-sided equivalence test in case of a significance decision, is bought at the price that the sample sizes required for ensuring the same level of power are considerably larger.

6.2

Mann-Whitney test for equivalence

The equivalence testing problem treated in the previous section can easily be embedded in a purely nonparametric framework. To see this, it suffices to recall the elementary fact that under the specific parametric assumption that both the Xi and the Yj are normally distributed with common variance σ 2 and arbitrary expectation ξ and η, respectively, the probability π+ , say, of obtaining an observation from the first distribution which exceeds some √ single observation drawn from the second, can be written π+ = Φ((ξ − η)/ 2σ). Since the the standard normal cdf Φ is well known to be both continuous and strictly increasing, this identity implies that the testing problem (6.2) is equivalent to H : π+ ≤ 1/2 − ε1 or π+ ≥ 1/2 + ε2 versus provided π+ and the

εν

K : 1/2 − ε1 < π+ < 1/2 + ε2 ,

(6.10)

are defined by

 √  π+ = P [Xi > Yj ] ; εν = Φ εν / 2 − 1/2 , ν = 1, 2 .

(6.11)

Since the Mann-Whitney functional π+ is well defined for any pair (F, G) of continuous distribution functions on IR1 such that Xi ∼ F ∀ i = 1, . . . , m ,

Yj ∼ G ∀ j = 1, . . . , n ,

(6.12)

it is natural to treat (6.10) as a nonparametric formulation of the problem of testing for equivalence of two continuous distributions of arbitrary shape. On the following pages, we will show how an asymptotically distribution-free solution to that testing problem can be based on the large-sample distribution of the U -statistic estimator W+ , say, of π+ . As is well known (see, e.g., Lehmann, 1975, p. 335), this estimator is simply given as the Wilcoxon twosample statistic in its Mann-Whitney form divided by mn so that we have W+ = (1/mn)

m  n  i=1 j=1

I(0,∞) (Xi − Yj ) .

(6.13)

Mann-Whitney test for equivalence

127

Not surprisingly, the role played here by the statistic W+ and the functional π+ will be analogous to that of the quantities U+ [→ p. 100, (5.42)] and q+ [→ (5.41)] in the derivation of a nonparametric equivalence test for paired observations. Irrespective of the shape of the underlying distribution functions F and G, the exact variance of W+ is given as Var[W+ ] = (1/mn) ·

2 π+ − (m + n − 1)π+ + (m − 1)ΠXXY + (n − 1)ΠXY Y ,

(6.14)

where ΠXXY = P [ Xi1 > Yj , Xi2 > Yj ], ΠXY Y = P [ Xi > Yj1 , Xi > Yj2 ]

(6.15)

(see, e.g., Lehmann, 1975, (2.20) ). Natural estimators of the functionals ΠXXY and ΠXY Y are given by the U -statistics 2 × m(m − 1) n m−1 m n   

 XXY = Π

I(0,∞) (Xi1 − Yj ) · I(0,∞) (Xi2 − Yj ) ,

(6.16a)

i1 =1 i2 =i1 +1 j=1

 XY Y = Π

2 × m n(n − 1) m n−1  

n 

I(0,∞) (Xi − Yj1 ) · I(0,∞) (Xi − Yj2 ) . (6.16b)

i=1 j1 =1 j2 =j1 +1

As is true for any two-sample U -statistic estimator with square integrable  XXY and Π  XY Y kernel (cf. Randles and Wolfe, 1979, Corollary 3.4.9), W+ , Π are consistent for the respective estimands π+ , ΠXXY and ΠXY Y . Hence, a consistent estimator of Var[W+ ] is given by σ ˆ 2 [W+ ] =

1 W+ − (m + n − 1)W+2 + mn

 XXY + (n − 1)Π  XY Y . (m − 1)Π

(6.17)

In view of the asymptotic normality of the Mann-Whitney statistic (see, e.g. Lehmann, 1975, p. 69), consistency of σ ˆ 2 [W+ ] implies that the statistic (W+ − π+ )/ˆ σ [W+ ] converges in law to a standard Gaussian variable whenever we have 0 < π+ < 1, m, n → ∞ and m/n → λ (with λ denoting some positive real number). Hence, we can apply once more the general theory outlined in

128

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

§ 3.4 leading to the conclusion that an asymptotically valid testing procedure for the equivalence problem (6.10) is obtained by using the decision rule: Reject nonequivalence if and only if     σ [W+ ] < CM W (α; ε1 , ε2 ) , W+ − 1/2 − (ε2 − ε1 )/2)/ˆ

(6.18)

with

CM W (α; ε1 , ε2 ) = 100α−percentage point of the χ2 − distri-

1/2 σ 2 [W+ ] . (6.19) bution with df = 1 and λ2nc = (ε1 + ε2 )2 /4ˆ

At the URL associated with this book, one finds also a tool for the quick and easy implementation of the testing procedure (6.18), appropriately termed Mann-Whitney test for equivalence in the research paper where its basic properties have first been rigorously established (Wellek, 1996). The program which had originally been written as another SAS macro and translated later on in R, is named mawi and requires as input data the actual values of the significance level α, the sample sizes m, n and the tolerances ε1 , ε2 , as well as the name of a file containing the raw data under analysis. In the output, the user finds in addition to the input data, the values computed for the Mannˆ [W+ ], the critical Whitney statistic W+ , its estimated standard deviation σ a   bound CM (α; ε , ε ) and an indicator which is set equal to 1 if and only if 1 2 W the data fall into the rejection region corresponding to (6.18). Example 6.1 (continued) In order to illustrate the use of the testing procedure (6.18), we reanalyze the blood pressure reduction data of Table 6.3 by means of the Mann-Whitney statistic. According to (6.11), the equivalence hypothesis −.50 < (ξ − η)/σ < 1.00 on two Gaussian distributions with means ξ, η and common variance σ 2 can be rewritten as .3618 < π+ < .7602. The corresponding specification of the εν in (6.10), (6.18), (6.19) is ε1 = .1382, ε2 = .2602. With these values and the raw data displayed in Table 6.3, the program mawi yields ˆ [W+ ] = .11133, CM W (.05; ε1 , ε2 ) = .30078. Since we have W+ = .41667, σ |.41667 − 1/2 − (.2602 − .1382)/2|/.11133 = 1.2964, the observed value of the test statistic clearly exceeds its critical upper bound. Thus it follows that the Mann-Whitney test for equivalence leads to the same decision as the parametric test derived in the previous subsection, namely acceptance of the null hypothesis of nonequivalence of both drugs with respect to their antihypertensive efficacy.

Mann-Whitney test for equivalence

129

In order to investigate the level and power of the Mann-Whitney test for equivalence in real applications with samples of finite or even small size, one has to rely on Monte Carlo simulation once more. The fact that the ordinary Mann-Whitney test is well known (see H´ ajek et al., 1999, pp. 96–7) to be optimal against logistic shift alternatives suggests to restrict such an investigation to constellations where the cdf and the density of the Yj has one of the standard forms listed in Table 5.16 and the distribution of the Xi is the result of shifting that of the Yj by a real constant ϑ. Accordingly, in the majority of simulation experiments run to study the finite sample behavior of the testing procedure (6.18), random numbers of the form Xi = Xi◦ + ϑ, ◦ , Y1◦ , . . . , Yn◦ ) as a sample of size Yj = Yj◦ were generated with (X1◦ , . . . , Xm m + n from a standardized Gaussian, Cauchy, uniform, exponential, Laplace and logistic distribution, respectively. In addition, one constellation was investigated where both distributions under comparison differ in form. The rationale behind the procedure of generating pairs of samples from distributions of this latter kind was the same as for the location models: The distribution from which the Yj were taken, was fixed (at N (0, 1), i.e., the standard Gaussian), and the center ϑ of the distribution of the Xi (belonging to the Gaussian family as well but with threefold standard deviation) was successively shifted from zero to the point at which the functional π+ [recall (6.11)] reaches 1/2 − ε1 , 1/2 + ε2 , and 1/2, respectively. For Xi = Xi◦ + ϑ, Yj = Yj◦ with Xi◦ and Yj◦ following the (continuous) cdf F◦ and G◦ , respectively, the functional π+ with respect to which equivalence of the underlying distributions is asserted under the alternative hypothesis of (6.10), admits the representation  ∞ π+ = [1 − F◦ (y − ϑ)]dG◦ (y) . (6.20) −∞

It is easy to verify that in any setting with F◦ = G◦ and F◦ (−x) = 1 − F◦ (x) ∀x ∈ IR, i.e., in any ordinary shift model with a symmetric common baseline cdf, the integral on the right-hand side of (6.20) coincides with the expression on the left of equation (5.50), provided, ϑ is replaced with ϑ/2 in the latter. Hence except for the exponential distribution, for all baseline cdf’s considered in § 5.4, the limits −ε1 , ε2 , say, of the equivalence range for the location parameter ϑ which corresponds to some given equivalence range (1/2 − ε1 , 1/2 + ε2 ) for π+ under the two-sample shift model generated by F◦ , can be determined by means of the formulae shown in the third column of Table 5.16. In the two-sample shift model obtained by setting F◦ (x) = Φ(x/3), G◦ (y) = Φ(y) such that the distributions under comparison differ in dispersion for any value of the location parameter, π+ admits a simple representation through the normal distribution function as well. It reads √ π+ = Φ(ϑ/ 10), in close analogy to the ordinary Gaussian shift model given by F◦ = G◦ = Φ. Making use of these relationships one obtains the values displayed in Table 6.6 which is a direct two-sample counterpart to Table 5.17.

130

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Specifying the equivalence limits to π+ by choosing ε1 = ε2 = .20 can be motivated as explained in § 1.7 [cf. Table 1.1, (ii)]. The other specification of ε1 , ε2 referred to in Table 6.6 corresponds to the equivalence range used in Example 6.1 [to be resumed below] for the standardized difference of the means in the Gaussian shift model. Table 6.6 Equivalence ranges for the location parameter ϑ corresponding to .30 < π+ < .70 and .3618 < π+ < .7602, respectively, in seven different models for the two-sample problem with Xi ∼ F◦ ( · − ϑ), Yj ∼ G◦ ( · ). Family of Distributions

Equivalence Range (−ε1 , ε2 ) for ϑ corresponding to 1/2 − ε1 < π+ < 1/2 + ε2 , for (1/2 − ε1 , 1/2 + ε2 ) = (0.3000, 0.7000)

Gaussian Cauchy Uniform Exponential Laplace Logistic N (ϑ, 9)/N (0, 1)

(−0.7416, (−1.4531, (−0.2254, (−0.5108, (−0.8731, (−1.2636, (−1.6583,

0.7416) 1.4531) 0.2254) 0.5108) 0.8731) 1.2636) 1.6583)

(0.3618, 0.7602) (−0.5000, (−0.9274, (−0.1494, (−0.3235, (−0.5770, (−0.8491, (−1.1180,

1.0000) 2.1324) 0.3075) 0.7348) 1.2070) 1.7127) 2.2356)

In the simulation study of the Mann-Whitney test for equivalence, a balanced design was assumed throughout, and the common sample size n was chosen from the set {12, 24, 36}. For each of these sample sizes and all 7 × 2 constellations covered by Table 6.6, the rejection probability of the test at the usual nominal significance level of 5% under ϑ = −ε1 , ϑ = ε2 and ϑ = 0 (⇔ π+ = 1/2) was determined on the basis of 100, 000 replications of the respective Monte Carlo experiment. As can be seen from the results of this study summarized in Table 6.7, the nominal level is strictly maintained in all settings with a symmetric choice of the equivalence limits, even for sample sizes as small as 12. The only cases where the test shows some anticonservative tendency are those where very small samples are used for establishing an equivalence hypothesis exhibiting marked asymmetry, and even then the extent of anticonservatism seems not really serious from a practical point of view. Under the ordinary shift model with a standard Gaussian cdf as baseline, the loss in power as compared to the optimal parametric procedure (i.e., the twosample t-test for equivalence derived in the preceding section) is roughly of the same order of magnitude as in the analogous paired-sample setting [cf. p. 105].

Mann-Whitney test for equivalence

131

Table 6.7 Simulated rejection probabilities of the Mann-Whitney test for equivalence at both boundaries of the hypotheses (columns 5, 6) and π+ = 1/2 (rightmost column) with samples of common size n = 12, 24, 36 from distributions belonging to the families listed in Table 6.6 and equivalence range (1/2 − ε1 , 1/2 + ε2 ) = ( .3000, .7000), ( .3618, .7602). [The italicized values give the power of the two-sample t-test for equivalence at level α = .05 against π+ = 1/2.] Family of Distributions (1/2 − ε1 , 1/2 + ε2 ) m Gaussian

n

Rejection Probability at π+ = 1/2 − ε1 1/2 + ε2 1/2

(.3000, .7000)

12 12

.05015

.04941

"

24 24

.04510

.04565

"

36 36

.04427

.04476

(.3618, .7602)

12 12

.05514

.04327

"

24 24

.05273

.04026

"

36 36

.04828

.03900

Cauchy

(.3000, .7000) " " (.3618, .7602) " "

12 24 36 12 24 36

12 24 36 12 24 36

.04925 .04644 .04466 .05107 .05143 .04967

.04780 .04536 .04442 .04336 .03939 .03962

.22020 .24605 .57929 .63971 .82186 .86027 .19727 .21013 .46811 .49669 .65010 .67635 .20180 .56918 .81617 .17994 .45973 .64708

Uniform

(.3000, .7000) " " (.3618, .7602) " " (.3000, .7000) " " (.3618, .7602) " "

12 24 36 12 24 36 12 24 36 12 24 36

12 24 36 12 24 36 12 24 36 12 24 36

.04503 .04312 .04267 .04872 .04892 .04928 .04631 .04471 .04285 .04949 .05028 .04782

.04397 .04282 .04291 .03800 .03709 .03824 .04557 .04404 .04322 .04054 .03879 .03860

.19944 .56956 .81578 .17954 .45624 .64447 .20216 .56989 .81782 .18135 .45580 .64542

Exponential

132

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.7 (continued) Family of Distribution Laplace

Logistic

N (ϑ, 9)/N (0, 1)

(1/2 − ε1 , 1/2 + ε2 ) m (.3000, .7000) " " (.3618, .7602) " " (.3000, .7000) " " (.3618, .7602) " " (.3000, .7000) " " (.3618, .7602) " "

12 24 36 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36

n 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36

Rejection Probability at π+ = 1/2 − ε1 1/2 + ε2 1/2 .04743 .04582 .04255 .05138 .05031 .04833 .04518 .04371 .04324 .05102 .05045 .04650 .04862 .04506 .04403 .05429 .05090 .04914

.04745 .04425 .04363 .04177 .03961 .03904 .04577 .04507 .04445 .03979 .03922 .03973 .05058 .04605 .04431 .04447 .04088 .03921

.20001 .56931 .81818 .17918 .46001 .64797 .20179 .57006 .81664 .17807 .45537 .64290 .17883 .47684 .73950 .16224 .39080 .58209

All in all, the simulation results shown in Table 6.7 allow the following conclusions: (i) As long as the equivalence range is chosen as an interval symmetric about 1/2 (which will be the case in the majority of practical applications), sample sizes of one dozen per group are sufficient to ensure that the Mann-Whitney test for equivalence is strictly valid with respect to the significance level. This is even true for pairs of underlying distribution function which satisfy the null hypotheses but exhibit gross departure from homoskedasticity. The anticonservative tendency to be observed in very small samples under some parametric submodels for markedly asymmetric choices of the equivalence limits is only slight and can be ignored for most practical purposes. (ii) The power attained by the Mann-Whitney test for equivalence under various shift models shows little sensitivity even to gross changes in form of the baseline distribution. Substantial differences in power can only be seen if any of the traditional location-shift models is contrasted with the model involving heteroskedasticity in addition to shift in location. In fact, heteroskedasticity (as well as other differences in form of the

Mann-Whitney test for equivalence

133

two distributions to be compared) has a marked negative impact on the efficiency of the test. (iii) Conclusion (iii) stated on p. 106 with regard to the efficiency of the signed rank test for equivalence under the Gaussian submodel relative to the (paired) t-test applies mutatis mutandis to the Mann-Whitney test for equivalence as well. By direct analytical arguments, some interesting and intuitively plausible properties of the (exact) power function of the Mann-Whitney test for equivalence can be established which hold under any model obtained by replacing the standard Gaussian cdf with an arbitrary continuous F◦ in the setting of the ordinary two-sample t-test. To be more specific, let us denote (ϑ, σ) the rejection probability of the test when applied with data by βFm,n ◦ such that Xi = σXi◦ + ϑ, Xi◦ ∼ F◦ ∀i = 1, . . . , m, Yj = σYj◦ , Yj◦ ∼ F◦ ∀j = 1, . . . , n. Then, the following statements hold true: (a)

F◦ symmetric about 0 or m = n ⇒ βFm,n (−ϑ, σ) = βFm,n (ϑ, σ) ∀ (ϑ, σ) ∈ IR × IR+ ; ◦ ◦

(b)

βFm,n (ϑ, σ) = βFm,n (ϑ/σ, 1) ∀ (ϑ, σ) ∈ IR × IR+ . ◦ ◦

The second of these results implies that in any (not only the Gaussian!) location-scale model for the two-sample setting which assumes homogeneity of the underlying distributions with respect to the scale parameter σ, the equivalence hypothesis which the test presented in this section is tailored for, is a statement about ϑ/σ rather than ϑ per se. This is one of several basic properties distinguishing the testing procedure under consideration from the interval inclusion procedure with Mann-Whitney based confidence limits to ϑ as proposed by Hauschke et al. (1990). In fact, the interval estimation method forming the basis of the latter presupposes a simple location-shift model as becomes obvious from its construction (cf. Lehmann, 1963). Hence, it cannot be taken for granted that the corresponding interval inclusion test keeps being valid if the underlying nonparametric model covers also pairs of distributions differing in dispersion or other shape parameters in addition to location. Thus, apart from the fact that both approaches relate to totally different formulations of hypotheses, it is clear that the test obtained on the preceding pages is (asymptotically) distribution-free in a much stronger sense than the interval inclusion procedure based on Moses∗ -Lehmann confidence limits. ∗ The

technique of computing nonparametric confidence limits for the shift in location between two continuous distributions of the same shape and dispersion by means of the MannWhitney null distribution has been originally introduced in a textbook chapter authored by Moses (1953, pp. 443-5).

134

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Mann-Whitney test for noninferiority If we want to test for noninferiority in the nonparametric two-sample setting (6.12) with underlying distribution functions F and G of the continuous type, this means that we have to consider the hypotheses H1 : π+ ≤ 1/2 − ε

versus

K1 : π+ > 1/2 − ε .

(6.21)

Since (W+ − π+ )/ˆ σ [W+ ] converges in law to a standard normal variable as m, n → ∞ (under the very weak conditions made precise above in the derivation of the Mann-Whitney test for two-sided equivalence), the general result stated in § 2.3 implies that an asymptotically valid test for (6.21) is given by the rejection region    σ [W+ ] > u1−α . (6.22) W+ − 1/2 + ε /ˆ Its behavior in finite samples can be assessed from the simulation results presented in Table 6.8 which, except for omitting the right-hand limit of the theoretical equivalence range to the target functional, is organized analogously to Table 6.7. As in the Mann-Whitney test for two-sided equivalence, exceedances of the target significance level which might be of some concern for practical purposes, can be observed, if at all, only for the smallest of the sample sizes investigated, i.e., m = n = 12. Under the homoskedastic Gaussian shift model, the loss in power as compared to the optimal parametric test for (6.21), is roughly the same as in the case of two-sided equivalence. With regard to (in-)sensitivity against changes in the form of the distribution functions compared, the results obtained for the noninferiority case are likewise well comparable to those shown in the previous table: The most marked differences across the models arise through allowing for heteroskedasticity which, for a variance ratio of 9, reduces the power against null alternatives by up to about 6%.

Mann-Whitney test for equivalence

135

Table 6.8 Simulated rejection probabilities of the Mann-Whitney test for noninferiority at level α = .05 at the boundary of the hypotheses (column 5) and π+ = 1/2 (rightmost column) with samples of common size n = 12, 24, 36 from distributions belonging to the families listed in Table 6.6 and the same specifications of the equivalence margin ε . [The italicized values give the power of the two-sample t-test for equivalence at level α = .05 against π+ = 1/2.] Family of Equivalence m Distributions margin ε Gaussian

Cauchy

Uniform

Exponential

n

Rejection Probability at π+ = 1/2 − ε 1/2

.2000

12 12

.04902

"

24 24

.04516

"

36 36

.04344

.1382

12 12

.05668

"

24 24

.04836

"

36 36

.05036

12 24 36 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36

.05146 .04846 .04412 .05744 .05315 .05036 .04748 .04348 .04215 .05607 .05014 .04708 .04908 .04447 .04260 .05722 .04937 .04765

.2000 " " .1382 " " .2000 " " .1382 " " .2000 " " .1382 " "

12 24 36 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36

.54167 .56858 .78273 .81853 .90821 .92911 .34380 .33276 .51296 .53508 .65860 .68286 .53895 .78933 .91015 .34597 .52233 .65860 .54198 .78192 .90783 .34687 .51843 .65595 .54265 .78407 .91012 .34678 .51946 .65444

136

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.8 (continued) Family of Distributions Laplace

Logistic

N (ϑ, 9)/N (0, 1)

6.3

Equivalence m margin ε .2000 " " .1382 " " .2000 " " .1382 " " .2000 " " .1382 " "

12 24 36 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36

n 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36 12 24 36

Rejection Probability at π+ = 1/2 − ε 1/2 .05055 .04547 .04332 .05791 .05098 .04882 .04824 .04443 .04322 .05694 .04999 .04766 .04966 .04424 .04410 .05744 .05119 .04839

.54191 .78196 .90967 .34464 .51735 .65821 .53956 .78391 .90729 .34545 .51625 .65516 .49628 .72957 .86861 .32071 .46791 .59737

Two-sample equivalence tests based on linear rank statistics

In this section, general results derived in Janssen (1999, 2000) will be exploited to establish two-sample equivalence tests which are based on Wilcoxon’s ranksum and the Savage statistic, respectively. From a practical point of view, these tests are to supplement the repertoire of non- and semiparametric equivalence testing procedures which are made available in other parts of this chapter for commonly occurring two-sample problems with continuous data. The first of these tests to which the procedures considered in the present section provide a useful alternative, is the Mann-Whitney test derived in § 6.2. A semiparametric equivalence test which, like the test based on the Savage statistic, is tailored for the analysis of survival data and allows for right censoring in addition, will be presented under the heading “log-rank test for equivalence.” Both of these tests are large-sample procedures so that, by construction, maintenance of the target significance level is guaranteed only asymptotically. Exact tests for interval hypotheses about distribution func-

Two-sample equivalence tests based on linear rank statistics

137

tions satisfying the proportional hazards assumption have been considered by Munk (1996). The general strategy for constructing linear rank tests for equivalence hypotheses is as follows: At a first stage, a specific semiparametric submodel is assumed and the critical constants for the test statistic computed from the exact distribution which the selected linear rank statistic follows under that submodel. At a second stage, the rejection probabilities of the test are studied by simulation under distributions outside the semiparametric submodel for which the construction was originally carried out, with a view to establishing that the exact computations done under the submodel provide satisfactory approximations to the operation characteristic of the test under the full nonparametric model for the two-sample setting with continuous observations. The semiparametric model which we start from is given by the relationship 1 − F (t) = (1 − G(t))θ

(6.23)

which has to hold for all t ∈ IR and some θ > 0. In classical nonparametric theory, each cdf F satisfying (6.23) is said to constitute a Lehmann alternative to G (see, e.g. Randles and Wolfe, 1979, § 4.3). In a survival context where both cdf’s have support [0, ∞) and are assumed to be differentiable, the class of all pairs (F, G) satisfying (6.23) is called a proportional hazards or relative risk model (cf. Kalbfleisch and Prentice, 2002, § 2.3.2). In developing the details of the approach, some additional pieces of notation will be used. The probability distributions on the real line defined by the cdf’s F and G under comparison, are denoted by P and Q, respectively. The proversion probability π+ defined in (6.11) will be explicitly written as a functional of (P, Q) setting  π+ = κM W W (P, Q) = Q((−∞, t]) dP(t) . (6.24) The second functional which will be used in the sequel as a measure of dissimilarity between the distributions from which the samples (X1 , . . . , Xm ) and (Y1 , . . . , Yn ) are taken, is usually named after Savage (1956). It is particularly useful in the analysis of samples consisting of positive values as obtained from survival distributions. In order to give a formal definition, let ΛP (t) = − log(1 − F (t)) denote the cumulative hazard function correspond t f (s) ing to F . Recall that ΛP (t) = 0 λP (s)ds holds where λP (s) = is 1 − F (s) the hazard rate at s whenever F has a density f . Denoting the cumulative hazard of the second distribution under analysis by ΛQ the Savage functional is defined by  (6.25) κSAV (P, Q) = ΛQ (t)dP(t) . An equivalent representation involving the two cumulative hazards only, reads  κSAV (P, Q) = exp(−ΛP (t))dΛQ (t) . (6.26)

138

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Interestingly, κSAV (P, Q) has an interpretation as the mean relative risk of the two populations. Actually, in the proportional hazards model (6.23), there holds ΛP = θΛQ implying that κSAV (P, Q) = 1/θ is exactly the (time independent) relative risk between the populations. Without the assumption of proportional hazards κSAV (P, Q) is given by an average. With a view to symmetry, it is more convenient to consider the logarithmic version of (6.25), namely  (6.27) κLM RR (P, Q) ≡ log ΛQ (t)dP(t) which is called the logarithmic mean relative risk functional. It is easy to see that κM W W and κLM RR are invariant with respect to strictly increasing transformations of the observations Xi and Yj . Under the semiparametric model (6.23), both functionals are simple functions of the scalar parameter θ only since we then have: κM W W (P, Q) = 1/(1 + θ) ,

κLM RR (P, Q) = − log θ .

(6.28)

Further important properties can be found in the Appendix to Janssen and Wellek (2008) where also proofs for all more technical results underlying the development presented here are given. Put in general terms, by a linear rank test for equivalence of F and G (or P and Q) with respect to some functional κ(·, ·), we mean a testing procedure which rejects the null hypothesis of relevant differences if and only if it turns out that m    aN RN (Xi ) < c2 . (6.29) c1 < SN ≡ m−1/2 i=1

The form of the test statistic SN appearing in this critical inequality is the same as in a linear rank test of the null hypothesis P = Q of the two-sided testing problem traditionally considered in deriving nonparametric tests for two independent samples. This is to say that RN (Xi ) stands for the rank of the ith observation from P with respect to the pooled sample (X1 , . . . , Yn ), and (aN (1), . . . , aN (N )) denotes a suitable score vector to be chosen in dependence of the functional in terms of which the hypotheses are formulated. Up to now, for nonparametric equivalence problems, no results about an optimal choice of the scores are available. However, as argued by Janssen and Wellek (2008), a promising way of choosing aN ( · ) is to adopt exactly the same specifications as lead to rank tests of the classical null hypothesis P = Q which are efficient against alternatives defined in terms of the selected functional. As shown by Janssen (2000), the efficient score vectors corresponding to κM W W and κLM RR have elements r WW aM − 1/2 (6.30a) (r) = N N +1 and r  1 RR aLM − 1, (6.30b) (r) = N N +1−q q=1

Two-sample equivalence tests based on linear rank statistics

139

respectively. Given the scores, the major problem left to solve is to determine the critical constants c1 , c2 which, of course, have to depend both on the target significance level α and the equivalence region specified by the alternative hypothesis. Setting φN ≡ I(c1 ,c2 ) (SN ), a first order approximation to the power of the equivalence test given by (6.29) is obtained by applying a central limit theorem derived by Janssen (2000). The accuracy is satisfactory for practical purposes in neighborhoods of the null model assuming identical distributions in both populations sampled, without restricting consideration to a specific semiparametric submodel. In order to state the result, let VN2 denote the variance of the linear rank statistic SN under P = Q. Furthermore, let ϑ stand for the value taken on by an arbitrary functional κ( ·, · ) in its centered version κ(P, Q) − κ(P, P). Then we can write  E(P,Q) (φN ) ≈ Φ

c2 − ϑm1/2 n/N VN



 −Φ

c1 − ϑm1/2 n/N VN

 .

(6.31)

When Mann-Whitney-Wilcoxon scores are used, the null variance of SN can be computed exactly from the equation VN2 =

n ≡ VN2 ; M W W . 12(N + 1)

(6.32a)

For the normalized sum of Savage scores, the exact formula for the null variance is relatively complicated (see, e.g. H´ ajek et al., 1999, Eq. (4.2.1.18)) and will be replaced with the approximation VN2

n ≈ N

1

2 n 1 + log(1 − u) du = ≡ VN2 ; LM RR . N

(6.32b)

0

The validity of (6.31) for any pair (P, Q) of underlying distributions has two major implications. Firstly, if the equivalence range for the centered functional ϑ is chosen to be (−ε1 , ε2 ), it suggests to determine the critical  constants c1 , c2 by solving the equations Φ (c2 + ε1 m1/2 n/N )/VN − Φ (c1 +      ε1 m1/2 n/N )/VN = α = Φ (c2 −ε2 m1/2 n/N )/VN −Φ (c1 −ε2 m1/2 n/N )/VN and to subsequently use the result as initial values for an iteration scheme based on the exact distribution of the test statistic. A second implication is that any exact solution established for a suitably chosen submodel can be used as an approximate solution for the full nonparametric model. The key fact which allows us to construct the tests in the semiparametric proportional hazards model (6.23) as exact procedures is that, under Lehmann alternatives, the distributions of linear rank statistics are known exactly (at least in principle) and do not depend on the “baseline” distribution function G . The task of deriving formulae for the exact distribution of SN in the proportional hazards model is greatly facilitated through representing test

140

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

statistics of that form in terms of random group indicators. For each r = 1, . . . , N , let " 1 if rank r is taken on by an observation ZN r = . (6.33) belonging to Sample 1 0 otherwise Then, the linear rank statistic given by the scores aN (1), . . . , aN (N ) can be written N  SN = m−1/2 aN (r)ZN r . (6.34) r=1

Furthermore, under (6.23), the exact joint distribution of the group indicators ZN r is well known (see, e.g., Savage, 1956; Kalbfleisch and Prentice, 2002, p. 106) to be given by the probability mass function   Pθ,m,n (ZN 1 , . . . , ZN N ) = (z1 , . . . , zN ) = m!n!

N ' r=1

N 

zr + (1 − zr )/θ , N  zq + (1 − zq )/θ

q=r N 

∀(z1 , . . . , zN ) ∈ {0, 1}N with

(6.35)

q=r

zr = m .

r=1

By means of (6.35), an exact expression for the distribution function of (6.34) can be established. Observing that any linear rank statistic is constant whenever N = m or N  = n holds, yields the boundary conditions SN = 0 N for m = 0 and SN = r=1 aN (r) if m = N for the following recursion formula which is very useful for computational purposes and holds under proportionality of hazards for each m, n ≥ 1 and x ∈ IR :

Pθ,m,n

(N 

) aN (r)ZN r ≤ x

r=1

m Pθ,m−1,n m + n/θ n/θ + Pθ,m,n−1 m + n/θ

= (N −1 

) aN (r + 1)ZN −1,r ≤ x − aN (1)

r=1

(N −1 

) aN (r + 1)ZN −1,r ≤ x

.

(6.36)

r=1

As special cases of (6.36), recursion formulae for the distribution functions of the Wilcoxon and Savage statistics are obtained. For the sake of simplicity, it is advantageous to work with the sum of the ranks themselves and the RR scores aLM (r)+1, respectively. Using the corresponding expressions for the N original test statistics SN requires just elementary steps of rescaling. Dealing

Two-sample equivalence tests based on linear rank statistics with SN ; M W W first, we set aN (r) = r so that that we have N (N + 1)/2 . Then, (6.36) simplifies to ) (m  Pθ,m,n RN (Xi ) ≤ x = i=1

m Pθ,m−1,n m + n/θ n/θ + Pθ,m,n−1 m + n/θ

(m−1  i=1 (m 

141 N r=1

aN (r) =

) RN −1 (Xi ) ≤ x − m ) RN −1 (Xi ) ≤ x − m

.

(6.36a)

i=1

RR An analogue of (6.36a) is obtained by setting bN (r) = aLM (r) + 1 with N LM RR (r) being defined as in (6.30b). These shifted Savage scores sum up to aN N , and the recursion formula reads ) (m    Pθ,m,n bN RN (Xi ) ≤ x = i=1

m Pθ,m−1,n m + n/θ n/θ + Pθ,m,n−1 m + n/θ

(m−1  (

i=1 m 

 m bN −1 RN −1 (Xi )) ≤ x − N bN −1

i=1



m RN −1 (Xi )) ≤ x − N

) ) .

(6.36b)

The recursion formula (6.36a) is equivalent to that of Shorack (1967) for the Mann-Whitney form of the Wilcoxon statistic. A recursion for the distribution of the Savage statistic was used by Munk (1996) (see also Davies, 1971). For a closer study of the properties of equivalence tests with rejection regions of the form (6.29), it is convenient to denote the centered version of the functional κ( ·, · ) for which the test statistic is tailored, by an extra symbol setting ϑ ≡ ϑ(P, Q) = κ(P, Q) − κ(P, P) . (6.37) The relationship between ϑ and the hazard ratio θ depends on the selected functional under consideration. For the Mann-Whitney-Wilcoxon functional κM W W and the log Savage functional κLM RR , there holds θ=

1 − 2ϑ , 2ϑ + 1

ϑ ∈ (−1/2, 1/2) ,

(6.38a)

and θ = exp(−ϑ),

ϑ ∈ IR ,

(6.38b)

respectively. In both of these cases, we write Eϑ (ϕN ) = E(P,Q) (ϕN ) for the rejection probability of the test under any pair (P, Q) of underlying distributions which

142

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

satisfy (6.23) with θ = exp(−ϑ). Power computations are considerably simWW or plified making use of the following results for the tests ϕN = ϕM N LM RR given by (6.29) for fixed c1 < c2 : ϕN = ϕN a) The function ϑ → Eϑ (ϕN ) is continuous. b) If there holds E0 (ϕN ) > α and min SN < c1 < c2 < max SN , then there exist equivalence margins 1 < 0 < 2 for ϑ such that E 1 (ϕN ) = E 2 (ϕN ) = α .

(6.39)

c) Denoting the distribution of any real-valued statistic TN on the sample space of (X1 , . . . , Yn ) under ϑ(P, Q) = ϑ by L(TN |ϑ), we have for each ϑ and all (m, n) MW W MW W L(SN |ϑ) = L(−SN | − ϑ) .

(6.40a)

In the balanced case m = n, there holds LM RR LM RR L(SN |ϑ) = L(−SN | − ϑ)

(6.40b)

in addition. All exact numerical results which will be presented below for the MannR Whitney-Wilcoxon case have been obtained by means of a MATLAB function implementing the recursion scheme (6.36a). Even for sample sizes of 100 each, execution time of this program on a standard Intel PC is reasonably short so that the algorithm is well suited for routine applications. The computational effort entailed in exact calculation of the distribution of the Savage statistic under arbitrary Lehmann alternatives is much higher than for the R program we used for determining Wilcoxon statistic. Another MATLAB the critical constants of the exact version of the Savage test for equivalence as well as for computing its power, is based on formula (6.35) rather than the recursion relation (6.36b). The program can conveniently be used as long as the larger of both sample sizes does not exceed 12. For larger sample sizes, we recommend to rely on the approximation obtained by applying (6.31) and check on its accuracy by means of Monte Carlo simulations. In view of the independence of the exact distribution of the baseline distribution function G, such simulations can easily be carried out by generating pseudo random observations from suitably scaled exponential distributions. With a view to power comparisons between the Wilcoxon and the Savage test, it is essential to note that in the nonparametric framework, different functionals (scales) require different specifications of the equivalence margins. Exploiting the relationships (6.38a,b) between the hazard ratio θ and the respective version of the centered functional ϑ, it is easy to establish a rule for adjusting to each other the equivalence regions to be used in the Wilcoxon and the Savage test. From (6.38a), we know that if ϑM W W ≡M W W −1/2 is

Two-sample equivalence tests based on linear rank statistics

143

chosen to be the functional of primary interest and the proportional hazards assumption holds true, the equivalence hypothesis − < ϑM W W <  is the same as

   1 1   1 + θ − 2 < 

(6.41a)

(6.41b)

for any  > 0. The latter statement is in turn equivalent to | log(θ)| < log (.5 + )/(.5 − ) , where, according to (6.38b), − log(θ) coincides with the centered Savage functional in its logarithmic version. Denoting the latter by ϑLM RR , this implies that ϑM W W falls in the equivalence interval (−, ) if and only if the distributions under comparison belong to the equivalence region     ϑLM RR  < log 0.5 +  (6.41c) 0.5 −  defined in terms of the Savage functional. Thus, if primary interest is in assessing equivalence in terms of the MannWhitney-Wilcoxon functional and the equivalence margin for ϑM W W is chosen to be  = 0.2 [recall Table 1.1 (ii)], it follows that a semi-parametric competitor to the rank-sum test for is given by the  equivalence    Wilcoxon Savage test of ϑLM RR  ≥ log(7/3) versus ϑLM RR  < log(7/3) = 0.847 . Comparing both tests in terms of their power functions leads to the couple of graphs shown in Figure 6.1 relating to the usual specification α = .05 of the (nominal) significance level and samples of size m = n = 12. Not surprisingly (in view of the local optimality of the Savage test for the traditional one-sided testing problem about two distribution functions satisfying the proportional hazards model — see, e.g., H´ ajek et al., 1999, p. 106), the Wilcoxon test turns out uniformly inferior with respect to power. Furthermore, the power function of the Wilcoxon test fails to cross the level-α line exactly at the boundary points of the equivalence range which, within the limited range of resolution provided by the graph, cannot be seen for the Savage test. This reflects the obvious fact that, as compared to the Savage statistic, the distribution of the simple sum of ranks exhibits much coarser discreteness.

144

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Figure 6.1 Exact power function of the Wilcoxon [dashed] and the Savage test for equivalence [solid line] at level α = .05 for two samples of sizes m = n = 12 from distributions satisfying a proportional hazards model. Illustrations In order to demonstrate the use of the linear rank tests discussed in this subsection, we first revert once more to the data set shown in Table 6.3. Replacing the individual observations with their ranks with respect to the combined sample yields: Moxonodin: 8; 23; 14; 19;

2;

Captopril: 18; 20; 6;

13; 15; 10; 5;

1;

16; 11; 7;

12; 9; 24; 17 3;

4; 22; 21 .

Summing up all ranks associated with the observations making up the first m MW W sample gives S˜N ≡ R i=1 N (Xi ) = 138. As follows from (6.39) and (6.40a), choosing a theoretical equivalence range to ϑM W W of the form (6.41a) implies that the critical interval of the exact test for equivalence based on  MW W S˜N is in turn symmetric about the expected value of m R (X i ) under i=1 N the traditional null hypothesis P = Q, with limits m(N + 1)/2 ∓ c . In view

Two-sample equivalence tests based on linear rank statistics

145

of (6.31), (6.32a) and (4.26), the critical constant c is approximately given by  (6.42) c ≈ mn  − mn(N + 1)/12 u1−α . For m = n = 12,  = .20 and α = .05, this yields c ≈ .3078. Taking the smallest integer being at least as large as this approximate value of c as an initial value for iterative determination of the exact c yields after a R function mentioned before, c = 5 . Since 138 few runs of the MATLAB falls outside the corresponding critical interval (145, 155), the exact Wilcoxon test for equivalence cannot reject the null hypothesis of relevant differences between both underlying distribution functions. The discrepancy between the approximate and the exact value of c should not be overestimated. First of all, sample sizes are obviously too small for expecting high accuracy of results based on first order asymptotics. FurtherMW W more, it must be kept in mind that even for m = n = 12, the range of S˜N is fairly large (with bounds 78 and 222) so that an approximation error of about 5 does not matter too much on that scale. Replacing unit with Savage scores, we obtain  the ranks  obtained in m from LM RR ≡ i=1 bN RN (Xi ) = 10.3242. In the moxonodin arm of the trial S˜N order to determine the limits of the critical interval to be used in the exact LM RR , we proceed in essentially the same way as in the exact test based on S˜N Wilcoxon test for equivalence. Again, for a symmetric specification of the two equivalence margins, thereis only a single critical constant c to be determined  m R b (X ) at one of the boundary points of from the distribution of N i i=1 N the equivalence hypothesis (6.41c) we are interested in. This time, the first order approximation based on (6.31) and (6.32) yields   * 0.5 +  mn mn log u1−α . (6.43) c≈ − N 0.5 −  N Plugging in the same values as before in the expression on the right-hand side of (6.43) yields c = 1.0547 as initial estimate of the exact critical constant. Repeated runs of the computational procedure described above show that this initial value has to be diminished to c = 0.9766 in order to make sure that the test is carried out the 5% level exactly. Since the  min a way maintaining observed value of i=1 bN RN (Xi ) is smaller than the right-hand limit of the corresponding critical interval (m − c, m + c) = (11.0234, 12.9766), the Savage test fails to lead to a positive decision either. As a second example, we analyze a subset of Kalbfleisch and Prentice’s (2002, pp. 378–9) Data Set I referring to a clinical trial conducted by the Veterans Administration Lung Cancer Study Group of the U.S. In both arms of this trial, patients were in particular stratified with regard to pretreatment status. We suppose that a preliminary test for equivalence of patients with and without previous treatment shall be performed in order to make sure that nonstratified randomization and the assumption of a common treatment effect for the whole study population were appropriate. Discarding a few patients

146

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

who provided censored values and restricting the analysis to the standard chemotherapy arm, there were m = 20 and n = 44 patients with and without previous treatment, respectively. In order to avoid technical complications, a few ties contained in the data set were broken at random. The empirical survival functions obtained from these data are shown in Figure 6.2.

Figure 6.2 Empirical survival functions observed in lung-cancer patients with [dashed] and without previous therapy [solid line]. The set of ordered ranks of the survival times making up the first of these two samples [↔ pretreatment yes] are: 4; 5; 6; 8; 9; 10; 12; 16; 31; 34; 40; 42; 48; 52; 53; 56;

25; 27; 59; 63.

The corresponding values of the (noncentered) Wilcoxon and Savage statistics are computed to be MW W S˜N = 600 and

LM RR S˜N = 18.6004 .

Using the same specification of the equivalence margin  as before, Equation (6.42) gives c = 62.4275 as an approximate value of the critical constant for the

Two-sample equivalence tests based on linear rank statistics

147

Wilcoxon test for equivalence. The computational scheme for determining the MW W described in the previous example does not exact critical interval for S˜N have to rely on the assumption that a balanced study design is used. Hence, we can proceed as before using 63 as the initial value in an iterative search for the largest integer c such that the exact value of Pθ m(N + 1)/2 − c <  MW W S˜N < m(N + 1)/2 + c at θ ∈ {3/7, 7/3} does not exceed α = .05 . In this way, we obtain c = 76 corresponding to a critical interval of (576, 724) MW W . Since the latter covers the observed value of the sum of ranks for for S˜N the pretreatment group the decision is in favor of equivalence. In order to compute the critical interval for the sum of Savage scores of the observations making up the first sample, we start with applying formula (6.43) which gives c = 5.5505 as an approximate value of the distance of both endpoints from m = 20 . However, in the unbalanced case the exact critical LM RR is no longer symmetric about its value expected under the interval for S˜N assumption of identical populations. Thus, we have to determine two different  integers c1 and c2 such that the probability content of m − c1 , m + c2 comes as close as possible to α both under θ = 3/7 and θ = 7/3. A heuristic algorithm for determining (c1 , c2 ) is as follows: In a first step, starting from the large-sample solution, as the largest positive number such  c2 is determined  LM RR < m + c2 ≤ α under θ = 3/7. From that there holds Pθ m − c2 < S˜N a set of Monte Carlo simulation experiments of the kind described above [→ p. 142], it has been found that the initial approximation has to be replaced with c2 = 4.98 in order to meet this condition. In order to ensure that the significance level is kept also under θ = 7/3, the left-hand critical limit has to be increased from m − c2 to m − c1 with c1 = 4.86, implying that the critical interval for the exact Savage test is estimated to be (m − c1 , n + c2 ) = (15.14, 24.98). Again, the observed value of the test statistic is included so that the qualitative decision is the same as has to be taken in the Wilcoxon test for equivalence with the same data. Robustness/sensitivity of the exact tests against violations of the proportional hazards assumption Table 6.9 shows some simulation results on the level and power of the exact linear rank test for equivalence with Wilcoxon scores, under two common location-shift models. For ease of comparison, the corresponding entries in Table 6.7 obtained under the same specifications for the asymptotic test derived in § 6.2, are added as italicized values.

148

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.9 Simulated level and power of the exact rank-sum test for equivalence under Gaussian and Cauchy shift models for the same sizes and specification of the equivalence margins as in the corresponding part of Table 6.7. [The italicized values are taken from the latter, showing the rejection probabilities of the asymptotic MannWhitney-Wilcoxon test of Section 6.2 under the same specifications.]

Family of Distributions Gaussian

Cauchy

m

n

Rejection Probability at ϑ = −0.20 +0.20 0.00

12 12

.04568 .05015 24 24 .04851 .04510 36 36 .04894 .04427

.04580 .20089 .04941 .22020 .04785 .59286 .04565 .57929 .04958 .83373 .04476 .82186

12 12

.04767 .20363 .04780 .20180 .05465 .59412 .04536 .56918 .05466 .83441 .04442 .81617

.04796 .04925 24 24 .05191 .04644 36 36 .05425 .04466

In order to study the behavior of the Savage test for equivalence when the two distribution functions under comparison do not satisfy the semiparametric submodel (6.23), an appropriate choice of a model for generating the data by Monte Carlo simulation is the lognormal distribution. In principle, the log transformation is dispensible since both the functional in terms of which the hypotheses are defined, and the test statistics remain invariant under monotone transformations of any kind applied to each observation. But Savage scores are commonly used in the nonparametric analysis of survival data, and the lognormal distribution is among the most popular survival models with nonproportional hazards. Under nonproportional hazards, one has to refer to the general definition (6.27) of the log Savage functional. In the simulation experiments whose results are summarized in Table 6.10, both distributions of log survival times had unit variance, and the expected values were determined through solving the equation ΛQ (t)dP(t) = θ ∗ with θ ∗ = 7/3 [↔ left column], 3/7 [↔ middle column], and 1 [↔ right column], respectively.

Two-sample equivalence tests based on linear rank statistics

149

Table 6.10 Level and power of the exact Savage test for equivalence under the lognormal model. Distribution underlying Sample 1 (P) : LN (μ, 1) , μ ∈ {−0.8444 , 0.0000 , 1.0630}; baseline distribution (Q): LN (0, 1). [The three values of μ correspond to values 7/3 , 1 , and 3/7 of the generalized Savage functional as defined in (6.26-7).]

m

n

Rejection Probability at μ = 1.0630 −0.8444 0.0000

12 24 36

12 24 36

.0241 .0161 .0127

.0633 .0786 .0760

.3129 .7560 .9279

The simulation results shown in both tables admit the following conclusions: – In the Gaussian shift model, the performance of the MWW rank-sum test with critical constants determined under the proportional hazards model is very satisfactory: the maximum rejection probability under the null hypothesis is slightly smaller than the nominal level throughout. – Replacing the Gaussian with the Cauchy distribution leads to a slight anticonservatism of the MMW test for equivalence whose extent seems tolerable for most practical purposes. – The Savage test turns out to be markedly more sensitive to violations of proportionality of hazards; its application in settings where restriction to this semiparametric model seems unrealistic must be discouraged.

Noninferiority versions of the exact semiparametric tests If the alternative hypothesis one aims to establish is given by the class of all pairs (P, Q) for which the centered functional ϑ of interest satisfies −Δ < ϑ < ∞, the natural form of the critical function of a suitable test is ϕ∗N = I(c∗ ,∞) (SN )

(6.44)

being the indicator function of the noninferiority analogue of a critical region of the form (6.29). The power function of ϕ∗N can be approximated (in the same sense as made precise above for the case of two-sided equivalence [recall the comments on

150

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

(6.31)] by means of the formula E(P,Q) (φ∗N )

 ≈1−Φ

c∗ − ϑm1/2 n/N VN

 .

(6.45)

For the practical implementation of the noninferiority versions of the linear rank tests considered above, (6.45) plays the same role as (6.31) in testing for equivalence in the strict sense. I.e., it can be used for finding a good initial estimate of the exact critical lower bound to the linear rank statistic under consideration. In the noninferiority case, the exact critical constant has to be determined as the (1 − α)-quantile of the distribution of SN under ϑ = −Δ. As in the case of two-sided equivalence, (6.45) is in particular a convenient starting point for computing the exact critical lower bound to the linear rank statistic for a test for noninferiority at prespecified level α in the proportional hazards model. In practice, this leads to using the same initial values for computing the exact value of c∗ as were used in the equivalence case for computing the left-hand bound c1 of the critical interval. Correspondingly, fixing the left-hand limit of the equivalence range for ϑ at the same value as in a two-sided equivalence problem relating to the same functional, the exact values of c∗ and c1 can be expected to differ only slightly. Revisiting briefly the first of the examples used above for illustration, we find this expectation largely confirmed. As before, let us specify Δ = .20 and Δ = log(7/3) = .847 for the test relating to ϑM W W and ϑLM RR , respectively. With these values of the equivalence margin, the critical lower bound MW W LM RR and S˜N to be used in the Wilcoxon and the Savage test for to S˜N noninferiority at exact level α = .05 is computed to be c∗M M W (α) = 148 and c∗LM RR (α) = 11.2304, respectively. On the other hand, the power against the specific alternative ϑ = 0 being of primary interest for the majority of practical applications, increases to .5338 [↔ MMW] and .6236 [↔ LMRR] when proceeding from the equivalence to the noninferiority version of the tests. Moreover, the power function as a whole is no longer concave from below but monotonically increasing (decreasing) in ϑ (θ) and takes every value in the open unit interval, as is generally the case with tests of one-sided hypotheses.

6.4

A distribution-free two-sample equivalence test allowing for arbitrary patterns of ties

The assumptions underlying the construction of a nonparametric twosample test for equivalence described in § 6.2 imply that the occurrence of ties between observations from different samples can be excluded with probability one. Accordingly, the procedure is unsuitable for analyzing data sets made up of independent observations from two different discrete distributions.

A distribution-free two-sample equivalence test allowing for ties

151

In this section we show how to generalize the Mann-Whitney test in such a way that arbitrary patterns of ties can be admitted without affecting the (asymptotic) validity of the procedure. In elaborating this idea, basic analogies with the derivation of a generalized signed rank test for equivalence dealt with in § 5.5 will become apparent. Obviously, the definition of the functional π+ [→ (6.11) , (6.24)] makes sense for any type of distributions which the X’s and Y ’s are taken from. However, in the noncontinuous case, identity of both distributions no longer implies that π+ takes on value 1/2. Instead, in the general case, the relationship F = G ⇒ π+ = 1/2 has to be replaced with F = G ⇒ π+ = (1/2) · (1 − π0 ) provided we define π0 as the functional

(6.46)



π0 = P [Xi = Yj ] =

(G − G )dF

(6.47)

assigning each pair of cdf’s as its value the probability of a tie between an observation from F and an observation from G independent of the former. (On the right-hand side of the second of the equalities (6.47), G stands for the leftcontinuous version of the cdf G as defined by G (y) = P [Yj < y] ∀ y ∈ IR.) In view of (6.46), it seems natural to use the distance of π+ /(1 − π0 ) from the point 1/2 as a measure for the degree of disparity of any two cdf’s F and G from which our two samples (X1 , . . . Xm ) and (Y1 , . . . Yn ) are taken. Adopting this as our basic measure of distance, we are led to formulate our testing problem as H : π+ /(1 − π0 ) ≤ 1/2 − ε1 versus

or π+ /(1 − π0 ) ≥ 1/2 + ε2

K : 1/2 − ε1 < π+ /(1 − π0 ) < 1/2 + ε2 .

(6.48)

For both of the basic functionals appearing in (6.48), there exists an U statistic estimator with a kernel which is just the indicator function of the respective event. Of course, the first of these estimators (for π+ ) is the same as that introduced in the previous section [recall (6.13)]. Analogously, a U statistic estimating π0 is given by 1  I{0} (Xi − Yj ) , mn i=1 j=1 m

W0 =

n

(6.49)

with I{0} (u) = 1 for u = 0 and I{0} (u) = 0 for u ∈ (−∞, 0) ∪ (0, ∞). Furthermore, it is natural to estimate the target functional in terms of which the hypotheses under assessment now have been formulated, by just plugging in (6.13) and (6.49) into the expression π+ /(1 − π0 ). Denoting this latter estimator by Q, we have by definition Q = W+ /(1 − W0 ) .

(6.50)

152

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Now, we proceed by studying the large-sample distribution of (6.50) which is readily obtained as soon as we know the joint asymptotic distribution of the two individual U -statistics involved. Assuming in the sequel that the sequence (m/N )N ∈IN of relative sizes of the first sample converges to some nondegenerate limit λ ∈ (0 , 1), say, it follows from the asymptotic distribution theory√for two-sample U -statistics (see, e.g., Randles and Wolfe, 1979, § 3.6) that N (W+ − π+ , W0 − π0 ) converges in law to a centeredbivariate 2 σ+N σ+0;N normal distribution. The covariance matrix ΣN = , say, of 2 σ+0;N σ0N this distribution can be computed exactly. The exact formulae contain also terms of order O(1/N ): √ N (W+ − π+ ) : (i) Variance of 2 σ+N =

N mn [π+

2 − (N − 1)π+ + (m − 1)ΠXXY + (n − 1)ΠXY Y ]

(6.51a)

ΠXXY = P [Xi1 > Yj , Xi2 > Yj ] , ΠXY Y = P [Xi > Yj1 , Xi > Yj2 ] .

(6.51b) (6.51c)

where

(ii) Variance of 2 σ0N =

√ N (W0 − π0 ):

N mn [π0

− (N − 1)π02 + (m − 1)ΨXXY + (n − 1)ΨXY Y ]

(6.52a)

where ΨXXY = P [Xi1 = Yj , Xi2 = Yj ] , ΨXY Y = P [Xi = Yj1 , Xi = Yj2 ] . (iii) Covariance of σ+0;N =

(6.52b) (6.52c)

√ √ N (W+ − π+ ) and N (W0 − π0 ):

N mn [(m

− 1)ΛXXY + (n − 1)ΛXY Y − (N − 1)π+ π0 ]

(6.53a)

where ΛXXY = P [Xi1 > Yj , Xi2 = Yj ] , ΛXY Y = P [Xi > Yj1 , Xi = Yj2 ] .

(6.53b) (6.53c)

Now let us define the function q( · ) by q(π+ , π0 ) = π+ /(1 − π0 ) and denote the result of evaluating the quadratic form ∇q(π+ , π0 )ΣN (∇q(π+ , π0 )) 2 [with ∇q(π+ , π0 ) as the gradient (row) vector of q( · )] by νN . Then, after straightforward calculations and rearrangements of terms we obtain the expression: 2 σ+N π2 σ2 2π+ σ+0;N 2 νN = + + 0N 4 + . (6.54) 2 (1 − π0 ) (1 − π0 ) (1 − π0 )3 Resorting to the so-called δ-method (cf. Bishop et al., 1975, § 14.6) once more,

A distribution-free two-sample equivalence test allowing for ties

153

√ √ we can conclude that N (Q−π+ /(1−π0 )) = N (q(W+ , W0 )−q(π+ , π0 )) converges weakly to a random variable following a centered normal distribution 2 . with variance ν 2 = limN →∞ νN 2 consists The first step of a natural approach to deriving an estimator for νN in plugging in U -statistic estimators of all functionals of the underlying cdf’s F and G appearing on the right-hand side of equations (6.51)–(6.53). The U -statistic estimators for the functionals indexed by subscript XXY are given by  XXY = Π

m−1 m n     2 I(0,∞) (Xi1 − Yj ) · m(m − 1)n i =1 i =i +1 j=1 1

2

1

I(0,∞) (Xi2 − Yj )

 XXY = Ψ

m−1 m n     2 I{0} (Xi1 − Yj ) · m(m − 1)n i =1 i =i +1 j=1 1

2

1

I{0} (Xi2 − Yj )

 XXY = Λ

m  1 m(m − 1)n i =1 1



 (6.55a)

 (6.55b)

n   I(0,∞) (Xi1 − Yj ) ·

i2 ∈{1,...,m}\{i1 } j=1

 I{0} (Xi2 − Yj ) . (6.55c)

The analogous estimators of ΠXY Y , ΨXY Y and ΛXY Y admit the representation:  XY Y = Π

m n−1 n     2 I(0,∞) (Xi − Yj1 ) · m(n − 1)n i=1 j =1 j =j +1 1

2

1

I(0,∞) (Xi − Yj2 )

 XY Y = Ψ

m n−1 n     2 I{0} (Xi − Yj1 ) · m(n − 1)n i=1 j =1 j =j +1 1

2

1

I{0} (Xi − Yj2 )

 XY Y = Λ

m  n  1 m(n − 1)n i=1 j =1 1

 j2 ∈{1,...,n}\{j1 }

 (6.56a)

 (6.56b)

 I(0,∞) (Xi − Yj1 ) ·  I{0} (Xi − Yj2 ) . (6.56c)

154

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Since each of the estimators defined in (6.13), (6.49), (6.55) and (6.56) is (strongly) consistent for its expectation (cf. Lee, 1990, § 3.4.2), it follows that 2 obtained from (6.51a) by replacing π+ , ΠXXY and the plug-in estimator σ ˆ+N  XXY and Π  XY Y , respectively, is consistent for σ 2 , and so ΠXY Y with W+ , Π +N 2 2 2 on for σ ˆ0N and σ ˆ+0;N . Finally, consistency of W+ , W0 , σ ˆ+N ,σ ˆ0N and σ ˆ+0;N √ 2 ensures that the asymptotic variance νN of the statistic N W+ /(1 − W0 ) (and hence a fortiori its limiting variance ν 2 ) can be consistently estimated by 2 = νˆN

2 2 σ ˆ+N W+2 σ ˆ0N ˆ+0;N 2W+ σ + + . 2 4 (1 − W0 ) (1 − W0 ) (1 − W0 )3

(6.57)

What is left to do then in order to complete the construction of a MannWhitney test for equivalence allowing for arbitrary patterns of ties in the data, is to apply the general approach of § 3.4 to the specific √ case that k = 2, TN = W+ /(1 − W0 ), θ = π+ /(1 − π0 ), σ = ν and τˆN = νˆN / N . Accordingly, we end up with the decision rule π+ 1 π+ 1 Reject H : ≤ − ε1 or ≥ + ε2 if and only if 1 − π0 2 1 − π0 2  √   ∗   N W+ /(1 − W0 ) − (1 − ε1 + ε2 )/2) νˆN < CM W (α; ε1 , ε2 ) ,

(6.58)

with

∗   2 CM W (α; ε1 , ε2 ) = 100α−percentage point of the χ − distri2 νN bution with df = 1 and λ2nc = N (ε1 + ε2 )2 /4ˆ

1/2

(6.59)

By Theorem A.3.4, it is a fact that the corresponding test is asymptotically valid over the whole class of all pairs (F, G) of distribution functions on the √ real line such that the limiting variance ν 2 of N W+ /(1 − W0 ) does not vanish. Inspecting specific families of discrete distributions corroborates the impression that this regularity condition is actually very mild. Even in the extreme case that both the Xi and the Yj are Bernoulli variables, the requirement that ν 2 has to be positive rules out merely those constellations in which at least one of the two underlying distributions is degenerate on its own. In order to reduce the practical effort entailed by carrying out the generalized Mann-Whitney test for equivalence to a minimum, another special computer program is provided at the URL accompanying this book, again both as a SAS macro and a R function. The program for which the name mwtie_xy has been chosen, performs all necessary computational steps from scratch, i.e., upon just providing the m + n raw data values as input.

A distribution-free two-sample equivalence test allowing for ties

155

Computational methods for grouped data Suppose that the set of values taken on by any of the observations in the pooled sample is given by {w1 , ..., wK } such that w1 < w2 < ... < wK and the number K of different groups of values is small compared to both sample sizes. Then, computations can be made considerably faster by reducing the raw data to group frequencies before calculating the various counting statistics involved. For elaborating on this idea, we need some additional notation. Let Mk and Nk be the number of X’s and Y ’s, respectively, taking on value wk (k = 1, . . . , K). Furthermore, define corresponding cumulative frequencies   Mkc and Nkc by M0c = N0c = 0, Mkc = kl=1 Mk , Nkc = kl=1 Nl , k = 1, . . . , K. Then, it is easy to verify that the individual U -statistic estimators required for computing the test statistic of (6.58) admit the following representations: 1  1  c Mk Nk−1 , W0 = Mk Nk ; W+ = mn mn

 XXY = Π

K

K

k=1

k=1

(6.60)

 1 (m − Mkc )(m − Mkc − 1) Nk , m(m − 1)n

(6.61a)

 1 Mk (Mk − 1) Nk , m(m − 1)n

(6.61b)

 1 Mk (m − Mkc ) Nk ; m(m − 1)n

(6.61c)

 1 c c Mk Nk−1 (Nk−1 − 1) , mn(n − 1)

(6.62a)

 1 Mk Nk (Nk − 1) , mn(n − 1)

(6.62b)

 1 c Mk Nk Nk−1 . mn(n − 1)

(6.62c)

K

k=1

 XXY = Ψ

K

k=1

 XXY = Λ

K

k=1

 XY Y = Π

K

k=1

 XY Y = Ψ

K

k=1

 XY Y = Λ

K

k=1

The above identities give rise to a simplified version of the code for the procedure to be found in the WKTSHEQ2 Source Code Package under the program name mwtie_fr. The program likewise processes the primary set of raw data. The (cumulative) frequencies appearing in expressions for the U -statistics involved are determined automatically and need not be available as entries in a user-supplied 2 × K contingency table.

156

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Example 6.2 In a study of a N-methyl-D-aspartate receptor 2B (NR2B) gene variant as a possible risk factor for alcohol dependence (Schumann et al., 2003), a single nucleotide polymorphism (SNP) located at position 2873 of the gene was used in genotyping m = 204 patients and n = 258 unrelated healthy controls. The polymorphism leads to a C (cytosine) to T (thymine) exchange. Table 6.11 shows the frequencies of the 3 possible genotypes found in both samples. Table 6.11 Observed distribution of SNP C2873T NR2B genotypes in patients with alcohol dependence and healthy controls.

Group

Genotype CC CT TT



Patients

103 (50.5)

84 17 (41.2) (8.3)

204 = m (100.0)

Controls

135 (52.3)

105 18 (40.7) (7.0)

258 = n (100.0)

Since the location of the mutation was suspected to be outside the gene region encoding the domain of an antagonist of the NMDA receptor assumed to play a major role in mediating the reinforcing effects of alcohol abuse, the authors of the study aimed at excluding the existence of a substantial association between SNP C2873T NR2B genotype and alcohol dependence. In a traditional study of this type being launched with the intention to establish the association, the standard inferential procedure is Armitage’s (1955) test for an up- or downward trend in a 2 × K contingency table with ordered column categories and fixed row margins (for more recent reviews of this method focusing on applications to genetic epidemiology see Sasieni, 1997; Devlin and Roeder, 1999). Experience shows that the decision to which this procedure leads rarely differs from that to be taken in a Mann-Whitney test corrected for the large number of ties occurring in a set of data taking on only three different values, namely 0, 1 and 2 (giving simply the number of mutated alleles in an individual’s genotype). This fact suggests analyzing the data set underlying Table 6.11 by means of the equivalence counterpart of the tie-corrected Mann-Whitney test derived above. Let the significance level be chosen as usual, i.e., α = .05, and fix the equivalence limits to π+ /(1 − π0 ) − 1/2 at ∓.10. Running the grouped-data version mwtie_fr of the respective program we obtain for the genotype frequencies shown above the following estimates: W+ = .29298 ,

W0 = .43759 ,

νˆN = .918898.

A distribution-free two-sample equivalence test allowing for ties

157

With these values and tolerances ε1 = ε2 = .10, the test statistic to be used according to (6.58) is computed to be √ √ N |W+ /(1 − W0 ) − (1 − ε1 + ε2 )/2| 462 |.52093 − .5| = .489579. = νˆN .918898 √ As the critical upper bound which N |W+ /(1−W0 )−(1−ε1 +ε2 )/2|/ˆ νN has to be compared to at the 5%-level, the program yields the value 0.70545 which clearly exceeds the observed value of the test statistic. Thus, the results of the study warrant rejection of the null hypothesis that there is a nonnegligible association between the polymorphism under study and alcohol dependence.

Simulation Study Samples from rounded normal distributions. In numerous applications, occurrence of ties between quantitative observations can suitably be accounted for by numerical rounding processes which the associated “latent” values on some underlying continuous measurement scale are subject to. If the underlying continuous variables have Gaussian distributions with unit variance and rounding is done to the nearest multiple of some fixed rational number r, say, then the distribution of the observable discrete variables is given by a probability mass function of the form 

z+r/2

pμ;r (z) = z−r/2

1 √ exp{−(u − μ)2 /2} du , z = j r, j ∈ Z . 2π

(6.63)

In the sequel, we will call any distribution of this form a rounded normal distribution and write Z ∼ Nr (μ, 1) for a discrete random variable Z having mass function (6.63). In a first set of simulation experiments on the generalized Mann-Whitney test for equivalence with discretized normally distributed data, rounding was done to nearest multiples of r = 1/4. Whereas the Yj were generated by rounding standard normal random numbers, three different specifications of μ were used for generating the Xi : (a) μ = −.695 ; (b) μ = +.695 ; (c) μ = 0 . The number −.695 was determined by means of a numerical search algorithm as that value of μ which ensures that Xi ∼ N1/4 (μ, 1), Yj ∼ N1/4 (0, 1) ⇒

π+ /(1−π0 ) = .30 . By symmetry, it follows that N1/4 (.695, 1) , N1/4 (0, 1) is a specific pair of distributions belonging to the right-hand boundary of the hypotheses (6.48) for the choice ε1 = ε2 = .20 . Of course, generating both the Xi and the Yj as samples from N1/4 (0, 1) provides a basis for studying the power of the test against the specific alternative that π+ /(1 − π0 ) = 1/2. As another setting involving samples from rounded normal distributions we investigated the case r = 1, i.e., rounding to the nearest integer, which of

158

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

course produces a much higher rate π0 of ties between X’s and Y ’s. For r = 1, the nonnull μ’s corresponding to π+ /(1 − π0 ) = .30 and π+ /(1 − π0 ) = .70 are computed to be ∓.591.  The mass functions of the three elements of the  family N1 (μ, 1)  μ ∈ IR corresponding to μ = −.591, μ = 0 and μ = +.591 are depicted in Figure 6.3 . Tables 6.12a,b show the rejection probabilities of the generalized MannWhitney test for equivalence at nominal level α = .05 at both boundaries of the hypotheses (→ size) and the center of the equivalence range (→ power) found in 100, 000 replications of the simulation experiments performed with data from rounded normal distributions. The results suggest that the test keeps the nominal significance level even when both sample sizes are fairly small. Furthermore, for given sample sizes the power of the test decreases markedly with the proportion of ties between observations from different populations. Table 6.12a Simulated rejection probabilities of the test defined by (6.58) with data from normal distributions rounded to the nearest multiple of .25. (From Wellek and Hampel, 1999, with kind permission by Wiley-VCH.)

m

n

20 40 60 10

20 40 60 90

Rejection Prob. Rejection Prob. Power at π+ /(1 − π0 ) at π+ /(1 − π0 ) at π+ /(1 − π0 ) = .30 = .70 = .50 .04429 .04337 .04245 .04830

.04374 .04279 .04253 .04959

.36408 .79949 .95070 .23361

Table 6.12b Simulated rejection probabilities of the test defined by (6.58) with data from normal distributions rounded to the nearest integer. (From Wellek and Hampel, 1999, with kind permission by Wiley-VCH.)

m

n

Rejection Prob. at π+ /(1 − π0 ) = .30

20 40 60 10

20 40 60 90

.04277 .04212 .04247 .04664

Rejection Prob. Power at π+ /(1 − π0 ) at π+ /(1 − π0 ) = .70 = .50 .04378 .04216 .04248 .04681

.20246 .54873 .80174 .18361

A distribution-free two-sample equivalence test allowing for ties

159

Figure 6.3 Normal distributions rounded to the nearest integer with ), μ = 0 ( ), and μ = .695 ( ). μ = −.695 ( (From Wellek and Hampel, 1999, with kind permission by Wiley-VCH.)

160

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Lehmann alternatives to discrete uniform distributions. Another parametric submodel of considerable interest for purposes of investigating the small sample properties of the test under consideration is given by the class of Lehmann alternatives to the uniform distribution on the set {1, . . . , k} of the first k natural numbers (k ∈ IN, fixed). By definition, each distribution belonging to this class has a probability mass function of the form pθ;k (j) = (j/k)θ − ((j − 1)/k)θ ,

j = 1, . . . , k,

θ > 0.

(6.64)

The symbol Uk (θ) is to denote any specific distribution of that form. In the first part of the simulations referring to the model (6.64), the number of mass points was fixed at k = 6, and the parameter θ varied over the set {θ1 , θ2 , θ0 } with (a) θ1 = .467 ; (b) θ2 = 2.043 ;

(c) θ0 = 1 .

These values were in such a way that for ε1 = ε2 = .20, the pair   determined  U6 (θ1 ) , U6 (θ0 ) and U6 (θ2 ) , U6 (θ0 ) belongs to the left- and the right-hand boundary of the hypotheses (6.48), respectively. Bar charts of the corresponding probability mass functions are displayed in Figure 6.4. The second half of the results shown in Table 6.13 for data generated from probability mass functions of the form (6.64) relates to Lehmann alternatives to the uniform distribution on the set {1, 2, 3}. In the case of these much coarser distributions, the values ensuring that Xi ∼ U3 (θ1 ), Yj ∼ U3 (1) ⇒ π+ /(1 − π0 ) = .30 and Xi ∼ U3 (θ2 ), Yj ∼ U3 (1) ⇒ π+ /(1 − π0 ) = .70 were computed to be θ1 = .494 and θ2 = 1.860, respectively. Again, the nominal significance level was set to α = .05 throughout, and all rejection probabilities listed in the tables are based on 100, 000 replications. Essentially, we are led to the same conclusions as suggested by the results for the rounding error model with underlying Gaussian distributions: For balanced designs, the generalized Mann-Whitney test for equivalence guarantees the nominal significance level even when both sample sizes are as small as 20. However, the rejection probability found in the case k = 6, (m, n) = (10, 90) at the right-hand boundary of the hypotheses shows that the test is not in general strictly conservative. Remark. It is worth noticing that ranking the four tables 6.12a – 6.13b with respect to the power of the test at the center of the equivalence interval yields for every combination of sample sizes the same ordering, namely (6.13b)(6.12b) (6.13a)(6.12a). This is exactly the ordering obtained by arranging the associated values of the tie probability π0 into a decreasing sequence: Straightforward computations show that π0 takes on value .333, .250, .167 and .062 for U3 , N1 , U6 and N1/4 , respectively.

A distribution-free two-sample equivalence test allowing for ties

161

Figure 6.4 Lehmann alternatives to the uniform distribution on {1, . . . , 6} with ), θ = 1 ( ), and θ = 2.043 ( ). (From Wellek and θ = .467 ( Hampel, 1999, with kind permission by Wiley-VCH.)

162

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.13a Simulated rejection probabilities of the test defined by (6.58) under Lehmann alternatives to the uniform distribution on {1, 2, . . . , 6}. (From Wellek and Hampel, 1999, with kind permission by Wiley-VCH.)

m

n

20 40 60 10 90

20 40 60 90 10

Rejection Prob. Rejection Prob. Power at π+ /(1 − π0 ) at π+ /(1 − π0 ) at π+ /(1 − π0 ) = .30 = .70 = .50 .04500 .04266 .04184 .04838 .04370

.04312 .04202 .04201 .05230 .04544

.26757 .68698 .89465 .25194 .24936

Table 6.13b Simulated rejection probabilities of the test defined by (6.58) under Lehmann alternatives to the uniform distribution on {1, 2, 3}. (From Wellek and Hampel, 1999, with kind permission by Wiley-VCH.)

m

n

Rejection Prob. at π+ /(1 − π0 ) = .30

20 40 60 10 90

20 40 60 90 10

.04531 .04270 .04149 .04690 .04203

Rejection Prob. Power at π+ /(1 − π0 ) at π+ /(1 − π0 ) = .70 = .50 .04566 .04321 .03984 .04772 .04212

.17858 .45012 .72160 .15050 .15188

Generalized Mann-Whitney test for noninferiority The noninferiority problem associated with (6.48) reads H1 : π+ /(1 − π0 ) ≤ 1/2 − ε versus K1 : π+ /(1 − π0 ) > 1/2 − ε .

(6.65)

In view of the weak convergence result underlying (6.58) we can apply again the general result of § 2.3. Specifying k = 2, TN = W+ /(1 − W0 ), θ = √ π+ /(1 − π0 ), σ = ν and τˆN = νˆN / N , this implies that using the rejection region

√

  N W+ /(1 − W0 ) − 1/2 + ε νˆN > u1−α (6.66)

A distribution-free two-sample equivalence test allowing for ties

163

yields an asymptotically valid testing procedure for (6.65) which we propose to call generalized Mann-Whitney test for noninferiority. The entries in Tables 6.14a through 6.15b were obtained by simulation under the same models used before for studying the small-sample behavior of the equivalence version of the test. The major changes as compared to the two-sided case relate to power which turns out to increase by up to 40 % when the equivalence range specified by the alternative hypothesis is left unbounded on the right.

Table 6.14a Simulated rejection probabilities of the generalized Mann-Whitney test for noninferiority with data from normal distributions rounded to the nearest multiple of .25.

m

n

Rejection Prob. at π+ /(1 − π0 ) = .30

20 40 60 10

20 40 60 90

.04482 .04309 .04367 .04780

Power at π+ /(1 − π0 ) = .50 .66571 .89985 .97340 .63065

Table 6.14b Simulated rejection probabilities of the generalized Mann-Whitney test for noninferiority with data from normal distributions rounded to the nearest integer.

m

n

Rejection Prob. at π+ /(1 − π0 ) = .30

20 40 60 10

20 40 60 90

.04415 .04238 .04086 .04733

Power at π+ /(1 − π0 ) = .50 .52300 .77213 .90060 .49468

164

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.15a Simulated rejection probabilities of the generalized Mann-Whitney test for noninferiority under Lehmann alternatives to the uniform distribution on {1, 2, . . . , 6}.

m

n

Rejection Prob. at π+ /(1 − π0 ) = .30

20 40 60 10 90

20 40 60 90 10

.04431 .04205 .04363 .04844 .04356

Power at π+ /(1 − π0 ) = .50 .59250 .84452 .94805 .55951 .56142

Table 6.15b Simulated rejection probabilities of the generalized Mann-Whitney test for noninferiority under Lehmann alternatives to the uniform distribution on {1, 2, 3}.

6.5

m

n

Rejection Prob. at π+ /(1 − π0 ) = .30

20 40 60 10 90

20 40 60 90 10

.04391 .04189 .04162 .04695 .04138

Power at π+ /(1 − π0 ) = .50 .46618 .71891 .85743 .44668 .44649

Testing for dispersion equivalence of two Gaussian distributions

In routine applications of the most frequently used inferential procedures, the preliminary check on the homogeneity of variances of two Gaussian distributions as a prerequisite for the validity of the classical two-sample t-test consists of an “inverted” conventional test of the null hypothesis of equality of both variances. In other words, the usual F -test is carried out and homogeneity of variances taken for granted whenever the result turns out insignificant. In this section we derive an optimal testing procedure tailored for establishing the

Dispersion equivalence of two Gaussian distributions

165

alternative hypothesis that the variances, say σ 2 and τ 2 , of two normal distributions from which independent samples X1 , . . . , Xm and Y1 , . . . , Yn have been taken, coincide up to irrelevant discrepancies. The region of still tolerable heterogeneity between σ 2 and τ 2 is defined as a sufficiently short interval around 1 which has to cover the true value of the ratio σ 2 /τ 2 . Accordingly, the present section deals with the problem of testing H : 0 < σ 2 /τ 2 ≤ ω12 versus

or ω22 ≤ σ 2 /τ 2 < ∞

K : ω12 < σ 2 /τ 2 < ω22

(6.67)

by means of two independent samples (X1 , . . . , Xm ), (Y1 , . . . , Yn ) such that Xi ∼ N (ξ, σ 2 ) , i = 1, . . . , m ,

Yj ∼ N (η, τ 2 ) , j = 1, . . . , n .

(6.68)

Of course, the tolerances ω12 , ω22 making up the common boundary of the two hypotheses of (6.67) are assumed to be fixed positive numbers satisfying ω12 < 1 < ω22 . Obviously, (6.67) remains invariant under the same group of transformations as the corresponding one-sided testing problem treated in Lehmann and Romano (2005, p. 220, Example 6.3.4). By the results established there we know that a test for (6.67) which is uniformly most powerful among all level-α tests invariant under transformations of the form (X1 , . . . , Xm , Y1 , . . . , Yn ) → (a + bx1 , . . . , a + bxm , c + dy1 , . . . , c + dyn ) with −∞ < a, c < ∞, b, d = 0, |b| = |d|, can be obtained in the following way: In a first step, the raw data X1 , . . . , Xm , Y1 , . . . , Yn are reduced to the statistic 2 Q = SX /SY2 ≡

m n  n−1  ¯ 2/ (Xi − X) (Yj − Y¯ )2 m − 1 i=1 j=1

(6.69)

which the ordinary F -test of the null hypothesis σ 2 /τ 2 = 1 is based upon. Under the Gaussian model (6.68), the distribution of Q is an element of the scale family associated with the central F -distribution with ν1 = m − 1, ν2 = n − 1 degrees of freedom, and the scale parameter , say, is related to the variances we want to compare simply by  = σ 2 /τ 2 . Hence, the testing problem (6.67) can be rewritten as ˜ : 0 <  ≤ 1 ∨ 2 ≤  < ∞ versus K ˜ :  1 <  < 2 H

$ (6.67)

$ and is carried with k = ωk2 , k = 1, 2. If an UMP level-α test exists for (6.67) out through Q in terms of the original observations X1 , . . . , Xm , Y1 , . . . , Yn , then the result is the desired UMPI test at the same level α for (6.67). Let R◦ denote a generic random variable following a central F -distribution with (ν1 , ν2 ) degrees of freedom, and fν1 ,ν2 ( · ; ) the density function of R◦

166

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

for arbitrary  > 0. As is rigorously shown in the Appendix [see p. 370], the family (fν1 ,ν2 ( · ; ))>0 (sometimes called family of stretched F -densities in the literature – cf. Witting, 1985, p. 217) is strictly totally positive of any order and hence in particular STP3 . Furthermore, continuity of the standard F -density fν1 ,ν2 ( · ; 1) with the respective numbers of degrees of freedom obviously implies that the function (q, ) → fν1 ,ν2 (q; ) is continuous in both of its arguments. Thus, the family (fν1 ,ν2 ( · ; ))>0 of densities satisfies all conditions of Theorem A.1.5 [→ p. 371]. Consequently, the desired UMP level-α $ exists indeed and is given by the rejection region test for (6.67) 

(1) (2) ˜ C˜α; ( ,  ) < Q < C ( ,  ) . (6.70) 1 2 1 2 ν1 ,ν2 α; ν1 ,ν2 (1) The critical constants C˜α;ν1 ,ν2 (1 , 2 ), k = 1, 2, have to be determined by solving the equations

Fν1 ,ν2 (C˜2 /1 ) − Fν1 ,ν2 (C˜1 /1 ) = α = Fν1 ,ν2 (C˜2 /2 ) − Fν1 ,ν2 (C˜1 /2 ) , 0 < C˜1 < C˜2 < ∞,

(6.71)

with Fν1 ,ν2 (·) denoting the cdf of the standard central F -distribution with (ν1 , ν2 ) degrees of freedom. The algorithm for computing the optimal critical constants admits considerable simplification along the lines described on p. 44 provided the equivalence range for  has been chosen as symmetric on the log scale, and the underlying parallel-group design is balanced with respect to the sample sizes so that the distribution of Q has the same number of degrees of freedom for numerator and denominator, respectively. In fact, for ν1 = ν2 = ν, we obviously have d Q−1 = Q under  = 1, and this implies that the distribution of Q under any  > 0 coincides with that of Q−1 under −1 . Hence, for ν1 = ν2 = ν, 1 = −1 ◦ , 2 = ◦ > 1, specifying T (X) = Q, θ = log , ε = log ◦ yields a setting which satisfies both of the conditions (3.10a), (3.10b) for symmetrizing the critical region of the corresponding uniformly most powerful test. Thus, by (3.11), we may reduce (6.70) and (6.71) in the case of a balanced design and symmetric equivalence limits to

 (◦) ˜ (◦) 1/C˜α; , (6.72) ν (◦ ) < Q < Cα; ν (◦ ) and

˜ ◦ ) − Fν,ν (C˜ −1 /◦ ) = α , C˜ > 1 , Fν,ν (C/

(6.73)

respectively. (◦) Of course, in (6.72), C˜α;ν (◦ ) stands for the (unique) solution to equation (6.73). Reducibility of (6.70) to (6.72) in the special case ν1 = ν2 = ν, 1 = −1 ◦ , (k) 2 = ◦ means that we may write both for k = 1 and k = 2 : C˜α; ν,ν (−1 ,  ◦) ◦

Dispersion equivalence of two Gaussian distributions

167

(3−k) = 1/C˜α; ν,ν (−1 ◦ , ◦ ). In view of the obvious fact that for any random variable Zν1 ,ν2 following a central F -distribution with (ν1 , ν2 ) degrees of freedom, we d

have Zν1 ,ν2 = 1/Zν2 ,ν1 , this reciprocity relation generalizes to (k) ˜ (3−k) −1 −1 C˜α; ν1 ,ν2 (1 , 2 ) = 1/Cα; ν2 ,ν1 (2 , 1 ) , k = 1, 2 .

(6.74)

(◦) Table 6.16 shows the value of the critical constant C˜.05; ν (◦ ) to be used in the F -test for equivalence at level α = .05 in symmetric settings with ν1 = ν2 = ν and 2 = 1/1 = ◦ , for ν = 10(5)75 and ◦ = 1.50(.25)2.50. The corresponding values of the power attained against the alternative that the population variances under comparison are equal (↔  = 1), are given in Table 6.17.

(◦) Table 6.16 Critical constant C˜.05; ν (◦ ) of the F -test for dispersion equivalence of two Gaussian distributions at level α = 5% with common sample size m = n = ν + 1 and 2 2 symmetric equivalence range (−1 ◦ , ◦ ) for  = σ /τ , for ν = 10(5)75 and ◦ = 1.50(.25)2.50.

ν

1.50

1.75

◦ = 2.00

2.25

2.50

10 15 20 25 30 35 40 45 50 55 60 65 70 75

1.05113 1.04572 1.04362 1.04307 1.04346 1.04450 1.04605 1.04803 1.05040 1.05314 1.05623 1.05967 1.06344 1.06754

1.06165 1.06048 1.06328 1.06849 1.07566 1.08461 1.09529 1.10756 1.12122 1.13590 1.15116 1.16654 1.18165 1.19620

1.07593 1.08259 1.09561 1.11396 1.13724 1.16457 1.19424 1.22424 1.25297 1.27963 1.30399 1.32612 1.34626 1.36464

1.09448 1.11432 1.14600 1.18805 1.23644 1.28538 1.33075 1.37118 1.40685 1.43839 1.46647 1.49167 1.51445 1.53518

1.11796 1.15830 1.21867 1.29055 1.36110 1.42349 1.47701 1.52302 1.56300 1.59816 1.62939 1.65740 1.68272 1.70576

168

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.17 Power attained at  = 1 when using the critical constants shown in Table 6.16 for the F -statistic.

ν

1.50

1.75

◦ = 2.00

2.25

2.50

10 15 20 25 30 35 40 45 50 55 60 65 70 75

.06129 .06786 .07511 .08313 .09198 .10174 .11250 .12433 .13732 .15156 .16710 .18402 .20234 .22207

.07350 .08905 .10778 .13024 .15699 .18852 .22510 .26666 .31260 .36179 .41267 .46361 .51317 .56028

.08986 .12011 .15978 .21057 .27303 .34530 .42279 .49986 .57194 .63666 .69332 .74219 .78396 .81944

.11071 .16328 .23646 .33006 .43531 .53850 .62999 .70663 .76907 .81917 .85903 .89053 .91529 .93466

.13650 .22032 .33743 .47162 .59665 .69906 .77824 .83793 .88235 .91509 .93904 .95644 .96902 .97805

In contrast to (6.73) which can be solved more or less by trial and error since ˜ solving the the expression on the left-hand side is obviously increasing in C, system (6.71) to be treated in the general case requires appropriate software tools. The program fstretch to be found in the WKTSHEQ2 Source Code Package, provides for arbitrary choices of α, νk and k (k = 1, 2) both the (k) critical constants C˜α;ν1 ,ν2 (1 , 2 ) determining the UMPI critical region (6.70), and the power against  = 1 (⇔ σ 2 = τ 2 ). Basically, the algorithm used by the program is an adaptation of the iteration scheme described in general terms on pp. 42–3, to the specific case that the STP3 family which the distribution of the test statistic belongs to is generated by rescaling a standard central F -distribution with the appropriate numbers of degrees of freedom, in all possible ways. Example 6.3 In a laboratory specialized in testing basic physical and chemical properties of innovative materials for potential use in dental medicine, the existing device for taking measurements of breaking strengths of solid plastic elements had to be replaced with technically modernized equipment of the same kind. Two systems A and B, say, were taken into final consideration with A being priced markedly lower than B. In view of this, the head of the laboratory decided to take his own data in order to assess the equivalence of both devices with respect to precision of the measurements they provide. For that purpose, a

Dispersion equivalence of two Gaussian distributions

169

total of 100 sample elements made of the same material were prepared. For one half of the elements, breaking strength was to be measured by means of A, for the other by means of B, respectively. Since all sample elements could be considered identical in physical properties, it was reasonable to regard pure measurement error as the only source of variability found in both sequences of values. Out of the 50 measurements to be taken by means of A, only m = 41 came to a regular end since the remaining 9 elements broke already during insertion into the device’s clamp. By the same reason, the number of values eventually obtained by means of device B was n = 46 rather than 50. All measurement values were assumed to be realizations of normally distributed random variables with parameters (ξ, σ 2 ) [↔ device A ] and (η, τ 2 ) [↔ device B], respectively. Precision equivalence of both devices was defined through the condition .75 < σ/τ < 1.33 on the standard deviations of the underlying distributions. From the raw data [given in kiloponds], the following values of the sample means and variances were computed: ¯ = 5.0298 , S 2 = .0766 ; Device A: X X ¯ Device B: Y = 4.8901 , S 2 = .0851 . Y

Running the program fstretch, one reads from the output list that in the UMPI test at level α = .05 for σ 2 /τ 2 ≤ .752 or σ 2 /τ 2 ≥ 1.332 ver2 sus .752 < σ 2 /τ 2 < 1.332 , the statistic Q = SX /SY2 has to be checked (1) ˜ for inclusion between the critical bounds C.05;40,45 (.5625, 1.7689) = .8971, (2) C˜.05;40,45 (.5625, 1.7689) = 1.1024. Since the observed value was Q = .0766/ .0851 = .9001, the data satisfy the condition for rejecting nonequivalence. However, the power to detect even strict equality of variances was only 26.10 % under the specifications made in this example. Testing for one-sided dispersion equivalence of two Gaussian distributions In some applicitions, it will be sufficient to establish the hypothesis that the variability of the observations Xi obtained under the experimental treatment, exceeds that of the reference population from which the Yj are taken, by a practically irrelevant amount at most. Put in formal terms, the testing problem we then are interested in is given by the following pair of hypotheses: H1 : σ 2 /τ 2 ≥ 1 + ε versus K1 : σ 2 /τ 2 < 1 + ε .

(6.75)

Similar arguments as those which were used above to show that (6.70) is the critical region of a UMPI level-α test for (6.67) allow us to conclude that the UMPI level-α test for (6.75) rejects the null hypothesis of a relevantly increased variability of the Xi if it turns out that 2 /SY2 < (1 + ε)Fm−1,n−1;α SX

(6.76)

For a proper use of (6.76), it is essential to note that Fm−1,n−1;α stands for the lower 100α percentage point of a central F -distribution with m − 1, n − 1 degrees of freedom.

170

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

The critical upper bounds to the observed variance ratio which have to be used according to (6.76) are shown in Table 6.18 for the same choice of the df ’s (assumed to be common to numerator and denominator) as covered by Table 6.16, and the equivalence margin ε ranging through {.10, .25, .50, 1.0, 3.0} . Remarkably, for small sample sizes, a decision in favor of the hypothesis of nonexistence of a relevant increase in variability in the population underlying the Xi as compared to the distribution of the Yj requires that the observed ratio of variances falls below unity even when the upper limit to the theoretical variance ratio allowed by that hypothesis is as large as 2.00. For ε = .50 and ε = 1.00, the entries in Table 6.18 are the noninferiority counterparts of the critical constants appearing in Table 6.16 for the choice ◦ = 1.50 and ◦ = 2.00, respectively. In accordance with the construction of both tests, the latter are always larger than the former, but the difference vanishes asymptotically as ν increases to infinity. The power attained in the F -test determined by the critical upper bounds listed in Table 6.18 against the null alternative σ 2 = τ 2 can be read from Table 6.19. Given the sample sizes and the upper limit of the equivalence range specified under the hypothesis, the power of the test for one-sided equivalence

Table 6.18 Critical upper bound of the F -test for one-sided dispersion equivalence of two Gaussian distributions at level α = 5% with common sample size m = n = ν + 1 and equivalence range (0 , 1 + ε) for  = σ 2 /τ 2 , for ν = 10(5)75 and ε ∈ {.10, .25, .50, 1.0, 3.0}. ν

0.10

0.25

ε = 0.50

1.00

3.00

10 15 20 25 30 35 40 45 50 55 60 65 70 75

0.36935 0.45768 0.51785 0.56253 0.59754 0.62602 0.64981 0.67011 0.68772 0.70319 0.71693 0.72926 0.74040 0.75053

0.41971 0.52009 0.58847 0.63924 0.67903 0.71138 0.73842 0.76149 0.78150 0.79908 0.81470 0.82870 0.84136 0.85288

0.50365 0.62410 0.70616 0.76709 0.81483 0.85366 0.88611 0.91379 0.93780 0.95889 0.97764 0.99444 1.00963 1.02345

0.67154 0.83214 0.94155 1.02278 1.08644 1.13821 1.18148 1.21839 1.25039 1.27852 1.30351 1.32592 1.34618 1.36461

1.34308 1.66428 1.88310 2.04557 2.17288 2.27643 2.36295 2.43677 2.50079 2.55705 2.60703 2.65185 2.69235 2.72921

Dispersion equivalence of two Gaussian distributions

171

uniformly exceeds that of the test for dispersion equivalence in the two-sided sense. This increase in power, which is most marked for larger sample sizes and a tight equivalence margin, reflects the fact that under σ 2 /τ 2 = 1 + ε, the probability mass of the interval between the upper critical bounds of both tests is small as compared to that of the left-hand part of the acceptance region for the equivalence case which belongs to the rejection region of the test for noninferiority. Exploiting the results of Section 6.5 for testing for equivalence of hazard rates in the two-sample setting with exponentially distributed data There is a close formal relationship between the testing problems treated in the core part of this section, and that of testing for equivalence of the hazard rates of two exponential distributions from which independent samples are obtained. In order to make this relationship precise, let us assume that instead of following normal distributions, the observations making up the two samples under analysis satisfy Xi ∼ E(σ) , i = 1, . . . , m ,

Yj ∼ E(τ ) , j = 1, . . . , n ,

(6.77)

where, as before [recall p. 55], E(θ) symbolizes a standard exponential distribution with hazard rate θ −1 > 0. Let us further assume that the hypotheses

Table 6.19 Power of the UMPI test (6.76) at level α = .05 against σ 2 = τ 2 attained with the sample sizes and equivalence margins appearing in Table 6.18.

ν

0.10

0.25

ε = 0.50

1.00

3.00

10 15 20 25 30 35 40 45 50 55 60 65 70 75

.06594 .07071 .07485 .07862 .08212 .08543 .08859 .09162 .09455 .09740 .10018 .10289 .10555 .10816

.09355 .10852 .12216 .13501 .14730 .15919 .17077 .18208 .19318 .20409 .21484 .22543 .23589 .24622

.14735 .18571 .22172 .25612 .28922 .32116 .35202 .38184 .41064 .43843 .46524 .49107 .51594 .53985

.27023 .36327 .44710 .52223 .58906 .64806 .69978 .74483 .78386 .81749 .84633 .87097 .89193 .90970

.67512 .83274 .91719 .96022 .98134 .99142 .99612 .99827 .99924 .99967 .99986 .99994 .99997 .99999

172

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

about (σ, τ ) between which we want to decide by means of these samples, read H : 0 < σ/τ < 1 or 2 ≤ σ/τ < ∞ versus K : 1 < σ/τ < 2 ,

(6.78)

and H1 : σ/τ ≥ 1 + ε versus K1 : σ/τ < 1 + ε .

(6.79)

Another application of the general results proved in § 6.5 of the book of Lehmann and Romano (2005) shows carrying out the UMP level α-test  that   $ in terms of m−1 m Xi n−1 n Yj yields a UMPI test for (6.78) for (6.67) i=1 j=1 at the same level, provided the numbers of degrees of freedom are specified as ν1 = 2m, ν2 = 2n in (6.70) and (6.71). The group of transformations with respect to which the invariance property holds, consists this time of all homogeneous dilations of any point in (m+n)-dimensional real space IRm+n . In other words, the practical implementation of the UMPI level-α test for equivalence of two exponential distributions with respect to the hazard rates differs from that of the modified F -test for dispersion equivalence of two Gaussian distributions only with respect to the rule for computing the test statistic from the raw observations and for counting the numbers of degrees of freedom. Thus, it is in particular clear that the program fstretch also enables us to carry out an optimal two-sample equivalence test for exponentially distributed data. Likem n wise, checking the observed value of m−1 i=1 Xi /n−1 j=1 Yj against the critical upper bound appearing in (6.76) as computed with ν1 = 2m, ν2 = 2n yields an UMPI level-α test for (6.79) under the model (6.77). Since the exponential scale-parameter family of distributions is the most widely used special case of a model which satisfies the proportional hazards assumption (6.23), the UMPI tests being available for two samples from exponential distributions can be viewed as natural parametric competitors to the exact semiparametric tests based on the linear rank statistic with Savage scores as have been derived in § 6.3.

6.6 6.6.1

Equivalence tests for two binomial samples Exact Fisher type test for noninferiority with respect to the odds ratio

One of the most obvious and important examples of a setting which is definitely not accessible to the translation-based approach to one-sided equivalence testing discussed in § 2.1, is that of a two-arm trial with a binary endpoint, i.e., an outcome variable taking on only two possible values labeled

Equivalence tests for two binomial samples

173

“response” and “nonresponse” in the sequel. Without loss of statistical information, the data obtained from a trial of this type can be reduced to the frequencies of responders and nonresponders observed in both treatment arms. The latter are conveniently arranged in a table of the well-known 2 × 2 layout. In order to fix notation, a generic contingency table of that type is displayed below. Table 6.20 Contingency table for the analysis of a two-arm trial with binary outcome-variable. Response Treatment

+

−

Σ

A

X (p1 )

m−X (1 − p1 )

m (1.00)

B

Y (p2 )

n−Y (1 − p2 )

n (1.00)

Σ

S

N −S

N

By assumption, X has a binomial distribution with parameters m, p1 [shorthand notation: X ∼ B(m, p1 )] and is independent of Y ∼ B(n, p2 ). As to parametrization of the corresponding family of joint distributions of (X, Y ), we start with adopting the view supported by the arguments given in § 1.6 that the odds ratio ρ = p1 (1−p2 )/(1−p1 )p2 provides a more adequate measure of dissimilarity of the two distributions under comparison than the difference δ = p1 − p2 between the responder rates in the underlying populations. Accordingly, we are seeking for a suitable test of H1 : ρ ≤ 1 − ε versus

K1 : ρ > 1 − ε

(6.80)

with some fixed 0 < ε < 1 . An optimal solution to this problem can be derived by way of generalizing the construction behind the exact test for homogeneity of B(m, p1 ) and B(n, p2 ) usually named after R.A. Fisher (1934, § 21.02). Like the latter, the optimal [precisely: uniformly most powerful unbiased (UMPU) — see Lehmann and Romano (2005, pp. 126–127)] test for (6.80) is based on the conditional distribution of X given the realized value s ∈ {0, 1, . . . , N } of the “column total” S = X + Y . The conditional p-value p(x|s) of the number x of responders counted in treatment arm A has to be computed by means of a probability distribution which, following Harkness (1965), is usually called extended hypergeometric distribution (cf. Johnson et al., 1992, § 6.11). In

174

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

this conditional distribution, any possible value of X is an integer x such that max {0, s − n} ≤ x ≤ min{s, m}. For any such x the precise formula for the conditional p-value reads p(x|s) =

  + m   m n (1 − ε)j j s−j j=x



min{s,m}

j=max{0,s−n}



m j



n s−j

 (1 − ε)j . (6.81)

Except for trivial cases like s = 1 or x = m, this formula is tractable for manual computations only if both sample sizes are very small. On the other hand, writing a program for the extended hypergeometric distribution is hardly more than a routine exercise, and the SAS system provides an intrinsic function for the exact computation of the corresponding cdf. As is the case for most other predefined SAS functions for special distributions, probhypr has a counterpart in R. It can be found in the package BiasedUrn and called using the function name pFNCHypergeo. Fortunately, the computational tools which can be made use of when applying the UMPU test for one-sided equivalence of two binomial distributions with respect to the odds ratio, go considerably beyond the availability of an algorithm for computing significance probabilities. In fact, both the power of the test against arbitrary specific alternatives, and sample sizes required for maintaining some prespecified power, are accessible to exact computational methods as well. Naturally, both of these tasks are much more demanding conceptually no less than technically than mere computation of exact conditional p-values. An algorithm for exact power computations will be outlined first. Let us keep both sample sizes m, n fixed and denote by (p∗1 , p∗2 ) ∈ (0, 1)2 any point in the region corresponding to the alternative hypothesis K1 of (6.80). By definition, the power of the test based on (6.81) against (p∗1 , p∗2 ) equals the nonconditional rejection probability, given the true distribution of X and Y is B(m, p∗1 ) and B(n, p∗2 ), respectively. In view of this, for each possible value s = 0, 1, . . . , N of the column total S = X + Y , the critical value kα (s), say, of the test has to be determined. Adopting the convention that, conditional on {S = s}, the rejection region be of the form {X > kα (s)}, kα (s) is obviously given as the smallest integer x such that max{0, s − n} ≤ x ≤ min{s, m} and p(x + 1|s) ≤ α. Furthermore, it is essential to observe that the test holds its (rather strong) optimality property only if in each conditional distribution of X the prespecified significance level α ∈ (0, 1) is exactly attained rather than only maintained in the sense of not exceeding it. In view of the discreteness of all distributions involved, this typically requires that on the boundary of the critical region {X > kα (s)}, i.e., for X = kα (s) a randomized decision

Equivalence tests for two binomial samples

175

in favor of the alternative hypothesis is taken with probability γ(s) = [α − p(kα (s)+1 | s )]/[p(kα (s) | s )−p(kα (s)+1 | s )] . Although randomized decisions between statistical hypotheses are hardly acceptable for real applications, it seems reasonable to carry out the power calculations both for the randomized and the ordinary nonrandomized version of the test defined by accepting on the boundary of the rejection regions always H1 . In fact, given some specific alternative (p∗1 , p∗2 ) of interest, the rejection probability of the randomized version of the test is an upper bound to the power attainable in any other unbiased test of (6.80). Moreover, knowledge of the power of the exact UMPU test allows us to estimate the maximum improvement which might be achieved by deriving more sophisticated nonrandomized versions of the test [see below, § 6.6.2]. The conditional power of the randomized test is given by β(p∗1 , p∗2 |s) = 

min{s,m}

j=kα (s)+1



m j



n s−j



 j

ρ∗ + γ(s)



min{s,m}

j=max{0,s−n}



m j

m kα (s)



n s−j



n s − kα (s)

 ρ∗ kα (s) ,



(6.82)

ρ∗ j

with ρ∗ = p∗1 (1 − p∗2 )/(1 − p∗1 )p∗2 . In order to compute its nonconditional power against the arbitrarily fixed specific alternative (p∗1 , p∗2 ), we have to integrate the functions s → β(p∗1 , p∗2 |s) with respect to the distribution of S. Unfortunately, the latter is simply of binomial form again only if (p∗1 , p∗2 ) lies on the diagonal of the unit square. Whenever we select a parameter combination with p∗1 = p∗2 , the distribution of S must be computed from scratch by means of convolution. The Fortran program bi2ste1 supplied in the WKTSHEQ2 Source Code Package implements this algorithm for sample sizes m, n with m + n ≤ 2000 and arbitrary choices of α, ε, p∗1 and p∗2 . In order to guarantee high numerical accuracy of the results, it uses (like the other Fortran programs to be found at the same URL) fourfold precision in floating-point arithmetic throughout. A double-precision version is made available as a DLL which can be called from R and produces results whose numerical accuracy is still fully satisfactory for practical applications. Unavoidably, the computational effort entailed in determining sample sizes required to guarantee some prespecified level β ∗ of power is much higher. The program bi2ste2 which has been constructed for serving that purpose, is based on an iteration algorithm whose efficiency strongly depends on a sensible choice of the initial values we let the iteration

176

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

process start from. In determining the latter, the exact nonconditional power s(m, n)) where β(p∗1 , p∗2 ) is approximated by the conditional power β(p∗1 , p∗2 |˜ s˜(m, n) denotes the integer closest to the expected value mp∗1 + np∗2 of S. Example 6.4 To provide an illustration of the Fisher type exact test for one-sided equivalence of two binomial distributions with respect to the odds ratio, we will use some part of the data from a comparative phase-IV trial of two antibiotics administered to treat streptoccal pharyngitis (Scaglione, 1990). At the time the trial was launched, clarithromycin was an experimental drug of which one knew from animal experiments that its toxicity was considerably weaker than that of the classical macrolid antibiotic erythromycin used as the reference medication. Hence, with regard to therapeutic efficacy, one-sided equivalence with erythromycin could have been considered a sufficient argument for preferring the new macrolid in the treatment of streptococcal infections. Specifying ε = .50 in (6.80), we find with the data of Table 6.21 by means of the SAS function mentioned in connection with the general formula for computing the p-value in the test under consideration [ → (6.81)] that p(98|195) = 1 – probhypr(213,106,195,97,.50) =.049933 . Thus, at the 5% level of significance, the data of the trial allow the conclusion that the odds of a favorable response to clarithromycin do not fall by more than 50% as compared to those of a positive outcome after administering erythromycin. For the power of the UMPU test at level α = .05 against the observed alternative (p∗1 , p∗2 ) = (.9245, .9065) [← relative frequencies of favorable responses shown in Table 6.21], the Fortran program bi2ste1 gives the result β = .579960 ≈ 58%. Omitting randomized decisions and keeping the nominal value of α equal to

Table 6.21 Successes and failures observed when treating β hemolytic streptococcal pharyngitis in patients less than 65 years of age by means of clarithromycin (A) and erythromycin (B), respectively. Response Treatment favorable nonfavor.

Σ

A

98 (92.45%)

8 (7.55%)

106 (100.0%)

B

97 (90.65%)

10 (9.35%)

107 (100.0%)

Σ

195

18

213

Equivalence tests for two binomial samples

177

the target significance level, the power drops to .505559 ≈ 51%. Accordingly, about every second two-arm trial using a binary endpoint and sample sizes m = 106, n = 107 will lead to a rejection of the null hypothesis  ≤ .50 if the true values of the binomial parameters are 90.65% (↔ Treatment B) and 92.45% (↔ Treatment A), and the test is applied in its conservative nonrandomized version. The sample sizes required in a balanced trial to raise the power of the randomized test against the same specific alternative to 80%, are computed to be m = n = 194 by means of bi2ste2.

6.6.2

Improved nonrandomized tests for noninferiority with respect to the odds ratio

We start with exploring the possibility of reducing the conservatism entailed in incorporating the critical bounds kα (0), . . . , kα (N ) of the exact Fisher type test into the acceptance region, through replacing the target significance level α with a maximally increased nominal level α∗ . Despite the conceptual simplicity of this approach which for testing traditional one- or two-sided hypotheses about  goes back to Boschloo (1970), the computational procedure is fairly demanding and requires a suitable software tool. The computational burden is markedly reduced by the fact that the maximum of the rejection probability of each test carried out through comparing the conditional p-value (6.81) with some fixed upper bound over the null hypothesis of (6.80) is taken on at the boundary. This follows from a result stated by R¨ ohmel and Mannsmann (1999a) referring to H´ ajek and Havr´ anek (1978) as a primary source for a rigorous derivation. All values of α∗ displayed in Table 6.22 have been computed by means of another Fortran program named bi2ste3 (which is again made available in the WKTSHEQ2 Source Code Package also as a DLL accessible from R). The program can be used for determining maximally raised nominal levels for arbitrary additional combinations of α, ε, m, n as long as the sum N of both sample sizes does not exceed 2000. Actually, each α∗ given in the above table represents a nondegenerate interval of nominal levels over which the size of the test remains constant. Of course, it makes no difference if one specific point in that interval is replaced with some other value producing a test of identical size. All what really matters is maximization of the size subject to the condition that the latter must not exceed the prespecified significance level (except for differences small enough for being neglected in practice). Detailed insights about the effectiveness of Boschloo’s technique are provided by comparing the power of all three tests (exact UMPU, conservative nonrandomized version, and nonrandomized conditional test at maximally increased nominal level) against all possible null alternatives specifying p1 = p2 for arbitrary p2 ∈ (0 , 1). Figure 6.5 shows these power curves for samples of size n = 100 each, noninferiority margin ε = .50, and .05 as the target significance level. For reference responder rates p2 between 25 and 75%, the

178

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.22 Nominal significance levels and sizes of improved conditional tests for (6.80) maintaining the 5%-level, for ε = 1/3, 1/2 and m = n = 10, 25, 50, 100, 200. [ Number in ( ): size of the nonrandomized test at nominal level α = .05.] ε n α∗ 1/3 10 .08445 " 25 .09250 " 50 .07716 " 75 .06295 " 100 .06064 " 200 .05974

Size .04401 .05083 .04991 .04997 .05006 .05015

(.02118) (.02728) (.02790) (.03797) (.03915) (.04108)

1/2 10 .10343 " 25 .07445 " 50 .06543 " 75 .06810 " 100 .06468 " 200 .05992

.04429 .05064 .04999 .05053 .05040 .04970

(.02869) (.03049) (.03551) (.03584) (.03761) (.04344)

Figure 6.5 Power function of the exact UMPU test (— —) for noninferiority with respect to the odds ratio, its nonrandomized counterpart (· · · · ·), and the improved nonrandomized test at increased nominal level (- - -) [ε = .50, α = .05, m = n = 100].

Equivalence tests for two binomial samples

179

power curve of the nonrandomized conditional test at the nominal level α∗ to be used according to the results listed in Table 6.22 is practically indistinguishable from that of the exact randomized UMPU test, and the width of the interval for which this holds true, increases with increasing common sample size n. With the traditional conservative nonrandomized test, the improved nonrandomized test shares the behavior of the power function at the boundaries of the range of p2 : As p2 → 0 or 1, the power converges to zero, reflecting lack of unbiasedness of both tests which cannot be avoided unless randomized decisions between both hypotheses are admitted. The use of the objective Bayesian approach for constructing an improved nonrandomized test for one-sided equivalence of two binomial distributions with respect to various parametric functions including the odds ratio, was investigated by Wellek (2005). Generalizing the argument made explicit by Box and Tiao (1973, § 1.3.4) for the one-sample case to the binomial two-sample setting leads to defining a noninformative reference prior as the product of two beta distributions with parameters (1/2, 1/2) each. Denoting the joint posterior distribution of the basic parameters (p1 , p2 ) associated with this prior ∗ (·), it can easily be shown that we have by P(x,y) ∗ P(x,y) [p1 ≤ p1 , p2 ≤ p2 ] = Ip1 (x + 1/2, m − x + 1/2) · Ip2 (y + 1/2, n − y + 1/2) (6.83)

where I· (a, b) stands for the beta distribution function with parameters (a, b) [for arbitrary a, b > 0] and (x, y) the realized value of (X, Y ). The density corresponding to (6.83) is Γ(m + 1) x−1/2 p (1 − p1 )m−x−1/2 · Γ(x + 1/2)Γ(m − x + 1/2) 1 Γ(n + 1) y−1/2 p (1 − p2 )n−y−1/2 , 0 < p1 , p2 < 1 . (6.84) Γ(y + 1/2)Γ(n − y + 1/2) 2   ∗ (·, ·) over the set K1 = (p1 , p2 )  p1 (1 − p2 )/(p2 (1 − p1 )) > Integrating f(x,y)  1 − ε and comparing the result with the usual cutoff 1 − α [recall § 2.4] leads to the following critical inequality which the observed counts (x, y) of responders have to satisfy in order to allow rejection of the null hypothesis in the objective Bayesian test of (6.80):  1   1 − I(1−ε)p2 /((1−p2 )+(1−ε)p2 ) (x + 1/2, m − x + 1/2) · 0  Γ(n + 1) y−1/2 n−y−1/2 ·p (1 − p2 ) dp2 ≥ 1 − α . (6.85) Γ(y + 1/2)Γ(n − y + 1/2) 2 ∗ f(x,y) (p1 , p2 ) =

For evaluating the left-hand side of this inequality, we prefer to rely on Gauss type integration (see, e.g., Davis and Rabinowitz, 1975, § 2.7) with abscissas chosen as the roots of a Legendre polynomial of sufficiently high degree (c.f.

180

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Abramowitz and Stegun, 1965, Table 25.4). Even when the number of abscissas is chosen as large as 96, the respective algorithm is fast enough for allowing one to search through the whole sample space {0, 1, . . . , m} × {0, 1, . . . , n} of (X, Y ) with regard to (6.85) in reasonably short computing time. Once the rejection region of the Bayesian test has been determined, its basic frequentist properties can easily be investigated by means of exact computational methods. In particular, the size of the critical region can be calculated. If the latter turns out to exceed the target significance level α, the nominal level can (and should be) adjusted by diminishing it as far as necessary to obtain a test which is exactly valid with regard to its level of significance. As before [recall, e.g., Tab. 5.9], we denote the largest nominal significance level to be used in (6.85) for obtaining a critical region of size ≤ α, by α∗ . Once the posterior probability of K1 has been computed for each point (x, y) in the sample space of (X, Y ), α∗ can be determined in the same way as described in § 5.2.3 for the problem of testing for one-sided equivalence of binomial proportions with correlated observations. Table 6.23 is the analogue of Table 6.22 for the objective Bayesian test, except for the entries put in parentheses. The latter are all larger than 5% and give the size of the critical region of the test at uncorrected nominal level. All values shown in Table 6.23 were computed by means of another program to be found in the WKTSHEQ2 Source Code Package under the program name bi2by_ni_or. Table 6.23 Nominal significance levels for the objective Bayesian test of (6.80) maintaining the 5%-level, for the same equivalence margins and sample sizes covered by Table 6.22. [ Number in ( ): size of the test at uncorrected nominal level α = .05.] ε n α∗ 1/3 10 .03547 " 25 .03895 " 50 .03992 " 75 .04023 " 100 .04039 " 200 .04063

Size .04401 .04370 .04565 .04551 .04545 .04535

(.06206) (.06580) (.06684) (.06724) (.06744) (.06771)

1/2 10 .03978 " 25 .04123 " 50 .04245 " 75 .04258 " 100 .04264 " 200 .04273

.04028 .04927 .04559 .04833 .04776 .04705

(.07340) (.07177) (.07122) (.07104) (.07094) (.07081)

Equivalence tests for two binomial samples

181

Figure 6.6 shows the result of overlaying the power curve for the Bayesian test at nominal level α∗ = .04264 for p1 (1 − p2 )/(p2 (1 − p1 )) ≤ 1/2 vs. p1 (1 − p2 )/(p2 (1 − p1 )) > 1/2 based on two samples of size 100 each, with the dashed line of Figure 6.5. From a practical point of view, the differences between both curves are negligible so that the objective Bayesian approach proves once more to be a promising way of constructing nonrandomized tests for discrete families of distributions.

Figure 6.6 Power function of the Bayesian test (— —) for noninferiority with respect to the odds ratio as compared to that of the improved nonrandomized conditional test of Fig. 6.5 (- - -) . (From Wellek, 2005, with kind permission by Wiley-VCH.)

6.6.3

Tests for noninferiority using alternative parametrizations

Although there are large fields of application (epidemiology being perhaps the most important of them) where considerations of odds ratios play a predominant role, not a few users of equivalence testing methods prefer to look at the difference δ = p1 − p2 between both binomial parameters. Notwithstanding the logical difficulties encountered in defining equivalence in terms of this apparently more intuitive measure of treatment effect [recall p. 14], the literature contains numerous contributions dealing with the corresponding nonin-

182

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

feriority testing problem (among many others see Makuch and Simon, 1978; Blackwelder, 1982; Rodary et al., 1989; Roebruck and K¨ uhn, 1995; Chan, 1998; Chan and Zhang, 1999). Another popular option for choosing the target parameter is the relative risk p1 /p2 (see, e.g., Miettinen and Nurminen, 1985; Farrington and Manning, 1990; Blackwelder, 1993; Chan, 1998; Nam, 1998). Each of these choices leads toa different form of the indifference zone  by which the alternative hypothesis (p1 , p2 ) | 0 < p2 < p1 < 1 of the classical one-sided testing problem about the two binomial parameters is enlarged when the question of noninferiority is raised. In principle, there are infinitely many ways of defining such an indifference zone. Preferably, the definition will be based on some parametric function g(·, ·) with constant value g0 on the diagonal {(p, p)|0 < p < 1} in the unit square and g(p1 , p2 ) < g0 for p1 < p2 . In order to keep the corresponding testing problem H1g : g(p1 , p2 ) ≤ g0 − ε vs. K1g : g(p1 , p2 ) > g0 − ε

(6.86)

tractable, the real-valued transformation g(· , ·) of the parameter space (0, 1)× (0, 1) of (p1 , p2 ) into the real line will usually be chosen to be continuous and strictly increasing (decreasing) in its first (second) argument. Under the additional assumption that g is totally differentiable with g1 , g2 as its two first-order partial derivatives, it seems reasonable to base a test for (6.86) on Wald’s statistic (cf. Rao, 1973, § 6e.3) which in the present setting admits the simple explicit representation Wg (X, Y ) = g(ˆ p1 , pˆ2 ) − g0 + ε   1/2 , (g1 (ˆ p1 , pˆ2 ))2 pˆ1 (1 − pˆ1 )/m + (g2 (ˆ p1 , pˆ2 ))2 pˆ2 (1 − pˆ2 )/n

(6.87)

where pˆ1 and pˆ2 denotes as usual the relative frequency of “successes” observed under treatment A and B, respectively. In order to construct a nonconditional test of exact level α based on (6.87) one has to determine the smallest w ∈ IR such that sup(p1 ,p2 )∈H1g Pp1 ,p2 [Wg (X, Y ) > w] ≤ α and use this w as a critical constant. As long as simple explicit formulae are available for both partial derivatives of g, adopting this technique (the basic idea behind it goes back to Suissa and Shuster, 1985) is a straightforward exercise. However, even on a high-speed computer, determination of the critical constant wg (ε, α), say, of a test of that type with satisfactory numerical accuracy might be a very time-consuming process except for very small sample sizes m and n. The reason is that it is in general not sufficient to restrict maximization of the rejection probability Pp1 ,p2 [Wg (X, Y ) > wg (ε; α)] to the boundary ∂H1g = {(p1 , p2 ) | g(p1 , p2 ) = g0 − ε} of H1g . In fact, except for parametrizations g satisfying rather strong additional conditions (as derived by R¨ ohmel and Mannsmann, 1999a), the full size sup(p1 ,p2 )∈H1g Pp1 ,p2 [Wg (X, Y ) > wg (ε; α)] of the rejection region under consideration will be greater than the value found

Equivalence tests for two binomial samples

183

by maximization over ∂H1g only. For some of the exact nonconditional tests for one-sided equivalence of two binomial distributions studied in the literature, it is only evident that they provide control over the significance level in this weaker sense whereas proving validity over the null hypothesis as a whole is a technically challenging problem. Perhaps the best known example of that kind is the test proposed by Chan for p1 −p2 ≤ −ε vs. p1 −p2 > −ε (see Chan, 1998; Chan and Zhang, 1999; Chan, 2003) whose validity was initially questioned by R¨ ohmel and Mannsmann (1999b) and later on proved by R¨ ohmel (2005). The objective Bayesian approach which we used in the previous subsection as an alternative method for constructing powerful nonrandomized tests for noninferiority with respect to the odds ratio, can likewise be adopted for any other parametric function g(· , ·) such that p1 → g(p1 , p2 ) has a well defined inverse g −1 (· ; p2 ), say, for arbitrarily fixed p2 ∈ (0, 1). Under this condition, the critical inequality of the objective Bayesian test for (6.86) can be written  1   1 − Imax{0,g−1 (g0 −ε;p2 )} (x + 1/2, m − x + 1/2) · 0  Γ(n + 1) y−1/2 n−y−1/2 ·p (1 − p2 ) dp2 ≥ 1 − α , (6.88) Γ(y + 1/2)Γ(n − y + 1/2) 2 generalizing (6.85) in the obvious way. In the remainder of this subsection, we confine ourselves to apply both approaches to the special case that noninferiority shall be established in terms of the nonscaled difference g(p1 , p2 ) = p1 − p2 ≡ δ of proportions. With this choice of g(· , ·) we obviously have g1 = g2 = 1, and (6.87) simplifies to X/m − Y /n + ε Wδ (X, Y ) =  1/2 . (6.89) (1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n) In view of the asymptotic normality of this test statistic, a convenient way of determining the corresponding lower critical bound wδ (ε; α) is through iteration level α∗ required in a test with rejection   over the nominal significance region Wδ (X, Y ) > u1−α∗ . Denoting the probability of this region under ∗ any parameter constellation (p1 , p2 ) ∈ (0, 1)2 by Qε;α m,n (p1 , p2 ), we can write # " n   ε;α∗ b(x; m, p1 ) b(y; n, p2 ) , (6.90) Qm,n (p1 , p2 ) = y=0

∗

ε;α x∈Xm (y)

 ε;α∗ (y) stands x, y˜) | 0 ≤ x ˜ ≤ m, 0 ≤ y˜ ≤ where Xm  for the y-section of (˜ x, y˜) > u1−α∗ . n, Wδ (˜ As to the form of the latter, we are in a similar situation as we encountered in § 5.2.2 [recall the footnote on p. 81]: In contrast to the statistic obtained from Wδ (X, Y ) by replacing the unrestricted variance estimator with the pooled estimator, Wδ (X, Y ) itself does not satisfy the convexity condition

184

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

usually (see, e.g., R¨ ohmel and Mannsmann, 1999a) named after Barnard from ε;α∗ (y) are left-bounded intervals in the which it could be inferred that the Xm sample space of X . However, in extensive numerical studies based on (6.90), we never found an exception from the rule that for each y = 0, 1, . . . , n, there is some integer ky ∈ {0, 1, . . . , n} such that ∗

ε;α Xm (y) = {x ∈ IN0 | ky ≤ x ≤ m} .

(6.91)

From the basic properties of the binomial distribution, it follows that whenever (6.91) holds true,  the maximum of the rejection probability of the test with critical region Wδ (X, Y ) > u1−α∗ over the null hypothesis H1 : p1 −p2 ≤ −ε is taken on the boundary of the latter whichallows one to restrict numerical  search to the line (p2 − ε, p2 ) | ε < p2 < 1 in determining the size of the test. In other words, taking the validity of (6.91) for granted, we can write ∗

∗

ε;α (Wδ ) = sup Qε;α SIZEm,n m,n (p2 − ε, p2 )

(6.92)

ε −ε maintaining the 5%-level, for the same values of the noninferiority margin ε and sample sizes covered by Table 6.24. [ Number in ( ): size of the test at uncorrected nominal level α = .05.] ε

6.6.4

n

α∗

∗

ε;α SIZEm,n (BAYδ )

.10 " " " " " " "

25 50 75 100 125 150 175 200

.01400 .04433 .02214 .02920 .03307 .03492 .03550 .03525

.02377 .04997 .03027 .03593 .03782 .04054 .04355 .04238

(.07179) (.11132) (.05700) (.06185) (.06310) (.06236) (.06051) (.06280)

.15 " " " " " " "

25 50 75 100 125 150 175 200

.03154 .03233 .02815 .03135 .03182 .03628 .04304 .04292

.03906 .04605 .03311 .04028 .03891 .04900 .04944 .04810

(.09307) (.06538) (.06650) (.06369) (.05923) (.05913) (.07171) (.06529)

Exact test for two-sided equivalence with respect to the odds ratio

In the remaining parts of this section, we keep supposing that we are given a 2×2 contingency table of the form shown in Table 6.20 [→ p. 173]. But now we are interested in establishing two-sided equivalence of the underlying binomial distributions rather than noninferiority of B(m, p1 ) to B(n, p2 ). Measuring, as in § 6.6.1, the distance between both distributions in terms of | − 1| with  = p1 (1 − p2 )/(1 − p1 )p2 , and p1 and p2 as the probability of a favorable response to treatment A and B, respectively, the problem of testing for equivalence in the strict sense reads H : 0 <  ≤ 1 or 2 ≤  < ∞ versus

K :  1 <  < 2 .

(6.94)

Of course, the limits 1 , 2 of the equivalence range for the odds ratio  have to be specified as positive real numbers satisfying 1 < 1 < 2 . If we keep denoting the number of experimental units responding to treatment A and B

Equivalence tests for two binomial samples

187

Figure 6.7 Power function of the asymptotic test for one-sided δ-equivalence —] as compared with that of the at corrected nominal level α0 = .01801 [— objective Bayesian test for of the same hypothesis [- - -] . (From Wellek, 2005, with kind permission by Wiley-VCH.)

by X and Y , respectively, then independence of X and Y implies that the joint probability mass function of both variables can be written    m n P [X = x, Y = y] = (1 − p1 )m (1 − p2 )n · x y 

 p (1 − p )  1 2 + (x + y) log[p2 /(1 − p2 )] , exp x log (1 − p1 )p2 (x, y) ∈ {0, . . . , m} × {0, . . . , n} . (6.95) Letting θ = log  = log[p1 (1−p2 )/(1−p1 )p2 ] , ϑ = log[p2 /(1−p2 )] , it is easily verified that we have 1 − p1 = (1 + eθ+ϑ )−1 , 1 − p2 = (1 + eϑ )−1 and hence . If the joint density of (X, Y) isde- (1−p1 )m (1−p2 )n = (1+eθ+ϑ )−m (1+eϑ )−n n fined with respect to the measure B (2) → (x,y)∈B (2) ∩{0,...,m}×{0,...,n} m x y on the σ-algebra of Borel sets in IR2 rather than the counting measure of the set {0, 1, . . . , m} × {0, 1, . . . , n}, we thus obtain the expression pθ,ϑ (x, y) = (1 + eθ+ϑ )−m (1 + eϑ )−n exp{xθ + (x + y)ϑ} .

(6.96)

188

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Obviously, (6.96) defines the generic element of a family of densities of the form considered in § A.2 [see in particular equation (A.16)] with k = 1, T = X, S = X + Y . In other words, the densities (6.96) make up a two-parameter exponential family in θ = log , ϑ = log[p2 /(1 − p2 )] with X and S = X + Y as a statistic sufficient for θ and ϑ, respectively. Hence, Theorem A.2.2 [ → p. 375] does apply implying that a uniformly most powerful unbiased (UMPU) level-α test for (6.94) is obtained by proceeding for each of the possible values s ∈ {0, 1, . . . , m + n} of the first column total S according to the following decision rule: ⎧ m,n Rejection of H if C1; α (s; 1 , 2 ) < ⎪ ⎪ m,n ⎪ ⎪ X < C2; ⎪ α (s; 1 , 2 ) ⎪ ⎪ ⎪ m,n m,n ⎨ Reject with prob. γ 1; α (s; 1 , 2 ) if X = C1; α (s; 1 , 2 ) . (6.97) m,n m,n ⎪ Reject with prob. γ2; (s; 1 , 2 ) if X = C2; (s; 1 , 2 ) ⎪ α α ⎪ ⎪ ⎪ m,n ⎪ Acceptance of H if X < C1; ⎪ α (s; 1 , 2 ) ⎪ ⎩ m,n or X > C2; α (s; 1 , 2 ) m,n m,n The critical constants Cν; α (s; 1 , 2 ), γν; α (s; 1 , 2 ) , ν = 1, 2 , have to be determined for each s by solving the equations C 2 −1

hm,n (x; 1 ) s

+

2 

γν hm,n (Cν ; 1 ) s

=α=

ν=1

x=C1 +1

+

2 

γν hm,n (Cν ; 2 ) , s

C 2 −1

hm,n (x; 2 ) s

x=C1 +1

0 ≤ C1 ≤ C2 ≤ m , 0 ≤ γ1 , γ2 < 1 .

(6.98)

ν=1

For any value  ∈ IR+ taken on by the true population odds ratio p1 (1 − p2 )/(1 − p1 )p2 , hm,n ( · ; ) stands for the probability mass function of the diss tribution of X conditional on the event {S = s}. As was already stated in § 6.6.1, this conditional distribution is a so-called extended hypergeometric distribution given by (x; ) hm,n s

    m n = x x s−x



min{s,m}

j=max{0,s−n}

   m n j , j s−j

max{0, s − n} ≤ x ≤ min{s, m} .

(6.99)

In the balanced case of equal sample sizes m = n, it is easy to verify that the conditional distributions of X satisfy the symmetry relation −1 ) = hn,n hn,n s (s − x;  s (x; ), max{0, s − n} ≤ x ≤ min{s, n}, s = 0, 1, . . . , 2n .

(6.100)

In view of (6.100) and Corollary A.1.7 [→ p. 372–3], in the case of a balanced design with common size n of both samples and a symmetric equivalence

Equivalence tests for two binomial samples

189

range, say (−1 ◦ , ◦ ) for the population odds ratio , the decision rule (6.97) defining a UMPU level-α test for (6.94) can be simplified to ⎧ if |X − s/2| < C(n, s; ◦ , α) ⎨ Rejection of H Reject with prob. γ(n, s; ◦ , α) if |X − s/2| = C(n, s; ◦ , α) . (6.101) ⎩ Acceptance of H if |X − s/2| > C(n, s; ◦ , α) Of the two critical constants appearing in (6.101), the first one has to be determined as    ∗ C(n, s; ◦ , α) = C = max C  s/2 − C, s/2 + C ∈ IN◦ , s/2 − C ≥ 



s/2+C−1

max{0, s − n} , s/2 + C ≤ min{s, n},

hn,n s (x; ◦ )

≤ α . (6.102)

x=s/2−C+1

As soon as the correct value of the critical upper bound C ∗ ≡ C(n, s; ◦ , α) to |X − s/2| has been found, the randomization probability γ(n, s; ◦ , α) can be computed by means of the explicit formula  γ(n, s; ◦ , α) = α −

s/2+C ∗ −1



hn,n s (x; ◦ )



x=s/2−C ∗ +1



∗ n,n ∗ hn,n . s (s/2 − C ; ◦ ) + hs (s/2 + C ; ◦ )

(6.103)

Furthermore, the condition for sure rejection of the null hypothesis given in (6.101) can alternatively be expressed in terms of a conditional p-value to be computed as x ˜s  pn;◦ (x|s) = hn,n (6.104) s (j; ◦ ) j=s−˜ xs

hn,n s (x; ◦ )

is obtained by specializing (6.99) in the obvious way, and where the upper summation limit is given by x ˜s = max{x, s − x} .

(6.105)

In fact, it is readily verified that the observed value x of X satisfies |x−s/2| < C(n, s; ◦ , α) if and only if the conditional p-value (6.104) does not exceed α. The system (6.98) of equations which determines the critical constants of the exact Fisher type equivalence test in the general case, is of the same form as that which had to be treated in a previous chapter [see p. 60, (4.19)] in the context of the one-sample binomial test for equivalence. Correspondingly, the algorithm proposed for solving the latter can be adopted for handling (6.98) with comparatively few modifications. Of course, the most conspicu(x; ν ) have to replace the binomial ous change is that the functions x → hm,n s probability mass functions x → b(x; n, pν ) (ν = 1, 2). Since the equivalence

190

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

range for  was assumed to be an interval covering unity, we can expect that an interval (C1 , C2 ) satisfying (6.98) when combined with a suitable (γ1 , γ2 ) ∈ [0, 1)2 , contains that point x◦ ∈ [max{0, s − n}, min{s, m}] which, as an entry into a contingency table with first column total s, yields an observed odds ratio exactly equal to 1. Elementary algebra shows that this number is x◦ = ms/(m + n). For the purpose of searching iteratively for the solution of (6.98), C1◦ = [[x◦ ]] + 5 proved to be a suitable choice of an initial value of the left-hand critical bound to X in numerous trials of the algorithm. Once more, both SAS and R allow fairly easy implementation of such a numerical procedure. The reason is that in both systems predefined functions for the cdf of any extended hypergeometric distribution with probability mass function given by (6.99) are available [recall p. 188]. The source codes of a program computing for any s ∈ {0, 1, . . . , m + n} the critical constants of the optimal conditional test for equivalence of B(m, p1 ) and B(n, p2 ) with respect to the odds ratio, can be found in the WKTSHEQ2 Source Code Package under the main file name bi2st. The computational issues to be tackled in connection with power analysis and sample size determination for the exact Fisher type test for equivalence are basically the same as those we had to consider in § 6.6.1 in connection with the noninferiority version of the procedure. The major modification to be effected in the two-sided case is that in the numerator of the expression for the conditional power [→ p. 175, (6.82)], {j ≥ kα (s) + 1} has to be replaced m,n m,n with the summation region {C1; α (s; 1 , 2 ) + 1 ≤ j ≤ C2; α (s; 1 , 2 ) − 1} bounded on both sides, and the properly weighted probability of the single point kα (s) with a weighted sum of the probabilities of the two boundary points of the s-section of the critical region. As before, the nonconditional power is obtained by integrating the function s → β(p∗1 , p∗2 |s) assigning to each s ∈ {0, 1, . . . , m + n} the conditional power given {S = s}, with respect to the distribution of S = X + Y under the specific alternative (p∗1 , p∗2 ) ∈ (0, 1)2 of interest. In view of the large number of steps involving computation of extended hypergeometric and binomial probabilities, respectively, the results obtained by means of this algorithm will be numerically reliable only if the precision of each intermediate result is kept as high as possible. This is again a reason for having recourse to maximum floating-point precision being available within the programming environment used for its implementation. Primarily, the programs provided in the WKTSHEQ2 Source Code Package for computing power and minimally required sample sizes for the Fisher type test for equivalence of two binomial distributions were coded in Fortran using real*16 floating-point arithmetic. The procedure named bi2aeq1 performs the   the power againstany specific  task of computing  alternative (p∗1 , p∗2 ) ∈ (p1 , p2 ) ∈ (0, 1)2  p1 (1 − p2 )/(1 − p1 )p2 ∈ (1 , 2 ) . It accommodates total sample sizes N = m + n of up to 2000. The other program, bi2aeq2, determines for a given specific alternative (p∗1 , p∗2 ) and prespecified power β ∗ to be attained against it, the smallest sample sizes (m∗ , n∗ ) required to guarantee that the rejection probability of the test does not fall short of

Equivalence tests for two binomial samples

191

β ∗ , subject to the side condition that the ratio of both sample sizes equals some arbitrarily fixed positive value λ. Except for the iterative determination of the critical constants to be used at each possible value of the conditioning statistic S and the above-mentioned modifications referring to the computation of the conditional power, the Fortran program bi2aeq1 and bi2aeq2 is a direct analogue of the corresponding program bi2ste1 and bi2ste2 for the exact Fisher type test for noninferiority, respectively. As in the case of testing for one-sided equivalence, of both programs, a double-precision version has been made available as a DLL which can be called from R. Again, diminishing the numerical precision of the individual arithmetic operations affects the final results only at decimal places being largely irrelevant for practical applications. Example 6.5 In a test preliminary to the final confirmatory assessment of the data obtained from a controlled comparative multicenter trial of the calcium blocking agent verapamil and a classical diuretic drug with respect to antihypertensive efficacy, it was the aim to rule out the possibility that patients with and without some previous antihypertensive treatment differ to a relevant extent in the probability of a favorable response to the actual medication. Table 6.26 shows the frequencies of responders and nonresponders observed in both strata of the study population during an 8 weeks’ titration period. The equivalence range for the odds ratio  characterizing the underlying (sub–)populations was set to (1 , 2 ) = ( .6667, 1.5000), following the tentative guidelines of § 1.7 for specifying the tolerances in formulating equivalence hypotheses about target parameters and functionals frequently studied in practice. The significance level was chosen as usual specifying α = .05. Table 6.26 Responder rates observed in the VERDI trial (Holzgreve et al., 1989) during an 8 weeks’ titration period in patients with and without previous antihypertensive treatment. Previous treatment with antihypertensive drugs

Response to trial medication + –

Σ

yes

108 (48.00%)

117 (52.00%)

225 (100.0%)

no

63 (52.94%)

56 (47.06%)

119 (100.0%)

Σ

171

173

344

192

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

With m = 225, n = 119, s = 171 and the specified values of the ν , the program bi2st outputs the critical interval (110, 113) and the probabilities .0214, .6322 of a randomized decision in favor of equivalence to be taken if it happens that X = 110 and X = 113, respectively. Since the observed value of X remained below the left-hand limit of the critical interval, the data shown in Table 6.26 do not allow to reject the null hypothesis that there are relevant differences between patients with and without previous antihypertensive treatment with respect to the probability of a positive response to the study medication. Running program bi2aeq1, the power of the UMPU test against the specific alternative that the true responder rates p1 and p2 coincide with the observed proportions .4800 and .5294, is computed to be as small as 16.19%. If the nonrandomized conservative version of the test is used this value drops even to 10.70%. A selection of results on sample sizes required in the exact Fisher type test for equivalence to maintain some prespecified power against fixed alternatives on the diagonal of the unit square are shown in Table 6.27. All entries into the rightmost columns have been computed by means of program bi2aeq2. One interesting fact becoming obvious from these results is that the efficiency of the test decreases with the distance of the common value of p1 and p2 from the center of the unit interval. Furthermore, the order of magnitude of the sample sizes appearing in the lower half of the table corroborates the view that the requirement 2/3 <  < 3/2 corresponds to a rather strict criterion

Table 6.27 Sample sizes required in the exact Fisher type test for equivalence at level α = .05 to maintain a prespecified power against selected alternatives p1 = p2 = p∗ on the diagonal of the unit square, for the balanced [↔ λ = m/n = 1] and various unbalanced designs [↔ λ > 1]. (1 , 2 )

p∗

POW

λ

m

n

N =m+n

(.4286, 2.3333)† " " " " (.6667, 1.5000) " " " "

.50 .40 .30 .20 .10 .50 .50 .50 .50 .50

.80 " " " " .60 " " " "

1.00 1.00 1.00 1.00 1.00 1.00 1.25 1.50 2.00 3.00

98 102 117 156 281 302 340 377 454 606

98 102 117 156 281 302 272 251 227 202

196 204 234 312 562 604 612 628 681 808

†

≈ (3/7, 7/3) – cf. pp. 16-7

Equivalence tests for two binomial samples

193

of equivalence of two binomial distributions, as suggested in § 1.7 [recall Table 1.1 (iii)]. Finally, it is worth noticing that keeping both the equivalence range (1 , 2 ) and the specific alternative of interest fixed, the total sample size N = m + n increases with the ratio λ of the larger over the smaller of both group sizes.

6.6.5

An improved nonrandomized version of the UMPU test for two-sided equivalence

Nonrandomized versions of UMPU tests for discrete families of distributions obtained by incorporating the whole boundary of the rejection into the acceptance region, are notoriously conservative except for unusually large sample sizes. This is in particular true for the exact Fisher type test for equivalence in the strict, i.e., two-sided sense of two binomial distributions as presented in the previous subsection. Fortunately, it will turn out that the conceptually very simple trick of raising the nominal conditional significance level by the maximum allowable amount, which leads to considerable improvements to the nonrandomized Fisher type test for the noninferiority setting [recall § 6.6.2], works in the present context as well. Apart from showing also the power of both the conventional and the improved nonrandomized version of the UMPU test, Table 6.28 is the direct two-sided analogue of Table 6.22 of § 6.6.2. It refers likewise to balanced designs with common sample sizes ranging over the whole spectrum of what is realistic for practical applications. The upper and the lower half of the table refers to what has been proposed in § 1.7 as a strict and liberal choice of a symmetric equivalence range for the odds ratio, respectively. All values appearing in the third column of Table 6.28 as a maximally raised nominal significance level α∗ have been computed by means of another Fortran program named bi2aeq3 which is the analogue of bi2ste3 for equivalence testing in the strict sense. The program enables its user to find maximally increased nominal levels for arbitrary combinations of values of α, ◦ , m and n, provided the total sample size N = m + n does not exceed 2000. As to the numerical accuracy attained in computing a maximally raised nominal level for any given configuration (α, ◦ , m, n), the situation is once more strictly analogous to that described in connection with the noninferiority case: there is always a whole interval of nominal levels over which the exact size of the nonrandomized conditional test remains constant so that it suffices to ensure that the resulting value is contained in that interval. Furthermore, a tolerance of 0.1% was specified throughout for a practically negligible anticonservatism of the improved nonrandomized test.

194

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.28 Nominal significance level, size and power against p1 = p2 = 1/2 of an improved nonrandomized Fisher type test for the problem (6.94) maintaining significance level α = 5%, for 1 = −1 2 = ◦ = 1.5000, 2.3333, and m = n = 25(25)100(50)250. ◦ , [Numbers in parentheses refer to the ordinary nonrandomized version of the exact Fisher type test using nominal level .05.] ◦

n

α∗

Size

1.5000 " " " " " " 2.3333 " " " " " "

25 50 75 100 150 200 250 25 50 75 100 150 200 250

.17300 .12000 .12000 .08188 .06500 .05875 .05891 .14989 .07500 .06875 .06250 .05875 .06000 .05812

.03717 .05017 .04582 .05052 .05079 .05041 .05005 .04893 .05054 .05060 .05020 .04941 .04972 .04968

Power against p1 = p2 = 1/2 (.00000) (.00000) (.00000) (.03915) (.04059) (.04153) (.04199) (.00000) (.02910) (.03576) (.03834) (.04064) (.04104) (.04352)

.04804 .07959 .06504 .13916 .22695 .34462 .47284 .13629 .38233 .63079 .81869 .95357 .98929 .99771

(.00000) (.00000) (.00000) (.05635) (.13747) (.27361) (.43801) (.00000) (.23560) (.53742) (.77034) (.94298) (.98580) (.99721)

Going through the fourth and fifth column of Table 6.28, one finds that even with sample sizes as large as 250 there remains a substantial margin for adjusting the size of the nonrandomized conditional test. In many cases, the gain in power achieved by raising the nominal over the target significance level exhibits an order of magnitude in view of which one has clearly to discourage from using the nonrandomized test in its conventional form for real applications.

6.6.6

Tests for two-sided equivalence with respect to the difference of success probabilities

In this subsection we consider the two-sided version of the testing problem upon which we focussed in § 6.6.3. In other words, we aim at establishing a satisfactory solution for the problem of testing H : −1 < δ ≤ −δ1 or δ2 ≤ δ < 1 versus K : −δ1 < δ < δ2 ,

(6.106)

where δ = p1 − p2 ,

(6.107)

Equivalence tests for two binomial samples

195

and δ1 , δ2 denote fixed numbers to be chosen from the open unit interval. In contrast to the odds ratio  = p1 (1 − p2 )/(1 − p1 )p2 , the simple difference δ of both success probabilities is not a mathematically natural parameter for the underlying class of products of two binomial distributions. Accordingly, we will not be able to exploit one of the major theoretical results presented in the Appendix for deriving an optimal solution to (6.106). Instead, we will adopt the basic idea behind the approach taken in § 5.2.2 to constructing a valid level-α test for equivalence with respect to δ of two binomial distributions from which a sample of paired observations is available. In the case of two independent samples we have now to deal with, a test statistic for (6.106) which satisfies the conditions of Theorem A.3.4 is given by |TN − (δ2 − δ1 )/2| = τˆN |(X/m − Y /n) − (δ2 − δ1 )/2| , [(1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n)]1/2

(6.108)

where the subscript N to which all limiting operations apply, denotes this time the total sample size m + n. Accordingly, the two-sample analogue of the decision rule (5.9) obtained in the McNemar setting, reads as follows: Reject nonequivalence if and only if |(X/m − Y /n) − (δ2 − δ1 )/2| < [(1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n)]1/2   (δ1 + δ2 )/2 Cα . [(1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n)]1/2

(6.109)

The function Cα (·) to be evaluated for determining the critical upper bound to the test statistic is defined as before so that the expression on the righthand side of (6.109) stands for the square root of the α-quantile of a χ2 distribution with 1 degree of freedom and (random) noncentrality  parameter (δ1 + δ2 )2 / 4 [(1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n)] . By Theorem A.3.4, we know that the testing procedure corresponding to (6.109) has asymptotic significance level it can be taken for granted that the √ √ α whenever to a positive variance of the statistic N TN = N (X/m − Y /n) converges √ limit as N → ∞. Since we obviously have that V ar[ N (X/m − Y /n)] = (N/m)p1 (1 − p1 ) + (N/n)p2 (1 − p2 ), this simply requires that the relative size m/N of sample 1 converges to some nondegenerate limit and at least one of the two binomial distributions under comparison be nondegenerate. In other words, (6.109) defines a test of asymptotic level α provided there exists some λ ∈ (0, 1) such that m/N → λ as N → ∞, and at least one of the primary parameters p1 and p2 is an interior point of the unit interval. A straightforward approach to transforming the asymptotic testing procedure (6.109) into an exactly valid test for the problem (6.106) starts from establishing an algorithm for the exact computation of its rejection probability under any fixed parameter constellation (p1 , p2 ). For brevity, let us

196

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

write 1 ,δ2 (p1 , p2 ) = Qδα;m,n

P(p1 ,p2 )

|X/m − Y /n − (δ2 − δ1 )/2| [(1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n)]1/2 (δ + δ )/2

< Cα

1

2

! . (6.110)

[(1/m)(X/m)(1 − X/m) + (1/n)(Y /n)(1 − Y /n)]1/2

1 ,δ2 (p1 , p2 ) which proves particularly convenient for A representation of Qδα;m,n computational purposes reads # " m   δ1 ,δ2 Qα;m,n (p1 , p2 ) = b(y; n, p2 ) · b(x; m, p1 ) , (6.111)

x=0

δ ,δ

1 2 (x) y∈Uα,n

where



 δ1 ,δ2 Uα;m,n (x) = y ∈ IN◦  y ≤ n, |x/m − y/n − (δ2 − δ1 )/2| [(1/m)(x/m)(1 − x/m) + (1/n)(y/n)(1 − y/n)]1/2

 (δ1 + δ2 )/2 . (6.112) < Cα [(1/m)(x/m)(1 − x/m) + (1/n)(y/n)(1 − y/n)]1/2 δ1 ,δ2 With regard to the form of the sets Uα;m,n (x) over which the inner sum has to be extended in computing exact rejection probabilities of the asymptotic test, the situation is quite the same as found in the analogous setting with δ1 ,δ2 (x) turn out to be paired observations: By explicit construction, the Uα;m,n intervals in the sample space of the second of the binomial variables involved. Since there is no argument in sight which allows to assert that this observation reflects a general mathematical fact, both computer programs bi2diffac, 1 ,δ2 (p1 , p2 ) bi2dipow involving evaluations of exact rejection probabilities Qδα;m,n by means of (6.111) are organized as their paired-observations counterparts [cf. the footnote on p. 81].

Now, we are ready to determine once more the largest nominal significance level α∗ which has to be substituted for α in order to ensure that the exact size of the asymptotic testing procedure does not exceed the target significance level. For the sake of avoiding excessively long execution times of the program bi2diffac provided for accomplishing that task, SIZEαB (δ1 , δ2 ; m, n) = max

sup 0 C, respectively. For convenience, let us arrange the pooled set of all pairs of time variables underlying a two-sample setting with survival data subject to random right-censoring, into a single sequence ((Tk , Ck ))1≤k≤N . Then, the data effectively available from the first sample are given by the m random pairs    (1)  ˜ (1) , . . . , Xm , D ˜m with X1 , D 1  1 Ti ≤ C i (1) ˜ for Xi = min(Ti , Ci ) , Di = , 1 ≤ i ≤ m. (6.117a) 0 Ti > C i Likewise, the censoring process reduces the second sample from (Tm+1 , Cm+1 ),     ˜ (2) , . . . , Yn , D ˜ n(2) where . . ., (Tm+n , Cm+n ) to Y1 , D 1 Yj = min(Tm+j , Cm+j ), ˜ (2) = D j



Tm+j ≤ Cm+j 1 for , 0 Tm+j > Cm+j

1 ≤ j ≤ n . (6.117b)

Log-rank test for equivalence of two survivor functions

203

Of course, the distributions whose equivalence we want to establish are those of the true survival times T1 , . . . , Tm (↔ Sample 1) and Tm+1 , . . . , Tm+n (↔ Sample 2). The survivor functions characterizing these distributions will be denoted by S1 (·) and S2 (·) so that we have by definition: S1 (t) = P [Ti ≥ t] ,

t ≥ 0,

S2 (t) = P [Tm+j ≥ t] ,

i ∈ {1, . . . , m} ;

t ≥ 0,

j ∈ {1, . . . , n} .

(6.118a) (6.118b)

Both S1 (·) and S2 (·) are assumed to be absolutely continuous and to satisfy an ordinary proportional hazards model (see, e.g., Kalbfleisch and Prentice, 2002, § 2.3.2). With no loss of generality, we treat S2 (·) as the baseline survivor function underlying the model. With this convention, the proportional hazards assumption is equivalent to the condition [cf. (6.23)] S1 (t) = [S2 (t)]θ for all t ≥ 0 and some θ > 0 .

(6.119)

A nonparametric test for one-sided equivalence of survivor functions has been developed by Freitag (2005) (see also Freitag et al., 2006). Although this test does not have to rely on the proportional hazards assumption, it has other limitations. In particular, it requires restriction to some fixed time window to be specified a priori, and the critical bounds to the test statistic have to be determined by means of bootstrap resampling methods.

6.7.1

Rationale of the log-rank test for equivalence in the two-sided sense

Generally, it seems intuitively plausible to define equivalent survivor functions by requiring the uniform distance between S1 (·) and S2 (·) to be sufficiently small. In other words, we can take equivalence of S1 (·) and S2 (·) for granted if we have ||S1 − S2 || ≡ sup |S1 (t) − S2 (t)| < δ for some δ > 0.

(6.120)

t>0

In view of the continuity of S2 (·), (6.119) implies that ||S1 − S2 || = sup |u − uθ | ,

(6.121)

0 0, || S2θ − S2 || < δ . Table 6.30 Numerical correspondence between equivalence limits referring to ||S1 − S2 || and the parameter θ relating both survivor functions under the proportional hazards model. δ ε log(1 + ε)

.05 .075 .1457 .2266 .1360 .2042

.10 .15 .20 .3135 .5077 .7341 .2727 .4106 .5505

.25 1.0000 .6931

Since both survivor functions under comparison have been assumed absolutely continuous, each of the true survival times Tk (k = 1, . . . , N ) has a well-defined hazard function, say λ1 (·) (for k = 1, . . . , m) and λ2 (·) (for k = m + 1, . . . , m + n), where λν (t) =

 d − log Sν (t) , t ≥ 0 , ν ∈ {1, 2} . dt

(6.125)

By the basic model assumption (6.119), these functions admit the representation λν (t) = λ2 (t)ezk β , t ≥ 0 , ν ∈ {1, 2} , k ∈ {1, . . . , N } 

where β = log θ and zk =

1 0

for

1≤k≤m . m+1≤k ≤N

(6.126)

(6.127)

Log-rank test for equivalence of two survivor functions

205

Figure 6.10 Region of survival curves being equivalent to a given baseline survivor function (solid line) with respect to uniform vertical distance in the proportional hazards model.

After these preparations, we are ready to construct an asymptotic testing procedure for H : ||S1 − S2 || ≥ δ versus K : ||S1 − S2 || < δ .

(6.128)

As explained above, under the proportional hazards assumption (6.119), the problem of testing (6.128) is the same as that of testing ˜ : |β| ≥ log(1 + ε) versus K ˜ : |β| < log(1 + ε) H

 (6.128)

provided β is the real-valued regression coefficient appearing in (6.126) and ε is determined as the unique solution to equation (6.124). It is easy to verify that in the special case of a proportional hazards model given by (6.126) and (6.127), the partial log-likelihood for β simplifies to (1)

log L(β) = d˜· β −

q˜  q=1

where

log(rq1 eβ + rq2 )

(6.129)

206

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES rqν = number of items at risk in the νth sample at the qth smallest failure time t(q) (ν = 1, 2 ; q = 1, . . . , q˜) , (1) d˜· =

m 

(1) d˜i = total number of failures in Sample 1.

(6.130)

i=1

Correspondingly, the maximum partial likelihood estimator βˆ is found by solving the equation q˜  q=1

rq1 eβ (1) = d˜· , −∞ < β < ∞ , rq1 eβ + rq2

(6.131)

and the observed information at β = βˆ is obtained as ˆ = IN (β)

q˜ 

rq1 rq2 eβ

q=1

(rq1 eβ + rq2 )2

ˆ

ˆ

.

(6.132)

Fairly mild conditions which can be made mathematically precise by specializing the regularity assumptions formulated by Andersen and Gill (1982, § 3) (for additional details see Andersen et al., 1993, § VII.2.2) ensure that ˆ say, such that there exists a finite positive constant, σ∗2 (β) ˆ → 1/σ 2 (β) ˆ as N → ∞ , N −1 IN (β) ∗ P

and

√ L ˆ → N (βˆ − β)/σ∗ (β) N (0, 1) as N → ∞ , for all β .

(6.133) (6.134)

ˆ another application of TheoIn view of these properties of βˆ and IN (β), rem A.3.4 shows that the rejection region of an asymptotically valid test for  and hence for (6.128) itself, is given by (6.128)

 1/2  1/2 ˆ ˆ ˆ log(1 + ε) |β| < Cα IN (β) IN (β) . (6.135) 1/2 ˆ log(1 + ε) in order The function ψ → Cα (ψ) to be evaluated at ψ = IN (β) to determine the critical upper bound to the absolute value of the standardized estimate of the regression coefficient is the same as defined in the general description given in § 3.4 of the asymptotic approach to constructing tests for equivalence. In other words, for any ψ > 0, Cα (ψ) stands for the square root of the lower 100α percentage point of a χ2 -distribution with df = 1 and noncentrality parameter ψ 2 . Thus, given the value of βˆ and its estimated standard error, the practical effort entailed in computing the (random) critical bound to be used in the log-rank test for equivalence with rejection region (6.135) reduces to writing a single line of programming code invoking the intrinsic ˆ log2 (1 + ε), respectively, or SAS function cinv with arguments α, 1 and I(β)

Log-rank test for equivalence of two survivor functions

207

the R function qchisq with the same arguments to be plugged in, in the same order. The standard statistical software packages provide the user also with an easy to handle tool for finding the solution to the partial likelihood equation (6.131): It suffices to run a procedure tailored for analyzing standard proportional hazards models with time-independent covariates. In the output list one will find both the value of βˆ (e.g., under the label “Parameter Esti−1/2 ˆ (as an entry into the column mate” in the SAS procedure phreg) and IN (β) “se(coef)” in the standard output of the R function coxph). Altogether, these hints should make it sufficiently clear that the practical implementation of the log-rank test for equivalence as given by the rejection region (6.135) reduces to steps very easy to carry out by anybody having access to professional statistics software. Example 6.6 For the Medulloblastoma II Trial of the SIOP (for details see Bailey et al., 1995), N = 357 children suffering from such malignoma of the cerebellum were enrolled. At some stage of the statistical analysis of the data obtained from this long-term trial, it was of interest to establish equivalence with respect to relapse-free survival between the subgroups of patients treated in big (8 or more cases accrued into the trial) and small centers, respectively. Sample 1 (↔ patients from big centers) was of size m = 176, exhibiting a (1) total of d˜· = 75 failure events. In the n = 181 patients of Sample 2 (↔ (2) small centers), d˜· = 70 relapses were recorded. The complete relapse-free survival curves obtained for both groups by means of standard product-limit estimation are shown in Figure 6.11. As the maximum distance δ considered compatible with equivalence of the underlying true survivor functions [recall (6.120)] let us choose δ = .15, i.e., the value in the middle between the two alternative specifications recommended in § 1.7 for defining equiva lence ranges for parameters taking values in (0, 1). By (6.124) and (6.128), this is the same as specifying the alternative hypothesis |β| < .4106 [→ Table 6.30] about the regression coefficient introduced in (6.127). With the data underlying Figure 6.11, solving equation (6.131) gives the ML estimate ˆ = 36.1981 as the inβˆ = .0944. Using this value in (6.132) yields IN (β) 1/2 ˆ |β| ˆ = .5680 . On the other formation observed at βˆ so that we get IN (β) √ ˆ hand, with ψN = 36.1981 · .4106 = 2.4704 and α = .05, for the critical 1/2 ˆ ˆ bound which the test statistic IN (β) |β| has to be compared with according √ to (6.135), we obtain Cα (ψˆN ) = .689235 = .8302 where, in SAS, .689235 is the value assigned to the variable csq, say, as the result of getting executed the statement csq=cinv(.05,1,2.4704**2) (in R, the analogous statement reads csq=qchisq(.05,1,2.4704^2) yielding the value .6892414). Since the 1/2 ˆ ˆ β| thus falls short of its critical upper bound, the observed value of IN (β)| test decides in favor of equivalence of the survivor functions under comparison.

208

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Figure 6.11 Medulloblastoma Trial SIOP II: Observed relapse-free survival curves by size of treatment center. Solid line: Big centers having enrolled ≥8 patients. Dashed line: Small centers with total accrual < 8. (From Wellek, 1993b, with kind permission by the International Biometric Society.) Simulation results on the size of the log-rank test for equivalence In order to study the size of the critical region (6.135), two sets of simulation experiments were performed. In the first set, the baseline survivor function was lognormal with parameters μ = 2 and σ 2 = 1 as given by S2 (t) = Φ(2 − log t), whereas the censoring distribution function was specified G(t) = 1 − e−t/50 irrespective of the true value of the hazard ratio θ. In the second set of experiments, the lognormal distribution LN (2, 1) was replaced by the Weibull distribution with form parameter c = 2 and unit scale parameter. The associated censoring times were generated from two different rectangular distributions inducing baseline (↔ Group 2) censoring rates of 20 and 80%, respectively. The results of studying the size of the critical region (6.135) under both types of models are summarized in Tables 6.31 a,b. Each entry is based on 40, 000 replications of the respective simulation experiments and gives the fraction of samples for which the test decides in favor of equivalence. None of the entries into the rightmost columns of the tables indicates relevant anticonservatism so that the level properties of the test are fully satisfactory even when performed with very small samples.

Log-rank test for equivalence of two survivor functions

209

Table 6.31a Estimated rejection probabilities of the log-rank test for equivalence at nominal level α = .05 under the null hypothesis ||S1 − S2 || ≥ .15 (↔ ε = .5077) in a model with lognormal baseline survivor function and exponentially distributed censoring times.† m

n

θ = 1/(1 + ε)

θ =1+ε

.04945 .05078 .05108 .04778 .05085 .04800 .04943

.05103 .05158 .05003 .05238 .04913 .04963 .05025

25 25 50 25 50 50 75 50 75 75 100 75 100 100 †

censoring rates: .188 for Group 2; .268 and .133 for Group 1 under θ = 1/1.5077 and θ = 1.5077, respectively

Table 6.31b Analogue of Table 6.31a for a Weibull baseline survivor function and uniform censoring distributions U(0, τ ) inducing various group-specific censoring rates ζ1 and ζ2 . m

n

θ = 1/(1 + ε)

θ = 1+ε

∗

.04913 .04703 .05000 .04548 .04940 .04978 .04848

.05068 .05125 .04843 .05063 .05030 .05110 .05165

25 25 1.748709∗∗ 50 25 " 50 50 " 75 50 " 75 75 " 100 75 " 100 100 "

.04838 .04875 .04995 .04918 .05050 .04723 .05090

.04820 .05375 .04850 .04995 .05045 .05028 .04705

25 25 50 25 50 50 75 50 75 75 100 75 100 100

∗ (ζ , ζ ) 1 2

τ 4.431133 " " " " " "

= (.2450, .2000) and (.1633, .2000) under θ = 1/(1 + ε) and θ = 1 + ε, respectively ∗∗ (ζ , ζ ) = (.5938, .5000) and (.4127, .5000) under θ = 1/(1 + ε) 1 2 and θ = 1 + ε, respectively

210

6.7.2

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

Power approximation and sample size calculation for the log-rank test for equivalence

As usual, for purposes of approximating the power and computing sample sizes required for attaining some prespecified power, we need an expression for the “theoretical” variance of the nonstandardized test statistic under any alternative of potential interest. In view of practical applications to the planning of equivalence trials with survival endpoints, it seems sufficient to restrict the class of alternatives of interest by the following assumptions: (I) Both groups are subject to the same independent censoring mechanism induced by a recruitment process running uniformly over some fixed accrual period [0 , τ ] . (II) The study protocol specifies a minimum follow-up period of length t◦ ≥ 0 for each patient. Since t◦ might also be equal to zero, studies allowing the admission of new patients during the whole duration of the trial are covered as a special case. (III) The trial under consideration is to be a balanced one, in the sense that we have m = n, N = 2n. Applying Theorem VII.2.2 of Andersen et al. (1993) it can be shown that under assumptions (I)–(III), the asymptotic variance of the normalized partial √ likelihood estimator N βˆ is given by ˆ = σ∗2 (β) where eβ v (β ; τ, t ) = 2 2

◦



t◦

0



(6.136)

  β −1 eβ + (S2 (t))1−e f2 (t) dt +

t◦ +τ

t◦

1 v (β ; τ, t◦ ) 2



eβ + (S2 (t))1−e

β

 −1 t◦ + τ − t f2 (t) dt τ

(6.137)

and f2 (·) denotes the density of the baseline survival distribution as assumed for Group 2. In general, the integral appearing on the right-hand side of (6.137) admits no representation in closed form. However, it is easy to evaluate numerically (e.g., R R or Maple ). In the specific case β = 0 [↔ alternative using Mathematica of exact coincidence of both survival distributions], (6.137) reduces to v 2 (0 ; τ, t◦ ) =

1 P [T < C] 4

(6.138)

where T denotes a random variable with density f2 (·), and C a censoring variable independent of T and following the uniform distribution over the interval [t◦ , t◦ + τ ] .

Log-rank test for equivalence of two survivor functions

211

Now, (6.133) and (6.134) suggest that for purposes of power approxima√ 1/2 ˆ ˆ everywhere in (6.135) and treat with N /σ∗ (β) tion, we replace IN (β) √ √ ˆ ˆ N β / σ∗ (β) as normally distributed with unit variance and mean N β◦ / ˆ where β◦ ∈ IR denotes the value of the log-hazard ratio β [recall (6.126), σ∗ (β) (6.127)] specified as the alternative of interest. Denoting the power of the test given by (6.135) against β = β◦ by Π◦ , this leads to the relationship √

 √ ˆ − N β◦ /σ∗ (β) ˆ − Π◦ ≈ Φ Cα N log(1 + ε)/σ∗ (β)

√  √ ˆ − N β◦ /σ∗ (β) ˆ . Φ − Cα N log(1 + ε)/σ∗ (β) (6.139) A natural way of establishing a method for estimating minimally required sample sizes is now to treat (6.139) as an equation in N with all other quantities as given numbers. The task of formulating an algorithm for finding the solution to this equation can be considerably simplified by √ making use √ ˆ ≈ N log(1 + ( N log(1 + ε)/σ∗ (β)) of another approximation, namely Cα√ ˆ ˆ being substituted ε)/σ√ N log(1 + ε)/σ∗ (β) ∗ (β) − u1−α [→ p. 65, (4.26) with for nε ]. Combining the latter with (6.139) and dropping the distinction between approximate and exact identities yields, after a few rearrangements of terms, the equation  √ N (log(1 + ε) − β◦ ) − u1−α Φ ˆ σ∗ (β)  √ N (log(1 + ε) + β◦ ) − u1−α = 1 + Π◦ . (6.140) +Φ ˆ σ∗ (β) In order to determine√N from (6.140), it is convenient to proceed as follows: ˆ N as an unknown positive number, say σ ˜ , and solve We first treat σ∗ (β)/ the equation     log(1 + ε) − β◦ log(1 + ε) + β◦  − u1−α + Φ − u1−α = 1 + Π◦ (6.140) Φ σ ˜ σ ˜ for σ ˜ which can be done by elementary numerical techniques since the expression on the left-hand side is obviously decreasing in σ ˜ . Denoting the  by σ solution to (√ 6.140) ˜ (β◦ ; α, ε, Π◦ ), we have to choose N such that it satˆ N =σ ˜ (β◦ ; α, ε, Π◦ ). In view of (6.136), this is equivalent to isfies σ∗ (β)/ determining the approximate total sample size required for attaining power Π◦ against the specific alternative β◦ ∈ (− log(1 + ε), log(1 + ε)) by means of −1  . N= σ ˜ 2 (β◦ ; α, ε, Π◦ ) v 2 (β◦ ; τ, t◦ )

(6.141)

Through v 2 (β◦ ; τ, t◦ ) [recall (6.137)], the resulting minimum sample size depends both on the selected baseline survival distribution and the censoring

212

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

mechanism, as has necessarily to be the case. In the special case of the “null alternative” β◦ = 0 under which both sur  reduces to Φ log(1 + ε) − u1−α = vivor functions actually coincide, (6.140) σ ˜ 1 = ˜ (0; α, ε, Π◦ ) is explicitly given by (1 + Π◦ )/2 so that σ σ ˜ (0; α, ε, Π◦ )   Φ−1 (1 + Π◦ )/2 + u1−α . Plugging this expression into (6.141) gives log(1 + ε) N=

[Φ−1 ((1 + Π◦ )/2) + u1−α ]2 v 2 (0; τ, t◦ ) log2 (1 + ε)

(6.141◦ )

as the null alternative version of (6.141). In view of (6.138), one can state that (6.141◦ ) exhibits a strict formal analogy with the well-known sample size formula derived by Schoenfeld (1981) for the log-rank test for conventional one-sided hypotheses. In fact, according to Schoenfeld’s result, (6.141◦ ) approximates the total sample size required to detect the alternative that the true hazard ratio equals 1 + ε, with power 1 − α in a test at nominal significance level 1 − (1 + Π◦ )/2. Simulation results on power and sample size approximation In the majority of real applications of the log-rank test for equivalence, sample size calculation will be done under the assumption that the trial aims at detecting perfect coincidence of the survival distributions under comparison. Accordingly, the bulk of the simulation results presented below as a basis for assessing the accuracy of the sample size approximation (6.141) refers to the special case (6.141◦ ). Table 6.32 contrasts the total sample size computed by means of our approximate formula with the simulated exact N required to attain prespecified power Π◦ against β = β◦ = 0 for various values of Π◦ and common censoring rates ζ. In the Monte Carlo experiments performed to generate the entries in the last two columns of this table, we again assumed both true survival distributions to be Weibull with shape parameter c = 2.0 and unit scale parameter. Furthermore, the associated values of the censoring variables were generated to satisfy conditions (I)–(III) of p. 210 with t◦ = 0 [↔ no minimum follow-up period, i.e., admission of new patients over the whole duration of the trial] and length τ of accrual period being adjusted to match with the prespecified event rate 1–ζ common to both arms of the trial. The equivalence range for the log-hazard rate β was chosen as in Tables 6.31 a,b so that under the hypothesis to be established, a vertical distance of δ = .15 at most was considered compatible with equivalence between two survivor functions. Each individual simulation experiment was run 10,000 times.

Log-rank test for equivalence of two survivor functions

213

Table 6.32 Simulated exact total sample sizes versus sample sizes approximated by means of formula (6.141◦ ), for nominal power Π◦ = .5, .8, .9 and common censoring rate ζ = .1, .2, .5, .8. [Survival distributions: Weibull with form parameter c = 2.0; censoring distributions: uniform.]

ζ .10 " " .20 " " .50 " " .80 "

Π◦ .50 .80 .90 .50 .80 .90 .50 .80 .90 .50 .80

Approx. Simulated N Exact Power 140 .4779 226 .7826 286 .9008 160 .4834 254 .7871 320 .8902 256 .5022 406 .7931 514 .8966 638 .4994 1016 .7923

Simulated Exact N 144 230 = Approx. N 164 258 328 = Approx. N 408 = Approx. N = Approx. N 1020

Qualitatively speaking, the accuracy of the suggested approximation to the total sample size required to detect null alternatives in the log-rank test with prespecified power, turns out surprisingly good. In fact, the discrepancy between the approximated and the simulated exact value, if existent at all, is found not to exceed 4 per group which seems negligible for all practical purposes. The subsequent table suggests that this conclusion is warranted all the more if the alternative β◦ of interest is chosen as some point between the center and the boundaries of the equivalence range specified for the target parameter under the alternative hypothesis. The simulation study performed to generate the entries in the rightmost column of Table 6.33 was designed as follows: Both survival distributions were assumed to be exponential with hazard rate λ2 = .80 (↔ baseline) and λ1 = λ2 (1 + ψε) (↔ Group 1), respectively, with ε specified as before (i.e., ε = .5077) and 0 < ψ < 1. The censoring distribution was assumed uniform over [0, τ ] = [0, 1.992030] for both arms of the trial ensuring that the censoring proportion had exact value ζ2 = .50 under the baseline hazard rate. The entries in column 3 have been obtained by evaluating the second integral (the first one vanishes for t◦ = 0) on the right-hand side of (6.137) with S2 (t) = e−λ2 t , f2 (t) = λ2 e−λ2 t , t > 0.  for the respective Column 5 shows the values of σ ˜ which solve equation (6.140) combination of β◦ and Π◦ when (α, ε) is fixed at (.05, .5077).

214

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

The main conclusion to be drawn from the results shown in Table 6.33 is that the exact power attained with samples of total size estimated by means of formula (6.141) approximates the nominal power even slightly better for nonnull alternatives as compared to settings where perfect coincidence of both survivor functions is the specific alternative of interest. Presumably, this just reflects the obvious fact that given the values of all other parameters involved, the total sample size required to detect the specific alternative β = β◦ increases with the distance of β◦ from the center of the equivalence range for β. Table 6.33 Exact versus nominal power against nonnull alternatives of the log-rank test for equivalence with total sample size estimated from (6.141). [Baseline survival distribution: exponential with hazard rate .80; censoring distribution: uniform, inducing event rates of 1 − ζ2 = .50 under baseline hazard.]

ψ

β◦

v 2 (β◦ ; τ, t◦ )

Π◦

σ ˜ (β◦ ; α, ε, Π◦ )

Appr. N

.126806 " " .129242 " "

.50 .80 .90 .50 .80 .90

.17497 .13640 .11988 .16332 .11676 .09945

258 424 548 290 568 782

.1 .049526 " " " " .25 .119499 " " " "

6.7.3

Simul. Ex. Pow. .4995 .7951 .8949 .5038 .8010 .8975

Log-rank test for noninferiority

In the one-sided case, it is only of interest to rule out that the survivor function S1 (·) associated with the experimental treatment, falls below the reference survivor function S2 (·) by more than some given threshold δ > 0 at some point on the time axis. In other words, the experimental treatment is considered noninferior to the reference if there holds S1 (t) > S2 (t) − δ for all t > 0. In the proportional hazards model (6.119), the corresponding testing problem H1 : S1 (t) ≤ S2 (t) − δ

for at least one t > 0 versus K1 : S1 (t) > S2 (t) − δ

for all t > 0

(6.142)

can easily been shown to be equivalent to ˜ 1 : β < log(1 + ε) . ˜ 1 : β ≥ log(1 + ε) versus K H

(6.143)

Log-rank test for equivalence of two survivor functions

215

˜ 1, K ˜ 1 ) directly The sign reversal entailed by proceeding from (H1 , K1 ) to (H reflects the obvious fact that large hazards rates are undesirable. In view of (6.133) and (6.134), it follows from the general result stated in § 2.3 that the rejection region of an asymptotically valid test for (6.143) con (1) (1)  sists of all points in the sample space of the primary observations T1 , D1 ,  (1) (1)   (2) (2)  . . . , Tm , Dm , T1 , D1 for which it turns out that



1/2 ˆ (6.144) βˆ − log(1 + ε) IN (β) < uα . The practical implementation of the corresponding decision rule is still simpler than that of the log-rank test for two-sided equivalence and needs no additional explanation. Under the assumptions made precise in the previous subsection, the power Π◦ of the asymptotic test with rejection region (6.144) against any alternative β◦ can be approximated through setting the second term on the right-hand side of Equation (6.140) equal to unity yielding √  N (log(1 + ε) − β◦ ) − u1−α . Π◦ = Φ (6.145) ˆ σ∗ (β) Recalling (6.136) and solving this equation for N yields after some trivial rearrangements of terms: 2  Φ−1 (Π◦ ) + u1−α N= . (6.146) [log(1 + ε) − β◦ ]2 v 2 (β◦ ; τ, t∗ ) In the special case that interest is in controlling approximately the power against the null alternative specifying β◦ = 0, (6.146) simplifies to

N=

2  Φ−1 (Π◦ ) + u1−α log2 (1 + ε) v 2 (0; τ, t∗ )

.

(6.146◦ )

According to (6.141◦ ), this coincides with the approximate total sample size required for achieving power 2Π◦ − 1 against β = 0 in the log-rank test for two-sided equivalence. Example 6.7 For illustration, we present the results of sample-size calculation for a clinical trial to be run for the purpose of establishing one-sided equivalence of interstitial brachytherapy and radical surgery as treatments of localized carcinoma of the prostate with time to tumor progression as outcome criterion of primary interest. The study was projected to extend over a total of 6.5 years, including an accrual period of τ = 5 years’ duration. Accordingly, minimum

216

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES

follow-up time was limited to t◦ = 1.5 [yrs] . The probability of remaining progression-free for at least 5 yrs was assumed to be P2 = .85 after radical surgery, and the largest tolerable difference between this baseline survival rate and the corresponding value P1 = S1 (τ ) for the brachytherapy arm defined to be δτ = .15 . These specifications imply that the equivalence margin for the regression coefficient β of the dummy covariate [treatment-arm indicator] Z is given by .85(1+ε) = .70 ⇔ 1 + ε = 2.194667 ⇔ log(1 + ε) = .786030 .

(6.147)

Sample size calculation was based on the assumption that both survival distributions are exponential. The power to detect the null alternative β◦ = 0 in a log-rank test for noninferiority at asymptotic level α = .05 was set at Π◦ = .80 . Starting from these assumptions and specifications, calculation √ of the asymptotic variance of the normalized partial likelihood estimator N βˆ by means of (6.138) is a fairly easy task. Under the selected alternative, both survival distributions are exponential with hazard rate − log(P2 )/τ = − log(.85)/5 = .0325 . The probability that a random variable T with distribution E(1/.0325) does not exceed an independent C uniformly distributed over [0, τ ] = [0, 5] is obtained by means of elementary calculations to be .120952 so that we have v 2 (0; 5, 1.5) = (1/4) P [T < C] = .030238 . Plugging in this value together with (6.147) and α = .05, Π◦ = .80 in our sample size formula (6.146◦ ) for the noninferiority case with β = 0 as the alternative of interest, yields 2  −1 Φ (.80) + u.95 [.841621 + 1.644854]2 = 330.93 ≈ 330 . = N= .030238 · .7860302 .018682 Since the assumption that the experimental therapy leads to exactly the same tumor progression hazard as the reference treatment was deemed a bit too optimistic, the following nonnull alternative was alternatively taken into consideration: P1 = S1 (τ ) = .85 , P2 = S2 (τ ) = .825 ⇒ .85exp{β◦ } = .825 ⇔ β◦ = .168636 . Leaving all other specifications invariant, we obtain v 2 (β; τ, t∗ ) = .027529 implying that the total sample size required for ensuring asymptotic power 80% has to be recomputed as 2  −1 Φ (.80) + u.95 = 589.19 ≈ 590 . N= 2 .786030 − .168636 · .027529 Comparing both results reflects the fact that the power of the log-rank test for (one-sided) equivalence is highly sensitive against misspecifications of the alternative.

Log-rank test for equivalence of two survivor functions

217

Simulation results on the log-rank test for noninferiority The simulation studies performed to investigate the small-sample behavior under specific alternatives of the noninferiority version of the log-rank were designed in exactly the same way as those which gave the results shown in Tables 6.32 and 6.33 for the case of testing for two-sided equivalence of survivor functions. Table 6.34 provides a basis for assessing the accuracy attainable by estimating total sample size through applying formula (6.146◦ ). The models for generating the data by Monte Carlo simulation were exactly the same as in the two-sided case, and regarding the differences between approximated and simulated exact values of N , the same qualitative statements can be made. Again, the accuracy of the suggested approximation to the sample size required when the focus is on null alternatives, turns out very satisfactory. Table 6.34 Results of repeating the simulation experiments leading to Table 6.32, for the noninferiority version of the log-rank test.

ζ .10 " " .20 " " .50 " " .80 "

Π◦ .50 .80 .90 .50 .80 .90 .50 .80 .90 .50 .80

Approx. Simulated N Exact Power 72 .4920 162 .7886 226 .8952 80 .4850 184 .7950 254 .8906 128 .4966 294 .8022 406 .8995 320 .4938 734 .8035

Simulated Exact N 74 166 = Approx. N 84 = Approx. N 256 = Approx. N = Approx. N = Approx. N 322 = Approx. N

Finally, all simulation experiments whose results are given in Table 6.33 were likewise repeated with the log-rank test for noninferiority. Comparing the entries in columns 4 and 6 of Table 6.35 within the same row leads to the conclusion that the accuracy of the large sample approximation (6.145) to the power of the one-sided version of the log-rank test for equivalence keeps being remarkably good even when the alternative of interest is chosen from a neighborhood of the null model.

218

EQUIVALENCE TESTS FOR TWO UNRELATED SAMPLES Table 6.35 Exact versus nominal power against nonnull alternatives of the log-rank test for noninferiority with total sample size estimated from (6.146). [For details on notation and the models used to generate the data see the description of Table 6.33.]

ψ β◦ .1 .049526 " " " " .25 .119499 " " " "

v 2 (β◦ ; τ, t◦ ) .126806 " " .129242 " "

Π◦ .50 .80 .90 .50 .80 .90

Appr. N 164 374 518 248 564 782

Simul. Ex. Pow. .4993 .7952 .8969 .4993 .7920 .8929

7 Multisample tests for equivalence

7.1

The intersection-union principle as a general solution to multisample equivalence problems

The existing literature on k-sample equivalence testing procedures is comparatively sparse. A reason for this fact might be that, at least for parametric settings, there is a straightforward yet often overconservative solution to problems of establishing equivalence of k distributions as soon as a test for the corresponding two-sample problem is available. The rationale behind this approach going back to Berger (1982) is a result which can appropriately be termed intersection-union principle because of its straight duality with Roy’s (1953) well-known union-intersection approach to multiple testing and interval estimation. The scope of potential applications of the principle is fairly wide and goes far beyond the specific multisample settings discussed in the subsequent sections of this chapter. In order to introduce the general idea, let us assume that we are considering an arbitrary but fixed number q ≥ 2, say, of elementary null hypotheses H1 , . . . , Hq to be tested against associated alternatives K1 , . . . , Kq . For each ν = 1, . . . , q, let Hν and Kν denote statements about some parametric function (in nonparametric settings: a functional of the distributions from which the data are taken) ην such that there is some testing procedure φν , say, valid at the selected nominal significance level α ∈ (0, 1). Finally, suppose that the rejection probability of test φν does not exceed α, under any possible parameter constellation (η1 , . . . ηq ) such that ην ∈ Hν , irrespective of how many and what other elementary null hypotheses are true. Then, the intersection-union principle states that the following decision rule defines a valid level-α test for the “combined” problem H ≡ H1 ∪ H2 ∪ . . . ∪ Hq

versus

K ≡ K1 ∩ K 2 . . . ∩ K q :

Reject H in favor of K if and only if each individual test φν rejects the null hypothesis Hν which it has been constructed for (ν = 1, . . . q) .

(7.1)

(7.2)

The proof of the result is almost trivial, at least if one is willing to adopt some piece of the basic formalism customary in expositions of the abstract

219

220

MULTISAMPLE TESTS FOR EQUIVALENCE

theory of statistical hypotheses testing methods. In fact, if we denote the combined test defined by (7.2) φ and identify each test involved with its critical function, it is clear that φ admits the representation φ = φ1 · φ2 · . . . · φq . The condition we have to verify, reads then as follows: E(η1 ,..,ηq ) (φ) ≤ α for all (η1 , .., ηq ) ∈ H

(7.3)

where E(η1 ,...,ηq ) (·) denotes the expected value computed under the parameter constellation (η1 , . . . , ηq ). Now, if (η1 , . . . , ηq ) belongs to H, this means by definition that there is some ν ∈ {1, . . . , q} with ην ∈ Hν . Furthermore, every critical function φ1 , . . . , φq takes on values in [0, 1] only, which implies that φ ≤ φν and thus E(η1 ,...,ηq ) (φ) ≤ E(η1 ,...,ηq ) (φν ). By assumption, we have E(η1 ,...,ηq ) (φν ) ≤ α for every (η1 , . . . , ηq ) with ην ∈ Hν , and hence the proof of (7.3) is already complete. It is interesting to note that the principle of confidence interval inclusion introduced in § 3.1 as a general approach to the construction of tests for single equivalence hypotheses, can be viewed as a special case of the intersectionunion principle. This follows simply from the fact pointed out on p. 34 that every interval inclusion test admits a representation as a double one-sided testing procedure. In order to apply the result to multisample equivalence testing problems, let θj be the parameter of interest (e.g., the expected value) for the ith distribution under comparison, and require of a pair (i, j) of distributions equivalent to each other that the statement K(i,j) : ρ(θi , θj ) < ε ,

(7.4)

holds true with ρ( · , · ) denoting a suitable measure of distance between two parameters. Suppose furthermore that for each (i, j) a test φ(i,j) of H(i,j) : ρ(θi , θj ) ≥ ε versus K(i,j) : ρ(θi , θj ) < ε is available whose rejection probability is ≤ α at any point (θ1 , . . . , θk ) in the full parameter space such that ρ(θi , θj ) ≥ ε. Then, by the intersection-union principle, deciding in favor of  “global equivalence” if and only if equivalence can be established for all k2 possible pairs, yields a valid level-α test for H : max{ρ(θi , θj )} ≥ ε i<j

versus

K : max{ρ(θi , θj )} < ε . i<j

(7.5)

As a specific instance, we get a quick solution to the problem of testing k Gaussian distributions with common unknown variance σ 2 for equivalence with respect to their unscaled means, say μ1 . . . , μk , if we proceed in the following way: Apply for each (i, j) the interval inclusion rule with standard tbased (1−2α)-confidence intervals for μi −μj . This yields a test for equivalence of N (μi , σ 2 ) and N (μj , σ 2 ) in the sense of |μi − μj | < ε whose validity is not affected by changes of any other functions of the whole vector (μ1 , . . . μk ) of means except for the difference within the given pair. Thus, deciding in favor   of “global” equivalence of all k distributions if and only if all k2 confidence

The intersection-union principle

221

intervals are included in (−ε, ε) yields a valid level-α test of H : |μi − μj | ≥ ε for at least one (i, j) versus K : |μi − μj | < ε for all (i, j) ∈ {1, . . . , k}2 . More often than not, the obvious advantages of this approach (conceptual simplicity of the underlying principle, ease of practical implementation of the resulting “global” decision rule) will be outweighed by the following drawbacks: 1. The test of H : maxi<j {ρ(θi , θj )} ≥ ε versus K : maxi<j {ρ(θi , θj )} < ε   obtained by applying the intersection-union principle with q = k2 and Hij : ρ(θi , θj ) ≥ ε vs. Kij : ρ(θi , θj ) < ε (1 ≤ i < j ≤ k) as the elementary testing problems can be markedly overconservative. 2. Looking at the maximum distance between pairs of distributions might not adequately translate an investigator’s notion of equivalence between all k distributions into a testable statistical hypothesis; for other distance measures, the intersection-union principle is typically no longer applicable. In the following sections of the present chapter, we show for selected settings frequently encountered in practice, how to replace the intersection-union test with a less conservative specific multisample test for equivalence.

7.2

F -test for equivalence of k normal distributions

The considerations of this (as well as of the subsequent) section refer to the classical univariate ANOVA one-way layout with normally distributed observations of common (yet unknown) variance σ 2 ∈ IR+ . Thus we suppose that the pooled data set to be analyzed exhibits the following structure: X11 , . . . , X1n1 X21 , . . . , X2n2 ............ .. Xk1 , . . . , Xknk

← ← ... ←

1st 2nd ... kth

sample, sample, ...... , sample,

from N (μ1 , σ 2 ) from N (μ2 , σ 2 ) ... ............ from N (μk , σ 2 )

In the balanced case n1 = . . . = nk = n, a natural measure of the degree to which the true underlying distributions deviate from the “ideal” of perfect pairwise coincidence is given by the squared Euclidean distance of the vector μ· /σ) 1k , where μ ¯· and 1k denotes the ordi(μ1 /σ, . . . , μk /σ) from the point (¯ nary arithmetic mean of the μi and the k-vector of ones, respectively. A useful way of extending the definition of this measure to arbitrarily unbalanced de˜· , say, of signs of the same kind consists of replacing μ ¯· with the average μ the sample-size–weighted population means ni μi , and the unit weight to the

222

MULTISAMPLE TESTS FOR EQUIVALENCE

n). The resulting generalized squared distance in the ith component with (ni /¯ squared Euclidean distance is then given by ψ2 =

k 

(ni /¯ n)(μi − μ ˜· )2 /σ 2 ,

(7.6)

i=1

with μ ˜· =

k 

ni μi /

i=1

k 

ni ,

n ¯=

i=1

k 

ni /k .

(7.7)

i=1

Using ψ 2 as our target parameter leads to the problem of testing H : ψ 2 ≥ ε2

versus

K : ψ 2 < ε2

(7.8)

with suitably fixed ε > 0. Clearly, in any balanced design, equivalence in the sense of (7.8) is satisfied if and only if the point with the standardized population means as coordinates lies in a k-dimensional sphere of radius ε around (¯ μ· /σ, . . . , μ ¯· /σ). In the case simplifies still of k = 2 samples with common size n1 = n2 = n, this condition √ further in that it turns out equivalent to |μ1 − μ2 |/σ < 2 ε . This suggests adopting the guidance given in Table 1.1(v) for specifying the tolerance ε in the √ two-sample case, also√for arbitrary multisample designs choosing ε = .36/ 2 ≈ .25 and ε = .74/ 2 ≈ .50, depending on whether the equivalence criterion is intended to be rather strict or comparatively weak. The testing problem (7.8) remains invariant under a large group G, say, of affine transformations which contains in particular arbitrary common rescalings and translations of all observations in the pooled sample (an exhaustive characterization of G involves canonical coordinates and is not required in the present context; for full mathematical details see Lehmann and Romano, 2005, § 7.1). The usual F -statistic for the one-way fixed-effects ANOVA is a maximal invariant with respect to this group of transformations. Dropping the constant factor n ¯ /(k − 1), it can be viewed as an estimator of the target parameter ψ 2 so that we write k  ¯i − X ¯ .. )2 (ni /¯ n)(X

ψˆ2 =

i=1

(N − k)

−1

ni k  

(7.9) (Xiν

¯ i )2 −X

i=1 ν=1

The distribution of (¯ n/(k − 1))ψˆ2 depends on the μi and σ 2 only through ψ 2 and is noncentral F with k − 1, N − k degrees of freedom (where N denotes  the total sample size ki=1 ni ) and noncentrality parameter λ2nc = n ¯ ψ 2 . It is thus an element of a family with strictly monotone likelihood ratios, and it follows that the class of all level-α tests for (7.8) which depend on the raw

F-test for equivalence of k normal distributions

223

data only through ψˆ2 contains a uniformly most powerful one. The critical region of this optimal test is given by

 n)Fk−1,N −k; α (¯ nε2 ) (7.10) ψˆ2 < ((k − 1)/¯ with Fk−1,N −k; α (¯ nε2 ) ≡ lower 100α-percentage point of a noncentral F -distribution with k − 1, N − k degrees of freedom and noncentrality parameter n ¯ ε2 .

(7.11)

In view of the maximal invariance of the statistic ψˆ2 with respect to the group G of all transformations of the full sample space leaving the testing problem invariant, the rejection region (7.10) defines an UMPI level-α test for (7.8). Its size is exactly equal to α, and the power against any specific alternative is strictly greater than the significance level (→ strictly unbiased testing procedure). The exact rejection probability under any parameter constellation depends only on ψ 2 and can easily be computed by means of the formula: nψ 2 ) < Fk−1,N −k; α (¯ nε2 ) ] (7.12) β(ψ 2 ) = P [ Fk−1,N −k (¯ where Fk−1,N −k (ψ˜2 ) stands for a variable which has an F -distribution with k − 1, N − k degrees of freedom and noncentrality parameter ψ˜2 . In particular, using SAS one will obtain the value of β(ψ 2 ) simply by submitting the statements c=finv(alpha,k-1,N-k,n_*eps**2); beta=probf(c,k-1,N-k,n_*psi**2); provided, the value specified for α, k, N , n ¯ , ε and ψ has been previously assigned to the variable alpha, k, N, n_, eps and psi , respectively. Example 7.1 In a comparative clinical trial of 4 different antihypertensive treatments, blood pressure was recorded continuously over 24 hours in each patient enrolled. The AUC (area under the curve) for the diastolic value divided by the length of the measurement interval was defined as the primary endpoint for assessing efficacy of treatment. The raw data [mm Hg] obtained in this way are listed in the following table:

224

MULTISAMPLE TESTS FOR EQUIVALENCE Table 7.1 Raw data of a 4-arm trial using the average of diastolic blood pressure recorded continuously over 24 hours as primary endpoint. Sample 1 112.488, 103.738, 95.815, 91.864,

86.344, 101.708, 102.479, 102.644

Sample 2 100.421, 101.966, 105.486, 98.662,

99.636, 94.137,

Sample 3 84.846, 100.488, 99.510, 89.005, 107.067

119.763, 108.200,

Sample 4 100.825, 100.255, 94.750, 107.129, 92.691, 93.111,

103.363, 98.246, 98.243

95.108,

105.931,

88.377, 89.367,

102.618, 106.204

103.736, 93.141, 82.209, 100.104,

108.254, 103.706,

105.983, 98.626,

93.230, 96.365,

95.325, 99.740,

100.288, 106.049,

The sample means and standard deviations computed from these raw data are shown in Table 7.2. Hence, in the present example, we have: N = 50 , k = 4 , n ¯ = 12.5 ;

¯ = 99.3849 ; X..

k 

¯i − X ¯ .. )2 = 1.2157 ; (ni /¯ n)(X

i=1

(N − k)−1

ni k  

¯ i )2 = 54.6976 ; (Xiν − X

i=1 ν=1

⇒

ψˆ2 = 1.2157/54.6976 = .0222 .

Specifying α = .05 and ε = .50, the critical upper bound of (7.10) comes out (using SAS) as c = (3/12.5) · F4−1,50−4; .05 (12.5 · .25) = .24 · finv(.05,3,46,3.125) = .24 · .30833 = .0740 . Since the observed value of the test statistic ψˆ2 falls below this bound, the UMPI test rejects nonequivalence. Finally, evaluating the right-hand side of equation (7.12) yields for the power of the test against the specific alternative of identical distributions: β(0) = P [ F3,46 (0) < (12.5/3) · c ] = probf(.30833,3,46,0) = 18.08% .

Modified studentized range test for equivalence

225

Table 7.2 Summary statistics for the raw data of Table 7.1 [ with i indexing the individual samples]. i 1 2 3 4

7.3

¯i X ni Si 10 99.8120 7.5639 12 99.2903 5.9968 13 100.0024 10.4809 15 98.6407 4.5309

Modified studentized range test for equivalence

Under the additional assumption that the one-way ANOVA layout we are dealing with is a balanced one satisfying n1 = . . . = nk = n, an alternative approach to testing for equivalence of an arbitrary number k of normal distributions with means μi ∈ IR and common variance σ 2 > 0 can be based on the studentized range statistic. Of course, such a change in the test statistic would make little if any sense without a corresponding reformulation of hypotheses in terms of a different parametric function viewed as the basic measure of dissimilarity of the distributions under assessment. In the present context, this measure is chosen to be the maximum of all pairwise distances between standardized population means which leads to replacing (7.8) with the testing problem ˜ μ : max |μi − μj |/σ ≥ δ H i<j

versus

˜ μ : max |μi − μj |/σ < δ , K i<j

(7.13)

with arbitrarily fixed equivalence margin δ > 0 . The studentized range statistic which will be denoted Rs in the sequel, is obtained by simply replacing both maxi<j |μi − μj | and σ with its sample analogue yielding Rs =

¯ (k) − X ¯ (1) X S

(7.14)

¯ (i) stands for the ith-smallest (i = 1, . . . , k) among the k sample where X means, and S 2 is the usual ANOVA estimator of the error variance as appearing in the denominator to (7.9). In the equivalence case, determining a suitable critical constant for the statistic Rs is a rather complicated problem which has been solved by Giani and Finner (1991). As shown by these authors, the critical bound rα; δ (n, k),

226

MULTISAMPLE TESTS FOR EQUIVALENCE

say, which the studentized range statistic has to be compared with in order to obtain a valid test for (7.13), has to be computed as the α-quantile of the distribution of Rs under the specific parameter constellation (μ1 /σ, . . . , μk /σ) = (−δ/2, 0, . . . , 0, δ/2) . Fortunately, a program has been made available by the same group of authors (Giani et al., 2005) which computes rα; δ (n, k) for arbitrary combinations of α, δ, n and k by numerical integration. Table 7.3 shows the critical upper bounds to Rs to be used in equivalence testing at level α = .05 for δ ∈ {1/2, 1} with up to 5 samples of size 10 and 50, respectively.

Table 7.3 Critical bounds for the studentized range test for equivalence at level α = .05, for δ = 1/2, 1, n = 10, 50 and k = 3, 4, 5 [computed by means of the program SeParATe, Version 3.0B (Giani et al., 2005)]. δ .50 " " " " " 1.00 " " " " "

n 10 10 10 50 50 50 10 10 10 50 50 50

k 3 4 5 3 4 5 3 4 5 3 4 5

rα; δ (n, k) .185896 .293307 .376803 .221237 .255186 .281114 .407372 .494329 .559521 .669046 .673074 .676400

Example 7.1 (continued) Suppose 99.8120 ± 7.5639 , 99.2903 ± 5.9968 , 100.0024 ± 10.4809 , 98.6407 ± ¯ i ± Si ) the data 4.5309 [cf. Table 7.2] would actually describe (in terms of X obtained in a balanced trial involving k = 4 groups of size n = 10 each. Then, the estimator of the error variance would take on the value S 2 = (7.56392 + 5.99682 + 10.48092 + 4.53092 )/4 = 55.8881, and for the studentized range statistic we would have √ ¯ (4) − X ¯ (1) )/S = (100.0024 − 98.6407)/ 55.8881 = .1821 . Rs = (X On the other hand, the critical bound to which Rs must be compared in a test at level α = .05 of maxi<j |μi − μj |/σ ≥ 1.00 versus maxi<j |μi − μj |/σ < 1.00 with four groups of 10 observations each is seen from the above table to

Modified studentized range test for equivalence

227

be r.05; 1.00 (10, 4) = .494329 . Hence, with the data and the specifications of this (synthetic) example, the modified studentized range test decides in favor of equivalence again. Before concluding the discussion of approaches to testing for equivalence of several homoskedastic normal distributions with respect to their standardized means, it seems worthwhile to point out the most important differences and similarities between the modified studentized range and the noncentral F -test of § 7.2: – Both tests are exactly valid with respect to the significance level, provided there are no deviations from the basic assumptions of the classical ANOVA model. – The critical constant of the noncentral F -test can easily be computed without recourse to nonstandard software. – The applicability of the studentized range statistic is restricted to balanced designs whereas the noncentral F -test allows for arbitrary imbalances between the sample sizes. – Power comparisons between the procedures are futile since the hypotheses they are tailored for are of grossly different shape (except for k = 2). For the special case of testing for equivalence of just three normal distributions with respect to their standardized means, a range test which is also applicable for unequal sample sizes, has been presented by Wiens and Iglewicz (2000) (see also Wiens and Iglewicz, 1999). The procedure does even allow for heteroskedasticity. However, its usefulness is largely restricted by the fact that the critical constants for the test statistic were determined under the assumption that the population variances are known.

7.4

Testing for dispersion equivalence of more than two Gaussian distributions

If homoskedasticity of the normal distributions behind the samples (X11 , . . . , X1n1 ), . . . , (Xk1 , . . . , Xknk ) cannot be taken for granted a priori, then it is natural that one tries to establish it by means of the actual data. In order to keep the description of a large-sample procedure for an equivalence hypothesis specifying homoskedasticity except for negligible heterogeneity of the variances as simple as possible, we restrict consideration first to balanced designs. Furthermore, it will be convenient to use the following additional

228

MULTISAMPLE TESTS FOR EQUIVALENCE

notation: ζi = log σi ,

ζ¯ = k−1

k 

log σi ;

(7.15a)

i=1

Zi = log Si ,

Z¯ = k−1

k 

log Si .

(7.15b)

i=1

A reasonable way of translating the phrase “homoskedastic except for irrelevant differences in variability” into a formal statement about a suitable parametric function consists in requiring that all logarithmic standard devia¯ tions ζi lie in a sphere of sufficiently small radius ε∗ around their average ζ. This leads to proposing the problem of testing Hσ :

k 

¯ 2 ≥ ε2 (ζi − ζ) ∗

i=1

versus

Kσ :

k 

¯ 2 < ε2 (ζi − ζ) ∗

(7.16)

i=1

with some suitably fixed ε∗ > 0. Some guidance for choosing the equivalence  ¯ 2 can be provided by noting that in the two-sample margin for ki=1 (ζi − ζ) case, dispersion equivalence in the sense of the above alternative hypothesis (1 + ε)−1 < σ1 /σ2 < 1 + ε with ε being related to Kσ reduces to the condition √ ε∗ by the equation ε∗ = (1/ 2) log (1 + ε). In view of this fact, generalizing the suggestions given in Table 1.1 for choosing the equivalence margin for the case of k = 2 variances leads to specifying ε∗ = .2867 or ε∗ = .4901, depending on whether a rather strict or more liberal criterion of equivalence is intended to apply. In order to construct a large-sample test for (7.16), we exploit the wellknown fact (see, e.g., Lehmann, 1986, p. 376) that the observed log-standard deviations Z1 = log S1 , . . . , Zk = log Sk are asymptotically normal and mutually independent estimators of the corresponding population parameters ζ1 = log σ1 , . . . , ζk = log σk . Since all of these estimators have the same asymptotic variance (2n)−1 , this means that for large n, the Zi = log Si have distributions N (log σi , 1/2n). In view of the mutual independence of all statistics depending on individual variables from different samples, this implies that for arbitrary values of the true σi ’s, we have k    ¯ 2 ≤ v 2 ≈ Gk−1, λ2 (v 2 ) ∀ v 2 > 0 P 2n (Zi − Z) nc

(7.17)

i=1

 ¯ 2 and Gk−1, λ2 (·) as the cdf of a χ2 -distribution with with λ2nc = 2n (ζi − ζ) nc df = k − 1 and noncentrality parameter λ2nc . In view of (7.17), an approximately valid test at level α for (7.16) is given by the rejection region 

(7.18) Qk,n < qk,n; α (ε∗ )

Dispersion equivalence of k Gaussian distributions where Qk,n = 2n

k  

log Si − (1/k)

k 

229

log Sι

2

(7.19)

ι=1

i=1

and qk,n; α (ε∗ ) = χ2k−1; α (2nε2∗ ) ≡ lower 100α percentage point of a χ2 -distribution with k − 1degrees of freedom and noncentrality parameter λ2nc = 2nε2∗ .

(7.20)

The power of this test against any alternative (σ1 , . . . , σk ) ∈ such that 2  k k  ˜ 2 < 2nε2 can be approximated =λ 2n i=1 log σi − (1/k) nc ∗ ι=1 log σι as  2  ˜2 ) ≈ G χk−1; α (2nε2∗ ) . (7.21) β(λ ˜2 nc k−1;λ IRk+

nc

Example 7.2 The data displayed in Table 7.4 are sample variances and corresponding logstandard deviations observed in k = 4 groups of common size n = 50. The Gaussian distributions behind the samples are to be tested for dispersion equivalence in the sense of (7.16) where the radius ε∗ of the equivalence sphere centered at the mean of the logarithmic population standard deviations is specified ε∗ = .2867 corresponding to a ratio of 1.5 at most in the two-sample case. The significance level is once more fixed at α = .05.

Table 7.4 Variances and logarithmic standard deviations computed from a raw data set consisting of k = 4 independent samples of common size n = 50. i Si2 Zi

1

2

3

4

49.4 45.7 43.1 53.6 1.9500 1.9110 1.8818 1.9908

For this data set, the value of the test statistic turns out as 2 · 50 ·

k  i=1

¯ 2 = 100 · .006735 = .6735 . (Zi − Z)

230

MULTISAMPLE TESTS FOR EQUIVALENCE

k ¯ 2 has On the other hand, the critical upper bound which 2n i=1 (Zi − Z) to be compared to, is readily computed (using once more the respective SAS function) to be χ23;.05 (2 · 50 · .28672 ) = cinv(.05,3,8.2197) = 2.85636. Since the value observed for the test statistic clearly falls short of this bound, the data set shown in the above table falls in the rejection region (7.18) so that we have to decide in favor of dispersion equivalence of the underlying normal distributions. In order to approximate the power of the test against the alternative of exact homoskedasticity, we invoke the SAS intrinsic function for the cdf rather the quantile function of the χ2 -distribution with k − 1 = 3 degrees of freedom, giving β(0) ≈ G3;0 (2.85636) = probchi(2.85636,3,0) = 58.57% . Although it is not our intention here to enter into a detailed investigation of the accuracy of the approximation (7.17) underlying the testing procedure given by (7.18–20), it should be mentioned that sufficient simulation results are available to justify the following qualitative statements: Even for large numbers k of distributions to compare, the probability of rejecting the null hypothesis of (7.16) according to (7.18) approaches the nominal significance level α rather quickly with increasing common sample size n, under parameter configurations lying on the common boundary of Hσ and Kσ . Furthermore, if nonnegligible discrepancies between α and exact rejection probabilities under parameter configurations (σ1 , . . . , σk ) satisfying 2 k k  = ε2∗ occur at all, the testing procedure is i=1 log σi − (1/k) ι=1 log σι found to be over- rather that anticonservative throughout. Generalization to the unbalanced case Unbalancedness of the design requires only minor modifications to the test. It suffices to replace in all sums appearing in the formula for the test statistic and the noncentrality parameter, unit weights by the weighing factors  ni /¯ n, with n ¯ = N/k = ki=1 ni /k. By plugging in these weights the alternative hypothesis of dispersion equivalence takes on the form Kσ :

k 

k

2  (ni /¯ n) ζi − nι ζι /N ,

i=1

(7.16 )

ι=1

and the test statistic has to be computed as Qk,n ≡ 2

k  i=1

k 2   ni log Si − nι log Sι /N . ι=1

(7.19 )

Nonparametric k-sample test for equivalence

231

Finally, the critical upper bound to which the observed value of Qk,n has to be compared, is given by  2 nε2∗ ) . qk,n; α (ε∗ ) = χk−1;α (2¯

7.5

(7.20 )

A nonparametric k-sample test for equivalence

The approach to multisample equivalence testing to be discussed in this section can be viewed as a natural extension of the Mann-Whitney test for equivalence of two continuous distributions of arbitrary shape as described in § 6.2. Although the basic idea behind this extension is almost self-explanatory, the construction is technically rather intricate involving a lengthy sequence of formulae for the various entries in the covariance matrix of a vector of U -statistics whose dimension grows with the square of the number of distributions under comparison. Instead of writing down all these details which would be rather tedious, we confine the exposition to an outline of major steps to be taken in carrying out the construction. First of all, we must define a sensible nonparametric measure of distance between all k distributions. This can be obtained by measuring the distance between each individual pair of distributions in the same way as in the twosample case and combining the resulting k(k − 1)/2 functionals into a single overall measure by means of a suitable algebraic operation. In order to make this idea precise, let us define for any 1 ≤ i < j ≤ k  + = P [Xi > Xj ] = Fj dFi (7.22) πij where Xi and Xj denote “generic” observations, independent of each other, from population i and j with cdf Fi and Fj , respectively. In view of the + + = 1/2, we proposed in § 6.2 |πij − 1/2| as obvious fact that Fi = Fj ⇒ πij a suitable nonparametric distance measure for the case of two samples. A + − 1/2| with natural generalization to the k-sample setting is to replace |πij + + + + the Euclidean distance between the vector (π12 , . . . , π1k , π23 , . . . , πk−1,k ) of all pairwise proversion probabilities (7.22), and that point in the (k(k − 1)/2)dimensional unit interval whose coordinates all equal 1/2. Using this distance in its squared form (which turns out technically a bit more convenient), we are lead to formulate the testing problem   + + (πij − 1/2)2 ≥ ε˜2 versus K : (πij − 1/2)2 < ε˜2 (7.23) H: i<j

i<j

with suitably specified equivalence margin ε˜2 > 0.

232

MULTISAMPLE TESTS FOR EQUIVALENCE

In order to keep notation sufficiently compact, we introduce an extra symbol for representing the squared Euclidean distance between any two vectors in (k(k − 1)/2)-dimensional space defining 

k(k−1)/2

d2 (u, v) =

(ui − vi )2 ,

u, v ∈ IRk(k−1)/2 .

(7.24)

i=1

Using this convention, the condition assumed to be satisfied under the alternative hypothesis of (7.23) can be written d2 (π + , 1/2) < ε˜2 , provided the symbols appearing as arguments to d2 ( · , · ) in that inequality are read as + + + + π + = (π12 , . . . , π1k , π23 , . . . , πk−1,k ) , 1/2 = (1/2) · 1k(k−1)/2 ,

(7.25)

with 1m denoting the vector of m 1’s for arbitrary m ∈ IN. Now, a natural choice of a statistic which an asymptotic test for establishing the hypothesis K : d2 (π + , 1/2) < ε˜2 can be based upon, is this: We estimate the squared distance between π + and 1/2 by that between the corresponding vector W + , say, of two-sample U -statistics, and the same point of reference in the parameter space [0, 1]k(k−1)/2 . Of course, for any 1 ≤ i < j ≤ k, the (i, j)-component of W + has to be defined by Wij+ =

nj ni  1  I(0,∞) (Xiμ − Xjν ) ni nj μ=1 ν=1

(7.26)

[recall (6.13)]. The most tedious part of the construction consists in establishing a procedure for estimating the asymptotic standard error of the estimated distance d2 (W + , 1/2). For that purpose, we need to know the full covariance matrix, √ (N ) say Σ(N ) = (σ(i1 ,j1 ),(i2 ,j2 ) )1≤i1 <j1 ≤k, 1≤i2 <j2 ≤k , of N W + . For each element of Σ(N ) , an exact expression can be found which is a function of the sizes of the samples involved, the appropriate components of π + , and functionals of (N ) the form P [Xi1 > Xj1 , Xi2 > Xj2 ]. The resulting formulae for σ(i1 ,j1 ), (i2 ,j2 ) differ markedly in structure, depending on the number of common elements in the set {i1 , j1 , i2 , j2 } of indices involved. For i1 = i2 , j1 = j2 , Wi+1 j1 and Wi+2 j2 refer to four different samples, which trivially implies that we have (N )

σ(i1 ,j1 ),(i2 ,j2 ) = 0 for all (i1 , j1 ), (i2 , j2 ) with i1 = i2 and j1 = j2 .

(7.27)

The other extreme is given by the diagonal elements of Σ(N ) : In this case, (N ) {i1 , j1 , i2 , j2 } reduces to a set of cardinality 2, say {i, j}, and σ(i,j), (i,j) is readily obtained by applying formulae (6.14), (6.15) for the computation of the exact variance of a single Mann-Whitney statistic. If the pairs (i1 , j1 ) and (i2 , j2 ) coincide with respect to the first component but differ in the second,

Nonparametric k-sample test for equivalence

233

we have

(N ) + + σ(i1 ,j1 ),(i2 ,j2 ) = (N/ni ) P [Xi > Xj1 , Xi > Xj2 ] − πij π 1 ij2  

 = (N/ni ) Fj1 Fj2 dFi − Fj1 dFi Fj2 dFi for i = i1 = i2 ,

(7.28)

and so on for the other possible constellations with #({i1 , j1 } ∩ {i2 , j2 }) = 1. Obviously, all these expressions converge to well-defined limits, say σ(i1 ,j1 ),(i2 ,j2 ) , as N → ∞, provided the relative size ni /N of each sample converges to some nondegenerate limit λi ∈ (0, 1) as the total sample size grows to infinity. Furthermore, it follows from well-known results of the asymptotic theory of generalized U -statistics (cf. Lee, 1990, § 3.7.1) that the existence of nondegenerate √ limits for all ni /N ensures weak convergence of the normalized vector N (W + −π + ) of pairwise Mann-Whitney statistics to a centered (k(k − 1))-variate normal distribution with covariance matrix Σ = limN →∞ Σ(N ) . Hence, another application of the δ-method yields √  2  N d (W + , 1/2) − d2 (π + , 1/2)   L → N 0, σ 2 [d2 (W + , 1/2)] as N → ∞ (7.29) with

  σ 2 d2 (W + , 1/2) = 4(π + − 1/2)Σ(π + − 1/2) .

(7.30)

Finally, under some mild additional regularity conditions, each entry in the exact covariance matrix Σ(N ) can be consistently estimated by means of a suitable function of two-sample (→ diagonal elements) or two- and three-sample U -statistics (→ off-diagonal elements). In   (N ) view of σ(i1 ,j1 ),(i2 ,j2 ) → σ(i1 ,j1 ),(i2 ,j2 ) ∀ (i1 , j1 ), (i2 , j2 ) , these estimators are a fortiori consistent for the respective elements of the limiting covariance ma (N ) the matrix obtained by replacing each trix Σ. Thus, if we denote by Σ (N ) σ(i ,j ),(i ,j ) with its consistent estimator, the asymptotic variance (7.30) of √ 1 12 2 +2 N d (W , 1/2) can be consistently estimated by  2  2  (N ) (W + − 1/2) . d (W + , 1/2) = 4(W + − 1/2)Σ σ ˆN (7.31)

(N ) σ(i1 ,j1 ),(i2 ,j2 )

Hence, we have √  2  N d (W + , 1/2) − d2 (π + , 1/2) L   → N (0, 1) as N → ∞ σ ˆN d2 (W + , 1/2)

(7.32)

under any k-tuple (F1 , . . . , Fk ) of continuous cdf’s such that Σ is positive definite and π + = 1/2. Accordingly, an approximate level-α test for (7.23) can be based on the decision rule: Reject nonequivalence in the sense of d2 (π + , 1/2) ≥ ε˜2 iff   √ ˆN d2 (W + , 1/2)) / N . d2 (W + , 1/2) < ε˜2 − u1−α · σ

(7.33)

234

MULTISAMPLE TESTS FOR EQUIVALENCE

Example 7.1 (continued) To illustrate the nonparametric approach to multisample equivalence testing described above, we reanalyze the data set introduced in Example 7.1. With the entries in Table 7.1, the k(k − 1)/2 = 6 pairwise Mann-Whitney statistics are computed to be: + + + W12 = .52500, W13 = .49231, W14 = .57333; + + + W23 = .44231, W24 = .56667, W34 = .58974.

Thus, the result of estimating the squared distance between π + and 1/2 is d2 (W + , 1/2) = (.52500 − .5)2 + (.49231 − .5)2 + (.57333 − .5)2 + (.44231 − .5)2 + (.56667 − .5)2 + (.58974 − .5)2 = .021888 . √  (N ) of N W + , we obtain For the estimated exact covariance matrix Σ ⎛

ˆ (N ) N −1 Σ

⎞ .01392 .00704 .01112 −.00386 −.00650 .00000 ⎜ .00704 .01271 .00777 .01012 .00000 −.01026 ⎟ ⎜ ⎟ ⎜ .01112 .00777 .01487 .00000 .00371 .00214 ⎟ =⎜ ⎟ ⎜ −.00386 .01012 .00000 .01227 .00440 −.01142 ⎟ ⎝ ⎠ −.00650 .00000 .00371 .00440 .01195 .00276 .00000 −.01026 .00214 −.01142 .00276 .01257

 (N ) on the right-hand side Plugging in the observed values of W + and N −1 Σ 2 + of (7.31), the empirical standard error of d (W , 1/2) is found to be   √ σ ˆN d2 (W + , 1/2) / N =  ˆ (N ) ) 2 · (.02500, −.00769, .07333, −.05769, .06667, .08974)(N −1 Σ 1/2 (.02500, −.00769, .07333, −.05769, .06667, .08974) = .045094 . Fixing the significance level at the conventional value α = .05, there remains to specify the equivalence margin ε˜2 for d2 (π + , 1/2). For the two-sample case, |π+ − 1/2| < .20 has been proposed in § 1.7 as a reasonable criterion of equivalence. This motivates to choose ε˜ as the radius of the smallest sphere around 1/2 in (k(k − 1))-dimensional space which contains all points with every coordinate equal to 1/2 ± .20. This leads to the specification ε˜2 = (k(k − 1))/2 · .202 so that we use ε˜2 = 6 · .04 = .24  √in the present example. ˆN d2 (W + , 1/2) / N = .045094, the critical With α = .05, ε˜2 = .24 and σ upper bound which the observed value of d2 (W + , 1/2) has to be compared to according to (7.33), is .24 − 1.644854 · .045094 = .165827. Hence, the raw data shown in Table 7.1 fall in the rejection region of our nonparametric test at level α = .05 for equivalence of the distributions from which the k = 4 samples under analysis are taken.

8 Equivalence tests for multivariate data

All testing procedures considered in this chapter are primarily tailored for data taken from a k-variate normal distribution, with k denoting a fixed natural number ≥ 2. The methods discussed in § 8.1 can be viewed as repeated measurement analogues of the equivalence tests obtained in §§ 7.2–3 for the ANOVA one-way layout. The remaining sections are devoted to multivariate analogues of the two-sample t-tests for equivalence of § 6.1 and the double one-sided testing procedure, respectively.

8.1 8.1.1

Equivalence tests for several dependent samples from normal distributions Generalizing the paired t-test for equivalence by means of the T 2 -statistic

Regarding the basic structure of the data under analysis, the setting considered in this subsection differs from that of the paired t-test [→ § 5.3] only by the fact that the number k of conditions [treatments, timepoints at which some quantity is repeatedly measured, etc.] under which the observations are taken from each subject in a sample of size n , is no longer restricted to 2. The pattern of intraindividual correlations is left completely unspecified, but all distributions involved have to exhibit Gaussian form. More precisely speaking, we start from the assumption that the data set consists of n mutually independent vectors (X11 , . . . , Xk1 ), . . ., (X1n , . . . , Xkn ) such that   (X1i , . . . , Xki ) ∼ N (μ1 , . . . , μk ), Σ , ∀ i = 1, . . . , n , (8.1) where

⎛

σ12 ⎜ σ21 ⎜ Σ≡⎜ . ⎝ ..

σ12 σ22 .. .

... ... .. .

⎞ σ1k σ2k ⎟ ⎟ .. ⎟ . ⎠

(8.2)

σk1 σk2 . . . σk2 denotes some positive definite (symmetric) matrix of order k × k .

235

236

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

2 = (μ1 − μ2 )2 / A natural way of generalizing the distance measure δ 2 /σD 2 + σ2 − 2σ12 ) underlying the paired t-test with symmetric specification of the hypothetical equivalence range, to the k-variate setting (8.1) is to use the Mahalanobis distance of a set of k − 1 contrasts (δ1 , . . . , δk−1 ) in the μ s from the origin as the parametric function of interest. Of course, the covariance matrix ΣD , say, with respect to which this distance has to be taken, is that of the corresponding contrasts in the components of the random vectors primarily observed. For definiteness, we choose these contrasts as pairwise differences between successive components of the respective vectors, defining (8.3) δj = μj+1 − μj , j = 1, . . . , k − 1 .

(σ12

D Then, the entries in ΣD , say σjl , are given by D σjl = σjl + σj+1,l+1 − σj,l+1 − σj+1,l , 1 ≤ j, l ≤ k − 1 .

(8.4)

The equivalence hypothesis we want to establish specifies that the true vector δ = (δ1 , . . . , δk−1 ) of mean differences between “adjacent” components of the X1 , . . . , Xn is sufficiently close to 0, with distances between pairs (d1 , d2 ) of  points in IRk−1 measured in terms of (d1 − d2 )Σ−1 D (d1 − d2 ) . Accordingly, the equivalence testing problem we are now interested in reads  −1  2 2 H : δΣ−1 D δ ≥ ε versus K : δΣD δ < ε .

(8.5)

It is important to note that for any fixed μ = (μ1 , . . . , μk ) ∈ IRk , replacing the δj as defined in (8.3) by any other vector δ˜ of contrasts does not change   the value of δΣ−1 D δ . Actually, if we denote by C the matrix by which μ  is premultiplied in generating the vector δ with components (8.3), and let ˜ be any other (k − 1) × k-matrix of rank k − 1 with C ˜ (1, 1, . . . , 1) = 0, C ˜  = C  A for elementary facts from linear algebra imply that there holds C some nonsingular (k − 1) × (k − 1)-matrix A . In view of this, both δ˜ and ˜ k−1 ) and D = (D1 , . . . , Dk−1 ) ˜ 1, . . . , D ˜ = (D δ, and the random vectors D whose expectations are given by the former, are related to each other through the transformation V = U A which induces the relationship ΣD˜ = A ΣD A ˜ and D . Making use of the identities between the covariance matrices of D  ˜ δ = δA , ΣD˜ = A ΣD A, only a little bit more algebra is required to show ˜ −1 δ˜ = δΣ−1 δ  indeed. that there holds δΣ ˜ D D An optimal test for the equivalence problem (8.5) can be based on the empir ical counterpart of the theoretical Mahalanobis distance δΣ−1 D δ as obtained by replacing δ and ΣD with ¯ ≡ n−1 D

n  i=1

and

(D1i , . . . , Dk−1,i )

(8.6)

Equivalence tests for several dependent samples from normal distributions 237 S D ≡ (n − 1)

−1

n 

¯ 1 , . . . , Dk−1,i − D ¯ k−1 ) (D1i − D

i=1

¯ 1 , . . . , Dk−1,i − D ¯ k−1 ) , (D1i − D

(8.7)

respectively. A well-known result from the classical theory of parametric multivariate inference (see, e.g., Anderson, 1984, § 5.4) states that the distribution of ˜ ¯ −1 D T 2 ≡ nDS (8.8) D  depends on δ and ΣD only through the actual value, say τ 2 , of δΣ−1 D δ , in that we have   d (8.9) T 2 (n − k + 1)/ (n − 1)(k − 1) = Fk−1,n−k+1 (nτ 2 ) ,

where Fν1 ,ν2 (ψ 2 ) stands for a random variable following an F -distribution with ν1 , ν2 degrees of freedom and noncentrality-parameter ψ 2 . Consequently, an exact level-α test for (8.5) consists of checking the observed data for inclusion in the critical region  2    T < (n − 1)(k − 1)/(n − k + 1) Fk−1,n−k+1; α (nε2 ) (8.10) with Fν1 ,ν2 ; α (ψ 2 ) denoting the lower 100α percentage point of the distribution of Fν1 ,ν2 (ψ 2 ) for any (ν1 , ν2 ) ∈ IN2 and ψ 2 ≥ 0 . Table 8.1 shows the result of computing the critical upper bounds to the statistic T 2 of (8.8) for three choices of the equivalence margin, five different values of the number k of conditions under comparison, and the same range of sample sizes n which is covered by Table 5.11 for the paired-sample setting. Omission of the bivariate case from the current table has been done with a view to the fact that for k = 2, we simply would have to tabulate the squares of the respective entries in the former. The power function of the T 2 -test for equivalence of k normal distributions to be analyzed on the basis of dependent samples can likewise be evaluated exactly by means of easily accessible computational tools. Actually, denoting the rejection probability of the test based on (8.10) under any parameter  2 2 configuration (δ , ΣD ) with δΣ−1 D δ = τ ∈ [0, ∞) by β(τ ), we can write   (8.11) β(τ 2 ) = P Fk−1,n−k+1 (nτ 2 ) ≤ Fk−1,n−k+1; α (nε2 ) . Against null alternatives specifying that the true vector δ of mean differences between the k treatments is zero, the power of the test turns out to be as shown in Table 8.2 which is otherwise organized in the same way as the preceding table. According to intuition, one would expect that, given everything else, the chance of establishing equivalence of the k marginal distributions when they are in fact identical in terms of their means, decreases with the number of treatment conditions under comparison. The exact numerical results obtained by means of formula (8.11) clearly confirm the correctness of this assumption.

238

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

2;k−1 2 Table 8.1 Critical upper bound T.05; n−1 (ε) to the T -statistic to be used when testing for equivalence of k normal distributions with dependent samples of size n ∈ {10, 20, . . . , 100}, for α = 5% and three different specifications of the equivalence margin ε to the root of the Mahalanobis distance of the vector of mean differences from the origin.

n

ε 3

10 10 10 20 20 20 30 30 30 40 40 40 50 50 50 60 60 60 70 70 70 80 80 80 90 90 90 100 100 100

0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00

0.15846 0.38473 3.05419 0.20122 0.96156 8.43408 0.26599 1.87865 14.74092 0.35166 3.01525 21.56257 0.46040 4.29730 28.72676 0.59379 5.68531 36.14076 0.75213 7.15530 43.74764 0.93439 8.69127 51.50940 1.13865 10.28185 59.39914 1.36262 11.91862 67.39698

4 0.53337 0.95198 4.17579 0.58067 1.63675 9.63571 0.68338 2.62803 16.01615 0.81262 3.81160 22.89350 0.96453 5.12787 30.10126 1.13775 6.54262 37.55055 1.33113 8.03435 45.18680 1.54332 9.58850 52.97355 1.77283 11.19460 60.88491 2.01813 12.84485 68.90172

k = 5 1.13575 1.74680 5.59033 1.10236 2.40659 10.96366 1.21517 3.43654 17.37638 1.36781 4.65233 24.28965 1.54568 5.99437 31.52912 1.74411 7.43024 39.00570 1.96069 8.93969 46.66554 2.19369 10.50898 54.47287 2.44165 12.12821 62.40237 2.70326 13.79002 70.43534

6 2.01545 2.83976 7.44170 1.74900 3.28112 12.43980 1.83910 4.30964 18.83065 1.99705 5.54054 25.75604 2.18794 6.89877 33.01357 2.40155 8.34955 40.50849 2.63342 9.87233 48.18555 2.88098 11.45348 56.00867 3.14247 13.08332 63.95258 3.41649 14.75464 71.99868

8 5.27403 6.73052 13.80923 3.42816 5.40902 15.95362 3.33994 6.27486 22.06458 3.45082 7.47309 28.92250 3.63237 8.82984 36.16660 3.85127 10.28915 43.66685 4.09472 11.82388 51.35659 4.35653 13.41794 59.19524 4.63317 15.06066 67.15558 4.92238 16.74442 75.21811

Equivalence tests for several dependent samples from normal distributions 239 Table 8.2 Power against the alternative μ1 = . . . = μk attained in the T 2 -test (8.10) when using the critical constants shown in Table 8.1 for the k-sample setting with dependent observations. n 10 10 10 20 20 20 30 30 30 40 40 40 50 50 50 60 60 60 70 70 70 80 80 80 90 90 90 100 100 100

ε 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00

3

4

k = 5

6

8

.06743 .15417 .68924 .09046 .35874 .96334 .11999 .58471 .99683 .15680 .75706 .99977 .20104 .86702 .99998 .25204 .93060 1.0000 .30831 .96508 1.0000 .36782 .98293 1.0000 .42842 .99186 1.0000 .48820 .99619 1.0000

.06604 .13883 .58339 .08695 .30494 .93324 .11286 .50349 .99289 .14413 .67871 .99939 .18081 .80707 .99995 .22249 .89061 1.0000 .26842 .94072 1.0000 .31758 .96903 1.0000 .36878 .98430 1.0000 .42082 .99224 1.0000

.06455 .12626 .49550 .08373 .26842 .89735 .10690 .44562 .98688 .13437 .61614 .99870 .16617 .75332 .99989 .20207 .85081 .99999 .24164 .91412 1.0000 .28424 .95255 1.0000 .32912 .97467 1.0000 .37547 .98687 1.0000

.06310 .11562 .41991 .08093 .24119 .85710 .10202 .40136 .97853 .12672 .56469 .99758 .15505 .70532 .99977 .18691 .81230 .99998 .22201 .88641 1.0000 .25994 .93418 1.0000 .30017 .96324 1.0000 .34211 .98011 1.0000

.06020 .09729 .29101 .07634 .20192 .76799 .09450 .33666 .95422 .11536 .48430 .99350 .13902 .62377 .99926 .16546 .74088 .99993 .19456 .83042 .99999 .22610 .89388 1.0000 .25979 .93616 1.0000 .29531 .96290 1.0000

˜ −1 δ˜ = δΣ−1 δ  Obviously, the argument behind the fact that δ˜ = δA ⇒ δΣ ˜ D D for any nonsingular (k − 1) × (k − 1)-matrix A, likewise applies when δ and −1 ¯ Σ−1 D are replaced with their empirical counterparts D and S D . Thus, the 2 statistic T of (8.8) is left invariant under arbitrary linear one-to-one transformations of the sample contrast-vectors D i , and can even be shown (see, e.g., Lehmann, 1986, § 8.2) to be maximal invariant with respect to this group

240

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

of transformations. Furthermore, in view of (8.10) and the maximal invariance of T 2 , the same arguments which lead in § 7.2 to the conclusion that the ANOVA F -test for equivalence of k normal distributions from which independent samples are given, can be used to infer that the test given by (8.10) maximizes the power uniformly among all invariant level-α tests for (8.5).

Example 8.1 In a study of the effects of deprivation on voluntary alcohol intake in alcoholpreferring rat lines, one of the objectives was to show that in some of the lines included in the experiments, the total intake of ethanol [g/kg/day] after the deprivation phase remained roughly constant over time. Measurements were taken on k = 4 consecutive days after the animals had been relocated to a setting providing free access to alcohol drinking solutions. Table 8.3 shows the arithmetic means and the empirical covariance matrix calculated from n = 20 data vectors. Table 8.3 Summary statistics for a sample of n = 20 repeated measurements of total ethanol intake [g/kg/day] taken at k = 4 consecutive days in rats of an alcohol-preferring line after deprivation. Day j = ¯j X X

S1j X S2j X S3j X S4j

1

2

3

4

5.0967989

5.0942015

5.2835529

5.1851779

1.521976271 1.710873546 0.881500432 0.541358142

0.549312992 −0.260377815 0.881500432 0.541358142 1.106519118 1.230386226 1.230386226 2.313748226

2.095013962 1.521976271 0.549312992 −0.260377815

¯ and the empirical coThe elements of the estimated contrast vector D variance matrix S D of D are easily obtained by replacing the population parameters appearing in Equations (8.3) and (8.4) with the respective entries in Table 8.3, leading to the values displayed in Table 8.4. Finally, as soon as ¯ and S D have been calculated, evaluating the quadratic form DS ˜  re¯ −1 D D D quires to write down just a single assignment statement in any matrix-oriented programming environment like SAS/IML or R. For the test statistic itself, we obtain T 2 = 1.7386013 which is slightly larger than the critical upper bound we read from Table 8.1 for n = 20, ε = .50 and k = 4 . Thus, specifying the equivalence margin to the squared Mahalanobis distance of the true value of δ from the origin to be .502 , we have to accept the null hypothesis of nonequivalence in the present case. Keeping in mind, that, according to the numerical

Equivalence tests for several dependent samples from normal distributions 241 Table 8.4 Estimated mean differences and variancescovariances of intraindividual differences between successive time points obtained with the data shown in Table 8.3.

1 ¯j D D S1j D S2j D S3j

−0.0025970

j= 2

3

0.1893514 −0.0983750

0.7619350 0.1432902 0.1432902 1.0543918 0.4695485 0.4640094

0.4695485 0.4640094 0.9594949

material shown in Table 8.2, the maximum power of the test is only slightly larger than 30% under the conditions of this example, it is not surprising that the decision eventually to be taken turns out negative.

8.1.2

A many-one equivalence test for dependent samples based on Euclidean distances

The distance measure used in the preceding subsection as a basis for constructing an equivalence test for several dependent samples from distributions of Gaussian form might be criticized for leading to equivalence regions whose geometrical shape strongly depends on the underlying covariance structure. The potential difficulties entailed with this fact are best clarified by visualizing the theoretical equivalence region corresponding to the alternative hypothesis D D = σ22 of both K of (8.5) in the bivariate case with identical variances σ11 components of the vector of intraindividual differences, for various values of the theoretical correlation coefficient D . As can be seen from Figure 8.1, it is a circle around zero for D = 0, but an ellipse whose major diameter increases to infinity as D → 1 whereas in the direction of the minor axis, the diameter remains bounded above by 2ε for any value of D . In this subsection, we keep the shape of the theoretical equivalence region independent of the covariance structure of D = (D1 , . . . , Dk−1 ) specifying D ) of under the alternative hypothesis that the vector (δ1 /σ1D , . . . , δk−1 /σk−1 standardized expected values has Euclidean distance < ε from the origin for some fixed ε > 0 . Of course, this means, that the hypothetical equivalence region is a sphere of radius ε around 0 in (k − 1)-dimensional space. Given the expected value μ = (μ1 , . . . , μk ) of the primarily observed vector X = k−1  δj /σjD )2 clearly depends on the choice of (X1 , . . . , Xk ), the value of j=1 the contrast matrix used for reducing X to the (k − 1)-dimensional vector D of intraindividual differences. Hence, the testing procedure to be derived in the sequel is tailored for settings where the contrast of interest is uniquely determined from the context. The prototype of such a setting occurs in any

242

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

2

1.5

1

0.5

0

−0.5

−1

−1.5

−2 −2

−1.5

−1

−0.5

0

0.5

1

1.5

2

Figure 8.1 Shape of the equivalence region in terms of Mahalanobis distance in the bivariate case with equal marginal variances and ε2 = .25, for three different values of the correlation coefficient D between the intraindividual differences. trial involving k ≥ 2 treatments of which one (say that indexed by j = 1) serves as a control for all others so that the only set of contrasts to be taken in consideration is given by δj = μj+1 − μ1 , j = 1, . . . , k − 1 .

(8.12)

For the corresponding transforms of observed variables, we likewise use the same symbols as in the preceding subsection redefining Dj and Dji as Dj = Xj+1 − X1 , j = 1, . . . , k − 1 ,

(8.13)

Dji = Xj+1,i − X1i , j = 1, . . . , k − 1 , i = 1 . . . , n ,

(8.14)

and

Equivalence tests for several dependent samples from normal distributions 243 respectively. Another price (additionally to having to dispense with invariance against relabeling the treatments under comparison) we will have to pay for replacing Mahalanobis with ordinary Euclidean distance is that the problem of testing H:

k−1 

 δj /σjD )2 ≥ ε2

versus

k−1 

K:

j=1

 δj /σjD )2 < ε2

(8.15)

j=1

can only be solved by means of asymptotic methods. A suitable starting point for the asymptotic construction is obtained by introducing the plug-in ˆ D , of the squared length of the vector (δ1 /σ1D , . . . , δk−1 /σ D ) estimator, say Q k−1 of standardized intraindividual mean differences. Defining D )≡ g(δ1 , σ1D , . . . , δk−1 , σk−1

k−1 

 δj /σjD )2 ,

(8.16)

j=1

ˆ D admits the representation Q D ˆ D = g(D ˆ 1 , S1D , . . . , D ˆ k−1 , Sk−1 Q ),

(8.17)

where ˆ j = n−1 D

n  i=1

n 1/2   ˆ j )2 Dji , SjD = (n − 1)−1 (Dji − D

(8.18)

i=1

for each j = 1, . . . , k − 1 . Now the major task is to derive the asymptotic distribution of the statistic ˆ D by means of the multivariate δ-method as described, e.g., in Bishop et al. Q (1975, § 14.6.3). In order to make the details precise, it is convenient to D = τjl for all 1 ≤ j, l ≤ p. simplify notation a bit by setting k − 1 = p and σjl Then, well-known results from the distribution theory for sample means and variances/covariances under the multivariate normal model (see in particular Anderson, 1984, Theorem 3.4.4) imply that  L √  D D ˆ 1 , S1D , . . . , D ˆ k−1 , Sk−1 n (D ) − (δ1 , σ1D , . . . , δk−1 , σk−1 ) → N (0, Σ∗ ) (8.19) as n → ∞ , with ⎛ 2 τ1 0 τ12 0 ... ⎜ 0 2 2 τ /2 0 τ /(2τ τ ) ⎜ 1 2 ... 1 12 ⎜ 2 ⎜ τ12 0 τ2 0 ... ⎜ ⎜ 2 2 0 τ12 /(2τ1 τ2 ) 0 τ2 /2 ... Σ∗ = ⎜ ⎜ ⎜ .. .. .. .. .. ⎜ . . . . . ⎜ ⎜ 0 τ2p 0 ... ⎝ τ1p 2 2 0 τ1p /(2τ1 τp ) 0 τ2p /(2τ2 τp ) . . .

⎞ τ1p 0 2 0 τ1p /(2τ1 τp ) ⎟ ⎟ ⎟ ⎟ τ2p 0 ⎟ 2 0 τ2p /(2τ2 τp ) ⎟ ⎟. ⎟ ⎟ .. .. ⎟ . . ⎟ ⎟ 2 τp 0 ⎠ 2 0 τp /2

(8.20)

244

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

Furthermore, the function g( · ) obviously has a differential anywhere in IR × IR+ × . . . × IR × IR+ , and the components of the gradient vector at any point (δ1 , τ1 , . . . , δp , τp ) in this space are easily calculated to be 2δj2 ∂g 2δj ∂g = 2 , =− 3 , ∂δj τj ∂τj τj

j = 1, . . . , p .

(8.21)

2 ˆ In order to determine √ the asymptotic variance, say σa [QD ], of the test statistic ˆ QD multiplied by n as a normalizing factor under an abritarily selected parameter configuration, we have to evaluate the quadratic form x → xΣ∗ x ∂g ∂g ∂g ∂g . With the explicit expressions appearing at x = , ,..., , ∂δ1 ∂τ1 ∂δp ∂τp in (8.20) and (8.21), this gives after suitable rearrangements of terms

ˆD] σa2 [Q

=

p p  

   2δj δl τjl /(τj2 τl2 ) 2 + δj δl τjl /(τj2 τl2 )

j=1 l=1

≡

k−1  k−1 

 D   D D D 2δj δl σjl /(σjj σllD ) 2 + δj δl σjl /(σjj σllD ) .

(8.22)

j=1 l=1

The reason why this formula is worth being presented in the present context is that the theorem underlying the δ-method implies that there holds √

L

ˆ D − QD )/σa [Q ˆ D ] → N (0, 1) , n(Q

(8.23)

with QD standing for the parametric function (8.16) of interest. Moreover, the ˆ D ] of the normalized test statistic can be consistently asymptotic variance σa2 [Q D ¯ j and S D ≡ and (n−1)−1 n (Dji − with D estimated by replacing δj and σjl jl i=1 ¯ j )(Dli − D ¯ l ), respectively, in (8.22) for all 1 ≤ j, l ≤ k − 1 . The explicit D formula for this plug-in estimator reads ˆD] = σ ˆa2 [Q

k−1  k−1 

 D   D D D ¯ l Sjl ¯ l Sjl ¯jD ¯jD 2D /(Sjj SllD ) 2 + D /(Sjj SllD ) ,

(8.24)

j=1 l=1

analogously to (8.22). Since the alternative hypothesis under consideration specifies that the true value of QD is sufficiently small, the one-sided rejection region to be used according to (2.6) in an asymptotically valid test for noninferiority has to be replaced with a region containing all points in the sample space for which the normalized test statistic centered about the equivalence margin ε2 falls below the lower α standard normal quantile. Thus, a largesample solution to the testing problem (8.15) rejects if and only if it turns out that  √  ˆ D − ε2 /ˆ ˆ D ] < uα . σa [Q n Q (8.25)

Equivalence tests for several dependent samples from normal distributions 245

Example 8.1 (continued) For a reanalysis of the data set underlying Table 8.3 by means of the test given by (8.25), we reduce all vectors primarily observed to intraindividual changes against the value measured on Day 1 [cf. (8.14)]. The means and empirical variances/covariances obtained with these recalculated difference vectors are ˆ D is easily computed shown in Table 8.5. With these values, the test statistic Q Table 8.5 Estimated mean differences and variancescovariances of intraindividual changes against the value measured on Day 1 as baseline for the data shown in Table 8.3.

1 ¯j D D S1j D S2j D S3j

−0.002597

j= 2 0.186754

3 0.088379

0.7619350 0.9052251 1.3747736 0.9052251 2.1029071 3.0364650 1.3747736 3.0364650 4.9295178

ˆ D = 0.0181785. For the estimated standard error of √nQ ˆ D , we find to be Q ˆ D ] = 0.3304173. Using through plugging-in into formula (8.24) that σ ˆa [Q the same specification of the equivalence margin as before, the left-hand side of (8.25) turns out to be −3.13766 which is distinctly smaller than u.05 = −1.64485. So, this time the decision is in favor of equivalence. As is the case for virtually all asymptotic testing procedures, a question of considerable importance is about its (maximum) rejection probability under the null hypothesis when used with samples of small or moderate size. In order to provide sufficient insight into this issue, another set of simulation experiments has been performed covering fairly different correlation structures D ) and distributions of the total squared distance ε2 of (δ1 /σ1D , . . . , δk−1 /σk−1 from the origin over the components of this vector. (Varying the parameter configuration by looking at different specifications of the δj and σjD leading to the same pattern of standardized expected differences is unnecessary since both the hypotheses and the testing procedure are obviously scale invariant). Table 8.7 shows the results of such a simulation study for dimension p = 4 of the vector of intraindividual observations to be assessed and the same specifications of the equivalence margin and the nominal significance level made in the example, and the four correlation matrices specified in Table 8.6.

246

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

A glance over the rejection probabilities obtained with samples generated under the null hypothesis clearly reveals that the accuracy of the large-sample approximation upon which the test (8.25) relies, is acceptable, if at all, only for an equi-correlation structure of (D1 , . . . , Dk−1 ) with low common value of the off-diagonal entries. Again [recall §§ 5.2.2, 5.2.3, 6.6.3, 6.6.6], a straightforward way of protecting oneself from grossly exceeding the target significance level is to reduce the nominal level as far as necessary. Tentatively, such a reduced nominal level α∗ can be determined through simulation under the parameter configuration which, according to the results presented in Table 8.7, came out least favorable (in the sense of yielding the largest rejection probability at the boundary of the null hypothesis) for the respective sample size. In terms of α − α∗ , the extent of the required corrections is huge: For the sample sizes covered by Table 8.7, we obtained the reduced levels 2.5 · 10−7 [→ n = 25], 10−6 [→ n = 50], 10−3 [→ n = 100], and .014 [→ n = 400], respectively. The power results shown in Table 8.8 hold for the test when performed at these corrected nominal levels and have also been computed by simulation. As can be expected in view of the behavior of the power function on the common boundary of H and K, the rejection probability strongly depends on the correlation structure also under null alternatives. Reducing the nominal significance level as much as necessary for keeping the maximum rejection probability under the null hypothesis below 5% also affects the qualitative result of the test in the above example: For n = 20, a slightly less extreme choice of α∗ (as compared to n = 25) turns out to work, namely α∗ = 2.5·10−6 . Accordingly, the observed value of the standardized test statistic has actually to −4.56479 rather than −1.64485. Since we √  ˆ to be2 compared ˆ D ] = −3.13766, the level-corrected test for equivσa [Q obtained n Q D −ε /ˆ k−1  alence in the sense of j=1 δj /σjD )2 < ε2 likewise fails to allow rejection of nonequivalence, as did the exact test for equivalence in terms of Mahalanobis distance. Table 8.6 Correlation matrices used for generating the data in the simulation experiments reported in Table 8.7. Correlation Matrix # (r) (r) (r) (r) 12 13 14 1 2 3 4

.25 .90 .25 −.65

.25 .90 .50 .35

.25 .90 .95 .45

23 24

(r)

(r)

34

(r)

.25 .90 .05 .35

.25 .90 .25 .35

.25 .90 .65 .95

Equivalence tests for several dependent samples from normal distributions 247 Table 8.7 Simulated rejection probabilities of the test (8.25) at the boundary of the hypotheses (8.15) for the correlation structures (1)–(4) of Table 8.6 and the following patterns of contributions of the components of (δ1 /τ1 , . . . , δp /τp ) to the assumed value ε2 = .25 of QD : (a) (δj /τj )2 = ε2 /p ∀j = 1, . . . , p; (b) (δ1 /τ1 )2 = ε2 , (δj /τj )2 = 0 ∀j = 2, . . . , p; (c) (δj /τj )2 = j ε2 /(p(p + 1)/2) ∀j = 1, . . . , p [ p = 4, ε = .50, α = .05 ; 100,000 replications per Monte Carlo experiment].

n

Correlation Matrix # Pattern of Standardized δ’s (r) (a) (b) (c)

25 " " "

1 2 3 4

.04977 .14779 .07452 .06394

.02211 .02957 .02133 .03295

.04759 .14378 .07411 .06983

50 " " "

1 2 3 4

.06802 .14659 .09218 .08176

.03490 .05104 .04576 .05369

.06496 .14670 .09385 .08441

100 " " "

1 2 3 4

.06667 .11849 .08691 .07789

.04424 .07822 .06441 .06286

.06346 .11954 .08709 .07891

400 " " "

1 2 3 4

.05873 .07982 .06778 .06663

.05071 .09123 .06758 .05921

.05731 .08087 .06879 .06569

Table 8.8 Simulated power of the level-corrected test (8.25) against the alternatives obtained by setting δj = 0 ∀ j = 1, . . . , p in the constellations covered by Table 8.7. n 25 50 100 400

Correlation Matrix # 1 2 3 0.01745 0.11521 0.03730 0.16601 0.25811 0.19388 0.84795 0.64818 0.75595 1.00000 0.99954 1.00000

(r) 4 0.04463 0.17744 0.72903 1.00000

248

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

8.1.3

Testing for equivalence with dependent samples and an indifference zone of rectangular shape

Another reasonable way of specifying under the alternative hypothesis to be established, an equivalence region whose shape does not depend on the pattern of associations between the components of the primary data vectors, is to require that for all pairs (j, l) with 1 ≤ j < l ≤ k, there holds |μj − μl | < εjl for suitable equivalence margins εjl > 0. The latter may be chosen as fixed constants involving no other unknown population parameters, or as multiples of the theoretical standard deviations, say σ(j,l) , of the differences Xj − Xl between the corresponding components of a random vector from the k-dimensional distribution under assessment. In both cases, the indifference zone making up the theoretical equivalence region, corresponds to  a rectangle in the k(k − 1)/2 -dimensional parameter space of all pairwise differences of expected values. We start with a consideration of the case that interest is on establishing equivalence in terms of standardized differences between the marginal means. The testing problem to be treated then, reads H : |μj − μl |/σ(j,l) ≥ ε for at least one (j, l) ∈ {1, . . . , k}2 with j < l versus K : |μj − μl |/σ(j,l) < ε ∀ 1 ≤ j < l ≤ k ,

(8.26)

2 with arbitrarily fixed ε > 0 . Of course, the variances σ(j,l) are related to the entries in the covariance matrix Σ of the primarily observed vectors (X11 , . . . , Xk1 ), . . ., (X1n , . . . , Xkn ) [recall (8.2)] by 2 σ(j,l) = σj2 + σl2 − 2σjl .

(8.27)

A straightforward solution to the problem (8.26) is obtained by applying the intersection union principle of § 7.1 with q = k(k − 1)/2 and paired t-tests for equivalence as building blocks. This yields the decision rule: Reject H in favor of K if and only if   √ ¯j − X ¯ l |/S(j,l) < F1,n−1; α (nε2 ) 1/2 n|X

∀ 1 ≤ j < l ≤ k,

(8.28)

with 2 S(j,l) = Sj2 + Sl2 − 2Sjl = (n − 1)−1

n     ¯j − X ¯ l ) 2 . (8.29) (Xji − Xli ) − (X i=1

From general facts stated in § 7.1 about intersection-union tests for problems in which the alternative hypothesis specifies that several elementary hypotheses hold true simultaneously, it follows that the procedure defined by (8.28) is exactly valid, in the sense that the type-I error risk is less than or equal to α under any parameter constellation belonging to H. Except for k = 2, exact power computations are largely impracticable even under null

Equivalence tests for several dependent samples from normal distributions 249 alternatives assuming μj − μl = 0 ∀ (j, l) . Table 8.10 shows, for the case of observations of dimension k = 5, the simulated power attained against special alternatives of that kind distinguished by 5 different correlation structures (including pairwise independence) as specified in Table 8.9, and equal marginal variances. The fact that for the first three correlation structures (labeled 0 through 2), the results are identical except for some slight variation due to the simulation error, may seem surprising at first sight. Mathematically, it is a consequence of a classical result (stated, e.g., in § 5.3 of Kotz et al., 2000)

Table 8.9 Correlation matrices used for generating 5-dimensional data vectors in studying the power of the union-intersection test given by (8.28). Correlation Matrix # (r) (r) (r) (r) (r) 12 13 14 15 0 1 2 3 4

.00 .25 .90 .15 −.65

.00 .25 .90 .25 .35

.00 .25 .90 .50 .45

.00 .25 .90 .95 .10

(r)

(r)

23 24 .00 .25 .90 .55 .35

(r)

25

.00 .00 .25 .25 .90 .90 .25 .25 .35 −.20

34 35

(r)

(r)

45

.00 .25 .90 .45 .95

.00 .25 .90 .25 .20

.00 .25 .90 .70 .10

Table 8.10 Simulated power of the intersection-union test (8.28) at nominal level α = .05 against null alternatives specifying μj − μl = 0 ∀ (j, l) for 5-dimensional observations, equivalence margin ε = .50, homogeneous marginal variances, and the correlation structures listed in Table 8.9 [100,000 replications per Monte Carlo experiment]. n 10 20 30 40 50 60 70 80 90 100

0 .00048 .02377 .17186 .41566 .63755 .79569 .89525 .94899 .97572 .98874

Correlation Matrix # (r) 1 2 3 .00048 .00051 .00132 .02400 .02399 .05045 .17000 .16985 .25610 .41294 .41413 .50967 .63892 .63935 .71179 .79808 .79959 .84440 .89424 .89521 .92058 .94868 .94795 .96149 .97550 .97541 .98122 .98930 .98885 .99120

4 .00081 .03756 .23052 .49344 .70018 .84135 .91893 .96157 .98113 .99150

(r)

250

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

on multivariate normal random variables with equi-correlated components: If, for each i = 1, . . . , n and 1 ≤ j < l ≤ k, there holds μj = μl = μ, σj = σl = σ0 and jl =  for some μ ∈ IR, σ0 > 0 and 0 ≤  < 1, then we √ d have Xji − Xli = σ0 1 −  (Zji − Zli ) where all Z’s are standard normally distributed and mutually independent. Clearly, this implies, that the joint   distribution of the k(k − 1)/2 -vector of pairwise t-statistics obtained from the sample (X11 , . . . , Xk1 ), . . ., (X1n , . . . , Xkn ) is the same as that computed from the unit normal vectors (Z1i , . . . , Zki ), i = 1, . . . , n, and thus does not depend on . Measuring the distance between pairs of marginal distributions in terms of nonscaled differences of their expected values leads to reformulating the testing problem (8.26) through dropping the standard deviations σ(j,l) . The resulting pair of hypotheses can be written in a more compact way as a couple of statements about the range among the population means μ1 , . . . , μk . Actually, when we define μ(1) ≡ min1≤j≤k μj , μ(k) ≡ max1≤j≤k μj and replace σ(j,l) with unity for all (j, l), (8.26) simplifies to H : μ(k) − μ(1) ≥ ε

versus

K : μ(k) − μ(1) < ε .

(8.30)

In this nonscaled version, the testing problem is likewise easy to solve through applying the intersection-union principle. The natural choice of the elementary tests for equivalence of pairs of marginal distributions is now the paired-samples version of the two one-sided t-tests procedure, or, equivalently, the interval inclusion test with central t-based confidence bounds to the respective mean within-subject differences. After suitable rearrangements of the double inequalities defining each of these k(k − 1)/2 elementary tests, the compound decision rule of the intersection-union procedure can be written: Reject H in favor of K if and only if  √  ¯ l | < ε − S(j,l) / n tn−1; 1−α ¯j − X |X

∀ 1 ≤ j < l ≤ k,

(8.31)

where tn−1; 1−α stands for the (1 − α)-quantile of the central t-distribution with n − 1 degrees of freedom, and all other symbols have the same meaning as in (8.28–9). In designing a simulation study providing useful insights into the behavior of the power function of this alternative testing procedure following the intersection-union principle, it must be kept in mind that, in contrast to the procedure using paired t-tests as building blocks, (8.31) is not invariant under rescaling the primary observations and their components. This is the reason why Table 8.11 shows the result of determining by simulation the power of the procedure against null alternatives, which differ not only with respect to the correlation structure but also the assumed common value of the marginal variances.

Equivalence tests for several dependent samples from normal distributions 251 Table 8.11 Simulated power of the intersection-union test (8.31) at nominal level α = .05 against null alternatives specifying μj − μl = 0 ∀ (j, l) for 5-dimensional observations, equivalence margin ε = .50 to the nonscaled range among the μj , different values of the standard deviation σ0 assumed to be common to all components of the observed random vector, and the correlation structures listed in Table 8.9 [100,000 replications per Monte Carlo experiment].

n

Correlation Matrix # (r)

0.70

0.85

25 " " " "

0 1 2 3 4

0.06078 0.22542 1.00000 0.46092 0.10677

0.00279 0.03192 0.99997 0.14465 0.00990

50 " " " "

0 1 2 3 4

0.66422 0.88827 1.00000 0.94622 0.66700

75 " " " "

0 1 2 3 4

100 " " " "

0 1 2 3 4

σ0 = 1.00

1.15

1.30

0.00002 0.00186 0.99852 0.02476 0.00036

0.00000 0.00002 0.98216 0.00229 0.00001

0.00000 0.00000 0.91410 0.00014 0.00000

0.27602 0.56387 1.00000 0.75659 0.33531

0.06867 0.24125 1.00000 0.47051 0.10851

0.00969 0.07193 0.99999 0.22677 0.01715

0.00054 0.01405 0.99991 0.08033 0.00107

0.94409 0.99317 1.00000 0.99603 0.90407

0.69393 0.90122 1.00000 0.95208 0.68614

0.36030 0.65685 1.00000 0.81519 0.41757

0.13813 0.36769 1.00000 0.59470 0.19045

0.03739 0.16435 1.00000 0.37301 0.05799

0.99285 0.99961 1.00000 0.99981 0.97371

0.90648 0.98548 1.00000 0.99212 0.86438

0.66334 0.88590 1.00000 0.94558 0.66086

0.37274 0.67308 1.00000 0.82536 0.42964

0.16575 0.41991 1.00000 0.63899 0.22348

Even when the common standard deviation σ0 of the components of the observed random vectors is set equal to unity, the power attained in the test (8.31) with some given sample size n turns out to be very sensitive to changes in the correlation structure. Through comparison with the entries in the previous table, it becomes evident that this property distinguishes the nonscaled version of the intersection-union test quite sharply from its counterpart (8.28) tailored for establishing equivalence with respect to the standardized differences of means. The most marked changes occur under transition from cor-

252

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

relation structure 0 [independence] to 2 [equal correlations of common value  = .90]. Taking an asymptotic point of view, that part of the phenomenon is fairly easy to understand: For jl =  ∀ (j, l) and σ1 = . . . = σk = 1, each of the elementary tests making up the procedure (8.31) isasymptoti√ ¯j − X ¯ l |/ 2(1 − ) < n |X cally equivalent to the test with rejection region  √  nε 2(1 − ) − u1−α [cf. p. 64]. Consequently, increasing the common correlation coefficient from zero to some nonzero value  < 1 has on its asymp¯j − X ¯ l ) = 0 the same effect as increasing totic power against μj − μl = E(X the equivalence margin by the factor (1−)−1/2 . Specifically, for ε = 0.50 and  = .90, this would amount to increasing the equivalence margin to ≈ 1.58. The difference to .50 intuitively explains the huge increase in power of the compound testing procedure as a whole.

8.1.4

Discussion

None of the testing procedures made available in this subsection for establishing equivalence of several normal distributions from which dependent samples are taken, seems fully satisfactory. The test based on Hotelling’s T 2 has the advantage of guaranteeing exact coincidence of the size of its critical region with the prespecified nominal significance level α. Furthermore, it satisfies a fairly strong optimality criterion (UMPI), and its power against null alternatives does not depend in any way on the covariance matrix. Unfortunately, the importance of these desirable properties has to be considerably relativized taking into account the fact [illustrated by Figure 8.1] that the theoretical equivalence region which the true parameter configuration can be said to belong to in the case of a positive decision, highly depends both in shape and geometric content on the correlation structure. Not a few users of the methods under discussion might feel unable to see why this is reasonable. The pros and cons of the procedure derived in § 8.1.2 are largely dual to those of the exact test for equivalence as defined in terms of Mahalanobis distance of the vector of expected pairwise intrasubject differences from zero. Measuring the distance of a vector of real numbers from the origin through Euclidean metric has a long tradition in many scientific areas, and the shape of the resulting region in the space of standardized expected values is spherical for any underlying correlation structure. However, the testing procedure obtained for the equivalence problem formulated in terms of Euclidean distances relies on first-order asymptotics, and convergence of the rejection probability under various parameter configurations falling on the boundary of both hypotheses to the prespecified significance level turned out to be extremely slow. Even apart from the problem of performing a test of maybe considerably increased size which can be addressed by reducing the nominal level, the power against null alternatives is fairly sensitive against changes in the correlation structure so that sample size determination requires much more

Multivariate two-sample tests for equivalence

253

detailed knowledge about the true parameter configuration as compared to the exact test based on Hotelling’s one-sample T 2 -statistic. Regarding the geometric shape of the indifference zone specified under the alternative hypothesis one aims to establish, an option comparable in intuitive appeal to the approach based on Euclidean metric, is to require pairwise equivalence of the marginal means for all possible pairs leading to a rect angular equivalence region in k(k − 1)/2 -dimensional space. Both testing procedures proposed in § 8.1.3 for that type of hypotheses formulation make use of the intersection-union principle being thus exactly valid in terms of the significance level though prone to conservatism. Remarkably, the extent of that conservatism is by no means dramatic as turns out when comparing the first of these procedures (tailored for establishing equivalence in terms of standardized expected values) to a competitor ensuring coincidence between nominal significance level and effective size of the critical region. Although having to rely on much more restrictive model assumptions (homoskedasticity and equality of all pairwise correlations), a procedure of this latter kind is that studied  √by Giani  and Finner (1991). For k = 5,  = 0.00, σ0 = 1.00 and δ0 ≡ ε σ0 1 −  = .70, these authors report their test to reach a power of 95% against μ1 = . . . = μk when performed with n = 76 observations at (nominal) level α = .05 . Since, in the homoskedastic√case with unit variances and vanishing pairwise correlations, we have σ(j,l) = 2 ∀ (j, l) and ε was set, according to (8.26), as the equivalence margin to (μj − μl )/σ(j,l) , this value √ has to be compared with the power of our test for ε = .70/ 2 = .4945 ≈ .5 and n = 76 under correlation structure (0) [recall Table 8.9], which, from the second column of Table 8.10, is estimated to fall in the interval from ≈ .895 through ≈ .95 (an extra simulation experiment run with the exact values of ε and n gave .92244). Taking into consideration that the intersection-union test based on pairwise t-tests for equivalence is valid under any covariance structure, the loss in power compared to Giani and Finner’s procedure seems far from being serious for practical purposes.

8.2 8.2.1

Multivariate two-sample tests for equivalence A two-sample test for equivalence based on Hotelling’s T 2

The test to be studied in this subsection can be viewed as the natural multivariate generalization of the two-sample t-test for equivalence of two homoskedastic univariate Gaussian distributions introduced in § 6.1 since it reduces to the latter in the case that k = 1 . The changes which have to be made to its one-sample analogue (8.10) in order to obtain the decision rule

254

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

to be used in the case of two samples from k-variate normal distributions are fairly similar to those leading from the classical one-sample T 2 -test to a test of the null hypothesis that the vectors of expected values of two k-variate Gaussian distributions with common though unknown covariance matrix coincide. Analogously to (8.1), the basic assumption is that we are given two independent samples (X11 , . . . , Xk1 ), . . ., (X1m , . . . , Xkm ) , (Y11 , . . . , Yk1 ), . . ., (Y1n , . . . , Ykn ) of k-variate observations of possibly different sizes m and n such that   (X1u , . . . , Xku ) ∼ N (μ1 , . . . , μk ), Σ , ∀ u = 1, . . . , m ,   (8.32) (Y1v , . . . , Ykv ) ∼ N (ν1 , . . . , νk ), Σ , ∀ v = 1, . . . , n . For the elements of the common (positive definite) covariance matrix Σ, the notation introduced in (8.2) will be adopted in the two-sample case as well. The equivalence hypothesis we are now interested in specifies that the (squared) Mahalanobis distance between the two population mean vectors μ and ν is smaller than some equivalence margin ε2 . Thus, the complete formulation of the testing problem is H : (μ − ν)Σ−1 (μ − ν) ≥ ε2 versus K : (μ − ν)Σ−1 (μ − ν) < ε2 .

(8.33)

An optimal test for (8.33) can be based on the two-sample counterpart of the T 2 -statistic (8.8) which is traditionally likewise named after H. Hotelling. Again, it can be represented as a normalized plug-in estimator of the population Mahalanobis distance. More precisely speaking, μ, ν and Σ  have, ¯ = m−1 m respectively, to be replaced with the empirical mean vectors X u=1  ¯ = n−1 n (Y1v . . . , Ykv ) , and the pooled covariance ma(X1u . . . , Xku ) , Y v=1 trix   1  ¯ 1 . . . , Xku − X ¯ k  X1u − X ¯ 1 . . . , Xku − X ¯k X1u − X N − 2 u=1 m

S=

+

n      Y1v − Y¯1 . . . , Ykv − Y¯k Y1v − Y¯1 . . . , Ykv − Y¯k ,

(8.34)

v=1

with N denoting the total sample size m + n . Using mn/N as a normalizing factor, the test statistic can now be written T2 =

mn ¯ ¯ −1 (X ¯ − Y) ¯  . (X − Y)S N

(8.35)

The distribution of T 2 under any (μ, ν, Σ) such that (μ − ν)Σ−1 (μ − ν) = δ 2 > 0, is well-known (see, eg., Anderson, 1984, § 5.2.2) to be given by   d T 2 (N − k − 1)/ k(N − 2) = Fk,N −k−1 (mnδ 2 /N ) ,



(8.36)

Multivariate two-sample tests for equivalence

255

where, as before, Fk,N −k−1 (ψ 2 ) denotes a random variable following an F distribution with k, N − k − 1 degrees of freedom and noncentrality parameter ψ 2 ≥ 0. Hence, the critical region of an exact level-α test for (8.33) can be written 

  (8.37) T 2 < k(N − 2)/(N − k − 1) Fk,N −k−1; α (mnε2 /N ) . Table 8.12 is the analogue of Table 8.1 showing the critical upper bounds to the two-sample T 2 -statistic to be used in testing (8.33) for the same three choices of the equivalence margin and the same range for the dimension k of the data vectors. The underlying design is assumed to be balanced so that n stands for the number of observations contained in each group. The sample size is let increase in steps of 5 from 10 through 50; beyond 50, the increment is 10. The power of the equivalence test with critical region (8.37) against any  specific alternative (μ, ν, Σ) with (μ − ν)Σ−1 (μ − ν) = δa2 ∈ [0, ε2 ) admits the explicit representation   β(δa2 ) = P Fk,N −k−1 (mnδa2 /N ) ≤ Fk,N −k−1; α (mnε2 /N ) (8.38) where Fk,N −k−1 (ψ 2 ) and Fk,N −k−1; α (ψ 2 ) has the same meaning as in (8.36) and (8.37), respectively. The tabulation of power values given in Table 8.13 relates to null alternatives under which both mean vectors μ and ν coincide and covers the same choices of n, k and ε as were made when compiling Table 8.12. Like its univariate counterpart [cf. § 6.1], the T 2 -test for equivalence of two multivariate Gaussian distributions with common covariance structure exhibits a number of noticeable mathematical properties. First of all, it is unbiased at the prespecified nominal significance level α ∈ (0, 1). Moreover, it is uniformly most powerful among all level-α tests for (8.33) which remain invariant under a large group of linear transformations of the sample space generated by several subgroups. The most important of these transformations are of the form (X1u , . . . , Xku ) → (a1 , . . . , ak ) + (X1u , . . . , Xku ) B , u = 1, . . . , m , (Y1v , . . . , Ykv ) → (a1 , . . . , ak ) + (Y1v , . . . , Ykv ) B , v = 1, . . . , n , with (a1 , . . . , ak ) ∈ IRk and B as an arbitrary nonsingular k × k matrix.

256

EQUIVALENCE TESTS FOR MULTIVARIATE DATA 2;k 2 Table 8.12 Critical constant T.05; n,n (ε) for the two-sample T -test for equivalence of two homoskedastic k-variate normal distributions with samples of common size n = 10(5)50(10)80 , significance level α = 5%, and equivalence margin ε ∈ {.25, .50, 1.00} to the root of the Mahalanobis distance of both mean vectors.

n

ε 2

10 10 10 15 15 15 20 20 20 25 25 25 30 30 30 35 35 35 40 40 40 45 45 45 50 50 50 60 60 60 70 70 70 80 80 80

0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00

0.12733 0.20187 0.96363 0.13462 0.26634 1.87953 0.14398 0.35190 3.01540 0.15460 0.46058 4.29694 0.16630 0.59394 5.68458 0.17903 0.75226 7.15429 0.19279 0.93449 8.69003 0.20762 1.13874 10.28043 0.22355 1.36270 11.91706 0.25891 1.86097 15.30460 0.29926 2.41433 18.81542 0.34498 3.01198 22.42484

3 0.43070 0.58383 1.64415 0.43722 0.68485 2.63189 0.45258 0.81354 3.81391 0.47170 0.96519 5.12926 0.49314 1.13827 6.54338 0.51636 1.33155 8.03467 0.54112 1.54367 9.58846 0.56727 1.77313 11.19428 0.59478 2.01839 12.84431 0.65371 2.55052 16.25412 0.71782 3.12986 19.78287 0.78711 3.74842 23.40714

k = 4 0.88562 1.11129 2.42416 0.87233 1.21902 3.44537 0.88469 1.37009 4.65792 0.90643 1.54725 5.99823 0.93303 1.74529 7.43302 0.96275 1.96163 8.94171 0.99478 2.19447 10.51043 1.02869 2.44231 12.12924 1.06423 2.70383 13.79070 1.13965 3.26322 17.21980 1.22036 3.86449 20.76433 1.30601 4.50119 24.40185

5 1.47559 1.76869 3.31564 1.41398 1.84702 4.32590 1.41115 2.00154 5.55069 1.42816 2.19094 6.90591 1.45464 2.40375 8.35489 1.48663 2.63514 9.87647 1.52231 2.88238 11.45676 1.56073 3.14364 13.08594 1.60131 3.41750 14.75675 1.68775 3.99852 18.20203 1.78003 4.61795 21.76010 1.87735 5.27009 25.40916

10 6.83045 7.47811 10.34322 5.54220 6.33757 10.01409 5.15320 6.14646 10.88636 4.99010 6.19976 12.10975 4.91821 6.35618 13.50519 4.89240 6.56813 15.00741 4.89325 6.81493 16.58506 4.91104 7.08603 18.22036 4.94032 7.37538 19.90197 5.02151 7.99502 23.37521 5.12215 8.65598 26.96377 5.23538 9.34877 30.64290

Multivariate two-sample tests for equivalence

257

Table 8.13 Power of the two-sample T 2 -test for equivalence at level α = .05 against the alternative that both mean vectors coincide, for the values of k, n and ε covered by Table 8.12. n 10 10 10 15 15 15 20 20 20 25 25 25 30 30 30 35 35 35 40 40 40 45 45 45 50 50 50 60 60 60 70 70 70 80 80 80

ε 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00 0.25 0.50 1.00

2

3

k = 4

5

10

.05815 .09044 .35808 .06270 .11998 .58400 .06757 .15678 .75651 .07278 .20102 .86665 .07836 .25201 .93038 .08432 .30827 .96496 .09067 .36778 .98287 .09743 .42838 .99182 .10462 .48816 .99617 .12034 .59964 .99920 .13790 .69519 .99984 .15736 .77290 .99997

.05767 .08691 .30400 .06193 .11282 .50257 .06646 .14410 .67794 .07127 .18077 .80652 .07637 .22244 .89026 .08178 .26838 .94051 .08749 .31753 .96891 .09353 .36872 .98424 .09989 .42077 .99221 .11362 .52312 .99821 .12872 .61771 .99962 .14521 .70056 .99992

.05715 .08366 .26723 .06115 .10684 .44449 .06537 .13432 .61519 .06982 .16611 .75261 .07451 .20201 .85032 .07946 .24158 .91382 .08465 .28418 .95237 .09011 .32905 .97457 .09584 .37540 .98681 .10810 .46926 .99670 .12147 .55996 .99924 .13594 .64322 .99983

.05667 .08083 .23976 .06043 .10194 .40006 .06439 .12664 .56357 .06854 .15498 .70446 .07290 .18684 .81168 .07747 .22194 .88600 .08225 .25987 .93392 .08726 .30010 .96309 .09250 .34204 .98003 .10365 .42858 .99460 .11573 .51471 .99866 .12875 .59641 .99969

.05473 .07069 .15938 .05781 .08629 .26961 .06093 .10366 .39667 .06415 .12309 .52700 .06748 .14462 .64744 .07094 .16824 .74932 .07452 .19387 .82931 .07823 .22137 .88827 .08206 .25057 .92942 .09014 .31319 .97436 .09875 .37977 .99162 .10791 .44811 .99749

258

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

Example 8.2 The data set we use for illustrating the two-sample T 2 -test for equivalence is taken from the same context as described in the brief introduction to Example 8.1. This time, two different lines of alcohol-preferring rats are to be assessed for similarity of the profiles obtained by measuring an animal’s total intake of ethanol [g/kg/day] on k = 4 consecutive days after alcohol deprivation. A sample of size n = 15 was available for the experiment from each line. Table 8.14 shows the arithmetic means and the empirical covariance matrix calculated from both samples of data vectors. Table 8.14 Summary statistics for two independent samples of repeated measurements of total ethanol intake [g/kg/day] taken in rats of two different alcohol-preferring lines at k = 4 consecutive days after deprivation. Day j = ¯j X

1

2

3

4

5.266324

4.7816957

4.7213656

5.1046539

X S1j X S2j X S3j X S4j Y¯j

6.8492111 −1.398337 −0.848002 −0.758188

−1.398337 −0.848002 1.0855432 0.4738331 0.4738331 1.0993696 0.5639499 0.6963447

−0.758188 0.5639499 0.6963447 0.6525258

5.3756573

4.4723527

4.8108581

5.0109942

Y S1j Y S2j Y S3j Y S4j

5.7589209 0.2711184 −0.284273 −0.254542

0.2711184 −0.284273 0.8766867 0.183452 0.183452 0.7138245 0.1820176 0.4123788

−0.254542 0.1820176 0.4123788 0.3939202

The additional computational effort required for determining the value of the test statistic T 2 with the entries in this table is almost negligible for a user working in a matrix-oriented programming environment. Both of the equations (8.34) and (8.35) can be translated into a single assignment statement then. Executing a brief script containing these statements yields the result T 2 = 1.404995. Setting the equivalence margin to the theoretical squared Mahalanobis distance between both distributions equal to ε2 = .25, the critical upper bound to which this value has to be compared, is read from Table 8.12 2;k 2 to be T.05; n,n (ε) = 1.21902 . Since the calculated value of T obviously fails to remain below this bound, the null hypothesis of relevant differences in terms of Mahalanobis distance cannot be rejected in the present case. According to Table 8.13, even if the the population mean vectors were exactly the same for both groups, the chance of coming out with a positive decision was only a bit

Multivariate two-sample tests for equivalence

259

larger than 10% given the fairly small size of both samples and the choice of the equivalence margin. Thus, with a view to power, the result is far from being surprising.

8.2.2

Behavior of the two-sample T 2 -test for equivalence under heteroskedasticity

The easiest way of handling settings where both population covariance matrices differ is to rely on the assumption that the equivalence version of Hotelling’s two-sample test is sufficiently robust against such deviations from the standard model under which the test is exact. The first step which has to be taken when one wants to investigate to what extent this assumption is warranted, consists of generalizing the hypotheses formulation (8.33) by replacing ordinary Mahalanobis distance with the quadratic form in the average ¯ ≡ (1/2)(Σ1 + Σ2 ) evaluated, as before, at (μ − ν) . Of covariance matrix Σ ¯ is of the same rank as Σ1 and Σ2 , namely k . course, this requires that Σ Under that additional restriction, generalizing (8.33) in a natural way leads to considering 

H : (μ − ν)(Σ1 + Σ2 )−1 (μ − ν) ≥ ε2 /2  versus K : (μ − ν)(Σ1 + Σ2 )−1 (μ − ν) < ε2 /2 .

(8.33∗ )

The question for what deviations between Σ1 and Σ2 the test with rejection region (8.37) keeps being valid in the sense of maintaining the significance level at least approximately under the null hypothesis of (8.33∗ ) has been addressed in another series of simulation experiments. All individual experiments performed for that purpose related to the same number of measurements taken per individual as in Example 8.2, namely k = 4. They differed with regard to which of the correlation structures, vectors of marginal variances and location-shift patterns specified in Table 8.15 were assigned to the populations in generating the data. The shift patterns were implemented in the following way: The distribution underlying the first sample was assumed to be centered about 0 ; the vector ν of expected values of the Yiv was set equal to  (s) (s)  cε δ1 , . . . , δk with cε being determined by solving the equation  (s)  (s) (s)  (s)  δ1 , . . . , δk (Σ1 + Σ2 )−1 δ1 , . . . , δk = ε2 /(2c2 ) , c > 0 . Table 8.16 shows the simulated rejection probabilities obtained for various combinations of sample sizes and parameter configurations generated by assigning one of the correlation structures, sets of marginal variances and location-shift patterns appearing in Table 8.15 to each of the multivariate normal distributions to be tested for equivalence with respect to the generalized Mahalanobis distance. The major conclusions to be drawn from these results are as follows:

260

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

(i) As long as the design is balanced with respect to sample size, heteroskedasticity does not lead to serious exceedances of the rejection Table 8.15 Correlation matrices, marginal variances and location-shift patterns used for generating 4-dimensional data vectors in studying the size of the T 2 -test for equivalence under heteroskedasticity. Correlation Matrix # (r)

12 13 14

(r)

23 24

(r)

34

1 2 3

.25 .25 .25 .90 .90 .90 −.65 .35 .45

.25 .25 .90 .90 .35 .35

.25 .90 .95

Vector of Marginal Standard Deviations # (q)

(r)

(r)

(q)

σ1

(r)

(q)

σ2

(q)

σ3

1.000 0.100 100.0

Location-Shift Pattern # (l)

δ1

δ2

δ3

δ4

1 2 3

1 1 1

1 0 2

1 0 3

1 0 4

(l)

1.000 0.100 100.0

(q)

σ4

1 2 3

(l)

1.000 0.100 100.0

(r)

(l)

1.000 0.100 100.0

(l)

probability under the null hypothesis over the threshold specified as the nominal significance level. More marked is a tendency towards anticonservatism. (ii) Under gross imbalance regarding the sample sizes, huge discrepancies between actual size of the test and the nominal significance level can occur both in the conservative and the liberal direction. A grossly anticonservative behavior of the test has to be expected when the larger of both samples is taken from the population with large entries in the covariance matrix, in particular large marginal variances. Switching this assignment leads to gross overconservatism.

Multivariate two-sample tests for equivalence

261

(iii) Given the sample sizes and both covariance matrices, moving along the common boundary of the hypotheses (8.33∗ ) through changing the Table 8.16 Simulated rejection probabilities of the two-sample T 2 -test for equivalence at the boundary of the hypotheses (8.33∗ ) with ε = .5 for various specific parameter combinations of the correlation structures, marginal variances and location-shift patterns specified in Table 8.15 . [The meaning of the symbols r, q and l is the same as in the previous table ; 100,000 replications per Monte Carlo experiment were carried out.] Sample 1 (r) (q) m

Sample 2 (r) (q) n

(l)

Rejection probability

1 " " " " " " "

2 " " " " " " "

10 50 10 50 10 50 10 50

2 " " " " " " "

3 " " " " " " "

10 50 50 10 50 10 50 10

1 " " " 2 2 3 3

.03782 .04779 .23852 .00209 .23960 .00198 .23811 .00199

1 " " " " " " "

2 " " " " " " "

10 50 10 50 10 50 10 50

3 " " " " " " "

1 " " " " " " "

10 50 50 10 50 10 50 10

1 " " " 2 2 3 3

.04505 .04963 .17244 .00592 .17548 .00574 .13933 .00703

2 " " " " " " "

3 " " " " " " "

10 50 10 50 10 50 10 50

3 " " " " " " "

1 " " " " " " "

10 50 50 10 50 10 50 10

1 " " " 2 2 3 3

.03698 .04816 .00193 .23763 .00193 .23823 .00189 .23777

262

EQUIVALENCE TESTS FOR MULTIVARIATE DATA location-shift pattern only slightly affects the rejection probability of the test.

In summary, we may conclude that applying the two-sample T 2 -test also under heteroskedasticity is an option as long as the study under consideration follows an approximately balanced design and Mahalonobis distance is generalized in the way made precise in (8.33∗ ). For considerable differences between both samples with regard to the number of data vectors contained, the homoskedasticity condition is crucial, and applying the test in presence of evidence against this assumption must be discouraged.

8.2.3

Multivariate two-sample tests for equivalence regions of rectangular shape

Even when all model assumptions are satisfied so that the T 2 -test for equivalence is exactly valid and UMPI, the shape of the underlying theoretical equivalence region is basically the same as in the one-sample case. Accordingly, there seems to be again considerable need for tests for inclusion of the true parameter point in an equivalence region of more intuitive form being independent of the covariance structure. In principle, both approaches followed in the one-sample case for modifying the shape of the theoretical equivalence region in a way making it invariant against changes of the common covariance matrix, can easily be adopted for the two-sample setting as well. The two-sample analogue of the asymptotic test for equivalence with respect to squared Euclidean rather than Mahalanobis distance derived in § 8.1.2 entails essentially the same problems with the low speed of convergence of the distribution of the suitably standardized maximum-likelihood test statistic to N (0, 1) . Hence, we do not pursue this approach here any further (for a detailed description of the procedure see Hoffelder, 2006) and confine ourselves to a brief consideration of the testing problem H : |μj − νj |/σj ≥ ε for at least one j ∈ {1, . . . , k} versus K : |μj − νj |/σj < ε ∀ 1 ≤ j ≤ k ,

(8.39)

with arbitrarily fixed ε > 0 . Obviously, this is another example of a testing problem which can easily be solved by means of the intersection-union principle. This time, the univariate two-sample t-test for equivalence of § 6.1 is the natural choice for testing each of the elementary hypotheses making up (8.39), and the corresponding rejection rule reads: Reject H in favor of K if and only if    ¯ j − Y¯j |/S˜j < F1,N −2; α (mnε2 /N ) 1/2 mn/N |X

∀ 1 ≤ j ≤ k,

(8.40)

Multivariate two-sample tests for equivalence

263

with S˜j2 =



m n 

   N ¯ j )2 + (Xju − X (Yjv − Y¯j )2 , N = m + n . (8.41) mn(N − 2) u=1 v=1

A prerequisite for applying the procedure in practice is some knowledge of its power against alternatives under which both multivariate Gaussian distributions from which the samples are taken, coincide. Except for the case of bivariate observations or observed vectors of any dimension with mutually independent components, exact power calculation is beyond the limits of computational feasibility. Hence, the entries in Table 8.17 which relates to multivariate normal observations of dimension k = 5 and four different correlation structures, were generated by means of simulation. Table 8.17 Simulated †) power of the intersection-union test (8.40) for equivalence of two k-variate Gaussian normal distributions against alternatives with μj − νj = 0 ∀ j = 1, . . . , k, for k = 5, α = .05, ε = .50, the correlation structures of Table 8.9, and common sample size m = n = 10(5)50(10)100. [100,000 replications per Monte Carlo experiment.] n 10 15 20 25 30 35 40 45 50 60 70 80 90 100 †)

0 .00001 .00003 .00015 .00062 .00234 .00757 .02015 .04430 .08247 .19782 .34400 .49256 .62420 .73078

Correlation Matrix # (r) 1 2 3 .00001 .00042 .00009 .00005 .00167 .00024 .00029 .00598 .00107 .00083 .01907 .00385 .00274 .05261 .01060 .00965 .11506 .02734 .02438 .20062 .05784 .05340 .29159 .10509 .09320 .38326 .16536 .22196 .54069 .31782 .37019 .66487 .46842 .51864 .75902 .60409 .64728 .82955 .71136 .74530 .87757 .79342

4 .00026 .00053 .00184 .00596 .01575 .03439 .07055 .11824 .18056 .32350 .46632 .59982 .70476 .78905

For r = 0 [↔ independence], the tabulated power values were computed exactly applying formula (6.7) and exponentiating the result.

264

EQUIVALENCE TESTS FOR MULTIVARIATE DATA

Not surprisingly, it becomes obvious from the above results that the power of the intersection-union test (8.40) highly depends on the correlation structure. Least favorable among the five correlation patterns investigated is pairwise independence of all components of the observed random vectors, and the largest power values are attained under the constellation indexed by r = 2 [↔ equal pairwise correlations of .90] . Under independence, 50 observations per group are required for excluding that the power of the test falls below the significance level. In order to guarantee a power of 80% under all correlation structures, both samples would have to consist of more than 100 data points.

9 Tests for establishing goodness of fit

9.1

Testing for equivalence of a single multinomial distribution with a fully specified reference distribution

In their simplest form, problems of testing for goodness rather than lack of fit involve a fully specified multinomial distribution, together with some sufficiently small “neighborhood” of distributional models of the same kind which according to the alternative hypothesis to be established by means of the data, contains the true distribution from which the sample has been taken. In such a setting the primary data set consists of n mutually independent random vectors (Y11 , . . . , Y1k ), . . . , (Yn1 , . . . , Ynk ) of dimension k ≥ 2 where (Yi1 , . . . , Yik ) indicates in what out of k mutually exclusive categories the ith sampling unit is observed to be placed. Thus, (Yi1 , . . . , Yik ) consists of exactly k − 1 zeros and a single one, and the probability that the nonzero component appears at the jth position is assumed to be the same number πj ∈ [0, 1] for each element of the sample. As usual in the analysis of categorical data, these vectors are aggregated to the corresponding cell counts (X1 , . . . , Xk ) form the beginning defining Xj ≡ #{i | i ∈ {1, . . . , n}, Yij = 1} for j = 1, . . . , k. Then, we  have kj=1 Xj = n, and the distribution of (X1 , . . . , Xk ) is multinomial with parameters n and π = (π1 , . . . , πk ) given by the probability mass function k '   x πj j /xj ! , P X 1 = x1 , . . . , X k = xk = n ! j=1 k

  x1 , . . . , x ˜k ) | x ˜j ∈ IN◦ ∀ j = 1, . . . , k, x ˜j = n . (9.1) (x1 , . . . , xk ) ∈ (˜ j=1

The usual shorthand notation for this distribution is M(n; π1 , . . . , πk ). The reference distribution of (X1 , . . . , Xk ) to which the true one is asserted to fit sufficiently well under the alternative hypothesis to be established by assessing the observed cell counts, is simply given by some specific choice of the parameter (row-) vector π, say π ◦ = (π1◦ , . . . , πk◦ ) ∈ (p1 , . . . , pk ) ∈  k [0, 1]k | j=1 pj = 1 . In order to quantify the degree of dissimilarity between the true and the target distribution of (X1 , . . . , Xk ), it seems reasonable to

265

266

TESTS FOR ESTABLISHING GOODNESS OF FIT

use the ordinary Euclidean distance between the associated parameter vectors which leads to the following formulation of the problem of testing for goodness of fit of M(n; π1 , . . . , πk ) to M(n; π1◦ , . . . , πk◦ ): H : d2 (π, π ◦ ) ≥ ε2

versus

K : d2 (π, π ◦ ) < ε2 .

(9.2)

As before [recall (7.24)], d2 ( · , · ) denotes the square of the metric in the Euclidean space of prespecified dimension. In other words, (9.2) has to be read with k  d2 (π, π ◦ ) = (πj − πj◦ )2 ∀ π ∈ Π (9.3) j=1

where Π denotes the parameter family  space for the underlying  of distrik butions, i.e., the hyperplane (p1 , . . . , pk ) ∈ [0, 1]k | j=1 pj = 1 in the kdimensional unit-cube. In the special case that there are just two different categories, (9.2) reduces √ √ to the problem considered in § 4.3, with (π1◦ − ε/ 2, π1◦ + ε/ 2) as the equivalence range to the binomial parameter p = π1 . For larger values of k, there is no hope of being able to solve (9.2) in a way yielding an exact optimal testing procedure. Instead, we will rely again on asymptotic methods starting from the well-known fact (see, e.g., Bishop et al., 1975, Theorem 14.3–4) that the normalized vector √ √ n(ˆ π1 − π1 , . . . , π ˆk − πk ) ≡ n(X1 /n − π1 , . . . , Xk /n − πk ) (9.4) of relative frequencies converges in law (as n → ∞) to a random variable which follows a k-dimensional Gaussian distribution with mean 0 and covariance matrix ⎛ ⎞ ⎛ 2 ⎞ π 1 . . . π1 π k π1 0 . . . 0 ⎜ 0 π 2 . . . 0 ⎟ ⎜ π 2 π 1 . . . π2 π k ⎟ ⎜ ⎟ ⎜ ⎟ (9.5) Σπ = D π − π  π ≡ ⎜ . . .. ⎟ . .. ⎟ − ⎜ .. ⎝ ⎝ .. .. ⎠ . . ⎠ . πk π1 . . . πk2 0 0 . . . πk Furthermore, it is obvious that π → d2 (π, π ◦ ) is totally differentiable on Π with gradient vector ∇d2 (π, π ◦ ) = 2(π − π ◦ ). These facts imply that we can exploit once more the so-called δ-method to derive the asymptotic π , π ◦ ) between true and distribution of the estimated squared distance d2 (ˆ target parameter vector. Writing √  π , π ◦ ) = 4 (π − π ◦ )(Dπ − π  π)(π − π◦ ) , (9.6) σa2 n d2 (ˆ we may thus conclude that √  L  √  2 n d (ˆ π , π ◦ ) − d2 (π, π ◦ ) → Z ∼ N 0, σa2 n d2 (ˆ π, π◦ ) as n → ∞ . (9.7)

Case of a fully specified reference distribution

267

Next, the expression on the right-hand side of (9.6) is readily expanded   k k k ◦ 2 ◦ ◦ into 4 j=1 (πj − πj ) πj − j1 =1 j2 =1 (πj1 − πj1 )(πj2 − πj2 )πj1 πj2 which is obviously a polynomial in (π1 , . . . , πk ). Since π ˆj is a consistent estimator of πj for each j = 1, . . . , k, the asymptotic variance (9.6) can be consistently estimated by replacing in this latter expression each cell probability with the homologous relative frequency. Denoting the resulting variance estimator by π , π ◦ ), we have vn2 (ˆ π, π◦ ) = 4 vn2 (ˆ

k 

(ˆ πj − πj◦ )2 π ˆj −

j=1 k k  

(ˆ π j1 −

πj◦1 )(ˆ π j2

−

πj◦2 )ˆ π j1 π ˆ j2

! .

(9.8)

j1 =1 j2 =1

√  In view of (9.7) and the consistency of vn2 (ˆ π , π ◦ ) for σa2 n d2 (ˆ π , π ◦ ) , an asymptotically valid test for the goodness-of-fit problem (9.2) is obtained by π , π ◦ ) as a normally distributed statistic with known variance treating d2 (ˆ −1 2 ◦ n vn (ˆ π , π ) and unknown expected value θ, say, about which the one-sided null hypothesis θ ≥ ε2 has been formulated. Accordingly, an asymptotic solution to (9.2) can be based on the following decision procedure: M(n; π1 , . . . , πk ) to M(n; π1◦ , . . . , πk◦ ) √ iff it turns out that d2 (ˆ π , π ◦ ) < ε2 − u1−α vn (ˆ π , π ◦ )/ n , Reject lack of fit of

(9.9)

with u1−α = Φ−1 (1 − α). Example 9.1 For the purpose of illustrating the use of the testing procedure derived in this section, it is tempting to revisit the elementary classroom example of assessing the compatibility of the results of casting the same play dice independently a given number n of times, with the Laplacean “equal likelihood” definition of probability. Suppose the number of casts performed in such an experiment was n = 100, and the following frequencies of the possible elementary outcomes were recorded:

Table 9.1 Absolute frequencies observed in a sequence of n = 100 casts of a given play dice. j

1

2

3

4

xj

17 16 25 9

5

6



16 17 100

With these data, the traditional χ2 -test for lack of fit yields 7.7600 as the observed value of the test statistic. The associated p-value, i.e., the upper

268

TESTS FOR ESTABLISHING GOODNESS OF FIT

tail-probability of 7.7600 under a central χ2 -distribution with 5 degrees of freedom, is computed to be p = .16997, indicating that the observed frequencies do not deviate significantly from those expected for an ideal dice. But is this really sufficient for asserting positively that the dice under assessment is approximately fair? In order to obtain a conclusive answer to this question, we perform the testing procedure defined by (9.9), at the conventional significance level α = .05 and with tolerance ε = .15 which corresponds in the binomial case to a theoretical equivalence interval of length ≈ 2 · .10. The necessary computational steps are readily carried out running the program gofsimpt, another SAS macro provided in the WKTSHEQ2 Source Code Package. (An R script serving the same purpose can be found there as well.) The squared distance between the empirical distribution shown in Table 9.1 and the theoretical distribuπ , π ◦ ) = .012933 tion given by πj◦ = 1/6 ∀j = 1, . . . , 6, turns out to be d2 (ˆ −1/2 ◦ vn (ˆ π , π ) = .009200. Hence, with with an estimated standard error of n the suggested specifications of α and ε, the critical upper bound which the π , π ◦ ) has to be compared to according to (9.9), is observed value of d2 (ˆ 2 .15 − 1.644854 · .009200 = .007367. Since the observed value of the test statistic clearly exceeds this bound, at the 5% level the data of Table 9.1 do not allow us to decide in favor of the (alternative) hypothesis that the dice under assessment is approximately fair. Thus, the example gives a concrete illustration of the basic general fact (obvious enough from a theoretical viewpoint) that the traditional χ2 -test of goodness of fit to a fully specified multinomial distribution is inappropriate for establishing the hypothesis of (approximate) fit of the true to the prespecified distribution. Some simulation results on the finite-sample behavior of the test As is the case for every multiparameter problem with a composite null hypothesis specifying an uncountable subspace of parameter values, performing a high-resolution search for the maximum exact rejection probability on the common boundary of the hypotheses of (9.2) or, still more, on H as a whole in a simulation study would entail a tremendous computational effort. Therefore, for the purpose of throwing some light on the effective size of the critical region of the test (9.9) when applied to finite samples, we confine ourselves to present simulated rejection probabilities only under a small yet sufficiently diversified selection of parameter configurations π ∈ Π with d(π, π ◦ ) = ε. Tables 9.2 a and b show such a representative set of rejection probabilities simulated under the null hypothesis for the “equal likelihood” problem with k = 4 and k = 6 categories, respectively, for the same specification of α and ε as in the above dice-casting example.

Collapsibility of multiway contingency tables

269

Table 9.2a Simulated exact rejection probability of the test (9.9) at nominal level α = .05 with n = 100 observations, under selected null configurations for k = 4, πj◦ = 1/4 ∀ j = 1, . . . , 4, and ε = .15 [40,000 replications of each Monte Carlo experiment].

π1

π2

π3

π4

Rejection Probability

.28879 .29655 .21029 .25388 .14393 .17500

.13000 .15000 .20000 .22000 .25000 .32500

.25000 .21000 .21000 .16000 .25000 .17500

.33121 .34345 .37971 .36612 .35607 .32500

.03025 .04250 .06513 .05098 .04143 .04540

Table 9.2b Analogue of Table 9.2a for k = 6.

π1

π2

π3

π4

π5

π6

Rejection Probability

.21057 .12296 .17296 .10283 .10211 .10543

.05000 .05000 .10000 .15000 .15000 .22790

.15000 .20000 .10000 .10000 .15000 .10543

.15000 .20000 .15000 .15000 .15000 .22790

.20000 .20000 .20000 .25000 .15000 .10543

.23943 .22704 .27704 .24717 .29789 .22790

.00835 .00833 .02685 .02575 .03943 .02113

The results displayed in the above pair of tables suggest that, given the sample size, the signed difference between nominal significance level and exact size of the critical region increases quite rapidly with the number of cells of the underlying contingency table. In fact, whereas for k = 4 the tabulated rejection probabilities indicate some tendency towards anticonservatism (whose extent seems to cause little harm for most practical purposes, to be sure), under the conditions of Example 9.1 the test fails to exhaust the target significance level by about 1%. Clearly, both of these undesirable effects can be removed at least in part by adjusting the nominal level to the maximum admissible degree, but working out the details is beyond our scope here. Tables 9.3 a and b give results from simulation experiments performed in order to study the second basic property of the test (9.9), namely, the relationship between its power against the specific alternative of perfect coincidence between true and model-derived distribution of (X1 , . . . , Xk ), and size n of the available sample. These latter data point to an interesting relationship corroborated by the results of additional simulation experiments with

270

TESTS FOR ESTABLISHING GOODNESS OF FIT

Table 9.3a Simulated exact power P OW◦ of the test (9.9) at nominal level α = .05 against the alternative that πj = πj◦ = 1/k ∀j = 1, . . . , k, for k = 4, ε = .15 and n = 50(25)200 [40,000 replications of each experiment]. n P OW◦

50

75

.23798 .44120

100

125

150

175

.63673 .77330 .87213 .92868

200 .96388

Table 9.3b Analogue of Table 9.3a for k = 6. n P OW◦

50

75

100

.16030 .42943 .66795

125

150

175

200

.83120 .92620 .96913 .98710

other values of k : The larger the number of categories, the steeper the increase of the power against π = π ◦ as a function of n. Both in the tetra- and the hexanomial case, n = 150 observations turn out sufficient to guarantee satisfactory power of the test (9.9) against the specific alternative of primary interest.

9.2

Testing for approximate collapsibility of multiway contingency tables

The concept of collapsibility plays an important role within the literature on log-linear models for multiway contingency tables (cf. Bishop et al., 1975, § 2.5) The precise meaning of the term in its strict sense is as2follows: Let q (X1...1 , . . . , Xk1 ...kq ) be the vector of frequencies counted in the ν=1 kν cells of a higher-order contingency table formed by cross-classifying each of n sampling units with respect to q categorical variables C1 , . . . , Cq with possible values {1, . . . , k1 }, . . . , {1, . . . , kq }, respectively. Then, the set {C1 , . . . , Cq } is called strictly collapsible across Cν (for prespecified ν ∈ {1, . . . , q}) if the loglinear model for the reduced table (X1...+...1 , . . . , Xk1 ...kν−1 +kν+1 ...kq ) obtained k by taking for each (i1 , . . . , iν−1 , iν+1 , . . . , iq ) the sum iνν=1 Xi1 ...iν−1 iν iν+1 ...iq contains exactly the same parameters as that for the original table, except for those related to Cν . It can be shown (see Bishop et al., 1975, p. 47) that this holds true if and only if Cν is independent of {C1 , . . . , Cν−1 , Cν+1 , . . . , Cq }. Accordingly, strict collapsibility of a set of categorical variables defining a given multiway contingency table is a special case of independence of a couple

Collapsibility of multiway contingency tables

271

of such variables, so that the general problem raised in the title of this section can be solved by constructing a test for approximate independence of two categorical variables A and B say, with possible values 1, . . . , r and 1, . . . , s, respectively. As usual, we denote by πij the probability that a specific sampling unit is observed to belong to cell (i, j) of the corresponding two-way table, and Xij stands for the number of sampling units counted in that cell. Of course, independence of the two characteristics r A and B holds if and only if we may s write πij = πi+ π+j ≡ ( ν=1 πiν )( μ=1 πμj ) ∀ (i, j) ∈ {1, . . . , r}× {1, . . . , s}. Thus, the target distribution to which the true distribution of the primary observations is asserted to be sufficiently similar under any hypothesis formalizing the notion of approximate independence, depends on the unknown parameters πij rather than being completely specified. Nevertheless, it will turn out to be comparatively easy to accommodate the approach of the preceding section to this more complicated situation. In order to give the details of the construction, we have to adapt the notation for the primary parameter vector by defining π as the vector of dimension r × s formed by putting the rows of the matrix (πij )(r,s) of cell probabilities one after the other. Furthermore, it will be convenient to use the following extra symbols for additional parametric functions to be referred to repeatedly in defining the hypotheses, as well as in deriving a test statistic upon which an asymptotically valid test for approximate independence of A and B can be based: s  πiν , i = 1, . . . , r ; (9.10a) i (π) ≡ ν=1

ζj (π) ≡

r 

πμj ,

j = 1, . . . , s ;

(9.10b)

 . . . , r (π)ζ1 (π), . . . , r (π)ζs (π) .

(9.10c)

μ=1

 g(π) ≡ 1 (π)ζ1 (π), . . . , 1 (π)ζs (π),

By analogy with the way of approaching the problem of testing for consistency with a fully specified distribution considered in § 9.1, it seems sensible to base the criterion for approximate independence of the classifications under consideration on the squared Euclidean distance r  s     2 πij − i (π)ζj (π) d2 π, g(π) =

(9.11)

i=1 j=1

between the vector π of true cell probabilities and the associated vector g(π) of products of marginal totals. Again, a natural estimator of that distance is obtained by substituting the vector π ˆ of observed relative frequencies for π throughout, where the components of π ˆ have this time to be written π ˆij =

272

TESTS FOR ESTABLISHING GOODNESS OF FIT

Xij√/n. Except for obvious changes in notation, the asymptotic distribution π − π) is the same as in the one-dimensional case [recall (9.5)]. This of n(ˆ     √ 2 2 ˆ , g(ˆ π ) − d π, g(π) is asymptotically normal with implies that n d π √   ˆ , g(ˆ π ) , say, which can be computed expectation 0 and variance σa2 n d2 π by means of the δ-method as σa2

r  s  √ 2    2 ∂ 2 ˆ , g(ˆ π) = nd π d π, g(π) πij ∂πij i=1 j=1

−

r  s  r  s  i1 =1 j1 =1 i2 =1 j2

  ∂   ∂ d2 π, g(π) d2 π, g(π) πi1 j1 πi2 j2 . (9.12) ∂πi1 j1 ∂πi2 j2 =1

For the purpose of deriving explicit expressions for the individual compo  nents of the gradient of π → d2 π, g(π) , we start with determining the partial derivatives of the (i, j)th term of the double sum appearing on the right-hand side of (9.11) [for arbitrarily fixed (i, j) ∈ {1, . . . , r} × {1, . . . , s}] with respect to πμν for μ = 1, . . . , r, ν = 1, . . . , s. From (9.10a), (9.10b) it is obvious that we have ⎧ 0 μ = i , ν = j  ⎪  ⎪ ⎨ ∂ i (π)ζj (π) ζj (π) μ = i , ν = j for . (9.12a) = i (π) μ = i , ν = j ⎪ ∂πμν ⎪ ⎩ μ = i, ν = j i (π) + ζj (π) Using (9.12a), it is readily verified that 2  1 ∂ πij − i (π)ζj (π) = 2 ∂πμν ⎧ 0 ⎪ ⎪ ⎪ ⎨ −π −  (π)ζ (π)ζ (π) i j  j  ij for ⎪ − πij − i (π)ζj (π) i (π) ⎪ ⎪    ⎩ πij − i (π)ζj (π) 1 − i (π) − ζj (π)

μ = i , μ = i, μ = i , μ = i,

ν ν ν ν

= j = j =j =j

. (9.12b)

Summing (9.12b) over (i, j) ∈ {1, . . . , r} × {1, . . . , s} gives:  r        ∂ 2 πiν − d π, g(π) = 2 πμν − μ (π)ζν (π) − ∂πμν i=1

s        i (π)ζν (π) i (π) − πμj − μ (π)ζj (π) ζj (π) . (9.13) j=1

π, Of course,√what we eventually need is a consistent estimator, say vn2 (ˆ  g(ˆ π )), of σa2 n d2 (ˆ π , g(ˆ π )) . In view of (9.12) and (9.13), for achieving this goal, it is sufficient to plug in the homologous relative frequency at every

Collapsibility of multiway contingency tables

273

place on the right-hand side of both equations where a population parameter, i.e., a component of the vector π appears. More precisely speaking, for any (μ, ν) ∈{1, . . . , r}  × {1, . . . , s}, the (μ, ν)-component of the gradient vector of π → d2 π, g(π) can be consistently estimated by  r       ˆ π ˆiν − i (ˆ dμν ≡ 2 π ˆμν − μ (ˆ π )ζν (ˆ π) − π )ζν (ˆ π) · i=1

s      π ˆμj − μ (ˆ π) − π )ζj (ˆ π ) ζj (ˆ π) . i (ˆ



(9.14)

j=1

  √ ˆ , g(ˆ π ) is Finally, the desired estimator of the asymptotic variance of n d2 π   ˆ11 , . . . , π ˆrs for ∂π∂11 d2 π, g(π) , obtained by substituting dˆ11 , . . . , dˆrs and π   . . . , ∂π∂rs d2 π, g(π) and π11 , . . . , πrs , respectively, on the right-hand side of (9.12), yielding r  s    vn2 π ˆ , g(ˆ π) = ˆij − dˆ2ij π i=1 j=1 r s r s    

ˆ i 1 j1 π ˆ i 2 j2 . dˆi1 j1 dˆi2 j2 π

(9.15)

i1 =1 j1 =1 i2 =1 j2 =1

Now we are ready to formulate the decision rule of an asymptotically valid test of     (9.16) H : d2 (π, g π) ≥ ε2 versus K : d2 π, g(π) < ε2 for fixed ε > 0 to be chosen a priori. The alternative hypothesis we want to establish according to (9.16) specifies that the true distribution is in acceptably close agreement with that hypothesized by the model of independence given both vectors of marginal probabilities, which seems to be an adequate translation of the notion of approximate validity of that model into a statement about the parameter characterizing the distribution from which the data are taken. After the technical preparations made in the preceding paragraphs, we know that asymptotically, the estimated squared distance g(π) can  of π from  2 π, g(π) as unknown be viewed as a normally distributed variable with d   ˆ , g(ˆ π ) . Hence, we get a expected value and fixed known variance n−1 vn2 π test of asymptotic level α for the problem (9.16) by applying the decision rule: Reject existence of relevant discrepencies between true distribution and independence model iff     √ d2 π ˆ , g(ˆ π ) < ε2 − u1−α vn π ˆ , g(ˆ π) / n . (9.17)

274

TESTS FOR ESTABLISHING GOODNESS OF FIT

Example 9.2 In one out of four different arms of a large multicenter trial of drugs used as first-choice medication for mild arterial hypertension, classifying the patients by gender and outcome of short-term treatment gave the 2 × 4 contingency table shown below as Table 9.4. It covers all patients who were randomized for administration of the calcium-entry blocking agent nitrendipine and treated in compliance with the study protocol. At a certain stage of their work, the statisticians responsible for the analysis of the results of the trial were confronted also with the problem of identifying those treatment arms for which a relevant gender-by-outcome interaction could be excluded. For the nitrendipine group, this question can be suitably addressed by applying decision rule (9.17) to the data of Table 9.4. Once more, we set the significance level to α = .05, and specify ε = .15 as the largest tolerable distance between the vector of true cell probabilities and the associated vector of products of marginal probabilities. Table 9.4 2 × 4 contingency table relating gender and treatment outcome on nitrendipine monotherapy in patients suffering from mild arterial hypertension. [Source: Philipp et al. (1997); Numbers in ( ): relative frequencies with respect to the overall total n.] Outcome Category 1 Female Gender

Male 

∗)

2∗∗)

3∗∗∗)

4†)



9 13 (.042) (.060) 24 18 (.111) (.083)

13 48 (.060) (.221) 20 72 (.092) (.332)

83 (.382) 134 (.618)

33 31 (.152) (.143)

33 120 (.152) (.553)

217 (1.000)

∗)...∗∗∗) Response at lowest, middle and highest dose step, †) Failure to reach target level of diastolic blood pressure

respectively at all three

dose steps

With X11 = 9, . . . , X24 = 72, the estimated distance between π = (π11 , ˆ, . . . , π24 ) and g(π)= (π1+ π+1 , . . . , π2+ π+4 ) is readily computed to be d π g(ˆ π ) = .028396. Evaluating formula (9.15) for the squared standard error   √ ˆ , g(ˆ π ) by hand or by means of a pocket calculator is of the statistic n d2 π a rather tedious process, even if the total number of cells of the contingency table under analysis is comparatively small. So we leave this job to another computer program ready for being downloaded from WKTSHEQ2 Source Code Package. Once more, it exists both in a SAS and a R version, with gofind_t

Collapsibility of multiway contingency tables

275

as program name. Running this macro, we read from the output vn = .017016. Thus, at the 5%-level and with  ε = .15, the critical upper bound which the ˆ , g(ˆ π ) has to be compared to, is computed to be observed value of d2 π √ .152 − 1.644854 · .017016/ 217 = .020600. Since .0283962 = .000806 is clearly smaller than this critical bound, we have to decide in favor of approximate independence between gender and treatment outcome in hypertensive patients taking nitrendipine according to the prescriptions of the protocol of the trial from which the data of Table 9.4 are taken (for details see Philipp et al., 1997). Example 9.3 In the light of the introductory remarks at the beginning of this section, we now turn to illustrating the use of decision rule (9.17) for testing for approximate collapsibility in the strict sense of a higher-order contingency table across a prespecified element of the set {C1 , . . . , Cq } of categorical variables involved. For this purpose, we confine ourselves to a brief discussion of a synthetic data set. Suppose the format of the table under analysis is 2 × 2 × 2 with observed cell frequencies as shown below, and the variable with respect to which we aim to establish collapsibility, is the first one. Suppose further that α and ε have been specified as in the previous example. Table 9.5a Observed 2 × 2 × 2 table to be tested for approximate collapsibility in the strict sense across the first binary classification. C2 +

+

C3

−

+

8

13

−

15

6

+

19

21

−

31

7

C1 −

In a preliminary step we have to rearrange the original array in a 2× 4 table with the combination of C2 and C3 defining the columns. Using the entries in the Table 9.5b as input data to the program gofind_t we obtain  rearranged  d π ˆ , g(ˆ π ) = .030332 and vn = .019190. Hence, the condition for rejecting the null hypothesis of relevant deviations from collapsibility is in the present √  ˆ , g(ˆ π ) < .152 − 1.644854 × .019190/ 120 = .019619 which in case that d2 π view of .0303322 < .001 is clearly satisfied.

276

TESTS FOR ESTABLISHING GOODNESS OF FIT Table 9.5b Rearrangement of Table 9.5a in a 2 × 4 array. B (↔ C2 × C3 ) 1 A (↔ C1 )

1 2 

2

3

4

8 13 15 19 21 31

6 7

 42 78

27 34 46 13 120

Simulation results on size and power Table 9.6 is an analogue of Tables 9.2 a, b for the testing procedure (9.17) as applied to a 2 × 4 contingency table containing roughly the same total number of observations as had been available in the real application described in Example 9.2. Of course, for the problem of testing for approximate independence of two categorical variables, a numerically accurate search for the maximum rejection probability on the common boundary of the hypotheses is still farther beyond the scope of computational feasibility than for the problem of establishing goodness of fit to a fully specified multinomial distribution. In fact, in the independence case, the alternative hypothesis is an uncountable union of hyperspheres of radius ε rather than a single set of that form. Accordingly, the simulation results displayed in the rightmost column of the next table allow only a rough qualitative assessment of the finite-sample behavior of the testing procedure (9.17) under the null hypothesis H of (9.16). Table 9.6 Simulated exact rejection probability of the test (9.17) at nominal level α = .05, with r = 2, s = 4, ε = .15 and n = 200, under various parameter configurations on the common boundary of the hypotheses (9.16) [40,000 replications per Monte Carlo experiment]. π11

π12

π13

π14

π21

π22

π23

π24

P [Rejection]

.05 .05 .05 .10 .10 .05 .25 .15 .20 .20

.05 .05 .15 .05 .10 .10 .25 .15 .05 .05

.05 .05 .10 .25 .10 .15 .05 .15 .05 .05

.05 .10 .35 .15 .15 .20 .05 .15 .05 .20

.65923 .48723 .07878 .18082 .24850 .23229 .04030 .25308 .16601 .27247

.05 .05 .05 .10 .10 .05 .20 .05 .30 .05

.05 .15 .15 .05 .20 .10 .15 .05 .15 .15

.04077 .06277 .07122 .11918 .00150 .11771 .00970 .04692 .03399 .02753

.07153 .06718 .06218 .05405 .05680 .05803 .05615 .06573 .06993 .05895

Collapsibility of multiway contingency tables

277

Again we find some anticonservative tendency whose extent seems small enough for being ignored in the majority of practical applications, but is clearly outside the range of deviations which can be accounted for by the simulation error. Since the results of additional simulation experiments not reproduced in detail here suggest that the convergence of the exact size of the asymptotic test to the target significance level is quite slow, it is recommended to rely on an adjusted nominal level α∗ < α whenever strict maintenance of the prespecified level is felt to be an indispensible requirement. For instance, replacing α = .05 with α∗ = .025 suffices for ensuring that even for the least favorable of the parameter constellations covered by Table 9.6 [→ row 1], the (simulated) exact rejection probability with a sample of size n = 200 no longer exceeds the 5%-bound. For completeness, we finish our discussion of (asymptotic) testing procedures tailored for establishing the approximate validity of customary models for contingency tables, with presenting some simulation results on the power of the procedure (9.17) to detect that the true parameter configuration π fits the model of independence exactly. Obviously, given the format of the two-way tables to be analyzed, there is an uncountable manifold of specific alternatives of that type. Singling out once more the case (r, s) = (2, 4), we studied the power under three configurations of marginal probabilities π1+ , π2+ ; π+1 , . . . , π+4 determining extremely different joint distributions of the Xij . Table 9.7 shows the rejection probabilities simulated under these parameter configurations for n ∈ {50, 100, 150}, ε = .15 and nominal levels α∗ which, according to our results on the size of the test (9.17), are small enough to ensure that the target level of 5% is maintained at least at all points on the common boundary of the hypotheses covered by Table 9.6. Table 9.7 Simulated exact power of the test (9.17) against three alternatives satisfying πij = πi+ π+j ∀ (i, j) for 2 × 4 arrays, with ε = .15, n = 50, 100, 150 and corrected nominal significance level α∗ [40,000 replications per Monte Carlo experiment]. n

α∗

50 .01 " " " " 100 .02 " " " " 150 .025 " " " "

π1+

π2+

π+1

π+2

π+3

π+4

P [Rejection]

.25 .50 .33 .25 .50 .33 .25 .50 .33

.75 .50 .67 .75 .50 .67 .75 .50 .67

.10 .25 .15 .10 .25 .15 .10 .25 .15

.40 .25 .15 .40 .25 .15 .40 .25 .15

.30 .25 .15 .30 .25 .15 .30 .25 .15

.20 .25 .55 .20 .25 .55 .20 .25 .55

.39788 .25395 .43115 .87578 .84405 .90630 .97925 .98458 .98758

278

TESTS FOR ESTABLISHING GOODNESS OF FIT

  Not surprisingly, the power against alternatives π such that d π, g (π) vanishes turns out to be very sensitive against gross changes to the shape of the marginal distributions. Furthermore, even if the nominal significance level α∗ is downgraded as far as necessary for ensuring that the rejection probability keeps below the target level of 5% under all null constellations studied for assessing the level properties of the test, a sample size of 100 seems sufficient for achieving reasonable power against alternatives satisfying the model of independence exactly.

9.3 9.3.1

Establishing goodness of fit of linear models for normally distributed data An exact optimal test for negligibility of interactions in a two-way ANOVA layout

Presumably, more often than not, existence of interaction effects will be considered as an undesirable lack of fit to the model which an investigator applying standard ANOVA techniques would like to rely upon. The obvious reason is that in presence of interactions, the main effects do not admit a direct interpretation. In fact, only in absence of interactions, a positive main effect associated with some level of a given factor justifies the conclusion that on an average, all experimental units assigned to that level do better than those assigned to another level with a negative effect of the same factor. Adopting this perspective, it is natural to consider that part of the analysis of an ANOVA two-way layout which deals with the interaction effects, as a preliminary check carried out with the intention to establish that the “ideal” additive model fits the data sufficiently well. For the moment, we restrict the description of a testing procedure, which allows us to exclude relevant deviations from additivity of the main effects, to the case of a strictly balanced design. In other words, we assume that the data set under analysis consists of r · s independent samples of common size n from normal distributions with the same unknown variance σ 2 > 0. Denoting the kth observation in group (i, j) by Xijk , we suppose that for each (i, j) ∈ {1, . . . , r} × {1, . . . , s}, the cell mean μij = E(Xij1 ) = . . . = E(Xijn ) can be additively decomposed in the usual way leading to the representation μij = μ + αi + βj + γij .

(9.18)

Of course, the parameters coming into play according to this sbasic model s are r assumed to satisfy the usual side conditions i=1 αi = 0 = j=1 βj , j=1 γij = r 0 ∀ i = 1, . . . , r, i=1 γij = 0 ∀ j = 1, . . . , s.

Linear models for normally distributed data

279

Now, it seems reasonable to consider the interaction effects negligible as soon as it can be taken for granted that the standardized γij are the coordinates of a point in rs-dimensional Euclidean space which lies in a spherical neighborhood of 0 of sufficiently small radius ε > 0. Accordingly, our hypothesis of absence of relevant interactions specifies that s r alternative 2 (γ /σ) < ε2 , so that the testing problem as a whole reads ij i=1 j=1 H : d2 (γ/σ, 0) ≥ ε2

versus

K : d2 (γ/σ, 0) < ε2 ,

(9.19)

provided γ stands for the vector consisting of the rows of the matrix (γij )(r,s) , 0 for the null vector in rs-dimensional space, and d(u, v) denotes the Euclidean distance between arbitrary points u, v ∈ IRrs . It is easily verified that this pair of hypotheses remains invariant under the same group of 1 : 1 transformations of the sample space IRnrs onto itself as the problem of testing the conventional null hypothesis that all interaction parameters vanish. (As is generally the case with problems exhibiting the same invariance structure as a pair of linear hypotheses about the mean of an independent vector of homoskedastic Gaussian variables, a precise characterization of this transformation group requires the introduction of a canonical coordinate system for the observed vector, which would lead the exposition away from what our real issue is here.) Hence, any invariant test of (9.19) is necessarily a function of the classical F -ratio for testing γij = 0 ∀(i, j) vs. γij = 0 for some (i, j) ∈ {1, . . . , r} × {1, . . . , s}. Discarding the constant factor n/(r − 1)(s − 1), this statistic can be viewed as the squared Euclidean distance from the origin of the least squares estimator of γ standardized with respect to the length of the residual vector for the nonrestricted model as the standard estimator of σ. Using a notation which is to be suggestive of this interpretation of the rescaled F -ratio, we write r  s 

γ /S, 0) = d2 (ˆ

¯ ij − X ¯ i. − X ¯ .j + X ¯ .. )2 (X

i=1 j=1 r  s  n 

,

(9.20)

¯ ij ) /(n − 1) rs (Xijk − X 2

i=1 j=1 k=1

r ¯ ¯ ¯ ij ≡ n Xijk /n , X ¯ i. ≡ s X ¯ ¯ with X k=1 j=1 ij /s , X.j ≡ i=1 Xij /r, and X.. ≡ r  s ¯ i=1 j=1 Xij /rs.   The distribution of nd2 (ˆ γ /S, 0)/ (r − 1)(s − 1) is well known (see, e.g., Lehmann and Romano, 2005, p. 292) to be noncentral F with (r − 1)(s − 1), (n−1)rs degrees of freedom and noncentrality parameter λ2nc = nd2 (γ/σ, 0), under any parameter constellation (μ, α1 , . . . , αr , β1 , . . . , βs , γ11 , . . . , γrs ) ∈ IR(r+1)(s+1) . Since the class of all distributions of that type constitutes a family with strictly monotone likelihood ratios, it follows that the class of all γ /S, 0) contains a uniformly most level-α tests of (9.19) based upon d2 (ˆ  powγ /S, 0)/ (r − 1)(s − 1) turns erful element which rejects H if and only if nd2 (ˆ

280

TESTS FOR ESTABLISHING GOODNESS OF FIT

out to be less than or equal to the α-quantile of a F -distribution with the same numbers of degrees of freedom and noncentrality parameter λ2nc = nε2 . In γ /S, 0) for the hypotheses view of the maximal invariance of the statistic d2 (ˆ (9.19) we can thus conclude that 

  γ /S, 0) < (r − 1)(s − 1)/n F(r−1)(s−1),(n−1)rs; α (nε2 ) (9.21) d2 (ˆ is the critical region of an UMPI level-α test for negligibility of interactions in a two-way ANOVA layout with rs cells. Of course, in order to render this statement true, the threefold subscribed symbol F must be defined in the same way as in the preceding chapters so that we have F(r−1)(s−1),(n−1)rs; α (nε2 ) = lower 100α-percentage point of an F -distribution with (r − 1)(s − 1), (n − 1)rs degrees of freedom and noncentrality parameter nε2 .

(9.22)

For the power of the test with rejection region (9.21) against any specific alternative with some given value, say ψ 2 , of d2 (γ/σ, 0), we get a direct analogue of formula (7.12) referring to the noncentral F -test for equivalence of k homoskedastic Gaussian distributions as discussed in § 7.2 . More precisely speaking, if we denote the rejection probability of the test under any parameter constellation (μ, α1 , . . . , αr , β1 , . . . , βs , γ11 , . . . , γrs ) such that d2 (γ/σ, 0) = ψ 2 by β(ψ 2 ), then we can write   β(ψ 2 ) = P F(r−1)(s−1),(n−1)rs (nψ 2 ) < F(r−1)(s−1),(n−1)rs;l,α (nε2 ) (9.23) with F(r−1)(s−1),(n−1)rs (nψ 2 ) representing a random variable whose distribution differs from that to be used for determining the critical constant only by changing the value of the noncentrality parameter from nε2 to nψ 2 . Example 9.4 In a randomized clinical trial of the antimydriatic effects (to be brought about for the purpose of reversing scopolamine-induced pupil dilation at the end of cataract surgery) of acetylcholine and thymoxamine, a total of 228 eyes was subdivided into the following four treatment arms: (1) (2) (3) (4)

placebo (pure solvent) acetylcholine thymoxamine acetylcholine + thymoxamine.

Randomization was strictly uniform so that n = 57 eyes were assigned to each of these treatment groups. The primary endpoint for assessing the efficacy of

Linear models for normally distributed data

281

all treatments was pupillary constriction [mm] as measured 5 minutes after application. The following table summarizes the data obtained from this trial (as cellwise arithmetic means and standard deviations, respectively): Table 9.8 Arithmetic means and standard deviations (in parentheses) of pupillary restriction [mm] measured in the four arms of a clinical trial of the antimydriatic effects of acetylcholine and thymoxamine following a classical 2×2-design [ n = 57 eyes treated in each group]. (Data from Pfeiffer et al., 1994.) Acetylcholine − Thymoxamine

+

−

+

−0.0386 (.2433) 1.5281 (.7975)

1.8561 (.6690) 3.4193 (.9801)

¯ 1. = 0.9088, X ¯ .1 = 0.7448, X ¯ 2. = 2.4737, With these values, we calculate X ¯ ¯ X.2 = 2.6377, X.. = 1.6912 yielding for the interaction parameters the estimates γˆ11 = −0.0386 − (0.9088 + 0.7448 − 1.6912) = −0.0009, γ11 = 0.0009, γˆ21 = −ˆ γ11 = 0.0009 and γˆ22 = γˆ11 = −0.0009. γˆ12 = −ˆ Furthermore, using the parenthesized entries in Table 9.8, we find that the residual variance of the data set under analysis with respect to the unrestricted model (9.18) is S 2 = (.24332 + .66902 + .79752 + .98012 )/4 = .5258. Thus, as estimated squared distance of γ/σ from the origin, we find here γ /S, 0) = 4 · .00092 /.5258 = 6.16 · 10−6 . d2 (ˆ Let us now suppose that the true interaction effects are to be considered negligible if γ/σ lies in a sphere of radius ε = .25 around 0 and that we want to perform the test at significance level α = .05. Then, using the appropriate intrinsic SAS function, the critical upper bound which the estimated standardized distance of γ/σ from 0 has to be compared to according to (9.21), is computed to be (r − 1)(s − 1) 1 F(r−1)(s−1),(n−1)rs; α (nε2 ) = finv(.05,1,224,3.5625) n 57 1 = · .125142 = .002195 . 57 γ /S, 0), the data of Since this is clearly larger than the observed value of d2 (ˆ Table 9.8 allow us to reject the null hypothesis that there is a nonnegligible

282

TESTS FOR ESTABLISHING GOODNESS OF FIT

interaction between the antimydriatic effects of both substances. Although a common sample size of almost 60 for each treatment combination is unusually large for a multiarm trial performed in a rather specialized field, it is still much too small for ensuring that the power of the UMPI test exceeds 80% against the alternative that all interaction effects vanish. In fact, for ψ 2 = 0, evaluating the right-hand side of (9.23) by means of the SAS function probf yields under our present specifications β(0) = P [F1,224 (0) ≤ 0.125142] = probf(.125142,1,224) = .27614 . Keeping α and ε fixed at α = .05 and ε = .25, respectively, at least n = 138 observations per cell would be necessary for raising the power against the same type of alternative to 80% or more. With ε = .50, n = 35 would suffice for meeting the requirement β(0) ≥ .80. Generalization of the UMPI test for negligibility of interactions to nonbalanced designs In an unbalanced two-way layout, the numbers nij of observations taken under the various treatment combinations are arbitrary natural numbers (discarding situations where some cells contain no data at all) allowed to be even pairwise different from each other. As holds generally true for analyses of classical linear models for multiway layouts, nonbalancedness induces complications both with regard to the computational techniques required for determining the test statistics and, still more important, interpretation of hypotheses. From a computational perspective, the crucial change refers to the numerator of the (rescaled) F -statistic (9.20) which in the unbalanced case can no longer be written simply as the sum of squares of the best linear estimates of ¯ ij −(X ¯ i. +X ¯ .j −X ¯ .. ) has to be replaced the γij in the full model (9.18). In fact, X ˆ ˆ ¯ ij − (μ ˆ+α ˆ i + βj ) where (μ ˆ, α ˆ i , βj ) denotes the least squares estimate with X of (μ, αi , βj ) in the interaction-constrained model μij = μ + αi + βj . With ˆ i , βˆj and μ ˆ admit no explicit representation as linear unbalanced data, the α ¯ ij but must be determined by solving a system functions of the cell means X of linear equations whose coefficient matrix exhibits no closed-form inverse (for details see, e.g., Searle, 1987, § 4.9). Of course, instead of the ordinary ¯ ij − (μ ˆ +α ˆ i + βˆj ) we have to form the weighted sum of squares of the γˆ ij ≡ X n where using weight-factors nij /¯ r s 1  n ¯ ≡ N/rs ≡ nij . rs i=1 j=1

(9.24)

Making the dependence of the numerator of the test statistic on these weights explicit by adding the subscript n [symbolizing the so-called incidence matrix

Linear models for normally distributed data

283

(nij )(r,s) ] to d2 ( · , · ), we have to replace (9.20) in the unbalanced case with r  s 

ˆ /S, 0) = d2n (γ

 2 ¯ ij − (μ ˆ+α (nij /¯ n) X ˆ i + βˆj )

i=1 j=1 nij r  s  

.

(9.25)

¯ ij )2 /(N − rs) (Xijk − X

i=1 j=1 k=1

From a practical point of view, it makes almost no difference whether computation of the test statistic has to be based on formula (9.25) or can be simplified to (9.20). In fact, any well-designed program package for use in statistical applications contains a procedure for computing F -statistics for all hypotheses of interest in analyzing ANOVA higher-way layouts with arbitrary incidence matrices. Denoting the F -ratio for testing the null hypothesis of no (n) interactions by F(r,s) (γ|μ, α, β), we have just to read the value of the latter from the output list generated by running a suitable program (like proc glm in SAS) and multiply it by (r − 1)(s − 1)/¯ n to obtain the observed value of ˆ /S, 0) as defined in (9.25). Clearly, this suggests using d2n (γ 

  ˆ /S, 0) < (r − 1)(s − 1)/¯ n F(r−1)(s−1),N −rs; α (¯ nε2 ) . (9.26) d2n (γ as the rejection region of the desired test for negligibility of interactions in the case of an arbitrary incidence matrix n = n1(r,s) . However, the hypotheses for which (9.26) gives a UMPI critical region can no longer be expressed in terms of a spherical neighborhood of the origin in the space of the standardized true γij . To see this, one has to exam(n) ine the noncentrality parameter λ2nc of the distribution of F(r,s) (γ|μ, α, β) =   2 n ¯ /(r−1)(s−1) dn (γˆ /S, 0) under an arbitrary parameter configuration with γ = 0. Discarding the factor n ¯ , λ2nc is obtained by substituting in the numerator ¯ ij by its exof (9.25) every single observation Xijk and hence every cell mean X 2 ˆ is a vector pectation μij , and dividing the resulting expression by σ . Since γ ¯ ¯ ˆ of linear functions, say gn (X11 , . . . , Xrs ), of the cell means with coefficients (n) determined by the nij , it follows that F(r,s) (γ|μ, α, β) is noncentral F with ¯ d2n (ˆ gn (μ11 , . . . , μrs )/σ, 0) (r − 1)(s − 1), N − rs degrees of freedom and λ2nc = n so that the hypotheses actually tested when using (9.26) as the critical region, are gn (μ11 , . . . , μrs )/σ, 0) ≥ ε2 H : d2n (ˆ versus

ˆn (μ11 , . . . , μrs )/σ, 0) < ε2 K : d2n (g

(9.27)

On the one hand, for any configuration (μ11 , . . . , μrs ) ∈ IR of true cell ˆn (μ11 , . . . , μrs )/σ, 0 means satisfying the no-interaction model exactly, d2n g vanishes and can thus be viewed as a generalized measure of distance between true and constraint model. On the other, the parameter sets corresponding to the hypotheses of (9.27) depend in a complicated way on the rs

284

TESTS FOR ESTABLISHING GOODNESS OF FIT

entries in the incidence matrix n and will be comparable in form to those considered in the balanced-data case only for small to moderate values of (maxi,j nij / mini,j nij ) − 1. In presence of gross differences between the cell frequencies, a more satisfactory solution to the problem of testing for negligibility of interactions might be obtained by keeping the hypotheses unchanged and constructing √  a large-sample2 test for (9.19) based on the asymptotic disγ /S, 0) − d (γ/σ, 0) . Up to now, this approach has not tribution of N d2 (ˆ been pursued in the literature so that nothing can be said about how the resulting procedure compares with that based on (9.26).

9.3.2

Establishing negligibility of carryover effects in the analysis of two-period crossover trials

In this subsection, we suppose that the data set under analysis stems from a standard two-period crossover trial involving two different treatments, say A and B. Trials of this type are very popular in various branches of clinical research and give in particular the standard case of a comparative bioavailability study to which the final chapter of this book is devoted in full length. The decisive feature of the two-period crossover design is that each experimental unit receives both treatments under comparison in temporal succession. Since there are of course two possible sequences in which the treatments can be administered the total sample of N = m + n subjects is randomly split into sequence groups A/B and B/A, say. In the first of these groups, each subject receives treatment A at the beginning of the first period of the trial, and is switched to treatment B in a second period. Usually, between the end of Period 1 and the beginning of Period 2, a sufficiently long washing-out period is scheduled. The only difference in the second sequence group B/A as compared to the first one is that the order of application of the treatments is reversed. It is furthermore assumed that the same univariate quantitative outcome measure is used in both periods to describe each subject’s biological condition after administration of the respective treatment. A convenient notation is obtained by defining Xki and Ykj to be the outcome observed in period k = 1, 2 in the ith and the jth subject of sequence group A/B and B/A, respectively. Thus, a complete description of the layout is as follows: Sequence Group

Period 1

Period 2

A/B B/A

X11 , . . . , X1m Y11 , . . . , Y1n

X21 , . . . , X2m Y21 , . . . , Y2n

At first sight, such a design looks simply like a special case of a twoway ANOVA layout with both factors varying over just two levels. However, there is a marked difference deserving keen attention: In the crossover

NEGLIGIBILITY OF CARRYOVER EFFECTS

285

case, the observations from different cells are not mutually independent but form pairs within each row. In other words, the data set obtained from a two-period crossover trial consists of two unrelated samples of bivariate observations rather than four samples made up of one-dimensional variables. By far the most popular approach to analyzing the data of a standard crossover trial is based on a simple parametric model originally introduced by Grizzle (1965). Of course, it allows in particular for correlations within all pairs (X1i , X2i ), (Y1j , Y2j ) and can be written (1)

+ εki ,

(2)

+ εkj ,

Xki = μk + Si Ykj = νk + Sj (1)

(2)

(1)

(1)

i = 1, . . . , m, k = 1, 2 ,

(9.28a)

(2)

j = 1, . . . , n, k = 1, 2 ,

(9.28b)

(2)

where all Si , Sj , εki , εkj are mutually independent with, (1)

(2)

Si , Sj and

(1)

(2)

ε1i , ε1j ∼ N (0, τ12 ) , (1)

∼ N (0, σS2 ) (1)

(2)

ε2i , ε2j ∼ N (0, τ22 ) .

(9.29) (9.30)

(2)

Usually, Si and Sj is called the (random) effect of the ith and jth subject in sequence group A/B and B/A, respectively, σS2 the between-subject variance, and τk2 the within-subject variance for the kth period. For the expected values μk , νk of the Xki , Ykj , Grizzle’s model assumes that these allow to be additively decomposed into a overall mean ω, a “direct treatment effect” φA or φB , a period effect πk , and for k = 2, a carryover effect denoted here λ(1) [↔ A/B] and λ(2) [↔ B/A], respectively. The corresponding model equations are μ1 = ω + φA + π1 ,

μ2 = ω + φB + π2 + λ(1) ,

(9.31a)

ν1 = ω + φB + π1 ,

ν2 = ω + φA + π2 + λ(2) .

(9.31b)

Clearly, the carryover effects reflect deviations from strict additivity of treatment and period effects and are thus another instance of interaction parameters. If they have some common value, say λ(0) , they can be totally dropped from the above model equations since then ω + λ(1) and ω + λ(2) are simply relabeled versions of the same overall mean. Thus, with regard to the carryover effects in a two-period crossover setting, negligibility means equality except for irrelevant differences. In authoritative expositions of inferential methods for the analysis of crossover trials (see, e.g., Jones and Kenward, 2003) in addition to Grizzle’s pioneering paper of 1965), it is much more clearly understood than in the literature on classical ANOVA methods, that the primary aim of inference on interactions consists in ruling out the possibility that such effects must be taken into account. The standard argument which is usually presented in favor of this view is that otherwise none of the differential main effects φA − φB

286

TESTS FOR ESTABLISHING GOODNESS OF FIT

and π1 − π2 admits unbiased estimation from the full data set. Nevertheless, also in this field, very little has changed over the years with regard to the usual practice of assessing the difference between both carryover effects: A preliminary test of the null hypothesis λ(1) = λ(2) is carried out and a nonsignificant result considered sufficient to warrant the conclusion that the λ(k) can be discarded when assessing the main effects. As a tried way around the flaw that this test is obviously directed to the wrong side, Grizzle suggested to raise the nominal significance level to α = .10, and this advice was adopted by Jones and Kenward in the first edition of their book. Of course, it is our aim here to show how this makeshift solution of the problem of establishing approximate additivity of treatment and period effects in a crossover layout can be replaced with a rigorous inferential procedure. It will turn out that the problem can readily be reduced in such a way that its solution requires no new ideas but simply an application of some of the methods developed in Chapter 6. In order to see this, let us recall the rationale behind the standard parametric test for the traditional two-sided problem λ(1) = λ(2) vs. λ(1) = λ(2) . The initial step, from which the rest follows almost automatically, consists of forming for each subject the total response by summing up his responses to both treatments. Defining Xi+ = X1i + X2i ∀ i = 1, . . . , m,

Yj+ = Y1j + Y2j ∀ j = 1, . . . , n ,

(9.32)

it is a straightforward exercise to verify by means of (9.28)–(9.31) that the Xi+ and Yj+ make up two independent samples from Gaussian distributions with common variance 2 = 4 σS2 + τ12 + τ22 (9.33) σ+ and expected values μ+ = E(Xi+ ), ν+ = E(Yj+ ) such that μ+ − ν+ = λ(1) − λ(2) .

(9.34)

In view of (9.33) and (9.34), it is clearly appropriate to formulate the problem of establishing negligibility of carryover effects in the model due to Grizzle (1965) as the problem of testing H+ : |μ+ − ν+ |/σ+ ≥ ε versus K+ : |μ+ − ν+ |/σ+ < ε

(9.35)

by means of the within-subject totals (9.32). Since the latter satisfy 2 Xi+ ∼ N (μ+ , σ+ ) ∀i,

2 Yj+ ∼ N (ν + , σ+ ) ∀j ,

(9.36)

(9.35) is simply a special case of (6.2) [→ p. 119]. Thus, there is a UMPI level-α test which rejects H+ if and only if the absolute value of the ordinary two-sample statistic T+ , say, calculated with the Xi+ and Yj+ as the raw data,  turns out smaller than F1,N −2; α (mnε2 /N ) [for the definition of F1,N −2; α , recall (9.22)].

NEGLIGIBILITY OF CARRYOVER EFFECTS

287

Example 9.5 It is instructive to compare the exact optimal test for (9.35) with the “inverted” two-sided t-test through applying both procedures to the same data set. Table 9.9 shows the results of a classical crossover trial of the efficacy of a certain oxygen gel [↔ (B)] in the improvement of dental plaque and gingival-inflammation, as compared with placebo and used at several places in the biostatistical literature for illustrating various techniques useful for the analysis of crossover trials (Brown, 1980; Jones and Kenward, 1989). The values displayed in the table represent improvements in a summary score of oral hygiene during each trial period. Table 9.9 Improvements in a summary score of oral hygiene observed in a crossover trial comparing a new oxygen gel (B) with placebo (A). (Data originally reported by Zinner et al. (1970); reproduced from Brown (1980), with kind permission by Wiley-Blackwell Publishing, Oxford.) Group A/B i

1

2

3

4

5

6

7

8

9

10

11

12

X1i X2i

0.83 1.00 0.67 0.5 0.50 0.83 1.0 0.67 0.67 0.33 0.00 1.17 1.83 2.17 1.67 1.5 2.33 1.83 0.5 0.33 0.50 0.67 0.83 1.33

Xi+

2.66 3.17 2.34 2.0 2.83 2.66 1.5 1.00 1.17 1.00 0.83 2.50 13

14

15

16

17

18

19

20

21

22

23

24

0.00 0.50 0.33 0.33 0.50 1.00 0.00 0.5 −0.50 0.17 1.00 1.00 0.67 1.83 1.50 1.50 1.17 1.67 1.33 1.5 2.83 2.33 1.33 1.67 0.67 2.33 1.83 1.83 1.67 2.67 1.33 2.0 2.33 2.50 2.33 2.67 25

26

27

28

29

30

31

32

33

34

1.33 0.33 0.67 0.83

2 1

4.00 0.83 0.50 0.5 0.50 2.17 0.67 0.17 1.67 1.33 1.5 1.67 1.33 1.17

2.00 1.16

3

4.17 2.50 1.83 2.0 2.17 3.50 1.84

Group B/A j

1

2

3

4

5

6

7

8

9

10

11

12

Y1j Y2j

1.67 2.5 1.00 1.67 1.83 0.50 1.33 1.33 0.50 2.17 1.67 1.5 0.33 0.5 −0.17 0.50 0.50 0.33 0.67 0.00 0.17 0.83 0.33 0.0

Yj+

2.00 3.0 0.83 2.17 2.33 0.83 2.00 1.33 0.67 3.00 2.00 1.5

288

TESTS FOR ESTABLISHING GOODNESS OF FIT Table 9.9 (continued) 13

14

15

16

17

18

19

20

21

22

23

24

1.33 1.5 1.33 0.67 1.67 2.50 1.83 0.83 2.33 1.17 1.33 1.33 0.50 0.5 0.00 −0.17 0.50 0.67 0.00 0.67 0.17 0.50 0.00 0.83 1.83 2.0 1.33 0.50 2.17 3.17 1.83 1.50 2.50 1.67 1.33 2.16 25

26

27

28

29

30

0.33 2.17 1.00 0.33 1.17 0.5 1.33 1.17 0.33 1.00 0.17 0.5 1.66 3.34 1.33 1.33 1.34 1.0

As argued by Jones and Kenward (1989, p. 39), the 28th subject of Group A/B should be removed as an outlier. With the remaining N = 33 + 30 within-subject totals, the t-statistic is computed to be *  1/2 33 · 30 (2.0552 − 1.7883)/ (32 · .6993312 + 29 · .7306542 )/61 T+ = 63 = 1.4810. In the conventional two-sided test, the associated p-value is .1438 so that the null hypothesis λ(1) = λ(2) has to be accepted also at the 10%-level. According to Grizzle’s rule there is thus no reason to suspect that the standard test for assessing the direct treatment and period effects might lead to invalid conclusions. Performing the t-test for equivalence of both distributions of within-subject totals instead, it seems appropriate to rely on a rather strict specification of the tolerance ε determining the hypotheses of (9.35). Following the recommendations put forward in § 1.5 this amounts to setting ε = .36, and with that value of ε the critical constant of the UMPI test for (9.35) at level α = .05 based on samples of size m = 33 and n = 30 is found to be (by means of the SAS intrinsic function for quantiles of arbitrary F -distributions) 1/2  = finv(.05,1,61,2.0366)**.05 F1,61; .05 (33 · 30 · .362 /63) = 0.173292 . Since the observed value of |T+ | is far beyond this bound, our formal test for absence of relevant carryover effects does not lead to a positive decision in the case of the dental hygiene study of Zinner et al. Having a look at the power of the test, this result is far from being surprising: Even under the null alternative λ(1) = λ(2) , the rejection probability is as low as 13.7%, and in a balanced trial, 133 subjects in each sequence group would be required for

NEGLIGIBILITY OF CARRYOVER EFFECTS

289

raising the chance of detecting perfect additivity to 80%. From a different perspective, this result corroborates the conclusion drawn by Brown (1980) from his analysis of the efficiency of the carryover relative to a simple parallel-group design: The frequently invoked superiority of the former holds only in those cases where additivity of the main effects can be taken for granted a priori and need not be established by means of a valid preliminary test providing reasonable power to detect that the restriction λ(1) = λ(2) is actually satisfied. Moreover, even if the preliminary test for carryover effects is performed in the logically adequate version treating the assumption of nonexistence of relevant carryover effects as the alternative hypothesis and the sample sizes are sufficiently large for ensuring reasonable power against the specific alternative λ(1) = λ(2) , serious doubts have to be raised regarding its use as part of a two-stage procedure. In fact, the results obtained by Freeman (1989) on the traditional two-stage procedure using the intraindividual differences between both periods for assessing the treatment effects only conditionally on a sufficiently large p-value of the conventional two-sided test for differences between the carryover effects, are likely to apply also to the equivalence version of the latter. In other words, a two-stage procedure starting with carrying out the UMPI test for (9.35) as a pre-test for making sufficiently sure that the assumption of negligibility of carryover effects holds at least approximately, tends to be anticonservative. Consequently, preliminary testing for negligibility of the carryover effects can only be recommended when a separate pilot sample has been made available not to be used in the main testing step, whether or not the test for differences between treatment effects will eventually be chosen to involve the observations taken in both periods.

Alternative approaches (i) Assessing raw rather than standardized effects. Sometimes, it might be preferable to measure the extent of the deviation from strict additivity of the main effects through the raw difference λ(1) − λ(2) between the carryover effects. In such cases, it is reasonable to replace the testing problem (9.35) with  : |μ+ − ν+ | ≥ ε H+

versus

 K+ : |μ+ − ν+ | < ε

(9.37)

and to apply the interval inclusion approach of § 3.1 to the latter. A guidance for specifying the equivalence range (−ε , ε ) in this alternative formulation of the problem of testing for negligibility of the carryover effects can be found in Brown (1980): Considering the power of the traditional two-sided test, this author recommends choosing ε = δ  /2 with δ  > 0 denoting the true difference one tries to detect in the standard test of equality of direct treatment effects. Suppose the authors of the study considered in the above Example 9.5 eventually aimed at detecting an effect of δ  = 1.00 of the oxygen gel

290

TESTS FOR ESTABLISHING GOODNESS OF FIT as compared with placebo. Then, in the interval inclusion test at level  ¯ +− would require that (−.50, .50) ⊇ (X α = .05 for (9.37), rejection of H+ + + + + + + ¯ − Y¯ + t61;.95 S˜ ), with S˜ denoting the standard erY¯ − t61;.95 S˜ , X + + ¯ ¯ ror of X − Y as estimated from the pooled sample. Since S˜+ can obvi¯ + − Y¯ + |/|T + | and we had X ¯ + = 2.0552, Y¯ + = ously be written S˜+ = |X 1.7883, T + = 1.4810, the 95%-confidence bounds to μ+ − ν+ are computed to be 0.2668 ± 1.67022 · .2668/1.4810 = 0.2668 ± .3009. Thus, with ε = .50 and at level α = .05, the null hypothesis of a nonnegligible absolute difference between the carryover effects must likewise be accepted.

(ii) Testing for negligibility of the carryover effects without relying on parametric distributional assumptions. In cases of doubt about the adequacy of the distributional assumptions (9.29), (9.30), it is generally advisable (cf. Jones and Kenward, 1989, § 2.8) to replace t-tests with their nonparametric counterparts. Of course, there is no reason for exempting the test for negligibility of the carryover effects from this recommendation. As we know from Chapter 6, there is a distribution-free analogue of the t-test for equivalence of two homoskedastic Gaussian distributions with respect to the standardized shift in location. With the computational tools presented in § 6.2, the Mann-Whitney test for equivalence of both distributions of within-subject totals is almost as easily carried out as the noncentral t-test. Specifying the equivalence range (1/2 − ε1 , 1/2 + ε2 ) = (.40, .60) for the target functional P [Xi+ > Yj+ ] (which, according to the results of § 1.7, is approximately the same as requiring |μ+ − ν+ |/σ+ < .36 in the Gaussian case), we obtain by means of the SAS macro mawi (or the R version of this program) with the data of Example 9.5: ˆ [W+ ] = .070076, CM W (.05; .10, .10) = .17259 . W+ = .59495, σ In view of |W+ −1/2|/ˆ σ [W+ ] = 1.3550 > CM W (.05; .10, .10), the decision is identical to that we were lead to take using the UMPI test at the same level for the parametric version of the problem.

9.4 9.4.1

Testing for approximate compatibility of a genotype distribution with the Hardy-Weinberg condition Genetical background, measures of HW disequilibrium

The so-called Hardy-Weinberg law plays a central role both in classical population genetics and modern genetic epidemiology of complex diseases. Since

Assessment of genetic equilibrium

291

genotyping is nowadays almost exclusively done at the highest level of resolution yielding data on single DNA bases, it suffices to restrict discussion to the Hardy-Weinberg model for biallelic markers called single nucleotide polymorphisms (SNPs) in contemporary genetic terminology. Except for loci on the sex chromosomes, genotyping in terms of SNPs generates trinomial frequency distributions as basic data. More precisely speaking, whenever a total of n individuals have been genotyped at some single nucleotide SNP involving two alleles denoted by A and B, the empirical genotype distribution found in the sample under consideration is given by the entries in the upper row of the following one-way contingency table: Genotype

AA AB

BB

Absolute Count Population frequency

X1 π1

X3 π3

X2 π2

 n 1.000

In traditional genetic notation, the corresponding allele frequencies are denoted by p [↔ A] and q [↔ B] where p ≡ π1 + π2 /2 ,

q ≡ π2 /2 + π3 .

(9.38)

The law discovered independently in 1908 by the British mathematician G.H. Hardy and the German physician W. Weinberg states that the distribution of a genotype involving two different alleles on each chromosome is reproduced in a 1:1 manner within the next generation if and only if the genotype frequencies (π1 , π2 , π3 ) can perfectly be reconstructed from the allele frequencies according to the following equations: π1 = p2 = (π1 + π2 /2)2 ;

(9.39a)

π2 = 2pq = 2(π1 + π2 /2)(π2 /2 + π3 ) ; π3 = q 2 = (π2 /2 + π3 )2 .

(9.39b) (9.39c)

It is important to note that, if the genotype frequencies for the next generation are denoted by (π1 , π2 , π3 ), then according to the basic rule of Mendelian population genetics we always have π1 = (π1 + π2 /2)2 , π2 = 2(π1 + π2 /2)(π2 /2 + π3 ) , and π3 = (π2 /2 + π3 )2 . Thus, Hardy-Weinberg’s law implies that, without “disturbing” influences from “outside,” a genotype distribution reaches equilibrium already in the next to current generation. Keeping in mind the boundary condition π1 + π2 + π3 = 1, it is easily verified that each of the above equations (9.39a–c) implies the other two. Thus, the genotype distribution under assessment is in strict Hardy-Weinberg equilibrium (HWE) if and only if there holds

√ ∀ π1 ∈ (0 , 1) . π2 = 2 π1 − π1 (9.40)

292

TESTS FOR ESTABLISHING GOODNESS OF FIT

The curve in the parameter space of (π1 , π2 ) given by this equation is usually graphically represented in a so-called DeFinetti diagram constructed through applying the following simple transformation of coordinates: (π1 , π2 ) → (π1 + π2 /2 , π2 ) .

(9.41)

One of the reasons why this transformation is very popular among statistical geneticists and genetic epidemiologists (see, e.g., Ziegler and K¨ onig, 2006, § 2.4) is that it maps that curve into the symmetric parabola with equation y = 2x(1 − x). The assessment of departure from HWE is still the most important and frequently used tool of quality control in association studies on genetic risk factors for complex diseases involving unrelated individuals. A standard strategy is to filter markers that do not conform with HWE prior to the conduct of genetic association tests (for details see Ziegler and K¨onig, 2006, Ch. 4). Unfortunately, the traditional statistical procedures to assess HWE are not tailored for establishing compatibility with the model which is the actual aim in the majority of applications of that type. Specifically, this statement holds true for the standard Pearson χ2 -test which to call a goodness-of-fit test is as misleading in the present context as in the settings considered in the first two sections of this chapter. Actually, the corresponding null hypothesis is that the distribution underlying the data is in agreement with HWE, and a significant test result indicates incompatibility of the observed data with the model. Accordingly, this approach allows only the identification of markers with the most severe deviations from HWE so that two basic issues remain. First, one cannot conclude that the remaining markers, for which the test does not reject its null hypothesis, do conform to the HW proportions of (9.39), since it may simply lack the power to detect even a gross violation of the equilibrium condition. The second and perhaps more troubling problem is that, given the large sample sizes that are nowadays typically used for studies of complex diseases, the null hypothesis of perfect HWE in the underlying population may be rejected because of high power for detecting a small, irrelevant deviation from HWE. A promising way around these difficulties consists of constructing a test of the null hypothesis that the true population frequencies (π1 , π2 ) correspond to a point in the parameter space which lies outside a suitable indifference zone around the curve determined by the family of all genotype distributions being in exact HWE, versus the alternative of equivalence with HWE. The first step to be taken on the way towards accomplishing this goal is to identify a function of the true genotype frequencies (π1 , π2 , π3 ) which gives a reasonable measure of HW disequilibrium (HWD). The specific measure of HWD we will use in the sequel combines mathematical convenience with ease of genetic interpretability. We start with elaborating briefly on the latter aspect. Clearly, there are many equivalent ways of re-expressing the complete set (9.39a–c) of conditions being jointly necessary and sufficient for HWE, through

Assessment of genetic equilibrium

293

a single restriction on the components of the parameter of the underlying trinomial distribution. For the present purpose, the most interesting of them is given by the statement 3 

 √  uj = 1 . (9.42) π2 = 2 π1 π3 ∀(π1 , π2 , π3 ) ∈ (u1 , u2 , u3 ) ∈ (0, 1)3  j=1

Equivalence between the statements (9.39a–c) and (9.42) (for a short formal proof see Wellek et al., 2009, Section A.1) implies that the degree of Hardy-Weinberg disequilibrium (HWD) is reflected by the extent to which √ the actual proportion π2 of heterozygotes differs from the proportion 2 π1 π3 of heterozygotes expected in a population which conforms exactly to HWE. This justifies considering the ratio √ ω ≡ (π2 /2) π1 π3 (9.43) as a measure of the relative excess heterozygosity (REH) and declaring the actual genotype distribution under analysis for being sufficiently close to HWE if and only if it can be taken for granted that there holds 1 − ε1 < ω < 1 + ε2 for suitably chosen equivalence margins 0 < ε1 < 1, ε2 > 0 . In § 9.4.3, we will argue that a choice which can well be recommended for routine applications in genetic association studies is to set the equivalence limits to ω equal to 1 − ε1 = 1/(1 + ε◦ ) , 1 + ε2 = 1 + ε◦ , with ε◦ = .4 . Figure 9.1 visualizes the corresponding equivalence region in the parameter space of (π1 , π2 ) in a DeFinetti diagram. The basic features of this band are • symmetry of both boundary curves about the line {(1/2, π2 )|0 ≤ π2 ≤ 1} • increasingness of the vertical width in the allele frequency p on the left and in (1 − p) on the right of the axis of symmetry. The subsequent subsection is devoted to the construction of an exact optimal test for establishing the (alternative) hypothesis that the genotype distribution generated through determining a given SNP in the population under study corresponds to a point in the (p, π2 )-plane lying inside the equivalence band shown in Figure 9.1. In § 9.4.3, the same testing problem will be addressed by means of the principle of confidence interval inclusion. In its asymptotic version, the interval inclusion test will turn out to be computationally very simple, making it better suited for applications in genomewide association studies involving many hundred thousands of SNPs as compared with the exact UMPU test.

9.4.2

Exact conditional tests for absence of substantial disequilibrium

First of all, we introduce a bit of additional notation setting θ≡

π22 , π1 (1 − π1 − π2 )

(9.44)

294

TESTS FOR ESTABLISHING GOODNESS OF FIT

Figure 9.1 Equivalence band obtained by applying the condition 1 − ε1 < ω < 1 + ε2 with ε1 = 2/7, ε2 = 2/5. [Middle solid line: HWE-curve (9.40), transformed by means of (9.41).]

ϑ ≡ π1 /(1 − π1 − π2 ) , and cn (θ, ϑ) ≡ (1 +

√ θϑ + ϑ)−n .

(9.45) (9.46)

The starting-point for the construction of an exact UMPU test for approximate compatibility of a SNP-based genotype distribution with HWE is given by the fact that the joint probability mass function of the absolute genotype counts X1 [↔ AA] and X2 [↔ AB] in a random sample of size n admits the representation pθ,ϑ (x1 , x2 ) =

n! θ x2 /2 ϑ(2x1 +x2 )/2 cn (θ, ϑ) . x1 !x2 !(n − x1 − x2 )!

(9.47)

Obviously, (9.47) implies that the possible of (X1 , X2 ) form √ distributions √   a two-parameter exponential family in log θ, log ϑ and (T, S), with the sufficient statistics being given by T = X2 , S = 2X1 + X2 . Comparing (9.44)

Assessment of genetic equilibrium

295

with (9.43), it is likewise obvious that the two parametric functions ω and θ are related through the simple equation θ = 4ω 2 ,

(9.48)

implying that each of them is a continuous and strictly increasing function of the other. Accordingly, the testing problem H : ω ≤ 1 − ε1 or ω ≥ 1 + ε2

K : 1 − ε1 < ω < 1 + ε2

versus

(9.49)

is the same as H : θ ≤ 4(1−δ1 ) or θ ≥ 4(1+δ2 ) versus K : 4(1−δ1 ) < θ < 4(1+δ2 ) , (9.50) provided, the δ’s are specified to be the following transforms of the ε’s: δ1 = 1 − (1 − ε1 )2 , δ2 = (1 + ε2 )2 − 1 .

(9.51)

In view of (9.47) √ and the continuity and strict increasingness of the transformation θ → log θ, it is clear that the testing problem (9.50) is another special case to which the general result stated in the Appendix as Theorem A.2.2 applies. Hence, we can conclude that a UMPU level-α test exists and is defined by the following decision rule to be followed for any fixed value s of the total number S of A alleles: ⎧ 1 2 ⎪ ⎨ reject H if Cα;s + 2 ≤ X2 ≤ Cα;s − 2 ; ν ν reject H with probability γα;s if X2 = Cα,s , ν = 1, 2 ; (9.52) ⎪ ⎩ 1 2 accept H if X2 ≤ Cα;s − 2 or X2 ≥ Cα;s + 2 . ν ν , γα;s (ν = 1, 2) have to be determined through The critical constants Cα;s solving the equations 2 Cα;s −2



1 +2 x2 =Cα;s

pθ1 ;s (x2 )

+

2 

ν ν γα;s pθ1 ;s (Cα;s ) = α =

ν=1 2 Cα;s −2



1 +2 x2 =Cα;s

pθ2 ;s (x2 )

+

2 

ν ν γα;s pθ2 ;s (Cα;s ),

(9.53)

ν=1

where pθ1 ;s (·) and pθ2 ;s (·) stands for the probability mass function of the conditional distribution of X2 given S = s under θ = θ1 ≡ 4(1 − δ1 ) and θ = θ2 ≡ 4(1 + δ2 ) , respectively. The conditional distribution of X2 which has been investigated by Stevens already in the 1930s, has a comparatively simple mathematical form. In order to provide an explicit formula for its mass function under any value of θ, we have first to note that, given any fixed value s ∈ {0, 1, . . . , 2n} of S, the set

296

TESTS FOR ESTABLISHING GOODNESS OF FIT

of possible values of X2 , say Wsn (X2 ), depends both on whether s is odd or even and smaller or larger than n. Actually, there holds: ⎧ 0, 2, 4, . . . , s for even s ≤ n ⎪ ⎪ ⎨  1, 3, 5, . . . , s odd s ≤ n n . (9.54) Ws (X2 ) =  0, 2, 4, . . . , 2n − s even s ≥ n ⎪ ⎪ ⎩ 1, 3, 5, . . . , 2n − s  odd s ≥ n Furthermore, let us define: Csn (j) =

n! ((s − j)/2)!j!(n − j/2 − s/2)!

for all j ∈ Wsn (X2 ) .

(9.55)

Then, the conditional probability of observing x2 heterozygotes in a sample where exactly s A-alleles are counted, is given by +   n x2 /2 Csn (x2 ) θ x2 /2 . (9.56) Pθ [X2 = x2 |S = s] = Cs (x2 ) θ x2 ∈Wsn (X2 )

An algorithm for computing the critical constants of the UMPU level-α test (9.52) for the equivalence problem (9.50) from Equations (9.53) is obtained through adapting the computational scheme presented in § 3.3 to the special case given by (9.56). An implementation allowing for sample sizes not exceeding 600 has been made available as a SAS macro in a Web appendix to Wellek (2004). An improved version with no limitation concerning the sample size is provided as accompanying material to Goddard et al. (2009) and can be found in the WKTSHEQ2 Source Code Package under the program name gofhwex both as a SAS macro and a R function. It is of some importance to note that the test given by the decision rule (9.52), is not only UMPU but also remains invariant under exchanging the counts X1 and X3 of both homozygous genotypes. The reason why this is a desirable property of any test for assessing compatibility of some biallelic genotype distribution for compatibility with HWE comes from the fact that labeling of the two alleles discriminated in the course of genotyping is completely arbitrary so that the same is true for assigning the two possible homozygous variants to the outer cells of the underlying contingency table. Accordingly, in a reasonable formulation of the testing problem, both hypotheses should be invariant against exchanging π1 and π3 , and it follows immediately from (9.43) that (9.49) satisfies this condition for any choice of the equivalence margins εν . It is not difficult to see that the conditional test obtained above has the analogous property in terms of X1 and X3 : Conditioning on {2X1 + X2 = s} leads to the same distribution of X2 as fixing the value of the sufficient statistic S = 2X1 + X2 at 2n − s, and it is obvious that 2X1 + X2 = 2n − s occurs if and only if s is the value taken on by 2X3 + X2 . Another nice property of the UMPU test (9.52) for approximate compatibility of a genotype distribution with the Hardy-Weinberg law is the possibility of carrying out exact power calculations. For the purpose of computing

Assessment of genetic equilibrium

297

the rejection probability βHWE (π), say, under an arbitrary configuration π = (π1 , π2 , π3 ) of genotype frequencies in the population, it is particulary convenient to use the representation βHWE (π) =

 (x1 ,x2 ,x3 )∈Xn

φ(x1 , x2 , x3 )

n! π x1 π x2 π x3 , x 1 ! x2 ! x3 ! 1 2 3

(9.57a)

with Xn = {(x1 , x2 , x3 ) | xj ∈ IN◦ ∀j = 1, 2, 3, x1 + x2 + x3 = n} and

⎧ 1 2 ⎪ 1 x ∈ C , C 2 ⎪ α;2x1 +x2 α;2x1 +x2 ⎨ ν ν φ(x1 , x2 , x3 ) = γα;2x . (9.57b) for x2 = Cα;2x1 +x2 , ν = 1, 2 1 +x2 ⎪ ⎪ ⎩ 1 2 0 x2 < Cα;2x or x2 > Cα;2x 1 +x2 1 +x2 ν ν , γα;2x recall (9.53).] [For the definition of the critical constants Cα;2x 1 +x2 1 +x2 Of course, the alternatives of major interest are those which specify that the population sampled is in perfect HWE. The power against any such al ternative is obtained by evaluating (9.57) at π = p2 , 2p(1 − p), (1 − p)2 for some p ∈ [0, 1] . Given the significance level and the equivalence margins, the form of the corresponding function of p (called the power function of the test in the sequel, despite the fact that it is actually only a section through the power function as the whole being defined on the two-dimensional domain {(π1 , π2 ) | 0 ≤ π1 + π2 ≤ 1}) highly depends on the sample size. Figures 9.2a-c show the power curves both of the exact UMPU test and its nonrandomized ν = 0 for ν = 1, 2 and all (x1 , x2 ) version [obtained through setting γα;2x 1 +x2 such that (x1 , x2 , n − x1 − x2 ) ∈ Xn in (9.57a)] for α = .05, the same specification of the equivalence band as in Figure 9.1, and n ∈ {200, 400, 800} . For each choice of the sample size, the power curves for both versions of the test are strictly unimodal and symmetric about p = 1/2. Furthermore, it becomes obvious from the graphs that the power of both the exact UMPU test and its nonrandomized counterpart approaches unity everywhere as n → ∞, with a speed of convergence which highly depends on the population allele frequency p . For large n, the differences between the nonrandomized and the exact version mainly reduces to the fact that the power converges to zero rather than α when p approaches one of the boundary points of the unit interval.

298

TESTS FOR ESTABLISHING GOODNESS OF FIT

Figure 9.2a Power of the conditional test for approximate compatibility with HWE against specific alternatives under which HWE is satisfied exactly, with ε1 = 2/7, ε2 = 2/5, α = .05 and n = 200. [Solid line: exact test; dashed line: nonrandomized version of the procedure.]

Figure 9.2b Analogue of Figure 2b for sample size n = 400.

Assessment of genetic equilibrium

299

Figure 9.2c Analogue of Figure 2a for n = 800.

Example 9.6 a) As regards the biological mechanism behind the genetic variant under study, the data set shown in Table 9.10 refers to a nonstandard case since the mutation consists in a loss of a single nucleotide rather than an exchange of some DNA base against another one. The total number of nondeleted alleles is 429, and for α = .05, ε = .40, n = 475, S = 429, the critical constants to be used in the UMPU test for approximate compatibility of the underlying population with HWE, are computed to be 1 1 2 = 215, γα;s = .9353, Cα;s = (by means of the program gofhwex) Cα;s 2 257, γα;s = .3552 . Since the observed number of heterozygotes is X2 = 233, the decision is in favor of equivalence of ω to unity so that the SNP Table 9.10 Genotype distribution observed through determining the deletion (D) allele of the angiotensin I converting enzyme gene (ACE) in the sample of controls in a study of possible associations between ACE and dementia. (Data from Richard et al., 2001.) Genotype

II

ID

DD

Absolute Count 98 233 144 Population frequency .2063 .4905 .3032

 475 1.000

300

TESTS FOR ESTABLISHING GOODNESS OF FIT passes the check for approximate compatibility with HWE (regarding the associated genotype distribution in the population of nondemented controls).

b) The data set shown in Table 9.11 refers to an ordinary SNP involving exchanges rather then deletions of nucleotides. Setting the significance level and the equivalence margins to the same values as before, the critical interval for X2 now to be used in the UMPU test (9.52) is found 1 2 to be (C.05;511 , C.05;511 ) = (167, 191). Since the observed value of the number of patients genotyped GT falls neither in the interior nor to the boundary of this interval, we have to accept the null hypothesis of relevant deviations from HWE in the underlying distribution of genotypes. Table 9.11 Genotype distribution with respect to a SNP within the endothelial nitric oxide synthase (NOS3) gene obtained in n = 393 individuals recruited as controls for a study on genetic risk factors for ischemic stroke. (Data from MacLeod et al., 1999.) Genotype

GG

GT

TT

Absolute Count 154 203 36 Population frequency .3919 .5165 .0916

 393 1.000

Noninferiority version of the test: Excluding existence of a substantial deficit of heterozygotes In genetic association studies, an important issue is the risk of inducing spurious associations between some genotype and the disease status under study through recruiting the subjects from mixture populations exhibiting marked genetic heterogeneity. With a view to this problem (usually called the problem of population stratification in genetic epidemiology — see, e.g., Ziegler and K¨ onig, 2006, § 10.4), a particularly relevant concern is to determine if there is excess homozygosity in the sample compared to the expected homozygosity under HWE. It is an immediate consequence of the concavity of the HWE curve [see Figure 9.1] that mixing different populations conforming with HWE always results in an excess of homozygotes, or in terms of the parametric function ω selected here, to a deficit of heterozygotes. Thus, for purposes of ruling out that population stratification has to be taken into account, it suffices to carry out a test tailored to establish the (alternative) hypothesis that the underlying genotype distribution corresponds to a point in the (π1 , π2 )-plane which does not fall below the lower boundary of the equivalence band shown in Figure 9.1 . Wherever such considerations apply, the testing problem (9.49) should be replaced by what is called in clinical applications of equivalence assessment

Assessment of genetic equilibrium

301

procedures the noninferiority version of that testing problem. Specializing the general considerations of Chapter 2 to the present setting, this leads to rewriting the hypotheses of interest as H1 : ω ≤ 1 − ε0 versus K1 : ω > 1 − ε0 .

(9.58)

The modifications to the decision rule (9.52) and the equations (9.53) determining the optimal critical constants which are required for obtaining a UMPU solution of (9.58) are largely analogous to those which were required in the binomial two-sample setting with the odds ratio as the distributional parameter of interest [recall § 6.6]. More precisely speaking, the decision rule we have to apply in carrying out a UMPU level-α test for (9.58) reads: ⎧ ⎪ ⎨ reject H1 if X2 ≥ Cα;s + 2 ; reject H1 with probability γα;s if X2 = Cα,s ; (9.59) ⎪ ⎩ accept H1 if X2 ≤ Cα;s − 2 . For determining the critical constants, there is only a single equation left to solve, namely  pθ0 ;s (x2 ) + γα;s pθ0 ;s (Cα;s ) , (9.60) x2 ≥Cα;s +2

where pθ0 ;s (x2 ) has to be defined as in (9.53) with substitution of θ0 = 4(1 − ε0 )2 for θ. For solving (9.60), another SAS macro is available in the WKTSHEQ2 Source Code Package under the filename gofhwex_1s.sas . The R-code of the same computational procedure can be found at the same place opening the file gofhwex_1s.R . Example 9.7 We illustrate the use of the noninferiority version (9.59) of the exact conditional test for absence of relevant deviations from HWE with an example taken from an association study of the impact of two APOE (Apolipoprotein E) promoter polymorphisms on the risk of Alzheimer’s disease. Table 9.12 shows the results of genotyping one of the control groups involved in this study for 491 A→T, the fist of the SNPs which were selected for that purpose. Table 9.12 Results of genotyping a sample of n = 133 controls in a study of the contribution of APOE promoter polymorphisms to the risk of developing Alzheimer’s disease. (Data from Lambert et al., 2002, Table 3.) Genotype

TT

AT

Absolute Count 6 53 Population frequency .0451 .3985

AA



74 133 .5564 1.000

302

TESTS FOR ESTABLISHING GOODNESS OF FIT

For n = 133 and s = 2 · 6 + 53 = 65, the critical constants to be used in (9.59) at significance level α = .05 and for the same (left-hand) equivalence margin as before, i.e., ε0 = 2/7 are obtained by means of the program gofhwex_1s to be Cα;s = 51, γα;s = .9376 . Thus, the observed value of X2 exceeds its critical lower bound implying that the null hypothesis of a relevant deficit of heterozygotes can be rejected. In contrast, under the same specifications, in the test (9.52) for approximate compatibility with HWE in the two-sided sense, the null hypothesis of relevant deviations from the model could not be rejected with the data of Table 9.12. Actually, running the program gofhwex 1 2 , C.05;65 ) = (49, 51) for n = 133 and thus a critical interval yields (C.05;65 whose intersection with the support Wsn (X2 ) of the conditional distribution of X2 [recall (9.54)] is empty. The exact power function of the one-sided test can be computed by essentially the same rationale being behind the graphs displayed in Figures 9.2a–c. The only change is that the critical function φ( · ) has to be specified on the basis of (9.60) rather than (9.52). Figure 9.3 shows the resulting power curve for a sample of size n = 400, together with that of the two-sided equivalence test. Except for the most extreme values of the allele frequency p in the population, the gain in power from discarding the upper boundary of the equivalence region is considerable.

Figure 9.3 Comparison of the power curve for the one-sided version of the test [solid line] with that of the test for approximate compatibility with HWE in the two-sided sense, for ε0 = ε1 = 2/7, ε2 = 2/5, α = .05 and n = 400.

Assessment of genetic equilibrium

9.4.3

303

Confidence-interval–based procedures for assessing HWE

Although the exact conditional testing procedure made available in § 9.4.2 is fully satisfactory from the statistical point of view, and its use for the analysis of association studies has been demonstrated in the genetic epidemiology literature (Goddard et al., 2009), the rationale behind it might not be easy to understand for applied researchers, and its practical implementation requires access to advanced statistical software. By such reasons the acceptance which the idea of replacing the classical lack-of-fit with goodness-of-fit tests for routine checks for HWE in large-scale genetic association studies will find, can be expected to improve substantially by providing procedures which are based on the principle of confidence interval inclusion as stated and proven as a general result in Chapter 3.1 of this book. An additional advantage of the availability of confidence procedures is that the resulting bounds and intervals can be used for decision making concerning large classes of testing problems involving hypotheses of very different forms. In particular, instead of assessing approximate compatibility of the underlying genotype distribution with HWE, each pair of confidence bounds can also be used for testing the traditional null hypothesis specifying that HWE holds exactly. Procedure based on the asymptotic distribution of the plug-in estimator of ω One natural approach to deriving an interval estimation procedure for the parameter ω consists in studying the asymptotic distribution of its natural estimator. This is obtained by replacing each population genotype frequency with its empirical counterpart in the expression on the right-hand side of Equation (9.43). Denoting this plug-in estimator by ω ˆ n , we have π ˆ2 X2 /n , =  ω ˆn = √ 2 π ˆ1 π ˆ3 2 (X1 /n)(X3 /n)

(9.61)

with π ˆj = Xj /n, j = 1, 2, 3 . Since the parameter space of ω is bounded on the left by zero, it is more promising to use the logarithm of ω ˆ n as a pivot for determining the desired confidence bounds. Based on the asymptotic distribution of log ω ˆ n which is again easily derived by means of the δ-method (full details of this derivation are given in the Appendix to Wellek et al., 2009), one obtains the following pair of confidence bounds for log ω at one-sided asymptotic confidence level 1 − α each: C lα (ˆ πn )

* ˆ2 1 1−π 1 , = log ω ˆ n − u1−α + n 4ˆ π1 π ˆ3 π ˆ2

C¯αl (ˆ π n ) = log ω ˆ n + u1−α

* ˆ2 1 1−π 1 , + n 4ˆ π1 π ˆ3 π ˆ2

(9.62)

304

TESTS FOR ESTABLISHING GOODNESS OF FIT

where u1−α denotes the (1 − α)-quantile of N (0, 1). The corresponding confidence interval for the relative heterzygosity ω on the original scale is given by *  

  ˆ2 1 1−π 1  ¯ , C α (ˆ π n ) , Cα (ˆ πn ) = ω ˆ n exp u1−α + n 4ˆ π1 π ˆ3 π ˆ2 * 

ˆ2 1 1 − π 1  ω ˆ n exp u1−α + . (9.63) n 4ˆ π1 π ˆ3 π ˆ2 For applications, it is desirable to represent the rejection region of the interval inclusion test for the problem (9.49) based on (9.62) or (9.63) as a subset of ˆ2 ) or, alternatively, by way of graphical visualizathe sample space of (ˆ π1 , π tion in a DeFinetti diagram. Again, following the better part of the genetic epidemiology literature, we adopt the latter frame of reference. The desired geometrical representation is obtained in the following way: For each possible value s/2n (s = 0, . . . 2n) of the allele frequency pˆ observed in a sample of size n, the confidence bounds of (9.62) are considered as functions of π ˆ2 , say cpˆ ; α (·) and c¯pˆ ; α (·), which in view of pˆ = π ˆ1 + π ˆ2 /2 are given by   cpˆ ; α (ˆ π2 ) = log π ˆ2 − (1/2) log(ˆ p−π ˆ2 /2) + log(1 − pˆ − π ˆ2 /2) *

1−π ˆ2 1 1 +π −u1−α n ˆ2 , (9.64a) 4(ˆ p−π ˆ2 /2)(1 − pˆ − π ˆ2 /2)   π2 ) = log π ˆ2 − (1/2) log(ˆ p−π ˆ2 /2) + log(1 − pˆ − π ˆ2 /2) c¯pˆ ; α (ˆ *

1−π ˆ2 1 + 1 . +u1−α n ˆ2 4(ˆ p−π ˆ2 /2)(1 − pˆ − π ˆ2 /2) π

(9.64b)

In order to determine the set of all π ˆ2 such that the asymptotic confidence interval assigned to the point (ˆ p, π ˆ2 ) satisfies the condition for rejecting the null hypothesis H : ω ≤ 1 − ε1 ∨ ω ≥ 1 + ε2 , we have to solve the equations cpˆ ; α (ˆ π2 ) = log(1 − ε1 ) ,

0≤π ˆ2 ≤ 2 min{ˆ p , 1 − pˆ}

(9.65a)

π2 ) = log(1 + ε2 ) , c¯pˆ ; α (ˆ

0≤π ˆ2 ≤ 2 min{ˆ p , 1 − pˆ} .

(9.65b)

and Neither one of the functions cpˆ ; α (·) and c¯pˆ ; α (·) is strictly monotonic so that the above equations typically admit several solutions. The natural way of determining the limits of the pˆ-section of the rejection region consists of selecting the solution that is nearest to the ordinate 2ˆ p(1 − pˆ) of the corresponding p ; α) point on the HWE curve on both sides. Denoting these values by π ˆ2;l (ˆ p ; α) [upper critical bound], respectively, it [lower critical bound] and π ˆ2;u (ˆ is possible (in particular for small sample sizes) that the interval determined by them is empty. Furthermore, for values of pˆ near the endpoints of the unit interval, a solution to equation (9.65a) and/or (9.65b) need not exist. p ; α) = In all these degenerate cases, we use the convention of setting π ˆ2;l (ˆ

Assessment of genetic equilibrium

305

p ; α) = 2ˆ p(1 − pˆ). Figure 9.4 shows the result of this construction, again π ˆ2;u (ˆ for the choice ε1 = 5/7, ε2 = 7/5 of the equivalence margins. The graph relates to the usual significance level α = .05 and a sample of size n = 200. In addition to the critical region of the asymptotic test based on (9.65a-b), it also shows the equivalence region specified under the alternative hypothesis. 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 9.4 Rejection region of the test for goodness of fit with HWE based on pairs of asymptotic 95%-confidence bounds for the relative excess heterozygosity parameter ω computed from a sample of size n = 200. [Dotted central line characterizes distributions being in exact HWE; outer dashed lines demarcate the theoretical equivalence band for ω obtained by specifying ε1 = 2/7, ε2 = 2/5.]

Exact conditional confidence limits The conditional distributions of the number X2 of heterozygotes given an arbitrarily fixed value s ∈ {0, . . . , 2n} for the total number S of A-alleles, constitute a one-parameter exponential family having in particular monotonically increasing likelihood ratios. Hence, exact confidence bounds of any prespecified level 1 − α can be computed for θ (and thus also ω) applying the usual procedure for STP2 families (cf. Lehmann and Romano, 2005, § 3.5) to the distribution given by (9.56) for each realized value s of the statistic S being sufficient for the nuisance parameter ϑ as defined in (9.45). Since each such distribution is of the discrete type and we do not want to rely on extraneous randomization as would be required for exhausting the non-coverage proba-

306

TESTS FOR ESTABLISHING GOODNESS OF FIT

bility α, we perform the construction in a conservative version fully analogous to that carried out in Bickel and Doksum (2001, pp. 244–6). In order to give a precise description of the procedure, we start with choosing an arbitrarily fixed point θ◦ in the range space of the basic parameter θ. Considering the p-values of the nonrandomized conditional tests of the one-sided null hypotheses Hl : θ ≥ θ◦ and Hr : θ ≤ θ◦ corresponding to this (l) (r) value as functions of θ◦ and denoting them by gx2 ; s,n (θ◦ ) and gx2 ; s,n (θ◦ ), respectively, we have    x /2 x /2 gx(l)2 ; s,n (θ◦ ) = Kn,s (x2 ) θ◦ 2 Kn,s (x2 ) θ◦ 2 , (9.66a) (2)

(2)

x2 ∈ Xn,s ∩ [0,x2 ]

(θ◦ ) = gx(r) 2 ; s,n

 (2) x2 ∈ Xn,s

x2 ∈ Xn,s x /2

Kn,s (x2 ) θ◦ 2

∩ [x2 ,∞)





x /2

Kn,s (x2 ) θ◦ 2

. (9.66b)

(2) x2 ∈ Xn,s

The monotone likelihood property of the underlying family of (conditional) distributions ensures (by another classical result stated, e.g., in Lehmann and Romano, 2005, as Theorem 3.4.1(ii)) that the second function is strictly increasing, implying strict decreasingness of the first function. Furthermore, it is obvious from their definitions that they are both continuous. Thus, each of the equations gx(l)2 ; s,n (θ◦ ) = α , 0 ≤ θ◦ < ∞ ;

gx(r) (θ◦ ) = α , 0 ≤ θ◦ < ∞ 2 ; s,n

(9.67)

has a unique solution. The first of them yields the desired upper confidence bound for θ with exact conditional coverage probability ≥ 1 − α. The solution (r) to gx2 ; s,n (θ◦ ) = α is a lower confidence limit with the same property. Of course, both values depend on x2 , s and n in addition to α which is made explicit by using the notation θ¯α ; n (x2 , s) and θ α ; n (x2 , s) for the exact conditional upper and lower confidence bound, respectively. The corresponding ¯ α ; n (x2 , s) for the relative excess heterozyconfidence bounds ω α ; n (x2 , s), ω gosity ω are eventually obtained through rescaling, i.e., as ω α ; n (x2 , s) =   θ (x2 , s)/2 , ω ¯ α ; n (x2 , s) = θ¯α ; n (x2 , s)/2 . Details of the correspondα;n

ing computational procedure are given in Wellek et al. (2009), and a set of SAS/IML modules for its implementation can be found in the WKTSHEQ2 Source Code Package [filename: cf_reh_exact.sas]. Confidence limits based on exact conditional mid-p-values The confidence bounds obtained by means of the procedure described above are exact but conservative, in the sense that the conditional coverage probabilities Pθ [θα ; n (X2 , S) < θ | S = s] , Pθ [θ¯α ; n (X2 , S) > θ | S = s] might be larger than required, i.e., > 1 − α . Such conservatism is clearly an intrinsic

Assessment of genetic equilibrium

307

property of confidence bounds based on families of nonrandomized tests for parameters of discrete distributions. A frequently recommended (see, e.g., Agresti, 2002, § 1.4.5) approach to reducing or even eliminating this conservatism consists of modifying the p-values involved by assigning a weight of 1/2 (instead of unity) to the probability that the test statistic takes on the observed value. In the present context, this so-called mid-p-value approach (l) (r) leads to replacing gx2 ; s,n (θ◦ ) and gx2 ; s,n (θ◦ ) in (9.66a–b) with (θ◦ ) ≡ gx(l∗) 2 ; s,n

 1  (l) gx2 −2; s,n (θ◦ ) + gx(l)2 ; s,n (θ◦ ) and 2  1  (r) (r) gx(r∗) g (θ ) ≡ (θ ) + g (θ ) , ◦ ◦ ◦ x2 +2; s,n 2 ; s,n 2 x2 ; s,n

(9.68)

respectively. The derivation of a confidence procedure from these modified p-values is fully analogous to the construction of exact confidence limits. In what follows, the resulting confidence bounds for ω are denoted ω ∗α ; n (x2 , s) and ω ¯ α∗ ; n (x2 , s) . The source code of a SAS/IML program for computing these statistics is stored in the file named cf_reh_midp.sas . It should be noted that the mid-p-value–based bounds are not guaranteed to maintain the target confidence level of 1 − α . However, numerical results from simulation studies of a variety of discrete families of distributions show that they are typically still conservative but to a markedly lesser extent than their exact counterparts. Illustration For illustration, we reanalyze the controls samples of a selection of the genetic association studies that were extracted by Wittke-Thompson et al. (2005) for consideration of the appropriate use of the classical tests for lack of fit to HWE. The sample sizes range from n = 58 to n = 801, and the observed ˆ n = 1.2964. The results relative excess heterozygosities from ω ˆ n = 0.7667 to ω of estimating in these samples 95%-confidence bounds for the relative excess heterozygosity by means of the three methods described above are shown in Table 9.13 in ascending order with respect to the observed value of the point estimate ω ˆ n of the relative excess heterozygosity. Specifying the hypothetical equivalence range for ω as before, approximate compatibility with HWE can be concluded by means of the asymptotic version of the interval inclusion test at nominal significance level .05 for 10 out of 19 samples listed in the table. Remarkably, only two of these equivalence decisions (relating to studies # 8 and 12) is affected by changing the method of constructing the confidence bounds to ω. More extensive comparisons between the asymptotic, exact and mid-p-based confidence limits revealed that the differences between the results of the mid-p and the asymptotic approach are distinctly smaller than

308

TESTS FOR ESTABLISHING GOODNESS OF FIT

those between either of them and the exact approach. This corroborates the rule stated by Agresti (2002, p. 21) that the mid-p method is markedly less conservative than the exact approach. Table 9.13 95%-Confidence bounds for the relative excess heterozygosity in the control subjects of 19 association studies reported by Wittke-Thompson et al. (2005). Study

Lower Confidence Bound Upper Confidence Bound

# X1 /X2 /X3 ω ˆn asympt. exact mid-p asympt. exact 1 3/23/75 0.7667 0.4235 0.3851 0.4168 1.3878 1.6439 2 75/35/6 0.8250 0.5280 0.4946 0.5213 1.2888 1.3937 3 61/212/237 0.8816 0.7487 0.7403 0.7470 1.0381 1.0465 4 66/80/31 0.8843 0.6841 0.6647 0.6790 1.1431 1.1638 5 297/258/71 0.8883 0.7651 0.7581 0.7637 1.0314 1.0384 6 57/192/197 0.9059 0.7632 0.7538 0.7612 1.0754 1.0848 7 197/349/186 0.9116 0.8071 0.8016 0.8058 1.0296 1.0337 8 58/97/48 0.9192 0.7292 0.7119 0.7246 1.1588 1.1752 9 77/41/6 0.9537 0.6186 0.5835 0.6127 1.4706 1.5891 10 193/393/215 0.9646 0.8587 0.8534 0.8574 1.0837 1.0876 11 98/233/144 0.9807 0.8421 0.8334 0.8400 1.1421 1.1492 12 23/83/67 1.0572 0.8082 0.7844 0.8024 1.3828 1.4122 13 62/65/15 1.0657 0.7797 0.7504 0.7727 1.4566 1.5020 14 16/76/79 1.0688 0.7966 0.7707 0.7909 1.4341 1.4755 15 112/132/33 1.0856 0.8740 0.8573 0.8704 1.3485 1.3678 16 27/26/5 1.1189 0.6691 0.6095 0.6522 1.8710 2.0542 17 114/560/533 1.1359 1.0179 1.0130 1.0171 1.2676 1.2725 18 174/45/2 1.2061 0.6397 0.6109 0.6554 2.2742 3.0015 19 98/77/9 1.2964 0.9206 0.8886 0.9176 1.8256 1.9182

mid-p 1.4570 1.3057 1.0370 1.1390 1.0304 1.0739 1.0285 1.1547 1.4917 1.0825 1.1402 1.3791 1.4550 1.4336 1.3462 1.8902 1.2671 2.5415 1.8401

Power, sample size and equivalence margin specification A major advantage of the interval inclusion test with asymptotic confidence bounds is that it allows for simple yet sufficiently accurate approximations to its power against arbitrary alternatives under which there holds ω = 1. In fact, if we denote the probability of rejecting the null hypothesis H of (9.49) in the test based on an asymptotic confidence interval computed from  (9.63) in a sample from a population satisfying π2 = 2 π1 (1 − π1 − π2 ) by βε∗1 ,ε2 ;α (π1 , n), then it can be shown (for a rigorous proof see Wellek et al., 2009) that for sufficiently large n, we can write  √ √ √ βε∗1 ,ε2 ;α (π1 , n) ≈ Φ n˜ ε2 · 2 π1 (1 − π1 ) − u1−α + √  √ √ Φ n˜ (9.69) ε1 · 2 π1 (1 − π1 ) − u1−α − 1 ,

Assessment of genetic equilibrium

309

where ε˜1 = − log(1 − ε1 ) , ε˜2 = log(1 + ε2 ) .

(9.70)

Choosing the equivalence interval symmetric on the log-scale through specifying ε2 = ε, ε1 = 1 − 1/(1 + ε) with ε > 0 and treating (9.69) like an ordinary (exact) equation to be solved for n, the minimum sample size required for ensuring a power of 1 − β, say, can be computed from n=

(u1−β/2 + u1−α )2 . √ √ (2 π1 (1 − π1 ))2 ε˜2

(9.71)

The sample size formula (9.71) is of considerable interest under a number of different aspects. In particular, it provides a basis for rationally justifying the recommendation of setting the equivalence margin ε at the value .40 which was used throughout this subsection in all examples etc. Elaborating this remark we start with making the largely plain statement that a positive result of any equivalence testing procedure is scientifically the more satisfactory the smaller the equivalence margins have been chosen. Notwithstanding this fact, it is clearly not reasonable to narrow the equivalence region to an extent that an unacceptably large proportion of SNPs would fail to be selected as HWE-compatible although they perfectly satisfy the model. These general considerations suggest to choose the equivalence margin as small as we can do under the following restrictions concerning (i) power, (ii) allele frequency, and (iii) sample size: (i) Over the whole range of values of frequency fAM I of the allele of major interest to be taken into account, the probability of rejecting lack of fit under a genotype distribution which exactly conforms to the model, must not be smaller than 90%. (ii) The power to detect an association in a case-control or cohort study is low for low values of fAM I . In current genomewide association studies (GWAs), the proportion of SNPs with fAM I < 0.1 associated with a disease is extremely low (see Johnson and O’Donnell, 2009). For SNPs defined eligible for inclusion in association studies, we therefore require 0.1 ≤ fAM I ≤ 0.5, corresponding to 0.01 ≤ π1 ≤ 0.25 under HWE. (iii) Current GWAs have up to 3,000 control subjects. The only exception are studies from Iceland, where more than 10,000 subjects served as controls for a variety of investigated diseases. Therefore, the required sample size should not exceed 3,000. √ Now, as a function of π1 which under HWE coincides with fAM I , the expression on the right-hand side of (9.71) is clearly decreasing on (0, 1/2]. Hence, the smallest equivalence margin leading to a sample size of n = 3000 for u1−β/2 + u1−α √ SNPs satisfying 0.1 ≤ fAM I ≤ 0.5, is calculated to be ε˜ = 2 · .1 · .9 · 3000

310

TESTS FOR ESTABLISHING GOODNESS OF FIT

2 · u.95 √ = .3337 . Retransforming the equivalence range −.3337 < 2 · .09 · 3000 log ω < .3337 to the original scale yields an equivalence margin of e.3337 − 1 = .3961 for ω itself so that we indeed end up with the proposal to specify ε = .4. =

Discussion As compared with the UMPU conditional test, the confidence-interval–based approach derived here provides the following major advantages: • conceptual lucidity; • ease of computational implementation; • flexibility with regard to changes of the equivalence margins and the form of the hypothesis to be established; • existence of simple closed formula for power and sample size. The first of these points is not specific to the problem of establishing approximate compatibility of a theoretical genotype distribution with the HWE model, and in general, conceptual simplicity of the confidence interval inclusion approach to equivalence testing does not ensure simplicity of computational implementation, as the exact and mid-p-value versions of the procedure show. However, the asymptotic version involves only elementary algebraic operations which makes it almost ideally suited as a tool for the working genetic epidemiologist. Moreover, as additional numerical results presented in Wellek et al. (2009) suggest, the corresponding rejection region is surprisingly similar to that of the exact conditional test even for moderate sample sizes. Even the most conservative of the confidence-limit based testing procedures yields rejection regions which are only slightly narrower than the critical region of the nonrandomized version of the UMPU test. This implies that in the present case, the loss in power entailed as the price to pay for computational simplicity is largely negligible from a practical point of view. One of the major advantages of the confidence-limit–based approach to HWE assessment over that via application of the exact conditional test is the ease of adaptation to different specifications of the margins under the hypothesis to be established. Actually, all that happens in applying one of the interval inclusion rules when the margins are changed, is that in the very final step a potentially different decision has to be taken. Computationally, interval inclusion rules remain completely invariant against numerical re-specification of the margins. In contrast, carrying out the exact conditional test with some other value of the equivalence margins requires re-execution of the full computational procedure for determining the bounds of the critical interval to X2 . Last but not least, for the UMPU test, power calculation is computationally quite demanding even for small or moderate values of n, and no analogue of formula (9.69) is available.

10 The assessment of bioequivalence

10.1

Introduction

Although the overwhelming majority of the existing literature on statistical tests for equivalence hypotheses refers to the analysis of comparative bioavailability trials (often also named bioequivalence trials, for the sake of brevity), the testing problems to be dealt with in the context of bioequivalence (BE) assessment are rather special in nature. This is because the regulatory authorities set up strict guidelines for the planning and analysis of bioequivalence studies quite soon after this type of trial had begun to play an important role in pharmaceutics and clinical pharmacology. According to these rules, every bioequivalence study has to be conducted as a phase-I clinical trial, which means that the subjects are healthy volunteers rather than patients suffering from a disease considered to be an indication for administering the drug under study. Furthermore, each subject has to be given several formulations of that drug in an experiment following a crossing-over scheme allowing for two different trial periods at least. The outcome measure to be used for characterizing the biological activity of the drug administered at the beginning of the respective period, has to be chosen from a very short list of pharmacokinetic parameters. The distributions of these outcome measures are of the continuous type throughout, modeled in the majority of cases in a fully parametric way, usually after logarithmic transformation of the primary measurements obtained during the trial. A complete exposition covering all statistical facets of the planning and analysis of BE trials gives sufficient material for a monograph of its own (see Chow and Liu, 2008; Hauschke et al., 2007; Patterson and Jones, 2005) so that we confine ourselves in this chapter to a discussion of the standard case of a bioequivalence trial involving two periods and as many different drug formulations. Actually, the restriction to this standard design is far from severe for practical purposes since real BE trials involving more than two periods and/or treatments are still a rare exception. Furthermore, even with regard to the 2×2 case, the controversy about the most appropriate statistical criterion for BE assessment is still very far from being settled and seems to be a never ending story within (bio-)statistical science. Not surprisingly, the major issue of the ongoing discussion refers to the conceptual level. In fact, from

311

312

THE ASSESSMENT OF BIOEQUIVALENCE

the mathematical and statistical perspective, it can be stated that there are very satisfactory solutions to an impressive manifold of problems put forward in this context. However, only a small number of them yield an intuitively plausible translation of the bio-medical question to be answered at the end of a BE trial into the formalism of decision making between statistical hypotheses. Thus, it can be argued that statistical BE assessment has developed into a field where more mathematically sophisticated solutions exist than problems worth the effort entailed in deriving solutions. In view of this situation, we prefer to concentrate on a careful discussion of the pros and cons of the most frequently applied approaches. Even within the scope of methods tailored for the analysis of BE trials following the standard design, we do not aim at a complete coverage of the existing literature. In return, the reader will find also some new ideas on the topic which lead beyond the solutions encouraged by the official guidelines or may even suggest to question the adequacy of the rationale behind the latter. The general frame of reference for the whole chapter is given by a conventional nonreplicated crossover design as had been considered from a totally different perspective in § 9.3.2. In the present context, the treatments under comparison are two drug products denoted “Test” (T ) and “Reference” (R), adopting the usage almost universally followed in the field of BE assessment. Typically and most commonly, T is a generic drug to be approved for the market, and R a reference listed drug containing chemically identical active ingredients. The total sample of subjects recruited to a standard BE trial consists of N healthy volunteers randomly assigned to one of the two possible sequence groups T /R and R/T . Although the randomization is usually performed under the restriction of generating a strictly balanced layout, by technical difficulties it might happen that a few subjects have been lost from one or the other group due to providing incomplete or otherwise insufficient pharmacokinetic data. Thus, we allow for arbitrary sample sizes m (↔ T /R) and n (↔ R/T ) in both sequence groups. The basic data set consists of two vectors per subject, whose components give the ordinates of the concentration-time profile recorded in both periods of the trial. (In the majority of BE studies, the concentration of the active ingredient of interest is measured in the blood plasma or serum.) Figure 10.1 shows a schematic example of such a pair of observed concentration-time profiles. Instead of analyzing the individual profiles as multivariate data, they are usually reduced in a preliminary step to a single real value by calculating one of the following pharmacokinetic characteristics: • area under the profile as a whole (AUC) • maximum concentration (Cmax ) • time at which the peak concentration Cmax was measured (tmax ).

Introduction

313

Figure 10.1 Example of a pair of concentration-time profiles observed in a given subject during both periods of a standard bioequivalence trial. (Redrawn from Rodda, 1990, with kind permission by Marcel Dekker, Inc.) Commonly, AUC and Cmax are considered as alternative estimates of the extent of the absorption process induced in a trial subject, whereas tmax is interpreted as measuring its rate. All three pharmacokinetic parameters are accepted as a reasonable choice for selecting a reference measure of bioavailability to be used as the endpoint variable in both periods and all subjects enrolled in the trial. The data set eventually to be analyzed for the purpose of BE assessment consists of two independent samples (X11 , X21 ), . . . , (X1m , X2m ), (Y11 , Y21 ), . . . , (Y1n , Y2n ) of pairs of one-dimensional random variables such that Xki = bioavailability measured in the ith subject (i = 1, . . . , m) of sequence group T /R during the kth period (k = 1, 2) ; Ykj = bioavailability measured in the jth subject (j = 1, . . . , n) of sequence group R/T during the kth period (k = 1, 2) . In order to ensure at least approximate normality of the distributions of these variables, it is generally recommended (see FDA, 2001, § VI.A) to determine their values by taking logarithms of the bioavailabilities computed from the individual concentration-time profiles. Thus, if the AUC has been selected as

314

THE ASSESSMENT OF BIOEQUIVALENCE

the measure of bioavailability of interest, then we have X11 = log(A11 ), Y11 = log(B11 ) with A11 and B11 denoting the area under the profile obtained in Period 1 in the 1st subject of group T /R and R/T , respectively, and so on. If not explicitly stated otherwise, we assume throughout (following once more the current official guidance for conducting in vivo BE studies) that the log-bioavailabilities Xki , Ykj satisfy the basic parametric model for the analysis of 2 × 2 crossover trials [see (9.28–31)] with the following slight modifications: The within-subject variability (usually interpreted as reflecting measurement error) is this time allowed to depend on treatment rather than period so that we let     (1) (2) 2 (10.1a) V ar ε1i = V ar ε2j ≡ σeT and

    (1) (2) 2 V ar ε2i = V ar ε1j ≡ σeR

(10.1b)

for all i = 1, . . . , m and j = 1, . . . , n. Furthermore, the direct treatment effects are now called formulation effects and written φT , φR instead of φA , φB . Except for § 10.5, we adopt the conventional view (supported by empirical evidence — see D’Angelo et al., 2001) that in analyzing a properly conducted BE trial, nonexistence of carryover effects can be taken for granted a priori, due to a careful procedure of eliminating the residual activity of the drug administered in the initial period during the washing-out interval. Of course, having renamed the treatment effects, this leads to the simplified model equations E(X1i ) = μ1 = ω + φT + π1 ,

E(Y1j ) = ν1 = ω + φR + π1 ,

E(X2i ) = μ2 = ω + φR + π2 (i = 1, . . . , m)

(10.2a)

E(Y2j ) = ν2 = ω + φT + π2 (j = 1, . . . , n) .

(10.2b)

In a broad sense, it is clear what bioequivalence of the two drug formulations means: sufficient similarity of the distributions of bioavailabilities measured after administration of T and R, respectively. However, there are obviously many ways of making this idea precise in terms of a hypothesis to be established by means of some suitable test of significance. The still most frequently used criterion focuses on the difference φT − φR of formulation effects. In the guidelines and the biostatistical literature on BE assessment, testing procedures allowing to decide whether or not a criterion of this type is satisfied, are called tests for “average bioequivalence.” Due to their overwhelming importance for biostatistical practice, the best established approaches to the assessment of average BE are discussed in the first of the core sections of this chapter. A totally different philosophy underlies the methods presented in § 10.3: “Individual bioequivalence” in the “probability-based” sense requires that in the underlying population, there is a sufficiently high proportion of

Methods of testing for average bioequivalence

315

subjects such that the bioavailabilities induced by T and R in the same individual are identical except for some pharmacologically irrelevant difference. Methods of testing for “population bioequivalence,” which will be discussed in § 10.4, aim at establishing that the distributions of bioavailabilities associated with T and R, are similar both with respect to location and variability. Finally, in § 10.5 we argue that any marked difference between the joint distribution of the (X1i , X2i ) and that of the (Y1j , Y2j ) is at variance with equivalence of both drug formulations with respect to the given measure of bioavailability. Accordingly, the testing procedure derived in this final section, are bivariate analogues of the univariate two-sample tests for continuous data discussed in Chapter 6.

10.2 10.2.1

Methods of testing for average bioequivalence Equivalence with respect to nonstandardized mean bioavailabilities

The traditional approach to BE assessment recommended even in the most recent official guidelines on statistical methods for the analysis of comparative bioavailability trials (FDA, 2001) as a standard procedure, focuses on the raw formulation effects φT , φR as appearing in the model equations (10.2). In the simplest version of the concept, average BE is defined to be satisfied if and only if the true value of φT − φR lies in a sufficiently small neighborhood of zero. By the so-called 80–125% convention, the radius of this neighborhood is specified (10.3) δ◦ = log(5/4) = .2231 , which can be motivated as follows: Suppose the period effects can also be dropped from the model equations (which, according to experience, holds true for the majority of real BE studies), and the within-subject variance is the same under both treatments (which is a standard assumption in the classical model for the 2 × 2 crossover). Then, the distribution of the nonlogarithmic bioavailability induced by formulation T and R is lognormal with parameters ω + φT , σS2 + σe2 and ω + φR , σS2 + σe2 , respectively. Hence (cf. Johnson et al., 1994, p. 212) the expected values, say μ∗T and μ∗R , of these distributions are given by μ∗T = exp{ω + φT + (σS2 + σe2 )/2}, μ∗R = exp{ω + φR + (σS2 + σe2 )/2}, so that their ratio is μ∗T /μ∗R = exp{φT − φR }. This shows that under the additional restrictions made explicit above, the condition |φT − φR | < log(5/4) is the same as 4/5 < μ∗T /μ∗R < 5/4, and it has become part of good pharmaceutical practice to consider the latter sufficient for neglecting the differences in the response to both formulations of the drug.

316

THE ASSESSMENT OF BIOEQUIVALENCE

Keeping the specification (10.3) in mind, assessment of average BE in the usual sense requires that we are able to perform a valid test of H : |φT − φR | ≥ δ◦

versus

K : |φT − φR | < δ◦ .

(10.4)

In order to see how such a test can be constructed, it is helpful to recall the standard parametric procedure of testing the conventional null hypothesis φT = φR in a 2 × 2 crossover design satisfying strict additivity of direct treatment and period effects. The basic step enabling an easy derivation of such a test consists in reducing the bioavailabilities (X1i , X2i ), (Y1j , Y2j ) observed in the successive trial periods, to within-subject differences measuring the change in response level from Period 1 to Period 2 (to be computed in that order throughout, irrespective of the sequence group to which a given subject has been assigned). Denoting this difference by Xi− (↔ Group T /R) and Yj− (↔ Group R/T ), respectively, we have Xi− = X1i − X2i ∀ i = 1, . . . , m, Yj− = Y1j − Y2j ∀ j = 1, . . . , n,

(10.5)

and it readily follows from (10.1) and (10.2) that Xi− ∼ N (φT − φR + π1 − π2 , σ 2 ) ∀ i = 1, . . . , m,

(10.6a)

Yj− ∼ N (φR − φT + π1 − π2 , σ 2 ) ∀ j = 1, . . . , n,

(10.6b)

with 2 2 σ 2 = σeT + σeR .

(10.7) Xi−

Yj−

Clearly, (10.6) implies that the within-subject differences and satisfy the assumptions of the ordinary two-sample t-test where the shift in location of both Gaussian distributions admits the representation μ− − ν− ≡ E(Xi− ) − E(Yj− ) = 2(φT − φR ) .

(10.8)

Thus, treating the Xi− and Yj− as the raw data, (10.4) is nothing but a special instance of the problem of testing for equivalence of two homoskedastic Gaussian distributions with respect to the absolute shift in location. As we know from § 3.1, the simplest solution of an equivalence testing problem of that kind is through an application of the principle of confidence interval inclusion. For a concise description of the corresponding decision rule, it is convenient to introduce the following notation making explicit that all statistics involved have to be computed from the inter-period differences defined by (10.5): ¯ − = (1/m) X

˜−

S

3( =

m  i=1

Xi−

, Y¯ − = (1/n)

n  j=1

Yj− ;

)   41/2 m n   1 1 − − − 2 − 2 ¯ ) + + (Xi − X (Yj − Y¯ ) . (m + n − 2) m n i=1 j=1

Methods of testing for average bioequivalence

317

In terms of these symbols, a pair of confidence bounds for φT − φR of the same one-sided level 1 − α reads ¯ − − Y¯ − ∓ S˜− tN −2; 1−α ) (1/2) · (X

(10.9)

with N = m + n and tN −2; 1−α denoting the (1 − α)-quantile of a central t-distribution with N − 2 degrees of freedom. Correspondingly, the level-α interval inclusion test of (10.4) is given by the decision rule: Reject average bioinequivalence if and only if it turns ¯ − − Y¯ − | < 2δ◦ − S˜− tN −2; 1−α . out that |X

(10.10)

Except for relying on confidence bounds of one-sided level 1 − α rather than 1 − α/2, this solution to the problem of average BE assessment has been proposed as early as in the seminal paper of Westlake (1972). Although the underlying criterion for the equivalence of two normal distributions with the same variance can be criticized for missing a crucial point (as will be explained in § 10.2.2), the method is still strongly recommended in the guidelines as a standard procedure. One of the undisputed advantages of the decision rule (10.10) is computational simplicity. Unfortunately, this advantage is more or less canceled out by the fact that exact power computations are rather complicated requiring numerical integration even if the null alternative φT = φR is selected. The power of the interval inclusion test (10.10) for average BE against an arbitrary specific alternative |φT − φR | = δ < δ◦ can easily be shown to admit the representation P OWδ◦ (δ, σ) =

v(δ◦ /σ;  m,n,α)3

0

*

  * mn mn ∗ 2(δ◦ − δ)/σ − vtα − Φ − · Φ N N 4 √ √  χ ∗ 2(δ◦ + δ)/σ + vtα N − 2 fN N − 2 v dv , −2

(10.11)

where    t∗α = tN −2; 1−α , v δ◦ /σ; m, n, α = mn/N 2(δ◦ /σ)/t∗α ,

(10.12)

χ and fN −2 (·) stands for the density function of a so-called χ-distribution with N − 2 degrees of freedom. By the usual definition (cf. Johnson et al., 1994, p. 417), the latter is explicitly given as χ 2−N/2 −u e fN −2 (u) = 2

2

/2

   · uN −3 Γ N/2 − 1 , u > 0 .

(10.13)

Numerical evaluation of the integral (10.11) can be carried out by means of the same technique recommended earlier [see § 3.2] for exact computation

318

THE ASSESSMENT OF BIOEQUIVALENCE

of marginal posterior probabilities for equivalence ranges. In the WKTSHEQ2 Source Code Package a couple of files with common basename pow_abe can be found containing R and SAS scripts of a program which enables its user to determine the power of the interval inclusion test (10.10) with higher accuracy than required for practical applications. The program allows for arbitrary specifications of the nominal significance level α, the sample sizes m, n in both sequence groups, and the parameters δ ∈ IR+ ∪ {0}, σ ∈ IR+ . Since δ can in particular be chosen as any point in [δ◦ , ∞), the procedure can also be used for studying the exact size of the test (10.10). Example 10.1 We illustrate the standard procedure of testing for average BE by applying it to the areas under the serum-concentration curves from time 0 to the last time with measurable concentration obtained in a bioequivalence study whose results have been available at the FDA web-site since December 1997, on a file named gen10.txt. Although the original trial aimed primarily at detecting a possible gender-by-treatment interaction, the data are analyzed here as in a conventional comparative bioavailability study. Table 10.1 displays the logAUC’s observed in each subject during both periods, together with the interperiod differences. Computing the empirical means and variances with the entries in the bottom lines of the two parts of the above table gives: ¯ − = -0.06058 ; S 2 − = .0277288 ; X X Y¯ − = -0.04515 ; SY2 − = .0336990 . Using these values of the sample variances, the pooled estimate of the standard ¯ − − Y¯ − is obtained as error of X  1/2 S˜− = (11 · .0277288 + 12 · .0336990)/23 · [1/12 + 1/13]1/2 = .070306 . Since the 95th percentile of the central t-distribution with N − 2 = 23 degrees ¯ − − Y¯ − | to be of freedom is t23; .95 = 1.713872, the critical upper bound to |X used in the interval inclusion test for average BE at level α = 5% is computed to be: 2 log(5/4) − .070306 · 1.713872 = .102648 . ¯ − − Y¯ − | = .015430, With the observed values of the sample means, we have |X so that applying rejection rule (10.10) leads to a decision in favor of average BE.  Taking the observed value of S˜− / 1/m + 1/n for the true value of the population standard deviation σ of a single within-subject difference gives σ = .175624, and under this assumption, the exact power of the test against δ = δ◦ /2 and δ = 0 turns out (running program pow_abe) to be .92415 and

Methods of testing for average bioequivalence

319

.99993, respectively. If the true value of σ is assumed to be twice as large, i.e., .351249, these power values drop to .45679 and .84831, respectively. Table 10.1 Logarithmically transformed AUC-values and inter-period differences computed from the FDA (1997) sample bioequivalence data set gen10. Sequence Group T/R i X1i X2i Xi−

1

2

3

4.639 4.093 4.222 4.501 4.353 4.353 0.138 −0.260 −0.131

4 4.549 4.580 −0.031

5 4.241 4.064 0.177

8

9

10

11

12

4.436 4.565

4.572 4.492

4.546 4.577

3.882 4.121

4.561 4.452

0.080 −0.031

−0.239

0.109

−0.129

6

7

4.279 4.309 4.618 4.380 −0.339 −0.071

Sequence Group R/T j

1

2

Y1j Y2j Yj−

4.746 4.560 0.186

4.521 4.486 0.035

8

9

10

11

12

13

4.369 4.363

4.445 4.598

4.714 4.831

4.018 4.199

4.145 4.129

4.449 4.539

0.006 −0.153

3

4

5

6

4.009 4.818 4.040 4.099 3.953 5.001 4.114 4.511 0.056 −0.183 −0.074 −0.412

−0.117 −0.181

7 4.140 3.816 0.324

0.016 −0.090

Modifications and extensions The interval inclusion test (10.10) has been considered by many authors from very different perspectives. Mandallaz and Mau (1981) showed that it admits a representation as a Bayesian testing procedure with respect to an noninformative reference prior for (μ− , ν − , σ), and Schuirmann (1987) pointed out its equivalence with a “combination” of two one-sided t-tests with shifted common boundary of the hypotheses [see also § 3.1]. From a mathematical point of view, it is perhaps its most striking property (becoming immediately ob-

320

THE ASSESSMENT OF BIOEQUIVALENCE

vious from (10.11), (10.12)) that, given any value of the (absolute) difference δ = φT − φR between the true formulation effects, its rejection probability converges to zero as σ → ∞, implying that there are specific alternatives against which the power is much smaller than the significance level α. Various attempts have been made to reduce or even totally eliminate this lack of unbiasedness of the standard procedure of testing for average BE. Anderson and Hauck (1983) proposed to use the rejection region 

    ¯ − | − 2δ◦ )/S˜− − F T ¯ − | − 2δ◦ )/S˜− < α , (−| D (10.14) FNT −2 (|D N −2 where FNT −2 (·) denotes the cdf of a central t-distribution with N −2 degrees of ¯ − − Y¯ − . The level and power properties of (10.14) were ¯− ≡ X freedom and D extensively studied by Frick (1987) and M¨ uller-Cohrs (1988) using numerical methods. First of all, the results obtained from these investigations show that the modification suggested by Anderson and Hauck leads to an anticonservative solution of the problem of testing for average BE which is not surprising. In fact, the function FNT −2 (·) is positive everywhere on IR so that (10.14) is      ¯ − | − 2δ◦ )/S˜− < α = (|D ¯ − | − 2δ◦ )/S˜− < a proper superset of FNT −2 (|D      ¯ − | < 2δ◦ − S˜− tN −2; 1−α ¯ − | < 2δ◦ + S˜− tN −2; α = |D (FNT −2 )−1 (α) = |D and thus of the critical region of the interval inclusion test which is well known to have exact size α. Aligning both tests with respect to size, the AndersonHauck procedure has better power against alternatives it detects with probability 60% at most. However, it still fails to be unbiased, but the bias is not as extreme as that of the interval inclusion test: For fixed δ ∈ [0, δ◦ ), the power of the Anderson-Hauck test against the alternative (δ, σ) converges to the nominal level rather than zero, as σ → ∞. Munk (1993) proposed a “mixture” of both tests defined by the rule that (10.10) has to be used for values of S˜− falling short of some cut-off value k∗ (depending on the target significance level α and the number N − 2 of degrees of freedom), and (10.14) with corrected nominal level α∗ < α for S˜− > k∗ . Finally, the testing problem (10.4) admits also of constructing a critical region which is strictly unbiased. This has been shown by Brown et al. (1997) adapting Hodges and Lehmann’s (1954) approach to deriving an unbiased test for the dual problem to be written H  : |φT − φR | ≤ δ◦ vs. K  : |φT − φR | > δ◦ in the present context. Unfortunately, the form of the unbiased critical region exhibits still more counterintuitive features than that of Anderson and Hauck’s test. In fact, as a function of the realized value s˜− of S˜− , the width of its horizontal sections is not only nonmonotonic, but even jagged. Like (10.14), the unbiased critical region contains a large subregion of points in the (d¯− , s˜− )-plane whose distance from the vertical axis is larger than the theoretical equivalence limit to the target parameter 2|δ| (see Figure 10.2), ¯ − is the natural estimator of 2δ. Since the notwithstanding the fact that D gain in power which can be achieved when using such curiously shaped critical regions instead of the triangular region corresponding to (10.10) seems not really relevant from a practical point of view, it is not surprising that neither

Methods of testing for average bioequivalence

321

of the modifications to the interval inclusion test mentioned above made its way in the routine of BE assessment.

Figure 10.2 Critical region of the interval inclusion test (lower triangular part of shaded region), the level-corrected Anderson-Hauck test ( ——) and the unbiased test (shaded region as a whole) for (10.4) with 2δ◦ replaced by unity and N − 2 = 1. [Exact size of all three regions: α = .25.] (Redrawn from Brown, Hwang and Munk, 1997, with kind permission by the International Mathematical Institute.)

As pointed out by Hauschke et al. (1996), the problem of testing for average BE on the basis of the inter-period (log) differences Xi− and Yj− differs fundamentally from an ordinary two-sample t-test setting with regard to the possibility of justifying the assumption of homoskedasticity. In fact, in the vast majority of real applications, it is reasonable to suppose that the withinsubject variability, if different from period to period at all, depends only on treatment as such and not on treatment order, which implies by (10.7) that we have V ar(Xi− ) = V ar(Yj− ) even for grossly different treatment-specific vari2 2 ances σeT and σeR . If one thinks it necessary to allow for heteroskedasticity of both distributions under comparison anyway, then it suggests itself to apply to (10.10) and (10.14) the same modifications which lead from the classical to

322

THE ASSESSMENT OF BIOEQUIVALENCE

a heteroskedasticity-corrected two-sample t-test. Adopting Welch’s (1938) approximate solution to the ordinary Behrens-Fisher problem, this means that 1/2 1 2 SX − + n1 SY2 − and FνˆT (·), we have to replace S˜− and FNT −2 (·) with S˜∗− ≡ m 2 2 , S denote the two sample variances, and νˆ stands respectively, where SX − Y− for the corrected (random) number of degrees of freedom to be determined from 2 2   2 SX (m/n)SY2 − 1 1 1 − · = + . 2 2 2 2 νˆ SX + (m/n)S m − 1 S + (m/n)S n − 1 − Y− X− Y− The resulting modification of the critical region proposed by Anderson and Hauck has been studied by Dannenberg et al. (1994) with respect to size and power. The simulation results obtained by these authors show in particular that the Welch-corrected Anderson-Hauck test exhibits a similar (slight) tendency towards anticonservatism as the original version (10.14) in the homoskedastic case. Of course, the simplest means of reducing or even eliminating this tendency is to apply Welch’s correction to the double t-test. A problem still easier to handle than heteroskedasticity but much more important for real applications, is nonnormality of the distributions of the inter-period differences Xi− , Yj− . Clearly, the classical parametric confidence ¯ − − Y¯ − ) ∓ S˜− tN −2; 1−α which the standard test (10.10) for average limits (X BE is based upon, can be replaced with any other pair of confidence bounds of one-sided level 1 − α each for the shift in location of two continuous distributions of identical shape. As is well known from the nonparametrics literature (see Lehmann, 1975, § 2.6), confidence bounds which are fully distributionfree in the two-sample shift model are given by suitable order statistics for the mn-dimensional vector of all differences Xi− − Yj− between observations from different samples. As was already mentioned in § 6.2, this distributionfree variant of the interval inclusion rule (10.10) has been recommended for assessment of average BE in several contributions to pharmaceutical and clinical pharmacology journals (among others, see Hauschke et al., 1990; Hauck et al., 1997).

10.2.2

Testing for scaled average bioequivalence

For a correct interpretation of the meaning of a positive result of the standard test for average BE as well as any of its variants, it is indispensable to note the following basic fact: According to the alternative hypothesis to be established in the parametric model, it makes no difference whether some sufficiently small value, say .10, of the difference E(Xi− ) − E(Yj− ) between the true means of the normal distributions under consideration, goes with a common (theoretical) standard deviation σ of .001 or 1000. Persumably, not many statisticians will disagree with the conclusion we came to in our general discussion [→ § 1.6] of sensible and less reasonable parametrizations of equivalence testing problems, that this conception of similarity between

Methods of testing for average bioequivalence

323

normal distributions is hardly tenable. If one tries to trace back the literature on statistical methods of assessing what is nowadays called average BE, one finds that except for notational variations, hypotheses formulation (10.4) was taken as definitively given already in the earliest pertinent contributions. So there is strong indication that it was not the idea of statisticians to measure the distance between two homoskedastic Gaussian distributions on the basis of the absolute means only. In the early seventies when biostatisticians first started research on methods for analyzing BE studies (Westlake, 1972; Metzler, 1974), questioning the adequacy of that measure would have amounted to challenging a well-established tradition within the pharmaceutical sciences including clinical pharmacology. Reformulating the problem of testing for average BE as ˜ : 2|φT − φR |/σ < ε ˜ : 2|φT − φR |/σ ≥ ε versus K H

(10.15)

gives several important advantages over the traditional formulation (10.4). First of all, in view of (10.7) and (10.8), the alternative hypothesis now specifies that the ratio of the shift in location of the two Gaussian distributions under comparison over their common standard deviation be sufficiently small. Thus, the underlying measure of distance is scale-free and accommodates to the obvious fact that two normal distributions with common variance σ 2 and given absolute difference .10 of their expected values are indistinguishable for all practical purposes if σ = 1000, whereas for σ = .001, the regions bounded by their density curves are almost perfectly disjoint. Furthermore, the arguments supporting a certain numerical specification of the tolerance ε do not depend on the special context of bioequivalence assessment, and for (10.15) there exists an exact unbiased test satisfying a strong optimality criterion and admitting power calculations in terms of a function which is predefined in advanced statistical packages and can thus be taken as giving explicit expressions. All tools required for carrying out the corresponding alternative test for (scaled) average BE as well as for planning a trial to be analyzed by means of it, have been provided in § 6.1. Thus, except for calculating the actual numerical values of the test statistic and the critical constant, nothing new comes into play if we base the analysis of a standard bioequivalence study like that considered in Example 10.1, on the two-sample t-test for equivalence. Example 10.1 (continued) Suppose the entries in the bottom rows of Table 10.1 are the realized values of N = 12 + 13 = 25 mutually independent random variables distributed as in (10.6), and we want to perform the UMPI test of § 6.1 at level α = .05 in order to prove statistically that the underlying normal distributions are equivalent in the sense of satisfying |μ− − ν− | = 2|φT − φR | < ε with ε = .74 [← Table 1.1, (v)]. Then, by (6.6), we have to compare the absolute value of the two-sample t-statistic, say T − , computed from the Xi− and Yj− , to a critical upper bound obtained as

324

THE ASSESSMENT OF BIOEQUIVALENCE

  1/2 C˜.05;12,13 (.74) = F1,23; .05 (12 · 13/25) · .742  1/2 = finv(.05,1,23,3.417024) √ = .111742 = .334278 , using the SAS function for quantiles of noncentral F -distributions. On the ¯ − = −0.06058, Y¯ − = −0.04515 and S˜− = 0.070306, other hand, we had X so that the observed value of the t-statistic is T − = −0.015430/0.070306 = −0.219469. Thus, the data shown in Table 10.1 lead to a positive decision in the test for scaled average bioequivalence as well [at the 5% level, and with equivalence range (−.74/2, .74/2) for φT − φR ]. Discussion (i) Generally, there is no reason to expect that assessing scaled average BE will lead to the same qualitative decision as any of the procedures being available for testing of equivalence with respect to the absolute means. In particular, it is not unusual that with a given data set, the test for scaled average BE has to accept inequivalence although raw average BE can be established even by means of the simple interval inclusion rule (10.10). At first sight, one might suspect that this is a contradiction to the asserted optimality of the t-test for equivalence. Actually, this would be true only if for some ε > 0, the null hypothesis of (10.15) implied that of (10.4) which is obviously not the case. By the same reason, power comparisons between tests for average BE in the ordinary sense and scaled average BE are pointless in any case. (ii) Researchers in the pharmaceutical sciences often base their reservation about applying the scaled criterion of average BE on the argument that the exact value of the population variance σ 2 of the inter-period (log-) differences is unknown to the experimenter. This would be a serious point only if transforming a given equivalence range for the absolute mean difference was the only way of coming to a consensus about what can be considered an adequate numerical specification of the equivalence limit to the standardized difference of means of two normal distributions with the same variance. Actually, as was explained in Chapter 1, nothing prevents from treating |μ− − ν− |/σ as the primary distance measure admitting direct specification of an equivalence limit without taking into consideration any other function of the three unknown parameters μ− , ν− and σ 2 involved in the problem. In addition, as is the case with any welldesigned study to be eventually analyzed by means of hypotheses testing methods, planning of a BE trial also involves calculation of minimally required sample sizes, and it follows from equation (10.11) that this presupposes knowledge of σ 2 even if the standard test for unscaled average

Individual bioequivalence: Criteria and testing procedures

325

BE shall be used. Long-term experience shows that despite a high variability of the bioavailabilities measured in the individual periods of a BE trial, the standard deviation of the inter-period differences between the bioavailabilities observed in the same subject is typically smaller than the equivalence limit 2δ◦ = .4462 to be specified for E(Xi− ) − E(Yj− ) according to the guidelines of the regulatory authorities. Accordingly, a positive result of a test for unscaled average BE in the majority of cases ensures at best that the shift in location of the two normal distributions under comparison does not substantially exceed their common standard deviation. Thus, it seems not unfair to state that from a statistical point of view, the condition satisfied by two drug formulations proven bioequivalent with respect unscaled averages is often remarkably weak. (iii) Another objection which frequently has been raised against the use of |μ− −ν − |/σ = 2|φT −φR |/σ as the basic distance measure for assessment of average BE is that an experimenter who deliberately “inflates” the variability of the observed bioavailabilities by careless handling of the measurement procedure, will be “rewarded” by enabling him to meet the criterion of BE even for a comparatively large value of |φT − φR |. A good deal of skepticism seems in order about the soundness of this argument as well. First of all, it is not clear how an imagined experimenter who adopts the strategy in mind, can protect himself from increasing the shift in location of the distributions at the same time. Even if he succeeds in this latter respect, one cannot but admit that virtually all inferential procedures are liable to biases induced by manipulation of the conditions under which the primary observations are taken. To put it differently, methods of statistical inference are made for extracting the maximum information from data collected by people who are willing to adhere to the principles of good scientific practice, rather than for providing protection against fraud and doctoring of data.

10.3 10.3.1

Individual bioequivalence: Criteria and testing procedures Introduction

The introduction of the concept of individual bioequivalence by Anderson and Hauck (1990) and Wellek (1990, 1993a) was the beginning of a process which, although overdue already at that time, is still going on and aims at emancipating bioequivalence assessment from taking into account only the first two moments of the distributions of the inter-period (log-) differences Xi− and

326

THE ASSESSMENT OF BIOEQUIVALENCE

Yj− . The idea behind the concept in its original version is simple enough, and its logical basis is easily grasped even by people exclusively interested in the pharmacological and clinical issues connected with bioequivalence testing: Even perfect coincidence of μ− = E(Xi− ) and ν− = E(Yj− ) by no means admits the conclusion that the proportion of individuals whose responses to both drug formulations exhibit the pharmacologically desirable degree of similarity, is sufficiently large. But precisely this has to be guaranteed in order to justify declaring the two formulations of the drug equivalent in the sense of allowing to switch from one to the other in the same individual without altering the response to a relevant extent (cf. Chen, 1997, p. 7). The question of how this notion can be translated into a statistical hypothesis admits a straightforward answer if we restrict the model for the 2 × 2 crossover design we have to consider when analyzing a standard BE trial, by assuming that the period effects also coincide and can thus be dropped from the expressions for the four cell means. Actually, this additional assumption is not as oversimplistic as it might seem at first sight. First of all, the proportion of real BE trials leading to a rejection of the null hypothesis π1 = π2 in a suitable test of significance, is extremely low (confirming a general statement of Chow and Liu, 2008, p. 56). Moreover, there is no problem to positively establish approximate equality of both period effects by means of a preliminary t-test for equivalence of the distribution of the Xi− to that of the −Yj− with respect to their standardized means. Thus, for practical purposes, it entails only a minor loss in generality to assume that through defining Zi = Xi− , i = 1, . . . , m ; Zi+j = −Yj− , j = 1, . . . , n

(10.16)

and ζ = φT − φR ,

(10.17)

Z1 , . . . , ZN becomes a single homogeneous sample of size N = m + n with Zl ∼ N (ζ, σ 2 ) ∀ l = 1, . . . , N , ζ ∈ IR, σ 2 > 0 .

(10.18)

In view of (10.16), (10.5) and the definition of the Xki , Ykj as logarithms of the selected measure of bioavailability, nonexistence of period effects clearly ensures that | exp{Zl } − 1| measures the dissimilarity of the responses to both drug formulations observed in the same subject making up the lth element of the pooled sample. Now, according to basic standards of clinical pharmacology, we can expect that switching over from one formulation of the drug to the other will not change its potential therapeutic effect induced in the lth individual as long as the intraindividual bioavailability ratio exp{Zl } remains within the limits set by the 80–125 % rule. More generally, i.e., without restricting attention to a particular specification of the range considered acceptable for an individual bioavailability ratio, a natural formalization of the basic concept of individual BE is obtained by requiring that   (10.19) P (1 + ε)−1 < exp{Zl } < (1 + ε) > π ∗

Individual bioequivalence: Criteria and testing procedures

327

where ε denotes a positive constant determining the tolerance for intraindividual differences in response levels, and π ∗ stands for the smallest value acceptable for the probability of observing bioequivalent responses in a randomly selected individual. The term “probability-based individual bioequivalence” (abbreviated to PBIBE in the sequel) has been used in the literature since the mid-nineties in order to distinguish (10.19) from an alternative condition proposed by Sheiner (1992) and Schall and Luus (1993) as a criterion of individual BE. This so-called moment-based criterion of individual BE is not discussed here because it presupposes that replicate crossover designs be used (for more details see FDA, 2001). In the parametric model given by (10.18), the probability of observing bioequivalent responses at the individual level can obviously be written P [(1 + ε)−1 < exp{Zl } < (1 + ε)] = πε (ζ, σ) ,

(10.20)

provided we define 

log(1 + ε) − ζ πε (ζ, σ) = Φ σ





− log(1 + ε) − ζ −Φ σ

 .

(10.21)

In § 10.3.2, we will treat P [(1 + ε)−1 < exp{Zl } < (1 + ε)] as a functional allowed to depend on the cdf of the Zl in an arbitrary way. The corresponding distribution-free test for PBIBE is based on a simple counting statistic and can thus be expected to leave a large margin for improvements in power when we assume that the expression on the left-hand side of (10.19) admits the representation (10.21). Although the problem of testing for a sufficiently large value of the parametric function πε (ζ, σ) looks fairly simple, a satisfactory solution is difficult to find by means of classical methods. In § 10.3.3, a Bayesian construction is proposed instead which can be shown by numerical methods to maintain the ordinary frequentist significance level without being overly conservative and to yield gains in power of up to 30 % as compared to the distribution-free procedure.

10.3.2

Distribution-free approach to testing for probabilitybased individual bioequivalence

If establishing the validity of (10.19) is accepted as a reasonable objective of bioequivalence assessment, we need a procedure for testing Hε(1) : pε ≤ π ∗

Kε(1) : pε > π ∗ , (10.22)  where pε symbolizes the probability of the event (1 + ε)−1 < exp{Zl } <   (1 + ε) = |Zl | < log(1 + ε) under an arbitrary cdf FZ (·), say, of the observed individual log-bioavailability ratios. Provided FZ (·) is continuous, versus

328

THE ASSESSMENT OF BIOEQUIVALENCE

the (one-sided) testing problem (10.22) admits an easy solution exhibiting attractive theoretical properties. To see this, let us introduce the statistic

  N+ ≡ # l ∈ {1, . . . , N }  |Zl | < log(1 + ε) (10.23) which simply counts the number of subjects in the sample who exhibit a relative bioavailability lying within the tolerated range. Of course, the distribution of N+ is B(N, pε ) [binomial with parameters N , pε ]. Hence, it is natural to decide between the hypotheses of (10.22) by carrying out in terms of N+ the UMP level-α test of the null hypothesis p ≤ π ∗ about the unknown parameter of a binomial distribution generated by repeating the same Bernoulli experiment independently N times. Clearly, this leads to the decision rule ⎧ (1) ∗ ⎪ Rejection of Hε N+ > kN (α) ⎪ ⎪ ⎪ ⎨ (1) Rejection of Hε with ∗ (10.24) if N+ = kN (α) , ∗ ⎪ ⎪ probability γN (α) ⎪ ⎪ ⎩ (1) ∗ N+ < kN (α) Acceptance of Hε where the critical constants have to be determined from ∗ kN (α)

 = min

 N    ∗ k ∈ IN0  b(j; N, π ) ≤ α ,

(10.25)

j=k+1

∗ γN (α)

 =

α−

N 

  ∗  b kN b(j; N, π ) (α); N, π ∗ , ∗

(10.26)

∗ (α)+1 j=kN

with b( · ; N, π ∗ ) as the probability mass function of B(N, π ∗ ) . Obviously, (10.24) is simply a variant of the one-sided sign test which implies in particular (1) (1) that it is completely distribution-free. Since the hypotheses Hε and Kε refer to the class of all continuous distributions of the individual observations Zl , mimicking Lehmann and Romano’s (2005, pp. 291) proof of the UMP property of the ordinary sign test, (10.24) can be shown to be uniformly most powerful among all level-α tests of the nonparametric version of the null hypothesis of individual bio-inequivalence in the probability-based sense. Under the acronym TIER (for Testing Individual Equivalence Ratios) coined by Anderson and Hauck (1990), the conservative, nonrandomized version of the procedure (10.24) gained some popularity for a time. Clearly, its most conspicuous practical advantage is extreme simplicity. In fact, neither for determining the critical constants nor the exact power against some specific alternative pε ∈ (π ∗ , 1), other computational devices than an extensive table or a program for the binomial distribution function are required. Therefore, presentation of an extra example illustrating the practical use of the method is dispensable here.

Individual bioequivalence: Criteria and testing procedures

329

With regard to its conceptual basis, the simple criterion (10.19) of PBIBE has rightly been criticized by several authors for potentially using a lower bound to the probability of observing equivalent bioavailabilities in a randomly selected subject which may be unattainable even in a study comparing the reference formulation to itself. A reasonable way around this potential difficulty recommended in the literature (see Schall, 1995, (2.3)) consists of replacing π ∗ by a bound scaled by relating it to a reliable estimate of the probability of finding the condition |Zl | < log(1 + ε) satisfied in a pseudotrial of two identical drug products. In the parametric submodel (10.18), this probability is clearly given by πε (0, σ) = 2 Φ(log(1 + ε)/σ) − 1 so that it seems natural to scale the bound π ∗ of (10.19) by replacing it with the number   ∗ πsc ≡ π ∗ · 2 Φ(log(1 + ε)/σ ∗ ) − 1 , (10.27) where σ ∗ denotes some realistic value of the population standard deviation σ of log-bioavailability ratios measured for the drug product under investigation. In contrast to the basic measurements taken in the individual period of a BE trial, σ 2 does not depend on the between-subject variance component σS2 [recall (9.29)] but reflects the technical and analytical precision of the measurement process as such. Thus, σ 2 is a substance-specific rather than biological constant, and a comparatively stable estimate might be obtained by pooling the point estimates over a variety of trials of the same or chemically related agents. Experience shows that in a BE trial performed in strict adherence to the established technical standards, the observed standard deviation of the Zl often remains below .20 . Using this as σ ∗ and .75 as the unscaled bound to P [|Zl | < log(1 + ε)] yields for ε = .25 :   ∗ πsc = .75 · 2 Φ(.2231/.2000) − 1 = .75 · .7353 ≈ .55 . A more sophisticated approach to scaling the criterion of PBIBE is available in the Bayesian framework developed in the next subsection.

10.3.3

An improved parametric test for probability-based individual bioequivalence

In this subsection, we consider the parametric version of the problem of testing for PBIBE which arises from replacing in (10.22) the functional pε with πε (ζ, σ) as defined in (10.21). In other words, we now put forward the problem of testing ˜ ε(1) : πε (ζ, σ) ≤ π ∗ H

versus

˜ ε(1) : πε (ζ, σ) > π ∗ K

(10.28)

by means of a sample Z1 , . . . , ZN of independent observations from N (ζ, σ 2 ) . Our construction of a Bayesian test for (10.28) will be based on the usual (see Box and Tiao, 1973, p. 92) reference prior for the expectation ζ and the standard deviation σ of the underlying Gaussian distribution. This means

330

THE ASSESSMENT OF BIOEQUIVALENCE

that a priori we treat the true value of the population parameter (ζ, σ) as the realization of a random variable (ζ, σ), say, inducing an improper probability distribution defined by the following density with respect to Lebesgue measure on IR2 : ⎧ −1 −∞ < ζ < ∞, σ > 0 ⎨σ for . (10.29) (ζ, σ) = ⎩ 0 −∞ < ζ < ∞, σ ≤ 0 The corresponding posterior density of (ζ, σ) given the realized values (¯ z , s) 1/2    N N ¯ 2 of the sufficient statistics Z¯ = i=1 Zi /N , S = (N −1)−1 i=1 (Zi − Z) , is well known (cf Box and Tiao, 1973, pp. 93-6) to be given as (¯z,s) (ζ, σ) =

√ √ √  √  χ N /σ ϕ N (ζ − z¯)/σ N − 1 (s/σ 2 ) fN N − 1 s/σ (10.30) −1 χ where ϕ(·) denotes the standard normal density and fN −1 (·) the density of a χ-distribution with N − 1 degrees of freedom [ → (10.13) with N augmented by 1]. Now, an algorithm for computing the posterior probability of the (alternative) hypothesis of (10.28) is obtained by exploiting the following basic facts: (1)

˜ε (i) For arbitrarily fixed σ > 0, the set of values of ζ satisfying K the representation

{ζ ∈ IR | πε (ζ, σ) > π ∗ } = − σ q(˜ ε/σ, π ∗ ) , σ q(˜ ε/σ, π ∗ ) ,

admits

(10.31)

where ε˜ = log(1 + ε) and q(˜ ε/σ, π ∗ ) has to be determined by solving the equation (10.32) Φ(˜ ε/σ − ψ) − Φ(−˜ ε/σ − ψ) = π ∗ , ψ > 0 . (ii) The solution of (10.32) exists and is uniquely determined for all σ falling below an upper bound σε∗ , say, defined by σε∗ = ε˜/Φ−1 ((1 + π ∗ )/2) .

(10.33)

[In view of Theorem A.1.5 (iii), this follows immediately from the fact that the left-hand side of (10.32) coincides with the power function of a test with critical region {−˜ ε/σ < U < ε˜/σ}, where U ∼ N (ψ, 1) .] (iii) Combining (i) and (ii) with (10.30) shows that the posterior probability of the alternative hypothesis {πε (ζ, σ) > π ∗ } of (10.28) can be written as

Individual bioequivalence: Criteria and testing procedures

 σε∗  √    ∗ Φ N q(˜ P(¯z,s) πε (ζ, σ) > π = ε/σ, π ∗ )− 0 √     z¯/σ − Φ N − q(˜ ε/σ, π ∗ ) − z¯/σ ·  √ √  χ N − 1 (s/σ 2 )fN N − 1 s/σ dσ . −1

331

(10.34)

The integral on the right-hand side of (10.34) can be evaluated with very high numerical accuracy by means of Gauss-Legendre quadrature. Since computation of individual values of the function σ → q(˜ ε/σ, π ∗ ) on the basis of (10.32) is very fast and the cumulative standard normal distribution function Φ(·) is predefined in any programming environment to be taken into consideration for the present purpose, the degree of the polynomial determining the abscissas can once more be chosen as large as 96. Implementations in SAS and R of the algorithm described above for the ˜ ε(1) given an arbitrary realization computation of the posterior probability of K ¯ S), can be found in the WKTSHEQ2 Source (¯ z , s) of the sufficient statistic (Z, Code Package under the program name po_pbibe. Using this computational tool, it is no problem to perform the Bayesian test with respect to the reference prior (10.29). For definiteness, its rejection region reads    

 (10.35) (¯ z , s) ∈ IR × IR+  P(¯z,s) πε (ζ, σ) > π ∗ > 1 − α . For the specific case α = .05, ε = .25, π ∗ = .75, and sample size N = 20, ¯ S) is displayed as the filled the corresponding set in the sample space of (Z, portion of the graph shown in Figure 10.3. Roughly speaking, it looks like a vertically compressed image of the region bounded by the upper solid curve which is the geometric representation of the alternative hypothesis under consideration. Since (¯ z , s) is the observed value of the natural estimator for (ζ, σ), it is clear that no test which is valid with respect to the significance level, can have a rejection region being larger than the subset of the parameter space specified by the alternative hypothesis one wishes to establish. Hence, the form of the Bayesian rejection region seems very reasonable.

332

THE ASSESSMENT OF BIOEQUIVALENCE

Figure 10.3 Rejection region of the Bayesian test at (nominal) level α = .05 for PBIBE, for ε = .25, π ∗ = .75 and N = 20. [Upper solid line: Boundary of the parameter subspace specified by the alternative hypothesis.] (From Wellek, 2000a, with kind permission by Wiley-VCH.) Numerical study of size and power of the Bayesian critical region The following result (rigorously proven in Wellek, 2000a) forms the basis of an algorithm for computing exact rejection probabilities of the Bayesian test with rejection region (10.35). (a) For any fixed point s◦ in the sample space of the statistic S, there exists ∗ (s◦ ) in the sequel, such that a nonnegative real number denoted z¯αε 

    z¯ ∈ IR P(¯z,s◦ ) πε (ζ, σ) > π ∗ > 1 − α =

∗ (s◦ ) , z¯αε ∗ (s◦ ) . − z¯α,ε (10.36) Thus, all horizontal sections of the rejection region (10.35) are symmetric intervals on the real line.

Individual bioequivalence: Criteria and testing procedures

333

∗ (s◦ ) is either zero [so that the corresponding section of (b) The number z¯αε (10.35) is empty], or it is uniquely determined by the equation   P(¯z,s◦ ) πε (ζ, σ) > π ∗ = 1 − α , z¯ ∈ IR+ . (10.37) ∗ (s◦ ), not only (c) There exists a positive real number s∗α,ε [depending, like z¯αε on α and ε, but also on N and π ∗ ] such that we have  ∗ (s) > 0 = 0, s∗α,ε  . s ∈ IR+ |¯ (10.38) zα,ε ∗ Let now βα,ε (ζ, σ) denote the rejection probability of the Bayesian test under any parameter constellation (ζ, σ) ∈ IR×IR+ . In view of the independence ¯ S), applying (a) – (c) yields of the two components of the sufficient statistic (Z, ∗ (ζ, σ): the following integral representation of βα,ε

 s∗α,ε  √

∗ βα,ε (ζ, σ) = zα∗ (s) − ζ)/σ − Φ N (¯ 0

 √ √ √ χ ∗ zα (s) − ζ)/σ ( N − 1 /σ) fN Φ N (−¯ −1 (s N − 1/σ) ds

(10.39)

χ where fN −1 (·) again stands for the density function of the square root of a variable following a central χ2 -distribution with N − 1 degrees of freedom. The formal structure of the integrand appearing on the right-hand side of this equation is similar to that of the function to be integrated in order to compute the posterior probability of the alternative hypothesis of (10.28) by means of (10.34). Hence, in evaluating (10.39), 96 point Gauss-Legendre integration ˜ ε(1) given works as well as it does in computing the posterior probability of K ¯ an arbitrary realization of (Z, S). Table 10.2 shows the results of applying formula (10.39) to various parameter constellations (ζ, σ) lying on the common boundary of the hypotheses (10.28) for ε = .25, π ∗ = .75, the nominal significance level α = .05, and three different sample sizes whose order of magnitude ranges from “small” (N = 20) through “moderate” (N = 50) to “large” (N = 100). For true standard deviations σ being smaller than or equal to .10, the differences between ∗ (ζ, σ) and the nominal α are clearly negligible, and even for the largest βα,ε values of σ to be taken into consideration for purposes of assessing the level properties of the Bayesian test, the differences nowhere go in the wrong, e.g., anticonservative direction.

334

THE ASSESSMENT OF BIOEQUIVALENCE Table 10.2 Exact rejection probabilities of the Bayesian test for PBIBE on the common boundary of the hypotheses [ α = .05, π ∗ = .75, ε = .25 (↔ ε˜ = .2231)]. (From Wellek, 2000a, with kind permission by Wiley-VCH.) ζ

σ

N

∗ βα,ε (ζ, σ)

.2164 .2164 .2164 .1894 .1894 .1894 .1557 .1557 .1557 .1163 .1163 .1163

.01 .01 .01 .05 .05 .05 .10 .10 .10 .15 .15 .15

20 50 100 20 50 100 20 50 100 20 50 100

.050 000 .050 009 .049 758 .050 000 .050 000 .050 000 .048 438 .049 401 .049 659 .035 674 .040 441 .043 053

.0709 .0709 .0709

.18 .18 .18

20 50 100

.028 230 .034 270 .038 209

Of course, (10.39) allows also numerically exact computation of the power ˜ ε(1) . Since of the Bayesian test against arbitrary specific alternatives (ζ, σ) ∈ K computing the rejection probabilities of the distribution-free procedure of testing for PBIBE under arbitrary values of pε ∈ (0, 1 is an elementary matter, we are able to systematically assess the gain in efficiency achievable from using the Bayesian test based on (10.35) instead of the so-called TIER procedure (10.24). Although the randomized version of the latter is of theoretical interest only, Table 10.3 compares the rejection probabilities of both the nonrandomized and the optimal version with that of the Bayesian test for all specific alternatives selected. Otherwise, we would not be in a position to discriminate true gains in power achieved by using the parametric test, from “spurious” improvements vanishing if the conventional

5% level is replaced  with a level “nat  ural” for the counting statistic Nε = # l ∈ {1, . . . N }  |Zl | < log(1 + ε) .

Individual bioequivalence: Criteria and testing procedures

335

Table 10.3 Exact rejection probabilities of the Bayesian test and both versions of TIER against various specific alternatives, for the case α = .05, π ∗ = .75, and ε = .25. (From Wellek, 2000a, with kind permission by Wiley-VCH.) ζ

σ

.16423 .07 .16423 .07 .16423 .07 .12127 .12 .12127 .12 .12127 .12 .03745 .17 .03745 .17 .03745 .17 .17132 .05 .17132 .05 .17132 .05 .11937 .10 .11937 .10 .11937 .10 .03891 .15 .03891 .15 .03891 .15 .15907 .05 .15907 .05 .15907 .05 .09456 .10 .09456 .10 .09456 .10 .01338 .135 .01338 .135 .01338 .135

πε (ζ, σ)

N

Bayes

TIER nonrandom

TIER random

.80 .80 .80 .80 .80 .80 .80 .80 .80 .85 .85 .85 .85 .85 .85 .85 .85 .85 .90 .90 .90 .90 .90 .90 .90 .90 .90

20 50 100 20 50 100 20 50 100 20 50 100 20 50 100 20 50 100 20 50 100 20 50 100 20 50 100

.14423 .25380 .41369 .12532 .23833 .40096 .08508 .19513 .36831 .34955 .68261 .92453 .33178 .67423 .92231 .23753 .61062 .91057 .67436 .97097 .99977 .63842 .96741 .99974 .52538 .95347 .99970

.06918 .19041 .27119 .06918 .19041 .27119 .06918 .19041 .27119 .17556 .51875 .76328 .17556 .51875 .76328 .17556 .51875 .76328 .39175 .87785 .98999 .39175 .87785 .98999 .39175 .87785 .98999

.12171 .20238 .31550 .12171 .20238 .31550 .12171 .20238 .31550 .26355 .53404 .79930 .26355 .53404 .79930 .26355 .53404 .79930 .50117 .88444 .99264 .50117 .88444 .99264 .50117 .88444 .99264

Overall the results displayed in Table 10.3 admit the following conclusions: • The Bayesian test dominates the nonrandomized TIER uniformly on the ˜ ε(1) . In many cases, the difference between selected grid of points in K the two power functions has an order of magnitude making further comments on the practical relevance of the improvement dispensable. • Even as compared to the randomized version of TIER, the Bayesian test comes out as clearly superior because for the few constellations with a

336

THE ASSESSMENT OF BIOEQUIVALENCE negative sign of the difference, the distance between the power functions is practically negligible.

Generalizations 1. Scaling of the criterion of PBIBE. The Bayesian framework admits the possibility of letting the factor to be used for scaling the bound π ∗ to the probability of obtaining a bioequivalent individual response, depend on the actual value of the standard deviation σ of the log-bioavailability ratios Zl . As explained on p. 329, the natural candidate for a function of σ whose values give suitable weight factors to π ∗ , is σ → πε (0, σ), so that we propose to modify the condition πε (ζ, σ) > π ∗ for PBIBE by requiring that πε (ζ, σ) > π ∗ · πε (0, σ). This leads to replacing (10.28) with ˜ (1) : πε (ζ, σ) ≤ π ∗ H ε πε (0, σ)

versus

˜ (1) : πε (ζ, σ) > π ∗ . K ε πε (0, σ)

(10.40)

As shown in the graphical representation of Figure 10.4, incorporation of that kind of scaling induces a radical change to the shape of the equivalence region specified by the alternative hypothesis to be established by means of ˜ ε(1) of (10.28) [→ outer contour plotted in Fig. 10.3], the data. In contrast to K ˜ (1) corresponds to a region in the (ζ, σ)-plane being unbounded in upward K ε direction such that the width of the horizontal sections increases to infinity as σ → ∞ . This implies in particular, that for whatever large a shift in the mean away from zero, scaled PBIBE in the sense of (10.40) is satisfied for any sufficiently extreme degree of variability. If the latter property is considered ˜ (1) whose undesirable, it might make sense to combine the Bayesian test for K ε rejection region is shown in Figure 10.4 as well [again for α = .05, ε = .25, N = 20, and a lower bound to the target parametric function of .75], with a test for reasonably low variability, as described in the following subsection in connection with a disaggregate test for population bioequivalence. 2. Accommodating nonnegligible period effects. If the period effects cannot be neglected (which is, as mentioned earlier, quite exceptional in bioequivalence trials), the difference (10.17) between the direct formulation effects can be written ζ = ζ1 − ζ2 where ζ1 and ζ2 denote the expectations of two normal distributions with common (unknown) variance σ∗2 from which independent [1] [1] [2] [2] samples Z1 , . . . , Zm and Z1 , . . . , Zn have been taken. Clearly, we have [1] [2] just to put Zi = Xi− /2, Zj = Yj− /2, and observe that the standard deviation σ we are interested in according to (10.21) is now given by σ = 2σ∗ .

Individual bioequivalence: Criteria and testing procedures

337

Figure 10.4 Rejection region of the Bayesian test at (nominal) level α = .05 for scaled PBIBE in the sense of (10.40), for ε = .25, π ∗ = .75 and N = 20. [Outer solid line: Boundary of the parameter subspace specified by the alternative hypothesis. (From Wellek, 2000a, with kind permission by Wiley-VCH.)]

Now, the posterior distribution of (ζ, σ∗ ) given the observed values of

1/2   [1] [2] [1] Z − Z¯ [2] and S∗ = (m + n − 2)−1 SQZ + SQZ with SQZ and  [2] [1] [2] SQZ as the corrected sum of squares for the Zi and the Zj , respectively has the same form as in the paired t-test setting considered in the body of this paper. In fact, as shown in § 2.5.1 of Box and Tiao (1973) the posterior density of (ζ, σ∗ ) with respect to the reference prior with (improper) density (ζ1 , ζ2 , σ∗ ) → σ∗−1 on IR2 × IR+ , is given by:

  ˆ ∗ · mn/N (1/σ∗ ) ϕ mn/N (ζ − ζ)/σ (ζ,s ˆ ∗ ) (ζ, σ∗ ) = √

√ χ (10.41) N − 2 (s∗ /σ∗2 ) fN N − 2 s /σ ∗ ∗ −2 ¯ [1]

χ where fN −2 (·) has to be defined as in (10.13). Hence, except for minor mod-

338

THE ASSESSMENT OF BIOEQUIVALENCE

ifications, the algorithms described above can also be used for computing posterior probabilities of the alternative hypotheses of (10.28) and (10.40), as well as exact rejection probabilities of the associated Bayesian tests for (scaled) PBIBE with data from bioequivalence trials involving nonnegligible period effects. Example 10.2 For illustration, we use the data set analyzed by means of a nonparametric procedure for the assessment of scaled average BE in Wellek (1996). It has been obtained from a standard comparative bioavailability study of a generic (T ) and the original manufacturer’s formulation (R) of the calcium blocking agent nifedipine. As is so often the case in practice of bioequivalence assessment, the total sample size has been rather small (N = 20). The complete listing of the raw data is given in Table 10.4. Although the primary analysis performed in the reference cited above used an approach allowing for unequal period effects π1 = π2 , the natural estimator ¯ − + Y¯ − ) of π1 − π2 (see, e.g., Jones and Kenward, 2003, § 2.3) turns (1/2) (X out to be rather small [-.0028] as compared to its standard error [.0357] . Hence there seems little harm in assuming that switching from the first period to the second per se does not change the average bioavailability of the active agent under study. This leads to treating the values of log(AU C1 /AU C2 ) from the subjects in sequence group T /R and those of log(AU C2 /AU C1 ) from the subjects randomized to R/T , as a single sample of size N = 20 from the same Gaussian distribution N (ζ, σ 2 ). The values of the sufficient statistics are readily computed to be Z¯ = −.05168 and S = .15559, respectively. Setting the nominal significance level α equal to the conventional value 5%, the tolerance ε determining the equivalence range for an intra-subject bioavailability ratio to .25, and the lower bound for ordinary and scaled PBIBE to .75, we are able to apply both versions of the Bayesian decision rule simply by means of the graphical devices provided by Figure 10.3 and 10.4. On the one hand, the observed point (−.05168 , .15559) fails to be included in the rejection region displayed in Figure 10.3 whose vertex is located below s = .14 . Accordingly, in the Bayesian test for nonscaled ˜ ε(1) of (10.28), the data PBIBE as specified by the alternative hypothesis K

Individual bioequivalence: Criteria and testing procedures

339

Table 10.4 Results of a bioequivalence trial of two formulations of the calcium-blocking agent nifedipine. (From Wellek, 1996, with kind permission by Wiley-VCH.) Subject ID (l) AU C1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

106.9 131.3 81.4 154.7 111.2 85.8 295.2 217.0 252.3 157.9 217.3 174.4 155.8 299.5 157.6 121.4 143.9 157.0 114.5 71.0

AU C2

SEQ

Zl

112.9 124.4 89.5 134.9 108.3 94.0 418.6 207.0 239.3 207.3 195.2 122.7 188.2 309.2 153.5 104.7 119.3 146.8 138.2 70.3

T /R T /R T /R T /R T /R T /R T /R T /R T /R T /R R/T R/T R/T R/T R/T R/T R/T R/T R/T R/T

−0.05461 0.05398 −0.09486 0.13695 0.02643 −0.09128 −0.34926 0.04718 0.05290 −0.27221 −0.10725 −0.35161 0.18893 0.03187 −0.02636 −0.14799 −0.18748 −0.06717 0.18813 −0.00991

shown in Table 10.4 do not allow a positive decision. On the other hand, (−.05168 , .15559) is obviously an inner point of the left half of the grey filled region shown in Figure 10.4 so that the Bayesian test for scaled PBIBE decides ˜ ε(1) [→ (10.40)] with the same data. Interestingly, in favor of the alternative K such a qualitative discrepancy between the final decisions to be taken also occurs if, in assessing the above data set, the conventional test (10.10) for average bioequivalence is contrasted with the t-test for scaled average bioequivalence (Wellek, 1991): At the 5% level, average bioequivalence in the sense of the well-known 80–125% criterion can be established, but scaled average bioequivalence cannot, even if a shift by up to a whole common standard deviation is defined compatible with equivalence of two homoskedastic Gaussian distributions.

340

10.4 10.4.1

THE ASSESSMENT OF BIOEQUIVALENCE

Approaches to defining and establishing population bioequivalence Introduction

With regard to coherence of terminology, the concept of population bioequivalence which has attained considerable popularity in the pertinent literature during the last 10 years, is hardly a fortunate choice. As a matter of fact, every inferential method of BE assessment aims at establishing a certain statement about the populations from which the subjects participating in the actual trial have been recruited by means of random selection. So the term population bioequivalence, which was originally coined by Anderson and Hauck (1990) and technically elaborated by Schall and Luus (1993), reveals almost nothing about the specific content of the concept it denotes. Nevertheless, after the preparations made in § 10.1, its technical content can readily be made precise. The idea is to require of bioequivalent drug formulations that the marginal distributions associated with the treatments in the two sequence groups be sufficiently similar both with respect to the treatment-related component of the mean, and total variability. Under the standard model for the 2×2 crossover design, the variances of the Xki and the Ykj depend only on treatment and are given by 2 ≡ σT2 T , V ar(X1i ) = V ar(Y2j ) = σS2 + σeT

(10.42a)

2 V ar(X2i ) = V ar(Y1j ) = σS2 + σeR ≡ σT2 R .

(10.42b)

In the existing literature (see, e.g., Schall and Luus, 1993; Holder and Hsuan, 1993), as well as in the guidelines of the regulatory agencies (FDA, 2001), socalled aggregate criteria of population BE have been preferred up to now. Instead of aiming to establish sufficient similarity both of φT to φR and of σT2 T to σT2 R , such an aggregate approach relies on a single real-valued function of all four parameters to be compared to some prespecified upper bound. As a so-called reference-scaled version of such an aggregate criterion, the most recent FDA guidance for BE trials recommends θP =

(φT − φR )2 + (σT2 T − σT2 R ) . σT2 R

(10.43)

There are several reasons to question whether it really makes sense to aggregate the moments under comparison exactly in the way leading to (10.43). (i) Except for the scale-factor 1/σT2 R , the parametric function to be assessed according to (10.43) combines a Euclidean distance (viz., that between

Defining and establishing population bioequivalence

341

the formulation effects) with an ordinary signed difference. The commonly referenced argument (see Schall and Luus, loc. cit.) for the use of this combination is based on the equation E[(YT − YR )2 − (YR − YR )2 ] = (φT − φR )2 + σT2 T − σT2 R . However, the latter presupposes that all three observed (log)-bioavailabilities YT , YR and YR are taken from different subjects, whereas in a real bioequivalence trial the drug formulations are administered consecutively to each individual. (ii) In principle, scaling the primary distance measure has much to recommend it, provided there exists an intrinsic relationship between the parametric function to be scaled and the parameter with respect to which standardization is done. However, the incommensurability of the two terms making up the numerator of (10.43) raises a question as to whether a single parameter can give an appropriate reference for both and hence for their sum. In fact, under the basic parametric model for two-period crossover BE studies [ → (9.28), (9.29), (10.1), (10.2)], the optimal estimator for φT − φR has a sampling variance which is proportional to the 2 2 sum of the two intra-subject variances σeT and σeR . Hence, a suitably scaled measure for the distance between the formulations with respect to average log-bioavailability is given by η2 =

(φT − φR )2 σ2

(10.44)

where 2 2 + σeR . σ 2 = σeT

(10.45)

1/σT2 R

as a scaling factor for the On the other hand, the adequacy of second term in the numerator of the parametric function introduced in (10.43) is obvious. Hence, it is natural to require of populationbioequivalent drug formulations that both η 2 and Λ = σT2 T /σT2 R − 1

(10.46)

should be sufficiently small. In order to circumvent the problems pointed out above, Wellek (2000b) proposed a disaggregate scaled criterion of population BE which requires that the two inequalities 2 2 ∗ (φT − φR )2 /(σW T + σW R ) < εφ

and

σT2 T /σT2 R < 1 + ε∗σ

(10.47a) (10.47b) ε∗φ , ε∗σ

must be satisfied simultaneously, for suitably chosen constants > 0. A formal inferential procedure for assessing population BE in this latter sense consists of performing a valid test of H∪ : Hφ ∨ Hσ

versus

K∩ : Kφ ∧ K σ

(10.48)

342

THE ASSESSMENT OF BIOEQUIVALENCE

where the elementary testing problems involved read in explicit formulation 2 2 ∗ Hφ : (φT − φR )2 /(σW T + σW R ) ≥ εφ

vs.

2 2 ∗ Kφ : (φT − φR )2 /(σW T + σW R ) < εφ

(10.48a)

and Hσ : σT2 T /σT2 R ≥ 1 + ε∗σ vs. Kσ : σT2 T /σT2 R < 1 + ε∗σ .

(10.48b)

Exploiting the intersection-union principle formulated and proven in § 7.1, testing of (10.48) reduces to performing a valid test for each of the elementary problems. But in view of (10.45), except for renaming the equivalence bound to |φT −φR |/σ, (10.48a) is the same as the problem of testing for scaled average BE in the sense of § 10.2.2. Hence, all that is really left to do for constructing a test for establishing the proposed disaggregate criterion of population BE, is to derive a solution for the one-sided testing problem (10.48b). This will be the topic of § 10.4.2. In § 10.4.3, the rejection region of the complete disaggregate testing procedure will be presented, and its use illustrated by reanalyzing the BE trial considered in Example 10.1 of § 10.2.1. The section will be concluded by presenting simulation results on the power of the disaggregate test, and the sample sizes required for attaining given power against selected specific alternatives, respectively.

10.4.2

A testing procedure for establishing one-sided bioequivalence with respect to total variability

The basic idea behind the construction of an exact level-α test for (10.48b) to be described in this subsection, goes back as far as the 1930s (Pitman, 1939; Morgan, 1939) and was exploited in a slightly modified version by Liu and Chow (1992) as well as Guilbaud (1993) for deriving tests for one-sided equivalence with respect to intra-subject rather than total variability of the pharmacokinetic measure of interest. As before [recall p. 313], let us denote the log-bioavailabilities observed during the kth period in a subject randomized to sequence group T /R and R/T by Xki and Ykj , respectively. Furthermore, for any fixed positive number ω let ω = X1i + ωX2i U1i

,

ω U2i = (1/ω)X1i − X2i

,

i = 1, ..., m ,

(10.49a)

ω V2j = (1/ω)Y2j − Y1j

,

j = 1, ..., n .

(10.49b)

and ω = Y2j + ωY1j V1j

,

ω ω , U2i ) Then, it follows from (9.28), (9.29), (10.1) and (10.2) that the pairs (U1i ω ω , V2j ) form samples from two bivariate normal distributions with posand (V1j sibly different mean vectors but the same covariance matrix. Hence, in particular the correlation within any pair of U ω ’s is the same as within any pair

Defining and establishing population bioequivalence

343

of V ω ’s, and the common correlation coefficient ρω , say, is easily computed to be σ 2 − ω 2 σT2 R ρω =  2 T T . (10.50) (σT T + ω 2 σT2 R )2 − 4ω 2 σS4 ω ω In view of the possible heterogeneity of the distributions of the (U1i , U2i ) ω ω and the (V1j , V2j ) with respect to the means, ρω cannot be estimated by the empirical correlation coefficient for the pooled sample consisting of all ω ω ω ω , U2i ) and (V1j , V2j ) together. Instead, pooling has to be done for the (U1i ω ω sample moments which have to be computed first separately for the (U1i , U2i ) ω ω and the (V1j , V2j ) which leads to the estimator

 Rω =



 SSX (1) + SSY (2) − ω 2 SSX (2) + SSY (1)     2 SSX (1) + SSY (2) + ω 2 SSX (2) + SSY (1)

2 !1/2 −4ω SP X + SP Y . 2

(10.51)

In this formula, SSX (k) and SSY (k) stands for the (corrected) sum of squares SP X and SP Y have of the Xki and Y kj (k = 1, 2), respectively. Similarly, ¯ 1 )(X2i − X ¯ 2 ) and n (Y1j − Y¯1 )(Y2j − Y¯2 ) (X − X to be defined as m 1i i=1 j=1 ¯ k = m Xki /m and Y¯k = n Ykj /n (k = 1, 2), respectively. where X i=1 j=1 √ The appropriateness of Rω with ω = 1 + ε∗σ as a test statistic for (10.48b) follows from the following properties: (i) If ω equals the true value of the ratio σT T /σT R of total standard deviations, then Rω has the same distribution as the empirical correlation coefficient for a single sample of size m + n − 1 from a bivariate normal distribution with diagonal covariance matrix. Equivalently, this means that for ω = σT T /σT R , the distribution of  √ Tω = m + n − 3 Rω / 1 − Rω2 (10.52) is central t with m + n − 3 degrees freedom. (ii) For fixed sums of squares SSX (k) , SSY (k) (k = 1, 2) and sums of products SP X, SP Y , and arbitrary ω1 , ω2 > 0, there holds the relation ω1 ≤ ω2 ⇒ Tω1 ≥ Tω2 .

(10.53)

Property (i) is plausible from the obvious fact that ρω vanishes for ω = σT T /σT R and that Rω is by definition the empirical correlation coefficient pooled over two independent samples from bivariate normal distributions with the same theoretical correlation ρω (a rigorous proof can be given by means of Theorems 7.3.2 and 7.6.1 of Anderson, 1984). The other property (ii) can

344

THE ASSESSMENT OF BIOEQUIVALENCE

be established by examining the derivative with respect to ω of the expression on the right-hand side of (10.51). Combining both properties shows that the decision rule Reject Hσ if and only if T√1+ε∗ < tm+n−3; α σ

(10.54)

defines an unbiased test of exact level α for (10.48b). Of course, this statement holds under the customary notational convention for central t-quantiles according to which tν; γ symbolizes the lower 100γth percentile of a central t-distribution with ν degrees of freedom (for any 0 < γ < 1).

10.4.3

Complete disaggregate testing procedure and illustrating example

Bearing clear that putting  equation (10.44) in mind, it becomes immediately ˜ of (10.15) coincide with the ε = 2 ε∗φ makes the alternative hypothesis K first condition required by our aggregate criterion of population BE. Thus, the testing sub-problem (10.48a) admits an optimal solution which is obtained by applying  the critical region (6.6) of the two-sample t-test for equivalence with ε = 2 ε∗φ and the within subject differences Xi− and Yj− as the primary observations. This gives the rejection region  

¯ − − Y¯ − |/S˜− < C˜α;m,n 2 ε∗ (10.55) |X φ ¯ − , Y¯ − and S˜− have to be computed in the way explained on p. 316. where X Now, according to the intersection-union principle, we have to combine (10.55) and (10.54) into the aggregated decision rule: Reject lack of population BE if and only if there holds both  ¯ − − Y¯ − |/S˜− < C˜α;m,n 2 ε∗ and |X φ *  √ 2 m + n − 3 R√1+ε∗ 1 − R√ < tm+n−3, α . ∗ σ

1+εσ

(10.56)

Example 10.1 (continued) We illustrate the use of the aggregate decision rule (10.56) by applying it to the same data set that has been analyzed in § 10.2 by means of the interval inclusion test for average BE and the two-sample t-test for scaled average BE. Table 10.1 [ → p. 319] displays the raw data in a format well suited to carrying out almost all necessary computational steps by means of a simple pocket calculator. To begin with the first component problem Hφ vs. Kφ [ → (10.48a)], we already know from § 10.2.2 that for ε∗φ = (.74/2)2 = .1369, the UMPI test rejects at the 5%-level. Thus, there remains to extend the analysis

Defining and establishing population bioequivalence

345

of the data set by carrying out the test for one-sided equivalence of the total variabilities derived in § 10.4.2 . The raw input data for computing the test statistic (10.52) are the entries in the middle rows of Table 10.1. Calculating the sums of squares and products appearing on the right-hand side of equation (10.51) with these values gives: SSX (1) = 0.5814 ,

SSX (2) = 0.3503 ,

SP X = 0.3133 ;

SSY (1) = 1.0280 ,

SSY (2) = 1.3879 ,

SP Y = 1.0059 .

If we specify ε∗σ = 1.00 for the constant which determines the upper equivalence limit in the hypotheses (10.48b), then plugging this value into (10.51) yields R√1+ε∗ = −.2715 . The corresponding value of the t-statistic (10.52) σ is T√1+ε∗ = −1.3229 . Since the .05-quantile of a central t- distribution with σ

m + n − 3 = 22 degrees of freedom is −1.7171, (10.54) implies that the null hypothesis Hσ of a relevant increase of the total variability associated with the test formulation of the drug, cannot be rejected. Hence, relying on the specifications εσ = 1.00 and α = .05, one-sided equivalence with respect to total variability cannot be established, and as a final consequence it has to be stated that the data of Table 10.1 do not contain sufficient evidence in support of population BE in the sense of the disaggregate criterion underlying (10.48). For comparison, we mention that an application of the aggregate criterion of population BE suggested in the current FDA guidance to the same data leads to an identical qualitative conclusion. In fact, the point estimate of the parametric function (10.43) is obtained to be θˆP = .4293, and taking B = 100, 000 bootstrap samples from the observed empirical distributions gave a (bias-corrected) upper 95%-confidence bound of 1.0609 for θP which is actually quite large.

10.4.4

Some results on power and sample sizes

If Π denotes the power of any statistical test obtained by means of the intersection-union principle against an arbitrarily selected specific alternative and each elementary test φν (ν = 1, ..., q) [cf. (7.3)] has rejection probability Πν under the same parameter constellation, then an application of the elementary Bonferroni inequality yields the relationship Π≥1−

q 

(1 − Πν ) .

(10.57)

ν=1

Using the expression appearing on the right-hand side of this inequality as a first order approximation to the power of the compound test can be recommended only for settings in which for any ν1 = ν2 , the acceptance regions of subtests φν 1 and φν 2 correspond to almost disjoint events. Furthermore, the approximation based on (10.57) is of real use for practice only if the power of

346

THE ASSESSMENT OF BIOEQUIVALENCE

each component test is a quantity easily accessible by direct computational techniques. Among the two individual tests making up our decision procedure for the assessment of disaggregate population BE, the first one admits exact power calculations as shown in § 6.1. In contrast to the two-sample t-statistic, the correlation coefficient R√1+ε∗ used for assessing possible differences between σ drug formulations in total variability, has a rather complicated distribution, and exact computation of nonnull rejection probabilities of the corresponding test would require numerical integration in several dimensions. Hence, in the case being our concern here, even the right-hand side of (10.57) cannot be evaluated exactly without recourse to advanced and time-consuming numerical methods. In view of this, the power of the test for one-sided equivalence with respect to total variability as well as that of the complete disaggregate testing procedure was investigated by means of Monte Carlo simulation. Table 10.5 Rejection probabilities of the t-test for scaled average bioequivalence ( ↔ Π1 ), the test for one-sided equivalence with respect to total variability ( ↔ Π2 ), and the disaggregate test for PBE as a whole ( ↔ Π ). [Except for Π1 , all probabilities are based on simulation, with 100,000 replications per experiment.] (Partially reproduced from Wellek, 2000b, with kind permission by John Wiley & Sons, Inc.)

†

m

n

ε˜φ †

ε∗σ

σT2 R /σS2

12 12 12 12 12 12 18 18 18 18 18 24 24 24 24 24 24

12 12 12 12 12 12 18 18 18 18 18 24 24 24 24 24 24

0.74 1.00 1.00 1.40 1.00 1.00 0.74 1.00 1.00 1.00 1.00 0.74 1.00 1.00 1.00 1.00 1.00

1.00 1.00 2.00 2.25 2.00 1.50 1.00 1.00 1.00 2.00 2.00 1.00 1.00 1.00 2.00 2.00 0.50

2.00 2.00 3.00 3.00 5.00 5.00 2.00 2.00 3.00 3.00 5.00 2.00 2.00 3.00 3.00 5.00 2.00

equivalence limit for 2(φT − φR )/

Π1 .24453 .57214 .57214 .90558 .57214 .57214 .44688 .81196 .81196 .81196 .81196 .63693 .92162 .92162 .92162 .92162 .92162



Π2 .57116 .57015 .84551 .88555 .82286 .69319 .74091 .74109 .67928 .95286 .94106 .84767 .84957 .79226 .98714 .98215 .47040

Π .13881 .32245 .48139 .80123 .47036 .39415 .32827 .60117 .55122 .77391 .76486 .53882 .78173 .72912 .91064 .90421 .43184

2 2 σW T + σW R

All simulation results shown in Table 10.5 refer to the 5% level and to balanced designs with equal numbers of subjects assigned to both sequence

Defining and establishing population bioequivalence

347

groups. As the only specific alternative of interest, perfect coincidence of the marginal distributions corresponding to the drug formulations, i.e., φT = φR and σT2 T = σT2 R is assumed. Inspecting the numerical material displayed in the table leads to the following major conclusions: • The minimum sample size required to guarantee a reasonable overall power Π of the disaggregate testing procedure (10.56), is a complicated function both of the equivalence limits ε˜φ , 1 + ε∗σ , and the ratio σT2 R /σS2 between the total and the inter-subject variance. • For fixed sample sizes m, n and given upper equivalence limit 1 + ε∗σ for σT2 T /σT2 R , the power Π2 of the test for equivalence in total variability decreases in σT2 R /σS2 . • The approximation to the overall power Π based on (10.57) is poor if the power of at least one of the two subtests is smaller than .70. However, if both Π1 and Π2 are larger than .80, the approximation is remarkably accurate.

10.4.5

Discussion

Technically, there is no problem to obtain a left-sided analogue of the test (10.54) for noninferiority with respect to total variability and to combine it with the right-sided version by means of the intersection-union principle into a test for equivalence in the strict, i.e., two-sided sense (for an unbiased test for two-sided equivalence with respect to intra-subject variability which is uniformly more powerful than the corresponding double one-sided test see Wang, 1997). However, reducing bioequivalence assessment with respect to variability to a one-sided testing problem of the form (10.48b) seems conceptually much more adequate than working with the two-sided version of that testing problem. To be sure, it is obvious that “hyperavailability” of a drug formulation under assessment in the sense of an increased population average may cause serious problems in patients treated with the drug. But it is hard to see any reason why a relevantly reduced variability of the pharmacokinetic measure of interest should be considered an undesirable property of a drug formulation which its potential consumer must be protected against. Hence, for purposes of establishing population bioequivalence, it seems fully sufficient indeed to give convincing evidence for the truth of (10.48a) and (10.48b) whereas the double two-sided criterion 2 2 ∗ ∗ 2 2 ∗∗ ∗ (φT − φR )2 /(σW T + σW R ) < εφ , 1 − εσ < σT T /σT R < 1 + εσ with εσ < 1 seems unnecessarily restrictive. This restrictiveness entails a rather drastic loss in the power of the associated testing procedure. For instance, replacing the equivalence region 0 < σT T /σT R < 2.00 by 0.50 < σT T /σT R < 2.00 in the situation covered by line 14 of Table 10.5 would imply that the power of the second subtest and of the overall testing procedure drops from .7923 and .7291 to .5866 and .5392, respectively.

348

THE ASSESSMENT OF BIOEQUIVALENCE

Finally, it seems worth noticing that for both elementary tests we did combine to form our disaggregate procedure, suitable nonparametric or distribution-free analogues are available. A distribution-free procedure for testing a hypothesis which coincides with that of (10.48a) in the standard parametric model is obtained by applying the Mann-Whitney test for equivalence discussed in § 6.2 to the within-subject differences Xi− and Yj− . Furthermore, nonparametric tests for one-sided equivalence with respect to intra-subject variability are provided in Section 2 of Liu and Chow (1992). The latter can readily be adopted for the problem of testing for one-sided equivalence with respect to total variability.

10.5

Bioequivalence assessment as a problem of comparing bivariate distributions

Although the same basic model is used for describing the primary observations, i.e., the log-bioavailabilities observed in the individual periods of the trial [recall p. 313], the view taken in this final subsection on the problem of BE assessment differs radically from all approaches treated up to now since no reduction of the pairs (X1i , X2i ), (Y1j , Y2j ) to univariate quantities will be carried out. The reason for this change of perspective is so elementary that one may ask why it has been totally ignored in the existing literature on the topic: If both formulations of the drug under study are actually identical, (Y11 , Y21 ), . . . , (Y1n , Y2n ) must be just a replicate sample from the same bivariate distribution from which the random sample (X11 , X21 ), . . ., (X1m , X2m ) making up the data set for sequence group T -R, has been taken. Thus, for identical formulations, we must in particular have E(X1i , X2i ) = E(Y1j , Y2j ) (for arbitrary i ∈ {1, . . . , m}, j ∈ {1, . . . , n}) which is incompatible with a nonzero difference both of the direct treatment and the carryover effects. In fact, if (10.2) is replaced with (9.31) [with (φA , φB ) being renamed (φT , φR )], then we have E(X1i ) − E(Y1j ) = φT − φR , E(X2i ) − E(Y2j ) = (φR − φT ) + (λ(1) − λ(2) ) so that φT = φR ⇒ E(X1i ) = E(Y1j ) and (φT = φR , λ(1) = λ(2) ) ⇒ E(X2i ) = E(Y2j ). As a clear conclusion of this simple argument, we can state that inequality of carryover effects is a specific form of inequivalence rather than a violation of the conditions for the validity of a suitable BE assessment procedure using the data from both periods of the trial. (Essentially the same fact has been pointed out from a nonequivalence perspective by Jones and Kenward, 2003, p. 44.) Thus, in what follows we adopt the view that assessment of BE requires comparison of the distributions underlying the pairs (X1i , X2i ) and (Y1j , Y2j ). Assuming the usual parametric model in the simplified version with treat-

Bioequivalence assessment as a bivariate problem ment-independent within-subject variances, we have     (X1i , X2i ) ∼ N (μ1 , μ2 ), Σ , (Y1j , Y2j ) ∼ N (ν1 , ν2 ), Σ ,

349

(10.58)

where the elements of the common covariance matrix Σ are given by σ12 = σS2 + σe2 , σ12 = σ21 = σS2 , σ22 = σS2 + σe2 .

(10.59)

Clearly, testing for equivalence of the two bivariate normal distributions can be done by means of any of the procedures considered in § 8.2. We focus here on the equivalence version of Hotelling’s two-sample T 2 -test and start with observing that in the bivariate case, Mahalanobis distance between two Gaussian distributions with common covariance matrix can be written Δ2 (μ, ν; Σ) = (μ1 − ν1 )2 σ22 − 2σ12 (μ1 − ν1 )(μ2 − ν2 ) + (μ2 − ν2 )2 σ12 . (10.60) 2 σ12 σ22 − σ12 In the bivariate approach, we require of bioequivalent drug formulations that the distance between the bivariate Gaussian distributions underlying the nonreduced data set obtained in both sequence groups, be smaller than some specified positive bound, say Δ2◦ . Accordingly, we propose to formulate the testing problem Hb : Δ2 (μ, ν; Σ) ≥ Δ2◦ versus Kb : Δ2 (μ, ν; Σ) < Δ2◦ ,

(10.61)

where the subscript b is added in order to indicate that this is the bivariate formulation of the problem of assessing bioequivalence with data obtained from a standard comparative bioavailability study. It is interesting to have a closer look at the geometric shape of the region in the parameter space which corresponds to the alternative hypothesis of (10.61). Under the restricted covariance structure (10.59) arising from the standard model for BE trials, Kb can easily be shown to hold true if and only if we have 3 2 (φT − φR ) + (λ(2) − λ(1) )/2 √ + σe / 2 2 41/2  (2) λ − λ(1)  < Δ◦ . (10.62) 2 σS2 + σe2 /2 Obviously, (10.62) defines a circular disk of radius Δ◦ around zero in a suitable two-dimensional space. The most important feature of this space is that it refers to a two-dimensional parameter distance   whose Euclidean  from the   origin increases on φT , φR , λ(1) , λ(2)  (φT − φR ) λ(1) − λ(2) > 0 both in |φT −φR | and |λ(1) −λ(2) | . Moreover, the circular disk corresponding to (10.62)

350

THE ASSESSMENT OF BIOEQUIVALENCE

provides a good approximation to the equivalence region specified by Kb even 2 2 and σeR have to be in cases where two different within-subject variances σeT taken into account. In order to see this, it suffices to note that (10.62) reflects the independence of the natural estimators of (φT − φR ) + (λ(2) − λ(1) )/2 and 2 2 (λ(2) −λ(1) )/2 implied by σeT = σeR . Moreover, in the general case, the square of the correlation coefficient between these estimators is easily computed to (σeT − σeR )2 (σeT + σeR )2 be which is typically still close to zero since 2 2 2 2 (σeT + σeR )(4σS2 + σeT + σeR ) the within-subject variances will be much smaller than the between-subject variance. In the bivariate case, the two-sample T 2 -statistic admits the explicit representation  mn ¯ 1 − Y¯1 )2 S22 − (X T2 = 2 N (S12 S22 − S12 )  2 ¯ 1 − Y¯1 )(X ¯ 2 − Y¯2 ) + (X ¯ 2 − Y¯2 )2 S12 , (10.63) 2S12 (X with S12 , S22 and S12 defined as in (8.34). Specializing (8.37) and (8.38), the critical region of an exact UMPI level-α test for (10.61) can be written 

  (10.64) T 2 < 2 (N − 2)/(N − 3) F2,N −3; α (mnΔ2◦ /N ) , and the power against any specific alternative (μ, ν, Σ) with Δ2 (μ, ν, Σ) = Δ2a ∈ [0, Δ2◦ ) is given by   (10.65) β(Δ2a ) = P F2,N −3 (mnΔ2a /N ) ≤ F2,N −3; α (mnΔ2◦ /N ) . Except for the change referring to the degrees of freedom, the notation in (10.64) and (10.65) is the same as in § 8.2.1. I.e., F2,N −3 (ψ 2 ) stands for a random variable following a noncentral F -distribution with 2, N −3 degrees of freedom and noncentrality parameter ψ 2 ≥ 0, whereas F2,N −3; α (ψ 2 ) denotes the lower 100α percentage point of this distribution. Of course, without the assumption of homoskedasticity and bivariate normality of the distributions under comparison, it cannot even be taken for granted that the rejection region (10.64) has size α, which raises the question of robustness against departures from one or the other of these assumptions. For higher-dimensional data, the sensitivity of the T 2 -test for equivalence against deviations from homoskedasticity has been investigated in Chapter 8. For the bivariate case, we conducted separate simulation studies to assess the impact of nonnormality and heteroskedasticity both on size and power to detect exact coincidence of both mean vectors. Table 10.6 shows the simulated rejection probabilities under constellations with non-Gaussian forms of the distributions, and first and second moments defining a point in the parameter space lying on the common boundary of the hypotheses and at the most extreme specific alternative, respectively.

Bioequivalence assessment as a bivariate problem

351

Table 10.6 Simulation results on the sensitivity of the T 2 -test for (bio-)equivalence against nonnormality of distributions.

†)

Distributional Shape

Reject. Prob. at Δ2 = Δ2◦

Power at Δ2 = 0

10 ∗ ∗ " .165 .020 " .20 .10 " .165 .020 " .20 .10

Gaussian Logistic " " Exponential " "

.0500†) .0487 .0416 .0428 .0416

.3581†) .3546 .3535 .3375 .3349

15 ∗ ∗ " .165 .020 " .20 .10 " .165 .020 " .20 .10

Gaussian Logistic " " Exponential " "

.0500†) .0496 .0480 .0451 .0431

.5840†) .5793 .5793 .5741 .5737

m

n

10 " " " " 15 " " " "

σS2

σe2

Exact values, depending on all parameters only through Δ2

The distributional forms appearing in the fifth column refer to the marginal  (2)  (1) (1)  [↔ group T /R], Dj , distributions of linearly transformed pairs Di , Zi (2)  [↔ group R/T ] with uncorrelated components. In the homoskedasZj (1)

2 2 tic case σeT = σeR , such transforms are readily obtained by setting Di = (1) (2) (2) X1i − X2i , Zi = X1i + X2i and Dj = Y1j − Y2j , Zj = Y1j + Y2j , respectively. Clearly, the invariance properties of the T 2 -test imply that it makes no  (1) (1)   (2) (2)  difference whether the (X1i , X2i ), (Y1j , Y2j ) or the Di , Zi , Dj , Zj are treated as the primary data. Table 10.7 shows analogous simulation results for bivariate Behrens-Fisher situations. For generating the data for the Monte Carlo experiments performed for the purpose of studying the influence of heteroskedasticity on the size of the critical region (10.64), Mahalanobis dis¯ = (1/2)(Σ1 + Σ2 ) of both tance was computed with respect to the average Σ group-specific covariance matrices, and the mean vectors μ, ν were specified ¯ attains the bound Δ2◦ set to equivalent hoin a way ensuring that Δ2 (μ, ν; Σ) moskedastic Gaussian distributions under the original alternative hypothesis Kb . Both series of simulation experiments refer to a nominal significance level of 5% and an equivalence bound Δ2◦ of unity. All rejection probabilities shown in the tables are estimates based on 40, 000 replications of the respective Monte Carlo experiment. The insights provided by the results into both aspects of the robustness problem arising in the present context are as clear-cut as encouraging:

• Both kinds of deviations from the original model affect mainly the size of the test.

352

THE ASSESSMENT OF BIOEQUIVALENCE

• The differences found between the nominal level and the true rejection probabilities on the boundary of the region of nonequivalence point to the overconservative direction. • Given the size of both samples, overconservatism is more marked under heteroskedasticity than deviations from the Gaussian form of the marginal distributions.

Table 10.7 Simulation results on the sensitivity of the T 2 -test for (bio-) equivalence against heteroskedasticity. m

n

σS2

2 σeT

2 σeR

10 10 .165 .005 .020 " " .20 .05 .20 15 15 .165 .005 .020 " " .20 .05 .20 20 20 .165 .005 .020 " " .20 .05 .20 †) Σ ¯

Reject. Prob. under Power against ¯ †) ) = Δ2◦ Δ2 (μ,ν; Σ μ =ν .0406 .0325 .0361 .0277 .0344 .0264

.3578 .3588 .5810 .5823 .7539 .7532

≡ (1/2)(Σ1 + Σ2 )

Example 10.3 As an illustration of the bivariate approach to testing for scaled average BE, we apply it to the log-bioavailabilities shown in Table 10.8. With these data, the sample means and pooled estimates of the covariance and both variances are calculated ¯ 1 = 4.8819 X Y¯1 = 5.0921 S12 = 0.1839

, , ,

¯ 2 = 4.8623 ; X Y¯2 = 5.0697 ; S22 = 0.1490 ; S12 = 0.1405

Plugging in these values in Equation (10.63) yields T2 =

 100 (4.8819 − 5.0921)2 · 20(.1839 · .1490 − .14052 ) .1490 − 2 · .1405 · (4.8819 − 5.0921) · (4.8623 − 5.0697)  +(4.8623 − 5.0697)2 · .1839 = 1.4643 .

Bioequivalence assessment as a bivariate problem

353

Table 10.8 Logarithmically transformed AUC’s measured in both periods of a standard bioequivalence trial of two other formulations of nifedipine [different from those referred to in Example 10.2]. Sequence Group T/R i X1i X2i

1

2

3

5.32833 4.60833 4.61290 4.91018 4.98835 4.53084 7

8

9

4

5

6

4.85399 4.82933 4.61250 5.00148 4.88278 4.68550 10

4.89988 4.48080 6.00856 4.83594 4.43013 5.72340

4.58419 4.63451

Sequence Group R/T j

1

Y1j Y2j

4.97670 5.32219 7 4.52988 4.79501

2

3

4

5.79190 5.37921 5.11251 5.51966 5.17163 4.98796 8

9

5

6

5.29075 4.64138 5.31198 4.34402

10

4.70318 5.35740 5.13770 4.50227 5.56355 5.17898

On the other hand, specifying Δ◦ = 1 and using the SAS intrinsic function finv for computing the noncentral F -quantile required for determining the critical upper bound to T 2 , we obtain the latter as 2

N −2 18 F2,N −3; α (mnΔ2◦ /N ) = 2 · finv(.05,2,17,5.00) N −3 17 = 2.1176 · 0.4551 = 0.9637 ,

provided of course, that the test has to be carried out at the usual level α = 5%. Since the observed value of T 2 clearly fails to remain under this bound, we have to accept the null hypothesis Hb of (10.61) with the present data, despite the fact that Δ◦ has been chosen to be much larger than recommended earlier [→ § 1.7] for the univariate version of the same test. It is of considerable interest to note that this negative result is almost exclusively due to a comparatively large estimated difference between the carryover effects, whereas estimating the difference of treatment effects gives a standardized value being as small as .0272. Thus, the data set shown in Table 10.7 is an instance of the possibility that the bivariate and the conventional

354

THE ASSESSMENT OF BIOEQUIVALENCE

univariate approach to testing for (scaled) average BE may lead to opposite conclusions. With regard to contributions to the existing literature dealing with the problem of BE assessment from a point of view declared explicitly to be multivariate in nature (see Chinchilli and Elswick Jr., 1997), it is important to note that a different kind of multivariateness is intended there. In fact, these authors propose to avoid reduction of the serum-concentration profile to simple univariate measures of bioavailability like AUC and Cmax by using a measure of distance between the profiles obtained from the same subject which integrates the distances at all relevant points on the time axis.

11 Tests for relevant differences between treatments

11.1

Introduction

Formally speaking, the most straightforward way of explaining what we mean by a a test for relevant differences, is to state that the testing problem to which such a procedure is tailored, arises from interchanging the two hypotheses making up an equivalence problem without modifying them otherwise, except for replacing the interval corresponding to the former alternative with its closure. Thus, in a generic formulation, all problems to be considered in this chapter read H∗ : θ ∈ [θ◦ − ε1 , θ◦ + ε2 ] versus

K∗ : θ < θ◦ − ε1 ∨ θ > θ◦ + ε1 . (11.1)

All symbols appearing in (11.1) have the same meaning as before [cf. in particular § 1.5] and need not to be defined again. The reason for including the endpoints of the interval of irrelevant deviations from θ◦ in H∗ is purely technical in nature: Generally, in a hypothesis testing problem of any form, it is thought to be desirable that the power function of any unbiased level-α test takes on α as its maximum value under the null hypothesis H0 . Given the power function is continuous, this condition is satisfied whenever the subset in the parameter space corresponding to H0 , is compact. From a conceptual perspective, problems of testing for relevant differences are much more directly related to traditional two-sided than equivalence problems. Actually, the null hypothesis H∗ of (11.1) is simply a generalization of the ordinary point null hypothesis specifying θ = θ◦ and reduces to the latter by choosing both relevance margins to equal zero. Replacing under the null hypothesis the single point θ◦ in the parameter space of θ by some nondegenerate interval, is a promising step toward finding a way around the problems connected with the plain fact that considering an effect being significant in terms of the result of a traditional two-sided test as substantial or scientifically relevant can be grossly misleading. Although the scope of potential applications of tests for problems of the form (11.1) is certainly not less broad than that of equivalence testing procedures, the role that inferential procedures of that type have played up to now in the practice of statistical data analysis has been marginal at best. This

355

356

TESTS FOR RELEVANT DIFFERENCES

holds true notwithstanding the fact that in the medical statistics literature it was pointed out already decades ago (cf. Victor, 1987) that analyzing studies run to demonstrate superiority of one treatment over the other by means of tests for relevant difference rather than tests of traditional null hypotheses should be the rule rather than the exception. Perhaps, one of the major reasons explaining this state of the matter has to do with complications involved in establishing sample size criteria for tests for relevant differences. Actually, as will become apparent from the results to be presented in the sections to follow, the power function of each reasonable test of (11.1) with θ as the only parameter upon which the family of distributions under consideration depends, is convex from below with a unique global minimum taken on at, or in a neighborhood of, the target value θ◦ . Thus, determining the sample size for a trial to be analyzed by means of a test of this form, requires specification of a point in the parameter space whose distance from θ◦ exceeds the relevance margins εν (ν = 1, 2) by some additional relevance threshold defining the alternative against which the desired level of power has to be attained. Accordingly, even in a one-parameter setting, complete statistical planning of a study whose confirmatory analysis is to be based on a test for a problem of the form (11.1), entails specification of three extra constants in addition to the significance level α . A pragmatic proposal we will follow in the remainder of this chapter for choosing the numerical values of both the margins εν and the specific alternative to be detected with given power, is to make use of the specifications proposed in Table 1.1 for equivalence testing in the following way: The interval [θ◦ − ε1 , θ◦ + ε2 ] of nonrelevant deviations will be set equal to what was declared there to be a strict specification of the equivalence range, and the specific alternative to one of the endpoints of the equivalence interval in its “liberal” version. Taking the paired t-test setting as a specific example, this amounts to setting under H∗ the limits ±.25 to δ/σ, whereas the prespecified power has to be attained against δ/σ = .5, and so on for the other settings covered by Table 1.1.

11.2

Exploiting the duality between testing for two-sided equivalence and existence of relevant differences

Essentially, given the margins, the only difference between a problem of testing for relevant differences in the sense of (11.1) and the corresponding two-sided equivalence problem is that null and alternative hypotheses switch their roles. Hence, it seems reasonable to generate tests for the former by reinterpreting an equivalence testing procedure available for the same setting in the following way: Decision has to be taken in favor of K∗ if and only if the equivalence

Exploiting the duality with testing for two-sided equivalence

357

test cannot reject its null hypothesis, implying that what was formerly a type-II error becomes now an error of the first kind, and the other way round. Representing any test by its critical function (which, except for randomized decisions, is simply the indicator of its rejection region) and indicating whether the test is to be used for establishing equivalence or existence of relevant differences by appending the subscript eq or rd to the symbol φ commonly used for critical functions of statistical tests, the relationship we have in mind can be made precise by writing: φrd = 1 − φeq .

(11.2)

Since the rejection probability under any θ of both tests is given by Eθ (φrd ) and Eθ (φeq ) where expectation has to taken with respect to the joint distribution of the random sample under analysis, it is clear that a minimum requirement for obtaining from (11.2) a valid level-α test of (11.1) is that we have (11.3) Eθ (φeq ) ≥ 1 − α for all θ ∈ [θ◦ − ε1 , θ◦ + ε2 ] . In other words, the equivalence test φeq to be started from, must satisfy the following twofold condition: (i) As a test of the null hypothesis H : θ ≤ θ◦ − ε1 or θ ≥ θ◦ + ε2 , φeq must be performed at (nominal)“significance” level α = 1 − α rather than α ; (ii) φeq has to be unbiased at this level α . It is important to note that, due to their lack of unbiasedness, confidence interval inclusion tests are generally not suitable for being inverted in tests for relevant differences through applying the duality principle (11.2). In order to see this, we consider the one-sample problem with normally distributed data of unit variance, symmetric choice ε1 = ε2 = ε of the margins, and θ◦ = 0 as the target value of the parameter θ = E(X) of interest. As shown in § 4.4 [cf. p. 64], the confidence interval inclusion test at level α = 1 − α of H : |θ| ≥ ε versus K : |θ| < ε rejects if and only if the√observed mean computed from n mutually ¯ n | < √nε − uα . Applying (11.2), this independent observations satisfies n|X is converted to the decision rule of rejecting H∗ if and only if it turns out that ¯ n ≥ ε − uα /√n or X ¯ n ≤ −ε + uα /√n X √ ¯ n − u1−α / n ≥ ε or X ¯ n + u1−α /√n ≤ −ε . ⇔ X The second of these conditions for rejecting H∗ can be rephrased by saying that the equal-tails (1 − 2α)−confidence interval for θ must not have a nonempty intersection with the indifference zone (−ε , ε) so that applying the duality principle (11.2) transforms the interval inclusion test for equivalence into an interval exclusion test for relevant differences. From the first formulation of the rejection rule of this interval exclusion test, it readily follows that

358

TESTS FOR RELEVANT DIFFERENCES

its rejection probability under any θ ∈ IR is given by √   √  1 − Φ n(ε − θ) + u1−α + Φ − n(ε + θ) − u1−α . For θ = ε, the probability of rejecting H∗ is thus √ 1 − Φ(u1−α ) + Φ(− n 2ε − u1−α ) . In view of 1 − Φ(u1−α ) = α and Φ(·) > 0, this shows that the rejection probability of the interval exclusion test at the common boundary of the hypotheses (11.1) exceeds α. Accordingly, the test obtained by means of the duality principle fails to maintain the significance level in the present case. Whenever the above conditions (i) and (ii) for an application of the duality principle are satisfied, the practical implementation of a test for relevant differences is straightforward. In a first step, one determines critical constants C1∗ , C2∗ , γ1∗ , γ2∗ (which, in multiparameter settings, might also depend on the value of some conditioning statistic) by solving the system (3.9) [see p. 42] with θν = θ◦ + (−1)ν εν , ν = 1, 2, and 1 − α instead of α. Subsequently, decision is taken in favor of the alternative hypothesis K∗ of relevant differences, if the observed value of the test statistic T (X) falls outside the interval [C1∗ , C2∗ ]. For C1∗ < T (X) < C2∗ , the null hypothesis H∗ of nonexistence of relevant differences is  accepted, and at the left- (right)-hand boundary of the critical region T ∩ − ∞, C1∗ ) ∪ (C2∗ , ∞), H∗ is rejected with probability γ1∗ (γ2∗ ). [As before, the symbol T stands for the range of the test statistic T (X) .] The fact that it is natural to consider any testing problem of the form (11.1) as a generalized two-sided problem in the usual sense, is corroborated by observing that even in the case of a one-parameter family being STP3 , there are no UMP tests for relevant differences. Actually, the results proven in §§3.7 and 4.2 of Lehmann and Romano (2005) imply that inverting a UMP level-(1 − α) test for equivalence by means of (11.2) yields a test of (11.1) which is uniformly most powerful only among all unbiased tests for the same problem. As is well known, the classical two-sided problem with a one-point null hypothesis about the parameter of a STP3 -family of distributions, likewise admits only a UMPU solution whereas a test which is uniformly most powerful among all valid tests, does not exist. Of course, exact UMPU solutions to problems of testing for relevant differences exist only for a comparably small number of settings. Often, one will have recourse to asymptotic methods to be obtained by applying the duality principle to an equivalence test of the form described in general terms in § 3.4 . All that has to be done for implementing such a testing procedure is that in the corresponding asymptotic test for equivalence, the critical constant to which the suitably centered and normalized test statistic is compared must be determined by computing the square root of the upper 100 (1−α)-percentage point of a χ2 -distribution with a single degree of freedom and noncentrality param2 2 eter ψ 2 = (ε1 + ε2 )2 /4 τˆN , with τˆN denoting a weakly consistent estimator for the asymptotic variance of the test statistic. Of course, rejection of the null

Solutions to some special problems

359

hypothesis of absence of relevant differences has again to be done if the test statistic falls to the left or the right of the critical interval. Finally, another promising option for addressing problems of testing for relevant differences for which no satisfactory solutions by means of exact finite-sample constructions are available, is to adapt the objective Bayesian approach to the hypotheses formulation now under consideration. Technically, this just requires replacing in the general decision rule (2.7) of a Bayesian test for noninferiority the set (θ◦ − ε, ∞) with (−∞, θ◦ − ε1 ) ∪ (θ◦ + ε2 , ∞) .

11.3 11.3.1

Solutions to some special problems of testing for relevant differences One-sample problem with normally distributed data of known variance

The setting to be considered briefly here is largely the same as in § 4.1 and need not be described in detail again. It suffices to recall that any testing problem involving interval hypotheses about the expected value θ, say, of a Gaussian normal distribution with known variance σ◦2 > 0 from which a sample (X1 , . . . , Xn ) of arbitrary size n is given, can be reduced without loss of statistical information to a pair of statements specifying that the expected value θ˜ of some normally distributed random variable Z with unit variance lies in- or outside some symmetric interval with endpoints ±˜ ε. If we denote the critical function of the optimal (UMPU) level-α test for ˜ ∗ : −˜ ε ≤ θ˜ ≤ ε˜ versus H

˜ ∗ : θ˜ < −˜ K ε or θ˜ > ε˜

(11.4)

based on this Z by ψ˜∗ (·), then it follows from the duality principle that  ∗ 1 for |z| > Cα; ε˜ , (11.5) ψ˜∗ (z) = ∗ 0 for |z| ≤ Cα; ε˜ ∗ where Cα; ε˜ denotes the unique solution of

Φ(C − ε˜) − Φ(−C − ε˜) = 1 − α ,

C > 0.

(11.6)

For computational purposes, it is more convenient to make use of the possibil∗ ity of expressing Cα; ε˜ explicitly through the quantile function of a noncentral χ2 -distribution with a single degree of freedom. The required expression is obtained from Equation (4.6) by replacing α with its complement α = 1 − α so that we can write  ∗ Cα; χ21; α (˜ ε2 ) . (11.7) ε˜ =

360

TESTS FOR RELEVANT DIFFERENCES

By means of (11.7), it is easy to determine the entries in Table 11.1 which is the relevant-differences analogue of Table 4.1. Substituting α for α in the approximation formula (4.26) yields √ √ ∗ Cα; nε − u1−α = nε + u1−α , (11.8) ε˜ ≈ and not surprisingly, it becomes obvious from the exact values shown in Table 11.1, that the accuracy of this approximation is almost perfect even for considerably smaller values of ε˜ than required in the equivalence case. ∗ Table 11.1 Optimal critical constant Cα; ε˜ [→ (11.6), (11.7)] for testing for relevant differences at the 5% level in the one-sample case with normally distributed data of known variance.

ε˜ .0 .1 .2 .3 .4 .5 .6 .7 .8 .9

0. 1.95996 1.96973 1.99855 2.04505 2.10699 2.18148 2.26539 2.35583 2.45047 2.54760

1. 2.64615 2.74544 2.84511 2.94496 3.04490 3.14487 3.24486 3.34486 3.44485 3.54485

2. 3.64485 3.74485 3.84485 3.94485 4.04485 4.14485 4.24485 4.34485 4.44485 4.54485

3. 4.64485 4.74485 4.84485 4.94485 5.04485 5.14485 5.24485 5.34485 5.44485 5.54485

4. 5.64485 5.74485 5.84485 5.94485 6.04485 6.14485 6.24485 6.34485 6.44485 6.54485

In order to illustrate the use of the critical constants computed from (11.6) or, equivalently, (11.7), let us suppose that we want to test for relevant differences of the expected value of a normal distribution with unit variance from zero specifying the indifference zone to range from −.25 to +.25 . If the size of the sample having been made available for that purpose is n = 100 and the test is to be carried out at the usual significance level α = .05, then we read from the third column of the above table that the observed value √ of the ¯ has to be compared with the critical bound 4.14485/ 100. sample mean X Thus, the existence of relevant differences can be declared established at the ¯ exceeds the bound 0.414485 which 5%-level if and only if it turns out that |X| is more than twice as large as the critical bound to be used in a traditional two-sided test for the same setting. The implications of this fact for the power of both tests are clear enough: As shown in Figure 11.1, introducing a relevance margin of ε = .25 considerably reduces the rejection probability of the test under any true parameter value θ ∈ IR , of course except those specific alternatives which are so extreme that even the power of the test for relevant

Solutions to some special problems

361

¯ relates to a samdifferences is arbitrarily close to unity. Generally, when X ple of n observations from (θ, 1), the  rejection probability  of any test with  √ N ¯ < −C ∪ C < √nX ¯ < ∞ is easily computed critical region − ∞ < nX exactly using the formula   √  √  P OW = Φ − C − nθ + Φ − C + nθ (11.9) The values behind both power curves displayed in Figure 11.1 were obtained in this way.

Figure 11.1 Power function of the UMPU test at level α = .05 for |θ| ≤ .25 vs. |θ| > .25 [solid line] and the conventional two-sided null hypothesis θ = 0 [dotted line] based on n = 100 observations from N (θ, 1). [Bold-drawn bars on horizontal coordinate axis demarcate the region or relevant differences.]

11.3.2

Two-sample t-test for relevant differences

The relevant-differences analogue of the equivalence testing problem considered in § 6.1 reads: H∗ : −ε1 ≤ (ξ − η)/σ ≤ ε2 versus K∗ : (ξ − η)/σ < −ε1 or (ξ − η)/σ > ε2 ,

(11.10)

where ξ and η are the expected values of two normal distributions with common unknown variance σ 2 > 0 from which independent samples X1 . . . , Xm

362

TESTS FOR RELEVANT DIFFERENCES

and Y1 . . . , Yn are given. The sizes m and n of both samples need not to be equal, and similarly, no restrictions on the two relevance margins ε1 , ε2 have to be made. Although the setting of the two-sample t-test involves a multiparameter family of distributions, like its equivalence counterpart, the problem (11.10) can be reduced by invariance to a pair of hypotheses relating to a single STP3 family. In this reduced version, it admits a UMPU solution which is readily obtained by applying the principle of duality to the critical region given by (6.3) and (6.4). Adopting the arguments used in § 6.6 of Lehmann and Romano (2005) in the one-sample analogue of problem (11.10) with ε1 = ε2 = 0, it can be shown that the test obtained by composing  the critical function of x − y¯)/ this UMPU test with the mapping (x1 , . . . , yn ) → mn(N − 2)/N (¯

 1/2 n m 2 2 x) + j=1 (yj − y¯) ≡ t(x1 , . . . , yn ), is UMP among all level-α i=1 (xi −¯ tests for (11.10) which are both invariant and unbiased. The practical implementation of this t-test for relevant differences between two homoskedastic Gaussian distributions can be accomplished with the same computational tools made available before for the two-sample t-test for equivalence. Actually, the rejection region can be written



 T < C˜α1  ; m,n (−ε1 , ε2 ) ∪ T > C˜α2  ; m,n (−ε1 , ε2 ) , (11.11) where the critical constants C˜αν  ; m,n ν = 1, 2, are determined from Equations (6.4) with substituting 1 − α for α. If the margins are chosen symmetrically setting ε1 = ε2 = ε for some ε > 0, (11.11) simplifies to

 |T | > C˜α ; m,n (ε) , (11.12) where C˜α ; m,n (ε) can be computed by determining the square root of the upper 100 α percentage point of an F -distribution with 1, N − 2 degrees of freedom and noncentrality parameter λ2nc = mnε2 /N . Clearly, the power of the test with rejection region (11.11) against any alternative (ξ, η, σ 2 ) with (ξ − η)/σ ≡ θ ∈ (−∞, −ε1 ) ∪ (ε2 , ∞) is given by  1  √ C˜α ; m,n (−ε1 , ε2 ) + β∗ (θ) = F T N −2; θ mn/N  2  √ 1 − FT C˜α ; m,n (−ε1 , ε2 ) , (11.13) N −2;θ

mn/N

where FNT −2;λnc (·) symbolizes the cumulative distribution function of a random variable which is noncentral t with df = N − 2 and noncentrality parameter λnc ∈ IR . In view of the above formula, it is an easy exercise to recalculate the entries in Tables 6.1 and 6.2 for the t-test for relevant differences. Of course, the specific alternative of interest must now be chosen as some point θ with |θ| > ε. The entries in Table 11.3 hold under the assumption that we specify θ = 2ε .

Solutions to some special problems

363

Table 11.2 Critical constant C˜.95; n,n (ε) of the two-sample t-test for relevant differences at level α = 5% in the case of a balanced design with n = 10(5)75 and relevance margin ε = .10(.10).50.

n 10 15 20 25 30 35 40 45 50 55 60 65

.10 2.15247 2.12304 2.12175 2.13028 2.14322 2.15838 2.17473 2.19172 2.20902 2.22645 2.24387 2.26122

ε .20 2.29572 2.32217 2.37146 2.42582 2.48048 2.53387 2.58545 2.63512 2.68291 2.72893 2.77330 2.81615

= .30 2.50159 2.58970 2.68894 2.78532 2.87655 2.96262 3.04402 3.12130 3.19497 3.26545 3.33309 3.39821

.40 2.73959 2.88182 3.02417 3.15753 3.28184 3.39825 3.50791 3.61176 3.71060 3.80506 3.89566 3.98282

.50 2.99103 3.18130 3.36387 3.53289 3.68971 3.83619 3.97396 4.10433 4.22831 4.34674 4.46027 4.56946

Table 11.3 Power against the alternative θ = 2 ε attained in the two-sample t-test for relevant differences for the sample sizes and margin specifications underlying Table 11.2.

n 10 15 20 25 30 35 40 45 50 55 60 65

.10 .06467 .07233 .07963 .08663 .09335 .09983 .10609 .11216 .11805 .12378 .12936 .13482

ε .20 .10159 .12512 .14640 .16616 .18485 .20276 .22008 .23691 .25335 .26945 .28524 .30075

= .30 .14937 .19137 .22979 .26609 .30085 .33431 .36659 .39774 .42778 .45672 .48457 .51133

.40 .20304 .26744 .32710 .38314 .43579 .48513 .53120 .57404 .61374 .65041 .68416 .71513

.50 .26168 .35179 .43381 .50815 .57493 .63441 .68694 .73301 .77315 .80792 .83788 .86358

Example 11.1 In 2005, the German Competence Network on Dementias started an actively controlled clinical trial to evaluate the efficacy and safety of an antidementive

364

TESTS FOR RELEVANT DIFFERENCES

combination therapy (galantamine plus memantine) in subjects with mildto-moderate stage of probable Alzheimer’s disease. In the control arm of the study, galantamine was administered as monotherapy, and random assignment of patients to the two arms was done in a 1:1 manner. As primary outcome criterion for assessing the antidementive effects, the change of the ADAS-cog score after 12 months of treatment was used. As is true for any rating scale based on a set of questionnaire items, the distribution of this variable is in principle discrete. However, in view of the comparatively large number of   possible values taken on by the ADAS-cog score range: {x ∈ IN0 | x ≤ 70} , when planning the trial one felt justified to rely on the standard parametric model assumptions behind the two-sample t-test. Sample-size calculation was based on the one-sided version of the test at the usual level α = .05 and the requirement that a true effect of size θ = (ξ − η)/σ = .40 should be detected with a power of 80% at least. Accordingly, the recruitment target was set to n = 78 per group. Suppose that the same trial is to be planned with a view to testing for relevant differences in terms of (ξ − η)/σ between the combination of both drugs and monotherapy fixing the margin at ε = .20 and leaving unchanged both the alternative of interest and the requirement on the minimum power. Then it becomes obvious from the entries in the second column of Table 11.3 that the size of both samples would have to be much larger than 78. By iterating formula (11.13) on m = n, the precise value of n turns out to be as large as 310. Mainly, this increase is due to introducing a nonzero relevance margin rather than replacing a one- with a two-sided test.

11.3.3

Exact Fisher type test for relevant differences between two binomial distributions

Another standard setting of considerable practical importance where applying the duality principle yields an exactly valid optimal test for relevant differences is the binomial two-sample problem with the odds ratio as the parameter of interest. Denoting the critical function of the equivalence test considered in § 6.6.4 for any fixed value s ∈ {0, 1, . . . m + n} of the column total S belonging to the contingency table summarizing the data set under analysis [see Table 6.20] by φ(·, s) and observing that this test has to be performed at (nominal) level α = 1 − α in the present context, we can write: ⎧ m,n 1 if X < C1; ⎪ α (s; 1 , 2 ) ⎪ ⎪ m,n ⎪ ⎪ or X > C2; ⎪ α (s; 1 , 2 ) ⎪ ⎪ ⎪ ⎪ ⎨ 1 − γ m,n (s; 1 , 2 ) if X = C m,n (s; 1 , 2 ) 1; α 1; α 1 − φ(x, s) = . (11.14) m,n m,n ⎪ 1 − γ (s;  ,  ) if X = C   (s; 1 , 2 ) 1 2 ⎪ 2; α 2; α ⎪ ⎪ ⎪ ⎪ m,n ⎪ 0 if C1; ⎪ α (s; 1 , 2 ) < ⎪ ⎪ ⎩ m,n X < C2; α (s; 1 , 2 )

Solutions to some special problems

365

All critical constants appearing in the definition of this decision rule have to be determined by solving the equations (6.98) with α being substituted for α. The algorithm provided in Chapter 6 for that purpose works for any choice of the nominal level so that the program bi2st can also be used for testing for relevant differences from unity of the odds ratio associated with the two binomial distributions under comparison. Since the construction carried out in § 6.6.4 ensures in particular that φ(X, S) / (1 , 2 ) versus K :  ∈ (1 , 2 ), 1 − φ(X, S) is unbiased at level α for H :  ∈ gives a test of H∗ :  ∈ [1 , 2 ] vs. K∗ :  < 1 or  > 2

(11.15)

whose rejection probability under H∗ does not exceed α. From the results proven in Lehmann and Romano (2005, § 4.4), it follows that it is unbiased in turn and maximizes the power uniformly among all unbiased level-α tests for (11.15). The practical application of the decision rule given by (11.14) is best illustrated by another concrete example. Example 11.2 One of the polymorphisms which were investigated in an association study on the genetic background of psoriasis was a SNP denoted TNFA-238 which belongs to a gene related to the tumor necrosis factor alpha. The proportion of carriers of at least one G-allele at the corresponding locus was found to differ considerably between cases and controls. The pertinent part of the data published by Reich et al. (2002) on the results of this study yield the following 2 × 2 contingency table: Table 11.4 Frequencies of carriers of at least one G-allele at a SNP located in a gene encoding the tumor necrosis factor TNF-α obtained in a genetic association study on psoriasis. (Data from Reich et al., 2002.)

Group

Presence of G-allele + –

Σ

Control

29 (8.41%)

316 (91.59%)

345 (100.0%)

Psoriasis

53 178 (22.94%) (77.06%)

231 (100.0%)

Σ

82

494

576

366

TESTS FOR RELEVANT DIFFERENCES

Considering the association between the SNP and the disease status as relevant only if the true odds ratio satisfies either  < 2/3 or  > 3/2, the basic step to be taken for determining the critical constants for the test (11.14) at level α = .05 consists of running program bi2st with .95, 345, 231, 82, 2/3, and 3/2 as the values assigned to its five arguments. The results it then returns are: C1 = 35, C2 = 62, γ1 = .2891, γ2 = .6703 . According to (11.14), this means that for the values of the marginal frequencies shown in Table 11.4, the null hypothesis of irrelevant deviations of the odds ratio from unity has to be rejected if the observed count X of carriers of a Gallele among the controls is less than 35 or larger than 62. In the exact UMPU test at level α = .05, randomized decisions in favor of K∗ have to be taken with probabilities 1−γ1 = .7109 (for X = 35) and 1−γ2 = .3297 (for X = 62), and for 36 ≤ X ≤ 61, the null hypothesis H∗ must be accepted. Since the realized value of X is much smaller than the lower critical bound, existence of a relevant association between TNFA-238 and the psoriasis condition can be declared established with the data presented by Reich et al. (2002). Computing the power of the exact randomized UMPU test (11.14) against arbitrary alternatives (p1 , p2 ) requires no more than running the program made available in § 6.6.4 for calculating the power of the exact Fisher type test for equivalence. Of course, both the significance level and the resulting power has to be replaced by its complement with respect to 1. The power of the nonrandomized version of (11.14) cannot be obtained in exactly the same way since before inverting the corresponding equivalence test, the boundaries m,n Cν; α (s; 1 , 2 ), ν = 1, 2, must be included in rather than excluded from its critical interval. In order to accommodate this slight modification, a separate program named bi2rlv1 is made available in the WKTSHEQ2 Source Code Package. The appropriate input to the latter is the target value of α itself rather than α , and the power of both versions of the test for relevant differences is output directly rather than as the complement of the correct value. Using this tool for computing the power which the test for relevant deviations of the odds ratio from 1 provides under the specifications made in the above example with the observed proportions .0841 and .2294 as the actual values of p1 and p2 , yields POW = .9352 and POWNR = .9194 for the exact and the nonrandomized version of (11.14), respectively. Adopting the computational tools made available in Chapter 6 for sample size determination with the exact Fisher type test for equivalence, for the case of relevant differences entails more far reaching modifications as compared with the computation of power. Actually, applying the algorithm behind the program bi2aeq2 with a (p1 , p2 ) falling outside the indifference zone specified by the null hypothesis of (11.15) would not work since, in the equivalence framework, the rejection probability of the test under such a parameter configuration has the meaning of a type-I error risk and is thus decreasing rather than increasing in the sample size(s). The result of adapting the algorithm with a view to this basic fact is the source code to be found in the WKTSHEQ2

Solutions to some special problems

367

Source Code Package under the base filename bi2rlv2. Table 11.5 shows a selection of results obtained by means of this procedure for the exact Fisher type test for relevant differences at level α = .05, relevance limits 1 = 2/3, 2 = 3/2, and target power 80%. All parameter points (p1 , p2 ) it covers lie on the same level curve defined by (p1 , p2 ) = 7/3 in the parameter subspace corresponding to K∗ . Table 11.5 Sample sizes required in the exact Fisher type test for relevant differences at level α = .05 to maintain power 80% against selected alternatives p1 = p2 = p∗ on the contour p1 /(1 − p1 ) = ∗ p2 /(1 − p2 ) with ∗ = 7/3 for λ ≡ m/n ∈ {.50, 1.0, 2.0} and relevance limits 1 = 2/3, 2 = 3/2. [For the motivation behind this choice and the specification of ∗ see § 1.7.] p1

p2

λ

.205882 " " .437500 " " .700000 " " .875000 " " .9296875 " "

.10 " " .25 " " .50 " " .75 " " .85 " "

0.5 1.0 2.0 0.5 1.0 2.0 0.5 1.0 2.0 0.5 1.0 2.0 0.5 1.0 2.0

m

n

373 746 525 525 828 414 214 428 292 292 448 224 208 416 273 273 404 202 348 696 443 443 634 317 556 1112 701 701 990 495

N =m+n 1119 1050 1242 642 584 672 624 546 604 1044 886 951 1668 1402 1395

As can be expected by intuition, the exact Fisher test for relevant differences shares with its equivalence counterpart the property that the required sample sizes are the larger the extremer the baseline responder rate p2 happens to be. Furthermore, the results shown in the above table confirm once more the general rule that, in terms of sample sizes, unbalanced designs are less cost-effective than sampling schemes with 1:1 allocation of the observational units to the treatment groups under comparison. Finally, it is worth mentioning that the technique used in § 6.6.5 to construct an improved nonrandomized version of the optimal conditional test works in

368

TESTS FOR RELEVANT DIFFERENCES

the present context as well. The steps to be taken for adapting the algorithm required for determining maximally increased nominal significance levels for the nonrandomized counterpart of the UMPU level-α test for the problem (11.15) are largely self-explanatory so that no further details are given here.

Appendix A Basic theoretical results

A.1

UMP tests for equivalence problems in STP3 families

A.1.1 Definition. Let (pθ (·))θ∈Θ be a family of real-valued functions with common domain X . Suppose that both X and the parameter space Θ is some simply ordered set. Moreover, for arbitrary n = 1, 2, . . ., let X (n) and Θ(n) denote the set of all increasingly ordered n-tuples of pairwise different points in X and Θ, respectively, and define ⎞ ⎛ pθ1 (x1 ) . . . pθ1 (xn )   x , . . . , xn ⎟ ⎜ .. .. .. Δn 1 ≡ det ⎝ (A.1) ⎠ . . . θ1 , . . . , θn pθn (x1 ) . . . pθn (xn ) for arbitrary (x1 , . . . , xn ) ∈ X (n) , (θ1 , . . . , θn ) ∈ Θ(n) . Then, the family (pθ (·))θ∈Θ is said to be strictly totally positive of order r ≥ 1 (abbreviated to STPr ) if for each n = 1, . . . , r we have that   x1 , . . . , xn Δn > 0 ∀ ((x1 , . . . , xn ), (θ1 , . . . , θn )) ∈ X (n) × Θ(n) . (A.2) θ1 , . . . , θn If (A.2) holds true for every n ∈ IN, the family (pθ (·))θ∈Θ is called strictly totally positive of order ∞ (STP∞ ).

Important special STP∞ families of probability densities on IR A.1.2 Lemma. Let X be a Borelian set in IR1 , μ(·) a σ-finite measure on X , and (Pθ )θ∈Θ a family of probability distributions on X such that Θ is a nondegenerate interval on the real line and for each θ ∈ Θ, a μ-density of Pθ is given by (A.3) pθ (x) = c(θ) exp{θx} h(x) , x ∈ X , with c(·) : Θ → IR+ and h as a Borel-measurable transformation from X to IR+ [↔ one-parameter exponential family in x and θ]. Then, (pθ (·))θ∈Θ is STP∞ .

369

370

BASIC THEORETICAL RESULTS

Proof. → Karlin (1968, p. 19). A.1.3 Lemma. For arbitrary ν ∈ IN and θ ∈ IR let pθ ( · ; ν) be the density (with respect to Lebesgue measure) of a noncentral t-distribution with ν degrees of freedom and noncentrality parameter θ. Then it follows that (pθ ( · ; ν))θ∈IR is STP∞ . Proof. → Karlin (1968, § 4(ii)).

A.1.4 Lemma. For any (ν1 , ν2 ) ∈ IN2 , let V denote a random variable whose distribution is central F with ν1 and ν2 degrees of freedom. Furthermore, let h (·) be the density function of  · V where  denotes any positive real number. Then, the family (h (·))>0 (occasionally called family of stretched F -densities with ν1 , ν2 degrees of freedom – see, e.g., Witting, 1985, p. 217) is STP∞ . Proof. Let (U1 , U2 ) be an independent pair of random variables such that ν1 U1 and ν2 U2 has a central χ2 -distribution with ν1 and ν2 degrees of freedom, respectively. Moreover, define for any (a, b) ∈ IR2+ g˜ν1 (a; b) ≡ b gν1 (ab) , g˜ν2 (a; b) ≡ b gν2 (ab) with gν1 (·) and gν2 (·) as the density function of U1 and U2 , respectively. Clearly, for any b > 0, the density function of b−1 · U1 and b−1 · U2 is then given by g˜ν1 ( · ; b)and g˜ν2 ( · ; b) respectively. d

By definition of V , we have hat  V = U1 /(−1 · U2 ). Hence, applying the well-known formula for the density of the product of two independent random variables of the absolutely continuous type (see, e.g., Kingman and Taylor, 1973, p. 299, Ex. 11.5.3), we can write for any z ∈ IR+ :  ∞  ∞ h (z) = y gν1 (yz) g˜ν2 (y; ) dy = g˜ν1 (z; y) g˜ν2 (y; ) dy . (A.4) 0

0

As shown by Witting (1985, Theorem 2.33 a) the families (˜ gν1 ( · ; y))y>0 and (˜ gν2 ( · ; ))>0 both exhibit the structure assumed in A.1.2 and are thus STP∞ . Hence, we can apply the composition theorem for pairs of strictly (totally) positive families (Karlin, 1968, p. 17) with X = Z = Y = (0, ∞), r = s = 1, K(z, y) = g˜ν1 (z; y), L(y, ) = g˜ν2 (y; ), σ = λ(0,∞) B yielding the proposed result.

UMP tests for equivalence problems in STP3 families

371

Existence and basic properties of UMP tests for equivalence in STP3 families of distributions A.1.5 Theorem. Let α be an arbitrary real number in (0 , 1), X a nondegenerate interval in IR, BX the Borelian σ-algebra on X , μ a σ-finite measure on BX whose support contains at least two different points, and (pθ (·))θ∈Θ an STP3 family of μ-densities on X with Θ exhibiting the form of some nondegenerate interval on the real line as well. Assume further that the function (x, θ) → pθ (x) is continuous in both of its arguments, and that some fixed pair (θ1 , θ2 ) of points in Θ with θ1 < θ2 is given. Then, the following statements hold true: (i) For the problem H : θ ∈ Θ \ (θ1 , θ2 ) versus K : θ ∈ (θ1 , θ2 ) there exists a UMP level-α test φ : X → [0, 1] given by ⎧ x ∈ (C1 , C2 ) ⎨1 (A.5) φ(x) = γi for x = Ci , i = 1, 2 , ⎩ 0 x ∈ X \ [C1 , C2 ] where

Ci ∈ X , i = 1, 2 , C1 ≤ C2 and  Eθi φ(X) ≡ φ pθi dμ = α for i = 1, 2 .

(A.6)

(ii) This φ minimizes the rejection probability Eθ φ(X) uniformly in θ ∈ Θ \ [θ1 , θ2 ] among all tests which satisfy (A.6). (iii) If the cardinality of the set which the dominating measure μ is concentrated upon, is greater than 2, then there exists a point θ◦ ∈ (θ1 , θ2 ) such that the power function θ → Eθ φ(X) of the UMP test is monotonically in- and decreasing on (−∞, θ◦ ] ∩ Θ and [θ◦ , ∞) ∩ Θ, respectively. Proof. A proof a weaker version of the theorem restricted to the special case that the underlying family of distributions exhibits the form (A.3), can be found in Lehmann and Romano (2005, Ch. 3.7). Proving the same result in full generality as repeatedly used in the present book, is left to the reader of Lehmann’s book as an exercise worked out by Kallenberg (1984, pp. 54–58).

372

BASIC THEORETICAL RESULTS

Simplified computation of the UMP test for equivalence under symmetry restrictions A.1.6 Lemma. Under the assumptions and notational conventions of Theorem A.1.5, let the limits of the equivalence range for θ be chosen as θ1 = −ε, θ2 = ε, with arbitrarily fixed ε > 0. Suppose further that under θ = ε the random variable which the hypotheses H and K refer to, has the same distribution as −X under θ = −ε. Then, a UMP level-α test for H : θ ∈ Θ \ (−ε, ε) versus K : θ ∈ (−ε, ε) is given by ⎧ |x| < C ⎨1 φ(x) = γ if |x| = C , (A.7) ⎩ 0 |x| > C where

and



  C = max x ∈ [0, ∞)  Pε [|X| < x] ≤ α ⎧ α − Pε [|X| < C] ⎪ ⎨ Pε [|X| = C] γ= ⎪ ⎩ 0

(A.8)

Pε [|X| = C] > 0 .

for

(A.9)

Pε [|X| = C] = 0

Proof. In view of (A.7) and (A.6), it suffices to show that P−ε [|X| < C] + γ P−ε [|X| = C] = α = Pε [|X| < C] + γ Pε [|X| = C] . The validity of the second of the above equalities is obvious since (A.8) and (A.9) simply give the solution to the equation Pε [|X| < x] + q Pε [|X| = x] = α, 0 ≤ x < ∞ , 0 ≤ q < 1. Furthermore, the assumption of the equality of the distribution of −X with respect to P−ε and that of X with respect to Pε trivially implies that we have P−ε [|X| ∈ B+ ] = Pε [|X| ∈ B+ ] ∀ B+ ∈ B[0,∞) . Putting B+ = [ 0, C) and B+ = {C}, respectively, this yields P−ε [|X| < C] + γ P−ε [|X| = C] = Pε [|X| < C] + γ Pε [|X| = C] , as required.

A.1.7 Corollary. Suppose the assumptions of Theorem A.1.5 are satisfied, and there exist isotonic continuous mappings T : X → IR, h : Θ → IR such that Z ≡ h(Θ) is an interval symmetric about zero and the distribution of −T (X) under h−1 (−ζ) is the same as that of T (X) under h−1 (ζ) for each ζ ∈ Z . Furthermore, let θ1 = h−1 (−ζ◦ ), θ2 = h−1 (ζ◦ ) for some ζ◦ ∈ Z ∩ IR+ , and define

UMP tests for equivalence problems in STP3 families 

  t∗ = max t ∈ [0, ∞)  Pθ2 [|T (X)| < t] ≤ α ,

373 (A.10)

C1 = T −1 (−t∗ ) , C2 = T −1 (t∗ ) , ⎧ α − Pθ2 [|T (X)| < t∗ ] ⎪ Pθ2 [|T (X)| = t∗ ] > 0 ⎨ Pθ2 [|T (X)| = t∗ ] ∗ γ = , for ⎪ ⎩ ∗ 0 Pθ2 [|T (X)| = t ] = 0 ⎧ C 1 < x < C2 ⎨ 1 φ(x)= . γ ∗ if x ∈ {C1 , C2 } ⎩ 0 x ∈ X \ [C1 , C2 ]

(A.11)

(A.12)

(A.13)

Then, φ is a UMP level-α test for H : θ ∈ Θ \ (θ1 , θ2 ) versus K : θ ∈ (θ1 , θ2 ). If we consider the rejection probability φ as a function of ζ = h(θ), the latter is symmetric about ζ = 0 and increases (decreases) on the left (right) of zero. Proof. Continuity and isotony of the mapping h and injectivity of the transformation T imply that the problem H versus K originally put forward is the same as that of testing H ∗ : ζ ∈ Z \ (−ζ◦ , ζ◦ ) versus K ∗ : ζ ∈ (−ζ◦ , ζ◦ ) in the family of distributions of T (X). T (X) For brevity, let us denote Ph−1 (ζ) , i.e., the distribution of T (X) under θ = h−1 (ζ), by Pζ∗ , and the image T (μ) of the basic measure μ under the transformation T by μ∗ . Then, it is readily verified by means of the transformation theorem for μ-integrals (see, e.g., Halmos, 1974, p. 163) that a density of Pζ∗ with respect to μ∗ is given by p∗ζ = ph−1 (ζ) ◦ T −1 . Since, by assumption, both T and h are continuous and isotonic, it is clear that (p∗ζ )ζ∈Z shares with (pθ )θ∈Θ the property of being an STP3 family satisfying the continuity conditions of Theorem A.1.5. Thus, all  are fulfilled for an application  prerequisites of the above Lemma A.1.6 with T (X), ζ, ζ◦ instead of (X, θ, ε). Accordingly, we can conclude that a UMP level-α test φ∗ for H ∗ versus K ∗ is obtained by computing the critical constants t∗ , γ ∗ from t∗

γ

∗



  = max t ∈ [0, ∞)  Pζ∗◦ [|T (X)| < t] ≤ α , ⎧ α − P ∗ [|T (X)| < t∗ ] ζ◦ ⎪ ⎨ Pζ∗◦ [|T (X)| = t∗ ] = ⎪ ⎩ 0

Pζ∗◦ [|T (X)| = t∗ ] > 0 for

(A.8∗ )

,

(A.9∗ )

Pζ∗◦ [|T (X)| = t∗ ] = 0

and defining φ∗ by

∗

φ (t)

⎧ ⎨ 1 = γ∗ ⎩ 0

|t| < t∗ for |t| = t∗ |t| > t∗

.

(A.7∗ )

374

BASIC THEORETICAL RESULTS

, we obviously have (A.8∗ ) ⇔ (A.10) and In view of Pζ∗◦ ≡ Ph−1 (ζ◦ ) = Pθ2 ∗ (A.9 ) ⇔ (A.12). Moreover, the assumptions made on the transformation T ensure that there holds φ(x) = φ∗ (T (x)) ∀x ∈ X with φ and (C1 , C2 ) defined as in (A.13) and (A.11), respectively. Hence φ is obtained by representing the UMP level-α test φ∗ for H ∗ vs. K ∗ as a function of x rather than t which in view of the equivalence of the testing problems (H, K) and (H ∗ , K ∗ ) implies that φ is indeed UMP among all level-α tests for the former. Thus, it only remains to show that the function ζ → Eh−1 (ζ) φ(X) is symmetric about zero and exhibits the monotonicity properties  stated in the sec ond part of the corollary. By φ = φ∗ ◦ T , Pζ∗ = T Ph−1 (ζ) , we may write for arbitrary ζ ∈ Z :    ∗ Eh−1 (ζ) φ(X) ≡ φ dPh−1 (ζ) = φ ◦ T dPh−1 (ζ) = φ∗ dPζ∗ . (A.14) T (X)

T (X)

By construction, φ∗ is a UMP level-α test for H ∗ : ζ ∈ Z \ (−ζ◦ , ζ◦ ) versus K : ζ ∈ (−ζ◦ , ζ◦ ) with ζ as the parameter of a family of distributions which satisfies the conditions of Theorem A.1.5. By part (iii) of A.1.5, it thus follows ∗ that ∗ the∗ interval (−ζ◦ , ζ◦ ) contains some point ∗ζ◦ such that∗ the function ζ → φ dPζ in- and decreases strictly on Z∩(−∞, ζ◦ ] and Z∩[ζ◦ , ∞), respectively. From (A.7∗ ), it is obvious that we may write φ∗ (T (x)) = ψ ∗ (|T (x)|) ∀ x ∈ X with ψ ∗ ≡ I[0,C ∗ ) + γ ∗ I{C ∗ } . Hence, we can replace the middle of the equations (A.14) with 

Eh−1 (ζ) φ(X) = ψ ∗ d |T |(Ph−1 (ζ) ) . (A.15) ∗

    The assumption (−T ) Ph−1 (−ζ) = T Ph−1 (ζ) clearly implies that the measure with respect to which the integral on  the right-hand side of (A.15) has to be taken, coincides with |T | Ph−1 (−ζ) , so that we have Eh−1 (−ζ) φ(X) = Eh−1 (ζ) φ(X) ∀ ζ ∈ Z which establishes the asserted symmetry of the power function ζ → Eh−1 (ζ) φ(X). But a function which is symmetric about zero can only attain a strict global maximum, if the latter is attained at 0.

UMPU equivalence tests in multiparameter exponential families

A.2

375

UMPU equivalence tests in multiparameter exponential families

A.2.1 Definition. Let (X , BX ) be a measurable space, μ a σ-finite measure on (X , BX ), Θ ⊆ IR, Ω ⊆ IRk , k ∈ IN . Suppose further that for each (θ, ϑ) = (θ, ϑ1 , . . . , ϑk ) ∈ Θ × Ω, Pθ,ϑ is a probability distribution on X having a μ-density of the form k

  ϑj Sj (x) , x ∈ X , pθ,ϑ (x) = c(θ, ϑ) exp θ T (x) +

(A.16)

j=1

with c(θ, ϑ) ∈ IR+ ∀ (θ, ϑ) and T (·), S1 (·),. . . , Sk (·) as BX −measurable is called a (k + transformations from X to IR . Then, Pθ,ϑ (θ,ϑ)∈Θ×Ω 1)−parameter exponential family in (θ, ϑ1 , . . . , ϑk ) and (T, S1 , . . . , Sk ). A.2.2 Theorem. In the situation described by the above definition, let Θ and Ω be a nondegenerate interval in IR1 and IRk , respectively, and (θ1 , θ2 ) a pair of points in Θ satisfying θ1 < θ2 . Then, at any level α ∈(0, 1), there exists a uniformly most powerful unbiased (UMPU) test x → φ T (x), S(x) for   H : (θ, ϑ) ∈ Θ \ (θ1 , θ2 ) × Ω versus K : (θ, ϑ) ∈ (θ1 , θ2 ) × Ω . (A.17) Given the value s of the statistic S ≡ (S1 , . . . , Sk ) : X → IRk , φ( · , s) : T (X ) → [0, 1] is obtained by constructing by means of Theorem A.1.5 a UMP level-α test for the problem H s : θ ∈ Θ\(θ1 , θ2 ) versus Ks : θ ∈ (θ1 , θ2 ) con T (X)|s cerning the family Pθ of conditional distributions of T (X) given {S = s} .

θ∈Θ

Proof. See Lehmann and Romano (2005, Ch. 4.4).

376

A.3

BASIC THEORETIC RESULTS

A sufficient condition for the asymptotic validity of tests for equivalence

A.3.1 Definition. For each N ∈ IN, let X (N ) = (X1 , . . . , XN ) be a vector of p-dimensional (p ∈ IN) random variables on a common probability space (Ω, A, P ). Suppose the distribution of X (N ) is determined by a fixed number k ≥ 1 (independent of N ) of p-dimensional distribution functions F1 , . . . , Fk in the following way: X1 , . . . , Xn1 (N ) Xn1 (N )+1 , . . . , Xn2 (N ) .. .

∼ ∼ .. .

F1 F2 .. .

Xnk−1 (N )+1 , . . . , Xnk (N ) ∼ Fk , where n1 (N )+. . .+nk (N ) = N . Furthermore, let θ(·) denote some functional on the underlying space of cdf’s, H the class of all vectors F = (F1 , . . . , Fk ) (N ) of cdf’s satisfying the null hypotheses H : θ(F ) ∈ Θ \ (θ1 , θ2 ) , and PF the common distribution of (X1 , . . . , XN ) under F . A test φN : IRpN → [0, 1] is said to be asymptotically valid at level α ∈ (0, 1) for H if we have lim sup EF (φN ) ≤ α N →∞

∀ F ∈H

(A.18)



where (N )

EF (φN ) ≡

(N )

φN (x1 , . . . , xN ) dPF (x1 , . . . , xN ) .

A.3.2 Lemma. For any s > 0 denote   cθα1 ,θ2 (s) = Cα s (θ2 − θ1 )/2

(A.19)

(A.20)

with Cα (ψ) as the square root of the α-quantile of a χ2 -distribution with a single degree of freedom and noncentrality parameter ψ 2 > 0 . Then cθα1 ,θ2 (s) − s (θ2 − θ1 )/2 → uα

as s → ∞

(A.21)

provided that uα = Φ−1 (α). Proof. → Wellek (1996, p. 707). A.3.3 Lemma. For each natural number N let SN be a positive random

Asymptotically valid tests for equivalence

377 PN

variable on some probability space (Ω, A, PN ) such that SN → c as N → ∞ for some c ∈ (0, ∞), q(·) a function from (0, ∞) into IR converging to some y ∗ ∈ IR as s → ∞, and (aN ) a sequence of positive real numbers with aN → +∞ as N → ∞. Then, we can conclude that PN

q(aN SN ) → y ∗

as N → ∞ .

(A.22)

Proof. → Wellek (1996, p. 707).   A.3.4 Theorem. Let X (N ) N ∈IN be as specified in A.3.1 with F as the underlying class of k-tuples of distribution functions, and assume that for each N ∈ IN, a test statistic TN on the sample space of X (N ) is given such that we have √ N (TN − θ(F ) L → Z ∼ N (0, 1) as N → ∞ ∀ F ∈ F (A.23) σ(F ) for suitable functionals θ(·) : F → IR and σ(·) : F → IR+ . Suppose further, that (ˆ σN )N ∈IN is a consistent estimator of σ(F ) for each F ∈ F , and cθα1 ,θ2 (·) is as in (A.20). If we then define ⎧ √ √  N |TN − θ◦ |/ˆ σN < cθα1 ,θ2 N /ˆ σN  (N )  ⎨ 1 φN X = for √ (A.24) √  ⎩ 0 N |TN − θ◦ |/ˆ σN ≥ cθα1 ,θ2 N /ˆ σN with θ◦ ≡ (θ1 + θ2 )/2, we can be sure that (φN )N ∈IN is asymptotically valid at level α ∈ (0, 1) for H : θ(F ) ∈ Θ \ (θ1 , θ2 ) versus K : θ(F ) ∈ (θ1 , θ2 ) . Proof. First we consider an arbitrarily √ such that θ(F ) ≥ θ2 and √ fixed F ∈ H ˆN / N = τˆN . By definition of write for brevity θ(F ) = θ, σ(F )/ N = τN , σ φN (·), we have   (N ) (N ) τN < cθα1 ,θ2 (1/ˆ τN ) − EF (φN ) = PF (TN − θ◦ )/ˆ   (N ) PF (TN − θ◦ )/ˆ τN ≤ −cθα1 ,θ2 (1/ˆ τN )   (N ) ≤ PF (TN − θ)/ˆ τN + (θ − θ◦ )/ˆ τN − cθα1 ,θ2 (1/ˆ τN ) < 0   (N ) τN + (θ2 − θ1 )/2ˆ τN − cθα1 ,θ2 (1/ˆ τN ) < 0 (A.25) ≤ PF (TN − θ)/ˆ by

θ ≥ θ2 , θ◦ = (θ1 + θ2 )/2 .

By assumption (A.23) and consistency of (ˆ σN )N ∈IN for σ(F ) we can write:   (N ) τN < w → Φ(w) as N → ∞ ∀ w ∈ IR (A.26) PF (TN − θ)/ˆ

378

BASIC THEORETIC RESULTS

and N

1/2

P

(N )

F

τˆN → σ ∗ as N → ∞ for some σ ∗ ∈ (0, ∞) .

(A.27)

In view of (A.20) and (A.21) all the conditions of Lemma A.3.3 are satisfied −1  (N ) if we put PN = PF , SN = N 1/2 τˆN , c = 1/σ ∗ , q(s) = cθα1 ,θ2 (s) − s (θ2 − ∗ 1/2 θ1 )/2 , y = uα , and aN = N . Hence, it follows that P

(N )

F

(θ2 − θ1 )/2ˆ τN − cθα1 ,θ2 (1/ˆ τN ) → −uα as N → ∞ .

(A.28)

Combining (A.28) with (A.26) gives   (N ) PF (TN − θ)/ˆ τN + (θ2 − θ1 )/2ˆ τN − cθα1 ,θ2 (1/ˆ τN ) < w → Φ(w + uα ) as N → ∞ ∀ w ∈ IR .

(A.29)

Taking w = 0 in (A.29) shows that the expression on the right-hand side of (A.25) converges to α as N → ∞ which completes the θ ≥ θ2 half of the proof. (N ) Let us now choose F ∈ H such that θ = θ(F ) ≤ θ1 . Then EF (φN ) can be bounded above in the following way:   (N ) (N ) EF (φN ) ≤ 1 − PF (TN − θ◦ )/ˆ τN ≤ −cθα1 ,θ2 (1/ˆ τN )   (N ) = 1 − PF (TN − θ)/ˆ τN + (θ − θ◦ )/ˆ τN + cθα1 ,θ2 (1/ˆ τN ) ≤ 0   (N ) ≤ 1 − PF (TN − θ)/ˆ τN − (θ2 − θ1 )/2ˆ τN + cθα1 ,θ2 (1/ˆ τN ) ≤ 0 by

(A.30)

θ ≤ θ1 , θ◦ = (θ1 + θ2 )/2 .

Since relations (A.26) and (A.28) hold true for any F with 0 < θ(F ) < 1 we get in direct analogy to (A.29)   (N ) PF (TN − θ)/ˆ τN − (θ2 − θ1 )/2ˆ τN + cθα1 ,θ2 (1/ˆ τN ) ≤ w → Φ(w − uα ) as N → ∞ ∀ w ∈ IR .

(A.31)

Specifying w = 0 in this latter relation shows that the right-hand side of (A.30) converges to 1 − Φ(−uα ) as N → ∞ so that the θ ≤ θ1 part of the proof is complete as well.

Appendix B List of special computer programs

Program Name bi2aeq1

Objective Power of the exact Fisher type test for equivalence bi2aeq2 Sample sizes for the exact Fisher type test for equivalence bi2aeq3 Increased nominal significance level for the nonrandomized version of the exact Fisher type test for equivalence bi2by_ni_del Corrected nominal levels for the objective Bayesian test for one-sided equivalence of two binomial distributions with respect to the difference bi2by_ni_or Corrected nominal levels for the objective Bayesian test for one-sided equivalence of two binomial distributions with respect to the odds ratio bi2diffac Corrected nominal significance level for the asymptotic test for equivalence of two unrelated binomial proportions with respect to the difference of their population counterparts bi2dipow Exact rejection probability of the asymptotic test for equivalence of two unrelated binomial proportions with respect to the difference of their expectations at any nominal level under an arbitrary parameter configuration bi2rlv1 Power of the exact Fisher type test for relevant differences bi2rlv2 Sample sizes for the exact Fisher type test for relevant differences

†)

Language → p. † Fortran, (R) 191 Fortran, (R)

191

Fortran, (R)

193

SAS/IML

185

SAS/IML

180

SAS, R

196

SAS, R

196

Fortran, (R)

366

Fortran, (R)

367

(R) indicates that a shared object accessible within R is available.

379

380 Program Name bi1st

LIST OF SPECIAL COMPUTER PROGRAMS

Objective Critical constants and power against the alternative p = (p1 + p2 )/2 of the UMP test for equivalence of a single binomial proportion to some given reference value bi2st Critical constants for the exact Fisher type UMPU test for equivalence of two binomial distributions with respect to the odds ratio bi2ste1 Power of the exact Fisher type test for noninferiority bi2ste2 Sample sizes for the exact Fisher type test for noninferiority bi2ste3 Increased nominal significance level for the nonrandomized version of the exact Fisher type test for noninferiority bi2wld_ni_del Corrected nominal levels for the Waldtype (asymptotic) test for one-sided equivalence of two binomial distributions with respect to the difference cf_reh_exact Exact confidence bounds to the relative excess heterozygosity (REH) exhibited by a SNP genotype distribution cf_reh_midp Mid-p-value – based confidence bounds to the relative excess heterozygosity (REH) exhibited by a SNP genotype distribution exp1st Critical constants and power against the null alternative of the UMP test for equivalence of the hazard rate of a single exponential distribution to some given reference value fstretch Critical constants and power against σ 2 = τ 2 of the UMPI test for dispersion equivalence of two Gaussian distributions gofhwex Critical constants of the exact UMPU test for approximate compatibility of a SNP genotype distribution with HWE

Language SAS, R

→ p. 60

SAS, R

190

Fortran, (R)

175

Fortran, (R)

175

Fortran, (R)

177

SAS/IML

184

SAS/IML

306

SAS/IML

307

SAS, R

57

SAS, R

168

SAS, R

296

LIST OF SPECIAL COMPUTER PROGRAMS Program Name gofhwex_1s

Objective Critical constants of the exact UMPU test for absence of a substantial deficit of heterozygotes as compared with a HWEconform SNP genotype distribution [↔ noninferiority version of the test implemented by means of gofhwex] gofind_t Establishing approximate independence in a two-way contingency table: Test statistic and critical bound gofsimpt Establishing goodness of fit of an observed to a fully specified multinomial distribution: Test statistic and critical bound mawi Mann-Whitney test for equivalence of two continuous distributions of arbitrary shape: Test statistic and critical upper bound mcnasc_ni Corrected nominal levels for the asymptotic test for noninferiority in the McNemar setting mcnby_ni Analogue of mcnasc_ni for the objective Bayesian test for noninferiority in the McNemar setting mcnby_ni_pp Posterior probability of the alternative hypothesis of noninferiority in the McNemar setting, given a specific point in the sample space mcnemasc Corrected nominal significance level for the asymptotic test for equivalence of two paired binomial proportions with respect to the difference of their expectations (McNemar setting) mcnempow Exact rejection probability of the asymptotic test for equivalence of two paired binomial proportions with respect to the difference of their expectations (McNemar setting) mwtie_fr Analogue of mwtie_xy for settings with grouped data mwtie_xy Distribution-free two-sample equivalence test for tied data: Test statistic and critical upper bound

381 Language SAS, R

→ p. 301

SAS, R

274

SAS, R

268

SAS, R

128

SAS, R

86

SAS/IML

89

SAS/IML

89

SAS, R

83

SAS, R

84

SAS, R

155

SAS, R

154

382

LIST OF SPECIAL COMPUTER PROGRAMS

Program Name Objective po_pbibe Bayesian posterior probability of the alternative hypothesis of individual bioequivalence postmys Bayesian posterior probability of the alternative hypothesis ε1 < δ/σD < ε2 in the setting of the one-sample t-test pow_abe interval inclusion test for average bioequivalence: exact power against an arbitrary specific alternative powsign Nonconditional power of the UMPU sign test for equivalence and its nonrandomized counterpart against the alternative p+ = p− sgnrk Signed rank test for equivalence of an arbitrary continuous distribution of the intraindividual differences to a distribution satisfying q+ ≡ P [ Di + Dj > 0 ] = 1/2 : Test statistic and critical upper bound srktie_d Generalized signed rank test for equivalence for tied data: Test statistic and critical upper bound srktie_m Analogue of srktie_d for settings where the distribution of the Di is concentrated on a finite lattice tt1st Critical constants and power against the null alternative of the one-sample t-test for equivalence with an arbitrary, maybe nonsymmetric choice of the limits of the equivalence range for δ/σD tt2st Critical constants and power against the null alternative of the two-sample t-test for equivalence with an arbitrary, maybe nonsymmetric choice of the limits of the equivalence range for (ξ − η)/σ

Language → p. SAS, R 331 SAS, R

38

SAS, R

318

SAS, R

73

SAS, R

101

SAS, R

111

SAS, R

113

SAS, R

96

SAS, R

122

Appendix C Frequently used special symbols and abbreviations

BE cdf iff df λ2nc STPr φ(·) UMP UMPU UMPI IN IN0 IR IR+

Z

#A IA (x) 1(r,s) d2 (u, v) ≡ d

= × P

bioequivalence cumulative distribution function if and only if number of degrees of freedom noncentrality parameter of a χ2 or F -distribution strictly totally positive of order r, with r ∈ IN or r = ∞ critical function of a test uniformly most powerful uniformly most powerful unbiased uniformly most powerful invariant set of natural numbers " " nonnegative integers " " real numbers " " positive real numbers " " integers number of elements of an arbitrary finite set A indicator function of a given subset A ⊆ IR at x ∈ IR r × s matrix with all elements equal to 1. squared Euclidean distance between any two vectors u, v of the same dimension equal by definition equal in distribution Cartesian product of sets

→

convergence in probability

→ X∼D U(a, b)

convergence in law random variable X follows distribution D uniform distribution over the interval (a, b) ⊆ IR

B(n, p)

binomial distribution with parameters n ∈ IN0 and p ∈ [0 , 1] probability mass function of B(n, p) evaluated at k ∈ {0, 1, . . . , n}

L

b(k ; n, p)

383

384

SPECIAL SYMBOLS AND ABBREVIATIONS

M(n; θ1 , . . . , θk ) multinomial distribution of dimension k  ≥ 2 with paramk eters n ∈ IN0 and (θ1 , . . . , θk ) ∈ [0 , 1]k , j=1 θj = 1 . (x; ρ) probability mass function at x of the conditional distribuhm,n s tion of X ∼ B(m, p1 ) given X + Y = s, with Y ∼ B(n, p2 ) independent of X and ρ = p1 (1 − p2 )/(1 − p1 )p2 E(σ) exponential distribution with scale parameter σ ∈ IR+ Γ(x) gamma function (complete gamma integral) at x > 0 N (ξ, σ 2 ) ϕ(·) Φ(·) Φ−1 (·) u1−α χ2ν; γ Cα (ψ)

Cα; ε˜ χ2ν; γ (ψ 2 ) fνχ (·)

normal distribution with expectation ξ ∈ IR and variance σ 2 ∈ IR+ density function of N (0, 1) cdf of N (0, 1) quantile function of N (0, 1) Φ−1 (1 − α), for arbitrary α ∈ (0, 1) γ-quantile (0 < γ < 1) of a central χ2 -distribution with ν degrees of freedom square root of the α-quantile (0 < α < 1) of a noncentral χ2 -distribution with a single degree of freedom and λ2nc = ψ 2 (ψ ∈ IR+ ) = Cα (˜ ε), with (−˜ ε, ε˜) as the equivalence range for the expectation of a Gaussian distribution with unit variance γ-quantile (0 < γ < 1) of a noncentral χ2 -distribution with ν degrees of freedom and λ2nc = ψ 2 density of a χ-distribution with ν degrees of freedom

FνT (·) tν; γ

cdf of a central t-distribution with ν degrees of freedom γ-quantile (0 < γ < 1) of a central t-distribution with ν degrees of freedom

T Fν; ψ (·)

cdf of a noncentral t-distribution with ν degrees of freedom and noncentrality parameter ψ ∈ IR γ-quantile (0 < γ < 1) of a noncentral t-distribution with ν degrees of freedom and noncentrality parameter ψ ∈ IR cdf of a central F -distribution with ν1 , ν2 degrees of freedom γ-quantile of a noncentral F -distribution with ν1 , ν2 degrees of freedom and λ2nc = ψ 2 random variable following a noncentral F -distribution with ν1 , ν2 degrees of freedom and λ2nc = ψ 2

tν; γ (ψ) Fν1 ,ν2 Fν1 ,ν2 ; γ (ψ 2 ) Fν1 ,ν2 (ψ 2 ) p+ p0 p−

P [ Di > 0 ] , with Di as the i th intra-subject difference obtained from a sample of paired observations P [ Di = 0 ] , with Di as above P [ Di < 0 ] , " " "

SPECIAL SYMBOLS AND ABBREVIATIONS q+ q0 q− U+ U0 π+

385

P [ Di + Dj > 0 ] , with Di , Dj as the intra-subject difference for two different observational units in a paired-data setting P [ Di + Dj = 0 ] , with Di , Dj as above P [ Di + Dj < 0 ] , " " " "

π0

U -statistic estimator of q+ U -statistic estimator of q0 P [ Xi > Yj ] , with Xi and Yj as independent observations from two different univariate distributions under comparison P [ Xi = Yj ] , with Xi and Yj as above

W+ W0

U -statistic estimator of π+ U -statistic estimator of π0

||S1 − S2 || supt>0 |S1 (t) − S2 (t)| , with S1 (·), S2 (·) as two survivor functions of the continuous type λν (t) hazard function at t ≥ 0 of the νth sample (ν = 1, 2) ˆ IN (β) observed information evaluated at the maximum likelihood estimate of the regression coefficient β ∈ IR φT , φ R πk σS2 2 2 σeT , σeR σT2 T , σT2 R Xki Xi− μ− Ykj Yj− ν− λ(1) , λ(2)

“direct” effect of drug formulation T (↔ “Test”) and R (↔ “Reference”), respectively kth period effect in a 2 × 2 crossover trial (k = 1, 2) between-subject variance in a standard BE trial within-subject variance associated with drug formulation T and R, respectively total variance of the bioavailability of drug formulation T and R, respectively bioavailability measured in the ith subject (i = 1, . . . , m) of sequence group T /R during the kth period (k = 1, 2) of a standard bioequivalence trial X1i − X2i E(Xi− ) bioavailability measured in the jth subject (j = 1, . . . , n) of sequence group R/T during the kth period (k = 1, 2) of a standard bioequivalence trial Y1j − Y2j E(Yj− ) carryover effect of the treatment administered in the first period of a 2 × 2 crossover trial for the 1st (↔ T /R) and 2nd (↔ R/T ) sequence group

References

Abramowitz, M. and Stegun, I. (1965) Handbook of Mathematical Functions. New York: Dover Publications, Inc. Agresti, A. (2002) Categorical Data Analysis. Second Edition. Hoboken, NJ: John Wiley & Sons, Inc. Altman, D.G. and Bland, J.M. (1995) Absence of evidence is not evidence of absence. British Medical Journal 311: 485. Andersen, P.K., Borgan, Ø., Gill, R.D. and Keiding, N. (1993) Statistical Models Based on Counting Processes. Berlin: Springer-Verlag. Andersen, P.K. and Gill, R.D. (1982) Cox’s regression model for counting processes: A large sample study. Annals of Statistics 10: 1100–20. Anderson, S. and Hauck, W.W. (1983) A new procedure for testing equivalence in comparative bioavailability and other clinical trials. Communications in Statistics: Theory and Methods 12: 2663–92. Anderson, S. and Hauck, W.W. (1990) Consideration of individual bioequivalence. Journal of Pharmacokinetics and Biopharmacology 18: 259–73. Anderson, T.W. (1984) An Introduction to Multivariate Statistical Analysis. Second Edition. New York: John Wiley & Sons, Inc. Armitage, P. (1955) Tests for linear trends in proportions and frequencies. Biometrics 11: 375–86. Bailey, C.C., Gnekow, A., Wellek, S., Jones, M., Round, C., Brown, J., Philips, A. and Neidhardt, M.K. (1995) Prospective randomized trial of chemotherapy given before radiotherapy in childhood medulloblastoma. International Society of Paediatric Oncology SIOP and the German Society of Paediatric Oncology GPO: SIOP II. Medical Pediatric Oncology 25: 166–78. Barnard, G.A. (1947) Significance tests for 2x2 tables. Biometrika 34: 123–38. Berger, J.O. (1985) Statistical Decision Theory and Bayesian Analysis. Second Edition. New York: Springer-Verlag. Berger, R.L. (1982) Multiparameter hypothesis testing and acceptance sampling. Technometrics 24: 295–300.

387

388

REFERENCES

Berger, R.L. and Boos, D.D. (1994) P values maximised over a confidence set for the nuisance parameter. Journal of the American Statistical Association 89: 1012–16. Berger, R.L. and Hsu, J.C. (1996) Bioequivalence trials, intersection-union tests and equivalence confidence sets. Statistical Science 11: 283–319. Berger, R.L. and Sidik, K. (2003) Exact unconditional tests for a 2×2 matched pairs design. Statistical Methods in Medical Research 12: 91–108. Bickel, P.J. and Doksum, K.A. (2001) Mathematical Statistics: Basic Ideas and Selected Topics. Vol. I. Second Edition. Upper Saddle River, NJ: Prentice Hall. Bishop, Y.M.M., Fienberg, S.E. and Holland, P.W. (1975) Discrete Multivariate Analysis: Theory and Practice. Cambridge, MA: MIT Press. Blackwelder, W.C. (1982) “Proving the null hypothesis” in clinical trials. Controlled Clinical Trials 3: 345–53. Blackwelder, W.C. (1993) Sample size and power for prospective analysis of relative risk. Statistics in Medicine 12: 691–8. Bondy, W.H. (1969) A test of an experimental hypothesis of negligible difference between means. The American Statistician 23: 28–30. Boschloo, R.D. (1970) Raised conditional level of significance for the 2 × 2table when testing the equality of two probabilities. Statistica Neerlandica 24: 1–35. Box, G.P.E. and Tiao, G.C. (1973) Bayesian Inference in Statistical Analysis. Reading, MA: Addison-Wesley. Brown, B.W. (1980) The crossover experiment for clinical trials. Biometrics 36: 69–79. Brown, L.D., Hwang, J.T.G. and Munk, A. (1997) An unbiased test for the bioequivalence problem. Annals of Statistics 25: 2345–67. Casella, G. and Berger, R.L. (1987) Reconciling bayesian and frequentist evidence in one-sided testing problems. Journal of the American Statistical Association 82: 106–11. Chan, I.S.F. (1998) Exact tests of equivalence and efficacy with a non-zero lower bound for comparative studies. Statistics in Medicine 17: 1403–13. Chan, I.S.F. (2003) Proving non-inferiority or equivalence of two treatments with dichotomous endpoints using exact methods. Statistical Methods in Medical Research 12: 37–58. Chan, I.S.F. and Zhang, Z. (1999) Test-based exact confidence intervals for the difference of two binomial proportions. Biometrics 55: 1202–9.

REFERENCES

389

Chen, M.L. (1997) Individual bioequivalence – A regulatory update. Journal of Biopharmaceutical Statistics 7: 5–11. Chester, S.T. (1986) Review of equivalence and hypothesis testing. In ASA Proceedings of the Biopharmaceutical Section. Washington: American Statistical Association, 177–82. Chinchilli, V.M. and Elswick Jr., R.K. (1997) The multivariate assessment of bioequivalence. Journal of Biopharmaceutical Statistics 7: 113–23. Chow, S.C. and Liu, J.P. (2008) Design and Analysis of Bioavailability and Bioequivalence Studies, Third Edition. Boca Raton, FL: Chapman & Hall/CRC Press. Cox, D.R. (1958) Some problems connected with statistical inference. Annals of Mathematical Statistics 29: 357–72. Cox, D.R. and Hinkley, D.V. (1974) Theoretical Statistics. London: Chapman & Hall. D’Angelo, G., Potvin, D. and Turgeon, J. (2001) Carry-over effects in bioequivalence studies. Journal of Biopharmaceutical Statistics 11: 35–43. Dannenberg, O., Dette, H. and Munk, A. (1994) An extension of Welch’s approximate t-solution to comparative bioequivalence trials. Biometrika 81: 91–101. Davies, R. (1971) Rank tests for “Lehmann’s alternative.” Journal of the American Statistical Association 66: 879–83. Davis, P.J. and Rabinowitz, P. (1975) Methods of Numerical Integration. New York: Academic Press. Devlin, B. and Roeder, K. (1999) Genomic control for association studies. Biometrics 55: 997–1004. Dunnett, C.W. and Gent, M. (1977) Significance testing to establish equivalence between treatments, with special reference to data in the form of 2 × 2 tables. Biometrics 33: 593–602. EMEA (1998) CPMP/ICH/363/96. http://www.emea.europa.eu/pdfs/human/ich/03636en.pdf9. EMEA (2005) CPMP/EWP/2158/99. http://www.emea.europa.eu/pdfs/human/ewp/21589en.pdf9. Farrington, C.P. and Manning, G. (1990) Test statistics and sample size formulae for comparative binomial trials with null hypothesis of non-zero risk difference or non-unit relative risk. Statistics in Medicine 9: 1447– 54. Feller, W. (1968) An Introduction to Probability Theory and Its Applications.

390

REFERENCES Volume I. Third Edition. New York: John Wiley & Sons, Inc.

Feller, W. (1971) An Introduction to Probability Theory and Its Applications. Volume II. Second Edition. New York: John Wiley & Sons, Inc. Firle, E.A. (1998) Asymptotically Distribution-Free Tests for Equivalence for Paired Observations Allowing for Ties. Diploma thesis, Department of Mathematics, University of Mainz, Mainz, Germany. Fisher, R.A. (1934) Statistical Methods for Research Workers. 5th Edition. Edinburgh: Oliver & Boyd. Fl¨ uhler, H., Grieve, A.P., Mandallaz, D., Mau, J. and Moser, H. (1983) Bayesian approach to bioequivalence assessment: An example. Journal of Pharmacological Sciences 72: 1178–81. Food and Drug Administration FDA (1997) Bioequivalence Studies-II. Gender Studies with non-replicate Designs. Center for Drug Evaluation and Research at http://www.fda.gov/bioequivdata. Food and Drug Administration FDA (2001) Guidance for Industry: Statistical Approaches to Establishing Bioequivalence. Rockville, MD: Center for Drug Evaluation and Research CDER. Food and Drug Administration FDA (2008) Orange Book, Version 12/2008. http://www.fda.gov/cder/orange/obreadme.htm. Freeman, P. (1989) The performance of the two-stage analysis of two treatment, two period crossover trials. Statistics in Medicine 8: 1421–32. Freitag, G. (2005) Methods for assessing noninferiority with censored data. Biometrical Journal 47: 88–98. Freitag, G., Lange, S. and Munk, A. (2006) Non-parametric assessment of non-inferiority with censored data. Statistics in Medicine 25: 1201–17. Frick, H. (1987) On level and power of Anderson and Hauck’s procedure for testing equivalence in comparative bioavailability trials. Communications in Statistics, Theory and Methods 16: 2771–8. Giani, G. and Finner, H. (1991) Some general results on least favorable parameter configurations with special reference to equivalence testing and the range statistic. Journal of Statistical Planning and Inference 28: 33–47. Giani, G., Straßburger, K. and Seefeldt, F. (2005) SeParATe, Version 3.0B. Diabetesforschungsinstitut an der Heinrich-Heine–Universit¨ at, Abt. Biometrie & Epidemiologie, D¨ usseldorf, Germany. Goddard, K.A.B., Ziegler, A. and Wellek, S. (2009) Adapting the logical basis of tests for Hardy-Weinberg equilibrium to the real needs of association studies in human and medical genetics. Genetic Epidemiology 33: 569–80.

REFERENCES

391

Grizzle, J.E. (1965) The two-period change-over design and its use in clinical trials. Biometrics 21: 467–80. Guilbaud, O. (1993) Exact inference about the within-subject variability in 2 × 2 crossover trials. Journal of the American Statistical Association 88: 939–46. ˇ ak, Z. and Sen, P.K. (1999) Theory of Rank Tests. San Diego: H´ajek, J., Sid´ Academic Press. H´ajek, P. and Havr´ anek, T. (1978) Mechanizing Hypothesis Formation. Berlin: Springer-Verlag. Halmos, P.R. (1974) Measure Theory. Berlin: Springer-Verlag. Harkness, W.L. (1965) Properties of the extended hypergeometric distribution. Annals of Mathematical Statistics 36: 938–45. Hauck, W.W. and Anderson, S. (1996) Comment on “Bioequivalence trials, intersection-union tests and equivalence confidence sets.” Statistical Science 11: 303. Hauck, W.W., Hauschke, D., Diletti, E., Bois, F.Y., Steinijans, V.W. and Anderson, S. (1997) Choice of Student’s t- or Wilcoxon-based confidence intervals for assessment of average bioequivalence. Journal of Biopharmaceutical Statistics 7: 179–89. Hauschke, D., Steinijans, V.W. and Diletti, E. (1990) A distribution-free procedure for the statistical analysis of bioequivalence studies. International Journal of Clinical Pharmacology and Therapeutics 28: 72–8. Hauschke, D., Steinijans, V.W. and Hothorn, L.A. (1996) A note on Welch’s approximate t-solution to bioequivalence assessment. Biometrika 83: 236–7. Hauschke, D., Steinijans, V.W. and Pigeot, I. (2007) Bioequivalence Studies in Drug Development: Methods and Applications. Chichester: John Wiley & Sons, Inc. Hodges, J.L. and Lehmann, E.L. (1954) Testing the approximate validity of statistical hypotheses. Journal of the Royal Statistical Society, Series B 16: 262–8. Hoffelder, T. (2006) Two-Sample Equivalence Tests for Multivariate Normal Data. Diploma thesis, Department of Mathematics, University of Mainz, Mainz, Germany. Holder, D.J. and Hsuan, F. (1993) Moment-based criteria for determining bioequivalence. Biometrika 80: 835–46. Holzgreve, H., Distler, A., Michaelis, J., Philipp, T. and Wellek, S. (1989)

392

REFERENCES Verapamil versus hydrochlorothiazide for the treatment of hypertension. Results of a long-term double-blind comparative trial. British Medical Journal 299: 881–6.

Hotelling, H. (1931) The generalization of Student’s ratio. Annals of Mathematical Statistics 2: 360–78. Hsu, J.C., Hwang, J.T.G., Liu, H.K. and Ruberg, S.J. (1994) Confidence intervals associated with tests for bioequivalence. Biometrika 81: 103– 14. Hsueh, H.M., Liu, J.P. and Chen, J.J. (2001) Unconditional exact tests for equivalence or noninferiority for paired binary endpoints. Biometrics 57: 478–83. Janssen, A. (1999) Testing nonparametric statistical functionals with application to rank tests. Journal of Statistical Planning and Inference 81: 71–93. Janssen, A. (2000) Nonparametric bioequivalence tests for statistical functionals and their efficient power functions. Statistics & Decisions 18: 49–78. Janssen, A. and Wellek, S. (2008) Exact linear rank tests for two-sample equivalence problems with continuous data. In Technical Report. University of Duesseldorf, D¨ usseldorf, Germany. Jeffreys, H. (1961) Theory of Probability. Third Edition. Oxford: Clarendon Press. Johnson, A.D. and O’Donnell, C.J. (2009) An open access database of genomewide association results. BMC Medical Genetics 10: 6. Johnson, N.L., Kotz, S. and Balakrishnan, N. (1994) Continuous Univariate Distributions. Volume 1. Second Edition. New York: John Wiley & Sons, Inc. Johnson, N.L., Kotz, S. and Balakrishnan, N. (1995) Continuous Univariate Distributions. Volume 2. Second Edition. New York: John Wiley & Sons, Inc. Johnson, N.L., Kotz, S. and Balakrishnan, N. (1997) Discrete Multivariate Distributions. New York: John Wiley & Sons, Inc. Johnson, N.L., Kotz, S. and Kemp, A.W. (1992) Univariate Discrete Distributions. Second Edition. New York: John Wiley & Sons, Inc. Jones, B. and Kenward, M.G. (1989) Design and Analysis of Cross-Over Trials. London: Chapman & Hall. Jones, B. and Kenward, M.G. (2003) Design and Analysis of Cross-Over Tri-

REFERENCES

393

als. Second Edition. Boca Raton, FL: Chapman & Hall/CRC Press. Kalbfleisch, J.D. and Prentice, R.L. (2002) The Statistical Analysis of Failure Time Data, 2nd Ed. Hoboken, NJ: John Wiley & Sons, Inc. Kallenberg, W.C.M. (1984) Testing Statistical Hypotheses: Worked Solutions. Amsterdam: Centrum voor Wiskunde en Informatica. Karlin, S. (1956) Decision theory for P´ olya type distributions. Case of two actions, I. In J. Neyman, ed., Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, CA: California University Press., 115–28. Karlin, S. (1957a) P´ olya type distributions, II. Annals of Mathematical Statistics 28: 281–308. Karlin, S. (1957b) P´ olya type distributions, III: Admissibility for multi-action problems. Annals of Mathematical Statistics 28: 839–60. Karlin, S. (1968) Total Positivity. Volume 1. Stanford, CA: Stanford University Press. Kingman, J.F.C. and Taylor, S.J. (1973) Introduction to Measure and Probability. Cambridge: Cambridge University Press. Kotz, S., Balakrishnan, N. and Johnson, N.L. (2000) Continuous Multivariate Distributions, Vol. 1: Models and Applications. New York: John Wiley & Sons, Inc. Lachenbruch, P.A. and Lynch, C.J. (1998) Assessing screening tests: extensions of McNemars’s test. Statistics in Medicine 17: 2207–17. Lambert, J.C., Araria-Goumidi, L., Myllykangas, L., Ellis, C., Wang, J.C., Bullido, M.J., Harris, J.M., Artiga, M.J., Hernandez, D., Kwon, J.M., Frigard, B., Petersen, R.C., Cumming, A.M., Pasquier, F., Sastre, I., Tienari, P.J., Frank, A., Sulkava, R., Morris, J.C., St Clair, D., Mann, D.M., Wavrant-DeVri`eze, F., Ezquerra-Trabalon, M., Amouyel, P., Hardy, J., Haltia, M., Valdivieso, F., Goate, A.M., P´erez-Tur, J., Lendon, C.L. and Chartier-Harlin, M.C. (2002) Contribution of APOE promoter polymorphisms to Alzheimer’s disease risk. Neurology 59: 59– 66. Lee, A.J. (1990) U-Statistics. Theory and Practice. New York: Marcel Dekker Inc. Lehmann, E.L. (1963) Nonparametric confidence intervals for a shift parameter. Annals of Mathematical Statistics 34: 1507–12. Lehmann, E.L. (1975) Nonparametrics: Statistical Methods Based on Ranks. San Francisco: Holden-Day, Inc.

394

REFERENCES

Lehmann, E.L. (1986) Testing Statistical Hypotheses. Second Edition. New York: John Wiley & Sons, Inc. Lehmann, E.L. and Romano, J.P. (2005) Testing Statistical Hypotheses. Third Edition. New York: Springer. Lindley, D.V. (1970) Introduction to Probability and Statistics from a Bayesian Viewpoint. Part 2: Inference. Cambridge: Cambridge University Press. Little, R.J.A. (1989) Testing the equality of two independent binomial proportions. The American Statistician 43: 283–8. Liu, J.P. and Chow, S.C. (1992) On the assessment of variability in bioavailability/bioequivalence studies. Communications in Statistics: Theory and Methods 21: 2591–607. Lloyd, C.J. (2008a) A new exact and more powerful unconditional test of no treatment effect from binary matched pairs. Biometrics 64: 716–23. Lloyd, C.J. (2008b) Exact P-values for discrete models obtained by estimation and maximisation. Australian & New Zealand Journal of Statistics 50: 329–45. Lloyd, C.J. and Moldovan, M.V. (2008) A more powerful exact test of noninferiority from binary matched pairs data. Statistics in Medicine 27: 3540–9. Lu, Y. and Bean, J.A. (1995) On the sample size for one-sided equivalence of sensitivities based upon McNemar’s test. Statistics in Medicine 14: 1831–9. MacLeod, M.J., Dahiyat, M.T., Cumming, A.M., Meiklejohn, D., Shaw, D. and St Clair, D. (1999) No association between Glu/Asp polymorphism of NOS3 gene and ischemic stroke. Neurology 53: 418–20. Makuch, R.W. and Simon, R. (1978) Sample size requirements for evaluating a conservative therapy. Cancer Treatment Reports 62: 1037–40. Mandallaz, D. and Mau, J. (1981) Comparison of different methods for decision-making in bioequivalence assessment. Biometrics 37: 213–22. Mart´ın, A.A. (1998) Fisher’s exact and Barnard’s tests. In S. Kotz, C. Reid and D.L. Banks, eds., Encyclopedia of Statistical Sciences, vol. 2. New York: John Wiley & Sons, Inc., 250–8. Mau, J. (1987) On Cox’s confidence distribution. In R. Viertl, ed., Probability and Bayesian Statistics. New York: Plenum., 346–56. Mau, J. (1988) A statistical assessment of clinical equivalence. Statistics in Medicine 7: 1267–77. McDonald, L.L., Davis, B.M., Bauer, H.R. and Laby, B. (1981) Algorithm

REFERENCES

395

AS161: critical regions of an unconditional non-randomized test of homogeneity in 2 × 2 contingency tables. Applied Statistics 30: 182–9. McDonald, L.L., Davis, B.M. and Milliken, G.A. (1977) A nonrandomized unconditional test for comparing two proportions in a 2 × 2 contingency table. Technometrics 19: 145–50. McNemar, I. (1947) Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12: 153–7. Mehring, G. (1993) On optimal tests for general interval hypotheses. Communications in Statistics, Theory and Methods 22: 1257–97. Mehta, C.R., Patel, N.R. and Tsiatis, A.A. (1984) Exact significance testing to establish treatment equivalence with ordered categorical data. Biometrics 40: 819–25. Meredith, M.P. and Heise, M.A. (1996) Comment on “Bioequivalence trials, intersection-union tests and equivalence confidence sets.” Statistical Science 11: 304–6. Metzler, C.M. (1974) Bioavailability — a problem in equivalence. Biometrics 30: 309–17. Miettinen, O. and Nurminen, M. (1985) Comparative analysis of two rates. Statistics in Medicine 4: 213–26. Miller, K.W., Williams, D.S., Carter, S.B., Jones, M.B. and Mishell, D.R. (1990) The effect of olestra on systemic levels of oral contraceptives. Clinical Pharmacology and Therapeutics 48: 34–40. Morgan, W.A. (1939) A test for the significance of the difference between the two variances in a sample from a normal bivariate population. Biometrika 31: 13–9. Morikawa, T., Yanagawa, T., Endou, A. and Yoshimura, I. (1996) Equivalence tests for pair-matched binary data. Bulletin of Informatics and Cybernetics 28: 31–45. Moses, L.E. (1953) Non-parametric methods. In H. Walker and J. Lev, eds., Statistical Inference. New York: Holt, Rinehart and Winston, 426–50. M¨ uller-Cohrs, J. (1988) Size and power of the Anderson-Hauck test and the double t-test in equivalence problems. In Proceedings of the SAS. Copenhagen: European Users Group Internations Conference., 30–8. Munk, A. (1993) An improvement on commonly used tests in bioequivalence assessment. Biometrics 49: 1225–30. Munk, A. (1996) Equivalence and interval testing for Lehmann’s alternative. Journal of the American Statistical Association 91: 1187–96.

396

REFERENCES

Nam, J. (1998) Power and sample size for stratified prospective studies using the score method for testing relative risk. Biometrics 54: 331–6. Nam, J. (2006) Non-inferiority of new procedure to standard procedure in stratified matched-pair design. Biometrical Journal 48: 966–77. Nam, J. and Blackwelder, W.C. (2002) Analysis of the ratio of marginal probabilities in a matched-pair setting. Statistics in Medicine 21: 689–99. Neyman, J. and Pearson, E.S. (1936) Contributions to the theory of testing statistical hypotheses I. Unbiased critical regions of type A and type A1. Statistical Research Memoirs 1: 1–37. Patterson, S. and Jones, B. (2005) Bioequivalence and Statistics in Clinical Pharmacology. Boca Raton, FL: Chapman & Hall/CRC Press. Pearson, E.S. (1947) The choice of statistical tests illustrated on their interpretation of data classified in a 2 × 2 table. Biometrika 34: 139–67. Perlman, M.D. and Wu, L. (1999) The emperor’s new tests. Statistical Science 14: 355–69. Pfeiffer, N., Grehn, F., Wellek, S. and Schwenn, O. (1994) Intraocular thymoxamine or acetylcholine for the reversal of mydriasis. German Journal of Ophthalmology 3: 422–6. Philipp, T., Anlauf, M., Distler, A., Holzgreve, H., Michaelis, J. and Wellek, S. (1997) Randomised, double blind multicentre comparison of hydrochlorothiazide, atenolol, nitrendipine, and enalapril in antihypertensive treatment: Results of the HANE study. British Medical Journal 315: 154–9. Pitman, E.J.G. (1939) A note on normal correlation. Biometrika 31: 9–12. Pratt, J.W. (1960) On interchanging limits and integrals. Annals of Mathematical Statistics 31: 74–7. Racine-Poon, A., Grieve, A.P., Fl¨ uhler, H. and Smith, A.F.M. (1987) A twostage procedure for bioequivalence studies. Biometrics 43: 847–56. Randles, R.H. and Wolfe, D.A. (1979) Introduction to The Theory of Nonparametric Statistics. New York: John Wiley & Sons, Inc. Rao, C.R. (1973) Linear Statistical Inference and Its Applications. Second Edition. New York: John Wiley & Sons, Inc. Reich, K., M¨ ossner, R., K¨ onig, I.R., Westphal, G., Ziegler, A. and Neumann, C. (2002) Promoter polymorphisms of the genes encoding tumor necrosis factor-α and interleukin-1β are associated with different subtypes of psoriasis characterized by early and late disease onset. The Journal of Investigative Dermatology 118: 155–63.

REFERENCES

397

Richard, F., Fromentin-David, I., Ricolfi, F., Ducimeti´ere, P., Di Menza, C., Amouyel, P. and Helbecque, N. (2001) The angiotensin I converting enzyme gene as a susceptibility factor for dementia. Neurology 56: 1593–5. Rodary, C., Com-Nougue, C. and Tournade, M.F. (1989) How to establish equivalence between treatments: A one-sided clinical trial in paediatric oncology. Statistics in Medicine 8: 593–8. Rodda, B.E. (1990) Bioavailability: Designs and analysis. In D.A. Berry, ed., Statistical Methodology in the Pharmaceutical Sciences. New York: Marcel Dekker Inc, 57–81. Rodda, B.E. and Davis, R.L. (1980) Determining the probability of an important difference in bioavailability. Clinical Pharmacology and Therapeutics 28: 247–52. Roebruck, P. and K¨ uhn, A. (1995) Comparison of tests and sample size formulae for proving therapeutic equivalence based on the difference of binomial probabilities. Statistics in Medicine 14: 1583–94. R¨ohmel, J. (2005) Problems with existing procedures to calculate exact unconditional p-values for non-inferiority/superiority and confidence intervals for two binomials and how to resolve them. Biometrical Journal 47: 37–47. R¨ohmel, J. and Mannsmann, U. (1999a) Unconditional non-asymptotic onesided tests for independent binomial proportions when the interest lies in showing non-inferiority and/or superiority. Biometrical Journal 41: 149–70. R¨ohmel, J. and Mannsmann, U. (1999b) Letter to the editor on “Exact tests of equivalence and efficacy with a non-zero lower bound for comparative studies” by S.F. Chan. Statistics in Medicine 18: 1734–5. Romano, J.P. (2005) Optimal testing of equivalence hypotheses. Annals of Statistics 33: 1036–47. Roy, S.N. (1953) On a heuristic method of test construction and its use in multivariate analysis. Annals of Mathematical Statistics 24: 220–38. Sasieni, P.D. (1997) From genotype to genes: Doubling the sample size. Biometrics 53: 1253–61. Savage, I.R. (1956) Contributions to the theory of rank order statistics - the two-sample case. Annals of Mathematical Statistics 27: 590–615. Scaglione, F. (1990) Comparison of the clinical and bacteriological efficacy of clarithromycin and erythromycin in the treatment of streptococcal pharyngitis. Current Medical Research and Opinion 12: 25–33. Schall, R. (1995) Assessment of individual and population bioequivalence us-

398

REFERENCES ing the probability that bioavailabilities are similar. Biometrics 51: 615– 26.

Schall, R. and Luus, H.G. (1993) On population and individual bioequivalence. Statistics in Medicine 12: 1109–24. Schoenfeld, D. (1981) The asymptotic properties of nonparametric tests comparing survival distributions. Biometrika 68: 316–9. Schuirmann, D.J. (1987) A comparison of the two one-sided tests procedure and the power approach for assessing equivalence of average bioavailability. Journal of Pharmacokinetics and Biopharmaceutics 15: 657–80. Schuirmann, D.J. (1996) Comment on “Bioequivalence trials, intersectionunion tests and equivalence confidence sets.” Statistical Science 11: 312– 3. Schumann, G., Rujescu, D., Szegedi, A., Singer, P., Wiemann, S., Wellek, S., Giegling, I., Klawe, C., Anghelescu, I., Heinz, A., Spanagel, R., Mann, K., Henn, F.A. and Dahmen, N. (2003) No association of alcohol dependence with a NMDA-receptor 2B gene variant. Molecular Psychiatry 8: 11–2. Searle, S.R. (1987) Linear Models for Unbalanced Data. New York: John Wiley & Sons, Inc. Selwyn, M.R., Dempster, A.P. and Hall, N.R. (1981) A Bayesian approach to bioequivalence for the 2 × 2 changeover design. Biometrics 37: 11–21. Selwyn, M.R. and Hall, N.R. (1984) On Bayesian methods for bioequivalence. Biometrics 40: 1103–8. Sheiner, L.B. (1992) Bioequivalence revisited. Statistics in Medicine 11: 1777– 88. Shorack, R. (1967) Tables of the distribution of the Mann-Whitney-Wilcoxon U-statistic under Lehmann alternatives. Technometrics 9: 666–77. Sidik, K. (2003) Exact unconditional tests for testing non-inferiority in matched-pairs design. Statistics in Medicine 22: 265–78. Starks, T.H. (1979) An improved sign test for experiments in which neutral responses are possible. Technometrics 21: 525–30. Steinijans, V.W. and Diletti, E. (1983) Statistical analysis of bioavailability studies: Parametric and nonparametric confidence intervals. European Journal of Clinical Pharmacology 24: 127–36. Steinijans, V.W. and Diletti, E. (1985) Generalisation of distribution-free confidence intervals for bioavailability ratios. European Journal of Clinical Pharmacology 28: 85–8. Stevens, W.L. (1938) Estimation of blood-group gene frequencies. Annals of

REFERENCES

399

Eugenics 8: 362–75. Stuart, A. and Ord, K. (1994) Kendall’s Advanced Theory of Statistics. Volume 1 - Distribution Theory. Sixth Edition. London: Edward Arnold. Suissa, S. and Shuster, J.J. (1985) Exact unconditional sample sizes for the 2 × 2 binomial trial. Journal of the Royal Statistical Society, Series A 148: 317–27. Tango, T. (1998) Equivalence test and confidence interval for the difference in proportions for the paired-sample design. Statistics in Medicine 17: 891–908. Ungersb¨ock, K. and Kempski, O.S. (1992) A study of the effect of increased intracranial pressure on the cortical microcirculation of New Zealand white rabbits. Department of Neurosurgery, University of Mainz. Unpubl. Techn. Report. Victor, N. (1987) On clinically relevant differences and shifted nullhypotheses. Methods of Information in Medicine 26: 155–62. Wang, W. (1997) Optimal unbiased tests for equivalence in intrasubject variability. Journal of the American Statistical Association 92: 1163–70. Welch, B.L. (1938) The significance of the difference between means when the population variances are unequal. Biometrika 29: 350–62. Wellek, S. (1990) Vorschl¨ age zur Reformulierung der statistischen Definition von Bio¨ aquivalenz. In G. Giani and R. Repges, eds., Biometrie und Informatik - Neue Wege zur Erkenntnisgewinnung in der Medizin. Berlin: Springer-Verlag, 95–9. Wellek, S. (1991) Zur Formulierung und optimalen L¨ osung des Bio¨aquivalenznachweis-Problems in der klassischen Theorie des Hypothesentestens. In J. Vollmar, ed., Bio¨ aquivalenz sofort freisetzender Arzneiformen. Stuttgart: Gustav Fischer Verlag, 91–109. Wellek, S. (1993a) Basing the analysis of comparative bioavailability trials on an individualized statistical definition of equivalence. Biometrical Journal 35: 47–55. Wellek, S. (1993b) A log-rank test for equivalence of two survivor functions. Biometrics 49: 877–81. ¨ Wellek, S. (1994) Statistische Methoden zum Nachweis von Aquivalenz. Stuttgart: Gustav Fischer Verlag. Wellek, S. (1996) A new approach to equivalence assessment in standard comparative bioavailability trials by means of the Mann-Whitney statistic. Biometrical Journal 38: 695–710.

400

REFERENCES

Wellek, S. (2000a) Bayesian construction of an improved parametric test for probability-based individual bioequivalence. Biometrical Journal 42: 1039–52. Wellek, S. (2000b) On a reasonable disaggregate criterion on population bioequivalence admitting of resampling-free testing procedures. Statistics in Medicine 19: 2755–67. Wellek, S. (2004) Tests for establishing compatibility of an observed genotype distribution with Hardy-Weinberg equilibrium in the case of a biallelic locus. Biometrics 60: 694–703. Wellek, S. (2005) Statistical methods for the analysis of two-arm noninferiority trials with binary outcomes. Biometrical Journal 47: 48–61. Wellek, S. and Michaelis, J. (1991) Elements of significance testing with equivalence problems. Methods of Information in Medicine 30: 194–8. Wellek, S., Ziegler, A. and Goddard, K.A.B. (2009) A confidence-limit-based approach to the assessment of Hardy-Weinberg equilibrium. In Technical Report, 2008/9 - 2. Department of Biostatistics, CIMH Mannheim. Westlake, W.J. (1972) Use of confidence intervals in analysis of comparative bioavailability trials. Journal of Pharmacological Sciences 61: 1340–1. Westlake, W.J. (1976) Symmetrical confidence intervals for bioequivalence trials. Biometrics 32: 741–4. Westlake, W.J. (1979) Statistical aspects of comparative bioavailability trials. Biometrics 35: 273–80. Westlake, W.J. (1981) Comments on “Bioequivalence testing – A need to rethink.” Biometrics 37: 591–3. Wiens, B.L. and Iglewicz, B. (1999) On testing equivalence of three populations. Journal of Biopharmaceutical Statistics 9: 465–83. Wiens, B.L. and Iglewicz, B. (2000) Design and analysis of three treatment equivalence trials. Controlled Clinical Trials 21: 127–37. Willems, J.L., Abreu-Lima, C., Arnaud, P., Brohet, C.R., Denis, B., Gehring, J., Graham, I., van Herpen, G. and Machado, H. (1990) Evaluation of ECG interpretation results obtained by computer and cardiologists. Methods of Information in Medicine 29: 308–16. Windeler, J. and Trampisch, H.J. (1996) Recommendations concerning studies on therapeutic equivalence. Drug Information Journal 30: 195–200. Witting, H. (1985) Mathematische Statistik I. Stuttgart: B.G. Teubner. Wittke-Thompson, J.K., Pluzhnikov, A. and Cox, N.J. (2005) Rational inferences about departures from Hardy-Weinberg equilibrium. American

REFERENCES

401

Journal of Human Genetics 76: 967–86. Ziegler, A. and K¨ onig, I.R. (2006) A Statistical Approach to Genetic Epidemiology. Weinheim: Wiley-VCH. Zinner, D.D., Duany, L.F. and Chilton, N.W. (1970) Controlled study of the clinical effectiveness of a new oxygen gel on plaque, oral debris and gingival inflammation. Pharmacology and Therapeutics in Dentistry 1: 7–15.